Advanced Analytics is the autonomous or semi-autonomous examination of data or content using sophisticated techniques and tools, typically beyond those of traditional business intelligence (BI), to discover deeper insights, make predictions, or generate recommendations. Advanced analytic techniques include those such as data/text mining, machine learning, pattern matching, forecasting, visualization, semantic analysis, sentiment analysis, network and cluster analysis, multivariate statistics, graph analysis, simulation, complex event processing, neural networks.
Traditional BI vs, Advanced Analytics
Advanced Analytics represents a collection of techniques used to model internal and external data to yield valuable insights that can drive business-improving actions. This collection of techniques is called “advanced” analytics to differentiate them from the traditional analytics approach normally accomplished using business intelligence (BI) systems.
The following four main ideas help you understand the differences between traditional BI and advanced analytics:
- Difference 1: Their Purpose - In traditional BI, data is processed to inform business users of the past performance of business operations. Thus, the data is gathered and aggregated into a clean format for reporting purposes. Nowadays, the expectation of traditional BI systems is extending into a new area, analysis-oriented BI. Organizations want to get a 360-degree view of their customers in a timely manner, identify the root causes of success or failure in business operations, and control as much future uncertainty as possible. These demands can’t be satisfied with traditional BI dashboards or reports alone. This is where advanced analytics comes in to solve complex business problems. In this way, advanced analytics serves as a trouble-shooting player in an organization rather than information provider.
- Difference 2: The Approach - There is a fundamental difference between traditional and advanced analytics, namely the process followed to design and solve a business problem. In traditional BI, the analysis is typically built to be repeatable. IT develops the reporting template and extracts certain information important to the business in assessing historical performance. Thus, the types of information analyzed and the format in which the information is presented is predefined. The advanced analytics techniques that have become more mainstream remind people there is another approach: a question or doubt is raised first, then a set of analysis is designed to dive into the data and mine the business insight to answer the question. In this approach, IT typically only provides the analytical platform. The business then directly collects what it wants. Advanced analytics software vendors provide a friendly user interface to allow people with varied backgrounds to utilize the data to find the answer to their questions. Often the software will guide the user through the techniques by helping select and process the relevant information from multiple resources.
- Difference 3: The Data Used - Data used for analysis in traditional BI is typically gathered from a data warehouse or a data mart. The BI platform connects to the data sources, queries and aggregates data to higher levels in a hierarchy, such as geography, time frame or category. Business users review the aggregated reports to understand historical operational performance. If any unusual observations are identified, they can drill down to lower/related levels for more detail. Characterizing traditional analysis as starting from large to small, advanced analytics starts from small. With the statistical methods and enhanced computational power, now we are able to capture the unique characteristics of each individual customers instead of analyzing customers at a segment level. So it becomes possible to make personalized marketing activities and improve marketing ROI. Besides structured data, unstructured data such as social comments, images and videos become valuable sources of information. Stream computing, furthermore, helps people access and process real-time information. Organizations can monitor market sentiment and brand engagement to measure effectiveness of a marketing campaign. They can also improve the product design by analyzing social comments. Thus, more complete, timely and various data are analyzed with advanced analytics.
- Difference 4: The Techniques - When we say advanced analytics, “advanced” refers to quantitative methods such as statistics, algorithms and stochastic processes. Although not all of the advanced analytics techniques are predictive, they are future-oriented since the key idea of the methods is to support data-driven decisions in the future. The advanced analytics techniques can be categorized into three functions:
- Descriptive Analysis: Descriptive analysis aims to understand an underlying phenomenon or process. The analysis will answer questions like, “What are the typical characteristics of customers who tend to churn?”, or, “Which products do consumers usually purchase together?”
- Predictive Analysis: Predictive analysis studies the hidden relationships between factors and outcomes and then forecasts or estimates an unknown value. For example, a predictive model will allow us to predict which customers are going to churn, or estimate how much revenue will be lost if temperatures drop 10 degrees.
- Simulation and Optimization: Simulation imitates the operation and characteristics of a process and summarizes the outcome. Optimization prioritizes the decision options based on a key performance index. For example, if we want to design a drive-through route for a restaurant, we can simulate the traffic and ordering process, compare the simulation outputs for several options, optimize the design and select the best choice.
In summary, while some of the underlying data may be consistent between traditional BI and advanced analytics activities, these two techniques vary significantly in the “what”, “why”, and “how” they are implemented. Advanced analytics wields the power to drive deeper, more strategic and more actionable insights from your data than traditional BI reporting.
An Example of Advanced Analytics
Cross-selling and Up-selling - By associating products with customers, purchasing behavior can be analyzed. Thereby, products can be cross-sold effectively. Advanced Analytics can help in identifying cross-sell and up-sell opportunities by analyzing purchase patterns using sophisticated techniques and algorithms. The idea of cross-selling translates well into just about any business situation. In the fast food industry, customers are often invited to try new products or established complementary items. For example, when an individual orders a hamburger at a local fast food restaurant, the server will often ask the customer if he would like a side item to go with the hamburger. If the restaurant is offering a new dessert, the server may also suggest to the customer that the new item may be a desirable complement to the hamburger. By employing this simple approach, the server may entice the customer into making another purchase above and beyond the one originally intended.
Benefits of Advanced Analytics
- 1. Collecting and analyzing your data can transform your business: Data is only ones and zeros until you turn it into insights that provide business impact. The integrated emergence of the Internet of Things (IoT) and advanced analytics marks an opportunity for organizations to gain views of their business that weren’t available before. IDC estimates companies that are leaders in using data assets to their advantage will capture $1.6 trillion more in business value than those that lag behind. Whether the data suggests a new business model or presents a potential new revenue stream, when you apply advanced analytics to the data from the Internet of Things, you can see new ways to transform your business.
- 2. Connecting things you already have makes economic sense: You have untapped data all over your business. By using the Internet of Things you can make use of these data sources to improve your business. Start with the things you have already—your systems, devices, sensors and data—and focus on the areas of your business that can yield data insights that provide quick return. A common place to start is operations by connecting systems and line-of-business assets, such as machines on your factory floor or trucks you use to deliver products, to improve performance visibility, which can streamline maintenance and help to reduce downtime.
- 3. Analysis of diverse data sources provides deeper insights: Start with your existing IT investments and build upon them to access diverse data sources. You can apply advanced analytics to the data those assets generate to gain deeper insight into what your customers want and your employees need.
- 4. Integrating new data streams delivers a more comprehensive view of your overall business: Whether you’re seeking a top-level view or drilling deep into certain metrics, integrating existing systems with new data sources, such as those from devices, can provide a more thorough picture of your business. Use these combined insights to create new business models and find new ways of staying ahead of the competition.
- 5. Process automation offers remote monitoring possibilities so you know the status and health of your assets: By 2020, Gartner estimates more than 85 percent of data will be automatically generated – from every device, sensor, upload, tweet, purchase, shipment and keystroke. Remote monitoring offers better visibility into operational status by collecting data from your assets and using that data to trigger automatic alerts or actions, such as maintenance requests. By automating alerts from your systems and devices, you can respond proactively to potential issues rather than reacting after the fact. You can also analyze previously untapped data to improve business outcomes.
- 6. Predictive maintenance provides opportunities to better manage and maintain your business assets: Take action earlier on emerging trends, streamline processes and avoid costly downtime by anticipating the maintenance needs of devices and other assets from insights within data. Dive deeper with asset optimization to gain comprehensive intelligence from advanced analytics that can inform your maintenance schedule or create new business opportunities altogether. For example, by gathering data from new or existing sensors and systems, companies can go beyond preventative maintenance to offer predictive and even preemptive maintenance to avoid costly downtime.
- 7. Your data can help your business learn and adapt: Businesses that learn and adapt will continue to thrive. By monitoring and analyzing data from multiple sources in near real time, you can enable your business to innovate and make the most of the situation. By analyzing purchasing behavior, for instance, you can predict what your customers will order and suggest those items and complementary products at their next visit, providing better service to your customers.
- 8. You gain more certainty when insights are easier to uncover: Spend less time wondering and more time taking action. Data insights can help you respond more quickly to competition, supply chain changes, customer demand, and changing market conditions. Collecting and analyzing data gives you quick insight into developing trends, so you can change your production activity, fine-tune your maintenance schedule, or find less expensive materials.
- 9. Bringing solutions to scale is smooth and efficient: New ideas are born when you work with new partners, new technologies, new assets and new data streams. Comparing results from different store locations, for instance, lets you identify the most successful services and roll them out nationwide. The Internet of Things lets you scale from the smallest data point to global deployments, and advanced analytics lets you turn those data insights into intelligent action.
Challenges of Implementing Advanced Analytics
Some of the concerns the authors have observed in the development and implementation of advanced analytics include:
- 1. Executive Ownership: Without buy-in from senior leadership and a clear corporate strategy for integrating predictive models, advanced analytics efforts can end up stalled at model development. In order to be effective, analytics efforts should involve the key executives who can help drive acceptance and change throughout the organization. Senior leaders should insist there be a clear correlation between the actions to be taken through model implementation and the expected business benefits to be realized. Without accountability for a targeted return on investment, organizations risk spending a lot of time “doing” versus “getting things done.”
- 2. IT Involvement: Failure to involve IT from the very beginning of the analytics journey can lead to significant issues down the road if technology gaps and limitations aren't understood up front. Modelers may find a way to get access to internal and external data, but without the help and involvement of IT, it is almost impossible to bring the models to life in the day-to-day operation of the organization
- 3. Available Production Data vs. Cleansed Modeling Data: Access to historical data for model development is very different from access to real-time data in production, and a strong model is only as good as its ability to be practically implemented within the technology infrastructure. Real life limitations may restrict the data that's available for historical modeling. Sometimes a proxy variable can be used for modeling until the data is available. Analytics initiatives often risk being stymied by the belief that data for modeling must be perfectly clean and organized. Predictive model development is not an accounting exercise, but rather a statistical process where numerous techniques allow the “dirt in data to be washed away.”
- 4. Project Management Office (PMO): Lack of clear ownership of the end-to-end journey is a common stumbling block for organizations that have struggled (and failed) in implementing their predictive models. Without the right project management structure in place, a clear cadence of project milestones, and the ownership of deliverables by pre-identified business owners, the project could be doomed before it starts. Most importantly, the PMO must be able to connect with all interested parties and adopt an agile approach.
- 5. End User Involvement and Buy-In: Lack of end user involvement in the planning, design and ultimate roll out of the predictive models can be detrimental to the efforts. For underwriting or claims models, involving underwriters, marketing, actuaries, claims adjusters, nurse case managers and special investigative unit (SIU) resources early in the process is critical. End users also have more insight into the business process and may be able to better identify potential gaps or roadblocks to successfully incorporate models in day-to-day operations. If the end users feel as if they have a stake in the predictive model roll out, then the company may be more likely to realize the potential financial benefits. If done correctly, some of the early doubters can eventually become analytics advocates.
- 6. Change Management: Organizations often fail to understand how predictive models change the current business and technology operations — policies, procedures, standards, management metrics, compliance guidelines and the like. Without the proper design, development and roll out of training materials to address the impacted audiences in the field and home office, the analytics journey can come to an abrupt end. Educating end users and other related stakeholders on how the model will be used on a day-to-day basis, and how their life may change, is important. A communication plan should be developed to answer frequently asked questions (FAQs), address common concerns, and help end users appreciate the strategic vision of the organization. Change management doesn't start and end with training; it begins on day one and lasts well beyond the roll out of the models.
- 7. Explainability vs. the “Perfect Lift”: It is important to balance building a precise statistical model with the ability to explain the model and how it produces results. What good is using a non-linear model or complicated machine learning method if the end user has no way to translate the drivers of the score and reason codes into actionable business results? Experience shows that a less complex statistical model development method yields results similar to those from more complex approaches, and a small sacrifice of predictive power can result in marked improvement in the explainability of technical model recommendations for end users.