Predictive analytics involves the use of data mining, mathematical modeling, and statistical analysis to provide actionable predictions and help drive the decision-making process. Traditionally, business intelligence has dealt with data access, reporting and limited analysis.

It has been helpful in answering questions about the past and to a certain extent in explaining current events, but traditional BI doesn’t provide any insight into the future. Historically, BI has helped businesses answer questions such as what happened, why did it happen, what was the problem and what actions are needed. Predictive analytics goes beyond this and helps the business with what is happening in real-time and what will happen next.

See Related Graphic 1: Predictive Analytics for What's Happening and What's Next

Predictive analytics is forward-looking and it provides actionable predictions based on trends, patterns, relations and correlations in data. It enriches the decision-making process in a big way by making the intelligent prediction available as the basis for the decision-making process. At a high level, predictive analytics applies a mathematical modeling and statistical analysis approach to the data to develop knowledge that can be used to predict future events. A very common example of predictive analytics is the one used by credit bureaus to develop credit scores for consumers by predicting the customer behavior.

Role in Advancing Business: Actionable Insights and Predictions

“In business, as in baseball, the question isn’t whether or not you’ll jump into analytics. The question is when. Do you want to ride the analytics horse to profitability … or follow it with a shovel?" – Rob Neyer, author and senior writer, ESPN.com

Predictive analytics has great commercial, scientific and social value. The purpose could be to decide the product marketing strategy, customer retention, risk management or product pricing. This exercise can’t be completed in isolation with the use of data, technology and statistical models alone. Business acumen plays a significant role in formulating insightful predictions that make sense and deliver to meet the business requirements. The ultimate purpose of the analytics is to help deliver on strategy and achieve defined goals.

It’s essential to define what to predict and how to use the predictions. There are a wide variety of trends, patterns and behaviors that can be identified, but the focus should be on getting the actionable insight according to the defined objective. Start with defining the objective of the data mining and predictive analytics, which could be risk management, fraud detection or antiterrorism. The following are some examples where predictive analytics is widely being used.

Strategic business planning: Predictive analytics enhances strategic business planning by eliminating the reliance on mere averages or guesswork and setting the focus on actionable intelligence. It enables businesses to better manage risk and uncertainty and helps with decision-making to manage the likely future events. Smart companies are integrating predictive analytics with decision-making processes to improve strategic-planning processes. Embedding the ability to intelligently predict the uncertain future and the ability to measure the impact of this uncertainty in strategic planning goes a long way in helping companies achieve their business objectives.

Risk: Adoption of predictive analytics lets companies effectively respond to changes in risk drivers and actually manage all material risks related to competition, capital markets and regulatory bodies. Predictive modeling makes risk data more relevant and actionable. It is widely used to predict risk for credit loan applications and plays a significant role in managing risk and deciding insurance policy premiums based on customer demography, financial information and claims history. In rapidly changing markets, the reward for being able to manage risk quickly and effectively is greater than ever.

Audit and fraud detection: In cases of audit and fraud detection, the goal could be to catch the incidence of fraud before it occurs or apply the predictive models to the large number of transactions to identify the ones that are more likely to be fraudulent. This is helping companies change their approach from being corrective to preventive in fraud management.

Medicine: Role of predictive analytics in patient, physician and facility management has been growing as the cost of medical care has come under scrutiny. In the field of medicine, it is being used in disease management to improve provider partnership and strengthen customer relationships. Patient profiles are used by physicians to predict hospital admission likelihood. Data mining is being used by hospitals to predict the length of hospital stay, which helps them better manage the patients, physicians and the facilities.

Miscellaneous: Predictive models are also being used in a wide range of scientific and social initiatives. Scientists use it to refine their predictions of global climate change. Knowledge discovery from spatial data is leading to a new era of climate change predictions. Predictive models are being used to help manage resources for student, faculty and curriculum planning in educational institutions. It is being used to help manage the natural resources and minerals. And it is assisting in predicting food contamination and disease outbreaks.

Customer Relationship Value and the Path to Profitability     

“The pillars of effective cross-selling are strengthened and enhanced by the effective application of predictive modeling and analytics.” – “The Intelligent Contact Center: Using Predictive Analytics to Generate Growth,” Christopher Checco and David Rook, Customer Chemistry

By capitalizing on the rigor of predictive analytics, companies can align business strategy with customer acquisition, development and retention programs to manage one of their most precious assets: customers. Individual business goals could be to sign up new customers, cross sell, up sell, reduce customer churn, etc. Retailers frequently use data mining and predictive modeling to identify customer behavior patterns and use this information discovery to target customers for direct marketing offers. This helps in building a competitive position, developing plans to maximize sales and increasing revenue.

Predictive analytics can assist at practically every customer touchpoint. It can be used in conjunction with interactive voice response systems, response modeling, market basket analysis, etc. It helps with determining the expected customer behavior, managing the changing customer needs and the ways to best address them. It helps with winning the right customers, improving customer relationship experience, and with the customer behavior expectations.

Customer lifetime value analysis along with data mining provides potential value and expected profitability from customers. Data analytics in the areas of campaign evaluation, churn modeling, pricing analysis and contact frequency analysis helps companies decide how to offer the right product at the right time. It sets the suitable price points encouraging the customers to make purchases while reducing the possibility of churn and managing the marketing campaigns.

Use of predictive analytics with a wide range of data to analyze factors such as pitch rates, close rates and procedure compliance helps identify the trends and provides an opportunity to correct the process. It also helps identify the possible reasons for not achieving the expected results, e.g., over pitching customers, offer fatigue and price point misalignment.

See Related Graphic 2: Predictive Analytics Enhances the Customer Lifetime Value and Profitability

The Process of Predictive Analytics: How Does it Work?

“Analytics themselves don't constitute a strategy, but using them to optimize a distinctive business capability certainly constitutes a strategy.” – “Competing on Analytics: The New Science of Winning,” Thomas H. Davenport and Jeanne G. Harris

Business requirements or problem definition: Driving predictive analytics based on input from businesspeople ensures the alignment of the outcome with business objectives. Success of the outcome hinges completely upon the ability to define the business requirement and the problem before addressing it. You should determine what business needs will be addressed and what business value will be delivered.

Data exploration and preparation: Predictive analytics begins with the definition of the data set to be used. The identified data set is profiled, cleansed and enriched to ensure data quality and to provide the complete picture. This can be summed up as defining the relevant data set, identifying relationships in data elements and ensuring data quality.

Statistical modeling and model validation: Once the data is ready, a statistical modeling framework is designed. Variance in data quality is accounted for in the statistical model because missing or incomplete information impacts the outcome. This stage forms the core of predictive data mining.

Predictive model deployment: Selected data set along with additional external data is fed to the predictive model to produce the actionable predictions.

Analysis of the results: After applying the data to the model, statistical analysis is performed and the results are analyzed to come up with actionable predictions. The outcome of this process is used to refine the business strategy and to act on strategic and tactical initiatives.

See Related Graphic 3: Predictive Analytics Iterative Process

A good example that highlights how the predictive modeling process works is the insurance industry practice of managing risk and policy premium pricing. A wide array of data elements, such as age, medical condition and employment, can be used to feed the predictive model to determine the policy premium for the insured. Additional variables such as credit scores, life choices and family can be added to the algorithm to improve the accuracy of the predictions. Another example could be to predict the likelihood of a customer purchasing a specific product in a defined time frame as a result of the mailing campaign. Marketing statistics, membership data, purchase patterns and customer demographics are identified as data sets; a statistical model is created and scored based on the objective of the exercise, and identified data sets are applied to the model to determine actionable predictions.

Framework, Technology and Architecture: Enabling Success

"We are becoming rational, analytical and data-driven in a far wider range of activity than we ever have been before." – Larry Summers, former president, Harvard University

A predictive analytics framework manages the link between the goals being served and the end-to-end process required to provide the actionable predictions to achieve those goals. It incorporates people, processes and technology to work with the steps of data exploration, data selection, data preparation, technology selection, statistical modeling, expert insights and analysis of the results. Such a framework ensures the use of best practices to go from inception to implementation. In order to derive meaningful results, it is imperative to safeguard against unacceptable practices within data management, statistical modeling and analysis. Continuous process improvement should remove potential bias from the existing information to avoid the risk of corrupting the end results.

The success of predictive analytics relies on the data set selection, the quality of the data being fed to the model and the statistical models being used to analyze the data. Counts and distribution of variables and observations play a significant role in the outcome because the predictive models rely on relationships between the data elements and various other influencing factors. Reliability of the outcome also depends on whether the data set of interest is complete. Inability to correctly identify the control set, the treatment set and incorrect use of the statistical procedures derails the analysis. This could result in probabilistic observations and guesswork, which can easily lead to confusion, whereas the goal of the exercise is to come up with actionable predictions.

A large variety of predictive models have been developed, including linear and nonlinear regression, logistic regression, support vector, neural networks and decision trees. There is no one-size-fits-all model, and these models are applied based on problem definition, scenario, data set and the desired outcome. Advancements in the areas of pattern recognition, statistical and mathematical techniques are used in these models.

Recent Advancements and Future Trends: Can it Become Pervasive?

In today’s world mining of text, Web and media (unstructured data) plus structured data mining, the term information mining is a more appropriate label. Mining a combination of these, companies are able to make the best use of structured data, unstructured text and social media. Static and stagnant predictive models of the past don’t work well in the world we live in today. Predictive analytics should be agile to adapt and monetize on quickly changing customer behaviors in our world, which are often identified online and through social networks.

Better integration of data mining software with the source data at one end and with the information consumption software at the other end has led to improvement in the integration of predictive analytics with day-to-day business. Even though there haven’t been significant advancements in predictive algorithms, the ability to apply large data sets to models and the ability to enable better interaction with business has led to improvements in the overall outcome of the exercise.

Predictive analytics is becoming more mainstream as a result of advanced machine learning capabilities, technology advancements and availability of large volumes of data. Can predictive analytics be pervasive in future? Absolutely! The technology is still maturing, but predictive analytics has huge potential, and recent advancements are very promising.

Social, Ethical and Regulatory Implications

It is imperative to consider the social, ethical and regulatory implications of predictive analytics. There is a need to exercise caution in using the outcome of data mining and predictive analytics to make decisions such as profiling an individual as a suspect or assigning a mortgage interest rate. In all of these cases, there is a probability that the data being applied to the statistical model or the model itself is skewed and could provide a result that leads to a decision violating individual rights or hurting the business instead of helping it. Considering the level of maturity in the predictive analytics space, the outcome should be used with human intelligence and various other factors to minimize the negative implications of the decisions.

Predictive analytics involves detecting relationships in data using predictive models and applying statistical analysis and business acumen to make actionable predictions leading to decisions impacting people. As an example, if the insurance industry uses a credit score as one of the factors in deciding policy premiums for its customers, then the insurance firm must be able to demonstrate how various other factors such as education, employment and marital status are being used along with credit score to decide the policy premium. To manage the implications, this information as a whole -- and not the outcome of predictive analytics alone -- should be the basis of the decisions.

The Promise of Data Mining and Predictive Analytics

With the current state of data mining technology, it wouldn’t be prudent to expect the outcome of predictive analytics to provide definitive outcomes such as flagging a fraud case or a terrorist with absolute conformity. At the same time, it is inappropriate to disregard the result of the exercise while constant advancements are being made to improve the accuracy of the outcomes. Predictive analytics is not a silver bullet. It is not a substitute for the intelligence that comes from people who have a deep understanding of the issues and know how to use systems, data and outcomes of predictive analysis. It is an enabler.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access