Predictive analytics is being embraced at an increasing rate by organizations that need to gain actionable and forward-looking insight from their data. Why? Companies realize that simply looking in the rearview mirror to obtain insight and make decisions is not enough to remain competitive. Companies want to better understand what actions their customers might take. They want to better predict failures in their infrastructure. The uses for predictive analytics are extensive and growing. Some examples include customer churn analysis, predicting insurance fraud and finding patterns in health related data.
Recently, Hurwitz & Associates published its inaugural "Victory Index for Predictive Analytics." The Victory Index is a market research assessment tool developed by Hurwitz & Associates that analyzes vendors across four dimensions: vision, viability, validity and value. In the course of our research, we surfaced a number of trends in the predictive analytics market that have important implications for companies considering deploying the technology. Five of these trends are discussed in this article.
1 | Providing Solutions Across the User Spectrum
Whereas the traditional user of predictive analytics was a statistician or other quantitative analyst, a change is occurring in user type. There is a shift toward business analysts as users and consumers of these products and services, and businesses want to expand the field of users even further. In fact, many organizations believe that anyone with knowledge of the business should be able to make use of predictive techniques. Vendors have responded to this demand in different ways. Some are providing more user-friendly graphical user interfaces and automating the process of building predictive analytics models. Some vendors provide tools with shortcuts, for example, suggesting the right predictive model for a user based on the data at hand.
Ease of use is a huge trend in predictive analytics today. However, since predictive models can be complex (and some remain best left to statisticians to build) some vendors are providing technologies for sharing the results of the models via interactive mashups and other Web interfaces. These strategies can help nontechnical users build and make use of predictive analytics to a certain level. Still others are offering predictive analytics in a software as a service model. Here, companies provide the vendor with their data and the vendor will produce the analysis for the company.
Regardless of who is using the technology, training will be necessary. The vast majority of end users we spoke to for our research, many of whom are using multiple predictive products, believe that training is a must, irrespective of the product used.
Predictive analytics can provide significant top-line benefits to companies, but in the wrong hands can lead to distracting or misleading results. If your company is planning to purchase predictive analytics tools or solutions, it is important to consider the skill level of the people using the technology and the range.
2 | Operationalizing Models
End users and vendors alike see great advantages in incorporating predictive models into business processes. For example, statisticians at an insurance company might build a model that predicts the likelihood of a claim being fraudulent. The model, along with some decision rules, could be inserted in the company's claims processing system in order to flag claims with a high probability of fraud. These claims would be sent to an investigation unit for further review.
In other cases, the model itself might be less visible to the end user. For instance, a model could be built to predict customers who are good targets for upselling when they call into a call center. The call center agent, while on the phone with the customer, would receive a message on specific additional products to sell to this customer. The agent might not even know that a predictive model was working behind the scenes to make this recommendation.
If your company is thinking of going down this route, make sure that the vendor you deal with can support this kind of deployment. For example, the vendor should provide model scoring as well the ability to create rules for decision-making and a platform for model deployment. In the insurance example, for instance, a rule might be that if the amount of the claim is less than $200, the claim goes to a separate category that isn't investigated because it would cost more to investigate than the claim is worth.
3 | Supporting Unstructured Data Analysis
End users realize they can gain significant insight from mining unstructured (i.e., text) data, which, used in conjunction with structured data, can provide a big lift to predictive models. For instance, in a worker's compensation claim, there may be notes regarding poor performance or the fact that the worker had been cited for misconduct. This information may not occur in structured data. Finding some of this information by manually reading through the notes is enormously time-consuming, and it is almost impossible to search through large amounts of data to identify patterns that might indicate fraud.
More vendors are offering solutions that marry structured data together with unstructured data. If your company has a lot of unstructured data and this kind of analysis might be on your roadmap, check with potential vendors to see how they deal with this kind of data. Do they offer text analytics technology? What sources can they extract data from? How easy is it to integrate the structured and unstructured data?
4 | BIG DATA
As companies gather greater volumes of disparate kinds of data (i.e., both structured and unstructured), they are looking for solutions that can scale. This kind of data includes data generated by RFID tags, social media and other channels. Real-time analysis of large amounts of data is also becoming more prevalent. For example, companies might use this capability for real-time ad placement. Or, a hospital might be interested in analyzing patient data from medical devices in real time. The buzz from vendors about "big data" solutions is growing louder. Some are providing solutions that work with big data appliances. Others are purposely built for analyzing big data.
Before embarking on a big data strategy, it is important to take a step back and ask what questions you are trying to answer with all of this data. Then it is important to look at your data sources - you may not need them all. Data consistency, integrity and other data management issues will be key to a big data strategy even before you start trying to analyze the data with predictive analytics.
5 | Open Source
Open source solutions are becoming increasingly important to the predictive analytics market because they enable a wide community to engage in innovation. Open source solutions are often used in academic institutions (at no cost), so students coming into the workforce are already familiar with them. But the openness of the software can also mean it is less accessible to nontechnical users. In fact, a number of end users we spoke to as part of the Victory Index commented on how they are struggling with using open source because of user interface and lack of performance. Vendors are trying to change that.
Established vendors are incorporating open source products, such as R, into their product suites. R is a free software environment for statistical computation and graphics. It consists of a language, tools and the ability to run programs from script files. New entrants may wrap their software and services around R to make it easier to use. Others are providing their own open source platforms.
These are just some of the trends that we're seeing in the predictive analytics market. While some of the core technology for predictive analytics has been around for decades, the market is now moving past the early adopter stage to become more mainstream. In fact, it is becoming red hot. Vendors are providing a range of solutions for this market across a range of problem types. There is a huge upside to implementing predictive analytics in terms of understanding customer behavior, being better able to compete and grow your top line. Companies embarking on predictive analytics will need to evaluate these products carefully to fit their specific needs.