Predictive Analytics That Rock Your World

I love predictive analytics. I mean, who wouldn't want to develop an application that could help you make smart business decisions, sell more stuff, make customers happy, and avert disasters. Predictive analytics can do all that, but it is not easy.

In fact, it can range from being impossible to hard depending on:

• Causative data. The lifeblood of predictive analytics is data. Data can come from internal systems such as customer transactions or manufacturing defect data. It is often appropriate to include data from external sources such as industry market data, social networks, or statistics. Contrary to popular technology beliefs, it does not always need to be big data. It is far more important that the data contain variables that can be used to predict an effect. Having said that, the more data you have, the better chance you have of finding cause and effect. Big data no guarantee of success.
• Data scientists. A data scientist is someone who can understand the desired business outcome, examine the data, and create hypotheses about how to establish predictive rules that can enable business outcomes such as increasing eCommerce upsell, keeping a production line running, or eliminating stock-outs. Data scientists need skills in mathematics, statistics, and often domain knowledge. Check out the solution to the 2008 Netflix Prize winner - formulas galore. That is serious science. Fortunately, many predictive analytics solutions aren’t as rigorous as this.
• Predictive analytics software. Data scientists must evaluate their hypotheses using software tools that can apply any combination of statistics and machine learning algorithms. IBM SPSS and SAS are two well-known analytics software tools used by data scientists. R Project is a popular open source choice. I am currently evaluating 10 vendors for the Forrester Wave™ evaluating big data predictive analytics solutions, which is scheduled for publication in fall 2012. If the data is big then you may need special processing platforms such as Hadoop or in-database analytics such as Oracle Exadata.
• Operational application. If you are lucky enough to find predictive rules, then you need to embed them in your application. Your predictive analytics software might have a way to generate code such as predictive analytics vendor KXEN. It is important that the data needed by the predictive rules is readily available. Predictive rules can also be enhanced with business rules management systems and complex event processing (CEP) platforms.

Ambitious Application Developers Can Learn This

Application development teams can differentiate themselves from their code-only peers by using predictive analytics to develop smarter apps. The computer scientist in you will love the gnarly big data and machine learning algorithms.

Good stuff.

This blog originally appeared at Forrester Research.