Predictive analytics is a source of fresh insight based on finding patterns in the tidal wave of transactional data that is stored and aggregated in existing transactional and data warehousing systems. In its predictive form, an inference about future behavior, performance or events is an essential part of the analytic model - the prediction is inside the model, not merely posed by the inquirer in a database query. Major drivers include:

The need to get to know the customer as a system of dispositions and behaviors. Firms need to relate to the customer as more than a series of clicks on a Web page or an aggregation of transactions. The view of the customer is transformed by predictive analytics. The customer is a bundle of dispositions and propensities - to buy, to jump to the competition or to be loyal. As the economy turns from cost cutting to generating incremental revenue, predictive analytics is an important method of producing results because it is able to determine a future outcome of customer, product and market behavior with a determinate, scientific probability.

The need to work smarter and reduce costs. During the economic slump of the past two and half years, firms have cut costs mercilessly, in some cases cutting capacities that are essential to recovery. However, firms recognize the need for a better way to reduce costs than across-the-board cuts that take out both waste and essential infrastructure together. Predictive analytic applications are one way to reduce the constraints of inventory, operations and risk management. These predictive applications are applied to cut costs through smarter forecasting, demand planning, supply chain optimization and credit worthiness.

Exploratory data warehousing. Forrester's Business Technographics April 2004 North American Benchmark study reports that fully 54 percent of respondents stated they had a centralized data warehouse of customer information either in production or they were in the midst of a rollout.1 While data warehousing is a mature technology, the advanced data warehouse is not. Next-generation data warehouses are now making inroads into complex problems in merchandising, optimizing product mixes at a store level where previously individual SKUs had to suffice. Retailers such as Ann Taylor and Marshall Field's are using the detailed POS data to analyze and track buying behavior as it lines up with customer loyalty across time, share of pocketbook and lifetime value. The availability of exploratory data warehousing - a clean, consistent body of information stored in a data warehouse - opens the possibility of predictive inferences about key markets, brands and customers.

Innovations in predictive analytic algorithms. Fast algorithms based on the research of Vladimir Vapnik and implemented by KXEN are posing new challenges to established market leaders such as Computer Associates (CA), SAS, SPSS and IBM. Genetic algorithms are inherently parallel, enabling large numbers of suboptimal predictive models to be invalidated simultaneously. These algorithms are surprisingly rapid in effectively searching complex, highly nonlinear, multidimensional search spaces. Upstart applications are largely limited to marketing, but it will not be long before the results are generalized to predictive analytics at large. Similar considerations apply to the inherent parallelism of genetic algorithms as commercially implemented by Genalytics, now also implemented in CA's Predictive Analytic Server. As these companies continue to have success in implementing genetic algorithms in commercial contexts, they will redefine and extend the limits of what is possible with the technology. This will drive the following trends regardless of the near-term success (or not) of one or two particular software companies:

Intensified competition between workbenches and wrappered applications. Innovations in fast algorithms are causing intensified competition by creating possibilities for benchmark shootouts where the upstart wins. Meanwhile, the competition between predictive analytic workbenches and predictive marketing applications will intensify with the pendulum swinging back in the direction of the workbenches. While algorithms have been incorporated into application suites - Siebel and PeopleSoft OEM predictive functions from Angoss, and Business Objects uses KXEN - the general-purpose workbench is and remains a viable platform for solving analytics problems in horizontal business domains.

Predictive analytics will assimilate predictive marketing. Consolidation is an ongoing trend, with Data Distilleries being acquired by SPSS in November 2003 but only recently brought back to market. Predictive marketing upstarts such as Sigma Dynamics are reportedly having trouble making their numbers. The one exception is Ingeneo, which won new accounts in Israel where it also has a development lab. Companies with only a single algorithm to their name such as Trajecta, Salford Systems or NeoVista - no matter how high performance - will struggle for market traction and either be acquired or fail outright. One-algorithm wonders are out of favor. The one hope: an OEM strategy such as that which has buoyed the fortunes of KXEN.

An architecture for every budget - desktop, in-database, server-based. The popularity of desktop tools for predictive analysis is holding strong. Tools such as S-Plus from Insightful, See5 from RuleQuest and SPSS Answer Tree are less expensive and less powerful than server-based options. After having stalled during the recession, in-database predictive analytics will get new impetus from the need to ship the function to the data rather than move the data to the function. For those enterprises that have large volumes of data in data warehousing systems, the line of least resistance is to analyze the data for predictive insights where it is - in the database.

However, diverse data stores will sustain the role of analytic servers. In-database predictive model building represents a game-changing architecture that can cannibalize the market of standalone analytic engines. However, given the diversity of data stores and databases in the multidivisional enterprise, most companies will continue to leverage the power, scalability and database independence of the analytic server dedicated to implementing the predictive model.

1. Forrester's Business Technographics April 2004 North American Benchmark study indicates that the adoption of customer data warehousing is reaching late maturity in the information technology adoption curve. In addition, data warehouses are meeting expectations at a high satisfaction level.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access