Continue in 2 seconds

Data Mining Market Consolidation Lessons Learned

  • February 01 2004, 1:00am EST

The consolidation in the data mining (predictive analytics) market during the past two years has been continuous and merciless. One of the key lessons for both users and providers of data mining technology and applications is that it is in the process of transitioning to predictive analytics.

Silicon Graphics' MineSet, which once boasted a substantial installed base, is not currently shipping a product. Trajecta was a data mining startup circa 1999 emerging from the incubation laboratory at Microelectronics and Computer Technology Corporation. Fair Isaac acquired the core Trajecta technology in 2002. Fair Isaac and HNC merged in August 2002. The sun has set on Business Miner from Business Objects, although Business Miner continues to have significant recognition in the desktop data mining market. Meanwhile, Business Objects has been offering KXEN predictive analytics as an encapsulated part of its Application Foundation and as a separately licensable product since January 2003 (initially bundled as part of the entire suite in June 2002). Instead of Darwin, Oracle now offers a migration path to Oracle9i Data Mining analytics.

In the face of a perfect storm of bad economic news and externalities, even top-notch technology can be culled from the flock as if by a Darwinian process, including tools with that name. The lessons of consolidation are described in the following paragraphs.

The promise of giving all different classes of end users access to data mining (predictive analytics) is oversold ­– data mining will require a cross-functional, collaborative effort and predictive modeling tools that support collaboration will have an advantage.

Data preparation can consume a substantial effort, and data mining tools require further productivity enhancements to address those requirements.

The data is to be found in the database and "in database data mining" is the technology of the future and always will be, unless ways of rendering the technology administratively usable and manageable can be found.

Templates and recommendation engines wrapped in applications are adequate for well-defined problems, but the advantage will otherwise go to those tools for predictive analysts that are able to support collaboration between business analysts, statisticians and information technology (IT) data preparation staff. Attempts to dispense with even one of these over the life cycle of the data mining application are rarely successful. Even a deployed, debugged application must have the underlying predictive model revalidated from time to time in the face of different data, different business circumstances or different results from the model itself. This will inevitably require the statistician and IT support staff.

There are a number of predictive analytics ("data mining") problems around churn, attrition, cross-selling and upselling. On the product side, demand planning and just-in-time inventory are defined well enough to be addressable by predefined applications (solutions) within their respective vertical markets in telecommunications, finance and logistics. Companies such as Unica, KXEN and Quadstone, and offerings wrapped as recommendation engines in applications from E.piphany and Siebel Analytics (even SAP, Oracle and PeopleSoft) have demonstrated success. However, mapping less well-defined problems to the available technology and predictive algorithms remains a challenge for statisticians in collaboration with business analysts and IT data preparation staff. Here, the business is up against the problem of "not knowing what they don't know." Unsupervised knowledge discovery holds great promise and always will, unless a framework for collaboration between statisticians, IT technology experts and business analysts is available to support the design, deployment and maintenance of the applications.

Prospective purchasers of predictive analytics software should take advantage of the buyers' market in qualifying alternative predictive products and approaches to data mining and predictive applications. Once a purchase has been made, manage and reduce the risk of technology lock-in by conforming to open standards such as predictive modeling markup language (PMML) and commodity hardware where feasible. When the business needs to design and implement a wide variety of predictive applications that cannot necessarily be defined in advance or readily encapsulated in off-the-shelf packages, end-user enterprises should choose a general purpose workbench such as SAS Enterprise Miner, SPSS Clementine, IBM DB2 Intelligent Miner for Data or Angoss KnowledgeSTUDIO. When they have well-defined problems requiring predictive analysis, enterprises should choose an analytic application from the marketing automation, customer relationship management and enterprise application vendors containing predictive analytics. In particular, enterprises should plan on obtaining the predictive technology wrapped in the application to address specifically delimited problems in customer buying behavior, upselling and cross-selling, campaign management, customer profiling, personalization and promotional applications.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access