The customer data integration (CDI) market is in a state of considerable flux as a multitude of diverse technologies compete for large-scale, mission-critical projects (see Figure 1). Analyzing data from a 150+ project database containing the vital signs of Global 2000 CDI projects, the CDI Institute is able to project future trends in mind share as well as current market share. During 2005, the average CDI software investment was $1.2 million with the typical large scale CDI project requiring systems integration (SI) fees ranging four to six times the amount spent on the CDI software. This demonstrates the increasing importance of application integration via master data management (MDM) as a catalyst for realizing ROI in large enterprises' multimillion-dollar customer relationship management (CRM) installations. Moreover, CDI is clearly one of the few remaining growth areas for both software vendors and systems integrators.

Figure 1: CDI Evolution (Source: CDI Institute)

What is CDI?

The CDI market is comprised of process and technology solutions for recognizing a customer at any touchpoint - while aggregating accurate, up-to-date knowledge about that customer and delivering it in an actionable form "just in time" to touchpoints. Both IT vendors and executive IT management at Global 2000 enterprises need guidance in this fast paced, high stakes market that is the convergence of multiple overlapping middleware markets - e.g., customer recognition, data quality, real-time analytics, data warehouseing, business process management, enterprise application integration (EAI), etc.

Clearly, CDI is an increasingly critical business strategy as well as a competitive differentiator for the world's largest software vendors (Oracle, SAP, Siebel) with a flurry of best-of-breed vendors (DWL, Initiate Systems, Siperian), systems integrators (Accenture, Alliance, BearingPoint, IBM Global Services) and data service providers (Acxiom, D&B, Experian) all chasing this market.

While most enterprises have infrastructure initiatives based on the technology platforms of strategic IT partners such as Oracle/PeopleSoft, SAP and Siebel, more than 75 percent of the IT professionals surveyed by the CDI Institute are actively considering purchases "outside the family" to facilitate connectivity between customer-facing applications and processes. CDI strategies systematize a panoramic or 360-degree view of the customer by aggregating and analyzing multiple sources of master customer information into a master "system of record." Such "master data hubs" represent the holy grail of CRM products from Microsoft, Oracle, Siebel and SAP as they enable a single version of the truth across the enterprise's customer-facing processes. It is now a realizable goal for IT organizations to buy rather than build such infrastructure, with more than 95 percent of financial services and life sciences enterprises actively looking to replace homegrown CDI solutions. Additionally, the spate of mergers and acquisitions occurring in the communications service provider industries (telco, cable, satellite) is further creating a need to integrate customer-facing databases and processes as the newly minted mega telcos look to gain operational efficiencies and competitive marketing advantage via mergers. Lastly, many IT professionals are clearly looking to such long-term and large-scale IT initiatives as "career longevity insurance," given that such enterprise-scale infrastructure is not amenable to offshore outsourcing, etc.

What are the Major CDI Implementation Styles?

The diverse nature of global business models requires variety in CDI implementation approaches or models (see Figure 2).

Figure 2: Top Five CDI Architectural Preferences within Global 2000 Enterprises (Source: CDI Institute)

Contemporary CDI solutions will vary by industry, for example, in terms of tactical approaches taken:

  • Pharmaceutical/life sciences adopt semi-batch, database-centric approaches to deploy master physician data to sales forces.
  • Financial services providers and online retailers require near real-time, business- and process-centric solutions to compete in the fast-paced B2C online world.
  • Governmental organizations are often restricted in the ways they are permitted to merge and analyze data on their citizens, thus skewing these architectures toward anonymous entity resolution.

For planning purposes, it is often important to know where you came from as well as where you are going. This holds true for an enterprise CDI strategy per the marketing mantra often recited by CRM vendors: CRM is a journey - an ongoing process - not a product or a way station. In that spirit, we outline the basic road map of CDI solution evolution (see Figure 3):

  • First-Generation CDI. These solutions are most often standalone databases or files managed by monolithic applications such as IBM's Transaction Processing Facility (used by airlines), CSC's Hogan banking system or even early generation Siebel Systems' sales force automation installations.
  • Second-Generation CDI. These solutions are "uni-modal" in that they are specialized for either batch or online updates and are database-centric master customer files. Frequently participating in bidirectional updates, these are either "data-centric" in using ETL (extract, transform and load) solutions or "process-centric" in depending heavily upon workflow.
  • Third-Generation CDI. These approaches offer "multi-modality" and are flexible enough to support batch and online performance, loosely and tightly coupled application models, etc. Specifically, this variation supports a service oriented architecture (SOA) model in terms of Web services for a "business services" approach. This approach also increasingly provides support for the federation model whereby individual business units retain some autonomy in managing their "master" data assets yet participate in a "union" or "confederation." Additionally, a third-generation solution must support the tremendous scalability requirements, as well as reliability and availability, associated with mission-critical infrastructure.
  • Fourth-Generation CDI. These systems provide extreme scalability because they are of the architecture required to support massive numbers of users such as business-to-consumer e-commerce or self-directed service centers. The systems also are designed to support unstructured content such as free-form text, video and audio in addition to structured database records.

Figure 3: CDI Solutions Genealogy (Source: CDI Institute)

Market Reality

In early 2005, each of the major application package vendors (Amdocs, Oracle, SAP, Siebel) updated their basic architectures to add support for service oriented architectures as well as "multi-modality" flexible enough to support both batch and online performance, in addition to loosely and tightly coupled application models. They joined the best-of-breed vendors in the category we'll call "third-generation/hybrid CDI." Furthermore, each added the ability to fine-tune performance and availability by user-tuning of the amount of master data persisted in the central hub. While some of these updates consisted of the fine-tuning of existing software, SAP's case was an example of an "extreme makeover" as the vendor shelved its prior release of SAP Master Data Management (MDM) and elected to place its architectural bets on the A2i software foundation (essentially product information management) product going forward.

During mid-2005, Cast Iron Systems (application router), Centerboard (virtual operational data store) and Ipedo were among the handful of CDI vendors proffering "CDI in a box [rack mount]" in the sense that they offered a utility software or hardware "appliance."

Fourth-Generation CDI

When SAP releases the next iteration of SAP MDM and Oracle releases its Product Data Hub, we can expect to see much greater support for unstructured information such as catalog images like that  provided by IBM's WebSphere Product Center (formerly independent software vendor Trigo). As the CDI landscape evolves to include more support for such master customer entities as product and supplier, the ability to integrate well with the unstructured content of product information management (PIM) tools associated with modern supply chains will become increasingly important. The relentless year-over-year growth in e-commerce will further dictate the need to integrate product content with customer processes as businesses are mandated to be able to properly present their product on the Web as well as in print and at kiosks.

Futures for CDI and MDM

Despite the modest uptake in BPEL as a standards-based approach to business process management (formerly known as workflow), it is still quite difficult to coordinate the process chains required for complex transactions that span application subsystems within the enterprise, let alone across the enterprise's extended supply chain. Nonetheless, vendors such as Oracle, SAP and Siebel will continue to drive up the food chain as they move their attempts to dominate the enterprise application infrastructure from data model-based hegemony to one based on process models (hence SAP's extrovert championing of NetWeaver). Such emphasis on process will further force partnerships (even mergers such as TIBCO and ObjectStar) as enterprise application integration (EAI) vendors and CDI vendors look to bolster their stories in data and process models.

Additionally, all CDI vendors will ratchet up their support for the complex hierarchies that represent the business entities of the corporate world - not just consumer householding as hierarchies. As corporate business models become increasingly distributed and sophisticated, the notions of global data synchronization and federated meta data will become increasingly vital.

Furthermore, during 2005 and 2006, the ETL vendors will continue to add near real-time capabilities to their (currently) primarily batch products - through either product enhancements or partnerships. Conversely, EAI vendors will add more robust batch capabilities to their primarily online transaction processing (OLTP) engines. Arguably, either of these two middleware capabilities can be used to build out master data synchronization. However, such vendors will face major competition on two fronts: the mega application suite vendors (e.g., Oracle Customer Data Hub, SAP Master Data Management, Siebel Universal Customer Master) and a new set of vendors focused on heterogeneous support while providing features not found in EAI and ETL.

By 2006/2007, relentless evolution of middleware will continue as developments in federated query processing, the XML Query (XQuery) standard, and business requirements for convergence of unstructured and structured information stores will force more consolidation within the middleware vendor ranks. By 2007/2008, continued decline in memory and CPU prices will foster a new set of database capabilities suitable to aggregating diverse information sources in real time as a standard feature found within mainstream database oligopoly (IBM, Microsoft and Oracle).

Strategic Initiatives

Given the marketing ramp-up by companies such as Oracle, SAP and Siebel, it has quickly become boardroom-level knowledge that CDI and MDM are "strategic" initiatives that deserve serious scrutiny - and soon. 

Aaron Zornes is founder and chief research officer for the CDI Institute ( with headquarters in San Francisco. Prior to the CDI Institute, Zornes founded and ran META Group's largest research practice for 14 years.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access