What's New in Data Quality Trends Related to CDI/MDM?
Many enterprises fail to consider the quality level of their customer or other master data (and its trusted sources) until very late in the process. As a result, many CDI/MDM implementations are duly jeopardized or even fail because of data quality issues - most of which are not identified until very late in the process. Moreover, it is more critical than ever that in a so-called "master" data system, organizations teach users how to understand the meaning and lineage of the data that is managed by the CDI/MDM system.
Clearly, DQ processes are critical to achieving value from CDI/MDM and to ensuring that the output of such infrastructure is well understood and trusted by the business.
During 2006-07, independent DQ vendors (e.g., Dun & Bradstreet, Firstlogic, PitneyBowes/Group1 and Trillium Software, et al) will struggle to compete against better-funded match/merge and data profiling capabilities increasingly integrated with megavendor CDI/MDM solutions (e.g., IBM Customer Center with WebSphere QualityStage, Oracle Customer Data Hub with Data Librarian). This scenario will be exacerbated by the consolidation trend of database vendors acquiring or building out their own DQ capabilities as part of their move into the CDI/MDM market. While both Oracle Customer Data Hub (and its recently acquired Siebel Universal Customer Master) and IBM's WebSphere Customer Center (formerly DWL Customer) will continue to have links to third-party DQ tools, there will be increasingly less demand for such third-party tools as the CDI/MDM products begin to better integrate with the DQ tools of their respective CDI/MDM vendor (e.g., Oracle Data Librarian, IBM WebSphere QualityStage, SAS/DataFlux). This has been the gloomy scenario which extract, transform and load (ETL) has faced as better than "good enough" ETL capabilities arrived in Microsoft Data Services, Oracle Warehouse Builder and SAS Warehouse Studio.
Additionally, the greater adoption of DQ through mega vendor CDI/MDM solutions is an area of opportunity for those DQ vendors whose strategy is specifically aligned with expanding the CDI/MDM ecosystem among enterprise software vendors.
Suffice to say, there are three compelling trends which every DQ solution vendor must keenly focus on during 2006 to ensure their very survival, let alone prosperity:
- Inherent support for SOA and XML Web services
- Greater explicit support for compliance solutions, and
- Stronger focus on integration accuracy post-DQ infrastructure installation.
Inherent Support for SOA and XML Web Services. Currently, no single CDI/MDM solution can offer the full range of services required for a robust CDI/MDM ecosystem - most notably, the independent CDI/MDM solutions all partner with multiple DQ providers for match/merge, data cleansing or data profiling. Clearly, in such a heterogeneous environment the notion of service-oriented architecture (SOA) and XML Web services not only increase overall CDI/MDM solution flexibility and the ability to more effectively support near real-time solutions, they also make it much easier to mix and match best-of-breed solution components from multiple vendors in the CDI/MDM ecosystem. This is extremely compelling because it enables IT organizations to more easily benefit from a wider range of DQ solution options.
Note that most DQ vendors (e.g., Firstlogic, IBM WebSphere QualityStage) have met this requirement whereby their output can be consumed in an SOA binding or loosely coupled architectures. Additionally, CDI/MDM solutions such as IBM WebSphere Customer Center (formerly DWL Customer) currently provide for DQ services to be directly consumed by their business process layer. Moreover, the trend is for even more integration between DQ and CDI/MDM, e.g., optimized interfaces with prebuilt templates and soft content.
Greater Explicit Support for Compliance Solutions. Compliance deadlines forced many companies to quickly create yet another one-off siloed application to address such specific needs. These freshly funded DQ silos must eventually be repatriated into the enterprise IT infrastructure, rather than compounding DQ issues by introducing separate islands of data quality. Overall, the DQ vendors recognize this need and its vital importance (in addition to an opportunity for a fresh upgrade sale of their DQ platform) in providing cleaner, more turn-key support for compliance solutions that enable DQ services to be more seamlessly integrated into the overall enterprise CDI/MDM architecture.
Stronger Focus on Integration Accuracy Post-DQ Infrastructure Installation. For many businesses, tremendous DQ-driven ROI will come from improving CDI/MDM accuracy. The more mundane ROI opportunities have already been exhausted for address cleansing and basic record matching in many large enterprises. During 2006-07, CDI/MDM accuracy offers the next greatest boost to ROI because it enhances the ROI not only of the CDI/MDM system directly, but also all other aspects of the business process that rely on an accurate "single customer view." This includes such key ROI areas as marketing (campaign modeling, data mining, analytics) and compliance (CAN-SPAM, USA Patriot Act, HIPAA, etc). Initiate Systems is one of the first vendors to truly embrace match accuracy as a competitive differentiator, as indicated in their marketing message.
Is There a Future for the Standalone DQ Vendor?
DQ solutions providers can maintain their best-of-breed edge by focusing on stronger international support while providing more complete and comprehensive locality data. Additionally, stronger emphasis on industry-specific vertical solutions will help establish defensible/dominant market segments that can better be protected against the generic megavendors. Because actual DQ feature/functions may see little in technology advances or breakthroughs, DQ vendors would be wise to invest more in domain-specific partners and marketing - e.g., by adopting industry-specific terminology and other domain content.
