How MDM and Data Services Platforms Can be Complementary

Accessing and integrating quality data is the next step in SOA

Information Management Special Reports, December 18, 2008

Eric Samson

As the service-oriented architecture model matures, data services and data services platforms are the emerging approach to solving the specific problem of data access in SOA. SOA applications tend to access multiple, heterogeneous data sources, adding an integration concern on top of the data access problem itself. In this context, master data management solutions can contribute nicely to solving data inconsistency issues.

The initial wave of SOAs had a profound impact on all software components. Enterprise services buses have emerged as the first visible foundation component around which other components can be integrated. Plenty of tools now exist to design, develop, deploy and compose services together in order to achieve more flexible information systems.

The second wave in SOA addresses data access and integration. Data services are specialized services that aggregate data coming from heterogeneous data sources and published in the form of reusable, business-friendly views, which can later be consumed by any client and used to manipulate data. DSPs are dedicated servers providing a robust means of hosting and administrating data services. The use of traditional data access application program interfaces prevents software architectures from getting all the expected service benefits, including flexibility, reusability and reduced deployment cycles. Data services and DSPs, on the other hand, enforce the specific characteristics of SOA: disconnected and asynchronous access, integration of multiple data sources, support for multiple client technologies and data exchange through XML.

One of the most important characteristics of data access in SOA is that it must integrate data obtained from multiple data sources, rather than typically accessing a single data source or silo of data like traditional applications. This is because one goal of SOA is to accelerate business by better interconnecting the various subsystems of the enterprise. New business applications typically access a large number of data sources. This trend is also reinforced by the SOA model itself, which tends to access applications through multiple business services instead of accessing their single database. The next generation of packaged applications (enterprise resource planning and customer relationship management) be delivered as services; even some databases are externally hosted and accessed by way of services now.

The moment simultaneous access of multiple data sources is considered, the question of data quality must be taken into account. Historically, MDM systems have been one solution to that problem. Some MDM vendors have already made attempts to turn their offer into SOA-enabled components.

Data quality is not necessarily a challenge for all business applications in SOA; often the metadata defined in the DSP is sufficient to determine which data to use and how to transform it. This is why it is not expected that all DSPs embed their own MDM solution. But in other situations where MDM functionalities are required, multiple integration solutions exist.

The first and most direct solution is for DSP programmers to write their own basic data quality features. Some DSP vendors provide sophisticated mechanisms to transform the raw data coming from the data sources, and this can support data cleansing features. While this approach is simple, it has several drawbacks, such as the ongoing maintenance of these MDM features; it’s more a development approach than a declarative one and runs the risk of data quality policies being spread in the data-access layers. It is unlikely that DSP developers will have time to design and code into their own homegrown data quality layers - the sort of robust mechanisms (such as versioning) found in mature MDM solutions.

A variation of the previously mentioned solution would be to use some MDM solutions available as Web services. Some vendors are starting to make several data quality features available on the Web (such as address reformatting, address checking and name deduplications, etc.), and DSPs can typically issue calls to these external services. However, manually composing these external data quality services could prove a tedious task. Certain DSP vendors offer the ability to dynamically compose services at runtime when needed - a feature that could greatly help in this effort.

A third approach consists of a technical and sales collaboration between a DSP and MDM vendor, with both working together to provide a well-integrated solution to their customers. This kind of integration could be easy to put in place between vendors; but, in fact, the MDM solutions are not enforcing any kind of standardization. This is in contrast to the DSP vendors; which are working together on defining standards for data access in SOA such as service data objects, defined as part of OASIS and data access service.

The lack of standardization among MDM vendors is probably due to the currently available MDM solutions originating from very different kinds of technologies, ranging from cleansing of customer data, products catalogs and bill of materials management, configuration management and versioning of data models as well as repositories of reference data. This makes integration of MDM solutions fully dependent on proprietary concepts, features, APIs, tools and file formats. The upside of integrating DSP and MDM technologies is that it could provide organizations with best-of-breed solutions for their particular requirements. It is interesting to note that some MDM vendors already provide access to their reference data and metadata through data services, suggesting a de facto market demand for integration of both technologies.

The fourth and final approach to integration entails a fully integrated MDM solution within a DSP delivered by a single vendor. This could be achieved through embedding an MDM solution into a DSP. MDM is not generally expected to be a specific component of a DSP, as a need exists for MDM solutions outside the context of a DSP. For those businesses able and willing to commit themselves to a particular vendor’s overall model, we can expect this greater level of integration to provide a smooth process for data management, including data quality steps through common metadata, unified tooling and shared