As the service-oriented architecture model matures, data services and data services platforms are the emerging approach to solving the specific problem of data access in SOA. SOA applications tend to access multiple, heterogeneous data sources, adding an integration concern on top of the data access problem itself. In this context, master data management solutions can contribute nicely to solving data inconsistency issues.
The initial wave of SOAs had a profound impact on all software components. Enterprise services buses have emerged as the first visible foundation component around which other components can be integrated. Plenty of tools now exist to design, develop, deploy and compose services together in order to achieve more flexible information systems.
The second wave in SOA addresses data access and integration. Data services are specialized services that aggregate data coming from heterogeneous data sources and published in the form of reusable, business-friendly views, which can later be consumed by any client and used to manipulate data. DSPs are dedicated servers providing a robust means of hosting and administrating data services. The use of traditional data access application program interfaces prevents software architectures from getting all the expected service benefits, including flexibility, reusability and reduced deployment cycles. Data services and DSPs, on the other hand, enforce the specific characteristics of SOA: disconnected and asynchronous access, integration of multiple data sources, support for multiple client technologies and data exchange through XML.
One of the most important characteristics of data access in SOA is that it must integrate data obtained from multiple data sources, rather than typically accessing a single data source or silo of data like traditional applications. This is because one goal of SOA is to accelerate business by better interconnecting the various subsystems of the enterprise. New business applications typically access a large number of data sources. This trend is also reinforced by the SOA model itself, which tends to access applications through multiple business services instead of accessing their single database. The next generation of packaged applications (enterprise resource planning and customer relationship management) be delivered as services; even some databases are externally hosted and accessed by way of services now.
The moment simultaneous access of multiple data sources is considered, the question of data quality must be taken into account. Historically, MDM systems have been one solution to that problem. Some MDM vendors have already made attempts to turn their offer into SOA-enabled components.
Data quality is not necessarily a challenge for all business applications in SOA; often the metadata defined in the DSP is sufficient to determine which data to use and how to transform it. This is why it is not expected that all DSPs embed their own MDM solution. But in other situations where MDM functionalities are required, multiple integration solutions exist.
The first and most direct solution is for DSP programmers to write their own basic data quality features. Some DSP vendors provide sophisticated mechanisms to transform the raw data coming from the data sources, and this can support data cleansing features. While this approach is simple, it has several drawbacks, such as the ongoing maintenance of these MDM features; its more a development approach than a declarative one and runs the risk of data quality policies being spread in the data-access layers. It is unlikely that DSP developers will have time to design and code into their own homegrown data quality layers - the sort of robust mechanisms (such as versioning) found in mature MDM solutions.
A variation of the previously mentioned solution would be to use some MDM solutions available as Web services. Some vendors are starting to make several data quality features available on the Web (such as address reformatting, address checking and name deduplications, etc.), and DSPs can typically issue calls to these external services. However, manually composing these external data quality services could prove a tedious task. Certain DSP vendors offer the ability to dynamically compose services at runtime when needed - a feature that could greatly help in this effort.
A third approach consists of a technical and sales collaboration between a DSP and MDM vendor, with both working together to provide a well-integrated solution to their customers. This kind of integration could be easy to put in place between vendors; but, in fact, the MDM solutions are not enforcing any kind of standardization. This is in contrast to the DSP vendors; which are working together on defining standards for data access in SOA such as service data objects, defined as part of OASIS and data access service.
The lack of standardization among MDM vendors is probably due to the currently available MDM solutions originating from very different kinds of technologies, ranging from cleansing of customer data, products catalogs and bill of materials management, configuration management and versioning of data models as well as repositories of reference data. This makes integration of MDM solutions fully dependent on proprietary concepts, features, APIs, tools and file formats. The upside of integrating DSP and MDM technologies is that it could provide organizations with best-of-breed solutions for their particular requirements. It is interesting to note that some MDM vendors already provide access to their reference data and metadata through data services, suggesting a de facto market demand for integration of both technologies.
The fourth and final approach to integration entails a fully integrated MDM solution within a DSP delivered by a single vendor. This could be achieved through embedding an MDM solution into a DSP. MDM is not generally expected to be a specific component of a DSP, as a need exists for MDM solutions outside the context of a DSP. For those businesses able and willing to commit themselves to a particular vendors overall model, we can expect this greater level of integration to provide a smooth process for data management, including data quality steps through common metadata, unified tooling and shared administration.
Now that we have seen how MDM and DSP solutions might be integrated, lets have a look at a few interesting use cases. An MDM vendor could embed a DSP into its product in order to deliver a data service layer to access and manipulate their internal reference data and metadata. We have already seen some MDM vendors publish some data services layers around their internal master data repositories. This eases the use of business process management systems, such as BPEL, SCA or workflow engines to define master data, as this definition could typically require actions from multiple kinds of users within the enterprise (IT people, marketing people and sometimes even end users). By using a standardized DSP instead of manually designing, coding and deploying data services, MDM vendors will save development time and can focus on their own features. They will also benefit from all the quality of service features (scalability, security, fault tolerance, etc.) supported by the DSP. This approach will enable the vendor to offer a wider range of data access programming interfaces (such as SDO, ODBC, JDBC, etc.) to their own users, depending upon what is supported by the DSP they have selected.
Conversely, DSP vendors could rely on an MDM solution for the management of their own metadata. This would add typical MDM advantages - governance and versioning - to their metadata. This might become an important benefit to large organizations as they begin to deploy more DSPs.
Another use case that naturally comes to mind is the possibility of accessing and manipulating master data repositories from a DSP as a new kind of supported data source. The natural ability of DSPs to work with multiple heterogeneous data sources makes access to the reference data much easier. Even if the master data is available as a table in a database or as a service layer, there is a need to have specific support for it in the DSP. That is because the master data often serves as a point of entry or as an intermediate lookup table, and additional transparent navigations will be required to access the actual or complementary data. Some DSP implementations support dynamic composition of data services and can combine successive calls into an execution plan at runtime. In this case, there is no need to predefine all possible combinations of data services at design time; the system will compose the required data services on its own on an as-needed basis. This kind of advanced DSP feature is extremely efficient in this use case, as the master data access steps will be transparently integrated into the global data access plan.
The last example we could consider regarding the added value of a DSP and MDM solution working together is the use of a DSP as the synchronization layer for MDM. Usually, MDM solutions provide a way to define the master data models and maintain the data, but they dont really provide a solution to synchronize updates between the mater data repositories and the operational data sources. This problem exists whether the updates are originally made in the master data and then replicated to operational data stores, or vice versa. Addressing this synchronization challenge, which could conceivably be quite complex to manage, falls on the development teams or system integrators. A DSP can be used here, probably in conjunction with other components (such as an ESB, for instance) to simplify the process of detecting changes and replicating them in the various heterogeneous subsystems.
These are just some examples of how the DSP and MDM models can complement each other. In the last two cases, the DSP can greatly simplify the adoption and deployment of MDM solutions within the enterprise.
MDM solutions have evolved so that they can be integrated into SOA environments. It is interesting to see that both MDM and DSP vendors often use the same classical M&A example to illustrate the business benefits they bring. Some basic integration of MDM and DSP can be easily built through consulting. Whatever approach is taken, the most valuable will require tight coupling of the two, so that both technologies can benefit from the advanced features of the other and can share some common components, such as repositories, metadata and tools.
As key components of global service-oriented enterprise information management, MDM systems and DSP working in conjunction with one another enhances their value well beyond the sum of the two individual parts.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access