The Build-and-Ready Approach
Information Management Magazine, October 2007
In today's complex computer application systems, customer data is spread widely across legacy and newer generation programs. With each new software application release, the challenges to consolidate customer data grows. Migration of the older legacy data into the newer applications without creating duplicates or bad records is critical to a successful release. As the number of data sources increases, so does the necessity to easily obtain a single enterprise view of customer information.
Source systems that store customer data are often composed of a mix between various network and computer technologies such as mainframes, Windows, Novell and security domains (LAN and WAN). Not all networks or their computers are able and have access to exchange required data easily between them. In order to create a successful customer data integration project, the ability to get data from various disparate systems is crucial. By utilizing a service-oriented architecture (SOA) approach, you can enhance your applications and enable them to share their silos of customer data. Building and exposing Web services allows a method for systems to send data to a common end point for processing.
Many projects around customer data integration (CDI) attempt to merge all of the data together from the various source systems before the system can be considered a production process or application. This approach is often difficult to achieve, especially when you have data that must be retrieved from various source systems and technologies. Each source system will have to be modified to query, extract and transfer the data to the integrated data repository. Often these projects will never receive funding due to the long delivery timeline required to modify each source system to retrieve the required data set. This reduces the ROI on the project because the systems cannot be utilized until all source systems are integrated.
Advertisement
Instead of creating all-or-nothing approaches to achieving CDI, I propose a build-and-ready approach. In this architectural style, you first determine what data you are trying to consolidate. In order to achieve this, think about the end goal of the data you need and then map to the originating source systems. You can create the required Web services to encapsulate appropriate business logic in order to process the data into the destination format. This allows for the transformation and loading of the data into a consolidated data store. Therefore, you now have a process, and the required building blocks have been built and are ready for use by your applications.
Once an agreement is made with the business that CDI is important and should be pursued, a plan can be created on how to implement the build-and-ready approach. The initial business requirements will lead to the selection of the customer data elements. Because the output of this effort is a customer data store and not an application, you have options on how to get the Web services built. The first would be building the Web services as part of a specific CDI project. For example, a project is sponsored to collect and cleanse customer data. As part of this project, the Web services are built to process the data. These Web services will then be available to the enterprise for use in future projects. This would be the ideal situation if funding for a project can be approved. The second option would be to build the required Web services as a standalone project. This requires selecting the basic customer data elements and building a foundation framework to process the data. As applications can be modified, the Web services are available for use by the enterprise. This would be more of an IT project, which allows the Web services to be built and does not get included in a specific application project.
One of the biggest challenges to an all-or-nothing approach is getting the applications working in unison, for instance, making all required application changes to allow the data to be submitted. Then you must coordinate the release schedule to make these changes in the production environment. Until all of the production applications are updated, the system cannot accept customer data for processing. Due to these complexities, it is more efficient to create an adaptable process that allows applications to feed the data as it is modified.
There are many projects that will be able to take advantage of a build-and-ready approach. Because the Web services are built and deployed for use, new projects can utilize the Web services to submit customer data into the new standard integrated environment. Likewise, they may even be able to leverage the new integrated data as the source data for a new application. Existing applications can be modified to utilize the Web service for sending or receiving customer data. This can be done as part of a regular maintenance cycle or application enhancement project.
In addition to system applications, external data can also be merged with internal data. Customer data that is purchased or cleansed outside of your organization can use the Web services to facilitate merging this data. Data resulting from mergers or acquisitions is also able to supply customer data to the Web services to quickly consolidate data into the integrated customer data repository. Because data from various sources, internal or external, can utilize the Web services, this flexible approach to integrating customer data enables you to build a quicker view of enterprise customer data.
In order to track back to the source systems, a CDI project needs to contain a legacy cross-reference application. This application allows users to know what the new global customer ID is now as well as what the customer ID was in the previous systems. This is often useful when portions of the original source system are still used. It provides users and other applications with the ability to query the source system and retrieve any required data.
In order to achieve a global customer ID, all previous instances of a customer must be associated with the new global ID. Initially this means updating the tables to store various versions of a customer. It also requires maintenance processes to ensure that new customer records are associated properly.
Page 1 of 2.
