Continue in 2 seconds

Data Diversity: Let It Be

  • June 01 2002, 1:00am EDT

One of the best things to ever hit the old information technology function must be Web services in support of enterprise application integration (EAI). For a while, we thought we were going to have to replace all of our legacy application systems and data, but no! We can now leave those old systems in place (many of which are running just fine, thank you) and use middleware to expose these systems and their data to the Web. EAI involves software used to integrate applications and processes within an enterprise or across enterprises. EAI is achieved by developing integration standards that facilitate interaction among disparate systems without the need for custom point-to-point application integration work. Data can remain in diverse formats yet be integrated with other relevant data to enable business transactions and decisions. Diversity of data is no longer something to be overcome or reengineered, but rather something to be leveraged.

However, EAI relies on knowing what data is where, which brings in our old friend meta data. Meta data has always been important, especially in data warehouse creation, as a passive guide to the pieces of data that are critical to an enterprise's continuity. Meta data has been housed in meta data repositories and has been important for application developers, business analysts and even users to understand what a piece of data means, where it comes from, how it's calculated and who uses it. However, the problem has been its passivity.

With EAI, meta data now assumes an even more critical role. EAI is dependent on all applications participating in a solution being willing to use the same business semantics. Additionally, with EAI, meta data must assume an active role. Data about the data needs to be sent within the message, along with data about the source and target applications related to the message. Meta data must enable the correct routing and transformation of data between applications.

A fancy term that is being used to describe meta data for EAI is "ontology." An ontology provides a formalized view reflecting a common understanding of data within a domain. Ontologies do what "enterprise data models" did previously. Ontologies explain entities, attributes and relationships. However, instead of just being a record of this information, ontologies provide a machine-readable common semantic layer that is active and can help actually route data between applications.

It sounds great, doesn't it? But the main problem with implementing an ontology within an EAI framework is complexity. We all know how complex it is to define the data that is important to an organization. It's difficult to define in the first place, difficult to maintain over time and difficult to understand. Developing an EAI ontology may seem too difficult an undertaking ­ yet we can't "not do it." It simply must be done. Technologies and products will be emerging to help us build ontologies and dynamically mediate discrepancies in business definitions, yet it will be enforcing the discipline of dealing with complexity within our organizations that will be the critical success factor. Not having an ontology will be worse than having one that isn't perfect.

This whole idea of EAI is relatively immature. Meta data models must be developed based on incomplete examples and guidelines available from industry resources. The ties between ontologies and business process models must be defined. Then integration brokers can use this meta data to route and transform data between systems. Interface definitions can be reused. Messages used to transmit entries to an operational system can be copied and delivered separately to an analytic application for business activity monitoring. Diverse data can remain diverse, yet be used for different purposes. The essence of EAI is dealing with heterogeneity because homogeneity is often an unrealistic expectation.

No one can "boil the ocean" and develop an enterprise-wide, detailed, full-function meta data repository for application integration. I'm not sure that could ever be achieved. However, application integration is here to stay. Its benefits far outweigh the complete replacement of legacy systems. We need to move forward in developing an ontology to support it ­ step-by-step, application-by-application and project- by-project. The net result of dealing with diverse data via an EAI ontology will be lower costs and improved functionality, and we will extend the life of our legacy systems by maintaining their independence but leveraging their data. Isn't that the goal of data diversity? Our organizations can speak words of wisdom if we let the data be.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access