This column is adapted from the book Universal Meta Data Models by David Marco & Michael Jennings, John Wiley & Sons, 2004.
In part two of this five-part series on Allstate's managed meta data environment (MME), I walked through their MME's overall technical solution and the role of data stewardship at Allstate (see the March issue of DM Review). In this installment, I will discuss the details of the meta data sourcing layer component of their MME.
Allstate has many different meta data sources that are brought into their MME. These sources include: logical data models, physical data models, codes, logical-to-physical data mapping and messaging for EAI.
Logical Data Models: One primary means of populating meta data in Allstate's MME is through the process of logical data modeling. Application development teams engage (and fund) the data administrator to help identify and model the entities, attributes, relationships and constraints on their data. They also create English-readable names and definitions for entities, attributes and nonenumerated domains. A data model is then created in one of the popular modeling tools, extracted and "checked into" the MME.
To accomplish the import, Allstate defined an XML schema that represents the data in the repository. An import utility was created to take that XML schema and load it into the repository. Whenever they need to get data into the repository, they just create a file or extract one from another tool in their standard format, and then the load can be easily accomplished.
Physical Data Models: Technical meta data describing the physical structure of data is extracted directly from the source database systems files. This process is relatively straightforward because an extraction of meta data (database, table, column and keys) is made from the database system files. Then, it is loaded into the MME through the meta data integration layer.
Codes: Another major source of meta data is from the codes analysts and their use of the codes management system application. The codes analysts research and document any data found to be in codes, creating new enumerated domains to represent the codes when needed, but with a focus on the reuse of existing domains whenever possible.
Codes analysts often find that the same data in two different applications is represented in two entirely different ways. This extends not only to the naming of the fields or columns, but also to both the codes and the values as well. The codes analyst will document the codes and business values used for each of the applications as well as the physical representation of the code for each. These distinct physical representations are called collections.
An enumerated domain can have one or more collections associated with it. The collections are different ways in which the information that a particular domain represents is implemented across the enterprise. The codes analysts will often designate one of the collections as "preferred" and encourage any new use of that domain to use that particular physical representation. The important thing, however, is to ensure, through rigorous definition of the domains, that what is actually the same data is not mistakenly represented by two different domains.
Logical-to-Physical Data Mapping: One of the most manual exercises involved in the population of the MME is the data mapping function for Allstate's data warehouse environment. Mapping is what allows data warehouse users to view what normally would be difficult-to-read physical column names (technical meta data) as easy-to-read logical names (business meta data). It also allows the business definitions for entities and attributes captured in the logical model to be viewed by the data warehouse users.
Mapping is performed by a data warehouse analyst primarily in two cases. The first is to map the physical and logical structures together. Through the MME, analysts simply access the meta data repository to pull up a list of tables on one side of the screen and a list of entities on the other, and proceed to match table/columns with entity/attributes. This "tie" is then captured in the MME. The second type of mapping is where table join keys are identified in different tables and "mapped" together.
Messaging for EAI: In any Global 2000 company or large government agency, system integration is always a major concern. Over the years, Allstate has successfully implemented many point-to-point systems; however, this approach is expensive to maintain.
Now at Allstate, instead of this point-to-point approach, messages deemed to be of interest or significance to the enterprise are now defined, and the meta data about those messages is stored in the MME. This capability allows an existing application to only perform one translation, from its own internal format (for the message content) to the "common" Allstate format for a given event, and it will be able to communicate with all other applications that use the same event. Likewise, a receiving application simply needs to translate from the common format to its own format to process the event. XML is used as the transportation syntax, and XML tag names are assigned to each item of data that the message contains.
The common messages are created through analysis of the two applications seeking to exchange the message with a focus toward the reuse of the message throughout Allstate's other applications. This process ensures consistency and reusability of these common messages. In addition, it leverages the codes information stored in the MME. When two applications exchange a message that contains codes, typically two different coding schemes are being utilized. While the message is being created, a new common code translation is used in these cases. All applications write code that either translates from their collection to the common collection or from the common collection to theirs, depending on their need. This eliminates point-to-point solutions and allows greater reusability of the messages across the enterprise.
Next month, I will present the details of Allstate's meta data delivery layer component of their MME.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access