Continue in 2 seconds

Managed Meta Data Environment: A Complete Walk-Through, Part 7

  • October 01 2004, 1:00am EDT

This column is adapted from the book Universal Meta Data Models by David Marco and Michael Jennings, John Wiley & Sons.

In last month's column, I presented the meta data management layer of a managed meta data environment (MME). That column discussed the following meta data management layer functions: archive, backup, database modifications, database tuning, environment management, job scheduling and load statistics.

In this month's column, I will complete my walk-through of the meta data management layer by discussing the remaining functions: purging, query statistics, query and report generation, recovery, security processes, source mapping and movement, user interface management and versioning.


This part of the meta data management layer defines the criteria for MME purging requirements. Your MME's specific purging requirements and criteria will be governed by its business requirements. As a general rule, meta data that is inaccurate or improperly loaded should be purged; all other meta data should be archived.

Query Statistics

Once the various meta data delivery layer reports and queries are generated, it is important to capture the user statistics associated with the access to these reports and queries. At a minimum, the meta data management layer needs to historically capture:

  • Who is accessing the report or query?
  • What report or query are they accessing?
  • When are they accessing the report or query?
  • How often are they accessing the report or query?
  • How long do they access the report or query?

Query and Report Generation

The reports and queries used in the meta data delivery layer are defined and generated in the report generation section of the meta data management layer. How this is accomplished depends on whether a custom meta data delivery application is developed or a meta data access tool is implemented. This component also needs to manage any publish and subscribe capabilities that are required.


There are many situations that can require a company to recover or reload a previous version of their MME: hardware failure, database corruption, power outage or errors in the meta data integration layer. The recovery process needs to be tightly tied to the backup and archiving facilities of the meta data management layer. Recovery processes may be manually built or utilize existing recovery processes within the meta data integration tool. Both approaches need to be integrated into the organization's existing application recovery processes.

Security Processes

The security processes manage:

  • Creation of new MME users.
  • Grouping of MME users.
  • Setting MME user privileges and profiles.
  • Firewalls/infrastructure.
  • Password management.
  • User location verification.
  • Meta data masking (data level or access level).

The extensiveness of security processes is dependent on the business requirements of the MME. For instance, the security requirements around a Department of Defense or Federal Bureau of Investigation MME are far greater than those of a bank MME.

Source Mapping and Movement

Meta data sources need to be mapped to the correct attributes and entities of the meta data repository. This mapping and movement process needs to be captured and managed in the meta data management layer of the MME.

User Interface Management

This involves the processes for the building, managing and maintaining the Web site (recommended user interface) that the end users will visit to navigate through the MME. Typically, the view (version of the Web site) end users see depends on their security privileges and profiles. A business user would not be interested in looking at program code changes; therefore, it makes sense to not have meta data reports or queries related to highly technical meta data available to the traditional business user through control of role access.


As meta data is changed, added and purged, it will be important for the meta data management layer to historically track (or version) it. There are two common techniques for meta data versioning. The first is the use of date stamps. Date stamping - assigning a date to each row of your meta model entities - allows a firm to make whatever changes are necessary to the meta data while maintaining its historical relevance.

The second technique is to assign version numbers to each row of the meta model. Version numbers are merely generated unique values (e.g., 1.1, 1.2, 1.a, etc.). Version numbers are more limiting than date stamping, which offers more meaning to MME users. Version numbers can be associated with specific dates and times; however, this adds additional complexity to the loading of the meta data and an additional join in the access queries and reports.

Another challenge of versioning is capturing meta data that will become current at some point in time in the future. For example, a new physical table may be moved into the production environment at a future date. To handle these versioning situations, an "effective dated rows" process can be useful. An effective dated rows process allows data to be entered into a group (table) for subsequent processing when the effective date becomes current. Effective dated data can be historical, current or future. Following are the key effective dated rows concepts:

Effective date: The date on which the row of data becomes effective; the date it can be acted on.

Effective status: Allows an application to select the appropriate effective dated rows when combined with the effective date (domain value list: "active" or "inactive").

Effective status comments: The current row is the first row of data with an effective date equal to or less than the system date and an effective status of "active". Only one row can be in this state. Future rows are meta data rows that have a date greater than the system date. Historical rows are rows of meta data that have an effective date less than the current row.

Figure 1 illustrates the effective dated rows technique. In this example, the current system date is January 20, 2004. The meta data row dated "2004-01-27" has an effective status of "inactive." However, once the current date reaches January 27, 2004, the meta data row dated "2004-01-18" will become a historical row and the row dated "2004-01-27" will have its "effective status" changed from "inactive" to "active."

Figure 1: Effective Dated Rows Table

Next month, I will conclude our eight-part MME odyssey by reviewing the meta data marts and the meta data presentation layer.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access