Related Links

Web Seminars

Malcolm would like to thank Fabio Corzo for his contribution to this article.

Trying to deal with enterprise information architecture is not easy. Most data practitioners have a hard enough time getting through one project at a time. Quite often they are working on several projects simultaneously as well as helping out with maintenance activities. Enterprise architecture is clearly bigger than any of this, but with so much busywork, it is difficult even to think about it. Furthermore, we are confronted with environments where the architecture is pretty much set, having evolved through the implementation of sets of legacy applications and the acquisition of vendor-supplied package solutions. Unfortunately, while none of this may be easy to deal with, architecture really does matter. It emerges as an issue in many endeavors, but it is especially important where data sharing or exchange happens. The past decade or so has seen an explosion in the implementation of business intelligence (BI), which can be viewed as extracting useful information from the data generated by the operational systems of an enterprise, possibly augmented by data brought in from outside.

A lot of people are unhappy about the results of their BI experiences. One of us (MC) teaches extensively about master data management, and a large segment of the audiences consists of project staff working on data warehouse or data mart projects whose outputs are unusable by the business. These BI applications are no doubt technically sound, but what they deliver has serious deficiencies. Because these deficiencies are inherent in the data being brought into the marts, the project staff tries to fight their way upstream, in terms of where the data comes from, to correct it. This heroic effort resembles some of the early European explorers trying to find the source of the River Nile. Actually, it is a lot more difficult. Even if our intrepid data mart builders could map out the terra incognita of the data landscape, they are probably going to come across great seas of polluted data they will be powerless to clean up, but which are still sources they must use. Perhaps they can partially decontaminate the stuff as it goes into the marts, and to that extent the knowledge gained from their explorations can be useful. However, this is far from solving the whole problem.

Architecture is the Culprit

One of the issues about enterprise information architecture is being able to think about it clearly - to conceptualize it or visualize it. We try to employ useful analogies, like the one just presented about exploring the data landscape. The reality is that enterprise information architecture is complex and consists of many different dimensions, each of which is a valid concept in its own right. This means it maps partially to a wide range of analogies. Unfortunately, no single one of these corresponds to the complete reality of architecture, and we are just going to have to build out our understanding of it one faltering step at a time.

That said, it is possible to ask the question if enterprise information architecture bears significant responsibility for failures in BI. Is it to blame for why rollups will not roll up, for why monthly reports that seem stable suddenly show weird blips of unbelievable data once in a while, for why ad hoc queries can never be relied on, and all the other BI misfortunes that we do not like to even name for fear we will conjure them into existence in our projects

We would answer that it is. Figure 1 shows what we term the Abstraction-Translation Paradigm of enterprise information architecture. In a nutshell, it visualizes the processes by which applications are created as passing through successive layers of abstraction and translation that ultimately enables business information to be stored as binary representation of facts in implemented databases. This data can then be moved into data marts for use in BI applications. However, it is still a binary representation of facts. To be used, it has to be transformed and translated through the same layers required in the creation of the operational application that produced it. Finally, it emerges back at the business level as information again. Given the complexity of this round trip for information, we should be amazed that BI applications ever produce anything useful, rather than disappointed that they have shortcomings.

The Abstraction-Translation Paradigm

The leftmost stack in Figure 1 represents the layers of analysis and design needed to build an operational application, and the roles played by the individuals who are responsible for what happens in each layer. It is divided up into a column for data and another column for business rules. These end up as physically implemented databases and application logic, respectively.

The process of implementing an operational application begins in the business, where a business analyst gathers business requirements, describes the current business processes and identifies the data used in these processes. The artifacts that are produced ideally include use cases, workflows and a conceptual data model with a glossary of business terms. These represent the business in what is termed the conceptual layer in Figure 1. Unfortunately, “conceptual” has all kinds of definitions, but here it is taken to mean a direct representation of the business.

Next, on the data side, a data analyst produces a data model. This contains an element of design. Even if it is for a current state, data modelers will always say they are producing a picture of how the business truly “sees” the data. Whatever truth there is in this assertion, we see business concepts such as vendors, customers and employees abstracted into “party” entities, inventions of surrogate keys, awkwardly named association entities and so on. We have definitely left the conceptual level and entered the logical level here.