NOV 30, 2007 11:07am ET

Related Links

Innovative Organizations Likely to have More Pervasive BI and Data Governance
September 2, 2014
Revolutionize Your Business Intelligence with Lean, High-Performance Solutions
August 21, 2014
Should You Always Obey Orders from Your Executives?
August 7, 2014

Web Seminars

Why Data Virtualization Can Save the Data Warehouse
Available On Demand
Essential Guide to Using Data Virtualization for Big Data Analytics
September 24, 2014

The Twin Towers of BI Babel


Malcolm would like to thank Fabio Corzo for his contribution to this article.

Trying to deal with enterprise information architecture is not easy. Most data practitioners have a hard enough time getting through one project at a time. Quite often they are working on several projects simultaneously as well as helping out with maintenance activities. Enterprise architecture is clearly bigger than any of this, but with so much busywork, it is difficult even to think about it. Furthermore, we are confronted with environments where the architecture is pretty much set, having evolved through the implementation of sets of legacy applications and the acquisition of vendor-supplied package solutions. Unfortunately, while none of this may be easy to deal with, architecture really does matter. It emerges as an issue in many endeavors, but it is especially important where data sharing or exchange happens. The past decade or so has seen an explosion in the implementation of business intelligence (BI), which can be viewed as extracting useful information from the data generated by the operational systems of an enterprise, possibly augmented by data brought in from outside.

A lot of people are unhappy about the results of their BI experiences. One of us (MC) teaches extensively about master data management, and a large segment of the audiences consists of project staff working on data warehouse or data mart projects whose outputs are unusable by the business. These BI applications are no doubt technically sound, but what they deliver has serious deficiencies. Because these deficiencies are inherent in the data being brought into the marts, the project staff tries to fight their way upstream, in terms of where the data comes from, to correct it. This heroic effort resembles some of the early European explorers trying to find the source of the River Nile. Actually, it is a lot more difficult. Even if our intrepid data mart builders could map out the terra incognita of the data landscape, they are probably going to come across great seas of polluted data they will be powerless to clean up, but which are still sources they must use. Perhaps they can partially decontaminate the stuff as it goes into the marts, and to that extent the knowledge gained from their explorations can be useful. However, this is far from solving the whole problem.

Architecture is the Culprit

One of the issues about enterprise information architecture is being able to think about it clearly - to conceptualize it or visualize it. We try to employ useful analogies, like the one just presented about exploring the data landscape. The reality is that enterprise information architecture is complex and consists of many different dimensions, each of which is a valid concept in its own right. This means it maps partially to a wide range of analogies. Unfortunately, no single one of these corresponds to the complete reality of architecture, and we are just going to have to build out our understanding of it one faltering step at a time.

That said, it is possible to ask the question if enterprise information architecture bears significant responsibility for failures in BI. Is it to blame for why rollups will not roll up, for why monthly reports that seem stable suddenly show weird blips of unbelievable data once in a while, for why ad hoc queries can never be relied on, and all the other BI misfortunes that we do not like to even name for fear we will conjure them into existence in our projects?

We would answer that it is. Figure 1 shows what we term the Abstraction-Translation Paradigm of enterprise information architecture. In a nutshell, it visualizes the processes by which applications are created as passing through successive layers of abstraction and translation that ultimately enables business information to be stored as binary representation of facts in implemented databases. This data can then be moved into data marts for use in BI applications. However, it is still a binary representation of facts. To be used, it has to be transformed and translated through the same layers required in the creation of the operational application that produced it. Finally, it emerges back at the business level as information again. Given the complexity of this round trip for information, we should be amazed that BI applications ever produce anything useful, rather than disappointed that they have shortcomings.

{{eval var=$ImageLine assign="Image1"}}{{$Image1|replace:'#Name#':"120107_chisholm_fig1_M.gif"|replace:'#Width#':"auto"|replace:'#Height#':"auto"|replace:'#Orientation#':"null"|regex_replace:'/#CaptionLine#(.*)#\/CaptionLine#/':""|replace:'#CaptionWidth#':""|replace:'#CaptionHeight#':""|replace:'#CaptionOrientation#':""|replace:'#ImageCredit#':""|replace:'#AltText#':""}}

The Abstraction-Translation Paradigm

The leftmost stack in Figure 1 represents the layers of analysis and design needed to build an operational application, and the roles played by the individuals who are responsible for what happens in each layer. It is divided up into a column for data and another column for business rules. These end up as physically implemented databases and application logic, respectively.

The process of implementing an operational application begins in the business, where a business analyst gathers business requirements, describes the current business processes and identifies the data used in these processes. The artifacts that are produced ideally include use cases, workflows and a conceptual data model with a glossary of business terms. These represent the business in what is termed the conceptual layer in Figure 1. Unfortunately, “conceptual” has all kinds of definitions, but here it is taken to mean a direct representation of the business.

Get access to this article and thousands more...

All Information Management articles are archived after 7 days. REGISTER NOW for unlimited access to all recently archived articles, as well as thousands of searchable stories. Registered Members also gain access to:

  • Full access to including all searchable archived content
  • Exclusive E-Newsletters delivering the latest headlines to your inbox
  • Access to White Papers, Web Seminars, and Blog Discussions
  • Discounts to upcoming conferences & events
  • Uninterrupted access to all sponsored content, and MORE!

Already Registered?

Filed under:


Comments (0)

Be the first to comment on this post using the section below.

Add Your Comments:
You must be registered to post a comment.
Not Registered?
You must be registered to post a comment. Click here to register.
Already registered? Log in here
Please note you must now log in with your email address and password.
Login  |  My Account  |  White Papers  |  Web Seminars  |  Events |  Newsletters |  eBooks
Please note you must now log in with your email address and password.