Continue in 2 seconds

The Limits of Business Intelligence: An Organizational Learning Approach

Published
  • June 01 2003, 1:00am EDT

Business intelligence (BI) is commonly considered to be "the knowledge gained about a business through the use of various hardware/software technologies which enable organizations to turn data into information."1 These technologies include data warehousing and the analytical tools that access data and turn it into useful information for business managers. The link between BI and an organization that uses it is that an organization collectively is a cognitive system: it senses the environment, makes a representation of it, acts on the basis of the representation and learns from the results of its actions, storing its experience as institutional memory. This article will examine the theoretical limits of BI technology as an aid to the process of organizational learning. This involves looking at how BI technology models the business reality that organizations encounter, how organizations and individual BI users define and access institutional memory and the limits to which BI technology can assist organizational learning. A subsequent article in DM Review's November issue will examine how BI designers can assess the risk imposed by these limitations and plan for successful BI implementations using organizational learning constructs.

Modeling the Business "Reality"

In the classic approach to BI system implementation, users and technicians construct a data warehouse (DW) that feeds data into functional data marts and/or "cubes" of data for query and analysis by various BI tools. The functional data marts represent business domains such as marketing, finance, production, planning, etc. At a conceptual level, the logical architecture of the DW attempts to model the structure of the external business environment that it represents. For example, tables in the DW might correspond to customer relationships, product lines, service histories and so forth. This technology arrangement is illustrated in Figure 1.


Figure 1: BI Suite of Tools

Users access this information with BI tools for business planning and decision making. Ideally, the process unfolds in a cycle of data acquisition (capture), analysis, decision making and action as illustrated in Figure 2.


Figure 2: The BI Cycle

Figure 2 illustrates that there is a transformation of data from its raw form into successively higher levels of usage. There appears to be a progressive distillation of the data generated in the business and externally to support the organizational processes of analysis, decision making and action. Diagrams similar to Figure 2 are sometimes used by BI vendors to illustrate how BI informs the overall organizational learning process. However, as we will see, this is not the case. Figure 2 is not organizational learning or even individual learning as it is understood by scholars in the field. There are several reasons for this. First, the nature of human perception imposes limits on the ability of the overall BI system to deliver complete results. Second, the real world is very complex, and there are limits on how well it can be "compressed" in a BI suite. Third, organizations don't actually learn in the cycle represented by Figure 2. Organizational cognition is much more nebulous, especially at the level of institutional memory. These limitations are sometimes invisible to BI users and implementers in spite of their involvement in the project, for reasons that we will examine.

The Limits of BI Design

DW architects and users usually proceed with the design and construction of the BI suite and its ancillary marts and tools as the solution that yields information needed for their business decision making and planning. Anyone who has participated in a DW effort knows how challenging such a project can be. Strong end-user involvement can mitigate implementation risk, but the perceptual issues related to just how effectively the BI suite models the real-world situation typically do not enter the project dialog. Nonetheless, these issues are real and can, in my opinion, account for much of the success or failure of BI projects.

Consider the situation illustrated in Figure 3. Here we see the chain of successive abstraction that is associated with the data modeling and design process needed for BI.


Figure 3: Levels of Abstraction in BI

By the time a user of the BI suite accesses information in the DW, he/she is already two levels removed from the actual phenomena that are being viewed through the lens of the BI suite. Depending on the preconceptions of the BI user, there may be additional levels of distance from real-world phenomena. These preconceptions have been termed mental models by Peter Senge in his management classic, The Fifth Discipline, and termed occupational cultures by Edgar Schein in his work on organizations. (2,3) They are the cognitive lenses through which users individually and collectively view and manipulate information. For example, if managers hold a view of the business as separate products and/or organizational "silos," they may not be receptive to information from the BI suite that shows that customers experience the business as a single entity and expect a consistent experience across all contact channels. In short, a BI system is an abstraction of the reality it is designed to analyze, and the preconceptions of its users can further cloud this abstraction if they are not aware of them.

In addition to its distance from the actual business reality that the DW attempts to model, there is the consideration of its ability to capture the complexity of real-world phenomena. A real-world business involves many elements and relationships, interacting dynamically. The philosophical idea behind this is that there may be one real world "out there," but there are many possible descriptions of it ­ some of them useful and others not so useful. This distinction is relevant because both organizations and individuals interact with the real world on the basis of their internalized descriptions of it.

The BI developer wants to create a description of the business environment in the DW that allows users to: 1) understand it in terms that are meaningful to them (e.g., KPIs); and 2) use this knowledge to predict and control the business environment. To accomplish these design goals requires the developer to make decisions about how the DW should model its external business reality, including some phenomena and excluding others. The DW becomes a summarized version or model of its external reality. Thus, the DW at the heart of the BI suite is a "compressed" version of the external business reality. It has become one way of describing the real world to its end users. To the extent that the BI suite is able to yield accurate analysis and prediction of the environment, we can say that the designer's compression of the external reality has captured its significant features; the DW is a kind of "shorthand" model of the real business world it attempts to analyze.

In fact, that is exactly what a good scientific theory is ­ an economical compression of the real world. Think of Einstein's famous energy/mass equation, E=mc2. In a few characters, this shorthand notation captured the essence of an enormous body of science knowledge and, thus, had both explanatory and predictive value. Einstein could do this because the external reality he sought to describe had sufficient structure to allow a concise equation to model it. Business reality is not always so cooperative! If the external business reality that the BI suite intends to capture and model has sufficient structure, it can be modeled and the BI suite might yield explanatory and predictive knowledge to its users. If it is too random, the BI project is unlikely to yield good results in the sense of Figures 2 and 3. Clearly, BI designers want to capture the relevant aspects of their users' business reality in a way that yields actionable results.

BI and Organizational Learning

Earlier, I stated that the link between BI and the organization that uses it is that an organization collectively is a cognitive system: it senses the environment, makes a representation of it, acts on the basis of the representation and learns from the results of its actions, storing its experience as institutional memory. Figure 2 illustrates how the BI suite might support the cognitive cycle of analysis, decision making and action. It is tempting to characterize the BI suite as the institutional memory component of organizational learning. However, this is only partly true. I want to show here that organizational learning and knowledge involve much more than BI technology and that BI can, at best, support only certain levels of business knowledge. To do this, we will have to examine how organizations "think" and how the structure of culture and knowledge informs learning.

Scholars have identified at least three separate internal cultures that characterize business organizations: the executive, technical and operator cultures.4 (Note: I would add to these a fourth occupational culture – sales and marketing.) Each has its own values and uses these values to judge (think about) what is important. These are:

  • The executive culture of senior officers and board members is concerned primarily with the firm's financial value and, thus, pays attention to revenue, capital structure, regulation, shareholder and board matters.
  • The technical culture of engineers, accountants and IT staff is focused on business processes, products and information; their chief concern is efficiency.
  • The operator culture of production workers and service staff is focused on how to get the work done; their chief values are teamwork and production.

It is tempting to map the needs of these three cultures to a specific data mart or set of cubes, and function-specific data marts and cubes usually do just that. However, an occupational culture needs a lot more than BI to assess environmental changes and to make decisions, and their activity is filtered through the lens of their primary occupational values. Thus, a CEO might focus obsessively on promoting the company's stock price and remain blind to the operational issues of business and service integration. (It appears that WorldCom did just that with its many acquired subsidiaries.) Or, IT technicians might strive to implement exotic technology without a view to its contribution to the value of the business or to customer service.
The lens of different values that each internal culture uses can result in these kinds of communication disconnects that are common within organizations. Even a data mart that is designed perfectly to match the needs of the financial staff might not provide good information to help them communicate effectively with the shop floor people or help them read the subtle intentions of senior management. The whole issue of mental models and occupational biases is that they are usually not visible to the people involved. Thus, it is up to the BI designer to initiate the process of end-user awareness to overcome this liability.

The Structure of Organizational Knowledge

Each occupational culture, as well as the whole organization, is a collective cognitive system that learns from the results of its actions; it does so through the lens of its own peculiar values. Collective experience is embodied as the institutional memory that each occupational culture and the organization as a whole accesses to inform its decisions. This memory consists of at least three levels of knowledge: artifacts, espoused values and basic assumptions.5 This is illustrated in Figure 4.


Figure 4: Levels of Organizational Knowledge

Artifacts are the visible and tangible products of internal culture in an organization and include items such as the physical layout, reward systems, important business processes, information systems (e.g., the BI suite), symbols of status, logos and dress codes. Some artifacts are common to the entire organization, and some are unique to each internal occupational culture.

Espoused values are what people in the organization say they believe; they are the organization's official view of itself and usually come into being as a result of management action (e.g., a mission statement or promotion of people who exemplify some desired behavior).

Basic assumptions are the real operative principles that underlay a culture. They are rarely observed easily and need to be deciphered from the more visible artifacts, espoused values and behavior. Basic assumptions may not be visible to outside observers or even to insiders and evolve when a solution to a problem works repeatedly and comes to be taken for granted so that it is embedded in the organization's unconscious knowledge.

The structural representation of organizational knowledge in Figure 4 as three levels of meaning does not look like the logical model of any DW I have seen, yet it has proven to be a good explanation for how an organization actually stores the information and values that are needed for group action and learning. In the context of organizational learning, the BI suite is clearly relegated to the artifact level (i.e., it is a thing that is used in learning but that exists at the level of data, facts, events, customers, products and other real world phenomena). This level of knowledge is very important, but gives little insight into the drivers of organizational behavior that are embedded in the other levels of institutional memory and that do not have an explicit representation as hard information. Artifacts such as BI and the information they contain are just the tip of the organizational knowledge "iceberg."

Previously, I noted that the purpose of BI design is to somehow compress the real world into a model that contains enough of the essence of the real-world phenomena to provide explanatory and predictive information. In the context of organizational knowledge as I have described it here, we see how difficult this really is. A DW can hardly capture or model the subtleties of the basic assumptions that drive an organization at its core; the DW is limited to capturing and analyzing objects that have a real world identity, such as customers and financial transactions. The level of basic values has even more of a psychological identity; espoused values may contain both real world (e.g., quality standards and mission statements) and psychological elements.

At the level of the individual, it is theorized that people carry in their minds an internalized model of organizational knowledge that they access and use to guide their own behavior in organizational situations.6 They use this knowledge to filter their experience and to decide how to respond and act as a kind of personalized internal BI suite! Thus, BI users are only looking at a subset of overall organizational knowledge ­ the artifact portion contained in the BI suite. Additionally, BI users filter this reality through their own occupational biases (mental models) and reference it to the other knowledge levels that they carry as their internalized version of institutional memory. The end result for individuals is that the BI suite can, at best, provide only partial information that is related to real-world tangibles and not to the more basic intangibles that lie under the surface of business life. Yet it is these intangibles and the ability to "read" them that is the measure of a person's organizational savvy and their personal survivability in their organization.

In recent years, organization development (OD) has come into its own as a way to facilitate the collective awareness that enables managers to identify the effect of their deeply held values and mental biases. OD is just now being applied to the design of enterprise systems such as BI and knowledge management.7 It appears that OD interventions could help with setting realistic expectations for BI systems in the context of organizational learning.

The Limits of BI

In this article, I have attempted to explain how BI functions in an organizational learning context and the limits this imposes on the technology. I have tried to show that the BI suite is a kind of compressed version or model of the overall business reality that is its subject and that its explanatory and predictive value is mitigated by the nebulous and dynamic structure of that world, including the occupational biases and mental models of BI users. At the level of organizational knowledge, the majority of such knowledge does not lend itself to representation as data and structured information. It exists as institutional memory both above and below the level of what a BI suite can model and interpret. This does not diminish the importance of BI as a tool to aid organizational learning and guidance, but it does position BI as a technology that is limited by what it can model of the business environment. My article in the November issue will examine how designers can assess their risk and plan for a successful BI implementation using organizational learning constructs.

References:
1. DMReview.com General Resources Glossary.
2. Senge, P. The Fifth Discipline: The Art & Practice of the Learning Organization. New York: Currency Doubleday, 1994.
3. Schein, E. Three Cultures of Management: The Key to Organizational Learning. Sloan Management Review, 38(1), 1996.
4. Schein (1996).
5. Schein, Edgar. Organizational Culture and Leadership. San Francisco, CA: Jossey-Bass, 1997.
6. Argyris, C., & Schon, D. Organizational Learning II. Reading, MA: Addison-Wesley, 1996.
7. Allee, V. The Knowledge Evolution: Expanding Organizational Intelligence. Newton, MA: Butterworth-Heinemann, 1997.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access