Gauge Your Data Warehouse Maturity
Information Management Magazine, November 2004
Many of us have managed data warehousing projects for years. Some have delivered highly strategic systems that are treasured by users and valued by top executives. However, others have struggled to sustain interest and funding in their data warehouses even though users are crying out for better, more accurate information.
What separates successful from struggling solutions? How does your data warehousing initiative compare to others in the industry? What will it take to get your data warehouse to the next level?
Many data warehousing managers ask these questions today. Unfortunately, there are no quick or easy answers.1 However, to provide some guidance, TDWI has developed a data warehousing maturity model that you can use to benchmark your progress. The model provides a quick way for you to gauge where your data warehousing initiative is now and where it needs to go next.
Advertisement
Six Stages
The maturity model consists of six stages: prenatal, infant, child, teenager, adult and sage. Business value increases as the data warehouse moves through each successive stage. (See Figure 1.)
Figure 1: Data Warehousing Maturity Model
Business value increases with each successive stage. Most organizations are currently in the child and teenager stages.
The stages are defined by a number of characteristics, including scope, analytic structure, executive perceptions, types of analytics, stewardship, funding, technology platform, change management and administration (for which we borrow concepts from SEI's Capability Maturity Model). This article will address just a few of these characteristics.
Organizations evolve at different rates through these six stages, and each may exhibit characteristics of multiple stages at a given time. As such, no one should expect to move cleanly and precisely from one stage to the next; however, there are two pivotal points in the evolution of any data warehouse/business intelligence (DW/BI) initiative, represented in Figure 1 as the "gulf" and the "chasm." Many DW/BI initiatives stall at these points. They remain stuck with one foot in the past and another in the future, unable to make a clean leap beyond. As a result, they never fully reap the benefits in the successive stages.
The primary way to overcome these obstacles is to change executive perceptions. To cross the gulf, executives must recognize that DW/BI is more than just a management reporting system and that the spreadsheets and desktop databases they rely on to run the business are actually undermining productivity and effectiveness. To cross the chasm -- which is much more difficult -- executives must perceive the DW/BI environment as a mission-critical enterprise resource that they own, direct and fund (not the IT department).
However, we're now getting ahead of ourselves. Let me briefly describe each stage and its major characteristics.
Prenatal Stage: Management Reporting
Most established organizations have management reporting systems that generate a standard set of static reports, which are printed and distributed to large numbers of employees on a regular basis, usually weekly, monthly or quarterly. Because the reports are hand-coded against legacy systems (or an operational data store), the IT department can't respond rapidly to requests for custom reports. This erodes IT's credibility and frustrates users who need quick access to information to do their jobs. This inflexibility is particularly intolerable to business analysts whose job is to crunch numbers on behalf of executives and power users who know their way around corporate information systems. Taking matters into their own hands, these individuals circumvent IT by extracting data from source systems and loading it into spreadsheets or desktop databases. This gives rise to our next stage.
Infant Stage: Spreadmarts
Spreadmarts are spreadsheets or desktop databases that function as surrogate data marts. Each contains a unique set of data, metrics and rules that do not align with other spreadmarts, management reports or analytical systems. Because spreadsheets are so ubiquitous, cheap and easy to use, spreadmarts proliferate like weeds -- organizations have dozens, if not hundreds or thousands, of these pernicious analytic structures.2
Spreadmarts prevent the organization (or CEO) from getting a clear, consistent picture of the enterprise. However, eliminating spreadmarts is difficult because they offer high local control at extremely low costs, making it difficult for organizations to cross the gulf between stage one and two. In fact, spreadmart users must sacrifice greater degrees of control over their analytical structures until the final two stages, when new development processes and analytical services align local and enterprise interests. 3 (See Figure 2.)
Figure 2: Local Control vs. Enterprise Value
It is difficult to eliminate spreadmarts because they offer maximum local control at minimum cost. Only when organizations enter the final two stages do local control and enterprise value align and accelerate together.
Child Stage: Data Marts
In the child stage, departments recognize the need to empower all their knowledge-workers with timely information and insight, not just the business analysts and executives who primarily benefit from spreadmarts.
A data mart is a shared, analytic structure that generally supports a single application area, business process or department. The departmental team gathers information requirements and tailors each data mart to meet the needs of the members in its group. They then provide knowledge-workers with an interactive reporting tool (e.g., OLAP or ad hoc query tool or parameterized reports). The tool lets knowledge-workers drill down or across a dimensional structure to follow trends and gain a deeper insight into events driving the process or tasks they manage.
However, data marts often fall prey to the same problems that afflict spreadmarts. Each data mart supports unique definitions and rules and extracts data directly from source systems. These so-called "independent" data marts do a great job of supporting local needs; however, their data can't be aggregated to support cross-departmental analysis. What's needed is a mechanism to integrate data marts without jeopardizing local autonomy. That's the hallmark of the teenager stage.
Teenager Stage: Data Warehouses
Page 1 of 3.
