Continue in 2 seconds

Back to the Future: Data Mart Consolidation, Part 1

Published
  • March 01 2003, 1:00am EST

For much of the last decade, conventional theories surrounding decision support architectures have focused more on cost than business benefit. Lack of return-on-investment (ROI) quantification has resulted in platform selection criteria being focused on perceived minimization of initial system cost rather than on maximizing lasting value to the enterprise. Often these decisions are made within departmental boundaries without consideration of an overarching data warehousing strategy.

This reasoning has led many organizations down the path of data mart proliferation. This represents the creation of nonintegrated data sets developed to address specific application needs, usually with an inflexible design. In the vast majority of cases, data mart proliferation is not the result of a chosen architectural strategy, but a consequence due to the lack of an architectural strategy.

To further complicate matters, the recent economic environment and ensuing budget reduction cycles have forced IT managers to find ways to squeeze every drop of performance out of their systems while still managing to meet users' needs. In other words, we're all being asked to do more with less.

The good news is that the data warehousing market is now mature enough that there are successes and best practices to be leveraged. There are proven methods to reduce costs, gain efficiencies and increase the value of enterprise data. Pioneering organizations have found a way to save millions of dollars while providing their users with integrated, consistent and timely information. The path that led to these results started with a rapidly emerging trend in data warehousing today – data mart consolidation (DMC).

I have learned that companies worldwide are embracing DMC as a way to save large amounts of money while still providing high degrees of business value with ROI. DMC is an answer to the issues many face today. There is a way to cut business intelligence (BI) costs and continue to deliver business value with BI.

The Benefits of the Program Approach

Tenets of sound business practices apply to data warehousing. One of these is the necessity to accomplish an objective in the most efficient manner. What is the most efficient way to accomplish data warehousing objectives?

It is the way that builds a data warehouse to solve specific needs, but does so in a manner that leverages previous investment in the architecture, tools, processes and people ­ and does not prohibit future growth. This enables an efficient, programmatic approach to data warehousing, created to serve information to the enterprise. By leveraging an integrated data warehousing approach, you will realize efficiencies generated by economies of scale.

Efficiency as it relates to DMC comes in three primary forms. There are true cost efficiencies involving the hardware, software and personnel carrying costs of the environment and switching these costs to a more manageable expense stream. Many in this study referred to these as "IT benefits," but lower total cost of ownership (TCO) and economies of scale are business benefits as well. With one data warehousing program as opposed to "many," fewer resources and processes need to be supported in an enterprise.

Secondly, there are efficiencies associated with having a "single version of the truth" to reference as opposed to engaging in internal "data warfare," spending most of the "analysis" time searching for data or "making do" with undesirable, outdated data. Many companies that implemented DMC were engaged in data warfare, but it is not simply a matter of whose data is better. In many organizations, the best data is not accessible or the users are not trained on the access method. A central warehouse helps set aside the politics of whose data is better by establishing a consistent, trustworthy source of information. Creating a single version of the truth drives internal efficiencies by focusing resources on the value-added activities of business rather than data-gathering activities.

Finally, there are system efficiencies to be gained by eliminating redundant processes. For example, although many are using the file delivery capabilities of operational systems to feed data to their data warehousing environment, getting data out of the source is still one of the most difficult tasks in data warehousing. Usually, the first extract request is not met with open arms. A second or third one can be impossible. This leads many to a "single extract, many load" architecture which solves some problems but not others.

Fortunately for those who have met the challenges, data warehousing has proved itself time and time again as a valid conduit for delivering data and data analysis into business processes, thereby improving them while helping the company achieve its stated goals. DMC allows organizations to reap the benefits of integrated, centralized data warehousing while delivering significant cost savings through internal efficiencies.

Desired Outcomes of DMC

Data warehousing is a process, not a project, and a journey rather than a destination. This applies to DMC as well. There are several forms that DMC can take including: merging data marts into a new warehouse; selecting an existing warehouse/mart and merging other warehouse/marts into it; and moving analytical functionality from other databases onto a data warehouse. The consolidation itself can leverage existing designs and reroute extract, transform and load (ETL) processes into the consolidated warehouse or consolidate designs as well as the platform.

Approaches to Data Mart Consolidation

Approaches and steps to DMC as well as maturity levels with DMC emerge:

  1. Rehosting ­ the process of taking database designs and ETL "lock, stock and barrel" and moving to a different platform as an effort to gain either performance or cost advantages. Often, the rehosting will be done onto a platform with existing data constructs, thereby expanding the utility of the platform.
  2. Rearchitecting ­ the process of merging database designs and, therefore, the data acquisition strategy for the data as well. Rearchitecting may involve picking the best model components from various models and/or it may involve more zero-based approaches (starting from scratch) that use requirements as the basis for the new model.

While there is no such thing as a cookie-cutter DMC process, there are common best practices and lessons which I will share in next month's column.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access