Continue in 2 seconds

Real-Time Data Warehousing

Published
  • March 01 1998, 1:00am EST

Every week, there's a new theory or argument in the data warehousing arena. Should organizations adopt an enterprise warehouse? Or, is an architected data mart a better way to go? If an organization decides to go the data mart way, then there is the question of a dependent or independent data mart solution? The real question is, do users care? No, I believe that what they care about is tapping corporate information to solve a business problem to compete and survive--in time.

Think & Act in Real Time

I believe the issue really revolves around time. There are people and organizations who want and need to plan entire enterprise data warehouses up front, gaining buy-in and agreement in advance about every detail of the warehouse from each participating department.

But after visiting hundreds of credible user sites, I find that it rarely works this way. The velocity of business today is such that it often doesn't make sense to plan for the ages. As Regis McKenna points out in his new book, Real Time,

"...almost all technology today is focused on compressing to zero the amount of time it takes to acquire and use information...to make decisions, to initiate action, to deploy resources, to innovate. We have to think and act in real time. We cannot afford to do otherwise."

The real question is how many business problems today can wait nine months--the minimum for an enterprise- wide data warehouse deployment? In fact, a recent estimate by the Data Warehousing Institute puts warehousing deployment cycles at the 24-36 month time frame. According to META Group, data warehouses and data marts have increasingly become a strategic business initiative--and what critical initiative can wait that long?

This accelerated pace of business is why data marts evolved in the first place. Marts can be deployed in as little as two months, largely because they generally don't cross organizational boundaries. This eliminates the time-consuming process of getting complete agreement among department managers. Marts allow a small group of people within a company to quickly set up a system to access and analyze a specific set of corporate data that they need to compete and survive.

I believe this deployment speed is why the data mart industry is in such good health; data marts have grown from 24 percent of all data warehousing implementations last year to fully 61 percent this year.1 Robertson Stephens found that marts deliver the same quality of decision at a fraction of the cost of data warehouses and, in fact, found that the average ROI for marts is 250 percent!

Design First, Implement Incrementally

Many organizations today are deploying data marts to provide answers to pressing business problems--but are first planning up front so that subsequent data marts will eventually snap together like Legos into a single, useful integrated data warehouse. Contrary to popular myths, data mart planning and fast execution are not necessarily mutually exclusive propositions.

This design-first, implement- incrementally approach provides the best of both worlds. Not only does it enable companies to reuse or combine information and fact tables later on, but it also makes the warehousing project far less daunting. According to data mart guru Ralph Kimball, "Data warehouse planners take refuge in carving off a little piece of the whole data warehouse by bringing it to completion and calling it a data mart."

Two techniques can be used to ensure that successive marts will inter-operate smoothly. The first is thinking through the structure, or the dimensions, of the data in the mart. Sometimes called "conforming dimensions," this process involves defining common data elements that will be used for successive data marts--resulting in a master suite of dimensions. Often this will include a fairly common set of information such as customers, location, time and products. Other departments can use these definitions and add their own, allowing the data warehouse to "grow up" in a coherent way. This planning process usually takes a few weeks.

The second technique is to keep track of all the data. Warehousing managers can tag each piece of data with tracking information, much like the tracking numbers used by small package carriers, ensuring that it can be easily located no matter where it ends up. This eliminates the time-consuming process of mining through various departmental data marts trying to locate a piece of information.

Certainly there are organizations, business problems and people that can wait for the company-wide agreement necessary for full-blown enterprise data warehouses. But, those types are getting fewer and farther between. Plan a little, implement incrementally and quickly, and reap the competitive benefits now.

1 Source: META Group

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access