One of the things that I keep seeing over and over again is the implementation of projects using inappropriate technology. More often than not, these projects end up with, for lack of a better term, confused systems as they meet some needs but not others. The thing that keeps these systems alive is the substantial investments of time and money. Even then, after awhile the money stops playing a role and these systems end up ignored or replaced. To ensure that real-time data warehousing endeavors don’t suffer the same fate, it’s important to recognize where it makes sense to implement them and, equally important, where not to implement them. I’ve found that real-time data warehousing makes sense in two primary areas: operational and business intelligence applications.

Operational Applications

Contrary to popular belief, successful data warehouses are utilized for operational applications. Billing and usage reporting applications seem to be the first operational applications to take advantage of real-time data warehousing at places such as credit card institutions and telecommunication companies. To keep up with the billing and usage volumes, these companies have been forced to migrate from nightly batching jobs to real-time or near real-time data processing. Data warehouses are the natural choice for this type of processing as aggregation and fast retrieval of large amounts of data is what they do best.

Other operational applications include inventory management and service provisioning, to mention a few. Many businesses have moved to just-in- time inventory, so timing is everything. Once provisioned for some service, the billing clock starts ticking so tracking when the service was provisioned is significant. The temporal element is the common denominator for operational applications that can benefit from real-time data warehousing.

Business Intelligence Applications

One of the current trends in business intelligence is the implementation of corporate dashboards. The sophistication of corporate dashboards is quite broad and ranges from a report that lists several key performance indicators (KPIs) to dashboards that mimic the look and feel of automobile dashboards right down to the round gauges with needles that fluctuate in real time. Timeliness of the dashboards is driven by the data freshness in the data warehouse. Dashboards that display real-time data activity typically have a real-time data warehouse behind them.

The kind of information reported by these dashboards is as varied as the dashboards themselves. Financial information typically takes first dibs followed by trend and operational information. Financial information includes up-to-date revenue, current exchange and interest rates, and stock prices to mention a few KPIs. Trend reports typically display KPIs across some predefined timeframe, such as rolling 24 hour view, while operational KPIs can include equipment utilization rates.

Customer relationship management (CRM) business intelligence is an area that has been getting quite a bit of attention and where real-time capability is a value add, especially in a Web environment. To illustrate this, consider an established retailer who wants to see whether launching a Web-based discount campaign will increase sales while driving additional traffic to their Web site. A real-time data warehouse that captures and analyzes the clickstream and sales activity from the Web site can provide the information necessary to determine the effectiveness of the discount campaign.

The Internet is definitely an area that can and should reap the benefits of real-time data warehousing. Clickstream data generates substantial transaction volumes that, in popular sites, can easily eclipse credit card transaction volumes. These data volumes alone justify having real-time data warehousing. From a business intelligence perspective, for clickstream data to make sense, it has to be aggregated into predefined buckets to allow for click activity analysis. Once again, data warehousing is the natural choice for this type of processing.

When Not to Implement Real Time

The trick with real-time data warehousing is to know when not to implement it. For example, I recently developed a data warehouse that relies on transactional data from several external sources. One source provides transactional data on a daily basis whereas the other sources provide data on a monthly basis. Although the client is interested in having real-time data, it simply does not make sense to enable a real-time capability in this case. The data is only available once a day at best, so having the infrastructure to process the data in real time is overkill and adds an unnecessary level of complexity.

In addition to data availability, the business itself dictates whether real time makes sense. Take, for example, the automotive world where dealer inventory plays a role. If the automobile manufacturer expects a dealer to have 60 days of inventory on average, does it make sense to put in the data warehouse a subject area for automobile dealer inventory that gets updated in real time? The answer to that depends whether there is benefit in reacting to this information within hours of seeing it. Unfortunately, there are no fast rules about when not to implement real-time data warehousing. However, this does not mean that taking a step back and seeing if the endeavor makes sense in the larger scheme of things is not useful.

Implementing real-time data warehouses makes sense in operational applications where either large data volumes or time play a significant role. Business intelligence applications, such as corporate dashboards and CRM and Web tracking applications, also benefit from having real- time data warehousing. On the other hand, data availability and the business itself dictate when it does not make sense to implement real-time data warehousing. All in all, knowing when to implement real-time data warehousing is as important as knowing when not to do it.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access