It has become popular to attach the term "real-time" to many concepts in business intelligence (BI) and data warehousing (DW) real-time BI, real-time enterprise, real-time warehouse, real-time ETL (extract, transform and load), real-time analytics, and so on.
I dislike this usage because it conveys a partial truth while obscuring the important issues. In our industry, the usage of "real-time" as an adjective has unfortunately become marketing fluff. Some have suggested the term "right-time" as a substitute; however, that begs the question about what is right.
The Webster dictionary defines "real-time" as the simultaneous recording of an event with the actual occurrence of that event. In engineering disciplines, the term is extended to describe a system that records and responds to an event within milliseconds.
The history behind the term is fascinating. German military coined the phrase during World War II to describe their new approach to battlefield management, which complemented their blitzkrieg tactics. In earlier wars, small-scale models of the battlefield were created at the headquarters located some distance from the battle. Troops marched on foot into their positions; hence, troop movements could be easily tracked by sending written messages back to headquarters. With highly mobile tanks, artillery and trucks, the old approach become hopelessly out of sync with the reality on the battlefield. Therefore, the Germans officers moved their headquarters closer to the action and relied on radio messages to continuously update their information. Hence, they could watch the battle unfold in "real time."
The key concept behind "real time" is that our artificial representation must be in sync with the real world so that we can respond to events in an effective manner. In today's technology, the data warehouse has become that artificial representation of our real business world. In this regard, the primary purpose of the data warehouse is to maintain a unified and consistent view of business reality. Doing so in a timely manner is only one aspect of fulfilling this purpose.
To clarify the relationship between time and business value, consider the value-time curve in Figure 1. At an initial time, a business event requiring a response occurs within some business process. At a later time, action is taken to respond to that event. For example, a customer requests information on a specific product, and then the company provides the information within a few seconds, minutes, hours, days, weeks or months.
Figure 1: The Value-Time Curve
The assumption behind this decay curve is that the longer the delay or latency of the response, the less business value accrues to the company. In this example, the value is the probability that the customer will purchase the product. The action time (or action distance) is the duration between the event and the action, while the net value is the business value lost (or gained) over this duration.1
In the spirit of microeconomic analysis, a second curve can be charted showing the cost-time relationship, as in Figure 2.2 This curve represents the cost required to design, build and operate the system that responds to the event. At some point, the two curves may intersect, implying that the system cost equals the business value. To the right of this point, the company benefits because value exceeds cost; to the left, the company loses because cost exceeds value. The good news is that the intersection point is moving steadily to the left, allowing companies to create systems that increasingly benefit the business.
Figure 2: Adding the Cost Curve
Within the context of data warehousing, there are three components to this action time, as shown in Figure 3. The data latency is the time required to capture (usually from some source transactional system), transform/cleanse and store this data in the warehouse, ready for analysis. The analysis latency is the time required to analyze and disseminate the results to the appropriate persons. The decision latency is the time required for a person to understand the situation, decide on a course of action and initiate it. For automated procedures, a person may not be directly involved; hence, decision latency would be very small.
Figure 3: Components of Action Time
Figures 1 through 3 assume that value decays rapidly after the business event. However, this may not be true in real business situations. Let's consider a few counter cases.
First, Figure 4 illustrates a situation in which value stays constant and then rapidly drops. For example, a customer orders a nice meal at a fine restaurant and has different expectations of service than at a fast-food restaurant. In fact, if the food is delivered too quickly, the customer may question the quality.
Figure 4: Inverse Decay Curve
Second, Figure 5 illustrates a situation in which value increases with time rather than decreases. The value "matures" through time. For example, a farmer does not harvest the crop a day after planting the seeds. The crop requires time to mature.
Figure 5: Maturing Value Curve
Finally, Figure 6 illustrates a situation that combines the previous two a slow increase followed by a sudden decline. This curve is often associated with perishable goods or services. For example, bananas sell slowly when green, but sell more rapidly as they turn yellow and then get discarded as soon as the brown spots appear. Likewise, tickets for a rock concert or airline seats to Chicago increase in value until the event occurs and then become worthless.
Figure 6: Perishable Value Curve
The conclusion is that real-time BI/DW must be understood in terms of real value to the business. We often misuse the term to restate the obvious, with an emphasis on doing things quicker whether or not it has value to the business.
Real-time BI/DW implies much more than time. It conveys a rich sense of urgency. As an enterprise, we should be fully engaged and highly responsive, with an in-your-face style of business. We should hone the organization so that normal business is conducted with excellence. We should respond to unexpected demands in a manner that is intelligent and timely (in that order). We should adapt to market and technology trends through continual redesign of our systems.
Real-time BI/DW is the prerequisite for all of these objectives. To paraphrase a recent statement by Stephen Brobst, a real-time enterprise without real-time BI/DW is a real dumb company moving really fast!
1. In earlier writings, the term "action distance" was used to emphasize that time was one of several factors that influenced the effectiveness of the action. See R. Hackathorn, BI Watch: Action Distance, DM Review, September 2002.
2. Adapted from Illustration 2.3 in C. White, Building the Real-Time Enterprise, TDWI Report Series, November 2003, p. 11.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access