Continue in 2 seconds

Deep History: What Good Is It?

  • March 01 2002, 1:00am EST

Data warehousing has brought many different aspects of information to the party. Prior to the data warehouse, there was no emphasis on data integration. Prior to data warehousing, data was stored so that it was optimal for transaction processing, not access and analysis. Prior to data warehousing, long-running transactions were thought to be abnormal activities that should be avoided at all costs.

Perhaps the most profound dimension of information that has been afforded by data warehousing is the recognition of the importance of historical data. Transaction systems stored data that was a month or a quarter old. Some accounting systems stored data at a summary level that was perhaps two years old. Data that was two years old was useful for establishing annual comparisons of revenue and other financial measurements of data. However, this was pretty much the extent of historical data.

There are, of course, the arguments that historical data is yesterday's newspaper and that anything older than about five minutes is ancient history. Many believe that businesses change so quickly that even yesterday's data is stale and worthless.

There may indeed be businesses that reinvent themselves every few moments, but there are not many of those businesses. Businesses that have been around any length of time have a certain regularity to their cycles. As long as customers ­ individuals and organizations ­ are creatures of habit, there will be a momentum that is built in the marketplace that is hard to ignore.

Deep history is data that is more than two years old and that is stored at a detailed level. Why is there a need to retain this deep history? It can be argued that business conditions are so different over the passage of lengthy amounts of time that no rational conclusions can be drawn from the behavior of a business five or ten years ago. In some circumstances, this may well be true. Yet for some types of analysis, even anecdotal inferences can prove insightful.

For example, consider that businesses are subject to long cycles of boom and bust or expansion and recession. In some cases, these cycles are twenty years in length. If a business has detailed records of how the business fared in the ups and downs of a business cycle, it is at least interesting to note how the business was affected, even if the conclusion cannot be drawn that today's cycles will affect the business in the same way. In other words, the recession of 1972 may have had entirely different causes than the recession of 2001. If that's the case, why should a corporation care how business fared in 1972?

The answer is that a corporation can learn a lot from understanding certain basic facts that emanate from a study of business over a substantial period of time, such as:

  • The signs that a recession was ebbing.
  • The length of time needed for recovery.
  • The first parts of the business to recover.
  • The parts of the business that never fully recovered.
  • The effect on the stock price and cash flow.
  • The effect on staffing levels.
  • The effect on market share.

Even though the recession of 1972 has very different characteristics than that of 2001, some basic business patterns of behavior remain the same. Those observations cannot be made – certainly at the detailed level – without deep historical data.
There are also other things that can be examined with deep historical data. Consider, for example, the rise and fall of interest rates. Some businesses are very sensitive to interest rates; therefore, those businesses are very interested in how they reacted historically to fluctuations in the rates. Deep historical data allows the organization to look at the company's reaction over time.

Other types of information that can be examined over time with deep historical data include:

  • The changing length of sales cycles.
  • The rise or fall of "deal size."
  • The effect of changes in packaging.
  • The changes in product mix.
  • The effect of competition.
  • The globalization of products over time.

There are many interesting aspects of business that can only be looked at over time – deep time. Merely keeping summarized data around for two or three years does not set the stage for the important kinds of analysis that can be performed with deep historical data.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access