Recently we moved and reorganized our company library. As part of this herculean effort, I was assigned the task of reviewing the giant cache of conference proceedings we had accumulated over the last decade of data warehousing. After careful deliberation, I determined that the conference proceedings from the very first data warehousing conference were probably out of date enough that we could recycle them. Before tossing them, I spent a few moments perusing the logos of the sponsoring companies from that seminal event. It was a sobering experience, as there were only two or three survivors from those early days. Every other one of the original "market leaders" has perished. I was reminded again of the immutable law of the technological universe: the only constant is change.

Within hours of disposing of those proceedings, I was signed to give a series of keynotes around the world on the future of data warehousing and business intelligence. As I thought back on the proceedings I had just reviewed, I was struck by how shortsighted the leaders of the day were and how skewed their predictions had been. I had very little evidence that I would be any more prescient; but I was hopeful that a neutral philosophy and ten years of field experience would weigh in my favor.

Given those hopeful parameters, here are some of my thoughts on where we're going as we head into the next century:

1. A turnkey, packaged world. The entry of Microsoft into the market and the accompanying comp etition killing nearly free price points have enabled the expansion of the marketplace by several orders of magnitude. The products that will be sold to this market of millions will be turnkey, packaged data mart and data warehouse systems. The major challenges that will arise from this market transformation are knowledge transfer, architecture and a rapidly changing systems integration/services market. The first challenge is to transfer knowledge to this new market of millions of customers. I suspect that we will eventually see the business intelligence cable TV channel ­ all BI, all the time ­ to accompany the multitude of mass market books, magazines, etc., that flood the market in conjunction with any mass market opportunity. The second, architecture, will be an ongoing challenge as vendors look to sell quick-hit, quick-impact systems that are nonarchitected and will simply be very usable stovepipes. Only the well-educated consumer, who demands integrated, architected solutions will survive and prosper. The last, a rapidly changing systems integration (SI) and services market, will be driven by the dramatic drop in market share of the typical massive, custom-built systems integration project of today. Currently, these "build" model systems are at least 85 percent of the market, whereas soon we'll have a world where 85 to 90 percent of the systems will be turnkey, packaged "buy" solutions. This will place heavy pressure, both in market share and pricing, on the services vendors. Yes, there will always be people needed; but if a customer is only spending $25,000 to $75,000 for a turnkey system including the server, the database and the analytical applications that solve their business problem, they are very unlikely to spend $300 to $500 per hour for the people to set it up and get it going. This will continue to drive ongoing consolidation in the services sector, with the eventual distillation of the market to two to three major BI services providers.

2. Industry-standard architectures. A pleasant outgrowth of the "packaging" of the BI market will be the rise of industry standard architectures. Most likely arising from the huge Microsoft based mid-tier market, these turnkey architectures and models will be driven by standards bodies centered around large vertical markets such as transportation, utilities, financial services, retail, healthcare, etc. There are early vendor-led efforts underway now, but those proprietary efforts are not likely to become de facto standards any more than the vendor driven proprietary meta data models were. As BI systems become as common as file servers, the typical industry-standards bodies will drive the definition and adoption of standard architectures and logical models. This will greatly aid those of us on the customer side of the equation to force the vendors to sell turnkey solutions that integrate smoothly into these standard architectures and models.

3. Mass market challenges. Data warehousing and what we think of today as business intelligence are about to become commodity items and deeply imbedded into the everyday lives of both business and consumers. One major challenge of this phenomenon is that the current vendors in our space are completely unprepared and probably incapable of servicing and supporting this market of millions of customers and hundreds of millions of users. A second, more pervasive and threatening challenge is that security and privacy issues are going to continue to become front-page societal and political issues. We're already seeing court cases, multimillion dollar settlements and headline stories regarding misuse of consumer information contained in the data warehouse systems of well-known brand names, especially financial services companies. We will need to see industry leadership arise that is less technology focused and vendor driven to meet and overcome these large-scale challenges if we are to successfully navigate these class V rapids.

Obviously, this is but a small sample of what lies in store for all of us who make our living in the world of data warehousing and business intelligence. To hear the rest of what I believe is likely to happen, you'll have to attend one of the conference keynotes and take home a copy of the conference proceedings.You can do your own assessment of my predictions in five or six years when you clean out your office libraries.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access