Where is the data warehousing/business intelligence (DW/BI) industry headed? Is that future state one of promise or problems? What are the current issues and trends by which that future state will be shaped? Finally, what present imperatives, if any, can we take away as we ponder these questions? Interpreting our industry's currents can be tricky. Nevertheless, let's sweep over the landscape at 50,000 feet and see what we can glean.
First, the industry is undergoing consolidation: Microsoft and ProClarity, IBM and FileNet, Oracle and Hyperion. These are some recent and well-publicized mergers, but the list goes on. This would basically seem to be a good thing. Consolidation means commoditization. Commoditization means better and more interoperable products and a lower total cost of ownership for customers. Or so we hope.
Second, the market is growing as measured by projected corporate expenditures. Gartner recently named BI the top spending priority among CIOs, and projects that strength in demand to continue over the next few years.1
Third, if the rapid proliferation of buzzwords and new three-letter acronyms (TLAs) being added to the industry lexicon is any indication, real innovation is occurring. CRM, SRM, BPM/CPM/EPM, MDM, BI2.0, EII, operational BI, etc.2 Alphabet soup, anyone? Joking aside, there is real value in many of these ideas, approaches and products, some of them with broad applicability and some more valuable when targeted to the right customer with the right need. The point is that new solutions are being continually invented, and invention doesn't typically happen in a vacuum, but in response to real, well-quantified (or at least well-identified), customer needs.
These are positive signs, are they not? Do they portend good things? Next, let's take a hard look at some of the more sobering indicators.
First, there is (and it seems to have been around for a while) widespread dissatisfaction among the business community with IT in general. Consider the IT offshoring movement. Is the drive to offshore simply a sign of competitive pressure, a flattening global economy and commoditization? Does it not also suggest subpar performance on the part of the IT industry? Why would an organization uproot, upend or heave off significant portions of its infrastructure, systems and human/knowledge assets if their value (however measured or perceived) ranged from adequate to exceptional? Further, consider some of the musings of business thinkers and leaders who publish within highly regarded periodicals such as the Harvard Business Review:
- Nicholas Carr in his 2003 article "IT Doesn't Matter" (the title says it all) argued that IT has reached the point of being a mass commodity and, therefore, is a risk to be managed, not a lever for differentiation and competitive advantage.3
- Robert Eccles, Liv Watson and Mike Willis, while championing the virtues of XBRL for integrated financial reporting, credited as HBR's breakthrough idea #10 for 2007 - nevertheless admit the following: " All this undoubtedly sounds too good to be true to managers who are rightfully jaded after decades of false promises that the next IT silver bullet is (this time, really!) just around the corner. "4
If it is safe to deduce from leading publications and the actions of business leaders, there is prevalent skepticism around the value of IT. It is against this backdrop that the EDW/BI scene is being played out.
Second, some of the growth in the EDW/BI market may be driven more by regulatory pressure than by inherent value. In a recent conversation, Paul Rasplicka, senior VP and portfolio manager of several midcap growth funds for AIM Investments, asked me this question, "How much of the growth in the BI industry do you think is attributable to government regulations, and how do you think the market will be impacted if those regulations are relaxed as is currently being discussed?" Being an EDW/BI practitioner myself, I initially downplayed the question, but on further reflection, came to appreciate it, for it would indeed be revealing to learn just how much of BI vendors' recent sales could be directly or indirectly attributed to Sarbanes-Oxley compliance. And what does that say, if some customers are only buying in order to satisfy regulatory requirements?
Third, even among industry analysts and experts, there is some confusion as to who the different players are and what precise value they bring to the table. Philip Howard of Bloor Research articulated this well in a recent article:
" The simple problem is this: there are too many BI tools (taking BI to have the broadest possible meaning) to get a clear picture of the market .... Given that this is the mess that BI is in, I think that there needs to be some clear holistic thinking about how all of this can be structured and put together in some reasonably simple way .... To conclude, I don't think I have an answer for this: I am merely pointing out the problem and suggesting that it needs resolving. It needs an architecture if you like."5
Fourth, among those of us who work and practice in this industry, there is widespread inability to articulate the value of what we do. Commonly offered explanations as to why an EDW/BI project should be undertaken and, concomitantly, key principles to observe in doing so, include the following:
- Decreasing the cost of information,
- Enabling better-informed decision-making, and
- It depends upon (and must adhere to) a specific business need.
Hardly compelling. That's not to say these explanations are completely devoid of worth, but they're nothing to rally around.
Finally, by way of anecdote, my personal conversations with numerous customers ranging from day-to-day users to corporate executives at various prominent companies suggest widespread disillusionment:
- One executive who stated that one particular BI solution, "Sure doesn't do a lot for a tool that costs so much."
- A finance executive who stated that all he knew about his EDW was that it cost a lot to build it, that data kept getting loaded into it but nothing really was being done with it.
- Well-trained users who have repeatedly told me that the numbers aren't right, or they have trouble issuing seemingly straightforward queries to their EDW.
- Fellow consultants who echo these refrains.
Be the first to comment on this post using the section below.