Note: would like to welcome R.H. Terdeman as a new columnist. Some of his columns will also be found at Bob Terdeman has extensive experience and a quirky view of the industry that he will share in his "From the Trenches" column. We welcome his expertise and hope you will enjoy the things he has to say in the coming months.

My coauthor and friend, Bill Inmon, and I were chatting the other day about those things that send us over the intellectual edge. In particular, it is individuals in any discipline that focus on what is "wrong" as opposed to what is "right." In data warehousing we often find those who are ready to declare a "failure." Interestingly, these critics are unwilling to make an intellectual contribution to a project in the formation phase but are too willing to step in when the going gets rough. Of course, these individuals generally have a contribution in the difficult phase of extensive tactical consulting at great expense as opposed to a few days of strategic advice that would have avoided the problem to begin with. Needless to say, they have all the answers. These types of individuals fail to recognize that data warehousing is not a science but an art form. Like any other art form, it involves constantly changing factors – many of which are not known at the time of project formation. Let’s examine a few of those factors.

Most important of all factors is the state of the business. The highest level of architecture within a business is the "business architecture." Yes, there really is such a thing as business architecture. Many companies start out with a simple premise; they sell widgets. Pretty soon they discover that some of their customers don’t know how to install the widgets, and thus they enter the service business. Next some of their customers decide that widget acquisition should be an operating expense, not a capital expense, and now they go into the widgets leasing business. This is the typical path as businesses try to constantly adjust to the changing requirements of their customers.

Let’s go back now to the data mart or data warehouse that was built for General Widget when it was only selling widgets. Did anyone know that General Widget was going into the service business? Did anyone know that some customers would want to lease widgets? One could argue that a great analyst would have foreseen these requirements and designed such capability. But even if the analyst had designed the capability in, the project manager, constrained by time and budget, would have probably removed the components from the project any way. The data warehouse came up and was used for sales and marketing and then had to be redesigned to meet the changing business requirements; is this a success or a failure?

Another classic case is the situation where the project team picks hardware and software before any parametric limits are known as to the size of the project. This equivalent to saying we need to move a load of freight from New York to Boston, we don’t know how much it weighs, we don’t know how much space it displaces but we are going to move it on a three-quarter-ton truck because that’s the type of truck we like. Is this a success or a failure? Well, from the point of view that we constricted the work to the truck we like, it’s a success. We might have to make 22 trips and, in the end, it is much more costly, but that is the nature of the organization and working within those culture limits it’s a success.

In both of the illustrations, while neither project obtained absolute success, each obtained a modicum of success as constrained by limited resources or limited access to critical strategic information. In realty, after personally visiting with 500 data warehousing teams, I have observed only one consistent type of failure and that involves organizational paralysis. That is the organization is so fearful to do anything that they wind up doing nothing. That’s right, any data mart or any data warehouse that brings business value is a success! The reality is every project is a partial disappointment to the project team and the business to some degree. Why? The project wasn’t cheap enough, the project missed someone’s expectation, the project took too long to implement, the project did not take into account a niche requirement, the project uses technology that was obsolete by the time it was operational, etc.

Using the correct criteria – did the business get some value out of the project – my personal experience (not a survey) says that more than 95 percent of the projects experience some success. What factors could improve the perceived level of success?

  1. Data warehousing projects must have access to the businesses strategic plan so as to anticipate requirements.
  2. The logical design should encompass business knowledge of the industry, even if this requirement does not immediately fall into the project plan.
  3. Three stakes in the ground are necessary for sizing: approximate data size, approximate user count and some tentative query mix projected at 36 months out. (Yes, you must be clairvoyant to meet this requirement.)
  4. Managing user expectation (don’t oversell the project).
  5. Stick to known technology. It’s tough enough to do a data warehousing project without experimenting with new technology unless it’s absolutely necessary.
  6. Higher one pathfinder, not a whole army.
  7. Data warehouse projects are committee endeavors; someone has to have the vision, someone has to run them, someone has to take responsibility.

Hope this stirred the pot a little, have fun see you next month!

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access