Last month we took a look at the data warehouse industry's progress on the deadly sins of tunnel vision, perspective and myopia. This month we'll cover the remaining sins and how we're doing overall.
The classic signs of ignorance have been teams assuming that data warehousing was just another online transaction processing (OLTP) type project, labeling a collection of replicated normalized tables as a data warehouse and not attaching any importance to meta data.
Of all the sins, we've made the most progress on ignorance. Many thousands of data warehouse warriors have attended seminars, classes and conferences. Many thousands have also gained hard-earned experience on data warehouse projects in their organizations. Although we are still saddled with lingering urban legends and the usual amount of FUD (fear, uncertainty and doubt) present in any technology market, we continue to shine the purifying light of knowledge and experience into these dank, dark hiding places of obfuscation and distortion.
The manifestations of greed are typically a six-month time frame for an enterprise scale project; planning to do parallel development of multiple subject areas with inexperienced teams; setting requirements that demand 100 percent loading of the available resources, thus precluding any headroom; and ongoing, pervasive scope creep.
We still see struggles in the area of greed, both at the team and vendor levels. At the team level, there is still too much greed when it comes to scope. Even given the amount of education that has taken place, there are still many teams that have very little real-world experience. It is all too easy for them to attempt more than they are capable of delivering in the time frames they commit to. I don't have any silver bullet for vendor greed. It is something that we all have to put up with. We do best by seeking to effect positive change by voting with our dollars.
Sloth is typified by using normalized data models in systems that are meant for direct user access, little to no data scrubbing or other information quality processes or functions, and reliance on technological silver bullets rather than the difficult and time-consuming work of data analysis and productive, user-oriented design.
Sloth is a sin where there is still tremendous opportunity for advancement. Teams tend to run out of gas just when they could make a tremendous difference in the ease of use of their systems. When it comes time to include value-added Y/N flags such as "Top 100 Customer Y/N" and derived data points such as "Total Units PYTD (prior year to date)" or "Total Units YTD (year to date)" in their designs, the teams will more often than not choose the easy path and delete them from the design. It is a real pity, as most of the value of the entire data warehouse system is manifested in these types of easy-to-use, powerful features.
By far, the greatest challenge we've had as an industry has been pride. All of us fall victim to the seductive voice of pride from time to time. Press, pundits, analysts, consultants, vendors, users and data warehousing teams are no exception.
If we could find an anecdote to pride, we could save the world much pain and suffering. Unfortunately, our field of business intelligence suffers the same as the world in general from doses of excessive pride. There are still plenty of projects that suffer dismal fates for no other reason than profuse pride by business leaders, project leaders, team members, consultants or other vendors. I have heard business leaders claim they don't need any help from anyone since they have all the answers. I have attended meetings where data warehouse team leaders have proclaimed that they didn't need business discovery because they knew what was best for the users and the business. I have witnessed consultants make incredible, fantastically outrageous claims of experience and capability. I have seen vendors engage in indescribable fits of pomposity. I wish I had an anti-hubris pill I could give them all, but I don't. Unfortunately, my experience mirrors the quote from Emily Bronte, "Proud people breed sad sorrows for themselves."
Overall, I'd give our industry a "B" at this stage of its development. We've come a tremendous way since the early days of bumbling around in the dark, inventing things as we went along. We've still got a way to go in several key areas including user-oriented design, meta data integration and cultural change management issues. Even though data warehousing is no longer in its infancy, we can all still profit by identifying the presence of and overcoming the seven deadly sins of data warehousing.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access