What do you think of when you hear the word "methodology?" For many of us, it conjures up images of tedious and costly projects that require baby-sitting by expensive consultants. But what about data warehousing methodologies? Are they more useful? What do they offer to project managers whose responsibility it is to bring the project to a successful conclusion?
A recent study by the Data Warehousing Institute (DWI) surveyed 21 data warehouse project managers on their most difficult challenges. Methodology was the third biggest menace--technology and education were number one and two. The three represented 87 percent of the problems identified. Interestingly, solving the third biggest problem--methodology--often minimizes technology and education issues as well.
A necessary tool for those overwhelmed with the task at hand, data warehousing methodologies have unfortunately gotten a bad name over the past couple years. This less-than-stellar reputation is due, in part, to vendors who have peddled their existing IT methodologies in a new wrapper, claiming these approaches will work for data warehousing too. These vendors don't understand that data warehouse projects are different from traditional IT projects. They are parallel in nature, have a different structure, different roles and expertise, a new vocabulary and, because of its enormous scope, other issues you don't find with common IT projects.
What are the components of a good data warehousing methodology? Comprehensive methodologies, although developed by different vendors, share similarities in their approach to warehouse development. At a minimum, a good warehousing methodology covers the following phases of the project:
Sponsorship, Leadership and Management: The number one cause of warehousing project failure is the lack of sponsorship from the business and the leadership in IT. This alliance must initiate the project and provide leadership throughout.
Requirements Gathering: "Build it and they will come" has proven to be the wrong approach. The correct method is to understand what information drives the measurement of the key processes that deliver the highest payback.
Data Warehouse Strategy and Architecture: Taking the right approach for the development of the strategy and architecture will ensure that the initial investments made in warehousing yield high returns on subsequent initiatives. Building incrementally with the future in mind is a key ingredient in the methodology.
Legacy System Assessment and Quality Analysis: "Quality, quality, quality" is the new battle cry of the warehousing project. The right methodology should address data quality issues thoroughly, or all other steps may be for naught.
Data Analysis, Modeling and Mapping: The methodology must address rapid deployment, expandable data architectures, building data models for decision making and a rules-based approach for data mapping.
Implementation and Training: Data warehouses cause great shifts in workloads and in how work is allocated. The shift from dependence on IT to the independence of the business user is a major cultural shift for most organizations. Training can help organizations with this paradigm shift.
Optimization and Deployment: Using the right methods for optimizing the warehouse and rolling it out to the organization are key in producing a usable warehouse. Rolling the warehouse out in stages, measuring and communicating results along the way, helps to ease the warehouse into the organization with minimal disruption.
Roles and Responsibilities of Warehouse Management: Ongoing support is often forgotten, but the roles and responsibilities of warehouse administrators and users must be structured, documented and understood by all.
When you purchase a data warehousing methodology or engage a vendor to employ their methodology, you may wonder what you actually get for your money. Although deliverables vary in scope and price, some common themes are emerging among these methodologies:
- Context-sensitive hypertext project plan at the task level with time estimates,
- Best practices approach,
- Staffing and resource recommendations, and
- Templates and forms for all deliverables.
Data warehousing methodologies have only recently begun to take shape, and they have some ground to make up in pursuit of the many acclaimed DW toolsets on the market. The next generation of data warehousing methodologies will likely be more flexible than these pioneering products--hopefully with an à la carte flavor that allows an organization to use specific phases without having to entirely reshape their current environment to fit that methodology. Until that day arrives, selecting and following a proven methodology (check references!) can get you past some common stumbling blocks that might otherwise trip up your warehouse project.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access