Editor's Note: This article is featured in the December 2000 Consultant's Guide, a supplement to the December issue of DM Review.
In a few short years, data ware housing has gone from theory to conventional wisdom. As organizations build their data warehouse/ corporate information factory environment, they are increasingly turning to consulting organizations to do the heavy lifting. There is a natural progression to the reliance on external talent, because there simply is not enough in-house talent and experience to build and populate the warehouse environment.
The following are some guidelines to assist you in the selection of a data warehouse consultant.
- If your consulting salesman wants you to sign a contract for longer than six months for the first iteration, find another firm. Data warehousing is not just another ERP contract. In many cases, the failures in the data warehousing/ corporate information factory world have come where the engagement is oversized. The essence of success in data warehousing comes in small fast steps, not huge steps. Do not sign any contract longer than six months.
- Do not engage more than five to an absolute maximum of ten people for your warehouse development. Under most circumstances, you do not need more than five people for warehouse development.
- Make sure the deliverables are well defined.
- Do not rely on firm or corporate credentials to assess the credibility of your consulting firm. When you contract with a consulting firm, you are signing on with individuals. Check the references of the individuals who will be doing the work on your contract. Your success will depend on the people who will be doing the work in your shop. Make sure you know whom the consulting firm is really going to be bringing in to do the work.
- Make sure your consulting firm understands architecture. After all that is what you are paying for. Remember, data marts are not a short cut to data warehousing and ultimately cost the corporation much time and money. Make sure your consulting firm understands the basics of architecture.
Competitive qualifications are those that are important but can be negotiated. They are the difference between the consulting firm that gets the business and the firms that don't get the business. The following are some competitive qualifications:
- Does your consultant have a spiral development methodology? Many consulting firms still have an operational waterfall approach to the development of systems. A spiral development approach to building the data warehouse is the only way to fly. Does your consultant know this?
- Does your consultant want to spend huge amounts of time interviewing the end user? While it is true that end-user involvement is crucial, spending huge amounts of time up front in the gathering of end-user requirements smacks of an older operations approach to building systems. The truth is that the end user is not going to know many requirements until the second or third iteration of development is completed.
- Does your consultant understand where a data mart is appropriate and where a data warehouse is appropriate? Does your consultant understand the difference between multidimensional design and normalized design? Does your consultant understand where each design does and does not fit?
- The volumes of data that accompany a properly built data warehouse are enormous. Coping with the volumes of data requires special design skills. In addition, the selection of technology is crucial to the success of managing very large volumes of data. What volumes of data has your consultant encountered? What approaches has your consultant used for managing large volumes of data?
- A data model is required for the building of the data warehouse environment. One approach is to build a custom model from scratch. Another approach is to find and use industry- specific models.
- In any data warehouse, there is a need for movement and transformation of data from the operational/legacy environment. In some cases, this movement is a large issue. In other cases the movement and conversion that needs to be done is not a large issue. One approach is to build the code that is needed by hand. Another approach is to use extract, transform and load (ETL) software for this purpose. Where there is any amount of transformation that needs to be done, the automated approach of ETL software is vastly superior. Does your consultant want to unload a small army of programmers on you to do manual integration and transformation coding?
- As shops become more experienced with data warehousing, they discover that exploration processing is an attractive activity that can be done. Exploration processing requires special design and architectural techniques. Has your consulting firm ever built an exploration warehouse? Has your consulting firm ever done true exploration processing?
- Managed properly, a data warehouse provides good response time and has orderly growth. In order to achieve good management, the data warehouse needs to be periodically monitored. The monitoring of the data warehouse is fundamentally different than the monitoring of the online transaction processing environment. The data warehouse monitor needs to do things such as: determine what data is and is not being used, determine who is using the warehouse, determine whether explorers are using the warehouse and to what extent, and so forth. The sophisticated consultant plans for monitoring from the outset of the data warehouse experience.
- Meta data management is an essential aspect of the data warehouse/ corporate information factory environment. Unfortunately meta data management especially meta data management across the enterprise is not particularly easy to accomplish. Most tools of meta data management have a very local and parochial focus. Notwithstanding that focus, the sophisticated consultant is prepared to address the needs for global data warehouse/corporate information factory meta data management, both at the local level and the global level.
Earlier it was stated that no data warehouse development team should be larger than ten people. Indeed, for optimal effectiveness this is true. However, there are some really large warehouses that are being built that defy this rule of thumb. If your warehouse happens to be one of those very large projects, then the optimal structure is in terms of groups of people working on the project independently. In that case, each group should be no larger than five to ten people.
These are some thoughts and guidelines to keep in mind as you go forward with the contracting process and the selection of a consultant for the development of your data warehouse/corporate information factory. If you follow these guidelines, you have a greater chance of implementing a successful and sustainable environment.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access