You've just been assigned to build a data warehouse. The term is not totally unfamiliar, as you've heard it at user group meetings and seminars, in idle conversations around the shop, to say nothing of the articles on the topic in the trade press or on various Internet sites in recent years. Just the same, you're not sure what one is, and how to go about actually building such a thing is as foreign as the Antarctic ice fields.
You are an experienced project manager, so you know you need a team to complete your assignment, and you have a general idea of what kind of folks belong on the team. The questions you need answered, then, relate to the details. How many people will you need? What skill sets are required? What role or roles will each person play? This article will attempt to provide some information to assist in the definition of roles and responsibilities for a data warehousing/ business intelligence project team.
Evalute Scope of Project
The first step is to evaluate the size and scope of the project, the goals to be achieved, level of management support and commitment, and other related factors. If the project involves creation of an enterprise data warehouse (EDW) with analytical and business intelligence tools, possibly data mining capabilities, and so on, you are looking at a multiyear, multimillion dollar project with potentially several teams and a large staff. (See Figure 1.) Development of a departmental data mart is likely to be a much smaller effort and will usually be completed in a much shorter time frame than an EDW. Also, smaller organizations with fewer transaction processing applications and less need for historical information are typically going to have smaller data warehouse projects.
Figure 1: Organization Chart for Large DW Team
While the number of people involved in the development of a data warehouse varies rather significantly depending on the size of the firm and the size and complexity of the project, many or all of the roles discussed in this article will need to be performed by someone during the course of the project. The number of people staffing the project will be dependent on the resources and budget, of course. What is the minimum required? Bill Inmon has said four. I'm not sure the answer is that cut and dried. I can see the potential for a one- or two-person project in a smaller organization. Using some of the newer toolsets, especially where the entire toolset comes from a single vendor, it is possible to build a complete data warehouse application with a very small team. (See Figure 2.) This is probably more likely to work on a data mart project than an EDW.
Figure 2: Organization Chart for Small DW Team
Such a small team is not likely to be the norm, or anything approaching the norm, in the near future. For the most part, data warehouses require a lot of planning, consume significant resources and are not going to be built by anything less than a fairly good-sized team that includes a broad set of skills. In many cases, a team of four members is probably going to produce an overworked and overstressed group, one prone to mistakes and errors in the development stages.
The roles and responsibilities identified in Parts 1 and 2 relate to development of a new data warehouse or data mart, not the support of an existing warehouse or mart which is the subject of Part 3. Also, this is not intended to be a definitive list, but rather a guide to the basic roles you might expect to see. Some roles might be combined or other roles might be needed which aren't defined here based on the specifics of your organization and project.
Data Warehouse (DW) Manager: (The actual titles can range from vice president of data warehousing in very large firms with large DW organizations to project manager in smaller companies and/or where the level of commitment to data warehousing is very limited.) This team member is responsible for working with and informing top management of the usage and impact of data warehousing within the organization, identifying and proposing projects which utilize data warehousing and business intelligence to provide support for corporate goals, managing and staffing projects, defining and gaining approval for budgets and schedules, and educating users on DW/BI applications and capabilities.
In many cases the DW manager will coordinate the activities of multiple team leads and analysts as they perform the development for different areas and/or iterations of the EDW and/or dependent data marts. In cases where the company is beginning its initial forays into data warehousing and business intelligence activities, the DW manager will be responsible for the initial requirements definition and selection of the first DW team members.
The DW manager may also work with end users and business applications analysts to develop high-level business process documentation which can be used to define DW/BI project priorities and iterations. As the data warehousing team grows and development continues, the DW manager becomes the liaison with senior management, the coordinator of the efforts of different project teams and primary decision- maker for design issues. In large projects, the DW manager will do little or no actual development work. Rather, this person provides the broad vision and direction, and works with senior management and end users to ensure that the team's efforts will provide the desired results. In smaller teams, this person may also function in development roles.
