Is there any recommendation as to the best way to set up a DW team structure? Is it better to have specialists who focus on their own specific areas (data modeling, extract and load, report creation, etc.) or generalists who can each handle the whole life cycle?


Sid Adelman’s Answer: You will absolutely need specialists. You cannot expect the person who models the data to also be an expert (and you need experts) in the ETL tool. The data warehouse is no place for generalists. Besides, the required level of activity will mean dedicated, full-time people in a number of the data warehouse roles.Les Barbusinski’s Answer: You need both. I’ve never seen a data warehouse or data mart project succeed when the project team consisted solely of narrowly focused specialists. It’s always a recipe for disaster. Someone on the team needs to have the “big picture” in order to insure that the finished product is a cohesive and useful whole…not just an ersatz collection of tables and processes.

This big picture function is usually performed by a data warehouse architect. Someone who is familiar with the business drivers of the enterprise and its industry and experienced in all of the data warehouse technical disciplines including data modeling, database design, source system data analysis, ETL script development, OLAP reporting, infrastructure design, etc. This person is usually responsible for a) defining system requirements, b) developing data models and database designs, c) identifying data integrity rules, d) designing report templates, and e) developing high-level architectural designs.

In essence, the architect is responsible for insuring that the proper source data is accurately and efficiently captured, cleansed, transformed, aggregated, stored, and delivered to the sponsoring organization via the appropriate distribution medium (e.g., Web-based OLAP tool, e-mail reports, wireless alerts, hard-copy reports, ad hoc data extracts, etc.).

Doug Hackney’s Answer: This is very dependent on the size of the organization and the internal/consultant mix. Small and mid-size companies will prize, even demand, multi-role people who can do many tasks. In large teams, multi-talented people are valuable in stepping and covering for people who are on vacation/holiday, sick or who leave the team. Multi-role team members are also valuable as the character of the project changes from a build concentration to a maintenance focus, and in the changing priorities as you work through a BI methodology. Generalist abilities are very valuable for project leads and project managers to prevent technical specialists from hoodwinking the leadership and for quick understanding of the challenges of various subteams at different stages of the project.

Specialist focus is valuable in consultant/contractor talent brought in to do specific tasks, i.e. ETL tool jockey, database tuner, etc. and in projects where there will be long term needs for specific tasks.

Scott Howard’s Answer: The architect and the project manager need to be generalist familiar with the entire life cycle, but you should rely on specialists and experienced technicians for all the specific tasks you mention. For example, a complex DW project is not the place to "stretch" the skills of a talented OLTP modeler. One must be schooled in multidimensional modeling and have the experience to succeed or the rest of the project dependent on this task's success will falter. Now this recommendation is relative to the project's scope and complexity. Very simple projects may very well be completely and adequately accomplished by a generalist, I've just never been witness to such success.

Michael Jenning’s Answer: It would be very difficult and/or costly to assemble a data warehouse team whose members are adequately experienced in the entire data warehouse life cycle (business analysis, source system and infrastructure analysis, ERD and dimensional modeling, DBMS functionality, ETL processing and tool experience, meta data management, data quality and cleansing methods and tools, front-end delivery (query, OLAP, ad hoc) methods and tools, and data warehouse project management). A suggested team structure would be data warehouse project manager, technical architect, ETL developer, data warehouse DBA, business requirements analyst, meta data architect, reporting/access analyst, quality assurance/build administrator, data warehouse operations/maintenance manager. This would need adjustment based on the size of the project and the structure of the organization.

Chuck Kelley’s Answer: I would create a team of specialists who have knowledge of the whole life cycle. For example, there are very few people who are excellent at presentation of data who are equally talented in data modeling or database administration. However, they could do the job if they need to. Having team members do what they are best at, but understand the rest, is what I would be looking for. Using myself as an example, I feel quite comfortable in modeling and building the database but would not want to build the user interface. However, I can build a (for example) Business Objects universe that would allow users to query the data warehouse, but it would not be as pleasing to the eye as someone who possesses the knowledge of presentation aesthetics (one reason is that I still like command mode stuff over mouse driven, but am slowly getting better at the mouse stuff!).

David Marco’s Answer: A good data warehouse team has a mixture of specialists and generalists. For example, if an OLAP tool is used, you will need someone that specializes on that tool. On the other hand, if your front-end developers only know your OLAP tool this will limit your ability to work with other tools or build custom applications.

Larissa Moss’ Answer: The most effective team structure is to create a core team, which operates like a "swat" team. (Think "peer programming" as in XP, or "think tank" as in NASA or JPL). This team would consist of three to five (seven max) individuals who brainstorm together, assign work to each other, review work products for each other, resolve problems together, make decisions together, etc. I suggest that the core team must be composed of specialists because of the complexity of various disciplines (logical data modeling as opposed to physical data modeling, source data analysis as opposed to extract and load, etc.) and because most technicians specialize in one of those disciplines. However, there should be at least one experienced generalist on this team (could be – and in most cases is – the project manager) who has acquired a broad skill set across the whole life cycle. This person can help maintain the think-tank atmosphere and help avoid the handing over of work products that frequently occurs between the specialists.

Joe Oates’ Answer: One of the biggest mistakes that I have seen for a data warehouse project is to NOT get specialists with the right experience and training for the different aspects of a data warehouse project. Specialists are certainly needed for data warehouse physical database design, ETL design and implementation and presentation tool design and implementation. Not having these specialists inevitably results in much higher costs, much longer schedules and in some cases, cancellation of the data warehouse project.

My recommendation to customers is that they get people from the vendor or some company specializing in the particular tool or skill and apprentice the customer people to these specialists. This is the fastest and most effective way of training that I have seen. Of course you also need a project manager who is a specialist in the data warehouse life cycle.

Clay Rehm’s Answer: You will receive a different opinion from as many different people you ask. I would prefer to have specialists such as data modelers, business analysts, ETL specialists, DBAs, data delivery specialists, etc. All of these roles need to be managed by a very good project manager.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access