What is the relevance of a work breakdown structure in a data warehouse project?

What is the relevance of a work breakdown structure in a data warehouse project? Can you give me an example?

Tom Haughey's Answer: It is often said that warehouses are different. In some ways, they are. Requirements are less tangible and predictable than in OLTP (online transactional processing). Queries are very data intensive, involving few or many tables, but with many, many rows. In OLTP, transactions are data selective, involving few or many tables and comparatively few rows. Metadata is always important, but in OLTP the meaning of fields is predetermined on a screen or report. In a warehouse, metadata (in some way, shape or form) becomes critical because you can ask any question. However, warehouse projects are real projects and, like all development projects, must be managed. To manage them, they need to follow a clear plan. In my experience, warehouse managers often have a more difficult job than those managing OLTP projects because there are so many pieces and sources to manage. Managers that don't follow a good plan typically have major problems, such as lots of rework before delivery, lots of surprises or even eventual failure. Two purposes of the work breakdown structure are to manage work and to ensure success. This is the same as in any project.

However, warehouse projects are unlike typical waterfall projects in that they are based on a RAD (rapid application development) approach. Three of the main principles of RAD are as follows:

Iteration: Division of work into small increments.
Timeboxing: Delivery of capability in short intervals. The first release typically around three to nine months (depending on complexity) and quarterly releases thereafter.
Prototyping: Early delivery of a prototype. Deliver working database one-third of the way through.

(Incidentally, most RAD projects I have observed follow an essentially waterfall process within a given increment. It's sort of like a bunch of smaller waterfalls!)

These three principles are very important because even the best of warehouse plans will invite failure if you ignore them. An example of a failure waiting to happen, even with a fully detailed plan, is a large data warehouse that will gather all requirements upfront and where the warehouse will be delivered all-at-once in three years. It is not the "large" that is the problem, but the "all requirements upfront" and the "all-at-once in three years." Even enterprise data warehouses are delivered piece-by-piece using these three (and other) principles. The feedback you can gather from increment to increment is critical to the success of the future increments. There are many examples of successful enterprise warehouses, but they have all adhered to such principles. [See my answer this month to " An Insurance System Data Warehouse" for an example of a work breakdown structure.]

Danette McGilvray's Answer: A work breakdown structure (WBS) is relevant for any type of project, and a data warehousing project is no exception. A work breakdown structure is a list of project tasks along with deliverables and owners responsible for ensuring the tasks are completed. (The owner may or may not be the person actually executing the task.) The WBS is sometimes known as a project plan, a workplan or a task list.

The work breakdown structure provides a foundation for a realistic project schedule and helps the project team breakdown the deliverables into manageable tasks that can be tracked. By identifying the tasks upfront, it helps reduce surprises about activities, time and resources needed for the project. An additional benefit comes in the form of greater commitment when team members are included in creating the work breakdown structure.

The goal of the task estimation process is to determine the tasks needed to complete the project, estimate effort and create a draft schedule for completion.

When developing the initial task list and timeline with a project team, it is helpful to use a facilitator. The facilitator expedites the process so the project manager and team members are free to focus on determining the actual tasks and effort needed. Depending on the size and scope of the project, you may have more than one project team creating their own project plan for their specific deliverables. The overall project manager then brings together the plans into a master plan. The approach below works whether you have one team or several teams.

Caution: Do not expect this plan to be set in stone. The plan will continue to change as the project progresses; new information is available; scope, resources and priorities change; deliverables are (or are not) completed on time, etc. The process of estimating and modifying your plan should be repeated many times throughout the project. Even the initial plan could take several iterations before you have gathered enough information to create a useable plan.

I will briefly explain a process I have used successfully for many projects.

Determine the general approach, high-level phases and the methodology to be used in the project.

This is usually done by the project manager and team leads. You may be using a methodology provided by a vendor, other experts or creating it yourself.
For example, high-level phases might include plan project; gather data requirements; develop data model; design and develop physical database; source, profile and map data; extract transform, load data; etc.

Gather the project team.

Manual technique:

For each high-level phase in the project, follow the steps below to develop your plan.
One technique is to use large sticky notes or index cards and masking tape. Have team members write tasks on the sticky notes and place on the wall or white board. Use one sticky note per task. It is easy to move the notes as you organize the order. Each sticky note will include as much information as you have at the time: task, owner, time estimates and dependencies.
Once the manual work is done, transfer the information to your project tracking software (such as Microsoft Project).