Data Warehouse Project Management

Information Management Magazine, May 2005

Tim Furey

Project management is one of the most challenging responsibilities in a data warehousing initiative. How can you ensure that your initiative achieves the desired results?

For most organizations, business intelligence and data warehousing are expensive initiatives that aren't entered into lightly. Both involve high risk, yet high reward, and are daunting tasks for even the most experienced project managers.

However, failure rates remain high, but not as a result of the integration challenges with evolving technology that we saw in the '90s. Now, there are other reasons causing budgets to be overrun, timelines to slip and the needs of stakeholders and expectations of executive management not to be met.

Best practice recommendations - the proven processes and techniques that are continuously refined through experience and research to achieve a desired result - are an answer to this enduring dilemma. They provide insight and guidance for leveraging technology, mitigating risk, and, most importantly, managing for success. The following list of recommendations, although a small subset, is clearly a step in the right direction. These recommendations have been gleaned from years of "lessons learned" and organized for one of the most challenging responsibilities in a new strategic data warehousing initiative: project management.

Corporate Vision and Data Warehousing Mission

It's imperative that project managers allocate sufficient time to gain a complete and comprehensive understanding of the corporate vision including:

• Short- and long-term objectives.
• The strategic plan to achieve those objectives.
• The challenges/obstacles that may impede the attainment of objectives.
• Key performance indicators (KPIs) used to measure performance to objectives.

The rationale is simple. The mission for a data warehousing initiative is to advance the strategies and objectives of a business and to ensure the return on investment can be measured in terms of operational efficiencies, business value and satisfaction. This method of strategic alignment establishes a solid, business-driven foundation for governing project life cycle decision making and activities such as prioritization, scope definition, requirements gathering and deployment options.

As the conduit for both stakeholders and project team members, project managers must ensure that no one ever loses sight of the corporate vision and data warehousing mission. It needs to be communicated continuously and, most importantly, at the project kickoff and at the beginning of every phase of the project life cycle.

Project Management Methodology

Project managers must utilize a proven methodology for managing the design, development and deployment of a data warehousing initiative. An experience-based approach will identify potential conflicts and misunderstandings at an early stage of a project, but will also offer the opportunity to reduce project risk, increase project team productivity, control project costs and accelerate the project timeline.

The planning phase of the project life cycle is the communication forum for educating stakeholders and project team members on the project management approach. During this phase, project managers should seek to communicate:

• Procedures for issue, risk and change management; project management monitoring and communication; project management quality assurance; and meeting schedules.
• Strategies for stakeholder user acceptance, support, training and marketing; configuration management; document management; backup and recovery; archive and restore; business recovery; ongoing operation and maintenance; and validation and qualification.
• Project scope.
• Project success criteria.
• Project life cycle methodology.
• The project plan.

The project plan serves as the means to inform all relevant resources of their roles, responsibilities and tasks in the project, as well as the expectations other resources will have of them in terms of availability, adherence to procedures, key milestones and the overall project timeline.

Project Status Monitoring and Communication

Data warehousing initiatives also require the coordination of numerous resources and tasks that must be integrated at the right time to ensure success. Project status monitoring and communication are proven techniques for achieving that objective, as well as for measuring performance, deploying a quality solution and assessing stakeholder satisfaction.

Project managers should monitor status through:

• Individual status reports submitted by each project team member at the end of each week.
• Ongoing updates to the project plan.
• Tracking of variances on costs, milestones, started tasks, completed tasks and task durations.
• Issue, risk and change management logs.
• Recorded deviations of developed software that have been measured against business requirements.
• Recorded deviations of deployed software that have been measured against technical specifications.
• Recorded deviations from executed test plans.

Project managers should communicate status through:

• Regularly scheduled Monday meetings with core project team members covering accomplishments, planned activities, issue management, risk mitigation and the updated project plan.
• Regularly scheduled Tuesday meetings with key stakeholders covering accomplishments, planned activities, issue management, risk mitigation, change management and the updated project plan. In addition, assessments should be made as to whether performance levels, from a project and resource perspective, are being met. If they are not, agreement should be reached on methods to achieve that objective.

Validation and Qualification

Before the planning phase, project managers should meet with key stakeholders to discuss the merits of implementing software validation and qualification procedures within the project life cycle. Validation covers the controls and procedures for documenting the development process (e.g., deliverables, formal quality assurance reviews, etc.), while qualification covers the testing against predetermined specifications, as well as defined system management procedures in a controlled environment.

It's best to thoroughly assess the benefits and consequences of these procedures before settling on an approach. However, this is not an easy task. Validation and qualification is expensive and resource intensive, sometimes adding as much as 10 to 30 percent onto the project cost. On the other hand, the cost for correcting a defect discovered after implementation can be as high as 200 percent of the development cost. Therefore, at a minimum, project managers must take the following points into consideration during this assessment process. Validation and qualification will:

• Identify defects earlier in the project life cycle, leading to lower project costs.
• Reduce long-term costs for software maintenance.
• Infuse a higher degree of confidence among stakeholders on product and service delivery capabilities.
• Ensure the consistent delivery of accurate, reliable and trusted information for decision-making processes.
• Enable peer and formal quality assurance reviews, which improves product quality and service delivery, and establishes an optimal educational and knowledge transfer forum for the project team.

Without some form of validation and qualification, there will be business, technical and financial consequences. They include, but are not limited to, discontented stakeholders, lower product quality, longer durations between incremental data warehouse deliveries, and higher project and data warehousing program costs.

Pilot/Proof of Concept

Page 1 of 2.