Continue in 2 seconds

Data Warehouse Project Management

Published
  • May 01 2005, 1:00am EDT

Project management is one of the most challenging responsibilities in a data warehousing initiative. How can you ensure that your initiative achieves the desired results?

For most organizations, business intelligence and data warehousing are expensive initiatives that aren't entered into lightly. Both involve high risk, yet high reward, and are daunting tasks for even the most experienced project managers.

However, failure rates remain high, but not as a result of the integration challenges with evolving technology that we saw in the '90s. Now, there are other reasons causing budgets to be overrun, timelines to slip and the needs of stakeholders and expectations of executive management not to be met.

Best practice recommendations - the proven processes and techniques that are continuously refined through experience and research to achieve a desired result - are an answer to this enduring dilemma. They provide insight and guidance for leveraging technology, mitigating risk, and, most importantly, managing for success. The following list of recommendations, although a small subset, is clearly a step in the right direction. These recommendations have been gleaned from years of "lessons learned" and organized for one of the most challenging responsibilities in a new strategic data warehousing initiative: project management.

Corporate Vision and Data Warehousing Mission

It's imperative that project managers allocate sufficient time to gain a complete and comprehensive understanding of the corporate vision including:

  • Short- and long-term objectives.
  • The strategic plan to achieve those objectives.
  • The challenges/obstacles that may impede the attainment of objectives.
  • Key performance indicators (KPIs) used to measure performance to objectives.

The rationale is simple. The mission for a data warehousing initiative is to advance the strategies and objectives of a business and to ensure the return on investment can be measured in terms of operational efficiencies, business value and satisfaction. This method of strategic alignment establishes a solid, business-driven foundation for governing project life cycle decision making and activities such as prioritization, scope definition, requirements gathering and deployment options.
As the conduit for both stakeholders and project team members, project managers must ensure that no one ever loses sight of the corporate vision and data warehousing mission. It needs to be communicated continuously and, most importantly, at the project kickoff and at the beginning of every phase of the project life cycle.

Project Management Methodology

Project managers must utilize a proven methodology for managing the design, development and deployment of a data warehousing initiative. An experience-based approach will identify potential conflicts and misunderstandings at an early stage of a project, but will also offer the opportunity to reduce project risk, increase project team productivity, control project costs and accelerate the project timeline.

The planning phase of the project life cycle is the communication forum for educating stakeholders and project team members on the project management approach. During this phase, project managers should seek to communicate:

  • Procedures for issue, risk and change management; project management monitoring and communication; project management quality assurance; and meeting schedules.
  • Strategies for stakeholder user acceptance, support, training and marketing; configuration management; document management; backup and recovery; archive and restore; business recovery; ongoing operation and maintenance; and validation and qualification.
  • Project scope.
  • Project success criteria.
  • Project life cycle methodology.
  • The project plan.

The project plan serves as the means to inform all relevant resources of their roles, responsibilities and tasks in the project, as well as the expectations other resources will have of them in terms of availability, adherence to procedures, key milestones and the overall project timeline.

Project Status Monitoring and Communication

Data warehousing initiatives also require the coordination of numerous resources and tasks that must be integrated at the right time to ensure success. Project status monitoring and communication are proven techniques for achieving that objective, as well as for measuring performance, deploying a quality solution and assessing stakeholder satisfaction.

Project managers should monitor status through:

  • Individual status reports submitted by each project team member at the end of each week.
  • Ongoing updates to the project plan.
  • Tracking of variances on costs, milestones, started tasks, completed tasks and task durations.
  • Issue, risk and change management logs.
  • Recorded deviations of developed software that have been measured against business requirements.
  • Recorded deviations of deployed software that have been measured against technical specifications.
  • Recorded deviations from executed test plans.

Project managers should communicate status through:

  • Regularly scheduled Monday meetings with core project team members covering accomplishments, planned activities, issue management, risk mitigation and the updated project plan.
  • Regularly scheduled Tuesday meetings with key stakeholders covering accomplishments, planned activities, issue management, risk mitigation, change management and the updated project plan. In addition, assessments should be made as to whether performance levels, from a project and resource perspective, are being met. If they are not, agreement should be reached on methods to achieve that objective.

Validation and Qualification

Before the planning phase, project managers should meet with key stakeholders to discuss the merits of implementing software validation and qualification procedures within the project life cycle. Validation covers the controls and procedures for documenting the development process (e.g., deliverables, formal quality assurance reviews, etc.), while qualification covers the testing against predetermined specifications, as well as defined system management procedures in a controlled environment.

It's best to thoroughly assess the benefits and consequences of these procedures before settling on an approach. However, this is not an easy task. Validation and qualification is expensive and resource intensive, sometimes adding as much as 10 to 30 percent onto the project cost. On the other hand, the cost for correcting a defect discovered after implementation can be as high as 200 percent of the development cost. Therefore, at a minimum, project managers must take the following points into consideration during this assessment process. Validation and qualification will:

  • Identify defects earlier in the project life cycle, leading to lower project costs.
  • Reduce long-term costs for software maintenance.
  • Infuse a higher degree of confidence among stakeholders on product and service delivery capabilities.
  • Ensure the consistent delivery of accurate, reliable and trusted information for decision-making processes.
  • Enable peer and formal quality assurance reviews, which improves product quality and service delivery, and establishes an optimal educational and knowledge transfer forum for the project team.

Without some form of validation and qualification, there will be business, technical and financial consequences. They include, but are not limited to, discontented stakeholders, lower product quality, longer durations between incremental data warehouse deliveries, and higher project and data warehousing program costs.

Pilot/Proof of Concept

Meeting with key stakeholders to discuss the merits of a pilot/proof of concept prior to the planning phase is advised. This "lessons learned" approach validates the "blueprint" for the data warehouse architecture and the data warehouse program. It not only mitigates the risk accompanying the first "product ionized" incremental delivery, but also builds credibility, support and momentum for the data warehouse in the eyes of the stakeholders and the executive management team.

The scope of the pilot/proof of concept should last no more than 30 to 45 days and include:

  • The deployment of a scaled-down version of the "componentized" data warehouse architecture in a single technical environment that leverages the designed technical processes for data extraction, staging, data verification, cleansing, consolidation and delivery.
  • The utilization of data warehouse software components/tools, such as extract, transform and load (ETL); business intelligence; and data modeling.
  • A multi-tier infrastructure for the database, mid-tier, Web services and client components.
  • The execution of a scaled-down data acquisition strategy from at least one but no more than two source systems.
  • The execution of a scaled-down data delivery strategy using a highly visible and value-add subject area.
  • The application of developed standards and procedures for data modeling, OS scripting, ETL development and administration, and business intelligence development and administration.
  • The execution of a scaled-down software qualification strategy, support and training strategy, and ongoing operation and maintenance strategy.

At the conclusion of the pilot/proof of concept, a joint assessment by both key stakeholders and project team members should be performed on the business, technology and program results. Armed with experience-based insight, informed decisions with actionable tasks can be accurately planned to ensure the success of the first incremental delivery and overall data warehouse initiative.

Data Warehouse Program

Unbelievably, there is more. The project manager's responsibilities extend beyond the realm of the technical infrastructure and the initial implementation in order to ensure that the data warehouse remains sustainable and maintainable. Therefore, before the planning phase, project managers should discuss the merits of the support and training strategy(s), as well as the ongoing operation and maintenance strategy, with key stakeholders.

The rationale is sound. One of the major predicaments within data warehousing today is that an exorbitant amount of time, effort and cost are expended and incurred to implement the data warehouse; however, not nearly enough is allocated to make it sustainable and maintainable. Thus, it is not uncommon to find:

  • Total cost of ownership (TCO) costs running high.
  • Technical environments not being optimally tuned.
  • Service level metrics not being monitored for continuous improvement.
  • Executive management and stakeholders giving unsatisfactory ratings on value and performance.

The strategies described earlier and those that follow address this predicament head-on and form the basis for a best practice data warehousing program.
Support Strategy must cover the roles and responsibilities of a multi-tier support infrastructure (e.g., tier 1, 2 and 3); engineered support processes to maximize the productivity of stakeholders and the utilization of support resources; policies and procedures to promote confidence and reliability in product support; and the governance on service level objectives to monitor and continually improve performance and customer satisfaction.

Training Strategy must cover seven key areas to ensure stakeholders can leverage the investment in the data warehousing initiative to its fullest extent. They include:

  1. Business Value Expectations: How the deployed business intelligence application can incrementally increase after-tax cash flow, advance the strategies and objectives of the business and achieve the defined intangible benefits.
  2. Business Process Optimization: How the application enhances operational processes.
  3. Application: How to leverage the application (e.g., dashboard, report, etc.) to manage an area of responsibility.
  4. Data: What data is available, what it means, why it is important to the business, where it is sourced, how it flows through the architecture and how it is organized and stored for easy access.
  5. Technology: Features and functions of the business intelligence tool(s); how to leverage it for reporting and analytics; and a series of tips, standards and procedures to guide development, publishing, navigation and scheduling.
  6. Data Quality: The type and number of quality checks being performed in the data warehouse, the business rules being adhered to and why the stakeholder should have confidence in the data being consumed for business analytics and reporting.
  7. End-User Support: Available support options. This covers all tiers: help desk support processes, tips and techniques documentation, FAQ documentation and application help documentation.

Ongoing Operation and Maintenance strategy must cover the policies and procedures to monitor, manage, improve and evolve the data warehouse. This includes, but is not limited to: change management; post implementation reviews; data warehouse standards and procedures covering development, configuration, administration, and performance monitoring and tuning; informal education workshops on business intelligence tools and technologies; KPI and service level reports governing utilization, technical performance, support, satisfaction and data quality; support logs that identify deficiencies in service and product delivery; and support repositories comprised of tips, techniques and frequently asked questions (FAQs).

What's the secret to great data warehouse project management? By now, you've probably guessed - it's great communication and planning. Whether dealing with internal constituents, vendors, partners, training or support, the onus is on the project manager to be a great conductor - ensuring all participants in the process remain in concert. It's the collective performance of the team that creates a great data warehouse!

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access