In a previous DM Review article, "An Intelligible Approach to Enterprise Business Discovery and Information Deployment" (April 1999), I outlined a broad methodology to follow for large scale decision support system (DSS) implementation. As part of this process, Phase 3 Iterative Project Deployment discusses establishing a controllable and repeatable project management planning process. This is certainly a key consideration for instituting successful and sustainable project management practices when undertaking short duration, iterative delivery of DSS business information systems.
Undertaking a sizable data warehouse or data mart project can be a confusing and overwhelming experience. The most difficult aspect of the project is often simply deciding how to begin and how to organize the project. As with any type of project, the key to success is to have a solid project plan. From a technical and architectural perspective, there is much literature available discussing how to build data warehouses and data marts; however, there is much less published information available on successful DSS project methodologies and processes.
Planning and developing DSS projects have unique challenges for project managers. For one, DSS and OLTP systems are not the same type of beast to tame and should be approached differently. The relatively short duration and iterative nature of DSS projects suggest that typical project approaches that are routinely used on OLTP projects may be somewhat inadequate for DSS endeavors. DSS projects do not necessarily deliver a finished, turnkey product as is typical with an OLTP system; rather each DSS project delivers another layer in an evolving information delivery system. Large OLTP development projects tend to be unique by delivering discrete functional capability to the business; however, the nature of DSS projects suggests that these projects are more about data integration than true application development. Thus, they are not as unique as large monolithic OLTP projects. DSS projects, once begun, have an inherent, repeatable nature that seeks to integrate and deliver more information better and faster from more sources via ongoing, evolving data warehousing or data mart efforts.
Generally speaking, most application development projects, especially OLTP projects, call for ground-up planning and uniquely customized project plans to build one-of-a-kind information systems. The detailed content of each individual OLTP project plan remains relatively unique from project to project. The traditional concept of OLTP project planning may likely include the notion of phases within the overall project wherein tasks are organized time-sequentially in large precedence groupings (or phases). For example, tasks organized in the initial analysis phase are completed prior to completing tasks in the succeeding design phase that, in turn, precedes the completion of tasks in the later development phase, etc.
In the DSS arena, projects tend to be smaller scale, shorter duration and very iterative, adding incremental value to an evolving data warehouse or data mart. Contrary to one-time-use OLTP project plans, DSS project plans may be formulated into functional templates to be used over and over again with relatively minor changes reflecting new data sources, new data targets and new delivery time lines. The project plan itself becomes part of the repeatable process of iterative and rapid DSS system deployment.
To assist in the accomplishment of this practice of having reusable project plans, the structure of the traditional OLTP project plan must be further broken down into more manageable groups. The phased structure of organizing tasks is retained; in addition, related functional tasks are grouped by the project planner into logical paths that span across the time-sequential phases. The trick is to successfully identify all of the seemingly disparate tasks that are somehow logically related and need to come together within a single point of control within the project as a logical path.
The initial working document that will be created is the project path map (see Figure 1). Each logical path is identified and defined in practical terms that are appropriate within a given team or organization. Each path will be managed by a path leader with responsibility for the success of that entire subset of the project and will require that particular skills and expertise be brought to bear throughout the project. These skilled resources will more than likely come together from several diverse sources (networking, operations, database administration, business subject-matter experts, etc.), many outside of the jurisdiction of the DSS project manager. Each path leader will manage a matrix organization of allocated persons serving as subject area specialists.
Figure 1: Graphical Representation of a Project Path Map
Next, the logical paths of tasks are integrated within the project planning process to form a cohesive development effort throughout the project life cycle. Project resources and time lines are applied to the tasks, and responsibility for individual tasks is assigned and managed at an appropriate level. This pathing of project tasks creates natural channels of work to be performed and managed; these logical paths serve as reusable templates for each repeated project iteration. This phased and pathed approach to DSS projects is analogous to building a multi- story skyscraper where the general contractor manages the building of the entire structure, while the subcontractors manage related functional tasks repeatedly as the building is erected floor by floor.
This approach to project planning is equally useful for both data warehouse and data mart projects. Of course, planning a data warehouse project will be much more complex and involved than planning less-complicated, dependent data marts.
Identifying Appropriate Paths
Isolating individual project tasks by logical path will make it is easier to delegate responsibility for these sets of related tasks rather than managing each task individually. Each path clearly and completely identifies a set of the pertinent tasks to be performed, their relative precedence and sequence, and their dependencies on other tasks within other parallel logical paths. Resources and time lines have not yet been assigned to the individual project tasks; simply, the purpose here is to identify clearly what is to be done within each functional area (path) within each progressive phase of the overall project.
The number of project planning paths is variable and depends on how finely you wish to isolate and define the functions of the development process. It may be most appropriate to define paths purely by function or process; alternatively, paths may be defined as dictated by the organizational structure of the team or participating groups. Dependencies and interaction of tasks between paths can be easily coordinated by each of the responsible path leaders. In the example presented here, nine paths are proposed within the project plan based on hypothetical functional aspects of the DSS development process.
To begin, create an initial list of possible named logical paths drawn from experience, then simply list the individual named project tasks to be performed within each proposed logical path without regard to precedence. For example, a named path may likely be data migration and within that path may be a named task such as load test data into staging area. The idea here is to create a task checklist within each path that will become the project path map. Later the project path map will be refined into a formal project plan with dependencies and constraints.
As the grouping or pathing process plays out, the initial names of the paths may change several times as logic and reality influence the grouping decisions. Similarly, the individual, initially named paths may split apart into two separate paths or join together into a single path. For example, the initial named path, data migration, may need to be split into two separate paths, data migration in and data migration out.
Once the task list has been completed for each of the proposed logical paths, then thought can be applied to the intra-path and inter-path relationships between the individual tasks. More than likely the number of individual identified tasks will fluctuate throughout the planning process as gaps are filled and redundancies eliminated. Each individual task should result in some specific deliverable or set of deliverables, whether it be a signed contract, a completed functional specification, a final design scheme, a loaded database, etc. Responsibility for the successful completion of all the deliverables for all the tasks within each given path will be delegated to the appropriate path leader within the project team who reports to the project manager.
On smaller projects, the same individual may be responsible for more than one path; however, it is still important to keep the agreed-upon paths distinct, because it is the repeatable process that is important not how one small project must be implemented due to resource restrictions. Combining two non-related paths undermines the integrity of the plan, even though the same person may have responsibility for both paths on the current project.
Hypothetical Sample Paths
For purposes of illustration, several sample paths are presented in Figure 1 that reflect distinct functions within a hypothetical DSS development project. A detailed list and discussion of individual tasks within each of these organized paths is beyond the scope of this publication. The following is a descriptive overview of the individual sample paths offered here.
Sponsor Path: This path is almost always forgotten, even though it has the potential for the greatest impact upon the project's success. The project sponsors have indispensable responsibilities to the project that should be included as part of the project plan.
This path includes functions and tasks related to senior executives acting as project sponsors driving DSS systems deployment as well as senior business and IT directors and managers responsible for implementing these systems. This path should include all of the key executive support tasks including identifying key business metrics, securing financial commitment for short-term and long-term DSS support, commitment to the immediate project, commitment to subsequent projects, expectation management, etc. Path responsibility: senior business executives or senior IT executives.
Project Management Path: Project management activities should be included in the project plan itself. All too often, project management functions and tasks are implied rather than explicitly defined and not allocated resources within the project plan. By including project management tasks as part of a dedicated path, project managers on various iterative projects have well-defined roles based on best practices.
This path includes functions and tasks related to first-level business and IT managers and senior project leaders who are responsible for the day-to-day management of the DSS project(s). This path should include all of the key project management tasks including drafting the overall DSS development plan, defining project scope, creating the DSS project plan, securing professional consulting services contracts, securing project resources, monitoring project schedule, etc. Path responsibility: business project manager or IT project manager.
Technology Architecture Path: The DSS technology architecture may be clearly stated in advance of actual project development work; however, most likely architectural decisions will be an evolving and iterative activity integral with the project development phases. This is especially true if new and innovative technologies are being deployed as part of the project deliverables. If architecture is not defined outside of the project development work, then it must be included within the project as a distinct path.
This path includes functions and tasks related to senior IT developers analyzing, designing and implementing the overall data warehouse and/or data mart technical architecture and development methodology. This path should include all of the key system architecture tasks including identifying technology infrastructure requirements, defining technology architecture, selecting infrastructure products, implementing technical infrastructure, etc. Path responsibility: senior IT systems architect.
Business Driver Path: This includes functions and tasks related to the business users, key subject-matter experts and power users responsible for clearly defining business requirements and ensuring that the DSS systems and applications support those business requirements. This path should include all of the key business and end-user related tasks including gathering business requirements, analyzing business requirements, defining business metrics, validating DSS data, validating DSS pilot systems and applications, etc.
Granted, many disciplines must coalesce to deploy a successful DSS project; however, it is very important to acknowledge that end-user involvement is especially important at all stages of DSS projects much more so than for OLTP projects. This higher degree of user involvement may be important enough to justify the creation of a separate business driver path within the DSS project plan. In the end, this path ensures that a usable system is delivered to appropriate business groups that are ready to use it. Path responsibility: business managers or senior business analysts.
Data Modeling/Meta Data Path: For ongoing iterative development projects, data modeling and meta data must be integrated into existing or developing schemes and structures, if appropriate. For pilot or start-up projects, these necessary schemes and structures must be defined and established as part of this path. The meta data represents the brain of the DSS system, and the data stores represent the heart and blood of the DSS system. Combined as defined by the data model, these elements are the core to a successful, iterative DSS system implementation.
This path includes functions and tasks related to data designers, DBAs and business subject-matter experts responsible for designing and implementing the business-oriented DSS data schemes. This path should include all logical and physical data modeling, resulting database and business rule definitions, and subsequent database build and support activities. This path should also include capturing business and technical meta data, meta data management, meta data sharing schemes, meta data standards, etc. Path responsibility: senior data modeler, senior DBA, data architect or senior business analyst.
Data Migration (Data-In) Path: For most DSS projects, whether building a data warehouse or a data mart, the data-in tasks may represent 60 to 70 percent of the effort of the entire project. Data migration is usually underestimated due, in part, to an assumption that the data is relatively clean and can be integrated easily. However, reality soon reveals that data-in is a much bigger job than initially planned. Without question, data-in deserves its own path to better manage this huge subgroup of tasks. Once successful data-in processes are established, then the overall project resources consumed by data- in tasks in future project iterations may diminish somewhat. For data warehouses and independent data marts, this path entails acquiring and integrating all data from the original source systems. For dependent data marts, this path entails sourcing all data directly from the data warehouse.
This path includes functions and tasks related to source system data extract, data cleansing, data transformation and target system data loading. This path should include all of the key data migration tasks including source system analysis, source/target data mapping, data extraction, data transformation, data cleansing, data staging, data integration, data loading, etc. Path responsibility: senior IT analyst or senior DBA.
Data Migration (Data-Out) Path: For data warehouses, this path entails creating data feeds for dependent data marts as well as supporting direct ad hoc queries not supported by the data marts. For data marts, this path entails supporting direct queries answering business questions and reporting known business metrics. This path is tightly coupled with the business driver path.
This path includes functions and tasks related to the deployment of all business intelligence (BI) software tools involved in the data warehouse and/or data mart project including data migration tools, middleware tools, query and reporting tools, OLAP tools, etc. Tasks include product evaluation and selection, product trial, product purchase, initial product training, product installation, product implementation and product administration. Path responsibility: senior IT analyst or senior business analyst.
Operations Path: The operations path must define all of those tasks leading up to as well as following the successful handover of the new DSS system from the development group to the operations group. Many times, post- development sustainability issues get lost in the shadow of the higher visibility development effort. Ongoing system sustainability is the primary responsibility of the operations path and is one of the long-term measures of project success.
This path includes functions and tasks related to system administration and DBA support for the DSS system and applications. This path should include all of the key operational tasks including defining of operational requirements, establishing operational plans, conducting capacity planning, backup and recovery capability, validating the ongoing operating plan, supporting the pilot data warehouse system, etc. Path responsibility: senior operations analyst or operations manager.
End-User Support and Training Path: The end- user support and training path must define all of those tasks leading up to as well as following the successful handover of the new DSS system from the technology group to the business-user group(s). This path has hidden perils similar to the operations path in that, many times, post- development sustainability issues on the business-user side get lost behind high-technology project deliverables. Ongoing business system usability and sustainability are the primary responsibilities of the end-user support and training path and are also long-term measures of project success.
This path includes functions and tasks related to end-user support as well as training for both end users and technical developers. This path should include all of the key training and ongoing support tasks including defining support requirements, establishing support plans, developing the data warehouse training curriculum, conducting end-user training, establishing help desk support, etc. Path responsibility: senior support analyst, support manager, senior training coordinator or training manager.
Outlining a workable project plan for data warehouse or data mart development should not be an overwhelmingly complicated undertaking. Start small and simply by organizing the known functional project tasks into logically related paths. Then order the tasks within each path by relative precedence. Also, identify dependent relationships between individual tasks in other paths within the overall project. Assign responsibility for the successful completion of each path of tasks to appropriate team leaders. This effort will yield a basic project path map from which the project manager can build a detailed project plan that includes time lines and resource allocations.
The detailed project plan will evolve from the basic project path map as finer project details such as resources and time lines are applied. In the end, by documenting the process and correcting flaws of the iterative DSS project plan, the initial project development effort will result in a well conceived and proven, repeatable project development methodology. Subsequent iterations of the data warehouse or data mart development cycle should flow more smoothly and quickly by following the established methodology and refined project plans. Project scope should always remain small and manageable to sustain rapid implementation cycles.
Previous related articles appear in the April 1999 and June 1999 issues of DM Review.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access