Best Practices in Change and Configuration Management for BI Projects

Information Management Special Reports, January 6, 2009

Stacy Bergert, Carl Wohletz, Jr.

Many business intelligence projects consider themselves successful if they simply get deployed into production. As the industry matures, more structure, consistency and a repeatable process for control and deployment of such projects is desired. It is necessary to establish a model for change and configuration management that is flexible enough to support frequent deployments and rigid enough to be documented and repeatable. The approach we recommend is tool agnostic and scales across BI and configuration management tools within the industry. This article highlights the configuration management processes successfully implemented for BI projects and shares some lessons learned in change and configuration management for such projects.

BI Projects Differ from Custom Transactional Software Development, but Configuration Management Goals Remain the Same

BI project implementations differ from transactional custom software development in various ways, including:

• No centralized code package to “build” and “migrate” because most of the projects are completed through the use of tools,
• Projects are data-centric, requiring significant data modeling, mapping and analysis and usually include more cross-project impacts in dealing with source systems,
• End users often access data in an ad hoc or self-service manner instead of through a static feature-driven interface.

While it is necessary to employ many of the same configuration management principles relevant to custom software development to BI work, the process for implementing these principles differs. In the BI world, tool-based configuration items include:

• Data models in modeling tools,
• Objects (mappings, workflows and jobs) in extract, transform and load tools,
• Query and analysis report structures or schemes in BI reporting tools and
  Metadata.

These tools usually do not automatically integrate with an organization’s existing change and configuration management tools, but this does not preclude their use for BI projects. Control, auditability, migration procedures for deployments, impact analysis and employment of best practices are vital configuration management concepts that still apply to BI projects.

Plan Early for Configuration Management Adoption to Ensure Consistency, Transparency, Stability and Control

Practitioners who understand the value of configuration management recognize that the organization can better trust what will be deployed to production when the organization controls the overall change process, documents and archives the necessary configuration items, and ensures a repeatable and consistent process from development completion through production deployment.

Once your organization has bought into the value that change and configuration management can bring, it is useful to have a starting point for implementation. Configuration management adoption should start with project planning that includes business, stakeholder and discipline constraints, time frames and key milestones. BI teams should initiate configuration management practices early in the lifecycle to enable the setup of configuration management as early as possible, so that governance and control can be phased in and have time for continuous improvement. The configuration management aspect of a BI project can be broken into three phases.

Phase 1: Project Setup

Setup starts with configuration and training of the change and configuration management suite of tools. If your organization has not determined the configuration management suite of tools, leverage configuration management industry research experts and articles to jump-start your selection process.

To ensure full traceability throughout the development lifecycle, we recommend you set up a requirements, testing, workflow/change management and source code repository for any BI initiative. Once the tools are in place, make sure to train teams on their expected use.

Phase 2: Configure “Build” Process

The term “build” within the BI world means employing the required steps for versioning of the artifacts intended for release and packaging them together.

Phase 3: Establish Enhanced Traceability

Configuration management processes establish traceability between artifacts from requirements to test cases to code changes. Most configuration management tools allow for traceability throughout the project lifecycle.

Configuration Management Best Practices for BI Require Process Understanding, Prototyping and Iteration

What follows is a list of critical concepts that will help you get started post-project setup and ensure your best chance for overall success:

Start small and adopt products as you refine the process. The planning process mentioned above is iterative. Start with one piece of your BI initiative to flesh out what works. This might be starting with one data mart, your staging area, an operation data store or even starting with one source system. Expect that the process you put in place will change, and make sure the documentation and communication channels are flexible enough to support iterative change.

Implement a consistent process for managing change. Change for BI initiatives often comes from a number of places, including business owners/stakeholder requests as well as impacts from source system changes. The group should decide how changes will be logged, managed, prioritized and updated and then ensure all changes are treated in the same manner. If your organization has a change-tracking tool already in house, it is a good idea to make use of this. If not, find a good spreadsheet template, freeware or shareware to use in the meantime.

Track impacts and dependencies. As you track change, make sure you have a way to also track and monitor cross-project impacts and/or dependencies to ensure coordination is proactive instead of reactive.

Have at least one resource dedicated to configuration management. You will clearly send the message that configuration management is a best practice and priority when you assign the role and set expectations. Depending upon the size and complexity of what you are trying to do, you may not require a full-time resource. The point is to ensure that at least a portion of someone’s time is dedicated to ensuring consistency and traceability, and that the agreed-upon processes are being followed.

Interview stakeholders to determine requirements and tradeoff priorities. Most projects require tradeoffs because most projects run into staffing or budget issues at some point in time. Knowing this, you want to interview management, architects, database administrators, analysts, modelers and developers early in the process to understand goals and priorities to better prepare your configuration management processes for handling tradeoffs.

Figure 1

Figure 2

Page 1 of 2.