Measuring success on an engagement typically begins with revisiting the business case. The business case represents one way to measure project success and answer the question, “How did we do?”
For your BI project, there are additional, nonfinancial gauges for “How did we do?” that should be monitored and reviewed. Ideally, this should happen throughout the engagement as an ongoing activity. Many times, though, project teams get caught up in the delivery and implementation and only ask, “How did we do?” as a part of project close-out or post-mortems. By this time, it can be too late to perform any meaningful rework, if needed, as the project has already wound down. Monitoring and measuring results as an ongoing activity provides the opportunity to take course-corrective actions before it is too late. Doing so will help to ensure that the project itself accomplishes what it set out to.
This is third article in the series entitled “Implementing the Business Intelligence Collaboration: Setup.” Part 1 focused on setup and Part 2 on execution. This article will focus on BI project reporting, metrics and measurement.
Reporting lies at the core of measuring the health of a business intelligence (BI) initiative, which should be easy because reporting is at the heart of BI, right? The reality is that BI teams often only analyze the organization’s data and not their own. Without the proper reporting mechanisms in place, it will be hard to know and gauge what’s been accomplished. Ideally, you should have access to several standardized or canned reports as well as the ability to generate ad hoc reports to help ensure you’ve accomplished what you set out to.
A few basic, yet critical, analytical reports that should be generated on all BI-C projects are discussed below:
- Gap analysis - As discussed in Part 2, the discovery phase is the primary data gathering phase of the project. Based on the extensive interviewing process, requirements, features and releases are defined and documented. Gap Analysis reports highlight how well the BI solution meets the business need. All requirements and features should be available and accessible from a central repository from which gap analysis reports can be generated.
- Requirements gap analysis reporting - Missing requirements are probably one of the most common issues encountered on IT projects. The opposite scenario, implementing more requirements than necessary or planned for (scope creep) is just as detrimental, if not worse. A gap analysis report for requirements can provide the necessary visibility to eliminate the risk associated with both of these scenarios. Metrics that have not been implemented will become visible; undocumented requirements that do not appear in the report but are included in a particular release can be flushed out using this report.
- Features gap analysis reporting - Feature gap analysis reports slice the data in a different way. This report highlights how well the BI features meet the business need. For instance, phase 1 of your project might implement a basic sales report and graphs that can drill into detailed sales metrics. Feature gap analysis reporting may highlight that this sales report meets fice low-priority business needs whereas an executive scorecard will meet 10 high-priority needs. Reporting on your project metadata in this manner will ensure that you are making the best decision for your business when planning your next BI release.
- Lifecycle genealogy - The duration of a BI-C project can vary, ranging from just a few weeks for extremely limited scope projects to multiphase engagements that span multiple years.
In either instance, a thorough audit trail is critical to ensure accurate coverage, align with stated business needs and prevent scope creep. The lifecycle genealogy report is an analytical and multidimensional report that can be used to provide detail on a BI project’s full lifecycle, from interview to release. A comprehensive genealogy report provides detail on linkages, relationships, priorities and impacts (as gathered and documented in the discovery phase). This report provides traceability by detailing in a backward fashion, from release all the way back to establishing the project’s charter. It can be drilled into and across, charted, graphed and manipulated to provide flexibility in getting at the right project information.
