This column is focused on measuring business intelligence (BI) success. Topics will cover The BI audit (TBIA) BI Capability Maturity Model, which is a blueprint for the audit of BI assets.
In this fourth and last column in the series of articles on measurement factors for BI assessment (i.e. the key performance indicators), the focus will be on the six KPIs highlighted in Figure 1. All the KPIs have been selected for the BI capability maturity model from business and technical drivers that are well recognized within the industry as keys to success. These are mostly common sense, straightforward, and just a repeat of well known industry terminology. These KPIs are important, however, in the manner in which each will be used in the BI audit– as measurement factors.
The next set of articles will describe the BI asset base, all components and key assessment features.
Figure 1: List of the BI Key Performance Indicators
Performance
Does the information get to the right user at the right time? In most cases, performance measurements in the BI arena mean response times, i.e., the time from the request for information to the return of an answer to the user. This is fine as long as there is an agreed upon and clear definition of the terms. Service level agreements (SLAs) are an important key to ensure consistent Business Information Quality. In any such service level agreement between the BI end user and the IT group, it is important for both groups understand and agree about the specifics. The right time for the user may be within five seconds, two days or two weeks. The IT specialist should know what is possible and only agree to a reasonable right time. These SLA response times should be correlated to the types of information queries, because there may be different response times associated with simple queries compared to complex reporting.
Ease of Use
A BI user may be interested only in standard reports. The BI user may also be a power user or analyst whose expectations are for complex, analytical capabilities. BI products must meet the requirements for each user, in a manner which is intuitive for that user. The interface should also evolve naturally with the capabilities of the user. We are moving toward more structured BI analytical programs and processing, which are focused on carefully leading the user through simple to complex research/processing. Check out the capabilities of your organization during the audit.
Comprehension
Do we have all the information that is required? During the audit, for every part of the BI asset base, this is a key question for assessment. This can apply to a review of the analytical platform and reporting – referring to the answering of a single question, a BI application or a complex analytical process. This assessment should be made, as well, for the Data Integration Platform and that single source of truth, the information presentation library. Have we anticipated the data needs for the organization to meet all the business objectives for some planned period of time?
Product Quality
How accurate is the information? Is it accepted as truth across the organization? How many replications of the same data element exist in how many separate locations across the organization? How do we keep them in synchronization? Or, do we even try? Some of the source data systems for the data warehouse (DW) may be using the same data fields in multiple ways? Do we check that out before we collect and dump it into a DW? Is there a gold or certified copy of the data somewhere in the organization which we know can be accepted as accurate? Are there stewards for the data who are responsible for maintaining the quality of the data? Is the information accurate, timely, up to date and is it consistent across the organization? Does anyone or everyone trust it? Product quality at the heart is data quality. Defining data quality is relatively easy. Ensuring data quality is extremely difficult.
Be the first to comment on this post using the section below.