KPI Reduction the Correlation Way

Published
• February 01 2005, 1:00am EST
More in

Once again we will tackle the challenge of reducing the 1,000+ metrics shipped "out-of-the box" with business intelligence and performance management applications to a manageable 10 to 15 key performance indicators (KPIs). Last month we discussed affinity analysis as one potential approach. This month the focus will be on a Six Sigma technique called correlation analysis, which will allow us to understand the relationships between individual metrics while simultaneously identifying potential KPI candidates. The e-commerce case study in this column will illustrate how correlation analysis can highlight the relationships between Web metrics and bottom-line financials. The approach is to first determine which metrics are related to the business financial drivers. The second step is to discover the interrelationships that exist among the Web metrics themselves. This prevents the inclusion of any two Web metrics that might be measuring the same process or outcome. The result of the analyses will be a reduced set of KPI metrics that are uncorrelated with each other, and yet related to financial impact.

Correlation analysis allows us to measure the degree of linear relationship between two variables. As you may recall from STAT 101, the correlation coefficient may take on any value between plus and minus one. The sign of the correlation coefficient (+ or -) defines the direction of the relationship, either positive or negative. A positive correlation means that as the value of one variable increases, the value of the other increases; as one decreases the other decreases. A negative correlation coefficient indicates that as one variable increases, the other decreases; as one decreases the other increases. Alternatively, the absolute value of the correlation coefficient measures the strength of the relationship. A correlation coefficient of r =.8 indicates a strong positive relationship between two measures, whereas a correlation coefficient of r =-.6 indicates a less powerful, albeit strong negative relationship between the two measures.

Figure 1: Web Metrics vs. Financials Correlation Matrix

In our e-commerce case study (see Figure 1), the correlation coefficient measures both the level of magnitude and directional impact of Web metrics (e.g., stickiness, frequency) on the bottom-line financials (e.g., revenue, operating income). Stickiness thus appears to have a significant relationship to revenue with a correlation coefficient of .9 (the value in the cell where stickiness and revenue cross). This makes sense because stickiness is a term used to characterize the attractiveness of a site, section or page, and is measured by the average number of page views, session duration and page depth. Thus stickiness today could be an excellent predictor of revenues tomorrow. It is also important to note that stickiness and page views are highly correlated at a .8 level. This implies that both are measuring similar characteristics, and that stickiness can be used as the KPI (and as a surrogate for page views). Keep in mind that the values above the diagonal mirror those values below the diagonal, and only those values of .5 or higher are shown. Although this is a simplistic example, this approach can be used with all "out-of-box" metrics to determine which metrics are the true KPI metrics. As a final note, it is important to remember that correlation does not always indicate causation. Before causation is validated, the relationship must pass an important criteria - the causal variable must always temporally precede the variable it causes.