# The Power of Metrics

Published
• May 01 2005, 1:00am EDT
More in

This month, we will continue our discussion on the use of control charts for setting key performance indicator (KPI) threshold levels. In its basic form, the control chart is a simple plot of a time sequence of group statistics (i.e., KPI data points). The points are then compared to control limits (i.e., KPI threshold levels), which capture the bandwidth of the variation due to the common causes (see April 2005, p. 47). As long as the KPI data points are within the control limits, then the underlying process being measured is statistically in control. This concept can also be used to develop KPI warning limits.

#### Control Chart Elements

As you may recall from your college Stat 101 course, stable business process outputs are randomly distributed with a mean µ and a variance s2 (or standard deviation of s). The baseline on the control chart is typically the arithmetic mean for all the KPI sample data plotted. If the outputs are normally distributed, it is a well-known statistical result that 99.74 percent of these output averages are within three standard deviations of the mean. In statistical parlance, the three standard deviations are referred to as the upper control limit (UCL) at +3s or the lower control limit (LCL) at -3s. In addition, there are also upper warning limits (UWLs) at +2s and lower warning limits (LWLs) at -2s that capture 95.0% of the output averages (or KPIs). The control and warning limits will be extremely helpful in identifying KPIs that are out of control, KPIs that are experiencing problematic trends and KPIs that are within the allowable process variability. The basic elements of the control chart are illustrated in Figure 1. In addition to this x-bar chart, a companion range chart can also provide information about the spread of the sample KPI metrics.
Figure 1: KPI x-bar Control Chart

#### Control Chart Tests

The rationale behind control charts is to quantify variation and identify problem areas by observing patterns in the KPI metrics plotted on the control chart. As we mentioned earlier, the distance from the mean baseline to the control limits can be divided into three equal parts of one sigma each. Where the sample KPI data points fall relative to the different sigma levels over time dictates whether the business user needs to be concerned about immediate process control problems, just track the process closely to catch any apparent emerging trends or not be concerned at all. A portfolio of statistical pattern detection rules have been developed to alert management to the current state of the business. The pattern detection rules in turn drive the combination of green, yellow and red visuals (i.e., stoplights, beacons and flags) that indicate the current state of the business. Following are several examples of the pattern detection rules.

A KPI visual would appear red if any of the following conditions occurred:
Individual data point rules

• any individual KPI data point above +3s or below -3s
• two out of last three KPI data points above +2s or below -2s
• four out of last five KPI data points above +1s or below -1s

Trend rules

• six KPI data points in a row trending up or down
• eight consecutive KPI data points on one side of the baseline
• 14 KPI data points in a row alternating up and down

A KPI visual would appear yellow if any of the following conditions occurred:Individual data point rules

• any individual KPI data point between +2s and +3s or between -2s and -3s
• three out of last five KPI data points above +1s or below -1s

Trend rules

• four KPI data points in a row trending up or down
• six consecutive KPI data points on one side of the baseline
• 2 KPI data points in a row alternating up and down

Alternatively, a KPI visual would appear green if none of these criteria are met.
Next month, we will illustrate how a KPI control chart can be employed to develop threshold levels and track ongoing trends.