Related Links

Web Seminars

Steve would like to thank colleague Kevin Haas for his helpful comments.

In October of last year, I received an email from a prospect in the IT department of a small company looking to implement an open source business intelligence (BI) application. Included in the note was a side request to help address the return on investment (ROI) of the proposed solution. When we followed up over the phone, ROI was at the front of the discussion. The prospect told me that before he could get serious about the initiative, he had to demonstrate the project's value to the CFO. He wondered if I had a spreadsheet that could provide the foundation for the calculations he needed.

I did have a spreadsheet that could help him tabulate the cost of deployment, but it, unfortunately, provided little guidance on quantifying benefits. And, despite being able to provide at least some assistance, the interaction raised several red flags for me. What was the company’s motivation for BI? Was the business side involved in the effort? Or, was the BI program primarily an IT initiative? How would the company articulate and quantify the business benefits? Like it or not, I was back in the BI ROI business.

In the mid to late 1990s when technology was king, I recall assisting customers with their ROI calculations. At that time, the ROI determination effort often seemed pro forma, almost as if it were a meaningless checklist item. Supported by their ROI analyses, companies were investing heavily in BI. Life was good.

In 2008, there is no shortage of opinion on BI ROI in the thought leadership market. Hardware and software vendors regularly report analyses demonstrating the positive return on their BI products. Go figure. A variant of this vendor-driven ROI review is a product evaluation by a recognized expert or survey firm focused on cost-benefit. Such reviews are generally more thoughtful, using currently accepted BI concepts to frame product ROI considerations. In fact, I've read more than one of these articles just to keep up with the celebrated authors before I even realized they were shilling for a product. While expert ROI reviews can certainly be useful, readers must be mindful that findings are preordained to a positive spin on a product.

Surveys of BI users fully demonstrate the difficulties of quantifying the hard benefits of BI. Many practitioners are content to simply contrast the performance of their BI solutions against going in expectations. Some observe soft benefits such as quicker and more accurate reporting and richer support for decision making. Metrics noted include both an increase in access to information and a decrease in the time of that access. Enabling more users with easier access is cited as well.

There are articles that tout a pure financial calculation approach to ROI, using precise mathematical formulas to specify the costs and benefits of BI. There's certainly appeal from the CFO perspective for such exercises. The problem, however, is that inputs to the formulas are often imprecise, and sometimes arbitrary estimations. Benefit projections three, four, five or more years out are risky at best and spreadsheet jockeys can readily tweak the inputs to build their case. Alas, this approach might be more the rule than exception.

My own philosophy has been to help customers identify the cost side upfront, then make full cost-benefit analysis a topic of a roadmap engagement that kick-starts the BI initiative. The key activity here is a high-level analysis conducted with a performance management lens that links business strategy to measurement through a series of if/then hypotheses. This method both engages business with IT and provides the inputs to solidify an understanding of program ROI. The approach is not unlike the “Business-Centric BI Methods,” espoused by Nancy Williams and Steve Williams in their excellent August 2004, DM Review Magazine article.

Of course, the performance-driven roadmap still requires customers to make a leap of BI faith to devote the time, energy and dollars to conducting the ROI study. Unfortunately, as BI blogger David Loshin commented, companies often will not make this commitment to identifying the benefits of BI. It seems Loshin suffered the same fate with his customer as I did with mine - a process that stalled without the requisite IT/business collaboration to determine ROI.