Steve would like to thank colleague Kevin Haas for his helpful comments.
In October of last year, I received an email from a prospect in the IT department of a small company looking to implement an open source business intelligence (BI) application. Included in the note was a side request to help address the return on investment (ROI) of the proposed solution. When we followed up over the phone, ROI was at the front of the discussion. The prospect told me that before he could get serious about the initiative, he had to demonstrate the project's value to the CFO. He wondered if I had a spreadsheet that could provide the foundation for the calculations he needed.
I did have a spreadsheet that could help him tabulate the cost of deployment, but it, unfortunately, provided little guidance on quantifying benefits. And, despite being able to provide at least some assistance, the interaction raised several red flags for me. What was the companys motivation for BI? Was the business side involved in the effort? Or, was the BI program primarily an IT initiative? How would the company articulate and quantify the business benefits? Like it or not, I was back in the BI ROI business.
In the mid to late 1990s when technology was king, I recall assisting customers with their ROI calculations. At that time, the ROI determination effort often seemed pro forma, almost as if it were a meaningless checklist item. Supported by their ROI analyses, companies were investing heavily in BI. Life was good.
In 2008, there is no shortage of opinion on BI ROI in the thought leadership market. Hardware and software vendors regularly report analyses demonstrating the positive return on their BI products. Go figure. A variant of this vendor-driven ROI review is a product evaluation by a recognized expert or survey firm focused on cost-benefit. Such reviews are generally more thoughtful, using currently accepted BI concepts to frame product ROI considerations. In fact, I've read more than one of these articles just to keep up with the celebrated authors before I even realized they were shilling for a product. While expert ROI reviews can certainly be useful, readers must be mindful that findings are preordained to a positive spin on a product.
Surveys of BI users fully demonstrate the difficulties of quantifying the hard benefits of BI. Many practitioners are content to simply contrast the performance of their BI solutions against going in expectations. Some observe soft benefits such as quicker and more accurate reporting and richer support for decision making. Metrics noted include both an increase in access to information and a decrease in the time of that access. Enabling more users with easier access is cited as well.
There are articles that tout a pure financial calculation approach to ROI, using precise mathematical formulas to specify the costs and benefits of BI. There's certainly appeal from the CFO perspective for such exercises. The problem, however, is that inputs to the formulas are often imprecise, and sometimes arbitrary estimations. Benefit projections three, four, five or more years out are risky at best and spreadsheet jockeys can readily tweak the inputs to build their case. Alas, this approach might be more the rule than exception.
My own philosophy has been to help customers identify the cost side upfront, then make full cost-benefit analysis a topic of a roadmap engagement that kick-starts the BI initiative. The key activity here is a high-level analysis conducted with a performance management lens that links business strategy to measurement through a series of if/then hypotheses. This method both engages business with IT and provides the inputs to solidify an understanding of program ROI. The approach is not unlike the Business-Centric BI Methods, espoused by Nancy Williams and Steve Williams in their excellent August 2004, DM Review Magazine article.
Of course, the performance-driven roadmap still requires customers to make a leap of BI faith to devote the time, energy and dollars to conducting the ROI study. Unfortunately, as BI blogger David Loshin commented, companies often will not make this commitment to identifying the benefits of BI. It seems Loshin suffered the same fate with his customer as I did with mine - a process that stalled without the requisite IT/business collaboration to determine ROI.
Those who share the frustration with the current state of BI ROI can find solace in an excellent Harvard Business Review (HBR) article by professors Clayton M. Christensen, Stephen P. Kaufman, and Willy C. Shah entitled Innovation Killers - How Financial Tools Destroy Your Capacity to Do New Things. As the title suggests, the authors are not big fans of the current obsession with single-minded financial analysis to determine the fate of new business initiatives.
For Christensen, et. al., a new business innovation like BI is often disadvantaged out of the gate by traditional discounted cash flow (DCF) and net present value (NPV) calculations. The problem is that the present value of business with BI is contrasted with a business without BI that is assumed to have unchanging performance. In fact, the authors argue, the do-nothing scenario more often results in a decline in performance over time, brought on by price and margin pressures from changes by the competition. This Parmenides' fallacy is further exacerbated by errors of estimation in the cash flows for out years, three or more years in the future, which compound mistakes in initial assumptions. And for Christensen, et. al., the out years are generally when the decline in performance accelerates in the do-nothing scenario.
All Information Management articles are archived after 7 days. REGISTER NOW for unlimited access to all recently archived articles, as well as thousands of searchable stories. Registered Members also gain access to:
- Full access to information-management.com including all searchable archived content
- Exclusive E-Newsletters delivering the latest headlines to your inbox
- Access to White Papers, Web Seminars, and Blog Discussions
- Discounts to upcoming conferences & events
- Uninterrupted access to all sponsored content, and MORE!