Ive been a subscriber to the Harvard Business Review (HBR) for about five years. In the aggregate, Im pretty happy with the articles, almost always reading at least two in their entirety. The writers - academics, consultants and prominent business professionals - at times provide keen insight on pertinent strategic and operational business concerns. I generally find the taxonomies or problem domain dimension specifications very useful. At least every other edition has something spot on pertinent for business intelligence (BI).
HBR is a marketing juggernaut as well. Im continually bombarded with emails and mailings on the latest HBR publications from business books, special finance, accounting, supply chain and customer relationship how tos to downloads of past magazine articles. And I must admit, the marketing often succeeds, though I purchase through Amazon.com when possible to save money.
Not too long ago, HBR launched a paperback series entitled Harvard Business Review on , that spans a wide range of on topics ranging from corporate strategy and marketing to leadership and supply change management. Each of the more than 50 titles is actually a collection of pertinent HBR articles from the extensive archives. The books can be quite convenient aggregators of information on specific business topics. Indeed, its interesting just to note the evolution in thinking in a given area over time.
I recently purchased Harvard Business Review on Making Smarter Decisions and read several articles in sequence that struck me as particularly relevant for educating executives on the business benefits of BI. The Hidden Traps in Decision Making identifies the business problem of flawed thinking that subverts many management decisions. Evidence-Based Management proposes a business solution for decision-making that revolves on a management embrace of hypotheses, facts, research and a supportive culture for an evidenced-based approach. Finally, Competing on Analytics, the technical solution, is the proverbial killer application that puts EBM in play with strategic commitments to data, intelligence and analytics. Interestingly, both Evidence-Based Management and Competing on Analytics were precursors to Harvard Business School (HBS) books that have received wide acclaim in the management and intelligence worlds.
Its not difficult to find writings on flawed management decision-making and its impact on firm performance. Every other issue of HBR, it seems, has a case study post mortem of a failed business initiative. The OpenBI Forum has written on The Halo Effect and Fooled By Randomness, both of which identify multiple examples of delusional thinking that have afflicted otherwise successful companies. A personal favorite is Why Smart Executives Fail, an academic survey and analysis of major corporate failures.
The Hidden Traps in Decision Making is a handy compendium categorizing the psychological missteps that can lead to flawed thinking and sabotage the best-intended decisions. The culprit is often not incompetence, but instead, heuristics, a process of discovering by intelligent guesswork rather than by evidence from an organized hypothesis. Heuristics ease decision-making by psychologically reducing complexity, often incorrectly simplifying the process. Rule-of-thumb, seat-of-the pants and other predispositions characterize potentially dangerous heuristic thinking.
Several psychological traps that befuddle business decision-makers are elaborated in The Hidden Traps in Decision Making. The anchoring trap is related to the often observed primacy effect, wherein disproportionate consideration is given to early information and initial impressions. The status quo trap, not surprisingly, is a bias towards the current order. In business, the status quo is often the easiest and seemingly less risky path. The confirming evidence trap leads us to seek out evidence that supports our perspective and interpret new information consistent with our biases. The framing trap can lead to different answers to essentially identical questions, depending on how the questions are positioned. The estimating and forecasting traps sometimes manifest in overconfidence in projections, sometimes in overly conservative estimates and other times in overemphasis on the ability to recall inputs to the projections. Alas, all traps play tricks on our decision-making abilities, often leading us unwittingly down a destructive decision path.
So whats the antidote to the flawed thinking that often drives business decision-making? How can businesses assure that damaging heuristics and delusions do not subvert their problem-solving approaches? Evidence-Based Management offers insight from a movement underway in medicine: the idea that decisions in medical care should be based on latest and best knowledge of what actually works.1
Dr. David Sackett and colleagues at McMaster University promote evidence-based medicine (EBM) as: the conscientious, explicit and judicious use of current best evidence in making decisions about individual patients.2 EBM is committed to applying and disseminating the latest and best evidence from medical research to the treatment of patients. Physicians who practice EBM opt to make decisions based on hypotheses, research and evidence, instead of dogma, obsolete medical school information, conventional but never-proven wisdom and treatments with which they are most comfortable.
The tenets of evidence-based medicine are relevant as well for business. Evidence-Based Management contrasts true evidence-based thinking with casual benchmarking, an often delusional exercise wherein companies uncritically emulate what are perceived to be the differentiators from best-performers in their space. Unfortunately, the research behind the benchmarking is often superficial, generally flawed and seldom distinguishes the factors that truly differentiate performance.
An evidence-based approach is built on the scientific method: generate hypotheses, conduct research, assemble evidence, draw conclusions and take actions. EBM represents a mindset change for management, mandating that it put aside conventional wisdom and experience for the results of testing and facts. Management treats the organization as an unfinished prototype and encourages trial programs, pilot studies and experimentation.3 The organization assumes an attitude of wisdom that humbly appreciates what it doesnt know, but acts on the best knowledge available while constantly questioning what it does know.
Management commitment to research for testing hypotheses and establishing evidence is critical for the success of EBM. EBM leadership companies buy in to rigorous methods for evaluating different approaches, often conducting small experiments to test new ideas. Yahoo typically has 20 or so experiments running at any time, manipulating site features like colors, placement of advertisements and location of text and buttons. These little experiments can have big effects.4 eBay CEO Meg Whitman opines: This is a completely new business, so theres only so much analysis you can do. Its better to put something out there and see the reaction and fix it on the fly.5
Perhaps the most critical determinant of the success of EBM is the willingness of the management team to put its decision-making ego aside, deferring to evidence. Evidence-based practice changes power dynamics, replacing formal authority, reputation and intuition with data.6 Jack Barksdale, then CEO of Netscape, put it best: If the decision is going to be made by the facts, then everyones facts, as long as they are relevant, are equal. If the decision is going to be made on the basis of peoples opinions, then mine count for a lot more.7
Competing on Analytics takes Evidence-Based Management to the next level of operational detail, proposing specific strategic, cultural and technological foundations for those companies wishing to lead the intelligence pack. For analytics powerhouses, it all starts with the CEO, who professes that analytics are central to company strategy. Senior executives commit to changing the way employees think, work and are evaluated, driven by an evidenced-based mandate. Aphorisms like: In God we trust. All others bring data. and Do we think this is true. Or do we know? become company mantras.
This strategic commitment is, of course, just a starting point. For leaders, the commitment to analytics is clearly articulated across the enterprise, focused on areas of optimal strategic benefit. Analytics companies establish a culture that is evidence-based, instilled with EBMs attitude of wisdom and enlightened trial and error. Leaders obsess on hypothesizing, measuring, evaluating and adapting, using the most rigorous research methods at their disposal. They experiment, often at small scale, using positive results to expand initiatives.
And, of course, analytics leaders make significant investments in technology to include supportive hardware, corporate intelligence data stores and BI platforms, as well as predictive modeling and optimization software. They strategically hire analytics experts and deploy/manage their programs across the enterprise. They put management and governance processes in place to assure their commitment to analytical decision-making is resolute. In short, they make evidence-based decision-making supported by analytics a top strategic priority. In the end, people and strategy, as much as information technology, gives such organizations strength.8
Musings on the ROI of BI Redux
The January 2008 OpenBI Forum column, Musings on the ROI of BI, discussed the frustrations companies often experience attempting to demonstrate the return on investment of BI. Soft benefits and capricious net present value analysis based on shaky assumptions seem the rule. The besieged BI team is on the defensive, struggling to justify continued support for their initiatives through subsequent stage-gate reviews. Evidence and intelligence live on the edge.
Companies that practice evidence-based management and make the strategic investment to compete on analytics, in contrast, probably see the ROI of their initiatives much differently. Rather than as a complement to their business processes that is evaluated as a stand-alone investment, analytics companies see their intelligence investments as central to their operations to their very existence. Analytics leaders can no more see themselves without the requisite intelligence infrastructure than they can without ERP or supply chain systems. The BI ROI considerations for evidenced-based, analytics-committed organizations are different, focused strategically, not tactically. It is when a company acknowledges this difference that it can be confident its on the way to analytics leadership.
- Jeffrey Pfeffer and Robert I. Sutton. Evidence-Based Management. Harvard Business Review on Making Better Decisions. Harvard Business School Publishing Corporation, 2007.
- David L. Sackett. Evidence-Based Medicine What It Is and What It Isnt. British Medical Journal, 1996.
- Jeffrey Pfeffer et al.
- Jeffrey Pfeffer et al.
- Jeffrey Pfeffer et al.
- Jeffrey Pfeffer et al.
- Jeffrey Pfeffer et al.
- Thomas H. Davenport. Competing on Analytics. Harvard Business Review on Making Better Decisions. Harvard Business School Publishing Corporation. 2007.
- John S. Hammond, Ralph L. Keeney and Howard Raiffa. The Hidden Traps in Decision Making. Harvard Business Review on Making Better Decisions. Harvard Business School Publishing Corporation: 2007.
- Jeffrey Pfeffer and Robert I. Sutton. Evidence-Based Management. Harvard Business Review on Making Better Decisions. Harvard Business School Publishing Corporation: 2007.
- Thomas H. Davenport. Competing on Analytics. Harvard Business Review on Making Better Decisions. Harvard Business School Publishing Corporation: 2007.
- Jeffrey Pfeffer and Robert I. Sutton. Hard Facts, Dangerous Half-Truths & Total Nonsense Profiting From Evidence-Based Management. Harvard Business School Press: 2006.
- Thomas H. Davenport and Jeanne G. Harris. Competing on Analytics - The New Science of Winning. Harvard Business School Press: 2007.
- Phil Rosenzweig. The Halo Effect - and the Eight Other Business Delusions That Deceive Managers. Free Press: 2007.
- Nassim Nicholas Taleb. Fooled by Randomness - The Hidden Role of Chance in Life and in the Markets. Thomson, Texere: 2004.
- Sydney Finkelstein. Why Smart Executives Fail, and What You Can Learn From Their Mistakes. Portfolio: 2003.