OCT 25, 2007 8:00am ET

Related Links

Business Analytics Services Spending to Reach $89.6 billion in 2018
September 15, 2014
Artificial Intelligence Meets the C-suite
September 15, 2014
Mayo Clinic to Use Watson to Bolster Trial Enrollments
September 11, 2014

Web Seminars

Essential Guide to Using Data Virtualization for Big Data Analytics
September 24, 2014
Integrating Relational Database Data with NoSQL Database Data
October 23, 2014

OpenBI Forum: Experiments in BI


I love watching football on those weekends in the fall when family demands are minimal. Sitting in front of the TV last Saturday and Sunday, I  had the opportunity to sort through the accumulation of business journals I subscribe to, including WSJ, BusinessWeek, The Economist,  Harvard Business Review, and Forbes. Alas, I'd fallen behind in the reading and was under pressure from the family to clean my periodicals house. What better way to catch up than by watching two college games on Saturday and two pro games on Sunday, all the while perusing wisdom from leading business pundits?


I'm not sure if BI is getting more press now than ever, but over the two days I came across eight to 10 articles that were quite pertinent for our audience. Early in Saturday's first game I decided to write October's column on BI in the business press. By the end of  second game, I had eliminated several articles as content, and by the end of Sunday's pro games decided to further limit my focus to experimentation in business.

{{eval var=$ImageLine assign="Image1"}}{{$Image1|replace:'#Name#':"miller-steve.jpg"|replace:'#Width#':"auto"|replace:'#Height#':"auto"|replace:'#Orientation#':"null"|regex_replace:'/#CaptionLine#(.*)#\/CaptionLine#/':""|replace:'#CaptionWidth#':""|replace:'#CaptionHeight#':""|replace:'#CaptionOrientation#':""|replace:'#ImageCredit#':""|replace:'#AltText#':""}}


Super Crunchers


Regular readers of the OpenBI Forum are probably nauseatingly familiar with my insistence on quality designs for establishing the validity of BI analyses (http://www.dmreview.com/news/1081924-1.html). Of course the platinum standard for design is a randomized experiment in which “subjects” are assigned by chance to treatment and control groups, thereby assuring that differences noted are due to the groups, not confounding factors. This argument was taken to another level of sophistication by Yale professor Ian Ayres in his latest book, Super Crunchers, which I reviewed in September. Ayres argues with a wealth of great examples that super crunching, the combination of predictive modeling and randomized testing, can provide significant lift for programs in government, education, health care and business.


Ayres shows his obsession with experiments again, this time with Yale colleague Barry Nalebuff, in a Forbes Magazine article of September, 3, 2007 entitled, appropriately enough, "Experiment." In this column, Ayres and Nalebuff argue that Wal-Mart, a noted BI juggernaut, could be even more successful with knowledge gleaned from randomized experiments than it is currently with historical observations. If, for example, Wal-Mart conducted tests of different retail space allocation ideas across randomly-assigned stores, the results would be more valid - and perhaps sustainable - than those from historical observation alone. Another illustration where experimentation could add to performance management is an insurance company's evaluation of its underwriting policies. The company will obviously use historical member and claims information for measurement, but that information alone cannot tell anything about prospects that were not accepted, only ones that were. Small randomized tests of those who would have been rejected, on the other hand, might reveal profitability foregone. Progressive Insurance, for example,  found that middle-aged, college-educated motorcycle riders were excellent, and profitable, risks. The Ian Ayres mantra: randomized experiments for business are important adjuncts to expert opinion for the formulation of business strategy. And randomized testing, especially in the Internet age, may be much less intrusive - and more accessible - than often thought.


Tom Davenport, author of Competing on Analytics, agrees with Ayres and takes the thinking one step further. In a Wall Street Journal interview, Davenport responded to a question on the foundation for successful deployment of business analytics: “I'd say the number one factor is really a leadership dimension. It's how committed are a company's senior executives to fact-based and analytical decision-making and the whole idea of experimentation as a way to learn rather than doing it out of gut feel or intuition.”




I'm a big fan of Amazon and its CEO, Jeff Bezos. Since the advent of Amazon Prime, which essentially covers an account's unlimited annual shipping costs for $75, I buy my books, CDs, DVDs, camera equipment, holiday gifts - just about anything I can - from Amazon. Amazon's collaborative filtering recommendation engine has probably sold me a dozen books and DVDs over the last two years alone. I continue to be amazed that I can order a book at 4:30 p.m. and have it delivered by 10:30 a.m. the next day.


Even as Time Magazine's Person of the Year for 1999, when it seemed all the market experts were trumpeting a new economic order fueled by the Internet that was different this time,  Bezos had his feet firmly planted on the ground, realizing his work to build a successful company was just beginning. Amazon is committed to germinating its strategy, patiently letting the model play out in the marketplace. And Bezos is not bullied by Wall Street experts on how he should run his company. The decision to launch Amazon Prime, decried as a money loser by analysts, has instead proven quite successful. What Amazon might be subsidizing in shipping costs it is more than recouping from the resulting deeper customer relationships. The market has acknowledged Amazon's success as well. The Amazon stock price is up fourfold in the past five years, doubling in the last year alone.


Get access to this article and thousands more...

All Information Management articles are archived after 7 days. REGISTER NOW for unlimited access to all recently archived articles, as well as thousands of searchable stories. Registered Members also gain access to:

  • Full access to information-management.com including all searchable archived content
  • Exclusive E-Newsletters delivering the latest headlines to your inbox
  • Access to White Papers, Web Seminars, and Blog Discussions
  • Discounts to upcoming conferences & events
  • Uninterrupted access to all sponsored content, and MORE!

Already Registered?


Comments (0)

Be the first to comment on this post using the section below.

Add Your Comments:
You must be registered to post a comment.
Not Registered?
You must be registered to post a comment. Click here to register.
Already registered? Log in here
Please note you must now log in with your email address and password.
Login  |  My Account  |  White Papers  |  Web Seminars  |  Events |  Newsletters |  eBooks
Please note you must now log in with your email address and password.