AUG 11, 2009 4:46am ET

Web Seminars

How Intelligent Digital Self-Service with Customer Analytics Can Lower Costs and Raise Revenue
July 29, 2014
Improve Omni-channel Shopping Experience with Product Information Management
August 21, 2014

The Drunkard's BI – Part I

Print
Reprints
Email

I spent last week vacationing in the Outer Banks of North Carolina. As usual, the weather, beach, family, fishing, seafood, etc. were delightful. And as an added bonus, I read what is certain to be my favorite BI book of 2009: The Drunkard's Walk, How Randomness Rules Our Lives, by Caltech physics professor Leonard Mlodinow. Vacation finally provided the book-reading time I couldn't find after encountering Mlodinow's splendid July 16, WSJ article, The Triumph of the Random, which  debunks commonly-held misperceptions of performance “streakiness” in sports and business. What are cited as performance runs of a generation, according to Mlodinow, might often in reality be lucky random sequences. 

A central theme of The Drunkard's Walk is that humans are too quick to assign cause and effect explanations for short-term findings that could well have been the result of such simple randomness. We appear to need the order of cause and effect thinking. Our stories change based on outcomes. Rather than examining the process of getting there, we seem to look backwards from results, manufacturing explanations on the fly to fit the findings, often revising our cause and effect thinking quickly – and comically. “That's why, for example, in spring 2007, when the stock of Merrill Lynch was trading around $95 a share, its CEO E. Stanley O'Neal could be celebrated as the risk-taking genius responsible, and in the fall of 2007, after the credit market collapsed, derided as the risk-taking cowboy responsible – and promptly fired.” The themes of randomness and causality – and their misinterpretations – are certainly not new, having been amusingly explored in Nassim Taleb's provocative Fooled by Randomness and Phil Rosen
zweig's highly-acclaimed business gem The Halo Effect.

Like Mlodinow, I'm fascinated by the attention given to Legg Mason executive Bill Miller (unfortunately, no relationship) and his Value Trust mutual fund performance streak of 15 consecutive years besting the S&P 500 market index benchmark. Before his fund rudely crashed several years ago – in turn, dissipating just about all its accumulated performance advantage over the S&P 500 built up over the years – Miller had been deified by an adoring financial media, his momentous streak testimony to an unparalleled investment prowess. Mlodinow quotes Credit Suisse-First Boston as noting the likelihood of of a fund beating the the S&P for even 12 consecutive years by chance alone to be either 1 in 4096, 1 in 477,000 or 1 in 2.2 billion, depending on assumptions – profoundly different numbers, but all quite small, indicating a highly unlikely event. 

The author, of course, disagrees, opining that Miller's accomplishment, while significant, could  have come from simple random fluctuations. With enough years and enough fund managers, he argues, a 15 year streak like Miller's could easily be the result of luck alone. Indeed, Mlodinow derives a likelihood of 3 in 4 that a result like Miller's steak of winning years would occur at least once in 40 years from a population of 1000 fund managers with investment skill no deeper than coin tossing. He then painstakingly details his probability calculation thinking. 

I was intrigued enough by the apparent dilemma to start my own analysis. My facility with complex probability calculations, however, is not in Mlodinow's league, and frankly isn't what it once was either. So instead of laboring through combinatorics, I decided to write a  few R scripts that use random number generation and Monte Carlo statistical computation to repeatedly simulate answers. The power of the MC technique is that over enough trials, the averages of the counts or other calculated parameters of interest should converge to the actual figures. Better for me to bludgeon an answer with a computer than finesse one with paper and pencil.

For my simulation, I assumed a 40 year time horizon with 1000 fund managers (or business managers or baseball managers – anyone with a dichotomous performance variable), parameters consistent with Mlodinow's. I also assumed that every year represented an independent trial, with a 50% chance each for both success and failure. So for each of the 1000 managers, I generated a random string of 40 zeroes and ones and looked for the largest “streaks” of repeated numbers. I replicated the experiment 1000 times and aggregated the results. 

Non-statisticians might be surprised by the findings. Runs of winners and losers are as much the rule as the exception, even when the chance of success and failure are equal at 50%. A five year run of winning is generally considered a mark of distinction. If you're a football coach it'll get you a new contract; if you're a CEO or fund manager, it'll make you very rich. But over the 40 year period, almost 47% of my “players” experienced winning streaks of 5 years or more, even though they had no skill. Of course, since success and failure are equally likely in the random scenario, the players chances of failure are identical to success – that is, if they're not fired before the streak unwinds. A ten or more year winning run is not out of bounds either, with 1.5% of lucky bumblers in that category.  Finally, my figures yield about a 42% chance of at least one positive streak the magnitude of Miller's 15 given 1000 players over the 40 year period.

That it's easy to confuse randomness with a more satisfying explanation is an important lesson for BI. Prudent analysts will make the random explanation the null hypothesis and systematically seek to disprove it. Without strong evidence to the contrary, the random hypothesis wins. 

Next week I'll discuss several other important points of emphasis of The Drunkard's Walk, including Bayes' Law and the theories of large numbers and measurement, and how difficulties understanding randomness and probability often trip up otherwise intelligent adults, providing more grist for the predictably irrational mill.

Get access to this article and thousands more...

All Information Management articles are archived after 7 days. REGISTER NOW for unlimited access to all recently archived articles, as well as thousands of searchable stories. Registered Members also gain access to:

  • Full access to information-management.com including all searchable archived content
  • Exclusive E-Newsletters delivering the latest headlines to your inbox
  • Access to White Papers, Web Seminars, and Blog Discussions
  • Discounts to upcoming conferences & events
  • Uninterrupted access to all sponsored content, and MORE!

Already Registered?

Filed under:

Advertisement

Comments (5)
I believe the 42% has been achieved assuming the same probability on beating on not beating the benchmark. As in this case the benchmark is a good share of the market, I really would like in reality (i mean with asymmetrical info, insider trading scandals etc) 50% was the probability to beat the market.
Posted by andrea c | Tuesday, August 11 2009 at 2:09PM ET
Some of the review about the book in Amazon seems to be very negative.
Posted by Udayan B | Wednesday, August 12 2009 at 12:07AM ET
Add Your Comments:
You must be registered to post a comment.
Not Registered?
You must be registered to post a comment. Click here to register.
Already registered? Log in here
Please note you must now log in with your email address and password.
Twitter
Facebook
LinkedIn
Login  |  My Account  |  White Papers  |  Web Seminars  |  Events |  Newsletters |  eBooks
FOLLOW US
Please note you must now log in with your email address and password.