I spent last week vacationing in the Outer Banks of North Carolina. As usual, the weather, beach, family, fishing, seafood, etc. were delightful. And as an added bonus, I read what is certain to be my favorite BI book of 2009: The Drunkard's Walk, How Randomness Rules Our Lives
, by Caltech physics professor Leonard Mlodinow. Vacation finally provided the book-reading time I couldn't find after encountering Mlodinow's splendid July 16, WSJ article, The Triumph of the Random
, which debunks commonly-held misperceptions of performance streakiness in sports and business. What are cited as performance runs of a generation, according to Mlodinow, might often in reality be lucky random sequences.
A central theme of The Drunkard's Walk
is that humans are too quick to assign cause and effect explanations for short-term findings that could well have been the result of such simple randomness. We appear to need the order of cause and effect thinking. Our stories change based on outcomes. Rather than examining the process of getting there, we seem to look backwards from results, manufacturing explanations on the fly to fit the findings, often revising our cause and effect thinking quickly and comically. That's why, for example, in spring 2007, when the stock of Merrill Lynch was trading around $95 a share, its CEO E. Stanley O'Neal could be celebrated as the risk-taking genius responsible, and in the fall of 2007, after the credit market collapsed, derided as the risk-taking cowboy responsible and promptly fired. The themes of randomness and causality and their misinterpretations are certainly not new, having been amusingly explored in Nassim Taleb's provocative Fooled by Randomness
and Phil Rosen
Like Mlodinow, I'm fascinated by the attention given to Legg Mason executive Bill Miller (unfortunately, no relationship) and his Value Trust mutual fund performance streak of 15 consecutive years besting the S&P 500 market index benchmark. Before his fund rudely crashed several years ago in turn, dissipating just about all its accumulated performance advantage over the S&P 500 built up over the years Miller had been deified by an adoring financial media, his momentous streak testimony to an unparalleled investment prowess. Mlodinow quotes Credit Suisse-First Boston as noting the likelihood of of a fund beating the the S&P for even 12 consecutive years by chance alone to be either 1 in 4096, 1 in 477,000 or 1 in 2.2 billion, depending on assumptions profoundly different numbers, but all quite small, indicating a highly unlikely event.
The author, of course, disagrees, opining that Miller's accomplishment, while significant, could have come from simple random fluctuations. With enough years and enough fund managers, he argues, a 15 year streak like Miller's could easily be the result of luck alone. Indeed, Mlodinow derives a likelihood of 3 in 4 that a result like Miller's steak of winning years would occur at least once in 40 years from a population of 1000 fund managers with investment skill no deeper than coin tossing. He then painstakingly details his probability calculation thinking.
I was intrigued enough by the apparent dilemma to start my own analysis. My facility with complex probability calculations, however, is not in Mlodinow's league, and frankly isn't what it once was either. So instead of laboring through combinatorics, I decided to write a few R scripts that use random number generation and Monte Carlo statistical computation to repeatedly simulate answers. The power of the MC technique is that over enough trials, the averages of the counts or other calculated parameters of interest should converge to the actual figures. Better for me to bludgeon an answer with a computer than finesse one with paper and pencil.
For my simulation, I assumed a 40 year time horizon with 1000 fund managers (or business managers or baseball managers anyone with a dichotomous performance variable), parameters consistent with Mlodinow's. I also assumed that every year represented an independent trial, with a 50% chance each for both success and failure. So for each of the 1000 managers, I generated a random string of 40 zeroes and ones and looked for the largest streaks of repeated numbers. I replicated the experiment 1000 times and aggregated the results.
Non-statisticians might be surprised by the findings. Runs of winners and losers are as much the rule as the exception, even when the chance of success and failure are equal at 50%. A five year run of winning is generally considered a mark of distinction. If you're a football coach it'll get you a new contract; if you're a CEO or fund manager, it'll make you very rich. But over the 40 year period, almost 47% of my players experienced winning streaks of 5 years or more, even though they had no skill. Of course, since success and failure are equally likely in the random scenario, the players chances of failure are identical to success that is, if they're not fired before the streak unwinds. A ten or more year winning run is not out of bounds either, with 1.5% of lucky bumblers in that category. Finally, my figures yield about a 42% chance of at least one positive streak the magnitude of Miller's 15 given 1000 players over the 40 year period.
That it's easy to confuse randomness with a more satisfying explanation is an important lesson for BI. Prudent analysts will make the random explanation the null hypothesis and systematically seek to disprove it. Without strong evidence to the contrary, the random hypothesis wins.
Next week I'll discuss several other important points of emphasis of The Drunkard's Walk, including Bayes' Law and the theories of large numbers and measurement, and how difficulties understanding randomness and probability often trip up otherwise intelligent adults, providing more grist for the predictably irrational mill.