From 1993-2004, I worked for a startup technology company that enjoyed great success and received many accolades. Actually, the company did extremely well between 1993 and 1999, but struggled 1999 through 2004, when it was sold. Company alumni keep in touch, gathering once a year or so to catch up and reminisce. On those occasions, theories abound on both the successes and failures of the firm. Great people, focused strategies and vanguard approaches are common explanations for success, while mediocre people, diffuse strategies and stale approaches are offered for the decline. Few explanations go beyond the inner workings of the company. Heaven forbid someone suggest the company was in large part the lucky beneficiary of the technology boom from 1993-1999. An unlucky victim of the Internet bust, on the other hand, is certainly palatable to the then-management team.
In reality, there's a randomness to company performance that, as deftly chronicled by Leonard Mlodinow in The Drunkard's Walk, is a much more powerful force than we'd like to believe. That's precisely the message of A Random Search of Excellence, Why “great company” research delivers fables and not facts, by Deloitte Consulting. If The Drunkard's Walk is the best BI book of the year, then A Random Search of Excellence is the year's best BI business analytics article – by far.
Like The Halo Effect and Eight other Delusions that Deceive Managers by Phil Rosenzweig and Hard Facts, Dangerous Half-truths and Total Nonsense, by Jeff Pfeffer and Bob Sutton, A Random Search of Excellence takes on the  “success studies” of management research that purport to have foolproof recipes for winning: if your business adopts the formulas that the success authors prescribe, it, too, will realize the fortunes articulated in their tomes. In Search of Excellence, Good to Great, Big Winners  and Big Losers, and What Really Works are examples of the success genre – and astonishingly popular books in the business world.
The Random Search authors primary problem with these books is that they are inattentive to the possibility they are studying random performance or luck rather than skill with their success companies. “By our measures, they are instead, by an overwhelming majority, studying a sample of firms with performance profiles that are statistically indistinguishable from fortunate random walks. In other words, they are not studying demonstrably great companies, and may very well be studying merely luck companies.”
As is the case with Mlodinow, the Random Search authors use a coin flipping metaphor to illustrate their point. An MIT professor asks all students in her class to stand up and toss a coin. Those with tails sit down, while the remainder flip a second time, and again the heads flippers continue playing while tails sit. After six or seven such rounds in a 70 person class, there is generally at least one “winner” remaining. The professor then asks that student if she can write a case study about his success story. I've duplicated the article's hypothetical 20 year random walk scenarios in R, finding extremes in company “performance” that, even though generated randomly with mean 0, show huge accumulated disparities over time, both positive and negative – certainly grist for success and failure story mills.
A Random Search notes, almost comically, that Campbell Soup is extolled as a winner in What Really Works, which assessed company performance in 1986-1996, while disparaged as a loser in Big Winners and Big Losers, which used an overlapping 1992-2002 time frame. Neither are right, of course: a look at the combined sixteen year period of both studies reveals that Campbell stock growth tracked closely with the benchmark Dow Jones index in the aggregate. Go figure. The moral is that there are many twists and turns to company performance. To get an accurate picture, one needs a large sample of time points spanning multiple economic environments. Check out the stock price performance of Oracle Corporation in the tech boom 1990's; now contrast that with its “growth” since 2000. Even with those extreme ups and downs, does anyone really doubt Oracle's a great company?
Realizing that terms like randomness don't particularly resonate in the hard-charging business world, the authors introduce the phrasing “special causes” and “common causes”  to distinguish company-idiosyncratic performance from common economic and industry environments that impact all competitors. Common causes are akin to “beta” in the investment measurement world; special causes much like “alpha” – a coefficient that differentiates the skills of portfolio managers. Truly great companies exhibit noticeable relative separation in special cause or alpha performance from their peers.
It's all well and good to identify yet again the folly of flawed conventional thinking and a body of management research. A Random Search doesn't stop there, though. In fact, the article's main contribution lies in its well-formulated and rigorous methodology for identifying truly successful companies. The authors first address the issue of what measures are best indicators of company performance, correctly rejecting commonly-used company stock market returns. They settle instead on return on assets (ROA) as the operating indicator they feel best measures the quality of management.
A Random Search uses information from a to-die-for comprehensive business database from 1966 to drive its computations. To hone in on measurable specific causes of company performance, the authors use sophisticated quantile regression techniques to statistically control for common causes like market share, industry, capital structure, age and survivorship. The ROA adjusted for these common causes is thus a more pure estimate of specific firm success.
The regression models  generate “decile rankings” for companies based on the adjusted ROA calculations. Each firm is assigned an ROA performance decile (0 to 9, the higher the better) for every year its performance is available. The authors looked at the movement of companies between deciles from year to year, an example being migrating from the 3rd decile (not so good) to the 8th decile (good). They then ran 1000 bootstrap simulations of 41 years of all 22,000 companies, testing the range of outcomes given the observed variability, providing a solid calibration on the frequencies of adjusted performance. When they finally accommodated their statistical findings with the number of companies “participating”, they got a clear view of success, able to note for example, how many consecutive years of 9th decile ratings constitute great performance.
Needless to say, the number of great companies identified with these rigorous methods is a good deal smaller than those identified by the management books. This is to be expected, of course, since A Random Search eliminates those companies from the “great” list that have simply enjoyed a run of common cause luck. Depending on the variables used in the regression models, the percentage of truly successful companies identified by A Random Search is in the range of just 4% to one third of those in the great companies books – quite a difference, indeed. Maybe there's some truth to the aphorism: “Better lucky than good.”

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access