JUL 26, 2007 1:00am ET

Related Links

Innovative Organizations Likely to have More Pervasive BI and Data Governance
September 2, 2014
Revolutionize Your Business Intelligence with Lean, High-Performance Solutions
August 21, 2014
Should You Always Obey Orders from Your Executives?
August 7, 2014

Web Seminars

Why Data Virtualization Can Save the Data Warehouse
Available On Demand
Essential Guide to Using Data Virtualization for Big Data Analytics
September 24, 2014

Chance and BI - Fooled by a Random Black Swan


Random Baseball Champs

Several years ago, my suburban Chicago town's high school won the state baseball championship. The team played terrific ball, running the eight-game, single-elimination playoff table after rising to the top in the early newspaper polls and remaining for the duration of the season. They were quite talented. Two seniors were drafted by major league baseball, and more than half a dozen were recruited to play in college.

Following the final game, a triumphant convocation was held in the school gymnasium to honor the conquering heroes. Coaches, school administrators and local politicos gloated in unison as the championship banner was hoisted. About 45 minutes into the ceremony, the emeritus ex-coach was invited to deliver a homily. He started out dutifully following the chorus of the others - acknowledging the team's skills, work ethic, resolve, resiliency, etc. - but then, unpredictably, veered off track.

Citing the need to remain humble in the face of success, the ex-coach reminded the team that they had caught several fortunate breaks in route to the championship. Had the opposing pitcher in the semifinals not been afflicted with uncharacteristic late-inning wildness, he noted, the champs might never have gotten the opportunity to salvage a game in extra innings they could well have lost in regulation. Or had several bang-bang plays not gone their way in two other nail-biting sectional games, the team might have been eliminated earlier in the competition. His message: enjoy the magnitude of your accomplishment, but remain humbled in the knowledge that there was at least some luck or good fortune in your success. His corollary warning? Be prepared to be on the other side of such fate at some time in the future.

The reception to the ex-coach's words was not very positive, to say the least. Team parents complained the speech cast a pall on the championship. How dare he minimize the team's accomplishments! The ex-coach attempted to clarify his remarks on several occasions, noting he meant no disparagement to the team, seeking only to ground the high schoolers for their volatile life journeys. He was quoted in the local newspaper as saying he witnessed such capriciousness firsthand many times in his 20 years of coaching. Twice when he absolutely "knew" he had the best team, he fell short in the tournament, losing to "inferior" competition. The year his team won state, the ex-coach had felt they'd be lucky to make the round of 16. In his view, talent, preparation - and luck - were needed to be a champion.

The Random Black Swan

Perhaps it is personal bias - or luck - but I seem lately to be uncovering more business-related books that interest me. Not the formulaic, feel-good, how-to-be-successful tomes, mind you, but meaty books that focus on intelligence, analytics, failure, delusions, psychology and randomness. Three years ago, I stumbled on Fooled by Randomness, by Nassim Nicholas Taleb, while browsing in Border's, and enjoyed the read thoroughly. This spring, I purchased Taleb's The Black Swan, a follow-up that examines the role of rare events with an extreme impact. I vacillate in my assessment of Taleb's prose: at one sitting, viewing his style as irreverent; at another, denouncing it as pompous. Even when the latter, however, I forgive him, for the depth and substance of his thinking certainly more than compensates for his meandering style.

Taleb's combined series, what is playfully called Fooled by a Random Black Swan, is very substantive stuff, challenging readers in philosophy, history/history of science, finance, economics, psychology and mathematics/probability. Taleb is an academic tour de force, having earned both MBA and Ph.D. degrees from prestigious schools. A one-time serious quants trader, he now appears to be more engaged in mundane scholarly pursuits.

The point of departure for Fooled by a Random Black Swan is that the human brain sees the world as less random, and conversely, more well-behaved than it actually is. We often mistake pure luck for skill, and indeed, often elevate lucky fools to guru status. We are wired for certainty, determinism and causality, even when they don't exist. We think linearly, continuously and symmetrically, elevating the bell curve to religious status.

Black swans, those events that are outliers, carry extreme impact, and are not predicted - but nevertheless explained post facto - are of much higher importance than we would like to attribute, according to Taleb. He contrasts two types of randomness, the utopian Mediocristan, which is close to equality, and behaves according to the bell curve with continuous progression; and the winner-take-all Extremistan, with extensive skewness in population values and progression in jumps. Mediocristan, typified by a population height distribution or IQ, is impervious to black swans. Extremistan, typified by a population wealth distribution or sizes of companies, is vulnerable to black swans. Alas, our lives are more influenced by Extremistan than we'd like to think.

Financial markets and trader performance are the grist for much of Taleb's randomness and black swan theorizing. He holds "acute successful randomness fools" from the investment world in highest disdain, citing arrogance and utter insensitivity to the role of pure chance in their success. Such fools almost routinely manifest the psychological flaws of overestimating the accuracy of their beliefs, of being married to their positions, of constantly rewriting history, and of denying their failures. More often than not, they regress from seven figure compensations to trader cemeteries overnight.

Random Wisdom for BI

There is much wisdom for both business intelligence and research to be gleaned from the combined Fooled by a Random Black Swan. The assumption of randomness that dominates Taleb's thinking is an excellent point of departure for BI. Randomness, or the absence of a systematic explanation for performance outcomes, represents the null hypothesis that must be disproven by intelligence/research. Only when a well-designed "study" finds strongly related performance factors should the explanation of simply random results be considered for rejection.

Get access to this article and thousands more...

All Information Management articles are archived after 7 days. REGISTER NOW for unlimited access to all recently archived articles, as well as thousands of searchable stories. Registered Members also gain access to:

  • Full access to information-management.com including all searchable archived content
  • Exclusive E-Newsletters delivering the latest headlines to your inbox
  • Access to White Papers, Web Seminars, and Blog Discussions
  • Discounts to upcoming conferences & events
  • Uninterrupted access to all sponsored content, and MORE!

Already Registered?


Comments (0)

Be the first to comment on this post using the section below.

Add Your Comments:
You must be registered to post a comment.
Not Registered?
You must be registered to post a comment. Click here to register.
Already registered? Log in here
Please note you must now log in with your email address and password.
Login  |  My Account  |  White Papers  |  Web Seminars  |  Events |  Newsletters |  eBooks
Please note you must now log in with your email address and password.