JUL 10, 2008 2:24pm ET

Related Links

Analytics: Needing to Know or Wanting to Know?
April 9, 2014
Many Utilities Yet to Understand Full Potential for Big Data Analytics
April 7, 2014
Majority of Health Organizations Have Not Implemented BI System
March 6, 2014

Web Seminars

April 29: Create a data protection strategy with open, software-defined storage
April 29, 2014
New Best Practices To Manage Customer Information
May 7, 2014
May 13: Cost-effective, scale-out backup in 1 solution
May 13, 2014

Who Needs Science?

Print
Reprints
Email

It’s a rare thing for a magazine editor to seek an interview with another magazine editor, but in the case of Chris Anderson, the editor in chief at Wired magazine, I didn’t hesitate. Anderson, who wrote the widely-praised book, The Long Tail, is now on a tangent that challenges the precepts of science by proposing that correlations among data are becoming more important than models, and that evidence of something being true can be more immediately useful than understanding why.

 

In a nutshell, “The End of Theory,” Anderson’s essay in July’s Wired, suggests that the vast amount of data and the computing muscle now available to statisticians has empowered them to provide valid observations in the absence of any underlying hypothesis, experiment or testing.

 

This goes a magnitude beyond Tom Davenport’s notion of analytic proficiency as competitive differentiation; it says we can assign confidence to certain truths in the absence of physical evidence. (In a twist, Davenport advocated the scientific method in his most recent column in BI Review/DM Review.)

 

One definition of agnosticism is belief in what can be seen, and we don’t usually put correlations in league with manifest objects, since correlations are relative, not absolute. Yet there are exceptions and precedents: we know of scientific theories that predicted subatomic particles before they were proven to exist, and it’s true that new discoveries occasionally come by accident. Science corrects its own stumbles as new discoveries allow. But with the onset of the data age, Anderson says we can skip some of that linear learning and jump straight to conclusions.

 

In other words, it’s Google. “Google’s philosophy is that we have an abundance of data and a shortage of useful conclusions to draw,” Anderson told me. “They made a business out of making data monetizeable and said, ‘we’re not going to be experts on anything other than data analysis.’ ‘We’re not going to be experts on language or semantics or taxonomy or what people want or why they’re looking for it. We’re simply experts on being able to sift this data statistically so that the latent knowledge in the network is exposed.’”

 

Anderson believes the original PageRank algorithm behind Google deserves a Nobel Prize for its agnosticism toward subject matter.

 

“It says that the network is its own answer, that we don’t know whether this page is more relevant than the next page. What we do know is that the statistics of the network suggest that the crowd believes so. The people who know these domains have voted and we have measured their votes. You end up with a library without theory, without a Dewey Decimal System or taxonomy, a library without an understanding of what’s in it. You end up with this emergent structure that comes out of being completely oblivious to the reasons why that structure exists in the first place.”

 

If I’d had time for a rebuttal, I imagine I’d have heard from Megan Burns at Forrester Research who believes that statistics surrounding Web surfing are more about behavior than about attitude, and can end up being more inferential than illuminating. I’d add that, unlike a scientific proof, behavior on the Web is dynamic and subject to change, even if a measure of truth can be captured in a snapshot. (I also recall that I chose to forget the proofs of calculus as soon as I had my first scientific calculator.)

 

But you’d be wrong to put Google at the root of Anderson’s thesis. A physicist by training, Anderson builds his argument mostly around biological research (which he’ll expand upon in an upcoming co-authored story in the scientific journal Nature). The essay in Wired is followed by a half dozen examples of the trend, including the shotgun gene-sequencing project of J. Craig Venter that retraced the circumnavigation of Captain Cook to discover thousands of new species of bacteria and other life forms that have never been physically encountered.

 

For all the value we attach to data, I still found it somehow perverse that Anderson’s thesis suggests that trust in data has now become an article of faith, and that Venter’s work now sounded existential. If you can prove the existence of a new species without being able to physically describe it, does it really exist?

 

I didn’t have a ready answer, so I asked if it didn’t feel like Huxley’s Brave New World to say, ‘we can believe things because they are so.'

 

“I know where you’re going and I think you’re probably right so I don’t want to over-generalize,” Anderson replied. “I’ve gotten many emails from biologists who don’t think what Craig Venter is doing is science. They say what he’s doing is observation.”

 

That got us into some tricky historical questions about what science actually is.

 

“What Darwin did for most of his career was observe,” Anderson continued. “Only later on did he come up with a theory to explain his observations. Only later did we come up with the scientific method with its hypothesis, synthesis and experiments and testable or falsifiable theories. In a sense, we have created a definition of science that excludes much of the world that could potentially have answers, or at least useful information for science, because we’ve constructed this method that is predicated on testable hypotheses.”

 

Scientific leaps have long been the irreversible measure of our advancement as a species, so I had to ask whether we should expect a qualitative backlash to the quantitative revolution.

 

Get access to this article and thousands more...

All Information Management articles are archived after 7 days. REGISTER NOW for unlimited access to all recently archived articles, as well as thousands of searchable stories. Registered Members also gain access to:

  • Full access to information-management.com including all searchable archived content
  • Exclusive E-Newsletters delivering the latest headlines to your inbox
  • Access to White Papers, Web Seminars, and Blog Discussions
  • Discounts to upcoming conferences & events
  • Uninterrupted access to all sponsored content, and MORE!

Already Registered?

Advertisement

Are you actively evaluating master data management technologies and their ability to scale and support emerging trends around big data, social and mobile?

Yes 61%
No 23%
Don't Know 9%
Not Applicable 6%

 

Twitter
Facebook
LinkedIn
Login  |  My Account  |  White Papers  |  Web Seminars  |  Events |  Newsletters |  eBooks
FOLLOW US
Please note you must now log in with your email address and password.