At this level of authority, leaders never start out by describing details of budding technologies. They’re more likely to seek passionate reasoning and dispassionate evidence in pursuit of business success. Their greatest concerns are timing, risk and opportunity, not project scale or feature functionality. Whether the issue is big data, mobility or cloud, their daily decisions are typically about when they should leap, how far and why.
It was in this frame of mind that I read the latest cover story in the Harvard Business Review, “Big Data: The Management Revolution,” which, coming from HBR, spoke to this kind of crowd.
What I liked about the story is that, after an explanation of big data and a couple of examples of its use, it was acknowledged that there’s actually little hard evidence right now that using big data will improve business performance. In fact, the authors, MIT professors Andrew McAfee and Erik Brynjolfsson, called it an “embarrassing gap,” (just the kind of reaction you could expect from a top IT leader).
I’ve briefly met Brynjolfsson, a storied researcher who runs MIT’s Center for Digital Business, at a couple of MIT’s events, so I called him to follow up. MIT’s initial response to the missing link, he explained, was to conduct executive interviews at 330 public North American companies, a sample they would marry to a separate set of objective business performance data.
What they found was that the more companies characterized themselves as data-driven, the better they performed on objective measures of financial and operational results. Companies in the top third of their industry in this regard were 5 percent more productive and 6 percent more profitable than their competitors.
It raises an obvious question though: Do companies perform better because they are truly data driven, or do good performing companies merely tend to say or believe they really are data driven?
That’s a concern with the study, Brynjolfsson told me. In order to reach a finding quickly and efficiently, researchers asked a variety of questions about things like how they use data, how much data they have and whether they used it in decision-making.
“But it was ultimately their own self reports,” he said. “We constructed the question so there wasn’t an obvious good answer and obvious bad answer. We just asked how they went about making their decisions. It’s a good start, but we should walk the walk and get more objective data, not just their attitudes,” which he feels nonetheless have a degree of validity.
With proof points still wanting, MIT is now testing other metrics like the amount of storage and the number of Hadoop programmers and users in companies and how that compares to sales and number of customers. Some not yet published information will show correlations between attitudes and these more objective metrics, the professor says, to give more validity to the previous study.
There will be more questions. The examples in the HBR article, as you’d expect, apply vast sets of pandemic disease data or huge retail customer information sets or FAA flight paths mashed up with weather details. It’s not a surprise that Johns Hopkins Medical Center or Sears Holdings or a global airline could make hay with this data; we’re used to Amazon or eBay doing this sort of thing already because it’s their business.
But how universally beneficial will the big data approach be to different kinds of organizations? Is big data the answer to how to sell more Hondas in Peoria? Is it compliant with controls and regulations? What are the risks of being wrong?
The point of all of this is that advocates will soon need to bring a real “A” game to leaders like those you meet at MIT symposia and those we talk to in our annual list. There will be plenty of simple vexing questions to answer. It will require data that speaks to and answers the business challenge, not an assumption that technology is an answer unto itself. These people have seen that before.
It’s about evidence to contradict the HiPPOs in the room, which is how Brynjolfsson and McAfee refer to the highest paid person’s opinion (whose gut instincts bully the fact-based decision process).
You can make two kinds of mistakes right now, the professor says. “You could go too aggressively into big data when you should have been more cautious or you can be too cautious when you should have been more aggressive. Either one of those is a bad thing to do.”
But either way you will be pitching a program to very smart people who know the difference between volume and nuance. Just because it’s big, data can’t be the bully either.
Paige