Doug Laney is a research VP at Gartner Research covering business analytics, big data use cases, “infonomics,” and other data-governance-related issues. He has also led a number of international analytics and information-management-related projects. Before joining Gartner, he launched the Meta Group's Enterprise Analytics Strategies research and advisory service and established and co-led the Deloitte Analytics Institute. He recently talked with Information Management’s editorial director John McCormick about one of the biggest big data challenges – the variety of data. What follows is an edited transcript of their conversation.


You have an interesting take on the biggest big data challenge. Explain it for us.

When we talk about big data, we characterize it in terms of the three-dimensional challenges -- the challenges of the increasing volume of data, velocity of data, and variety of data.

From a data management standpoint, the biggest challenge is the variety of data.

Why is that such a big challenge? 

Over the last 20 years or so, the number of datasets that an organization is trying to make sense of has grown from just being a customer database and a sales database and a production database and a human resource database to include datasets that are unstructured -- things like email and data coming in from Internet of Things and the variety of data that comes from endogenous data sources.  And then there’s outside data sources from partners, from suppliers, from customers, from open data sources like government organizations.  And from social media.

Leading companies are thinking much more broadly about the wealth of data that’s available outside their own four walls and thinking about their information ecosystem. These organizations that leverage data from outside data sources are going to be the ones that drive and survive.

Now, the challenge with big data is not around technology; it’s around leadership and skills -- leadership from the perspective of going beyond basic hindsight-oriented business intelligence.  We still see so many organizations caught up in responding to users who are telling IT “give me my data,” and “make me a pretty pie chart,” stuff like that.  And that’s not where the value in big data lies. 

The value of big data doesn’t lie in building bigger pie charts.  It’s in the advanced analytics that become more meaningful and utilitarian when you have this broad-based and deep detailed data.

Patterns and semantics and sentiment analysis and machine learning and all those kind of techniques can be a bit scary to business leadership and IT leadership because they don’t understand those techniques very well. Analytics becomes a bit of a black box for them.  And so getting leaders to appreciate the use cases for these various kinds of techniques and the kind of technologies that go beyond just traditional SQL RDBMSes is a bit of a trick.

So, who in the organization gets this?

The “hoodies,” in our vernacular, are the ones who are being more proactive and experimental with data and looking for opportunities in the data. Then they’re bringing data proactively to the business and saying, “Hey, look what we found.  Is this potentially interesting?  Is it potentially useful, actionable?  If so, let’s work together on developing that idea.”

And that’s where the real transformation to real innovation takes place.

I have a library that I’ve been compiling for a number of years on how organizations have been innovating with information, Innovators are much more oriented toward doing diagnostics or predictive or even prescriptive analytics, and that’s where the real value comes, the real transformation happens, the real innovation occurs.  It’s a hard pill for some business leaders to swallow.  It’s a scary place. 

What talent -- what skill -- do today’s business leaders need to bring on to their staffs?

From a skills perspective, you’re looking at bringing on folks who have data-science oriented skills. People who not only have deep statistical chops but also can put data and analytics in the context of business drivers.

That’s what we found when we studied what companies are looking for in data scientists.  They’re not just looking for people who have deep statistical skills.  If they did, then they’d just hire statisticians.  They’re looking for people who can curate and integrate and prepare the data and then put it into the context of a business model and think about how to read business models.

Granted, those people are hard to find. Our research shows that data scientist compensation is at about a 50- to 70-percent premium over BI analysts or statisticians.

But I think the up-and-coming business leaders over the next 10, 15, 20 years are going to be those who fully appreciate the power of advanced analytics and multi-structured data -- and are thinking about using data in ways to actually innovate, to create new products and services, not just to enhance performance. 

Let’s get back to the variety of data.  Are there things that IT organizations can do to get a better handle around the variety of data they have?

Well, this kind of goes back to one of my core research topics -- what I call “infonomics,” or the economics of information. 

The unfortunate thing that I find in most organizations is that they have a better accounting of their office furniture than their information assets.

Information assets are really the thing that’s driving competitive advantage, differentiation, performance, innovation.  So you have to wonder how is it that organization don’t have an accounting for -- or even an inventory of – their information assets?  I argue that it’s because they’re not on the balance sheet. The accounting aristocracy has not evolved to allow an organization to capitalize their information assets, and because of that, information is considered kind of a second class asset. 

We have this conflict between the insurance industry, the accounting profession, and the courts don’t often recognize information as an asset or as property.  But organizations that are truly benefitting in big ways from information are starting to manage it and leverage it and recognize it as an asset.

And an inventory of information assets needs to go beyond just what you’re capturing. You need to know what data’s available from your partners and suppliers and customers.  You need to know what data’s available from open data sources, from government organizations, local and national, from syndicated data providers, from social media. And you need to be able to think creatively about how to leverage those data sources. 

It takes some kind of business leader to see those ideas at a high level and to manage the kinds of people who are looking into that more experimentally and in a speculative nature.

A lot of companies have research efforts that are focused on their core business, whether they’re in manufacturing or oil and gas or even retail. But what they don’t have is an R&D function that is experimenting with information. Every day I come across clients that say, “Well, that’s not really our business.”  Well, yeah, it is.  Information is your business. 

So creating an R&D function for information is really, really important. And, yeah, there are going to be things you do with data that are more traditional.  Obviously, you need to continue to do financial reporting -- you know the hindsight-oriented financial reporting and all that regulatory reporting that’s required.

But let’s make sure that the data that we have, our information assets, are driving business value. 

It’s our duty to develop some ways to help chief data officers or data governance professionals to value data as if it were a balance sheet asset. 

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access