San Diego -- Call it Part II of the moot court convened to decide the reality or hype of big data at the TDWI World Conference this week.
Consultants/pundits Marc Demarest and Mark Madsen squared off as the disciple and the bomb tosser to big data’s business reality check, a mock trial of legitimacy full of hyperbole meant to entertain more than to define either man’s belief system (which they did eventually get around to).
Demarest: "Data is no longer a convenience, it is no longer an asset we need to keep clean and tidy. It is in fact a factor of production. It is essential to the way we deliver our goods and services to our markets of choice."
Madsen: "The benefits of this stuff are kind of suspect. There are very narrow use cases … the poster children of this revolution tend to be social media companies, Web companies, getting you to buy stuff by clicking on things..."
And so on.
It was Part II because the pair put on a similar show at TDWI in Las Vegas 10 months ago. The repeat in San Diego might have been for the benefit of a new audience and folks who had never heard these two go at it. In any case it was inside humor befitting a TDWI 8 a.m. breakfast audience.
Madsen: "Perhaps we’ve been a little too authoritarian with our databases but with the alternative, the CSV [comma separated value file] as the standard of data with no metadata, no anything … we replace the center of the city with a shanty town that buns down every five or 10 years … I think I’ll go with the longer lasting building."
Demarest: "Since the beginning of the industry we’ve had this magical thought … if we dump data in a pretty form on the desktops of decision-makers, good things will happen .. [they’ll]take the data, make an Excel spreadsheet, put it on the file server with 10,000 other spreadsheets … make a pretty power point and take it to a meeting … [where] we’ll lose sight of the data and make decisions on a political basis…"
And so on.
But it said something that two regarded consultants (Mark runs ThirdNature, Marc is at Noumenal), Ralph Kimball anointed dimensional experts no less, pretty much described an identical horizon to what they saw 10 months ago. That’s an eternity for folks who have migrated to Internet time. And with examples that were general and drenched in metaphor you could say they weren’t bringing much ammunition to the table.
Maybe this wasn’t the place for that. But if this audience was new to the boss’s discussion at a TDWI conference as organizers said, we’ve got some tire kicking left to go. TDWI attracts a distinctly talented and gainfully employed audience. I can only imagine a lot more evidence and digestion awaits for the broader business side to try to figure what all this big data stuff means. For them, as research and poster children arrive, it’s still going to require that we see a lot more big data in action.
When Madsen and Demarest finally did step out of character, they were philosophical in a constructive kind of way.
Madsen said his honest view of reality was somewhere between the extremes and the key thing to maintain is market focus. Differentiation does not come from commodity BI technology and we’re at a point where every organization of a certain size already has that as a cost of doing business.
“Best of breed doesn’t mean throw out all the old stuff ... Maybe that old data warehouse is good but maybe we’ve got to write it on a new parallel database.” And, Madsen adds, if we like the outcomes we find in analytics, we can scale that where appropriate.
Demarest said it would be a disservice to call big data a “technological revolution.” The pattern will go on, he said, where franchise vendors will build or absorb succeeding generations of new technologies as they mature. “We will be doing new things … and initially we may be buying that technology from new companies but over time our supplier base will stabilize much as it has in the last five years.”
The revolution, he figures, is that we’ll return to business value by way of analytics, understanding how our infrastructure is wired and that will help “move to real time market signal based analysis that affects your company and what you need to do about it.”
It’s hard to predict how much of that arrives and how soon, but we’ll know it when we see it.