Big Data Pointless without Integration

Ventana Research
FEB 25, 2013 9:41am ET

Big data involves interplay between different data management approaches and business intelligence and operational systems, which makes it imperative that all sources of business data be integrated efficiently and that organizations be able to easily adapt to new data types and sources.

Get access to this article and thousands more...

All Information Management articles are archived after 7 days. REGISTER NOW for unlimited access to all recently archived articles, as well as thousands of searchable stories. Registered Members also gain access to:

  • Full access to including all searchable archived content
  • Exclusive E-Newsletters delivering the latest headlines to your inbox
  • Access to White Papers, Web Seminars, and Blog Discussions
  • Discounts to upcoming conferences & events
  • Uninterrupted access to all sponsored content, and MORE!

Already Registered?

Comments (1)
I'd go one step further and say big data without integration is not only pointless, but undoable. Big data often only becomes big data when aggregated with data in other systems.

I'd also like to add that entity resolution and identification (particularly when the entity is a person, organization or group) are significant barriers to integrating big or small data. Yet, these two primary causes of data dis-integration are only infrequently mentioned as barriers to IM. In my experience they tend to be the underlying cause of at least three of the five barriers indicated in the chart on Barriers to IM.

Inconsistent resolution and identification of entities is the big culprit prohibiting data integration, because the big data related to an entity hangs off the small amount of data required to properly and correctly identify the entity to begin with. And as the affiliations among entities (persons, organizations and groups) become entities themselves...and increasingly of interest, the resulting mess from trying to reconcile ill-defined, inconsistently identified entities is too daunting for many enterprises. This explains the demands of Google and Facebook for veronymity when the entity is a person.

This data mess can be likened to entropy. Although I am not advocating that data is subject to the physical laws of entropy, (it's not) you can't ignore these two insights from Stephen Hawking:

"...entropy...measures the degree of disorder of a system. It is a matter of common experience that disorder will tend to increase if things are left to themselves. One can create order out of disorder, but that requires expenditure of effort." And,

"...the entropy of an isolated system always increases, and that when two systems are joined together, the entropy of the combined system is greater than the sum of the entropies of the individual systems."

Just substitute "data" for "things" and we sort of get a law for big data integration.

Posted by Peter P | Tuesday, February 26 2013 at 3:04PM ET
Add Your Comments:
You must be registered to post a comment.
Not Registered?
You must be registered to post a comment. Click here to register.
Already registered? Log in here
Please note you must now log in with your email address and password.