Unless you have been hiding in a cave for the past two years, you will certainly agree that the hype surrounding big data has been dramatically rising, probably faster than any technology hype to date. And if you believe Gartner, you won’t argue that big data is just about to reach the all-dangerous “peak of inflated expectations” before starting to slide into the infamous “trough of disillusionment” of the hype cycle. After all, this slide into the trough happens to all technologies as they mature, usually right after they “cross the chasm.” Some technologies make it across, some don’t. The ones that come out, do so stronger and deflated of their hype.
Not everyone has embarked into big data yet. So it hardly came as a surprise when a client told me “I don’t have a big data project.” It became interesting, however, when that client showed architecture diagrams, on which were liberally splattered terms like Hadoop, HDFS, YARN, Cassandra and Hive. The explanation is very simple: For this organization, there is nothing special about big data. It is “just data.” Yes, they are using cutting-edge platforms, that enables them to harvest and leverage more data, faster and more efficiently. But when they deployed Teradata 20 years ago, or Netezza 10 years ago, they did not call it anything special. So why succumb to the hype now?
And indeed, big data means so many different things that it’s now just an over-hyped marketing term (that I am as guilty of using as everyone else). But at the end of the day, whether it is social data, sensor data, transaction data, dark data, exhaust data, it is just data. The shape, complexity and volume of data keeps evolving. The speed at which it is needed keeps evolving. The use cases keep evolving. But this evolution is a continuum, even if its pace keeps accelerating.
The 451 Group uses the term “total data.” I like that term, because it encompasses everything: large and small data, fast and slow data, dynamic and static data. And indeed, big data cannot be viewed in isolation. A “big data project” is not only about harvesting, say, social data; it’s also about consolidating it with existing CRM and/or PLM data, financial records and more. It’s not only about again, say collecting weather information; it’s also about correlating it with historical sales and supply chain data to optimize inventory.
IT hypes do die. Some of them leave a solid technology foundation in their wake, which becomes the bedrock upon which the next generation of IT is built. Big data is one of these hypes. What it will leave behind is Just Data.