When you think about data management concepts, often the majority “opinion” or current mindset, limits what we think we can and cannot accomplish. If something is against the current mindset, that task or challenge cannot be accomplished. If something is not against, but not quite aligned with, the current mindset, the challenge is worthy but should not be undertaken.
Arthur C. Clarke described this process by which new ideas come into being as:
- It can't be done.
- It probably can be done, but it is not worth doing.
- I knew it was a good idea all along!
Prior to the age of relational databases, there were (and still are …) hierarchical and navigational data stores that were the underpinning of the workloads for mainframe and other legacy computing platforms. These data stores met the needs of the time. But due to the mindset of the day, often technologists would say, “Why would you need to change? These approaches work fine.”
Starting in the 1980s, there was a change to this majority opinion. Relational theories and implementations were an excellent solution to particular operational data management problems associated with the best way to store information for those workloads. However, it was only after beating back that hierarchical and navigational mindset.
Then in the 1990s, data warehousing and analytical demands created the need for data schema changes. Denormalization and star schemas for analytical storage and processing grew in popularity. However, despite the relatively recent change to data management mindsets, it was required to fight against third normal form (3NF) majority opinions: “Star Schemas can’t be done!” and “Yes you can repeat all of that data, but why would you want to?” However, despite this, Clarke’s process applies and denormalization became more popular and people began to say, “I knew Star Schemas were a good idea after all!”
Now, I am hearing from technologists who haven’t completely bought into the NoSQL paradigm:
“Unstructured and multi-structured probably can be done, but it's not worth doing. It would mess with all my < >”
In this posting, I am not suggesting that NoSQL is now the majority “opinion.” However, I am opposed to using a single paradigm to rule out options. Personally, I would like to start making all options available and not have to go through all three steps.
What say the readers?
Have we reached an era of data management enlightenment where options #1 and #2 can be discarded as positional? Are we moving into an era where NoSQL will push relational datastores into the same “legacy” (read old school) category as the mainframe?
Post your comments below or contact me directly via Twitter at @JohnLMyers44 using the hashtag #noodlingNoSQL.