Today’s DBA may feel somewhat like Dr. Who—traversing alternate application “universes” in which different rules and demands weigh on their decisions.

One universe—the universe of Big Data—is one in which people understandably tend to stress the “Big” part of data. NoSQL and Hadoop offerings, which often take center stage in conversations, are good at handling massive scale. As long as you’re okay with sacrificing immediate consistency, these solutions are excellent for use cases that don’t require making sense of data quickly and easily.

For Big Data applications that are all the rage now—evaluating behavioral trends, weather patterns, etc.—it doesn’t hurt if data consistency lags behind initial computations, and small discrepancies are forgivable.

Yet, there is a whole other “universe” in which even the slightest bit of inaccuracy spells doom for your application. These “high-value” workloads involve several steps, each of which must be performed immediately and accurately or else money, inventory or other assets may be lost.

This is the universe in which e-commerce, online gaming, ad tech and other applications reside—one where there simply is no room for error, lest you lose customers. But what happens if you need to handle millions of these types of transactions at once?

To ask it another way, what happens when the universes of high-value transactions and Big Data collide?

To survive in such a high value, heavy workload “multiverse”, companies need the best of both MySQL and NoSQL. They need the ACID compliance properties that ensure that database transactions are processed reliably, that comes with the former…but at the impressive scale characteristic of NoSQL.

However, achieving both massive scale while retaining complete data accuracy is not easy. MySQL’s (and by extension most MySQL drop-in replacements’) single-node (server) architecture hits a wall quick; once you max out capacity, you have to move to a bigger server, or employ unnatural feats to scale. And unfortunately, cloud deployments don’t help either; these databases don’t scale in the cloud the way other cloud applications do because they’re stuck on a single instance (server).

Resisting the Temptation to “Mess With Nature”

Gaming, ad tech and e-commerce are three examples of industries that have fully entered the multiverse of heavy, high-value workloads. For these industries, it is equally important to process massive numbers of transactions and retain complete data accuracy, but traditional database technology puts these aims at odds with each other.

If B-grade sci-fi has taught us anything, it is that messing with nature—whether it be tweaking with the space-time continuum, or resurrecting dinosaurs—is dangerous business.

Similarly, the kind of coding changes required to scale MySQL beyond its natural limits (such as those involved in sharding), or to ensure complete data integrity in NOSQL, wreak havoc on the very applications that the databases are supposed to power, making them fragile and much more complex to manage.

So companies find themselves in a bind. They need to scale, but NoSQL solutions aren’t an option because of data integrity issues. They need ACID compliance, but MySQL isn’t an option because of scalability issues.

NewSQL Aor A Big Data, Accurate Data Multiverse

Do you always have to choose between scalability and integrity, straddling the universes of Big Data and Accurate Data? Fortunately, there’s a way out of this conundrum.

In the past, emerging companies that hit the big time had no choice but to enter into onerous agreements with big-box vendors, the only products in the marketplace that could handle these workloads. However, software-defined database services and cluster-oriented solutions have made it possible to scale SQL-based databases “out” (i.e., by adding instances) rather than up (i.e., migrating to more powerful servers), in the data center or the cloud, all without being forced down the path of contractual and technological vendor lock-in.

When the universes of Big Data and high-value data collide, companies no longer have to compromise one over the other.

(About the author: Mike Azevedo is chief executive officer at Clustrix)

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access