The Year When Cloud Overwhelms the Database
The “Year of the Cloud” has come and gone. Now what? Our adoption has increased and we’ve realized some concrete bottom-line cost savings, but we would be fooling ourselves if we thought the hard work was behind us.
The database has been neglected in terms of tooling, mindshare and adoption in the cloud. As a result, our database administrators (DBAs) will be forced to support databases in the public cloud without a safety net. Today, we can easily deploy and update our applications in the cloud. Tomorrow, we are unprepared to do the same for our databases. In fact, due to the increases in efficiency we see on the development, test and system administration side, I’m petrified that DBAs will be held to a standard that is unfair and unattainable without some foundational work.
Standing up new, empty database instances in the cloud is as simple as a new application server. Sadly, DBAs simply don’t have the tools to properly manage and deploy the inescapable influx of database changes in these new cloud database instances. Even worse, DBAs are unable to quickly verify the current schema version on these cloud database instances. Because of that lack of knowledge, a real risk exists that they may update or terminate the wrong database instance, causing the loss of customer and company data.
There is a HUGE difference between pushing out a change to an application versus a change to a database. If someone releases a bad application, no problem; just roll it back. Copy over the new, bad version with the old, good version. Problem solved! But that is impossible to do with the database. A DBA can’t simply copy over the existing database with a previous schema. Now multiply that challenge by the hundreds, thousands or hundreds of thousands of database instances that will soon exist in the cloud across the enterprise. The complexity and danger of the task is enough for a DBA to dig his or her heels in and refuse to move to the cloud.
I predict that these difficulties and breakdowns will have an exponential impact in 2016, because whether DBAs like it or not, databases will be moving to the public cloud. IDC predicts today’s cloud database market of $746M (2014) will grow to more than $5.7B in 2019. A significant portion of that multi-year growth will take place in 2016. But fear not, there are pieces of advice and ways to help get ahead of these challenges and prepare for the inevitable.
1) The end game in the cloud will be a marathon, not a sprint. Cloud allows developers to go faster and be more efficient in order to have greater impact on the business. When it comes to the database, many DBAs might act under the false assumption that setting up new cloud database instances and migrating production data is all they really need to do to keep pace. Not so. Those are one-time occurrences that need to be handled deftly, to be sure. But the real challenge that DBAs will face in the cloud will be making sure changes can be consistently and repeatedly deployed and reflected across environments. The speed increase will not be static. As we continue to adopt the cloud for all applications, the corresponding changes to databases will continue to increase in frequency and number. Hold on to your hat!
2) DBAs will do well to clean house prior to migrating to the public cloud. For a long time, DBAs have warned us of potential issues with migrating to the cloud. They know any friction or slowness they have in updating the database on-premise will be multiplied by the large volume of cloud database instances. With that foresight, and given the reality that the move to cloud eventually becomes inevitable, DBAs will want to bring flexibility and agility to their application pushes for the database NOW. If they don’t, proliferation of cloud database instances will ultimately make things more confusing, cluttered and slow. With a sound, organized foundation laid, the move to cloud becomes academic. DBAs will simply change the target from on-premise servers to the cloud and not miss a beat.
3) DevOps’ influence will continue to spread. And that includes to the database layer. DevOps is one of the most compelling methodologies to help IT organizations stay in synch and on pace. There’s no reason it shouldn’t extend to all steps of the application release process, including the very last one.
4) DBAs will need to look out for themselves. In a select few companies, the development team or line of business are concerned enough for DBAs to make sure they have the resources they need to keep up with application release cycles. Then there are the rest of the companies. Often, DBAs are left to fend for themselves and ordered to “just get the job done.” Leading up to the shift to cloud, DBAs need help. They must make that known in order to secure the budgetary dollars to invest in the right software tools and keep their work/life balance intact. DBAs shouldn’t have to work from behind nor should they be forced to skip vacations, soccer practices or family dinners.
I don’t mean to present a doomsday scenario with these predictions; cloud is an amazing technology that brings with it the flexibility and scalability we have demanded for decades. As with any new technology, a large amount of preparation and planning will be needed, but I believe DBAs are up to the challenge. After all, we’d be fooling ourselves again if we thought we could make the move to the cloud without them.
(About the author: Robert Reeves is CTO and co-founder of Datical)