Okay, I'll admit it. Three years ago I recall sitting in this chair and feeling a little bored with the same old topics at the data grindstone. It was all important stuff and remains so, but felt a little like old news. 2008-2009 saw the serious emergence of data governance, a welcome business development (that was also a sign of more maturity in the space).
It was also a few years ago that the appliance and massive parallel processing markets really got rolling. We saw virtualization and the cloud model for hardware and software pop up. As a reporter, I was taken with a lot of this, but wanted to not get too far ahead of myself as I had a decade ago with the middleware and SOA phenomena. Who would have thought that it was the bottom end of infrastructure hardware and software that was going to be the game changer for IT and business going forward?
It's not happening everywhere, but boy, is it happening. In the last two weeks I've spoken with Dave Powers at Eli Lilly who's rapidly taking all of the pharma giant's entire R&D, collaboration (and research mindset) into the cloud. I spoke to Paul Sikora at the University of Pittsburgh Medical Center, who has moved the entire enterprise application infrastructure for dozens of hospitals from 167 UNIX servers to 14.
I'll be writing more on those stories soon, but for the moment, I rang Carl Olofson at IDC, who put out an interesting report on some of these trends last week. His prediction? Within five years most data warehouses will be stored in columnar fashion; most OLTP databases will be augmented or reside entirely in-memory; most large databases will scale horizontally through clustering; and finally, lots of data collection and reporting will fall to databases that have no formal schema.
Carl's DBMS message, right there in the report, was, "everything you know is wrong." For someone who'd been in the business for so long, I asked him how that could come to be so quickly.
"When I said 'everything you know is wrong,' it wasn't to say everything wasn't just fine when the mainstream databases were developed," Olofson told me. "Newer databases are responding to different economics in which systems have many processors and processes of many cores."
There's enough bought and paid for infrastructure and enough inventory on vendor shelves to guarantee the shift of cloud, virtualization and a bunch of other developments won't be consumed overnight.
But I've got a feeling that when some big infrastructure refresh cycles come up for you, lot of people are going to be surprised and pleased at what they see on the market.