Research from market analyst firm IDC predicts a new generation of DBMS technology will drive significant change in the data warehousing and database market over the next five years.

The IDC study, "The Third Generation of Database Technology: Vendors and Products That Are Shaking Up the Market," finds that the current pace of development and adoption signals change in four areas.

First, most data warehouses will come to be stored in columnar fashion. Second, most OLTP databases will be augmented or reside entirely in-memory. Third, most large database servers will achieve horizontal scalability through clustering. And finally, many data collection and reporting issues will be met by databases that have no formal schema.

The predicted shift is largely driven by infrastructure upgrades in the form of cheaper processing and memory that lowers cost and brings older and fringe-computing ideas back to viability. "Ten years ago you had to lay the data out on disk in a fairly straightforward way where your main mission was to optimize disk access and reduce I/O as much as possible," says IDC Research VP Carl Olofson. "What's happened in buffering, in indexing, in-memory and 64-bit addressability changes the equation."

Olofson says new and familiar vendors are have stepped up with solutions to mainstream problems that will do well in the next technology refresh. "As people continue to invest in data management they might ask if they are using the best tool for the job."