Gartner's 19 In-memory Databases for Big Data Analytics
Amid the big data boom, the in-memory database market will enjoy a 43 percent compound annual growth rate (CAGR) – leaping from $2.21 billion in 2013 to $13.23 billion in 2018, predicts Markets and Markets, a global research firm. What’s driving that demand? Simply put, in-memory databases allow real-time analytics and situation awareness on "live" transaction data – rather than after-the-fact analysis on "stale data,” notes a recent Gartner market guide. Here are 19 in-memory database options mentioned in that Gartner market guide.
This database “offers a flash-optimized in-memory key-value store. This is a hybrid DRAM/addressable flash system,” according to Gartner.
This “relational in-memory DBMS is fully ACID and SQL-99 compliant.?It has the ability to run completely in memory, on disk or in a hybrid mix of disk and memory. Built-in replication feature for five-nines high availability (HA) and scalability,” according to Gartner.
This is a “document-style NoSQL DBMS. It uses a memory-first architecture, where data goes first into memory, but allows for overflow to disk. Version 3.0 allows for tuning of memory versus disk for performance. HA is enabled in memory using node-to-node and cross-data-center replication,” Gartner stated.
This is a “table-style NoSQL DBMS, which which has in-memory database structure capabilities,” Gartner said.
This is a “column store in-memory MPP DBMS for data warehousing and analytical solutions. It runs in hybrid DRAM and disk mode. Added capabilities like EXACloud (cloud offering) and EXACluster (providing disaster recovery and?simple cluster administration),” Gartner noted.
“DB2 with BLU Acceleration adds to the existing row store, an in-memory column store initially designed to support analytical use cases. DB2 with BLU Acceleration provides improved data compression, query performance and reduced administration. Meanwhile, dashDB is a fully managed data warehouse cloud solution. It is based on IBM BLU Acceleration in-memory, columnar processing and leverages Netezza in-database analytics, R and other tools for analytics,” Gartner said.
This “in-memory row-store MPP database specializes in analytical and data warehousing use cases. Kognitio offers its technology as software only, a database appliance, or database SaaS (Kognitio Cloud). Kognitio supports a range of connectors such as Hadoop and Amazon Web Services' Simple Storage Service (Amazon S3), which enable its usage in data discovery use cases,” Gartner said.
This is a “row-based relational IMDBMS with full ACID compliance; it’s used for OLTP applications with roots as a small footprint-embedded DBMS. It has added optional enterprise capabilities, such as persistent/hybrid storage, clustering, HA and columnar data layout,” Gartner said.
This relational DBMS is “for both transactional and analytical models in a single DBMS offering having complete SQL capabilities and ACID compliance. It offers the unique ability to compile SQL to machine code, push execution across distributed nodes and allow for the machine code to be reused for subsequent queries,” Gartner said.
“An in-memory column-store for analytical processing debuted with SQL Server 2012. An in-memory row store for transactional support surfaced with SQL Server 2014. Combining both supports HTAP more efficiently,” Gartner noted.
“Oracle TimesTen for transactional capabilities can also be used as an in-memory cache for the Oracle RDBMS. Oracle TimesTen is also an option for the adaptive in-memory cache for analytics on Exalytics. Oracle Database 12c in-memory option, which became available for sale to customers in mid-2014, adds in-memory columnar capabilities for analytical processing leaving the rest of the RDBMS unchanged,” Gartner said.
This is a “hybrid in-memory and disk column-store product with a bitmap-like index structure called the High Performance Compressed Index, which uses the latest compression techniques to allow analytics to be performed in the compressed format. ParStream enables fast data loading capabilities through its parallel microbatch loading technology, which distributes the loading process and the data across the cluster. ParStream can be used to analyze large volumes of information flowing at low latency (such as network monitoring data, clickstreams, price quotes, sensor data, or meter readings),” Gartner said.
Pivotal GemFire XD “combines the capabilities of an IMDG and an in-memory RDBMS. It is a memory-centric SQL DBMS serviced by multiple nodes in a "shared nothing" architecture with query processing, indexing and user-defined functions optimized for a distributed grid. This is the first hybrid-style in-memory data management platform, combining an IMDBMS and elements of an IMDG,” Gartner stated.
ActivePivot is an “in-memory, columnar analytical engine that provides real-time aggregation of data with proprietary compression algorithms that supports the reduction of memory requirements and horizontal scalability. It allows performing continuous queries and distributing the results to subscribers using streaming APIs, offers support for simulation and "what if" analysis. It can be fed from multiple sources including JDBC, Java Message Service, comma-separated values, XML and APIs for developing new connectors,” Gartner stated.
This is an “in-memory NoSQL (key-value store) cloud DBMS using database platform as a service. It combines the in-memory NoSQL DBMS and an IMDG allowing for a scalable programming environment with HA,” Gartner said.
“SAP Hana is an IMDBMS column store delivered as an appliance on various hardware platforms. While it initially supported only analytical use cases, it now also supports transactional applications. SAP Hana has various deployment models: on-premises as an appliance, in the cloud (private and public), and as a hybrid of both on-premises and cloud. SAP now offers SAP Business Suite support on Hana, allowing it to also support transactional processing along with analytical processing on the same data store and thereby delivering on its promise to support a HTAP,” Gartner said.
“Teradata Intelligent Memory (TIM) is an in-memory store designed to enhance the performance of analytics beyond what can be achieved through standard caching. It was added as part of Teradata Database 14.10. TIM is automated; the software decides which data is most used ("hot") and places this data into memory. Data is processed using a memory-friendly column store,” Gartner said.
This SQL DBMS is “ACID compliant, supporting ODBC and JDBC connectivity options. It is also used as an embedded DBMS in addition to an enterprise DBMS. Its "universal cache" feature accelerates all major DBMSs on disk with solidDB acting as a relational, SQL-based in-memory data cache,” Gartner said.
“VoltDB is an open-source DBMS with a community and an enterprise edition with HA/DR capabilities. It offers an SQL ACID-compliant database that combines in-memory processing and a "shared nothing" scale-out architecture to process various transactions on commodity servers. It supports the relational and the JSON model. It supports analytical capabilities including real-time queries to provide context and state- to-streaming data, integration with multiple data stores including Hadoop, Netezza, HP Vertica and rapid data export to distributed message queues such as Apache Kafka and RabbitMQ,” Gartner said.
Although 19 vendors were listed in Garner’s in-memory database market guide, that does not imply an exhaustive list, the market researcher noted. Poke around the industry and you’ll surely find more in-memory options. For more Information Management slideshows, visit here.
In-memory databases are designed for big data applications and real-time analytics. Here are 19 in-memory databases that Gartner mentioned in a recent market overview report.
Big Data Skills and Pay Trends That Have Top Impact
The Seven Truths for Governing Big Data for Business Success
The Dos and Don’ts of Predictive Analytics
20 Top Master Data Management Solutions for 2016 (Part 2)
20 Top Master Data Management Solutions for 2016 (Part 1)
14 Business Impacts of a Cyberattack
Gartner’s Top 10 Information Security Technologies
5 Tips to Keep Cybersecurity Strong this Summer