As a business-side guy who covers technology I’m not normally attracted to millisecond performance, throughput or the bragging rights associated with all that stuff. But on this day I find myself surrounded with news and views on data appliances, which speak to just such factors, and yet hint at the business outcomes I spend my time worrying about.


It started with this year’s rounds of user conferences and chatting up several new data appliance vendors I’d frankly never met or heard of before. Next, I received a report and took a briefing with Jim Kobielus, Forrester Research’s new point man on data appliances. This was followed by a big appliance-type release from Teradata this week. Finally, I did a good bit of background work before interviewing Stuart Frost, the founder and CEO of DATAllegro for an upcoming issue of DM Review.


My latest crash course made me aware of just how hot and contentious the data appliance market is today. It has gone basically from zero to monstrous potential -- upwards of $30 billion in hardware and software by some estimates -- in a half-dozen years. Two years ago IDC slapped a 70 percent five-year compound average growth rate for data appliances.


So back to my mission; what are we talking about and why do we care? As most people agree, a data appliance (and in our world, a data warehouse appliance) is basically a combination of optimized hardware and high performance software in a box. It’s a pre-tuned, relatively low cost turbocharger of throughput and analytics that breezes through data tasks previously ground out and cursed over whole days by those who manage big arrays of servers and disk drives. Phil Russom, senior research manager at TDWI and a regular DM Review contributor, credits the term data appliance to Foster Hinshaw of Netezza, the six-year-old grandfather of the business. But Russom also points out that the idea of purpose-built high performance data warehousing software and hardware goes back 30 years with Teradata, though the company didn’t like the idea of calling the package an appliance – at least until recently. Others like DATAllegro’s Frost are happy to call their appliances updated and improved versions of Teradata’s vision.


That’s part of the problem. Until fairly recently, the largest enterprise data warehouse (EDW) database vendors hemmed and hawed as data appliances slowly clawed into the dreams of both database administrators and business analysts. Now the big guys are all in, though the responses of IBM, Oracle and Microsoft sometimes look more like stacks of existing products than the sexy little black boxes produced by Netezza, DATAllegro, Greenplum, Dataupia and a few others.


What the big companies offer is the breadth of all that the EDW entails and requires: ETL, BI, data quality, not to mention the data warehouse itself. What the appliance vendors offer is a lower-cost means to an end.


“Netezza popularized the concept of a data warehouse appliance as first and foremost a tactical deployment for very high speed, high volume complex analytical processing for BI reporting, data marts, OLAP and so forth,” Kobielus told me. “It was really a data accelerator, not a complete enterprise data warehouse, but rather something you could put in front of your EDW to offload some of the processing,” and other heavy lifting assigned to data warehouses.


That has been the sweet spot for the appliance vendor, who now wants space on the ground floor of business intelligence. But you can’t just buy an appliance and have an EDW. So the big questions I wound up with are, now that data appliances are scaling and proving their cost-effectiveness, are they going to 1) be an integral piece of data warehouse infrastructure and 2) supplant exiting data warehouse infrastructure going forward.


Over time, the answers appear to be yes and yes, based on the inherent missions of the data appliance. In the near term they need to prove they can scale to petabyte levels of data and improve performance across different workloads. If they are not quite there yet, Kobielus figures they could be in as little as a year from now. That ought to set the stage for partnerships, if not outright acquisitions of the best data appliance vendors.


The end game would be something like EDW in a box, which is something like what Teradata began offering this week with its 2500 model, a high performance appliance with a database that can also support outside applications such as CRM. Kobielus also calls IBM’s Balanced Warehouse and blade appliances EDW in a box, and is fond of its flexibility, though you could argue about the packaging. Oracle and Microsoft are also offering purpose-built optimized packages with partners such as HP or Dell. HP is also working in the arena with NeoView and NonStop, and Sun has partnered as well.


But it’s fair to say that the big vendors are chasing the market discovered by Netezza and that appliance vendors such as DATAllegro have equal if not more mature relationships with partners EMC and Dell. “Teradata especially and IBM are feeling the heat, but you’re seeing them come back with answers,” Kobielus says.


That should make the next year pretty interesting for the enterprise data warehouse market. If nothing else, the appliance vendors have dragged the big vendors back to the drawing board, the result being that we’ll see better performance, quicker deployments and lower price points as the market coagulates. That should be good news for data-strapped companies everywhere.


For information on Kobielus’ new data appliance report from Forrester Research, click here. For Teradata’s press release, click here. Watch for my interview with Stuart Frost of DATAllegro in the June issue of DM Review.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access