June 30, 2011 – With financial backing and a big first client in Yahoo!, startup Hortonworks plans to build out an open source Hadoop software framework to handle and install software for massive amounts of data.

Venture capital firm Benchmark Capital joined Yahoo! in founding Hortonworks, which is comprised of many former Apache Hadoop project engineers and contributors, including newly minted CEO Eric Baldeschwieler, according to a news release. Yahoo! provided early prototype funding of Hadoop in 2005, and the massive Internet corporation uses Hadoop across its advertising and email platforms.

Hortonworks’ mission is the acceleration of development and adoption of Apache Hadoop for greater ease of consumption by enterprises and tech vendors looking for storage and processing of big data through open source. Baldeschwieler, who formerly held the title of VP of software engineering for the Hadoop team at Yahoo!, says the growth of contributions and use of Hadoop in the last five years shows its promise for adoption going forward.

“We anticipate that within five years, more than half the world’s data will be stored in Apache Hadoop. We’ve assembled a top caliber team committed to the Apache open source community and with the technology and business expertise to deliver value to the big data market,” said Baldeschwieler in a news release announcing the start of Hortonworks.

David Menninger, vice president and research director at Ventana Research, says the expanding market around Hadoop should support many vendors, though he sees Hortonworks with a leg up from its large testing lab and relationship with Yahoo!. He expects Hortonworks engineers to focus on known issues with Hadoop distribution, such as manageability and high availability.

“Hortonworks’ place is to try to bring the most important features into the core so the community doesn’t have to turn to multiple vendors to get a robust, complete Hadoop stack,” Menninger says.

Forrester Research senior analyst James Kobielus, who has written extensively on recent trends with Hadoop, sees Hortonworks’ “brain trust” of Hadoop engineers associated with Yahoo! continuing to contribute open source development as a plus for that community’s capabilities with big data, especially with growing interest in data storage in the cloud. Kobielus says Hortonworks has a unique place in the immature Hadoop market to spread best practices, compared with providers like Cloudera and MapR that have some training in Hadoop, but more along the lines of a traditional software and services business.

“Somebody needs to play those roles. There’s no clear standards group in the Hadoop space … whereas Hortonworks is part open source software business but also, in many ways, an evangelist for all things Hadoop. That sets them apart,” Kobielus says.

After months of speculation around a new Yahoo!-backed Hadoop initiative, Hortonworks officially opens for business in July with a staff of 25, according to an announcement from Yahoo!.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access