The shortage of skilled data professionals continues to get plenty of press. And for good reason, according to Girish Pancha, chief executive officer at Streamsets Inc.

Pancha cited the continued skills gap and the struggle for trustworthy applications to manage data as among the biggest challenges to many firms this year. His comments came at the recent Strata & Hadoop World conference in New York.


Information Management: What are the most common themes that you heard among conference participants?

Girish Pancha: “There was a great deal of technology confusion amongst some of the people I spoke with, including some exasperation with keeping track of all of the open source projects, many of which seem to overlap and are difficult to distinguish from one another.”


IM: What are the most common challenges that attendees are facing with regard to data management and analytics?

Pancha: “Of the three things I heard about mostly two were perennial and one was new. The two old saws were the challenge of finding skilled data professionals and struggling with operationalizing applications for timely, trusted and reliable data use. The new one was a struggle to move mainframe data into Hadoop, which may indicate increasing adoption in more conservative industries such as banking and telecom.”


IM: What are the most surprising things that you are hearing from attendees regarding their data management initiatives?

Pancha: “While all the press and hype has been around streaming analytics, the majority of the near-term pain is still in taking full advantage of plain old structured and semi-structured data being moved in batch mode. I suppose there is something to be said for learning to walk before trying to run.”


IM: How do these themes and challenges relate to your company’s market strategy this year?

Pancha: “Our mission is to help enterprises harness their data in motion, which speaks directly to the struggle of getting data where it needs to be, all the time and as efficiently as possible. This is not a Hadoop problem per se, it is really about making it easy for enterprises to build and run many-to-many dataflow topologies as a continuous disciplined operation.”


IM: What does your company view as the top data management and data analytics issues in 2016?

Pancha: “As it relates to data in motion, where we specialize, there are three big issues we see as most critical for enterprises. First, solving the ingestion operations problem for Hadoop continues to be important, as this determines whether you realize the potential of the storage/compute platform. Second, architecture for stream processing. Use of streaming analytic platforms will increase pressure on the upstream data ingestion function since there is little to no time for manual data preparation as there would be with batch data. Lastly, creating a data strategy that simultaneously embraces both new technologies like Hadoop and Spark and incumbent technologies like mainframes and RDBMS.“

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access