Organizations taking more of their data and analytics to the cloud
The use of cloud computing for analytics continues to accelerate, and that trend is having dramatic impact on both user organizations and the vendor companies that work in this space.
“The growing number of cloud-based solutions, and the interest among organizations to move more and more of their data and analytics to the cloud, are combining to make it possible for organizations to take strategic advantage of the cloud rather than just a tactical approach,” according to Jon Bock, vice president of products and marketing at Snowflake.
But that focus on the cloud also increases the vulnerability of data, and vendor companies have to respond with more secure analytics products.
“We’ve made investments in security because we’re increasingly seeing companies exploring putting sensitive data into cloud platforms,” Bock explains. “We’ve worked on the features and certifications required to meet regulatory and compliance requirements such as SOC 2 and HIPAA.”
Bock attended the recent Strata & Hadoop World conference in San Jose, where he shared his views on the top themes of the show and the top concerns of attendees. Most noticeably, the show has grown to be about much more than Hadoop.
“Strata & Hadoop this year has seen enormous buzz about artificial intelligence (AI) from vendors,” Bock says. “That’s a noticeable shift from the past, where big data platforms around Hadoop and discussions on data science ruled the day.”
“The shift to AI is surprising to those of us who’ve been in the industry for some time,” Bock continues “AI was a widely hyped idea a few decades ago that never seemed to deliver on its promise. What’s emerging now is a more pragmatic view of AI, namely how can analytics be integrated and embedded into processes, decisions, and applications.”
Another theme: “People are shifting focus to outcomes and results, moving past what had previously been technology-driven projects and focus (e.g. how do I deploy Hadoop, how do I integrate noSQL with my overall data environment, etc.),” Bock explains. “People are increasingly being asked to show results, not just projects, and so they are being much more aggressive about evaluating technologies with an eye on time to value.
The Strata & Hadoop show has also evolved to focus on analytics in general.
“Analytics is being embedded everywhere,” Bock noted. “Companies are looking to incorporate live insights from analytics directly into software and services. That’s a marked shift from previous approaches; in which analytics was processed outside the application and then only the results were made available to the application. With that older approach, applications were working with analytics results that were quickly stale. The new approach allows the analytics to be updated constantly for always up-to-date results.”
Bock says he is also encouraged that organizations seem to be rediscovering the importance of data management and governance.
“In the explosion of excitement about data analytics and big data, most aspects of data governance—security, controls, metadata, cataloging, etc.--were bypassed or minimized as people focused on understanding and deploying new underlying data infrastructure,” Bock says. “However, as people scale their use of new infrastructure, they’re coming back to thinking about how to manage all of those aspects in order to avoid confusion and chaos.”
Impacts on vendors
What are the impacts of all of this on Snowflake’s market strategy?
“Snowflake has doubled down on democratizing access to powerful data platforms for analytics,” Bock explains. “That includes releasing several new capabilities last year focused on eliminating complexity and manual administration (including our automated concurrency scaling feature) as well as launching a new solution partner program to get customers connected with experts in data solutions.”
“Snowflake’s strategy continues focus on delivering the most compelling engine to power analytics,” Bock says. “Building a data warehouse specifically for the cloud has made it possible for Snowflake to deliver the elasticity, flexibility and performance needed for modern applications. Many of Snowflake’s customers, both established enterprises and new startups, are using Snowflake to power the analytics that they are embedding in those applications.”
But the focus on the cloud is not new for the firm.
“Snowflake has been focused on the cloud from day one,” Bock says. “We saw that cloud offers huge advantages for data analytics—provided that software is designed to take full advantage of it. The fact that our data warehouse is built for the cloud has allowed us to deliver unique performance, elasticity, and flexibility that people need to support their data analytics projects.”
The move to cloud computing was not the only common experience to emerge from the show. Another was complexity.
“People are struggling to manage the complexity of their data analytics infrastructure—as the number of platforms and tools has proliferated, the complexity of their environment has become almost unmanageable,” Bock says.
“People are also beginning to realize the need to bring a modern approach to data management and governance to their data infrastructure,” Bock continues. “Now that they’re working to move their big data efforts from exploration and POCs of new technology into production, they’re realizing that doing that without important security, controls, and metadata will not scale.”
Finally, asked what the most surprising thing he heard from attendees was, Bock noted “The failure rate of analytics projects is surprisingly high. Everyone at the show has a big data analytics project, but in conversations with them we’re hearing that a lot of people are still struggling to make those projects successful.”
“A number of people that we talked to are almost at the breaking point, considering abandoning their current technology investments to try something different in the hope of breaking through the challenges that have kept their projects from being successful,” Bock says.