Competitive edge with data still elusive for many organizations
The focus around data this year is all about artificial intelligence, but most organizations find themselves struggling with how to successfully gain competitive advantage with the technology.
That is the take of Vineet Tyagi, chief technology officer at Impetus Technologies, who reflected on the three biggest challenges for attendees of the recent Strata conference in San Jose, Calif.
According to Tyagi, data professionals say their three biggest concerns this year are:
- “Finding the right balance of what to keep on the traditional data repositories and when to sunset them. The key challenge they are facing is implementing the logical data lake strategy that includes all enterprise data platforms and enables the right positioning of workloads based on the best price and performance.”
- “Being able to discover, analyze and consume data across the enterprise seamlessly with as little latency as possible.”
- “How to begin the first data science and artificial intelligence project.”
This last point definitely puts an organization at competitive disadvantage, since the leading trends now are the implementation of artificial intelligence and machine learning.
Another top trend is the “enterprise-wide transition to self-service business intelligence, data science and data preparation leading to a major transition in how IT teams work with business,” Tyagi says.
The data scientist still rules
The role of data scientist was predicted to be the top technology job again this year, and Tyagi finds no argument on that point.
“Data science and machine learning were the biggest and most common themes we heard at the show,” Tyagi says. “Additionally, the transition needed for the enterprise to become real time self-service (data prep, data science, BI etc.) was the other theme we noticed. These themes are perfectly aligned to what we were telling our customers to prepare for two years ago.”
One of the biggest surprises to come out of the conference is perhaps the degree to which organizations have adopted Hadoop, and in many cases, are moving onto their next phase. This was not doubt a big reason for the change in the name of the conference this year, dropping Hadoop World from the Strata title.
“Big data Hadoop and Spark are now a given for all enterprises,” Tyagi says. “Some attendees are questioning whether they need Hadoop on-premise, or if should they be a cloud-only IT shop.”
Indeed, cloud-only architectures are increasingly becoming mainstream, Tyagi says.
The bottom line for many organizations, however, is that achieving true value from data, and using it to drive business decisions, is no easy task.
“Despite managing and acquiring data assets unlike before, businesses still struggle in gaining a competitive edge with their data,” Tyagi says. IT teams and decision makers are realizing they are experiencing bottlenecks and badly need to transition to self-service to better enable business success.”