Big data projects are increasingly finding their way into hybrid cloud environments, which suggests that advanced data projects may be significantly on the rise.
That is the take of Sumit Sarkar, chief data evangelist at Progress, who spoke to Information Management about what attendees at the recent Strata & Hadoop World conference in San Jose, CA, revealed about their present initiatives and challenges.
Information Management: What are the most common themes that you heard among conference attendees and how do those themes align with what you expected?
Sumit Sarkar: A common theme I heard repeatedly during Strata & Hadoop World was that big data platforms are either deployed on-premises or in the cloud, however, the data sources people want to access are often in a hybrid environment instead.
Our team predicted this would be the case in the future, but did not expect the notion to be true today as it indicates more advanced big data projects are in the works than we anticipated.
IM: What are the most common data challenges that attendees are facing?
SS: The biggest challenges I heard from attendees revolved around getting all the data they needed for a project at the right level of detail. Hosted cloud applications such as SaaS expose varying levels of details from their APIs, but many big data analytics projects are all about the low level details. Accessing a wide range of business systems across different platforms and RDBMS technologies presented a different set of challenges for the attendees I spoke with.
IM: What are the most surprising things that you heard from attendees?
SS: I was surprised at the interest from attendees in the management of data lakes in particular; and the supporting infrastructure for management across data preparation, ingestion, lineage, and quality, etc.
Big data projects are beginning to transition from the ETL model to ELT, and data lakes bring the data closer to the powerful big data programming frameworks and tools. This elevates the importance of the “Exact” and “Load” components of ETL with data movement.
IM: What do you view as the top data issues or challenges in 2016?
SS: Progress sees the importance of data connectivity across a wide range of data sources deployed in hybrid environments as a top priority in data management and analytics this year. These data sources range from mainframes and on-premises ERPs to CRM and marketing systems deployed as SaaS. Without quick and fluid data connectivity, organizations will struggle to access data from disparate sources and won’t be able to utilize that data for efficient business operations.
IM: How do these themes and challenges relate to our company’s market strategy this year?
SS: In 2016, Progress is focusing its efforts around enabling organizations to become more agile and deliver compelling customer experiences. Data connectivity is critical in this process, as digital businesses produce cloud and mobile application deployments that require access to core back office systems.
Data analytics are also needed to make sense of the many new types of data that are being collected from omni-channel customer touch points. Progress provides the industry’s broadest portfolio of connectivity (think “E” and “L” for big data integration as mentioned above) exposing an open standard API (ODBC/JDBC for data analytics and movement, or REST/OData for application development) that can be delivered on-premises or in the cloud.
Our company has spoken to several organizations that are building web applications on top of big data platforms and are now exploring data connectivity to accelerate their business operations. This is a very promising sign of successful big data adoption.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access