Data Science conference looks at four top reasons big data projects fail
Data professionals gathered at the Data Science New York conference this week to learn what trends will most impact investments in data analytics and data management in 2019, and how their organizations can benefit from best practices of leading firms.
Information spoke with Robert C. Whetsel, chief data scientist with the U.S. Department of Defense, about his session on “Why Big Data Efforts Are Failing to Deliver.” According to Whetsel, there are four primary reasons leading to this problem:
- Organizations lack of Big Data Foundations
- Silos of knowledge
- Lack of finding people with the right "BIG" skill sets
- Tool and technology mismatch
Information Management: What is the most important message or lesson that you hope attendees of the Data Science conference took away from your presentation?
Robert Whetsel: Data is not done in isolation, it takes a diverse team with diverse skills, and leadership allowing the workforce to innovate and create.
IM: What are the most exciting data management or analytics projects that you are involved with currently, and what are you learning from them?
Whetsel: Mmmmmmm…, my organization is well behind industry curve. I am way more excited in what is happening in industry. They are pushing boundaries that I can’t even see in my far horizon with the maturity of our programs.
IM: What do you find to be the biggest challenges with implementing and managing these projects, and how are you tackling those challenges?
Whetsel: The culture is the largest challenge that I face. One small step over an extremely long period of time.
IM: What do you believe will be the real top trends in data management, data analytics or data governance in 2019?
Whetsel: In my organization, hopefully we will learn we have gotten it wrong. Data is a concatenation of bit strings it is that simple. We continue to try to normalize it before we understand what we are looking at.
This breaks relationships that we have not discovered. As well as, having one data model does not gain the outcomes we desire. More importantly, nor does having a single standard in the way we use them or common anything delivers the elusive interoperability that all our program craves.
IM: Based on your experience and observations, do you believe most organizations are prepared for those trends?
Whetsel: No, most of our organizations are looking back not forward.
IM: Do you believe all the attention currently being paid to so-called emerging technologies such as artificial intelligence, machine learning and automation is warranted, or are those technologies being over-hyped?
Whetsel: These things go in cycles. AI was big, then gone, and big again. We tend to look at the shiny new thing, but AI is not new. The trend in AI is becoming more hype than real. The real AI has potential, but we have folks living in the hype and missing the real.
IM: Looking back over your career, and looking ahead to what you anticipate will be the top trends this year, are you enjoying your role as a data scientist (a) more; (b) less; (c) the same – and why so?
Whetsel: The same. My culture took 100 years to mature to what we have now. I will be long retired before we move the science forward in my organization.