Slideshow 6 Problems Big Data Will Make Worse

  • November 15 2012, 11:51am EST
8 Images Total

Here are half-dozen problems big data will probably make worse in the near-term.

Matching Requirements with Integration

Digging into your requirements for business data may turn into easier avenues toward finding the right data. It may also expose a deeper problem with integration that befuddles data projects, big or small. Dr. Darshan Desai writes: “To manage growing volumes of big data, it is crucial to create a fast, efficient and simple data integration environment. Despite the technological advancements, these tools and technologies are still new and not easily usable in an enterprise environment. Often, these tools require large technical teams; the hardest part is balancing the effectiveness of the technology with the capital and operational cost constraints.”

Content Continues Below

Developing a Single EDW

If you were still in the process of combining and constructing one data warehouse for all your enterprise functions, big data may stomp out those plans for good. In its expectations of disruptive tech trends for 2013, Gartner Research writes that the maturity of “strategic big data” will move enterprises toward multiple systems – content management, data marts, specialized file systems, for example – tied together with data services and metadata to make a “logical” enterprise data warehouse.

Relying on Existing Infrastructure

Many early big data adopters are hitting a wall on one aspect of implementation in particular: belief that present information infrastructure is sufficient. According to a survey of 1,144 business and IT professionals involved in some stage of a big data program, most enterprises should not expect to simply layer on more analytics programs or solutions. “On the surface, a combination of adding storage and one or more larger servers can support the growth of an information management foundation. However, it is important to understand that anticipating and architecting the infrastructure is key to delivering the business value of the intended business case,” the survey authors wrote.

Mounting Social, Privacy Concerns

One of the lures over big data plans is the promise of finding profits in social media posts and public sources. Rumbling beneath that are quandaries among the legal community and at the government level about business big data plans with personal information including information stored in the cloud. And the IEEE and other industry organizations are taking their own cautionary view on the actual returns from “noisy” data sets like Facebook posts and tweets.

Content Continues Below

Backing Plans with Talent

You’re working out the “how” with big data, but what about the “who”? At the recent Strata-Hadoop World conference in New York City, analyst and blogger Steve Miller reviewed a keynote led by Berkeley mathematician Cathy O’Neill, who opined that academia is “not currently aligned” with big data skills on working with messy sets and business questions. O’Neill and others say higher education remains the best chance to address expectations of a limited pool of talent worldwide to handle big data.

Anticipating Static Results

Bringing on big data may also bring on a seismic shift in enterprise attitudes on IT projects and returns. Chris Ford, founder and managing director of the Chicago Business Intelligence Groups, writes: “Harnessing the power of big data means you have to add new technologies to your infrastructure. It also means you may have to start applying a new mindset to your organization. It also means trying things that may, in the long run, not be feasible. Think of it as a construction or remodeling process, where unexpected events are inevitable. Virtually no one gets it perfect the first time, but that shouldn’t deter you from starting now. In fact, it’s best to work out problems while you have time on your side.”

For more on big data ...

Click here for a landing page dedicated to strategies, trends and news in the big data space. All photos used with permission from ThinkStock.