Data Management Gets Big
May 24, 2011 – To process the mounting data from social networking sites, complex machines and retail tracking, organizations must introduce new management techniques and emerging technologies for big data processing, according to a report from Forrester.
The new report, “Big Opportunities in Big Data,” runs through strategies for preparing to deal with incoming data sets that are 100 to 1,000-times larger than what IT systems currently handle.
Forrester finds benefits in existing data management methods and technology, like online transaction processing for fixed database schema, and very large databases for the terabyte and low petabyte levels. But the Forrester report pegs much of the future on big data processing, a mix of virtual infrastructure and massive parallel processing. Big data processing has unique capabilities with voluminous data in areas such as poorly or lightly structured data, and simply structured data streams, like those from medical sensors of RFID.
Aside from more obvious issues with latency, consistency and isolation with unstructured data processing, maturity with processing technology varies greatly, Forrester notes. Infrastructure as a service and workload management tools in some cloud offerings may provide the most immediate capabilities with big data. However, choice of a data management and application processing technology that isn’t fully developed could leave organizations planning for the absence of integrated support, according to Forrester.
Lead author and Forrester analyst Brian Hopkins said developing an approach and technology plan for big data processing now will prepare businesses for changes and developments over the next few years.
“[It] requires your business to shift its thinking about business data from a cost to an asset. It also requires that IT avoid the temptation to view big data as a point solution in its own silo,” Hopkins wrote in the report.
Recommendations to get started from Hopkins and Forrester co-authors Alex Cullen and Mackenzie Cahill include the development of a business capability map to direct data, an evaluation of how and where data will be integrated, updating your technology roadmap, and maintaining a holistic approach with information architecture.