Searching for Smart Data: All-In-One, Automated Big Data Applications
What would life be like without a smartphone? There are still some of us roaming about that can actually remember a time, back before the earth’s crust cooled, when mobile phones served one single function as a calling device.
In the article, “Smartphones Poised to Take Over Mobile Phones in 2013: Ten Reasons Why,” writer Don Reisinger cited a new study from research firm International Data Corporations that predicted, “For the first time, smartphone shipments in 2013 will outpace mobile device shipments.” Reisinger listed 10 reasons for this shift, including, among others, emerging markets around the world (particularly in South America and Asia), Apple’s iPhone touch screen technology introduced in 2007, Google’s open source Android operating system, lower costs, better quality and the rapidly growing middle class in China.
Smartphones are arguably the most useful electronic tool for managing daily life. This all-in-one, handheld device does all the heavy lifting by eliminating the need for any other standalone device, namely telephones, computers, cameras, video recorders, TVs and much more. The question for those of us in big data is this:
If people can surf the Web, send and receive email, track appointments, take pictures, shoot video, get directions, weather, and stock prices, and do their banking on a smartphone, why can’t the business world enjoy the same single-source technology to access and manage big data applications?
Call it Smart Data
Companies need a single, intuitive technology that presents an effective, efficient view of all information gleaned from unstructured, semi-structured and structured data to make informed daily decisions regarding critical issues. Up to this point, most of the talk swirling around big data has focused on the technology itself — Hadoop, columnar databases, in-memory databases, etc. At the most recent Strata conference held this past February in Santa Clara, CA, dozens of vendors showcased database hardware and software, going into nuts-and-bolts microscopic detail about how their technology works. But busy executives and company directors don’t care about technology; they care about solving their problems. On more than one occasion when various software engineers discussed underlying technology, the conference attendees were left confused and ultimately disengaged. CEOs, CIOs, CFOs and CMOs want applications with preconfigured and predefined tools that solve business problems — a “smart data” dashboard, if you will, that represents a single, effective view of their business or function, such as marketing campaign optimization, smart meter analytics for smoothing power generation curves, product pricing elasticity models, etc.
Companies want a tool that contains everything they need in one package, like a smartphone. As it was made clear at the Strata Conference, today it only exists in pieces and parts. If you ask businesspeople who work in different disciplines (marketing, healthcare, retail, etc.), “What do you need?” you’ll get a consistent answer based on their department function and industry type. Marketing directors want to drill down into social media sentiment as it relates to their structured data. Health care professionals need to tie together electronic medical records and clinical trial data. Retailers need to manage supply with demand, tracking and comparing structured sales data with regard to region, climate, trends and demographic data. But few technology firms are currently working toward focusing on the types of applications that solve these business needs. Maybe it’s too soon? It shouldn’t be. The technology exists, but the vision to build business applications is missing.
There are at least two parallels here. It took at about a decade for data warehousing to gain traction in the industry because businesses and DW firms didn’t take advantage of the existing technology. Once software firms like MicroStrategy, BusinessObjects (now part of SAP) and others allowed business users to generate ad hoc reports that provided insights about sales, operations and more, data warehouse technology was off and running. Second, while software giant SAP may not be the most elegant technology architecture, SAP beat out the competition in the ERP arena because their ERP applications actually solved business problems. They are now one of the largest software firms in the world. JD Edwards accomplished similar feats for the financial services sector.
Who’s going to win the race in solving this electronic dashboard puzzle in the big data space, similar to what SAP achieved with ERP space? Some clues lie within ERP itself, since the purpose of ERP is to facilitate the flow of information between all business functions (financial, human resources, supply chain management and marketing) inside the boundaries of an organization and to manage the connections with outside stakeholders. Whether it’s people, products or services, ERP is the central hub of all data within an organization. ERP dashboard modules should be fully integrated and provide real-time applications. This means data entered into one ERP module is immediately and automatically updated and reflected in all of ERP’s functional areas and then can be integrated, structured and available for planning, management and operations of the overall business, not just one area of the business.
Another piece of the puzzle involves sales and operations planning, which serves as the umbrella over the ERP management process. S&OP distills the information from ERP and, based on that intelligence, guides decision-making for questions such as: Do I buy more materials? Do I reduce my inventory? Do I expand or contract my workforce? The data is used to minimize, or mitigate, the risk of excessive purchases or lost opportunities due to lack of resources and information.
Successfully integrating big data analytics into an all-in-one smart data dashboard is the final piece to the puzzle. Like the innovators of the smartphone, the firm that is first to successfully market a preconfigured “everything a business executive needs” in one big data application that pulls from all three data types (structured, semi-structured and unstructured) will have a huge competitive advantage, transforming how enterprises do business forever.