Related Links

Web Seminars

Big data has finally arrived.

With cheap, seemingly infinite storage and powerful computing technology, the infusion of the Internet and clouds of CPUs, businesses are capturing enormous amount of data.

From clickstreams, point of sale records, call center events and ATM transactions to Facebook, blogs, emails, RFID and mobile device data, the influx of information is now impossible for the human brain to analyze and comprehend.

Being analytics savvy is no longer an option for most businesses but a matter of survival. If a company can gain insight into customer, product, channel, pricing and supply chain and predict the best customers, best pricing and most profitable products by analyzing business data, it will leave competitors in the dust. Analytic competition is increasingly the norm, and gets more sophisticated every day as companies strive to "out-data-mine" their competitors.

Meanwhile, there has been raging debate about business analytics versus business intelligence in recent years. Many pundits and analysts have drawn their own line to differentiate BA from BI. Some have proclaimed the death of BI upon the rise of BA. The debate raises serious question about the current state of BI and the relevancy of BI architecture, as well as the technology, process and organization that surrounds it.

Historically, BI implementations have often stopped at data gathering and reporting and failed to dig further into analytics for actionable information and data-driven decision-making. But BI as a concept and theory never was, nor should be, just about collecting and reporting data. In its full promise, BI provides historical, current and predictive views of business operations that supports better business decision-making. Unfortunately, BI in practice too often fails to deliver on the promise of helping business acquire intelligence and make fact-based decisions. And, all too frequently, IT as the owner of BI systems and technologies fails to understand business from the beginning and does not provide valuable information or help guide business decisions. In short, BI often lacks both business and intelligence.

BA, on the other hand, has been overhyped as the silver bullet to fix most of the problems BI can't address today. As capable and promising as BA is, most information management companies need to overcome common challenges to compete on analytics: misaligned IT and business objectives, data quality failures, data silos, inconsistent data definitions, information latency, a lack of standards and performance problems.

It's important to remember that business analytics isn't a new concept, and has been practiced since the Efficiency Movement in the late 19th century. It was then that Frederick Winslow Taylor postulated that, by analyzing work, the "one best way" to do a task would be found.

However, the "big data" era introduced an astronomical amount of data, the wealth of information that companies can potentially unlock and the depth and breadth in which organizations can perform analysis, enable technology and apply analytical methods of the 21st century.

In the past, BA processes have run mostly in silos. Business managers and analysts often used desktop tools to build models and analytics outside of the BI architecture framework. This approach exacerbates problems in data quality and governance and lacks an enterprise-wide approach for data integration and standards. Furthermore, in order to perform deep analytics, data needs to be captured and modeled at the most granular level, and as a result, the amount of data for processing could quickly overwhelm a personal computer. The scope of analysis and performance of such analytical processes are limited due to the constraints of desktop or laptop PC power.

The good news is that existing BI architecture and its infrastructure, which includes powerful BI and database servers, are able to transform and adapt to meet the new analytical requirements. The intent here is to focus on the architecture aspect of BI and BA and propose a new solution with an integrated approach.

The Traditional BI Architecture

The traditional BI architecture shown in Figure 1 integrates enterprise data from multiple sources, which could be internal and external, into an operational data store. Internally, many organizations have critical corporate data in transactional systems, such as ERP, CRM and point of sale. External data can be sourced from industry benchmarks, census, business bureau and more. Data sources can also be both structured and unstructured, such as emails, Web pages and blogs. Data is then extracted, transformed and loaded from the ODS to the enterprise data warehouse in a formatted structure suited for analysis.

It's crucial that the enterprise solution implemented gathers all relevant data for analytics to provide a holistic view of analyzed subject and a single version of the truth. This could include, for example, customer, product, pricing, sales and revenue data. Additionally, OLAP cubes and data marts can be built to focus analytics on a specific domain. This first step accomplishes data collection and preparation for reporting and analysis.

On top of the data service layer, BI reporting and visualization tools serve as a presentation layer providing financial, operational and regulatory reports through a business semantic layer, which translates technical terms into business language that executives, managers and analysts can understand. For the most part, the BI presentation layer is very heavy on reporting but light on advanced analytics.