The ability to assemble and analyze information has become a powerful source of competitive advantage for many organizations. However, most companies have not harnessed this capability and are operating at a clear competitive disadvantage. The reasons? It has been excessively expensive and difficult to retrieve and analyze corporate information - until now. In particular, affordable business intelligence (BI) solutions have not been available to the small and medium-sized business (SMB) market, and while many large companies have been able to afford these solutions, they have been frustrated in their attempts to effectively implement and utilize BI capabilities within a reasonable timeframe. The BI environment is changing rapidly as new solutions are emerging that allow many more companies (and a broader group of users within these companies) to access key information and create significant business value. These next-generation solutions offer full BI capabilities, rapid implementation with many fewer demands on a company’s business and technology resources;,dramatically lower cost and lower risk than traditional solutions, and the flexibility to accommodate business changes as they occur. These new solutions will have a major impact on the BI marketplace, providing companies with critical business information and a valuable competitive edge. Increasingly, many leading companies recognize that access to information and analytics is integral to successfully operating and growing their businesses. By being able to view and analyze information throughout their organization, they are able to understand the profitability of market segments and sharpen marketing approaches; improve their ability to serve their customers; better manage their supply base; more rapidly integrate acquisitions; and enhance their overall efficiency and operational performance. Companies such as Wal-Mart, Dell and Amazon have focused analytics on supply chain costs and inventory optimization; Harrah’s, Capital One and Barclay’s have leveraged analytic and BI tools to enhance customer selection, loyalty and service; and MCI, Marriott and Verizon have focused on pricing, profitability and financial analysis.1 The abundance of data now available from both internal and external sources presents a tremendous opportunity for companies to leverage information for competitive advantage. Over the past two decades, large and small companies have implemented new enterprise software, particularly transaction systems, to help them manage their businesses. These new systems include broad enterprise resource planning (ERP) or related management capabilities such as financial management and accounting, order management, asset management, customer relationship management, marketing and sales management, supply chain management, transportation, procurement and related functionality. Transaction systems provide essential capabilities and allow companies to operate much more efficiently, but also generate large volumes of very valuable data. Paradoxically, access to this transaction data is often limited. The data that is organized and stored to optimize the transaction system has fundamentally different requirements than the data needed for rapid access, analysis and reporting. Unfortunately, end users cannot easily extract data, combine it with data from other transaction systems, analyze it or generate reports - precluding their ability to look at data in new ways or gain new insights. The compelling need to access and leverage this valuable data, which is resident but not available to many companies, has driven the growth of BI.

Challenges of Traditional BI

Historically, implementing BI solutions has not been for the weak of heart or those lacking money, resources or time. BI programs are large, lengthy, complex and expensive undertakings that demand careful management. According to industry experts, more than 50 percent of large BI programs fail to deliver to expectations, and many of these projects are abandoned as a direct result of their implementation complexities and/or their failure to deliver anticipated value. Traditional approaches to BI systems necessarily lead to multiphased and very complex projects. Providing information from multiple transaction systems to senior executives and managers for business analytics involves an extensive set of activities - data movement, data transformation, presentation layers, data analytics tools, master data management (MDM), metadata management and overarching data governance. And the lifecycle for implementing BI systems will progress through many steps, including requirements analysis, source systems analysis, gap analysis against existing capabilities, solution architecture, data model development, change management for data governance and implementation. Serving the analytic needs across all functions of the company complicates the design of data models and data transformation processes. Inevitably, companies will start with a goal of a single data management framework, but then implement multiple instances at any or all of the layers of the operational data store (ODS), dimensional data warehouse and data marts. Solution architecture and design must involve the evaluation of multiple special purpose technologies for extract, transform and load (ETL), standard and ad hoc reporting, custom analytics, MDM and metadata management. The project lifecycle is extended by the need for validation of findings at each phase of the project and the complexity of vendor technology selection for multiple best-of-breed technologies or integrated BI platforms. Further, the organization is faced with obtaining the core competencies for each of the technologies and layers of the data management process. In the best case, the typical implementation of BI entails lengthy, multiphase project lifecycles and the investment in multiple technologies. The obvious issues with this are the investment in project resources, the investment in technology, managing data quality and organizational acceptance. The more subtle risk is the ability to respond to changing business strategy, both in the initial project lifecycle as well as for system upgrades. When a company attempts to seize new opportunities, the BI infrastructure must keep pace with the need to evaluate metrics of market entry and performance management. Faced with this challenge, business analysts in both large and small companies often choose to move some or much of reporting and analysis to technologies that they can maintain and understand, most notably Excel spreadsheets and Access databases. While the data management and data governance process is severely compromised when data lands in the desktop tools environment, the more serious issue is how the data is sourced. Business users will extract data from whichever environment is available to them: source transaction systems, ODS, data warehouse or data marts. Inevitably, companies lose any attempt to achieve a single version of the truth. Data management and traditional data warehouse techniques (e.g., conjoined dimensions) are lost because there is no consistency (and no predictability) on the sourcing of data for business analysis. So while these solutions fill a gaping hole in information access, they are neither an enduring nor an ideal answer.

Broader Audience for BI

As you can see, the challenges of creating practical, flexible and economical data access and data integration are significant. These challenges have become preemptive barriers for SMBs and major failure points for too many large companies. But these challenges also form principles that must be incorporated in the next generation of BI solutions. We need simplified data integration and a much faster time frame to implement a BI solution. Why must we procure multiple technologies (ETL tool, data model/database and analytic/reporting software), spend many months (or years) and require enormous business and technical resources for data mapping and weaving these applications together? We need more flexibility to accommodate changes in business strategy and company direction. Why must we distract and waste considerable business and technical resources to revisit our entire data integration framework when we have a new acquisition or an upgrade to a new version of our BI software? We need easier user adoption and the ability to make BI available to a broader group of users. Why can’t we have more easy-to-use, intuitive user interfaces that do not require endless training in order to be fully utilized? We need the ability to rapidly access the full data set and to drill down to the most detailed level of data without losing the ability to scale. We need the ability to report off of large data sets (down to the individual transaction and not from some high-level data summary in a data cube), so we can solve business problems and not just identify that we have a problem. Why must we organize our data in subsets of data marts and data cubes and be limited to the data we can access? We ask these questions to be somewhat provocative, but also to break through traditional barriers that many in the market assumed were immovable. The good news is that we are finding that these barriers can be overcome. New technologies, unfettered by the constraints of the old approach, are now designed to address the needs of business users. There are several next-generation BI software products emerging on the market today. Some of these solutions provide an integrated data extraction, data schema and analytics/reporting solution built from the ground up as a single, integrated product. This approach provides a tremendous advantage in the cost, resources, time and risk of implementation because little to no data mapping/integration is required across the solution stack. And a few of the next-gen BI solutions have revolutionized database design, offering reporting with drill downs off the full data set. These solutions eliminate the need for data marts and data cubes, while providing powerful information to users. Some of the more mature next-gen products have demonstrated repeatedly that a full BI solution can be implemented as quickly as in six or eight weeks, versus the six to 12-plus months required for traditional approaches. These solutions have proven to be robust and scalable, handling up to billions of rows of data in several customer implementations. These next-generation BI solutions deliver a truly integrated BI platform that offers great promise for the SMB market and for larger companies as well. The SMB model, with its need for flexibility, will benefit in additional ways from the rapid implementation and “resource lite” approach that some of the next-generation solutions provide. Some solutions enable companies to incorporate changes to their business model (e.g., acquisitions, new data sources, etc.) even more rapidly than the six-week time frame it takes for the initial implementation. BI technology has achieved great success through the years, enabling many large companies to leverage their information assets to compete on analytics. Today, next-generation BI solutions are providing these same capabilities to SMBs - and at the same time, delivering rapid implementation, lower costs, the flexibility to accommodate business changes and dramatically lower risk of failure. Some of these new solutions are delivered through a software as a service (SaaS) model and offer a very intuitive, easy-to-learn and easy-to-use Web interface that simplifies and accelerates user adoption. These solutions enable SMBs to grow profitability without adding IT headcount for training, software management and troubleshooting. Understandably, the benefits of next-gen solutions are valuable to companies of all sizes, giving them a 360-degree view into what’s happening in their organizations today and enabling them to realize business results immediately. The next wave of BI solutions are meeting evolving industry challenges, delivering a powerful value proposition to the market and providing cost-effective, competitive advantage for companies of all sizes. References:

  1. Thomas H. Davenport. “Competing on Analytics.” Harvard Business Review, January 2006.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access