The Next Generation of Business Intelligence: Operational BI

Information Management Magazine, May 2005

Colin White

A business intelligence (BI) system is a key component of a company's IT framework. It is the component that enables business users to report on, analyze and optimize business operations to reduce costs and increase revenues. Most companies use this component for strategic and tactical decision making where the decision-making cycle may span a time period of several weeks (e.g., campaign management) or months (e.g., improving customer satisfaction).

Competitive pressures, however, are forcing companies to react faster to changing business conditions and customer requirements. As a result, there is now a need to use BI to help drive and optimize business operations on a daily basis, and, in some cases, even for intraday decision making. This type of BI is usually called operational business intelligence.

The objective of this article is to explore the business requirements for operational BI and to review different types of operational BI processing, technologies and products that support those requirements. The objective of operational BI is to make more timely business decisions, and, therefore, it has a close relationship to the subject of real-time or right-time BI processing.¹

Types of IT and BI Processing

IT systems support three main types of application processing (see Figure 1): business transaction (BTx) processing, BI processing and collaborative processing. BTx processing drives day-to-day business operations and supports business activities such as order entry, inventory control, shipping, billing and so forth. BI processing reports on and analyzes BTx processing, and provides information about how well this processing is meeting business requirements. Business users employ the output produced by BI applications to optimize BTx processing to more closely match business goals and requirements. This optimization process involves discussions between business experts about possible ways of improving business processes. The interaction between these business users is enabled by collaborative processing.

Figure 1: Types of Business Processes and Data

In a traditional BI system environment, the time between events occurring in BTx systems and action being taken based on BI system output is relatively long - a matter of days, weeks or months. The strategic and tactical decision making supported by a traditional BI environment is reactive in nature and is based on summarized and historical data. This long decision-making cycle allows the BI system to be loosely connected to related BTx and collaborative applications. Batch ETL jobs can be used to extract operational source data and load it into a data warehouse; and reporting applications can be used to produce and burst reports, and distribute the results to Web-based desktop and mobile users.

The objective of an operational BI system is to react faster to business needs and to anticipate business problems in advance before they become major issues. This style of processing requires tighter connections between the BI system and its BTx and collaborative counterparts. It also requires more timely (i.e., zero- or low-latency) detailed BTx data. How timely this data needs to be will vary by company and application. Telephone and credit card companies using BI to detect fraud will need the data to be as close to real-time as possible, whereas the use of BI to optimize supply chain and procurement operations is less time-sensitive.

Operational BI Reporting

Operational BI is not new -- companies have been doing operational reporting for many years. When mainframes dominated the IT landscape, operational reporting consisted primarily of batch production reporting jobs that were run overnight against live BTx data to avoid affecting the performance of online BTx processing. Prime-time ad hoc reporting was usually restricted to parameterized queries that were used to retrieve information about a specific customer, order, product, etc. In this environment, production and ad hoc report output often contained encoded information, such as product codes, that only a business expert could interpret.

As the number of data sources proliferated and business needs caused online operations to be extended (approaching 24 hours in some cases), it became clear that another solution had to be found for supporting operational reporting. One popular approach is to collect and consolidate detailed BTx data into an operational data store (ODS). The use of an ODS allowed batch and ad hoc operational reporting at any time without directly affecting online BTx application performance. It also provided a single and integrated view of BTx data, and enabled the BTx data to be transformed into a more usable and readable format.

The downside of using an ODS is the performance impact on BTx processing when capturing data changes for consolidation into the ODS, and the need to store and manage duplicate data. Another disadvantage for certain types of applications is the data latency introduced by copying data from BTx data sources into the ODS. It is both difficult and expensive to create a close to real-time ODS. Data warehouse appliances from companies such as Datallegro and Netezza can help reduce the cost of managing ODS data where the amount of data is in the order of one to three terabytes. For larger ODS environments, solutions from companies such as Teradata are more appropriate.

Another solution for doing operational reporting is to replicate the BTx source data into a second identical copy. The replicated copy is usually a real-time copy that can be used for disaster recovery, in addition to operational reporting. The problem here, of course, is that this approach doesn't support the reformatting of BTx source data and doesn't overcome the issue of having to report against multiple dispersed BTx data stores.

The advent of BTx application packages from companies such as Oracle and SAP helped solve some of the problems associated with operational reporting. These companies offer suites of BTx applications that are based on a single logical business view of BTx data. This approach provides a simpler view of BTx data and, in theory, reduces data proliferation. These application packages also often contain operational reporting applications that reduce the need to build custom applications. Even though reporting against live operational data still impacts BTx application performance, improved price/performance of computer hardware does allow a company to reduce this performance impact by using a larger hardware configuration.

Page 1 of 3.