There have been several major waves of development and expansion in the business intelligence (BI) industry since the 1980s, and now we're on the cusp of another: BI standardization.
To date, BI has undergone rapid evolution and growth in three major directions:
Physically. BI tools have evolved across platforms. Starting with mainframes and minicomputers, BI tools have been ported to desktop and client/server platforms and, most recently, to the Web.
Horizontally. BI tools have evolved from mono-modal toolsets geared to specific types of analytical users (e.g., analysts, developers, managers, casual users, etc.) to multi-modal analytical suites (i.e., integrated query, reporting and analysis) customizable to any user type.
Vertically. Leading BI players are integrating the BI "stack" by supplementing BI suites with data integration tools to supply the raw material (i.e., data warehouses) and with graphical development workbenches and packaged analytic applications to create the finished product (i.e., applications).
With each successive wave, different groups in organizations have purchased new BI tools and products, creating a mishmash of BI functionality and vendor loyalties. BI standardization attempts to bring order to the haphazard dissemination of BI capabilities within organizations by replacing multiple tools with a single BI platform.
Standardizing on one BI platform offers substantial financial and strategic benefits. However, it can also backfire on companies if they are not careful. How do you select one platform from the many currently in use within your organization? How do you avoid political and cultural battles that can cost more to resolve than the BI standardization effort will save? How can you be sure the new BI platform will scale "up" to support all users and scale "out" to support their diverse sets of analytical needs? In most cases, the answer is to tread carefully, plan thoroughly and go slowly.
It Makes Perfect Sense
The business logic for BI standardization is too compelling to resist. According to a recent TDWI-Giga Research survey, organizations possess an average of 3.4 business intelligence tools.1 However, I have come across many organizations with a dozen or more BI tools and some with as many 25.
The overhead costs of supporting multiple BI tools add up quickly. Organizations must pay software licenses and annual maintenance fees for each BI product they own. Additionally, many BI tools require dedicated servers, thus increasing hardware, operating system and administration costs. In addition, end-user training and support costs climb with each additional BI product. Organizations must assign and train different technical experts to configure each BI tool and develop custom reports. Finally, groups duplicate efforts to research and select BI tools when they could be doing other work.
Chief financial officers and BI executives are now working closely together to reduce the burden of supporting multiple toolsets. More than half of the respondents (57 percent) in the TDWI-Giga Research survey said their organization had a "very high" or "high" desire to standardize its BI portfolio. One-quarter (25 percent) had already standardized their BI platform and one-third (34 percent) planned to do so within 24 months. Twenty-two percent weren't sure of their organization's plans.
However, reducing costs and redundant work effort is only half of the equation. Organizations that want to extend the benefits of BI to a wider audience can't do so unless they first create a standard platform for both data and analysis. On the data side, this calls for the creation of an enterprise data warehousing (EDW) infrastructure (using either a logical or physical architecture). By nature, EDW projects are complex and expensive; however, when successful, they deliver the holy grail of data warehousing a "single version of truth."
Yet, to gain the full benefit of a standard data environment, organizations must also standardize their BI environment. There is no point in creating a single version of truth if every user and group creates reports that calculate metrics differently or uses different time series or attributes to display common data values. Moreover, it's nearly impossible to distribute and manage BI on an enterprise scale unless an organization consolidates everything onto a single BI platform.
TDWI research shows that the allure of "enterprise BI" is as powerful as the need to reduce costs. When TDWI conference attendees were asked, "What is driving your organization to standardize on a single BI tool?" 40 percent said "to deploy BI on an enterprise scale" and 38 percent said "to reduce support costs." The remaining attendees had no plans to standardize on a BI tool or had other reasons to consolidate tools.
Before your organization launches headfirst into a BI standardization project, it needs to understand the potential problems and pitfalls that often afflict such endeavors. There are numerous technical, political and cultural problems that lay hidden beneath the initial benefit/cost analysis, which may cause organizations to rethink their strategy for sweeping the BI environment clean.
Politics. The biggest challenge in any BI standardization project is politics. The nastiest battles occur after a merger or acquisition when two firms have large, dissimilar BI environments. Users and administrators who have become attached to a BI tool or environment will fight to protect their investments and preferred methods of viewing and distributing information. Choosing one environment over another is a political nightmare.
Culture. On a smaller scale, most users would rather not change the way they access, view and analyze information. It takes extra time and effort to learn new tools and become familiar with new reports. Most users will complain in a Dilbert-like fashion if they are forced to relearn new ways of doing old things that add little value to the tasks they manage.
In addition, many users fear change they fear that they won't be as effective with the new tools or techniques as they were with the old ones; they worry that the information they rely on will disappear, undermining their effectiveness and reputation; or they fear that they will lose "control" of the information and not be able to "spin" the results in a favorable light. Before you can move forward with a BI standardization effort, you need to assess these fears rational or not and develop plans to allay them.
Technology. Outside of the social infrastructure, there may be insurmountable technical constraints as well. Some remote offices may use slow dial-up lines, making ad hoc analysis via the Web impossible. Security concerns may limit the use of downloadable Java applets or other "thick" Web client alternatives. Finally, many BI tools lack the infrastructure required to scale up to support hundreds or thousands of users who have diverse analytical requirements. (In other words, you can't just "push" static HTML reports out to users and expect them all to be happy!)
Given this range of problems, following are a few things to consider before consolidating multiple BI tools.
Changing BI Tools Costs Money. Although maintaining multiple BI tools duplicates efforts and expenditures, swapping out BI tools in midstream is not cheap either! First, you need to purchase an enterprise BI license, which costs approximately $500,000 (for a 1,000-user license, not including training or premium support services). Plus, most vendors charge approximately 20 percent per year in maintenance fees to obtain product upgrades and basic support.2
In addition, you will need to retrain users and administrators that have been using other BI tools and rewrite existing reports in the new tool. These "retooling" expenses can be so high that many organizations "grandfather" existing BI tools; they only enforce the BI standard in new projects or engagements.
Many BI Tools are Embedded into Existing Business Processes. Changing a BI tool may involve changing, or at least disrupting, an existing business process. For example, a report may be embedded within an operational application that helps security auditors identify potential fraudulent activity and prepare evidence for an on- site interview. Problems arise if a new BI tool doesn't support the same types of analytical calculations, output formats, delivery channels and external logic used in the existing application. If that's the case, the organization will need to decide whether to customize the tool or reengineer the business process, neither of which is inexpensive nor quick. These types of applications are also excellent candidates for "grandfathering."
New Tools Threaten Careers. Many IT and power users develop skills and expertise in BI tools. Eliminating these tools threatens their career path, job security and reputation within the firm. Undoubtedly, many of these individuals will vigorously resist the introduction of new BI tools. It's best to get ahead of such defensive posturing by enlisting the technical literati to help you migrate to the new environment. A good strategy is to appeal to their desire to learn new technologies and solve technical problems. An "overtime" bonus for participating in the retooling effort also keeps morale headed in the right direction.
Usage Isn't Guaranteed. The most difficult part about implementing BI tools is getting users to use them. Many will revert to old ways of doing things, especially if the decision to migrate is largely political (i.e., following a merger or acquisition) and doesn't make their jobs any easier. Organizations need to aggressively market and sell the new BI environment as well as provide requisite training through all possible channels. Of course, this costs money as well, which should be factored into the initial cost/benefit analysis. In the end, sometimes it's necessary to go cold turkey shutting off the old environments before people will migrate to the new platform.
Selecting the Right Tools
Once you've assessed your political and cultural liabilities and come up with a workable strategy, you will need to select a BI tool on which to standardize. This is not as easy as it may sound. The requirements for an enterprise BI platform are more stringent and comprehensive than departmental BI implementations that serve a homogeneous user population. If you boil down the hundreds of requirements that fill the current crop of requests for proposals (RFPs) for BI standardization projects, organizations are looking for "breadth" and "depth" in a BI platform. By breadth, I mean does it scale to support thousands of users? By depth, I mean does it support the diversity of users, from casual report consumers to sophisticated report authors?
Fortunately, leading BI tools vendors have made great strides in the past three years rounding out their BI portfolios. Today's BI suites cover the range of user requirements, from interactive dimensional analysis to pixel-perfect report authoring. (However, beware as most vendors only introduced production reporting modules in 2003.) Most vendors are still struggling with the scalability issue. Most BI tools started out as desktop Windows products and still stumble in delivering a bulletproof, server-centric architecture that can rightfully claim a spot in a legitimate corporate data center without recrimination.
There are some telltale signs of scalability in a BI product. For instance, you should evaluate the way it queues requests, clusters processing, leverages memory, uses caches, leverages database capabilities and monitors system performance. However, the only true way to determine scalability is to test the toolset with your own data in your environment using your own reports. Vendors will balk at doing this because it is very expensive and may lead to nothing but a black eye for them. However, it's your only defense against a multimillion- dollar investment that never gets off the ground.
BI standardization will drive our industry for the next several years. Organizations are leaving too much money on the table to ignore the benefits of consolidating multiple BI tools, yet wise organizations understand that they cannot wipe the slate clean overnight. They may need to evolve gradually over a period of years to a standard BI environment or risk certain mutiny.
1. We suspect the average is low because many organizations that have yet to implement a data warehousing environment attend TDWI conferences. The survey was conducted at the November 2003 TDWI conference in San Diego, California.
2. Based on figures in the "Evaluating BI Suites" course I teach at TDWI. Vendors estimated list prices for a 1,000-user scenario containing a mix of user interaction modes and tools for design and administration.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access