OLAP (online analytical processing) has been around for more than 30 years under one name or another. For much of that time, it has meandered peacefully down an information technology (IT) backwater; but no longer. Oracle and Microsoft are now using integrated OLAP servers as weapons to bash each other as well as competitors such as IBM. According to "The OLAP Report" (www.olapreport.com), the OLAP market is now larger than $3 billion; and, unlike others, it is not yet dominated by a "big three."

Despite this increased prominence, there haven't been any detailed surveys of real-world OLAP usage – until now. Survey.com recently published the first such study.

More than 1,300 people took part in the study. After screening out respondents without current OLAP installations, 644 respondents remained. Altogether, people from 45 countries participated, though almost 70 percent were from the U.S. Other countries with a significant representation included the UK with 8.4 percent and France with 2.4 percent. (See Figure 1.)


Figure 1: Geographic Distribution of the Sample

Applications

Sales analysis proved the most popular OLAP application; enterprise resource planning (ERP) data was the most likely to be analyzed. However, higher-value applications such as planning/forecasting and profitability/yield analysis were only just behind, suggesting that OLAP is not just used in bread-and-butter reporting applications. Financial applications such as budgeting and consolidation were in use in almost one-third of OLAP server sites. Interestingly, some fashionable applications, such as balanced scorecard and clickstream analysis, came far down the list with fewer than 20 percent of organizations using OLAP for either.

Success Rates

Overall, OLAP deployments seem to be very successful. Approximately 85 percent of respondents would like to deploy them more widely (though almost 40 percent cited cost as a deterrent), and most plan to purchase more of the current product in use. Overall levels of customer satisfaction are also high. More than 70 percent of respondents said that their projects had largely or completely met their business goals, and less than three percent reported that the goals had only partially been met. There were surprisingly large differences in the goal satisfaction rates between the users of different products.

Seagate Info users seemed to have a much lower goal achievement rate than users of other products. Only 43 percent said that they had largely or fully met their business goals. At the other extreme, MicroStrategy, Express and iTM1 sites seemed to be a little more successful than the rest, with more than 80 percent saying that they had largely or fully met their business goals.

Data warehousing cynics might question these figures, and with good reason. Although most respondents say they are happy with their current implementations, there is evidence of previous failures in the high rate of discontinued use of OLAP products. On average, 40 percent said they had abandoned using OLAP products; and in the case of one leading product, the discontinuance rate was a scary 56 percent.

Problem Areas

Anyone who has implemented a business intelligence application in a big company doesn't need a reminder that organizational issues cause far more problems than product shortcomings. Indeed, scarred veterans might be surprised that merely one-third of the respondents reported company politics as one of their top three problems. Predictably, 32 percent found that data quality was a major problem. "Requirements changed before project was completed" was cited as a major problem by 24 percent.

However, the products themselves fared surprisingly well. Only slightly more than half the respondents reported even a single technical problem as a "top-three" issue. According to the survey, the best products – Microsoft OLAP and Oracle Discoverer – had only half the rate of technical problems reported for the most troublesome products.

Conversely, there were an average of 1.4 reported nontechnical problems per respondent such as company politics, poor quality data and failure to agree on requirements. Yet, a lucky 20 percent reported no significant problems at all.

Query Performance

Among the technical problems, the most important was query performance, with approximately one in six respondents citing it as a major issue. There were big differences between products, with about five times as many users of the worst product complaining compared to the least troublesome. Users of desktop OLAP and ROLAP (relational online analytical processing) had the most problems, while OLAP server products did best.

Figure 2 shows the percentage of respondents of ten major products reporting query performance as one of their top three problems. The numbers in parentheses are the numbers of respondents for each product.


Figure 2: Proportion of Respondents Reporting Query Performance as a "Top Three" Issue

Software Reliability

Perhaps surprisingly, software unreliability was rarely a problem. Only 5.6 percent of respondents cited it as a major issue. The best products had fewer than two percent of users complaining, and less than seven percent of the users of even the least reliable products were dissatisfied.

Zealous Purchasers

Could this reassuring lack of technical problems have encouraged organizations to buy with "irrational exuberance?" I ask because the size of the shelfware mountain is awesome. Indeed, for several OLAP vendors, shelfware is probably their biggest single product.

Shockingly, the average reported shelfware rate is as high as 39 percent for the 10 most popular products. In other words, for every three licensed seats deployed, two languish on the shelf – paid for, but unused. The most enthusiastic over-purchasers appear to be Cognos customers, with 62 percent shelfware. In contrast, the prudent Essbase buyers only have a measly 15 percent shelfware.

Despite the towering shelfware mountain, most sites intend to buy more: 54 percent of sites say they intend to purchase more, and only 12 percent say they do not. It seems that once people try OLAP, they're hooked.

Marketing and Sales Effectiveness

There are big differences in how effectively OLAP products are marketed and sold. For example, Essbase was the OLAP product most likely to have been evaluated (a measure of marketing effectiveness); however, more Microsoft OLAP was purchased and installed than any other product. A remarkable 80 percent of the people who considered Microsoft OLAP deployed it, but only 46 percent of the larger number of people who considered Hyperion Essbase did the same. Curiously, IBM's Essbase clone, DB2 OLAP Server, did not do nearly as well, with only a 24 percent success rate when evaluated by IBM's loyal DB2 base.

Oracle's Parallel Universe

Unlike IBM customers, Oracle customers are far more loyal not just to Oracle, but to its whole culture. Not only do they seem more inclined to buy and happier than average with Express and Discoverer, but they have markedly different behavior in other respects as follows:

  • Only 47 percent of Express sites use Windows NT/2000 servers versus 76 percent of non-Oracle OLAP server sites.
  • A remarkable 40 percent of Oracle sites still use Netscape for Web OLAP versus only 14 percent of non- Oracle sites.
  • Just 20 percent of Express sites use Excel as an OLAP front end versus 70 percent of others who have the choice.

Oracle sites are also more likely to prefer Java as their Web architecture, and to use non-Windows PCs as browser clients.

The Microsoft Effect

It's also clear that Microsoft OLAP is taking a big bite out of the market. Released at the beginning of 1999, it had more respondents using it than any other OLAP product. This was because such a high proportion of those evaluating it went on to deploy it (often without evaluating any competitive products).

Microsoft OLAP was mainly selected because of its low cost, not because people had high expectations. However, it appears to be performing well with its users, as do the Oracle OLAP products. Although the typical Microsoft OLAP database size was quite modest, it actually had more users with very large databases than either Essbase or Express!

Given Microsoft's less-than- stellar reputation for product stability, it is remarkable that Microsoft OLAP users had significantly fewer product quality complaints than those of any other leading OLAP product. This might have been because typically their recent applications have yet to be deployed on an enterprise scale. However, the evidence is that Microsoft OLAP users do have a lower ratio of product-related versus non product-related problems. This suggests that in equally ambitious deployments (measured by the environmental problems reported), Microsoft users report fewer technical problems than users of any other OLAP product. This, along with Microsoft OLAP's very low price, might explain why Microsoft OLAP users were nearly the most loyal of all those reporting.

Web OLAP

For several years, many observers have confidently assumed that OLAP applications were mainly deployed via Web browsers. The survey found that this is not yet true. Despite the hype about the Web, only 18 percent of sites claim 100 percent Web deployment; a larger 28 percent do not use the Web at all. The median is about 40 percent Web deployment.

MicroStrategy had the greatest prevalence of Web deployment, and Applix iTM1 the least. Neither statistic is a surprise, as MicroStrategy has focused on Web delivery for several years, while most iTM1 users have always accessed OLAP applications via spreadsheets. Seagate Info and Essbase also reported below-average Web deployment rates.

As expected, Web deployment rates varied geographically. As the sample had a North American bias, it almost certainly overstates the worldwide Web OLAP usage.

Preferred Web Architectures

We asked desktop OLAP users their preferred browser architecture (Java, ActiveX, DHTML, etc.). Considering how significant the Web is in users' plans, we might have expected that they would have strong preferences for particular Web architectures, but this did not prove to be the case.

One-third of the respondents said they had no strong preferences, and this was well ahead of those who expressed any particular preference. The most popular choice of those who expressed a preference was Java, but this was largely due to the Oracle users, of whom 37 percent preferred this architecture. BusinessObjects users also had a slightly higher than average preference for Java.

Conversely, Brio users supported their own vendor's architecture, by (slightly) preferring the plug-in approach, while Cognos users preferred pure HTML, which happens to be the architecture of choice for PowerPlay users. Few users wanted an ActiveX approach. In general, it seems, users seem to support whatever architecture their supplier does, either because they chose the product for its Web architecture (which seems very unlikely) or because they have been "educated" by their chosen supplier.

The bizarre finding is that few users said that XML (extensible markup language) or XSL (extensible stylesheet language) was their preferred Web architecture, and hardly anyone asked for DHTML (dynamic hypertext markup language) – despite the fact that these are now the preferred routes for most OLAP tools vendors. Perhaps the vendors have yet to reeducate their customers?

Overall, users are happy with most of today's leading OLAP products, but many of yesterday's stars seem to have faded. There were very few active users of some reasonably well-known products such as Gentia, Holos, Pilot, SAP BW and SAS MDDB; and those few users were less satisfied and loyal than average. Customers certainly seem to buy more seats than they need, in some cases actually ending up with more shelfware than deployed seats.

OLAP is certainly moving to the Web, but much more slowly than forecast; and, so far at least, there is minimal interest in deploying OLAP to platforms other than Windows PCs.

Information about the full, detailed survey report (which includes 76 tables and charts, with full analysis on the sales and usage of ten leading products plus limited analysis on 12 minor products) is available from http://www.survey.com/reports/olap .

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access