In September and October 2006, IDC and DM Review conducted a survey on business intelligence (BI), commonly associated with query, reporting and OLAP technology.1 Throughout this study, IDC used the term business analytics (BA) to describe a superset software category that includes tools and applications for tracking, storing, analyzing, modeling and delivering data in support of automating decision-making and reporting processes. 743 people responded.
IDC's research into the BA market has identified that the market moves in 15-year cycles. The first of these periods, from 1975 to 1990, was characterized by production reporting on mainframes. The second 15-year period, from 1990 to 2005, saw the beginning of the modern era of BI characterized by end user-friendlier client/server-based BI. Eventually query, reporting and OLAP technology migrated from client/server to Web-based architecture with the development of broad suites of BI platforms. 2005 was another turning point in the BI market and ushered in the new wave of investment in business analytics (BA) by organizations in all industries. The current market cycle is expected to last until 2020 and will be focused on expanding the reach of BA to more users both inside and outside the organization.
The goal of reaching more users will be achieved through two primary methods - faster and easier access to accurate and relevant information using appropriate tools and integrating or embedding BA functionality into business process sets.
Although Albert Einstein said, "Not everything that counts can be counted and not everything that can be counted counts," organizations in all industries are collecting and storing an increasing amount of data generated by internal transactional systems as well as external content sources. The challenge of what to measure and how to agree on key performance indicators (KPIs) is a point of frustration for both IT and business. However, most organizations are willing to err on the side of caution and deal with more data rather than discarding it.
The number of source systems feeding a typical data warehouse tended to be between two and nine source systems, according to 56 percent of the respondents. The current size and expected 12-month growth rate of data warehouses, according to the survey results, showed that 40 percent of data warehouses are more than 1TB in total size (see Figure 1), with 18 percent of organizations expecting their largest data warehouse to at least double in size during the next year. The largest percent (37 percent) expect a 25 to 50 percent growth rate for their data warehouse. There are still a significant number of smaller warehouses, but the expected growth suggests increased need for storage and scalable tools to sift through the data. The 2006 survey, as in previous years, shows the expected growth rate of larger data warehouses to outpace that of smaller data warehouses.
Figure 1: What is the current size of your organization's data warehouse?
Compounding the complexity of providing the most relevant data is the increased demand among employees for more rapid access to the data as well. As decision cycles shrink, organizations are looking for near real-time information to be delivered to their decision-makers. As a result, companies are seeking to implement data warehouses that refresh their data in real time. Respondents at 13 percent of organizations refresh their data warehouse in real time. The data suggests that while organizations are interested in real-time refreshes, such projects are going to take longer than expected due to both technical and business reasons.
With the massive amounts of data collected, managed and refreshed at a more rapid rate, organizations will be seeking to distribute business information to a wider audience of decision-makers at all levels. IDC research conducted prior to this survey indicated that 70 percent of organizations are planning to increase the number of internal users of BI tools over the next 12 months. Additionally, 33 percent of organizations plan to provide direct BI tool access to suppliers, 40 percent to customers/members and 20 percent to government agencies.
Challenges and Shortcomings
Given the trend to expand the BA user base, organizations will be faced with a number of challenges that affect the success rate of BA projects. Typical challenges can be segmented into technology and business challenges, including securing executive support, identifying user requirements, deciding on KPIs, managing data quality and master data, and selecting appropriate tools and interfaces for different user groups.
BA projects can include a broad range of software across a technology stack, including data integration, data warehouse management and the various analytic applications and BI tools used by end users. Figure 2 presents the success rates of assorted components to a BA project.
Figure 2: Success Rate of Various BA Project Components
As IT and business users come together in business analytics competency centers to tackle the BA needs of their organizations and improve the success level of such projects, it is important to examine some key trends in the BA software market over the next 12 months. Survey respondents were asked about their future plans on some key topics.
Consolidation of the number of query and reporting tools. With the competitive landscape undergoing important evolutionary changes, the maturity of query, reporting and analysis products means that they are increasingly being treated as a commodity where differentiation among brands becomes less apparent or relevant and price becomes more important. Historically, query and reporting tools have been departmental purchases, which has led to the proliferation of software tools within most large and midsized organizations. The fact is, the core query and reporting functionality required by various departments differs very little, furthering the argument of the benefits that can be gained from standardizing to one set of BI tools. However, increased demand for BA functionality also means that IT is unlikely to be able to fulfill all the BA needs of business end users, suggesting that at some level departmental purchases will continue. Proof of this trend is apparent in the survey results where 73 percent of organizations indicated that they are consolidating the number of BI tools (see Figure 3). However, it is noteworthy that 29 percent of the total are actually consolidating to a single corporate standard while 44 percent are simply reducing the number of tools in use. This suggests the current multitool environment will persist.
Frequency of BA software upgrades. BA software vendors want and need rapid adoption of newer versions of their software to increase satisfaction levels and reduce support costs. It also ensures a steady stream of maintenance revenue. When adoption of upgrades slows greatly, customers are sending a signal that they do not see additional benefits in new products relative to the costs of implementing and supporting the new products. According to the survey, 45 percent of organizations upgrade BI software within the first year or as soon as it becomes available. Close to 80 percent of organizations upgrade their BI software within two years of release.
Figure 3: Are you consolidating the number of query and reporting tools in your organization?
Rapid adoption of new versions of BA software is an indicator that the BA software market is still healthy and that customers see the value of new features and functionality. These findings are consistent with those about the relatively low use of open source BI software discussed later. With the increase in functionality of these open source BA suites, IT departments that spend the time working with these new tools may find them suitable substitutes to commercial products. However, any significant impact, as measured by spending levels on such products, of such open source BI software is likely not going to be felt for several years until more organizational success stories and positive experiences with open source offerings begin to reverse the skepticism among many BA buyers toward open source solutions.
Interest in open source BI software. It makes sense that a tool perceived to have just enough functionality at a low cost would be appealing to many companies with simpler reporting requirements. Several community projects and commercial companies have emerged to address the potential market for open source BI software. Interest among respondents for these offerings was modest, with 18 percent evaluating the products. However, the majority of respondents indicated no interest in the coming year. These tools will need to mature and prove themselves in the market before wider adoption can occur. Companies will continue to feel comfortable in allocating budgets toward commercial products, especially as system integrators largely choose to recommend these products and offer resources skilled in their implementation. The functionality available in open source products may be suitable as a replacement for commercial products, but skepticism still abounds. In markets where software is directly facing end users rather than just IT employees, open source alternatives have been slowly adopted. IDC does not believe open source BA products will have significant impact on the market in the coming year.
Interest in subscription-based BA software. Subscription-based BA software is increasingly attractive to BA software buyers. Figure 4 shows that a healthy 30 percent of respondents indicated they have interest in such an offering. Pricing is definitely a factor in adopting subscription-based software products, but outsourcing the support and maintenance is also greatly appealing to resource-constrained companies. As with the open source BA software market, there are few products to choose from, and customer success stories are not readily or abundantly available to offer a compelling reason for the market to shift at a large rate to subscription-based BA software over the next year.
Figure 4: Would you be interested in a subscription-based solution?
Integration of BA software with other software functions. BA software will be increasingly delivered not just as a standalone solution but also as a component to a broader solution requiring other related software. This will require BA software to be integrated with a variety of other software functions to provide additional insight and context about the business information being analyzed. New solutions will include functionality for better information access, collaboration and business process management. Portal software (followed by master data management) is the most likely offering to be integrated with BA software among the choices provided. Interestingly, on average, technologies dealing with unstructured content are lower on the priority scale for integration with BA software over the next 12 months. Also, compliance applications stand out as separate solutions not integrated with BA solutions, but there is healthy interest in such integration in the near term.
- Brian McDonough. Dan Vesset. "The Latest Trends in the Business Analytics Market." IDC Analytics and Data Warehousing Software Survey, October 2006.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access