Press articles and industry conferences tend to focus on the latest and more sophisticated information technologies. In business intelligence (BI) and data warehousing, this includes evolving technologies such as real-time processing, business performance management, executive dashboards and predictive analysis. As a result, well-established approaches get shortchanged, and it is often forgotten that traditional technologies such as enterprise reporting still form the underpinnings of most BI systems.
Attending and presenting at business intelligence conferences provides me with an ideal opportunity to track the level of interest in any given topic. While there is no doubt that many people want to learn about new and advanced BI technologies, a significant percentage of the audience is still trying to solve more basic problems that are often ignored in the rush to address the latest industry buzzwords. There is still high interest in traditional technologies because companies are still facing many issues when trying to implement them. In the enterprise reporting area, for example, companies are constantly fighting complexity and data integration issues in their pursuit to produce accurate, consistent and timely operational reports.
If you have any doubts about the demand for improved enterprise reporting, take a look at the latest announcements from leading BI vendors such as Business Objects (acquisition of Crystal Decisions), Cognos (ReportNet), MicroStrategy (Report Services) and Microsoft (Reporting Services), which clearly show that this area of BI is still of paramount importance. During 2004, we are likely to see high awareness of this topic and some intense vendor competition.
Why do we continue to focus on this topic, given that we have been doing enterprise reporting for more than 30 years? What is different about the new products, and where are we heading with enterprise reporting? To answer these questions, we need to trace the evolution of enterprise reporting from a user, vendor and technology perspective.
Early enterprise reporting applications consisted of batch applications written by IT. These applications accessed centralized operational transaction databases and produced huge volumes of paper on a daily, weekly or monthly basis. The advent of client/server computing introduced new interactive query tools that supported the creation of ad hoc reports. The benefits of this approach were that it removed the necessity for business users to wade through vast amounts paper to find the information they needed and that business users did not have to wait for IT staff to produce the reports they required.
Although client/server reporting was a major step forward, many reporting issues still remained. One problem was that the new tools did not make business users as self-sufficient as IT staff had hoped. Many business users found the tools too complex to use and simply did not have the time to learn how to apply them efficiently. Users were also confused about how to locate the information they required in the IT system. To make matters worse, the new tools had limited formatting and printing capabilities, and often only supported new relational database products -- legacy files and databases could not be accessed. The net result was that many batch reports continued to be run. This situation has improved as reporting products have been enhanced and new Web-based tools have been introduced. Many companies, however, continue to run complex enterprise reports as batch overnight jobs, even though the output of these jobs can now be burst and distributed electronically to users across the Web.
Another issue with client/server interactive reporting tools was that unlike batch applications that ran during off-peak periods, ad hoc queries were run during the day, frequently consuming large amounts of machine resources and impacting the performance of operational transactions. The solution here was to extract operational transaction data into a separate data warehouse and to use the ad hoc tools against the data warehouse instead of operational files and databases. This reduced the decision support load on the operational transaction systems. A data warehouse also offered the benefit of integrated source data, which provided better data quality and consistency. It also enabled data to be summarized and kept for historical reporting and analysis.
The situation in most organizations today is that reporting is done against both live operational transaction data and data warehouse information, depending on application needs and data latency requirements. When intensive reporting and formatting is required, applications are developed by IT experts using enterprise reporting tools. When simpler and ad hoc reporting is required, business users can use Web-based tools to access and format data. Dispersed, inconsistent and out-of-date data continues to be a major issue.
The present objective of BI vendors is to provide a single enterprise reporting environment that enables IT and business users to develop both complex and simple reports. This is the focus of many recent announcements. The BI vendors have begun to realize that many new BI technologies are not growing as fast as expected, and they need instead to address some of the more basic information problems facing organizations in order to maintain and increase revenue.
In parallel with the trend for BI vendors to focus more on enterprise reporting, the relational database management system (DBMS) vendors are steadily enhancing their products to improve support for mixed workloads of operational transaction and decision support processing. They are also releasing better workload and query management tools.
This leaves the problem of data integration and consistency. The enterprise reporting tools have steadily enhanced access to a wide range of data stores, but there is no easy solution for data integration issues. One technology that offers promise for certain types of reporting is enterprise information integration (EII). As I pointed out in my February DM Review column, there is general confusion about EII and an ongoing debate about its relationship to data warehousing.
It is important to realize that although EII cannot magically solve complex data quality and consistency problems, its value lies in its ability to isolate business users from the need to know where data is located. This is achieved by placing a semantic or meta data layer between the applications accessing data and the data stores being accessed. If this semantic layer is used in conjunction with a business information model that defines where the master version of any given piece of business information is located, then this can have a major positive effect on the usability and accuracy of operational reports. This is especially the case in the area of real-time processing (support center access to customer data, for example). EII is not, however, a substitute for the benefits offered by data warehousing for historical reporting and analysis, or in many cases, the building of an operational data store for providing integrated operational data. EII technology can, of course, be used in conjunction with data warehousing tools to simplify the access to operational source data when building a data warehouse.
The DBMS vendors are beginning to focus on the EII area; however, to date, their emphasis has been primarily on performance, scalability and query optimization. IBM is an exception. In addition to enhancing DB2, the company is also putting significant development and marketing emphasis on its DB2 Information Integrator product. The key to success in EII is support for the semantic layer and meta data. As is often the case, new startup companies often provide the better support when it comes to new technology. Companies to watch here include Composite Software, Nimble and MetaMatrix. Application integration vendors, such as BEA with Liquid Data, are also moving into the EII space.
Enterprise reporting then is alive and well. New and improved reporting products are appearing on the market and are worth evaluating. Enterprise reporting problems cannot be solved, however, until data integration issues are also fixed. For this reason, when looking for new enterprise reporting solutions, you must also address improvements in data integration.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access