In his bestselling book How to Become CEO (Hyperion, 1998), author Jeffrey Fox described how one of the U.S. automakers, under mounting pressure for fuel efficiency during the 1970s energy crisis, called on its engineers to redesign its cars to be lighter. But the seasoned, veteran engineers argued that doing so was unsafe, impractical and impossible. So, the automaker brought in younger, newly minted engineers who unceremoniously proceeded to shed hundreds of pounds off the cars without negatively impacting safety. Unlike the longtime engineers, the newer engineers succeeded because they were not constrained by preconceptions as to what was possible. Essentially, they succeeded because they didn't know any better!
Similarly, organizations are long overdue for a fresh look at how they can provide enterprise-wide business intelligence (BI), unbiased by long-held misconceptions as to what a BI implementation must entail.
The critical task of providing actionable enterprise data often falls victim to the same limited thinking displayed by those veteran automobile engineers, particularly when such flawed logic is applied to admirable but imprudent efforts to implement pervasive BI across an organization. Many IT professionals mistakenly believe the only way to get enterprise BI into the hands of those who need it is to implement complex BI tools supported by even more complex data management systems. Too often, these solutions prove to be overkill, placing undue burdens on the IT department and forcing companies to incur hefty, avoidable costs.
As BI expert Colin White has observed, disillusionment with existing BI software is already well under way. Some companies are starting to rebel, demanding easier and cheaper BI solutions. These companies, White notes, typically have fewer IT resources and skills necessary to implement BI projects. As a result, they may be struggling to implement even basic BI capabilities.1
This demand for disruptive solutions challenges the long-held misconceptions about BI. Companies today must pay new attention to the overlooked unstructured sources of actionable enterprise data, which can dramatically simplify the demands of providing the right information to the right person at the right time.
Unstructured Data: Separating Value from Hype
Traditional BI systems typically rely on structured information (databases), which usually comprises only a small portion of all enterprise information. However, the vast majority of enterprise information is found in unstructured data sources; that is, information sources outside of databases. Much hype has been devoted to the concept of harvesting unstructured data, while technology writers still disagree about what unstructured data is and whether it has any value at all.
Experts have a wide range of definitions for unstructured data. Some include any sort of potentially informative document, even hand-written notes. Others dismiss unstructured data as an oxymoron and as a gibberish term that is not helping anyone solve their data problems.2 The debate reveals an important kernel of truth: For an unstructured data source to be of authentic value as an enterprise BI source, the process of transforming a certain unstructured data source into actionable structured data must be very simple and reliable, without exception.
This key criterion eliminates the distraction of more exotic unstructured data sources, such as freeform text, handwritten notes, etc., and enables us to recognize that the single most valuable supply of unstructured data in the enterprise is the existing reports and business documents already produced by every organization.
Organizations running enterprise resource planning (ERP) systems, for example, already own a library of existing canned reports. All of the work is already done for them there is no coding to do, no security to work out, and all the information that they need is embedded within the delivered standard report.3 For example, SAP offers canned reports numbering in the several thousands.
Organizations maintaining a huge historical database of an ERP (or other core system) often face a frustrating reality: most of the effort and cost associated with their complex BI tool is intended to allow the organization to create and work with customized views of the very same enterprise data that already appears within various existing ERP report outputs.
Another key attribute of existing reports is that they already contain business rules, which transform raw data collected by enterprise applications into actionable information. These business rules do not exist within the database. Instead, they are executed by the application at runtime, when an existing, or canned, report is created.
For example, a health care organization trying to enable more effective materials management found that reports produced by its ERP solution included critical and complex first in, first out (FIFO) inventory turns calculations that had been performed at report runtime. As a result, it was virtually impossible to replicate the same calculations using a BI tool or report writer. Imagine the frustration of seeing crucial inventory turns data on an existing health care report, with no apparent way to actually work with that data interactively.
Putting Existing Reports to Work as a BI Source
Clearly, the existing reports already produced within the organization comprise its greatest unstructured data asset, and putting those reports to use should be a top priority. The problem many organizations face is that conventional wisdom still says reports cannot be used as a source of BI because they are static; they do not allow the end user to ask questions about the data and see different data views. Thanks to report mining technology, this is no longer true. Existing reports can now easily become live, actionable data for easy analysis, with no new programming required.
Report mining enables the intelligent recognition and parsing of data within existing reports and business documents - typically in plain text or PDF format - into a valid data table, complete with optional new calculated fields of data and database lookups that allow the inclusion of additional data located elsewhere. Report mining also facilitates sorting, filtering, combination with other data and summarization with subtotals and grand totals, as well as easy export to a data cube, Excel, PDF, online form and other applications that allow users to analyze the information. Reports, particularly when intelligently indexed and archived within a report mining-enabled enterprise report management system, can become a wellspring of easily accessed and manipulated data for programming-free BI.
This approach also expands the spectrum of available business information to include reports and business documents provided from external sources, such as outside data processing firms for organizations that have outsourced their core transaction processing, or special industry-specific documents provided by an outside entity, like explanation of benefits and other health care-related documents. As a result, external reports suddenly become a source of actionable information.
Report mining is delivering the kind of solution companies have been demanding for simplified BI. Since the approach is programming-free, the process of transforming an unstructured data source (i.e., existing report output) into a structured data source (i.e., a customized data table, ready for analysis and export) is in fact easier than the IT back-office work needed to map raw databases together behind the scenes.
As organizations seek to free themselves from complicated BI tools, its important to note that report mining passes the unstructured data hype test and warrants an in-depth review as an innovative disruptive BI solution. Organizations may find that report mining allows them to achieve pervasive BI at a dramatically lower cost and complexity than traditional BI solutions.
- Colin White. The Need for Easier and Lower Cost Business Intelligence. Business Intelligence Network, September 19, 2007.
- Josh Berkus. Unstructured Data as an Oxymoron. ITToolbox, September 1, 2005.
- SAP. The Three Biggest Misconceptions About SAP Reporting - And How to Dispel Them. SAP Professional Journal, May/June 2002.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access