I can tell you that widespread use of text mining and unstructured BI search is still on the horizon in most mainstream corporate settings. It is an unscientific finding, but I have connections, I am pretty good at digging out stories and I've had a tough time finding end users with an unstructured or integrated structured/unstructured story to bolt onto their traditional BI reporting/performance management work.Or maybe they have a story but it's too early to share. (I'm sure the National Security Agency could tell me a hell of a text mining story, but they're not talking.) With unstructured and semi-structured data playing a greater role in the business intelligence plans of enterprises there's every reason to believe that BI search and text analytics could redefine reporting in the future. At least one person thinks search will redefine reporting in the near future, though not exactly as you might think. According to a new report from The Data Warehouse Institute's Phil Russom, BI search will have a twist that could have a big impact on business intelligence itself.
I admit I haven't fully formulated scenarios or use cases for BI search, but I've mostly envisioned it as searching for existing reports and maybe looking up a nugget to support a reporting need. Russom's view may not be the first such take, but it did get me thinking. "Rudimentary BI search to find a report among thousands of reports is nice and useful, but advanced configurations are far more iconoclastic," he tells BI Review. "For instance, using BI search you can find sections of reports - even small artifacts like queries and data references - then assemble those to create a new report."
We've spent a good bit of time talking about the shortage of skilled database and business analysts to support growing demand for business intelligence. Then again, if Russom is right, it is plausible that advanced engines and comprehensive, intuitive or otherwise intelligent metadata layers could get past some of this heavy lifting down the road. Let's lift the vision right from the report's text:
"Looking into our crystal ball for a moment, we see a possible future in which most BI will be operated through the search box. BI search can already find and deliver reports or sections of them, fetch data, and paste found items into new reports. If it catches on, BI search stands a good change of significantly changing report authoring and delivery. This would be a radical simplification of these two complex BI tasks, which in turn could help more new users embrace BI and older users be more productive."
There are plenty of practical reasons - including the absence of a mechanism - that explain why the search box is not presently a turnkey reporting solution. Yet we do know that there is much demand for incorporating unstructured data into a data warehouse feed, which can only drive the trend. TDWI's survey, for example, predicts the greatest increases with technologies that convey natural language (unstructured) information in text such as voice recognition (up 81 percent in three years), wikis (81 percent), RSS feeds (68 percent), taxonomies (70 percent), instant messaging (69 percent), and document management systems (61 percent). Though these numbers have risen from almost nothing in a short time as the report notes, to someone like me who is always looking for signs of what the future working interface will look like, it's pretty compelling. Plus, I've spoken with Russom off and on during his days at Giga and Forrester Research, and he's never strayed too far into the ether without a compass. By the way, Russom's research indicates that about half of enterprise data is unstructured, compared to the 80 percent or more predicted elsewhere - but why quibble. The point is the same, he told me: "Organizations need to start addressing these untapped resources so that decisions are based on a complete view of corporate performance instead of the current partial view."
The TDWI report, BI Search and Text Analytics: New Data Warehouse Sources, takes us through the progression of expanding search undertakings: searching for reports in a single BI platform; searching across multiple platforms; searching report metadata versus other report content; searching for report sections (versus whole reports); searching non-BI content along with reports; BI search as a subset of enterprise search; and searching for structured data that is indexed or referenced elsewhere. As demand for unstructured data increases alongside the need to transform it into some sort of structure for a data warehouse feed, the search angle of ad hoc reporting may become more apparent. In any case, this interesting report (summary here) suggests that BI search and text analytics are becoming part of the technology stack and that could have a variety of interesting effects on data warehousing, data integration and reporting.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access