Subsequent to our latest columns about self-service and the public proclivity to mine unstructured data for personal objectives, we were invariably led to the subject of text analytics. Some questions are obvious, but we were interested to learn more about the status quo of text analytics in the hands of corporate business, its uptake and current use. Given the value of the search box for users of any purpose, I wanted to know how businesses were pursuing this for internal purposes, and how text analytics is being integrated with traditional BI programs.For this I was fortunate to come across the work of Fern Halper, a partner at Hurwitz & Associates, who has spent a lot of time lately tracking and documenting the uptake of text analytics. I have gained a lot of respect for Judith Hurwitz and her staff over the last many years, and Judith herself will be contributing a column to the next issue of BI Review.

But back to the subject. As it turns out, text analytics in corporate hands is both real and still emerging. "Obviously we saw a convergence of text analytics and business intelligence and search and to some degree content management although the latter was sort of lagging," Halper tells me. "Then we saw the acquisition of ClearForest, [by Reuters] and other small players being grabbed up." The net effect Halper saw was that rapidly maturing technology had caused a shift in the relevance of the market, and that corporations specifically were seeking to align text analytics alongside traditional BI programs.

Halper looked at three groups of companies: those that weren't going to adopt text analytics in the near term, those that were evaluating it and those that had already implemented it. "For the ones that had already implemented it, the paradigm was to use text analytics as part of their business intelligence, they were looking at it as an extension of what they were already doing." While there was some ambiguity when it came to segregating enterprise content management and Web content management, a large majority of respondents said yes, text analytics were meant to be integrated with their BI programs.

The marriage of structured and unstructured content has always been difficult to summarize neatly. Last year I interviewed data warehousing authority Bill Inmon who, in his DW 2.0 model, was suggesting there be a staging area for "structuring" unstructured content in the data warehouse paradigm. Inmon somewhat reluctantly refined his own model while conceding that the traditional data warehouse was never meant to accommodate unstructured information. Comments have come down on both sides of this argument's viability, but Halper agrees that "structuring" unstructured information is at the core of business's interest.

Lest we get lost in the weeds, Halper says the drivers for this still awkward union start with focus on the customer. Seventy-five percent of organizations responding to her survey were planning to deploy text analytics were considering customer care solutions. A vast majority said they were extracting text and combining it with structured information for use by data mining and BI tools. Tough as this sounds, respondents said they were finding a high value solution in the combination.

Halper points to financial services as a leader in the BI/text analytics marriage, but finds the subject increasingly relevant in any vertical that tackles issues of customer service and warranty or service support. Again, the maturity of the technology is making this possible. "I remember when years ago when I was at AT&T, we wanted to get at this unstructured information to predict customers that were going to disconnect the service but the technology wasn't there and we couldn't get through the call center logs. A lot of those questions can be answered now and people are thinking about e-discovery and all the emails, and that is another hot area. The customer care stuff is already happening but we have to remember that it's still an early adopter market."

Unlike the Web analytics we've documented recently, traditional BI providers are already integrating text analytics with their products. We mentioned ClearForest; more recently Business Objects made its text intentions clear with the acquisition on Inxight. There's every reason to believe we'll soon see text analytics as an integrated function of your favorite BI tools.

Time to start thinking about how you will use it. So many programs, so little time in the day. There's more information available at

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access