FREE email newsletters 
Organizations to Understand and Predict Customer Behavior with Text Analytics
Information Management Special Reports, August 25, 2009
If you want to know what someone thinks, just ask. Better yet, just listen. Unsolicited feedback is everywhere and oftentimes people will tell you what they’re thinking without any prompting.
The world is a noisy place and the advent of modern communication technologies has resulted in nearly nonstop dialogue in countless locales. This oversharing of information might seem useless to some, but this “babble” actually has incredible hidden meaning, relationships, patterns and trends.
Companies today would be shortsighted to ignore what their customers are saying about their products and services, in their own words. Those opinions — likes and dislikes — are essential nuggets and reveal much more insight than traditional demographic or transactional data.
This is why organizations are now using text analytics to not only listen” to the voice of the customer, but to respond strategically. Text analytics allows organizations to analyze this unstructured data and harness the power of the written or spoken word by understanding what customers are thinking, expecting and demanding.
The Voice of the Customer Has Gotten Louder
The challenge in analyzing customer insights used to be gathering enough data to make informed decisions. The Internet, email, call center notes and Web 2.0 media, such as blogs, wikis and social networks, have created a new challenge: pulling out the rich content from endless streams of text data.
In fact, businesses today have more opportunities to communicate with their customers, constituents and employees at a time when those customers expect and demand greater levels of intimacy from the companies with which they choose to do business. The goal of truly scalable one-to-one marketing is becoming more achievable every day.
Additionally, there has been a fundamental shift in dynamics of customer opinion measurement. Whereas businesses traditionally controlled the timing and pace of conversations, Web 2.0 technologies have provided customers with a platform to express their views.
People are expressing impromptu opinions on their own terms, just like they always have, but now their comments are permanently plastered in blogs, forums and wikis for everyone to see — including potential customers and competition. The Internet has essentially become a highly visible, very scalable focus group in the clouds.
Nick Patience, managing analyst and research director of information management at The 451 Group, agrees. “With the rise of social media, marketing professionals can gain significant insight from brand or reputation management applications that spider the Web and analyze sentiment, registering the tone of individual consumer input and plumbing available public opinion,” he said. “Internally focused voice-of-the-customer applications use sentiment analysis with transcribed call center exchanges, providing insight into client issues on an individual or aggregate level.”
The Value of Text Analytics
Marketing researchers have long known the benefits of inserting open-ended questions into their survey questionnaires as a natural complement to traditional multiple choice and rating scale questions. However, the traditional workflow for processing this information was both labor-intensive and frequently provided results of questionable reliability. Survey research professionals scanned through thousands of responses and then allocated them to a predefined set of categories.
In order to reduce data processing times, teams of coders would often worked together to categorize the same response. This caused problems of inter-rater reliability. The subtlety and nuance of language makes the analysis of text data a partly subjective exercise. Reasonable people may read the same text and have different interpretations.
Text analytics provides a way to reduce both the processing time and ambiguity of measuring customer opinion. It also enables business users who aren’t survey research professionals to make use of their text data. By combing their domain expertise with sophisticated software technology, they can mine the depths of the text in their survey responses, call centers, social networks and emails and gain a crucial window in to the minds of their customers.
Natural Language Processing and Sentiment Analysis
There are various ways to analyze text, with the differences in each wide ranging. For instance, purely statistical techniques or search engine-derived approaches can yield some value, but will result in a bag-of-words output for problems of even modest complexity. Frequency counts of keywords also won’t provide the compelling, actionable information that’s desired.
The science of numbers is mathematic, and math doesn’t add up with text. If it’s about words and concepts, then the appropriate science is linguistics.
Statistical methods can’t properly address the ambiguity and complexity contained in human languages. Text analysis requires a process that yields to accuracy and traceability. The most comprehensive text analytics solution incorporates natural language processing, which analyzes text as spoken or written by human beings. By analyzing language syntax — understanding words in relation to each other — users can identify key concepts and how they are linked to each other within a sentence, paragraph or document. It’s the meaningful interpretation of the text that provides value, not summary statistics on the words themselves.
NLP also enables more sophisticated sentiment analysis. Sentiment analysis discerns attitudes and opinions about persons, places or things. Knowing how one concept relates to another enables users to define patterns that can be detected within text. For example, in analyzing call center notes, such as “Your customer service is terrible,” sentiment analysis would identify a thing (customer service) and an associated qualifier (terrible). Creating a pattern such as “Customer service + ” would ensure that similar statements, though worded differently, would be detected.
All industries have their own jargon and customized terminology. An integral part of both NLP and sentiment analysis is the ability to create and use libraries containing terms predominantly used within a given business domain. The ability to maintain these libraries or use them as starting points to derive similar but different libraries protects a user’s investment and helps deliver consistent results.
Page 1 of 2.
