Text analytics, also known as text mining, is a technology that turns that unstructured information into structured information so that it can be properly analyzed by business intelligence systems. From there, the data can be used to guide business changes and increase customer satisfaction.
With text analytics, businesses have the power to collect, store and analyze information from multiple corporate systems in a single, high-performance environment.
Text analytics opens the floodgates to new insights by allowing companies to analyze unstructured, free-form data in the same way structured data has been analyzed in enterprise data warehouses. Systems that interact with customers are inherently filled with a large amount of unstructured, free-form text.
For example, notes entered from a call center, open-ended responses on a customer survey, comments posted on the Internet—all are defined as unstructured text.
There are many approaches and techniques used to turn text into structured information. Each approach has varying levels of accuracy and utility. In this blog, we will explore those techniques and how they can be used in combination to uncover hidden insights stored within the text.
The Importance of Unstructured Text in a Customer Experience CX program
Currently, there is an explosion of free-form text information being generated by consumers. Studies show that as much as 80% of the information that is created in a corporation is free-form or text in nature.
At the same time, computer technology can not accurately process and understand language in its traditional form because computers are made to simply match patterns, compare and sort.
Therefore, companies are missing or ignoring a large percentage of the valuable information that could be helpful to their business.
Since the revolution of the Internet, paying attention to this type of information has become even more important. Consumers now generate an incredible amount of online content by posting comments that are publicly available to everyone.
Most compelling, this is information that is not being said to the companies themselves, but to the world at large.
Companies have numerous internal systems such as call centers, email and automated feedback systems to gather and manage customer information.
However, public Internet comments are posted for all to see, providing low-cost access to relevant customer thoughts and feelings about a company and its competitors.
Businesses and their competitors can use this information to do competitive research, understand general market trends, and pinpoint emerging problems early on in the product development life cycle.
Transaction or feedback surveys typically contain one or more verbatim questions such as, “How can we improve?” or “Please describe the problem you had.” Responses to these are typically very helpful individually.
But what if you had a few thousand survey responses? How would you summarize them? For these reasons, businesses are turning to text analytics systems and technologies to automatically process and analyze text in all its forms and transform it to be utilized in identifying trends, early warning signs, product issues, suggestions for improvement, and cries for help from customers.
Answering the “Why” Question
Traditional business intelligence systems that analyze structured data are very good for statistically reporting the current state of customers and markets. Sales are up or sales are down. Customers are more satisfied, or customers are less satisfied.
This region seems to be performing better than that region. Although these are important facts to understand, the key insights that are missing are why those things are happening now.
Answering the “why” behind the data is typically not possible, even with investments in interpolation, modeling, and statistical analysis on traditional structured data.
However, when you combine structured data with unstructured data, such as freeform replies to open-ended survey questions or comments on the Internet, you add another layer of depth that can give you a complete picture.
One of the most common things that can be learned using text analytics is when a customer expresses some sort of positive or negative emotion in conjunction with a company or brand interaction.
For example, you can see what customers are saying about a poorly performing product, why customers in a specific region for a specific type of product and for a specific time period are unhappy, and what were the key issues that drove low satisfaction.
Text analytics is the key to understanding these questions. Well-designed surveys will typically ask for customers to rate products or services, then ask “Why did you give us that rating?” or “Why were you dissatisfied with our service?”
The answers to those questions provide powerful insights. However, until recently this has been difficult to analyze.
Businesses have traditionally relied on verbatim coding systems where outsourced vendors or analysts manually review a random sample of a few hundred responses, and then create codes to categorize them into common issues.
Although manually reviewing a sample of responses provides some level of accuracy, there are some inherent flaws in that process.
First and foremost is that you are not looking at all of the data. If you have thousands or hundreds of thousands of responses, you are only able to cost effectively analyze a small fraction of the available information.