W H ITE PA PE R
Sentiment
Measuring and Analyzing Emotion in
Customer and Agent Interactions
Contents
Introduction............................................................................................................3
What is Sentiment?................................................................................................3
How Does Sentiment Work?..................................................................................3
How is Sentiment Used?........................................................................................5
Quality Management..............................................................................................5
Agent Enablement and Coaching..........................................................................5
Product Performance Monitoring...........................................................................5
Sales Effectiveness................................................................................................6
Process Improvement Analysis..............................................................................6
Adaptive Survey Collection....................................................................................6
Taking Sentiment to the Next Level.......................................................................7
Journey Excellence Score......................................................................................7
AutoDiscovery.......................................................................................................9
Automatic Categorization......................................................................................9
Query Coverage Analysis.......................................................................................9
Anomaly Detection..............................................................................................10
The Future of AI Analytics....................................................................................10
Introduction
Many years have passed since the advent of Big Data, and the analytics programs that gain insight into
that data. Many iterations of analytics technology have evolved over those years. One of the most influential
achievements of analytics technology is the ability to accurately measure human behavior, and human
emotion. But with analytics of any flavor, it is incumbent on the people using the insights to understand the
purpose of the measurements at their fingertips and to take action.
Read on to learn about sentiment analytics as the foundation of AI modeling for understanding customer and
agent interactions, and how they influence customer satisfaction, loyalty, and experience. Whether it be
agents, products, or processes, every part of the business needs to be aware of highs and lows to remain
relevant and competitive. Beyond sentiment analysis, an even more dynamic future awaits through analytics of
customer experience quality outside the contact center.
What is Sentiment?
As it applies to customer service and contact centers, sentiment is generally referred to as a method of
measuring emotion in customer and agent interactions. The reason behind this measurement is to analyze
these customer and agent interactions to uncover areas of the business that need improvement, to monitor
areas that are critical to customer loyalty and retention, and to monitor agent behaviors.
Using analytics and the Big Data already available through many sources and across many areas of the
business, it is possible to create data sets from any number of channels such as phone calls, emails, chats,
social media posts – any channel stored as voice or text in a database – and probe into these interactions for
greater insights into where customer sentiment is low or high, and why.
Sentiment is a machine learning (AI) model trained to measure whether
a customer interaction is positive, negative, or neutral, on a relational scale.
How Does Sentiment Work?
To be truly effective for businesses, the sentiment measurement needs to be sophisticated enough to identify
the relative emotion of agents and customers separately for more accurate results. First and foremost,
language models are employed to identify positive and negative words and phrases, whether spoken or text
based. The semantics of this are key, however, because positive phrases can offset negative ones. Likewise,
words and phrases need to be scored carefully within the context in which they are spoken. Some words that
may normally be used in a positive manner can also indicate sarcasm or frustration.
3
Sentiment | Measuring and Analyzing Emotion in Customer and Agent Interactions
A sample discovery word cloud identifies
conversations with words and phrases such as
“awesome” or “no problem,” but these statements
can be positive or negative depending on the
context of the conversation.
Beyond language modeling, other layers are added to further calibrate and tune sentiment scores for
accuracy. For example, laughter detection can indicate a positive change in an otherwise negative
conversation. Cross-talk (where the agent and customer talk over each other) might indicate confusion or
frustration. Changes in pitch and tone, or speaking rate, can signal changing emotions during the interaction.
Interactions are scored as negative, positive or neutral on a relational scale. They are also scored as starting
positive and moving to negative, or the other way around, for the purposes of identifying that category of
interactions for root cause analysis. Both positive and negative interactions can have a number of contact
reasons, whether it be something wrong with a process or product, or frustration with an agent. Interactions
that begin positive and end negative can also have many reasons behind them, such as an agent’s
confusion trying to help the customer or customers not liking what they are being told (e.g., in a collections
or billing scenario).
On a more granular level, accuracy of sentiment scoring is further improved with different weights applied to
each quartile of an interaction. Studies have shown that the latter portion of an interaction drives customers’
reported satisfaction more heavily than the former. A final scoring calibration is the length of an interaction.
The longer the interaction, the more opportunities there are to drive a sentiment score. As such, it becomes
important to normalize longer interactions with shorter ones so that they score within similar scale.
With all of these details covered, machine learning is then applied to more than half a million customer
interactions and their resulting survey scores across industry verticals. With continued training and
development, the result is a powerful AI sentiment model that accurately scores all interactions for further
root-cause analysis. As false positives become fewer and fewer in the machine learning algorithm, accuracy
and speed to insight improves.
The capability scores every single interaction, but it is important to understand that the measurement is
most successful in aggregate. While identifying one negative phone call is still worthy of note, it doesn’t help
solve the larger picture of how negative calls are trending, and what topics are driving those negative calls.
Understanding volume and trends is the real value behind sentiment analysis, and there are many uses for
which it can deliver value. As such, one of the most powerful uses for sentiment is when it is combined with
other analytics data, such as topics and queries, or when using metadata to filter specific agents or teams.
4
Sentiment | Measuring and Analyzing Emotion in Customer and Agent Interactions
How is Sentiment Used?
QUALITY MANAGEMENT
A well-established use case for sentiment analysis today is with quality management. Using sentiment
to score agent interactions removes the need for random sampling because the interactions are already
scored. It also empowers quality programs to analyze 100% of interactions to get a true sense of trends
and the reasons behind them. Negative sentiment calls don’t necessarily mean that agent behavior needs
improvement, so understanding where negative interactions occur in the contact center as well as who they
are associated with presents the opportunity to uncover issues with products, processes, and with agent
behaviors.
Instead of random sampling, a workflow can be set up for review of negative sentiment calls. These calls
can be sent to different queues for further review based on any criteria. For example, if a new product line
is seeing an upward trend in negative interactions, those interactions can be sent for review by the quality
manager for that product line. If an agent’s behavior caused a customer to get angry, that interaction can be
sent to the agent’s supervisor for review or to an evaluations queue.
Setting up a quality management workflow
for review of negative sentiment interactions
can be a powerful way to uncover issues with
products, processes, or agent behaviors.
AGENT ENABLEMENT AND COACHING
For agent quality and evaluations, sentiment is especially useful for ranking team members individually,
ranking teams alongside other teams, or even showing site by site comparisons in sentiment. This method of
scoring helps to reveal top performers and bottom performers, which is especially useful for understanding
what behaviors make a top performer’s sentiment so high as well as where a bottom performer might need
coaching assistance. Understanding the behaviors that make a top performer successful helps supervisors
and trainers create effective coaching packages that can be tailored to address low performers, and
sentiment trend monitoring helps ensure that those training packages are effective.
PRODUCT PERFORMANCE MONITORING
The power of sentiment scoring is that it works on any data set, from any viewpoint. A quick search in a
data set for a product name can reveal a sentiment model that tips to the negative or positive at any given
point in time. Queries and reports built around that product allow for better sentiment monitoring to see
how customer satisfaction diminishes or grows as the product matures. Being able to deep-dive into the
individual interactions can uncover product defects that need correction, customer confusion due to lack of
documentation, or agent confusion about how to help the customer due to self-help gaps, to name a few.
5
Sentiment | Measuring and Analyzing Emotion in Customer and Agent Interactions
SALES EFFECTIVENESS
When an organization puts a sales initiative into place, tracking sales effectiveness and the sentiment around
sales attempts can be very helpful for understanding where employees are struggling or doing well. Sales take
on different flavors, as they can be net new sales, up-sell to existing customers, or cross-selling to customers
with related products. Sentiment analysis surrounding the different types of sales attempts can further reveal
which employees are strong or weak in these specific areas, and targeted coaching can be provided to improve
overall revenue numbers. Outside of the employees, sentiment around the sales initiatives themselvescan be
indicators of whether the intention behind the initiatives are successful in operation, or need refinement.
PROCESS IMPROVEMENT ANALYSIS
Just as agent behaviors can drive sentiment, so can broken processes. As explained previously, building quality
workflows so that negative sentiment interactions can be evaluated often reveals situations where there are
issues outside of the agents’ or customers’ control that are causing the poor experience. Broken processes
can be problems that leave agents unable to help customers, business processes that cause unintended
consequences, or even missing processes for unanticipated events.
For example, one retail product line had two models that were almost identical. The documentation for both
models was the same, but their button configuration was slightly different. When people called in confused
about how to use the product, the agents were reading the same documentation as the customers and were
unable to understand why the troubleshooting wasn’t working. Both agent and customer concluded that the
product was defective.
But after they sent replacement models to these customers, the same customers called in again saying the
replacement wasn’t working either. It was a simple process problem of the documentation not specifying the
difference between the two models, with unintended consequences. By analyzing the sentiment of interactions
surrounding that product model, it became clear that there was an issue. But in this case, the product was
not defective.
If businesses build workflow queues and staff individuals who can identify and fix broken processes such as this
documentation and self-help problem, the improved efficiency will directly affect sentiment and overall customer
satisfaction. Trend monitoring and reporting over time can help ensure the process improvements
are successful.
ADAPTIVE SURVEY COLLECTION
When it comes to Voice of the Customer, sentiment analysis is especially powerful because it scores every
single interaction customers have with a business. Traditional survey collection is known to result in a dichotomy
of very positive to very negative feedback, with very little data in between. Additionally, the major challenge of
survey collection is that the data set is indicative of only a very small number of customers’ experiences. At
worst, some companies report a less than 10% response rate to their surveys.
6
Sentiment | Measuring and Analyzing Emotion in Customer and Agent Interactions
An innovative way to refresh the usefulness of the VOC program is to build an adaptive survey system. For
interactions that end negatively, the adaptive survey program could send a brief survey asking if that customer
would like to be contacted back to resolve their issue. If a negative interaction occurs around a product mention,
billing, account management, etc., surveys can be sent that address the specific issue so that the customer has
an opportunity to further explain the problem.
On the other hand, if a positive sentiment interaction is detected, a survey could be sent asking if the customer
is especially pleased with a product or experience and if they might provide a product review or a positive
social-media mention. In this way, the survey collection is more likely to produce substantive feedback that can
then be qualified and quantified with the larger analytics program.
A NICE benchmark report
showing the explosion in
the number of surveys sent
each year, with declining
response rates year over
year. Sentiment analysis
can provide indirect
feedback to augment
survey programs’ visibility
into customer experience.
Taking Sentiment to the Next Level
As a machine learning, AI powered solution, sentiment analysis presents endless possibilities. It has been
proven to be predictive of NPS (Net Promoter Score), and sentiment is often considered synonymous with
the term, “Predictive NPS.” As such, it can be a helpful method of understanding how to improve customer
experience and drive sentiment further into the positive.
JOURNEY EXCELLENCE SCORE
Improving customer experience doesn’t just mean the customer’s experience when interacting with agents.
The customer’s experience with a business involves touchpoints outside the contact center as well. Most
customers have vastly more engagement with a company online than they do with an agent in the contact
center, and picking up the phone is rarely the first interaction in the overarching customer journey. Experience
with self-service channels such as the website or mobile applications, retail store visits, mailed promotions
or account updates, or the time it takes for an order to be received, to name a few, can directly or indirectly
affect how the customer views the business. Being able to accurately measure customer experience begins with
pulling together data from all sources where a customer may have explicitly or implicitly provided feedback
7
Sentiment | Measuring and Analyzing Emotion in Customer and Agent Interactions
along their journey with the business and putting it together in a timeline so that experiences as they move
from touchpoint to touchpoint across channels can be better understood and analyzed. Contact center
interactions are an invaluable source for measuring sentiment, but sometimes situations occur prior to the
interaction that may affect the customer’s sentiment when they do make direct contact.
The Journey Excellence Score (JES) is an amalgamation of many different methods of measuring customer
experience quality, delivered as a score from 0 to 100. It uses many customer experiences indicators,
including but not limited to the following:
• Interactions involving specific contact reasons like cancellations or complaints
• Events such as a closed sale, upgrade, or lost sale.
• Survey responses
• Sentiment scores on contact center interactions
• Journey “effort,” often expressed in terms of the number and duration of interactions
The Journey Excellence Score provides the next level of sentiment scoring, effectively offering businesses
the ability to predict sentiment by finding customer journeys that need improvement. Just as sentiment is an AI
machine learning model, so is JES. It is trained with a robust dataset prior to deployment, and its performance
is maintained by retraining the model periodically to prevent data drift (this is typical maintenance for a
machine learning model).
The Journey
Excellence Score
is a machine
learning AI model
that combines
explicit and implicit
customer feedback
with customer effort
data to produce a
score between 0
and 100.
The resulting information is presented in a dashboard that presents overall Journey Excellence Score for the
entire dataset in aggregate, also broken down into different business processes, contact reasons, or intent
across multiple interactions. The model also recommends journey types for further investigation. To learn
more about the Journey Excellence Score, download the white paper.
8
Sentiment | Measuring and Analyzing Emotion in Customer and Agent Interactions
AUTODISCOVERY
The role of sentiment in analysis continues to grow from a single reporting metric to a value that contributes to
bigger picture metrics such as the Journey Excellence Score, or as in the AutoDiscovery module, a first class
filter working alongside AI generated categories to slice into data models. AutoDiscovery consists of three main
capabilities: Automatic Categorization, Anomaly Detection, and Query Coverage Analysis. These are designed
to help reduce time to insight, reduce effort surfacing high volume or low sentiment categories, detection
of anomalous activity in contact center trends, and better understanding of how well queries are covering
priority conversations for the business.
As the name “AutoDiscovery” implies, these modules are most helpful in the discovery phase of analytics.
Discovery is the phase when data is mined for insights that are not yet known, quantified, or qualified. These
tools are designed to surface unknowns and quantify them automatically by mining the data without human
supervision, so that work can be focused exactly where in the data further analysis is needed.
AUTOMATIC CATEGORIZATION
The unsupervised machine learning model for Automatic Categorization uses customer data to automatically
identify clusters of topics and relationships amongst the clusters within the interaction data. The clusters are
visualized by size and color via metric filters such as sentiment, volume, interaction duration, trend anomalies,
average handle time, volume and cross-talk providing for quick insight to the topics occurring in the data and
associated attributes.
Automatic categorization
visually represents
clusters of topics by
sentiment and volume,
and the relationships
between them.
QUERY COVERAGE ANALYSIS
Query Coverage Analysis maps existing manually curated structured queries amongst the topics discovered
in automatic categorization, providing the analyst an easy mechanism for identifying topics that are not
currently being quantified or tracked. This makes it easy to see where fast action might be taken to close
knowledge gaps, while also providing a confidence level for topics already being monitored. In some cases,
a query might provide coverage of a topic, but be missing additional information that indicates that the topic is
not fully quantified – the visualization of this gap makes it possible to strengthen existing queries.
9
Sentiment | Measuring and Analyzing Emotion in Customer and Agent Interactions
ANOMALY DETECTION
It is conceptually intuitive to mine data for high volume topics that have unusual occurrence patterns. But
what if there is a pocket of low volume conversations that have a high value to the business? Anomaly
detection automatically identifies phrases and topics whose “arrival pattern” differs from its typical pattern.
By automatically identifying these changes in trends, Anomaly Detection can help surface emerging topics
whose volume is likely to grow due to a newly uncovered problem, as well as uncovering topics that might never
otherwise be identified through day to day analysis of big data and high volumes of interactions.
The Future of AI Analytics
Big Data is just getting bigger, and no human can handle the task of manually uncovering the valuable insights
that have the power to change how organizations deliver on high customer expectations. Sentiment analytics
represents one of the early AI models that delivers insights across a myriad of scenarios. Using its capabilities
not just as a single metric but as a first class filter among other AI based metrics is just one use case for
how predictive AI analytics is taking over analytics programs. Additionally, the need for data insights beyond
just interactions data and into the full customer journey opens the door for predictive AI models such as the
Journey Excellence Score.
Using supervised, semi-supervised, or unsupervised machine learning to mine Big Data for the purpose of
improved customer experience, among many other benefits to businesses, is the natural progression for
modern analytics programs. AI models simply need enough data to train them to be accurate enough to
operate efficiently and accurately once deployed. Data sets can be used to build models that can predict
customer churn, personality traits needed in contact center staffing, sales opportunities, behavioral insights,
cross-functional insights, improved customer understanding, and on and on. The possibilities are endless.
To learn more about how AI is revolutionizing customer experience and contact center
analytics, visit www.nexidia.com.
Copyright © 2019, NICE Ltd. All rights reserved.
This document and the information herein are the exclusive property of NICE, Ltd. and/or its affiliated companies (Affiliates) and shall not be disclosed, in whole or in
part, to any third party or utilized for any purpose other than the express purpose for which it has been provided. All information included in this document, such as text,
graphics, photos, logos and images, is the exclusive property of NICE or an Affiliate and is protected by United States and international copyright laws. Permission is
granted to use, view and photocopy (or print) materials from this document only in connection with the products to which this document relates and subject to the terms of
license applicable to such products. Any other use, copying, distribution, retransmission or modification of the information in this document without the express prior written
permission of NICE or an Affiliate is strictly prohibited. In the event of any permitted copying, redistribution or publication of copyrighted material, no change in, or deletion
of, author attribution, trademark legend or copyright notice shall be made.
Nexidia – Headquarters +1 (866) 355 1241
3565 Piedmont Road NE, Building Two, Suite 400, Atlanta, GA 30305, USA
Nexidia – UK +44 (0)207 002 3000
160 Queen Victoria Street, London EC4V 4BF, United Kingdom nexidia.com
10
Sentiment | Measuring and Analyzing Emotion in Customer and Agent Interactions
