Emotion AI
Seth Grimes
Alta Plana Corporation
@sethgrimes
March 5, 2020
2008: emotionai.com
“If we want computers to be genuinely
intelligent, to adapt to us and to interact
naturally with us, then they will need to ability
to recognize and express emotions, and to
have what has come to be called ‘emotional
intelligence.’”
-- Rosalind Picard, Affective Computing, 1997
Affective Computing
1997
Bloom’s Taxonomy of Educational Objectives, 1956:
Affective, Cognitive & Psychomotor
« Le cœur a ses raisons que la raison ne connaît
point. »
-- Blaise Pascal
“All models are wrong, but some are useful.”
-- George Box
Emotion models: Geneva
Emotion Wheel
https://www.unige.ch/cisa/files/6914/6720/3988/2005_Scherer_SSI.pdf
https://www.unige.ch/cisa/gew
Emotion models: Plutchik
http://sentic.net
Plutchik via SenticNet
“As a framework,
SenticNet consists of a set
of tools and techniques for
sentiment analysis
combining commonsense
reasoning, psychology,
linguistics, and machine
learning.”
What about Sentiment Analysis? Text sourced…
https://dl.acm.org/doi/10.1145/945645.945658,
October 2003
https://dl.acm.org/doi/10.1561/1500000011,
January 2008
Bing Liu, 2010
Sentiment -> Emotion points
• Both sentiment and emotion are subjective.
• Emotion models may be fine-grained and hierarchical.
• Emotion categories aren’t orthogonal; emotion is
compositional.
• Sentiment is situational and not simply a one-dimensional
projection of emotion.
Consider…
Disclaimer
I use A LOT of commercial product images in the slides
that follow. These are illustrations and not meant as
recommendations.
Emotion:
Many sources
Emotion AI
“Emotion mining is the science of detecting, analyzing, and
evaluating humans' feelings towards different events, issues,
services, or any other interest.”
Emotion synthesis enhances the ability of a machine to provide
meaningful, contextual responses, by conveying an appropriate
emotional state through words, voice, and expression.
Emotion induction aims to evoke a certain emotional response or
affective state.
Examples …
Emotion
in Text:
Lexicons
Emotion in Text
https://saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm
“Parsing Text for Emotion Terms:
Analysis & Visualization Using R”
Emotion in
Text: Parsing,
Stats
https://datascienceplus.com/parsing-text-for-emotion-terms-analysis-visualization-using-r/
Emotion in Text
Voice :: Aspiration
Speech https://www.phon.ucl.ac.uk/courses/spsci/iss/week9.php
“Voice cues are commonly divided into those related to: (a) fundamental
frequency (F0, a correlate of the perceived pitch), (b) vocal perturbation (short-
term variability in sound production), (c) voice quality (a correlate of the
perceived ‘timbre’), (d) intensity (a correlate of the perceived loudness), and (e)
temporal aspects of speech (e.g., speech rate), as well as various combinations of
these aspects (e.g., prosodic features).”
http://www.scholarpedia.org/article/Speech_emotion_analysis
Voice Bots
Human =
Good
https://voicebot.ai/2019/11/27/alexa-is-learning-to-speak-emotionally/
https://voicebot.ai/2019/11/21/correct-call-to-action-recall-by-users-is-twice-
as-high-for-human-voices-as-synthetic-for-voice-apps/
Voice Rendering…via Markup
https://developer.amazon.com/en-US/blogs/alexa/alexa-skills-kit/2019/11/new-alexa-emotions-and-speaking-styles
Voice Rendering
https://www.sciencedirect.com/science/article/abs/pii/S0167639317303187
Conversational AI:
Unemotional
https://reprints.forrester.com/#/assets/2/664/RES144416/reports
“Conversational Intelligence & Behavioral Prediction Insights from
Voice: Our Oliver engine offers ASR+ with a sophisticated layer of
emotion recognition metrics & behavioral KPIs, not only from what is
being said but also from the how it is said.”
https://www.ipsoft.com/2019/08/29/emotional
-intelligence-in-conversational-ai-why-amelia-
is-a-leader/
“When Amelia is working with customers or coworkers, she may sense frustration,
anger, or even sadness. In these moments, she’s able to calibrate her tone and
phraseology in order to be considerate of how her counterpart is feeling. With this
emotional intelligence, she can build closer connections and relationships between
a company and their customers and employees.
“The language Amelia uses to interact with customers can vary from system to
system and even from project to project. She can use slang, humor and even
sarcasm, or she can speak in direct professional terms.”
MIT Nexi Robot, 2008
SoftBank Pepper, 2015
Facial Coding
SRI + Toyota
Counterpoint
Counterpoint 2
Affect recognition, a subset of facial
recognition that claims to “read” our
inner emotions by interpreting the
micro-expressions on our face, has
been a particular focus of growing
concern in 2019—not only because it
can encode biases, but because it
lacks any solid scientific foundation to
ensure accurate or even valid results.
Call for speakers through March 31: lt-innovate.org
Emotion AI
Seth Grimes
Alta Plana Corporation
@sethgrimes
March 5, 2020

Emotion AI