Vidyavardhini’s College of Engineering &

Technology
Department of Computer Engineering

Natural Language Processing (NLP)

Module 1. Introduction to NLP

Code/Sub: - CSDC7013/ NLP /VII

By Prof. Sridhar S

1
Course outcomes:
On successful completion of course learner
should able to :

CO1: Identify Challenges of NLP and ambiguities in

natural language.
Contents:
History of NLP
Generic NLP system
levels of NLP
Knowledge in language processing
Ambiguity in Natural language
Stages in NLP
Challenges of NLP
Applications of NLP
Introduction to NLP

A language : a set of rules or set of symbol.

Symbol are combined and used for conveying

information or broadcasting the information.
Communication Typical communication episode
S (speaker) wants to convey P (proposition) to H (hearer)
using W (words in a formal or natural language)
1. Speaker 2. Hearer
Intention: S wants H to Perception: H perceives
believe P words W” (ideally W” =
Generation: S chooses W)
words W Analysis: H infers possible
Synthesis: S utters words meanings P1,P2,…,Pn for
W W”
Disambiguation: H infers
that S intended to convey
Pi (ideally Pi=P)
Incorporation: H decides
to believe or disbelieve Pi
Introduction to NLP
Natural Language Processing (NLP) is a
tract of Artificial Intelligence and
Linguistics, devoted to make computers
understand the statements or words
written in human languages.

NLP can be defined as the automatic (or

semi-automatic) processing of human
language.

‘Language Technology’ or ‘Language

Engineering’.
Natural Language Processing
 Natural Language Processing (NLP)

1. Natural Language Understanding

● NLU is a subfield of NLP that focuses
specifically on machine understanding
of meaning, intent, context, and
ambiguity in text or speech.
2. Natural Language Generation
● Taking some formal representation of
what you want to say and working out a
way to express it in a natural (human)
language (e.g., English)
Comparative Chart

Feature NLP NLU

Broad (processing of Narrow (understanding
Scope
natural language) language)
Syntax, structure,
Focus Meaning, context, intent
transformation
Tokenization, stemming, Sentiment analysis,
Examples
translation intent detection
Type General umbrella field Subset of NLP
Applications of Nat. Lang. Processing
Machine Translation
Database Access
Information Retrieval
Selecting from a set of documents the ones
that are relevant to a query
Text Categorization
Sorting text into fixed topic categories
Extracting data from text
Converting unstructured text into structure
data
Spoken language control systems
Spelling and grammar checkers
Natural language understanding
Raw speech signal
Speech recognition

Sequence of words spoken

Syntactic analysis using knowledge of the grammar

Structure of the sentence

Semantic analysis using info. about meaning of words

Partial representation of meaning of sentence

Pragmatic analysis using info. about context

Final representation of meaning of sentence

Speech Recognition (1 of 3)

Input Analog Signal Freq. spectrogram

(microphone records voice) (e.g. Fourier transform)

Hz

time
 Natural Language Understanding
Input/Output data Processing stage Other data used

Frequency spectrogram freq. of diff.

speech recognition sounds
Word sequence grammar of
“He loves Mary” syntactic analysis language
Sentence structure meanings of
semantic analysis words
He loves Mary
Partial Meaning context of
Ξx loves(x,mary) pragmatics utterance
Sentence meaning
loves(john,mary)
 A Virtual Personal Assistant

Apple : February 2010

uses voice queries and a natural-language user

interface to answer questions,
make recommendations, and
perform actions by delegating requests
to a set of Internet services.
 Apple : February 2010

Cortana: April 2, 2014

Amazon Alexa: November 2014

Google Assistant: May 18, 2016

 A Virtual Personal Assistant:

• Does Things for You

focus on task completion

• Gets What you Say

intent understanding via conversation

• Gets to Know You

learns and applies personal information
Generic NLP System

(c) 2009 Siri, Inc.

(c) 2009 Siri, Inc.
 Knowledge in Language Processing

• Phonetic & Phonological Knowledge

• Morphological Knowledge
• Syntactic Knowledge
• Semantic Knowledge
• Pragmatic Knowledge
• Discourse Knowledge
• World Knowledge

(c) 2009 Siri, Inc.

 Ambiguity in NLP
Different types of Ambiguities
Lexical Ambiguity: is the ambiguity of a single word.
• A word can be ambiguous with respect to its
syntactic class.
Eg: book, study.
For eg: The word " silver" can be used as a noun, an
adjective, or a verb.
• She bagged two silver medals.
• She made a silver speech.
• His worries had silvered his hair.

(c) 2009 Siri, Inc.

 Ambiguity in NLP
Different types of Ambiguities
Lexical Ambiguity:
• Lexical Ambiguity can be resolve by category
disambiguation. i.e, parts part-of-speech [POS]
tagging.

• Parts part-of-speech tagging is the process,

assigning a part of speech or lexical category
such as noun, verb, pronoun, adverb, adjective
etc. to each word in a sentence.

(c) 2009 Siri, Inc.

 Ambiguity in NLP
Different types of Ambiguities
1. Lexical Semantic Ambiguity:
• occurs when a single word is associated with
multiple senses.
• Words have multiple meanings for such sentences.
• Consider the sentence
Ex. Bank, Pen, Fast, etc.
Ex.
The tank was full of water.
I saw a military tank.
" I saw a bat."
(c) 2009 Siri, Inc.
 Ambiguity in NLP

Different types of Ambiguities

2. Syntactic Ambiguity: structural ambiguities
Two kinds:
• Scope Ambiguity and
• Attachment Ambiguity.

(c) 2009 Siri, Inc.

 Ambiguity in NLP
Scope Ambiguity:
• Scope ambiguity involves operators and quantifiers.
• Example:
• Old men and women were taken to safe locations.
The scope of the adjective (i.e., the amount of text it qualifies) is
ambiguous.
That is, whether the structure (old men and women) or ((old men
and women)?
• The scope of quantifiers is often not clear and creates ambiguity.
Every man loves a woman.
The interpretations can be, For every man there is a woman and
also it can be, there is one particular woman who is loved by every
man.

(c) 2009 Siri, Inc.

 Ambiguity in NLP
Attachment Ambiguity:
• A sentence has attachment ambiguity if a constituent
fits more than one position in a parse tree.

• Attachment ambiguity arises from uncertainty of

attaching a phrase or clause to a part of a sentence.

Consider the example:

• The man saw the girl with the telescope.

Ambiguous: whether the man saw a girl carrying a

telescope, or he saw her through his telescope.
(c) 2009 Siri, Inc.
 Ambiguity in NLP
3. Semantic Ambiguity:
❖ Occurs when meaning of the words themselves can be
misinterpreted.
❖ Happen when sentence contains an ambiguous word or phrase.
❖ Ex. “ The car hit the pole while it was moving”
❖ Interpretation: the car, while moving, hit the pole
❖ Interpretation: The car hit the pole while pole was moving

❖ First interpretation is preferred based on model of world.(i.e.

distinguish what is logical from what is not)

(c) 2009 Siri, Inc.

 Ambiguity in NLP
Discourse Ambiguity:
❖ Discourse level processing needs a shared world or knowledge.
Interpretation is carried out using this context.
❖ Anaphoric ambiguity: entities that have been previously
introduced into the discourse.
❖ Ex. The horse ran up the hill. It was very steep. It soon get tired.
❖ Anaphoric reference of “it” in two situations create ambiguity.
❖ Interpretation: steep applies to surface hence “it” can be hill.
❖ Interpretation: tired applies to animate object hence “it” can be
horse.

(c) 2009 Siri, Inc.

 Ambiguity in NLP
Pragmatic Ambiguity:
❖ Refers to a situation where the context of a phrase gives it multiple
interpretation. (Hardest task in NLP)
❖ Problem Involves: (Highly complex task)
🡪 processing user intention
🡪 sentiment
🡪 belief world etc.
❖ Ex. I love you too.
❖ Interpretation: I love you ( just like you love me)
❖ Interpretation: I love you ( just like someone else does)
❖ Interpretation: I love you ( and I love someone else)

❖ Highly Complex to resolve all these kinds of ambiguities.

(c) 2009 Siri, Inc.
 Statistical Approaches of Ambiguity Resolution in NLP

❖ Probabilistic Model
❖ Part-of-Speech Tagging
🡪 Rule-Based Approaches
🡪 Markov Model Approaches
🡪 Maximum Entropy Approaches
🡪 HMM- Based Taggers
❖ Machine Learning Approaches

(c) 2009 Siri, Inc.

Stages of NLP

(c) 2009 Siri, Inc.

(c) 2009 Siri, Inc.
Some linguistic terminology
Morphology: the structure of words.
For instance, unusually can be thought of as composed of
a prex un-, a stem usual, and an afx -ly.
composed is compose plus the insertional afx -ed: a
spelling rule means we end up with composed rather than
composeed..
Syntax: the way words are used to form phrases. e.g., it
is part of English syntax that a determiner such as the will
come before a noun, and also that determiners are
obligatory with certain singular nouns.
Semantics: Compositional semantics is the construction
of meaning (generally expressed as logic) based on syntax.
Pragmatics: meaning in context.
Classical symbolic methods
Morphological analyzer
Parser (syntactic analysis)
Semantic analysis (transform into a logical form, semantic
network, etc.)
Discourse analysis
Pragmatic analysis
Morphological analysis
Goal: recognize the word and category

Using a dictionary: word + category

Input form (computed)
Morphological rules:
Lemma + ed -> Lemma + e(verb in past form)
…
Is Lemma in dict.? If yes, the transformation is
possible
Form -> a set of possible lemmas
Syntactic Analysis
Rules of syntax (grammar) specify the possible
organization of words in sentences and allows us to
determine sentence’s structure(s)
“John saw Mary with a telescope”
John saw (Mary with a telescope)
John (saw Mary with a telescope)
Parsing: given a sentence and a grammar
Checks that the sentence is correct according with the
grammar and if so returns a parse tree representing the
structure of the sentence

NLP - Prof. Carolina Ruiz

Syntactic Analysis - Grammar
sentence -> noun_phrase, verb_phrase
noun_phrase -> proper_noun
noun_phrase -> determiner, noun
verb_phrase -> verb, noun_phrase
proper_noun -> [mary]
noun -> [apple]
verb -> [ate]
determiner -> [the]

NLP - Prof. Carolina Ruiz

Syntactic Analysis - Parsing
sentence

noun_phrase verb_phrase

proper_noun verb noun_phrase

determiner noun

“Mary” “ate” “the” “apple”

NLP - Prof. Carolina Ruiz

Semantic analysis
john eats an apple. Sem. Cat (Ontology)
proper_noun v det noun object
[person: Mary] λYλX eat(X,Y) [apple]

np animated non-anim
[apple]

np vp person animal food …

[person: Mary] eat(X, [apple])

s vertebral … fruit …
eat([person: Mary], [apple])

apple …
Parsing & semantic analysis
Rules: syntactic rules or semantic rules
What component can be combined with what component?
What is the result of the combination?
Categories
Syntactic categories:Verb, Noun, …
Semantic categories: Person, Fruit, Apple, …
Analyses
Recognize the category of an element
See how different elements can be combined into a
sentence
Problem: The choice is often not unique
Write a semantic analysis grammar
S(pred(obj)) -> NP(obj) VP(pred)
VP(pred(obj)) -> Verb(pred) NP(obj)
NP(obj) -> Name(obj)
Name(John) -> John
Name(Mary) -> Mary
Verb(λyλx Loves(x,y)) -> loves
Discourse analysis
Anaphora
He hits the car with a stone. It bounces back.
Understanding a text
Who/when/where/what … are involved in an event?
How to connect the semantic representations of different
sentences?
What is the cause of an event and what is the consequence of
an action?
…
Pragmatic analysis

Practical usage of language: what a sentence means in

practice
Do you have time?
How do you do?
Uses context of utterance
Where, by who, to whom, why, when it was said
Intentions: inform, request, promise, criticize, …
Handling Pronouns
“Mary eats apples. She likes them.”
She=“Mary”, them=“apples”.
Handling ambiguity
Pragmatic ambiguity: “you’re late”: What’s the speaker’s
intention: informing or criticizing?
Pragmatics
Uses context of utterance
Where, by who, to whom, why, when it was said
Intentions: inform, request, promise, criticize, …
Handling Pronouns
“Mary eats apples. She likes them.”
She=“Mary”, them=“apples”.
Handling ambiguity
Pragmatic ambiguity: “you’re late”: What’s the speaker’s
intention: informing or criticizing?

NLP - Prof. Carolina Ruiz

Tactical generation: converts meaning representations to strings. This may
use the same grammar and lexicon as the parser.
Challenges – ambiguity
Word sense ambiguity
Challenges – ambiguity
Word sense / meaning ambiguity

Credit: http://stuffsirisaid.com
Challenges – ambiguity
PP attachment ambiguity

Credit: Mark Liberman, http://languagelog.ldc.upenn.edu/nll/?p=17711

Challenges – ambiguity
Pronoun reference ambiguity

Credit: http://www.printwand.com/blog/8-catastrophic-examples-of-word-choice-mistakes
Challenges – language is not static
Language grows and changes
e.g., cyber lingo
Challenges--language is compositional

Carefully
Slide
Challenges--language is compositional
小心: 地滑:
Carefully Slide
Careful Landslip
Take Wet Floor
Care Smooth
Caution
Applications of NLP

The following list is not complete, but useful systems have been
built for:

spelling and grammar checking

optical character recognition (OCR)
screen readers for blind and partially sighted users
machine aided translation (i.e., systems which help a
human translator)
lexicographers' tools
Applications of NLP

lexicographers' tools
information retrieval
document classification (filtering, routing)
document clustering
information extraction
question answering
summarization
text segmentation
exam marking
Thank You

Any Questions