Semantics and Pragmatics in NLP:

Requirements for Representation

• In Natural Language Processing (NLP),
semantics and pragmatics focus on
understanding and modeling the meaning
of language.
• Representing these aspects accurately is
crucial for enabling machines to interpret
and generate human language effectively.

1
 Semantics in NLP
• Semantics deals with the literal meaning of words, phrases, and
sentences. It aims to represent:
1. Word meanings (Lexical Semantics):
1. Individual word senses.
2. Relationships between words (synonymy, antonymy, hypernymy,
etc.).
2. Compositional Semantics:
1. Combining word meanings into phrases and sentences based on
syntactic structure.
2. Meaning derived from grammar and word order.
3. Sentence Meaning:
1. Truth conditions: What must be true for a sentence to be true.
2. Logical forms and propositions.

2
 Requirements for Representation in
Semantics
1. Expressiveness:
1. Ability to encode all aspects of meaning (e.g., tense, aspect, modality).
2. Handle ambiguities and polysemy (multiple meanings of a word).
2. Formalism:
1. Logical frameworks like First-Order Logic (FOL), Description Logics (DL),
or semantic graphs (e.g., AMR - Abstract Meaning Representation).
2. Must enable reasoning about meanings and relationships.
3. Compositionality:
1. Representations should follow the Principle of Compositionality, where the
meaning of a whole is derived from its parts.
4. Context-Independence:
1. Focus on literal, dictionary-defined meanings of words and structures.
5. Efficiency:
1. Must support real-time semantic parsing for applications like question-
answering and text summarization.
3
 Pragmatics in NLP
• Pragmatics focuses on contextual meaning, how language is used in real-world
situations, and the speaker's intent. It handles:
1. Disambiguation:
1. Resolving meaning based on situational context.
2. Example: "He is cold" could mean temperature or emotional state.
2. Implicature:
1. Indirect or implied meanings.
2. Example: "Can you pass the salt?" is a request, not a question about ability.
3. Deixis:
1. Words whose meaning depends on context (e.g., "this," "that," "here," "now").
4. Speech Acts:
1. Actions performed via language (e.g., promising, commanding, questioning).
5. Coreference Resolution:
1. Identifying references to the same entity (e.g., resolving "he" to a specific person).
4
 Requirements for Representation in
Pragmatics
1. Context-Awareness:
1. Representation must incorporate context, including:
1. Linguistic context: Previous discourse or sentences.
2. Situational context: The environment or scenario where language is used.
3. Speaker and listener roles: Social and conversational norms.
2. Dynamic Meaning:
1. Handle changes in meaning based on time, location, and dialogue.
3. World Knowledge:
1. Must integrate external knowledge (e.g., common sense, domain-specific facts).
2. Example: Understanding “John kicked the bucket” as an idiom for death, not a literal action.
4. Uncertainty Modeling:
1. Probabilistic frameworks to deal with ambiguities and multiple interpretations.
5. Conversational Structures:
1. Represent dialogue-specific features like turn-taking, politeness, and implicatures.
6. Multi-Modal Integration:
1. In real-world applications, combine linguistic input with non-linguistic cues (e.g., gestures, tone,
visual context).
5
 Requirements for Representation in
Pragmatics
1. Context-Awareness:
1. Representation must incorporate context, including:
1. Linguistic context: Previous discourse or sentences.
2. Situational context: The environment or scenario where language is used.
3. Speaker and listener roles: Social and conversational norms.
2. Dynamic Meaning:
1. Handle changes in meaning based on time, location, and dialogue.
3. World Knowledge:
1. Must integrate external knowledge (e.g., common sense, domain-specific facts).
2. Example: Understanding “John kicked the bucket” as an idiom for death, not a literal action.
4. Uncertainty Modeling:
1. Probabilistic frameworks to deal with ambiguities and multiple interpretations.
5. Conversational Structures:
1. Represent dialogue-specific features like turn-taking, politeness, and implicatures.
6. Multi-Modal Integration:
1. In real-world applications, combine linguistic input with non-linguistic cues (e.g., gestures, tone,
visual context).
6
 Key Challenges in Representing
Semantics and Pragmatics
1. Ambiguity:
1. A single word or sentence may have multiple meanings (e.g., polysemy,
homonymy).
2. Requires contextual understanding to resolve ambiguities.
2. Idioms and Figurative Language:
1. Example: “Break the ice” requires understanding beyond literal word
meanings.
3. Real-World Knowledge:
1. Semantic and pragmatic understanding often requires extensive knowledge
bases (e.g., encyclopedic knowledge, ontologies).
4. Temporal and Spatial Constraints:
1. Representing meanings that change over time or depend on spatial relations.
5. Scalability:
1. Handling the vast variety of meanings, contexts, and domains in real-world
language. 7
 Representation Techniques
• For Semantics
1. First-Order Logic (FOL):
1. Formal system to represent entities, properties, and relationships.
2. Example: ∀x(Human(x)→Mortal(x)).
2. Semantic Networks:
1. Graph-based structures representing relationships between concepts.
2. Example: WordNet.
3. Distributional Semantics:
1. Word embeddings (e.g., Word2Vec, GloVe, BERT).
2. Represent meaning as high-dimensional vectors based on usage context.
4. Abstract Meaning Representation (AMR):
1. Semantic graph representation for sentences.
2. Captures entities, events, and relationships.
8
 Representation Techniques
• For Pragmatics
1. Dialogue Systems:
1. Use task-specific frameworks like Reinforcement Learning or
Transformers for dialogue management.
2. Example: ChatGPT, Siri.
2. Coreference Resolution Models:
1. Use machine learning to link pronouns or anaphora to antecedents.
3. Knowledge Graphs:
1. Integrate context-specific facts into discourse understanding.
4. Probabilistic Approaches:
1. Bayesian networks or Markov models to capture uncertainties in
pragmatic reasoning.
9
 Representation Techniques
1. Question Answering (QA):
1. Extracting precise answers using both semantics (e.g., factual
meaning) and pragmatics (e.g., context).
2. Machine Translation:
1. Handling idiomatic expressions and context-sensitive meanings.
3. Dialogue Systems:
1. Chatbots and virtual assistants that interpret both literal and
implied meanings.
4. Summarization:
1. Capturing key points while maintaining context and intent.
5. Sentiment Analysis:
1. Understanding nuances in opinion or emotion.
10
 First-Order Logic (FOL) in NLP
• First-Order Logic (FOL), also known as Predicate Logic, is a formal
representation system that enables machines to model and reason
about the meaning of natural language.
• It plays a critical role in NLP for tasks requiring structured, logical
reasoning, such as question answering, knowledge representation,
and semantic parsing.

11
 Core Concepts of First-Order Logic

1. Basic Components
FOL builds on propositional logic by adding more expressive
power, including quantifiers, predicates, and objects. Its
components include:
1.Constants:
•Represent specific entities or objects.
•Example: John, Paris.
2.Variables:
•Represent entities or objects generically.
•Example: x,y,z.
3.Predicates:
•Represent properties or relationships between entities.
•Example: Loves(John,Mary): "John loves Mary."
12
 Core Concepts of First-Order Logic
4.Functions:
•Map entities to other entities.
•Example: FatherOf(John)=Jack: "Jack is the father of John."
5.Connectives:
•Logical operators to combine propositions:
4.And (∧)
5.Or (∨)
6.Not (¬)
7.Implies (→)
8.Equivalence (↔)
•Quantifiers:
•Specify the scope of variables:
•Universal Quantifier (∀): "For all."
•∀x(Human(x)→Mortal(x)): "All humans are mortal."
•Existential Quantifier (∃): "There exists."
•∃x(Cat(x)∧Black(x)): "There exists a black cat."
13
 Core Concepts of First-Order Logic
2. Semantics
•The semantics of FOL involves interpreting symbols in a specific domain,
where:
•Domain: The set of all possible entities.
•Interpretation: Assigns meanings to constants, functions, and predicates.

14
 Why Use FOL in NLP?
• FOL provides a systematic framework for representing and reasoning about language. It
is particularly useful for:
1. Semantic Parsing:
1. Converting natural language into structured logical forms.
2. Example: "Every cat sleeps" → ∀x(Cat(x)→Sleeps(x)).
2. Inference:
1. Deduce new facts from given statements using logical rules.
2. Example: If Loves(John,Mary) and Loves(Mary, Pizza), infer Loves(John,Pizza)
under certain contexts.
3. Handling Ambiguity:
• Provides explicit formalism to handle scope ambiguities in language.
• Example: "Every student likes some book" has two interpretations:
• ∀x(Student(x)→∃y(Book(y)∧Likes(x,y))): "Each student likes at least one book."
• ∃y(Book(y)∧∀x(Student(x)→Likes(x,y))): "There is one book liked by all
students."
15
 Why Use FOL in NLP?
4. Knowledge Representation:
1. Encoding facts and relationships in knowledge bases like WordNet or
knowledge graphs.
5. Natural Language Understanding:
1. Applications such as question answering, dialogue systems, and
summarization.

16
 How FOL is Applied in NLP

1. Semantic Representation
•FOL serves as a formalism to represent sentence
meanings.
•Example: "All birds can fly."
• FOL: ∀x(Bird(x)→CanFly(x)).
•Example: "Some birds cannot fly."
• FOL: ∃x(Bird(x)∧¬CanFly(x)).

17
 How FOL is Applied in NLP

2. Logical Inference
•Logical inference is the process of deriving
conclusions based on FOL statements.
•Example:
• Input: ∀x(Human(x)→Mortal(x)) and Human(Socrates).
• Inference: Mortal(Socrates).

18
 How FOL is Applied in NLP

3. Semantic Role Labeling

•Using FOL predicates to represent roles like Agent,
Patient, Instrument, etc.
•Example: "John hit the ball."
• Hit(Agent=John,Patient=Ball).

19
 How FOL is Applied in NLP

4. Question Answering (QA)

•Convert natural language questions into FOL queries and evaluate them
against a knowledge base.
•Example: "Is Socrates mortal?"
• Query: Mortal(Socrates).
• Answer: True (if Mortal(Socrates) is deduced from the knowledge base).
5. Dialogue Systems
•Represent dialogue semantics using FOL for reasoning about conversational
context.
•Example: "Can you bring me a glass of water?"
• FOL: ∃x(Glass(x)∧Water(x)∧Bring(User,x))..

20
 Strengths of FOL in NLP

1.Expressiveness:
1.Captures complex meanings and relationships.
2.Compositionality:
1.Combines meanings of smaller units to form larger units.
3.Inference Capability:
1.Enables reasoning and derivation of new facts.
4.General Applicability:
1.Works for structured data in knowledge bases and
ontology systems.
21
 Challenges of Using FOL in NLP
1. Scalability:
1. Representing and reasoning over large datasets is computationally expensive.
2. Ambiguity and Vagueness:
1. Natural language is inherently ambiguous, but FOL requires precise definitions.
2. Example: "Tall" might require additional predicates like
TallForHuman(x)TallForHuman(x)TallForHuman(x).
3. Limited Expressiveness for Probabilities:
1. FOL is deterministic, lacking mechanisms for handling uncertainty or
probabilistic reasoning.
4. World Knowledge Integration:
1. Requires extensive knowledge bases to handle real-world scenarios.
5. Context and Pragmatics:
1. FOL struggles to represent context-dependent meanings (e.g., deixis,
implicatures).
22
 Example Use Case: Semantic Parsing for
QA

• Input: "Which cities are in France?“

• Parsing:
• Natural Language: "Which cities are in France?“
• FOL Query: ∃x(City(x)∧In(x,France)).
• Reasoning:
• Using knowledge base, infer x=Paris, x=Lyon.
• Output: "Paris, Lyon."

23
 Description Logics (DL) in NLP
• Description Logics (DL) are a family of formal knowledge representation
languages used to represent the structured relationships between concepts, roles,
and individuals.
• They provide a balance between expressiveness and computational efficiency,
making them valuable for tasks in Natural Language Processing (NLP) that
involve semantic representation, reasoning, and ontology-based applications.

24
 Key Features of Description Logics

1. Concepts:
1. Represent sets of entities or classes.
2. Example: Human, Animal, Book.
2. Roles:
1. Represent relationships between entities or concepts.
2. Example: hasAuthor, parentOf.
3. Individuals:
1. Represent specific entities or objects.
2. Example: John, Paris, HarryPotterBook
4. Axioms:
1. Define properties of concepts, roles, and individuals.
2. Example: Human⊑Animal : "Humans are animals."
25
 Key Features of Description Logics

5. TBox (Terminological Box):

•Represents the schema or conceptual knowledge (e.g., class
hierarchies, relationships).
•Example: Mammal⊑Animal.
6. ABox (Assertional Box):
•Represents facts or assertions about specific instances.
•Example: hasAuthor(HarryPotterBook,J.K.Rowling).
7. Reasoning Capabilities:
•Subsumption: Determine if one concept is a subset of another.
•Instance Checking: Determine if an individual belongs to a concept.
•Consistency Checking: Verify if the knowledge base is logically
consistent.
•Classification: Organize concepts into a hierarchy.

26
 Description Logics Syntax and
Semantics
• Syntax
• DL provides constructors to build complex expressions from basic concepts and roles:
• Conjunction (⊓): Intersection of two concepts.
• Example: Human⊓Male: "Male humans."
• Disjunction (⊔): Union of two concepts.
• Example: Dog⊔Cat: "Dogs or cats."
• Negation (¬): Complement of a concept.
• Example: ¬Mammal: "Non-mammals."
• Universal Restriction (∀): Specifies that all related entities satisfy a property.
• Example: ∀hasChild.Human: "All children are humans."
• Existential Restriction (∃): Specifies the existence of at least one related entity satisfying
a property.
• Example: ∃hasChild.Human: "Has at least one human child."
• Role Inclusion: Specify sub-roles.
• Example: parentOf⊑relatedTo. 27
 Description Logics Syntax and
Semantics

• Semantics
• The semantics of DL is based on set theory:
• Concepts: Interpreted as subsets of a domain.
• Roles: Interpreted as binary relations over the domain.
• Axioms: Specify relationships between concepts, roles, and
individuals.

28
 Importance of Description Logics in
NLP
1. Formal Representation of Meaning:
1. Provides a structured way to represent the semantics of natural language using
concepts, roles, and axioms.
2. Reasoning Support:
1. Enables logical inference to derive new information or check consistency
within a knowledge base.
3. Ontology-Based NLP:
1. Ontologies represented in DL (e.g., OWL - Web Ontology Language) provide
a framework for tasks like semantic search and knowledge graph construction.
4. Disambiguation:
1. Helps resolve ambiguities by linking terms to specific concepts in an
ontology.
5. Integration with Knowledge Graphs:
1. Works seamlessly with semantic web technologies and knowledge graphs for
applications like question answering and entity linking.
29
 Applications of Description Logics in
NLP
1. Semantic Search:
•DL enables semantic reasoning over documents by linking terms to ontology concepts.
•Example: Searching for "scientist who developed relativity" could retrieve "Einstein" based on an ontology
defining scientists and their contributions.
2. Question Answering (QA):
•Natural language questions can be converted into DL queries to retrieve answers from a knowledge base.
•Example:
• Question: "Who is the author of Harry Potter?"
• DL Query: ∃hasAuthor.HarryPotterBook.
3. Information Extraction:
•Extract structured information from text and map it to ontological concepts.
•Example: Extracting relationships like marriedTo(PersonA,PersonB) from sentences.
4. Entity Linking:
•Disambiguating mentions in text by linking them to concepts in a knowledge base.
•Example: Resolving "Paris" to either the city or the person in context.
5. Natural Language Understanding (NLU):
•Use DL to represent and infer meanings of sentences in tasks like paraphrasing and summarization.
6. Dialogue Systems:
•Represent user intents and system actions in a structured manner to reason about dialogue state. 30
•Example: Request⊓∃wantsToBuy.Car: "User requests to buy a car."
 Example: Using DL for NLP Tasks
• Ontology Construction
• Define a knowledge base for a domain:
• Concepts: Person, Book, Author .
• Roles: hasWritten, hasTitle.
• Axioms:
• Author⊑Person.
• Book⊓∃hasWritten.Author.
• Query Answering
• Question: "Which authors have written books?"
• DL Query: ∃hasWritten.Book
• Text Understanding
• Sentence: "John wrote a novel."
• DL Representation:
• Author(John)
• hasWritten(John,Novel). 31
 Challenges of Description Logics in
NLP
1. Scalability:
1. Reasoning can be computationally expensive for large knowledge bases.
2. Complexity of Ontology Design:
1. Requires domain expertise to construct accurate and comprehensive
ontologies.
3. Ambiguity in Language:
1. Natural language is inherently ambiguous, which can be challenging to model
in a rigid formalism like DL.
4. Dynamic Knowledge:
1. Difficulty in handling constantly evolving knowledge and concepts.
5. Limited Expressiveness:
1. Some nuances of natural language, like probabilistic or fuzzy meanings, are
hard to capture.
32
Difference

33
Difference

34
Difference

35
FOL vs Other Semantic Representations

36
 Comparison Summary
1. Expressiveness vs. Efficiency:
1. FOL is more expressive, capable of representing a wider range of logical
constructs, but at the cost of undecidability in reasoning.
2. DL is less expressive but focuses on decidability and computational efficiency,
making it suitable for practical applications like ontology reasoning.
2. Use in NLP:
1. FOL is typically used in tasks requiring complex semantic reasoning, such as
inference and logical form parsing.
2. DL is commonly used for ontology-driven tasks like semantic search, entity
linking, and reasoning over knowledge graphs.
3. Reasoning Capabilities:
1. FOL supports more general reasoning but at a high computational cost.
2. DL reasoning is optimized for hierarchical and relational reasoning, ensuring
that queries and checks can be performed effectively.

37
 Example Comparison in NLP
• Representation of "All humans are mortal"
• FOL: ∀x(Human(x)→Mortal(x))
• DL: Human⊑Mortal
• Querying for "Who are humans that are mortal?"
• FOL:
• Use inference rules to derive from the given logical statements.
• DL:
• Use reasoning over the ontology to classify instances under Mortal.

• FOL and DL serve different purposes in NLP and AI. FOL is ideal for tasks
requiring complex and flexible reasoning, while DL is better suited for
structured semantic representation and efficient reasoning in ontology-based
systems.

38
 Syntax-Driven Semantic Analysis
• Syntax-driven semantic analysis is a technique in Natural
Language Processing (NLP) where the meaning of a sentence is
derived based on its syntactic structure (usually a parse tree) and
the semantic rules associated with it.
• This approach ties syntactic constituents (like noun phrases, verb
phrases, etc.) to their semantic counterparts, allowing us to
construct the meaning of the whole sentence compositionally.

39
 Core Concepts of Syntax-Driven
Semantic Analysis
1. Syntactic Parsing:
•The syntactic structure of a sentence is determined using a context-free
grammar (CFG) or similar parsing technique.
•Example: Producing a parse tree for a sentence.
2. Semantic Rules:
•Each syntactic rule in the grammar is associated with a semantic rule.
•These rules specify how the meanings of the subparts (children in the parse
tree) combine to form the meaning of the parent.
3. Compositional Semantics:
•The meaning of the sentence is built by recursively applying the semantic
rules from the leaves of the parse tree to the root.
4. Output Representation:
•The output of semantic analysis can be in forms such as logical forms (e.g.,
predicate logic), lambda calculus expressions, or structured data.

40
 Example: Syntax-Driven Semantic
Analysis
• Sentence:
• "John gives Mary a book."
• Syntactic Parsing:
• Using a CFG, we get the following parse tree:
S -> NP VP
VP -> V NP NP
NP -> ProperNoun | Det Noun

S
/ \
NP VP
/ / \
John V NP NP
gives Mary a book
41
 Example: Syntax-Driven Semantic
Analysis
• Semantic Rules for Composition:
1. S → NP VP:
1. Combine the subject's meaning (NP) with the predicate's meaning (VP).
2. Rule: S.semantic=apply(VP.semantic,NP.semantic)
2. VP → V NP NP:
1. The verb and two objects (direct and indirect) combine to form a predicate.
2. Rule: VP.semantic=λx.λy.λz.V.semantic(x,y,z) where x,y,z, are arguments.
3. NP → ProperNoun:
1. A proper noun refers to an entity.
2. Rule: NP.semantic=EntityReference(ProperNoun)
4. NP → Det Noun:
1. Combine determiner and noun to reference an entity.
2. Rule: NP.semantic=EntityReference(Noun,Det)
5. V → "gives":
42
1. Rule: V.semantic=λgiver.λrecipient.λobject.give(giver,recipient,object)
 Example: Syntax-Driven Semantic
Analysis
• Semantic Construction:
1. NP1=John.semantic=EntityReference(John).
2. NP2=Mary.semantic=EntityReference(Mary).
3. NP3=a book.semantic=EntityReference(book,a).
4. V.semantic=λx.λy.λz.give(x,y,z).
• Step-by-Step Composition:
1. Combine V (gives) with NP3 (a book):
1. λy.λz.give(giver,y,z) where z=book.
2. Combine with NP2​(Mary):
1. λx.give(x,Mary,book)
3. Combine with NP1(John):
1. give(John,Mary,book).
• Final Representation:
• give(John,Mary,book): A predicate indicating that John is the giver, Mary is the
recipient, and the book is the object. 43
 Advantages of Syntax-Driven
Semantic Analysis
1. Compositionality:
1. Builds meaning systematically by leveraging syntax and semantic
rules.
2. Structured Representation:
1. Produces logical forms or semantic graphs that can be directly
used for reasoning or querying.
3. Automation:
1. Semantic rules tied to grammar rules enable automated
construction of sentence meaning.

44
 Applications in NLP

1. Machine Translation:
1.Ensures correct mapping of meaning from one language
to another.
2. Question Answering:
1.Converts questions into logical forms for querying
knowledge bases.
3. Natural Language Understanding:
1.Helps systems interpret user input meaningfully.
4. Dialogue Systems:
1.Enables semantic understanding of user intents. 45
 Challenges

1. Ambiguity:
1. Syntax alone may not resolve semantic ambiguities (e.g., word sense
disambiguation, pronoun resolution).
2. Complex Sentences:
1. Handling nested structures or complex clauses increases difficulty.
3. Domain-Specific Knowledge:
1. Requires integration of ontologies or knowledge bases for domain-specific
interpretation.

Syntax-driven semantic analysis is a powerful method that builds meaning based on

grammatical structure. By combining syntactic parsing with semantic rules, it enables
the extraction of formal representations of meaning, making it crucial for NLP tasks
such as machine translation, question answering, and semantic search. However, it
requires careful handling of ambiguities and complex sentence structures to achieve
46
robust performance.
 Semantic Attachments
• Semantic attachments are mechanisms used in Syntax-Driven
Semantic Analysis to link syntactic structures (produced during
parsing) to their corresponding semantic interpretations.
• They provide a way to associate meaning with syntactic rules,
enabling the construction of sentence-level meaning from smaller
units.
• In essence, semantic attachments specify how the meaning of
individual components (like words or phrases) should be combined
according to a given syntactic structure.
• They are typically defined as functions or expressions attached to
syntactic rules in a grammar.

47
 Key Concepts
1. Syntactic Rules with Semantics:
1. In a grammar, each syntactic production rule is annotated with a semantic
attachment that defines how the semantic meanings of the components are
composed.
Example (in Backus-Naur Form, BNF):
S→NP VP [S.meaning=VP.meaning(NP.meaning)]

2. Compositional Semantics:
•Semantic attachments help achieve compositional semantics, where
the meaning of a complex expression is built by combining the
meanings of its parts.
3. Representations:
•The output of semantic attachments is typically represented in forms
like logical forms (e.g., predicate logic), lambda expressions, or
structured data for further reasoning.
48
 How Semantic Attachments Work
• Step 1: Syntactic Parsing
• A syntactic parser analyzes the input sentence and produces a parse
tree based on a predefined grammar.
• Step 2: Semantic Rules Application
• As the parse tree is constructed, semantic attachments are applied to
nodes in the tree according to their syntactic rules.
• Step 3: Meaning Composition
• The meanings of terminal nodes (words) are combined using the
semantic rules at each non-terminal node, eventually producing a
meaning representation for the entire sentence.

49
 Example: Sentence with Semantic
Attachments
• Sentence:
• "John eats an apple."
• Grammar Rules with Semantic Attachments:
1. S → NP VP:
1. Semantic Attachment: S.meaning=VP.meaning(NP.meaning)
2. VP → V NP:
1. Semantic Attachment: VP.meaning=λx.V.meaning(NP.meaning,x)
3. NP → ProperNoun:
1. Semantic Attachment: NP.meaning=EntityReference(ProperNoun)
4. V → "eats":
1. Semantic Attachment: V.meaning=λx.λy.eat(y,x)
5. NP → Det Noun:
1. Semantic Attachment: NP.meaning=EntityReference(Det,Noun)
50
 Step-by-Step Semantic Analysis
1. Parsing "John":
•Rule: NP → ProperNoun
•Semantic Attachment: NP.meaning=EntityReference("John")
2. Parsing "eats":
•Rule: V → "eats"
•Semantic Attachment: V.meaning=λx.λy.eat(y,x)
3. Parsing "an apple":
•Rule: NP → Det Noun
•Semantic Attachment: NP.meaning=EntityReference("apple","an")
4. Parsing "eats an apple":
•Rule: VP → V NP
•Semantic Attachment: VP.meaning=λx.eat(EntityReference("apple","an"),x)
5. Parsing "John eats an apple":
•Rule: S → NP VP
•Semantic Attachment: S.meaning=VP.meaning(EntityReference("John"))
Final Logical Representation:
•eat(John, apple)\text{eat(John, apple)}eat(John, apple)
•This representation states that the action "eat" has "John" as the subject and "apple" as the 51
object.
 Applications of Semantic Attachments

1. Natural Language Understanding (NLU):

1. Interpreting user inputs for virtual assistants or chatbots.
2. Machine Translation:
1. Converting meanings from one language to another while maintaining
semantic accuracy.
3. Question Answering:
1. Mapping questions to logical queries for knowledge bases.
4. Dialogue Systems:
1. Understanding and responding to user utterances.
5. Semantic Search:
1. Enhancing search engines by associating query terms with semantic
meanings.

52
 Challenges
1. Ambiguity:
1. Multiple possible meanings for a word or phrase can complicate semantic attachment
design.
2. Complex Structures:
1. Handling nested or recursive syntactic structures requires sophisticated semantic
rules.
3. Domain Knowledge:
1. Semantic rules often rely on domain-specific knowledge, making generalization
difficult.
4. Integration with Context:
1. Semantic attachments typically work at the sentence level, requiring additional
mechanisms for handling context across sentences.
Semantic attachments provide a systematic way to derive sentence meaning from
syntax, ensuring compositionality in NLP systems. By linking syntactic rules to their
semantic counterparts, they enable the automated generation of structured semantic
representations, crucial for tasks like machine translation, question answering, and
natural language understanding. 53
 Word Senses
• A word sense refers to a specific meaning or interpretation of a word. Since
many words can have multiple meanings based on the context, these meanings
are categorized into different senses. For example:
• The word "bank" can refer to a financial institution or the side of a river. These
are two distinct senses of the word.

54
 Relations between Senses
• The relations between different senses of a word can be complex and involve
several linguistic concepts, including:
1. Polysemy: A single word having multiple meanings that are related by extension
or metaphor. For example, the word "head" can mean the top of the body, the
leader of a group, or the front of a group. These senses are related through a
general concept of "position or prominence."
2. Homonymy: Two words that share the same form (spelling or pronunciation) but
have different meanings that are not related. For example, "bat" (an implement
used in sports) and "bat" (a flying mammal).
3. Synonymy: Words that have similar meanings. For example, "happy" and
"joyful".
4. Antonymy: Words that have opposite meanings. For example, "hot" and
"cold".
5. Hyponymy: A relationship where the meaning of one word is a more specific
instance of another. For example, "dog" is a hyponym of "animal".
6. Hypernymy: A broader term that encompasses more specific instances. For 55
example, "animal" is a hypernym for "dog".
 Thematic Roles
• Thematic roles (also called theta roles) are linguistic roles that a word (usually
a noun or noun phrase) plays within the context of a sentence. These roles are
part of the meaning of a sentence and describe the relationship between the verb
and its arguments. Common thematic roles include:
1. Agent: The doer of an action. E.g., in "John kicked the ball," John is the
Agent.
2. Theme: The entity that is affected by the action. E.g., in "John kicked the
ball," the ball is the Theme.
3. Experiencer: The entity that experiences a sensation or state. E.g., in "She felt
the cold," "she" is the Experiencer.
4. Goal: The endpoint of an action or motion. E.g., in "She walked to the store,"
"store" is the Goal.
5. Source: The starting point of an action or motion. E.g., in "He came from
Paris," "Paris" is the Source.
6. Instrument: The means by which an action is performed. E.g., in "She cut the
paper with scissors," "scissors" is the Instrument.
7. Recipient: The entity receiving something. E.g., in "He gave her the book," 56
 Selectional Restrictions

• Selectional restrictions are constraints placed by verbs on the types of

arguments they can take. These constraints are based on the verb's meaning and
its associated thematic roles. For instance:
• The verb "eat" typically requires a Theme that is something edible (e.g.,
"apple," "pizza"). You wouldn’t typically say "He ate the car," because "car"
doesn't fit the selectional restriction of something that can be eaten.
• "Give" requires a Recipient (someone to receive) and a Theme (something to
give), so "She gave a book to him" makes sense, while "She gave a book to
the tree" doesn’t, as "tree" doesn't fulfill the role of a Recipient in this context.
• These restrictions help to ensure that sentences make sense by aligning verb-
argument relationships with the expected real-world roles and types of entities.

57
 Word Sense Disambiguation (WSD) in
NLP
• Word Sense Disambiguation (WSD) is the process of identifying the
correct sense or meaning of a word in a given context when the word has
multiple meanings (polysemy).
• It is a crucial task in Natural Language Processing (NLP) to ensure that
systems interpret words accurately based on their usage in a sentence or
document.
Importance of WSD
• Contextual Understanding: Improves the comprehension of ambiguous
words in machine translation, search engines, and chatbots.
• Improved Accuracy: Enhances the performance of NLP applications like
question answering, sentiment analysis, and text summarization.
• Real-world Applications: Helps in domains like information retrieval
and semantic web where precise understanding of text is critical.
58
 Example of Ambiguity in Word Sense

• Consider the word "bank":

• Sense 1: A financial institution (e.g., "I deposited money in the
bank.")
• Sense 2: The side of a river (e.g., "We sat on the river bank.")
• WSD determines which sense is intended by analyzing the
surrounding words and context.

59
 Approaches to WSD

1. Knowledge-Based Approaches
1. Use external lexical resources like dictionaries or ontologies (e.g., WordNet).
2. Methods:
1. Lesk Algorithm: Compares the overlap of context words with dictionary
definitions.
2. Semantic Similarity: Determines the most related sense by measuring
semantic distances in an ontology.
2. Supervised Learning Approaches
1. Train models on annotated datasets where word senses are pre-labeled.
2. Techniques:
1. Naive Bayes, Decision Trees, SVMs: Use features like neighboring
words, part-of-speech, and context windows.
2. Neural Networks: Apply deep learning models for contextual
disambiguation.
60
 Approaches to WSD

3. Unsupervised Learning Approaches

•Do not rely on labeled data; instead, use clustering techniques
to group contexts.
•Techniques:
•Word Clustering: Group similar usages of a word based
on statistical patterns.
•Word Embeddings: Utilize vector representations (e.g.,
Word2Vec, GloVe) to infer word sense from context.
4. Hybrid Approaches
•Combine knowledge-based and data-driven methods to
leverage both structured resources and context-driven insights.

61
 WSD
Challenges in WSD
•Ambiguity in Data: Context might not always provide clear cues for
disambiguation.
•Lack of Annotated Data: Supervised methods require large, labeled datasets
which are hard to create.
•Domain-Specific Variations: Word meanings can change depending on the
application domain.
Applications of WSD
1.Machine Translation: Ensures accurate translation of ambiguous words based on
context.
2.Information Retrieval: Improves search engine relevance by interpreting query
intents.
3.Question Answering Systems: Enhances the precision of answers by
understanding the meaning of key terms.
4.Text-to-Speech Systems: Helps determine pronunciation for words with
62
homographs (e.g., "lead" as a metal vs. "lead" as a verb).
 WSD using Supervised Learning
Methods
Supervised WSD relies on labeled datasets where each occurrence of a word is annotated
with its correct sense. Machine learning models are trained on these datasets to predict the
sense of words in new contexts.
•Key Steps:
1.Data Collection: Gather corpora with words annotated with their senses (e.g., SemCor,
Senseval datasets).
2.Feature Extraction:
1. Context Words: Words surrounding the target word.
2. Part of Speech (POS): POS tags of nearby words.
3. Syntactic Features: Dependency relations or parse trees.
4. Collocations: Frequently co-occurring words.
3.Model Training: Use algorithms like:
1. Naive Bayes
2. Decision Trees
3. Support Vector Machines (SVMs)
4. Deep Learning (e.g., LSTMs, Transformers)
63
4.Prediction: Apply the trained model to determine the word sense in unseen text.
 WSD using Supervised Learning
Methods
Example:
•For the word "bank":
•Input: "I deposited money in the bank."
•Features: Nearby words ("deposited", "money"), POS tags (noun),
syntactic relations.
•Prediction: Financial institution sense.
Advantages:
•High accuracy when annotated data is available.
•Can learn complex patterns from large datasets.
Limitations:
•Requires large labeled corpora, which are costly and time-consuming to
create.
•Struggles with unseen senses or new words. 64
 WSD using Dictionary & Thesaurus-
Based Methods
• These methods rely on external lexical resources like dictionaries, thesauri, or ontologies (e.g.,
WordNet) to disambiguate word senses.
• Key Techniques:
1. Lesk Algorithm:
1. Compares the overlap between the context of the target word and the definitions (glosses) of its
senses in a dictionary.
2. Example:
1. Word: "bank"
2. Context: "river bank"
3. Definitions:
1. Sense 1: "A financial institution."
2. Sense 2: "The land beside a river."
4. Overlap: Words like "river" match the second sense.
2. Semantic Similarity:
1. Measures the relatedness of senses by analyzing their position in a lexical hierarchy (e.g.,
WordNet).
2. Uses similarity metrics (e.g., path similarity, Wu-Palmer similarity).
3. Thesaurus Lookup:
65
1. Matches synonyms or related terms in the thesaurus to the context of the target word.
 WSD using Dictionary & Thesaurus-
Based Methods
Example:
•For the word "plant":
•Input: "The plant grows in the garden."
•Dictionary senses:
• Factory or industrial building.
• Living organism.
•Matching context words ("grows", "garden") disambiguates it to the second sense.
Advantages:
•Doesn't require annotated corpora.
•Utilizes readily available lexical resources.
Limitations:
•Limited to the coverage of the dictionary or thesaurus.
•Often fails to capture subtle contextual cues.
66
 WSD using Bootstrapping Methods
• Bootstrapping is a semi-supervised approach that uses a small amount of
labeled data to iteratively label and learn from large amounts of unlabeled
data.
• Key Process:
1. Seed Data:
1. Start with a small, annotated dataset as the seed.
2. Iterative Learning:
1. Train a basic model on the seed data.
2. Apply the model to label unlabeled examples.
3. Add confidently labeled examples to the training set.
3. Refinement:
1. Repeat the process, gradually expanding the labeled dataset and
improving the model.
67
 WSD using Bootstrapping Methods
Example:
•For the word "java":
1.Seed Data:
1. Labeled sentences: "Java is a programming language."
2.Iteration:
1. Model predicts the sense for new sentences like "I learned Java for
software development."
2. Confident predictions are added to the training set.
3.Refinement:
1. After several iterations, the model achieves better disambiguation.

68
 WSD using Bootstrapping Methods

Advantages:
•Efficient use of limited labeled data.
•Adaptable to domain-specific applications.
Limitations:
•Propagation of errors if initial predictions are incorrect.
•Requires careful tuning to avoid overfitting.

69
 Comparison of Methods

•Supervised WSD provides high accuracy but requires extensive labeled datasets.
•Dictionary & Thesaurus-Based WSD is simpler but relies on the availability and
quality of lexical resources.
•Bootstrapping Methods offer a balanced approach, leveraging small labeled
70
datasets and large unlabeled corpora for scalable WSD.
 Word Similarity and Compositional
Semantics
• Understanding word similarity is a fundamental task in Natural
Language Processing (NLP).
• It enables systems to determine how closely related two words are in
terms of their meaning.
• Two common approaches to calculating word similarity are
Thesaurus-based methods and Distributional methods.
• Additionally, Compositional Semantics extends this understanding
to complex expressions.

71
 Word Similarity Using Thesaurus-Based
Methods
• Thesaurus-based methods rely on lexical resources like WordNet,
Roget's Thesaurus, or similar databases that encode relationships
between words, such as synonyms, antonyms, and hierarchical
structures.
• Key Concepts:
1. Synonyms and Hypernyms:
1. Synonyms: Words with similar meanings (e.g., happy and joyful).
2. Hypernyms: Generalized terms for specific words (e.g., animal is
a hypernym of dog).
2. Semantic Hierarchies:
1. Words are organized in a tree or graph structure (e.g., WordNet).
2. Similarity is computed based on their distance or depth in this
72
structure.
 Word Similarity Using Thesaurus-Based
Methods
• Methods:
1. Path-Based Similarity:
1. Measures the shortest path between two words in a lexical hierarchy.
2. Example:
1. WordNet path similarity: The shorter the path, the more similar the words.
2. Information Content (IC):
1. Computes similarity based on the amount of shared information between
words.
2. Example:
1. Resnik's Similarity: Uses the most specific shared ancestor (common
subsumer) in a hierarchy to calculate similarity.
• Example:
• Words: cat and dog.
• In WordNet, both share a hypernym (mammal).
• Shorter paths or shared ancestors with high IC yield higher similarity scores.
73
 Word Similarity Using Thesaurus-Based
Methods

• Advantages:
• Interpretability: Relies on human-defined relationships.
• Accuracy: High precision for well-documented words.
• Limitations:
• Dependency on lexical resources, which may lack coverage for
domain-specific terms.
• Fixed structure limits adaptability to evolving language use.

74
 Word Similarity Using Distributional
Methods
• Distributional methods are based on the Distributional
Hypothesis, which states that words appearing in similar contexts
tend to have similar meanings.
• Key Concepts:
1. Context Window:
1. Defines a range of words around a target word to capture its
usage patterns.
2. Example:
1.For the word bank, nearby words like money, deposit, and
loan suggest a financial institution.
2. Vector Representations:
1. Words are represented as vectors in a high-dimensional space
based on co-occurrence statistics. 75
 Word Similarity Using Distributional
Methods
• Methods:
1. Count-Based Models:
1. Constructs co-occurrence matrices using raw counts of word-context pairs.
2. Applies dimensionality reduction (e.g., Singular Value Decomposition) to capture
latent semantic relations.
2. Prediction-Based Models:
1. Learns word embeddings by predicting word-context relationships.
2. Examples:
1. Word2Vec (CBOW and Skip-gram models).
2. GloVe (Global Vectors for Word Representation).
3. Cosine Similarity:
1. Measures similarity between word vectors using the cosine of the angle between them.
• Example:
• Words: king and queen.
• Vector operations like king - man + woman ≈ queen demonstrate semantic
relationships. 76
 Word Similarity Using Distributional
Methods

• Advantages:
• Scalable: Handles large corpora effectively.
• Captures nuanced relationships beyond explicit synonyms.
• Limitations:
• Dependency on large, high-quality text corpora.
• Struggles with rare or domain-specific words.

77
 Compositional Semantics
• Compositional Semantics deals with how meanings of individual words combine
to form the meaning of larger expressions, such as phrases, sentences, or even
entire documents.
• Key Principles:
1. Principle of Compositionality:
1. The meaning of a complex expression is determined by the meanings of its
parts and the rules used to combine them.
2. Example:
1. Red car: Combines the meanings of red (color) and car (vehicle).
2. Syntax-Semantics Interface:
1. Syntactic structures guide how meanings are combined.
2. Example:
1. The dog chased the cat ≠ The cat chased the dog.

78
 Compositional Semantics
• Methods:
1. Formal Logic-Based:
1. Uses First-Order Logic to represent meanings and relationships.
2. Example:
1. Sentence: All dogs bark.
1. Logical Form: ∀x (Dog(x) → Bark(x)).
2. Vector-Based Compositionality:
1. Combines word vectors to represent phrases or sentences.
2. Examples:
1. Addition: Vector(red car) = Vector(red) + Vector(car).
2. Multiplication or Tensor-Based Models for more complex combinations.
3. Neural Network Approaches:
1. Deep learning models like Transformers (e.g., BERT, GPT) encode entire
sentences, capturing contextual and compositional meanings.
79
 Compositional Semantics
• Example:
• Sentence: The red apple is on the table.
• Combines individual meanings of red, apple, table, and syntactic relations to form a
coherent interpretation.
• Applications:
1. Machine Translation:
1. Accurately combines word meanings in source and target languages.
2. Question Answering:
1. Parses questions to derive logical forms and find accurate answers.
3. Sentiment Analysis:
1. Analyzes complex phrases like not entirely bad to understand nuanced sentiment.
• Challenges:
• Ambiguity in combining word meanings (e.g., idiomatic expressions like kick the bucket).
• Difficulty in modeling world knowledge or context beyond syntax.

80
 Compositional Semantics

• Thesaurus-Based Methods provide interpretable,

structured insights but require comprehensive lexical
resources.
• Distributional Methods are data-driven, flexible, and
capture contextual nuances effectively.
• Compositional Semantics extends the understanding of
word meanings to larger expressions, enabling applications
like machine translation and sentiment analysis.

81