Computational Semantics Torbjörn Lager Department of Linguistics Uppsala University
Jan 02, 2016
NLP1 - Torbjörn Lager 2
Computational Semantics
Compositional, logical semantics* Computational lexical semantics Word sense disambiguation* Text categorization Information extraction Information retrieval
NLP1 - Torbjörn Lager 3
Logical Semantics Example
John laughed laughed'(j)
Nobody laughed x[laughed'(x)]
But this is just translation! What's semantic about that?
NLP1 - Torbjörn Lager 4
What Is the Name of This Business?
Truth conditional semantics Model theoretic semantics Logical semantics Formal semantics Compositional semantics Syntax-driven semantic analysis
Compositional, logical, truth conditional, model theoretic semantics ....
NLP1 - Torbjörn Lager 5
An Important Tradition
We use language to talk about the world
Semantics is something that relates sentences (or utterances) of language and the outside world
There are other ideas about meaning, but in this tradition we don't believe in them!
Natural language
The outside world
NLP1 - Torbjörn Lager 6
Truth Conditional Semantics
Meaning = Truth conditions Examples:
"John whistles" is true iff John whistles
"John visslar" is true iff John whistles
"Ogul fautu seq" is true iff...
Natural language
The outside world
NLP1 - Torbjörn Lager 7
Model Theoretic Semantics
We don't know what the world is really like, so let's talk about a model of the world instead
Such a model does (usually) consists of individuals, sets of individuals, functions and relations. i.e the sort of things set theory talks about
Truth becomes truth relative to a model
Natural language
Model
The outside world
NLP1 - Torbjörn Lager 8
Compositional Semantics
The Compositionality Principle: The meaning of the whole is a
function of the meaning of the parts and the mode of combining them.
The meaning of a complex expression is uniquely determined by the meaning of its constituents and the syntactic construction used to combine them.
Natural language
Model
The World
NLP1 - Torbjörn Lager 9
Truth Conditional, Model Theoretic and Compositional Semantics Combined
A simple model M: Domain:
{John, Mary, Paul} Interpretation:
Names: "John" refers to John , "Mary" refers to Mary, etc.
Verbs: "whistles" refers to {John, Paul}
Example "John whistles" is true in
M iff the individual in M referred to as "John" is an element in the set of individuals that "whistles" refer to.
Richard Montague (1970): "I reject the contention that an important theoretical difference exists between formal and natural languages"
NLP1 - Torbjörn Lager 10
Translational Semantics
Account for the meanings of natural language utterances by translating them into another language.
It could be any language, but only if this language has a formal semantics are we done.
Natural language
Logical Form Language
Model
The World
NLP1 - Torbjörn Lager 11
Why Logical Semantics?
Account for ambiguity "every man loves a woman"
Allow evaluation e.g. by database lookup
Allow logical inference Every man who whistles is happy John is a man John whistles Therefore: John is happy
NLP1 - Torbjörn Lager 12
Applications of Logical Semantics
NLU systems Semantics + 'World knowledge' --> 'understanding'
Information Extraction Machine translation
LF as (part of an) interlingua
Dialogue Systems
NLP1 - Torbjörn Lager 13
Grammar and Logical Form
S -> NP VP[S] = [VP]([NP])
NP -> john[NP] = j
VP -> whistles[VP] = x[whistles'(x)]
[john whistles] = whistles'(j)
cf. The principle ofcompositionality
Have same truth conditions
NLP1 - Torbjörn Lager 14
Beta Reduction (Lambda Conversion)
[S] = [VP]([NP]) [NP] = j [VP] = x[whistles'(x)]
Beta reduction rule: u() where every occurrence of u in is replaced by
x[whistles'(x)](j) application whistles'(j) reduction
NLP1 - Torbjörn Lager 15
Grammar and Logical Form
S -> NP VP[S] = [NP]([VP])
NP -> john[NP] = P[P(j)]
VP -> whistles[VP] = x[whistles'(x)]
[john whistles] = whistles'(j)
NLP1 - Torbjörn Lager 16
From Logical Form to Truth Conditions
whistles'(j) is true iffthe individual (in the model) denoted by 'j' has the property denoted by 'whistles'
cf. "John whistles" is true iff John whistles
NLP1 - Torbjörn Lager 17
Beta Reduction (Lambda Conversion)
[S] = [NP]([VP]) [NP] = P[P(j)] [VP] = x[whistles'(x)]
Beta reduction rule: u() where every occurrence of u in is replaced by
P[P(j)](x[whistles'(x)]) application x[whistles'(x)](j) reduction whistles'(j) reduction
NLP1 - Torbjörn Lager 18
A Larger Example
S -> NP VP [S] = [NP]([VP])
NP -> DET N [NP] = [DET]([N])
DET -> every[DET] = Q[P[z[Q(z) P(z)]]]
N -> man [N] = x[man'(x)]
VP -> whistles[VP] = x[whistles'(x)]
[every man whistles} = z[man'(z) whistles'(z)]
NLP1 - Torbjörn Lager 19
A Larger Example (cont'd)
[S] = [NP]([VP]) [NP] = [DET]([N]) [DET] = Q[P[z[Q(z) P(z)]]] [N] = x[man'(x)] [VP] = x[whistles'(x)]
Q[P[z[Q(z) P(z)]]](x[man'(x)]) application P[z[x[man'(x)](z) P(z)]] reduction P[z[man'(z) P(z)]] reduction P[z[man'(z) P(z)]](x[whistles'(x)]) application z[man'(z) x[whistles'(x)](z)] reduction z[man'(z) whistles'(z)] reduction
NLP1 - Torbjörn Lager 20
Alternative Ontologies
[john whistles] =
ex[isa(e,WhistleEvent) agent(e,x) named(x,"john")]
[john whistles] = whistles'(j)
NLP1 - Torbjörn Lager 21
Alternative Meaning Representation Formalisms
Discourse Representation Theory
[every man whistles} =
man(x) whistles(x)
x
NLP1 - Torbjörn Lager 22
Semantics Research
How to design a nice meaning representation language? Ontology? How to treat a particular NL construct?
in a compositional way? and end up with correct truth conditions? and still be elegant and clean?
How to deal with things like time and events, propositional attitude reports, reference to non-existent individuals, etc.?
How to solve a particular semantic puzzle? How to design a nice syntax-semantics interface? How to design a nice semantics-pragmatics interface? What is the role of inference in semantic processing? How to account for things like presuppositions?
NLP1 - Torbjörn Lager 23
Semantic Puzzles
Why is "Every man loves a woman" ambiguous, but not "Every man loves Mary"?
What's wrong with the following argument: "Nothing is better than a long and prosperous life. A ham sandwich is better than nothing. Therefore, a ham sandwich is better than a long and prosperous life."
The morning star = the evening star, still"John believes that Venus is the morning star" may be true, and at the same time, "John believes that Venus is the evening star", may be false.
Everything written on this slide is false.
NLP1 - Torbjörn Lager 25
Example: Senses of "interest"
From the LDOCE
1. readiness to give attention, 2. quality of causing attention to be given3. activity, subject, etc., which one gives time and
attention to4. advantage, advancement, or favour5. a share (in a company, business, etc.)6. money paid for the use of money
NLP1 - Torbjörn Lager 26
WSD Examples
At the same time, the drop in interest rates since the spring has failed to revive the residential construction industry.
Cray Research will retain a 10% interest in the new company, which will be based in Colorado Springs.
Although that may sound like an arcane maneuver of little interest outside Washington, it would set off a political earthquake.
NLP1 - Torbjörn Lager 27
Why Word Sense Disambiguation?
Well, compositional logical semantics doesn't deal with word meaning, so...
Machine translation A non-disambiguated Russian translation of: “The spirit is
willing but the flesh is weak” gave “The vodka is good but the meat is rotten”
Information Retrieval When searching the web for info about companies buying
shares in other companies, you don’t want to retrieve the information about interest rates.
Provide clues to pronounciation "banan" -> BAnan or baNAN
NLP1 - Torbjörn Lager 28
Approaches to WSD
Deep (but brittle) WSD 'Selectional restriction'-based approaches Approaches based on general reasoning with 'world
knowledge'
Shallow and robust WSD Machine learning approaches
Supervised learning Unsupervised learning Bootstrapping approaches
Dictionary-based approaches Various combinations of methods
NLP1 - Torbjörn Lager 29
Machine Learning Approaches
Training data in the form of annotated corpora Decide on features on which to condition Preprocessing Steps
Context trimming Stemming/Lemmatizing Part-of-speech tagging Partial Parsing
Use a machine learning algorithm Enter the training-test cycle
NLP1 - Torbjörn Lager 30
Dictionary-Based Approaches
Lesk (1986)
Find the dictionary definition that overlaps most with the definitions for the words in the ambiguous word’s context.
Problem 1: A lot of computation. Problem 2: Definitions are usually too short
NLP1 - Torbjörn Lager 31
Lesk Example
Lexicon entries:
pine 1 kinds of evergreen tree with needle-shaped leaves
2 waste away through sorrow or illness
cone 1 solid body which narrows to a point
2 something of this shape whether solid or hollow
3 fruit of certain evergreen tree
Example: .... pine cone ...
The third sense for "cone" is selected here, since two of the (content) words in its entry, "evergreen" and "tree", overlap with words in the entry for "pine", and this doesn't happen for the other senses.
NLP1 - Torbjörn Lager 32
SENSEVAL-2: Second International Workshop on Evaluating Word Sense Disambiguation Systems
The purpose of SENSEVAL is to evaluate the strengths and weaknesses of WSD programs with respect to different words, different varieties of language, and different languages.
Two types of task over 12 languages. In the "all-words" task, the evaluation is on almost all of the content
words in a sample of texts. In the "lexical sample" task, first we sample the lexicon, then we find
instances in context of the sample words and the evaluation is on those instances only.
All-words: Czech, Dutch, English, Estonian Lexical sample: Basque, Chinese, Danish, English, Italian, Japanese,
Korean, Spanish, Swedish About 35 teams participated, submitting over 90 systems.
NLP1 - Torbjörn Lager 33
The Swedish Lexical Sample Task
40 lemmas: 20 nouns, 15 verbs and 5 adjectives. Together representing 145 senses and 304 sub-senses. 8,718 annotated instances were provided as training
material 1,527 unannotated instances were provided for testing. A lexicon - the GLDB (Gothenburg Lexical Database) -
complete with morphological information, definitions, language examples, etc., was also available.