74.793 NLP and Speech 2004 Feature Structures Feature Structures and Unification.

74.793 NLP and Speech 2004Feature Structures

Feature Structures and Unification

Feature Structures - General

• Feature structures describe linguistic attributes or features like number, person associated with words or syntactic constituents like noun phrase.

• Feature structures are sets of features and values, e.g. hat [Number sing ]

buys [Person 3 ][Number sing ]

Feature Structures - Agreement

Feature structures can be collected in one ‘variable’ called agreement.

buys agreement [Person 3]

[Number sing]

Feature Structures, Grammar, Parsing Feature Structures• describe additional syntactic-semantic information,

like category, person, number, e.g.goes <verb, 3rd, singular>

• specify feature structure constraints (agreements) as part of the grammar rules

• during parsing, check agreements of feature structures (unification)

example

S → NP VP <NP number> = <VP number>

S → NP VP <NP agreement> = <VP agreement>

Feature Structures as ConstraintsUngrammatical sentences like

“He go”

or “We goes”

can be excluded using feature constraints.

example

S → NP VP <NP agreement> = <VP agreement>

S → NP VP <NP number> = <VP number>

<NP person> = <VP person>

Add to feature structure category cat:

buys cat verb

agreement [Person 3 ]

[Number sing]

Feature Structures and Categories

Compare and combine feature structures:

he buys

buys cat verb

agreement [Person 3 ]

[Number sing]

he cat noun

agreement [Person 3 ]

[Number sing]

Feature Structures and Unification 1

S → NP VP <NP number> = <VP number><NP person> = <VP person>

buys cat verb

agreement [Person 3 ]

[Number sing]

he cat noun

agreement [Person 3 ]

[Number sing]

Using Feature Structures

Unification of Feature StructuresAgreement is checked by the unification operation

according to the following rules:

[featurei valuei] |_| [featurei valuei] = [featurei valuei]

[featurei valuei] |_| [featurei valuej] = fail

if valueivaluej

[featurei valuei] |_| [featurei undef.] = [featurei valuei]

[featurei valuei] |_| [featurej valuej] = featurei valuei

featurej valuej

if featurei

featurej

Features and Subcategorization 1

NP modifiers or Verb complements

central noun + modifiers + agreement

central verb + complements + agreements

“... the man who chased the cat out of the house ...”

“... the man chased the barking dog who bit him ...”

Agreements are passed on / inherited within phrases, e.g. agreement of VP derived from Head-Verb of VP:

<VP agreement> determined by <Verb agreement>

<NP agreement> determined by <Nom agreement>

Features and Subcategorization 2

NP modifiers or Verb complements:

central noun + modifiers + agreement central verb + complements + agreements

“... the man who chased the cat out of the house ...”“... the man chased the barking dog who bit him ...”

Agreements are passed on / inherited within phrases, e.g. agreement of VP derived from Head-Verb of VP:

<VP agreement> determined by <Verb agreement><NP agreement> determined by <Nom agreement>

Semantics

Semantics

Distinguish between

• surface structure (syntactic structure) and

• deep structure (semantic structure) of sentences.

Different forms of Semantic Representation

• logic formalisms

• ontology / semantic representation languages – Case Frame Structures (Filmore)– Conceptual Dependy Theory (Schank)– DL and similar KR languages – Ontologies

Semantic Representations

Semantic Representations based on some form of (formal) Representation Language.

– Semantics Networks– Conceptual Dependency Graphs– Case Frames– Ontologies– DL and similar KR languages

Constructing a Semantic Representation

General: Start with surface structure Derived from parser. Map surface structure to semantic structure

Use phrases as sub-structures. Find concepts and representations for

central phrases (e.g. VP, NP, then PP) Assign phrases to appropriate roles

around central concepts (e.g. bind PP into VP representation).

Ontology (Interlingua) approach

• Ontology: a language-independent classification of objects, events, relations

• A Semantic Lexicon, which connects lexical items to nodes (concepts) in the ontology

• An analyzer that constructs Interlingua representations and selects (an?) appropriate one

(based on Steve Helmreich's 419 Class, Nov 2003)

Semantic Lexicon

• Provides a syntactic context for the appearance of the lexical item

• Provides a mapping for the lexical item to a node in the ontology (or more complex associations)

• Provides connections from the syntactic context to semantic roles and constraints on these roles

Deriving Basic Semantic Dependency (a toy example)

Input: John makes tools

Syntactic Analysis:cat verbtense presentsubject  

root johncat noun-proper

object  root     toolcat nounnumber plural

Deriving Basic Semantic Dependency

John-n1syn-struc

root johncat noun-proper

sem-struchuman

name john

gender maletool-n1

syn-strucroot toolcat n

sem-structool

Lexicon Entries for John and tool

Relevant Extract from the Specification of the Ontological Concept Used to Describe the Appropriate Meaning of make:

manufacturing-activity...

agent humantheme artifact

…

Meaning Representation - Example make

John-n1syn-struc

root johncat noun-proper

sem-struchuman

name johngender male

tool-n1syn-struc

root toolcat n

sem-structool

Relevant parts of the (appropriate senses of the)lexicon entries for John and tool

The basic semantic dependency component of the TMR for

John makes tools

manufacturing-activity-7

agent uman-3theme set-1

element toolcardinality > 1

…

Semantic Dependency Component

try-v3syn-struc

root trycat vsubj root $var1

cat nxcomp root $var2

cat vform OR infinitive gerund

sem-strucset-1 element-type refsem-1

cardinality >=1refsem-1 sem event

agent ^$var1effect refsem-2

modalitymodality-type epiteucticmodality-scope refsem-2modality-value < 1

refsem-2 value ^$var2sem event

Constructing an IL representation

For each syntactic analysis: Access all semantic mappings and contexts

for each lexical item. Create all possible semantic representations. Test them for coherency of structure and

content.

REQUEST-INFO-130 THEME DEVELOP-2601.PURPOSE DEVELOP-2601.REASON TEXT-POINTER why INSTANCE-OF REQUEST-INFO

DEVELOP-2601THEME SET-2555AGENT NATION-97PHASE CONTINUOUS

TIME FIND-ANCHOR-TIME INSTANCE-OF DEVELOP

TEXT-POINTER developing

NATION-97HAS-NAME Iraq

INSTANCE-OF NATIONTEXT-POINTER Iraq

SET-2555 ELEMENT-TYPE WEAPONCARDINALITY > 1

INSTRUMENT-OF KILL-1864 THEME-OF DEVELOP-2601 INSTANCE-OF WEAPON

TEXT-POINTER weapons

KILL-1864 THEME SET-2556 INSTRUMENT SET-2555 INSTANCE-OF KILL

TEXT-POINTER destruction

SET-2556 THEME-OF KILL-1225 ELEMENT-TYPE HUMAN

CARDINALITY > 100 INSTANCE-OF HUMAN

TEXT-POINTER mass

“Why is Iraq developing weapons of mass destruction?”

Word sense disambiguation Constraint checking – making sure the

constraints imposed on context are met Graph traversal – is-a links are

inexpensive Other links are more expensive The “cheapest” structure is the most

coherent Hunter-gatherer processing

Logic Formalisms

Lambda Calculus

Semantics - Lambda Calculus 1

Logic representations often involve Lambda-Calculus:• represent central phrases (e.g. verb) as -

expressions -expression is like a function which can be applied

to terms• insert semantic representation of complement or

modifier phrases etc. in place of variables

x, y: loves (x, y) FOPL sentence

xy loves (x, y) -expression, function

xy loves (x, y) (John) y loves (John, y)

Semantics - Lambda Calculus 2

Transform sentence into lambda-expression:

“AI Caramba is close to ICSI.”

specific: close-to (AI Caramba, ICSI)

general: x,y: close-to (x, y) x=AI Caramba y=ICSI

Lambda Conversion:

-expr: xy: close-to (x, y) (AI Caramba)

Lambda Reduction:

y: close-to (AI Caramba, y)

close-to (AI Caramba, ICSI)

Semantics - Lambda Calculus 3

Lambda Expressions can be constructed from central expression, inserting semantic representations for complement phrases

Verb serves

{xy e IS-A(e, Serving) Server(e,y) Served(e,x)}

represents general semantics for the verb 'serve

Fill in appropriate expressions for x, y, for example 'meat' for y derived from Noun in NP as complement to Verb.

References

Jurafsky, D. & J. H. Martin, Speech and Language Processing, Prentice-Hall, 2000. (Chapters 9 and 10)

Helmreich, S., From Syntax to Semantics, Presentation in the 74.419 Course, November 2003.