CSE391 – 2005 NLP 1 Events From KRR lecture. CSE391 – 2005 NLP 2 Ask Jeeves – A Q/A, IR ex. What do you call a successful movie? Tips on Being a Successful.

CSE391 – 2005 NLP1

Events

• From KRR lecture

CSE391 – 2005 NLP2

Ask Jeeves – A Q/A, IR ex.

What do you call a successful movie?• Tips on Being a Successful Movie Vampire ... I shall

call the police.• Successful Casting Call & Shoot for ``Clash of

Empires'' ... thank everyone for their participation in the making of yesterday's movie.

• Demme's casting is also highly entertaining, although I wouldn't go so far as to call it successful. This movie's resemblance to its predecessor is pretty vague...

• VHS Movies: Successful Cold Call Selling: Over 100 New Ideas, Scripts, and Examples from the Nation's Foremost Sales Trainer.

Blockbuster

CSE391 – 2005 NLP3

Ask Jeeves – filtering w/ POS tag

What do you call a successful movie?• Tips on Being a Successful Movie Vampire ... I shall

call the police.• Successful Casting Call & Shoot for ``Clash of

Empires'' ... thank everyone for their participation in the making of yesterday's movie.

• Demme's casting is also highly entertaining, although I wouldn't go so far as to call it successful. This movie's resemblance to its predecessor is pretty vague...

• VHS Movies: Successful Cold Call Selling: Over 100 New Ideas, Scripts, and Examples from the Nation's Foremost Sales Trainer.

CSE391 – 2005 NLP4

Filtering out “call the police”

call(you,movie,what) ≠ call(you,police)

Different senses, - different syntax, - different participants

you movie what you police

CSE391 – 2005 NLP5

“Meaning”

• Shallow semantic annotation– Captures critical dependencies, – Highlights participants– Reflects clear sense distinctions

• Supports training of supervised automatic taggers

• Works for other languages

CSE391 – 2005 NLP6

Cornerstone: English lexical resource

• That provides sets of possible syntactic frames for verbs.

• And provides clear, replicable sense distinctions.

AskJeeves: Who do you call for a good electronic lexical database for English?

CSE391 – 2005 NLP7

WordNet – Princeton (Miller 1985, Fellbaum 1998)

• On-line lexical reference (dictionary)– Nouns, verbs, adjectives, and adverbs

grouped into synonym sets

– Other relations include hypernyms (ISA), antonyms, meronyms

CSE391 – 2005 NLP8

WordNet

• http://www.cogsci.princeton.edu/~wn/

CSE391 – 2005 NLP9

WordNet – call, 28 senses1. name, call -- (assign a specified, proper name to; "They named their son David"; …) -> LABEL2. call, telephone, call up, phone, ring -- (get or try to get

into communication (with someone) by telephone; "I tried to call you all night"; …)

->TELECOMMUNICATE3. call -- (ascribe a quality to or give a name of a common

noun that reflects a quality; "He called me a bastard"; …)

-> LABEL4. call, send for -- (order, request, or command to come;

She was called into the director's office"; "Call the police!")

-> ORDER

CSE391 – 2005 NLP10

WordNet – Princeton (Miller 1985, Fellbaum 1998)

• Limitations as a computational lexicon– Contains little syntactic information

• Comlex has syntax but no sense distinctions

– No explicit lists of participants– Sense distinctions very fine-grained, – Definitions often vague

• Causes problems with creating training data for supervised Machine Learning – SENSEVAL2

• Verbs > 16 senses (including call)• Inter-annotator Agreement ITA 73%, • Automatic Word Sense Disambiguation, WSD 60.2%

Dang & Palmer, SIGLEX02

CSE391 – 2005 NLP11

WordNet: - call, 28 senses

WN2 , WN13,WN28 WN15 WN26

WN3 WN19 WN4 WN 7 WN8 WN9

WN1 WN22

WN20 WN25

WN18 WN27

WN5 WN 16 WN6 WN23

WN12

WN17 , WN 11 WN10, WN14, WN21, WN24

CSE391 – 2005 NLP12

WordNet: - call, 28 senses, Senseval2 groups (engineering!)

WN5, WN16,WN12 WN15 WN26

WN3 WN19 WN4 WN 7 WN8 WN9

WN1 WN22

WN20 WN25

WN18 WN27

WN2 WN 13 WN6 WN23

WN28

WN17 , WN 11 WN10, WN14, WN21, WN24,

Loud cry

Label

Phone/radio

Bird or animal cry

Request

Call a loan/bond

Visit

Challenge

Bid

CSE391 – 2005 NLP13

Grouping improved scores: ITA 82%, MaxEnt WSD 69%

• Call: 31% of errors due to confusion between senses within same group 1:– name, call -- (assign a specified, proper name to; They

named their son David)– call -- (ascribe a quality to or give a name of a common

noun that reflects a quality; He called me a bastard)– call -- (consider or regard as being;I would not call her

beautiful)

– 75% with training and testing on grouped senses vs.– 43% with training and testing on fine-grained senses

Palmer, Dang, Fellbaum,, submitted, NLE

CSE391 – 2005 NLP14

Proposition Bank:From Sentences to Propositions

(Predicates!)

Powell met Zhu Rongji

Proposition: meet(Powell, Zhu Rongji)Powell met with Zhu Rongji

Powell and Zhu Rongji met

Powell and Zhu Rongji had a meeting

. . .When Powell met Zhu Rongji on Thursday they discussed the return of the spy plane.

meet(Powell, Zhu) discuss([Powell, Zhu], return(X, plane))

debate

consult

joinwrestle

battle

meet(Somebody1, Somebody2)

CSE391 – 2005 NLP15

Capturing semantic roles*

• Richard broke [ ARG1 the laser pointer.]

• [ARG1 The windows] were broken by the hurricane.

• [ARG1 The vase] broke into pieces when it toppled over.

SUBJ

SUBJ

SUBJ

*See also Framenet, http://www.icsi.berkeley.edu/~framenet/

CSE391 – 2005 NLP16

Word Senses in PropBank• Orders to ignore word sense not feasible for 700+ verbs

– Mary left the room– Mary left her daughter-in-law her pearls in her will

Frameset leave.01 "move away from":Arg0: entity leavingArg1: place left

Frameset leave.02 "give":Arg0: giver Arg1: thing givenArg2: beneficiary

How do these relate to traditional word senses in VerbNet and WordNet?

CSE391 – 2005 NLP17

WordNet: - call, 28 senses, groups

WN5, WN16,WN12 WN15 WN26

WN3 WN19 WN4 WN 7 WN8 WN9

WN1 WN22

WN20 WN25

WN18 WN27

WN2 WN 13 WN6 WN23

WN28

WN17 , WN 11 WN10, WN14, WN21, WN24,

Loud cry

Label

Phone/radio

Bird or animal cry

Request

Call a loan/bond

Visit

Challenge

Bid

CSE391 – 2005 NLP18

Overlap with PropBank Framesets

WN5, WN16,WN12 WN15 WN26

WN3 WN19 WN4 WN 7 WN8 WN9

WN1 WN22

WN20 WN25

WN18 WN27

WN2 WN 13 WN6 WN23

WN28

WN17 , WN 11 WN10, WN14, WN21, WN24,

Loud cry

Label

Phone/radio

Bird or animal cry

Request

Call a loan/bond

Visit

Challenge

Bid

CSE391 – 2005 NLP19

Overlap between Senseval2Groups and Framesets – 95%

WN1 WN2 WN3 WN4

WN6 WN7 WN8 WN5 WN 9 WN10

WN11 WN12 WN13 WN 14

WN19 WN20

Frameset1

Frameset2

developPalmer, Babko-Malaya,Dang SNLU 2004

CSE391 – 2005 NLP20

Sense Hierarchy • PropBank Framesets – coarse grained distinctions

20 Senseval 2 verbs w/ > 1 FramesetMaxent WSD system, 73.5% baseline, 90% accuracy

– Sense Groups (Senseval-2) intermediate level (includes Levin classes) – 69%

• WordNet – fine grained distinctions, 60.2%

CSE391 – 2005 NLP21

Maximum Entropy WSDHoa Dang, best performer on Verbs

• Maximum entropy framework, p(sense|context)

• Contextual Linguistic Features– Topical feature for W: +2.5%,

• keywords (determined automatically)

– Local syntactic features for W: +1.5 to +5%, • presence of subject, complements, passive?• words in subject, complement positions, particles, preps,

etc.

– Local semantic features for W: +6% • Semantic class info from WordNet (synsets, etc.)• Named Entity tag (PERSON, LOCATION,..) for proper

Ns• words within +/- 2 word window

CSE391 – 2005 NLP22

Context Sensitivity

• Programming languages are Context Free

• Natural languages are Context Sensitive?– Movement– Features– respectivelyJohn, Mary and Bill ate peaches, pears and apples,

respectively.

CSE391 – 2005 NLP23

The Chomsky Grammar Hierarchy

• Regular grammars, aabbbb S → aS | nil | bS

• Context free grammars, aaabbb S → aSb | nil

• Context sensitive grammars, aaabbbccc xSy → xby

• Turing Machines

CSE391 – 2005 NLP24

Recursive transition nets for CFGs

• s :- np,vp.• np:- pronoun; noun; det,adj, noun; np,pp.

S1 S2 S3

NP VP

S

S4 S5 S6

det noun

NPnoun

pronoun

pp

adj

CSE391 – 2005 NLP25

Most parsers are Turing Machines• To give a more natural and

comprehensible treatment of movement

• For a more efficient treatment of features

• Not because of respectively – most parsers can’t handle it.

CSE391 – 2005 NLP26

Nested Dependencies and Crossing Dependencies

John, Mary and Bill ate peaches, pears and apples, respectively

The dog chased the cat that bit the mouse that ran.

The mouse the cat the dog chased bit ran.

CF

CF

CS

CSE391 – 2005 NLP27

Movement

What did John give to Mary?

*Where did John give to Mary?

John gave cookies to Mary.

John gave <what> to Mary.

CSE391 – 2005 NLP28

Handling Movement:Hold registers/Slash Categories

• S :- Wh, S/NP

• S/NP :- VP

• S/NP :- NP VP/NP

• VP/NP :- Verb