CS4705 Part of Speech tagging - Columbia Universitykathy/NLP/2019/ClassSlides/Class8-POS/Cla… · CS4705 Part of Speech tagging 9/30/19 Some slides adapted from: Dan Jurafsky, Julia

CS4705PartofSpeechtagging

9/30

/19

1 Some slides adapted from: Dan Jurafsky, Julia Hirschberg, Jim Martin

Announcements• Readingfortoday:• Readingfornext0me:• Weareturningtosyntax.Today:part-of-speechtagging

• Homework2isout

2

Gardenpathsentences• Theolddogthefootstepsoftheyoung.

• Thehorseracedpastthebarnfell.

• Theco5onclothingismadeofgrowsinMississippi.

3

GardenpathsentencesN• Theolddog|thefootstepsoftheyoung.

• Thehorseracedpastthebarnfell.

• Theco5onclothingismadeofgrowsinMississippi. 4

GardenpathsentencesV• Theolddog|thefootstepsoftheyoung.

• Thehorseracedpastthebarnfell.

• Theco5onclothingismadeofgrowsinMississippi. 5

Gardenpathsentences• Theolddogthefootstepsoftheyoung.

VBD• Thehorseracedpastthebarn|fell.

• Theco5onclothingismadeofgrowsinMississippi. 6

Gardenpathsentences• Theolddogthefootstepsoftheyoung.

VBNVBD• Thehorseracedpastthebarn|fell.

• Theco5onclothingismadeofgrowsinMississippi. 7

Gardenpathsentences• Theolddogthefootstepsoftheyoung.

• Thehorseracedpastthebarnfell.

• Theco5onclothingismadeofgrowsinMississippi.

8

9

10

Whatisawordclass?• Wordsthatsomehow‘behave’alike:

• Appearinsimilarcontexts• Performsimilarfunc0onsinsentences• Undergosimilartransforma0ons

• 9(orso)tradi0onalpartsofspeech• Noun,verb,adjec0ve,preposi0on,adverb,ar0cle,interjec0on,pronoun,conjunc0on,

11

POSexamples• N nounchair,bandwidth,pacing• V verb study,debate,munch• ADJadjec0ve purple,tall,ridiculous• ADVadverb unfortunately,slowly,• P preposi0onof,by,to• PROpronoun I,me,mine• DETdeterminer the,a,that,those

12

POSTagging:DeDinition• Theprocessofassigningapart-of-speechorlexicalclassmarkertoeachwordinacorpus:

13

the koala put the keys on the table

WORDS TAGS

N V P DET

WhatisPOStagginggoodfor?•  IsthefirststepofavastnumberofCompLingtasks•  Speechsynthesis:

•  Howtopronounce“lead”?•  INsult inSULT•  OBject obJECT•  OVERflow overFLOW•  DIScount disCOUNT•  CONtent conTENT

•  Parsing•  NeedtoknowifawordisanNorVbeforeyoucanparse

• Wordpredic0oninspeechrecogni0on•  Possessivepronouns(my,your,her)followedbynouns•  Personalpronouns(I,you,he)likelytobefollowedbyverbs

•  MachineTransla0on

14

15

16

Openandclosedclasswords• Closedclass:arela0velyfixedmembership

•  Preposi0ons:of,in,by,…•  Auxiliaries:may,can,willhad,been,…•  Pronouns:I,you,she,mine,his,them,…•  Usuallyfunc0onwords(shortcommonwordswhichplayaroleingrammar)

• Openclass:newonescanbecreatedallthe0me•  Englishhas4:Nouns,Verbs,Adjec0ves,Adverbs•  Manylanguageshaveall4,butnotall!•  InLakhotaandpossiblyChinese,whatEnglishtreatsasadjec0vesactmorelikeverbs. 17

Openclasswords•  Nouns

•  Propernouns(ColumbiaUniversity,NewYorkCity,ElsbethTurcan,MetropolitanTransitCenter).Englishcapitalizesthese.

•  Commonnouns(therest).Germancapitalizesthese.•  Countnounsandmassnouns

•  Count:haveplurals,getcounted:goat/goats,onegoat,twogoats•  Mass:don’tgetcounted(fish,salt,communism)(*twofishes)

•  Adverbs:tendtomodifyac0onsorpredicates•  Unfortunately,Johnwalkedhomeextremelyslowlyyesterday•  Direc0onal/loca0veadverbs(here,home,downhill)•  Degreeadverbs(extremely,very,somewhat)•  Manneradverbs(slowly,slinkily,delicately)

•  Verbs:•  InEnglish,havemorphologicalaffixes(eat/eats/eaten)•  Ac0ons(walk,ate)andstates(be,exude)

18

• Manysubclasses,e.g.•  eats/V⇒eat/VB,eat/VBP,eats/VBZ,ate/VBD,eaten/VBN,ea0ng/VBG,...

• Reflectmorphologicalform&syntac0cfunc0on

19

Howdowedecidewhichwordsgoinwhichclasses?

• Nounsdenotepeople,placesandthingsandcanbeprecededbyar0cles?But…

Mytypingisverybad.*TheMarylovesJohn.

• Verbsareusedtorefertoac0ons,processes,states•  ButsomeareclosedclassandsomeareopenIwillhaveemailedeveryonebynoon.• Adverbsmodifyac0ons•  IsMondayatemporaladverboranoun?

20

21

DeterminingPart-of-Speech• Ablueseat/Achildseat:nounoradj?

• Sometests•  Syntac0c

•  Ablueseat Achildseat•  Averyblueseat *Averychildseat•  Thisseatisblue *Thisseatischild

• Morphological•  Bluer *childer

• Blueisanadjec0ve,butchildisanoun 22

DeterminingPart-of-Speech• Preposi0onorpar0cle?A.  Hethrewoutthegarbage.B.  Hethrewthegarbageoutthedoor.

C.  HethrewthegarbageoutD.  *Hethrewthegarbagethedoorout.

• outinAisapar0cle,inBisapreposi0on 23

ClosedClassWords• Idiosyncra0c• Closedclasswords(Prep,Det,Pron,Conj,Aux,Part,Num)areeasier,sincewecanenumeratethem….but• Partvs.Prep

•  Georgeeatsuphisdinner/Georgeeatshisdinnerup.

•  Georgeeatsupthestreet/*Georgeeatsthestreetup.

• Ar0clescomein2flavors:definite(the)andindefinite(a,an)

24

POStagging:Choosingatagset•  TodoPOStagging,needtochooseastandardsetoftagstoworkwith

• Couldpickverycoarsetagsets•  N,V,Adj,Adv.

• BrownCorpus(Francis&Kucera‘82),1Mwords,87tags

• PennTreebank:hand-annotatedcorpusofWallStreetJournal,1Mwords,45-46tags•  Commonlyused•  setisfinergrained,

•  Evenmorefine-grainedtagsetsexist25

PennTreeBankPOSTagset

26

UsingtheUPenntagset• The/DTgrand/JJjury/NNcommmented/VBDon/INa/DTnumber/NNof/INother/JJtopics/NNS./.

• Preposi0onsandsubordina0ngconjunc0onsmarkedIN(“although/INI/PRP..”)

• Exceptthepreposi0on/complemen0zer“to”isjustmarked“to”.

27

POSTagging• WordsoqenhavemorethanonePOS:back•  Thebackdoor=JJ• Onmyback=NN• Winthevotersback=RB• Promisedtobackthebill=VB

• ThePOStaggingproblemistodeterminethePOStagforapar0cularinstanceofaword. 28

These examples from Dekang Lin

HowdoweassignPOStagstowordsinasentence?WhatinformationdoyouthinkwecouldusetoassignPOSinthefollowingsentences?

•  Timeflieslikeanarrow.•  Time/[V,N]flies/[V,N]like/[V,Prep]an/Detarrow/N

•  Time/Nflies/Vlike/Prepan/Detarrow/N•  Fruit/Nflies/Nlike/Va/DETbanana/N•  Fruit/Nflies/Vlike/Prepa/DETbanana/N•  The/Detflies/Nlike/Va/DETbanana/N 29

HowhardisPOStagging?Measuringambiguity

30

Canyouthinkofsevensentenceswhereineachone“well”isusedwithadifferentpartofspeech?

31

PotentialSourcesofDisambiguation• ManywordshaveonlyonePOStag(e.g.is,Mary,very,smallest)

• Othershaveasinglemostlikelytag(e.g.a,dog)• Buttagsalsotendtoco-occurregularlywithothertags(e.g.Det,N)

•  Inaddi0ontocondi0onalprobabili0esofwordsP(w1|wn-1),wecanlookatPOSlikelihoodsP(t1|tn-1)todisambiguatesentencesandtoassesssentencelikelihoods

32

HiddenMarkovModelTagging• UsinganHMMtodoPOStagging

• AspecialcaseofBayesianinference

• Relatedtothe“noisychannel”modelusedinMT,ASRandotherapplica0ons

9/30

/19

33

POStaggingasasequenceclassiDicationtask• Wearegivenasentence(an“observa0on”or“sequenceofobserva0ons”)•  Secretariatisexpectedtoracetomorrow

• Whatisthebestsequenceoftagswhichcorrespondstothissequenceofobserva0ons?

•  Probabilis0cview:•  Considerallpossiblesequencesoftags•  Choosethetagsequencewhichismostprobablegiventheobserva0onsequenceofnwordsw1…wn.

9/30

/19

34

GettingtoHMM•  Outofallsequencesofntagst1…tnwantthesingletagsequencesuchthatP(t1…tn|w1…wn)ishighest.

•  Hat^means“oures0mateofthebestone”

•  Argmaxxf(x)means“thexsuchthatf(x)ismaximized”

9/30

/19

35

GettingtoHMM• Thisequa0onisguaranteedtogiveusthebesttagsequence

• Intui0onofBayesianclassifica0on:

• UseBayesruletotransformintoasetofotherprobabili0esthatareeasiertocompute

9/30

/19

36

UsingBayesRule

9/30

/19

37

Likelihoodandprior

n

Twokindsofprobabilities(1)• Tagtransi0onprobabili0esp(ti|ti-1)

• Determinerslikelytoprecedeadjsandnouns•  That/DTflight/NN•  The/DTyellow/JJhat/NN•  SoweexpectP(NN|DT)andP(JJ|DT)tobehigh•  ButP(DT|JJ)tobe:

• ComputeP(NN|DT)bycoun0nginalabeledcorpus:

Twokindsofprobabilities(2)• Wordlikelihoodprobabili0esp(wi|ti)

• VBZ(3sgPresverb)likelytobe“is”• ComputeP(is|VBZ)bycoun0nginalabeledcorpus:

9/30

/19

40

AnExample:theverb“race”

• Secretariat/NNPis/VBZexpected/VBNto/TOrace/VBtomorrow/NR

• People/NNScon0nue/VBto/TOinquire/VBthe/DTreason/NNfor/INthe/DTrace/NNfor/INouter/JJspace/NN

• Howdowepicktherighttag?

9/30

/19

41

Disambiguating“race”

9/30

/19

42

•  P(NN|TO)=.00047•  P(VB|TO)=.83•  P(race|NN)=.00057•  P(race|VB)=.00012•  P(NR|VB)=.0027•  P(NR|NN)=.0012•  P(VB|TO)P(NR|VB)P(race|VB)=.00000027•  P(NN|TO)P(NR|NN)P(race|NN)=.00000000032•  Sowe(correctly)choosetheverbreading,

9/30

/19

43

SummaryPartsofspeech•  What’sPOStagginggoodforanyhow?•  Tagsets•  Sta0s0csandPOStagging•  Next0me:

•  HMMTagging

44

Homework2

45