Time Tracks and Time Segments. Rethinking the Way to Look at Texts Dan Cristea “Alexandru Ioan Cuza” University of Iași, Faculty of Computer Science Romanian Academy Iași branch, Institute of Computer Science [email protected]
TimeTracksandTimeSegments.RethinkingtheWaytoLookatTexts
DanCristea“AlexandruIoanCuza”UniversityofIași,FacultyofComputerScience
RomanianAcademyIașibranch,[email protected]
Ourperceptionoftime
• Sherememberedthewordsthatsome.mes,atthe.meofadolescence,hermothertoldher."Youareapersonapprehended,right?...
TashAw:Mapoftheinvisibleworld
SYNASC,Timișoara,26thSeptember2016
Ourperceptionoftime
SYNASC,Timișoara,26thSeptember2016
Ourperceptionoftime
• Inthekitchen,wai.ngforthewatertoboil,MargaretwonderedhowwasitthatshejumpedsonaturallytoembraceAdam.Shecouldnotrealizewhathadmadeherfeelsohappytoseehimagain.
TashAw:Mapoftheinvisibleworld
SYNASC,Timișoara,26thSeptember2016
Ourperceptionoftime
SYNASC,Timișoara,26thSeptember2016
Goals• Acomputationalrepresentationoftemporalphenomenaoccurringinliterarytexts,preparingthusinvestigationtoolsthatwouldmakepossibletotacklethesephenomenabyautomaticanalysis
• Therewerereadersperceive:– changesofthecurrentdirectionoftime,– flashbacksandflashforwards,– temporalruptures,– perspectivesofdifferentnarratorsonthesamestory
Inthistalk• Timecanbebettervisualisedasthreadsthatinterlaceandseparate,– muchlikerailwaysdoincomplexrailyards:.metracks
• ProposeanXMLnotationfortheserepresentations• Showafewexamplesthatputinevidencedifferenttemporalmanifestations– commute,rupture,joinandsplitpoints
• Makeapreliminaryevaluationoftheannotationsonasmallcorpus
• Freelythinkfurther.Noconclusions!
Ourworkisframedin…
• TemporalReasoning(andtemporallogic:Allen,etc.)
• Annotationconventions/standardstocopewithtimeasmentionedinlanguageandtext
• Muchinterest:– informationextraction– questionanswering– textualentailment
SincetheACLworkshoponSpa.alandTemporalReasoning(2001),LRECworkshoponAnnota.onStandardsforTemporalInforma.oninNaturalLanguage(2002)
SYNASC,Timișoara,26thSeptember2016
Relatedwork– Anotation:TimeML
– TIMEX3:markupexplicittemporalexpressions(times,dates,durations,etc.)– SIGNALannotatefunctionwordsthatindicatehowtemporalobjectsaretobe
relatedtoeachother(e.gon,during,when,if,etc.)– TLINK-TemporalLinks:thetemporalrelationshipholdingbetweeneventsor
betweenaneventandatime– SLINK-SubordinationLink:relationsbetweentwoevents,oraneventanda
signal– ALINK-AspectualLink:relationshipbetweenanaspectualeventandits
argumentevent
– EVENT JamesPustejovskyetal.(2003).TimeML:RobustSpecificationofEventandTemporalExpressionsinText,AAAITechnicalReportSS-03-07
SYNASC,Timișoara,26thSeptember2016
Relatedwork
– Asystem(theTARSQIToolkit)thatrecognises:• temporalexpressions(TimeEx)• events• relationsbetweenthem
Verhagen&Pustejovsky:TemporalProcessingwiththeTARSQIToolkit,Coling2008
SYNASC,Timișoara,26thSeptember2016
Relatedwork– NarrativeContainers:assumedwindowsprovideleftandrightboundariesforwhenunanchoredeventsmostlikelyoccurred;sizeparameterizedbyDocumentCreationTime(DCT)andtheperiodicityofthesource
– NarrativeScope:thetimespanbetweentheearliestandlatesteventsdescribedinthedocument
– NarrativeTime:thecurrenttemporalanchorforeventsinadocument;temporalfocusshiftsasthereadermovesthroughthedocument
=>themannerinwhicheventsarelinked(related)totheDCTinTimeML Pustejovsky&Stubbs(2011).IncreasingInformativenessin
TemporalAnnotation,ProceedingsoftheFiIhLawWorkshop(LAWV),Portland
Relatedwork• ThemodelofNarrativeContaineranditsrelationswrtto:– Channel:isthedocumentwrittenorspoken?– Productioncircumstances:howwasthedocumentdistributed?broadcast,newswire,dailypublication
– Style:whatformatwasusedtopresenttheinformation?
– Presenceofatemporalanchor:whetheranarticlecontainedaNarrativeTimeinthefirstsentenceofthedocument
Pustejovsky&Stubbs(2011).IncreasingInformativenessinTemporalAnnotation,ProceedingsoftheFiIhLawWorkshop(LAWV),Portland
However…
• Fictionnevermentionedinthesestudies– mostinterestattachedtoprocessnewswiresandbroadcastedinformation
• Lessornoattentiongiventothelinkbetweeneventsandcharactersinvolved– andthereismorethanthemereroleofagentinanevent:JohnmetAnawhenexi.ngtheOperaHouse.Hesaid:Amazingvoices!
SYNASC,Timișoara,26thSeptember2016
Weareinterestedinconnectedsequencesofevents
• Inhistoricalstudies,dramaturgyandnarratology– 1906:storyline:
Amer.PhysicianMar.86/1:Hehadtoconfesshisinabilitytodothestory-linejus.ce.
– 1907:.meline–Alineusedtoillustrateorrepresentachronologicalsequenceofevents;achronology;achronologicalsequenceJ.W.ADAMSONPract.ofInstr.II.iv.275:A.melineforEnglishhistory…canbemadewithgreatadvantagewhentheclassistakingitsfirstconnectedcourseinEnglishhistory.
OxfordEnglishDictionarySYNASC,Timișoara,27thSeptember2016
Thankyou,Ja
mesDaven
port!
Weareinterestedinconnectedsequencesofevents
• InNLP– TimeLine:arepresentationofeventswhicharechronologicallyordered,ingeneralspecificforanentity(acharacterorparticipantinanaction,ageographicalplaceorregion)
– StoryLine:groupsofinteractingTimeLines,ormergersoftwoormoreTimeLineswherethesamecharactersorentitiesaretakingpartintheaction.Akindoftextstructurewhichisnotrelatedtotheflowoftime.
ChambersandJurafsky(2008).UnsupervisedLearningofNarrativeEventChains.InACL.
Laparra,AldabeandRigau(2015).FromTimeLinestoStoryLines:Apreliminaryproposalforevaluatingnarratives.InACL-IJCNLP.
Weareinterestedinconnectedsequencesofevents
• InNLP– .meline(orfabula):asetofeventinstancesorderedintime(anchoredtotimeexpressionsorrelativetoeachother)andasetofbridgingrelationsbetweeneventswithexplanatoryandpredictivevalue(risingaction,climaxandfallingaction)
– storyline:thetimelinemadeofthoseeventswhichhavethehighestconnectivityscore
⇒ detectstorylineinmassivestreamsofnews⇒ notconcernedwithdecipheringthetemporalstructureoftexts
Vossen,CaselliandKontzopoulou(2015).Storylinesforstructuringmassivestreamsofnews.InACL-IJCNLP
Motivationandgoal
• Largetextsaredifferentthennews– news:sequencesofevents,opinions,commentaries– novels:eventsareseldompresentedinnon-sequentialorder,sameeventsareinterpretedfromdifferentperspectives
• Charactersparticipateinevents• Eachhasher/hisowntrajectory/destiny/evolution• Althoughcharacters’trajectoriesintersectintime,onewouldliketodistinguish/separatethem
SYNASC,Timișoara,26thSeptember2016
The.meyardmodel
• ATimeYard(TY)isarepresentation(resemblingadiagram)ofthecharacters’destinies(atoo…preciousterm)inatext,unfoldedonthestorytimeaxis
• ATYismadeofTimeTracks(TTs)• ATTismadeofTimeSegments(TSs)• AbookisasequenceofTimeSegments
SYNASC,Timișoara,26thSeptember2016
TimeTracks
• ATTdisplaysthestorydevelopmentofonecharacterorastablegroupofcharactersoveraperiodofstorytime.– TTsaredecipheredbythereaderashavingaunityofcharacters,actionanddevelopment
– OneTTcanbenarratedbyoneormorerelators– TTsarenotnecessarilycharacterisedbyaunityofplace
SYNASC,Timișoara,26thSeptember2016
BorderpointsofTTs
• Timetrackscanbeborderedby:– startpoints–wheretheystart,– endpoints–wheretheyfinish,– joinpoints–inwhichtwodifferenttimetracksunify,
– splitpoints–whereonetimetracksplitsintotwoseparatetimetracks.
SYNASC,Timișoara,26thSeptember2016
TimeSegments
• ATSgroupsasequenceofeventsofonecharacterorastablegroupofcharactersoveraperiodofstorytime,thatisuninterruptedlytoldinaspanoftext.– oneTSisnecessarilytoldinacontinuousspanoftext– inaTS,thestorytimeevolveslinearly– oneTScanbenarratedbyonlyonerelator
SYNASC,Timișoara,26thSeptember2016
BorderpointsinTSs• IfinnertoTTs,TSscanbeborderedby:– rupturepoints–wherethestorycommutesfromonetimetrackontoadifferentone
– commutepoints–inwhich,althoughthestorygoeson,itissaidfromadifferentperspective,byadifferentstoryteller;oritmovesforthorbackonthesameTT.
• IfattheextremeofTTs,TSscanalsobeborderedby:– startorsplitpoints–ontheleftside– joinorstoppoints–ontherightside.
SYNASC,Timișoara,26thSeptember2016
Margaretinthekitchen…
• TS1:[Inthekitchen,wai.ngforthewatertoboil,Margaretwonderedhowwasitthat]
• TS2:[shejumpedsonaturallytoembraceAdam.]
• TS3:[Shecouldnotrealizewhathadmadeher]• TS4:[feelsohappytoseehimagain...]”
SYNASC,Timișoara,26thSeptember2016
TTsandTSsinMargaret&Adam’sepisode
commutepoints
SYNASC,Timișoara,26thSeptember2016
Margaretandhermother…
• TS1:[Sherememberedthewords]• TS2:[thatsome.mes,atthe.meofadolescence,hermothertoldher."Youareapersonapprehended,right?...]
SYNASC,Timișoara,26thSeptember2016
TTsandTSsinMargaret&hermother’sepisode
SYNASC,Timișoara,26thSeptember2016
Splitpoints
• TS1:[Uptoclimbintothetruckanddisappearafterthetarpaulinfalldown,]
• TS2:[hefollowedthemslowly,withhisclumsygait.]
SYNASC,Timișoara,26thSeptember2016
Joinpoints
• TS1:[Itwasonlywhenshecametothedoorwaythatsherealized]
• TS2:[someonewasthere,lyingonthefrontsteps.Hewasaboy,ateenager,almostcrouchedinthewomb.]
SYNASC,Timișoara,26thSeptember2016
TashAw’s“Mapoftheinvisibleworld”bird’seyetimeyard
SYNASC,Timișoara,26thSeptember2016
ReadingtheTYdiagrams
• Misses/discontinuities⇒ horizontalaxisinaTYrepresentsthestory-time⇒ notcompulsorythatatextincludesacompletedescriptionofaninterval
⇒ segmentsmaynotfillcontiguouslyacertainTT⇒ discontinuities:pauses,misses(interruptedlinesinthefigureabove)
SYNASC,Timișoara,26thSeptember2016
ReadingtheTYdiagrams
• Trackingcharacters’destinies(developments)=>inboldinthefigure:Adam’strajectory=>amereselectionoperationinthesetofTTelements=>producefocusedsummaries
SYNASC,Timișoara,26thSeptember2016
NotshowninTYdiagrams
• Ambiguoustemporalplacements:whentheexactorderofTSscannotbedetermined=>moreTSsbelongtothesameTT=>buttheirorderisundefined
SYNASC,Timișoara,26thSeptember2016
NotshowninTYdiagrams• Personalinterpretations:
=>differentTSsaretoldbydifferentrelators
=>thesameTSistoldbydifferentrelators
AnXMLnotation–inadditiontothoseofTimeML
• ElementsmarkingTimeTracks:<TT NAME ACTORS ID LEFT-EP RIGHT-EP/>
– LEFT-EP:“START”|IDofaSPLITend-point– RIGHT-EP:“STOP”|IDaJOINend-point
• ElementsmarkingTimeSegments<TS ID IN-TT TIME TYPE PER>...</TS>
– IN-TT:IDofaTT– TIME:IDofTIMEelement– TYPE:NARration,REMembers,SUPposition,GENeralknowledge,FICtions
– PERspective:acharacter(entity)
AnXMLnotation
• Elementsmarkingtimeconstraints<TIME ID FROM REL TO/>
– FROMandTO:IDsofTS– REL:BEFORE,IMMEDIATELY-BEFORE,AFTER,IMMEDIATELY-AFTER,SIMULTANEOUSLY,INCLUDES
• Elementsmarkingendpoints<ENDPOINT ID TYPE TT1 TT2/>
– TYPE:“JOIN”or“SPLIT”– TT1andTT2:IDsofthetwoTTsthateitherjoinorsplit
SYNASC,Timișoara,26thSeptember2016
Thecorpus
• FewpagesfromthesamenovelannotatedbyaclassofmasterstudentsinCL– interestedtoevidenceinter-annotatoragreement
• TashAw:Mapoftheinvisibleworld(Romanianversion)–currentlyunderannotation(byAndreeaMacovei)– tobeusedintrainingaTYparser
SYNASC,Timișoara,26thSeptember2016
Evaluation• InterestedtoappreciateinterannotatoragreementonperceivingTTsandTSs
• 2smallfragmentsfromMotIWçv
StudentstakingTSsasnewTT
çv
çv
StudentsmissingsomeTTsTTsareobjective,noticeable
Continuation
• OngoingannotationofthewholeMofIWnovel
• ATYvisualiserunderconstruction• ATYannotatorthatwillworkhandinhandwiththevisualiser
SYNASC,Timișoara,26thSeptember2016
Butwe,ourselves,liveontimeyards…
• Weliveourlivessequentially– Ilivedwithinmyfamilyuntiltheageof16,thenfortwoyearsIwentinanothercitytofollowahighschool,thenfor5yearsIwasastudentinBucharest,andoneyearbeforefinishingthefacultyIwasmarried,thenIcametoIașiand4yearslatermyfirstdautherwasborn,andwhenshewas5Igotdivorced,oneyearlaterImarriedagaina.s.o.
SYNASC,Timișoara,26thSeptember2016
OurTYs
• Myfirstdaughter’sstory…• Ourlivesmeet,thensplit,thenmeetagain…• Wemeetmanypeopleduringourlives,eachhavingher/hisowntrajectory
=>likerailways...
SYNASC,Timișoara,26thSeptember2016
WhenIsay:Ilivedwithinmyfamilyun.ltheageof16…
• Actually,eachdayIwenttoschool,intheholidaysIwenttomygrandparentshouseinthecountryside,andIwentmany.mestocinema…
• …andmyparentswerenotnearmeallthetime.=>lookedatitunderthelens,atimesegmentrevealsmanymore(finer)segments=>italsodependsontheperspec&ve…
SYNASC,Timișoara,26thSeptember2016
Atimetrackperspective
• Somebodythinks/believesthatsomebodyelsehasacertaintrajectory=>coursesofeventscanbeseendifferentlythroughdifferentperspectives=>sometimesTTsofacommunityofpeoplearegrosslyseenintertwined,justlikemakingasingledestiny:collectivities,peoplelivinginacountry=>althougheachhasher/hisown(developing)TT
SYNASC,Timișoara,26thSeptember2016
Adigression
• GENLIAS(www.genlias.nl)–digitisationandtrascriptionofoneofthebiggestcollectionofbirth,marriageanddeathcertificatesintheworld:nowapprox.15millioncertificateson50millionpeople=>32millioncert.
• LINKS–conectsthenamesincertificates=>tracesthetrajectoryofthepopulationandrebuildsfamilies
SYNASC,Timișoara,26thSeptember2016
Justimagine…
• …thatthereisadevicethatrecordstheTTofeachoneofus=>technologicallyfeasible=>perhapsnotverymuchwanted…(sentimentofBBoppression,leaks,etc.)
• BUT...=>pagesofhistorycanbeautomaticallywritten,withaproperintelligentsummarisationagent=>onecanleavepartsofher/hislifetotheposterity...
SYNASC,Timișoara,26thSeptember2016
Anapplication:MyDiary(ForgetMeNote)
• AnAssistedLeavingproject:inthebenefitofoldpersonsindangerofAlzheimerdisease…– theAgent“follows”theMasterfromawakingtillgoingtobed
– itperceiveswalkings,drivings,encounters,etc.– attheendofthedaytheagentengagesadialoguewiththeMaster• toclarifyambiguousmoments• toexerciseMaster’smemory
SYNASC,Timișoara,26thSeptember2016
Anapplication:“Takingtheforeignlanguageteacherwithme”
• Anagentabletoteachatouristthelanguageofthecountryshe/hetravelsin,inadynamic,interactivemanner.– awareofthesituationtheuserhappenstobein– assistsher/him– offerslinguisticsupport,adaptabletotheprogresstheuserismakinginlearningthelanguage• drillingexercisesadaptedtothesituation,• explanationsoftherealityaroundher/himbyappealingtoculturalidiosyncrasies,• realtimesupportinconversations,etc.
Acknowledgements• HumanitasPublishingHousefortheRomanianversionofTashAw’snovel
• TheWylieAgency,UK,fortheaccepttouseforreseachtheoriginalEnglishversionofTashAw’snovel
• MyclassofmasterstudentsinCL,fordebateswhileestablishingtheannotationconventionsandparticipationinthefirstannotationexperiment
• AndreeaMacoveiforannotatingthenovelandco-authoringpartsofthisresearch
• EmiPeiaforimplementingtheannotatorandtheTYvisualiser
SYNASC,Timișoara,26thSeptember2016