Temporal Reasoning in Natural Language Processing Andrew Gibbs November 16 th, 2004.

Temporal Reasoning in Temporal Reasoning in Natural Language Natural Language

ProcessingProcessing

Andrew GibbsAndrew Gibbs

November 16November 16thth, 2004, 2004

““The central component of any The central component of any knowledge representation that knowledge representation that supports Natural Language is the supports Natural Language is the treatment of verbs and treatment of verbs and timetime.” – .” – James AllenJames Allen

Classifying Temporal Classifying Temporal ExpressionsExpressions

Types of TimeTypes of Time

Time PointTime Point Instantaneous point assignment with some Instantaneous point assignment with some

transition in the worldtransition in the worlde.g. light turning on, someone finding a pene.g. light turning on, someone finding a pen

IntervalInterval Extended stretch over which some event Extended stretch over which some event

occursoccurse.g. “John drove his car to work at 5pm.”e.g. “John drove his car to work at 5pm.”

DurationDuration All intervals have durationsAll intervals have durations

e.g. five minutes longe.g. five minutes long Points cannot have durationsPoints cannot have durations

Interpreting Points and Interpreting Points and Intervals of TimeIntervals of Time

T1 < T2T1 < T2 point/interval T1 occurs before point/interval T2point/interval T1 occurs before point/interval T2

T1 : T2T1 : T2 interval T1 meets interval T2, or point T1 defines interval T1 meets interval T2, or point T1 defines

the beginning of interval T2, or point T2 defines the beginning of interval T2, or point T2 defines the end of interval T1the end of interval T1

T1 T1 ⊆ T2⊆ T2point/interval T1 is contained in interval T2point/interval T1 is contained in interval T2

Temporal Sentence ClassesTemporal Sentence Classes

Stative PropositionsStative Propositions Describes a state.Describes a state. Lacks defined ending point.Lacks defined ending point.

E.g. “Jack is happy.”E.g. “Jack is happy.”

Activity PropositionsActivity Propositions Describes an ongoing activity.Describes an ongoing activity. Occurs over an interval of time.Occurs over an interval of time.

E.g. “Jack is running.”E.g. “Jack is running.”

Telic PropositionsTelic Propositions Describes something that is brought to completion.Describes something that is brought to completion. AchievementAchievement

E.g. “Jack recognized the man.”E.g. “Jack recognized the man.” AccomplishmentAccomplishment

E.g. “They climbed the mountain.”E.g. “They climbed the mountain.”

Aspectual ClassAspectual Class Can Can Be Be True True at a at a point?point?

Can Be Can Be True True during during an an IntervalInterval

TemporaTemporal l Modifier Modifier inin

Stative PhraseStative Phrase YesYes YesYes NoNo

ActivityActivity NoNo YesYes NoNo

AchievementAchievement YesYes NoNo YesYes

AccomplishmenAccomplishmentt

NoNo YesYes YesYes

Parsing Text for Temporal Parsing Text for Temporal ExpressionsExpressions

Markers for TimeMarkers for Time Noun/Noun Phrase/Proper NounNoun/Noun Phrase/Proper Noun

““day”, “Friday night”, “Wednesday”day”, “Friday night”, “Wednesday” Prepositional PhrasePrepositional Phrase

““in a week”in a week” AdjectiveAdjective

““current”, “future”current”, “future” AdverbAdverb

““recently”, “hourly”recently”, “hourly” Adjective/Adverb PhraseAdjective/Adverb Phrase

““two weeks ago”, “nearly half an hour ago”two weeks ago”, “nearly half an hour ago” NumberNumber

3 (as in “He arrived at 3.”)3 (as in “He arrived at 3.”) Subordinate ClausesSubordinate Clauses

“…“…when the market stabilized”when the market stabilized”

Examples of Current MethodsExamples of Current Methods

LogicsLogicsTense LogicTense LogicInterval-based Temporal LogicInterval-based Temporal Logic

TIMEX2TIMEX2TimeMLTimeMLDAML Ontology of timeDAML Ontology of time

Tense LogicTense Logic

S – the time of speechS – the time of speech

E – the time of the event/stateE – the time of the event/state

R – the reference timeR – the reference time

Tense LogicTense Logic

Jack sings.Jack sings. simple present: S=R, E=Rsimple present: S=R, E=R

Jack sang.Jack sang. simple past: R<S, E=Rsimple past: R<S, E=R

Jack will sing.Jack will sing. simple future: S<R, E=Rsimple future: S<R, E=R

Jack has sung.Jack has sung. present perfect: S=R, E<Rpresent perfect: S=R, E<R

Jack had sung.Jack had sung. past perfect: R<S, E<Rpast perfect: R<S, E<R

Jack will have sung.Jack will have sung. future perfect: S<R, E<Rfuture perfect: S<R, E<R

S – the time of speechS – the time of speechE – the time of the event/stateE – the time of the event/stateR – the reference timeR – the reference time

Tense LogicTense Logic

Jack is going to sing.Jack is going to sing.

posterior present: S=R, R<Eposterior present: S=R, R<E

Jack was going to sing.Jack was going to sing.

posterior past: R<S, R<Eposterior past: R<S, R<E

Jack will be going to sing.Jack will be going to sing.

posterior future: S<R, R<Eposterior future: S<R, R<E

S – the time of speechS – the time of speechE – the time of the event/stateE – the time of the event/stateR – the reference timeR – the reference time

Interval-based Temporal Interval-based Temporal LogicLogic

Only based on intervals.Only based on intervals. 13 basic binary relations between time 13 basic binary relations between time

intervals: intervals: before, after, overlaps, before, after, overlaps, overlapped by, starts, started by, finishes, overlapped by, starts, started by, finishes, finished by, during, contains, meets, met finished by, during, contains, meets, met by, equal toby, equal to

Incomplete temporal information common Incomplete temporal information common in natural-language is captured by a in natural-language is captured by a disjunction of several of these relations.disjunction of several of these relations.

A

B

A

B

A

B

A

B

A

B

A

B

A

B

A is EQUAL TO B

A is BEFORE B

B is EQUAL TO A

B is AFTER A

A MEETS B

B is MET BY A

A OVERLAPS B

B is OVERLAPPED BY A

A STARTS B

B is STARTED BY A

A FINISHES B

B is FINISHED BY A

A is DURING B

B CONTAINS A

Interval-based Temporal Interval-based Temporal LogicLogic

1.1. PropertiesProperties hold over hold over every subintervalevery subinterval of an of an interval. Thus, the meaning of interval. Thus, the meaning of Holds(p,T)Holds(p,T) is that is that property property pp holds over interval holds over interval TT.. ׂׂ John was sleeping during the night.John was sleeping during the night.

2.2. EventsEvents hold hold only overonly over a whole intervala whole interval and not and not over any subinterval of it. Thus, over any subinterval of it. Thus, Occurs(e,T)Occurs(e,T) denotes that event denotes that event ee occurred at time occurred at time TT.. ׂׂ John broke his leg on Saturday at 6 P.M.John broke his leg on Saturday at 6 P.M.

3.3. ProcessesProcesses hold over hold over some subintervalssome subintervals of the of the interval in which they occur. Thus, interval in which they occur. Thus, Occurring(p,T)Occurring(p,T) denotes the process denotes the process pp is is occurring during time occurring during time TT.. John is walking around the block.John is walking around the block.

TIMEX2TIMEX2

• Developed by DARPA Translingual Developed by DARPA Translingual Information Detection, Extraction, Information Detection, Extraction, and Summarization (TIDES) in 2001and Summarization (TIDES) in 2001

• Automatically annotates sentences Automatically annotates sentences with tags describing temporal with tags describing temporal informationinformation

• Focuses on temporal markers (key Focuses on temporal markers (key words)words)

TIMEX2 Tag AttributesTIMEX2 Tag AttributesAttributeAttribute FunctionFunction ExampleExample

VALVAL Contains normalized Contains normalized form of the date/time.form of the date/time.

VAL=“1964-10-16”VAL=“1964-10-16”

MODMOD Captures temporal Captures temporal modifiers.modifiers.

MOD=“APPROX”MOD=“APPROX”

SETSET Identifies expressions Identifies expressions denoting sets of times.denoting sets of times.

SET=“YES”SET=“YES”

PERIODICITYPERIODICITY Captures the period Captures the period between regularly between regularly recurring times.recurring times.

PERIODICITY=“PIM”PERIODICITY=“PIM”

GRANULARITYGRANULARITY Captures the unit of time Captures the unit of time denoted by each set denoted by each set member in a set of member in a set of times.times.

GRANULARITY=“G3D”GRANULARITY=“G3D”

NON_SPECIFICNON_SPECIFIC Identifies non-specific Identifies non-specific expressions.expressions.

NON_SPECIFIC=“YES”NON_SPECIFIC=“YES”

COMMENTCOMMENT Contains any comments Contains any comments the annotator wants to the annotator wants to add.add.

COMMENT=“context COMMENT=“context garbled”garbled”

TIMEX2 ExamplesTIMEX2 Examples I was sick yesterday.I was sick yesterday.

I was sickI was sick <TIMEX2 VAL=“2004-11-15”> <TIMEX2 VAL=“2004-11-15”> yesterdayyesterday </TIMEX2></TIMEX2>..

Two years ago, the dance club drew about 100 Two years ago, the dance club drew about 100 students each week.students each week. <TIMEX2 VAL=“2002”> <TIMEX2 VAL=“2002”> Two years agoTwo years ago </TIMEX2> </TIMEX2>,, the the

dance club drew about 100 students dance club drew about 100 students <TIMEX2 <TIMEX2 VAL=“2002” SET=“YES” GRANULARITY=“G1W” VAL=“2002” SET=“YES” GRANULARITY=“G1W” PERIODICITY=“F1W”> PERIODICITY=“F1W”> each weekeach week </TIMEX2> </TIMEX2>..

A major earthquake struck Los Angeles three years A major earthquake struck Los Angeles three years ago today.ago today. A major earthquake struck Los AngelesA major earthquake struck Los Angeles <TIMEX2 VAL= <TIMEX2 VAL=

“2001-11-16”> “2001-11-16”> three years agothree years ago <TIMEX2 VAL=“2004- <TIMEX2 VAL=“2004-11-16”> 11-16”> today today </TIMEX2> </TIMEX2></TIMEX2> </TIMEX2>..

TIMEX2 Point vs DurationTIMEX2 Point vs Duration

Point in Time:Point in Time:

He was happy five days ago.He was happy five days ago.

He was happyHe was happy <TIMEX2 VAL=“2004-11- <TIMEX2 VAL=“2004-11-11”> 11”> five days agofive days ago</TIMEX2></TIMEX2>..

Duration: Duration:

He was happy for five days.He was happy for five days.

He was happy forHe was happy for <TIMEX2 VAL=“P5D”> <TIMEX2 VAL=“P5D”> five daysfive days </TIMEX2> </TIMEX2>..

TimeMLTimeML

• Developed over a six-month period, Developed over a six-month period, funded by ARDA.funded by ARDA.

• Automatically annotates sentences with Automatically annotates sentences with tags describing temporal and event tags describing temporal and event information.information.

• Focuses on content rather than key words.Focuses on content rather than key words.

TimeMLTimeML

Extends the TIMEX2 annotation of Extends the TIMEX2 annotation of attributesattributes

Reasons with contextually underspecified Reasons with contextually underspecified temporal expressions: temporal expressions: last week, in recent last week, in recent yearsyears

Identifies signals determining Identifies signals determining interpretation of temporal expressionsinterpretation of temporal expressions Temporal Prepositions: Temporal Prepositions: for, during, on, atfor, during, on, at Temporal Connectives: Temporal Connectives: before, after, whilebefore, after, while

TimeMLTimeML

Identifies all classes of event expressionsIdentifies all classes of event expressions Tensed verbs: Tensed verbs: has left, was captured, will resignhas left, was captured, will resign Stative adjectives and other modifiers:Stative adjectives and other modifiers: sunken, sunken,

stalledstalled Event nominal:Event nominal: merger, Military Operation, Gulf War merger, Military Operation, Gulf War

Creates dependencies between events and Creates dependencies between events and timestimes Anchoring: Anchoring: John left on Monday.John left on Monday. Orderings: Orderings: The party happened after midnight.The party happened after midnight. Embedding: Embedding: John said Mary Left.John said Mary Left.

TimeML Example 1TimeML Example 1John left two days before the attack.

John<EVENT eid=“e1” class=“OCCURRENCE” tense=“PAST” <EVENT eid=“e1” class=“OCCURRENCE” tense=“PAST” aspect=“PERFECTIVE”>aspect=“PERFECTIVE”>

left</EVENT></EVENT><MAKEINSTANCE eiid=“ei1” eventID=“e1”/><MAKEINSTANCE eiid=“ei1” eventID=“e1”/><TIMEX3 tid=“t1” type=“DURATION” value=“P2D” <TIMEX3 tid=“t1” type=“DURATION” value=“P2D” temporalFunction=“false”>temporalFunction=“false”>

2 days</TIMEX3></TIMEX3><SIGNAL sid=“s1”><SIGNAL sid=“s1”>

before</SIGNAL></SIGNAL>

the<EVENT eid=“e2” class=“OCCURRENCE” tense=“NONE” <EVENT eid=“e2” class=“OCCURRENCE” tense=“NONE” aspect=“NONE>aspect=“NONE>

attack</EVENT></EVENT><MAKEINSTANCE eiid=“ei2” eventID=“e2/><MAKEINSTANCE eiid=“ei2” eventID=“e2/>

TimeML Example 2TimeML Example 2Bill wants to teach on Monday.

BillBill<EVENT eid=“e1” class=“I_STATE” tense=“PRESENT” aspect=“NONE><EVENT eid=“e1” class=“I_STATE” tense=“PRESENT” aspect=“NONE>

wantswants</EVENT></EVENT><MAKEINSTANCE eiid=“ei1” eventID=“e1”/><MAKEINSTANCE eiid=“ei1” eventID=“e1”/><SLINK eventInstanceID=“ei1” signalID=“s1”<SLINK eventInstanceID=“ei1” signalID=“s1”subordinateEvent=“e2” relType=“MODAL”/>subordinateEvent=“e2” relType=“MODAL”/><SIGNAL sid=“s1”><SIGNAL sid=“s1”>

toto</SIGNAL></SIGNAL><EVENT eid=“e2” class=“OCCURRENCE” tense=“NONE” <EVENT eid=“e2” class=“OCCURRENCE” tense=“NONE”

aspect=“NONE”>aspect=“NONE”>

teachteach</EVENT></EVENT><MAKEINSTANCE eiid=“ei2” eventID=“e2”/><MAKEINSTANCE eiid=“ei2” eventID=“e2”/><SIGNAL sid=“s2”><SIGNAL sid=“s2”>

onon</SIGNAL></SIGNAL><TIMEX3 tid=“t1” type=“DATE” temporalFunction=“true” value=“XXXX-<TIMEX3 tid=“t1” type=“DATE” temporalFunction=“true” value=“XXXX-

WXX-1”>WXX-1”>

MondayMonday</TIMEX3></TIMEX3><TLINK eventInstance=“ei2” relatedToTime=“t1” <TLINK eventInstance=“ei2” relatedToTime=“t1”

relType=“IS_INCLUDED”/>relType=“IS_INCLUDED”/>

DAML Ontology of TimeDAML Ontology of Time

Funded by DARPAFunded by DARPA Still under developmentStill under development Built with the intention of creating more Built with the intention of creating more

accurate search engines.accurate search engines. Parses natural language in web-pages to Parses natural language in web-pages to

determine the content.determine the content. Built-in facilities for fast temporal reasoning.Built-in facilities for fast temporal reasoning. Based upon Interval-based Temporal logic.Based upon Interval-based Temporal logic. Integrated with TimeML for the annotation Integrated with TimeML for the annotation

of text.of text.

DAML example axiomDAML example axiom

<axiom id=“2.2-1”><axiom id=“2.2-1”>

before(T1, T2) && before(T2, T3) --> before(T1, before(T1, T2) && before(T2, T3) --> before(T1, T3)T3)

</axiom></axiom>

Summary of DAML-Summary of DAML-Time/TimeMLTime/TimeML

TimeML Annotation of TextTimeML Annotation of Text

↓↓

Algorithms for Automatic TimeML annotation Algorithms for Automatic TimeML annotation of textof text

↓↓

Interpret annotations in DAML-TimeInterpret annotations in DAML-Time

↓↓

Reason in DAML-Time to match requests Reason in DAML-Time to match requests with serviceswith services

Example QueryExample Query

• I want the latest book by John McCarthy by I want the latest book by John McCarthy by next Tuesday.next Tuesday.

Author: John McCarthyAuthor: John McCarthy

Book: Formalizing Common SenseBook: Formalizing Common Sense

Date: 1998Date: 1998

Price: $24.95Price: $24.95

Author: John McCarthyAuthor: John McCarthy

Book: LISP 1.5Book: LISP 1.5

Date: 1968Date: 1968

Price: $16.95Price: $16.95

Example QueryExample Query

• I want the I want the latestlatest book by John McCarthy by book by John McCarthy by next Mondaynext Monday..

Author: John McCarthyAuthor: John McCarthy Ships within 5 days.Ships within 5 days.Book: Formalizing Common SenseBook: Formalizing Common SenseDate: Date: 19981998Price: $24.95Price: $24.95

Author: John McCarthyAuthor: John McCarthyBook: LISP 1.5Book: LISP 1.5Date: 1968Date: 1968Price: $16.95Price: $16.95

ReferencesReferences Allen, James F., Allen, James F., Natural Language UnderstandingNatural Language Understanding, The , The

Benjamin/Cummings Publishing Company, Menlo Park, California, Benjamin/Cummings Publishing Company, Menlo Park, California, (Addison-Wesley Publishing Company, Reading, Massachusetts), (Addison-Wesley Publishing Company, Reading, Massachusetts), 1995 Pages 406-4101995 Pages 406-410

B. Han and A. Lavie. B. Han and A. Lavie. A Framework for Resolution of Time in A Framework for Resolution of Time in Natural LanguageNatural Language. . TALIP Special Issue on Spatial and Temporal TALIP Special Issue on Spatial and Temporal Information ProcessingInformation Processing, 2004 , 2004 http://www-2.cs.cmu.edu/~alavie/papers/BenH-TALIP-04.pdfhttp://www-2.cs.cmu.edu/~alavie/papers/BenH-TALIP-04.pdf

Kannan, A. Geetha, TV. Kannan, A. Geetha, TV. Temporal Reasoning with Intelligent Temporal Reasoning with Intelligent DatabasesDatabases. Anna University 2000 . Anna University 2000 http://www.ncst.ernet.in/kbcs/vivek/issues/11.4/kannan/kannan.htmhttp://www.ncst.ernet.in/kbcs/vivek/issues/11.4/kannan/kannan.htmll

Galton, Anthony. Galton, Anthony. Temporal LogicTemporal Logic. Stanford Encyclopedia of . Stanford Encyclopedia of Philosophy, 2003 Philosophy, 2003 http://plato.stanford.edu/entries/logic-temporal/http://plato.stanford.edu/entries/logic-temporal/

Ligozat, Gerard. Ligozat, Gerard. Representation of Space and TimeRepresentation of Space and Time. . http://cslu.cse.ogi.edu/HLTsurvey/ch9node4.htmlhttp://cslu.cse.ogi.edu/HLTsurvey/ch9node4.html

References (cont’d)References (cont’d) Pustejovsky, J., J. Castano, R. Ingria, R. Saurí, R. Gaizauskas, A. Pustejovsky, J., J. Castano, R. Ingria, R. Saurí, R. Gaizauskas, A.

Setzer, G. Katz (2003) Setzer, G. Katz (2003) TimeML: A Specification Language for TimeML: A Specification Language for Temporal and Event ExpressionsTemporal and Event Expressions. In . In IWCS, International IWCS, International Workshop of Computational Semantics.Workshop of Computational Semantics. Kluwer Academic Kluwer Academic Publishers.Publishers.

Hobbs, Jerry R., Ferguson, G., Allen, J., Hayes, P., Niles, I., and Hobbs, Jerry R., Ferguson, G., Allen, J., Hayes, P., Niles, I., and Pease, A. 2002 Pease, A. 2002 A DAML Ontology of TimeA DAML Ontology of Time. . http://www.cs.rochester.edu/~ferguson/daml/http://www.cs.rochester.edu/~ferguson/daml/

Ferro, Lisa. Ferro, Lisa. Instructional Manual of the Annotation of Instructional Manual of the Annotation of Temporal ExpressionsTemporal Expressions. MITRE, 2003 . MITRE, 2003 http://www.mitre.org/work/tech_papers/tech_papers_04/ferro_tideshttp://www.mitre.org/work/tech_papers/tech_papers_04/ferro_tides/ferro_tides.pdf/ferro_tides.pdf

Shahar, Yuval. Shahar, Yuval. Temporal Reasoning in Clinical DomainsTemporal Reasoning in Clinical Domains. 1994 . 1994 http://www.ise.bgu.ac.il/courses/trp/Shahar-1994.chapter3.dochttp://www.ise.bgu.ac.il/courses/trp/Shahar-1994.chapter3.doc

Hobbs, Jerry R. Hobbs, Jerry R. Ontologies for the Semantic Web: Time and Ontologies for the Semantic Web: Time and SpaceSpace. 2003 . 2003 http://www.racai.ro/EUROLAN-2003/html/presentations/JerryHobbshttp://www.racai.ro/EUROLAN-2003/html/presentations/JerryHobbs/1/1