Extracting Temporal and Causal Relations between Events

Extracting Temporal and Causal Relations between Events

Paramita Mirza

Under the supervision of Sara Tonelli

ACL SRW 2014

http://www.newsreader-project.eu/

What if computers could read the NEWS?

Recording events as stories

Event Extraction

On September 2008, Porsche increased its shares by another 4.98%, in effect taking control of the company.

6 Jan 2009 – Porsche has been on a quest to takeover VW for more than two years.

NAMED ENTITY TIME EXPRESSION EVENT

PhD topic

timeanchor

e1: increased- Porsche- shares- 4.98%

e2: taking control- Porsche- the company

e3: quest to takeover- Porsche- VW

t1: September 2008 t2: 6 Jan 2009 t3: two years

BEFORE

CAUSE

DURINGON ON

EVENT PARTICIPANT EVENT RELATION EVENT FACTUALITY

TEMPORAL RELATION CAUSAL RELATION

CAUSE

BEFORE

The Relationship between Events

IS_INCLUDED

Temporal Relations

Causal Relations

Typhoon Haiyan struck the eastern Philippines on Friday,

killing thousands of people.

Temporal Constraint of Causalitycause BEFORE effect

creating event timelines, multi-document summarization

predicting future events, risk analysis,decision making support

Research Questions

“Given a text annotated with events and time expressions, how to automatically extract temporal relations and causal relations between them?”

“Given the temporal constraint of causality, how to utilize the interaction between temporal relations and causal relations for building an integrated extraction system for both types of relations?”

Research Methodology

Temporal Relation

Extraction

1st

• Finding ways to improve the current state-of-the-art performance on temporal relation extraction

Causal Relation

Extraction

2nd

• Creating a standard benchmarking corpus for evaluating causal relation extraction

• Building an automatic extraction system for event causality

PhD!

Integrated System for Temporal

and Causal Relations

3rd

• Utilizing the interaction between temporal and causal to build an integrated system for temporal and causal relations

Research Methodology

Temporal Relation

Extraction

1st

• Finding ways to improve the current state-of-the-art performance on temporal relation extraction

Temporal Relation Extraction

• Common approach dividing the task:– Identifying the pairs of entities having a temporal link

• Often simplified, rule-based approach:– Main events of consecutive sentences– Pairs of events in the same sentence– An event and a time expression in the same sentence– An event and the document creation time

– Determining the relation types• Often regarded as a classification problem, supervised learning

approach: – Given an ordered pair of entities (e1, e2), the classifier has to

assign a certain label (temporal relation type)

TempEval-3 (2013)

• Shared task on temporal and event processing• Automatic identification of temporal expressions, events, and

temporal relations within a text annotated with TimeML Task F1 Precision Recall

Task A –Temporal Expression 90.30% 93.09% 87.68%Task B – Event Extraction 81.05% 81.44% 80.67%Task ABC – Temporal Awareness 30.98% 34.08% 28.40%Task C1 – Temporal Relations (identification + classification)

36.26% 37.32% 35.25%

Task C2 – Temporal Relations (only classification)

56.45% 55.58% 57.35%

Low performances on temporal relation extraction!

Classifying Temporal Relation Type

• Supervised classification approach• Support Vector Machines (SVMs) algorithm• TempEval-3 dataset for training & evaluation• Feature engineering: event attributes,

temporal signals, event duration, temporal connectives (disambiguation), etc.

• Bootstrapping the training data: inverse relations and closure

“Given a pair of entities (e1, e2), which could either be event-event, event-timex or timex-timex1, the classifier has to assign a certain label (temporal relation type).”

1so few number of pairs, so they are not considered

IS_INCLUDEDBEFORE

EVENT EVENTTIMEX

a BEFORE bb AFTER a

a IBEFORE bb IAFTER a

a BEGINS b b BEGUN_BY a

a ENDS b b ENDED_BY a

a DURING b b DURING_INV a

a INCLUDES bb IS_INCLUDED a

a SIMULTANEOUS b

a IDENTITY b

Microsoft Corp. broke sales records in 2010 when it released its Kinect.

a b

ab

ab

a

a

a

a

a

b

b

b

b

b

Relation event-event event-timextp fp fn tp fp fn

BEFOREAFTERIBEFOREIAFTERBEGINSBEGUN_BYENDSENDED_BYDURINGDURING_INVINCLUDESIS_INCLUDEDSIMULTANEOUSIDENTITY

18663

0000010012

209

18640

0000010024

3335

40104

12001010

394561

6

8214

00000000

27114

00

17700000020

1340

01

1415

56112211

1511

60

Classifying Temporal Relation Type (result)

Can be improved by including causality as a feature?

System F1 Precision RecallTRelProUTTime-1,4UTTime-3,5UTTime-2NavyTime-1NavyTime-2JU-SCE

58.48%56.45%54.70%54.26%46.83%43.92%34.77%

58.80%55.58%53.85%53.20%46.59%43.65%35.07%

58.17%57.35%55.58%55.36%47.07%44.20%34.48%

Best performing system!

Paramita Mirza and Sara Tonelli. 2014. Classifying Temporal Relations with Simple Features. In Proceedings of EACL 2014.

TempEval-3 Task

Research Methodology

Causal Relation

Extraction

2nd

• Creating a standard benchmarking corpus for evaluating causal relation extraction• Annotation guidelines for adding causal information in TimeML

• Building an automatic extraction system for event causality

C-SIGNAL and CLINK TimeML annotation

- EVENT- TIMEX3- SIGNAL- TLINK

+ Causality

- C-SIGNAL- CLINK

• C-SIGNAL → textual elements indicating the presence of causal relations• Prepositions • Conjunctions• Adverbial connectors• Clause-integrated expressions

because of, as a result of, due to, …because, since, so that, … as a result, so, therefore, … the result is, that’s why, …

• CLINK → a directional one-to-one relation where source = causing event and target = caused event(optional) c-signalID = ID of related C-SIGNAL

Causal ConceptsDynamics Model based on Force Dynamics Theory (Talmy, 1988)

• Captures the concept of causality, along with its related concepts, in terms of three dimensions:– the patient tendency for the result– the presence of concordance between the affector and the patient– the occurrence of the result

• Able to distinguish the concept of CAUSE from ENABLE, which is not available in the counterfactual model

• Was tested by linking it with natural language• The causality concepts can be lexicalized as verbs (Wolff and

Song, 2003):– CAUSE-type cause, influence, persuade, prompt, …– ENABLE-type aid, allow, enable, let, …– PREVENT-type block, constrain, prevent, restrain, …

CLINK: explicit causal constructions linking two events (source to target)

• Basic construction– The purchaseS caused the creationT of the current building

– The purchaseS enabled the diversificationT of their business

– The purchaseS prevented a future transferT

• Expressions with affect verbs affect, influence, determine, change

– Ogun CAN crisisS affects the launchT of the All Progressives Congress

• Expressions with link verbs link, lead, depend (on)

– An earthquakeT in North America was linked to a tsunamiS in Japan

• Periphrastic causatives– The blastS prompts the boat to heelT violently

– The oxygenS lets the fire getsT bigger

– The poleS restrains the tent from collapsingT

• Expressions with C-SIGNALs– Iraq said it invadedT Kuwait because of disputesS over oil and money

Manual Annotation

• TimeBank corpus from TempEval-3– with gold events, time expressions and temporal relations

• Inter-annotator agreement (on 5 documents):– 0.844 Dice’s coefficient on C-SIGNAL– 0.73 Dice’s coefficient on CLINK

Annotation EVENT C-SIGNAL CLINKManual 3933 78 144Manual-w/o new events 3872 78 95Automatic (rule-based) 3872 59 52

Paramita Mirza, Rachele Sprugnoli, Sara Tonelli and Manuela Speranza. 2014. Annotating causality in the TempEval-3 corpus. In Proceedings of CAtoCL 2014.

Causal Relation Extraction System

• To be presented at COLING 2014!

Paramita Mirza and Sara Tonelli. 2014. An Analysis of Causality between Events and its Relation to Temporal Information. (to appear) In Proceedings of COLING 2014.

Research Methodology

PhD!

Integrated System for Temporal

and Causal Relations

3rd

• Utilizing the interaction between temporal and causal to build an integrated system for temporal and causal relations

Temporal and Causal: the Interaction

• Temporal constraint of causality:“The cause happened BEFORE the effect”

• Bethard et al. (2008) on corpus analysis:– 32% of CAUSAL relations in the corpus did not have an

underlying BEFORE relation– “The walls were shaking because of the earthquake."

• Rink et al. (2010) makes use of temporal relations as a feature for classification model of causal relations– Causal relation extraction evaluation: F-score 57.9%

Integrated System for Temporal and Causal Relations

Temporal Expressions

Event Extraction

Temporal Relation

Extraction

Temporal & Causal Relation Extraction

Causal Relation Extraction

Cascading? Order?One pass for all? e.g. CRF?Online algorithm?

How?

Thank you!

CAUSE

BEFORE

Paramita closes the presentation so the question-answering session may start.