TIMEN: An Open Temporal Expression Normalisation Resource

TIMENAn Open Temporal Expression

Normalisation Resource

H.Llorens, L.Derczynski, R.Gaizauskas, E. Saquete

Outline ● Introduction: Timex normalisation● Related work● Problem: reinventing the wheel once and again

● Proposal: TIMEN● Evaluation● Conclusions● Further Work

Timex NormalisationTemporal information extraction subtask.

Timex: linguistic expression of a time point or interval.

Normalisation: semantic interpretation of timexes.

Temporal Expression (TIMEX)Linguistics/Variability/RelativityJune 2012, next month, 06/2012this morning 7 a.m.3 days and 3 hoursweekly

Timex normalizationISO 8601/Invariable interpretation2012-062012-05-24T07:00PT3D3HXXXX-XX-WXX

Timex Normalisation (II)Useful for a variety of NLP applications: IR, QA, Summarization, etc. I went to the cinema yesterday. When did he go to the cinema? 2012-05-23 The main advantage of normalisation is having timexes in standard time representations (e.g., gregorian calendar).

event timex

Value: 2012-05-23

Related Work There are many approaches to timex normalisation ● Pre TempEval-2

○ TempEx (2000), GUTime (2005), Chronos (2004), TERSEO (2005), TimexTag (2005), TEA (2006), DANTE (2007)...

● TempEval-2 (2010)○ HeidelTime, TRIPS/TRIOS, TIPSem/TIPSemB...

Similarities and differences● Approaches have slightly different architectures and

show slightly different performances on tests.

● But all the approaches are rule-based and in general they use the same normalization strategies.

● & also require the same parameters to perform the task.○ DCT: document creation time (deictic) (2 days ago: 2012-05-22)○ Reference time: time talked about (anaphoric)

(2 days before: 2012-05-20)○ Tense: Resolution direction (October)

Past (2011-10), Present/Future (2012-10)

The problemReinventing the wheel once and again● Implementation of high-performance approaches is

costly and it is done all the times from the scratch.● all the approaches are similar: rule-based with similar

normalization rules and strategies.● none is meant to be reused and refined by others.

Proposal: TIMENCharacteristics:

● Open philosophy: meant to be reused and refined (even across languages)

● Not only meant for computer scientists:

○ the algorithms (source code) and normalisation rules (db of user-friendly rules with a documented syntax) are separated.

● Independent from other timex processing tasks

● Multi-platform and easy integration

TIMEN Library ArchitectureExample:timex: three days agoDCT:2012-05-24normtext: 3_day_agopattern: Num_TUnit_agoonly 1 rule matches.normalized value: 2012-05-21

Example2:timex: October 202 rules matchingdisambiguation20 probably a dayrather than a yearbecause <32

Rule base sample (English)

TIMEN integration

TIMEN community ● Open-source software:

http://code.google.com/p/timen/ ● Crowd extension of the rule set (interactive

web interface to upload and check new rules): http//timen.org

* new rules only accepted if they improve the performance on the current dataset or new examples (human reviewed). Eg: New Year's Eve

EvaluationExperiments:● Normalization accuracy of TIMEN

● Performance gain in s-o-a approaches by integrating TIMEN

Datasets:● TempEval-2 test-set

(already known for approaches, mainly common dates and duration)

● TimenEval dataset (new, unknown for appr., balanced among different timex types)

Normalisation accuracy

yesterday2012Octoberdailymorning...

TIMEN

2012-05-2320122012-10xxxx-xx-xx2011...

correctcorrectincorrectcorrectincorrect...

normalisationgold timexes

e.g. TOTAL: 100 timexes to normalise e.g. TOTAL: 90 correct normalizations

RESULT: 90/100 --> 90% ACCURACY

Normalisation accuracy ● TIMEN shows a high performance even in this first

version (only 76 rules). ● TimenEval accuracy is lower. This corpus is more

heterogeneous (times/sets) and normalization is more difficult.

TEST SET NORMALISAION ACC

TempEval-2 0.90

TimenEval 0.68

Performance gain

Approach X recognized timexes

built-innormalisationof Approach X

Originalnormalisation

Performance gain = New accuracy - Original accuracy

TIMEN Newnormalisation

Performance gain (TempEval-2) "known data" ● Replacing built-in normalization approaches of the

systems by TIMEN generally improves their performance in TE2 testset.

● Tested (current) versions of the systems may have been developed/updated being aware of this data. What does it happen with data which is new for them?

System built-in norm. TIMEN norm. Err. Redution

TIPSemB 0.83 0.89 35%

HeidelTime 0.94 0.94 0%

TERNIP 0.76 0.92 66%

Performance gain (TimenEval) "new data" ● Using new data, the built-in approaches performance

decreases in general.● TIMEN favours the normalization performance for all the

systems.

System built-in norm. TIMEN norm. Err. Redution

TIPSemB 0.57 0.67 23%

HeidelTime 0.72 0.74 7%

TERNIP 0.70 0.72 66%

Conclusions ● We presented an open tool for timex normalisation:

TIMEN. ● ADVANTAGES:

○ High performance (above recent approaches).○ Easily integrated in any timex recognition

approach.○ Can be improved by the community (open philosophy),

and avoids re-development from scratch.○ Available: http://timen.org and Google code

Further Work ● Community-based extension and refinement

of TIMEN (rulebase). ● Extensive evaluation of TIMEN in various

languages (Spanish, Chinese, Italian and Danish).

TIMEN: An Open TIMEX Normalisation Resource

THANK YOU!QUESTIONS?

http://timen.org

H.Llorens, L.Derczynski, R.Gaizauskas, E. Saquete