Leveraging Learning To Rank in an Optimization Framework for Timeline Summarization Giang Binh Tran, Anh Tuan Tran, Nam Khanh Tran, Mohammad Alrifai, Nattiya Kanhabua L3S Research Center & University of Hannover, Germany 1 SIGIR Workshop TAIA’13, Dublin August 1, 2013
14
Embed
Leveraging Learning To Rank in an Optimization Framework for Timeline Summarization
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Leveraging Learning To Rank in an Optimization Framework for Timeline Summarization
Giang Binh Tran, Anh Tuan Tran, Nam Khanh Tran, Mohammad Alrifai, Nattiya Kanhabua
L3S Research Center & University of Hannover, Germany
1
SIGIR Workshop TAIA’13, Dublin August 1, 2013
Timeline Summarization
2
News Topic: Arab Spring What and how did it happen? A summarization with the temporal structure (list of daily key events) Example:
• 11 Feb 2011: Egypt President Hosni Mubarak resigned • 15 Feb 2011: protests broke out against Muammar
Gaddafi’s regime • 03 Mar 2011: Egypt Prime Minister Ahmed Shafik resigned
Example
Day Summaries of key events
Important dates 3
Related work • Timeline Summarization:
• Chieu et al. (SIGIR’04): • burstiness + interest score (~sum TFxIDF similarity to
neighbor sentences) • Yan et al. (SIGIR’11):
• Topic relevancy + coverage + coherence + diversity based on word distribution
Unsupervised manners
Our approach: learn from expert-created timeline summaries, and optimize with
different criteria
4
Sentence Ranking Model
TIMELINE
Date Summary
2011-08-29 Eni CEO meets with members of the rebel government.
2011-09-08 Gaddafi vows to fight on
……. ……
Learning Algorithms
Manually created
Timelines
Optimization
Rs
Ranked Sentences
5
Learning to Rank sentences • Assumption
• Day summaries are created from input news articles (e.g. BBC timelines BBC news articles)
• Generate Training Data automatically Relevance R(s) ~ Textual Similarity (s, DS ) A sentence with higher similarity to Day Summary (DS) is more likelihood to be selected as a part of summary
• Feature extraction
Surface: length, stop/non-stop words,#pronouns, position. Coherence: #temporal/logical/causal signals Topic: sum/avg TFIDF, logodds, cross entropy, semantic similarity to document abstract Temporal: popularity, has temporal expression Event: probability to describes the main events in term of top word pairs
6
Optimize Timeline Generation N-gram-based computation • Novelty Avoid duplication in a day summary when selecting s • Continuity Generate timeline as a flow of information (connecting the dots between day summaries) Maximize Using dynamic programming
7
Evaluation Dataset: Timeline17 (www.l3s.de/~gtran/timeline) 4650 articles collected from wellknown news agencies (e.g., BBC, CNN,.) 17 Timelines from 9 Topics : BP Oil Spill, Haiti Earthquake, H1N1, Financial Crisis, Lybian War, ... Leave-one-out strategy „In-house“ experiment: timeline generated from BBC news should be compared against BBC expert-generated timeline
8
9
Metric ROUGE n-gram based measurement (overlapped n-grams between generated day summary and expert-created day summaries - Precision/Recall/F-measure) ROUGE-1 uses uni-grams, ROUGE-2 uses bi-grams, ROUGE-S* uses skipped bi-grams
Chieu et al. (Chieu et al. SIGIR 2004) MEAD: traditional multi-document summarization system ETS (Yan et al. SIGIR 2011)
10
Michael Jackson Death trial, example
2009-07-28 Dr Murray 's home is also raided . 2011-05-02 The trial is delayed again , as Dr Murray 's lawyers ask for extra time to prepare for new prosecution witnesses . ----------------------- 2009-07-29 Court documents filed in Nevada show that Dr Murray is heavily in debt , owing more than $ 780,000 in judgements against him and his medical practice, outstanding mortgage payments on his house , child support and credit cards .
11
BBC Timeline (ground truth) 2009-07-28 (Ok) Police raid Jackson doctor 's home 2011-05-02 In Los Angeles , lawyers for Dr Conrad Murray had asked for a delay to prepare for new prosecution witnesses . ---------------------- 2009-07-29 (Bad) Michael Flanagan of the DEA describes the operation Police have searched the Las Vegas home and offices of Michael Jackson 's doctor as part of a manslaughter investigation into the singer 's death .
Ours
H1N1 – Continuity v.s. NonContinuity
12
Without Continuity 2009-04-25 The World Health Organisation has warned countries to be on alert for any unusual flu outbreaks after a swine flu virus was implicated in possibly dozens of human deaths in Mexico . 2009-04-26 The World Health Organisation said at least 81 people had died from severe pneumonia caused by the flu - like illness in Mexico .
With Continuity
2009-04-25 The World Health Organisation has warned countries to be on alert for any unusual flu outbreaks after a swine flu virus was implicated in possibly dozens of human deaths in Mexico . 2009-04-26 The influenza strain that has struck Mexico and the United States involves , in many cases, a never-before-seen strain of the H1N1 virus ..
Thank you very much!
13
14
Novelty computation (s: sentence, S: set of sentences)
Continuity computation (s: sentence, DS (d_i-1_) is the previous day summary