RETRIEVING RELEVANT AND INTERESTING TWEETS DURING LIVE TELEVISION BROADCASTS Rianne Kaptein, Yi Zhu, Gijs Koot, Judith Redi and Omar Niamut | TNO & TU Delft
Aug 15, 2015
RETRIEVING RELEVANT AND INTERESTING TWEETS DURING LIVE TELEVISION BROADCASTSRianne Kaptein, Yi Zhu, Gijs Koot, Judith Redi and Omar Niamut | TNO & TU Delft
LIVE EVENT: VOLVO OCEAN RACE
2 | Retrieving relevant and interesting tweets during live television broadcasts
RELEVANT & INTERESTING TWEETS
3 | Retrieving relevant and interesting tweets during live television broadcasts
RESEARCH QUESTION
o How can we design an event profiler that generates a set of query terms to
retrieve relevant and interesting tweets during an event?
o Compared to following the event title / hashtag
o Not only consider relevancy, but also interestingness
ADAPTIVE EVENT PROFILER
Event Title
from TV guide
(Initial) Search
Keywords
Search keywords
to addRule based preprocessing, e.g. remove words like “live” and dates
Set of Twitter Search Results
Search / Stream search keywords using the Twitter API
Show tweets in display /Write tweets to database/
Relevance Feedback
Repeat regularly during event to pick up new keywords
ADAPTIVITY
To cope with the dynamically changing nature of Twitter we will use a moving
window for data collection
Each n minutes we analyse the last n minutes of activity on Twitter
To avoid topic drift, each iteration will be based on the initial search keywords,
all previous search keywords will be considered again
KEYWORD TYPES
Main keyword: Starting point, event title or hashtag
Relevant keywords: Tweets containing a relevant keyword are considered
relevant, and are shown to the user in the interface
Candidate keywords: Potentially relevant keywords, are followed during time
period t to see if they can be promoted to a relevant keyword in the next
timeperiod t+1
7 | Retrieving relevant and interesting tweets during live television broadcasts
ADAPTIVE EVENT PROFILER STEPS (PART 1)
1. In period t follow the main keywords and the relevant keywords to retrieve
a set of tweets.
2. From the set of tweets containing the main keywords, extract the top n
most frequent terms, hashtags and usernames. These terms are the new
candidate keywords.
3. In period t + 1 follow the main keywords, the relevant keywords and the top
n candidate keywords to retrieve a set of tweets.
4. For each keyword generate a Maximum Likelihood Estimation language
model based on the tweets that contain the keyword
5. For each candidate and relevant keywords, assess the similarity of its
language model and that of the main keywords using Kullback-Leibler
divergence.
ADAPTIVE EVENT PROFILER STEPS (PART 2)
6. Select the k keywords (either candidate or relevant) most similar to the
main keyword, and designate them as relevant keywords for the upcoming
period t + 2.
7. In period t + 2 follow the main keyword and the k relevant keywords to
select relevant tweets.
8. Go to 1.
EXPERIMENT
Event: Sochi Winter Olympics 2014
Broadcasted live on Dutch National Television
Sports event: Speed skating finals, popular sport in The Netherlands
10 | Retrieving relevant and interesting tweets during live television broadcasts
EXPERIMENT:SET-UP
2 Sessions with 10 participants
Participants watched 25 minute live broadcast.
3 keyword levels (high, middle, low)
Click to like a tweet or not
Post-experiment questionnaire to evaluate the relevancy and interestingness
of a subset of the tweets displayed
EXPERIMENT: RESULTS CLICKS
5.959 clicks
1.729 different tweets
Average of 300 clicks per user
41/44 tweets evaluated in questionnaire
30 tot 35% of the tweets were liked
13 | Retrieving relevant and interesting tweets during live television broadcasts
# tweets # positive fraction
Tweets containing main keyword
721 0,42
Tweets not containing main keyword
1008 0,32
EXPERIMENT: RESULTS QUESTIONNAIRE
15 | Retrieving relevant and interesting tweets during live television broadcasts
EXPERIMENT: RESULTS QUESTIONNAIRE
Higly relevant terms:
#sochi2014
#10km
Schaatsen
Kleibeuker
Usernames skaters
Irrelevant terms:
Unrelated usernames
Relevant terms:
zilver
vrouwen
16 | Retrieving relevant and interesting tweets during live television broadcasts
IRRELEVANT TWEETS
Found keyword is used in a different context
Word can mean something in another language
Keyword is too general
18 | Retrieving relevant and interesting tweets during live television broadcasts
UNINTERESTING TWEETS
Simple statements like: “Watching ….”,
“Looking forward to ..”
Commercial tweets
Direct messages part of a (private) conversation
19 | Retrieving relevant and interesting tweets during live television broadcasts
INTERESTING TWEETS
Tweets containing a picture or movie
Tweets from participants and visitors of
the event
Inside information
Fun facts
20 | Retrieving relevant and interesting tweets during live television broadcasts
CONCLUSIONS
• Event profiler retrieves significantly more likeable tweets than following a
single keyword
• Without introducing too much noise.
• Relevant tweets are not necessarily interesting, but interesting tweets are
usually relevant.