Real-time Social Media Analytics through Semantic Annotation and Linked Open Data Diana Maynard, Mark Greenwood, Ian Roberts, George Windsor and Kalina Bontcheva What sentiments are expressed? Collector Backup Processor G A T E Embedded G A T E ímir M Queue Real-Time Big Data Analysis Data collection via Twitter streaming API Documents stored as JSON and annotated via GCP Documents indexed via MIMIR Search and visualisation via MIMIR/Prospector • • • • What kind of questions can we answer? Find all positive sentiment expressions about the “UK economy” theme in tweets written by Labour candidates for constituencies in Greater London”. Which political issues do people care about the most? Which ones do they engage with and why? Which party's manifesto is the most positive, and how does this influence public voting? • • • Sentiment visualisation: where did Miliband go wrong? Main topics and related sentiment of discussion on Twitter generated by the #Leadersdebate, between the leaders of the 7 main parties. Milliband was negative about the economy and the NHS • • Topic visualisation Where did people talk about the economy? • Research partially supported by the European Union/ EU under the Information and Communication Technologies (ICT) theme of the 7th Framework Programme for R&D (FP7) DecarboNet (610829) http://www.decarbonet.eu and Nesta http://nesta.org.uk Semantics is Crucial! Information isn't always explicit in the text We need semantic information in order to aggregate knowledge, e.g. all tweets from MPs in a certain geographical region • • Climate change is less boring than other political topics Climate change isn't mentioned frequently by politicians But it shows high levels of engagement: - frequent retweets - contains sentiment - optimistic - contains URLs and @mentions People engage more with climate change than other topics because they feel they can actively contribute • • • Analysing Social Media Real-time social media analytics needed by journalists, companies, governments etc. Processing and understanding text from social media is hard: messages are often brief, ungrammatical, and noisy Interpretation is heavily dependent on temporal, geographical, and social network context • • • Political Futures Tracker Sentiment Analysis Mimir Semantic Index Linked data (DBpedia, YourNextMP, NUTS) Topic Detection MP / Candidate Recognition Tokenizer Normalizer Part-of- Speech Tagger Named Entity Recognizer News G A T E g e n e r a l a r c h i t e c t u r e f o r t e x t e n g i n e e r i n g -20 0 20 40 60 80 100 120 UK economy Media and communications Wales Scotland Regulation reform Children and young people Higher education Borders and immigration National Health Service Public health Number of Tweets Expressing Negative Sentiment Positive Sentiment #ge2015 #leadersdebate #bbcdebate #snp #labour #ukip #ge15 #plaid15 #votesnp #snpout #registertovote #generationvote #cosplay #cameron #voteconservative #bbcdp #election2015 #nhs #manifesto #tories #conservative #libdems #janner #scotland #conservativemanifesto #conservatives #bbcqt #glasgow #genderequality #generalelection #miliband #greensurge #bbcbreakfast #votesnpgetsexy #vote #carecrisis #peoplenotpolls #libdemmanifesto #lordjanner #askbbchugh #1milliondisabledscovotes #believeinbritain #tory #sexysocialism #chickendave #indyref #csainquiry #ukpoverty #murdoch #stillyes #votelabour #ukipmanifesto #leanne15 #ukhousing #callclegg #greens #covdebate #votegreen2015 #votesnpgetscotland #skills #manufacturing #mymanifesto #uk #angryed #victorialive #challengersdebate #invisibledave #voteukip #righttobuy #cameronmustgo #modernpolitics #tacticalvote #hustings #sheffieldissuper #housing #5moreyearsfordc #wheresdave #teamnigel #plaidcymru #hookmeaduck #streatham #massdebate #signedbus #skynews #ncapm #clegg #banksy #votematch #toriesout #london #iamanimmigrant #politics #welfare #1pun #gove #indyref2 #ids #subctour #green #anarchy #novote #thechoice #news #wales #domesticviolence #tvdebate #labourmanifesto #paedomps #everydaysexism #property #osa #awkward #battlefornumber10 #youhadonejob #electionyorks #austerity #homesforbritain #fracking #olsx #voteneilhay #occupy #sheffield #dementiapromise #libdem #redtoriesout #scotland2015 #newsnight #leeds #abcnews24 #csa #manchester #beyondtheballot #greenparty #ddj #uknews #mentalhealth #coalition #greenmanifesto #policychallenge #llanelli #wato #ukelection2015 #farage #election #projectfear2 #dataviz #womeninpolitics #scotnight #bbc #nigelfarage #ukpolitics #osborne #allianceparty #elections #r4today #ldge2015 #gen2015 #values #votegreen #eu #workingforwales #fail #bbcnews #blukip #emptychair