Making More Sense Out of Social Data

Making More Sense Out of Social Data Harith Alani h+p://people.kmi.open.ac.uk/harith/

@halani

harith-alani

@halani

4th Workshop on Linked Science 2014— Making Sense Out of Data (LISC2014) ISWC 2014 -‐ Riva del Garda, Italy

Topics

•  Social media monitoring"•  Behaviour role analysis"•  Semantic sentiment "•  Engagement in microblogs"•  Cross platform and topic studies"•  Semantic clustering"•  Application examples"

Take home messages •  Social media has many more challenges and

opportunities to offer"

•  Fusing semantics and statistical methods is gooood"

•  Studying isolated social media platforms is baaaad … or not good enough … anymore!"

Sociograms •  Capturing and graphing social

relationships"•  Moreno founder of sociograms and

sociometry"•  Assessing psychological well-being

from social configurations of individuals and groups"

h+p://diana-‐jones.com/wp-‐content/uploads/EmoRons-‐Mapped-‐by-‐New-‐Geography.pdf

Friendship Choices Among Fourth Graders (from Moreno, 1934, p. 38

Computational Social Science

Behaviour role Analysis “A field is emerging that leverages the capacity to collect and analyze data at a scale that may reveal pa+erns of individual and group behaviours.”

Original slide by Markus Strohmaier h+p://gking.harvard.edu/files/LazPenAda09.pdf

“what does exisRng sociological network theory, built mostly on a foundaRon of one-‐Rme “snapshot” data, typically with only dozens of people, tell us about massively longitudinal data sets of millions of people .. ?”

!

Social semantic linking …. in 2003

Alani, H.; Dasmahapatra, S.; O'Hara, K.and Shadbolt, N. IdenRfying communiRes of pracRce through ontology network analysis. IEEE Intelligent Systems, 18(2) 2003.

•  Domain ontologies

•  SemanRcs for integraRng people, projects, and publicaRons

•  IdenRfy communiRes of pracRce

•  Browse evoluRon of social relaRonships and collaboraRons

Linking scientists …. in 2005

•  Who is collaboraRng with whom?

•  How funding programmes impacted collaboraRons over Rme?

Alani, H.; Gibbins, N.; Glaser, H.; Harris, S. and Shadbolt, N. Monitoring research collaboraRons using semanRc web technologies. ESWC, Crete, 2005.

data sources

gatherers and

mediators

ontology knowledge repository (triplestore)

applicaRons

Bigger data, greater sociograms

Social Media

Jan 29, 2013

In-house Social Platforms

Tools for monitoring social networks

Reputation Monitoring

•  http://www.robust-project.eu/videos-demos "

Challenges and Opportunities •  Integration"

–  How to represent and connect this data?"

•  Behaviour"–  How can we measure and

predict behaviour?"–  Which behaviours are good/bad

in which community type?"

•  Community Health"–  What health signs should we

look for? "–  How to predict this health?"

•  Engagement"–  How can we measure and

maximise engagement? "

•  Sentiment"–  How to measure it? "–  Track it towards entities and

contexts? "

Patterns

SemanRc Web & Linked Data

SemanRc SenRment Analysis

Macro/Micro Behaviour Analysis

ini#ators)

lurkers)

followers)

leaders)

Community Engagement

Table 1: Correlation Coefficients of dimensions

Dispersion Engagement Contribution Initiation Quality PopularityDispersion 1.000 0.277 0.168 0.389 0.086 0.356Engagement 0.277 1.000 0.939** 0.284 0.151 0.926**Contribution 0.168 0.939** 1.000 0.274 0.086 0.909**Initiation 0.389 0.284 0.274 1.000 -0.059 0.513Quality 0.086 0.151 0.086 -0.059 1.000 0.065Popularity 0.356 0.926** 0.909** 0.513 0.065 1.000

Figure 7: Cumulative density functions of each dimension showingthe skew in the distributions for initiated and in-degree ratio

same forum and do not deviate away, at the other ex-treme very few users are found to post in a large rangeof forums. For initiated (initiation) and in-degree ratio(popularity) the density functions are skewed towardslow values where only a few users initiate discussionsand are replied to by large portions of the community.Average points per post (quality) is also skewed to-wards lower values indicating that the majority of usersdo not provide the best answers consistently.These plots indicate that feature levels derived from

these distributions will be skewed towards lower values,for instance for initiated the definition of high for thisfeature is anything exceeding 1.55x10−5.The distribution of each dimension is shown in Fig-

ure 8 for each of the 11 induced clusters. We assessthe distribution of each feature for each cluster againstthe levels derived from the equal-frequency binning ofeach feature, thereby generating a feature-to-level map-

Figure 8: Boxplots of the feature distributions in each of the 11 clus-ters. Feature distributions are matched against the feature levels de-rived from equal-frequency binning

ping. This mapping is shown in Table 2 where certainclusters are combined together as they have the samefeature-level mapping patterns (i.e. 5,7 and 8,9). Wethen interpreted the role labels from these clusters, andtheir subsequent patterns, as follows:

• 0 - Focussed Expert Participant: this user typeprovides high quality answers but only within se-lect forums that they do not deviate from. Theyalso have a mix of asking questions and answeringthem.

• 1 - Focussed Novice: this user is focussed within afew select forums but does not provide good qual-ity content.

• 2 - Mixed Novice: is a novice across a mediumrange of topics

6

StaRsRcal Analysis

Technologies

MODELLING AND LINKING SOCIAL MEDIA DATA

June 25, 2013

•  SIOC is an ontology for representing and integrating data from the social web"

•  Simple, concise, and popular"

sioc-project.org

Semantically-Interlinked Online Communities (SIOC)

SRll seeking the one size that’ll fit all

SIOC for Discussion forums •  SIOC is well

tailored to fit discussion forum communities"

•  Needs extension to fit other communities, such as microblogs and Q&A"

Twitter in SIOC •  Microblogs"

•  No forum structure"

IBM Connections in SIOC

SAP Community Network in SIOC

BEHAVIOUR ROLES

h+p://www.smrfoundaRon.org/wp-‐content/uploads/2008/12/disRnguishing-‐a+ributes-‐of-‐social-‐roles.png

Why we monitor behaviour? •  Understand role of people in a community

•  Monitor impact of behaviour on community evolution

•  Forecast community future

•  Learn which behaviour should be encouraged or discouraged

•  Find the best mix of behaviour to increase engagement in an online community

•  See which users need more support, which ones should be confined, and which ones should be promoted

Linking networks

27/

<?xml version="1.0"?>!<rdf:RDF! xmlns="http://tagora.ecs.soton.ac.uk/schemas/tagging#"! xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"! xmlns:xsd="http://www.w3.org/2001/XMLSchema#"! xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"! xmlns:owl="http://www.w3.org/2002/07/owl#"! xml:base="http://tagora.ecs.soton.ac.uk/schemas/tagging">! <owl:Ontology rdf:about=""/>! <owl:Class rdf:ID="Post"/>! <owl:Class rdf:ID="TagInfo"/>! <owl:Class rdf:ID="GlobalCooccurrenceInfo"/>! <owl:Class rdf:ID="DomainCooccurrenceInfo"/>! <owl:Class rdf:ID="UserTag"/>! <owl:Class rdf:ID="UserCooccurrenceInfo"/>! <owl:Class rdf:ID="Resource"/>! <owl:Class rdf:ID="GlobalTag"/>! <owl:Class rdf:ID="Tagger"/>! <owl:Class rdf:ID="DomainTag"/>! <owl:ObjectProperty rdf:ID="hasPostTag">! <rdfs:domain rdf:resource="#TagInfo"/>! </owl:ObjectProperty>! <owl:ObjectProperty rdf:ID="hasDomainTag">! <rdfs:domain rdf:resource="#UserTag"/>! </owl:ObjectProperty>! <owl:ObjectProperty rdf:ID="isFilteredTo">! <rdfs:range rdf:resource="#GlobalTag"/>! <rdfs:domain rdf:resource="#GlobalTag"/>! </owl:ObjectProperty>! <owl:ObjectProperty rdf:ID="hasResource">! <rdfs:domain rdf:resource="#Post"/>! <rdfs:range =…!

Linking people via sensors, social media, papers, projects

•  Integration of physical presence and online information"•  Semantic user profile generation"•  Logging of face-to-face contact"•  Social network browsing"•  Analysis of online vs offline social networks"Alani, H.; Szomszor, M.; Ca+uto, C.; den Broeck, W.; Correndo, G. and Barrat, A.. Live social semanRcs. ISWC, Washington, DC, 2009

0"

0.2"

0.4"

0.6"

0.8"

1"

1.2"

1" 5" 9" 13" 17" 21" 25" 29" 33" 37" 41" 45"

H.Index"F2F"Degree"F2F"Strength"

Online+offline social networks

•  What’s your social configura-on? •  What does it say about you? •  And what you’ll become?

Barrat, A.; C., Ca+uto; M., Szomszor; W., Van den Broeck and Alani, H. Social dynamics in conferences: analyses of data from the Live Social SemanRcs applicaRon. ISWC, Shanghai, China, 2010.

h+p://www.tehowners.com/info/Popular%20Culture%20&%20Social%20Media/Online%20CommuniRes.jpg

Clustering for identifying emerging roles

–  Map the distribution of each feature in each cluster to a level (i.e. low, mid, high)

–  Align the mapping patterns with role labels

Table 1: Correlation Coefficients of dimensions

Dispersion Engagement Contribution Initiation Quality PopularityDispersion 1.000 0.277 0.168 0.389 0.086 0.356Engagement 0.277 1.000 0.939** 0.284 0.151 0.926**Contribution 0.168 0.939** 1.000 0.274 0.086 0.909**Initiation 0.389 0.284 0.274 1.000 -0.059 0.513Quality 0.086 0.151 0.086 -0.059 1.000 0.065Popularity 0.356 0.926** 0.909** 0.513 0.065 1.000

Figure 7: Cumulative density functions of each dimension showingthe skew in the distributions for initiated and in-degree ratio

same forum and do not deviate away, at the other ex-treme very few users are found to post in a large rangeof forums. For initiated (initiation) and in-degree ratio(popularity) the density functions are skewed towardslow values where only a few users initiate discussionsand are replied to by large portions of the community.Average points per post (quality) is also skewed to-wards lower values indicating that the majority of usersdo not provide the best answers consistently.These plots indicate that feature levels derived from

these distributions will be skewed towards lower values,for instance for initiated the definition of high for thisfeature is anything exceeding 1.55x10−5.The distribution of each dimension is shown in Fig-

ure 8 for each of the 11 induced clusters. We assessthe distribution of each feature for each cluster againstthe levels derived from the equal-frequency binning ofeach feature, thereby generating a feature-to-level map-

Figure 8: Boxplots of the feature distributions in each of the 11 clus-ters. Feature distributions are matched against the feature levels de-rived from equal-frequency binning

ping. This mapping is shown in Table 2 where certainclusters are combined together as they have the samefeature-level mapping patterns (i.e. 5,7 and 8,9). Wethen interpreted the role labels from these clusters, andtheir subsequent patterns, as follows:

• 0 - Focussed Expert Participant: this user typeprovides high quality answers but only within se-lect forums that they do not deviate from. Theyalso have a mix of asking questions and answeringthem.

• 1 - Focussed Novice: this user is focussed within afew select forums but does not provide good qual-ity content.

• 2 - Mixed Novice: is a novice across a mediumrange of topics

6

Table 2: Mapping of cluster dimensions to levels

Cluster Dispersion Initiation Quality Popularity0 L M H L1 L L L L2 M H L H3 H H H H4 L H H M5,7 H H L H6 L H M M8,9 M H H H10 L H M H

• 3 - Distributed Expert: an expert on a variety oftopics and participates across many different fo-rums

• 4 - Focussed Expert Initiator: similar to cluster0 in that this type of user is focussed on certaintopics and is an expert on those, but to a large ex-tent starts discussions and threads, indicating thathis/her shared content is useful to the community

• 5.7 - Distributed Novice: participates across arange of forums but is not knowledgeable on anytopics

• 6 - Focussed Knowledgeable Member: con-tributes to only a few forums, has medium-levelexpertise (i.e. he/she is neither an expert nor anovice) and has medium popularity

• 8,9 - Mixed Expert: medium-dispersed user whoprovides high-quality content

• 10 - Focussed Knowledgeable Sink: focusseduser who has medium-level expertise but who getsa lot of the community replying to them - hence asink. Differs from cluster 6 in terms of popularity.

6. Analysis: Community Health

Deriving a community’s role composition providescommunity operators and hosts with amacro-level viewof how their community is operating and how it is func-tioning. Understanding what is a healthy and unhealthycomposition in a community involves analysing how agiven role composition has been associated with com-munity activity, interaction or some other measure in thepast and reusing that knowledge. Forums and communi-ties operating within the same platform may also differsuch that what turns a community healthy in one loca-tion may be different from another. In this section wedescribe how community analysis is possible throughour presented approach to derive the role compositionof a community using semantic rules.

6.1. Experimental Setup

To demonstrate the utility of our approach we anal-ysed each of the 33 SAP communities from 2009through to 2011. Figure 9 shows how our dataset wasdivided into the tuning section - i.e. the first half of2008 in which we derived our clusters and aligned themto roles (as described in Section 5) - and the analysissection. We began with the 1st January 2009 as our col-lect date by taking a feature window 6 months prior tothis date (going back to the 2nd half of 2008) in whichwe measured the behaviour dimensions for each com-munity’s users. In order to gauge the role compositionin a community over time we move our collect date onone week at a time and use the 6-months prior to thisdate as our feature window. As Figure 9 demonstrateswe repeat this process until we reach 2011.

Figure 9: Windows used for a) tuning of the clusters and the derivationof roles and b) the analysis of community health. Role compositionis derived every week from 2009 onwards using a 6-month windowgoing back from the collection date.

By measuring the behaviour dimensions of individ-ual users in individual communities we are able to inferthe roles of the users using the semantic rules describedin Section 4. This provides a micro-level assessment ofthe roles that individual users assume. We can then lookat the macro-level by deriving the role composition of agiven community at a given point in time by measuringhow many users have a specific role. Such role compo-sition analysis allows for predictions to then be made.To demonstrate the application of such analysis we per-formed three distinct experiments (each designed to ex-plore one of our three aforementioned research ques-tions):

1. Composition Analysis: assesses the average rolecomposition in each community and clusters thembased on the compositions. We also pick out eachcommunity’s most popular role and measure whatpercentage of the community that role covers.

2. Activity Increase/Decrease: we perform a binaryclassification task such that at timestep t = k + 1we predict whether the community’s activity (i.e.number of posts) has increased or decreased since

7

•  1 - Focussed Novice: focussed within a few select forums but does not provide good quality content.

•  2 - Mixed Novice: a novice across a medium range of topics

•  3 - Distributed Expert: expert on a variety of topics and participates across many different forums

….

Mapping of cluster dimensions to levels

Encoding Roles in Ontologies with SPIN

•  Bottom Up analysis"–  Every community member is

classified into a “role”"–  Unknown roles might be identified"–  Copes with role changes over time "

iniRators

lurkers

followers

leaders

Structural, social network, reciprocity, persistence, participation

Feature levels change with the dynamics of the community

Associations of roles with a collection of feature-to-level mappings e.g. in-degree -> high, out-degree -> high

Run rules over each user’s features and derive the community role composition

Behaviour role extraction from Social Media Data

Angeletou, S; Rowe, M, and Alani, H. Modelling and analysis of user behaviour in online communiRes. ISWC 2011, Bonn, Germany

Correlation of behaviour roles with community activity

Forum on CommuRng and Transport Forum on Rugby Forum on Mobile Phones and PDAs

•  How certain behaviour roles impact activity in different community types?"

Community types •  So do communities of different types behave differently?

•  Analysed IBM Connections communities to study participation, activity, and behaviour of users

•  Compare exhibited community with what users say they use the community for –  Does macro behaviour match micro needs?

Community types Community

Wiki Page Blog Post Forum Thread

Wiki Edit Blog Comment Forum Reply

Bookmark Tag

File

§  Data consists of non-private info on IBM Connections Intranet deployment

§  Communities: §  ID §  Creation date §  Members §  Used applications

(blogs, Wikis, forums)

§  Forums: §  Discussion threads §  Comments §  Dates §  Authors and

responders

Community types •  Muller, M. (CHI 2012) identified five distinct community types in

IBM Connections:"–  Communities of Practice (CoP): for sharing information and network"–  Teams: shared goal for a particular project or client"–  Technical Support: support for a specific technology"–  Idea Labs Communities: for focused brainstorming "–  Recreation Communities: recreational activities unrelated to work.

"•  Our data consisted of 186 most active communities:"

–  100 CoPs, 72 Teams, and 14 Technical Support communities "–  No Ideas of Recreation communities"

Behaviour roles in different community types

Macro

Rowe, M. Fernandez, M., Alani, H., Ronen, I., Hayes, C., Karnstedt, M.: Behaviour Analysis across different types of Enterprise Online Communities. WebSci 2012

•  Members of Team communities are more engaged, popular, and initiate more discussions

•  Technical Support community members are mostly active in a few communities, and don’t initiate or contribute much!

•  CoP members are active across many communities, and contribute more

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Churn Rate

FPR

TPR

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

User Count

FPR

TPR

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Seeds / Non−seeds Prop

FPR

TPR

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Clustering Coefficient

FPR

TPR

•  Machine learning models to predict community health based on compositions and evolution of user behaviour

•  Churn rate: proportion of community leavers in a given time segment.

•  User count: number of users who posted at least once.

•  Seeds to Non-seeds ratio: proportion of posts that get responses to those that don’t

•  Cluster coefficient: extent to which the community forms a clique.

Health categories

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Churn Rate

FPR

TPR

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

User Count

FPR

TPR

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Seeds / Non−seeds Prop

FPR

TPR

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Clustering Coefficient

FPR

TPR

False Positive Rate

False Positive Rate False Positive Rate

False Positive Rate

True

Pos

itive

Rat

e Tr

ue P

ositi

ve R

ate

True

Pos

itive

Rat

e Tr

ue P

ositi

ve R

ate

The fewer Focused Experts in the community, the more posts will received a reply! There is no “one size fits all” model!

Behaviour roles and community health

Rowe, M. and Alani, H. What makes communiRes Rck? Community health analysis using role composiRons. SocialCom 2012, Amsterdam, The Netherlands.

SEMANTIC SENTIMENT ANALYSIS

Semantic sentiment analysis on social media

•  Range of features and statistical classifiers have been used in social media sentiment analysis in recent years

•  Semantics have often been overlooked

–  Semantic Features

–  Semantic Patterns

•  Semantic concepts can help determining sentiment even when no good lexical clues are present

Sentiment Analysis

hate negative honest positive inefficient negative Love positive …

Sentiment Lexicon

I hate the iPhone

I really love the iPhone

Lexical-Based Approach

Learn Model

Apply Model

Naïve Bayes, SVM, MaxEnt , etc.

Training Set

Test Set

Model

Machine Learning Approach

Semantic Concept Extraction •  Extract semantic concepts from tweets data and incorporate

them into the supervised classifier training.

Fig. 1. Measuring correlation of semantic concepts with negative/positive sentiment. These se-mantic concepts are then incorporated in sentiment classification.

OpenCalais and Zemanta. Their experimental results showed that AlchemyAPI per-forms best for entity extraction and semantic concept mapping. Our datasets consist ofinformal tweets, and hence are intrinsically different from those used in [10]. There-fore we conducted our own evaluation, and randomly selected 500 tweets from the STScorpus and asked 3 evaluators to evaluate the semantic concept extraction outputs gen-erated from AlchemyAPI, OpenCalais and Zemanta.

No. of Concepts Entity-Concept Mapping Accuracy (%)Extraction Tool Extracted Evaluator 1 Evaluator 2 Evaluator 3AlchemyAPI 108 73.97 73.8 72.8Zemanta 70 71 71.8 70.4OpenCalais 65 68 69.1 68.7Table 2. Evaluation results of AlchemyAPI, Zemanta and OpenCalais.

The assessment of the outputs was based on (1) the correctness of the extractedentities; and (2) the correctness of the entity-concept mappings. The evaluation resultspresented in Table 2 show that AlchemyAPI extracted the most number of conceptsand it also has the highest entity-concept mapping accuracy compared to OpenCalaisand Zematna. As such, we chose AlchemyAPI to extract the semantic concepts fromour three datasets. Table 3 lists the total number of entities extracted and the number ofsemantic concepts mapped against them for each dataset.

STS HCR OMDNo. of Entities 15139 723 1194No. of Concepts 29 17 14

Table 3. Entity/concept extraction statistics of STS, OMD and HCR using AlchemyAPI.

Impact of adding semantic features •  Incorporating semantics increases accuracy against the

baseline by:

–  6.5% for negative sentiment,

–  4.8% for positive sentiment

–  F1 = 75.95%, with 77.18% Precision and 75.33% Recall

Saif, H., He, Y. and Alani, H. SemanRc senRment analysis of twi+er. ISWC 2012, Boston, US.

Destroy(((Invading(Germs((

Nega%ve'Concept'Nega%ve'

•  OK, but what about such cases?

•  Can semantics help?

Semantic Pattern Approaches •  Apply syntac-c and seman-c processing techniques

•  Use external semanRc resources (e.g. Dbpedia, Freebase) to idenRfy semanRc concepts in Tweets

Trojan Horse

Threat

Hack

Code

Malware

Program

Dangerous

Harm

Spyware

•  Extract clusters of similar contextual semanRcs and senRment, and use as pa+erns in senRment analysis

Features

MaxEnt Classifier

Accuracy F-Measure

Minimum Maximum Average Minimum Maximum Average

Syntactic

Twitter Features -0.23 3.91 1.24 -0.25 4.53 1.62POS -0.89 2.92 0.79 -0.91 5.67 1.25Lexicon -0.44 4.23 1.30 -0.38 5.81 1.83Average -0.52 3.69 1.11 -0.52 5.33 1.57

Semantic

Concepts -0.22 2.76 1.20 -0.40 4.80 1.51LDA-Topics -0.47 3.37 1.20 -0.68 6.05 1.68SS-Patterns 0.70 9.87 3.05 1.23 9.78 3.76

Average 0.00 5.33 1.82 0.05 6.88 2.32Table 6: Win/Loss in Accuracy and F-measure of using different features for sentiment classifica-tion on all nine datasets.

classifier described in Section 4.2. Note that STS-Gold is the only dataset among theother 9 that provides named entities manually annotated with their sentiment labels(positive, negative, neutral). Therefore, our evaluation in this task is done using theSTS-Gold dataset only.

Features Accuracy

Positive Sentiment Negative Sentiment Neutral Sentiment Average

P R F1 P R F1 P R F1 P R F1Unigrams 48.28 92 79.31 85.19 6.67 7.69 7.14 22.22 25 23.53 40.3 37.33 38.62LDA-Topics 58.62 92 79.31 85.19 31.82 53.85 40 36.36 25 29.63 53.39 52.72 51.6Semantic Concepts 55.17 92 79.31 85.19 25 38.46 30.3 30.77 25 27.59 49.26 47.59 47.69SS-Patterns 60.34 92 79.31 85.19 34.78 61.54 44.44 40 25 30.77 55.59 55.28 53.47

Table 7: Accuracy and averages of Precision, Recall, and F measures of entity-level sentiment classification using differentfeatures.

Table 7 reports the results in accuracy, precision (P), recall (R) and F1 measure ofpositive, negative and neutral sentiment classification performances from using unigrams,semantic concepts, LDA-Topics and SS-Patterns features. Generally, our SS-Patternsoutperform all other features including word unigrams in all measures. In particular,merely using word unigrams for classification gives the lowest performance of 48.24%and 38.62% in accuracy and average F1. However, augmenting the feature space withSS-Patterns improves the performance significantly by 12.06% in accuracy and 14.85%in average F1. Our SS-Patterns also outperform LDA-Topics and semantic conceptsfeatures by at least 1.72% and 1.87% in accuracy and average F1.

As for per-class sentiment classification performance, we observe that all featuresproduce high and similar performances on detecting positive entities. This is becauseclassifiers trained from either feature set fail in detecting the sentiment of the sameentities. Moreover, it seems that detecting negative and neutral entities are much moredifficult tasks than detecting positive ones. For example, unigrams perform very poorlyin detecting negative entities with a F1 less then 8%. Although the performance improvesa lot by using SS-Patterns, it is still much lower than the positive classification perfor-mance. For neutral sentiment classification the performance is the lowest with unigrams(F1 = 23.53%) while it is the highest with SS-Patterns (F1 = 30.77%). Such varyingperformance might be due to the uneven sentiment class distribution in the entity dataset.As can be noted from Table 2, positive entities constitute 50% of the total number ofentities while the neutral and negative entities form together the other 50%.

Win/Loss in Accuracy and F-‐measure of using different features for senRment classificaRon on all nine datasets.

Based on 9 Twi+er datasets

Tweet-Level Sentiment Analysis

Hassan S., He, Y., Miriam F.and Harith A., SemanRc Pa+erns for SenRment Analysis of Twi+er, ISWC 2014, Trento, Italy

55.00

57.00

59.00

61.00

63.00

65.00

67.00

Accuracy F1

Unigrams LDA-‐Topics SemanRc Concepts SS-‐Pa+erns

Gold standard of 58 enRRes

Entity-Level Sentiment Analysis

Hassan S., He, Y., Miriam F.and Harith A., SemanRc Pa+erns for SenRment Analysis of Twi+er, ISWC 2014, Trento, Italy

ENGAGEMENT ANALYSIS

ONLINE ENGAGEMENT

Forum on a celebrity

Forum on transport

Different Engagement Patterns

Different Engagement Parameters

Different Engagement Parameters

… “few people took part”

•  309 invitees from media, academia, and public engagement bodies"

•  2 invitees contributed to the site, with

2 edits!!

Recipe for more engaging posts?

Ask the (Social) Data

•  What’s the model of good/bad tweets?"•  What features are associated with each group?"

Feature Engineering

initial tweet that generates a reply. Features which describe seed posts can bedivided into two sets: user features - attributes that define the user making thepost; and, content features - attributes that are based solely on the post itself.We wish to explore the application of such features in identifying seed posts, todo this we train several machine learning classifiers and report on our findings.However we first describe the features used.

4.1 Feature Extraction

The likelihood of posts eliciting replies depends upon popularity, a highly subjec-tive term influenced by external factors. Properties influencing popularity includeuser attributes - describing the reputation of the user - and attributes of a post’scontent - generally referred to as content features. In Table 1 we define user andcontent features and study their influence on the discussion “continuation”.

Table 1. User and Content Features

User FeaturesIn Degree: Number of followers of U #

Out Degree: Number of users U follows #List Degree: Number of lists U appears on. Lists group users by topic #Post Count: Total number of posts the user has ever posted #

User Age: Number of minutes from user join date #Post Rate: Posting frequency of the user PostCount

UserAge

Content FeaturesPost length: Length of the post in characters #Complexity: Cumulative entropy of the unique words in post p λ

of total word length n and pi the frequency of each word!

i∈[1,n] pi(log λ−log pi)

λUppercase count: Number of uppercase words #

Readability: Gunning fog index using average sentence length (ASL) [7]and the percentage of complex words (PCW). 0.4(ASL+ PCW )

Verb Count: Number of verbs #Noun Count: Number of nouns #

Adjective Count: Number of adjectives #Referral Count: Number of @user #

Time in the day: Normalised time in the day measured in minutes #Informativeness: Terminological novelty of the post wrt other posts

The cumulative tfIdf value of each term t in post p!

t∈p tfidf(t, p)Polarity: Cumulation of polar term weights in p (using

Sentiwordnet3 lexicon) normalised by polar terms count Po+Ne|terms|

4.2 Experiments

Experiments are intended to test the performance of different classification mod-els in identifying seed posts. Therefore we used four classifiers: discriminativeclassifiers Perceptron and SVM, the generative classifier Naive Bayes and thedecision-tree classifier J48. For each classifier we used three feature settings:user features, content features and user+content features.

Datasets For our experiments we used two datasets of tweets available on theWeb: Haiti earthquake tweets4 and the State of the Union Address tweets.5 The

4 http://infochimps.com/datasets/twitter-haiti-earthquake-data5 http://infochimps.com/datasets/tweets-during-state-of-the-union-address

•  Focus Features"–  Topic entropy: the distribution of the author across community forums"–  Topic Likelihood: the likelihood that a user posts in a specific forum given his post history"

•  Measures the affinity that a user has with a given forum"•  Lower likelihood indicates a user posting on an unfamiliar topic"

initial tweet that generates a reply. Features which describe seed posts can bedivided into two sets: user features - attributes that define the user making thepost; and, content features - attributes that are based solely on the post itself.We wish to explore the application of such features in identifying seed posts, todo this we train several machine learning classifiers and report on our findings.However we first describe the features used.

4.1 Feature Extraction

The likelihood of posts eliciting replies depends upon popularity, a highly subjec-tive term influenced by external factors. Properties influencing popularity includeuser attributes - describing the reputation of the user - and attributes of a post’scontent - generally referred to as content features. In Table 1 we define user andcontent features and study their influence on the discussion “continuation”.

Table 1. User and Content Features

User FeaturesIn Degree: Number of followers of U #

Out Degree: Number of users U follows #List Degree: Number of lists U appears on. Lists group users by topic #Post Count: Total number of posts the user has ever posted #

User Age: Number of minutes from user join date #Post Rate: Posting frequency of the user PostCount

UserAge

Content FeaturesPost length: Length of the post in characters #Complexity: Cumulative entropy of the unique words in post p λ

of total word length n and pi the frequency of each word!

i∈[1,n] pi(log λ−log pi)

λUppercase count: Number of uppercase words #

Readability: Gunning fog index using average sentence length (ASL) [7]and the percentage of complex words (PCW). 0.4(ASL+ PCW )

Verb Count: Number of verbs #Noun Count: Number of nouns #

Adjective Count: Number of adjectives #Referral Count: Number of @user #

Time in the day: Normalised time in the day measured in minutes #Informativeness: Terminological novelty of the post wrt other posts

The cumulative tfIdf value of each term t in post p!

t∈p tfidf(t, p)Polarity: Cumulation of polar term weights in p (using

Sentiwordnet3 lexicon) normalised by polar terms count Po+Ne|terms|

4.2 Experiments

Experiments are intended to test the performance of different classification mod-els in identifying seed posts. Therefore we used four classifiers: discriminativeclassifiers Perceptron and SVM, the generative classifier Naive Bayes and thedecision-tree classifier J48. For each classifier we used three feature settings:user features, content features and user+content features.

Datasets For our experiments we used two datasets of tweets available on theWeb: Haiti earthquake tweets4 and the State of the Union Address tweets.5 The

4 http://infochimps.com/datasets/twitter-haiti-earthquake-data5 http://infochimps.com/datasets/tweets-during-state-of-the-union-address

Classification of Posts

Seed Posts Non-Seed Posts

§  Binary classification model

§  Trained with social, content, and combined features §  80/20 training/testing

§  Identify best feature types, and

top individual features, in predicting post classification

Engagement on Boards.ie •  Which posts are

more likely to stimulate responses and discussions?"

•  What impacts engagement more; user features, post content, forum affinity?"

•  Which individual features are most influential?"

Top Features for Engagement on Boards.ie

Rowe, M.; Angeletou, S. and Alani, H. AnRcipaRng discussion acRvity on community forums. SocialCom 2011, Boston, MA, USA.

•  Content features were key!"•  Best predictions were achieved when combining user, content, and focus features"•  URLs (Referral Count) in a post negatively impact discussion activity"•  Seed Posts (posts that receive replies) are associated with greater forum likelihood"activity levels, and because it has already been used in other

investigations (e.g., [14]).Boards.ie does not provide explicit social relations be-

tween community members, unlike for example Facebook andTwitter. We followed the same strategy proposed in [3] forextracting social networks from Digg, and built the Boards.iesocial network for users, weighting edges cumulatively by thenumber of replies between any two users.

TABLE IDESCRIPTION OF THE BOARDS.IE DATASET

Posts Seeds Non-Seeds Replies Users1,942,030 90,765 21,800 1,829,465 29,908

In order to take derive our features we required a windowof !-days from which the social graph can be compiled andrelevant measurements taken. Based on previous work overthe same dataset in [14], we used a similar window of 188days (roughly 6-months) prior to the post date of a given seedor non-seed post. For instance, if a seed post " is made attime #, then our window from which the features (i.e., userand focus features) are derived is from # − 188 to # − 1. Inusing this heuristic we ensure that the features compiled foreach post are independent of future outcomes and will notbias our predictions - for example a user may increase theiractivity following the seed post which would not be a trueindicator of their behaviour at the time the post was made.Table I summarises the dataset and the number of posts (seeds,non-seeds and replies) and users contained within.

V. CLASSIFICATION: DETECTING SEED POSTS

Predicting discussion activity levels are often hindered byincluding posts that yield no replies. We alleviate this problemby differentiating between seed posts and non-seeds through abinary classification task. Once seed posts have been identifiedwe then attempt to predict the level of discussion that suchposts will generate. To this end, we look for the best classifierfor identifying seed and non-seed posts and then search for thefeatures that played key roles in distinguishing seed posts fromnon-seeds, thereby observing key features that are associatedwith discussions.

A. Experimental Setup

For our experiments we are using the previously describeddataset collected from Boards.ie containing both seeds andnon-seeds throughout 2006. For our collection of posts webuilt the content, user, and focus features listed in section IIIfrom the past 6 months of data leading up to the date on whichthe post was published - thereby ensuring no bias from futureevents in our dataset. We split the dataset into 3 sets using a70/20/10% random split, providing a training set, a validationset and a test set.

Our first task was to perform model selection by testing fourdifferent classifiers: SVM, Naive Bayes, Maximum Entropyand J48 decision tree, when trained on various individual fea-ture sets and their combinations: user features, content features

and focus features. This model selection phase was performedby training each classifier, together with the combination offeatures, using the 70% training split and labelling instancesin the held out 20% validation split.

Once we had identified the best performing model - i.e.,the classifier and combination of feature set that produces thehighest $1 value - our second task was to perform featureassessment, thereby identifying key features that contributesignificantly to seed post prediction accuracy. For this wetrained the best performing model from the model selectionphase over the training split and tested its classification accu-racy over the 10% test split, dropping individual features fromthe model and recording the reduction in accuracy followingthe omission of a given feature. Given that we are performinga binary classification task we use the standard performancemeasures for such a scenario: precision, recall and f-measure- setting % = 1 for an equal weighting of precision andrecall. We also measure the area under the Receiver OperatorCharacteristic curve to gauge the relationship between recalland fallout - i.e., false negative rate.

TABLE IIRESULTS FROM THE CLASSIFICATION OF SEED POSTS USING

VARYING FEATURE SETS AND CLASSIFICATION MODELS

! " #1 "$%User SVM 0.775 0.810 0.774 0.581

Naive Bayes 0.691 0.767 0.719 0.540Max Ent 0.776 0.806 0.722 0.556J48 0.778 0.809 0.734 0.582

Content SVM 0.739 0.804 0.729 0.511Naive Bayes 0.730 0.794 0.740 0.616Max Ent 0.758 0.806 0.730 0.678J48 0.795 0.822 0.783 0.617

Focus SVM 0.649 0.805 0.719 0.500Naive Bayes 0.710 0.737 0.722 0.588Max Ent 0.649 0.805 0.719 0.586J48 0.649 0.805 0.719 0.500

User + Content SVM 0.790 0.808 0.727 0.509Naive Bayes 0.712 0.772 0.732 0.593Max Ent 0.767 0.807 0.734 0.671J48 0.795 0.821 0.779 0.675

User + Focus SVM 0.776 0.810 0.776 0.583Naive Bayes 0.699 0.778 0.724 0.585Max Ent 0.771 0.806 0.722 0.607J48 0.777 0.810 0.742 0.617

Content + Focus SVM 0.750 0.805 0.729 0.511Naive Bayes 0.732 0.787 0.746 0.658Max Ent 0.762 0.807 0.731 0.692J48 0.798 0.823 0.787 0.662

All SVM 0.791 0.808 0.727 0.510Naive Bayes 0.724 0.780 0.740 0.637Max Ent 0.768 0.808 0.733 0.688J48 0.798 0.824 0.792 0.692

B. Results: Model Selection

1) Model Selection with Individual Features: The resultsfrom our first experiments are shown in Table II. Lookingfirst at individual feature sets - e.g., SVM together withuser features - we see that content features yield improvedpredictive performance over user and focus features. On dis-cussion forums content appears to play a more central role

•  Lower informativeness is associated with seed posts"

–  i.e. seeds use language that is familiar to the community"

Top Features for Engagement on Twitter

which we found to be 0.674 indicating a good correlation between the two listsand their respective ranks.

Table 4. Features ranked by Information Gain Ratio wrt Seed Post class label. Thefeature name is paired within its IG in brackets.

Rank Haiti Union Address1 user-list-degree (0.275) user-list-degree (0.319)2 user-in-degree (0.221) content-time-in-day (0.152)3 content-informativeness (0.154) user-in-degree (0.133)4 user-num-posts (0.111) user-num-posts (0.104)5 content-time-in-day (0.089) user-post-rate (0.075)6 user-post-rate (0.075) user-out-degree (0.056)7 content-polarity (0.064) content-referral-count (0.030)8 user-out-degree (0.040) user-age (0.015)9 content-referral-count (0.038) content-polarity (0.015)10 content-length (0.020) content-length (0.010)11 content-readability (0.018) content-complexity (0.004)12 user-age (0.015) content-noun-count (0.002)13 content-uppercase-count (0.012) content-readability (0.001)14 content-noun-count (0.010) content-verb-count (0.001)15 content-adj-count (0.005) content-adj-count (0.0)16 content-complexity (0.0) content-informativeness (0.0)17 content-verb-count (0.0) content-uppercase-count (0.0)

Fig. 3. Contributions of top-5 features to identifying Non-seeds (N) and Seeds(S).Upper plots are for the Haiti dataset and the lower plots are for the Union Addressdataset.

The top-most ranks from each dataset are dominated by user features includ-ing the list-degree, in-degree, num-of-posts and post-rate. Such features describea user’s reputation, where higher values are associated with seed posts. Figure3 shows the contributions of each of the top-5 features to class decisions in thetraining set, where the list-degree and in-degree of the user are seen to correlateheavily with seed posts. Using these rankings our next experiment explored theeffects of training a classification model using only the top-k features, observing

•  Top are list-degree, in-degree, informativeness, and #posts"

"•  Top are list-

degree, time of posting, in-degree, and #posts"

Rowe, M., Angeletou, S., Alani, H. PredicRng Discussions on the Social SemanRc Web. ESWC, Crete, 2011

which we found to be 0.674 indicating a good correlation between the two listsand their respective ranks.

Table 4. Features ranked by Information Gain Ratio wrt Seed Post class label. Thefeature name is paired within its IG in brackets.

Rank Haiti Union Address1 user-list-degree (0.275) user-list-degree (0.319)2 user-in-degree (0.221) content-time-in-day (0.152)3 content-informativeness (0.154) user-in-degree (0.133)4 user-num-posts (0.111) user-num-posts (0.104)5 content-time-in-day (0.089) user-post-rate (0.075)6 user-post-rate (0.075) user-out-degree (0.056)7 content-polarity (0.064) content-referral-count (0.030)8 user-out-degree (0.040) user-age (0.015)9 content-referral-count (0.038) content-polarity (0.015)10 content-length (0.020) content-length (0.010)11 content-readability (0.018) content-complexity (0.004)12 user-age (0.015) content-noun-count (0.002)13 content-uppercase-count (0.012) content-readability (0.001)14 content-noun-count (0.010) content-verb-count (0.001)15 content-adj-count (0.005) content-adj-count (0.0)16 content-complexity (0.0) content-informativeness (0.0)17 content-verb-count (0.0) content-uppercase-count (0.0)

Fig. 3. Contributions of top-5 features to identifying Non-seeds (N) and Seeds(S).Upper plots are for the Haiti dataset and the lower plots are for the Union Addressdataset.

The top-most ranks from each dataset are dominated by user features includ-ing the list-degree, in-degree, num-of-posts and post-rate. Such features describea user’s reputation, where higher values are associated with seed posts. Figure3 shows the contributions of each of the top-5 features to class decisions in thetraining set, where the list-degree and in-degree of the user are seen to correlateheavily with seed posts. Using these rankings our next experiment explored theeffects of training a classification model using only the top-k features, observing

HaiR Earthquake

State Union Address

former dataset contains tweets which relate to the Haiti earthquake disaster,covering a varying timespan. The latter dataset contains all tweets publishedduring the duration of president Barack Obama’s State of the Union Addressspeech. Our goal is to predict discussion activity based on the features of a givenpost by first identifying seed posts, before moving on to predict the discussionlevel.

Within the above datasets many of the posts are not seeds, but are insteadreplies to previous posts, thereby featuring in the discussion chain as a node.In [13] retweets are considered as part of the discussion activity. In our workwe identify discussions using the explicit “in reply to” information obtainedby the Twitter API, which does not include retweets. We make this decisionbased on the work presented in boyd et.al [4], where an analysis of retweetingas a discussion practice is presented, arguing that message forwards adhere todifferent motives which do not necessarily designate a response to the initialmessage. Therefore, we only investigate explicit replies to messages. To gatherour discussions, and our seed posts, we iteratively move up the reply chain - i.e.,from reply to parent post - until we reach the seed post in the discussion. Wedefine this process as dataset enrichment, and is performed by querying Twitter’sREST API6 using the in reply to id of the parent post, and moving one-step ata time up the reply chain. This same approach has been employed successfullyin work by [12] to gather a large-scale conversation dataset from Twitter.

Table 2. Statistics of the datasets used for experiments

Dataset Users Tweets Seeds Non-Seeds RepliesHaiti 44,497 65,022 1,405 60,686 2,931

Union Address 66,300 80,272 7,228 55,169 17,875

Table 2 shows the statistics that explain our collected datasets. One canobserve the difference in conversational tweets between the two corpora, wherethe Haiti dataset contains fewer seed posts as a percentage than the Uniondataset, and therefore fewer replies. However, as we explain later a later section,this does not correlate with a higher discussion volume in the former dataset. Weconvert the collected datasets from their proprietary JSON formats into triples,annotated using concepts from our above behaviour ontology, this enables ourfeatures to be derived by querying our datasets using basic SPARQL queries.

Evaluation Measures The task of identifying seed posts is a binary classifi-cation problem: is this post a seed post or not?. We can therefore restrict ourlabels to one of two classes: seed and non-seed. To evaluate the performance ourmethod we use four measures: precision, recall, f-measure and area under theReceiver Operator Curve. Precision measures the proportion of retrieved postswhich were actually seed posts, recall measures the proportion of seed postswhich were correctly identified and fallout measures the proportion of non-seedposts which were incorrectly classified as seed posts (i.e., false positive rate). We

6 http://dev.twitter.com

neg pos

05

1015

2025

30

Length

neg pos0.0

0.5

1.0

1.5

Complexity

neg pos

010

2030

40

Readability

neg pos

−4−2

02

4

Polarity

•  Top influential features do not match those found for Board.ie or for two non-random Twitter datasets"

Top Features for Engagement on Twitter – Earth Hour 2014

neg pos

510

1520

2530

Length

neg pos

0.6

0.8

1.0

1.2

1.4

complexity

neg pos

−4−3

−2−1

01

23

polarity

neg pos

01

23

45

67

mentions

!

Fernandez, M., Cano, E., and Alani, H. Policing Engagement via Social Media. CityLabs workshop, SocInfo, Barcelona, 2014

Top Features for Engagement on Twitter – Dorset Police

•  Top 4 features share 3 with Twitter Earth Hour dataset"

Publications about social media

by Katron Weller -‐ h+p://kwelle.files.wordpress.com/2014/04/figure1.jpg

Moving on … §  How can we move on

from these (micro) studies?

§  Are results consistent across datasets, and platforms?

§  One way forward is: §  Multiple platforms §  Multiple topics

Papers studying single/multiple social media platforms

Survey done on all submi7ed papers to Web Science conferences







Apples and Oranges

•  We mix and compare different datasets, topics, and platforms

•  Aim is to test consistency and transferability of results

7 datasets from 5 platforms

Seed posts are those that receive a reply Non-seed posts are those with no replies

Pla1orm Posts Users Seeds Non-‐seeds Replies

Boards.ie 6,120,008 65,528 398,508 81,273 5,640,227

Twi+er Random 1,468,766 753,722 144,709 930,262 390,795

Twi+er (HaiR Earthquake)

65,022 45,238 1,835 60,686 2,501

Twi+er (Obama State of Union Address)

81,458 67,417 11,298 56,135 14,025

SAP 427,221 32,926 87,542 7,276 332,403

Server Fault 234,790 33,285 65,515 6,447 162,828

Facebook 118,432 4,745 15,296 8,123 95,013

Data Balancing Pla1orm Seeds Non-‐seeds Instance Count

Boards.ie 398,508 81,273 162,546

Twi+er Random 144,709 930,262 289,418

Twi+er (HaiR Earthquake)

1,835 60,686 3,670

Twi+er (Obama State of Union Address)

11,298 56,135 22,596

SAP 87,542 7,276 14,552

Server Fault 65,515 6,447 12,894

Facebook 15,296 8,123 16,246

Total 521,922

For each dataset, an equal number of seeds and non-seed posts are used in the analysis.

Classification Results

Feature P R F1

Social 0.592 0.591 0.591

Content 0.664 0.660 0.658

Social+Content 0.670 0.666 0.665

(Random) (HaiR Earthquake)

(Obama’s State Union Address)

P R F1

0.561 0.561 0.560

0.612 0.612 0.611

0.628 0.628 0.628

P R F1

0.968 0.966 0.966

0.752 0.747 0.747

0.974 0.973 0.973

Feature P R F1

Social 0.542 0.540 0.539

Content 0.650 0.642 0.639


P R F1

0.650 0.631 0.628

0.575 0.541 0.521

0.652 0.632 0.629

P R F1

0.528 0.380 0.319

0.626 0.380 0.275

0.568 0.407 0.359

Feature P R F1

Social 0.635 0.632 0.632

Content 0.641 0.641 0.641


§  Performance of the logisRc regression classifier trained over different feature sets and applied to the test set.

Effect of features on engagement Boards.ie

β

−2−1

012

Twitter Random

β

−0.50.00.51.0

Twitter Haiti

−6e+16−4e+16−2e+16

0e+002e+164e+166e+16

Twitter Union

β

−0.8−0.6−0.4−0.2

0.00.2

Server Fault

β

−1.0−0.5

0.00.51.01.52.0

SAP

β

−10

−5

0

5

Facebook

β

−0.10.00.10.20.30.40.5

In−degreeOut−degreePost CountAge

Post RatePost LengthReferrals CountPolarity

ComplexityReadabilityReadability FogInformativeness

Logistic regression coefficients for each platform's features

Comparison to literature

§  How performance of our shared features compare to other studies on different datasets and platforms?

Positive impact Negative impact

Mismatch Match


Positive impact Negative impact

Mismatch Match


Let’s Share More Data!

Semantic Clustering •  Statistical models play important roles in social data

analyses

•  Keeping such models up to date often means regular, expensive, and time consuming retraining

•  Semantic Features are likely to decay more slowly than lexical features

•  Could adding semantics to the models extend their value and life expectancy?

Cano, E., He, Y., Alani, H. Stretching the Life of Twitter Classifiers with Time-Stamped Semantic Graphs. ISWC 2014, Trento, Italy.

Semantic Representation of a Tweet

<dbp:Barack_Obama>

American

dbprop:nationality

<skos:Nobel_Peace_Price_laureates>

dcterms:subject

<dbo:PresidentOfUnitedStateofAmerica>

rdf:type

<dbp:Hosni_Mubarak>

<skos:PresidentsOfEgypt>

<dbp:CNN>

<dbp:Egyptian_Arabic>

<skos:Arab_republics>

<dbp:Egypt>

<skos:English-language_television_stations>

dcterms:subject

dcterms:subject

dbprop:languages<dbp:Country>

rdf:type

rdf:type

Evolution of Semantics •  Renewed DBpedia Graph snapshots are taken over time"•  Semantic features updated based on new knowledge in

DBpedia"

v3.6 v3.7 v3.8

<Barack_Obama>

<Budget_Control_Act_of_2011>

<UnitedStatesPresidentialCandidates>

<Hawaii>

<MechelleObama>

spouse

birth1place

wikiPageWikiLink

wikiPageWikiLink

Experiments

Cross-‐Epoch

2010-‐2011 2010-‐2013 2011-‐2013 Average

F1 F1 F1 BoW 0.634 0.481 0.261 0.458

Category 0.683 0.539 0.524 0.582

Property 0.665 0.557 0.502 0.603

Resource 0.774 0.544 0.445 0.587

Class 0.691 0.665 0.669 0.675

Extending fitness of model to proceedings epochs

•  12,000 annotated tweets"•  Adding Classes as clustering features provide best performance"

Same-‐epoch

2010-‐2010 2011-‐2011 Average

BoW 0.831 0.875 0.845

APPLICATIONS

1. "Fish where the fish is"–  one interface to access multiple SNS"–  layman monitoring of users and topics "

2. "My consistency first"–  communicating with users in own

constituency"–  find local groups, events, and topics"

3. "What are their needs, complaints, and preferences?"–  what citizens talk about, complain about"–  what are the top 5-10 topics of the day"

4.  Who should I talk to?"–  who are the influential citizens"–  whom to engage with"

5.  What about Tomorrow?"–  which topics will get hotter?"–  which discussions are likely to grow

further?"

6.  Presence and popularity"–  what writing recipe to follow to reach more

people"

7.  Privacy"–  concerns on citizens’ privacy when

extracting info"–  concerns on their own privacy with 3rd

party SNS access tools"

What policymakers really want from Social Media?

Interviews with 31 policymakers

Wandhöfer, T.; Taylor, S.; Alani, H.; Zoshi, S.; Sizov, S.; et al. Engaging poliRcians with ciRzens on social networking sites: the WeGov Toolbox. IJEGR, 8(3), 2012

Demo

Monitoring SCN "Monitoring of evolution of community activities and level of contributions in SAP Community Networks – SCN "

SCN Behaviour

"Community managers can monitor behaviour composition of forums, and its association to activity evolution "

For Education

https://twitter.com/OpenUniversity/status/346911297704714240

FB Groups

Sentiment

Macro Behaviour

Micro Behaviour

Topics

Course tutors Behaviour Analysis

SenRment Analysis

Topic Analysis

Real Rme monitoring

•  How acRve the engaged the course group is?

•  How is senRment towards a course evolving?

•  Are the leaders of the group providing posiRve/negaRve comments?

•  What topics are emerging? •  Is the group flourishing or

diminishing? •  Do students get the answers

and support they need?

Thomas, K.; Fernández, M.; Brown, S., Alani, H. OUSocial2: a plaxorm for gathering students’ feedback from social media. (Demo) ISWC 2014, Trento, Italy.

DEMO

Thanks to colaborators

Thanks to ..

Gregoire Burel

Miriam Fernandez

Smitashree Choudhury

Hassan Saif Lara Piccolo Thomas Dickensen

Elizabeth Cano

Matthew Rowe

Keerthi Thomas

Sofia Angeletou

Heads-up

Semantic Patterns for Sentiment Analysis of Twitter Thursday 16:00 - Session: Social Media"

Semantic Patterns for Sentiment Analysis of Twitter Thursday 15.40 - Session: Social Media"

OUSocial2: a pla1orm for gathering students’ feedback from social media (DEMO) The Topics they are a-‐Changing — Characterising Topics with Time-‐Stamped Seman\c Graphs (POSTER)"!Automa\c Stopword Genera\on using Contextual Seman\cs for Sen\ment Analysis of Twi_er (POSTER)

User Profile Modeling in Online Communities !Sunday 2:05 pm - SWCS Workshop"

Making More Sense Out of Social Data

Data & Analytics

community health

social networks

social semantic linking

online community

community type

behaviour roles

inhouse social platforms

home messages social