Election 2010: The View from Twitter

Election 2010: The View from Twitter

Axel Bruns / Jean BurgessARC Centre of Excellence for Creative Industries and Innovation, [email protected] – @[email protected] – @jeanburgesshttp://mappingonlinepublics.net – http://cci.edu.au/

Image by campoalto

Project: New Media and Public Communication

• ARC Discovery (2010-12) – A$410.000

• Axel Bruns (CI), Jean Burgess (SRF) – QUT, Brisbane

• Lars Kirchhoff, Thomas Nicolai (PIs) – Sociomantic Labs, Berlin

• Project blog: http://mappingonlinepublics.net/

Year 1 Year 2 Year 3

Social network sources:

· YouTube· Flickr· Twitter· blogs

Research tools:

· network crawler· content scraper· content analysis· network analysis

Research tool development and baseline data

Baseline information:

· data extraction· content creation

statistics· patterns in terms

and themes· baseline social

networking map· interconnections

between social network spaces

Content creation patterns

Changes over time:

· short-term statistics· regular / seasonal

patterns

Cluster profiling:

· common themes / patterns

· lead users

Focus on specific events

Cultural dynamics:

· rapid spread of new ideas

· communication across clusters

· thematic discourse analysis

· relationship with main- stream media coverage

Methodology – Twapperkeeper

Analysis

Capture

Identification

#Hashtag Archive

Tweet Statistics and

@Replies

Patterns of Activity over

Time

Networks of @Replies

(short/long term)

Tweet Texts

Keyword /Key PhraseMapping

Data Processing – Twitter

• Typical data structure (#ausvotes):

Data Processing – Twitter

• Tools:

• Gawk – Scripting tool für CSV processing (open source)

• Excel – Data aggregation, pivot tables and charts

• Leximancer / WordStat – Keyword extraction, co-occurence matrices

• Gephi – Network analysis and visualisation (open source)

# Extract @replies for network visualisation## this script takes a CSV archive of tweets, and reworks it into network data for visualisation## expected data format:# text,to_user_id,from_user,id,from_user_id,iso_language_code,source,profile_image_url,geo_type, # geo_coordinates_0,geo_coordinates_1,created_at,time## output format:# from,to,tweet,time,timestamp## the script extracts @replies from tweets, and creates duplicates where multiple @replies are# present in the same tweet - e.g. the tweet "@one @two hello" from user @user results in# @user,@one,"@one @two hello" and @user,@two,"@one @two hello"## Released under Creative Commons (BY, NC, SA) by Axel Bruns - [email protected]

BEGIN {print "from,to,tweet,time,timestamp"

}

/@([A-Za-z0-9_]+)/ {

a=0 do {

match(substr($1, a),/@([A-Za-z0-9_]+)?/,atArray)a=a+atArray[1, "start"]+atArray[1, "length"]

if (atArray[1] != 0) print $3 "," atArray[1] "," $1 "," $12 "," $13

} while(atArray[1, "start"] != 0)

}

# filter.awk - Filter list of tweets## this script takes a CSV or other list of tweets, and removes any lines that don't include RT @username# the script preserves the first line, expecting that it contains header information## script expects command-line argument search={searchcriteria} _before_ the input CSV filename# enclose the search term in quotation marks if it contains any special characters## e.g.: gawk -F , -f filter.awk search="(julia|gillard)" tweets.csv >filteredtweets.csv## expected data format:# CSV or simple list of tweets, line-by-line## output format:# same as above, listing only retweets## Released under Creative Commons (BY, NC, SA) by Axel Bruns - [email protected]

BEGIN { getline print $0

}

tolower($0) ~ search {

print $0

}

2010

-Jun-

23 0

0

2010

-Jun-

23 0

1

2010

-Jun-

23 0

2

2010

-Jun-

23 0

3

2010

-Jun-

23 0

4

2010

-Jun-

23 0

5

2010

-Jun-

23 0

6

2010

-Jun-

23 0

7

2010

-Jun-

23 0

8

2010

-Jun-

23 0

9

2010

-Jun-

23 1

0

2010

-Jun-

23 1

1

2010

-Jun-

23 1

2

2010

-Jun-

23 1

3

2010

-Jun-

23 1

4

2010

-Jun-

23 1

5

2010

-Jun-

23 1

6

2010

-Jun-

23 1

7

2010

-Jun-

23 1

8

2010

-Jun-

23 1

9

2010

-Jun-

23 2

0

2010

-Jun-

23 2

1

2010

-Jun-

23 2

2

2010

-Jun-

23 2

3

2010

-Jun-

24 0

0

2010

-Jun-

24 0

2

2010

-Jun-

24 0

3

2010

-Jun-

24 0

4

2010

-Jun-

24 0

5

2010

-Jun-

24 0

6

2010

-Jun-

24 0

7

2010

-Jun-

24 0

8

2010

-Jun-

24 0

9

2010

-Jun-

24 1

0

2010

-Jun-

24 1

1

2010

-Jun-

24 1

2

2010

-Jun-

24 1

3

2010

-Jun-

24 1

4

2010

-Jun-

24 1

5

2010

-Jun-

24 1

6

2010

-Jun-

24 1

8

2010

-Jun-

24 1

9

2010

-Jun-

24 2

0

2010

-Jun-

24 2

1

2010

-Jun-

24 2

2

2010

-Jun-

24 2

3

2010

-Jun-

25 0

0

0

1000

2000

3000

4000

5000

6000

7000

Total

Prelude: Leadership #spill

23 June, 19:00-00:00:

Speculation

24 June, 08:00-15:00:

Party Vote & Aftermath

#spill Discussion Network (Node size: indegree [most @replies received]; node colour: outdegree [most @replies sent])

#ausvotes: Overall Activity (17 July – 24 Aug. 2010)

#ausvotes: Discussion Network(17 July to 25 Aug. 2010 / All @replies / Node size: Indegree / Node colours: betweenness centrality)

#ausvotes: Mentions of the Parties (normalised per day)

#ausvotes: Mentions of the Leaders (normalised per day)

#ausvotes: Mentions of the Leaders (cumulative)

#ausvotes: Key Themes

#ausvotes: Key Themes (normalised per day)

#ausvotes: Distractions (normalised per day)

Labor’s Twibbon Campaign RTs

#ausvotes: Distractions

Notes and Limitations

• Twapperkeeper relies on #hashtags

• Problem if #hashtags are inconsistent/unclear

• Follow-on @replies and retweets may not continue to use #hashtags

• Casual commenters may not use #hashtags in the first place

• May miss early developments – e.g. #hashtag standardisation

• Twitter as a subset of society:

• Broadband policy and Internet filter over-, asylum seekers underrepresented

• #hashtag use is a further sign of self-selection

• Need to look to Twitter firehose for more comprehensive picture

• Need to track baseline activity to understand how exceptional #ausvotes was

• See more at mappingonlinepublics.net – up next: time-based animations...

• Or find us at @snurb_dot_info and @jeanburgess

http://mappingonlinepublics.net/

@snurb_dot_info@jeanburgess

Image by campoalto