#Microposts16 - Comparing Social Media and Traditional Surveys around the Boston Marathon Bombing

Cody Buntain@[email protected] of Maryland

Erin McGrath, Gary LaFree{ecmcgrath,lafree}@umd.eduSTART Center, UMD

Jennifer [email protected] of Maryland

Comparing Social Media and Traditional Surveys Around the Boston Marathon Bombing

1

#Microposts201611 April 2016

Montreal, Quebec CA

Surveys are mature, ubiquitous, and

powerful instruments

2 - Introduction

Social media platforms produce huge data sets and let users respond to events

instantly

3 - Introduction

0

0.4

0.8

1.2

1.6

Face

book*

China

USA

Twitte

r*

Popu

latio

n (B

illion

s)

Instagram

Twitter

Facebook

1 100 10,000 1,000,000 100,000,000

New posts

Since I’ve been talking

4 - Introduction

5 - Introduction

Socialmediaisn’t

representa1ve!

Surveysarebe7er!Socialmediais

cheap,big,and

easy!

The Argument

What is the point of this talk?

6 - Introduction

- Introduction7

The Point

To compare social media and survey data

Compared across: • Costs • Data Types • Relevance • Validity

- Introduction8

Grounding the Point

9 - Case Study

What makes the Boston Marathon Bombing a good case study?

The Bombing from Multiple Perspectives

• Three-wave survey on public perceptions of law enforcement

• The bombing happened to occur between waves

• Post hoc survey on information seeking/sharing during crises

• Included respondents exposed to Boston Marathon coverage

• Analysis of public response to the bombing on Twitter

• 134 million tweets from April 2013

10 - Case Study

Why Twitter?

11 - Case Study

12 - Results

What are the strengths and weaknesses of surveys and social media?

Four Axes of Comparison

• Data types

• Relevance

• Cost

• Validity

13 - Results

Comparing Data Types

14 - Results

Survey Data

Social Media

Data

• Attitudinal data

• Behavioral data

• Demographic data

Comparing Data Types

15 - Results

Survey Data

Social Media

Data

• Attitudinal data

• Behavioral data

• Demographic data

Comparing Data Types

16 - Results

Survey Data

Social Media

Data

• Attitudinal data

• Behavioral data

• Demographic data

Comparing Data Types

17 - Results

Comparing Data Types

18 - Results

Boston PD Follower CountsBoston-Related Twitter Activity

Comparing Data Types

19 - Results

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

04-01

04-03

04-05

04-07

04-09

04-11

04-13

04-15

04-17

04-19

04-21

04-23

04-25

04-27

04-29

Percen

tageofT

weetsCon

tainingEm

o=on

Date

Anger Disgust Fear Happiness Sadness Surprise

Emotions in Boston-related Tweets from the US

Comparing Data Types

20 - Results

Self-reported “willingness”

Sentiment Towards Police in the US

Takeaway: Social media and survey data provide

different types of data

- Results

Comparing Data Types

21

With some overlapAnd both are useful for different purposes.

Four Axes of Comparison

• Data types

• Relevance

• Cost

• Validity

22 - Results

Comparing Relevance

23 - Results

• Temporal Relevance

• Topical Relevance• Geographic Relevance

Comparing Relevance

24 - Results

Temporal Relevance

2 mon

ths

March ‘

13

April ‘13

May ‘1

3

June

‘13

July ‘

13

Augus

t ‘13

Comparing Relevance

25 - Results

Topical Relevance

April 1 ‘

13

April 8 ‘

13

April 15

‘13

April 22

‘13

April 29

‘13

May 6

‘13

“police”

in

Twitte

r

#delhirape Protests

Comparing Relevance

26 - Results

Geographic Relevance

Distribution of Twitter Users Target Country for Survey

Comparing Relevance

27 - Results

Geographic Relevance

Distribution of Twitter Users Target Country for Survey

???

Takeaway: Social media data is more timely than survey

data, but surveys can better target topical and geographic responses for higher quality.

- Results

Comparing Relevance

28

Four Axes of Comparison

• Data types

• Relevance

• Cost

• Validity

29 - Results

- Results

Comparing Cost

30

- Results

Comparing Cost

31

$300/month

$11/month

+ $3,000/month

$3,311/month

Average one-time fee:

$70,000

- Results

Comparing Cost

32

Takeaway: Storing large sets of social media data,

contracting with a data reseller, and processing it at scale is significantly cheaper than

running nationally representative surveys.

- Results

Comparing Cost

33

Four Axes of Comparison

• Data types

• Relevance

• Cost

• Validity

34 - Results

- Results

Comparing Validity

35

- Results

Comparing Validity

36

Takeaway: The typical definition of validity is difficult (and perhaps undesirable) to apply to social media

data.

- Results

Comparing Validity

37

What is the point of this talk?

38 - Conclusions

Complements and Synergies

• Use both social media and surveys where possible

• Social media may provide first approximation when time and cost are factors

• Survey data can be of higher quality, more targeted, and can ask deeper questions

39 - Conclusions

Cody Buntain@codybuntain

[email protected]

University of Maryland

Comparing Social Media and Traditional Surveys Around the Boston Marathon Bombing

40

#Microposts201611 April 2016

Montreal, Quebec CA

Thank you! Questions?