Presentation for:

Boston | Geneva | Mumbai | San Francisco | Seattle | Washington FSG.ORG

Presentation for:

Measuring Social Media Impact:A closer look at the Knight Foundation’s

Community Information Challenge

RWJF Social Media Measurement Meeting

Princeton, NJApril 25, 2013

2

FSG.ORG

© 2013 FSG

Are communities becoming…

More informed?

More engaged?

Two Questions Guided the Evaluation of the Community Information Challenge

“I didn’t know how easy it could be to care for the

environment.”

“I am going to see if my neighborhood can

start a recycling program.”

3

FSG.ORG

© 2013 FSG

We Utilized the Following Practices in Measuring Grantees’ Social Media Activities

• Understand the goals of social media activity

• Identify the metrics of success

• Track changes with “offline” data

IMPACT Evaluation Guide

4

FSG.ORG

© 2013 FSG

Understand the Goals of Social Media Activity

5

FSG.ORG

© 2013 FSG

We Needed a Framework to Understand the Impact of a Diverse Set of Information Projects

6

FSG.ORG

© 2013 FSG

Our Social Media Measurement Focused on Project-Level Outcomes

Reaching target audience

Engaging target audience

7

FSG.ORG

© 2013 FSG

We Began Identifying the Metrics of Success

Increasing Reach Increasing Engagement

• More (new) people visiting the website over time

• More people subscribing to get information updates

• More people attending project-specific trainings, events, or meetings

• More (new) people commenting and contributing information online

• People taking action online

• People sharing information with their networks

• People participating in trainings, events, or meetings

Selected

We needed to look at changes that would take place online and offline

8

FSG.ORG

© 2013 FSG

Then We Mapped Outcomes to Key Performance Indicators (KPIs)

Increasing Reach• More (new) people visiting the website over

time

• More people subscribing to get information updates

• More people attending project-specific trainings, events, or meetings

Lesson Learned: Solicit the help of social media experts to identify the most relevant social media metrics

Increasing Engagement• More (new) people commenting and

contributing information online

• People taking action online, e.g., submitting contest entries

• People sharing information with their networks

% of visits by new visitors over time

# of Fans, Followers, registered users

# of attendees

# comments, content uploads

Donations, sign-ups, “conversions”

Retweets, forwards, hosting events

9

FSG.ORG

© 2013 FSG

Entering Data in a Spreadsheet Provided Us with Information That We Could Use for Benchmarking

Time trends

Google Analytics

Facebook Insights

Twitter and Tweetreach

YouTube“Custom”

metrics

10

FSG.ORG

© 2013 FSG

We Plotted Grantee Data to Better Understand Project Reach

Source: Anonym zed data from KCIC Year-End Evaluation Report (2010)

-2% -1% 0% 1% 2% 3% 4% 5% 6% 7%-3%

-2%

-1%

0%

1%

2%

3%

4%

5%

Average Monthly Growth in Visits

Ave

rage

Mon

thly

Gro

wth

in P

erce

ntag

e of

V

isits

from

New

Vis

itors

Projects in this quadrant have shown the greatest growth in reach through their website

11

FSG.ORG

© 2013 FSG

So What about Impact?

12

FSG.ORG

© 2013 FSG

Track Changes with “Offline” Data

Does filling out an online petition or sharing photos through a nonprofit Flickr account provide information about participation in “real world” activities?

OnlineEngagement

OfflineEngagement

We needed to examine the intersection of online and

offline behaviors

13

FSG.ORG

© 2013 FSG

Some Evidence of Real-World Engagement Could Be Found Online

GrowWNY, Buffalo NY

Source photo by Kate Mini Hillman on GrowWNY Flickr photostream (http://www.flickr.com/photos/growwny/6035588130/in/photostream.

What actions are people taking?

14

FSG.ORG

© 2013 FSG

We Collected Data Using More “Traditional” Evaluation Methods to Fully Understand Impact

Additional Data WereCollected Using…

Grantee Surveys

Interviews

Qualitative (Content) Analysis

Audience or Post-Event Surveys

Registered User Surveys

How is social media being used? By whom?

How are attitudes changing?

Are people acting differently?

15

FSG.ORG

© 2013 FSG

If online “action” is leading to end-game impact?

If we are reaching the already converted?

What “good” looks like?

Many Questions Still Remain

How Do We Know…