Yes We Can Measure Social Media: True Tales from the Social Media Measurement Trenches Central Ohio Chapter, PRSA September 10, 2009 Katie Delahaye Paine CEO [email protected]www.kdpaine.com http:/kdpaine.blogs.com Member, IPR Measurement Commission www.instituteforpr.org
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Yes We Can Measure Social Media: True Tales from the Social Media Measurement Trenches Central Ohio Chapter, PRSASeptember 10, 2009Katie Delahaye [email protected]:/kdpaine.blogs.comMember, IPR Measurement Commissionwww.instituteforpr.org
Agenda
Basic DefinitionsBasic rules of MeasurementCase StudiesThe Tools, Tips & Techniques You
Need
Why Measure?
“The main reason to measure objectives is not so much to reward or punish
individual communications manager for success or failure as it is to learn from the
research whether a program should be continued as is, revised, or dropped in favor of another approach ”
James E. Grunig, Professor Emeritus, University of Maryland “If we can put a man in orbit, why can’t we determine the effectiveness of our communications? The reason is simple and perhaps, therefore, a little old-fashioned: people, human beings with a wide range of choice. Unpredictable, cantankerous,capricious, motivated by innumerable conflicting interests, and conflicting desires.”
Ralph Delahaye Paine, Publisher, Fortune Magazine , 1960 speech to the Ad Club of St. Louis
First, the numbers
1. By 2010 Gen Y will outnumber Baby Boomers….96% of them have joined a social network
2. Social Media has overtaken porn as the #1 activity on the Web 3. Years to Reach 50 million Users: Radio (38 Years), TV (13 Years),
Internet (4 Years), iPod (3 Years)…Facebook added 100 million users in less than 9 months…iPhone applications hit 1 billion in 9 months.
4. 80% of companies use LinkedIn as a primary tool to find employees5. The fastest growing segment on Facebook is 55-65 year-old females 6. Ashton Kutcher and Ellen Degeneres have more Twitter followers
than the entire populations of Ireland, Norway and Panama 7. 80% of Twitter usage is on mobile8. There are over 200,000,000 Blogs 9. 54% = Number of bloggers who post content or tweet daily 10.78% of consumers trust peer recommendations , only 14% trust
advertisements 11.18% of traditional TV campaigns generate a positive ROI , 90% of
people that can TiVo ads do 12.Hulu has grown from 63 million total streams in April 2008 to 373
million in April 2009 13.More than 1.5 million pieces of content (web links, news stories,
blog posts, notes, photos, etc.) are shared on Facebook…daily.
4
Conquering your fears
A measurement timeline
Page 7
Old School 21st Century
You are a party planner, not a communicator
Why Traditional Metrics (AVEs) are like buggy whips
They both confuse activity with outcomeThe goal is to arrive at your destination, not
to have a faster horse
They both were based on a flawed premise
Comparing PR to Advertising is like comparing the Surrey With a Fringe On Top to the Space Shuttle – they serve different purposes
They both were rendered obsolete by technology
Page 8
The definition of timely has changedThe definition of reach has changed
GRPs & Impressions are impossible to count (an irrevelvant) in social media
The definition of success has changedThe answer isn’t how many you’ve reached,
but how those you’ve reached have responded Page 9
Old School PR 21st Century Role of PR
Social Media renders everything you know about measurement obsolete
12 Signs that it’s the end of measurement as we know it 1. The Dept of Defense considers Twittering and other
forms of social media critical to national security2. Sodexo cut $300K out of its recruitment budget
using Twitter3. Facebook USERS translated the site from English
to Spanish via a Wiki in less than 4 weeks and cost Facebook $0
4. BMC Software measures communications effectiveness based on contribution to EPS
5. United Linen & Pitney Bowes use soc.med to get ahead of competition, keep customers
6. HSUS generated $650,000 in new donations from an on-line photo contest on Flickr
7. NWF increased wildlife spotting as well as members with its Twitter account
8. The Red Cross measures the effectiveness of Twitter via lives saved and harm avoided
9. ImmunizeBC measures success in terms of vaccines given, awareness AND traffic
10. IBM 1000+ people tweeting & receives more leads, sales and exposure from a $500 podcast than it does from an ad
11. 11 Mom’s turned around Walmart’s image and delivered measureable increases in sales.
12. A social media campaign for War Child delivered a 38% increase in donations and 300 new volunteers
The New Rules of Communications
You aren’t in control and never have beenThere is no market for your message You become what you measureShe/he with the most data winsBehind every Tweet or Post is a person Empower employees, rely on customersEnable the conversations—it’s going on, with
or without youSpin is dead, long live transparency – you
can’t fake it so be who you are and see who is pleased Crowdsourcing will beat outsourcing every
time
The Engagement Decision Tree
The measurement forks in the road
Marketing/leads/sales/mission
Reputation/relationships
To fix this Or get to this
Goals drive metrics, metrics drive results
14
Goal
Metrics
Change the conversation, improve your reputation
Improve your reputation
Listen first, then respondStop doing stupid things
Negative coverage over time
Correlation exists between traffic to the ASPCA web site and the organization’s
Tracking toolsGoogle Analytics OmnitureWebTrends Web Side Story
What do you need to measure?Outputs?
Did you get the coverage you wanted?
Did you produce the promised materials on time and on budget?
Outtakes?Did your target audience see the messages?
Did they believe the messages?
Outcomes?Did audience behavior change?
Did the right people show up?
Did your relationship change?
Did sales increase?
Goals, Actions and Metrics Goal Action Output Metric Outtake
MetricOutcome Metric
Increased on-line reservations
Revamp website
Amount of content on web site
% perceiving state as a destination
% increase in web traffic and reservations
#1site for visitors to NH
Increase staffing and resources for communications
Increased exposure of “visit NH” message
Increased perception of NH as an an extreme destination
% increase in agreement with the statement
Website is preferred site for information
Add content, features to web site, keep up to date
% increase in traffic
% agreeing with the statement
# 1 rankings, and time spent on site
The 7 steps to Social Media ROI
1. Define the “R” – Define the expected results?
2. Define the “I” -- What’s the investment?
3. Understand your audiences and what motivates them
4. Define the metrics (what you want to become)
5. Determine what you are benchmarking against
6. Pick a tool and undertake research7. Analyze results and glean insight,
take action, measure again
Step 1: Define the “R”
What return is expected?
What were you hired to do?
If you are celebrating complete 100% success a year from now, what is different about the organization?
If your department was eliminated, what would be different?
23
Step 2: Define the “I”
What is the investment? PersonnelAgency compensationSenior Staff time Opportunity costRaw costs/hr costs vs material costs.
24
Step 3: Define your audiences and how you impact them
There is no “audience.” There are multiple constituencies Should you blog or Twitter? Don’t ask me, ask your customers List every stakeholder
Where do they go for information?What’s important to them?What is the benefit of having a good relationship with that stakeholder group?What’s important to them?Where do they go for information?What do you want them to know?
Understand your role in getting the audience to do what you want it to do
Step 4: Define your Key Performance Indicators (KPIs)
26
The Perfect KPIGets you where you want to go
(achieves corporate goals)Is actionableContinuously improves your
processesIs there when you need it
KPIs should be developed for: Your own propertiesDifferent tacticsOther influential sites
Revenue KPIs
Average order value and lifetime value against other channelsCost per customer acquisition vs other channels Out of pocket vs labor. % desirable conversations vs competitors and over time Share of recommendations (by influencers) compared to the competition
Potential KPIs for the blogs you control
Engagement IndexNumber of unique usersReturning versus new readersReferring source statisticsLinks from other sitesConversation Index: The ratio of blog comments to blog posts (where applicable)Total time spent on the siteThe popularity of the content itself, which gets the most views Traffic from blog to web siteSales
28
KPIs for External blogs and other Consumer Generated Media
Share of positioningShare of rants vs. ravesShare of positives/negativesShare of visibilityShare of quotesShare of brand benefits mentionedTypes of conversationsOptimal content score
Emerging benchmarks Past PerformanceThink 3
PeerUnderdog nipping at your heelsStretch goal
Whatever keeps the C-suite up at night
Step 5: Define your benchmarks
30
Relative positive vs. negative tone in SM vs. MSM
31
First: find out what already existsWeb analyticsCustomer Satisfaction dataCustomer loyalty data
Second: Decide what research is needed to give you the information you need:
Step 5: Conduct research (if necessary)
Step 6: Selecting a measurement tool
Objective KPI Tool
Increase inquiries, web traffic, recruitment
% increase in traffic#s of clickthrus or downloads
Google Analytics, Clicktracks, Web trends
Increase awareness/preference
% of audience preferring your brand to the competition
SurveyMonkey, Zoomerang
Engage marketplace Conversation index greater than .8Rankings
TypePad, Technorati
Communicate messages
% of articles containing key messagesTotal opportunities to see key messagesCost per opportunity to see key messages
Dominance Focal point 3 Not a focal point -1Visibility Headline mention 2 Top -20 % of story 0 Minor mention -2Target publication Top Tier 2 2nd tier 0 Not on target list -2
Total Score 10 0 -10
Optimal Content Score
Standard classifications of discussion
• Acknowledging receipt of information
• Advertising something• Answering a question• Asking a question• Augmenting a previous
post• Calling for action• Disclosing personal
information• Distributing media• Expressing agreement• Expressing criticism• Expressing support• Expressing surprise• Giving a heads up
• Responding to criticism• Giving a shout-out• Making a joke• Making a suggestion• Making an observation• Offering a greeting• Offering an opinion• Putting out a wanted ad• Rallying support• Recruiting people• Showing dismay• Soliciting comments• Soliciting help• Starting a poll• Validating a position
Your tool box also needs to include: 3. A way to measure
engagementThe conversation index=• Ratio of posts to
comments Relationship studiesThe engagement index
41
Managing engagement on your own property
% increase or decrease in unique visits In the past month, what % of all sessions represent more than 5 page views % of sessions that are greater than 5 minutes in duration % of visitors that come back for more than 5 sessions % of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand % of visitors that become a subscriber % of visitors that download something from the site % of visitors that provide an email address
Courtesy of Eric Peterson
For all institutions, most postings were simply making an observation or distributing media.
Page 43
3
6
1
1
7
36
1
29
5
15
14
2
16
1
2
12
7
2
6
2
24
787
3
2
203
12
12
46
11
1
3
2
1
4
1
4
3
6
2
1
13
2
2
1
13
2
6
18
4
1
1
5
35
3
17
2
8
9
1
1
1
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Acknowledging receipt of information
Advertising Something
Answering a question
Asking a question
Augmenting a previous post
Calling for action
Disclosing personal information
Distributing media
Expressing criticism
Expressing support
Expressing surprise
Giving a heads-up
Giving a shout-out
Making a suggestion
Making an observation
Offering an opinion
Playing a game
Rallying support
Recruiting people
Showing dismay
Share of Conversation Types
Arizona State
Michigan State
Penn State
Purdue University
University of Michigan
44.2%
6.5%
30.9%
49.5%
100.0%
100.0%
100.0%
1.6%
53.9%
100.0%
26.9%
23.1%
10.8%
38.7%
72.7%
10.9%
15.5%
46.1%
66.6%
27.3%
35.1%
39.7%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Acknowledging receipt of information
Advertising Something
Answering a question
Asking a question
Augmenting a previous post
Calling for action
Disclosing personal information
Distributing media
Expressing criticism
Expressing support
Expressing surprise
Giving a heads-up
Giving a shout-out
Making a suggestion
Making an observation
Offering an opinion
Playing a game
Rallying support
Recruiting people
Showing dismay
Share of Engagement by Conversation Type - Institutional Blogs
Arizona State
Michigan State
Penn State
Purdue University
University of Michigan
cx
Share of conversation vs share of engagement
Page 44
2
2
1
2
1
6
5
3
1
1
1
1
1
1
1
1
1
1
2
2
2
1
2
1
2
1
4
2
1
4
2
1
1
4
1
6
7
6
2
2
2
2
1
3
2
3
1
0 2 4 6 8 10 12 14 16 18 20
Faculty
Students
Research, Physical Sciences
Courses
Research, Earth Sciences
Projects, Non -Research
Financials
Alumni Topics
Research, Life Sciences
Staff
Admissions
Legal News
Other
Research, Agriculture
Policies
Institution, Overall
Campus Life
Research, Social Sciences
Share of Subject
Peer 1
Michigan State
Peer 2
Peer 3
Peer 4
15.3%
68.7%
100.0%
4.4%
33.3%
96.8%
28.6%
34.9%
12.5%
43.3%
28.6%
13.0%
38.3%
100.0%
23.6%
66.7%
6.3%
28.6%
20.8%
2.3%
95.6%
33.2%
5.8%
28.6%
100.0%
86.8%
13.0%
31.0%
22.1%
3.2%
71.4%
43.5%
18.8%
94.2%
56.7%
14.2%
13.2%
53.2%
28.4%
21.1%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Admissions
Alumni Topics
Campus Life
Community Relations
Courses
Events
Faculty
Financials
Institution, Overall
Inventions
Legal News
Other
Partnerships
Policies
Projects, Non - Research
Research, Agriculture
Research, Earth Sciences
Research, Life Sciences
Research, Other
Research, Physical Sciences
Research, Social Sciences
Staff
Students
Share of Engagement by Subject - ,External Blogs
Peer 1
Michigan State
Peer 2
Peer 3
Peer 4
The vast majority of discussion in external blogs is neutral.
Page 45
23
29
12
14
20
5
8
4
1
4
0
5
10
15
20
25
30
University of Michigan Purdue University Penn State Michigan State Arizona State
Share of Tone
Negative
Neutral
Positive
71%
3%
29%
94%
83%
42%
58%
6%
14%
58%
42%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Arizona State Michigan State Penn State Purdue University University of Michigan
Share of Engagement by Tone - External Blogs
Negative
Neutral
Positive
Aspects of relationships
Control mutualityTrustSatisfactionCommitmentExchange relationshipCommunal relationship
46
Components of a Relationship IndexControl mutuality
In dealing with people like me, this organization has a tendency to throw its weight around. (Reversed)This organization really listens to what people like me have to
say.Trust
This organization can be relied on to keep its promises.This organization has the ability to accomplish what it says it
will do.Satisfaction
Generally speaking, I am pleased with the relationship this organization has established with people like me.Most people enjoy dealing with this organization.
CommitmentThere is a long-lasting bond between this organization and
people like me.Compared to other organizations, I value my relationship with
this organization moreExchange relationship
Even though people like me have had a relationship with this organization for a long time; it still expects something in return whenever it offers us a favor.This organization will compromise with people like me when it
knows that it will gain something.This organization takes care of people who are likely to reward
the organization.Communal relationship
This organization is very concerned about the welfare of people like me.I I think that this organization succeeds by stepping on other
people. (Reversed)
How to implement relationship metrics
Step 1: Conduct a benchmark relationship studyStep 2: Implement PR programStep 3: Conduct a follow up
relationship studyStep 4: Look at what’s changed
Look for failures firstCheck to see what the competition is
doing Then look for exceptional successCompare to last month, last quarter,
13-month averageFigure out what worked and what
didn’t workMove resources from what isn’t
working to what is
Step 7: Analysis - -Research without insight is just trivia
49
Best Practices:
Correlations to bottom-line impact
DonationsMembershipsSign-upsLeads
Using SMM for planning
Define the time frame, market/topic you want to studyUse Google News,
Technorati or Radian6 to identify the conversations around the topic Analyze the
conversations for type, tone and positioningLook at share of
positioning, tone or conversation
Benchmarking against your peers
Looking at what the best doSetting goals
accordinglyUse data to persuade
recalcitrant spokespeople
Social Media in CrisisListen instantly to a wide
range of influencersIdentify weaknesses in
communications, customer service, or in the product
Improve your reputation
Listen first, then respondStop doing stupid things
Past performance: tonality of blog content
Tonality of Coverage Over Time
4 9 5 9
27
37 43
91
17
914
12
0
25
50
75
100
125
150
Oct Nov Dec Jan
2006 2007
Men
tions
Positive
Neutral
Negative
Tonality of all blog postings
Total, 10%
Total, 71%
Total, 19%
The competitive landscape
Technorati mentions with high authority
Cingular7%
Sprint7%
Verizon10%
T-Mobile75%
US Cellular1%
Company "sucks" mentions in Technorati with high authority
US Cellular2% Cingular
16%
Sprint12%
Verizon19%
T-Mobile51%
Using SMM for planning
The environmental scanDefining issues
in a marketSelecting a
positioning that works
Using SMM in a Crisis
Social Media in Crisis
Listen instantly to a wide range of influencersIdentify weaknesses in communications, customer service, or in the product
Case Study: Establishing benchmarks at Georgia Tech
Quantity and quality of discussion of Georgia Tech and four peer institutions across relevant user-generated media (UGM) channels in order to:
• Establish performance benchmarks• Observe user habits to inform UGM
strategies• Understand the influence of traditional
media on UGM channels• Provide support for funding of UGM
programs
Case Study: Georgia Tech
User Generated Media 57
During a crisis, UGM channels more likely to be negative
UGM amplified negative traditional media coverage.
Unusual negative stories, like MIT’s fake bomb scare, became popular on social bookmarking sites.
Negative news linked to politics was a mainstay on external blogs.
Facebook profiles amplified each of these effects, and also included critical pieces from campus newspapers.
0.0%0.0%0.0%3.8%
1.9%
14.5%
10.6%
1.9% 0.5%1.3%
13.2%
0.8%
YouTube(1,718 | 194)
SocialBookmarking
(310 | 5)
InstitutionBlogs
(317 | 12)
FacebookPopularTopics
(76 | 21)
External Blogs
(367 | 26)
TraditionalMedia
(2,802 | 154)
Georgia Tech Only All Institutions
Percent of Content Considered Negative Per Channel
58
UGM channels offered equal opportunity for message communication
18%
18%
42%
24%
42%
40%
YouTube (N=194)
Social Bookmarking(N=5)†
Institution Blogs(N=12)†
Facebook PopularTopics (n=22)†
External Blogs(N=26)†
Traditional Media(N=154)
Percent of Georgia Tech Content That Communicated One or More Strategic Messages
Overview of Key Metrics
Bookmark.
Facebook
Ext. Blogs
Inst. Blogs
YouTube
MSM
SOV 2% — 8% 9% 11% 7%
Popularity
230 bkmks
500/mo.
—20
links150k views
—
Engagement
59 cmts
1 day13
cmts2-12 cmts
2 cmts —
% Positive
20% 32% 54% 50% 15% 15%
% Negative
0% 0% 4% 0% 1% 2%
Strat. Mess.
40%† 18%† 42% 42%† 18% 38%
Peer 1 was the competitive leader in all but YouTube, where Peer 4 and Peer 3 led.Actions attributed to individuals were responsible for most content, except on YouTube.
† Small base size. Findings are directional only.
Top 5 Subjects of discussion in each channel
Rank Order
Facebook YouTube Social Bookmarking
External Blogs
Institutional Blogs
1 Campus Life
Events Courses Faculty Campus Life
2 Sports Campus Life
Projects, Non-Research
Research, Physical Sciences
Events
3 Technology Faculty Research, Physical Sciences
Institution Overall
Institution Overall
4 Product Services
Courses Events Expert Commentary
Institution Sub-Groups
5 Events Institution Overall
Faculty Events Admissions
Few subjects appear across all forms of social media, so tailor outreach accordingly
Peer 1 dominates social bookmarking, institutional blogs, Peer 4 leads on Facebook
Focus on Facebook
Less than one percent of users used network-level discussion features.
By September, discussion hosted by freshman groups decreased 99%.
Almost 1/3 of content posted to profiles was related to a home institution.
22% of Facebook discussion was related to the asking and answering of questions, second only to advertising (30%).
56% of questions went unanswered, but most were not related to the institution.
High school students accounted for 8% of all questions. Almost all of their queries were answered.
Where people get the content they share on Facebook
Sources of content
Genre of content
Influence of traditional media
On average, bloggers included as many as six links to external content in a post, the number three source being traditional news media sites.
Links to its newsroom accounted for 26% of links to mit.edu on blogs.
On Facebook, traditional news media sites were the source of 25% of popular items posted to profiles.
One third of content on social news sites was from traditional media sources.
Twice as many hard news stories were posted to social news sites as features.BBC Boston Globe CNET CNN
EurekAlert! Google News Los Angeles Times The New York Times
Pittsburgh Post-Gazette San Francisco Chronicle Washington Post
Selected Traditional Media Outlets Among Popular Sources of Content
Facebook Recommendations
Limit engagement with Facebook to contact with group officersDo NOT participate in discussions on
the network wall or discussion boardProvide administrators of freshman
groups with links to online resources no later than April Consider using Facebook to create
with other specific audiences like parents, graduating seniors and campus leadersDo not consider Facebook an
appropriate vehicle for research discussions
Understanding brand ownership of online video content
N=2,555,691
Peer Organizations
4.33%
Your Organization0.18% Other
Organizations8.65%
Individual Users86.84%
Use ownership to signal brand participation
Provide alerts for possible brand management issues
YouTube Recommendations
Use YouTube as a vehicle for strategic message communicationTailor materials related to high
profile competitions Prepare media infrastructure for
increased emphasis on online videoEncourage faculty members to be
subjects of videos
Focus on Social Bookmarking
In the event of a crisis, expect seeding from local papersThursday & Friday saw the greatest
number of seeds. GIT’s status as a technical
institution is an asset in the social bookmarking environment Few strategic messages appeared
in social bookmarking sites
External Blog Recommendations
Consider external blogs an opportunity for third-party endorsementsTreat influential external bloggers as you
would industry analysts or key reportersFocus efforts on blogs written by more
than one person, particularly in engineering and special focus areas Avoid local mainstream media blogsFocus on top-tier media outlets as key
sources of content for bloggersInclude blogger-friendly features in the FT
online newsroom – particularly video In a crisis, expect bloggers to collect
background from personal web pages, user profiles and/or project sites
Focus on Institutional Blogs
Departments generated the most number of blog postings/ inbound links among peer institutions Most blogs are written by individualsThe location of links played the largest role
in driving commentsTechnology drove the largest number of
posts, but personal life drove commentsMost posts consisted of making an
observation, most comments asked questionsPhotographs were most frequently used
multimedia contentInstitutional bloggers were significantly
more likely to be positive toward their home institutions than mainstream journalistsCurrently enrolled students wrote one in
five comments
Recommendations for Institutional Blogs
Recruit faculty to blogGuide message communicationsTailor institutional blogs to the
audiences looking for more in-depth informationEncourage bloggers to be opinionatedMix in personal subjects Leave frequency of posting up to the
discretionof the bloggerRemove abandoned blogs Unify blogs with easy-to-find thematic
lists of bloggersMake it easy to share content from your
institutional blogs – ie. lots of music and visuals
Thank You!
For more information on measurement, read my blog: http://kdpaine.blogs.com or subscribe to The Measurement Standard:
www.themeasurementstandard.comFor a copy of this presentation
go to: http://www.kdpaine.comFollow me on Twitter: KDPaineFriend me on Facebook: Katie