People talk, We Listen Social Media Measurement Master Class A presentation to US Army Worldwide Public Affairs Symposium Crystal City, VA May , 2009 Katie Delahaye Paine CEO [email protected]www.kdpaine.com kdpaine.blogs.com Member, IPR Measurement Commission www.instituteforpr.org
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
People talk, We Listen
Social Media Measurement Master Class
A presentation to US Army Worldwide Public Affairs SymposiumCrystal City, VA
“The main reason to measure objectives is not so much to reward or punish
individual communications manager for success or failure as it is to learn from the
research whether a program should be continued as is, revised, or dropped in favor of another approach ”
James E. Grunig, Professor Emeritus, University of Maryland “If we can put a man in orbit, why can’t we determine the effectiveness of our communications? The reason is simple and perhaps, therefore, a little old-fashioned: people, human beings with a wide range of choice. Unpredictable, cantankerous,capricious, motivated by innumerable conflicting interests, and conflicting desires.”
Ralph Delahaye Paine, Publisher, Fortune Magazine , 1960 speech to the Ad Club of St. Louis
2
People talk, We Listen
Communications then and now
Traditional role of Marketing & Communications
21st Century Role of PR
People talk, We Listen
People talk, We Listen
Page 4
12 Signs that it’s the end of the world as we know it 1. The Dept of Defense considers Twittering and other
forms of social media critical to national security2. BestBuy measures 85% lower turnover as a result of its
Blue Shirt community3. BMC Software tracks social media benefits direct to its
bottom line and EPS.4. NWF is using Twitter to spot, identify and protect wildlife 5. American Red Cross can measure improvement in
disaster management using Twitter 6. ASPCA correlates increases in on-line donations and
increased membership with its social media efforts. 7. HSUS generated $650,000 in contributions from a Flickr
photo contest 8. Wal-Mart is training and empowering its employees
closest to the customers to be the ones engaging online and provides social media tools to enable them to engage: my.walmart.com
9. Intel is using Facebook and internal Twitter tool to help employees engage with customers
10. Dell has made more money with Twitter than Twitter has 11. Comcast uses Twitter as crm tool, @comcastcares is the
new consumer affairs model12. The examples for this presentation came from people
Step 4: Define your Key Performance Indicators (KPIs)
Cost savingsEfficiency
Cost per message communicatedCost per new recruit acquired
Productivity: Increase in employee engagement/moraleLower turnover/recruitment costs
Engagement: Ratio of posts to comments% of repeat visitors% of 5+min visitors% of registrations
Trust:Improvement in relationship /reputation scores with customers and communities (Loyalty/Retention)
Thought leadership: Share of quotesShare of opportunities
Message penetrationPositioning on key issuesImprovement in favorable/unfavorable ratioImprovement in Optimal Content Score (OCS)
18
People talk, We Listen
What makes a perfect communications KPI?
Gets you where you want to go (achieves corporate goals) Is actionable by individuals as well as departmentsContinuously improves your processes Is there when you need it
People talk, We Listen
Why an Optimal Content Score?
You decide what’s important:Benchmark against peers and/or competitorsTrack activities against OCS over time Positive:
Mentions of the brandKey messagesPositioningVisibility
Negative OmittedNegative toneNo key message
20
People talk, We Listen
How to calculate Optimal ContentQuality score +1 0 -1
Dominance Focal point 3 Not a focal point -1Visibility Headline mention 2 Top -20 % of story 0 Minor mention -2Target publication Top Tier 2 2nd tier 0 Not on target list -2
Total Score 10 0 -10
Optimal Content Score
People talk, We Listen
Emerging benchmarksEngaged = 3-13 comments per postHyper-engaged = 15-35 comments per postAfter 3 days most comments are done, 14 days maxSocial Bookmarking momentum = 1 submitted item every other dayMessage should be communicated in 2 out of 5 blogs
Past PerformanceThink 3
Peer organization in Navy/Marines/Airforce/CoastGuardThe enemy Another similar organization that you admire
Whatever keeps the C-suite up at night
Step 5: Define your benchmarks
22
People talk, We Listen
Overview of Key Metrics
Bookmark.
Facebook
Ext. Blogs
Inst. Blogs
YouTube
MSM
SOV 2% — 8% 9% 11% 7%
Popularity
230 bkmks
500/mo.
—20
links150k views
—
Engagement
59 cmts
1 day13
cmts2-12 cmts
2 cmts —
% Positive
20% 32% 54% 50% 15% 15%
% Negative
0% 0% 4% 0% 1% 2%
Strat. Mess.
40%† 18%† 42% 42%† 18% 38%
Peer 1 was the competitive leader in all but YouTube, where Peer 4 and Peer 3 led.Actions attributed to individuals were responsible for most content, except on YouTube.
† Small base size. Findings are directional only.
People talk, We Listen
Top 5 Subjects of discussion in each channel
Rank Order
Facebook YouTube Social Bookmarking
External Blogs
Institutional Blogs
1 Campus Life
Events Courses Faculty Campus Life
2 Sports Campus Life
Projects, Non-Research
Research, Physical Sciences
Events
3 Technology Faculty Research, Physical Sciences
Institution Overall
Institution Overall
4 Product Services
Courses Events Expert Commentary
Institution Sub-Groups
5 Events Institution Overall
Faculty Events Admissions
Few subjects appear across all forms of social media, so tailor outreach accordingly
People talk, We Listen
First: find out what already existsWeb analyticsCustomer Satisfaction dataCustomer loyalty data
Second: Decide what research is needed to give you the information you need:
Message content analysisSurveys
Step 5: Conduct research (if necessary)
25
People talk, We Listen
Step5: Selecting a measurement tool based on your KPIs
Objective Metric Tool
Increase inquiries, web traffic, recruitment
% increase in traffic#s of clickthrus or downloads
Google Analytics, Omniture
Increase awareness/preference % of audience preferring your brand to the competition
Survey Monkey, Zoomerang,
Engage marketplace Conversation index greater than .8Rankings
Type pad, Technorati, Social Mention, ISPY
Communicate messages % of articles containing key messagesTotal opportunities to see key messagesCost per opportunity to see key messages
Your tool box also needs to include: 2. A way to analyze that content
Automated vs. Manual Census vs random sampleThe 80/20 rule – Measure what matters because 20% of the content influences 80% of the decisionsDashboards aggregate data
• Advertising something• Answering a question• Asking a question• Augmenting a previous
post• Calling for action• Disclosing personal
information• Distributing media• Expressing agreement• Expressing criticism• Expressing support• Expressing surprise• Giving a heads up
• Responding to criticism• Giving a shout-out• Making a joke• Making a suggestion• Making an observation• Offering a greeting• Offering an opinion• Putting out a wanted ad• Rallying support• Recruiting people• Showing dismay• Soliciting comments• Soliciting help• Starting a poll• Validating a position
Your tool box also needs to include: 3. A way to measure
engagementThe conversation index=• Ratio of posts to
comments Relationship studiesThe engagement index
31
People talk, We Listen
People talk, We Listen
Page 32
A Proposed Engagement Index
ClickthruDonations/
ordersSignups
Time on siteRepeat visits
Forwards/links
/comments
RelationshipsTone/content of
conversationMembership
An engagement index?
Output Outtake Outcome
+ +
People talk, We Listen
Share of conversation vs share of engagement
Page 33
2
2
1
2
1
6
5
3
1
1
1
1
1
1
1
1
1
1
2
2
2
1
2
1
2
1
4
2
1
4
2
1
1
4
1
6
7
6
2
2
2
2
1
3
2
3
1
0 2 4 6 8 10 12 14 16 18 20
Faculty
Students
Research, Physical Sciences
Courses
Research, Earth Sciences
Projects, Non - Research
Financials
Alumni Topics
Research, Life Sciences
Staff
Admissions
Legal News
Other
Research, Agriculture
Policies
Institution, Overall
Campus Life
Research, Social Sciences
Share of Subject
Peer 1
Michigan State
Peer 2
Peer 3
Peer 4
15.3%
68.7%
100.0%
4.4%
33.3%
96.8%
28.6%
34.9%
12.5%
43.3%
28.6%
13.0%
38.3%
100.0%
23.6%
66.7%
6.3%
28.6%
20.8%
2.3%
95.6%
33.2%
5.8%
28.6%
100.0%
86.8%
13.0%
31.0%
22.1%
3.2%
71.4%
43.5%
18.8%
94.2%
56.7%
14.2%
13.2%
53.2%
28.4%
21.1%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Admissions
Alumni Topics
Campus Life
Community Relations
Courses
Events
Faculty
Financials
Institution, Overall
Inventions
Legal News
Other
Partnerships
Policies
Projects, Non - Research
Research, Agriculture
Research, Earth Sciences
Research, Life Sciences
Research, Other
Research, Physical Sciences
Research, Social Sciences
Staff
Students
Share of Engagement by Subject - ,External Blogs
Peer 1
Michigan State
Peer 2
Peer 3
Peer 4
People talk, We Listen
The vast majority of discussion in external blogs is neutral.
Page 34
23
29
12
14
20
5
8
4
1
4
0
5
10
15
20
25
30
University of Michigan Purdue University Penn State Michigan State Arizona State
Share of Tone
Negative
Neutral
Positive
71%
3%
29%
94%
83%
42%
58%
6%
14%
58%
42%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Arizona State Michigan State Penn State Purdue University University of Michigan
Share of Engagement by Tone - External Blogs
Negative
Neutral
Positive
People talk, We Listen
For all institutions, most postings were simply making an observation or distributing media.
Page 35
3
6
1
1
7
36
1
29
5
15
14
2
16
1
2
12
7
2
6
2
24
787
3
2
203
12
12
46
11
1
3
2
1
4
1
4
3
6
2
1
13
2
2
1
13
2
6
18
4
1
1
5
35
3
17
2
8
9
1
1
1
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Acknowledging receipt of information
Advertising Something
Answering a question
Asking a question
Augmenting a previous post
Calling for action
Disclosing personal information
Distributing media
Expressing criticism
Expressing support
Expressing surprise
Giving a heads-up
Giving a shout-out
Making a suggestion
Making an observation
Offering an opinion
Playing a game
Rallying support
Recruiting people
Showing dismay
Share of Conversation Types
Arizona State
Michigan State
Penn State
Purdue University
University of Michigan
44.2%
6.5%
30.9%
49.5%
100.0%
100.0%
100.0%
1.6%
53.9%
100.0%
26.9%
23.1%
10.8%
38.7%
72.7%
10.9%
15.5%
46.1%
66.6%
27.3%
35.1%
39.7%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Acknowledging receipt of information
Advertising Something
Answering a question
Asking a question
Augmenting a previous post
Calling for action
Disclosing personal information
Distributing media
Expressing criticism
Expressing support
Expressing surprise
Giving a heads-up
Giving a shout-out
Making a suggestion
Making an observation
Offering an opinion
Playing a game
Rallying support
Recruiting people
Showing dismay
Share of Engagement by Conversation Type - Institutional Blogs
Arizona State
Michigan State
Penn State
Purdue University
University of Michigan
cx
People talk, We Listen
People talk, We Listen
Page 36
The numbers your web analytics guru should give you every month*1. % increase or decrease in unique visits 2. How many sessions on our blog or web site
represent more than 5 page views 3. In the past month, what % of all sessions
represent more than 5 page views 4. % of sessions that are greater than 5 minutes
in duration 5. % of visitors that come back for more than 5
sessions 6. % of sessions that arrive at your site from a
Google search, or a direct link from your web site or other site that is related to your brand
7. % of visitors that become a subscriber 8. % of visitors that download something from
the site 9. % of visitors that provide an email address* Courtesy of Eric Peterson36
People talk, We Listen
Aspects of relationships
Control mutualityTrustSatisfactionCommitmentExchange relationshipCommunal relationship
37
People talk, We Listen
Components of a Relationship IndexControl mutuality
In dealing with people like me, this organization has a tendency to throw its weight around. (Reversed)This organization really listens to what people like me have to say.
TrustThis organization can be relied on to keep its promises.This organization has the ability to accomplish what it says it will do.
SatisfactionGenerally speaking, I am pleased with the relationship this organization has established with people like me.Most people enjoy dealing with this organization.
CommitmentThere is a long-lasting bond between this organization and people like me.Compared to other organizations, I value my relationship with this organization more
Exchange relationshipEven though people like me have had a relationship with this organization for a long time; it still expects something in return whenever it offers us a favor.This organization will compromise with people like me when it knows that it will gain something.This organization takes care of people who are likely to reward the organization.
Communal relationshipThis organization is very concerned about the welfare of people like me.I I think that this organization succeeds by stepping on other people. (Reversed)
38
People talk, We Listen
How to implement relationship metrics
Step 1: Conduct a benchmark relationship studyStep 2: Implement PR programStep 3: Conduct a follow up relationship studyStep 4: Look at what’s changed
People talk, We Listen
Your tool box needs to include:
4. A way to quantify it all
HITS= How Idiots Track SuccessEyeballs – CompeteGoogle AnalyticsPanelsSurveys
40
People talk, We Listen
Research without insight is just trivia
What works, what doesn’t?What needs to be done? What are you communicating?What tools work best?
Step 7: Analysis
41
People talk, We Listen
Data mining the numbers you have
Look for failures firstCheck to see what the competition is doing Then look for exceptional successCompare to last month, last quarter, last yearFigure out what worked and what didn’t work
People talk, We Listen
Best Practices:
Correlations to bottom-line impact
DonationsMembershipsSign-upsLeads
Using SMM for planning
Define the time frame, market/topic you want to studyUse Google News, Technorati or Radian6 to identify the conversations around the topic Analyze the conversations for type, tone and positioningLook at share of positioning, tone or conversation
Benchmarking against your peers
Looking at what the best doSetting goals accordinglyUse data to persuade recalcitrant spokespeople
Social Media in CrisisListen instantly to a wide range of influencersIdentify weaknesses in communications, customer service, or in the product
Improve your reputation
Listen first, then respondStop doing stupid things
People talk, We Listen
Using SMM in a Crisis
Social Media in Crisis
Listen instantly to a wide range of influencersIdentify weaknesses in communications, customer service, or in the product
People talk, We Listen
Case Study: Engagement vs mentions
Users were positively engaged with advertisements
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Georgia-Pacific Kimberly-Clark Weyerhaeuser
Share of Engagement by Tone for March 2009
Negative Neutral Positive
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Georgia-Pacific Kimberly-Clark Weyerhaeuser
March 2009 Share of Tone by Company
Negative Neutral Positive
Client Competitor 1 Competitor 2
Client Competitor 1 Competitor 2
People talk, We Listen
By percentage, individuals were more engaged with Client subjects than competitors
46
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
March 2009 Share of Engagement by SubjectGeorgia-Pacific Kimberly-Clark Weyerhaeuser
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
March 2009 Discussion by Subject
Georgia-Pacific Kimberly-Clark Weyerhaeuser
(Engagement is the average number of comments per post made to a blog)
Client Competitor 1 Competitor 2
Client Competitor 1 Competitor 2
People talk, We Listen
Discussion of virgin vs. recycled fiber in tissue
47
0
10
20
30
40
50
60
70
80
Blogs youtube Twitter
13
2 2
42
7 9
22
163
Me
nti
on
s
Company Mentions by SourceMarch 2009
Georgia-Pacific
Kimberly-Clark
Weyerhaeuser
Beyond the layoffs, blogs also discussed WY’s decision to close the popular bonsai tree display at its corporate HQ, formerly open to the public.
Client
Competitor 1
Competitor 2
People talk, We Listen
Household product discussion jumped from discussion of a Greenpeace report on toilet tissue
0
20
40
60
80
100
120
Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar
Union activity and environmental concerns drove negative discussion
Four mill closings and other layoffs drove WY’s negative discussion.
3 31
6 6
3
8 7 6 5
8
5
1
2 24
11 1115
12
8
2
19
8 13
6
4 3 2
7
4
6
7
5
6
3
2
3
0
5
10
15
20
25
30
Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar
2008 2009
Me
nti
on
s
Share of Negative Discussion Over Time
Georgia-Pacific
Kimberly-Clark
Weyerhaeuser
Client
Competitor 1
Competitor 2
People talk, We Listen
Case Study: Establishing benchmarks at Georgia Tech
People talk, We Listen
Quantity and quality of discussion of Georgia Tech and four peer institutions across relevant user-generated media (UGM) channels in order to:
• Establish performance benchmarks• Observe user habits to inform UGM
strategies• Understand the influence of traditional
media on UGM channels• Provide support for funding of UGM
programs
Case Study: Georgia Tech
People talk, We Listen
Influence of traditional media
• On average, bloggers included as many as six links to external content in a post, the number three source being traditional news media sites.
• Links to its newsroom accounted for 26% of links• On Facebook, traditional news media sites were
the source of 25% of popular items posted to profiles.
• One third of content on social news sites was from traditional media sources.
• Twice as many hard news stories were posted to social news sites as features.BBC
Boston Globe CNET CNN
EurekAlert! Google News Los Angeles Times The New York Times
Pittsburgh Post-Gazette San Francisco Chronicle Washington Post
Selected Traditional Media Outlets Among Popular Sources of Content
People talk, We Listen
Where people get the content they share on FacebookSources of content
Genre of content
People talk, We Listen
Understanding brand ownership of online video content
N=2,555,691
Peer Organizations
4.33%
Your Organization0.18% Other
Organizations8.65%
Individual Users86.84%
Use ownership to signal brand participation
Provide alerts for possible brand management issues
People talk, We Listen
YouTube Recommendations
Use YouTube as a vehicle for strategic message communicationTailor materials related to high profile competitions Prepare media infrastructure for increased emphasis on online videoEncourage faculty members to be subjects of videos
People talk, We Listen
Focus on Social Bookmarking
In the event of a crisis, expect seeding from local papersThursday & Friday saw the greatest number of seeds. Few strategic messages appeared in social bookmarking sites
People talk, We Listen
External Blog Recommendations
Consider external blogs an opportunity for third-party endorsementsTreat influential external bloggers as you would industry analysts or key reportersFocus efforts on blogs written by more than one person, particularly in engineering and special focus areas Avoid local mainstream media blogsFocus on top-tier media outlets as key sources of content for bloggersInclude blogger-friendly features in the FT online newsroom – particularly video In a crisis, expect bloggers to collect background from personal web pages, user profiles and/or project sites
People talk, We Listen
Focus on Institutional Blogs
Departments generated the most number of blog postings/ inbound links among peer institutions Most blogs are written by individualsThe location of links played the largest role in driving commentsTechnology drove the largest number of posts, but personal life drove commentsMost posts consisted of making an observation, most comments asked questionsPhotographs were most frequently used multimedia contentInstitutional bloggers were significantly more likely to be positive toward their home institutions than mainstream journalistsCurrently enrolled students wrote one in five comments
People talk, We Listen
Recommendations for Institutional Blogs
Guide message communications don’t dictateTailor institutional blogs to the audiences looking for more in-depth informationEncourage bloggers to be opinionatedMix in personal subjects Leave frequency of posting up to the discretionof the bloggerRemove abandoned blogs Unify blogs with easy-to-find thematic lists of bloggersMake it easy to share content from your institutional blogs – ie. lots of music and visuals
People talk, We Listen
Thank You!
For more information on measurement, read my blog: http://kdpaine.blogs.com or subscribe to The Measurement Standard: www.themeasurementstandard.comFor a copy of this presentation go to: http://www.kdpaine.comOr call me at 1-603-868-1550Or email me at [email protected]
Where to find me:
Follow me on Twitter: @kdpaineFriend me on Facebook: Katie PaineSkype: KDPaineLinked In: Katie Delahaye PaineFlickr: kdpaineandpartnersYouTube: kdpaineandpartners