Make Children’s Lives Better: Measuring Media Relations at UNICEFDOUN I A KCHI ER E , PROJECT M A N AGER, M EA SUR EM EN T, UN I CEF
DR . DAVI D ROCK LA ND, PA RT N ER / CEO K ETCHUM GLOBA L R ESEARCH & A N A LYT ICS
1
Agenda
2
The Principles of Communications Measurement1
Measurement is Based on Setting Goals First2
Voice, Reach and Engagement Measurement3
Taking it Further: Outcome and Organizational Results Measurement
4
The Principles of Communications Measurement
3
20102000 2015
Clip Books…
• Messy
• Cumbersome
• Time Consuming
Evolution of PR
4
Barcelona Principles 2.0
Barcelona Principles Version 1
The Principles of Communications Measurement
5
• Our Original Goal: End years of debate as to whether metrics such as AVEs and multipliers should be used, and provide a foundation and hierarchy.
The Seven PrinciplesOriginal 2.0
1. Importance of Goal Setting and Measurement2. Measuring the Effect on Outcomes is Preferred to
Measuring Outputs3. The Effect on Business Results Can and Should Be
Measured Where Possible4. Media Measurement Requires Quantity and Quality5. AVEs are not the Value of Public Relations6. Social Media Can and Should be Measured7. Transparency and Replicability are Paramount to
Sound Measurement
1. Goal Setting and Measurement are Fundamental to Communication and Public Relations
2. Measuring Communication Outcomes is Recommended Versus Only Measuring Outputs
3. The Effect on Organizational Performance Can and Should be Measured Where Possible
4. Measurement and Evaluation Require Both Qualitative and Quantitative Methods
5. AVEs are not the Value of Communication6. Social Media Can and Should be Measured
Consistently with Other Media Channels7. Measurement and Evaluation Should be Transparent,
Consistent and Valid
6
Measurement is Based on Setting Goals First
7
Measurement is based on setting goals first
Who?
What?
How Much?
By When?
8
Goals should be defined as quantitatively as possible
Bad Goal Versus Good GoalBAD GOAL GOOD GOAL
Drive media coverage Through targeted media relations, reach 10M target audience members by the end of 2010. Deliver messages in 60% of all coverage.
Change the perception from a company that makes computers to a company that is a leader in global business solutions
Achieve 65% strategy comprehension in top 3 boxes on pulse survey fielded at end of year
Create “card envy” among affluent consumers in the San Francisco market
Increase awareness from 57% to 60% for the credit card brand among the affluent target (HHI $125K+) in San Francisco within campaign timeframe (May 2012 –Dec 2012)
Set emotional connection with consumers (consideration)
By the end of 2009, raise 1st choice consideration (by 5%) of available lens and eyeglass among GenXers, 40-45 years old
9
UNICEF’s Global Communication and Public Advocacy Strategy
1. Put the rights and well-being of the most disadvantaged children at the heart of
the social, political, and economic agendas,
2. Support shifts in public policy,
3. Fuel social engagement,
4. Increase private and public resources for children.
10
Shifting our communication model to support in realizing the rights of every child, especially the most disadvantaged ones
UNICEF’s Audiences
11
Youth and Millennials Middle Class
The power to bring change is expanding as people and communities make their voices heard –• reach out more directly to civil society• leverage the collective force of individuals uniting around causes• Work closely on advocacy issues to connect with vulnerable communities and children.
Strategy Pillars
12
Voice
Engagement
Reach
Cross-Cutting
Be the leading voice for – and with- Children
Reach 1 Billion people around the world
Engage 50 Million people acting in support of children
Contribute to the positioning of UNICEF’s Brand
Measurement Framework: Theory of Change
13
Activities
Inputs
Output
Outcomes
Internal planning and allocation of resources
Focus on target media channels (Voice and reach)
Focus on individuals and UNICEF’s key audiences (engagement and advocacy)
Focus on UNICEF strategic goals
Quantity and quality of communications activities
Impacts
Glocal rollout
14
61Country Offices are currently rolling out the global strategy.
16Country office have media monitoring companies in place and over 10 are currently hiring a service provider.
5329 COs and 24 NatComs have been engaged in the M&E onboarding process.
11are regularly tracking the KPIs proposed in the new global strategy.
R68% of all KPIs were completed
. 38%Continuously reporting on the global scorecards
Learning from past experience
1) There is no ‘one model fits all’ approach to all countries. Flexibility is key to achieve our
objectives
2) Human and financial resources are key to a successful implementation
3) Change-management efforts to inculcate the importance of measurable results
4) Measuring Advocacy impact is complex and requires looking beyond UNICEF mentions
5) Collecting and analyzing data allows establishing benchmarks
6) Data triggers our thinking, as a first step
15
Voice, Reach and Engagement Measurement
16
• Quantity of Communications Activitieso Metric: Number of Communications Activities. Examples:
– Distribute X number of publications with 3+ key messages by the end of 2017 targeting X number of influencers and governments
– Publish X number of content pieces on digital/social media by the end of 2017 containing 3+ messages about children’s rights
• Quality of Communications Activities o Metric: Quality of Communications Activities. Examples:
– Overall communications effectiveness
– Reach attendance at X events by X targeted individuals by the end of 2017
• Share of Voice o Metric:% Share of voice. Examples:
– On all stories related to children’s issues
– On Facebook at Twitter
Voice KPIs
17
• Quantity of People Reachedo Metric: Impressions to target/other audiences. Examples:
– Reach 1 billion people with UNICEF messages
– UNICEF receives substantive mention in top tier publications
– Number of people reached via mobile phones
– Volume of traffic to websites and blogs
– People attending events
• Quality of Message Deliveryo Metric: Media algorithm score. Example:
– Using a media algorithm, achieve a quality score of 60+ on -100 to +100 scale in top tier publications
– Achieve at 50% or higher number of articles that focus solely on UNICEF in conjunction with improving the lives of children
Reach KPIs
18
• Quantity and quality of social/digital mediao Metric: Deep interactions with UNICEF content. Examples:
– Increase rate of users who like, comment or share UNICEF’s Facebook posts by X% by the end of 2017
– Increase number of re-Tweets of UNICEF’s content by X% by the end of 2017– Increase UNICEF mentions in digital media by X% by the end of 2017– Number of followers and supporters on line
• Behavioral o Metric: Actions by target audiences. Examples:
– Volunteers– Pledges– Signatories– Contacts to legislators/regulators
Engagement KPIs
19
Quarterly ReportsOverview
20
Quarterly ReportsVoice analysis
21
Quarterly ReportsVoice analysis
22
1,7
19
1,0
34
602
245
193
73
77%
46%
27%
11% 9%3%
0%
25%
50%
75%
100%
0
500
1000
1500
2000
Features message
UNICEF mentioned at
least twice
Features UNICEF
spokesperson
Endorsement by Ambassador
UNICEF Photo/Graphic
Call to action
Perc
en
tag
e o
f Co
vera
ge
Vo
lum
e o
f Art
icle
s
Impact Measures
Prominence
24%
35%
41%
Headline/Lead
paragraph mention
Prominent mention
Passing mention
1%
37%
62%
Unfavorable
Neutral
Favorable
ProminenceFavorability
24%
4%
6%
5%
12%3%
7%
15%
3%
21%UNICEF
International Red Crescent/Red
CrossMedecins Sans Frontieres
Oxfam
Save the Children
UNFPA
WHO
World Bank
World Food Programme
UNHCR
Traditional Media Share of Voice
Quarterly ReportsReach & Engagement Analysis
23
Quarterly ReportsBrand Analysis
24
Campaign ReportsClimate Change – COP21
Objectives:
Put children at the heart of the global conversation about climate change, highlighting the effect of climate change on the poorest and the most vulnerable children.
Engage all stakeholders in the call for ambitious commitments to scaling up action and investment in climate change education, training, and awareness raising.
Urge investment in children when implementing national climate mitigation and adaption plans and protecting children on the move who were displaced as a result of climate change or climate-related impacts.
25
Campaign Reports Implementation
26
Campaign Reports Monitoring online and broadcast media
27
Key message Penetration (top messages):1. “Nearly 160 million children live in high or extremely high drought severity zones.”2. “Drought will cause crops to fail, livestock to die and family incomes to decrease, which can lead to greater food
scarcity and undernutrition.”3. “Following COY, some of the young people will present the project [digital youth mapping] initiative at COP21.”
Press Clipping Volume of coverage
Analysis between 10th November to 12th December 17th
Campaign Reports Timeline and Top Tier Media Mentions Triggers
28
Campaign Reports Monitoring social media
29
Hashtags used by UNICEF
#COP21, #climatechange,
#youthtakeover,
#earthtoparis,
#eyesonparis,
#worldupsidedown,
#uniceflive, #COY11,
#ActNowFT
Analysis between 10th November to 12th December 17th
Campaign Reports Analysis of Social Media
30
Tweet Time: 5:35Impressions: 286,337Engagements: 1,915Engagement Rate: 0.69%
Share of Voice
on issues
related to
children &
climate change
Post Type: VideoReach: 4,068,031Likes: 31,608 | Comments: 1,101Shares: 12,765 |Post Clicks: 175,418
Analysis between 10th November to 12th December 17th
Recommendations & Lessons Learned
Carefully choosing key moments to deliver key messages so that UNICEF’s voice is not crowded out by competing events.
Targeting relevant key audiences using tailored channels, stories, languages and calls to actions.
Developing clear calls to action to ultimately drive traffic to the UNICEF website.
Leveraging spokespeople and goodwill ambassadors to amplify UNICEF’s voice in traditional media and to reach new audiences
31
Taking it Further: Outcomes and Organizational Results
32
R2 = .668
If LLS decreased its current annual direct mail spend 10% and spent it on online, it would lose 5,975 direct mail leads, but gain 29,020 leads from online, a net gain of 23,045 or potentially $64,526,466 (23,045 X $2,800 average amount raised per participant in FY 10).
Channel Leads* Generated Per $1,000
Total Leads Generated (Annual)
Direct Mail 7 59,747
Radio 5 4,645
Online 33 6,249
PoP H 10 2,270
PoP P 23 11,126
APPROACH: Regression analysis was used to determine the relationship between marketing spend made via direct mail, PoP H, PoP, radio, and online (independent variables) and two outcomes, leads and final registration.
ANSWER: Online was the most cost effective marketing spend, almost five times as efficient as direct mail. Strategic professional PoP outreach was also more effective than hodgepodge PoP outreach.
Model 1:Leads § = 96.51 + 0.007(Direct Mail )+ 0.005(Radio) + 0.033(Online) + .010(PoP H) + .023(PoP P) + 136(If summer) +168(If fall)
Leukemia & Lymphoma Society: Analytics
33
34
Cleveland Clinic, while one of the premier health institutions in the United States, could improve on key metrics compared to its greatest competitors: Johns Hopkins and Mayo Clinic.
Specifically, Cleveland Clinic looks to KGRA to guide them on improving reputation and key behaviors among its target audience, influencers who lead and impact trends and policies.
Key behaviors of interest are:
• Likelihood to travel to Cleveland Clinic for a serious or life-changing medical condition
• Likelihood to refer to Cleveland Clinic as a leader in healthcare policy in the U.S.
Business Challenge
• Analyzed survey data using advanced statistical analysis including factor analysis and regression models to determine the most effective channels, messengers, media channels, message topics, capabilities to improve the reputation and key behaviors
• Applying this research, KGRA developed recommendations to optimize external communications strategy.
Solution
35
CC Reputation
and Key
Behaviors
Messages
Trusted
Visionary
Innovative
Messengers
Referral from Friend/Family
Seeing CC Doctors Quoted
Media Topics
Healthcare Policy
Best Doctors
Channels
Daily/National Newspapers
National TV Network News
Knowledge of Capabilities
Brain Cancer/TumorsUrological Cancer
Kidney CancerUrological Issues
Recommended Components of CC’s 2016 Communication Strategy
36
Moving Cleveland Clinic’s reputation closer to competitors will require small incremental changes in key drivers over time.
The example below demonstrates how transparency is integral part of Cleveland Clinic’s key messages, and how these message drivers could close the reputation gap with Cleveland Clinic’s competitors if each were improved by one point.
TRANSPARENCYpredicts
41 % to 61%
of CC’s Key Messages
KEY MESSAGESTrusted, Innovative & Visionary
predicts
51 % to 58%
of CC’s Reputation and Key Behaviors
8.4
9.0 8.90.3
8
9
10
CC Mayo JH
Improvement in Reputation
10
9
8CC May
oJH
CC Reputation Score vs. Competitors if Performance of all Three Key Message
Drivers Increased by 1-point
Questions?