Measurement 101 Measuring Public Relations Roe Roi Ms 2008

Measurement 101:Measurement 101:Measuring Public Relations Measuring Public Relations Impact on ROI by its ROEImpact on ROI by its ROE

Don W. Stacks, Ph.D.School of Communication

University of MiamiCoral Gables, FL 33124

The Institute for Public RelationsCommission on PR Measurement & Evaluation

[email protected]

Overview of theOverview of theEvaluation ProcessEvaluation Process ROI/ROE or

ROE+ROI? Best Practices Benchmark Realistic Goals

& Measurable Objectives

Set Timeline

Establish Methods Qualitative Quantitative “Triangulate”

Measure Evaluate

Against Benchmark

Recommend Action

What to measure?OUTPUTS

What companies/agencies do

OUTCOMESWhat stake-, stockholders & customers do

OUTTAKESWhat stake-, stockholders & customers understand

Best Practices

David Michaelson and Sandra Macleod, The Application of “Best Practices” in Public Relations Measurement and Evaluation Systems, Public Relations Journal, 1 (Fall 2007), www.prsa.org/prjournal/fall07.html.

All public relations actions, whether they be adding a client or preparing for a campaign begin with an analysis of secondary and historical data about the client and its publics.

John W. HillFounder, Hill & Knowlton

1. Benchmark1. Benchmark

BenchmarkBenchmark Data to be evaluated against Available from secondary sources Available from primary sources

Focus Groups Surveys Interviews Participant-Observation

2. ROI?2. ROI? Return on Investment? It’s not your traditional ROI

Financial indicators Sales, Overall Gross, Top Line Growth, etc.

Nonfinancial indicators Credibility, Trust, Relationship, Reputation

Impact of Expectations (ROE) Based on business assumptions

ROI (cont’d)ROI (cont’d) Decision-making is

basically the same in all organizations

All communication (messages) should

Set objectives Determine strategies

to meet objectives Implement strategies

to bring strategies to life

Research can be divided into three general phases Developmental Refinement Evaluation

Communication research is behavior-driven and knowledge-based

Donald K. Wright, as cited in Don W. Stacks, Primer of Public Relations Research, 2nd ed. New York: Guilford Press, in press.

Traditional Thought

Marketing Advertising Public Relations ROI

•Sales Units

•Gross Profits

•800 call tracking

•Reach

•Ad Equivalency

•Placement

•Clips

•Promotion

Marketing impacts on Advertising which impacts on Public Relations

Don W. Stacks, Primer of Public Relations Research, 2nd ed. New York: Guilford Press, in press.

Contemporary ThoughtFinancialIndicators• Unit sales• Gross profits• Expenses

NonfinancialIndicators• Credibility• Trust• Reputation• Relationships• Confidence

Stakeholder-Stockholder Expectations

(ROE)

ROI

A Public Relations Model:Return on Expectations*

Credibility

Relationship

Reputation

Trust

Outcome

Stakeholders

Stockholders

ROE ROI

Confidence

Don W. Stacks, Primer of Public Relations Research, 2nd ed. New York: Guilford Press, in press.

Defining Expectations Credibility

The underlying concept that establishes how people rate trust, relationship, and reputation

Trust A perception that the

evaluation object is dependable

Relationship A perception of

association between people and some entity

Reputation The historical

relationship between people and some entity

Confidence: An expectation of an

outcome

3. Goals & Objectives3. Goals & Objectives Goal: General

outcome expected by campaign end

Objective: Very specific projected outcome

Outputs: individual communication elements

Impact of specific tactics Written Visual Verbal

Objectives are Cause & Effect related Specific to the problem

Types Informational

General and specific knowledge

Motivational (expectation) Attitudinal and belief

oriented Behavioral (most

important!) Actual counts, $$$, and

so forth

Objectives (cont’d.)Objectives (cont’d.)

Informational Objective(s)

Motivational Objective(s)

Behavioral Objectives

Business objective(s)

4. Timeline4. Timeline

Time

Development (Evaluation) Refinement (Evaluation) Final Evaluation

Secondary/Benchmark

Informational/Evaluation

Motivational/Evaluation

Behavioral/Evaluation

Planned benchmarked evaluations

Surveys and Polls Descriptive Explanatory Attitude Opinion Polls

Content Analysis Descriptive Readability Readership

Communication Audit

Delphi Study Focus Group Field Observation

Participant-Observe

Interview Case Study Triangulation

5. Public Relations 5. Public Relations MethodsMethods

Qualitative or Quantitative Qualitative or Quantitative Methods?Methods?

Qualitative: Questions of definition, value and policy Intense, but small sample In-depth knowledge vs. Generalizability Examples

Focus Groups Participant-Observation Informal Observation In-depth Interviewing Case Study

Methods (cont’d.)Methods (cont’d.) Quantitative: Questions of definition and

fact Scientific Large samples Generalizability vs. In-depth understanding Reliable, representative sampling Examples

Surveys (descriptive, explanatory, attitude) Opinion polls Delphi studies Experiments

Triangulation

Secondary Qualitative

Quantitative

Methods (cont’d.)Methods (cont’d.)

Triangulation Use both qualitative and quantitative

methods to better describe, understand, predict, and control public relations campaigns

Provides both representative sampling and in-depth knowledge of the publics or audiences under study

Takes the case study into the “real” world

6. Roadmap to Measuring Behavior

Behavior

Opinions

Attitudes

Beliefs

Values Motivation to BehaveInfluenced by

Messages Aimed At:

Measurement MatrixMeasurement Matrix

Objective/Phase

Information

Motivation Behavior

Benchmark Developmental

Refinement

Final

7. Sample ROE Evaluation*7. Sample ROE Evaluation*

Core or Brand Values

Public/Target Audience

Yourself Your Family Your FriendsOthers Like

YouOthers who might use

*Source: Don W. Stacks, Ph.D. All rights reserved

8. Impact of new media8. Impact of new media Traditional reputational evaluation &

measurement was almost static – things didn’t change fast

New media has addressed four new elements in evaluation Speed Real time measurement The “rat’s tail” effect The “Power of 1”

New media New media

Speed: There is less reliance on editorial

review and more on being the first to report

Producing much more error in what is being communicated, often with disastrous effects

Real time measurement: Requires attention to social media on a

continuous basis Outcomes often result in “Dashboard

Analysis” and reliance on numbers without understanding what they mean

New media New media

“Rat’s tail”: There are thousands of people who

consider themselves “educated” with opinions & they use the social media & are often picked up via web-search engines

The good news is that only a few are really influential at any given point in time & they can be identified through systematic social media space & their influence evaluated through their followers

The bad news is that influentials often change over time, often due to what they say not the logic of their analyses

New media New media “Power of 1”:

It only takes one negative or bad report to shift attention from positive to negative

Influencers are now found in both internal & external publics

Monitoring takes on new meaning/legal consequences

Hypothetical social media evaluation example

Client or product

Blogger/Op. Ldr.

Audience

One-way

Two-way asymm.

Two-way symm.

0 +

-

0

- +.2

.1.6

+/0/1 = Msg. position. x= % msg. volume

+ .4

.1.05

.85A

B

C

Physicians

DrugCompany

Interpretation: A is neutral, no relationship, and little impact; B is negative, one-way relationship with minimal negative impact; C is positive , engaged in two-way symm relationship with major impact. On audience (physicians) producing mutual engagement with client. Recommendation: Focus on C while watching A and B

9. Conclusion(s)9. Conclusion(s)1. Without a benchmark, we have no way to

evaluate2. Evaluation is process-oriented: knowledge-

based and behaviorally-driven3. We can evaluate public relations, but what

we evaluate toward is not exactly what some call “ROI”; ROE must be constantly evaluated to better predict ROI

4. We can collect, analyze, evaluation and recommend courses of action

Measurement 101:Measurement 101:Measuring Public Relations Measuring Public Relations Impact on ROI by its ROEImpact on ROI by its ROE

Don W. Stacks, Ph.D.School of Communication

University of MiamiCoral Gables, FL 33124

The Institute for Public RelationsCommission on PR Measurement & Evaluation

[email protected]