This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Presentat ion approved for Publ ic Release:12-4611. Dist r ibut ion Unl imi ted * A B C – A c c r e d i t e d i n B u s i n e s s C o m m u n i c a t i o n b y t h e I n t e r n a t i o n a l A s s o c i a t i o n o f B u s i n e s s C o m m u n i c a t o r s
Measuring Communication - Simple Approaches That Can Change Your Image
An organization’s brand/reputation, comes from the inside out.
Employees treat customers as management treats employees.
– Effective communication improves employee engagement
– Engaged employees deliver better for the customer
Here’s an example.
| 4 |
Scenario - Call center at “the Bureau”
| 5 |
Call center agent, Jill, answers question and
completes service applications for citizens
| 6 |
Goal: communication with Jill will align her
performance with the Bureau’s goals
Communication Desired
interaction with
citizens
| 7 |
Head of call centers needs quality improved (response accuracy)
| 8 |
His e-mails, speeches, newsletters have stressed
quality – but haven’t produced change
| 9 |
He wants your help getting his quality message through
“What other communication should I do? I thought maybe a special newsletter each month focused just on quality, like industry best practices. Then maybe a contest between units and then spotlight the quality superstars in the newsletter each month. What do you think?”
| 10 |
Stepping back for a moment… The communication flow – classic model
Communication Flow
Sender Message Channel Receiver Results
Noise
Feedback
| 11 |
How does the basic model work in our large and complex organizations?
| 12 |
Communicator Perspective
Stakeholder Perspective (e.g. employees)
Client Program Owner Perspective
Sender Message Channel Receiver Results
Organizational roles that tie to steps in the model
Communication Flow
Noise
Feedback
| 13 |
Planned communication in an organization Steps that operationalize the model
Communicator Perspective
Plan Deploy Communication
Activities
Stakeholder Perspective (e.g. employees)
Frame of Reference
Communication
Outcomes
Client Program Owner Perspective
Stakeholder Behaviors
Results
Communication Flow
| 14 |
Planning & measures are intertwined Planning starts at the end, works backward
Communicator Perspective
Plan Deploy Communication
Activities
Stakeholder Perspective (e.g. employees)
Stakeholder Assessment
(Frame of Reference)
Communication Outcomes
(Knowledge &
Attitude)
Client Program Owner Perspective
Stakeholder Behaviors
Results
Planning Flow
| 15 |
Communication planning Explore measures for all steps, not just the end
Communicator Perspective
Plan
Measures
Deploy
Measures
Stakeholder Perspective (e.g. employees)
Stakeholder Assessment
(Frame of Reference)
Measures
Communication Outcomes
(Knowledge &
Attitude)
Measures
Client Program Owner Perspective
Stakeholder Behaviors
Measures
Results
Measures
Planning Flow
| 16 |
How to Measure Communication
Overview of Key Tools
Qualitative
– Interviews
– Focus Groups (and Structured Group Interviews)
– Polls
Quantitative
– Surveys
– Counting
| 17 |
Qualitative Data Gathering
Top tools Best for
understanding
Examples
• Focus groups
• Interviews
• Polls
• Depth—by probing
• To understand the
range of issues on a
topic
• Story-telling nature of
dialogue can uncover
forces around
change—enablers
and barriers to
change
• What people know
• How people feel
about a topic
• Frame of reference
on a topic
| 18 |
Quantitative Data Gathering
Top tools Best for
Examples
Surveys
Prediction
• Breadth—how widely
shared is an opinion
• Precise, for tracking
changes over time.
• Comparison over
time, and between
organizations
• Correlation analysis
for prediction
Counting
Outputs
• Number of articles
• Event attendance
• Percentage of
articles dedicated to
each strategy
| 19 |
Communication planning 1. Results
Communicator Perspective Stakeholder Perspective
(e.g. employees) Client Program Owner
Perspective
Results
Plan What to measure How to measures
Questions to ask:
Outcomes:
• What needs to change and why?
• How will you know if you’ve
accomplished your goal?
Examples:
• Customer satisfaction
• Budget requirements
• Congressional
requirement
Examples:
• Survey
• Financial statements
• GAO/committee
feedback
Outputs:
• What changes in outputs are needed
to achieve the goals?
• How will you measure progress?
• Number
• Quality
• Cost
• Counting
• Sampling
• Cost accounting
| 20 |
Results
Outcomes and Outputs
Will be measured by the program owner
Outcome: Higher satisfaction from
– GAO/Congress
– Citizens
Output: Higher quality calls
| 21 |
Communication planning 2. Targeted stakeholder behavior
Communicator Perspective Stakeholder Perspective
(e.g. employees) Client Program Owner
Perspective
Stakeholder Behaviors
Plan What to measure How to measure
Questions to ask
• Who needs to do what differently to
produce the output changes?
• How will you measure the behaviors?
• What will change to enable the new
behaviors?
Examples
Actual behaviors
• Applying new training
• Using new procedures
Behavioral indicators
Examples
Business process
measures
Survey questions
| 22 |
Behaviors
Not clear
Ron assumes the agents have leeway.
Sees his communication as motivation.
| 23 |
What to tell Ron?
One approach
“Yes! I can help you with a quality-focused newsletter and other
ways to get out your message.”
“AND— – how do you feel about first letting me do some feedback sessions
to get a sense of what will and won’t work. I might find better
options for you to consider.”
| 24 |
Communication planning 3. Stakeholder assessment – change readiness
Communicator Perspective Stakeholder Perspective
(e.g. employees)
Stakeholder Assessment
1. Frame of Reference for the change
Client Program Owner Perspective
Plan What to measure How to measure
Questions to ask the program owner
Client hypothesis about reaction to the
change:
• What changes will enable the new
behaviors or remove barriers?
• What is the stakeholder’s frame of
reference on what you are asking
them to do differently?
• What might they see as enabling or
blocking them?
Examples
Validate with the
stakeholder:
• Will you be able to do
this, in light of the
planned support?
Examples
Focus group
Interview
| 25 |
Conduct stakeholder assessment Focus groups at call centers
| 26 |
Finding: Quality measured quarterly, at the office level
| 27 |
Quantity Metric–
Every 30 minutes, Group Level
Finding:
Quantity measured every 30 minutes, at the group level
| 28 |
Frame of reference determines
what Jill will notice, how she will interpret it
Communication
Frame of Reference
(Enablers & Barriers)
Resulting
interaction
with citizens
| 29 |
Stakeholder assessment:
Focus groups determine Jill’s frame of reference
Frame of Reference
(Enablers &
Barriers)
Elements of frame of
reference
• Leadership
• Mission & Vision
• Culture
• Mgt. practices
• Policies &
procedures
• Work unit climate
• Individual needs
• Task & individual
skills
(See Burke-Litwin model)
Resulting
interaction
with citizens
| 30 |
What would you tell Ron?
What would you recommend
to him as a next step?
Might this change how Ron
sees you?
What to tell Ron?
| 31 |
Communication planning
3. Stakeholder assessment – communication fitness
Communicator Perspective Stakeholder Perspective
(e.g. employees)
Stakeholder Assessment
2. Frame of Reference on
communication
Client Program Owner Perspective
Plan What to measure How to measure
During deployment:
How to assess the effectiveness of each
communication activity during
deployment.
Past experience:
Usefulness of communication channels
and approaches with this stakeholder.
“Fitness” of communication
Examples:
- Clarity
- Right channels
- Amount
- Timing
- Exposure
During deployment
• Survey
• Poll
• Focus group
Past experience
• Compilation of
previous data
• Communication audit
“Fitness” measures.
What do they think of
our stuff? Does it “fit”
their needs?”
| 32 |
Example:
Ongoing assessment of "fitness" of messages
Weekly survey of employees and managers
Hottest topic of the week
Statistically reliable sample
| 33 |
Example "Fitness" measures Message penetration and usefulness
| 34 |
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Q7. How would you rate the
amount of communication you've
received on this topic? (Running
average for "About right" is 68%)
Q6. How important is the topic of
"new system procedures" to you?
(Running average for "Important" is
65%)
Q5. Overall, how satisfied were
you with this communication?
(Running average for "Satisfied" is
86%)
Q4. To what extent do you feel
this communication was timely?
(Running average for "Timely" is
82%)
Timely
Satisfied
Important
About right Too much Too little
Unimportant
Dissatisfied
Not timely
Example "Fitness" measures Timeliness, satisfaction, importance, amount
| 35 |
Communication planning 4. Communication outcomes
Communicator Perspective Stakeholder Perspective
(e.g. employees)
Communication Outcomes
Knowledge &
Attitude
Client Program Owner Perspective
Plan What to measure How to measure
What knowledge and attitude will
produce the targeted behaviors, as seen
through the stakeholder’s frame of
reference?
Examples
• Knowledge and attitudes
(perceived)
• Actual knowledge
Examples
• Survey
• Poll
• Focus group
| 36 |
Example knowledge measure
| 37 |
Example measure of feelings
9%
23%
17%
20%
31%
13%
38%
25%
0%
25%
0%
5%
10%
15%
20%
25%
30%
35%
40%
A great deal Somewhat A little None I’m not sure yet.
To what extent do you feel the new Performance Plan system will help the agency reach its goals?
Employees
Sups
| 38 |
Example: Measuring knowledge and feelings
for course correction
Situation:
– Executive Committee interactive video town hall
Topic:
– Transformation update – two years into effort
Update on each of five change strategies
Scripts weighted heavily on “why this strategy is important”
Measurement Plan
– Pre-event random survey – two questions for each strategy
Know: “How important do you feel this is to the agency transformation?”
Feel: “How effectively do you think this strategy is being implemented?”
– Recommend possible script changes
– Post-event repeat survey
| 39 |
| 40 |
Measuring knowledge and feelings
Pre-event survey (two days prior)
Several strategies showed:
• good scores for importance
(feeling)
• lower scores for
implementation (knowledge)
Comments indicated lack of
knowledge about early
implementation efforts and
results.
Updated script for these
strategies to weight more heavily
on pilots and early
implementation efforts.
Measuring knowledge and feelings
Post-event survey (one hour after event)
Scripts adjustments were
successful.
| 41 |
Communication planning 5. Deploy communication activities
Communicator Perspective
Deploy communication
activities
Stakeholder Perspective (e.g. employees)
Client Program Owner Perspective
Plan What to measure How to measure
What types of communication will best
convey the targeted knowledge and
attitudes?
Communication Outputs
Examples:
• # of articles and events
• % of content supporting
each business strategy
• % of stakeholders who
provide input on a change
• # of awards for a desired
behavior
• # informal leader
conversations
Examples
• Counting
| 42 |
Targeting value change with
messages Primary communication activities
Content, such as intranet
pages, publications and formal
presentations.
Interaction, such as input to
decisions, dialogue with
managers, and collaboration
with colleagues.
Leadership example, such as
how leaders signal what they
value through promotions,
recognition, budget, their time
and informal comments.
| 43 |
Output measures Content analysis – weekly data collection
| 44 |
Themes Totals
% of total
editorial
space E-Bulletin Managers Bulletin Magazine Webbie
Example output measure: Percentage of total editorial space,
by category, for all six channels
1% 0%
6%
28%25%
27%
14%17%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Vision Mission Values Goals Strategic
Priorities
Administrative Recognition Quotes
Example output measure: Percentage of total editorial space,
by category, for E-Bulletin
1% 1%3%
25%
19%
23%
8%
13%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Vision Mission Values Goals Strategic
Priorities
Administrative Recognition Quotes
Example output measure: Percentage of total editorial
space, deep dive in “goals” coverage
28%
3%5%
2% 2%
25%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Goals Total Goal 1: Develop
a…
Goal 2: Prepare
and equip…
Goal 3: Continue
to transition the
…
Goal 4: To serve
the public's
needs by…
Goal 5: Develop
an enterprise
approach to…
Communication Measures Integrated with Communication Planning
Communicator Perspective
Communi-cation plan elements
Measures for each element
5.
Tactics to Deploy
Communication Outputs
Stakeholder Perspective
3.
Stakeholder Assessment
(Frame of Reference)
Change Readiness
Communication Fitness
4.
Communication Outcomes
Knowledge and
Feelings
Client Program Owner Perspective
2.
Stakeholder Behaviors
Specific behaviors needed
Changes that will enable the
behaviors
1.
Results
Outcomes and outputs
| 48 |
Does anyone really care?
Measures can move your “brand” from overhead, toward operations.
Communications Executive Council survey: 2012 vs. 2007 data
Responsibilities of communication function – #1 increase: Social Media 80% (from zero) – #2 increase: Measurement 82% (from 64%)
Top communication strategic priorities for 2013
– #1: Employee engagement – #2: Improving leadership communication – #3: Building company presence in social media
Strategies to improve communication function effectiveness – #1 Improved consultative partnership with business leaders – #2 Deeper business acumen on the team
CEC 2012 report: Taking Your Communications Budgeting and Planning to the Next Level
| 49 |
What next?
Deeper topics
– Design and use tools for valid and reliable data
– Communication audits
Broader topics
– Looking at measures across organizations and time