Arizona advocacy
Post on 30-Nov-2014
209 Views
Preview:
DESCRIPTION
Transcript
Evaluating Advocacy:Dilemmas, Tactics, and Methods
Julia CoffmanCenter for Evaluation Innovation
jcoffman@evaluationinnovation.org
October 7, 2011
Three Questions
How is evaluating advocacy different?
What can we measure about advocacy?
How can we measure it?
How is evaluating advocacy different?
Challenge: The policy environment is complex, and that makes attribution hard.
What is my uniquecontribution?
Solution: If accountability is the purpose, demonstration of contribution
is expected, not attribution.
Shifting Politics Shifting Economics New Partners
Challenge: Advocacy strategies shift in response to the environment.
EV
AL
UA
TIO
N
ADVOCACY
Solution: Advocacy is a good opportunity to integrate or embed evaluation for learning
(not just accountability).
Challenge: Timeframes can be unpredictable.
Advocacy Timeframe
1 Year
Goal
Reporting Timeframe
1 Year 1 Year
10 Years5 Years1 Year
Goal
…progress…
Solution: Assess progress, not just the end result.
What can we measure about influencing?
Measure meaningful things. Don’t just count what is easy to quantify.
Have realistic expectations.
What are other
influencers doing?
What’s the opposition
doing? What are you doing and
who are you targeting?
What’s the political context?
Where is your issue in the policy
process?
Policy Influence
…progress…
Measure the changes made along the way, not just the end result.
INTERIM OUTCOMES
Interim outcomes are the changes you expect as you work toward your goal.
Think about the changes you will see in your audiences.
AUDIENCES
DECISION MAKERS
OU
TCO
MES
ACTI
ON
WIL
LAW
AREN
ESS
PUBLIC INFLUENCERS
WHO will change as a result of your work?
HOW will they change as a result of your work?
Use the framework to think about interim outcomes.
Where are your audiences?How far do you need to move them?
AUDIENCES
DECISION MAKERS
ACTI
ON
WIL
LAW
AREN
ESS
PUBLIC INFLUENCERS
Increase quality of child care
Child care providers
Parents of young
children
INTE
RIM
OU
TCO
MES
Legislators
Increased media coverage
Increased collaboration
among advocates
Increased willingness to act
Increased knowledge
Changed attitudes or beliefs
New and active advocatesIncreased salience
Reframing of the issue
New donors
Increased or diversified funding
Increased issue visibility or recognition
New and active high-profile champions
Activities and
Outputs
Policy Goals
Interim OutcomesAwareness
Will
Action
Increased capacity to act
Increased personal or collective
efficacy
How can we measure it?
Interviews
Surveys
Focus Groups
Polling
Traditional Evaluation Methods
System Mapping
Network Mapping
Advocacy Capacity Assessment
Critical Incident Timelines
Intense Period Debriefs
Media Scorecards
Research Panels
Crowdsourcing
Snapshot Surveys
Intercept Interviews
360-Degree Critical Incident Debriefs
Policy Maker Ratings
Media Tracking
ECCO Analysis
Policy Tracking
Non-Traditional Methods
Charting and Mapping
Survey and Interview
Debriefing and Rating
Media/Messaging Tracking
Bellwether Methodology
Champion Tracking
Bellwether Methodology
► Policymakers
► Administrators
► Other Advocates
► Funders
► Business
► Media
Bellwethers are:
“Influentials” in the public and private sectors whose positions require that they are politically informed and that they track a broad range of policy issues
Developed by Harvard Family Research Project
Bellwether Methodology
What are the top 3 Issues on the children’s (ages 0-18)
policy agenda?2005
(n=40)2008
(n=44)
1. K-12 education2. Health care3. Preschool
1. K-12 education2. Health care3. Preschool
Yes 84%
No 14%
Don't Know 2%
2008(n=44)
Can preschool help address the achievement gap?
How would you address the achievement gap?
2008 FULL SAMPLE (n=72) %
1. K-12 reform 67%
2. Education funding/resources 47%
3. Nonschool supports (health care, family involvement,after school)
32%
4. Supplemental school programs (ELL, literacy programs)
29%
5. Preschool/early care and education 21%
6. Other 13%
Bellwether Methodology
Very likely 9%
Somewhat likely 20%
Not very likely 50%
Not at all likely 20%
2008(n=44)
What is the likelihood that California will
increase preschool investments in the
next 3 years?
Bellwether Methodology
Policymaker Ratings
Support Influence Confidence
Ratings are completed for a whole governing body or defined group of policymakers
Developed by Harvard Family Research Project
Policymaker Ratings
35
71
15
3
35
55
23
7
0
10
20
30
40
50
60
70
80
Not at all supportive
Somewhat supportive
Supportive Extremely supportive
Year 1
Year 2
Developed by Harvard Family Research Project
Policymaker Ratings
Alameda
AlpineAmador
Butte
Calaveras
Colusa
Costa
Del Norte
El Dorado
Fresno
Glenn
Humboldt
Imperial
Inyo
Kern
Kings
Lake
Lassen
Los Angeles
Madera
Marin
Mariposa
Mendocino
Merced
Modoc
Mono
Monterey
Napa
Nevada
Orange
Placer
Plumas
Riverside
Sacramento
Benito
San Bernardino
San Diego
San FranciscoJoaquin
Obispo
Mateo
Barbara
Santa ClaraSanta
Shasta
Sierra
Siskiyou
Solano
Sonoma
Sutter
Tehama
Trinity
Tulare
Tuolumne
Ventura
Yolo
Yuba
Contra
San
San
SanLuis
San
Santa
Cruz
Stanislaus
Low Support
Medium Support
High Support
= Increase in Support
* Hypothetical Data
Developed by Harvard Family Research Project
Policymaker Ratings
Low Influence High Influence
Low Support
High Support
68 legislators 57% of the legislature
17 legislators 14% of the legislature
13 legislators 11% of the legislature
80% Democrat20% Republican
100% Assembly
85% Democrats15% Republican
75% Assembly25% Senate
50% Assembly50% Senate
67% Assembly33% Senate
49% Democrat51% Republican
64% Democrat36% Republican
22 legislators 18% of the legislatureLo Med Hi
Confidence Rating
70%5% 25%
Lo Med Hi
Confidence Rating
60% 40%
Lo Med Hi
Confidence Rating
100%
Lo Med Hi
Confidence Rating
40% 60%
Developed by Harvard Family Research Project
Wrap Up
Stay anchored in what you can reasonably change.
It’s okay to prioritize and focus on what is most important to assess.
Be creative—advocacy evaluation is an emerging field.
www.evaluationinnovation.org
Center for Evaluation Innovation
top related