Transcript
Outcome MappingMonitoring and Evaluating Policy Influencing8 December 2009Simon Hearn, s.hearn@odi.org.uk
Brief definition of OM
• A participatory method for planning, monitoring and evaluation
• Focused on changes in behaviour of those with whom the project or program works
• Oriented towards social & organizational learning
2
3
The Problem
Policy change can be:
• Complex (involve a confluence of actors and factors)
• Unstable (independent of project duration)
• Non-linear (unexpected, emergent, discontinuous)
• Two-way (intervention may change)
• Beyond control (but subject to influence)
• Incremental, cumulative (watersheds & tipping points)
5
1. Contribution, not Attribution
• Change processes involve interactions among multiple actors and factors
2. There is a limit to our influence
Sphere of Control
Sphere of Influence
Sphere of Interest
Inputs Activities Outputs Outcomes Impacts
3. Identify and focus on key agents of change
Develop enthusiasm to address topic
Learn in partnership
Develop awareness
and enthusiasm
Challenge existing beliefs
High
Gen
eral
leve
l of
alig
nmen
t
Low
Low HighInterest in specific topic
1. Map actors on the matrix
2. Identify which are the most influential
3. Who do you work with directly?
4. Outcomes as progressive behaviour changes
(Deep transformation)
(Active engagement)
(Early positive responses)
Love to see
Like to see
Expect to see
...policy change examples
• Discourse changes• Procedural changes• Content changes• Attitudinal changes• Behavioural changes
5. Use a broad range of strategies
causalpersuasiv
esupportiv
e
Aimed at the actor
Aimed at the actors
environment
6. M&E on three levels
12
Initiative Policy Actor
outcomes(behaviour changes in the actors)
implementation(interventions by the program)
relevance & viability(actions of the program)
Strategies
7. Can’t prove causation
• But can make a plausible estimate based on:– Timing: Did the change happen after the activity?– Logic: Is it reasonable to expect that these inputs
would contribute to the change?– Expert Judgment: Do knowledgeable people –
including those involved – agree with the contribution claim?
– Alternative Explanations: What other factors could explain the change?
8. Clarify intent from the outset
• Need a systematic approach to choosing areas for influencing, setting objectives and defining outcomes
• This will enable a systematic method for collecting monitoring information and evidence of change
Evaluating REGLAP
• RAPID Outcome Assessment– Outcome Mapping– Episode Studies– Most Significant Change
• Tracks key changes in target actors and surrounding environment
• Makes a judgement on the contribution of project activities on the changes
• Recognises external events
Evaluating REGLAP
1. Document review2. Interviews with project team3. Workshop with project team, key
stakeholders, external experts
Evaluating REGLAP
1. Clarify...– Policy objectives– Key policy actors targeted– Desired changes in and around these actors
2. Collect evidence of...– Activities undertaken and outputs of strategies– Internal changes within the programme– Changes in behaviour of key actors– Other changes in the policy environment
3. Map the evidence and discuss linkages
Evaluating REGLAP
• Strategy• Management • Outputs • Uptake• Outcomes and Impact
Somewhat
BP 0 1 2 3 4 5 6 7 8 9 10
Project
EE
Po
lic
y E
nvi
ron
me
nt
Po
licy C
han
ge
BP 0 1 2 3 4 5 6 7 8 9 10
Project
EE
Po
lic
y E
nvi
ron
me
nt
Po
licy C
han
ge
Step 1: Describe the policy environment
Before TodayBP 0 1 2 3 4 5 6 7 8 9 10
BP1
BP2
BP3
BP4
BP5
BP6
BP7
Project
EE
Po
lic
y E
nv
iro
nm
en
t
year/month
Po
licy
Ch
an
ge
Step 2: Identify key policy actors
Before TodayBP 0 1 2 3 4 5 6 7 8 9 10
BP1 0 5
BP2 0 4
BP3 0 8
BP4 0 7
BP5 0 4
BP6 0 9
BP7 0 3
Project
EE
Po
licy
En
viro
nm
ent
year/month
Po
licy C
han
ge
Before TodayBP 0 1 2 3 4 5 6 7 8 9 10
BP1 0 5
BP2 0 4
BP3 0 8
BP4 0 7
BP5 0 4
BP6 0 9
BP7 0 3
Project
EE
Po
licy
En
viro
nm
ent
year/month
Po
licy C
han
ge
Step 3: Describe the behaviour of the key actors
Before TodayBP 0 1 2 3 4 5 6 7 8 9 10
BP1 0 1 2 3,4 5
BP2 0 1 2,3 4
BP3 0 1 2 3 4,5,6 7 8
BP4 0 1 2 3 4 5 6 7
BP5 0 1 2 3 4
BP6 0 1,2 3,4,5 6,7,8 9
BP7 0 1, 2 3
Project
EE
Po
lic
y E
nv
iro
nm
en
t
year/month
Po
licy
Ch
an
ge
Step 4: Map the key changes in behaviour
Before TodayBP 0 1 2 3 4 5 6 7 8 9 10
BP1 0 1 2 3,4 5
BP2 0 1 2,3 4
BP3 0 1 2 3 4,5,6 7 8
BP4 0 1 2 3 4 5 6 7
BP5 0 1 2 3 4
BP6 0 1,2 3,4,5 6,7,8 9
BP7 0 1, 2 3
Project 0 1 2,3 4,5 6 7 8 9 10
EE 0 1 2 3,4 5 6 7 8 9 10 11
Po
lic
y E
nvi
ron
me
nt
year/month
Po
licy C
ha
ng
e
Before TodayBP 0 1 2 3 4 5 6 7 8 9 10
BP1 0 1 2 3,4 5
BP2 0 1 2,3 4
BP3 0 1 2 3 4,5,6 7 8
BP4 0 1 2 3 4 5 6 7
BP5 0 1 2 3 4
BP6 0 1,2 3,4,5 6,7,8 9
BP7 0 1, 2 3
Project 0 1 2,3 4,5 6 7 8 9 10
EE 0 1 2 3,4 5 6 7 8 9 10 11
Po
lic
y E
nvi
ron
me
nt
year/month
Po
licy C
ha
ng
eStep 5: Map project changes and external influences
Before TodayBP 0 1 2 3 4 5 6 7 8 9 10
BP1 0 1 2 3,4 5
BP2 0 1 2,3 4
BP3 0 1 2 3 4,5,6 7 8
BP4 0 1 2 3 4 5 6 7
BP5 0 1 2 3 4
BP6 0 1,2 3,4,5 6,7,8 9
BP7 0 1, 2 3
Project 0 1 2,3 4,5 6 7 8 9 10 11
EE 0 1 2 3,4 5 6 7 8 9 10 11
Po
licy
En
viro
nm
ent
year/month
Po
licy C
han
ge
Before TodayBP 0 1 2 3 4 5 6 7 8 9 10
BP1 0 1 2 3,4 5
BP2 0 1 2,3 4
BP3 0 1 2 3 4,5,6 7 8
BP4 0 1 2 3 4 5 6 7
BP5 0 1 2 3 4
BP6 0 1,2 3,4,5 6,7,8 9
BP7 0 1, 2 3
Project 0 1 2,3 4,5 6 7 8 9 10 11
EE 0 1 2 3,4 5 6 7 8 9 10 11
Po
licy
En
viro
nm
ent
year/month
Po
licy C
han
ge
Step 6: Determine level of influence
Influences:DirectIndirectExternal
top related