Demistyfing equity-focused evaluations [EFE] Marco Segone, UNICEF Evaluation Office Anne-Claire Luzot, M&E Regional Advisor, UNICEF CEECIS EO is developing a resource package on EFE. This presentation is therefore a work in progress
Dec 16, 2015
Demistyfing equity-focused evaluations
[EFE]
Marco Segone, UNICEF Evaluation Office
Anne-Claire Luzot, M&E Regional Advisor, UNICEF CEECIS
EO is developing a resource package on EFE. This presentation is therefore a work in progress
Outline
1. What does Equity mean?2. Why does Equity matter? 3. What is an equity-focused evaluation?4. How to manage an Equity-focused evaluation?
What does equity means?
Defining Equity
• Equity means that all children have an opportunity to survive, develop, and reach their full potential, without discrimination, bias or favoritism
• The disparities between population groups that are not driven by biology, are avoidable and unfair are termed inequities.
• Equity is therefore based on notions of fairness and social justice
All rights for all children everywhere, by prioritizing the most deprived
Graph 1: % of children (7-14 years old) out of school, by race/ethnicity
2000 Baseline and goals for national average, black and white children
2010 Goal: reduce by 50% the national average and the equity ratio between black and white children
5.5
2.8
3.8
2.2
6.9
3.1
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
Equ
ity R
atio
: 1
,8
Equ
ity r
atio
: 1
,4
Black children
White children
National average
Drivers: Income
Source: DHS data. From NAGA pg. 27
UNICEF
Under-five mortality rate in Central Asian countries,by wealth of households
0
55
110
Kazakhstan Kyrgyzstan Uzbekistan Tajikistan
per 1
000
Poorest 60% Richest 40%
Drivers: locations
District level Data - Nyanza ProvinceUNICEF UNICEF
Proportion of children aged 0-59 months living in households thatdo not provide adequate support for early learning and development, by
worst-off and best-off regions
0
20
40
60
Mo
nte
neg
ro
Mac
edo
nia
Ser
bia
Bel
aru
s
Geo
rgia
BiH
Alb
ania
Kaz
akh
stan
Uzb
ekis
tan
Kyr
gyz
stan
Taj
ikis
tan
Eastern Europe Central Asia
Best-off region Worst-off region
%
Drivers: education
UNICEF UNICEF
Proportion of children aged 0-59 months living in households thatdo not provide adequate support for early learning and development, by level of
education of mother/caretaker
0
25
50
Mo
nte
neg
ro
Mac
edo
nia
Ser
bia
Bel
aru
s
Geo
rgia
Bo
snia
an
dH
erze
go
vin
a
Alb
ania
Kaz
akh
stan
Uzb
ekis
tan
Kyr
gyz
stan
Taj
ikis
tan
Eastern Europe Central Asia
Basic or less Higher than basic
%
Drivers: Gender
The majority of out-of-school children continue to be girls (53%); achieving gender parity would mean 3.6 million more girls in primary school
Drivers: Disabilities
90% of children with disabilities in developing countries do not attend school
Multiple Dimensions: Who is not learning? Disparity in learning
How is equity different from or similar to the Human Rights Based Approach and CRC?
• UNICEF’s foundation strategy of a human rights based approach underpins the equity agenda.
• Equity cannot be effectively pursued outside of a human rights framework, while human rights cannot be realised while inequity persists.
• Reaching the marginalized and excluded has always been integral to UNICEF’s work (Progress for Children 2010). We are doing it. But are we doing it sufficiently and everywhere?
12
Why does equity matter?
Why equity
Equity has a significant positive impact in reducing povertyPoverty is very sensitive to income distribution changes, and small improvements in income distribution can have an important positive effect in reducing poverty. Equity has a positive impact on economic growth
Equity and economic growth rates are directly related, with higher equity having a positive impact not only on poverty reduction but also on economic growth itself. Higher equity can therefore benefit the poor in two ways: by increasing the overall growth rate and average income, and by letting them share in that growth, since growth with less inequity benefits the poor more.
Why equity
Equity has a positive impact in the construction of a socially fair and democratic societyUnequal opportunities of social groups in society is often a significant factor behind social unrest, which may lead to crime or even violent conflict, with negative effects for the social cohesion of a society. Prolonged inequity may lead to the “naturalization” of inequityThe long “stability” of inequity in several countries makes it something “natural”. When inequity is perceived as a natural phenomenon, society develops theoretical, political and ideological resistances to identify and fight inequity as a priority in public policies.
Inequity constitutes a violation of human rights and hampers the equitable achievements on Human Development and MDGs
What is an equity-focused
evaluation?
What is an equity-focused evaluation?
A judgment made of the relevance, effectiveness, efficiency, impact and sustainability of interventions on equitable development results.
Purposes of equity-focused evaluation
Accountability Organizational learning and improvement Evidence-based policy advocacy Contribute to Knowledge Management National Capacity development for equity-
focused M&E systems Empowerment of worst-off groups
Empowering worst-off groups, including children, through Equity-focused evaluation processes
• Involve worst-off groups:– To ensure relevance, effectiveness, ownership of
evaluation– Focus on Evaluation process use to empower
worst-off groups through the evaluation process– Including children
• Understanding the cultural, political economic and social context that influence inequity, including perception of, and attitudes to, equity
Why are equity-focused evaluation required?
• Because “conventional” evaluations have some potential limitations in evaluating equity-focused interventions:
– going beyond conventional quantitative data to analyze behavioral change, complex social processes and attitudes, and collecting information on difficult-to-reach socially marginalized groups.
– In addition, conventional evaluation are often weak on process and contextual analysis.
Is equity-focused evaluation a completely new approach?
No. While some new analytical tools are introduced, most of the EFE data collection and analysis techniques are built on existing approaches
The emphasis is on refining and refocusing existing technics – and enhancing national capacities to use those technics
HOW to manageEquity-focused
evaluations?
Key Steps
1. Preparing for the EFE
1
2. Preparing the ToR of
an EFE
3. Designing the
evaluation
4. Collecting/analysing evidence
432
5. Findings, conclusion and
reccs
65
6. Utilizing the evaluation
MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS
Key Steps
1 432 65
MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS
1. Preparing for the EFE
. Preparing for the Equity-focused evaluation
• Determining the evaluability of the intervention’s equity dimensions
• Identifying evaluation stakeholders, including worst-off
• Identifying intended use by intended users• Identifying potential challenges in promoting and
implementing Equity-focused evaluations
Key Steps
1 432 65
1. Preparing for the EFE
2. Preparing the ToR of
an EFE
Preparing the ToR of an EFE
• Defining the scope and purpose of the evaluation• Framing evaluation questions focusing on equity• Selecting technically-strong and culturally-sensitive
evaluation team• Ensuring appropriate budget
Framing evaluation questions focusing on equity
DAC criteria adapted for EFE
•Relevance•Impact•Effectiveness•Efficiency•Sustainability
Key Steps
1 432 65
1. Preparing for the EFE
2. Preparing the ToR of
an EFE
3. Designing the
evaluation
Designing the evaluation
• Selecting the appropriate evaluation framework– Theory of Change– Bottleneck analysis
Bottleneck supply and demand framework: factors affecting use of services by vulnerable populations
31
Use of services by vulnerable population
Supply side factors
• Coverage •Budgets and available resources•Culturally acceptable services• Culturally sensitive staff
Demand side factors
• Knowledge,
Attitudes and Practice of vulnerable groups• Community ownership factors affecting access• distance• cost of travel and fees• available transport• cultural constraints• time poverty
Contextual factors• Economic• Political• Institutional• Legal and administrative• Environmental
Designing the evaluation
• Selecting the appropriate evaluation framework• Selecting the appropriate evaluation design
Selecting the appropriate evaluation design and tools
Use of services by vulnerable population
Supply side factors
• Coverage •Budgets and available resources•Culturally acceptable services• Culturally sensitive staff
Demand side factors
• Knowledge,
Attitudes and Practice of vulnerable groups• Community ownership factors affecting access• distance• cost of travel and fees• available transport• cultural constraints• time poverty
Contextual factors• Economic• Political• Institutional• Legal and administrative• Environmental
Mix-Methods
Designing the evaluation
• Selecting the appropriate evaluation framework• Selecting the appropriate evaluation design• Selecting the appropriate evaluation methods
Key Steps
1. Preparing for the EFE
1
2. Preparing the ToR of
an EFE
3. Designing the
evaluation
4. Collecting/analysing evidence
432 65
MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS
Selecting the appropriate evaluation methods
Approaches to be used to evaluate complex equity-focused policies and
programmes
• Systems approach to evaluation• Unpacking complex policies into components
that can more easily be evaluated• Pipeline designs• Policy gap analysis• Using other countries or sectors as the
comparison group• Concept mapping• Portfolio analysis
Key Steps
1. Preparing for the EFE
1
2. Preparing the ToR of
an EFE
3. Designing the
evaluation
4. Collecting/analysing evidence
432
5. Findings, conclusion and
reccs
65
MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS
Key Steps
1. Preparing for the EFE
1
2. Preparing the ToR of
an EFE
3. Designing the
evaluation
4. Collecting/analysing evidence
432
5. Findings, conclusion and
reccs
65
6. Utilizing the evaluation
MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS
Utilizing the evaluation
• Preparing the evaluation report and alternative forms of reporting
• Disseminating the evaluation and preparing a Management Response
Do you want more?
Visit MyM&E
Remember
This is NOT what I was expecting!!!!!
Invest time/energy in designing a good quality ToR with all main intended users, and make sure consultants understand it
and are comfortable with it
Whose fault is it?
Thank you
. Group work
• Identifying potential challenges in promoting and implementing Equity-focused evaluations
• Identify possible way to address the challenges• What support would you need?