Systemic impact evaluation Evaluating impacts of agricultural projects The example of SOGUIPAH in Guinea AFD - J. Delarue
Jan 04, 2016
Systemic impact evaluationEvaluating impacts of agricultural projects
The example of SOGUIPAH in Guinea
AFD - J. Delarue
What do we want ?
• Rigorously estimating impact (counterfactual approach)
• Accounting for impact heterogeneity
• Evaluating impacts on the lifetime of the project’s outputs
What is SOGUIPAH?
• An agroindustrial company in Guinea: SOGUIPAH (1987), produces rubber (exports) and palm oil (national market)
• Land allocation : 23 000 ha Farmers deprived of land
• SOGUIPAH helps other farmers develop perennial cropping systems : A model farmer : 1 ha oil palm, 2 ha rubber and 0.5 irrigated rice
SOGUIPAH
Evaluative questions
• What are the impacts of SOGUIPAH on the farmers’ income on the lifetime of the plantations?
Impacts of land allocation Impacts of out-grower scheme
• Who is impacted by the project? directly Indirectly
• Why is there such an impact?
Evaluation context and methodological challenges
An evaluation 15 years after the project started Very little baseline and follow-up data
Estimating impacts : a counterfactual approach
Counterfactual at village level
Counterfactuals for the different types of farms
Matching on observables +
their combination in a system
Agrarian system analysis : matching villages
History
Region
Agricultural land use
EcosystemSocial
relationships
Village WITH SOGUIPAH
Village WITHOUT SOGUIPAH
Farming system analysis : matching farms
Tools Labour
Land
Family
Cropping systems and breeding systems
Labour calendar
Land allocation
Cash calendar…
Baseline situation
WITH SOGUIPAH
WITHOUT SOGUIPAH
RESULTS :
5 farming systems before SOGUIPAH
11 trajectories with SOGUIPAH
7 trajectories without SOGUIPAH
Agrarian system analysis : activities and results
ACTIVITIES :
1. In depth interviews (focus groups, farms)
2. Landscape observations
3. Secondary data
… and triangulation
RESULTS :
2 comparable villages
Understanding interactions of SOGUIPAH with the agrarian system (land market, labour market, prices…)
Indentification of farming systems
Farming system analysis : activities and results
ACTIVITIES :
1. In depth interviews (recall method at farm level)
2. Field observations
3. Productivity analysis nowadays
… and triangulation
RESULTS :
Comparable farms before SOGUIPAH
Trajectories of farming systems with and without SOGUIPAH
Economic results with and without SOGUIPAH for a variety of cropping systems
What is the impact we want to estimate ??
income
1987 2005
A cross-section-based measure of average impact on the population is not what we’re looking for !
Impact ?
0
10 000 000
20 000 000
30 000 000
40 000 000
1987 1997 2007 2017 2027 2037
GNF
con
stan
t
Kpoo 3
Galaye 5
Mesure d'impact ex post
Projectionex ante
Modeling economic results with and without SOGUIPAH for one farming system
2005
Translating trajectories into a simple economic model based on financial analysisAssessing future impact based on structural trends
(yields) and a few assumptions (prices) Impact on the lifetime of the plantations
Accounting for heterogeneity
0
4 000 000
8 000 000
12 000 000
16 000 000
1987
1990
1993
1996
1999
2002
2005
2008
2011
2014
2017
2020
2023
2026
2029
2032
2035
2038
2041
GN
F c
on
stan
t
Guilamou 1
Galaye 1
Mesure d'impact ex post
Projection ex ante
0
10 000 000
20 000 000
30 000 000
40 000 000
1987 1997 2007 2017 2027 2037
GN
F c
on
stan
t
Kpoo 3
Galaye 5
Mesure d'impact ex post
Projectionex ante
0
4 000 000
8 000 000
12 000 000
16 000 000
1987
1990
1993
1996
1999
2002
2005
2008
2011
2014
2017
2020
2023
2026
2029
2032
2035
2038
2041
GN
F c
on
stan
t
Kpoo 4
Galaye 1
Mesure d'impact ex post Projectionex ante
0
1 000 000
2 000 000
3 000 000
4 000 000
5 000 000
6 000 000
7 000 000
1987
1989
1991
1993
1995
1997
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
2019
2021
2023
2025
2027
2029
2031
2033
2035
2037
2039
2041
2043
GN
F co
nsta
nts
Kpoo 5
Galaye 4
Mesure d'impactex post
Projection ex ante
Impacts are highly differentiated ! Average : + 200 €/y ranging from – 200 €/y to + 1530 €/y
Conclusions about the systemic impact evaluation
• A qualitative approach which allows the quantification of impacts
• A very simple economic model based on high quality data
• It assesses the impacts along the life time of the project’s outputs
• It takes into account the complexity of real world
• It provides an in-depth understanding of the project effects and allows formulating recommandations