Extract of configura.onal analysis table: Thinking and evaluating systemically The use of PIALA (Participatory Impact Assessment & Learning Approach) for evaluating commodity chain development in Ghana Adinda Van Hemelrijck ([email protected] ) Glowen Kyei-Mensah ([email protected] ) ASSESS to what extent impacts occurred (or not) DEBATE where/how impacts can be enhanced EXPLAIN why impact occurred (or not) PIALA was developed with support from IFAD (Interna.onal Fund for Agricultural Develop- ment) and BMGF (Gates Founda.on) to address challenges of complexity in evalua.ng agricul- tural market system development. Challenges: • how to evaluate ‘open systems’ (medium-N) • rigorous causal inference in the absence of credible control groups • how to trigger learning about contribuFons to sustainable impact PIALA pilots: • Ghana 2015, Roots & Tubers Improvement and Marke5ng Programme (RTIMP) • Vietnam 2013, Doing Business with the Rural Poor Programme (DBRP) IntroducFon PROCESS: • Reconstruct and visualise the programme’s Theory of Change (ToC) • Iden.fy causal claims and mechanisms on which to focus • Ar.culate assump- Fons and formulate quesFons • Create shared under- standing and owner- ship of the ToC and focus of evalua.on DESIGN DECISIONS: • Scope & scale Important for drawing conclusions about contribu.ons to and sustainability of impact • Level engagement Important for ownership and systemic learning PHASE 1: Focusing & framing PROCESS: • Sample ‘open systems’ and popula.ons within these systems • Design parFcipatory methods and survey to inquire the causal links in the ToC • Collect data using the same set of methods across the sample • Inquire causes and explana.ons by using ‘sensing’ methods (e.g. causal flow mapping) and by cross-checking and collaFng data along the ToC DESIGN DECISIONS: • Counterfactual approach Important for sampling of ‘systems’ and popula.ons PHASE 2: CollecFng & linking PROCESS: • Organise sense- making workshops to engage stake- holders in valuing contribu.ons to impact • Analyse and compare different systemic configuraFons across the sample of ‘systems’ DESIGN DECISIONS: • Scale and level of engagement Important for solid debate enabling ‘voice’ (e.g. nr of workshops and % beneficiaries at local and aggregated levels) PHASE 3: Analysing PIALA Purposes Causal claims & links in the ToC of the RTIMP: M2b: Training & starter pack for commercial seed growers to multiply certified R&T seeds C3a: R&T processors grow and develop into GPCs that are profitable enterprises O3: Enhanced R&T processed volumes of high quality at scale O2: Enhanced R&T productivity and production at scale M2c: Farmer Field Forums (FFF) engage farmers, extension agents and researchers in developing, demonstrating and promoting appropriate R&T production technologies C2a: Resource-poor R&T farmers & seed producers gain access to and adopt improved R&T seed varieties, technologies & inputs to improve crop husbandry, soil fertility and pest management practices C2b: Resource-poor R&T farmers organise and register as FBOs that can access credit and bargain better market prices C1b: Resource-poor R&T processors, farmers & seed producers commercialize and establish effective supply chain linkages C1a: R&T supply chain farmers & processors are capable of developing and implementing viable business and marketing plans C3c: R&T supply chain farmers and processors gain access to business financing and market-linking services M3b: Subsidized upgrading of advanced R&T processors into Good Practice Centres (GPCs) that demonstrate and promote good quality processing & management practices C3b: R&T supply chain processors gain access to and adopt standardized processing technology and good quality management practices O1: R&T supply chain actors effectively solve their supply & demand issues and timely obtain technical support, resulting in sustainable and inclusive CCs linked to old and new markets I2: Improved R&T- based livelihoods for the rural poor in CC catchment areas M2a: R&D for developing bio-agents M1c: Information, Education & Communication (IEC) about CC support services, inputs and technologies M1a: Training of resource-poor farmers and processors involved in the R&T supply chains in business development and marketing M3c: Co-financing of R&T supply chain farmers and processors by matching 40% RTIMP funds with 50% loans from PFIs and 10% self-financing through the Micro-Enterprise Fund (MEF) M3a: Training of artisans to produce and maintain standardized processing equipment for R&T supply chain processors and GPCs I1: Rural poor people in CC catchment areas have increased access to food & income to sustain an active and healthy life M1: District Stakeholder Forums (DSFs) for addressing supply & demand issues and technical support needs of R&T supply chain actors members M1b: Supply Chain Facilitation (SCF) and market linking through the Initiative Fund (IF) E C1 E C3c E C1 E O2 E O1 E C3b Contribution Claim of RTIMP Component 3 ê Enhanced Processing (O3) Contribution Claim of RTIMP Component 2 ê Enhanced Production (O2) Contribution Claim of RTIMP Component 1 ê Enhanced Market-Linking (O3) Contributions of RTIMP Components 1, 2 & 3 ê Improved Livelihoods (I2) DSF FFF GPC MEF MEF GPC Evidence Strength FFF C2a+C2b àO2 Evidence Strength M1c+M1b+ O3+O2 +O1àC1b DSF Evidence Strength O1+O2+O3 àI2 Evidence Strength (M3c)+C1a+M3b àC3c (M3b)+C3c àC3b à O3 M2a+M2b+ (M2c) àC2a C1a+(M1) àO1 Tano North (Apesika) (CZ) 1 1 1 1 3 6 5 5 5 5 4 4 5 5 5 Techiman (CZ) 1 1 1 1 4 5 5 5 5 5 4 4 5 5 5 Gomoa East (SZ) 1 1 1 0 2 5 3 5 5 5 4 4 5 5 6 Assin South (SZ) 1 1 1 1 3 4 4 6 5 4 3 3 4 4 4 Birim Central (CZ) 1 1 1 1 3 3 4 5 5 4 3 4 4 4 5 Nkwanta South (NZ) 1 1 1 0 3 4 5 5 4 5 3 3 5 4 5 Upper West Akim (CZ) 1 1 1 1 2 4 4 5 5 4 3 3 5 4 5 Ashanti Mampong (CZ) 1 1 1 1 3 4 5 5 5 5 3 3 5 4 5 West Gonja (Damongo) (NZ) 1 1 1 0 3 4 5 5 4 5 3 3 5 4 5 Abura Asebu Kwamankese (SZ) 1 1 1 1 3 3 5 5 5 6 3 3 5 4 4 Nanumba North (NZ) 1 1 N/A N/A 5 5 5 3 3 5 4 5 East Gonja (NZ) 1 1 N/A N/A 4 3 5 3 3 5 4 5 Central Gonja (NZ) 1 1 N/A 2 3 5 5 4 5 2 2 5 4 5 Suhum (CZ) 1 1 1 0 3 4 5 4 4 5 2 3 5 3 5 Adansi South (CZ) 1 1 1 1 2 4 5 4 3 4 2 2 3 3 5 Ahafo Ano South (CZ) 1 1 1 0 2 2 4 5 4 5 2 2 5 3 5 Kintampo South (CZ) 1 1 N/A N/A 4 4 3 2 3 5 3 5 Wa East (NZ) 1 1 0 0 2 2 5 4 5 5 2 3 5 3 5 North Dayi/ Kpando (SZ) 1 1 1 0 2 2 6 2 3 5 2 2 5 2 5 Agona East (SZ) 0 0 1 0 2 3 6 4 4 4 2 2 6 2 6 Pru (CZ) 0 0 N/A N/A 2 2 4 2 2 5 2 5 Ho Municipal (SZ) 1 1 0 0 3 2 5 5 5 5 3 3 5 2 5 Tano North (Dua Yaw Nkwanta) (CZ) 1 1 1 1 2 2 4 3 3 4 2 2 3 2 3 Wassa Amenfi West (SZ) 0 0 N/A 1 2 5 4 4 5 1 1 6 2 5 Kumasi Metropolitan (CZ) 0 0 N/A 1 1 4 1 1 4 1 1 4 1 4 Sampled districts ê Scoring of consistency & strength of each causal link and evidence (0-6) Presence of mechanisms (1=yes ; 0=no) PIALA pilot in Ghana 30 random supply chain systems of 4 commodi.es in 25 districts in 8 regions 836 random households 1180 quasi-random beneficiaries (45% women) 750 parFcipants (> 30% beneficiaries) in 23 local and 1 na.onal sensemaking workshops PIALA is an approach, not a single methodology. It can embed any method, so long it is consistent with principles of systemic thinking and enabling voice. PIALA phases & elements PHASE 1: Focusing & framing the evaluation PHASE 2: Collecting & linking the data PHASE 3: Analysing contributions RIGOUR INCLUSIVENESS FEASIBILITY Systemic ToC approach Participatory mixed-methods Multi-stage sampling of/in ‘open systems’ Participatory sensemaking Configurational analysis 6 hbps://www.ifad.org/topic/ overview/tags/piala