IEE internal - e

An Independent External Evaluationof the International Fund

for Agricultural DevelopmentOffice of Evaluation

An Independent External Evaluationof the International Fund

for Agricultural Development

Office of Evaluation

September 2005

© International Fund for Agricultural Development, 2005

i

Background1. In 2002, the Consultation on the Sixth

Replenishment of IFAD’s Resources (IFAD VI) endorsed a proposal to undertakean Independent External Evaluation of IFAD.In February 2003 the Governing Councilapproved the Report on IFAD VI and decided to launch the IEE. In April 2003, theExecutive Board reviewed a report on the IEE by the Chairperson of the EvaluationCommittee, endorsed its recommendations ongovernance and organizational arrangements,and entrusted the Director of the Office ofEvaluation (OE) with the overall supervision of the IEE. Among other things, the ExecutiveBoard decided that the Board would receive,review and comment on the draft final report of the IEE.

2. The IEE reviewed IFAD’s performance forthe years 1994 to 2003. This was a period ofextraordinary challenges for the develop-ment community in general and for develop-ment agencies focusing on the rural sector inparticular, and one which witnesseddecreasing official development assistance tothe agricultural sector. At the same time,IFAD’s portfolio evolved from a focus onagriculture to encompass a broader agendaof rural development. At a global level, theinternational community adopted theMillennium Development Goals (MDGs),signalling a renewed commitment to tacklingpoverty, including, in particular, ruralpoverty, and a number of other challengesalready familiar to IFAD.

Objectives and output3. Against this backdrop, the IEE set out to

determine the relevance of IFAD’s missionand results, and its impact in reducing ruralpoverty. It aimed to be both a summativeand formative evaluation. Not only did theIEE concentrate on the results and impact ofIFAD’s activities but it also sought to assessthe effectiveness of key corporate andmanagement processes, through whichIFAD’s policies, programmes and projectshave been developed and implemented. As such, the IEE has a scope that is muchbroader than any comparable evaluation of a multilateral development organizationundertaken so far.

4. The output of the IEE consists of five deliverables. The first four — the inceptionreport, the desk review report, the synthesisreport on country visits (including the tencountry working papers and the two reportson the review of human resources manage-ment and governance/institutional issues), andthe draft final report — were distributed indraft form for comments to IFAD’s manage-ment, the steering committee, the two seniorindependent advisors and OE as well as to thethree List Convenors, who ensured circulationwithin their Lists. The fifth deliverable is thepresent IEE final report.

Governance structure5. The Executive Board clarified that the IEE is a

multi-stakeholder evaluation that is owned bythe Board.

Foreword

6. The Director of OE served as the focal pointfor the IEE consultants, ensuring that theirwork complied with the agreed terms of reference, methodology and processes as wellas good evaluation practice. For this purposethe Director of OE provided the consultantswith inter alia written comments on all deliver-ables and draft reports. In keeping with theindependent status of the IEE, the Director ofOE neither contested nor supported the IEEfindings, for which he was not responsible.The Director of OE also played a role as interlocutor between IFAD management andthe Board on the one hand and the consult-ants on the other, and submitted five statusreports on the progress of the IEE to theExecutive Board.

7. A steering committee, composed of represen-tatives of nine IFAD Member States, wasestablished on 12 June 2003 to serve in anadvisory capacity to the Director of OE. TheIEE benefited from the advice, guidance andcomments provided by the steeringcommittee, which endorsed the terms of refer-ence for the IEE that were prepared by theDirector of OE, selected the team of consult-ants and provided comments on all draft IEEreports. The steering committee consisted ofrepresentatives of the following MemberStates: Algeria, Brazil, Canada (elected to theChair), Denmark, India, Mali, Nigeria,Sweden and the United States. In addition,representatives of the following MemberStates also attended meetings of the steeringcommittee as observers: Belgium, Cameroon,Egypt, Finland, Germany, Mozambique,Norway, Spain, Switzerland, the UnitedKingdom, and the Bolivarian Republic of Venezuela.

8. Two senior independent advisors – ProfessorM.S. Swaminathan and Dr Robert Picciotto –supported the Director of OE in his task. Bothadvisers are internationally renowned for theirindependence of mind, commitment to inde-pendence in evaluation, and expertise indevelopment evaluation as well as in the fieldsof rural and agricultural development. Theirrole was to assess the organization, method-ology and findings of the IEE in order tofurther reassure the Executive Board that the

IEE was being carried out in an independentand rigorous manner, in line with interna-tional best practice in evaluation and state-of-the-art knowledge in rural development. Thetwo senior independent advisors have now alsoprovided their final comments on the method-ology, process and outcome of the IEE. Theyare reproduced in annex 6 of this document.

9. The management of IFAD facilitated the IEEby allowing unrestricted access to informationand staff. Management provided the consult-ants with detailed comments on all deliverablesand proactively came forward with relevantinformation to enhance the quality of the IEE.In addition, management established the IEEcoordination group within IFAD, with theAssistant President, Programme ManagementDepartment (PMD), as its Chair. This groupensured coordination between the consultantsand various IFAD units, thus greatly facili-tating the work of the consultants.

10. The IEE governance was undoubtedlycomplex, yet overall, it worked well, largelydue to careful management and the commit-ment of all stakeholders to finding ways tomaximize the opportunity offered by the IEEto improve IFAD’s performance in the future.The IEE process was aided by the inde-pendent status of OE, which reports directly tothe Executive Board, the owner of the IEE.

The consultants’ team11. As required by the Executive Board, the IEE

consultants were recruited as a unit throughan open and competitive international biddingprocess that was completed under the supervi-sion of the Director of OE with the support ofspecialized IFAD services. The selectionprocess took approximately seven months andresulted in the selection of ITAD Ltd from theUnited Kingdom. The company assembled ateam of independent consultants under theleadership of Mr Derek Poate, Director ofITAD. The full list of the IEE consultants’team members, their roles and affiliations isincluded on page xi of this document.

12. Within the limitations imposed inter alia bythe lack of data on self-evaluation, the teamperformed the daunting task of evaluating

ii

F O R E W O R DA

nIn

depe

nden

tEx

tern

alEv

alua

tion

ofth

eIn

tern

atio

nal

Fun

dfo

rA

gri

cult

ura

lD

evel

op

men

t

over ten years in the life of IFAD, undertakingits enquiries in IFAD and in the field with ahigh degree of professionalism. In complexand comprehensive evaluations such as theIEE, it is always possible to detect areas thatwould have benefited from additional analysisas well as further synthesis, e.g. on IFAD’sbudget and budget process. However, overallthe consultants have produced a report thatvery much meets the requirements of theterms of reference and is in line with themethodology and processes that were agreedat the outset. The report is based on a solidanalysis and has produced findings andrecommendations that have important implications for the future of IFAD.

Distinctive features of the IEE13. A comprehensive independent evaluation

such as the IEE is relatively new to develop-ment organizations. Few multilateral develop-ment organizations have been evaluated inthe past. And those that were evaluated weremostly led by a small number of donors. Thestage is set to change, however. Recently,other United Nations organizations andinternational financial institutions haveundergone multi-stakeholder independentevaluations that have been spearheaded bytheir governing bodies – or are about to doso. It is fair to say that in recent years thistype of independent evaluation has becomeincreasingly important – as also attested by a recent initiative undertaken by theDevelopment Assistance Committee in thisfield called “A New Approach to EvaluatingMultilateral Organizations’ Performance”.

14. The IEE is the first truly independent andcomprehensive evaluation in the Fund’shistory. IFAD’s experience of the IEE, and thelessons gained in this process, may serve as auseful reference and provide guidance forfuture independent external evaluations ofmultilateral development organizations. TheIEE has a number of distinctive featuresworthy of mention:n The IEE is an independent and external

process, recognized as such by its owner, theExecutive Board, and by the internationaldevelopment community. That principle is reflected in the governance structure,

selection criteria for consultants and in therandom sampling of countries and projectsreviewed by the IEE.

n The emphasis on impact assessment issomething to which most other inde-pendent institutional evaluations have notdevoted sufficent attention in the past. TheIEE has demonstrated that impact assess-ment is possible, albeit with some method-ological limitations.

n The IEE is also a corporate evaluation thataddresses aspects of IFAD’s governance,management processes and functions, suchas human resource management and pastand current change initiatives. This provedto be a particularly challenging task callingfor specialized expertise over and abovethat usually required for evaluating devel-opment effectiveness. Yet, this aspect of theIEE has made an indispensable contribu-tion to assessing IFAD’s policies andperformance in terms of delivering itsprogrammes and projects.

n Random sampling of countries and projectsensured that the sample was free of biasthat could undermine the independence,impartiality and credibility of the IEE.

n The use of benchmarking allowed IFAD’sperformance to be compared with relevantaspects of similar development organiza-tions. Benchmarking is a difficult taskowing to the dearth of truly comparabledata and documented benchmarks.However, future benchmarking will befacilitated by current efforts aimed atharmonizing processes, criteria and datawithin multilateral organizations.

n An unprecedented level of transparencyand interaction between stakeholdersduring the IEE process: all deliverables andinterim reports submitted by the consult-ants were widely disseminated and writtencomments thereon were provided by IFADmanagement, the steering committee andOE. They were also made available to thethree List Convenors, who in turn dissemi-nated them to representatives of otherMember States within their Lists. At times,this process was cumbersome and time-consuming, but it contributed to mini-

iii

mizing surprises at the end of the IEEprocess and to fostering IFAD ownership of the IEE findings. Furthermore, the manyinteractions and comments have enhancedthe quality of the IEE and contributed to its transparency and credibility, inter aliathrough a process that included publicdebate on the issues raised by the IEE,including the most critical and sensitive ones.

Key findings and futureimplications

15. There is a clear evidence trail linking thefindings of the IEE to the analytical section ofthe report. The IEE’s main conclusion is thatIFAD has a relevant, clear and distinctive roleto play in reducing rural poverty. To achievethis, however, the Fund will need to improveits performance. Three main findingsemerging from the IEE have implications for IFAD’s future performance.

16. Firstly, the IEE concludes that IFAD’s overallportfolio performance is similar to that ofcomparable multilateral development organi-zations, and that only half the projects evalu-ated had made more than a modest impact.Furthermore, the sample scored poorly oninnovation. There is clearly no scope forcomplacency, given the continuing widespreadand pervasive rural poverty and the effortsstill required to achieve the MDGs. Thisimplies, in particular, that IFAD shouldaddress the causes of low impact, increase itsefficiency and become a more systematicpromoter of innovations that could be scaledup and replicated by others. The Fund mustalso improve its internal policy formulation aswell as its external policy dialogue and part-nerships with other development actors.

17. Secondly, many of the past change initiativesundertaken by IFAD management were notexplicitly aimed at improving effectiveness anddid not yield the expected results. Recently,however, a number of promising change initia-tives have been launched. The IEE concludesthat if it is to meet its many challenges, IFAD will need to implement deeper, more far-reaching changes and a new operatingmodel. In particular, IFAD needs to overhaulits management processes and actively pursuethe new human resources policy.

18. Thirdly, on the issue of governance, the IEEadvocates that the Executive Board shouldplay a greater role in overseeing the develop-ment effectiveness of the Fund’s actions.

19. Finally, the IEE offers a number of recommen-dations that emerge clearly from the areasidentified by the evaluation as in need ofimprovement. Those recommendations meet the requirements of the IEE terms ofreference inasmuch as they indicate clearpolicy and strategic directions that IFADshould pursue as well as action needed toenable the fund to improve its performanceand meet the expectations of the internationaldevelopment community.

Luciano Lavizzari, Director, Office of EvaluationRome, September 2005

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

iv

F O R E W O R D

The report on the Independent ExternalEvaluation (IEE) of IFAD was prepared by ITAD Ltd, a consulting firm based in the United Kingdom. The firm assembled a team of 26 consultants from 14 countries, plus 10 national evaluation teams led by Derek Poate,Director of ITAD. The IEE was conducted underthe overall supervision of the Director of theOffice of Evaluation, on behalf of the ExecutiveBoard of IFAD.

The team would like to thank the many representatives of governments, civil society anddonors, and above all, people living in ruralareas, whose collaboration and support providedan invaluable contribution to the fieldworkconducted by the IEE in ten countries.

The IEE was provided with advice and guidanceby a Steering Committee that met three times inRome under the chairmanship of FrançoiseMailhot of Canada. The committee consisted ofthe following members: Djerir Habiba, Algeria;Baastian Philip Reydon, Brazil; Esther Lonstrup,Denmark; Govindan Nair, India; ModiboMahamane Touré, Mali; Gabriel Lombin,Nigeria; Stefan Molund, Sweden; and MaureenGrewe, United States of America. Throughoutthe process, Steering Committee membersprovided valuable advice and comments toenhance the quality of the IEE and ultimately itsfinal report. The team would like to thank themall for their noteworthy contributions.

The IEE also benefited from the contributionsof the following Member State representativeswho attended Steering Committee meetings asobservers: Bernard De Schrevel and PhilipHeuts, Belgium; Médi Moungui, Cameroon;Maryam Mousa, Egypt; Heidi Pihlatie, Finland;Bernd Dunnzlaff, Germany; Carla Elisa Mucavi,Mozambique; Margaret Slettevold, Norway;Luis Cuesta Civís, Spain; Lothar Caviezel,Switzerland; and Peter Reid, United Kingdom of Great Britain and Northern Ireland. Theteam would also like to thank the governmentsand agencies involved for generously allowingtheir staff to participate actively in the Steering Committee.

Throughout the IEE process, the flow ofcommunication between the Office ofEvaluation and IFAD’s Governing Bodies wasgreatly facilitated by the Convenors of the threeLists (A,B & C) of Member States. In particular,the team notes the contribution of List A:Charles Parker, Canada followed by MargaretSlettevold, Norway; List B: Nasr-EddineRimouche, Algeria, followed by Carlos PozzoBracho, Bolivarian Republic of Venezuela; andList C: Médi Moungui, Cameroon.

The IEE was partly funded by the generouscontributions of the governments of Belgium,Canada, Denmark, Norway, Sweden,Switzerland, and the United Kingdom.

v

Acknowledgements

The team gratefully acknowledges the invaluableguidance and insights it received from the twoeminent senior independent advisers, ProfessorM.S. Swaminathan and Dr Robert Picciotto, whoassisted the Director of the Office of Evaluation inthe supervision of the IEE.

The IEE also benefitted from the advice andassistance of Mona Bishay, Tariq Husain, HadiShams, and Roger Slade, who were also part ofthe Technical Evaluation Panel for the selection of the team of consultants.

The international competitive bidding processwould not have been as smooth and efficient, had it not been for the support of Ana Knopf,Assistant President, Finance and AdministrationDepartment of IFAD, Theresa Panuccio, Charalambos Constantinides and, in particular,Tiziana Carpenelli.

IFAD’s General Counsel, Christian Codrai,provided constant and impeccable legal guidancethroughout the IEE process, together with Cynthia Licul and Nancy Kaplan.

Within the Office of Evaluation, a substantivecontribution to the IEE process was provided by Ashwani Muthoo who, together with

Mark Keating, Louise Daniel, and Sile O’Broinalso ensured that communication aspects of theIEE were handled flawlessly. Invaluable adminis-trative and logistic support was provided by ateam composed of Lucy Ariano, Anna Benassi,Laura Morgia and Kendra White.

The team would like to express its appreciationfor the exceptional degree of cooperation andsteadfast support from many colleagues in IFADat every level of the organization, who are far toonumerous to mention here. The team is espe-cially grateful to James Carruthers, AssistantPresident, Programme Management Department,who, as chair of the IEE Coordination Groupwithin IFAD, was instrumental in ensuring thatcoordination between the team and IFAD staffwas handled smoothly and sensitively, whichgreatly enhanced the team’s work.

Finally, the team thanks IFAD’s senior manage-ment for its strong and unwavering support ofthe IEE process. In particular, the team is deeplyindebted to Lennart Båge, President of IFADwho, from the outset, openly embraced the IEEas an opportunity to review and strengthen theorganization, and whose support was pivotal togalvanizing interest in and contributions to theIEE at all levels in IFAD.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

vi

A C K N O W L E D G E M E N T S

Table of Contents

Abbreviations and Acronyms viii

IEE International and National Team xi

Executive Summary 1

Main Report:Chapter 1. The evaluation and its context 15

Purpose of the independent external evaluation 15Evaluation framework and methods 16Rural development in a changing world 17IFAD’s evolving mandate 18

Chapter 2. The project portfolio: performance and impact 22Resource allocation trends 22Portfolio trends and performance 26Project and programme performance 30Country programmes and policy influence 38Impact on rural poverty 42

Chapter 3. Corporate processes: management performance 55A framework to assess how IFAD’s results are derived from corporate performance 59Leadership and governance 60Policy and strategy development 64Human resources management 65Partnerships 69Management processes 71Knowledge management and learning 76Searching for the new business model 77

Chapter 4. Future performance 79Conclusions 79Recommendations 85

AnnexesAnnex 1: Bibliography 91Annex 2: List of People Interviewed 93Annex 3: Evaluation Methods and Instruments 96Annex 4: Synthesis of Country Working Papers 108Annex 5: Terms of Reference of the IEE 146Annex 6: The Comments of the IEE Senior Independent Advisors 159Annex 7: IFAD’s Management Response to the IEE 162Annex 8: Excerpt from the Minutes of the Eighty-Fourth Session of

the Executive Board 173

vii

ACBA Agricultural Cooperative Bank of ArmeniaACC Aid concentration curveADF Asian Development FundAfDB African Development BankAfDF African Development FundAFESD Arab Fund for Economic and Social DevelopmentARDE Annual Review of Development EffectivenessARRI Annual Report of Results and Impact of IFAD OperationsAsDB Asian Development BankAsDF Asian Development FundBCIE Central American Bank for Economic IntegrationBOAD West African Development BankBSF Belgian Survival FundCAF Andean Development CorporationCDG Credit Development GroupsCI Cooperating InstitutionCOSOP Country Strategic Opportunities PaperCPE Country Programme EvaluationCPIA Country Policy and Institutional AssessmentCPM Country Programme ManagerCSO Civil Society OrganizationCWP Country Working PaperDAC Development Assistance Committee (OECD)DRR Desk Review ReportEAD External Affairs Department (IFAD)EB Executive BoardEBRD European Bank for Reconstruction and DevelopmentEC Evaluation Committee (IFAD)ECP Extended Cooperation ProgrammeEKSYST Evaluation Knowledge SystemERT External Review Team FAO Food and Agriculture Organization of the United NationsFIDAFRIQUE Internet-based network of organizations and projects dedicated to

fighting rural poverty in West and Central AfricaFIDAMERICA Internet-based network of organizations and projects working with

the rural poor in Latin America and the CaribbeanFLM Flexible Lending Mechanism

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

viii

Abbreviations and Acronyms

ix

FPPP Field Presence Pilot ProgrammeFY Financial yearGC Governing CouncilHIPC Dept Initiative for Heavily Indebted Poor CountriesHIV/AIDS Human immunodeficiency virus/ Acquired immunodeficiency syndromeHR Human resourcesHRM Human resources managementIA Internal AuditIADB Inter-American Development BankIAPC Impact Achievement in the Project Cycle (an IFAD working group)IBRD International Bank for Reconstruction and Development (World Bank)IDA International Development AssociationIEE Independent External Evaluation (IFAD)IFAD International Fund for Agricultural DevelopmentIFAD V and VI Fifth and Sixth Replenishments of IFAD’s ResourcesIFI International Financial InstitutionIMI Initiative for Mainstreaming InnovationIR Inception Report (IEE)LFG Lake Fisheries GroupLGS Loans and Grants SystemM&E Monitoring and evaluationMDG Millennium Development GoalMTR Mid Term ReviewNEC National Evaluation Counterpart (for the IEE)NET National Evaluation Team (for the IEE)NRM Natural Resources ManagementODA Official Development AssistanceOE Office of EvaluationOECD Organization of Economic Co-operation OSC Operational Strategy and Policy Guidance CommitteePA Western and Central Africa Division (Africa I) (IFAD)PAG Pond Aquaculture GroupPBAS Performance Based Allocation SystemPCR Project Completion ReportPDIP Project Development and Implementation PartnershipPDT Project Design TeamPF Eastern and Southern Africa Division (Africa II) (IFAD)PI Asia and the Pacific Division (IFAD) PL Latin America and the Caribbean Division (IFAD)PMD Programme Management Department (IFAD)PN Near East and North Africa Division (IFAD)PPMS Project Portfolio Management SystemPRP Process Re-engineering ProgrammePRPP Progress Report on the Project PortfolioPRSP Poverty Reduction Strategy PaperPSR Project Status ReportPT Technical Advisory Division (IFAD)REA Rapid External Assessment (of IFAD in 1994)RIMS Results and Impact Management SystemRPA Regional Poverty AssessmentRRP Report and Recommendation of the PresidentSCP Strategic Change ProgrammeSPM Special Programming MissionTA Technical assistanceTAG Technical Assistance GrantTOR Terms of ReferenceTRC Technical Review CommitteeUNOPS United Nations Office for Project ServicesWFP World Food Programme

Acronyms of Sampled Projects from the Country Visits

AqDP Bangladesh Aquaculture Development ProjectAMSDP United Republic Agricultural Marketing Systems Development Project

of TanzaniaASP Armenia Agricultural Services ProjectBADP Pakistan Barani Area Development ProjectCORREDOR Peru Development of the Puno-Cusco Corridor ProjectEDNASP Egypt East Delta Agricultural Services ProjectMARENASS Peru Management of Natural Resources in the Southern Highlands ProjectNAFP Mozambique Nampala Artisanal Fisheries ProjectNWASP Armenia North-West Agricultural Services ProjectPAMA Mozambique PAMA Support ProjectPAMER Burkina Faso Rural Microenterprise Support ProjectPD-PAPE Guinea Smallholder Development Project in North Lower Guinea PD-PEF Guinea Smallholder Development Project in the Forest RegionPDRSO Burkina Faso South West Rural Development ProjectPF-ACP Pakistan Pat Feeder Command Area Development ProjectPRODESIB Bolivia Sustainable Development Project by Beni Indigenous PeoplePROSAT Bolivia Small Farmers Technical Assistance Services ProjectRFSP United Republic Rural Financial Services Programme

of TanzaniaSSWRDSP Bangladesh Small-Scale Water Resources Development Sector ProjectWNRDP Egypt West Noubaria Rural Development Project

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

x

A B B R E V I AT I O N S A N D A C R O N Y M S

xi

Name Role AffiliationInternational team

Derek Poate Team Leader ITADAlison Evans Senior Expert, Policy Analysis and Resource Allocation FreelanceIan Hill Senior Expert, Desk Review of Asia Region and

Human Resources FreelanceJohn Rowley Core Team, Knowledge Management and Burkina Faso FreelanceMary Underwood Core Team, Bolivia FreelanceAndy Batkin Core Team, Project Cycle Management and

United Republic of Tanzania FreelanceChris Barnett Core Team, Partnerships, Armenia ITADChris Brewster Human Resources Henley College of

ManagementNeil Chalmers Data management and systems FreelanceParvin Sultana Bangladesh ITADRidley Nelson Pakistan FreelanceEB (Ted) Rice Peru FreelanceChris Mowles Egypt FreelanceMunhamo Chisvo Mozambique Jimat Development

ConsultantsMonique Munting Guinea FreelanceJane Stanton Contract Administration ITAD

Name Role AffiliationLeaders of national evaluation teams1

Alexander Poghossian Armenia Alpha Plus ConsultingMahbub Alam Bangladesh Freelance Javier Gómez Bolivia Centro de Estudios para el Desarrollo

Laboral y AgrarioFernand Sanou Burkina Faso BIFACHossam Hussein Egypt North South ConsultantsIbrahima Diakité Guinea Freelance Verona Parkinson Mozambique Agema Consultoria & Services LdaCarolina Trivelli Peru Instituto de Estudios PeruanosAbdul Wasay Pakistan Innovative Development StrategiesAthumani Liviga United Republic of Tanzania NCG United Republic of Tanzania

1 Full lists of the consulting team in each country can be found in the Country Working Papers.

IEE International and National Team

1

Introduction and Approach1. This report presents the findings,

conclusions and recommendations of theIndependent External Evaluation (IEE) ofIFAD. The evaluation was conceived duringthe Consultation on the Sixth Replenishment of IFAD’s Resources and planned by theGoverning Council to be completed in timefor consideration in advance of the SeventhReplenishment. The main objectives are todetermine IFAD’s contribution to ruralpoverty reduction, examine the relevance of the organization’s mission and objectives,assess corporate learning and performance,and make recommendations on policy directions and steps to improve IFAD’s performance.

2. The evaluation was designed around a frame-work of impact and corporate performanceobjectives, and used generic criteria of relevance, impact, effectiveness, efficiency and sustainability to evaluate performance.The main period of reference was from 1994to 2003, but trends and initiatives both before 1994 and since 2003 have been takeninto account.

3. Data collection was conducted in two stages:(i) a desk review based at IFAD headquartersthat examined all relevant IFAD documents

and a sample of 21 country programmes (withparticular emphasis on 42 loan projects andseveral grants), and interviewed staff aboutcorporate processes and the performance ofprojects and programmes; (ii) visits to a sub-sample of 20 projects in ten countries. At bothstages, the IEE made extensive use of semi-structured individual and group interviewsand used ratings (based largely on IFAD’sevaluation methodology) in order tosummarise judgements. As stipulated by theExecutive Board (EB) of IFAD, the countriesand projects were selected randomly from allfive of the geographical regions in whichIFAD operates. Details of methodology are in Annex 3.

4. In addition to its scope and time constraints,the IEE methodology was limited by anumber of factors, including: the dearth ofreliable self-evaluation data from IFAD and itspartners; a somewhat upward or optimisticbias in the responses given by project benefici-aries and officials during self-assessment; andthe small number of completed projectsincluded in the sample. Notwithstanding theselimitations, the IEE has confidence in theveracity of the evaluation findings which areto be found in fourteen intermediate reportsand which are drawn together and summa-rized in this final report.

Executive Summary

The Evolving Mandate (Chapter 1 page 18)2

5. IFAD was set up in 1977 with a uniquelyexclusive focus on the rural sector. Initially itwas intended to fund projects that had beenidentified and designed by others. But quicklyit was seen that partner interests were muchbroader than food and agriculture and theirproject designs did not give sufficient focus to the needs of the poorest rural groups.Consequently the Fund moved into identifica-tion and project design, and in the mid 1990sadopted a new goal to lead global efforts inhelping the world’s poorest. By the early2000s IFAD had further expanded its missionto include a range of actions for enabling therural poor. The continuity with past strategieswas clear, but the new direction brought adirect focus on empowerment, on the role of markets and non-farm income and employment, and on decentralization and governance.

6. The broadening of IFAD’s mission is largely aproduct of changing ideas about poverty andhow development assistance should be deliv-ered. Wider trends include a move away froma project focus on agricultural productiontowards a more direct approach to povertyreduction, the adoption of poverty reductionstrategies under the Debt Initiative forHeavily Indebted Poor Countries (HIPC)framework, the international agreement onthe Millennium Development Goals (MDGs),and pressure for donors to align their objec-tives around those goals and to harmonizetheir ways of working. IFAD has attempted toretain a distinctive focus, largely by empha-sizing its catalytic and innovative roles, byarticulating a stronger role for partnershipsand policy dialogue, and by claiming a role asan advocacy organization for the rural poor.But as the number of actors and the ways ofdoing business in the rural sector change soIFAD’s niche has become less clear. IFAD,however, continues to emphasize area devel-opment as few others do, and it is the onlyinternational organization established to focusexclusively on the situation of the rural poor,and particularly the poorest. This mandatecontinues to be relevant, and it is furtherlegitimized by the importance recent progress

reports on the MDGs attach to the develop-ment of the rural sector. Indeed, the problemthat IFAD was created to tackle continues andis likely to continue in the foreseeable future.The critical challenge now is for IFAD todemonstrate its distinctive role based on aclearly defined approach to innovation,targeting, partnership and policy dialogue all directed to delivering results for the ruralpoor, and particularly the poorest, as part of the concerted global effort to reach the MDGs.

Resource Allocation Trends (Chapter 2 page 22)

7. The first step in the analysis of impact was toexamine the trends in resource allocation.IFAD’s Lending Policies and Criteria direct ittowards the poorest countries and poorestpopulations, but also require it to ensure a fairgeographic distribution of resources. Analysisof the flow of IFAD aid relative to the distribu-tion of population living below a dollar perday shows that the majority of IFAD’sresources went to the poorest countries.IFAD’s resource allocation is broadly pro-poor,but levels of lending per poor rural person aregenerally higher in less-poor countries, whichreflect a variety of factors: a tendency tosmaller rural populations, potentially highercosts of reaching poor groups in remote areas,and greater absorptive capacity in the lesspoor countries. Regional allocation shares haveensured broad geographic coverage alongsidean increasing focus on sub-Saharan Africa.Given that Africa is the continent least likely toreach the MDGs, this focus is appropriate. As aconsequence both Asia and Latin Americahave experienced declines in lending volumeduring 2001-2004. Both China and Indiareceive less assistance from IFAD than theirshare of the absolute numbers of poor peoplemight warrant.

8. IFAD has successfully leveraged cofinancingcommitments at the overall rate of USD 1.3 forevery USD 1.0 approved directly, but althoughthe volume is impressive it fluctuates wildlyfrom year to year, involves a very large numberof partners, many contributing very smallamounts, and has fallen compared with theyears before 1994. IFAD faces challenges to

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

2

E X E C U T I V E S U M M A R Y

adapt traditional cofinancing approaches intopooled sector programmes and general budgetsupport, and to conform with good practiceharmonization and alignment principles.3

9. The Fund is currently introducing a perform-ance based allocation system (PBAS), intendedto bring a more transparent framework toguide country allocations. PBAS will takeaccount of need, country and sector institu-tional and policy performance, and portfolioperformance. The approach follows that ofother international financial institutions (IFIs),but with some IFAD-specific features. WhilePBAS has the potential to help the Fund bemore selective in the level of resources andchoice of instruments within countries andsectors, it is not clear that IFAD has the neces-sary range of instruments to adopt a moredifferentiated country approach. Experiencefrom other IFIs suggests that a more selectiveresource allocation approach under PBAStends to mean higher levels of non-lendingand grants relative to lending in difficultpolicy contexts. The current set of IFADloans, technical assistance grants, and morerecently country grants, offers some flexibility,but is not based on a systematic approach toworking in different policy environments.

Portfolio Performance (Chapter 2 page 26)

10. Analysis of the portfolio is complicated by thedifficulties faced by the Fund in classifyingprojects and components. Over the periodfrom 1994 to 2003 aggregate OfficialDevelopment Assistance (ODA) flows to agri-culture declined. IFAD maintained its shareuntil 1999, since when there has been adecline. While the data on aggregate flowshave their limitations one implication is that,despite its specific mandate, IFAD was not ableto counteract the downward trend in aid flowsto agriculture in the 1990s, either directly byincreasing the share of its own resources goingto agriculture (although the impact would stillhave been modest given the scale of the reduc-tion from the International DevelopmentAssociation [IDA]) or indirectly, by persuadingother, larger donors to maintain or increasetheir resources going to agriculture. RatherIFAD has broadened its scope into rural

development. There has been a clear increasein project components delivering institutionalsupport, local capacity building, rural financialservices and rural infrastructure, while thefocus on input supply, fishing, irrigation andrural enterprise has declined. Componentdefinitions muddy the analysis, but there issome evidence that the portfolio is becomingmore complex, with inherent problems ofmanagement and technical support.

11. Portfolio performance data shows IFAD on apar with most other IFIs but there is no roomfor complacency. Compared with the WorldBank, for which rural data are available, IFADis underperforming, although the direction ofchange in performance showed improvementbetween 2000 and 2003. Poorer performanceis also found in the elapsed times betweenExecutive Board approval and loan signing,and then from loan signing to effectiveness.IFAD experiences longer times than otherIFIs and there are marked variations acrossthe regions. Long elapsed times betweenapproval and effectiveness usually indicatethat significant pre-implementation activitiesare required in order for a project to goahead, or that projects being presented to theBoard are below full ‘readiness’ - oftenmeaning that critical conditions have not beenmet by the borrower, or that cofinancing ornecessary institutional arrangements have notbeen forthcoming. In general, such delaysindicate a problem in the project design stage.

3

2 The page reference for each subheading directs the reader backto the relevant key points box, or main text where the findingsfor the topic were presented.

3 Alignment is about development organizations following strate-gies that reflect country development priorities; harmonization isabout development organizations adopting common aid policies(to avoid contradictions) and modalities to reduce transactioncosts to the recipient country.

Project and ProgrammePerformance (Chapter 2 page 30)

12. Analysis of performance is based on thesample of twenty, randomly selected projects,across the ten countries. All projects werestudied using a common methodology andinstruments, adapted for difference betweennew entrants that became effective within thepast year or so, and older or closed projects.A detailed presentation of the findings can befound in Annex 4. The findings support veryclosely those from the Office of Evaluation’s(OE) Annual Report of Results and Impact ofIFAD Operations (ARRI) in 2002 and 2003.

13. Relevance. The fit between objectives andneeds is high across all the sampled projects.But the IFAD strategic framework, and to alesser extent the regional and country strate-gies do not provide effective filters for selec-tivity. Because the Fund’s strategic guides areso inclusive, there is no clear basis on whichto turn down a project. In five projects fromthree countries it is doubtful whether theprojects were good investment choices forthe Fund.

14. Effectiveness. Two thirds of projects are rated as satisfactory, but half suffer majorimplementation problems and only becomeeffective after a mid-term redesign. Thisdelays effectiveness and undermines effi-ciency. There is a need for flexibility andresources to make adjustments earlier duringimplementation – an issue strongly linked to supervision.

15. Targeting. Using project documents the deskreview rated targeting as high, but evidencefrom the countries is that it is less effective inpractice. Country strategic opportunitiespapers (COSOPs) and project documents lackclear analysis of criteria to identify the poorand do not provide guidelines for implemen-tation. Leakage of benefits to the non-poor isa problem, but one that provides IFAD anopportunity to develop improved means toassist governments in their delivery of ruralservices to the poorer sections of society.

16. Efficiency. Projects are rarely subjected toeconomic analysis and insufficient attention is

paid to indicators of cost-effectiveness, espe-cially for projects where the end use ofinvestments is not known ex ante. About halfof IFAD projects do not represent a good useof resources invested and only 45 per cent of the sample was rated as havinga ‘high’ or ‘substantial’ level of efficiency.

17. Borrower performance. As noted in the deskreview, institutional analysis is often inade-quate, especially assessments of projectmanagement capacity. Less than satisfactoryproject management is a major factor thatundermines implementation in the earlyyears. There is a need to plan managementarrangements better and for the Fund tolearn from lessons in those countries whereopen competition for recruitment of manage-ment teams has been used to good effect.

18. Supervision. IFAD is required by its operatingrules to contract supervision to cooperatinginstitutions (CIs). This has long been identi-fied as a weakness and has been criticized byprevious evaluations. The findings in thisstudy support those criticisms. TheIndonesia country programme evaluationand interviews by the IEE with cooperatinginstitutions reveal that IFAD pays less thane.g. the World Bank and the United NationsOffice for Project Services (UNOPS) regardas the full costs. Quality of supervision asjudged by ratings in the country visits issubstantially weaker than quality assuranceestimates for the World Bank’s rural port-folio. CIs tend to focus on financial and fidu-ciary aspects to the neglect of developmentissues. This shortcoming contributes todesign issues being deferred until mid-term.IFAD needs to monitor and intervene moreeffectively during supervision. The combina-tion of contracted-out supervision and theindividual nature of the links between acountry and the country programmemanager (CPM) give rise to a fundamentalstructural weakness in IFAD’s headquarter-to-field relationship.

Country Programmes andPolicy Influence (Chapter 2 page 38)

19. The COSOP was introduced in 1995 toreplace the general identification and special

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

4


programming missions. At first it was a low-cost strategy paper, but it has been developedwith the intention of articulating a countryprogramme with synergy between projects,and across loans, grants and policy dialogue.It does not yet reach those expectations,though a few isolated good examples indicatethe potential of the approach. More oftenthan not the COSOP is found to be driven bythe current and planned portfolio, with weakanalysis of IFAD’s comparative advantageand rationale for engagement in the country.More recently the Programme ManagementDepartment (PMD) has devoted time toimproving quality.

20. There is interesting, but patchy evidence ofpolicy influence in the sampled projects inMozambique and the United Republic ofTanzania, and other examples from IFAD’sportfolio in Bangladesh. IFAD’s mandatecalled for evidence from projects andprogrammes to be used to engage in policydebate since the very beginning. But theFund has neither developed sufficientcapacity to undertake detailed policy analysisin such a way as to equip staff to promotesuch a dialogue, nor developed a model topromote policy influence. Even where goodand long-standing relations exist between theFund and national partners, examples fromcountries such as Egypt and Indonesia reveal that IFAD has not engaged in policy discussion.

21. A similar finding emerges from IFAD’spartners in other development organizationsat country and international levels.Interviews with sector and regional staff ofthe World Bank, Asian Development Bankand Inter-American Development Bank(IADB) suggest that IFAD is known for itsproject work in hard-to-reach rural areas orwith marginalized groups. The Rural PovertyReport 2001 is acknowledged by others inthe field, but more generally IFAD is notknown as a source of policy ideas or as aleader of policy debates about rural develop-ment and rural poverty reduction. IFAD’ssize and resources limit its ability to fulfil its policy, advocacy and innovation rolesacting alone.

Impact on Rural Poverty (Chapter 2 page 42)

22. Assessing IFAD’s impact is a central elementin the terms of reference. It is a challenge forthe IEE because the sample covers projects atdifferent stages of completion. Nevertheless,the methodology tried to take these differ-ences into account, and where appropriateresults are quoted for closed and nearly closedprojects rather than for the whole sample.Comparisons are also made with the ARRI.

23. Unsurprisingly, impact varies widely. The IEEencountered a few very good projects, butacross the sample as a whole impact onpoverty was variable. Only a little over half ofthe sampled projects showed a satisfactoryoverall impact on poverty. Moreover, most of the component elements of impact areassessed as modest or negligible while the like-lihood of sustainability falls dramatically at ornear project closure. Such variable perform-ance might be acceptable if it were associatedwith an innovative and risk taking approachto tackling poverty. But the sample scorespoorly on innovation.

24. The two projects rated highest, in Peru, havestriking characteristics: a rich innovativecontent; project design and managementflexibly subordinate to community demands;design adapted to reach poorer groups;committed national directors; and a strongdriving force from the CPM who, as ithappens, is based in Peru. The resultingimpacts combine rising levels of food security with a dramatic effect on beneficiaryself esteem.

25. IFAD’s traditional areas of expertise remainstrong: agricultural production and foodsecurity, plus some notable benefits from roadconstruction, irrigation work and financialservices. Non-traditional areas such as healthand education are small components of theportfolio yet impact is reasonable and commu-nity-led water supply seems particularly effective and appreciated.

26. Performance in other areas is more mixed.Fewer than half the sampled projects havegrappled with the challenging issues of

5

environmental conservation and developmentand of these nine, four show satisfactoryimpact. Interventions to promote social capitaland to give voice appear to bring significantshort-term gains, but where new institutionsare created sustainability is very much indoubt. Longer term support and clear exitstrategies need to be better articulated.

27. Innovation is seen as central to the achieve-ment of IFAD’s mandate. IFAD defines innovation in a broad way and while it hasaspirations to be an innovator, evidencesuggests otherwise. The IEE sample of opera-tions clearly indicates that while there are afew highly innovative projects (as well asothers that contain innovative elements),many are not. IFAD’s contribution to thecapture, learning, promotion and replicationof innovation also appears unsystematic andinadequate given its corporate mission.

28. The assessment of project performance revealsweaknesses: relevance is high, and effective-ness good after mid-term redesigns, butprojects lack effective approaches to targeting,are held back by inefficient supervisionarrangements, and experience declining likeli-hood of sustainability as they mature. The keyissues affecting impact on rural poverty areportfolio performance, quality assurance anddelays to implementation which need delib-erate attention by IFAD managers. All reflectaspects of corporate management andprocesses which are analysed in Chapter 3.

Corporate Processes:Management Performance(Chapter 3 page 55)

29. IFAD developed a simple but effectivebusiness model in its early years after itstarted to identify and design projects. Themodel hinged on a one-to-one relationshipbetween the then project controller (nowCPM) and the country, and is referred to asthe CPM-country model. It enabled highlycustomized project designs. It was a relativelylow cost approach, gave great flexibility toCPMs and built on the technical skills ofconsultants and partner agencies. However,changing ideas about development assistanceled, at the country level, to programme and

sector wide approaches driven by nationalpriorities and poverty reduction strategies.This was linked to changing priorities withinIFAD, that emphasized leadership, policyinfluence and innovation. These changescalled for a different approach, new skills andnew ways of working. An important part ofthe IEE has been to examine whether IFAD’sbusiness model has evolved sufficiently toenable the Fund to respond effectively tothese new demands. The elements of thebusiness model and their impact on corporateperformance are discussed in relation to lead-ership and governance, policy and strategydevelopment, human resource management,partnerships, management processes, andknowledge management and learning.

Leadership and Governance (Chapter 3 page 60)

30. Corporate oversight is the responsibility of theGoverning Council (GC) and Executive Board(EB). Business arrangements are similar tothose of other IFIs and provide for regularmeetings with records of business. Otherarrangements contribute to governance, suchas a policy on disclosure in 2000, an InternalAudit (IA) office, a Joint Appeals Board of staffand management for investigating staff-relatedcomplaints, an Oversight Committee and the2003 evaluation policy.

31. Policy is the concern of both the GC and EB,but in practice the main instrument of direc-tion has been the replenishment processes.The Fourth Replenishment in 1992 markedthe first real engagement by Member Stateswith IFAD policy and strategy. IFAD V intensi-fied the process and established for the firsttime a monitorable programme of change.IFAD VI took that process further, bringing asignificant new agenda, including PBAS, theResults and Impact Management System(RIMS), a new evaluation policy and thepiloting of the field presence initiative. Thenew agenda attempts to bring IFAD into linewith other multilaterals, but sits uneasily withthe older idea of IFAD as a small, hybridorganization, focused around innovation andexperimentation. In practice, the policy orien-tations of IFAD V and VI have dominated thework of both management and the EB.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

6


32. The EB meets only three times a year for twodays each session. This is far less than compa-rable regional IFIs and the World Bank, evenallowing for IFAD’s smaller portfolio. Withsuch infrequent and short meetings theagenda is very crowded, with large volumes ofmaterial, not always available well in advance.These characteristics taken together with thelimited prior experience of Directors in theoperations of an IFI, lack of training, andabsence of guidance through terms of refer-ence, result in a Board that does notadequately fulfil its role.

33. Current arrangements do not foster results-based scrutiny of development effectivenessbecause the Board lacks the tools and orienta-tion. Two sub-committees, audit and evalua-tion, serve the Board. Neither, however, hasdeveloped a comprehensive approach to port-folio quality and development effectiveness.Since mid-2004, detailed scrutiny of theFund’s programme of work has been startedby the Audit Committee. In December 2004,policy development as well as self-evaluationwas brought within the purview of theEvaluation Committee (EC).

Policy and StrategyDevelopment (Chapter 3 page 64)

34. The original business model, with its emphasison a low-cost flexible arrangement betweenthe Fund and countries, led to a highlycustomized approach to project design.Vestiges of that style remain and have heldback moves to adopt more normative policiesgoverning operations and internal proce-dures. As a result, policy development haslagged behind the broadened mandate and islow compared to other IFIs. Awareness ofIFAD policies and strategies among partnerorganizations at country level and internation-ally is also low.

35. The strategic framework 2002-2006 providesa clear sense of mission and the three strategicobjectives provide staff with a common refer-ence and a clear basis for communication. Butneither it nor the regional strategies provide aclear guide to greater operational selectivityor increased development effectiveness.

Instead the content is largely permissive, as isthe treatment of IFAD’s main target group.

36. The number of CPMs in PMD, constant since1994, are too few for IFAD to have been ableto cope with new demands. CPMs areexpected to foster leadership at country leveland among partners, use knowledge gener-ated from within the programme to find inno-vative solutions, and to catalyze the work ofpartners to scale-up promising interventions.Use of consultants to support these functionshas often been effective in the short term, butat the expense of limited learning and institu-tional development within the Fund. Theneed to develop the skills of CPMs has notbeen recognized as a crucial area for humanresources skills development.

Human Resources Management (Chapter 3 page 66)

37. IFAD’s human resources are the sole source ofinitiative, action and results. Management ofthese resources impacts directly on the opera-tional performance of the Fund. The IEEinvestigated human resources management(HRM) in some detail, with extensive meetingswith groups and individual managers and staff,plus a staff survey that attracted a responsefrom more than two thirds of all personnel.

38. Human resource (HR) policies have beenconservative and administration based. Theyhave failed to establish a culture of standardsetting, accountability and performancemanagement. Despite apparent limitationsimposed by the zero real growth budgetconstraint, demands from IFAD V and VI ledto an increase of 43 per cent in both staff andconsultant inputs between 1994 and 2003. Atthe same time the numbers of loans per yearremained fairly static resulting in a fall in theFund’s operating efficiency.

39. The profile of staff and consultants in IFADshows some important characteristics: n There has been a significant improvement

in the gender balance of staff and consult-ants between 1994 and 2003, with anoverall increase of 50 per cent in femaleemployees, and increases at all levels of the organization.

7

n Almost all staff who have worked for atleast three years get promoted. This meansthat many if not most posts are upgraded,leading to grade creep - in itself a disincen-tive to genuine career advancement efforts.

n Almost 50 per cent of professionals at P4and P5 level will retire within the next fiveto six years, representing a problem, interms of loss of institutional memory, butalso an opportunity for regeneration.

n There is a very high reliance on consult-ants, with a professional staff to consultantratio of 0.92.

40. The organizational culture is seen to workcounter to IFAD’s aims to be an innovative,learning organization. Extensive use ofconsultants limits the internal development ofinstitutional knowledge. Surveys reveal a lackof trust between management and otheremployees owing in part to failures of commu-nication. Managers are thought to be risk averse.

41. Poor HRM has been a major contributor tovariable project performance. A new humanresource policy was approved in 2004. Iffully implemented it will address many ofthese problems but will require radicalchanges in the culture of the organizationthat must receive the priority attention ofsenior management.

Partnerships (Chapter 3 page 69)

42. IFAD’s catalytic role calls for the Fund to workin partnership both to leverage funds and topromote replication and scaling up of invest-ments. IFAD V and VI established a newpolicy environment for partnerships, but theguidance was confusing and contradictory.Unwittingly, the replenishments led to anuncritical use of the term and failed to fosterclear objectives and improved ways of workingthat would bring strategic benefits. There areexceptions, the relationship with the BelgianSurvival Fund (BSF), the emerging relation-ship with the Consultative Group to help thePoor, the partnerships with NGOs throughemerging networks such as FIDAMERICAand FIDAFRIQUE, but generally no system-atic evidence has been compiled by IFAD toshow that partnerships lead to greater impact

or effectiveness. Specific gaps are to be foundat country level, where IFAD has notresponded to moves by other donors towardsdecentralized operations and has not devel-oped new approaches to partnership working.This is of particular concern in the light of the Rome Declaration (2003) and the ParisDeclaration (2005), both of which placespecific emphasis on the need for more effective donor partnerships as a way todeliver more timely and cost-effective develop-ment assistance in support of national devel-opment strategies.4

Management processes (Chapter 3 page 71)

43. IFAD staff and management have never beenslow to re-examine processes and initiatechange. The initial years of the Fund saw asteady evolution of practice as the originalmodel of cofinancing projects developed byother IFIs was replaced by IFAD-initiateddesigns that reflected IFAD’s approach. Acornerstone of that approach was andremains the freedom of action granted toCPMs, under which they controlled the rela-tionship with the government, the identifica-tion of projects, the technical design process,and the cooperating institution during imple-mentation. This model is characterized bythe IEE as a ‘free-agent’ CPM. The yearssince 1994 have seen a large number ofpowerful initiatives and participatoryprocesses that have been self critical aboutthe need to reform. Many have been directedat the project cycle, others cast more widely.But it is significant that, unlike the WorldBank, where similar change was driven byanalysis of performance, none of the initia-tives described here have been stimulated byanalysis of development effectiveness. Theyhave been process-driven on issues of effi-ciency, rather than performance-driven byanalysis of impact. None of the initiatives hasbeen evaluated for impact on IFAD’s devel-opment effectiveness.

44. IFAD’s business model and resourceconstraints directed attention towardsimproving the project cycle. At first the aimswere to achieve better quality and to managecosts; later they changed to coping with the

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

8


policy demands from IFAD V and VI. Thecurrent strategic change programme willintroduce improved information technologyand financial management systems. However,the large number of initiatives over the pastdecade have not been well connected nor seenthrough to recognizable outcomes, a failure ofmanagement that has left staff sceptical aboutwhether such changes make any difference.The historical overview in Box 11, page 73charts a logical development of initiatives inresponse to operational needs. But the IEE inthe desk review argues that IFAD’s niche andcomparative advantage were poorly articu-lated; that innovation is misused and misun-derstood; and that projects are both over andunder-designed. Nor, despite so many initia-tives to improve quality has the Fundsucceeded in instituting an effective qualityassurance system to analyse project designs.

45. The need for the role played by the CPM tochange has been recognized but not resolved,so the present free-agent CPM model remainsa constraint on ways of working that detractsfrom projects, programmes, knowledgemanagement and innovation. At best itenables entrepreneurial CPMs to flourish. For many, it leaves an unchallenged andunder-managed working environment thathas created a quality assurance process thatprovides neither effective support, noranalysis and accountability for quality. Thus,the Fund has not yet succeeded in developinga business model that responds to the Fund’scurrent challenges.

Knowledge Management andLearning (Chapter 3 page 76)

46. Knowledge has always been part of IFAD’scurrency. But the important role of knowl-edge to IFAD’s mission was given shape in the1994 Rapid External Assessment (REA) andquickly led to the IFAD V: Plan of Action with objectives to be innovative, a knowledgeinstitution, a catalyst and a leader in ruralpoverty reduction.

47. A wide range of practical initiatives wasmounted and tools developed, many similar tothe initiatives in other IFIs and developmentorganizations in the mid-1990s. But IFAD’s

draft knowledge management strategy wasnever actually approved and management’sinterest drifted. In sum, an organizationalculture with poor communications, aversionto risk, limited team working and no staffdevelopment, has not fostered learning.

48. In the desk review report (DRR) the IEEtested IFAD’s performance against a modelof knowledge management that charts thesequence of gathering information, identi-fying its value, storing, and disseminating itto IFAD’s own operations and to partners. A fundamental weakness is that IFAD has apoor record of data collection and self-evalu-ation. Project arrangements for monitoringand evaluation (M&E) of loans and grantshave been widely criticized inside IFAD itselfand in a variety of studies, and there is a lowpriority given to finding information fromexternal sources – despite so many ‘partner-ships’. Mechanisms with potential, such ascore learning partnerships and agreements at completion point, have had limited impact and even been counter-productive.The website has brought speedy internalretrieval of information, and staff make good use of that together with informal links to meet their personal needs. There are plenty of media available for dissemina-tion, but respondents in IFIs and at countrylevel report low levels of awareness, pooraccess and minimal utilization. There has been no monitoring and follow up to communications.

49. As a result, awareness of policies and strate-gies among partner organizations at countrylevel and internationally is low, and with theexception of the Rural Poverty Report 2001,

9

4 Both the Rome and Paris high level meetings took placeoutside the evaluation period for the IEE. No systematicreview of how IFAD has responded to the two Declarationswas therefore possible. Observations since 2003 suggest,however, a lag between IFAD’s corporate commitment to thealignment and harmonisation agenda and practical changesat country and field level in the short to medium run.

IFAD is absent from contemporary develop-ment literature.

Conclusions (Chapter 4 page 79)50. The period under review has been one of

change. Direct support to agriculture hasbeen replaced by wider enabling actions tofoster rural growth, and the internationalcommunity has formed a new consensusaround the MDGs. Against this backdropIFAD has expanded its mission. The need tomake rapid and sustained progress towardsthe MDGs places a premium on the combinedefforts of the international developmentcommunity to support national poverty reduc-tion goals. The quality of aid has come sharplyinto focus. In this context, the IEE concludesthat IFAD’s mandate – to provide additionalresources to improve the wellbeing of therural poor – remains fundamental. But thereis a danger that IFAD’s agenda has becometoo broad and distant. Weaknesses haveemerged in its management and delivery. TheFund needs to focus on its core distinctivenessas an innovator, where its potential compara-tive advantage lies.

51. Resource allocation and portfolioperformance. The evaluation has found thatIFAD’s portfolio of projects is broadly pro-poor, but resource allocation has been largelyad hoc, and not fully responsive to regionaland country needs, or to borrower perform-ance. The introduction of the PBAS shouldhelp the Fund to be more systematic in itsselectivity. But given IFAD’s small size, thereare bound to be concerns about the size offuture allocations to countries that are at thelower end of the policy and institutionalperformance scale (what are often calledfragile states) where the risk of conflict orinsecurity may be higher and the need forlong term support greater than elsewhere.How PBAS will influence the prospect forIFAD’s work in these areas is yet to be seen.

52. Assessment of the performance of the portfolio is complicated by the difficulties of benchmarking against organizations withdifferent approaches to measurement anddifferent operational procedures. Theevidence suggests that IFAD’s portfolio

performance is similar to some comparatoragencies, but this judgment is hampered bythe limited use made of self-evaluation andthe lack of systematic analysis generated fromthe current approach to quality assurance.

53. Project and programme performance.Relevance is high with respect to nationaldevelopment priorities and IFAD corporatestrategies but the test is a weak one as neitherthe strategic framework, nor country strategiesprovide an effective filter for project selectivity.COSOPs have largely been aggregations ofproject ideas, with little synergy betweenprojects or instruments. Only the most recentare starting to show some analysis of whereIFAD can best add value. Policy influence atproject and programme level is minimal.

54. Evidence about effectiveness and targetingboth point to weaknesses in the designprocess, highlighted originally in the DRR.Many projects experience implementationproblems, not all within IFAD’s control, but a significant proportion are associated withweak project management that reflects poorinstitutional analysis during design.

55. The majority of projects achieve their projectlevel objectives, but the causal link betweenobjectives and poverty impacts is often unclearand frequently over-ambitious. The resultsstructure that underpins the logframe has notbeen adopted by IFAD and consequentlythere are many examples of inconsistentlanguage, poorly defined objectives and inap-propriate or unworkable indicators. There is ahigh frequency of project redesign, reflectingpoor quality at entry, weak economic andinstitutional analysis and problems withmanagement and supervision. Lastly, effi-ciency is rarely assessed through economicanalysis and only half of IFAD’s projects canbe regarded as a good use of resources.

56. Policy influence at both project andprogramme level is minimal. Some importantexamples of policy influencing based onproject experiences do exist, but most takeplace within a vacuum – reducing the likeli-hood of both wider strategic policy engage-ment and longer term institutionalisation.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

10


57. Development impact. A few high-achievingprojects have delivered significant gains inhousehold food security and communityempowerment, through innovative, commu-nity-driven approaches. But only half theprojects sampled achieved more than modestimpact. Much more needs to be done to makean enduring contribution to the MDGs. Thelikelihood of benefits being sustained is onlymodest overall.

58. Innovation is a raison d’etre for IFAD, but theevidence reveals major shortcomings inIFAD’s approach. There is a lack of clarity inoperational practice, a tendency to view it asan end rather than a means, and a lack ofattention to both innovation and scaling-up inproject objectives.

59. Corporate performance. The IEE analysishas identified a number of weaknesses incorporate performance. One of the principalreasons for such weakness is that the originalbusiness model has not evolved to meetpresent demands and does not currentlydeliver the type of catalytic empowermentprogramme to which the Fund aspires. At theheart of this lies the continuing CPM-countrymodel of working. Corporate performancehas also been affected by issues affecting otherelements of the business model discussed inparas 30 – 49. The need for change has beenrecognized and in the past ten years there hasbeen an almost continuous process ofresponding to changing needs. However, theresulting changes have neither gone deepenough nor far enough and have been heldback by an organizational culture that isrooted in the original role of the Fund as acomplementary funding institution.

60. Corporate governance. Current arrange-ments meet the basic needs of the organiza-tion, but a crowded agenda and shortduration of meetings have limited the abilityof the EB to promote improved developmentperformance. Improved functioning of theBoard with a focus on quality assurance andperformance management processes at corpo-rate level has the potential to deliver clearerstrategic direction and more demanding oversight.

61. Policy development. IFAD has relatively fewoperational or sectoral policies and fewgoverning internal processes and procedures.Policy development is inadequate compared toother IFIs and has not kept pace with IFAD’sevolving strategic agenda. Awareness ofpolicies and strategies among partner organi-zations at country level and internationally islow. But recent attempts to address the policy‘deficit’ have potential. The implications of apolicy influencing role were not picked up in terms of skills development and the impli-cations for organization and management of CPMs.

62. Human resource policies have been veryconservative, and focused only on permanentstaff, despite the high proportion of tempo-rary and consultant employees. The corporateculture has failed to provide incentives forinnovation, failed to foster trust betweenmanagement and other staff and led to poorvertical and horizontal communication. Thenew policy has the right elements for change,but will require enormous commitment anddrive to see it through.

63. Partnerships are essential for the Fund toperform its catalytic role. But the currentapproach has led to a proliferation of weakarrangements that lack strategic relevance,and have neither objectives nor indicators tomonitor performance.

64. Management processes. The need forchange has been recognized and is reflectedin a series of initiatives over the periodstudied for the evaluation. These have been driven more by a search for efficiencythan the need to improve impact. The flatmanagement structure with consensusdecision-making has not provided adequatestrategic direction and drive to followthrough on change agendas. The roles andstructure of the CPMs and regional divisionshave stayed largely the same and notresponded to the evolving mission. With solittle evidence of the value of change staffare, on the whole, sceptical about the efficacyof change, though the most recent initiativeshave been more effectively implemented.

11

65. Learning and knowledge are critical to thesuccess of the Fund. Efforts have been madeto promote knowledge management, butthey have been undermined by weaknesses inthe organizational culture. Recent initiativeshave made an appreciable impact, in partic-ular the disclosure policy and website, butthere is a substantial communications gap,with low levels of awareness about IFAD’spolices and strategies among partner organi-zations, both at country level and interna-tionally. IFAD’s small size and resources, and extensive use of consultants limit itslearning ability.

Implications for the Fund66. IFAD is at a pivotal point in its history. The

level of performance that emerges from thisevaluation shows that while the Fund has arelevant, clear and distinctive role to play, itneeds to achieve a better performance, onethat is characterized by more effective innovation and focus on poor people in hard-to-reach circumstances. Without that,Member States inevitably will ask if it is effi-cient to continue disbursing those loans andgrants through the Fund. There is a clearchoice. Management can continue to intro-duce isolated reforms and react to changingcircumstances. Or it can embrace, as the IEErecommends, a more fundamental set ofimprovements to revitalize the Fund and shiftto a new path. The recommendations aredesigned collectively to achieve that changeand the Seventh Replenishment periodprovides a clear time frame during whichresults should be delivered and assessedbefore the next replenishment.

Recommendations(Chapter 4 page 85)

67. The way forward is to build on the processesof change which have engaged IFAD inrecent years. The challenge is to define andimplement a programme that succeeds whereprevious attempts have failed. Managementhas operational and strategic responsibility,but the Board must provide direction and a conducive authorizing environment. Therecommendations are developed under six headings:

Managing the change

68. The most difficult task is to come up with achange plan that is credible in the face of pastfailures, and is convincing both to the EB and to staff. The scale and gravity of changewhich encompasses the HR policy, a newbusiness process, and pressure to achievemeasurable improvements in developmenteffectiveness by the Eighth Replenishmentmeans the stakes for IFAD are high and theoutcome cannot be left to chance. In suchcircumstances, management theories argue fora more authoritarian approach. The IEEtherefore recommends as an overarchingobjective that a person is appointed –reporting only to the President – with broadexecutive powers and charged with the task ofsetting performance-based objectives anddriving through change to revitalize the Fund.

Address causes of low impact

69. The evaluation has identified that developmentperformance of IFAD projects and countryprogrammes must be improved. The existingquality assurance mechanism needs to be radi-cally improved to examine new designs andproject cycle processes, such as supervision andmid-term reviews, with transparent analysis andlinks through staff accountability to HRperformance management. Examples that haveproved effective in other organizations spanportfolio reviews of key themes such as arrange-ments for targeting the poor, or for tacklinggender issues; self-evaluation of projects; anddevelopment of internal and operationalpolicies, and good practice guidelines.

Develop a new business model

70. The need for a new business model is evidentboth from the expanding scope and breadthof work of the Fund, as well as the changingapproaches to development planning andorganization at country level. The recom-mendations are designed to give more flexi-bility to management and remove historicalconstraints that are no longer relevant.Specific elements should include: n changing the free-agent, arms length

CPM model;

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

12


3

1

2

n a zero-base budgeting exercise to re-assessneeds across IFAD as a whole and to real-locate savings from the Strategic ChangeProgramme (SCP), and funds currentlyused for consultants and supervision, intonew staff positions and a new structure forthe CPMs, regional divisions and technicalsupport in PMD;

n use of PBAS for greater selectivity anddifferentiation of services within countries;

n implementation of the new humanresources (HR) policy;

n preparing a strategy to influence policy atcountry and international levels, linked toa revitalized knowledge managementprocess and building on the recent goodwork to improve communications;

n creating effective mechanisms to reform orreplace the Operational Strategy andPolicy Guidance Committee (OSC)/Technical Review Committee (TRC)/Project Design Team (PDT) to provide amore incisive challenge to new policies,programmes and projects;

n releasing the Fund from the long-standingrestriction on project supervision; and

n expanding outposting of a proportion ofCPMs with a special focus on large countryprogrammes, or those with a high innova-tive content or in difficult environments.

Adopt smarter ways to encourage skills and learning

71. To gain access and to contribute toemerging policy ideas, to help in identifyinginnovation, to help improve potential forscaling up and the learning of lessonsthrough applied research, and to help buildthe knowledge base for its role as anadvocate for the rural poor, the Fundshould engage in a strategic long-term part-nership with a development research organ-ization of world-class standing in researchand policy analysis. IFAD is too small to beeffective at these demanding roles actingalone. Such a relationship would be a truepartnership, with the joint-venture able tobe innovative and take risks. The relation-ship would deepen understanding and tech-nical knowledge of policy issues drawingempirical evidence from IFAD’s projects.

72. In order to help the Fund tackle the newdemands on technical supervision andquality assurance, and release CPM’s time formore technical work, we recommend that theFund looks beyond working with individualconsultants and engages commercial and not-for-profit organizations to manage opera-tional tasks such as aspects of the grantsprogramme, portfolio reviews, quality assur-ance and technical supervision at countrylevel, with a specific aim to develop anddisseminate lessons and policies. This wouldbring new approaches to quality assurance,address weaknesses in institutional learning,support the changing business model, andimprove accountability.

73. Project level M&E remains weak and needs tobe re-energized together with selective in-depth studies of innovations during imple-mentation on a small sample of projectsconsidered to have high learning potential.

74. At a more detailed level the Fund shouldsupport reforms under the HR policy andimprove the visibility of the work of individualmembers of staff.

Clarify IFAD’s strategic niche; re-assert its complementary role

75. IFAD needs to clarify its role primarily as aninnovator in policy, institutional and opera-tional terms rather than as a purveyor of fairlyroutine projects which closely mirror theapproaches of larger development organiza-tions. IFAD’s comparative advantage does notlie in competing with the other IFIs but inbeing a progenitor of well tested innovativeideas and approaches that can be replicatednationwide by others with greater resources.

76. The evaluation evidence is clear that the Fundneeds to clarify this role better, linked to morestrategic selectivity at country and sector level.A more systematic approach would havecertain key features. Firstly, a link to a knowl-edge management system in which lessonsfrom IFAD and other sources are identifiedand disseminated; secondly, recognition ofinnovation in project designs by inclusion asan objective, with associated arrangements forflexibility, risk-taking and evaluation; and

13

5

4

thirdly, the identification of potential partnersfor scaling-up at the commencement of projectimplementation, to foster ownership andshared learning. The new initiative for main-streaming innovation (IMI) will help contributeto capacity development. The approach shouldinvolve more risk with greater attention tolearning from and disseminating results. Thisimplies a more discriminating management ofgovernment and non-governmental partner-ships and the strategic use of grants whererisks are significant.

Provide direction for development effectiveness

77. Reform proposals in recent years confirm the Board’s commitment, but limitations inorganization, meetings and the choice ofinstruments have held back real progress. As regards organization, the Board needs amechanism to scrutinize work programmesand budgets more effectively, and to examinedevelopment effectiveness, although changesto the scope of work of the audit and evaluation committees have now been made.Executive Directors should at least have clear terms of reference to define the role.Management systems to analyse the portfolio and its processes need to be revital-ized to provide directors with an analyticalview of operations, rather than isolatedglimpses of single countries and projectsthrough evaluations.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

14


6

Purpose of the independent external evaluation

1.1 During its deliberations, the Consultation onthe Sixth Replenishment of IFAD’s Resourcesendorsed a proposal to have an independentexternal evaluation of IFAD. Accordingly, theGC decided that the evaluation should beplanned and begun in 2003 and completed in2004, in time to allow for full consideration ofthe IEE report by the Executive Board, priorto its recommendation to the GoverningCouncil on the Consultation on the SeventhReplenishment of IFAD’s Resources.

1.2 The objectives of the evaluation are set out insection II of the terms of reference, and state:The main objective of the IEE is to determineIFAD’s contribution to rural poverty reduction, theresults and impact it has achieved in this area, andthe relevance of the organization’s mission andobjectives in relation to international developmentgoals and the national development strategies ofIFAD borrowing countries. The evaluation isfurther expected to assess whether and what IFADhas learned from past experience and how theFund’s policies and operations have evolved inresponse to lessons learned from that experience, andfinally, to offer recommendations on the policy direc-tions IFAD should pursue and other steps it shouldtake to improve its future performance 5

1.3 This objective encapsulates three strands: n development results and impact in regard

to rural poverty reduction;

n the corporate learning process andperformance of policy and operations; and,

n forward looking recommendations toimprove future performance.

1.4 This document is the final report. It drawstogether the findings from an extensive bodyof work comprising an inception report (IR),desk review report (DRR), ten working papersfrom countries where randomly selectedprojects were studied, and two additionalbackground papers dealing with IFAD’sgovernance and HRM. Management hascommented on all intermediate materialsincluding the country working papers andbackground papers. Any factual errors thatwere highlighted have been corrected.

1.5 The report is in four chapters. This chapterdescribes the context and sets the scene with areview of IFAD’s evolving mandate. Chapter 2presents the findings about IFAD’s portfolio ofprojects, their performance during implemen-tation, impact on rural poverty and influenceon policy under four sub-headings: resourceallocation trends; portfolio trends andperformance; project and programme

15

The evaluation and its context

5 IFAD (2003) Independent External Evaluation of IFAD, Termsof Reference, para. 3. Rome.

Chapter 1

performance; and impact on rural poverty.Chapter 3 examines how the Fund’s corporateprocesses and management have performed,and traces the link between that performanceand the Fund’s development impact. Chapter4 takes stock of the findings, draws conclu-sions and makes recommendations to improvefuture performance. The report is supportedby the first four annexes. Annex 1 is a bibliog-raphy, Annex 2 lists people who were inter-viewed during the study (excluding those inthe country visits who are reported in thecountry working papers). Annex 3 presentsthe evaluation sample, methodology andinstruments that were used. Annex 4 containsadditional analysis from the country visits.The report also draws on earlier IEE productsand extensive reference is made to themthroughout the text. To simplify presentationthe IEE inception report is referred to as IR;the desk review report as DRR; and thecountry working papers as CWP followed bycountry name.

Evaluation framework andmethods

1.6 A framework for the evaluation was developedin the IR. It was constructed from analysis ofthe objectives set for IFAD in its originalmandate and subsequently modified or rein-terpreted through later policy initiatives. Theframework is reproduced in Table 1.

1.7 The evaluation framework is the structure ofissues to be examined – based on the studyobjectives of development impact and corpo-rate performance. Evaluation criteria aregeneric and follow the Organization ofEconomic Cooperation/Development AssistanceCommittee (OECD/DAC) guidelines: impact,relevance, effectiveness, efficiency and sustain-ability. These are described in Annex 3, para.4.23. The framework and criteria were used to create sets of questions that defined theenquiries of the evaluation. They can bereferred to in the IR, Table 7, pages 23 to 32.

1.8 Following the design, the evaluation wasconducted in two stages of data collection.First, a desk review, with a small group ofteam members based in Rome, followed byvisits to a sample of ten countries to study a

total of twenty randomly sampled projects.Details of the sampling, methods and evalua-tion instruments are all given in Annex 3. Inall areas of enquiry the approach was to tryto collect primary and secondary data from a variety of sources (documents, interviews,databases) in order to triangulate thefindings.6 A large proportion of the staff ofIFAD (some 25 per cent) were interviewedindividually or in small groups, and over 350 staff out of an estimated complement of around 450 completed the HR on-line questionnaire.

1.9 Two pillars of the approach were the randomsample of countries and projects, and use ofordinal ratings to express evaluation judge-ments.7 There are strengths and weaknesses toboth approaches. The random sample drawnfrom all newly approved projects in the years1994 to 2003 gave rise to a sample of varyingages. Five projects were at a very early stage ofimplementation; five were around mid-term;four were at a late stage or near completion;and six had closed. The spread of ages affectsevaluation judgement in several ways. It isdifficult to assess likely impact and sustain-ability during the early years of implementa-tion. A characteristic of many IFAD projects isa major review with some redesign at mid-term, so early implementation progress can bevery different from later years.

1.10 Experience from other development organi-zations suggests that expectations of impactand sustainability are more positive in theearly years and decline over time. A sampleof closed projects would have given betterevidence for impact and sustainability. Butgiven IFAD’s relatively long implementationperiods, closed projects would have beendesigned around the start of the evaluationperiod and would not reflect later changes incorporate procedures and policies in the waythat new entrants do. On balance, the advan-tage of the latter is thought to outweigh thedisadvantage of the former. In the analysis,care is taken to distinguish findings aboutclosed and late-stage projects from earlyones. Supplementary evidence is alsopresented from other OE evaluations ofclosed projects.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

16

T H E E VA L U AT I O N A N D I T S C O N T E X T

17

1.11 Ratings require judgement and judgementsneed to be consistent if the results are to besound. Annex 3 paragraphs 4.21 to 4.26describe in detail the ways in which the IEEteam have used written guidance and peerreview to develop as robust and contestableset of findings as practical. The evaluation alsostayed as closely as possible to OE’s method-ology to enable comparisons to be made withthe ARRI.

Rural development in a changing world

1.12 The problem which IFAD was created totackle continues. An estimated 1.1 billionpeople live in extreme poverty on less thanone dollar a day (1993 purchasing powerparities). This is down from 1.24bn in 1990,

but still represents over one in five of theworld’s population.8 In its 2001 Rural PovertyReport, IFAD9 estimated that three-quartersof the acute poor live in rural areas. Althoughthis figure will decline over time with growingurbanization, the proportion living in ruralareas is expected to remain as high as 60 percent in 2025. Amongst the poorest the 2004-2005 Chronic Poverty Report10 estimates that

Table 1 Evaluation framework

Evaluation framework Related objectives from IFAD’s mandate and subsequent strategies

1 Development results and impact on rural poverty reduction

1.1 Policy influence through dialogue n strengthen policiesae

and advocacy n strengthen institutionsa

1.2 Project impact of loans and technical n introduce improved or expanded food production in assistance grants (TAGs) the poorest food deficit countriesa

n improve the nutritional level of the poorest populationsa

n strengthen capacity of the rural poor and their organizationsd

n improve equitable access to productive natural resources and technologyd

n increase access of the poor to financial assets and marketsd

2 Corporate performance of policy and operations

2.1 Resource allocation n funding allocated on the basis of objective economic and social criteria with regard to the needs of low income countries and fair geographic distributionb

n enhance catalytic impactc

2.2 Policy and strategy development n leadership in rural poverty eradicationc

and implementation

2.3 Partnerships n development of national partnerships, regional and international coalitionscef

2.4 Project cycle management n design and implementation of innovative, cost-effective and replicable programmesce

2.5 Knowledge management n to manage innovation, learning processes influencing and scaling upc

2.6 Human resource management n catalytic institution with highly motivated, well-trained and efficient staffc

Sources: a: IFAD Mandate Article 2; b: IFAD Mandate Article 7; c: IFAD Strategic Framework 1998-2000; d: IFAD Strategic Framework 2002-2006; e: Consultation on the Fifth Replenishment; f: Consultation on the Sixth Replenishment .

6 An explanation of triangulation is given in Annex 3, para. 4.2.

7 The use of random sampling was explicitly laid down by the IFAD Executive Board.

8 World Bank Group PovertyNet website.9 IFAD (2001) Rural Poverty Report 2001. The challenge

of ending rural poverty. OUP, Oxford.10 Chronic Poverty Research Centre (2004) The Chronic

Poverty Report 2004-2005.

between 300 and 420 million people arechronically poor, that is people in long termpoverty and who may be economically activebut unable to escape poverty because of theterms of their employment, their lack of accessto productive assets, or because of socialbarriers. The draft task force report from theUN Millennium Project on what is needed toachieve the MDG 1 states that they will not bemet without rapid progress in poverty reduc-tion in rural areas.11

1.13 While poverty persists in many countries andregions, the global context for agriculturaland rural development has seen unprece-dented change in recent decades.12 The greenrevolution productivity increases, which werekey to rural poverty reduction from the 1960’shave tailed-off. In Africa, not only has cerealproduction grown more slowly than popula-tion, but most of that growth has come fromincreased areas rather than productivity.13

For much of Africa, the potential benefits ofdemographic transition are now being eatenaway by the HIV/AIDS epidemic. For tropicalcommodity farmers, rapidly rising productionin the face of only steady increases in demandhave led to the lowest prices since recordsbegan.14 The character of rural space is alsobeing changed by the spread of global marketrelations, technical change and the graduallyshrinking contribution of the agriculturalsector to national GDP.

1.14 In the wider development assistance environ-ment, direct support to agricultural development has been replaced by the ruraldevelopment model, founded on the enablingrole of rural infrastructure and social provi-sion. More recently, the adoption of thePoverty Reduction Strategy (PRSP) approachunder the HIPC framework and the interna-tional agreement on the MDGs has created anunprecedented focus around the overarchinggoal of poverty reduction and the need formulti-sectoral efforts on the part of bothgovernments and the international donorcommunity to achieve the target of halvingpoverty by 2015. Related are strong pressuresfor donors to align and harmonize their waysof working in support of country-level povertyreduction strategies.

IFAD’s evolving mandate1.15 The Fund was set up in 1977 to promote

and fund projects to promote food securityand the livelihood and wellbeing of the ruralpoor, and in particular the poorest ruralgroups in the poorest, food deficit coun-tries.15 At the time it was a unique institu-tional concept, being the only internationalfinancial institution mandated to focus itsactivities exclusively on the rural sector, andthe only one to focus on providing fundingfor projects that were to be designed, imple-mented and supervised by others. Theassumption was that such a mandate wouldboth sharpen the focus of recipient govern-ments and the international community onthe wide-spread problems of rural areas, andcreate a simple, cost-efficient mechanism forchannelling additional funding into thedevelopment assistance system.

1.16 Relatively quickly it became apparent thatprojects designed by others did not alwaysgive sufficient focus to the needs of the ruralpoor, and in particular to the poorest ruralgroups. As a consequence the Fund movedinto project activity, taking a much moreactive role in the identification and design ofits projects. It also gradually evolved frombeing a pure funding institution to a devel-opment institution with a wider range ofinstruments – lending, grants and policydialogue – and a much broader corporateremit. This culminated in the mid-1990s inIFAD adopting a new goal ‘to lead globalefforts in helping the world’s poorest’ bybecoming a catalytic, knowledge-based organization focused on the design andimplementation of innovative, cost effectiveand replicable programmes with sustainableimpact (Box 1).

1.17 As IFAD’s mandate has evolved so has itsmission. From an initial focus on targetingresources to improve food production andnutrition amongst the poorest and mostmarginal populations in rural areas, IFADhas broadened its mission to include a rangeof actions for enabling the rural poorthrough improved access to financial assets,markets, productive resources and tech-nology and strengthening the capacity of the

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

18


19

Box 1 IFAD’s evolving strategic objectives

The basic mandate of the Fund is set out in the Agreement Establishing IFAD. Article 2 states that: the objectiveof the Fund shall be to mobilize additional resources to be made available on concessional terms for agriculturaldevelopment in developing Member States. In fulfilling this objective the Fund shall provide financing primarily forprojects and programmes specifically designed to introduce, expand or improve food production systems …Article 7 of the Agreement goes on to specify a focus on the poorest populations and the poorest food deficit countries.

In 1985, IFAD’s Annual Report confirmed its three main strategic objectives as higher food production, reducedmalnutrition and lower levels of rural poverty … In 1990 the Annual Report states that over the years IFAD’sinvestment priorities have shifted as the Fund has learned increasingly effective and specific strategies for ruralpoverty alleviation. A growing focus on the provision of credit funds has been the one notable trend … Otherpriorities are fisheries, small scale irrigation, marketing, storage, processing, entrepreneurial capacity and ruralmicro-enterprise.

A statement of IFAD’s Vision in 1995 further broadened the Fund’s objectives asserting that IFAD should be aleader in showing the way on poverty eradication by unleashing the capacities of poor rural people themselves;that IFAD should emphasize the design and implementation of innovative, cost-effective and replicableprogrammes and that IFAD should also be a catalytic and knowledge-based organization mobilizing knowledgeand resources to assist the cause of the rural poor.

The gradual broadening of IFAD’s mission is further evident in the strategic framework for 1998-2000 MeetingChallenges in a Changing World which describes IFAD’s core business as: innovative pilot projects andprogrammes in rural and agricultural development; projects and programmes focusing on poverty eradication,household food security and new markets for marginal areas, and the formation of effective partnerships withother development institutions and organizations.

The evolution of the Fund’s strategic objectives culminates in the most recent strategic framework 2002-2006Enabling the Rural Poor to Overcome their Poverty which identifies three strategic objectives:

n Strengthen the capacity of the rural poor and their organizationsn Improve equitable access to productive natural resources and technologyn Increase access of the poor to financial assets and markets

Further objectives to enhance catalytic impact are: harnessing knowledge and disseminating to a broad spectrumof national and international partners; supporting the development of national partnerships among the poor,governments, private sector and civil society; building regional and international coalitions and establishing institu-tional and policy frameworks in support of the poor.

rural poor and their organizations to engagewith institutions and markets (Box 1). Therehave been some constant themes over theyears including a focus on the role of women(now gender equity), targeting and theparticipation of beneficiaries, but there arealso new themes including an explicit focuson empowerment, on the role of marketsand non-farm activities, on decentralization,policy change and institutional development.The current strategic framework also high-lights IFAD’s potential role as an advocacyorganization for the rural poor.

1.18 The mandate continues to be relevant butIFAD’s niche has become less clear.Persistent poverty and hunger in rural areas

11 UN Millennium Project Interim Report of Task Force 1 onPoverty and Economic Development. February 2004.www.unmillenniumproject.org/documents.

12 Ashley, C. and Maxwell, S. (2001) Rethinking rural devel-opment Development Policy Review, Volume 19.

13 See, inter alia, Dorward et al (2004) A Policy Agenda forpro-poor agricultural growth, World Development vol. 32issue 1, January 2004.

14 See, inter alia, Peter Robbins (2003) Stolen Fruit: TheTropical Commodities Disaster, Zed Books.

15 …the Fund shall provide financing primarily for projectsand programmes specifically designed to introduce,expand and improve food production systems….takinginto consideration: the need to increase food productionin the poorest food-deficit countries…and the importanceof improving the nutritional level of the poorest popula-tions in developing countries and the conditions of theirlives. Article 2 of the Agreement Establishing theInternational Fund for Agricultural Development.

and the global effort to reach the MDGsensure that IFAD's mandate to focus on therural poor is as relevant today as it was in1976. But IFAD’s original focus onsupporting small-scale domestic food produc-tion and nutrition has been superseded asIFAD has followed other government anddevelopment agencies and shifted its supportto agriculture to the broader challenges ofrural development. A review of a number ofbilateral and multilateral donor strategies foragricultural and rural development reveals agrowing level of convergence around issuesthat figure strongly in IFAD’s 2002-2006strategic framework, including support forrural institutions, market linkages, equitableaccess to production factors, micro-finance,gender equity and participation.16 It is diffi-cult to determine what role IFAD has itselfplayed in this growing convergence. TheRural Poverty Report 2001 was clearly animportant report, which in turn influencedIFAD’s strategic framework, but few otheragencies refer directly to it.17 IFAD staffacknowledge that for much of the 1990s, asagricultural ODA was declining, they found itdifficult to convince other agencies of theneed to stay engaged in the agriculturalsector.18 The re-emergence of a globalcommitment to poverty reduction at the endof the 1990s, the HIPC initiative and theintroduction of PRSPs were crucial inbringing about a renewed commitment bythe international community to povertyreduction and to rural development. But asthe number of actors and the ways of doingbusiness in the rural sector have changed,19

so has IFAD’s distinctiveness.

1.19 IFAD has attempted to retain a distinctivefocus largely by emphasizing its catalytic andinnovative role through its loan projects andby articulating a stronger role for the Fundin partnerships and policy dialogue. Butcritical challenges remain. The currentstrategic framework provides IFAD with aclear sense of mission and purpose, but asnoted in the DRR, it does not provide ameaningful guide to its operating niche orvalue-added. With relatively few changes in the operating model and the range ofinstruments at its disposal the Fund’s ability

to work effectively in changing countrycontexts is in danger of being compromised.In moving towards a broader focus onenabling the rural poor, IFAD’s specific focuson the poorest areas and most vulnerablegroups is in danger of being eclipsed (Box 2).

1.20 A key question for IFAD now is whether itshould be looking to extend its mandatefurther to embrace a multi-sectoral approachto rural development and engage in newpolicy areas – including new aid modalities -related to rural poverty reduction, orwhether it should focus on being an innova-tive, pro-risk institution that works specifi-cally to support the agricultural productivepotential of the poor, often in difficult andmarginal areas. Many more agencies are nowdoing the former20 but the latter is notwithout its own dilemmas – not least how tobalance support to the poor in marginalareas with support in higher potential areasand how to balance the risk-reward of inno-vation amongst target groups that havelimited resources and limited capacity toabsorb further risks. In principle, this is anarea where IFAD should be able to exercisespecific expertise, but it requires a muchclearer operational approach to innovation,to assessing risk and reward and a clearerframework for identifying poverty target groups.

1.21 The 1994 rapid external assessment of IFADnoted that the big problem for the Fund is that theworld is a tougher place in which to be fresh andinnovative.21 It also noted that the realdanger for the Fund is being outpaced andovertaken by other organizations, includingsome of the larger NGOs, doing the samethings. The situation has not changed muchin 2004. The challenge for IFAD is todemonstrate its distinctive role based on aclearly defined approach to innovation,targeting, partnership and policy dialoguewith a clear focus on delivering results forthe rural poor, and particularly the poorest,as part of the concerted global effort to reachthe MDGs.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

20


21

16 Organizational priorities for agricultural and/or rural devel-opment were reviewed for Denmark, Finland, Germany, the Netherlands, the United Kingdom, Sweden, the UnitedStates Agency for International Development, the WorldBank, the Food and Agriculture Organization of the UnitedNations (FAO), AfDB, Asian Development Bank (AsDB) andthe European Commission (EC).

17 A number of agencies refer to the rural poverty projectionsprovided by the Rural Poverty Report but not to its policy or operational implications. There are, nevertheless, manycommon themes now emerging which were also reflectedin the Rural Poverty Report. See also the section in Chapter 2 on IFAD’s policy influence.

18 Staff interviews in PMD and the Technical Advisory Division (PT).

19 The volume of ODA going to rural areas is still well belowwhat is considered necessary to meet the MDGs, but thenumber of national and international actors now engagedin the sector has increased hence the need for more effec-tive partnerships and increased attention to policy coher-ence. Ashley and Maxwell (2001) also note the growingheterogeneity of rural areas themselves, particularlybetween high and low potential, well connected andweakly connected, peri-urban and remote areas, as well asthe impact of demographic transitions and the spread ofmarket relations on the character of the rural space.

20 See footnote 13.21 Report of the Rapid External Assessment of IFAD (1994)

page 11.

Box 2 Who are IFAD’s target groups?

An emphasis on targeting and particularly the poorest areas and groups in the rural space, has been a hallmark of

IFAD’s identity from the very beginning. Article 7 of the Agreement Establishing IFAD states that in allocating its

resources the Fund shall be guided by the need to increase food production and to improve the nutritional level

of the poorest populations in the poorest food deficit countries and in other developing countries. In 1980 IFAD’s

Annual Report notes the challenges of tacking rural poverty and states that clearly a great deal more needs to be

done to explore new approaches for reaching the poorest and hitherto largely by-passed population groups with

effective assistance. While the ‘poorest’ in this context are not defined, the perception that IFAD was established

to tackle poverty amongst groups left out of mainstream development is confirmed in an article in West Africa in

1988 which noted that IFAD has been something of a model development agency. It was launched

to help the poorest of the poor, those who the bigger agencies don’t or can’t reach (cited in the IFAD Annual

Report 1987).

More recently, however, the breadth of IFAD’s target group has expanded and turned into a much more generic

focus on ‘enabling the rural poor’. The 2003 Regional Portfolio Review for Eastern and Southern Africa is illustra-

tive, referring to various target groups ranging from ‘the needy’, the ‘poorest’ to the ‘active poor’ and also the

‘non-poor’ (mainly as indirect beneficiaries). The strategic framework 2002-2006 emphasizes the obstacles of day-

to day vulnerability faced by the rural poor, but the strategy does not highlight the particular condition of the

poorest or chronically poor, nor does it provide a distinctive strategy for targeting the poor in marginal or lagging

areas, in weak performing states or in conflict-affected zones. While the IEE Desk Review found that overall IFAD

was strong in geographical targeting - identifying largely poor areas - it was less clear that the groups identified

to participate in IFAD financed projects specifically included the poorest - the chronically poor or ultra poor. While

acknowledging that working with the poor, often in remote or difficult locations, is one of the hardest challenges

faced by an IFI, the view of the IEE is that as IFAD’s mission has expanded it has lost sight of some of its distinctive

focus on the poorest and most vulnerable groups. This is at a time when other IFIs and agencies are increasingly

focusing on broad-based actions to improve opportunities and services in rural areas (market access, rural roads,

extension and rural credit) potentially at the expense of the needs of the chronically poor and marginal or lagging

areas (Chronic Poverty Report 2004, MDG1 Interim Report 2004).

Table 2 Financial comparison between IFAD and other IFIs

IFAD World Bank AfDB AsDB IADB

Authorized capital (USD million) 190 811.0 29 732.7 51 997.0 100 951.0

Borrowings (USD million) N/A (19 000.0) (6 277.3) (4 141.0) (9 309.0)

Outstanding debt (USD million) N/A (103 017.0) (6 056.7) (26 259.0) (50 821.0)

Net income (USD million)22 68.6 5 344.0 256.7 436.0 2 433.0

Disbursements (USD million) 258.8 18 940.0 1 424.9 3 816.0 8 902.0

Cofinanced operations (USD million) 124.9 3 000.0 2 948.0 2 418.0 1 320.0

Loans approved (USD million) 424.0 18 500.0 1 614.2 6 104.8 6 810.0

Grants approved (USD million) 20.3 1 232.0 63.3 176.5 63.6

Other funding approved (USD million) 27.6 - 1 094.5 35.7 -

Administrative expenditure (USD million) 45.2 1 038.0 220.0 252.6 372.5

Number of projects approved 25 240 - 66 -

Resource allocation trends2.1 Compared with other IFIs, IFAD is a small

organization with an annual lendingprogramme in 2003 a quarter of the size ofthe AfDB and around two per cent of theWorld Bank’s (IBRD and IDA) lendingprogramme. But IFAD is a sizeable player inthe agriculture and rural sector in a numberof countries and its focus is to leverage impact

well above its financial size, partly by cofi-nancing and partly by facilitating the scalingup of investments.

2.2 IFAD’s project portfolio is broadly pro-poor.IFAD’s Lending Policies and Criteria make itclear that priority attention should be givento the poorest countries and the poorestpopulations, both with respect to the volume

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

22

The project portfolio: performance and impact

Figures are based on Annual Reports for 2003, except for AfDB which is based on the Annual Report 2002. World Bank includes IBRD andIDA. AfDB includes AfDB, African Development Fund (AfDF) and Nigeria Trust Fund.

Chapter 2

of lending and the terms of concessionallending.23 Comparing the distribution ofIFAD disbursements24 for 2000-2002 withthe distribution of the world’s absolute poorliving below USD 1/day confirms that IFADlending is generally pro-poor (Figure 1). The aid concentration curve shows that mostof IFAD resources were disbursed in thepoorest countries (above the diagonal line inFig. 1), with a smaller percentage - largely onordinary terms - allocated to less-poor andmiddle-income countries (below the diagonalline in Figure 1). The aid concentrationcurve also shows that proportionately moreIFAD assistance is concentrated in thepoorest but relatively small countries.Roughly 50 per cent of IFAD assistance in2000-2002 went to low-income countriesmaking up about 20 per cent of the world’spoor, most of them in sub-Saharan Africa.The larger countries, some of them better-offon average, with relatively large poor popula-tions, such as China, India and Indonesiareceived proportionately less of the Fund’sassistance, while some of the small middle-income countries (at the top end of the distribution) with smaller poor populationsreceived proportionately more.

2.3 When set against other agencies, the distribu-tion of IFAD’s project assistance is broadlysimilar with one or two key differences. Theshape of IFAD’s concentration curve reflectsits engagement with middle-income countriesas well as low-income countries. IDA andAfDF, on the other hand, only lend to lowincome countries, hence their aid concentra-tion curves remain well above the 45º line.The portfolios for the other agencies alsocover all sectors and not just lending to theagricultural and rural sector, which mayexplain the higher proportion of funds goingfrom these agencies to the larger poor coun-tries – China, India, Nigeria – compared toIFAD. Finally, IFAD’s assistance is significantlymore pro-poor than the UN overall reflectingthe very broad mandate of the UN.

2.4 IFAD lending per poor rural person ishigher in less-poor countries. While theoverall distribution of IFAD’s project port-folio is pro-poor, on a per capita basis it is

23

22 IFAD figure drawn from Annual Report 2002.23 IFAD Lending Policies and Criteria state that the poorest

countries, whose food problems require priorityattention, will be the main focus of the Fund’s operations, both with respect to volume of lendingand terms of concessional lending. The criteria fordefining food priority countries include (a) low per capitaincome of under USD 500 a year in 1975 prices (b) aprojected cereal deficit by 1985 of 500 000 tons or moreand/or a cereal deficit of 20 per cent or more as aproportion of estimated cereal consumption (c) thedegree of protein-calorie malnutrition in terms of theproportion of the population malnourished. In additioncountries with a per capita income of USD 500 orless (in 1975 prices) should be covered…and that theabsolute poorest among them should furthermorereceive special attention No specific definition of theabsolute poorest is given.

24 ODA data drawn from DAC development statisticsdatabase. The methodology draws on recent work byBaulch (2003, 2004) and involves the estimation of aidconcentration curves (ACC). Grateful thanks to MakibaYamano and Bob Baulch of the Institute of DevelopmentStudies for help in creating these ACCs. Aid concentra-tion curves provide a useful graphical device for showingwhether the distribution of a donor’s development assistance is targeted toward or away from the poorestcountries. If most of a donor’s aid goes to the poorestcountries then the aid concentration curve for thatagency will lie above the diagonal 45° line. If it is directedmostly to the less poor and middle-income countries thenthe curve will lie below the diagonal line. The x-axismeasures the cumulative share of national populationsliving under USD 1/day calibrated by the average grossnational income (GNI) for each country. The poorestcountries are captured at the beginning of the distribution. The y-axis measures the cumulative share of aid provided by each agency to recipient countries.

Cu

mu

lati

veSh

are

of

Aid

per

Ag

ency

(%)

Cumulative share of population (low and middle income countries)living under USD 1 per day (%)

0 10 20 30 40 10090807060500

10

20

30

40

50

60

70

80

90

100

AfDF

WB (IDA)

IFAD

UN, all

Figure 1 The Distribution of IFAD ODA comparedwith IDA, AfDF and the United Nations(2000-2002)

the rural poor in better off countries thatreceive proportionately more of IFAD’sresources. IFAD lending commitment perrural poor person ranged from about tencents to about USD 20 per annum between1994 and 2003. Higher levels are foundmainly in better off countries, such as Chile,Panama, Paraguay and Tunisia, while lowerlevels are found in the poorer and morepopulous countries such as Bangladesh,India and Viet Nam. The reason for higherlevels of lending per poor person in less poorcountries is not immediately clear but mayinclude smaller but persistent levels of acutepoverty amongst difficult to reach groups,and therefore higher delivery costs, togetherwith greater capacity to absorb funds in lesspoor countries. The aggregate data do notreveal who the target group is for IFADfinancing in these countries25 and whetherdelivery costs are higher compared withpoorer countries, but given the findings ofthe latest Millennium Project report onMDG achievement,26 there are reasons to be concerned about the mismatch betweenper capita lending levels in less poor coun-tries and those in the poorest and morepopulous countries.27

2.5 Regional shares remain a fundamental partof IFAD’s resource allocation framework.28

The regional distribution of IFAD lendingduring the IEE period is largely a product ofregional allocation shares established in 1997to ensure a “fair geographic distribution ofresources”. The principle is that no onecountry or region should receive a dispro-portionate share of IFAD resources in anyperiod.29 Since the mid 1990s, there hasbeen, nevertheless, a concerted effort toensure a rising share of resources for sub-Saharan Africa. Figure 2 presents dataaveraged across a three/four year intervaland confirms a relatively broad geographicalspread across the whole IEE period, with arising volume of resources going to sub-Saharan Africa accompanied by a markeddecline in lending volumes to Asia during2001-2003. With Africa the region least likelyto achieve the MDGs by 2015, focusingresources here is clearly appropriate,although lending levels per capita are still

clearly below those of some of the smaller,better off countries of Latin America andNorth Africa.

2.6 IFAD has leveraged significant levels of cofinancing but volumes vary widely acrossregions and years and actual disbursementsare known to lag behind commitments. Ofthe USD 9 billion in total project financingover the IEE period, the Fund directlycontributed USD 3.9 billion. Cofinancingcommitments by IFIs and donors amountedto USD 2.4 billion, and contributions fromgovernments and beneficiaries a further USD 2.7 billion. This means that for everyUSD 1 approved directly by IFAD, it raisedcommitments of roughly USD 1.3 from othersources – a strong indication of the Fund’scontinuing role as a catalyst for resourcemobilization.30 While the total volume of cofinancing is impressive, in reality it varieswidely from year to year (Figure 3) as well asacross regions. The share of cofinancing intotal project financing has also fallen overtime, from an average of 30 per cent for theperiod up to 1994 to an average of 25 percent since 1994.

2.7 Cofinanced arrangements involve a widerange of partners: bilateral donors, multilat-eral banks, funds, foundations and NGOs –around seventy at the last count, with manycontributing very small amounts towardssingle components or subcomponents.31

This inevitably involves significant transac-tions costs for IFAD; costs that have beenrising as the number of partners expandsand as donors themselves become moredecentralized in their resourcing decisions.32

The unpredictability of some cofinancedflows, particularly grant flows for technicalassistance, and the problem of raising timelycounterpart funds is also a major problemaffecting project implementation.33

2.8 A detailed study of the costs of cofinancingfor borrowing governments was not possibleas part of the IEE, but some clear challengesdo emerge for IFAD given the current aidclimate. First, how to adapt the traditionalproject cofinancing model to the moregeneral move by donors into pooled sector

24

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

T H E P R O J E C T P O R T F O L I O : P E R F O R M A N C E A N D I M PA C T

programmes and general budget support.There are some initial signs that IFAD isconsidering alternative ways to engage moredirectly with sector programmes but how this will impact on the current approach to project cofinancing is still not clear.35

Second, how to ensure that IFAD’scofinancing approach conforms with goodpractice harmonization and alignment princi-ples.36 In principle, cofinancing should helpto reduce costs imposed on borrowers as longas participating donors are willing to stream-line their procedures and rely more directlyon government reporting and monitoringmechanisms. Little evidence is available so far on how IFAD has progressed in thisdirection. Demonstrating progress will be acritical challenge for IFAD in advance of theSecond High-Level Forum on Harmonizationand Alignment for DevelopmentEffectiveness in early 2005.

2.9 Decisions about lending volumes at countrylevel have, until recently, been based on alimited assessment of regional and countryneeds and an ad hoc assessment of borrowerimplementation performance. This hasworked, up to a point, but the approach isincreasingly incongruent with best practiceamongst IFIs.37 A PBAS proposed during theIFAD VI round and currently being finalized

160

140

120

100

80

60

40

20

0

USD

mill

ion

s

Africa I Africa II SSA ASIA LAC NAfrica/Europe

Region

1994-1996

1997-2000

2001-2003

25

Source: PPMS

Figure 2 Average IFAD lending by region and period(by approval year)34

Co-Finance Domestic

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003

%o

fTo

talP

roje

ctFi

nan

ce

40

35

30

25

20

15

10

5

0

Source: PPMS

Figure 3 Cofinancing and domestic contributions as ashare of total project finance (by approval year)

25 Although the Regional Strategy for Latin America doesindicate a special focus on indigenous populations and hardto reach groups and areas.

26 United Nations Development Programme (2005) Investing inDevelopment – A Practical Plan to Achieve the MillenniumDevelopment Goals. Millennium Project, Report to the UNSecretary General. New York.

27 A similar finding was found in an analysis of the World Bank’srural portfolio in 2001. Here the range of Bank rural lendingcommitment per rural poor person was between USD 1 andUSD 20 per annum between 1994 and 2000. Higher levels oflending per rural poor person were found mainly in richercountries and lower levels in poorer countries. A study byBeynon (2003) found that the share of IFAD concessionalassistance going to the poorest countries remained high ataround 83 per cent, although the ratio of ODA per headgoing to high versus low poverty countries fell slightly from1.32 to 1.22 – meaning that high poverty countries receivedroughly 1.2 times as much per head as low poverty countriesin 1999/00. This was still well above the average for multilat-erals as a whole (0.74 in 1999/00) but below the equivalentratio for IDA, the AfDF and United Nations Children’s Fund.

28 Even surviving the early rounds of the new PBAS. 29 Regional shares are estimated using a rolling three-year

average.

30 Actual funding tends to fall below commitments, but comparative data are not available for analysis.

31 The count is based on data in the Project PortfolioManagement System (PPMS) and covers all sources of finance for IFAD projects during the IEE period.

32 During staff forums, several CPMs noted the increased timespent mobilizing co-financing with the shift, particularlyamongst bilateral donors, to more decentralized ways ofworking at country level.

33 See recent regional portfolio reviews and the Annual Reviewof Portfolio Performance.

34 IFAD’s two Africa regions are designated Africa I and II. They are broadly west and central, and east and southern,respectively. A list of countries can be found in any IFADannual report.

35 Recent sector-wide approaches paper discussed at theSeptember 2004 EB, Eastern and Southern Africa RegionalStrategy 2001.

36 Rome Declaration on Harmonization 2003; OECD/DAC GoodPractice Guidelines; Marrakesh Action Plan 2004 etc.

37 Nevertheless, IFAD is the first within the UN community toadopt a PBAS.

by IFAD management38 is intended to intro-duce a more transparent ex ante frameworkfor guiding country level allocations based ona combined assessment of need, country andsector institutional and policy performanceand portfolio performance. IFAD’s PBAS isconsistent with approaches adopted by otherIFIs, drawing as it does on the CountryPolicy and Institutional Assessment (CPIA)developed by IDA but, unlike others it alsoincludes a separate assessment of the qualityof the rural policy and institutional environ-ment in each country.

2.10 PBAS is still work in progress and is notexpected to come fully on stream until2005/2006. Some of the potential benefitsinclude a clearer institutional focus onperformance at project, sector and countrylevels and a stronger link between IFAD opera-tions and country strategies, in particular rein-forcing the link between IFAD operations andrural sector policy dialogue, an area alreadyidentified in the desk review as criticallylacking. But PBAS is fundamentally aboutdeciding how much and what type of resourcesdifferent countries should get based on,amongst other things, their policy and institu-tional performance. A key question that arisesis whether IFAD has the necessary range ofinstruments to adjust its level of engagementeffectively in response to the PBAS exercise.Experience from other IFIs suggests that amore selective resource allocation approachunder PBAS tends to mean higher levels ofnon-lending and grants relative to lending indifficult policy contexts.39 The current set ofIFAD loans, technical assistance grants, andmore recently country grants, offers some flexi-bility, but is not based on a systematic approachto working in different policy environments.There is, for instance, no policy paper settingout how IFAD instruments are best deployed indifferent country contexts or settings; nor isthere a clear normative framework guidingmedium term engagement in poor performingsectors or countries, or with policy frameworkssuch as PRSPs, although a draft policy paperfor engagement in post-conflict countries isnow available. Given the relatively small size ofIFAD’s lending programme, a further concernrelates to the equitable treatment of countries

that rank low on the CPIA or rural policyperformance scale. The Review of IDA 10-12(OED 2002) finds that even in a fairly wellestablished performance based allocationsystem such as that operated by IDA the consis-tent and transparent application of perform-ance criteria to ensure the full equitable treat-ment of countries is a continuing challenge.40

This clearly has implications for how IFAD andother agencies work with poor populations indifficult policy environments where the risk ofconflict or insecurity is likely to be higher andthe need for long term support greater thanelsewhere. Many of these environments arethose highlighted in the recent UN MillenniumProject Report 2004 as needing particularassistance and support if countries, particularlythose in Africa, are to get close to achieving theMDGs. How PBAS will influence the prospectfor IFAD’s work in these areas is yet to be seen.

Portfolio trends and performance

2.11 Aggregate ODA flows have returned to 1994levels but total flows to agriculture continue todecline. Following a period of severe declineduring the mid 1990s, total flows of ODA todeveloping countries have begun to rise onceagain. Flows to agriculture, and particularlymultilateral flows have, however, continued todecline for most of the decade, from a peak ofUSD 4.4 billion in 1996 to around USD 2.9billion in 2002. A major contributing factorbehind the decline in aggregate flows to agri-culture is the changing pattern of multilaterallending, with IDA in particular moving awayfrom direct lending to agriculture, partlybecause of a poor track record in agriculturallending and partly because of a policy shift infavour of a more enabling approach to ruralgrowth and poverty reduction.

2.12 IFAD lending has largely mirrored this trend.While maintaining its contribution to total agri-culture ODA41 at around 5-7 per cent for muchof the decade (around 13 per cent of multilat-eral flows), from 1999 IFAD flows to agricul-ture began to decline, falling to around 2.4 percent of total flows in 2002 (or 8 per cent ofmultilateral flows) (Figure 4). Much of thedecline is explained by a reorientation of IFADODA away from agriculture to the broader

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

26


category of ‘multisector’ which includes,amongst other things, activities related to ruraldevelopment (Figure 5).42 While the data ishighly aggregated one implication is that,despite its specific mandate, IFAD was not ableto counteract the downward trend in aid flowsto agriculture in the 1990s, either directly byincreasing the share of its own resources goingto agriculture (although the impact would stillhave been modest given the scale of the reduc-tion from IDA) or indirectly, by persuadingother, larger donors to maintain or increasetheir resources going to agriculture. RatherIFAD has followed the trend of other agenciesinto rural development.

2.13 Data drawn from IFAD’s own project portfoliomanagement system (PPMS) shows the shift inIFAD’s portfolio, but less clearly. (Table 3.)The share of IFAD lending going to agricul-ture43 varies quite widely over the IEE period,without an obvious trend. When aggregatedwith the share going to fishing, livestock andresearch, which more closely matches theOECD/DAC definition, there appears to be a clearer downward trend.44 The share oflending in rural development also showsevidence of increasing over time (again withsome year on year variance), in line with theOECD figures, as does the share of supportgoing to marketing and the new FlexibleLending Mechanism (FLM).45

2.14 Classification by project type is not a satisfac-tory way of capturing changes in the portfolio,however. The category of rural development is too imprecise, given that it more or lessdefines what IFAD is about rather than a

27

38 Criteria and Principles for the Development and Operation of a Performance-Based Allocation System in IFAD(REPL.VI/5/R.3); Progress Report on Implementation of thePerformance-Based Allocation System (EB 2004/82/R.30).

39 Asian Development Bank.40 The performance based allocation system of the World Bank

is under scrutiny as the CPIA on which it is based measuresinitial conditions and structural factors more than it doesperformance. Only indicators that are relevant to IFAD’smission should be selected in order to guide the choice ofinstruments and the direction of lending in individual countries.

41 Includes forestry and fishing.42 These data are based on OECD/DAC statistics and classifica-

tions. The definition of agriculture, fishing and forestry

covers agricultural sector policy, agricultural development andinputs, crops and livestock production, agricultural credit,cooperatives and research, all aspects of forestry policy anddevelopment and fishing. This is a more inclusive definitionthan used by IFAD internally which separates out agriculturefrom livestock, fishing, marketing and research.

43 Agriculture according to PPMS includes: food crops, forestry,fruit trees, horticulture, industrial cash crops, input supply,institutional support to agricultural bodies, land improvement,land reform, mechanization, pest control, seeds, soil andwater conservation.

44 The number of observations may not be sufficient to signify a trend, but definitely point in the direction of a change.

45 The number of FLM loans is now capped until an evaluationis carried out in the near future.

1994 1995 1996 1997 1998 1999 2000 2001 2002U

SDm

illio

n

1600

1400

1200

1000

800

600

400

200

0

AFDF

AsDF

IDA

IFAD

Source: OECD/DAC International Development Statistics

Figure 4 IFAD and IFI ODA flows to agriculture 1994-2002

Economic infrastructure Production Sectors(Agriculture)

Multisector Commodity Aid/GeneralProg. Ass

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003

300

250

200

150

100

50

0

USD

mill

ion

Source: OECD/DAC International Development Statistics

Figure 5 IFAD ODA flows by selected sector 1994-2002

Table 4 Portfolio performance – comparing major IFIs

Institution Portfolio performance measure47 % of Total portfolio Year

AfDF Projects ‘at risk’ (number) 31 2003

World Bank//IDA a Projects ‘at risk’ (number) c 15 2002

IADB b Alert Status (number) 29 2003

AsDB Projects ‘at risk’ (number) 15 2003

IFAD Underperforming projects48 (number) 21 2003

Table 3 Portfolio composition by project type (per cent of IFAD finance) 1994-2002

Project Type 1994 1997 2000 2001 2002

Agriculture 38.1 50.2 36.6 14.7 37.4

Fishing 2.5

Credit 2.3 10.2 4.6 6.4 6.8

Irrigation 24.1 2.8 4.2 7.6

Livestock 3.3 4.5

Marketing 1.8 4.1 4.4

Rural 11.8 25.7 18.0 45.3 26.7

Research 20.5 4.7 10.4 3.0

Settlement 5.0

Programme Loan

FLM 26.2 19.5 16.7

specific area of activity or project objective.The vast majority of the projects financed byIFAD also have multiple components, most ofwhich cut across a number of project types.This points to the need for a radicallyimproved system for classifying and moni-toring loan projects as part of ongoing portfolio review. Examination of projectsubcomponents provides a slightly moredetailed story, although still bedevilled by defi-nitional problems.46 Figures 6 and 7 clearlyshow the increasing focus on institutionalsupport, local capacity building, rural financialservices and rural infrastructure. An increasein activity in these areas is to some extent atthe expense of areas such as input supply,fishing, irrigation and rural enterprise,although Figure 7 also suggests that as well as focusing more on institutional change andcapacity building IFAD continues to finance

many of its more traditional areas includinganimal production, food crops and credit. Theinherent danger, particularly for a small insti-tution is that as the agenda expands so theportfolio becomes spread more thinly. Someearly indications can be found in a small butsignificant increase in the average number ofsubcomponents per project. The evidence istentative but raises concerns that an increas-ingly ambitious agenda may be resulting in anincreasingly complex and diffuse portfolio.

2.15 IFAD’s portfolio performance is similar tocomparator agencies, although there is noroom for complacency. Comparing IFAD’sportfolio performance measure with that ofthe other major IFIs shows that the number ofongoing projects underperforming in IFAD isslightly higher than the number of projects atrisk in the World Bank and the AsDB, but

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

28


Source: PPMS

Source: Respective annual reports on portfolio or loan/project performance (ARPP) for each institution.a World Bank figures relate to fiscal year.b An equivalent figure based on the number of problem projects could not be found in 2003.c A refinement of the ‘flags’ used to determine ‘at risk projects’ was made by the World Bank in Financial Year (FY02). If these changes

had come into effect before the ARPP was published it would show the percentage of ‘at risk projects’ as closer to 19 per cent forFinancial Year (FY02).

well below the equivalent measures for theAfDB and IADB (Table 4). This comparisonnevertheless includes all types of lendingactivity across all sectors, whereas the ruraland agricultural sectors are known to bemore risky and difficult to work in. A moreappropriate comparison is thereforebetween IFAD and the portfolio of otherIFIs in the rural sector. Such a comparisonis currently only possible with the WorldBank (Table 4). Taking 2002, which is theonly directly comparable year, the dataconfirm that IFAD is underperformingrelative to the World Bank. More important,however, are the trends. First, there isevidence of a modest improvement inIFAD’s portfolio performance since 2000,with an indication that things improvedfurther in 2003. Second, the share ofprojects at risk in the Bank’s rural and agricultural portfolios declined noticeablybetween 1997 and 2002, pointing to theeffects of a vigorous programme of portfolioimprovement across the institution.49

2.16 The need for IFAD to continue to addressportfolio performance is evident from thedata. Reasons for continued underperfor-mance are considered in more detail in the rest of this chapter but some initialhypotheses based on the DRR include theover-designed, under-designed nature ofIFAD financed projects and problems withproject supervision, particularly the ‘arms-length model’ of supervision which contrastsstrongly with the more hands-on approachof the World Bank and most other IFIs.50

29

46 Subcomponent descriptions are often imprecise and appearseveral times per project. The data were cleaned for thepurpose of this analysis.

47 IFIs, except for IFAD, use a standardized methodology forassessing portfolio performance. Problem projects refer toprojects rated less than satisfactory in implementation and/orin the likelihood of achieving development objectives.Projects ‘at risk’ relate to actual problem projects andpotential problem projects (i.e. those with 3 or more ‘at risk’flags ). IFAD uses underperforming projects which refers toprojects that are substantially below target or showing littleor no progress towards development objectives. This isclosest to the concept of problem projects used by other IFIs.Currently there is no equivalent measure of ‘at risk’ for theIFAD portfolio.

48 This is a self-evaluation estimate with no independent verification.

49 IADB produces a figure for problem projects and projects onalert status in agriculture. The figures for 2002 show 7.9 percent of agricultural projects in problem project status, whilethe figures for 2003 reveal 12.5 per cent of agriculturalprojects in problem status and 32 per cent in alert status.

50 The over-designed, under-designed characteristic was identi-fied in the DRR (page 35). It describes the situation whereappraisal documents, working papers and TRC discussionsaddress many issues in considerable depth for every project.However it is not clear that the core group of design issues –those on which project performance and impact ultimatelydepend – have been sharply identified and are given mostattention by formulators, PDTs, TRCs and appraisers.

Rural enterprise

Marketing

Rural Infrastructure

Roads/tracks

Rural financial services

Local capacity building

Institutional support

Credit

Tech Transfer

Irrigation

Input Supply

Food production

Forestry

Fishing

Animal production

200319981994

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

20

16

12

8

4

0

2

6

10

14

18

Nu

mb

ero

fp

roje

cts

wit

hsu

b-c

om

po

nen

t

Animal

product

ion

Fishin

g

Fore

stry

Food

product

ion

Input Su

pply

Irrig

atio

n

Tech

Tran

sfer

Credit

Insti

tutio

nalsu

pport

Loca

l cap

acity

buildin

g

Rural f

inan

cial s

ervic

es

Roads/t

rack

s

Rural I

nfrastr

ucture

Mar

ketin

g

Rural e

nterp

rise

1994

2003

Portfolio composition – number of projects with each subcomponent 1994 and 2003

Source: PPMS

Figure 7

Portfolio composition by subcomponentvalue – share of subcomponents in IFADfinance 1994, 1998, 2003

Figure 6

Source: PPMS

Table 5 Comparing sectoral portfolio performance

Institution Portfolio Year Projects at risk orunderperforming projects (%)

IFAD Total 2000 25.7

2001 24.9

2002 24.9

World Bank Rural developmentb 1997 a 24

2002 18

Agriculturec 1997 27

2002 16

2.17 Another measure of portfolio performance isthe time taken for projects to get from Boardapproval to loan effectiveness. Long elapsedtimes between approval and effectiveness canindicate that significant pre-implementationactivities are required in order for a projectto go ahead, alternatively it can indicate thatprojects being presented to the Board arebelow full ‘readiness’ - often meaning thatcritical conditions have not been met by theborrower, or that cofinancing or necessaryinstitutional arrangements have not beenforthcoming. Table 6 estimates elapsed timesover the IEE period, while Table 7 comparesaverage elapsed times for 1994-2003 acrossthe IFAD regions. Table 8 provides somerecent comparison with other IFIs.

2.18 Over the IEE period elapsed times appear tohave increased,51 while variations acrossregions are significant. Regional PortfolioReviews suggest that longer elapsed times arelargely due to difficulties encountered byborrowers in meeting conditions of loan effec-tiveness, a particularly significant problem inLatin America. In some cases this is becauseIFAD has wanted to ensure that a particulartarget group is fully reflected in project imple-mentation arrangements, in other cases it isdue to the slow passage of critical legislation orbecause of wider institutional or political insta-bilities. More flexible lending approaches mayalso increase the pre-implementation phase as

triggers and benchmarks are worked out forsuccessive lending tranches. What is not clear iswhether the increase in elapsed times is sugges-tive of a more systemic problem in the way thepre-implementation phase is handled withinIFAD, pointing to the need for actions to betaken earlier on at the project design stage toensure that borrower commitment is securedand that appropriate institutional arrange-ments are in place to take forward an effectiveimplementation phase. Compared with otherIFIs, elapsed times in IFAD are at the high endof the distribution which points to the need fora clearer justification from IFAD managementon whether this is matched by more effectiveprocesses and stronger results.

Project and programme performance

2.19 IFAD’s mandate is fundamentally about theidentification and design of sustainableprojects and programmes that enhance thewellbeing of the rural poor. It has alreadybeen described how, over time, IFAD hasbroadened its mandate to take on anincreasing role in project activity and supportfor project implementation. In this sectionthese initiatives are discussed and assessedusing the results of twenty project evaluationsfrom the country visit phase (a fuller treat-ment is provided in Annex 4). The focus inthis section is on the performance of the indi-vidual projects and the country programmes

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

30


Source: IFAD Review of Portfolio Performance 2003; World Bank ARPP FY02. Due to differences in reporting a directly comparable timeseries is not available.a World Bank years are all fiscal years.b Rural development is a thematic classification in the World Bank relating to all investment and policy-based operations with a rural

development focus or content. c Agriculture is a sectoral classification relating specifically to investment and policy based operations supporting agricultural activities.

Table 8 Portfolio performance – IFAD and the IFIs

Institution Elapsed time between Executive Board approval and loan effectiveness

AfDB (2003) 8.4 months

ADF 2003 +/-24 months

Word Bank FY02 8.2 monthsa

IADB (2003) 12.0 months (median)

14.0 months (mean)

IFAD (2003) 17.0 months

31

Table 6 Average elapsed times (months) between Executive Board and loan effectiveness byyear (by loan effectiveness year) 52

EB to Signing to EB to signing effectiveness effectiveness

1994 4.3 7.0 11.4

1995 4.7 8.1 13.0

1996 3.1 8.3 11.5

1997 4.8 8.3 13.2

1998 6.8 8.6 15.6

1999 3.9 10.8 14.7

2000 5.2 9.0 14.2

2001 5.0 10.7 15.8

2002 7.4 9.8 17.3

2003 4.9 12.1 17.0

Source: PPMS

Table 7 Average elapsed times (months) between Executive Board and loan effectiveness by region (by loan effectiveness year)

EB to Signing to EB to signing effectiveness effectiveness

Western and 5.0 10.0 15.1Central Africa (PA)

Eastern and 4.3 9.5 13.9Southern Africa (PF)

Asia and the Pacific (PI) 3.0 7.3 10.3

Latin America and 9.1 11.3 20.5the Caribbean (PL)

Near East and 4.2 8.7 12.9North Africa (PN)

Source: PPMS

a Investment projects only

51 The estimates here are slightly higher than the esti-mates given in the Progress Report on the ProjectPortfolio April 2004.

52 All data are based on the PPMS. Project 241 (Haiti)which experienced inordinately long delays followingapproval, mainly because of protracted political diffi-culties, was excluded from the analysis.

Key Points:

n IFAD’s assistance is broadly pro-poor, although per capita lendinglevels to the poorest and mostpopulous countries are belowthose in less poor countries.

n The resource allocation model usedup to 2003 is out of step with bestpractice. PBAS offers a more trans-parent criterion for identifyingpolicy and institutional challengesat country and sector level, but it is not clear how equipped IFAD isto respond in more selective waysconsistent with PBAS. Concernsabout the full and equitable treat-ment of countries, particularlythose at the lower end of thepolicy and institutional scale is afeature of well established PBAsystems and is likely to be a continuing concern for IFAD given its limited resources.

n ODA trends suggest a generalmove away from direct agriculturalsupport towards more multisec-toral rural support. Despite itsspecific mandate IFAD appears notto have been able to convinceother donors of the merits ofcontinuing to resource the agricultural sector. While IFADmaintained its contribution to agriculture for much of the 1990sit has more recently broadened its focus in the direction of ruraldevelopment, institutional changeand capacity building along withother donors working in the sector.

n Portfolio performance data showsIFAD on a par with most other IFIsbut there is no room for compla-cency. IFAD’s figures are closer tothose in the World Bank in the late1990s, indicating the need for amajor push on portfolio improve-ment to bring IFAD up to speedwith its comparators.

Box 3 Characteristics of the country sample

IFAD works across a range of country contexts. There are middle income and low income countries, countries thatare strong performers in terms of policy and institutional reform and others that are lagging. There are countries ontrack to meet the MDGs by 2015 and others that are seriously off track. Some countries are emerging out ofprolonged periods of conflict and others are moving deeper into it. The IEE sample of countries spans a number ofthese different characteristics. Table 9 lists each country by its most recent position on IDA’s CPIA, which measurescountry institutional and policy performance. Six out of the eight countries to which this measure applies appearamongst the better performers. Despite good macro performance, at least five of the countries in the sample areoff-track in terms of meeting the MDGs, all of them except for Pakistan are in sub-Saharan Africa. Mozambique hasalso recently emerged from a prolonged period of conflict, while Pakistan faces significant security risks linked to itsgeopolitical position. Two of the countries in the sample are middle-income countries - Egypt and Peru. The sampleis unfortunately too small to test the statistical correlation between country characteristics and project performance,but they are clearly relevant in our understanding of the challenges of achieving sustained poverty impact.

to which they relate. The mechanisms andcorporate processes by which IFAD’sapproach to programme and project develop-ment has changed over time are discussed inmore detail in Chapter 3. The followingsection considers the impact and sustainabilityof these projects leading to conclusions abouthow well IFAD is doing in delivering on itsmandate and mission.

2.20 Table 10 provides a summary of the mainperformance ratings generated by the projectevaluations. In terms of relevance to corpo-rate objectives and country developmentpriorities, IFAD financed projects are doingwell. All the projects evaluated sought toreduce poverty, either directly or indirectly,and poverty reduction is one, if not the over-arching priority of the majority of IFADborrowing countries. Effectiveness is substan-tial in two-thirds of cases, but this leaves onethird of projects in the sample underachieving

against development objectives.53 Efficiency issatisfactory – high or substantial – in only halfof the projects evaluated and less than satisfac-tory in the other half. IFAD financed projectsof course rely heavily on the performance ofthe borrower for successful implementation.In most of the IEE projects the borrowerperforms reasonably well, although there areclear grounds for improvement. The perform-ance of cooperating institutions and IFADcombined is marginally lower than that of theborrower, reflecting mainly problems withsupervision and delayed action on the part ofIFAD to correct underperforming projects.While the sample contains a few outstandingprojects, the IEE finds that the current oper-ating model fails to lift project performancemore generally.

Benchmarking performance2.21 There is a high degree of similarity between

the IEE results and those of the ARRI 2003.Table 11 shows a strong similarity in theresults produced by the IEE and those generated by the ARRI for 2002 and 2003.The implication is that, while the sampleswere drawn rather differently, together theyprovide a fairly representative slice of IFAD investments.

2.22 In terms of other comparator organizations,the only documented benchmark is the WorldBank’s rural sector portfolio. Figures for theongoing portfolio have already been compared(Tables 4 and 5). For completed projects,World Bank figures show that on average

Table 9 Country policy and institutional assessment, 2003

CPIA quintile IEE sample countries

First Quintile Armenia, United Republic of Tanzania

Second quintile Bangladesh, Bolivia, Burkina Faso, Pakistan

Third quintile Mozambique

Fourth quintile Guinea

Fifth quintile -

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

32


Note: The CPIA does not include Egypt and Peru becausethey are both MICs.

67.5 per cent of rural sector projects achieveda satisfactory outcome between 1994 and 2003,compared to 68.9 per cent for the portfolio asa whole.55 But there was a rising trend in thelast few years and by FY02 the share of satis-factory completed rural projects had risen to81 per cent, reflecting substantial efforts onthe part of the Bank to improve portfolioquality in the rural sector.56 The Bank’soutcome rating is a combination of relevance,efficacy (effectiveness) and efficiency.Averaging the IEE results for all 20 projectsacross the three criteria produces an ‘outcomerating equivalent’ of 70.7 per cent for theperiod, which is close to the Bank estimate,but when restricted to completed projects thefigure falls to 61 per cent.57 The sample ofclosed projects is very small and the compar-ison should be taken as indicative at best, but the indication is that, IFAD cannot afford to be complacent about its levels ofproject performance.

Relevance of IFAD investments2.23 Project evaluations point to a high degree of

congruence between project objectives andcountry development priorities, IFAD strate-gies and beneficiary needs.58 Still, in a quarterof the projects surveyed, there are strongreservations by the IEE country evaluationteam about whether the investment decisionwas in fact the most relevant one, eitherbecause of concerns about fit with IFAD’scurrent strategic objectives (Pakistan), orbecause of a mismatch between IFAD’s choice

33

Table 11 Comparison between IEE results and OE’s ARRI 2002 and 2003

% high and IEE ARRI ARRI substantial sample 2003 2002

Relevance 100 90 80

Effectiveness 67 70 60

Efficiency 45 50 50

53 This is about the same share of unsatisfactory projects across thetotal portfolio identified by Wappenhans in 1992 in the WorldBank. The Wappenhans report, which focused on the quality ofthe entire World Bank portfolio and not just its rural and agricul-tural projects (which at that time were doing worse than theaverage for the Bank as a whole) triggered a major programmeof portfolio improvement in the World Bank.

54 The modal class(es) are shaded. It was considered inappropriateto rate effectiveness of projects in the very early stages of imple-mentation: Barani Area Development Bank (BADP), Pakistan andthe Agricultural Marketing Systems Development Project(AMSDP), the United Republic of Tanzania, hence N for effective-ness is 18 rather than the full sample of 20.

55 World Bank Annual Review of Development Effectiveness 2003.Operations and Evaluation Department, Washington D.C. Thefigures are calibrated by the number of projects and not bydisbursements. If results are weighted by disbursement the figurerises to 77 per cent for rural sector projects and 78 per cent forthe portfolio as a whole.

56 Partial results for FY03 indicate a return to the 1994-03 average,although a significant number of FY03 exits had not been evalu-ated at the time the Annual Review of Development Effectiveness(ARDE) was released.

57 The Bank’s figures are for completed projects only. The IFADsample of current projects includes those at varying stages ofimplementation. It is difficult to make a coherent assessment ofperformance until close to the end, and preferably after imple-mentation has been completed.

58 Four projects Sustainable Development Project by BeniIndigenous People (PRODESIB), Small Farmers TechnicalAssistance Services Project (PROSAT), North-West AgriculturalServices Project (NWASP), Agricultural Services Project (ASP) cite a high degree of congruence with IFAD strategic documents (the strategic framework, the relevant regional strategy and theCOSOP) though this partially because the project design predatesthese strategies.

Table 10 Summary of project performance ratings54

(sample size: N = 20 projects except where stated) High Substantial Modest Negligible N

Relevance: The extent to which the project fits country development priorities, IFAD strategy and beneficiary needs. 60% 40% - - 20

Targeting: The extent to which the design targeted the right people with appropriate activities. 10% 55% 35% - 20

Effectiveness: Achievement (or expected achievement) of the development intervention’s major objectives. - 67% 33% - 18

Efficiency: the economical conversion of inputs/ resources into outputs and outcomes. 20% 25% 50% 5% 20

Performance of the borrower: The overall performance of project management and implementers. 25% 35% 40% - 20

Performance of CI and IFAD: The overall performance of IFAD and the supervisors. 10% 45% 45% - 20

of instrument and the wider aid coordinationcontext in the country (the United Republic ofTanzania), or because of concerns that IFAD’sunilateral support of a particular area ofgovernment policy has limited its options topursue a more progressive and potentiallymore relevant development agenda (Egypt,Box 4). This finding points up a widerproblem with IFAD’s strategies, identifiedduring the desk review.

2.24 IFAD’s strategic statements are highly permis-sive and sufficiently imprecise to make almostany work with the rural poor seem to berelevant. The IEE desk review (page 16)notes about the IFAD Strategic Framework2002-2006: The intellectual and policy frameworkis largely permissive, ruling almost everything inand very little out. Regional strategies arestrong on the quality of analysis, the clarity ofobjectives (although it is less clear that theseconstitute a good guide to resource allocation)and for setting out principles to guidecountry programming. But they are weakerin their recognition of other international bestpractice and knowledge about growth andrural poverty reduction at regional level. Andthe biggest weakness of all is the lack of anyclear indicators for monitoring progressagainst regional objectives. (DRR paras 3.33-34) A review of 20 COSOPs found them to besatisfactory in terms of corporate alignment,the degree of local ownership and stake-holder engagement, the quality of analysisand targeting issues. In many ways COSOPsare still functioning largely as an aggregation

of project ideas rather than as country strate-gies able to provide selectivity and focus. In particular, COSOPs are not providing therigorous filter required at project identifica-tion. They are strong on analysis but weak inarticulating IFAD’s development niche. Linkswith in-country processes, such as PRSPs, andarticulating ways of catalysing others to takeforward replicable ideas also need improve-ment. The vagueness of innovation and poor arrangements for M&E highlight the weaknesses.

Effectiveness of implementation2.25 Only two thirds of IFAD financed projects

are expected to achieve development objec-tives. The rating of effectiveness is a measureof the achievement of development objectivesas defined by the project. In other words,“how well has the project done what it set outto do?”59 Based on this measure there arecertainly some high performing projects inthe IEE sample but, equally, serious imple-mentation problems are a common occur-rence (Box 5).

2.26 Of the 16 projects in the sample whereimplementation is well underway orcompleted, half have suffered from majorimplementation problems (not including the two Bolivia projects, which were forcemajeure interrupted). Three projects (twoin Guinea, and PDRSO in Burkina Faso)reveal major mutually reinforcing weak-nesses: the size of the working area, thetarget population, and the number of

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

34


Box 4 Concerns about relevance in Egypt

In Egypt IFAD has unilaterally supported government efforts to reclaim new lands and provide viable livelihoods for

the dispossessed and unemployed largely through agricultural investments. Development of the reclaimed desert

areas is not without controversy, however. In addition, IFAD's emphasis on supporting agricultural investments does

not necessarily accord with the highest priorities of targeted beneficiaries, given the absence of even the most basic

infrastructure and services on most of the settlements. Indeed, there is some debate as to whether IFAD is working

with the poorest in Egypt as a whole - with farmers in Upper Egypt being poorer and an increasing number of

agencies turning their attention to this region.60 Furthermore, IFAD’s main partner, the Ministry of Agriculture and

Land Reclamation, is not a natural partner for participatory, bottom-up planning and intervention, plus the more

innovative aspects of community development and gender work – all critical dimensions of IFAD’s perceived

comparative advantage. One way forward could be to introduce the Ministry to the innovative work of NGOs such

as CARE whose action research on small-farmer cash cropping in Upper Egypt has recently been recognized with a

World Bank grant.

components were too great, and the selec-tion of the project management team wasbelow standard.

2.27 The extent of implementation difficulties isfurther revealed by the fact that half of theprojects reviewed by the IEE wereredesigned at mid-term, many of themsubstantially. It is notable that redesignissues become concentrated at the point ofthe mid-term review (MTR), rather thanbeing resolved earlier in implementation.Seven projects in the sample (35 per cent)were sorely delayed with very low effective-ness for some years until major modificationfollowed the respective mid-term reviews.62

While design changes are not uncommon inany development project and are, to someextent, desirable given shifts in the projectcontext, the number of major redesigns and the consequences for effectiveness (andefficiency) in the first few years of imple-mentation suggest that some serious modifications to IFAD’s project instrumentsare required.63

35

Box 5 Project design issues have impinged on effective implementation

Design is an essential component of aid quality, and the quality of aid has a tremendous impact on its effective-ness.61 The DRR (Chapter 3) provides a comprehensive account of the main weaknesses in IFAD project design.There is satisfactory definition of the project objective, outputs and the logical connection between them, but weakdefinition of practical indicators and targets for tracking performance, reflecting indecision about adopting thelogical framework. Treatment of sustainability is weak; adjectives and aspirations are not matched by practical design features. Appraisal of the implementation capacity, motivations and support requirements ofproject managers, partners and monitors is weak overall, and anti-corruption measures are not specified in project documentation.

Many of these design issues undermine on-the-ground effectiveness, and the IEE country studies are clearly illus-trative of this point:

n The Rural Financial Services Programme (RFSP) (the United Republic of Tanzania) has a confusing and overlyprescriptive design, and most of the 27 risks identified in the logframe should have been clarified and internalized during formulation and loan negotiations. If no mitigation of risk was possible at design since theproject was seeking to pursue a new and largely untested approach, it may have been preferable for thiscomponent to be explicitly described as a pilot initiative.

n South West Rural Development Project (PDRSO) (Burkina Faso): the logic of the project was weak with poorlystated development objectives framed around a project means, for the community to identify its own prioritiesand ways of working. Also, there was a lack of environmental impact assessment (of road building) and a highrisk of developing valley bottoms with only weak mechanisms for reducing risks.

n Barani Area Development Project (BADP) (Pakistan): the logic of the intervention deserved more attention,including the fact that 80 per cent of income comes from off-farm sources, the problems with public servicesreaching the poor, and, the project appraisal document’s ignorance of the consequences of far reaching devolution in Pakistan.

59 This assessment tends to overstate potential effectiveness assome projects have poorly structured objectives, a pointexplained in Annex 4.

60 For the Egypt programme as a whole, the CPE (2004, page 5) confirms the assessment of the IEE and notes thatthere is less relevance to the poorest.

61 As empirically demonstrated by Wane, W. (2004), TheQuality of Foreign Aid: Country Selectivity or DonorsIncentives?, World Bank Policy Research Working Paper3325, June 2004.

62 These were: AqDP (Bangladesh), PDRSO (Burkina Faso), East Delta Agricultural Services Project (EDNASP) (Egypt),Smallholder Development Project in the Forest Region (PD-PEF) (Guinea), Smallholder Development Project inNorthern Lower Guinea (PD-PAPE) (Guinea), Nampala ArtisanalFisheries Project (NAFP) (Mozambique) and Pat FeederCommand Area Development Project (PF-ACP) (Pakistan).

63 The FLM is intended to offer a more phased, adaptableapproach to project design allowing for changes insurrounding conditions, but no systematic evaluationresults are available yet to assess whether or not the FLM is proving more effective than the traditional IFADproject approach.

Effectiveness of targeting2.28 Pro-poor project designs do not always lead to

effective targeting on the ground. A review ofthe literature confirms that the quality ofpoverty targeting is strongly linked to betterproject performance.64 IFAD has identifiedtargeting as a key mechanism for ensuringthat the poorest and most marginal ruralgroups benefit from IFAD-financed invest-ments, but this awareness is undermined bymixed messages emanating from phrasessuch as the ‘economically active poor’ and‘rural poor who have productive capability’which lack a clear and simple interpretation.From the project documents, the DRR ratedIFAD’s approach to targeting as largely satis-factory at the design stage. The countrystudies, however, found several problemswith translating these designs into effectivetargeting mechanisms on the ground. Sixty-five per cent of the sample are ratedhigh/substantial for targeting overall - poorareas and poor functional groups (small-holders, artisanal fishers) - but only 40 percent are rated high/substantial for targetingthe poorest groups in the target area (Table 12).65

2.29 The varied performance on targeting at fieldlevel is a result of weakly developed targetingmechanisms at the design phase and imple-mentation mechanisms that are inadequate to prevent benefit capture. This includes afailure to provide explicit targeting guidanceand procedures for implementing partners;to ensure that design and implementationmechanisms prevent capture by better-off

groups, and insufficient attention to trackingand monitoring the ongoing effectiveness oftargeting and feeding this information backinto project management decisions.66 A firstrequirement is a clearer policy framework ontargeting to guide project work. The absenceof clear design criteria to identify poorgroups, or targeting guidelines for imple-mentation, has been identified in a forth-coming OE evaluation on decentralization inthree country programmes, currently underpreparation. The study points out that if thetargeted poor are not clearly identifiable, itwould be difficult to attribute changes intheir circumstances to IFAD.67

Efficiency 2.30 About half of IFAD projects do not represent

a good use of resources invested. Only 45 percent of the project sample was rated as havinga high or substantial level of efficiency, withhalf showing a modest performance. Theseresults are similar to the 2003 ARRI, whichrated 50 per cent as high/substantial.

2.31 Economic analysis and planning are notalways in line with the current nature ofinvestments. The DRR notes weaknesses inthe quality of economic analysis, in particularthe continued use of the traditional crop-budget approach even when a substantialshare of project loans is either financing activi-ties where the end use is largely unknown exante (social investments, microfinance institu-tion (MFI) strengthening) or is financing indi-rectly-productive processes and capital invest-ments (group formation).68 More meaningful

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

36


Table 12 Assessment of targeting 69

(Sample size: 42 projects) % % Sample satisfactory unsatisfactory size

Overall targeting (design) 83 17 42

Overall targeting (implementation) 65 35 20

Women (design) 81 19 42

Women (implementation) 58 42 19

Poorer people (implementation) 40 60 20

Source: IEE desk review and Form 5 ratings.

efficiency indicators are required to promotecost-effectiveness. Pakistan PF-ACP illustratesthese concerns, becoming uneconomic whenthe previous AsDB funded rehabilitationproject is included in the analysis. The projectcould only be economically justified on a‘sunk costs’ argument, yet this issue was never addressed at appraisal. There was noeconomic analysis at project completion(though AsDB say they will attempt one).

Borrower performance2.32 Some of the implementation difficulties

noted above to do with non-appearance ofbudgeted co-finance or difficulties flowingfrom both design and implementationarrangements are largely beyond IFAD’scontrol. The evaluation also looked at theperformance of the borrower, which ismainly evident in the arrangements forproject management.

2.33 Efficiency is often compromised by less thansatisfactory project management.70 In 40 percent of projects, the performance of theborrower in project management and imple-mentation is rated as modest. Difficulties withproject management are by no means uniqueto IFAD but notable is the relatively slowresponse by several IFAD CPMs to emergingmanagement problems and the tendency towait until MTR to put in place correctionalmeasures. The long wait for re-design under-mines efficiency, at least in the short run.

2.34 Good examples of project management doexist, and the country visits found that the use of open competition has been a significantfactor. This is supported by the studies ofPeru, the United Republic of Tanzania andMozambique PAMA Support Project (PAMA)where the management teams were hiredusing open competition. The selection ofcapable managers and project staff, combinedwith strong ownership of project-level objec-tives, translates into improved decision-makingas well as on-the-ground performance. Otherproject management units seen as performingwell include: Armenia’s North WestAgricultural Services Project (NWASP) and Agricultural Services Project (ASP);Mozambique’s Nampala Artisanal Fisheries

Project (NAFP); Burkina Faso’s RuralMicroenterprise Support Project (PAMER) andBangladesh’s Small-Scale Water ResourcesDevelopment Sector Project (SSWRDSP).

2.35 But, in many cases project management is lessthan satisfactory. In eight out of 20 projects(40 per cent), the performance of theborrower is rated as modest. A further five of the projects in the sample are very new,and there has not been much time formanagement problems to emerge. The deskreview (paragraphs 3.83-3.85) also pointed toconcerns with project management, namelythe analysis of current capacity and commit-ment amongst intended implementers.Weakness in assessment of the institutionalsetting means that management arrangementsdo not tackle underlying problems. There aretwo main issues that have emerged from thecountry studies: (i) that major managementproblems are allowed to occur in the firstplace, an issue linked to IFAD’s distanceduring implementation and, arms-lengthsupervision noted later in this chapter; and,(ii) where less severe management problemsdo exist they persist largely unchecked andfor a number of years.

37

64 Coady, D., Grosh, M. and Hoddinott, J. (2003) A Review ofTargeting Approaches. World Bank, Social Safety NetsPrimer Notes, Issue 10, 2003.

65 There are no clear definitions within IFAD for what is meantby the poor and poorest. Global definitions of ‘the poor’relate to all those living below USD 2/day and ‘the poorest’(or absolute poor) to those living on less than USD 1/day.Such definitions are not widely used in the project docu-ments sampled by the IEE. In assessing whether or notprojects were effectively targeting the poor and poorestgroups the IEE teams relied largely on project M&E data(where it was available) and assessments and rankings bybeneficiaries and project management teams.

66 In a recent review of targeting in 122 projects in 48 coun-tries, Coady, D. et al (2003: 64-5) concluded that, thequality of implementation matters tremendously to thetargeting outcome.

67 IFAD (2004) Thematic Evaluation of IFAD’s Performance andImpact in Decentralizing Environments: Experiences fromEthiopia, the United Republic of Tanzania and Uganda. OE

68 The Decision Tools for Rural Finance (2003) does includebest practice methods for use within MFI-supportedprojects, but this appears to be an exception.

69 The percentage satisfactory and unsatisfactory is the aggre-gate of high/substantial and modest/negligible respectively.

70 The primary assessment of borrower performance by the IEEhas been through the project management arrangements.

Supervision performance of cooper-ating institutions, and IFAD support

2.36 The current arrangements for supervision can be made to work, but more often than not supervision underperforms. (Box 6) The findings on supervision match thoseproduced by OE, which suggests that thesupervision model used in IFAD supportedprojects may not be the most effective toenhance project performance, particularlygiven IFAD’s approach to innovation andreplication.71 Nine of the twenty projects (45 per cent) are rated modest for supervisionperformance (an assessment of both CI andIFAD), and the remaining 55 per cent showsome specific concerns although overall theyare rated as substantial. This comparesunfavourably with the latest assessment by thequality assurance group in the World Bankwhich finds that the quality of supervision inrural projects was fully satisfactory in 90 percent in FY02.72

2.37 The CPM-project relationship is a criticalelement of IFAD’s supervisory role, because theCPM is the only representative of IFAD able toreview technical issues and ensure that objec-tives, targeting and implementation practicesremain in line with IFAD’s policies. While mostCPMs are viewed positively, the project staffusually have little interaction with the rest ofIFAD. In Armenia for instance, the CPM(s) andcooperating institutions for both NWASP andASP are viewed positively by project manage-ment, but beyond this small group, there arefew other IFAD staff actively involved. Similarly

in Burkina Faso, there is a low level of engage-ment between project staff and other IFADstaff based in Rome. Few IFAD documents aremade available to project-level staff and famil-iarity with broader IFAD objectives is relativelylow. The narrow base of contact betweencountry authorities and IFAD limits the scopefor interaction, places a high responsibility onthe CPM, and reduces the ability of colleaguesin IFAD to ‘challenge’ decisions about projects.These points are a feature of the current IFAD business model, developed in Chapter 3,paras. 3.2 et seq.

2.38 The IEE sample of countries is too small todraw conclusions on field presence and directsupervision. In Mozambique and the UnitedRepublic of Tanzania, the liaison officer initia-tive has helped bridge the communicationgap, but for some stakeholders this has beenlessened by the absence of any real authoritydelegated to the liaison officers within IFAD.In Peru, the out-posted CPM has contributedto the portfolio especially with increasedimplementation support (though with a lackof influence in Bolivia). In Armenia, the directsupervision of NWASP was broadly successful,but not without cost. Indeed the CPM nowprefers a central role in supervision missionsbut with administrative and fiduciary respon-sibilities delegated to a competent CI.

Country programmes and policy influence

2.39 The introduction of the COSOP in 1995 wasintended to provide a medium term frame-

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

38


Box 6 Making supervision work within the current system

In Armenia, supervision by both the CI and the CPM are viewed positively by project management and welcomedas part of the process of improving performance. The proactive and pivotal role of the CPM is a strong contributory factor, including:

n An active input during identification and formulation to ensure good project design

n Asserting some influence in ensuring that a capable supervisor is selected

n Using the CI to its strengths, including checking loan conditions, disbursements, and providing fiduciary expertise

n Visiting projects at least once a year, and up to three times during the first critical stages

n Developing a good relationship with project management, including frequent communication (such as quick responses to emails/faxes)

n Active participation on mid-term review missions, including field visits and report writing

work for IFAD operations. The initial aim wasnot to be comprehensive but to focus onIFAD’s specific role, niche and future direc-tion in the country. Over time the guidancedeveloped to include the relationship betweenthe Fund’s strategic and regional objectivesand interventions at the country level plus thelink with national policy processes and opportunities for engaging in pro-poor policy dialogue.

2.40 At first few IFAD resources were devoted tothe COSOP process.73 More recently, PMDhas devoted considerable time to improvingthe quality of COSOPs. The DRR found thatthere was evidence of improvement in thelater COSOPs, including an increasedemphasis on identifying opportunities for pro-poor policy dialogue, but many were still functioning largely as an aggregation of project ideas. Evidence from the countryvisits provides a similar story. In few cases are COSOPs considered to be the basis of acoherent country programme. The forth-coming OE study of decentralization identifiesgaps in plans for pro-poor advocacy andpolicy dialogue.74

2.41 The IEE found disappointing levels ofsynergy between projects, between differentaid instruments (loans and grants) andbetween projects and policy processes. In thefew cases where synergy does occur, it hasbeen used to good effect; Peru is a notableexample. Elsewhere, a multiplicity of grantfacilities and modalities has led to a loss offocus, lack of strategic orientation and noprioritization. With the exception of the agricultural research programme, reported in the desk review, there has been littlesystematic evidence to inform policy. Afterprotracted delays a new grants policy was

39

Key Points:

n Relevance: The fit between objectives andneeds is high. But the IFAD strategic frame-work, and to a lesser extent the regional andcountry strategies do not provide effectivefilters for selectivity. There is no clear basison which to turn down a project. In partic-ular, there is no real challenge to newproposals at identification.

n Effectiveness: Two thirds of projects are ratedas satisfactory, but half suffer major imple-mentation problems. Redesign is concen-trated at the MTR, and not dealt with earlier.There is a need for flexibility and resourcesto make adjustments earlier during imple-mentation – an issue strongly linked tosupervision problems.

n Targeting: Is less effective in practice thandesign documents suggest. COSOPs andproject documents lack clear analysis ofcriteria to identify the poor or guidelines for implementation. Benefit leakage to lesspoor groups is potentially a key issue forIFAD to assist governments in their deliveryof rural services to the poorer sections of society.

n Efficiency: Half the project sample was rated as having a modest level of efficiency.Economic analysis and planning has not kept pace with the current portfolio ofinvestments. Institutional analysis is ofteninadequate, and there is a need to planmanagement arrangements earlier – learningfrom lessons where open competition hasbeen used.

n Borrower performance: the better examplesof project management appear to be associ-ated with teams recruited by open competi-tion. Poor performance elsewhere is linkedto weak institutional analysis during projectdesign and the arms-length relationship byIFAD during implementation.

n Supervision: Quality is substantially weakerthan for the World Bank’s rural portfolio.IFAD needs to select, monitor and intervenemore effectively when supervision underper-forms. There is a real gap in the develop-ment focus of CIs. There are structural weaknesses in the headquarter-to-field relationships arising from the individualnature of the links between a country andthe CPM, and contracted-out supervision.

71 Evaluation of Supervision Modalities (2003). 72 Although because of the different approaches to supervi-

sion in the two institutions, the quality of supervision ismuch more a measure of World Bank performance alonethan it is in IFAD where the role and performance of CIs is central.

73 Only USD 20 000 per COSOP.74 Op cit.

adopted in 2003 and there are some earlysigns that new approaches will counter pastcriticisms, but it is too early for the IEE toassess likely outcomes.75

2.42 Although COSOPs include a focus on policydialogue evidence of policy influence ispatchy. In the country sample, both the RuralFinancial Services Programme (RFSP) and theAgricultural Marketing Systems DevelopmentProject (AMSDP) in the United Republic ofTanzania have specific policy-developmentcomponents. RFSP has played a significantrole in contributing to national policy reformin microfinance provision and cooperativeorganization. In Mozambique, both PAMAand NAFP have specific components focusingon policy change, and IFAD has made somenotable inroads. In NAFP the main achieve-ments have been (i) the adopting of a nationalfisheries policy and strategy; (ii) new legisla-tion to legalize fisheries co-managementcommittees; and (iii) new policy and legisla-tion on fishing zones and nets. In PAMA keypoints of policy influence are (i) draft legisla-tion on registration of farmers associations;and (ii) a position paper on modifications tothe agricultural concession’s policy. In othercases, Burkina Faso, Guinea and Pakistanthere is no evidence of any real policydialogue emerging out of project experiences.

2.43 There are also concerns about implementa-tion and sustainability. Evidence of influencedoes not necessarily mean that either policycapacities or policy regimes have significantlychanged. Under Mozambique/NAFP, despitethe positive legislative and rule changes, therelaxation of closed season restrictions hasbeen abused by industrial fishing vessels andfishers from other provinces and could lead torapid depletion of fish reserves. Relaxation ofclosed season restrictions to allow the poor tobenefit continuously from a natural resource,without a strong exclusion mechanism hasalso fuelled leakage of the benefits to the rich.Moreover, enforcement of the three mileexclusion zone ended when the projectresources dried up leading to intensifiedcompetition amongst small and large fishers.Government’s lack of continuity in theenforcement of new policies, largely because

of weak capacity, is limiting the impact ofIFAD in the area of long term policy and institutional change. As the Mozambique CWPnotes A project to influence a change in policy isincomplete unless it also supports the creation ofsustainable capacity for its enforcement.

2.44 IFAD has not established, despite its commit-ment, a separate policy dialogue function at country level. Using projects andprogrammes to engage at policy level hasbeen part of IFAD’s mandate from the verybeginning. Despite its longstanding commit-ment to developing a dialogue on rural policyissues, the IEE finds little evidence that this ishappening systematically or on any scale atcountry level, either with governments or withother development partners. While projectscan provide a platform for policy level discus-sions, experience from other developmentorganizations suggests that it is often the useof non-lending instruments, especiallycountry-specific analytical work and relatedcapacity building, that is critical to developingeffective processes of policy dialogue.76 TheIEE finds that IFAD is yet to develop sufficientcapacity to undertake detailed policy analysisbuilding on its operational experiences insuch a way as to equip staff to take forwardsuch dialogue. As the DRR noted, analyticalcapacity at the centre remains diffused acrossdivisions and individual CPMs and feedbackloops between operations and knowledgesystems at the centre remain largely ad hoc.The country visits also found relatively weaksynergy between lending and non-lending(technical assistance grants) activities atcountry level, with very limited use of non-lending instruments to pursue policy issuesbeyond the projects themselves.

2.45 This perspective is reinforced by a number ofcountry examples, including Egypt where thenature of dialogue with partner organizations– both governmental, non-governmental anddonor – is considered to be underdeveloped,despite longstanding engagement with theMinistry of Agriculture and Land Resources.And in Guinea where, notwithstanding thedifficulties of the operating context, IFAD hasnot had a technical input into the definition ofnational policy to reduce poverty, culminating

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

40


in the PRSP, and has been without a presencein most significant policy debates in thecountry.77 In the United Republic of Tanzania,despite some important project-level achieve-ments, no government or donor informantcould cite evidence of a wider IFAD impact onnational or sector policy, nor could they citeinstances in which IFAD has taken the lead inpromoting ideas or approaches to ruralpoverty reduction. Similar findings emergedfrom OE’s CPE of Indonesia (Box 7).

2.46 IFAD has recognized that a lacuna existswithin its capacity to engage in policydialogue, and that the scope of potentialdialogue has broadened in line with IFAD’soverall mission, from a focus on maximizingthe impact of projects to identifying criticalrural development bottlenecks and creating a‘better environment for the rural poor’. Therecent creation of a Policy Division is intendedto facilitate a more systematic process ofengagement on policy issues across the organi-zation and with development partners. Manychallenges remain however, including devel-oping a coherent programme of work for thePolicy Division and a clear sense of how policyengagement at local, national and interna-tional levels can be better incorporated intoIFAD’s core business.

2.47 IFAD has not yet become a leader on ruralpoverty issues in international circles. TheREA 1994 raised the expectation that IFADshould become “the foremost agent forcoherent and rational activity in ruralpoverty diminishment.” Later, IFAD’s visionstatement for the renewal of the institutionstates that IFAD should be the leader in showingthe way and galvanizing energies to eradicaterural poverty and hunger. The strategic frame-work 2002-2006, somewhat less ambitiously,states that IFAD will seek to influence regionaland international policies that shape rural develop-ment options. A key question for the IEE is to

41

75 The changing modality of grants under the 2003 grantspolicy is likely to have a significant effect. Early examplessuch as the use of a,large country grant for the NepalLeaseholder Forestry and Livestocks Programme Phase IIappear to overcome some of the criticisms of past operations, but it is too recent to predict if this is evidence of a genuine change.

76 World Bank/OED Annual Review of DevelopmentEffectiveness 2003.

77 The Guinea CWP finds that “IFAD’s image is also that of an agency institutionally absent from the country with littleinvolvement in the management of projects and virtually nopresence in policy debates. This is in contrast with itsimportance as a financing agency. Other donors financiallyless important have greater visibility and political presence.

Box 7 Policy effects in Indonesia – results of the country programme evaluation

One of the challenges for IFAD is in using its project-level experience to engage in policy-related dialogues at sub-national and national levels. The most recent COSOP for Indonesia sees IFAD engaging in policy dialogue as ameans of creating a ‘favourable pro-poor rural development framework in Indonesia’. The COSOP identifies areasfor policy dialogue, including decentralization for better project management, the participation of communitybased organizations in project implementation, the promotion of cooperatives at the grassroots level and landreform. It underscores the need for IFAD to coordinate with IFIs like the World Bank and AsDB in order to increasethe effectiveness of its policy dialogue with the Government of Indonesia. A country programme evaluation by OEin 2004 finds that contrary to its strategic objectives extensive enquiries among aid agencies in Indonesia revealedthat IFAD has not engaged in such dialogues in country. Similar findings were made among international NGOswho remain active in rural development. Even with a key partner, the AsDB, the Fund seems not to engage in effec-tive policy discussion. Similarly, IFAD has not engaged in the intensive ongoing policy and implementation discus-sions between donors and government about decentralized development at district level, or with microfinance atnational level, despite these being explicitly identified in the COSOP. In terms of policy dialogue directly withgovernment, IFAD has good relations with BAPPENAS (the National Planning Board) and the Ministry of Agriculture“but there is no evidence that it has influenced the salient features of the policy landscape.” At the micro level,however, and through its projects, some influence is discernable, for instance, in working with NGOs in projectimplementation and in helping agricultural line agencies adapt to decentralization.

IFAD/OE Country Programme Evaluation, July 2004

what extent IFAD has assumed a leadershiprole in rural development consistent with itscorporate aspirations.

2.48 Interviews with a number of close developmentpartners78 suggest that IFAD is largely knownfor its project work in hard-to-reach rural areasor with marginalized groups. This work isgenerally appreciated. IFAD is not widelyknown, however, as a source of policy ideas oras a leading influence on policy debates aboutrural development and rural poverty reduc-tion, although it was asked by the UnitedNations Economic and Social Council in 2003to take the lead role in convening the HighLevel Segment on Promoting an IntegratedApproach to Rural Development. Some otherspecific references were made by developmentpartners to IFAD ‘leading’, for instance, inWest Africa Roots and Tubers, in communitydriven development in the Niger Delta and inits work with indigenous groups in LatinAmerica, but where leadership was noted it wasgenerally considered to be at a low level - theproblem, noted several informants, was “lack of strategic policy influence” and “scaling up.”IFAD’s current lack of leadership is a short-coming that could be corrected through moreselective and stronger strategic alliances withinternational and regional organizations that

bring the necessary technical knowledge andanalytical capacity to add to IFAD’s long-standing field experience.79

Impact on rural poverty2.49 Providing an assessment of the impact and

sustainability of IFAD supported projects is acritical part of the IEE; it represents one of itsmost important contributions and one of itsbiggest challenges, not least because the samplecovers projects at different stages of completionacross ten different countries. Nevertheless asthe Annex 3 makes clear, every attempt wasmade to ensure a rigorous approach that tookinto account not only the stage of the projectbut also the challenges posed by differentcountry contexts.80 Figure 1 in Annex 3 setsout the steps by which data from differentsources were brought together to enable thecountry evaluator to make the overall ratingsas set out in Table 13. The reporting on impactfollows the OE methodology as far as possibleto ensure comparison with existing countryprogramme evaluations and with the ARRI.

2.50 Unsurprisingly, project impact varies widely.A few high impact projects contrast withlower performance overall. The two projectsrated most highly in terms of impact are theManagement of Natural Resources in theSouthern Highlands Project (MARENASS)and the Development of the Puno-CuscoCorridor Project (CORREDOR) in Peru.These strong performers reveal a number ofimportant characteristics:81

n A remarkably rich innovative content, withpublic finance limited to technical assis-tance (TA) and no project finance used forinvestments. A strong demand-drivenapproach lets groups decide what to dowith TA, resulting in a surprisingly largeamount of family resources invested in theactivities supported by project TA.

n Project design and management were attimes subordinate to community demands.In MARENASS, the original conservationprogramme was subjected to communityvote. They gave less priority to conserva-tion, but the project management andCPM concluded that the potential benefitsof full empowerment were extraordinaryand should be respected.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

42


Key Points:

n IFAD’s existing products and instru-ments are not used to their full poten-tial, and the limited options availableplace constraints on what can beachieved. In particular, there is a lackof synergy between projects andbetween instruments at country level.

n IFAD has not developed a model forinfluencing partners and policies ateither country or international level.

n IFAD’s capacity to engage in policydialogue has not kept pace with itsoverall mission. Staff development has neglected this aspect of the CPMs’ work.

n IFAD is not regarded as a leader onrural poverty issues, although there are specific areas of IFAD expertise that are recognized.

Table 13 Summary of investment impact 82

(Sample size: 20 projects) High Substantial Modest Negligible N

Overall impact on poverty 83 11% 44% 44% - 18

Impact on income 6% 47% 47% - 17

Impact on women 22% 22% 33% 22% 18

Agricultural production and food security 24% 24% 53% - 17

Physical assets - road and irrigation infrastructure 7% 36% 57% - 14

Financial services 20% 7% 73% - 15

Social infrastructure and services - education and skills 25% 42% 17% 17% 12

Social infrastructure and services - health and drinking water 10% 50% 40% - 10

Environment and common property resources 33% 11% 33% 22% 9

Social capital and empowerment 17% 6% 78% - 18

Policies, institutions and regulatory framework - 28% 6% 67% 18

Private sector development 29% 29% 29% 12% 17

Sources: Form 5 ratings, plus ratings based on CWP evidence (IEE surveys).

n Design was adapted to reach poorergroups: CORREDOR initially usedbusiness plans that biased towardseducated farmers and would-be urbanbusinessmen. A simpler formula (thebusiness profile) was introduced so thatgroups in the community could take part.

n A strong driving force from the CPM (whois locally resident), the project directorsand a stable group of consultants andassociates in Peru’s ministries and socialscience research centres.

The resulting impacts included:n Rising levels of food security directly

attributable to the project, not so much ingrains, potatoes and other basic foodstuffsbut in the fruits, vegetables, meat, milkand other protein deliverables.

n Dramatic impact on beneficiary self-esteem, with communities and groupsenabled to manage their activities andplan new ones, plus challenge governmentand agencies to help protect and enhancetheir own interests.

2.51 Across the sample as a whole 55 per cent ofprojects are judged to have achieved or areexpected to achieve a satisfactory impact onpoverty (high/substantial) while 44 per cent

are judged to be falling below expectations(Table 13). As already noted, these results arethe aggregated outcome of an immense diver-sity of performance, but what is notable is howclosely they compare with the ARRI for 2002and 2003, which rated impact on poverty ashigh or substantial in 50 per cent of projectsand modest for the remainder. More tellingly,however, is that amongst the closed projects inthe sample, where a full ex post assessment ofimpact is possible, 50 per cent are judged tohave had a less than satisfactory impact on poverty.

43

78 In particular interviews were held with representativesworking on the rural sector at corporate and regional levelsat the World Bank, IADB and AsDB.

79 Not all informants saw it as desirable for IFAD to assume amore strategic policy role, however, seeing IFAD’s valueadded in terms of smaller, targeted actions that otheragencies cannot do rather than the kind of detailedanalytics that other agencies can do.

80 Contextual factors are to some extent already taken intoaccount through the different project designs.

81 Based on the IEE Country Working Paper: Peru, whichassessed CORREDOR and MARENASS projects.

82 The modal class(es) are shaded. Projects were not rated ifthey were at a very early stage or if the specific domainwas not relevant to their objectives.

83 The likely impact on poverty, considering both the totalnumber of people benefiting and the size of the benefits accruing.

Income and household assets2.52 In terms of impact on family or household

income levels, 47 per cent of projects arerated as substantial and 47 per cent modest,the remaining 6 per cent are rated high.Based on beneficiary surveys, food consump-tion and cash income from sales show thestrongest change, with one half to two thirdsof respondents reporting improvements.Income changes from employment are muchless marked, as are the income benefits fromtime savings (see Table 14).

2.53 Low impact on income is partly the result oflow project coverage and over optimism aboutassociated employment effects. In BurkinaFaso, the PAMER microenterprise develop-ment interventions have been more successfulin increasing and stabilizing incomes than inthe PDRSO integrated project. UnderPAMER, a majority (68 per cent) say thatthere have been important or substantialincreases in income from sales. Yet, the over-whelming majority of respondents reportedno changes in income from employment,household assets, time saving, the use of cashcrops or due to the animal rearing. In Bolivia,PRODESIB shows only modest results interms of raising family income – with 39 percent of the sample stating that there had been

no change. In Mozambique, a generallymoderate impact on income is seen by bothprojects. In NAFP this is possibly a reflectionof the limited employment impact of theproject, though expectations of the futureimpact are greater. For PAMA, better infor-mation has meant that farmers are now sellingat viable prices, but the change in income isnot significant without a correspondingincrease in production and competitionamongst traders.

2.54 In some cases increases in income have led toimprovements in household assets. Thesampled projects in Peru provide two notableexamples. In MARENASS, a key result hasbeen the increase in the number and value ofhome and farm assets that was entirely self-financed (as stimulated by competitions andprizes). CORREDOR has also gained substan-tial leverage on family investments, in this caseby awarding contracts for TA to the winnersof the competitions. In Bangladesh, underSSWRDSP, all beneficiary types reportimprovements in house fabric and construc-tion, with most now having a tin roof insteadof straw. In Guinea (under PD-PEF) therehave been improvements in living conditions,and the (even temporary) increase in revenuehas played a part, with earnings often invested

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

44


Box 8 What does a ‘modest’ impact on poverty look like?

In Burkina Faso, PDRSO aimed to counteract the trend of progressive impoverishment and degradation of naturalresources. The project has however led to a disappointing level of impact, and there has been limited engagementby beneficiaries. Literacy training is appreciated and there have been improvements in hygiene. Training in theconstruction of compost bins has also made a positive contribution to production (though benefiting certain types of farms and mostly men). In all other areas however, the project’s contribution has been little or non-existent. Rural credit has largely been a failure with very few people benefiting and rather more suffering negativeeffects. The evaluation team encountered a general sense of dissatisfaction and widespread reports of broken promises.

In Bolivia, PRODESIB aimed to promote the sustainable self-development of the indigenous peoples in Beni throughcapacity building measures at grassroots level. Overall, the project has helped strengthen indigenous organizationsin the land reform process. The most positive changes have been amongst scholarship students who have gainedfrom the increased income earning opportunities following training. In general though, the land titling and trainingprogramme has shown modest results: in terms of raising family income, 49 per cent of beneficiaries said that theirincome had increased slightly, with 39 per cent stating no change. Improvements in food consumption are alsoreported as modest, with 51 per cent of PRODESIB beneficiaries citing small improvements and 33 per cent nochange. In the land titling process there has been little tangible increase in family access to (and use of) forestresources - with the project tending to reinforce community relationships rather than address underlying changesin the existing system.

in (long lasting) housing improvements. Theroad rehabilitation under PD-PAPE is alsosaid to have improved living conditions withbetter accessibility to health services, and to alesser extent, education. In Armenia, underNWASP and ASP, beneficiaries notedimprovements in income and food security,though with little tangible difference in house-hold assets. In Mozambique, under NAFP, theimpact on household assets was confined to alimited number of individuals.

Agricultural production, productivity and food security

2.55 Research, extension and organizational interventions, often supported by credit orirrigation, to increase crop, home garden,livestock and fishery production for consump-tion or sale are historically core areas of IFADinvestment. The sample shows considerablevariability in impact across this domain, with48 per cent of projects achieving a satisfactorylevel of impact (high or substantial) and 53 per cent underachieving. Several examplesof projects leading to increased productionexist: A demand-driven TA approach in

Bolivia (PROSAT) has led to widely welcomedimprovements in food security, income, cropand livestock production. In MozambiqueNAFP has increased smallholder fish production and the functioning of markets.Production and land-use has increased in all the visited SSWRDSP (Bangladesh) sub-projects, due to better flood control, drainageand irrigation facilities, plus the introductionof new high yielding varieties and high valuecrops. Under AqDP, 80 per cent of benefici-aries state that they eat at least a “bit better”as a result of pond and fishery developmentthrough AqDP.

2.56 Table 15 shows that over half (52 per cent) of beneficiaries interviewed had observed ahigh or substantial increase in crop produc-tion for own consumption as a result of theproject. The observations for cash crops and livestock are more modest – with themajority, nearly two thirds, saying they hadseen modest, negligible or no change in theseproduction domains, an outcome that maybe associated with the low prices notedearlier in the report.

45

Table 14 Income effects of IFAD project interventions

The effect on: % % % % % Totalsimproved improved stay about deteriorated/ other

substantially moderately the same worsened % N

Food consumption 15 50 28 5 0 100 1329

Cash income from sales 13 46 35 5 0 100 1313

Wage earning/employment 9 39 46 4 2 100 1176

Time savings 11 27 39 15 7 100 1213

Source: Beneficiary survey (Form 4) data.

Table 15 Beneficiary perspectives of changes in farm production

Change in production of: % high or % modest % no change/ Totalssubstantial or negligible other % N

Food crops 52 26 22 100 1314

Cash crops 37 20 42 100 1286

Livestock for food 38 27 35 100 1259

Livestock for cash 33 27 39 100 1202


2.57 There are also some noticeably poor projectsfailing to make much impact at all. In BurkinaFaso, under PDRSO, the majority of respon-dents noted no (or insignificant) changes inthe production of cash crops, animal rearingfor profit, income from salaries or time saving.In Guinea, under PD-PEF any increase inyields and change to multiple croppingpatterns has been jeopardized by the return of leased land to the landowners.

Physical infrastructure2.58 Public good investments in roads, irrigation

and other water infrastructure are the largeststand alone investments that IFAD makes. Aswell as capital works, they typically includestaff and operating costs in establishingarrangements for operation and maintenanceand water-user groups. Of the 20 projectssampled, 14 have road and/or water invest-ments. Overall, the impact of IFAD invest-ments in road and water infrastructure provi-sion is assessed as modest. Of the 14 projects,only one, MARENASS (Peru) is clearly highimpact. Two (PDRSO, Burkina Faso and PD-PEF, Guinea) are notably poor.

2.59 MARENASS is not a typical publicly-financedinfrastructure investment. Indeed its small,household-level irrigation developmentsresult from a project approach that couplesdemand-driven TA supply with communitycompetitions – leading to a high leverageeffect on household investment. It also hasweaknesses, with a recent study showing sub-optimal irrigation efficiency of some schemes.But overall, the popularity and benefit ofirrigation, its sustainability, and the fact thatit was entirely household-financed make thisa high impact investment in a difficultcontext. The innovative intervention designhas enabled the project to convert irrigationfrom being a hard-to-deliver public good intoa private good.84 Elsewhere four projectstypify the implementation, technical, mana-gerial and sustainability difficulties encoun-tered by irrigation interventions: PF-ACP(Pakistan), NWASP and ASP (Armenia) andWNRDP (Egypt). They also typify thesubstantial benefits for farm households who do achieve reliable water supply to their land.

2.60 In terms of road investment, both projects inMozambique (NAFP and PAMA) show positiveimpacts, with the opening up of fish landingsites to district and provincial consumptioncentres.85 Similarly, in AqDP (Bangladesh),roads were developed as a supporting activitywithin the project, and highly appreciated bythe communities served. Indeed, lake fisheriesgroup members have gained substantiallyfrom improved access to water bodies for fisheries, though a third of the non-fisheryrelated community development groupmembers (mostly poor women) reportedlosing access to waterbodies.

2.61 At the other extreme, the road developmentwithin PDRSO (Burkina Faso) has seen poorcommunity coordination and slow implemen-tation and payments, all within a failingproject which has required major restruc-turing. In Guinea, PD-PEF has performedpoorly, with a small volume of infrastructurebeing completed, and many schemes poorlydesigned, unfinished, silted and not main-tained. Organization of management groupsfor the infrastructure schemes has been weakand non-sustainable. In the United Republicof Tanzania, under AMSDP, a significantunder-estimation of unit costs at design islikely to reduce the volume of (cofinanced)road construction the project is able todeliver. In NAFP (Mozambique) there arestrong reservations about the maintenance ofrehabilitated roads: whilst government policyis now to ensure resource allocation to roadsrehabilitated through aid funds, only one ofthe five roads rehabilitated through NAFPhave benefited from regular maintenance.Sustainable mobilization of resources forpolicy implementation is an area where IFADcould contribute an innovative solution.

Financial services2.62 The DRR commended IFAD’s recent work

in establishing a state-of-the-art policy forinvestments in rural financial services; for its successful participation in the 2003 donor-peer review process, and for the ‘decisiontools’ publication to support field implemen-tation. The country studies reveal that,despite this, field operations are changingonly slowly. Of the 15 projects promoting

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

46


different types of rural finance provision,only four are achieving a substantial or highimprovement in access to credit productssuitable for the rural poor. The impact of the rest is modest, and some very modest indeed.

2.63 There are some notable examples of goodperformance in the rural financial servicessector. In Armenia, under both NWASP andASP, IFAD has supported the development ofthe Agricultural Cooperative Bank (ACBA).After initial start-up problems, ACBA is nowthe third largest (and only cooperative) bankin the country, plus the main lender to ruralareas - where previously the banking sectorhad focused on low risk, high returnborrowers from the main city. Also underASP, rural financial services continue to beextended through the Aniv Foundation -whose borrowers are mostly small ruralenterprises, in contrast to most other organi-zations who focus on medium-sized enter-prises. The prospects for RFSP (the UnitedRepublic of Tanzania) are also good,although it is too early to assess impact. Theproject benefits from being focused on asingle sector and from competent manage-ment, although there are indications that thecurrent drive to increase the number ofparticipating credit cooperatives and theirmembership may be at the expense ofbuilding sufficiently competent governanceand financial management to enable them toretail bank credit.

2.64 Most other credit interventions in the sampleare failing in one or more respects. Recoveryand sustainability has been poor, groupleaders have dominated fund access andmanagement, and project demands have attimes conflicted with bank requirements.Under NAFP (Mozambique) for example,financial services have had a limited impactoverall, with NAFP tending to target mostlyboat owners (the rich minority) and sufferingfrom a cumbersome registration process andpoor training. Overall, the majority ofsampled projects display the familiar range of weaknesses of traditional project creditschemes and overall impact in terms ofcreating sustainable access is modest. Urgent

attention is required to bring ongoingprojects into compliance with IFAD’s newrural finance policy.

2.65 In 2003, the ARRI also noted mixed perform-ance in the area of rural finance, as it did theprevious year. The report made threegeneral observations:n Grassroots, group-based credit and savings

institutions have often proved moresuccessful than official, subsidised credit schemes.

n Repayment rates by members of women’sgroups for unspecified small, short-termloans have generally been very high.Repayment of longer-term specified loansto individual farmers has been much lower.

n The need to ensure institutional and finan-cial sustainability is often either overlookedwhen the credit schemes were established,or remains a challenge in a number of cases.

Impacts on health and education2.66 Although health and education investments

are not a major feature of IFAD-supportedprojects, they have generally had a positiveimpact. Half of the project sample involvesassociated grant investments in health facilitiesand/or drinking water. Overall 60 per cent arerated as having a high or substantial impact.Investments in drinking water supply areespecially popular amongst beneficiaries. In Guinea for example (and despite otherfailings of the project), the boreholesconstructed under PD-PEF have been largely

47

84 The TA plus community-competition methodologies whichachieved this result are a successful example in a very poorcontext of the publicly-funded, privately-provided model,driven by consumer choice, which is now the subject ofexperiment for public service delivery in many countries.

85 Although, as will be noted later, the sustainability of roadinvestment has been a major worry in Mozambique.

successful and are appreciated, particularly bywomen. Similarly, education investments haveshown a relatively high level of impact, with67 per cent rated as high or substantial. Inthe sample of projects, 12 have either refur-bished schools or developed non-formalliteracy and other non-agricultural skills. InBolivia, 86 per cent of PDRSO respondentsreport improvements in adult literacy, whilstin PAMER, which develops rural microenter-prises, 77 per cent of respondents say skillshave improved.

The environment and common property resources

2.67 Few projects have made serious attempts tograpple with the many challenges andcontradictions of agricultural development,environmental conservation and the empow-erment of communities. There has been amixed impact of IFAD projects on the envi-ronment. Nine of the 20 projects sought toconserve the environment and encourage thesustainable use of natural resources. Impactis rated as high /substantial in four andmodest/negligible in five. MARENASS tookthe bold step to elevate the demand-drivengoal above that of natural resource manage-ment. This has resulted in considerable

leverage on households who have invested inconservation practices, although the heavilydegraded (communally held) pasturescontinue to deteriorate. In Guinea, both thesampled projects had environmental objec-tives but achieved very little. This is despitewidespread concern of shrinking fallow-periods, extensive logging, slash-and-burn,and the environmental impact of extensiveroad construction in PD-PAPE. There wasalso no environmental analysis or mitigationmeasures in PD-PAPE to address roadconstruction through fragile areas.

Social capital and institutional impact2.68 In recent years IFAD’s portfolio has shifted

from a strong emphasis on production andproductivity to a much broader poverty andempowerment agenda (Chapter 1). As such, itsinterventions are not just about transferringtechnologies and methods, but involve a muchgreater focus on the way production is organ-ized and the institutions required for opera-tions and maintenance, marketing, finance andother inputs. A key part of this transition is agrowing emphasis on developing pro-poorinstitutions that will enable the poor to helpthemselves and ensure operations and mainte-nance and financial sustainability.

2.69 IFAD interventions have not had muchsuccess in fostering social capital or in creatingnew and sustainable institutions. The findingsof the IEE country studies reflect OE evalua-tion findings: where IFAD-supported projectsstrengthen existing local institutions theimpact is positive. The downside is thatexisting institutions are not always representa-tive of the poorest groups or women. Butcreating new institutions appears to posemore difficulties.86 Institutions are not sustain-able unless people have a good sense ofmutual trust. If there is little social capital, theinstitutions fall apart. That has led the IEE toan overall rating that shows much lowerperformance than the ARRI in 2003.

2.70 In Pakistan, under PF-ACP, village andwomen’s group organization was a significantachievement within a difficult project envi-ronment, but much of that achievementremains dependant on continued inputs

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

48


Key Points:

n While the impact on poverty is modestoverall, there have been some gains inIFAD’s traditional areas of expertise:agricultural production and foodsecurity, plus a few notable examples ofroad construction, irrigation works andfinancial services.

n Health and education are not signifi-cant IFAD investment streams, yetimpact is reasonable, and community-led water supply seems particularlyeffective and appreciated by beneficiaries.

n The contribution to environmentalconservation and development is mixed,and few projects show signs of havingseriously grappled with the many chal-lenging issues that exist.

from the project via a sub-contracted NGO.In Bangladesh, under AqDP, the projectsupported private sector initiatives for indi-vidual and group based aquaculture inleased private ponds, and also through lakefisheries groups managing larger waterbodies in the form of cooperative businesses.Early examples of the latter have hadproblems with the dominance of non-fisherstakeholders. In Pakistan, the MansehraVillage Support Project (the predecessor tothe sampled BADP) formed many groups,but the impact over time is generally poor.

2.71 Part of the problem of ‘new institutions’ is thatthey are dependent on continued projectsupport for their longevity. An alternativeapproach in Armenia shows that this is not anecessary condition. Here the ASP has taken adeliberate approach to investing in key insti-tutions for each component, on the assump-tion that it is these organizations that will bearound long after the project ceases to exist.Consequently the ACBA bank (rural credit)and the Aniv Foundation (small enterprisefinance) have become the only significantinstitutions in the lives of project beneficiaries,neither the ‘NWASP’ or ‘project co-ordinatingunit’ make much sense outside of these two institutions.

2.72 IFAD investments also look to support moregeneralized empowerment amongst the poor,both at the level of individual self-esteem and collective engagement in policy processesthat affect their lives. The evidence frombeneficiary surveys suggests that a small

majority (59 per cent) feel more confident tospeak out and assert their rights as a result ofcontact with IFAD-supported projects, withmany feeling more confident in interactingwith public institutions.87

Sustainability2.73 Sustainability is considered along two key

dimensions. Firstly, whether the stream ofbenefits accrued as a result of the project iscertain (or likely) to continue after closure.And secondly, whether the institutionalchanges induced by the project are likely to continue after closure.

Sustainability of impact2.74 Sustainability of impact is substantial in

just under two-thirds of projects, but if theanalysis is restricted to those ten projects thatare nearly or actually closed, the proportionfalls to less than half (Table 16). This matches

Table 16 Sustainability of impact and institutional impact

High Substantial Modest Negligible N

Sustainability of impact: The likelihood of project-induced benefits continuing after project closure - 61% 39% - 18

Late and closed projects only - 40% 60% - 10

Sustainability of institutional impact: The likelihood of institutional (state, private, civil) changes induced by the project, continuing after project closure 6% 35% 53% 6% 17

Late and closed projects only - 22% 66% 11% 9

Sources: Form 5 ratings.

49

86 The OE Country Programme Evaluation for Indonesia(2004) has similar findings.

87 The OE report on decentralization [IFAD (2004) ThematicEvaluation of IFAD’s Performance and Impact inDecentralizing Environments: Experiences fromEthiopia, the United Republic of Tanzania andUganda] identifies two issues that further develop thefindings of the IEE: first, the need to help develop thecapacity of elected bodies as representatives of the people;second, for projects to plan and implement mechanisms forgovernment organizations to respond to people’s voice.

the ARRI 2003 finding that sustainability issubstantial in 50 per cent of closed projectsand modest in the remaining 50 per cent.The evaluation of likely sustainabilitydepends partially on the views of the benefi-ciaries themselves. It is therefore notsurprising that during the early stages of aproject there are relatively high expectationsthat project benefits will endure.88 Theseexpectations are clearly modified in the laterstages of implementation. The fact that somany projects provide services for a fixedperiod of implementation and are neithermainstreamed nor lead to follow-on projects,is also a contributory factor.

2.75 In MARENASS (Peru), findings generallysupport the notion that farm practices thatrequire little or no additional cash would bemost likely to survive, while instruments thatrequired cash to continue would suffer most.A high level of sustainability is expected forthe most prominent popular practices, suchas the irrigation systems, stables andorchards. Respondents were slightly lesspositive about the sustainability of method-ologies such as the use of paid TA, competi-tions, prizes and bank accounts. ForCORREDOR, the answers from participantssuggest that they are involved in these newbusinesses for the long haul. It is too early toassess failure rates though the outlook ispositive (with 78 per cent saying they willcontinue). For PRODESIB (Bolivia) some 87 per cent and 100 per cent of respondentsfrom phase 1 and 2 respectively thought thatthe benefits were sustainable. It is probablethat the benefits from land titling will bemaintained as they are firmly embedded inBolivian state structures and are unlikely tobe repealed. In Bangladesh: 70-90 per centof the respondents think that the coopera-tives established under SSWRDSP are likelyor certain to continue. Under AqDP, 80 percent of members of pond aquaculturegroups and credit development groups(CDGs) think their groups will continue, yetparticipants in the lake sites are less opti-mistic and 66 per cent of women of thesesites do not expect their CDGs to continue.It also seems likely that the benefits resultingfrom ASP (Armenia) will continue in the

longer term. ASP continues many of theinterventions undertaken under its prede-cessor (NWASP), for which many projectoutputs (irrigation; agricultural credit)continue to be utilized.

2.76 But, amongst certain projects there areserious concerns about sustainability. Tworecently closed projects (NAFP, Mozambiqueand PF-ACP, Pakistan) have had a substantialimpact on poverty but in both cases sustain-ability of the net benefits is seriously inquestion. In NAFP, there are reservations,especially about the maintenance of rehabili-tated roads. In PF-ACP, the likelihood ofproject-induced benefits continuing afterclosure is a major concern. While the NGOwill continue with credit, the capacity tosustain other community needs is doubtful.Moreover, there are already reports of majorrehabilitation required for minors, withmuch drainage choked or misused. In thecase of Egypt, where the projects are facili-tating the settlement and further develop-ment of economic activities on new lands,benefits are thought likely to continue byboth beneficiaries themselves and by otherstakeholders. The concerns are that withoutthe projects continuing to act as a catalyst togovernment action services to farmers willdiminish and given the harshness of condi-tions, it is difficult to imagine some of themajor gains of the projects being sustainable.

2.77 The sustainability of institutional impact islargely unsatisfactory. Of the 17 projects thatwere rated, 59 per cent are rated as modestor less.89 Of those nine projects in the latestages of implementation or closed, 77 percent are rated as modest or less. This findingamplifies the finding on the impact ofprojects on social capital and organizationand underscores the challenges IFAD (andothers) face in developing sustainable pro-poor institutions, especially in difficult policyand institutional contexts.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

50


Innovation and replication2.78 Innovation is seen as central to the achieve-

ment of IFAD’s mandate.90 The ability toscale-up successful and replicable innovationsenhances IFAD’s value-added and has adirect impact on poverty.

2.79 IFAD defines innovation in a broad way andwhile it has aspirations to be an innovator,evidence suggests otherwise. The IEE sampleof operations clearly indicates that whilethere are a few highly innovative projects (as well as others that contain innovativeelements), many are not. IFAD’s contributionto the capture, learning, promotion andreplication of innovation also appears unsystematic and inadequate given its corporate mission.

2.80 One approach is to consider innovation in terms of what is new or different in aparticular country context (a new type ofmicrofinance organization, a new agriculturaltechnology) or what is new or different at thecommunity or village level (more commonly

understood to be technology transfer). Interms of the latter, 55 per cent of the projectsample is considered to be innovative –essentially transferring existing knowledgeabout technologies and ways of working tocommunities that have not encounteredthem before (Table 17). In terms of thewider-scale, only 25 per cent can be consid-ered innovative. This finding is supported byOE’s 2001 Evaluation of Innovation whichstates that The majority of IFAD’s innovations arenot really ‘new’, although they may be new to theproject area involved.

2.81 For many though, IFAD’s comparativeadvantage is not simply about being at thecutting edge of development; rather IFAD’sstrength lies in the promotion and applica-tion of such innovations. This can be seen interms of three broad levels of innovation: (i) creating new technologies or approachesto development; (ii) promoting new ideasand ways of working; (iii) the diffusion ofestablished technologies and approaches tonew areas.

51

Key Points:

n The likelihood of project-inducedbenefits continuing after projectclosure is modest overall, especiallywhere only late or closed projects areconsidered.

n While interventions have generallycreated new institutions, theircontinued existence is undermined byweak attention to sustainability issues.

n Focus should be more on realistic exitstrategies, longer-term support to insti-tutions, and developing projects on theback of existing institutional structures.

Table 17 An assessment of innovation

% high % substantial % modest % negligible Sample size

National innovation 15 10 30 45 20

Local innovation 15 40 35 10 20

Source: IEE ratings based on CWP evidence.

88 Seven out of ten (70 per cent) of the earlier projects arerated as substantial for sustainability of impact, with 20 per cent not rated.

89 Three out of twenty are not rated for ‘Sustainability of institutional impact’: AMSDP, the United Republic ofTanzania; BADP, Pakistan; SSWRDSP, Bangladesh.

90 IFAD V states clear objectives to strengthen IFAD’s role “as aleading source of knowledge on the eradication of ruralpoverty”, and specifically: (i) enhance its participation inpolicy dialogue and analysis; (ii) take a more structuredapproach to documentation and evaluation of field-basedinnovations; (iii) step up efforts towards building strategicpartnerships. (Desk report, paragraph 3.20).

2.82 Amongst the project sample, the Perucountry programme stands out as anoutstanding example of innovative practice.A series of four Peruvian projects (includingMARENASS and CORREDOR) are regardedas highly innovative – an assessmentsupported by the 2002 OE thematic evalua-tion on IFAD’s innovative programming. The IEE has identified some thirteen ‘inno-vations’ with regard to the Peru programme.

2.83 There are also examples of IFAD promotingnew ways of working. In Mozambique,NAFP is considered to be fairly innovativethrough the integrated approach to ruraldevelopment. In Burkina Faso, underPAMER, there are some innovations throughthe bringing together of technical trainingand market awareness. In WNRDP (Egypt),several small-scale initiatives (such as usingorganic crops to gain access to Europeanmarkets) have been innovative. In Pakistan,the PF-ACP management team was forced to improvise innovative solutions to animmensely challenging water re-routing.

2.84 But, the overall evidence from the projectsample shows that there is little to distinguishIFAD’s work from that of other developmentagencies. Much of development is aboutbringing new technologies and methods toindividuals and communities. Indeed whilesome practices are new to particular locationsand villages, the practices themselves areoften not especially innovative. Examplesinclude projects in Bangladesh, BurkinaFaso, Guinea and Pakistan.

2.85 And in some country contexts, virtually anydevelopment initiative can be described as‘innovative’. Settlers in Egypt, the post-Soviet context in Armenia and post conflict in Mozambique are all examples from the sample.

2.86 Despite a few highly innovative projects, plusothers which contain and promote innovativeelements, IFAD does not contribute to thecreation, promotion, replication and lessonlearning in any systematic way. The OEEvaluation of Innovation (paragraph 13)supports this view: The current approach to

innovation is individualized, decentralized andunsystematic, and is determined by individual andchance factors rather than by a well-defined andacceptable sequential process. While innovations dotake place, this fragmented, ad hoc style does notlend itself to good use of IFAD resources.Knowledge management should perform akey role in learning from and promotinginnovations, yet this is an aspect that the OEInnovation Evaluation considered has notreached its full potential.91 A more system-atic approach would have certain keyfeatures. Firstly, a link to a knowledgemanagement system in which lessons fromIFAD and other sources are identified anddisseminated; secondly, recognition of inno-vation in project designs by inclusion as anobjective, with associated arrangements forflexibility, risk-taking and evaluation; andthirdly, the identification of potentialpartners for scaling-up at the commencementof project implementation, to foster owner-ship and shared learning.

2.87 A new IMI has been started and a key paperwas presented at the December 2004 EB. The approach acknowledges that changes inculture and learning are necessary elements of success. The IEE welcomes the draftdocument which appears to recognize manyof the underlying issues identified in this eval-uation. The approach would be strengthenedif the text had clearer links to core initiativessuch as the new HR policy, to demonstratehow changes in HR will support innovation.The approach would also benefit from a moredetailed presentation of the behaviouralchanges that are needed to achieve theoutcomes, and a set of objective monitorableindicators of change and outcomes.

Concluding remarks2.88 The project portfolio as a whole is broadly

pro-poor, but the evidence demonstratesthat the overall range of IFAD’s investmentslacks strategic coherence: past resource allo-cation has been based on a limited assess-ment of regional and country needs, and adhoc assessments of borrower performance;the PBAS being introduced may improvethis, but IFAD has a very narrow range ofinstruments with which to engage with

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

52


countries with weak institutional and policyenvironments; and the current strategicframework has fostered a broadening ofproject components in support of empower-ment objectives, but with a correspondingincrease in the diversity of skills needed tosupport the portfolio.

2.89 The impact of IFAD’s projects is variable, asit is in most development organizations, butwith a little under half of all projects fallingbelow expected levels of poverty impactthere is a need for better performance. Thesample points to several key lessons that arelinked to achieving higher impact overall:

n A strong demand-driven approach, whereproject design and management are attimes subordinate to community demands

n Design that is adapted to reach poorer groups

n The CPM is a strong driving force, along-side a stable group of consultants andlocally embedded associates

n Close support by IFAD during implementation

n Reliable impact data, to substantiatesuccess/failures and learn for future interventions

n Where appropriate, the inclusion of healthand education interventions especiallycommunity-led potable water supply

n Building on existing institutional struc-tures, wherever they can be efficiently used to further project objectives

n Planning for sustainability

2.90 Lessons from projects that have notperformed so well underscore the impor-tance of these success factors:

n Problems start with poor project design, in particular weak institutional analysis,especially of implementation capacity,which leads to ineffective arrangements for project management; a lack of atten-tion to design for sustainability, especiallyto foster new institutions; and a lack ofclear performance indicators and targetsto monitor performance.

n Economic analysis of costs and benefits hasnot kept pace with project designs andlittle use is made of ex post analysis tolearn from actual performance.

n Targeting falls below the aspirations setout in project design documents, andhighlights a policy vacuum in IFAD toprovide guidance and procedures oneffective mechanisms.

n And targeting, like other implementationissues, is relatively neglected duringimplementation as a result of the arms-length and distant relationship IFADexperiences in some countries, workingthrough a contracted cooperating institu-tion for project supervision. Technicalproblems are dealt with slowly, oftendelaying change until mid-term. Such asluggish response runs counter to theevaluative, learning, flexible approachneeded to manage innovative projects.

n COSOPs have been slow to achieve thehoped-for move into country programmes,and policy influence has not yet materialized, either at country or international levels.

2.91 Finding valid comparisons with other IFIs isdifficult owing to variations in terminology,definitions and operational practice. But onthe best data available, IFAD lags its comparators on measures of projects at risk,time to project effectiveness, and quality of supervision.

53

91 The OE Indonesia CPE also highlights that: In order todevelop programmes at the cutting edge of rural develop-ment and to fulfill its mandate effectively, IFAD must bemore responsive and innovative… Stronger analysis ofknown technologies during project design and criticallearning from IFAD’s own experience (written and oral)would greatly enhance IFAD’s approach, its programmeperformance and its credibility with development partners.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

54


2.92 The analysis of portfolio and projectperformance, and project impact, highlightsthose factors that have stimulated impact andwhere better performance might lead tosuccess. Many of the issues identified such ascountry strategy, project designs, targeting,economic analysis, sustainability and theprocesses of implementation support andsupervision are within the scope of IFAD’soperational procedures and management.The ways in which IFAD has approachedthese issues and the implications forimproving the Fund’s performance areexamined next in Chapter 3.

55

3.1 In order to understand the reasons whydevelopment performance has been as variedas described in Chapter 2, the IEE examinedthe ways in which IFAD works to deliver loansand grants. Corporate processes werereviewed in the IEE inception report and aschematic presentation was used as a basis fordefining those processes to be studied in thisevaluation (IR, Figure 1, page 19). Threebroad processes were identified: developmentof policy, development of strategy and theproject cycle processes to deliver loans andgrants.92 These are supported by corporatefunctions such as human resource manage-ment, administration and financial manage-ment. The way of working with theseprocesses is sometimes described as IFAD’sbusiness model, and that shorthand is used in this chapter.

3.2 IFAD developed an efficient business model totransfer resources through targeted projects.93

The original approach was a narrow focus onfood production and nutrition, targetedmainly at the poorest groups. After an initialperiod when IFAD only financed projectsidentified and designed by others, it led to away of working that followed the classicproject cycle to provide loan and grantfinance for stand-alone investment projects toincrease food production and nutritionamong poor rural people in defined locations.The model can be characterized as ‘the enclave

production project’ and was recognizable assimilar in many respects to the work of otherIFIs at the time, though they did not targetthe poorest groups.

3.3 Detailed analysis of country situation tookplace through general identification missionsto identify stand-alone projects. Links betweenIFAD and the country were handled by aproject controller (now CPM) in a close rela-tionship with a government department, mostcommonly agriculture. The projects wereformulated by teams of consultants, orsubcontracted to agencies such as FAO. Insome instances, IFAD provided cofinancing toprojects identified by other donors, orengaged in joint identification missions.Implementation was by project management

Corporate processes: managementperformance

Chapter 3

92 The IEE uses the term process to mean a sequence of activi-ties with a defined input and output that contributes to anoverall objective. Complex functions such as the design andsupervision of a project are a collection of numerous sub-processes.

93 A ‘business model’ describes how an organization definesits products, how it selects its clients, how it promotes andmarkets its products and services, and how it configures itsresources. There are no fixed conventions about how themodel is described. It can be expressed as a set ofprocesses or in simpler language such as the ‘bait andhook’ approach e.g. computer printers (cheap bait) and inkcartridge refills (expensive hook). IFAD adopted the terms‘operating models’ and ‘business cases’ under the processre-engineering programme design stage.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

56

C O R P O R AT E P R O C E S S E S : M A N A G E M E N T P E R F O R M A N C E

unit organizations, mostly staffed by regulargovernment personnel. Supervision wasundertaken by CIs. A cornerstone of theapproach was the freedom of action grantedto CPMs, under which they controlled therelationship with the country, the identifica-tion of projects, the technical design process,and the relationships with government andthe cooperating institution during implemen-tation. The role is characterized by the IEE asa ‘free-agent’ CPM.

3.4 Figure 8 sets out a simplified illustration of ageneric business model prior to the first re-engineering, as interpreted by the IEE. The keyfeatures are that the model was primarily a wayof managing the project cycle. The projectcontroller (now CPM) took a central role inorchestrating the process and managed workthrough consultants and the CI.

3.5 Several key strengths underpin this model: arelatively low-cost approach, in keeping withthe original mandate to provide complemen-tary finance; high flexibility for CPMs; andbuilding on the technical skills of consultantsand partner agencies. But the low-cost, arms-length, highly individual format was,with hindsight, ill-suited to respond to new challenges.

3.6 The working environment for developmentorganizations at country level has changed.IFAD’s niche area has become crowded withother lending donors moving into povertyand rural poverty (if not agriculture per se),together with an expansion of grant-aid bilat-eral and NGO donors with strong capabilitiesin the areas of participation, communitydriven development and empowerment.94

Development planning has also changedunder several influences: the move toprogrammes rather than projects, develop-ment of national sectoral plans, and donorharmonization initiatives coalescing aroundnationally-driven, multi-sectoral programmesdeveloped under poverty reduction strategies.IFAD’s cherished specificity of working withand through government is now claimed asnormal good practice by most organizations.

3.7 These external shifts challenge the IFADbusiness model by altering the focus ofengagement away from a bilateral exchangebetween IFAD and the government towards a country-driven multilateral process withnumerous and diverse participants.

3.8 At the same time, IFAD’s own changingstrategy and processes have prompted newapproaches. Objectives to catalyse, be a leader,

ImplementationNegotiationEB Approval

CounterpartGovernmentDepartment

CooperatingInstitution

CounterpartGovernmentDepartment

Evaluation

Mid-termReview

ProjectController

(CPM)

SpecialProgramming

Mission

Project Formulation

AppraisalConsultants Consultants

OSC/TRC

Consultants

Consultants

Figure 8 Original (simplified) business model

57

influence policy and promote innovation allrequire skills to gather information, synthe-size, learn and disseminate, work withpartners and scale up. The strategic frame-work 2002-2006 is a significant broadening of IFAD’s original mandate, implying both achange in how things are done (more policydialogue, and institutional orientation) and in the composition of investments (less agri-cultural production, more on organizations,small and medium enterprises, microfinance,market access etc.). All these developmentshave been prompted by clear evidence ofresponsiveness, both by management and the EB, to the changing environment.

3.9 These changes imply a shift to a new businessmodel to generate ‘learning and replicatingempowerment programmes’. Such a modelwould develop loan and grant financeprogrammes targeted at nationally-deter-mined poverty reduction priorities, with astrong element of learning and innovation,linked to scaling-up through work withpartners. Figure 9 presents the IEE interpre-tation of the elements of a business model thatbrings together the shifts in the external envi-ronment and IFAD’s own changing strategyand processes. The CPM is still dominant, butno longer in a position to play a single central

role. They are shown in a variety of differentpositions in the figure, corresponding todistinct tasks. These tasks demand a higherdegree of interaction with a wider range ofgovernment departments and with otherdonors working in rural areas at countrylevel, the development of a country strategythat brings together loans, grants and policydialogue, and the forging of a coherentprogramme to support innovation and scalingup. The CPM has an increasingly fracturedrole to play, demanding a range of skills fromstrategy development, through project design,to policy dialogue. One result has been anexpansion in reliance on consultants (notshown in the figure to avoid over-crowding,but active in most areas where arrows link text).

3.10 During the period under review a series ofreforms and initiatives have been under-taken, some at the prompting of the Board,many at the behest of management, todevelop the Fund. Table 18 lists the key elements.

Independent Evaluation

Quality AssuranceSupervision

Self Evaluation

COSOP

LOANS GRANTS

POLICYDIALOGUE

CPM

CPM

CPMCPM

CPM

IFAD Policydevelopment

Strategic Framework (SF)& Regional Strategies –

giving direction

Participatorycountry analysis

interpretation of SF& Regional Strategies

Alignment withNational Policies &

donor harmonisation

Sector &poverty analysis

Implementationsupport

Innovation

Scaling-upPartners &

Co-financingResources

Pro-poorrural

interventions

Impact AnalysisMDG impact

Portfolio AnalysisKnowledge Management

Learning

Figure 9 A new business model responsive to the changing demands on IFAD

94 The involvement of more organizations does not imply that resources for rural development are in any way over-subscribed.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

58


Table 18 Key events in the development of operational processes since 1993

Date Initiative and policy development

1992 IFAD IV

October 1993 Report of PMD task force on rationalization of the project cycle

Cost of project development had risen to USD 575 359 in 1990-92.

n reduced the stages of project design from 5 to 3; initiated the Project Development Fund; proposed to reduce project development costs to USD 450 000

January 1994 New operational procedures announced:

n Inception paper replacing general identification missions

n Project development team (PDT) introduced including CPM, PT and OE, with Finance and Administration Department and the Office of the General Counsel joining at appraisal stage

n Beneficiary involvement in formulation emphasized, leading to creation of in-country resource groups in some countries

n Formal TRC review process, with post-appraisal memo indicating compliance

n Appraisal to emphasize implementation support

June 1995 Project cycle re-engineering working group report:

n Seeking to reduce costs within zero-growth budget

n List of current difficulties, including: PDT performance; technical quality assurance and implementation support

n Recommendations include: new portfolio management system; COSOP (originally conceived as a strategy paper with a small budget of USD 20 000, but in practice acting as project pipeline identification); strengthened PDT function and in-country resource group

n Proposal to further reduce project development costs from the then-current USD 433 000 to USD 315 990 and release funds for early implementation support (Senior management did not approve this release.)

July 1995 Re-engineered project cycle - pilot phase including PDT, TRC and OSC stages

December 1995 Report of information systems re-engineering working group report

1997 Direct supervision pilot programme (approved by the GC)

January 1998 Mainstreaming of re-engineered project cycle, including logframe training for 45 staff

December 1998 Impact achievement in the project cycle (IAPC) group formed

1998-99 Continuing internal concern that PDT mechanism is not functioning effectively

June 1998 PMD instruction: PDT established as a mechanism running from inception to EB approval

June 1999 Pilot thematic groups established (livestock, rural finance, project design and implementation, microenterprises)

Late 1990's on OSC meetings increasingly seen to duplicate previous TRCs, with declining senior management attendance

2000 IFAD V

IAPC Report recommendations including: strengthened PDT to run from inception, through EB approval to project completion, including external expertise; increased direct IFAD implemen-tation support; key file based on the logframe; start-up workshop based on participatorylogframing

New approach to evaluation leading to new OE evaluation processes introduced; all evaluations to have agreement at completion point/understanding at completion point and core learning partnership

Rural finance policy

Process re-engineering programme (PRP)

Eight processes were reviewed: (1) human resources; (2) strategy and finance; (3) support services; (4) information technology; (5) knowledge management; (6) impact management; (7) product development; (8) partnerships management

Items (1) to (5) went to the EB in December 2000. Resulted in an allocation of a budget of USD 26 million: USD 1.0 million for design of the processes; USD 15.5 million for the five business cases approved by the EB; USD 9.5 million remains available, use to be decided

59

Table 18 (continued) Key events in the development of operational processes since 1993

2001 President Båge assumes office

PRP renamed Strategic Change Programme

OE introduced ARRI

April 2001 ‘Participatory Approaches for an Impact-Oriented Project Cycle’ published, following internal discussions around IAPC report

June 2001 Project design document and key file introduced

December 2001 Analysis of project development costs for the PDFF identified average budget allocations for project development of USD 335 000

2002 Rural finance decision tools

Managing Impact for Rural Development – a Guide for Project M&E

2003 IFAD VI

2003 PDD and key file revised

Three thematic groups formed to collate information from PDT meetings

Management teams: SMT, IMT and PMDMT instigated to improve communications, harness skills, and promote efficiency and effectiveness

EB approval of Field Presence Pilot Programme (FPPP); Initiative for Mainstreaming Innovation

Rural Enterprise Policy; Mainstreaming Gender Plan of Action 2003-2006

Grant Financing Policy

Sourcebook on Pro-Poor Institutional and Organizational Analysis and Change

OE became independent of management; EB approved IFAD evaluation policy

2004 ongoing Human resources policy approved by EB

Discussions underway to replace OSC approval stage with project design assessments to go in batches to president's policy forum meetings

First CPM forum held in April

Thematic group/learning approach extended through 'learning notes' covering 18 topic areas

Development of detailed proposals for PBAS and RIMS

Source: IEE DRR Table 6

3.11 The range of issues and continuity of changeis testament to an underlying recognition ofthe need to improve performance. Analysisin the desk review concluded that reformsacross a number of areas: the knowledgesystem; strategy development; project cyclemanagement; quality assurance; supervision;and the policy framework, have not yetsucceeded in improving the Fund’s perform-ance to the levels sought by those initiatives.95

This chapter examines the current situation,major initiatives and changes to corporateprocesses and explains why IFAD’s perform-ance has not lived up to expectations.

3.12 To help readers appreciate how IFADcompares with other IFIs, Table 19 sets outsome comparative statistics about staffingand administrative costs. Owing to differences in the ways staff are classified and

information is reported, these data should beinterpreted as broad comparisons.

A framework to assess howIFAD’s results are derivedfrom corporate performance

3.13 Figure 10 sets out a performance frame-work that is used to structure the discus-sion about corporate processes. The modelis simple and identifies the role of leader-ship to define and motivate policy and

95 DRR paras 5.12 to 5.42

strategy, human resources, partnerships andother resources.110 These act through theinstitutions’ processes to deliver impact. Thequality of these business processes is centralto the second fundamental question of thisevaluation, Are the skills and resources of IFADused in the best possible way, given IFAD’s overar-ching goal of supporting rural development andhelping countries eradicate rural poverty?111

3.14 In the early years of the Fund, processesfocused on development of loans and grants,with procedures to manage identification,formulation, appraisal, approval, implementa-tion, supervision, closure and evaluation.Since 1994 corporate, regional and countrystrategies have superseded identificationmissions, and the broader objectives of being a catalyst, providing leadership, andpromoting learning and innovation for subse-quent scaling-up have brought new elementsto the business model. The components of the framework are examined in turn.

Leadership and governance112

3.15 Governance arrangements are similar to thepractice of other comparable organizations.IFAD is managed by a President who actsboth as Chairman of the Executive Boardand chief executive. This role combines theoutward-looking tasks of strategic orientationand external relations with those of day today management (such as chairing the OSC). The President is assisted by a Vice President(currently with main responsibilities to assist in day-to-day management of the Fund anddeputize for the President)113 and threeAssistant Presidents who head the ExternalAffairs, Finance and Administration, andProgramme Management Departments.

3.16 The primary governance structure of IFADis the GC in which each of the 163 MemberStates is represented. The Council hasultimate responsibility for key matters suchas approval of new membership, appoint-ment of the President, approval of the

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

60


Table 19 IFI comparisons of administrative expenses and human resources

IFAD World Bank AfDB96 AsDB IADB

IBRD IDA

Overheads:

Admin. expenses (USD million) 97 42.3 882 220 252.6 372.5

Percentage of total programme 98 8.8% 7.3% 7.4% 3.8% 5.1%

Human resources:

Number of professional staff 99 202 620 836 1 396

Number of administrative staff 100 264 404 1 119 516

Number of ‘other’ staff 101 n/a 4 356 n/a

TOTAL 102 466 8 800 1 028 2 311 1 912

Percentage professional staff 43% 60% 36% 73%

Percentage administrative staff 103 57% 29% 39% 48% 27%

Staff diversity:

Number of women staff 298

Percentage of total staff 104 64% 52%

No. of women professional staff 105 83 241 531

Percentage of total staff 106 18% 24% 25% 29% 38%

Country presence:

Number of country offices 107 100 7 24 28

No. of staff located in field offices 108 16 3 000 362 540

Percentage of total staff 109 3% 30% 16% 28%

Unless otherwise stated figures are based on the Annual Report 2003 for the World Bank, AsDB and IADB and the Annual Report2002 for AfDB.

61

Leadership & Governance

Processes &Instruments Impact

Policy & Strategy

People

Partnerships & Resources

Policy dialogue, leadership, learning & innovation

Enablers Results

Figure 10 Institutional performance framework

96 Figures are for the AfDB group.97 Administrative expenses (USD million). For IADB, this is

the total administrative expenses, before reimbursementfrom funds in administration, and additions for capitalincreases. Sources: World Bank (IBRD), p38; AfDB, website;AsDB, p27; IADB, p92. IFAD Annual Report 2002.

98 Percentage of total programme (administrativeexpenses). This is calculated as: administrative expensesdivided by (total loan and grant programme plus administra-tive expenses).

99 Number of professional staff. IFAD figures from IEEhuman resources report Table 2. Sources: AfDB, p56; AsDB,p27 & 35; IADB, p93 and website.

100 Number of administrative staff. For IFAD, these are cate-gorized as ‘General Service category’. For AsDB, this istermed ‘General Services staff’. For AsDB, this includes 13staff on special leave without pay (AsDB 2004: 35).Sources: AfDB, p56; AsDB, p27 & 35; IADB, website.

101 Number of ‘other’ staff. AsDB, these are categorized as‘national officers’. Sources: AsDB, p27.

102 TOTAL (number of staff). Sources: World Bank, p135;AsDB, p27, IADB, website.

103 Percentage administrative staff. All figures calculatedbased on figures given in the Annual Reports and on thewebsite (IADB only), except for the World Bank. Sources:World Bank, p135.

104 Percentage of total staff (women). Sources: WorldBank, p135.

105 No. of women professional staff. Sources: IADB, p93.

106 Percentage of total staff (women professionals). Forthe World Bank and AfDB, the percentages are given in theannual reports – though for AfDB the figure on page 56 isstated as, “25 per cent of job offers being made to femaleprofessionals”, rather than actually employed staff.Percentages for AsDB and IADB are calculated based onactual figures given in their Annual Reports. Sources: World Bank, 135; AfDB, pxxii and 56.

107 Number of Country offices. For AsDB, this includesresident missions (16) plus regional missions (1), countryoffices (1), representative offices (3), a special office (1) and extended missions (2). Sources: World Bank, p135;AfDB website; AsDB, website; IADB, website.

108 Number of staff located in field offices. For IFAD 15liaison officers and one CPM. For the World Bank, this is an approximate figure. For AsDB, this is the balance fromfigures given in the Annual Report. For AsDB, this includes:69 professional staff and 293 local staff. Sources: WorldBank, website; AsDB, p35; IADB, p93.

109 Percentage of total staff (located in field offices). Allfigures calculated except for World Bank where percentageis given in Annual Report. Sources: World Bank, p135.

110 The model is derived from the European Foundation forQuality Management Excellence Model, adapted for publicand voluntary sector organizations, with terminology appro-priate to IFAD. It is a simplification of the diagram used inthe inception report. (www.quality-foundation.co.uk)

111 IEE terms of reference. The other fundamental question wasIs IFAD properly focused on its rural development mission?

112 This section draws on a separate IEE paper titled‘Governance and Institutional Issues’

113 In practice the Vice President’s role is wide-ranging.

administrative budget, and adoption ofbroad policies etc. The role of the GC withregard to policy and the budget is signifi-cant, but in other respects it is the EB that isresponsible for executive management. TheExecutive Board is made of up of 18 electedand 18 alternate members. It meets threetimes a year for two days.

3.17 Other arrangements contribute to goodgovernance. A new policy on disclosure in2000 has helped increase accountability andtransparency. Internal controls include IA andan Oversight Committee, though both arerelatively recent functions. The 2003 policy tocreate an independent OE and a new evalua-tion policy are both positive contributions. But

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

62


weaknesses exist in the operation of thesearrangements. The Audit Committee notedconcerns about a low level of implementationof IA recommendations in 2002; a situationthat had improved by the end of 2004; theoversight committee is essentially passiveresponding to allegations and moves toexpand the remit of the EC to include IFAD’spolicies have been resisted by management.

3.18 IFAD V and VI have been the main drivers ofpolicy change during the evaluation period.In practice the replenishment process hasbecome a major driver of internal policychange within IFAD (see Box 9). IFAD IV, setagainst a background of declining resourcesand a major restructuring of member votingrights, marked the first real engagement byMember States in discussions about IFADstrategy and policy. Previous replenishmentshad focused almost exclusively on financialpledges. IFAD V intensified the focus in theform of the IFAD V: Action Plan, which forthe first time put in place a framework formonitoring actions to enhance IFAD opera-tional processes. The plan of action sparkedsome important internal initiatives, includinga renewed emphasis on country strategy andthe gender action plan, but the consultationprocess also proved protracted and theresulting matrix of actions unwieldy andlacking clear prioritization.

3.19 IFAD VI began with a tighter, bettermanaged process, but the late tabling ofissues by at least one member of List A to theconsultation left IFAD management essen-tially on the back foot and facing a signifi-

cant new policy agenda – including theintroduction of PBAS, a new evaluationpolicy, RIMS and the piloting of IFAD fieldpresence. The new agenda seeks to bringIFAD into line with other multilaterals, at atime when donors have competing claims ontheir multilateral contributions and thepressure to demonstrate results is growing.IFAD has responded ably to the IFAD VIagenda, but the process continues to absorbconsiderable staff and management time,and past experience suggests that a periodof consolidation may be required to ensurethat the full impact of the latest initiatives isfelt before moving on in new policy direc-tions. Both IFAD V and IFAD VIReplenishments posed wide-ranging changeagendas for IFAD with only limited attentionto prioritization and costing. This will be acritical challenge for the next replenishmentalong with a focus on IFAD’s medium tolong term development effectiveness and itsstrategic niche within a changing interna-tional development context.

3.20 A crowded agenda and infrequent meetingslimits effective decision-making. EB meetingsare far fewer in number and shorter induration than for comparable organizations.114

The agenda is crowded with large volumes ofwritten material often provided at short notice,especially in translated versions. Directors haveto prioritize issues they wish to concentrate on.Some rely on the skills of their colleagues intechnical areas such as finance or audit. Someseek guidance from their governments. Butfew members have the resources to call onmuch support to review Board documents.

Box 9 The replenishment consultation process

A consultation group of IFAD Member States is established by the GC to run the replenishment process. The group

comprises 100% of List A and B representatives and 12 representatives from List C (four per sub-list). The group is

chaired by the President. The formal process begins with a plenary of the consultation group in which IFAD manage-

ment presents a position paper on key policy issues/areas for the forthcoming replenishment. The paper is discussed

by all participants of the process. The President then sits with the various convenors of Lists, plus some invited

‘friends’ from the Board and a few key staff and senior managers for informal consultation about overall direction

for the next replenishment. This can be a fairly protracted process, involving several stages. As consensus is reached

in the consultation group a report is prepared. The terms of the forthcoming replenishment are encapsulated in the

report and in a resolution setting out the necessary financial and voting mechanisms. This is then passed by the GC.

63

3.21 Board members differ in the experience, skillsand training that they bring to the role andthere are no terms of reference for the post.Experience of other boards by directors ismore likely to be connected with UN organi-zations than with IFIs, which suggests that feware familiar with the ways other IFIs havemeasured performance and acted to improvedevelopment effectiveness.

3.22 Scrutiny of key functions such as the annualprogramme of work and budget, and of eval-uations, is delegated to the Audit andEvaluation committees respectively. The workof the Audit Committee covers the financialaspects of the budget but has only started toinclude the work programme as such sinceApril 2004. The EC examines OE’s outputs,but prior to December 2004 did not reviewthe Fund’s policies or self evaluationproducts. Proposals were being developed atthe time of this evaluation to expand the ECremit. But during the period under evalua-tion, both committees had significant gaps intheir coverage.

3.23 The number of agenda items leaves little timefor effective debate. Directors have a widerange of interests. Some want to discuss loansand grants in detail; others devote moreattention to policies. The crowded agendaresults in too little time for effective debate onall topics. New policies are sometimesdiscussed at informal Board seminars thatprecede EB sessions, and these create betteropportunities for discussion. But the currentagenda includes approvals of all loans andgrants and review of COSOPs, which havereached a stage where the Board is unable toinfluence content or direction. Some directorsdescribe these discussions and decisions as perfunctory.

3.24 Current EB arrangements do not fosterresults-based scrutiny of development effec-tiveness. There has been a long tradition ofevaluation in IFAD and the work of OE isformally scrutinized through the EC. Butthe focus on evaluation has left other aspectsof development effectiveness untouched.The IEE views the term development effec-tiveness as embracing the efficiency of devel-

opment of new loans, grants, policies andother instruments, the effectiveness of theirimplementation, and their developmentoutcomes. There are no procedures toanalyse quality of the designs of loans,grants and COSOPs, and all self-evaluationby management is currently outside thescope of the Evaluation Committee.

3.25 Proposals to amend the terms of reference(TOR) and rules of procedure of the EC inpreparation for the December 2004 EB wereintended to broaden the scope of the EC inline with experience from other IFIs.Changes would result in the EC reviewingoperational policies, to ensure the feedbackbetween lesson learning and adoption isreflected in policies; and reviewing manage-ment’s self-evaluation reports such as theportfolio review of project performance(PRPP) and any future revisions to RIMS. Afurther proposal to de-link the timing of theEC from the EB would permit more time forrecommendations by the EC to be taken intoaccount by Board members.

3.26 The combined effects of a Board with a tightschedule of meetings with crowded agendaand limited analysis of the Fund’s activities hasbeen to leave directors with an inadequatefactual basis on which to manage IFAD’sperformance. As a result, debate aboutreforms is driven by the policy orientation ofthe replenishment exercises, without the foun-dation of clear analysis of current effective-ness. There is a lack of consensus within theEB about the development effectiveness andefficiency of IFAD and the nature of reformthat is required. This has contributed to

114 The Board of the World Bank is in continuous session; theBoard of Directors of AsDB met for 48 formal and 24informal sessions in 2003 (AsDB Board of Directors AnnualReport 2003); the Board of the AfDB met for 43 formalsessions, 26 informal and 24 committee sessions in 2003(AfDB Annual Report 2003).

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

64


strong differences of opinion about the poten-tial efficacy of PBAS and RIMS. The relevanceof both initiatives is questioned by manymembers and are seen as responding to theinterests of a narrow group of countries.

3.27 A few changes offer the potential for a largeimprovement in governance. There is a clearneed to improve the orientation of the workof the EB and enable the Board to lead in thepursuit of performance. Strategic direction,policy and development performance need toreceive a greater share of time and attention.This could be done through changes to theduration of meetings and the terms of refer-ence of the Audit and Evaluation Committees.Proposals are made in Chapter 4.

Policy and strategy development

3.28 IFAD’s development of policies has laggedbehind the broadened mandate. IFAD has fewoperational or sectoral policies and relativelyfew policies governing internal processes andprocedures, compared to other IFIs (sixthematic/sectoral, and four operational policypapers were produced between 1994-2003plus a series of position/discussion papers andinformal policy statements).

3.29 At various times there have been efforts toexpand the coverage of operational policypapers – the EB paper from December 1998entitled Planned Policy Papers noted a seriesof operational policy issues (good governance;recurrent and local costs) that should beaddressed. More discussion with the EB wascalled for. However, there was no tangiblefollow-up action. Since then policy develop-ment has been variously handled by differentunits, ED (now subsumed within ExternalAffairs Department (EAD) and the Technical

Advisory Division (PT). A new Policy Divisionwas established in 2003 to facilitate a moresystematic process of policy developmentacross the organization.115

3.30 A core issue appears to be the preponderanceof informal policy products. For example, thediscussion paper on transforming rural insti-tutions prepared for the 2003 GC meeting isrelevant to operationalization of the strategicframework, but its status as a statement ofIFAD policy is unclear. Operational policiescover a number of important areas of IFADactivity – links with HIPC, the new policy ongrant financing and the FLM – but againcover relatively limited territory compared toother IFIs.

3.31 The few policy products that have been devel-oped are rarely communicated widely beyondRome. Few corporate policy documents areknown or made available to country projectstaff. During the country visits it emerged thatproject staff on the United Republic ofTanzania RFSP and PF-ACP Pakistan knewnothing about the rural finance policy ordecision tools, nor were they aware of IFAD’sattempts to create a centre of excellence inrural micro-finance. The M&E guide has notbeen widely disseminated across ‘virtual IFAD’.Efforts to communicate across projects and viavirtual networks such as FIDAFRIQUE havebrought some gains but not in terms of explicitpolicy guidance. Experiences are similaramong partner agencies, where staff at head-quarters, working both in operational andsupport units, consider themselves to be uninformed about IFAD’s policies.

3.32 Resistance to adopting a normative style of working has inhibited development ofcomparative advantage. The 1994 REA identified the problem of the Fund being toodiffuse, in their telling judgement ‘there areas many IFADs as there are projectcontrollers’.116 But their recommendationshave not taken hold. Implicit approaches topolicy development have dominated withlesson learning about policy issues concen-trated at the level of individual CPMs or PTstaff. A widespread resistance to normativemodels and ways of working has limited

Key Point:

The current arrangements for governancemeet the basic requirements of the Fund,but a crowded agenda, infrequent andshort duration of meetings reduce the EB’seffectiveness as an executive unit.

65

policy development in IFAD. Staff are morecomfortable with the idea of sui generisapproaches. To a large extent that reflects acontinued adherence to the free-agent CPMmodel. CPMs are responsible for ensuringthe application of policies in the projectdevelopment process and as a basis for policydialogue at national level, but IFAD has sofew of these types of policies that there arelarge areas where staff must operate without policy guidance.

3.33 In the face of resistance to conformity, thePDT/TRC/OSC process has had mixed successin ensuring that policy guidance, where itexists, is consistently applied. The OSC inparticular has not performed this roleadequately. And once projects move intoimplementation, IFAD is dependent onsupport from their cooperating institutions.Monitoring the application of policies underimplementation should be the primaryresponsibility of CIs but CIs rarely performthis role and, more fundamentally, are rarelyaware of IFAD’s policies. A wider range ofinstruments and more effective challenges toproject designs are needed to improve consis-tency and raise the quality of IFAD’s products.

3.34 The IFAD Strategic Framework 2002-2006provides a clear sense of mission but is toopermissive to guide targeting and selectivity.The current strategic framework is the thirdstrategic product since 1994. IFAD Vision1995 and strategic framework 1998-2000 setout ‘new’ ways of working but did not presenta clear rationale to guide IFAD investments orstrategic objectives to bring greater coherenceand impact to the portfolio.

3.35 The strategic framework 2002-2006 providesa clearer sense of mission and the threestrategic objectives provide staff with acommon reference and a clear basis forcommunication both internally and externally.The content reflects international thinking onthe role of institutional change for long termpoverty reduction; strengthening social capitaland empowerment of the poor and influ-encing public policy better to reflect the needsof the rural poor. It does not, however, makeclear operational links to the MDGs, to the

provision of global public goods or how IFADwill engage in different institutional/policyenvironments or with national policyprocesses such as PRSPs. There is no clearsignal regarding IFAD’s primary target group– i.e. is it the rural poor, the productive poor,the poorest of the poor, the vulnerable. etc. Inthese omissions it fails to provide a clear guideto greater operational selectivity or increaseddevelopment effectiveness. The content islargely permissive, as is the reference toIFAD’s main target group.

3.36 IFAD is striving to be an innovator, a knowl-edge institution, a catalyst and a leader. Thesenew ambitions bring new pressures to theproject cycle. Instead of developing stand-alone projects, the intention is that CPMsfoster leadership at country level and amongpartners, use knowledge generated fromwithin the programme to find innovative solutions, and catalyse the work of partners to scale-up promising interventions. But thereare a limited number of professional staff oradministrative resources available for policy-related work and the transactions costs arehigh in such a non-conformist environment.CPM numbers have not changed in the pastdecade. The multiple demands on CPM timemake it difficult to fully engage with Romebased or country level policy initiatives, andthe project cycle processing stages have notchanged to reflect these emerging roles.

Human resources management3.37 Human resources are central to the success of

a knowledge based organization such as IFAD.Management of these resources impactsdirectly on the success of the organization inachieving its objectives. This section summa-rizes findings from the IEE human resource

115 There are as yet no clear terms of reference for the PolicyDivision (EO) and its relationship with PMD is not yetclearly defined.

116 Head et al (1994) page 12 for the criticism about arti-sanal approaches, paucity of operational guidelines andresulting diversity of practices; page 13 for recommenda-tion to become more programmatic and strategic.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

66


report. It presents trends in numbers of staffand consultants over the review period,discusses the ways in which IFAD’s HR aremanaged, examines aspects of the corporateculture and the need for change in thatculture, and assesses the impact on the effec-tiveness of the organization. It is based on areview of relevant documentation related tohuman resource management in IFAD, theInternational Civil Service Commission andother UN organizations data from the HRDivision, unstructured interviews with staff atall levels, and a systematic questionnaire tostaff to ensure that views expressed in inter-views were representative; over two thirds ofall staff responded to the questionnaire.

3.38 Staff and consultant numbers have increasedsignificantly over the period 1994 – 2003.IFAD was established as a relatively smallorganization with a deliberate intention toconstrain organizational growth. However,between 1994 and 2003 staff and consultant

numbers increased by 43 per cent.117 Therehave been increases in professional staff,support staff and consultants in all depart-ments. The increase in staff and consultancieshas not been accompanied by an increase inthe number of loans over the same periodalthough operational changes such as thedirect supervision pilot have brought a smallincrease in demand for consultants, so asimple estimate suggests that that efficiencyhas decreased significantly from 15 staff andconsultant years per loan in 1994. There hasbeen an increase in the number of grants,and a change in the nature of IFAD’sapproach to operations with a wider focus onregional strategies, policy dialogue, directsupervision and implementation support, andbroader stakeholder management. However,much of this change has mainly affectedoperations in PMD, impacting in particularon CPMs. The number of CPMs hasremained constant at 35 from 1994 to 2003.The inclusion of OE in the Office of thePresident has resulted in a significantincrease in staff and consultants in the Officeof the President and Vice President, but it isless easy to see why staff and consultantnumbers have risen so dramatically in EADand the Finance and AdministrationDepartment (FAD), by 73 per cent and 45 per cent respectively, even with thedemands of the SCP and other initiativesintroduced at the request of the Board.

3.39 Increases in staff and consultant numbers havebeen financed from both the regular adminis-trative budget and a variety of extra-budgetaryresources and supplementary funds. Datapresented by FAD show that between 1994and 2003 there was a 12 per cent increase inregular posts financed from the administrativebudget. All other increases in both temporarystaff and consultants were financed fromextra-budgetary resources or supplementaryfunds provided mainly by Denmark, Finland,France, Italy, Japan, the Netherlands and theUnited Kingdom, for associate professionalofficers or specific activities.

3.40 The profile of staff and consultants in IFADshows some important characteristics. First,there has been a significant improvement in

Key Points:

n IFAD has few operational or sectoralpolicies and relatively few policiesgoverning internal processes and procedures.

n Staff are more comfortable with theidea of sui generis approaches andthere has been a widespread resistanceto normative models.

n The IFAD Strategic Framework 2002-2006 provides a clearer sense of missionand the three strategic objectivesprovide staff with a common referenceand a clear basis for communication.But it fails to provide a clear guide togreater operational selectivity orincreased development effectiveness.The content is largely permissive, as is the reference to IFAD’s main target group.

n The numbers of staff in PMD,unchanged since 1994, are too few forIFAD to have been able to develop theprocesses and skills to be an innovator,a knowledge institution, a catalyst anda leader, especially in such a non-conformist environment.

67

the gender balance of staff and consultantsbetween 1994 and 2003, with an overallincrease of 50 per cent in female employees,and increases at all levels of the organization.Secondly, the majority of staff who haveworked for at least three years have beenpromoted. This means that many if not mostposts are upgraded, leading to grade creep.That in itself is likely to be a disincentive togenuine career advancement efforts. Thirdly,almost 50 per cent of professionals at P4 andP5 level will retire within the next five to sixyears, representing both a problem, in termsof loss of institutional memory, and an oppor-tunity for regeneration. Finally, IFAD has avery high reliance on consultants, with aprofessional staff to consultant ratio of 0.92.This is said to provide specialist expertise notavailable in house, but many consultants areused repeatedly with little systematic assess-ment of their performance.

3.41 A new human resource policy is being intro-duced that will involve radical changes to pastconservative HRM practices in IFAD. In thepast, human resource management in IFADwas seen as the province of one department,rather than a function for all managers. It wasessentially an administrative, bureaucratic,entitlements-controlling culture, that wasprocess-oriented rather than being an effec-tive contributor to the operation of the Fund.This unsatisfactory situation was recognizedby senior management. With the appointmentof a new Director for HR and in-housecommittee work, a new HR policy has beenapproved by the Board and a new andcomprehensive draft of the HR policies andprocedures manual was prepared in May2003 and is being further developed. Thepolicy is based on the International CivilService framework for human resourcemanagement with regard to compensationand benefits, employment, career manage-ment and good governance and compareswell with HR strategies in other internationalorganizations. Performance evaluation againstagreed core competencies lies at the heart ofthe new policy, and the fact that managementof people is fundamentally the responsibilityof line managers, rather than a separate HRdepartment. It is line managers who know

how well individuals perform and must takeresponsibility for the career development ofindividual staff, so that they contribute effec-tively to achieving IFAD’s goals.

3.42 Implementation of the new HR policy willrequire significant changes in the role of theHR division. As managers accept their role inmanaging their staff, so there will be lesspressure on the HR division to act as a‘manager-substitute’, more pressure to act as afacilitator and policy adviser, and more oppor-tunity for the division to act as a monitor andadviser to senior management in such areas asemployment numbers and costs, and staffcapabilities. It is unlikely that the division as it stands will be able to sustain the new rolerequired of it. This is not a question of struc-ture or numbers of staff but rather of skillsand experience. The division has been re-structured so that four HR officers areresponsible for coordinating with linemanagers all aspects of human resourcemanagement in a particular department.Current staffing levels in the HR division arealmost exactly in line with the median forinternational organizations. However, thereare few HR specialists in the division whilemost have experience only of previous conser-vative administration-based HR management.

3.43 Implementation of the new HR policy willrequire a significant change in the culture ofthe organization. The introduction of a moremodern HR policy will make significantchanges to the role of line managers and achange in the culture of the organization. It isnot clear to the IEE that this is fully appreci-ated – from interviews with managers there isas yet little evidence of visible support for suchchanges and the implications may need to beexplained more forcefully throughout theorganization. In particular, it is not clear thatthe dynamic, continuous and never-ending

117 Management estimates the increase to be 37 per cent, andalthough it is not clear how this figure has been derivedfrom the data available to IEE, it still represents a significantincrease in the complement of personnel.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

68


nature of the change, that is dependent onresponding to both internal and externalneeds, is fully appreciated. The magnitude ofthe cultural change required is reflected instaff perceptions: of management approaches,the role and capabilities of managers,performance management and accountability,attitudes to innovation and risk, opportunitiesfor learning, and change initiatives.

3.44 These perceptions are discussed in moredetail in the IEE human resource report, butin summary, staff view senior management asremote and failing to focus on the corebusiness of the organization. This is attributedto poor vertical communication, though it isacknowledged that there is now greater trans-parency and the internal web site has made asignificant improvement. Although seniorprofessionals view the structure as flat andnon-hierarchical, junior professionals andmost general service staff view IFAD as veryhierarchical and there is very strong gradeawareness throughout the organization.Many managers see themselves as technicalspecialists providing services, rather than asmanagers of talent arranging for others tocomplete tasks to relevant standards, partlybecause they are not trained or prepared forthat role, and only a minority are supportiveof such a change. Staff are not convinced thatIFAD has good managers, mainly becausethey do not feel that the best people areselected as managers. There is little belief thatmanagers or staff in general are judged ontheir performance or held accountable fortheir actions so there are few incentives forconsistently good performance: all staff getpromoted at some point. There is littleevidence of staff maintaining their levels ofexpertise due to very limited training anddevelopment provision, while internal orexternal rotation is rare.

3.45 IFAD stresses the importance of innovationbut innovative programmes are by definitionrisky, and staff feel there is no clear manage-ment policy or guidance as to whether IFAD’sobjective is relatively risk free resourcetransfer or the development of new conceptsand programmes that may be inherently risky.Many staff judged IFAD as risk-averse, due in

part to a lack of trust between staff andmanagers. Finally, staff are willing to acceptchange and many suggest IFAD needs tochange even further. However, there is a wide-spread feeling that despite the many changeinitiatives not much really changes, leading toa suspicion of, or even cynicism about change.Notwithstanding all these factors, most stafffeel that the objectives of the organization areimportant and are proud to work for IFAD,but the new HR policies will have to bestrongly promoted to overcome the culturalconstraints identified above.

3.46 Poor human resource management has beena major contributor to the modest projectperformance. The corporate culture reflectedby the factors discussed above impactsadversely on the achievement of IFAD’sobjectives and the performance of projects.In particular, the failure of managers tomanage staff, rather than acting as seniorspecialists, the lack of consistent evaluation ofperformance and accountability, the lack ofrewards or incentives for outstanding workand a perception that the organization isrisk-averse do not consistently promote the

Key Points:

n There has been a significant increase of43 per cent in both staff and consult-ants between 1994 and 2003. Humanresource policies have been conserva-tive and administration based, andhave failed to establish a culture ofstandard setting, accountability andperformance management.

n Surveys reveal a lack of trust betweenmanagement and other employeesowing in part to failures of communication. Managers are thought to be risk averse whichdiscourages innovation.

n Poor HRM has been a major contrib-utor to variable project performance.The new human resource policy willaddress many of these problems butwill involve radical changes in theculture of the organization that must become a priority for senior management.

69

identification and implementation of highquality innovative initiatives. This iscompounded by a failure to ensure that staffmaintain their expertise and are exposed tothe latest thinking on development issuesthrough training and development, andsecondments or rotation. In fact, IFAD’straining budget is very small. Suspicion ofchange may mean that changes are adoptedfor cosmetic reasons, rather than to intro-duce fundamental improvements.

Partnerships3.47 Partnerships are intended to make IFAD more

effective. IFAD has always worked in closeassociation with both national and interna-tional organizations. Policy directions underIFAD V and VI set out more specific ambi-tious objectives for partnership working.These were to help the Fund replicate andscale up through cofinancing; complement or underpin macroeconomic reformprogrammes; share institutional capacity andcomparative advantages; exchange knowledgeon policy and practices; increase multi-stake-holder coordination; be effective in creation of sustainable productive private sector invest-ment in rural areas; leverage resourcesthrough cofinancing; and play a catalytic rolebeyond the country level. Partnership workingwas identified as a pivotal process in helpingIFAD catalyse policy and resources

3.48 Partnership working needs clear objectivesand practical arrangements. The guidancegiven in the IFAD V and VI was confusingand contradictory and neither informed norbuilt on the approach in the strategic frame-work 2002-2006, which lacks any principles todirect management. This has created a situa-tion where there is limited institution-wideunderstanding of the potential benefits ofmany partnerships.

3.49 Most current partnerships lack a clearstrategic rationale and there is no process bysenior management or the EB to test theirobjectives against IFAD’s goal and niche – toestablish relevance. In practical terms, part-nership agreements often lack a clear state-ment of the shared objectives and indicatorsto measure progress. The Internal Audit

supports this judgment with evidence of a lack of clear criteria and priorities for the NGO ECP grants, but it holds true for most arrangements including supplementary funds.118

3.50 An initiative with the World Bank in 2001 tocreate a rural partnership initiative illustratesthe good intentions. A design document setsout the context and identifies global andcountry level actions, proposed for Bangladesh,El Salvador, Nigeria and Yemen. But the textappears to exist only in a draft form, objectiveslack specificity, the work has never been devel-oped through to action programmes (so thereare no indicators of performance) and fundingis unresolved.119 The IEE has been unable tofind any reports of progress or achievements.

3.51 There is poor internal communication aboutthe range and purpose of partnerships. Aproliferation of partnerships contributes toignorance and uncertainty amongst IFADstaff, where CPMs face significant transactioncosts in dealing with the large numbers oforganizations with which IFAD works anddealing with their different modalities.

3.52 Relationships are important to IFAD’s objectives but not all relationships are part-nerships. Partnership implies a relationshipthat goes beyond a contractual or administra-tive arrangement. It involves trust and a will-ingness and ability to take risks in pursuit of a

118 Internal Audit Report AR/99/01, May 1999.Supplementary funds have amounted to between USD 8 million and USD 12 million per year since the mid1990s. They are treated as extra-budgetary funds. There isno consolidated report of utilization and effectiveness. An IA report of June 2000 (AR/00/06) called for an integrated information system for supplementary funds(that would cover many partnerships). A progress reporton the SCP in April 2004 predicted such a system wouldbecome functional in mid-2004.

119 IFAD office memoranda dated 21 January 2002 from vande Sand to Codrai, and to PMD Division Directors (bothunreferenced); World Bank – IFAD Rural PartnershipInitiative, Draft (7) Working Paper December 2001.

Box 10 The Belgian Survival Fund

Since 1983 the Belgian Survival Fund (BSF) has been working with IFAD, under the auspices of the Joint Programme(JP). IFAD’s partnership with the BSF is highly relevant to its own strategic objectives, and enables a more holisticapproach to rural development: the BSF.JP focus on health, sanitation and basic social services provides an impor-tant platform for improving household income and food security. The BSF.JP also enhances IFAD’s role in other ways,such as: (i) through grant funding in countries not (yet) eligible for loan-financing (e.g. Somaliland; support towomen’s groups in North Kivu, Democratic Republic of the Congo); (ii) the direct funding of NGOs (e.g. KenyaWomen Finance Trust in Kenya); and, (iii) BSF-funded rehabilitation programmes (e.g. the Ethiopia RehabilitationProgramme for Drought-Affected Areas).

Global evaluations undertaken during the first phase (1983-1995) clearly demonstrate that the collaboration hasbeen instrumental in the Fund’s reaching out to the most marginalized and destitute rural populations in sub-Saharan Africa – with social activities financed by BSF often becoming entry points for income-generating activ-ities.120 Another overall evaluation is scheduled for 2005, and this will include an evaluation of the IFAD-BSF partnership.

The JP programme support unit is involved in all stages of IFAD’s project cycle: internal project design(PDT/TRC/OSC); implementation, follow-up and supervision (with the CIs); evaluation and, thematic reviews (post-conflict, etc). The administrative and fiduciary procedures of the BSF.JP however, sometimes affect the efficiency ofthe partnership. In Burkina Faso, under PDRSO, the IEE found that management lines had been made “morecomplex by the partnership with the Belgian Survival Fund which has special responsibility for the health compo-nent of the project”.121 Similarly, a recent completion evaluation by the BSF itself found that: (i) uncertainties andinitial delays in BSF headquarters delayed the expeditious launching of the project; (ii) further component changesmade by the BSF headquarters slowed down the launch of the project; (iii) direct BSF contacts with the project havealso been limited.122 These findings are supported by an internal audit report (AR-00-05) which labelled the BSF.JPbudget process as ‘cumbersome’.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

70


common objective.123 The term partnershiphas in fact been retrofitted to describe arange of pre-existing inter-organizationalrelationships and alliances, including thosewith cooperating institutions and co-finan-ciers. For example, the contribution ofsupplementary funds is often held to be apartnership arrangement. Many are in factdefined by contracts or memoranda of under-standing. IA found a lack of a clear strategicframework, difficulty in reconciling the needsof IFAD with the conditions of donors, andfrequent use of the funds to supplementactivities under the regular programme ofwork and budget.124 These examples showhow the imposition of a partnership labelconfuses a much more straightforward relationship that should be managed in anappropriate but different way.

3.53 There is virtually no systematic evidence toshow how partnerships are enhancing IFAD’simpact on rural poverty. There are exceptions,with some evidence of a catalytic role with theCoalition and the long-running achievementsof partnership with BSF, noted in Box 10.

.54 The IA commented on a lack of evaluation or other evidence of the NGO/ECP grantsprogramme and argued that effectiveness hasbeen undermined by low priority and inade-quate monitoring.125

3.55 The country studies found little evidence ofpositive use of partnership working. IFAD isperceived by the majority of stakeholders asbeing distanced from projects, both geograph-ically and organizationally. IFAD has notresponded to moves by other donors towardsdecentralized operations and has not devel-oped new approaches to partnership workingat country level.126

3.56 Partnerships with recipient governments arecentral to IFAD’s work, but so (increasingly) is partnership working with other donoragencies. Effective donor collaboration is at the centre of the international community’scommitment to improved aid effectiveness.Both the Rome (2003) and Paris (2005)Declarations on Harmonization and Alignmentstress the need for greater complementarityand coherence in donor support. The most

71

recent Paris Declaration includes a set ofperformance indicators to track progress withthe agenda, including the increased use ofcommon arrangements or procedures amongstdonors for delivering assistance, sharedanalysis and field missions, a reduction in thenumber of parallel implementation structures(PIUs etc.) and coordinated donorprogrammes consistent with recipient develop-ment strategies. These are not partnerships inthe full meaning of the word. But movestowards harmonization require a shift in the‘culture of partnership’ within many develop-ment organizations, including within IFAD.Although IFAD has recently tabled papers tothe Board setting out possible ‘new ways ofworking’, observation by the IEE team suggestsa significant lag with practical changes in theway IFAD is doing business on the ground.

Management processes3.57 The need for change was recognized before

the REA and boosted by that assessment.IFAD staff and management have never been slow to re-examine processes andinitiate change. The initial years of the Fundsaw a steady evolution of practice as theoriginal model of cofinancing projects developed by other IFIs was replaced byIFAD-initiated designs that reflected IFAD’speculiar approach.

3.58 The REA stimulated a wider process of changeafter 1994: the re-engineering programme(strategic management, COSOPs, PPMS,corporate score card) 1994-1998; IFAD Vision,1995; strategic framework 1998-2000; PRP2000 leading into the SCP 2001-2005; IFAD V:Plan of Action 2000-2002; strategic framework2002-2006; regional strategies 2002; IFAD VI:Plan of Action 2004-2006.127 It is notable,though that despite the REA drawing attentionto the limited use of budgeting as a manage-ment tool, movement towards a medium tolong-term strategic planning and budgetingprocess (as opposed to the strategic frame-work) was not a feature of the era, thoughmoves towards strategic planning haverecently been initiated (see Table 18).

3.59 It is significant that, unlike the World Bank,where similar change was driven by analysis of performance,128 none of the initiativesdescribed here have been stimulated byanalysis of development effectiveness. They have been process-driven on issues ofefficiency, rather than performance-driven byanalysis of impact. None of the initiatives hasbeen evaluated for impact on IFAD’s develop-ment effectiveness.

Key Points:

n Working in partnership has the potential to support the Fund’s objectives of being a catalyst and innovator.

n But the objectives set out in the Fifthand Sixth Replenishments have led toan uncritical use of the term and failedto foster clear objectives and improvedways of working that would bringstrategic benefits to the Fund.

n Partnerships need clearly stated jointobjectives and monitorable indicatorsset within the overall framework ofthe Rome and Paris Declarations.

n With a few notable exceptions there isno evidence of enhanced impactthrough partnerships.

120 Source: IFAD and Belgian Survival Fund Joint Programme:Strategy Paper 2001-2011, 1 September 2000, page 1.

121 Unfortunately, there were few BSF.JP grants covered underthe IEE random sample of projects.

122 Paragraph 169, Completion Evaluation Report: Support toWomen’s Groups in North Kivu, The Democratic Republicof the Congo, Report Number 1416, August 2003.

123 A UN definition is ‘A voluntary and collaborative agree-ment between one or more parts of the United NationsSystem and non-state actors in which all participantsagree to work together to achieve a common purpose orundertake a specific task and to share risks, responsibili-ties, competences and benefits’. Reported in ProgrammeManagement Department (2004).

124 Internal Audit Report AR/00/06, June 2000.125 Op cit. AR/99/01.126 See Annex 4.127 DRR Tables 3 and 6 and associated text; para 4.55 et seq.128 World Bank (1992) Effective implementation: Key to

development impact. Washington (The WapenhansReport).

3.60 The large number of initiatives have not beenwell connected nor seen through to recogniz-able outcomes. The historical overview in Box 11 charts a logical development of initia-tives in response to operational needs. But atthe end of the process the IEE in the deskreview argued that IFAD’s niche and compar-ative advantage were poorly articulated; thatinnovation is misused, misunderstood andmisguided; and that projects are both overand under-designed.129 Despite so manyinitiatives to improve quality the Fund hasnever instituted a system to analyse projectdesigns and report on quality.

3.61 Review of the underlying analyses thatsupported changes to the project cyclereveals well-conceived processes in whichstaff were self-critical and honest aboutfailings in the system. Yet the proposedchanges have not been developed to thepoint where real benefits are seen to haveemerged. Several examples from the deskreview illustrate this:

n The project design team, PDT, was anearly initiative to introduce team working.The principle is good, but in practice stafffind it hard to devote the time necessary.CPMs can choose who to nominate, so canavoid serious debate about their plans. Asa result, the only effective challenge in thedesign process comes when the AssistantPresident PMD intervenes in the TRCmeetings.

n PPMS was established to improve thestorage, analysis and access to informationto help manage operations. A review ofPPMS by IA in 1999 found the system notbeing used by the main intended targetgroup for reasons that included an incom-plete database, limited reporting facilitiesand no integration with other systems. Bythe time of the IEE the level of use hadnot improved, the database was morecomplete in some areas such as basicproject information, but others such as use of consultants had not been madeoperational; and the expected inclusion ofthe PSR as a reporting facility had beendropped owing to apparent limitations in computing power.130

n The desk review found a proliferation ofisolated IT systems unable to share informa-tion – loans and grants system (LGS), PPMS,TAGs, supplementary funds and the NGOECP data base. The data on PPMS is heavilyfrontloaded towards project design and nota useful tool for portfolio management.There have been problems associated withnot computerizing the project status reports.At present cross-referencing with PPMS hasto be done manually and there is no incen-tive to use as a management tool by CIs.The fragmented and under-used systemsseverely limit the extent to which perform-ance can be analyzed, both at an operationallevel for senior management, but moreimportantly, for reporting to the EB.

n The re-engineering working group in1995 proposed a saving of USD 100 000from design to be used for implementationsupport. Staff assert this was never imple-mented. Although the five regional divi-sions track their own resource use, theabsence of transparent reporting atdepartmental level (through PPMS orfinancial reporting) about costs incurredduring the project cycle has underminedconstructive debate about resource alloca-tion and divisional efficiency, by the EBand by senior management.

n The logframe had been around from someyears, but a renewed interest was shownwith a training programme in 1998. Yetcurrent projects have very poorly workedexamples; the results structure that under-pins the logframe has not been adopted byIFAD as a means of bringing consistencyand precision in language and the defini-tion of objectives and indicators to projectplanning; and the logframe has not beenused as a means to structure supervisionand project reviews. The value added fromcurrent usage is minimal.

n The corporate scorecard flourished verybriefly in 2000, never to be seen through asa reporting tool.

3.62 An indication of the underlying problems can be seen in the design of the PRP. This USD 26 million initiative was planned to bringunder one resource envelope the disparate

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

72


Box 11 A central concern was to develop the project and programme cycle

n Between 1982 and 1988 IFAD’s project cycle was characterized by cofinancing projects from the pipeline of

World Bank and regional IFIs. This reduced both the development costs and length of design cycle. But increas-

ingly, IFIs moved into policy-based lending.

n IFAD brought design in-house. Special programming missions analysed rural poverty and defined IFAD’s

country strategy. To delineate the target group, farm diagnostic studies, socio-economic surveys, studies of

NGOs, gender, socio-anthropological aspects of target groups, and all aspects of farming systems and the

institutional framework were reviewed. Project design costs increased from USD 400 000(1982-84) to

USD 575 000 (1990-92).

n The concept of quality was vague and undefined. The project cycle was becoming longer, but mainly to improve

quality of project selection and the design of IFAD specificity. Often the additional investigations required as a

result of internal reviews were not relevant to achieve better implementation but only served an improved

marketability within IFAD.

n The result of the task force of 1993131 and the re-engineering working group of 1995132 led to more changes

in the project cycle - a shift from design to implementation - seeking more continuity and flexibility in the

project cycle. The high degree of uncertainty in IFAD projects was thought to require a larger degree of flexi-

bility to allow for adjustments and changes during implementation.133

n The COSOP was introduced to provide a more strategic and programmatic framework for the development of

new opportunities in a country. The average cost of design was reduced by USD 100 000 – and the intention

was for this to be allocated to early and continuing implementation support.

n There was a clear call for change in corporate culture: teamwork, decentralized decision making and resource

management (a cost centre approach). Critical to achieving these changes were the underlying incentives and

signals governing staff behaviour. Reviews also emphasized the need for attention to increased workload and

training of staff in management skills.

n Over this period projects changed from typical blue-print and input oriented in the eighties and early 1990’s to

more flexible and process oriented projects in the mid and late 1990’s. IFAD started to supervise directly a group

of 15 projects to validate if direct involvement and accountability for project implementation would provide

better results on the ground and generate knowledge as part of IFAD’s own learning.

n Monitoring and evaluation of indicators, and evidence of impact were still weak and only slowly improving. The

logframe was introduced to assist in this. Teamwork remained a difficult target to achieve: project design teams

were improved but remained highly dependent on a CPM’s interest and motivation, and time availability of

colleagues in PT, OE, OL and FC, as well as incentives and rewards. Thematic groups were established and func-

tioned fairly well but suffered from the same drawbacks as the PDTs.

n The IAPC working group was originally set up in 1998 to work on mainstreaming the logframe.134 It was

converted into the IAPC in the context of the Process Re-engineering Programme in 2000. The working group

addressed a range of issues (in all it made 21 recommendations) but not very different from the past reviews.

For example it was agreed that PDT and PDIP were key to the whole project cycle. It was agreed to have a more

unified design document and a key file (with the logframe) that would constitute the focus of discussions

during TRC and OSC and constitute the basis for monitoring through the stages of life of a project. The key

file would also assist in knowledge management by centring discussion around the core concern of poverty

reduction while underlining the approach to exploit IFAD’s comparative advantages. IFAD started lending more

flexibly (FLM).

129 DRR Summary para. 12130 Internal Audit Report AR/98/04, March 1999131 IFAD (1993).132 IFAD (1995).

133 Similar interests at the World Bank led to the introduction of the LIL (Learning and Innovation Loan, and AdaptiveProgramme Loan.

134 Cooke et al (2000).

Sources: DRR para 3.46 et seq.; Programme Management Department (2004) 73

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

74


changes being proposed for human resources,strategy and finance, information technologyand support services. Internal Audit examinedthe design phase and concluded that there wasa variable degree of management commitmentand support; specific responsibilities had notbeen defined, agreed and communicated;there was a lack of focus in the work effort;process analysis was limited and not inte-grated; the information technology proposals(a major driver for the changes in order tointegrate and update systems) were not clearlylinked to the business proposals; and it was notclear how PeopleSoft modules would meet therequirements to replace the LGS (a concernthat has since been proved to be wellfounded).135 One of the first acts of thecurrent President was to revise and rescheduleits scope and management.

3.63 The impact of these changes has been under-mined by a lack of drive and commitment tonew ways of working, as reflected in theapproach to human resources management.But this has been influenced by a complexinteraction of several factors: pressures todeliver projects to the Board (a featurecommon to other IFIs); disincentives in workpractices that have held back a move awayfrom the underlying business model with afree-agent CPM; arrangements for resourceallocation and budget management that havenot promoted performance-related practices;‘horizontal’ structures at the levels of seniormanagers and directors that lead to consensusbased decisions; and constraints in workingpractices such as the extensive use ofoutsourced consultants, headquarters-basedworking, contracted-out supervision and zero-growth budget constraint.

3.64 Signs of new commitment may be seen in the more recent past with the creation of the‘IFAD Management Team’.136 Started inApril 2002 three meetings per year havebeen achieved since September 2003. Theoriginal driving force was the HR policy, buttime is now divided between policy issuesand HR. It is a positive move to facilitate andmanage change in a more participatorymanner, but too recent to have yielded evalu-able results.

3.65 The need for the CPM’s role to change hasbeen recognized but not resolved. One of the main distinguishing features of IFAD’sbusiness model has been the role of the CPM. Numbers have been fairly constant atabout 35 over the review period distributedbetween six and eight in each region. Their roles are several: to manage the port-folio in (usually) two to three countries,policy dialogue, project development andimplementation support, contribute to evaluations, promote innovation, and scalingup, and manage the administrative process.In addition, the broadening of IFAD’sstrategic framework has added new tasks: to develop and manage ‘partnerships’ andother relationships; to contribute to learningprocesses such as thematic groups; and towork on ad hoc task forces, prepare material for the EB, and support othercorporate processes.137

3.66 CPMs are the main point of contact betweenIFAD and its stakeholders, especially atcountry level. Their numbers and the PMDorganizational structure impose difficultconstraints on ways of working. Their role isa solitary one. No other IFAD staff have theopportunity to develop a competence indealing with their countries. At best, the onlyother staff who join them on mission are theregional economist and technical specialistsfrom PT. Solitary working means they have ahigh degree of control over how often theytravel and for how long; over how much toget involved personally with missions, who tohire and how much to delegate to consult-ants, and how to engage with governmentand with donor harmonization initiatives atcountry level.138

3.67 With such small numbers, the PDT approachto peer review does not provide an effectivesupport mechanism, and in practice comeslate in the process for fundamental change.The limited resource of professional supportin PT leaves CPMs isolated and with highresponsibility. Not only is there littlesupport, but until 1999 no performanceassessment either. As a result, performanceamong CPMs is acknowledged by seniormanagement to be very patchy.139 The best

75

have well-planned schedules a year or morein advance; spend over 100 days a year travelling to their countries, and meetProject Management Unit staff every year,twice if not in very remote locations; theyuse trusted consultants to play the role ofpeer review and help in the search for inno-vation; they lead MTR missions in person,are present the whole time and draft theaide-memoir themselves, if not the fullreport. At headquarters they are active inthe policy forums, participate in OE corelearning partnerships, may have a role or be a focal point in a current policy taskforce, join in thematic group meetings, and prepare papers for senior management.This is a high standard to aspire to. Notevery CPM can perform at that level, butboth staff and managers argue to the IEEthat there is insufficient management ofCPMs’ time and performance, and that is amajor factor in the uneven performance ofIFAD as a whole.140

3.68 Change has not improved efficiency and has left many staff sceptical of furtherchange.141 Despite all these cost-reducingand efficiency-promoting measures, thepaucity of management information is suchthat there is no convincing evidence oflasting improvements in the use of resources,and quality or efficiency in development ofIFAD’s products. In fact, IFAD has reacted to the change agenda by increasing its use of temporary and part-time staff in order tomeet the expanding demands. The IEE deskreview report drew attention to the hiddengrowth within the so-called zero-growthbudget.142 These numbers have beenconfirmed during the HR study and show an overall fall in the Fund’s efficiency overthe review period.

3.69 Not only have the changes failed to bite, they have left staff deeply sceptical about the ways they have been introduced andmanaged. The human resources study found change initiatives are perceived, cynically by many staff, to lack clarity ofobjectives, not to have adequate proceduralguidelines, to add to the workload and not to be adequately resourced.

Key Points:

n IFAD has welcomed and embracedchange, with a series of powerful initia-tives and participatory processes thathave been honest and self critical aboutthe need to reform. But the focus wason improving efficiency rather than adrive to enhanced impact.

n IFAD’s operating model and resourceconstraints directed attention towardsimproving the project cycle. At first theaim was better quality and managingcosts; later it changed to coping with thepolicy demands from IFAD V and VI. Thecurrent programme will introduceimproved information technology andfinancial management systems.

n Many project cycle initiatives have notbeen seen through to effective comple-tion – a failure of management that has left staff sceptical about changesmaking any difference.

n The free-agent CPM model remains aconstraint on ways of working thataffects projects, programmes, knowledgemanagement and innovation. At best it enables entrepreneurial CPMs to flourish. For many, it leaves an unchallenged and under-managedworking environment with little technical support.

n The combination of partially-imple-mented initiatives, changing role butunchanging organization of CPMs, andlimited technical support has created aquality assurance process that providesneither effective support, nor analysisand accountability for quality.

135 Internal Audit Report AR/00/08, December 2000.136 A forum of senior managers. Related initiatives are the

PMD Team and CPM Forum.137 Some 20 departmentally-based and 20 corporate working

groups have been created and in operation between 1995and 2004. (Source: IFAD-IEE coordinating group).

138 Workload is difficult to quantify. A sample of 24 CPMs(some 63 per cent of the total) for the period 1997 to2003 (excluding retirees and those moved to other posts)shows a workload distribution as follows: 25 per centsupervise 3 or 4 projects; 54 per cent supervise 5 to 7;and 21 per cent supervise 8 or more up to 11. Abouthalf the sample have brought between 2 and 4 newprojects to Board approval; half have presented 5 or 6projects each. (Source: IFAD-IEE coordinating group).

139 IEE interviews.140 IEE interviews.141 Findings from IEE staff survey.142 DRR para. 4.48.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

76


Knowledge management and learning

3.70 IFAD has been a knowledge organizationsince it recognized the need to design its ownprojects. A realization that IFAD needed toapply its own experience and skills if projectswere to be effective at rural poverty alleviationwas an early driver of the shift away from cofi-nancing projects identified by other IFIs to apredominantly self-identified approach. At thetime, phrases such as the ‘learning organization’and ‘knowledge management’ were not in use.With hindsight it is perhaps unfortunate thatthe REA took place in 1994, when the ideasfrom Senge’s path-breaking work weregaining ground and being adopted by otherIFIs, especially the World Bank.143 Theprominence given by the REA to recommen-dations to become a learning organizationmay have overshadowed the more funda-mental objective of being effective at workingwith governments to tackle rural poverty; adistraction of means over ends.

3.71 The direction advocated by the REA hasbecome a core feature of IFAD’s rhetoric andthe link to innovation developed strongly, firstin the strategic framework 1998-2000, whereit was one of five strategic thrusts, and then inthe IFAD V: Action Plan, 2000-2002, wherethe four major roles of innovation, knowledgeinstitution, catalyst and leader were defined.

3.72 Efforts to promote knowledge management(KM) have followed similar models to otherdevelopment organizations. The period from1995 to 2002 saw the creation of a KM facilita-tion and support unit (later disbanded), whichprepared a knowledge strategy. Other initia-tives included thematic groups, a new policy topromote disclosure of documents, entry of KMinto the work programmes and strategies ofthe regional divisions, and by 2002 re-creationof a communications division.144

3.73 These broad initiatives were accompanied bymany specific developments: the website (forboth internal and external users), EKSYST,PDTs, and the TRC. There has been noshortage of skills or instruments. But the KMstrategy was never actually approved, theprominence of KM and management focus on

initiatives was steadily diluted and by the timeof the 2002 ERT, progress in knowledgemanagement was judged to have been slow.

3.74 The organizational culture has underminedchange. Learning and the sharing of knowl-edge is done in the ways people work ratherthan in the tools that are used, though toolscan help ways. IFAD’s strategy was moreabout tools than ways of working. Poor hori-zontal and vertical communications, resourcesconstraints that limit effective team working,reliance on consultants, minimal attention toperformance measurement and poor account-ability through simple means such as namingauthors on documents, have combined toreduce the flows of information.

3.75 In the desk review report the IEE testedIFAD’s performance against a model ofknowledge management that charts thesequence of gathering information, identi-fying its value, storing and disseminating it toIFAD’s own operations and to partners. Afundamental weakness is that IFAD has a poorrecord of data collection.145 Project arrange-ments for M&E of loans and grants have beenwidely criticized in a variety of studies, andthere is a low priority given to finding infor-mation from external sources – despite somany ‘partnerships’. Mechanisms with highpotential, such as core learning partnershipsand agreements at completion point, havehad limited, often individual impact. Thewebsite has brought speedy internal retrievalof information, and staff make good use ofthat plus informal links to meet their personalneeds. There are plenty of media available fordissemination, but respondents in IFIs and atcountry level report low levels of awareness,poor access and minimal utilization. Therehas been no monitoring and follow up tocommunications, although the April 2004discussion paper on communications proposesformative, process and outcome evaluations.

3.76 An opportunity was missed in 2001 when OEevaluated IFAD’s innovation capacity. Some ofthe contributory work was undertaken byconsultants and their report provides a sharpand insightful analysis of the ways in whichIFAD’s culture inhibits communication and

77

sharing of information. The study was criticalof learning mechanisms, identified blockagesto innovation, found poor direction bymanagement and poor decision-making. Italso identified drivers of innovation. But thereport was disseminated at a difficult timethat coincided with the appointment of a newPresident who had to resolve a difficult finan-cial crisis, and the agreement at completionpoint process had the detrimental effect ofmisrepresenting some key findings.146

3.77 Ambitions to provide leadership and influ-ence have not yet borne fruit. Enquiriesduring the country visits and in meetingswith staff of other IFIs confirmed thatexchange of information takes place prima-rily through direct contact and only whenpeople share a common task. This has clearand important implications for the ways

CPMs interact in headquarters and workwith partners at country level. The COSOPshave had a negligible impact at promul-gating IFAD policies or strategies, orincreasing understanding about rural devel-opment. There are minimal references toIFAD’s work in relevant publications andstudies by other IFIs and a very low level ofawareness about operational documentssuch as the strategic framework, regionalstrategies, and regional poverty assessments.The 2001 Rural Poverty Report is the onlystudy with any wide exposure. Staff in theAgriculture and Rural DevelopmentDepartment of the World Bank, an obviouspoint of professional contact, report thatthey have few personal relations with IFADstaff and little awareness of publications.147

Searching for the new business model

3.78 IFAD still seeks a new business model basedon its catalytic role to generate ‘learning andreplicating empowerment programmes’.Despite all the changes described here, theshift in the de facto business model has nottaken effect. The reasons are grounded in theunderlying constraints of the Fund. They areinterwoven in a complex way, but the issuesare clear and revolve around four aspects: thefree-agent CPM; the zero-real growth budgetconstraint; the organizational culture; and theheadquarters-centred operations.

Key Points:

n The important role of knowledge toIFAD’s mission was given shape in theREA and quickly led to the IFAD V: Planof Action with objectives to be innova-tive, a knowledge institution, a catalyst and a leader in rural poverty reduction.

n A wide range of practical initiativeswas mounted and tools developed, butthe knowledge management strategywas never actually approved andmanagement’s interest appeared todrift. The organizational culture withpoor communications, aversion to risk,limited team working and no staffdevelopment, never fostered learning.

n Weak arrangements for project M&Eand use of performance informationmeant that even knowledge fromIFAD’s own projects was not well recycled.

n Communications and disseminationboth have strengths. The disclosurepolicy and website have brought practical benefits. But awareness ofpolicies and strategies among partnerorganizations at country level andinternationally is low and IFAD isabsent from contemporary develop-ment literature.

143 Senge, Peter M (1990) The Fifth discipline: The art andpractice of the learning organization. CurrencyDoubleday, New York.

144 See DRR Annex 6 for comparisons with the Departmentfor International Development, Japan InternationalCooperation Agency, the Swedish InternationalDevelopment Corporation Agency, EBRD and World Bank.

145 DRR Annex 6, para 4.1 et seq.146 DRR Annex 6, para 3.22.147 The recent World Bank publication ‘Agricultural

Investment Sourcebook’, February 2004, makes virtuallyno reference to IFAD and has nothing on innovation fromIFAD. Out of 494 citations for specific references orselected readings IFAD is only referenced once, for the2001 Rural Poverty Report. IFAD is not listed in anAppendix of 43 key websites. In another example, onlythree IFAD projects (two from Peru and one from VietNam) were included in the 100 case studies on “upscalingfor poverty alleviation” that were presented in 2004 at aUNDP-World Bank conference in Shanghai.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

78


3.79 The small scale of complementary-financeoperations directed the Fund towards its keyfeatures of headquarters-based operationsand small operational unit. At first the Funddid not even design projects. Once designwas the norm, the image of the organizationfostered the free-agent CPM, few in number,entrepreneurial in outlook and able todesign customized projects to fit the specialissues of IFAD’s mandate. Customized opera-tions on a small scale enabled each new loanto be described in the annual reports andencouraged an anecdotal style of reportagethat would ultimately resist quantifiedmeasures of performance and normativeways of working.

3.80 IFAD’s strength has always been its CPMs.But the unconstrained work environmentwithin which they flourished discouragedcollegiate practices such as the PDT,common tools like the logframe and thedevelopment of policy and strategy frameworks that would set boundaries.Knowledge management too was poorlyserved by the culture of individualism.

3.81 Burgeoning costs of operations drove thefirst attempts at reform from within.148 Theobjective was to reduce the costs of designand improve plans for implementation. Soonafter, the REA set in motion the dialogue thatled to the policy directions of IFAD V and VIand the SCP. Improvements to the projectcycle pulled the organization in two ways atonce. The pressure to reduce costs and workwithin the President-imposed zero realgrowth budget stimulated a lighter designprocess, but staff wanted the saving in coststo be used to give implementation support inthe early years. Implementation supportimplied a shift away from design and towardsthe supervision role denied by IFAD’s oper-ating rules. The REA called for those rules tobe changed but this was never formallypursued by management. At the same time,

the call to be more strategic and program-matic put more pressures on the Fund to beanalytical, just as general identification andspecial programming missions were replacedby the COSOP.

3.82 The expanded mandate and drive toimprove quality could never be costless.Projects had to decline in number or thenumber of staff and their skills had to grow,or new instruments had to be found. Inpractice, staff numbers grew, funding a fewposts through supplementary funds andother mechanisms, but mainly driving up the reliance on consultants. The consultantsare the worker bees of IFAD. Loyal andcommitted; many have had long workingrelations. Some work for few other clients.They have brought skills, experience andenabled the free agent CPM to perform. Buttheir high level of involvement has under-mined institutional learning, in the IFADhierarchy they carry little real influence, and the mission-style contracting model hasneither brought continuity nor fostered waysof adding value from their contributions.

3.83 Understanding the organizational culture has been a feature of this evaluation. Theemergence of a strong grade awareness, poorcommunications consensus management andorganizational silos has militated against thelearning culture that could take risks andpromote innovation, and encouraged therigid simplicities of the consultant worker. A headquarters-centred operation has heldback flexibility in change and made reformto the design-implementation-supervisionbalance harder to effect.

148 Report of PMD Task Force on Rationalization of theProject Cycle (1993).

79

Conclusions

The changing context 4.1 The period since IFAD’s establishment has

seen unprecedented change in the globalcontext for agricultural and rural develop-ment. The period covered by the IEE hasalso witnessed a major reaffirmation by theinternational development community of thecentrality of poverty reduction, culminatingin a new global consensus focused aroundachieving the Millennium DevelopmentGoals. The focus of development assistancehas also changed. External support for theagricultural sector has plummeted in favourof a much wider set of enabling actions forrural growth and development, accompaniedby a sharper focus on partnership and policyperformance in support of national povertyreduction strategies.

4.2 Against this backdrop IFAD has expanded itsmandate and mission. From a funding institu-tion with a specialist focus on improving foodproduction and nutrition, IFAD is now a fully-fledged development organization looking to‘lead global efforts in helping the world’spoorest’ through innovation, scaling-up andgreater policy engagement. The latest strategicframework charts an increasingly ambitiousagenda, emphasizing the Fund’s catalytic rolein enabling the rural poor to overcome their povertythrough harnessing knowledge, buildingregional and international coalitions andhelping to establish institutional and policyframeworks that support the poor.

4.3 The need to make rapid and sustainedprogress towards the MDGs places apremium on the combined efforts of theinternational development community tosupport national poverty reduction goals.The quality of aid has come sharply intofocus. In this context, the IEE concludes thatIFAD’s mandate – to provide additionalresources to improve the wellbeing of therural poor – remains fundamental. But thereis also danger that IFAD’s agenda hasbecome too broad and too distant from itsinitial aspirations of assisting the poorestgroups in the poorest countries. A clear focusis needed on core attributes with the poten-tial for comparative advantage, primarilyinnovation. The IEE also finds a series ofcritical weaknesses in IFAD’s managementand delivery of its strategic agenda, includingits ability to develop and replicate innovative,learning-based project approaches, whichneed serious attention if the quality and longterm development impact of IFAD assistanceis to be assured.

Resource allocation and portfolio performance (Chapter 2 page 26)149

4.4 The evaluation has found that IFAD’s portfolioof projects is broadly pro-poor and largelyresponsive to the Fund’s expanding mandate,

Future performance

Chapter 4

149 The page reference for each subheading directs thereader back to the relevant key points box, or main textwhere the findings for the topic were presented.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

80

F U T U R E P E R F O R M A N C E

but the existing resource allocation model isincreasingly out of step with best practice inthe aid community. The introduction of theperformance-based allocation system shouldhelp the Fund to be more transparent andsystematic in its engagement at country level,although there are serious doubts aboutwhether IFAD is fully tooled-up to respond toa more differentiated model of countryengagement. The Fund maintained its contri-bution to agriculture for much of the 1990s,but along with other donors began to moveinto the broader area of rural development bythe beginning of 2000. Declining resources foragricultural production remain a concern,especially with a renewed focus on the need toboost small farm productivity in lagging MDGcountries and hard to reach areas.

4.5 Assessment of the performance of the portfoliois complicated by the difficulties of bench-marking against organizations with differentapproaches to measurement and differentoperational procedures. The evidence suggeststhat IFAD’s portfolio performance is similar tocomparator agencies, but this judgment is seri-ously hampered by the limited use made ofself-evaluation and the lack of systematicanalysis generated from the current approachto quality assurance. The absence of a soundportfolio analysis restricts the ability of theExecutive Board to provide strategic directionto management, places the Fund at a disad-vantage when negotiating with donors, andreduces its effectiveness as a leader and itsability to influence policy.

Project and programme performance(Chapter 2 page 30)

4.6 IFAD investments are found to be relevantwith respect to national development prioritiesand IFAD corporate strategy; but the test is aweak one. The strategic framework provideslittle indication as to IFAD’s operational nichewhile regional and country strategies provideonly a weak filter for project selectivity withincountry contexts. Up until now COSOPs havebeen little more than aggregated project ideas,although the most recent COSOPs are makingstrides in the right direction. Limited analyt-ical work at sector level provides weak justifi-cations for what IFAD should be doing in

relation to other partners, and tends to feedproject ambitions rather than a holistic view of where IFAD can best add value. Policydialogue at country level is sporadic at bestand does not systematically add value overand above project investments. It is hamperedby the transient field presence of the Fund, anissue that is being examined through the FieldPresence Pilot Programme. Relatively littlechange has taken place in the identificationprocess over the years and IFAD is seen to beout of step with critical changes in aid harmo-nization efforts in at least some of itsborrowing countries.

4.7 Evidence about effectiveness and targetingboth point to weaknesses in the designprocess, highlighted originally in the deskreview report. Many projects experienceimplementation problems, not all withinIFAD’s control, but a significant proportionare associated with weak project managementthat reflects poor institutional analysis duringdesign. The details of implementation, espe-cially in key areas such as targeting, are lesseffective in practice than would be expectedfrom the expectations set out in design docu-ments, a reflection as well of gaps in policyand operational guidance.

4.8 While the majority of IFAD investmentsachieve their project-level objectives, thecausal link between objectives and povertyimpacts is often unclear and frequently overlyambitious. Changes to the project cycle havebrought some improvements, but the resultsstructure that underpins the logframe has notbeen adopted by IFAD as a means of bringingconsistency and precision in the language anddefinition of objectives and indicators.

4.9 Poor design should be tackled during thequality assurance process. The current systemis not delivering good quality products.Problems are exacerbated by the arms lengtharrangement between IFAD and the projectonce implementation starts and the cooper-ating institution does supervision. Repeatedcalls for the Fund to provide more implemen-tation support through changes in the projectcycle have not materialized. Weak initialimplementation is further evidenced by the

frequency of project redesign, which is lessfrequently a sign of project adaptability andmore often about poor quality at entry, unsat-isfactory economic and institutional analysis,and inadequate project management andsupervision. The limited field presence of theFund further reduces its ability to act tosupport implementation. Improving the inter-action of these factors through improvedquality is one of the major challenges forIFAD in the future.

Policy influence (Chapter 2 page 38)

4.10 There are limited options to IFAD’s existingproducts and instruments and a lack ofsynergy between projects at country level.Despite the aspiration for the Fund to influ-ence policy, it has never developed an opera-tional policy or model to guide that influence,nor have staff skills been developed in theseareas. IFAD’s size and resources limit its abilityto fulfil its policy advocacy and innovationroles acting alone. One of the few emergingareas of influence is in rural finance and theclose collaboration that is developing with theConsultative Group to Assist the Poor andothers. Policy influence at both project andprogramme level is minimal and not helpedby the extensive use of individual consultants.When it happens, it is highly localized andrarely joined up with more coordinatedefforts at policy dialogue by other develop-ment partners. Some important examples ofpolicy influencing based on project experi-ences do exist, but most take place within avacuum – reducing the likelihood of bothwider strategic policy engagement and longerterm institutionalization.

Development impact (Chapter 2 page 42)

4.11 Central to IFAD’s approach to developmentassistance is the bottom-up approach to projectdesign and implementation, which not onlycreates greater project ownership but also thelikelihood of higher and more sustainedimpact on the poorest groups.

4.12 While the IEE finds several good examples of such impact being achieved, direct andindirect impacts arising from IFAD investmentsare rated as modest overall. Improved crop

production for own consumption is a key areaof significant impact, together with improvedincomes from crop sales. In contrast, improvedwellbeing from employment, improvements inroad and water infrastructure provision as wellas financial services are rated as being modest,with some crucial exceptions. Impact is alsobelow expectations in IFAD’s new core area ofsocial capital and empowerment. The share ofbenefits from IFAD investments going to thepoorest groups was also found to be modest,either because of benefit capture by less poorgroups in the target area or because IFAD hadexplicitly targeted beneficiaries outside of thepoorest areas.

4.13 The few stellar projects that do stand out haveproduced significant gains in terms of house-hold food security and individual and commu-nity empowerment. Reasons include a highlyinnovative approach to beneficiary involve-ment, flexible project design, responsiveproject management with significant commu-nity involvement, and a highly engaged CPMworking largely outside of constraints fromIFAD headquarters.

4.14 Overall, only half of the projects sampledachieved more than a modest impact onpoverty. Not so very different from other IFIsworking in rural development, but evidenceenough that, for an organization specializingin rural development, much more needs to bedone to deliver key development results andan enduring contribution to achievement ofthe MDGs.

4.15 An interesting and encouraging finding fromthe sampled projects has been the positiveimpact of components dealing with health,education and water supply. These non-tradi-tional investments occur in a minor proportionof projects, and often depend on cofinancingby a partner, but they address the wider ambi-tions of the MDGs, have brought tangiblebenefits and should, where clear value-addedcan be demonstrated, be continued.

4.16 Achieving sustainability is a fundamentaldevelopment challenge. IFAD has attemptedto address sustainability by ensuring moreincentive compatible project designs,

81

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

82


addressing capacity gaps and ensuring localownership. Nevertheless, the likelihood of thebenefits of IFAD investments being sustained is modest overall. This puts it on a par with theWorld Bank and others, but raises questionsabout the value-added of IFAD investments.

4.17 Over the IEE period innovation has become araison d’etre for IFAD. The IEE concurs thatinnovation and experimentation are criticalareas for the Fund, but finds major shortcom-ings in the delivery of expectations around theinnovation agenda, a lack of clarity about whatinnovation means operationally, and atendency to view innovation as the end ratherthan the means. Even the most basic aspects,such as building the objectives of innovationand scaling-up into project designs, has notbeen done systematically. A new initiative tomainstream innovation offers promise andrecognizes the need for cultural change. Butto be effective it will need to link more closelyto planned changes in human resources, andset targets and objectively monitorable indica-tors that reveal the extent of progress.

Corporate performance and the business model (Chapter 3 page 55)

4.18 Like other IFIs, IFAD works in contexts thatare both complex and challenging. Its targetgroup is often hard to reach; it also works with multiple partners, both local andnational. Delivering on an effective assistanceprogramme is undeniably a major challenge inthese circumstances – but IFAD was foundedprecisely to address this challenge and needsto ensure that its systems, practices and proce-dures are fully aligned to ensure maximumdevelopment impact.

4.19 IFAD developed an efficient business modelto address its original mandate. But theevolving agenda and major changes in theworking environment at country level showthe current business model to be unsuited todelivering a high quality development assis-tance programme. Initiatives to addressimprovements in IFAD’s ways of workinghave not been lacking over the IEE period, in fact, the period is marked by an almostcontinuous attempt to re-wire the organiza-tion to changing global agendas, but the view

of the IEE is that change has neither gonedeep enough nor far enough.

4.20 IFAD’s business model urgently needs atten-tion. Past change efforts, while mobilizingstaff and management to focus on keybusiness practices have not yet succeeded inimproving the Fund’s development impact.Obstacles to change are rooted in the visionof IFAD as largely a funding institution, andfrom the organizational culture and practicesthat have evolved around it, especially thearms-length relationship with partners atcountry level, through consultants and coop-erating institutions, and the headquarters-centred operation.

Corporate governance (Chapter 3 page 60)

4.21 Current arrangements for governance meetthe basic requirements of the Fund, but acrowded agenda, a lack of training andguidance for Board Members, and shortduration meetings have limited the executivefunction of the EB, including the space toarticulate a clear focus on development effec-tiveness. The basic structures of the Board,Audit Committee and Evaluation Committeeare adequate. Other constituents of goodgovernance such as the IA, the OversightCommittee and an Independent Office ofEvaluation, are in the main relatively recentinnovations but have positive potential.Improved functioning of the Board, togetherwith better quality assurance and perform-ance management processes at corporatelevel have the potential to deliver strategicdirection and leadership.

Policy development (Chapter 3 page 64)

4.22 IFAD has relatively few operational or sectoralpolicies and few governing internal processesand procedures. Policy development is lowcompared to other IFIs and appears not tohave kept pace with IFAD’s evolving strategicagenda. Typical examples are guidance foreffective pro-poor targeting of project inter-ventions, and arrangements for the sustainablemanagement and maintenance of infrastruc-ture. A resistance to normative models haslimited the development of formal policyguidance and the subsequent development ofoperational instruments to take the strategic

83

agenda forward. Awareness of policies andstrategies among partner organizations atcountry level and internationally is low. Recent attempts to address the policy ‘deficit’,including the creation of a Policy Division,have potential. Ambitions to provide leader-ship and policy influence – with the recentexception of rural finance - have not yet bornefruit. The implications of a policy influencingrole were not picked up in terms of skillsdevelopment and the implications for organi-zation and management of CPMs. These arekey shortfalls in the current business model.

Human resources management (Chapter 3 page 66)

4.23 People are IFAD’s major operating cost and itsonly source of initiative and action. They arelargely responsible for the achievement ofIFAD’s development results. Human resourcespolicies until recently have been conservativeand administration based and focused only onfull-time staff. Wider change processes havebeen slow to recognize the importance ofhuman resources, in particular the demands ofbecoming an innovative, knowledge institutionand a catalyst in the wider international arenademands not only new ways of working to gainvalue from consultants, but also new skills andcapabilities from IFAD staff.

4.24 The staff survey reveals a corporate culturethat has not adequately focused on accounta-bility and performance, that lacks real trustbetween management and other employeesand where both horizontal and verticalcommunication links have, until recently,been unsatisfactory.

4.25 The new human resources policy containsthe right elements to tackle these problems,but it is not clear that the nature and the full magnitude of the changes to the role ofline managers and the culture of the organi-zation is fully appreciated, or that there issufficient support for the changesthroughout the institution.

Partnerships (Chapter 3 page 69)

4.26 Partnership working is vital to support theFund’s objectives of being a catalyst and inno-vator, but objectives set out in the Fifth and

Sixth Replenishments have led to an uncriticaluse of the term and failed to foster clear objec-tives and improved ways of working. Strategicalliances have been entered into only sporadi-cally, with the notable exception of BSF andothers, there is little evidence of enhancedimpact. A much more systematic and selectiveapproach to partnerships and alliances isneeded to promote innovation and scaling up,and to maximize potential value added inpursuit of the MDGs.

Knowledge management and learning(Chapter 3 page 76)

4.27 Knowledge and learning are critical to aneffective aid organization and shouldunderpin IFAD’s role as an innovator andleader in rural poverty reduction. Efforts topromote knowledge management havefollowed similar models to other develop-ment organizations, but weaknesses in orga-nizational culture in IFAD have underminedimprovements in knowledge management.The approach has lacked clear leadershipand a sustained commitment by seniormanagement. Knowledge management hasalso suffered from poor performance inarrangements for project-level M&E frag-mented database systems, limited use of selfevaluation and a lack of analysis of qualityassurance and portfolio performance.Recent initiatives have made an appreciableimpact, in particular the disclosure policyand website, but there is a substantialcommunications gap, with low levels ofawareness about IFAD’s polices and strate-gies among partner organizations, both atcountry level and internationally.

Management processes (Chapter 3 page 71)

4.28 IFAD staff and management have not beenslow to re-examine processes and initiatechange. The need for change in the underlyingbusiness model has been recognized, but theroles of the CPM and of the regional divisionshave stayed largely the same and many of theinitiatives to improve project cycle and qualityassurance processes have not been carriedthough to effective outcomes. There is little orno evidence that these changes have improvedefficiency. Arrangements for resource allocationand budget management have not promoted

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

84


performance-related practices, and the hori-zontal layers of senior managers and directorshave encouraged consensus decision-making.The number of CPMs and the PMD organiza-tional structure impose constraints on ways ofworking. In particular, the CPM-country one-to-one relationship has been under pressurecoping with the additional demands made forCPMs to develop strategy, identify and planloans and grants, manage partnership relationsfor innovation and scaling-up, engage in policydialogue with governments and developmentagencies, and participate in self evaluationsand knowledge management. The pressureshave resulted in even greater use of consult-ants and less time for all CPMs to engage inpeer review that all acknowledge is essential.

4.29 Communication across departments isimproving in the new horizontal manage-ment team approach, but still IFAD remainschallenged by its lack of integrated ways ofworking, a systematic approach to perform-ance management and a medium to longterm strategic planning and budgetingprocess. Much of its long term planning istainted by the constant pressure to raise andmobilize resources through the replenish-ment round – and the external agendaswhich drive that process – and through individual project investments. The flatmanagement structure has failed to provideadequate strategic direction and drive tofollow through on change agendas. Staff are,on the whole, sceptical about the efficacy ofnew change processes.

Future prospects4.30 A thread runs through this tale, and that is

the role of senior management. Many of thechanges that were promoted had a soundrationale and compare well with work inother development agencies. But so manyinitiatives have wilted before they flowered.Necessary practices such as constructive chal-lenges to projects and country programmesduring design; development of policies andstrategies that define IFAD’s comparativeadvantage; and the promotion of a learningorganization have all been started but notcarried through to the point where systemicchange has occurred. The flat management

structure and a tendency to consensus-baseddecisions is a central factor. But perhaps themost important reason for the limited successof many of these initiatives has been the lackof focus on development effectiveness.Changes have not been driven by analysis ofperformance, and management has notrecognized the centrality of improved humanresource management to realize change. Thefindings of the 2001 evaluation of innovationoffered a way to open the debate on theunderlying problems about how IFAD works.But it came at a difficult time with a newPresident facing a financial crisis, and as aresult it was passed over. That failure toconfront the underlying problems is sympto-matic of the era and highlights the challengeto be faced in the immediate future.

4.31 IFAD is at a crucial point in its history. IFAD’smandate is relevant but its distinctiveness is indanger of being overshadowed by a rapidlychanging development context. The level ofperformance that emerges from this evalua-tion shows that, with many notable achieve-ments to its credit, there is no room forcomplacency. If the Fund is not able toachieve a better performance, or one that ischaracterized by more distinctive innovationand focus on poor people in hard-to-reachcircumstances, member countries inevitablywill ask if it is efficient to continue disbursingthose loans and grants through the Fund.There is a clear choice. Management cancontinue to introduce isolated reforms andreact to changing circumstances but theevidence from this evaluation provides littleassurance that such an approach will improveperformance. Or it can make as the IEErecommends a fundamental shift onto a newpath. The recommendations set out next aredesigned collectively to achieve that change.The advent of the Seventh Replenishmentoffers a clear time frame during which resultsshould be delivered and assessed before thenext replenishment.

85

Recommendations

4.32 The following recommendations build on the processes of change which have engagedIFAD in recent years, and to tackle the underlying issues that have held back theirsuccessful adoption. The findings of this eval-uation point to a management failure by theFund. Problems have been identified, actionshave been taken, but little has been seenthrough to change the way IFAD works. The challenge in the future is to define andimplement a programme that succeeds whereprevious attempts have failed. The Board willneed to consider carefully how such an actionprogramme can be developed and managedin order to ensure a successful outcome.

4.33 The operational and strategic responsibilitieslie with management, who need to tackle theperformance shortcomings identified in thisevaluation and set in place new arrange-ments to change the corporate culture thatwill enable the Fund to reach its potential.The magnitude of this change should not be underestimated. Management will needboth support and direction. The role of theBoard is to direct the change and create aconducive environment that establishesIFAD’s role for the medium term. Therecommendations are listed here anddescribed in the following sections.

n Managing the change

n Address causes of low impact

n Develop a new business model

n Adopt smarter ways to encourage skills and learning

n Clarify IFAD’s strategic niche; re-assert itscomplementary role

n Provide direction for development effectiveness

Managing the change (paragraph 4.28)150

4.34 There is a need for deep and far reachingchange in the organization. A key question forthe IEE has been to understand why thecurrent management structure has not deliv-ered the necessary change in the past. Theevidence about previous change processes

supports the IEE conclusion that they havelacked strategic drive and commitment bysenior management - a consequence of theconsensus decision-making that the flatmanagement structure has fostered. Thecurrent President has demonstrated a resolveto improve performance, but global strategyand external relations leave little time forexecutive management. The most difficulttask is to come up with a change plan that iscredible in the face of past failures, and isconvincing both to the Executive Board andto staff.

4.35 IFAD must plan how to approach the processof change, taking into account lessons andexperience from management of change.151

Firstly, business process redesign is consideredby management experts to be a type of trans-formation that is challenging and hard tomanage. Secondly, evidence from the pastsuggests that staff will respond well to goodcommunication and the opportunity to nego-tiate details and arrangements. But there hasbeen resistance, especially in the degree ofcommitment shown by managers, and thisneeds to be overcome. Thirdly, in view of theevidence about development performance, the stakes for IFAD are high and the outcomecannot be left to chance. This is exacerbatedby the short time frame to show results by thenext replenishment. In such circumstances,management theories argue for a moreauthoritarian approach to change. The IEEtherefore recommends as an overarchingobjective that a person is appointed with broadexecutive powers and charged with the task of setting performance-based objectives anddriving through change to revitalize the Fund.This executive would be empowered aschampion of change; he or she would bringrenewed energy and create a sense of urgency.A managing director would be one option forthis role. Clear leadership and executive

150 The paragraph numbers next to each heading referencethe reader back to the relevant text in the precedingsection on conclusions.

151 See, for example, Jick (1991); Mohrmann et al (1989);Bennis et al (1969)

1

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

86


authority is necessary to achieve success and alater recommendation highlights the criticalrole of the Executive Board in this process.The IEE recognizes the cost implications ofsuch a change but considers that the positioncould replace the current post of VicePresident. A first task would be to develop astrategy for change with targets, indicators anda time-line that delivers measurable changebefore the 8th Replenishment. Such a planmust have practical and credible objectivesand must build on complementary changesuch as implementation of the humanresources policy.

Address causes of low impact (paragraphs 4.11 and 4.28)

4.36 The evaluation has identified that develop-ment performance of IFAD projects andcountry programmes must be improved.There is good evidence from the World Bankthat improvements can be achieved throughtried and tested techniques of portfoliomanagement.152 Actions include independentquality assurance reviews; portfolio analysis of key themes such as arrangements fortargeting, gender, M&E; self evaluation ofprojects; transparent analysis of portfolioperformance linked to staff accountability;development of internal and operationalpolicies and good practice guidelines, etc. The Fund needs to instigate rigorous means to improve quality.

4.37 Current arrangements for quality assuranceshould be updated with a broader mandate to examine new designs and project cycleprocesses, such as supervision and mid-termreviews, with transparent analysis and linksthrough staff accountability to humanresources performance management. Themechanism should draw on internal skills but,owing to the small size of the Fund, include a significant proportion of outside experts.Related improvements for portfolio analysisare needed, for example, in data managementand accessibility through the PPMS and PSRs.

4.38 In line with current proposals the mandate ofthe Evaluation Committee should be extendedto include evaluation of self-evaluationprocesses and of policies.

Develop a new business model (paragraphs 4.2 4.9 4.18 4.22 4.23 4.28)

4.39 The need for a new business model is evidentboth from the expanding scope and breadthof work of the Fund, as well as the changingapproaches to development planning andorganization at country level. The free-agent,arms length CPM model is no longer efficientgiven the wider range of skills beingdemanded of IFAD and the need to promotethe catalytic, innovative style of work to whichthe Fund aspires. The main concerns in themodel are the configuration of resources, waysof working, and the definition and differentia-tion of instruments (discussed below).

4.40 Recommendations here call for a significantchange in the way resources are used and start by asserting that PMD is understaffed forthe extensive roles demanded of its staff. Wepropose that a zero-base budgeting exercise beused to re-assess needs across IFAD as a whole.The funds currently used for consultants, fortemporary staff, for supervision, and savingsfrom the SCP, should be pooled and reallo-cated, partly into new staff positions and a newstructure for the CPMs, regional divisions andtechnical support in PMD. Most importantly,a move away from the CPM-country one-to-one relationship, will provide countries with a better, broader-based service with access to a wider pool of skills and ideas.

4.41 It remains to be seen how PBAS can be inter-preted to influence new operations. A moredifferentiated approach is likely to require ahigher focus on the types of instruments andmix of policy dialogue, grants and loans IFADneeds to apply in low performing countriesand difficult policy environments. Furtherdevelopment of the COSOP is an importantpart of this process. The IEE supports movesby IFAD’s management to make the COSOP amore effective filter for projects and todemonstrate closer links to national develop-ment strategies and the work of other donors.

4.42 Development of operational policies has beenslow, with a resistance to normative working.But the Fund would be better placed toengage in policy influence with a more struc-tured set of internal policies and guidelines.

2

3

87

Urgent candidates for policy developmentinclude a policy paper setting out how IFADinstruments are best deployed in differentcountry contexts including those being ‘fast-tracked’ as part of the United Nations MDGinitiative; a framework for medium termengagement with national poverty reductionstrategies and how IFAD can best work incountries where weak governance continues tohold back options for development financing;lessons and techniques for poverty targeting,support for newly created institutions and thedevelopment of approaches for sustainabilityin areas such as maintenance of infrastructure.

4.43 The Fund should also prepare a strategy forpolicy influencing at country and internationallevels, linked to a revitalized knowledgemanagement process and building on therecent good work to improve communications.This should include consideration of howgrant funds can be used for analytical work at country level in support of countryprogrammes and to inform policy dialogue.

4.44 The new human resources policy will be anintegral part of changing the way IFAD worksand implementation of that is a separaterecommendation. In addition, there shouldbe a thorough and detailed review of compe-tency against policy objectives. Skills in areassuch as rural poverty analysis, policydialogue, partnership working and innova-tion need to be taken into account moreforcefully and in more detail than just aninventory of staff qualifications.

4.45 The OSC/TRC/PDT approach to peer reviewand quality assurance has a sound structurebut fails to perform. The approach needs tobe re-examined to provide an effective chal-lenge to new policies, programmes andprojects at an early stage in their develop-ment, so that peer review can be effective. In such a small organization it is hard tocreate an effective independent challenge.Approaches to be explored could include theuse of independent external organizations orconsultants to act as reviewers. Peer reviewrequires sufficient staff time and genuineindependence. The pressures on CPMs andthe non-competitive culture make both hard

to achieve within the Fund. The process ofdeveloping new ideas for loans and grantsshould include an element of competitiveaccess to preparation funds so that resourcescan be applied to the best new proposals.

4.46 The Fund should be released from the long-standing restriction on project supervision.Evidence is not yet available about the costsand benefits of direct supervision. The IEEdoes not necessarily advocate direct supervi-sion in all cases and envisages a future modelin which the majority of supervision iscontracted out under arrangements such asare currently used, but that decision shouldbe left to management to enable flexibility insupport according to the country circum-stances and age of the project. The supervi-sion model should be sufficiently flexible toenable borrower countries to play a strongerrole, and to encompass working throughdiverse organizations including commercialconsultancies and local NGOs in countrieswith vibrant civil societies.

4.47 Evidence about the current pilot field presenceis inconclusive in this evaluation. The currentField Presence Pilot Programme has not beenunderway for long enough to evaluate,although the examples seen during thecountry visits suggest that liaison officer postsare not perceived to be at a sufficiently seniorlevel to be effective. Equally, the strongevidence in support of the outposted CPM in Peru is not reinforced by the Bolivia CPE,currently being finalized by OE. The IEEconsiders there is evidence that outposted staffbring substantial benefits, but not to the extentthat decentralizing all CPMs would be a wisepolicy. We consider there is sufficient evidenceto argue that outposting of CPMs should beexpanded with, for example, a special focus onlarge country programmes, or those with a

152 The quality assurance group established in the World Bankin 1995 and reporting to the President has been instru-mental in raising the quality of project supervision andoverall project quality in the Bank. The quality assurancegroup has pioneered many low cost, real time methods of assessment.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

88


high innovative content or in difficult environ-ments. Such a policy should be at the discre-tion of management, and we would envisage a large minority, such as 30 per cent, to beoutposted in the future for flexible periods oftenure without any fixed commitment to anyparticular countries. A flexible approach tooffice infrastructure should be used to helpmoderate costs. Such field experience shouldbe seen as a highly positive attribute in HRperformance assessment.

Adopt smarter ways to encourage skills and learning (paragraphs 4.10 4.20 4.23 4.26 4.27)

4.48 The core recommendation here is that thenew human resources policy should be imple-mented in full. The human resources reportprepared as part of the IEE noted the ambi-tious changes that this policy seeks to intro-duce. It has the potential to move the Fundaway from a culture that reflects the UNCommon System, with rigid distinctions,grading structure, bureaucratic requirementsand entitlements culture. The HR pilotprojects (of which IFAD is one) are a bold stepto move away from this towards a moremodern approach. Such a change will need tobe supported from the highest level, to ensurethat there is a relevant culture change andskills improvement in the HR Division, andtake advantage of the forthcoming set ofretirements from senior grades and the intro-duction of managerial competencies to changethe management style in the organization.

4.49 The difficulties IFAD has experienced intrying to develop in-house learning andknowledge management are documented inthe evaluation. It is unreasonable for anorganization of IFAD’s size to be able tohouse all the skills necessary for its complexportfolio and evolving policy environment.The recommendations here concern a newhigh-level partnership and new ways ofworking with consultants.

4.50 We recommend that the Fund engages in a strategic long-term partnership with a devel-opment organization of world-class standing inresearch and policy analysis. Such a relation-ship would have to be a true partnership, with

the joint-venture able to be innovative andtake risks, and not an employer-consultantrelationship. The partnership would deepenunderstanding and technical knowledge ofpolicy issues and relate them to IFAD’s fieldexperience. The arrangement could includestaff exchanges, joint development of a clearerset of policy products, and action-research inthe context of IFAD operations.

4.51 Improved record keeping and performanceassessment is needed in general for allmanagement of consultants. The expandeddemands made on IFAD have led to increaseduse of individual consultants, which hasbrought an added burden on CPMs tomanage them and on the organization to copewith the contractual administration. In orderto help the Fund tackle the new demands ontechnical supervision and quality assurance,we recommend that the Fund looks beyondworking with individual consultants andengages commercial and not-for-profit organi-zations to manage operational tasks such asaspects of the grants programme, portfolioreviews, quality assurance and technical super-vision. Such contracts would include a specificaim to develop and disseminate lessons andpolicies. This would retain the IFAD-consul-tancy work relationship, but bring added valuein terms of institutional learning and account-ability, both of which are lost in the currentsystem. It would also reduce transaction costsassociated with individual recruitment. Thisapproach is used extensively by some bilateraldonors and some multilateral developmentorganizations, whose experience can be usedto inform the Fund.

4.52 The Fund’s approach to knowledge manage-ment needs to be revitalized. Awareness ofthat can be seen in the proposals for theinnovation management initiative, for whichknowledge management is essential. Butknowledge management needs to be tackledin a more comprehensive manner, as it hasthe potential to reinforce other processes,such as improved quality assurance andhuman resource management. A key elementis to improve learning from IFAD’s ownprojects and partnerships. Project level M&Eremains weak and needs to be re-energized

4

89

with dedicated technical support in PT orthrough a consultancy or partnership. But in addition, the Fund should commissionrandomized empirical testing153 of innova-tions during implementation on a smallsample of projects considered to have highlearning potential. These could be under-taken through the research and policy part-nership or via consultants. Self evaluation alsoneeds to be developed so that all staff partici-pate fully in objective performance reviews ofloans, grants and policy, rather than defer-ring responsibility to independent evaluation.

4.53 At a more detailed level the Fund shouldsupport reforms under the HR policy andimprove the visibility of the work of individualmembers of staff by: ensuring that all reports,including Board papers, display the names ofthe authors; creating incentives for publica-tions relevant to IFAD’s mandate; andrewarding initiatives to take career breaksthrough exchanges with other developmentorganizations, especially at field level.

Clarify IFAD’s strategic niche; re-assert its complementary role (paragraphs 4.3 4.17)

4.54 IFAD’s specialist mandate provides a powerfulimperative for the Fund to take a central rolein demonstrating how different approaches torural development reduce poverty andcontribute to achievement of the MDGs. Thisrequires IFAD to clarify its role primarily as aninnovator in policy, institutional and opera-tional terms rather than as a purveyor of fairlyroutine projects which closely mirror theapproaches of larger development organiza-tions. IFAD’s comparative advantage does notlie in competing with the other IFIs but inbeing a progenitor of well tested innovativeideas and approaches that can be replicatednationwide by others with greater resources. Amore systematic approach to innovation wouldhave certain key features. Firstly a link to aknowledge management system in whichlessons from IFAD and other sources are iden-tified and disseminated; secondly recognitionof innovation in project designs by inclusion asan objective with associated arrangements forflexibility, risk-taking and evaluation; andthirdly the identification of potential partners

for scaling-up at the commencement of projectimplementation to foster ownership andshared learning.

4.55 While this is not new ground for IFAD, theevaluation evidence is clear that the Fundneeds to clarify this role better, linked to morestrategic selectivity at country and sector level.The approach should involve more risk andpossibly more failure with greater attention tolearning from and disseminating results. Thisimplies a more discriminating management ofgovernment and non-governmental partner-ships and the strategic use of grants whererisks are significant. By emphasising learningthroughout its operations IFAD could substan-tially increase and deepen its contribution topolicy change, both locally and internationally.A more proactive approach to policy engage-ment is crucial, more clearly defined andbetter resourced. IFAD needs to ensure that ithas both the instruments and the capacity todeliver. The new IMI offers promise todevelop this capacity (see Annex 4 Box 8).

4.56 Greater flexibility is needed in matchinginstruments, particularly loans and grants, tospecific areas within IFAD’s niche. Acquiringand testing innovative approaches is risky andfew if any countries can be persuaded toborrow for these purposes. Moreover, fragilestates and the poorest countries need moregrant assistance than loans and grantresources are needed for policy dialogue aswell. These arguments strongly suggest theneed to expand the grant resources availableto IFAD, if the organization is to be held toaccount for performing in its niche areas.Successful innovation is geared towards replication and scaling-up which, like policyinfluence, need to feature as specific and measurable objectives of projects if CPMs areto be held accountable.

153 This approach uses experimental designs or close approxi-mations to enable change to be attributed to develop-ment interventions. It would provide authoritativeevidence about the success or failure of innovations andenhance learning for IFAD and its partners.

5

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

90


4.57 IFAD’s small size should mean greater flexi-bility and an ability to engage where otherorganizations find they are constrained. Butthe vision is not one of IFAD going it alone.The current climate places a significantpremium on partnerships and alliances andIFAD needs to develop smarter, more strategicways of engaging at both local and globallevels to ensure that its work directly comple-ments the work of others. The strategic part-nership described above could provide thenecessary analytical skills to support IFAD’sinnovative approaches in the field, engage inexperimental learning and test new ideas forpolicy engagement. In this context, the workof the Policy Division needs to be underlinedwith a clear statement of role and objectives.

Provide direction for development effectiveness (paragraphs 4.4 4.21 4.27)

4.58 The drive towards improved performancemust be mandated by the Board. Reformproposals in recent years confirm the Board’scommitment, but limitations in organization,meetings and the choice of instruments haveheld back real progress. As regards organiza-tion the Board needs a mechanism to scruti-nize work programmes and budgets moreeffectively and to examine development effec-tiveness – by which we mean the efficiency offormulation of new projects, programmes,policies and partnerships; their effectivenessduring implementation and their developmentoutcomes. Changes to the scope of work of theAudit and Evaluation Committees are neededto enable these tasks to be undertaken fromthe current structure, though evaluation mightbe better renamed a committee of develop-ment effectiveness under such a model.

4.59 The current low frequency and limitedduration of meetings is one cause of thecrowded agenda and the Board shouldconsider ways to lengthen the time it isconvened in order to devote sufficient time todeal with all business at an appropriate level ofdetail. The number and duration of meetingsis within the power of the Board to change.The IEE also recommends that approval ofmost loans and grants should be delegated tothe President, subject to the other recommen-

dations for quality assurance being imple-mented. The Board would always retain theright to scrutinize projects, especially those of an innovative or unusual nature. Thesechanges would enable more time to bedevoted to policy development and develop-ment effectiveness.

4.60 The ability of the Board to provide leadershipwill depend in part on the quality of informa-tion it receives, and in part on the ways inwhich information is used. Executive manage-ment of an international public body is aserious task and the post of Executive Directorshould at least have clear terms of reference to define the role. In view of the infrequentmeetings and activities of the Board, aprogramme of training and support to newdirectors might hasten the speed at which they become effective in their role.

4.61 As regards quality of information, manage-ment systems to analyse the portfolio and itsprocesses need to be revitalized to provideDirectors with an analytical view of operations,rather than isolated glimpses of single coun-tries and projects through evaluations. This isparticularly important for the Board to fulfilits role of monitoring the process of change,by examining indicators of portfolio perform-ance and milestones that track implementationof the change agenda. Specifically, qualityassurance, PPMS, the PSR process, self evalua-tion and cost analyses of staff and consultantsactivities are all in need of improvement andeasier accessibility.

4.62 A key area for the Board to set standards is toensure that new partnerships, working rela-tionships, supplementary funds and policyinitiatives all have clear and realistic objectivesrelevant to IFAD’s mandate, and have objec-tively monitorable indicators of developmentoutcomes. Greater benchmarking of theperformance systems and standards of otherIFIs would enable Directors to be more effective in directing management.

6

Ashley, C. and Maxwell, S. (2001) Rethinking ruraldevelopment. Development Policy Review, SpecialIssue, Volume 19, Number 4

Bennis, Warren G., Kenneth D Benne, and RobertChin (Eds) (1969) The Planning of Change. Hole,Rinehart and Winston, New York

Beynon, Jonathan (2003) Poverty efficient aid alloca-tions – Collier/Dollar revisited. Economics andStatistics Analysis Unit Working Paper 2. ODI,London

CENTRIM and IDS (2001) IFAD’s capabilities as apromoter of replicable innovations. Brighton, Sussex

Chronic Poverty Research Centre (2004) The ChronicPoverty Report 2004-05 www.chronicpoverty.org

Coleman, G (1990) ‘Problems in project-level moni-toring and evaluation: evidence from one majoragency,’ Journal of Agricultural Economics, 41, pp149-161

Cooke, Rodney & 16 others (2000) Report of the workinggroup on impact achievement through the project cycle(IAPC) (mimeo)

COP and IFAD (1999) MoU between COP and IFAD forthe Global Mechanism

Dabelstein, Niels, Victor Hugo Morales Melendez,Adnan Bashir Khan, Lothar Caviezel, OsvaldoFeinstein, Gabriel Lombin, Robert van den Berg(2002) External Review of the Results and Impact ofIFAD Operations. Copenhagen

Head, Ivan L., Ali Ahmed Attiga, Martin EnriquePineiro, Yves Rovani and Djibril Sene (1994) TheChallenge of Rural Poverty: The Role of IFAD. Reportof the Rapid External Assessment of IFAD. Rome

IFAD (1976) Agreement Establishing the International Fundfor Agricultural Development (revised through to 1997)

IFAD (1977) By-laws for the Conduct of Business of IFAD(Revised 1997). Rome

IFAD (1977) Financial Regulations of IFAD (Revised1997). Rome

IFAD (1977) Rules of Procedure of the Governing Council(Revised 1997). Rome

IFAD (1978) Lending Policies and Criteria (3rd Revision,1998). Rome

IFAD (1993) Report of the PMD Task Force on the rationali-zation of the project cycle – Draft (mimeo)

IFAD (1994 to 2003) Programme of work and administra-tive budget for IFAD (Annual). Rome

IFAD (1995) Proposed Project Development Cycle:Recommendations to the Re-engineering Task Force.Draft (mimeo)

IFAD (1998) General conditions for AgriculturalDevelopment Financing. Rome

IFAD (2000) Detailed design of the process re-engineeringprogramme EB 2000/71/R.9 Rome

IFAD (2000) Evaluation of the IFAD/NGOs ExtendedCooperation Programme: Executive Summary andAgreement at Completion Point, IFAD, Rome

IFAD (2000) Highlights and News: 10th IFAD/NGOConsultation: Tripartite Partnerships for poverty allevia-tion and food security through programmes, 30 May-2 June 2000, Pune, India

IFAD (2000) Partnerships for Eradicating Rural Poverty.Report of the Consultation to Review theAdequacy of the Resources Available to IFAD2000-2002. GC 24/L.3.

IFAD (2000) Rural Finance PolicyIFAD (2000) Strengthening IFAD’s support to the develop-

ment of effective and efficient monitoring and evaluationsystems methodological study (Interim Paper).Evaluation Committee – Twenty-Fifth Session EC2000/25/W.P.3/Rev.1 Rome

IFAD (2001) Evaluation of IFAD’s capacity to promotereplicable innovation Phase I, A preliminary assessment.Office of Evaluation and Studies. Rome

IFAD (2001) Executive Board, Minutes of the Severty-FourthSession. Rome

91

Bibliography

Annex 1

IFAD (2001) Guidelines. Project Design Document andKey File (Revised 2003). Programme ManagementDepartment. Rome

IFAD (2001) Process reengineering programme(Design stage) Impact delivery and communicationprocess: Operating model and business case Rome

IFAD (2001) Strategic Framework for IFAD 2002-2006.Rome

IFAD (2002) Procedure for the Review of Country StrategicOpportunities Papers (COSOPs) by the ExecutiveBoard. EB2002/77/R.12 Rome

IFAD (2002) Annual Report 2001. RomeIFAD (2002) Annual Report on Results and Impact of

IFAD OperationsIFAD (2002) Enabling the Rural Poor to Overcome their

Poverty: Report of the Consultation of the SixthReplenishment of IFAD's Resources (2004-6)REPL.VI/5/R.2

IFAD (2002) IFAD’s capacity as a promoter of replicableinnovations in cooperation with other partners.Understanding at Completion Point and ExecutiveSummary. OE Report No. 1325. Rome

IFAD (2002) Progress Report on the IFAD V: Plan of Action(2000-2002). Executive Board – Seventy-SeventhSession EB 2002/77/R.31 Agenda Item 13. Rome

IFAD (2002) Progress Report on the Project Portfolio.Executive Board – Seventy-Fifth Session EB 2002/75/R.12/Rev.1 Agenda Item 6. Rome

IFAD (2002) Strategic Framework for 2002-2006: Enablingthe rural poor to overcome their poverty, March 2002,IFAD, Rome

IFAD (2003) A Methodological Framework for ProjectEvaluation. Office of Evaluation and Studies. Rome

IFAD (2003) Evaluation of IFAD’s Technical AssistanceGrants Programme for Agricultural Research. ReportNo. 1377. Office of Evaluation

IFAD (2003) Evaluation of Supervision Modalities in IFAD-supported Projects, OE

IFAD (2003) Framework for a results management system forIFAD-supported country programmes EB 2003/80/R.6Rome

IFAD (2003) IFAD Evaluation Policy. Executive Board –Seventy-Eighth Session EB 2003/78/R.17/Rev.1Agenda Item 8(c). Rome

IFAD (2003) IFAD policy for grant financingEB 2003/80/R.5/Rev.1. Rome

IFAD (2003) Independent External Evaluation of IFAD,Terms of Reference, Para. 3 Rome

IFAD (2003) Progress Report on the Popular Coalition toEradicate Hunger and Poverty, Governing Council –25th Session, GC 26/L.11, IFAD, Rome

IFAD (2003) Progress report on the process re-engineeringprogramme (Strategic Change Programme)EB 2003/79/R.36 Rome

IFAD (2003) Progress report on the process re-engineeringprogramme (Strategic Change Programme) GC 26/L.10

IFAD (2003) Progress Report on the Project Portfolio.Executive Board – Seventy-Eighth Session EB2003/78/R.16 Agenda Item 7. Rome

IFAD (2003) Report of the panel on the performance-basedallocation system EB 2003/80/R.3 Rome

IFAD (2003) Rural Enterprise PolicyIFAD (2004) Bolivia Country Programme Evaluation.

OEIFAD (2004) Consolidated Financial Statements for the years

ended 31 December 2003 and 31 December 2002IFAD (2004) Country Programme Evaluation of Indonesia

OE, RomeIFAD (2004) Discussion paper on communications directions

for IFAD 2004-2007, Parts I and II – Draft 4. RomeIFAD (2004) IMT Report. RomeIFAD (2004) Programme of work and administrative budget

for IFAD and its Office of Evaluation. RomeIFAD (2004) Scope and guiding principles of SCP. RomeIFAD (2004) Thematic Evaluation of IFAD’s Performance

and Impact in Decentralizing Environments: Experiencesfrom Ethiopia, the United Republic of Tanzania andUganda. OE

IFAD (2005–forthcoming) Egypt Country ProgrammeEvaluation. OE

IFAD (nd) Policy and Resource Trends Likely to Affect IFADJick, Todd D, (1991) Implementing Change Harvard

Business School Case N9-491-114 HarvardMoohrmann, Allan, S. Mohrmann, G. Ledford,

T, Cummings, and E.E. Lawler Jrs (eds.), (1989)Large-Scale Organizational Change. Jossey-Bass,San Francisco

PMD (2004) PMD Retreat January 2004 Office Memorandum

Programme Management Department (2004) Report offirst CPM Forum, 14-15 April 2004

Project cycle reengineering working group (1995)Proposed project development cycle: Recommendations tothe re-engineering task force – draft (mimeo)

Senge, Peter M (1990) The Fifth discipline: The art andpractice of the learning organization. CurrencyDoubleday, New York

UNDP (2005) Investing in Development – A PracticalPlan to Achieve the Millennium Development Goals.Millennium Project, Report to the UN SecretaryGeneral. New York

Watkins, Francis and Csaky, Corinna (2003).Partnerships, Volume 1 - Literature Review EdinburghResource Centre for DFID Evaluation Department

World Bank (1992) Effective Implementation: Key to development impact. Washington

World Bank (2001) IDA’s Partnerships for PovertyReduction FY94-FY00. Operations EvaluationDepartment. Washington

World Bank (2002) Annual Report on PortfolioPerformance. Washington

World Bank (2004) Agricultural Investment Sourcebook.Agriculture and Rural Development Department.Washington

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

92

A N N E X 1 : B I B L I O G R A P H Y

IFAD

Family name Given name Division/ Office

Abdouli Abdelhamid PNAbhyankar Uday OPAngelucci Lucia International

Land Coalition

Audinet Jean-Philippe OEAureli Massimo FMBåge Lennart PresidentBaldwin Brian FCBamba Zoumana EAD/ESBeavogui Mohamed PABenassi Anna OEBen-Senia Mohamed PABettink Willem PMDBishay Mona OEBouzar Khalida Global

MechanismBrett Nigel PICalcagni Antonia FCCappai Stefania FC/LCarruthers James PMDCassani Robert ERCastellucci Theresa FMCleveringa Rudolph PTConstantinides Bambis IA

Cooke Rodney PTCorona Patricia FCDaniel Louise OEde Willebois Ides PFDellanave Carla FCDemirag Ulac PADe Petro Daniela FHDommel Henri PTElhaut Thomas PDEl-Zabri Tawfiq PNFaisal Mohammad PFFarmosi Elisabeth PMDFarrant Ruth FCFelloni Fabrizio OEFrancis David Centrim,

Brighton University

Gil Pilar PLGiorgi Carina PLGlikman Pablo PLGracia Charles FCHaidara Hamed PAHamilton-Peach Julian PTHaralambous Sappho EOHassani Mohammed PNHeinemann Edward PFHopkins Raul PLHowe Gary PFHuynh Cong My FC

93

List of people interviewed 154

154 Details of people interviewed in connection with specific linesof enquiry such as the country studies or annexes of the deskreview and final reports can be found in those documents.

Annex 2

Kayser Katherine OEKennedy Sean PTKhadka Shyam PIKherallah Mylene PNKimani Beatrice FHKingsbury David PAKlavins Sandra FCKnopf Ana FADKuhn Christiane PNLavizzari Luciano OELembo Alessandro PFLohlein Daniela EOLongo Roberto PFLubbock Annina PTManssouri Mohamed PAMarzin Sylvie PAMathur Shantanu PTMcDonald Louise PFMcGhie John FCMcGuire Sandra EC - by

telephoneMedrano Rojas Ofelia PAMelina Brigitte FHMicallef Maureen FAMoore Bruce International

Land Coalition

Muthoo Ashwani OENichols Flemming OENsimpasi Luyaku Loko PAOkong’o Miriam PFOlsson Gunilla EOOmar Rasha PN

Palmeri Chase OEPedersen Henning PNPena-Montenegro Raquel PLPugerup Kare OPQuijandria Benjamin Consultant,

ex PLRahman Abdalla PNRath Thomas PIRebuelto Ana Lia PLRenard Nadine PTRice Theresa PMDRihane Fawzi EADRodriguez Doris PLRomagnoli Giuliana FHRota Antonio PTRoy Phrang EADRubio Ladislao PLRyden Per Global

MechanismSaint Ange Perin PASaitto Allegra FCSidahmed Ahmed PTSilveri Paolo OESlama Abdelmajid PN Sorensen Jens PFSourang Cheikh OMStigliano José FMTounessi Mohamed PAToure Abdoulaye PATrupke Hermi PDVentimiglia Stefano FHWholey Douglas PTWieland Ursula PD

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

94

A N N E X 2 : L I S T O F P E O P L E I N T E R V I E W E D

IFAD Executive BoardHabiba Djerir Board Participant (Algeria)Fernández Ariel Board Director (Argentina)De Schrevel Bernard Board Director (Belgium)Moungui Médi Board Director (Cameroon)Parker Charles Board Director (Canada)Mousa Mariam Board Director(Egypt)Guillouët Alain Board Director (France)Nair Govindan Board Participant (India)Morales Víctor Hugo Board Participant (Mexico)Lombin Gabriel Board Director (Nigeria)Al-Aquil Ahmed Ben Bouleiman Board Director (Saudi Arabia)Caviezel Lothar Board Director(Switzerland)Reid Peter Board Participant (United Kingdom)Jaskowiak Mark M. Board Director (United States)Bloomgarden David Board Participant (United States)Brubaker Andrew Board Participant (United States)Pozzo Bracho Carlos Board Participant (Bolivarian Republic of Venezuela)

Other OrganizationsWorld BankBaah-Dwomoh Joseph Sector Director, Rural, Environmental and Social Development,

Central AfricaByerlee Derek Senior Policy and Strategy Adviser, Agriculture and Rural

Development Department, Socially Sustainable Development (ESSD) Network, World Bank

Ganguly SushmaLister Douglas Cluster Coordinator, Tunisia and Morocco; Acting Sector

Manager, Rural and Social, Middle East and North Africa (MENA)

Munshi Meena Senior Economist, Rural Development Sector Unit, South Asia Region

Okidegbe Nwanzi Rural Strategy Adviser, Agriculture and Rural Development Department (ARD)

Perumpillai-Essex Jeeva Rural Development Specialist (Mozambique)Piriou-Sall Suzanne Rural Development and Decentralization Specialist, Africa

Region (Guinea)Umali-Deininger Dina Lead Agricultural Economist, South Asia Rural Development Unit

IADBArbelaez Jorge Perez Policy and Programming Specialist, Strategic Planning and

Budget Departmentde Souza Francisco Senior Natural Resource Management Specialist,

Basilio F Environment and Natural Resource Management Division, Regional Operations Department 3

Gaskin-Reyes Camille Chief, Regional Operations Support OfficeKuechemann Christof Deputy Chief, Social and Institutional Development,

Department of Sustainable DevelopmentMeins Bertus J Chief, Cofinancing Division Financial Support Services

Sub-department/Cofinancing and Export Promotion DivisionPaulson Arne Chief, Portfolio Management and Project Monitoring OfficeSumpsi Jose Maria Rural Development Unit, Department of Sustainable

Development

AsDBAmerling Carl B Principal Evaluation SpecialistEdwards David Director, Evaluation Division 2, Operations Evaluation

DepartmentEichenberger Joseph Vice PresidentErquiaga Philip C Principal Director, Office of Cofinancing OperationsGeissler Guido Planning and Policy Specialist, Strategy and Policy DepartmentKiy M Zeki Senior Cofinancing Officer, Office of Cofinancing OperationsRoche Frederick C Director, South Asia Department, Agriculture, Environment

and Natural ResourcesRyu Ki Hee Project Specialist (Water Resources), South Asia Department,

Agriculture, Environment and Natural ResourcesShrestha Omkar L Principal Programmes Specialist, South Asia DepartmentSuzuki Eisuke Director General, Operations Evaluation DepartmentTritasavit Paritha Cofinancing Officer, Office of Cofinancing Operationsvan Heeswijk Jan P M Director General Regional and Sustainable Development

Department

95

1. Introduction1.1 This annex sets out the principal features of

the methodology used for the IndependentExternal Evaluation. Full detail of the plannedmethods and instruments was given duringthe design phase of the study and can befound in the IEE inception report, Chapter 4and Annexes 2, 3 and 4. This annex recapsthe salient points and provides detail abouthow the instruments were used and adaptedin the course of the study, together withobservations on the robustness of theapproach and its strengths and weaknesses.

1.2 The discussion is in three parts. First thesampling of countries and projects isdescribed. Next, the instruments used fordata collection are reviewed, starting with thedesk study and then the country visits andother sources of data. Last, the methods usedfor data collection, processing and analysis.This includes a note on the use of ratings.

2. Sampling of countries andprojects

2.1 In response to the expressed wish of theExecutive Board the selection of countries andprojects was undertaken by random sampling.A total of 21 countries (four from PA, PF, PLand PN, and five from PI) were randomlyselected for desk study, with probability ofextraction proportional to the number ofprojects in each country. Then ten countries

(two from each region) were again randomlyselected from the initial selection, again withprobability proportional to the number ofprojects in each country. None of the countriesselected for field visit presented any particularsecurity concern.

2.2 Following the country selection 42 projectsfrom the 21 selected countries and four TAGs(taken from a list of IFAD TAGs approved bythe Executive Board between 1994 and 2003,with an amount above USD 100 000) wereselected through simple random extraction.Two projects were selected in each country.In the case of PL, since only one project was

present in one of the selected countries(Chile), three projects were selected randomlyin Bolivia for desk study. Out of these three,two were randomly selected for field visit. Keyfeatures of the procedure are:n The number of countries sampled in each

region was in proportion to the share ofthat region in IFAD’s portfolio of loanprojects. Four countries were sampled in allregions except Asia and the Pacific, fromwhich five were sampled, (Tables 1 and 2).

n Countries were sampled at random withprobability proportional to number of loan projects.

n A subsample of two countries in eachregion was drawn randomly with proba-bility proportional to number of loanprojects for the country visits.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

96

Evaluation methods and instruments

Annex 3

n Two projects were sampled by simplerandom sample for detailed study in eachcountry. If a country had less than twoIFAD projects three projects were selectedfrom another country and details recorded.

n All country specific technical assistancegrants were reviewed in the sample coun-tries. An additional sample of four TAGswith total costs greater than USD 100 000was also sampled randomly.

2.3 The sample was drawn in the OE office usingnumbered balls and a ‘lottery’ machine. Theprocedure was witnessed by a group of 40 ormore IFAD staff.

97

Table 1 Regional distribution of the portfolio

Region IFAD % of Sample1994-2003 Total No.

PA Number of countries 18 19% 4

Value of the portfolio USD million 614 17%

Number of projects 55 20%

Average no. projects per country 3.1

PF Number of countries 18 19% 4




PI Number of countries 17 18% 5




PL Number of countries 25 26% 4




PN Number of countries 19 20% 4




Total Number of countries 97 100% 21



Average no. projects per country 2.9 -

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

98

A N N E X 3 : E VA L U AT I O N M E T H O D S A N D I N S T R U M E N T S

Table 2 Countries and projects for desk and field study (field selections are shaded)

Countries in Africa I Projects in Africa I

Burkina Faso South West Rural Development Project

Rural Microenterprise Support Project

Ghana Village Infrastructure Programme

Rural Enterprise Project – Phase II

Guinea Smallholder Development Project in the Forest Region

Smallholder Development Project in the North Lower Guinea

Senegal Village Organization and Management Project

Agricultural Development Project in Matam – Phase II

Countries in Africa II Projects in Africa II

Eritrea Eastern Lowlands Wadi Development Project

Gash-Barka Livestock and Agricultural Development Project

Mozambique Nampula Artisanal Fisheries Project

PAMA Support Project

Rwanda Intensified Land Use Management Project in the Buberuka Highlands

Rural Small and Micro-enterprise Promotion Project

United Republic of Tanzania Rural Financial Services Programme

Agricultural Marketing Systems Development Programme

Countries in Asia and the Pacific Projects in Asia and the Pacific

Bangladesh Small-scale Water Resources Development Sector Project

Aquaculture Development Project

India Maharashtra Rural Credit Project

Jharkhand-Chattisgarh Tribal Development Programme

Mongolia Arhangai Rural Poverty Alleviation Project

Rural Poverty-Reduction Programme

Nepal Poverty Alleviation Project in Western Terai

Western Uplands Poverty Alleviation Project

Pakistan Pat Feeder Command Area Development Project

North-West Frontier Province Barani Area Development Project

99

Countries in Latin America and the Caribbean Projects in Latin America and the Caribbean

Bolivia Sustainable Development Project by Beni Indigenous People

Small Farmers Technical Assistance Services Project

Management of Natural Resources in the Chaco and High Valley Regions Project

Chile Agricultural Development Project for Peasant Communities and

Smallholders of the Fourth Region

Guatemala Programme for Rural Development and Reconstruction in the Quiché

Department

Rural Development Programme for Las Verapaces

Peru Management of Natural Resources in the Southern Highlands Project

Development of the Puno-Cusco Corridor Project

Countries in Near East and North Africa Projects in Near East and North Africa

Armenia North-West Agricultural Services Project

Agricultural Services Project

Egypt East Delta Newlands Agricultural Services Project

West Noubaria Rural Development Project

Jordan Agricultural Resource Management Project in the Governorates of Karak

and Tafilat

National Programme for Rangeland Rehabilitation Development – Phase I

The former Yugoslav Southern and Eastern Region Rural Rehabilitation Project

Republic of Macedonia Agricultural Financial Services Project

Sampled TAGs

Programme for the Establishment of the Near East and North Africa Management Training in Agriculture

Global Cassava Development Strategy

Validation and Delivery of New Technologies for Increasing the Productivity of Flood-Prone Rice Lands in

South and Southeast Asia

Programme for Poverty Alleviation and Enhanced Food Availability in West Africa (Yam)

3. Data collection instruments3.1 In order to develop a comprehensive and

logical body of evidence to support thefindings and conclusions of the evaluation, astructured approach was used, starting withan evaluation framework, set out in the inception report, Chapter 3, Table 5. Thisframework was used to structure and developquestions from the terms of reference, as setout in the inception report Tables 6 and 7.Those questions then formed the body oftopics for interviews and data collection. As faras possible, questions were rationalized intoformal checklists and questionnaires, asdescribed here.

Desk review3.2 The largest part of the work during the desk

study phase was to review information avail-able at IFAD concerning policies and strate-gies, projects and TAGs. All reviews of loanprojects, TAGs and COSOPs made use of astructured review, proforma drafts of whichwere presented in the inception reportAnnexes 3 and 4. These ensured:

n A consistent approach by all teammembers and across all documents

n Cross-referencing of sources of informa-tion to underpin the subsequent analysisand provide a trail of evidence

n A documented summary of factual information

n For each topic, a rating or summary assessment by the reviewer (described later in section 4)

Country visits 3.3 The main objective of the country visits was

to validate and enrich the desk review andto generate new information that wouldconfirm or refute the conclusions of thedesk review155. The work was coordinatedby an international consultant collaboratingclosely with a national evaluation team, withguidance and assistance from a national evaluation counterpart identified by IFAD,to organize access to selected IFAD-supported projects and relevant officials and other stakeholders.

3.4 Drafts of the main data collection instrumentswere set out in the Inception Report, Annex 4,and comprised:

n A checklist to summarize responses fromproject management, with ratings for theproject as a whole and specific componentswhere applicable (known as Form 2).

n A checklist to summarize responses fromother stakeholders, with ratings for theproject as a whole and specific componentswhere applicable (known as Form 3).

n A questionnaire to investigate perceptionsof project benefits and impact from partic-ipants, suitable for use with individuals orin group interviews (known as Form 4).The form was structured to correspondwith topics listed under the OE method-ological framework for evaluation‘domains of impact’.156

n A set of overall evaluation ratings of rele-vance, efficiency, effectiveness and impact(known as Form 5).

3.5 With the exception of Form 5, all the Formswere designed as basic documents, which thenational evaluation team was asked to modifyin order to match the situation of the project.In most instances modifications were to dropa question because the component was notrelevant (for example, investment in irriga-tion) or to add questions to explore moreproject-specific topics (such as the competitionprocess in Peru, MARENASS). In no instancewas the phrasing or coding of responseschanged within the basic set of questions. InBurkina Faso, Guinea and Mozambique, Form4 was translated into French and Portugueseas appropriate. In Pakistan BADP, an addi-tional survey was designed to investigateevidence of benefits and impact in a fore-runner project (Manshera Village SupportProject) because the sampled project had notbeen in implementation for sufficiently long togenerate evidence of benefits. All additionalquestions were analysed in-country as part ofthe CWP report.

3.6 All teams were provided with customizedsoftware in MS Excel and MS Access, for dataentry and processing of the data. Not allcountry teams made use of the facility. In

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

100


Guinea and Burkina Faso data were enteredusing a variant of SPSS and later converted tothe ITAD system.

3.7 The international consultant and nationalevaluation team were provided with a struc-tured outline for the text of the CWP, andwere required to follow the format, thoughadditional sub-headings and annexes were permitted.

Other surveys3.8 In addition to the data collection planned

during the inception phase, questionnairesand semi-structured topic lists were designedfor the following purposes:

n The survey of staff as part of the humanresources study

n A survey of consultants to contribute to thehuman resources study and analysis ofcorporate processes

n Interviews with members of the Executive Board

n Interviews with staff based at the headquarters or regional offices of someIFIs and UNOPS

4. Data collection, processing andanalysis methods

4.1 This section deals with the methods by whichthe evaluation was carried out. Information isprovided about sources of data and the waysin which facts were triangulated by enquirywith different people or other sources.157

Next is a review of the phasing of the fieldwork. A discussion of ratings follows, withexplanation of the definitions used and meansof achieving consistency. The last sectiondescribes the analysis process.

Sources of information and practical triangulation

4.2 At all stages of the work the IEE has endeav-oured to identify as diverse a set of sources ofinformation as could reasonably be managed,and to find ways of crosschecking findings bymeans of reference to documents, perform-ance information held on databases, and theviews of key informants. The main featuresare described separately for the desk reviewand country visits.

Desk review 4.3 The desk review used several sources of data:

COSOPs or any pre-dating strategy state-ments; documentation relating to projectpreparation; implementation/completion stage documentation including supervisionreports and the project status report (PSR)summaries, and where available, mid-termreviews, completion and evaluation reports;interviews with CPMs and other IFAD staff,especially from OE and PT.

(a) Loan and grant project documentation review

4.4 The end-point of the loan preparationprocess, and the key official documentdefining each loan project (and which allsampled projects will have in common) is theReport and Recommendation of the President(RRP). The desk study assessed the individualRRP for the sampled loan projects and TAGs,leading to summary rated assessment acrossthe whole sample. Background informationwas drawn from the formulation, appraisal,technical review and OSC documentspreceding the RRP, and from discussions with the CPMs and other staff concerned asrequired. However, because the RRP repre-sents the formal corporate definition of theproject at entry, and marks the conclusion ofan extensive process of formulation, qualityassurance and loan/grant negotiation, it isappropriate that the RRP, as approved, wasthe central document of review.

101

155 IFAD (2003) Independent External Evaluation of IFAD,Terms of Reference, para. 16

156 IFAD (2003) Methodological Framework for Evaluation. OE

157 Triangulation is a process by which enquiries about atopic are made (a) with at least two and preferably threeor more respondents in the expectation that contrastingviewpoints will lead to a more accurate representation ofthe true nature of the topic being studied (e.g. for awater users association, a member of the committee,farmers with land in different locations, male and femalecultivators etc.); and (b) using more than one data collec-tion technique (e.g. a group meeting, a focus group anda questionnaire survey). The term triangulation is takenfrom the means of locating a point by surveying, for usein mapping and navigation, etc.

(b) Implementation progress4.5 Because different CIs provide supervision

reports on different formats, it is difficult tocompare supervision reports across the wholesample. However, annual PSRs exist for allsampled projects, based in large part on thepreceding supervision reports. The desk studyassessed the most recent PSRs for all sampledprojects. Where mid-term review, completion,evaluation or other documentation were avail-able, they were also reviewed.

(c) CPM and staff interviews4.6 CPMs were interviewed at various stages in

the process. All CPMs of sampled projectswere interviewed individually more thanonce (with the exception of the CPMoutposted to Peru, who was only interviewedduring the country visit phase). Clarificationand elaboration on issues arising with indi-vidual projects were sought from CPMs andother staff such as OE, with experience of the country or region. CPMs and other staffwere also invited to participate in discussiongroups on policy and strategy development,knowledge management, partnerships,human resources and project cycle manage-ment. Specific aspects of methodology aredescribed in the relevant annexes to the deskreview report.

Country visit4.7 The country visits used four main sources of

data: interviews with project management andtheir government colleagues; interrogation ofthe project monitoring data; interviews withkey partners such as NGO and other donorswith knowledge of the project and of IFAD’swork in the country; individual and focus-group interviews with project beneficiaries.

(a) Project management interviews4.8 This category includes the formally-assigned

project managers, their staff and TA, togetherwith directly implementing partners includingthe ‘parent’ government, local governmentand parastatal units, and sub-contracted andother implementing entities, and NGOs.

(b) Project M&E system4.9 Wherever possible, up-to-date project

performance data was obtained through

project office systems, to update supervisionreports and other documents available inRome. Unfortunately, most systems are moregeared to documentation of activities andexpenditure, than to issues of developmenteffects. Management information system datawere used to estimate the number of projectbeneficiaries for each project component overa given period of time.

(c) Partners4.10 This category of other stakeholders includes

development partners who have knowledge ofIFAD projects or who are familiar with IFAD’srole and activities past or present, but are notdirectly responsible for project implementa-tion (co-financiers; central and local govern-ment units; donors; NGOs; other agenciesand key informants). In a few projectinstances (Bangladesh and Egypt) thenumbers of other stakeholders who were sufficiently well-informed to be able to giveopinions on the project were so few that thisform was not completed.

(d) Beneficiaries4.11 The majority of IFAD projects are multi-

sectoral, with several different interventions.One IFAD typology lists over 50 categories ofproject intervention, several of which are likelyto be included within a single sampled project.The lag-time for these different types of inter-vention to impact on poverty varies greatly.The means chosen to assess and compareimpact information across this diversity ofinterventions, impact timescales and linkageswas to ask beneficiaries themselves how theyview the relevance, benefits, sustainability andother key features of the various interventionsthey are involved in, and to assess theirresponses on a common standard.

4.12 The terms of reference for the evaluationstressed the need for the team to quantifyimpact as far as possible (para. 11(b)). Theresource envelope available for the evalua-tion, and tight timescale, means that datacollection was restricted to single-visit inter-views with beneficiaries. This means thatmost of the information was based onmemory recall by respondents. (There weresmall but significant exceptions where

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

102


interviews were supplemented by directobservation of assets and activities.) Theresponses to questions in Form 4 must therefore be understood to be a self-assess-ment of project activities and benefits, in line with the OE definition of impact,explained later.

4.13 The approach maximized the value of infor-mation by working wherever possible withgroups using participatory techniques, andwith individuals or households to cover thesame material. Classification of the dataenables analysis to be undertaken accordingto location, project, age of the project,component cluster, and socio-economiccharacteristic of the respondents. In fact, themain value of the surveys has been toprovide a trail of evidence to support theconclusions in each CWP, though someaggregate tables from the whole data set have been used in the final report.

4.14 Country programme. Although the mainfocus of the evaluation was the two projectssampled in each country, the evaluationexamined the whole portfolio to under-stand its composition, coherence andperformance. This was done in two ways.Firstly, details of all projects: their size,objectives, location, implementation andsupervision arrangements were obtainedfrom PPMS and by discussion with CPMs.That data included the most recent PSR.Secondly, working through the nationalevaluation counterpart at country level, the team interviewed key informantsamong partners and project management(see below). To the extent that otherprojects have links to any of the sub-sectors,target groups or locations of the sampledprojects, further information was soughtabout their current implementation experi-ence and performance.

Phasing of country visit fieldwork4.15 The country visits were phased in order

to allow time both for the IEE national evaluation team and for national stakeholders,to prepare for the evaluation. Most countryvisits followed a four-stage pattern.

(a) Preparation phase 4.16 During the desk review period the nominated

international evaluator :

n Consulted with the CPM – checking avail-able data and sources and drawing up apreliminary stakeholder analysis

n Made contact with the national evaluation counterpart

n Started to shortlist candidate local consultants

n Planned a programme for the reconnais-sance visit including initial meetings withkey stakeholders where feasible

n Circulated draft briefing documents about the evaluation

(b) Reconnaissance visit4.17 All countries received a short advance visit by

the international consultant or an IEE coreteam member. This enabled initial contacts tobe with key stakeholders, providing theopportunity to introduce purpose andapproach of study in advance of more detailedevaluation discussions.

Key tasks were:n meet national evaluation counterpart

n recruit national consultant team

n refine/agree specific country approach andlocation of field work

n identify training/briefing needs of consultants

n develop and agree detailed workplan andtiming – especially preparatory work to bedone on logistics, field studies andsecondary data

n draw up roles, responsibilities and contrac-tual arrangements

n validate/modify the preliminary stakeholder analysis

n begin process of developing national timeline of key policy changes/events relating to development and poverty reduction

n hold initial meetings/briefings for key stakeholders/partners as appropriate(government and project agencies)

n obtain or locate sources of appropriatenational information/statistics and projectlevel information

103

(c) National evaluation teams had to meetthe following criteria for selection n No previous direct connection with IFAD

as per the conflict of interest in IEE TORsn Knowledge of rural poverty, livelihoods of

the poor and the agriculture sector n Experience of undertaking field level,

participatory evaluationsn Gender balance of teamn Understanding of monitoring and evalua-

tion concepts, project, programme andpolicy issues

n Evidence of ability to produce goodquality, concise summary reports

n Good reputation for quality of workamong other donors and track record ofdelivering outputs on time

n Ability to organize logistics and back upfor international consultant, the completion of the field work andreporting requirements

4.18 The intended plan was for the internationalconsultant to visit the country twice. In theintervening period the national evaluationteam was to continue field level work andsourcing secondary data as required.Exceptions were Pakistan and the UnitedRepublic of Tanzania where constraints inthe schedule of the international consultantled to a single visit being used. InBangladesh, the international consultantwas based in Dhaka and so had a morecontinuous pattern of work.

(d) First field visit4.19 This first visit focused on key stakeholder

meetings, briefing for the national team andinitial field work. Early field work wasreviewed to identify and rectify any problemsor gaps in both process and products. Keytasks were: n Meetings with project management team

and other key stakeholdersn Conduct briefing/orientation workshop

with in country teamn Initial field visits to project locations

and testing of planned methods and approaches

n Review workshop with in country team

n Detailed workplan agreed for continuationof field work and reporting

(e) Second field visit4.20 This visit was planned to take place after, or

very close to, the end of the field work. Thefocus was on review, analysis and feedback tostakeholders. Key tasks were:n Progress review with national

evaluation teamn Analysis of field data and estimation of

beneficiary numbersn Follow up/feedback meetings with

key stakeholdersn Drafting country working papern Wrap up meetings to which project staff,

representatives of government and otherrelevant stakeholders were invited

Ratings and analysis4.21 The primary tool for expressing judgements

about performance in the evaluation is theuse of ordinal ratings. This approach buildson long-established practice for evaluation ofdevelopment projects by donors and IFIs.158

Ratings were used in the IEE in three ways: n The desk review checklist of questions

required the reviewer to ‘score’ the qualityof the topic. In these instances a standardscale of rating devised by the IEE wasapplied across all questions. For example:The congruence of the project approach with the COSOP (or equivalent at the time of project inception) is: 5 very good; 4 good; 3 satisfactory; 2 poor; 1 very poor; n/a not applicable.

n Questions on the beneficiary survey Form4 were rated using the same four-pointscale as for evaluation criteria (high,substantial, modest, negligible), but theinterpretation of that scale was set out onthe form for guidance.

n The overall ratings of performance againstevaluation criteria (Form 5) were under-taken following the rating methodologydeveloped by OE, which draws on thatadvocated by the OECD/DAC and isconsidered good practice among develop-ment agencies. For example: Relevance –the extent to which the project fits country development priorities, IFAD strategy and

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

104


beneficiary needs is: 4 high; 3 substantial; 2 modest; 1 negligible.

4.22 Figure 1 sets out the steps followed toachieve the impact and performance ratings.It illustrates the way in which differentstrands of enquiry were brought togetherinto a final judgement. Following that, thedefinitions used and means of achievingconsistency are described.

Definitions of criteria4.23 The inception report set out the evaluation

criteria to be followed by the IEE. This wasbased on definitions adopted by OE, in orderto enable comparisons between OE evalua-tions and the IEE. Table 3 reproduces those definitions.

4.24 The definitions are recognizable as derivedfrom the OECD/DAC.159 Apart from modifi-cations to make the terminology specific toIFAD, the only significant difference

concerns the criterion of impact. TheOECD/DAC definition is ‘The positive andnegative changes produced by a develop-ment intervention, directly or indirectly,intended or unintended.’ IFAD’s modifieddefinition applies the perspective of the ruralpoor. No doubt this accords with IFAD’sdesired specificity, but it is an unconventionalapproach that contrasts with other develop-ment organizations. The difference between

105

Figure 1: Steps to develop impact and project performance ratings

Core teamreview &challenge

Evaluator’sjudgment

Form 5 ratings

Group participatory Form 4 Individuals

Interviews –beneficiaries

Interviews –other

stakeholders

Analysis

Semi-structuredtopic list

Interviews –managers,supervisors

Field visits

RatingsDesk review

Figure 1 Steps to develop impact and project performance ratings

158 Examples include the methodology used by theOperations Evaluation Department of the World Bank forvalidation of project implementation completion reports;by the European Commission in the monitoring ofprojects and by bilateral donors.

159 See http://www.oecd.org/document/22/0,2340,en_2649_34435_2086550_1_1_1_1,00.html

effectiveness and impact needs to be under-stood. Effectiveness looks at how well theproject delivers what it was planned to do.Achievement of impact depends on furtherbehavioural change by participants and bycircumstances outside the project’s control,such as movements in prices, or weatherconditions. It is possible to have high effec-tiveness and low impact, if external factorsundermine the achievement of benefits byparticipants. It is less common, but notunknown, for lower effectiveness and higher impact, where unintended factorsreinforce benefits.

Definitions of rating values4.25 Ratings are ordinal measures, which means

they are ordered (high is better than substan-tial), but there is no relative or absolute scale.High, rated 4, does not imply twice theperformance of modest, rated 2. Given thatthese are relative measures, it is helpful forraters to be given guidance in order toachieve consistency. The IEE did that in three ways:n For the desk study, the team worked

collectively at IFAD headquarters andcross checked the interpretation of thescores by mutual reference to the sampleprojects. The form design also requires ashort narrative explanation for the ratingvalue and a cross reference to specific textin the documents to aid the justification.

n On beneficiary survey Form 4, becausethe evaluators would be working individu-ally and through the national evaluationteam, guidance was given about howresponses to each question should beinterpreted. For example, the questionHave household assets changed? 4-high; 3-substantial; 2-modest; 1-negligible; 0-nochange; 9 negative change included theguidance A high change indicates acquisitionof a major new item; substantial would be theimprovement to the quality of an item, such asreplacing an older with a newer one; modestmeans more limited improvements or expansion.A negative change is where an asset has deteri-orated or fallen into disuse during the periodbeing reviewed. Similarly, for the questionHas your farm production changed over thereference period? 4-high; 3-substantial; 2-modest; 1-negligible; 0-no change; 9negative change the guidance was Thisquestion divides farm production into crops andlivestock for food, and crops and livestock forcash sales. A high change implies an improve-ment of 40 per cent or greater; substantial isbetween 20 per cent and 40 per cent; modest isbetween 5 per cent and 20 per cent.

n The responses to the beneficiary surveyForm 4 are in effect a form of self evalua-tion. It was noted in several countries thatmany responses were optimistic, especiallythose made at an early stage in projectimplementation. The use of triangulation

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

106


Table 3 Statement of evaluation criteria

Criteria Definitions derived from IFAD’s methodological framework for evaluation

Impact The changes in the lives of the rural poor, intended or unintended — as they and their partnersperceive them at the time of the evaluation — to which IFAD interventions have contributed, as well as the likely sustainability of changes.

Relevance A measure of the extent to which objectives are consistent with (i) the rural poors’ perceptionsof their needs, potential and aspirations; (ii) the economic, social and policy environment; (iii) IFAD’s mandate, strategic framework and policies at the time of design; (iv) IFAD’s regionalstrategies; (v) IFAD’s current country strategy as formulated in the COSOP; and (vi) the country’scurrent poverty-reduction policies and strategies.

Effectiveness The extent to which major objectives were achieved or are expected to be achieved

Efficiency A measure of how economically inputs are converted into results

Sustainability The probability of continued, long-term benefits from a development intervention

(described above) enabled these views tobe tempered by the interpretation ofother respondents. The summary ratingsin Form 5, described next, all reflect anassessment of the diversity of evidence, not only self-evaluation findings.

n For the summary ratings in Form 5, theteam planned to follow the OE method-ology. That does not, however, define howto interpret the values of ratings otherthan giving guidance for effectiveness of:80 per cent or greater as high, 60-80 percent substantial; 30-60 per cent modest;and less than 30 per cent negligible.160

How to calculate the percentages is notexplained. Instead the team relied on achallenge process after the CWPs werefirst drafted. This is described in the next section.

Consistency and comparisons acrossprojects and countries

4.26 Confidence that the ratings are consistentacross countries and projects is important for the veracity of the study. Consistency wasachieved in four ways:

n Firstly, six of the ten country visits wereled by or initiated by a team member whohad undertaken the desk reviews and wastherefore already conversant with themethodology and had helped to developthe forms and ratings.

n Secondly, once the first draft of CWPs wassubmitted, one of the most experiencedmembers of the team reviewed all drafts inperson, in order to prepare the synthesis.In this process he developed summaryratings for all IFAD’s impact domains(linked to Form 4) which were then sentback to all the country authors to beaccepted or challenged – a form of peer review.

n Third, at the same time, apparent incon-sistencies in the Form 5 ratings were alsohighlighted and the CWP author asked toprovide a more robust explanation orreconsider the rating.

n Fourth, all the team minus two countryauthors met in the UK in September 2004and discussed aspects of the findings, withparticular reference to the ratings and anyoutstanding problems.

107

160 IFAD 2003 op cit. para. 24, footnote 6.

1. Introduction1.1 This Annex summarizes the findings of the

IEE visits to ten countries and 20 loanprojects, conducted over the period Aprilto July 2004. The purpose of this phase ofthe IEE is to build up a picture of impactand performance through the lens of IFADloan investments, policy dialogue andcountry strategies. As such, it respondsdirectly to the call in the IEE TOR to focuson the assessment of sustainable impact andeffectiveness of IFAD’s development initiatives,including IFAD supported projects, policydialogue, advocacy, corporate policies and strate-gies. Little impact information proved to beavailable in Rome during the desk review,consequently the country visits are muchthe most important source of impactevidence, and the structure of this report isorganized accordingly. Following a descrip-tion of the country visit methodology,Section 2 assesses the overall impact ofIFAD loan investments. This is expandedin the third section, which clusters theevidence by impact domain. Section 4presents findings of impact at country-levelof IFAD’s main ‘catalytic’ objectives(country programmes; innovation; knowl-edge sharing; organizational partnerships;and policy dialogue). Section 5 presentsIEE findings on project performance,including relevance, effectiveness and efficiency.

Methodology and data collection1.2 Ten countries and 20 loan projects from five

continents were randomly sampled from allloans coming effective between January 1994and December 2003.161 The full list ofsampled country projects is contained in Table 14, at the end of this Annex. This tableincludes basic information on each project, itsimplementation stage and type of interven-tion. The stage of implementation is impor-tant. Five projects were newly effective and a further five were only mid-way throughimplementation. This has an importantbearing on the ability of the evaluation teamto assess the likelihood of sustainable impact.In this summary, the projects are referred toby the acronyms given in Table 14.

1.3 An experienced international developmentconsultant led each country visit. Each IEEcountry team leader made a reconnaissancevisit to establish contact with the sampledprojects and the national evaluation coun-terpart responsible for liaison and introduc-tions, and identify and contract the nationalevaluation team responsible for data-collec-tion and analysis. The national evaluationteams were selected from companies andindividuals with a proven history of ruraldevelopment and poverty-project datacollection expertise, but without any signifi-cant record of work with IFAD. Followingthe reconnaissance visit, the country team

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

108

Synthesis of country working papers

Annex 4

leaders made one or two further visits tocomplete the evaluation.

1.4 The team leaders and the national evaluationteams worked together to develop samplingand visit methodology for the collection ofprimary data from beneficiaries, imple-menters and other informants. Lists ofproject management staff, other imple-menting agencies, and local and nationalpartners (individuals and organizations withknowledge of IFAD or the projects, butwithout direct implementation responsibility)were prepared. Meetings with the CI super-visors were arranged.

1.5 The CWPs draw on a rich variety of primaryand secondary quantitative and qualitativeinformation, including:

n Survey of beneficiary perceptions

n Structured interviews with projectmanagement and other implementers

n Structured interviews with local andnational partners and key informants162

n Focus group discussions with beneficiaries,management, implementers and partners

n Project records, quarterly and annualreports, monitoring data

n Interviews with CI supervisors

n Supervision, mid-term, completion andevaluation reports

n External reports and studies

1.6 The CWPs (available separately) bringtogether findings about IFAD’s impact, policyobjectives and processes from all of thesesources.163 Of the 20 sampled projects, onlysix were closed at the time of the IEE visits.This means that a full ex-post assessment ofimpact was only possible in six projects. Theremaining fourteen are still under implemen-tation, with five at an early stage. The assess-ments for these projects, and particularly forthe earliest ones, required making verycareful judgements of expected impact, effec-tiveness, and sustainability. A critical part ofmaking these judgements was the triangula-tion of evidence from different sources –both primary and secondary – and the exten-sive experience of the national and interna-

tional evaluators involved. Notwithstandingthe challenges, the IEE team is confident ofthe quality and the consistency of the assess-ments made in the country reports and inthis summary.

1.7 The approach taken throughout thissummary follows IFAD evaluation procedure.The ratings presented in the text are basedon a four point ordinal scale: high (4),substantial (3), modest (2) and negligible (1).The measure of central tendency used is themode (the most frequently occurring rating).Where a set of ratings has two modes (twoequally occurring rating frequencies), OEpolicy of reporting the higher one as theaverage is also used in this summary (seeARRI 2003, Annex 5 paragraph 7). Thiscould be viewed as biasing presentationupwards, but because this convention is usedwithin the ARRI, it is respected here. Impactis assessed according to ‘domains’ that trackclosely with OE’s methodological frameworkfor project evaluation. Sustainability istreated separately as are innovation and partnership, which are assessed as potentialcontributors to impact i.e. as means to anend. Unlike the OE methodology, innova-tion/replicability and scaling-up are nottreated as separate impact areas.

109

161 The sampling methodology – a stratified random samplewith probability proportional to numbers of projects ineach region – was described in Annex 2 of the inceptionreport.

162 Drafts of the data-collection forms for beneficiaries,project management and partners were presented inAnnex 4 of the inception report. The drafts underwentfurther refinement and testing before the country visits,but the final forms used were very similar to the draftsshown at inception, in content and approach.

163 It is important to note that the CWPs are not full evaluations of the country programme (i.e. CPEs), butrather evaluations of two sampled projects per country.They do however take account of broader aspects of theprogramme, including organizational partnerships, use ofgrants, policy influence, etc.

2. Impact on rural poverty2.1 This section of the Annex summarizes the

findings from the ten CWPs in relation to theimpact of IFAD investments. Except wherestated, the assessment of impact is made onthe same basis as the OE methodologicalframework for evaluation and the OE ARRI,combining an assessment of coverage, the sizeof change achieved, and its sustainability, as aresult of the project intervention.164

2.2 Overall, the CWPs find that project impact isvariable: A few high performers contrast withmodest impact across most domains (Table2). The two projects rated high by the IEEare MARENASS and CORREDOR, both inPeru. These strong performers show thefollowing characteristics:165

n A rich innovative content, with public financelimited to TA and no project finance used forinvestments: A strong demand-drivenapproach let groups decide what to do withTA, resulting in a surprisingly large amountof family resources invested in the activitiessupported by project TA.

n Project design and management were at timessubordinate to community demands. InMARENASS, the original conservationprogramme was subjected to communityvote. Communities gave less priority toconservation, but project management andCPM concluded that the potential benefitsof full empowerment were extraordinaryand should be respected.

n Design was adapted to reach poorer groups:CORREDOR initially used business plansthat biased towards educated farmers andwould-be urban businessmen. A simplerformula (the business profile) was intro-duced so that groups in the communitycould take part.

n A strong driving force from the CPM, hisstable group of consultants and associates inPeru’s ministries and social science researchcentres. Locally resident CPM providingclose support during implementation.

The resulting impacts included:n Rising levels of food security directly attrib-

utable to the project, not so much in grains,potatoes and other basic foodstuffs but inthe fruits, vegetables, meat, milk and otherprotein deliverables.

n Dramatic impact on people’s self-esteem,with communities and groups enabled tomanage their activities and plan new ones,plus challenge government and agenciesto help protect and enhance their own interests.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

110

A N N E X 4 : S Y N T H E S I S O F C O U N T R Y W O R K I N G PA P E R S

Box 1 The country context

IFAD works across a range of country contexts, including some difficult environments. The IEE sample spans anumber of different country contexts. Table 1 lists each country by its most recent position on IDA’s CPIA, whichmeasures country and institutional performance. Six out of the eight countries to which this measure applies areamongst the better performers. At least five of the countries in the sample are off-track in terms of meeting theMDGs. All of them, except for Pakistan, are in sub-Saharan Africa. Mozambique has also recently emerged from aprolonged period of conflict, while Pakistan faces significant security risks linked to its geopolitical position. Two ofthe countries in the sample are middle-income – Egypt and Peru. The sample is too small to test the statistical corre-lation between these characteristics and impact and effectiveness, but they are clearly relevant in understandingthe challenges of achieving sustained poverty impact.

Table 1 Country policy and institutional assessment, 2003

CPIA quintile IEE sample countries

First Quintile Armenia, United Republic of Tanzania

Second quintile Bangladesh, Bolivia, Burkina Faso, Pakistan

Third quintile Mozambique

Fourth quintile Guinea

Fifth quintile -

Note: Egypt and Peru are not rated under the CPIA exercise.

111164 OE’s ARRI 2003 describes impact ratings as …a combinationof an assessment of how much has changed; the contribu-tion of the project to that change; the scale of change (e.g.number of households affected); and the likely sustainabilityof that change. (ARRI 2003, paragraph 33). The same para-graph goes on to say: These (assessments) are more oftenbased on the informed judgements of the evaluators ratherthan on empirical measurements. That also applies in thisreport although the IEE was able to draw on a larger varietyof data sources, including primary research, than is availablefor OE interim or country programme evaluations.

165 Based on the IEE Country Working Paper: Peru.

166 The domains shown here are an expanded version of the sixused by OE. The model class(es) are shaded. The sample size(N) varies because some projects were ‘not rated’ as eitherthey were in the early stages of implementation or it wasconsidered inappropriate, in the context of project objec-tives, to rate impact (such as for health, education, etc).

167 The likely impact on poverty, considering both the total number of people benefiting and the size of the benefits accruing.

Box 2 What does a ‘modest’ impact on poverty look like?

In Burkina Faso, PDRSO aimed to counteract the trend of progressive impoverishment and degradation of naturalresources. The project has however led to a disappointing level of impact, and there has been a limited engage-ment of beneficiaries. Literacy training is appreciated and there have been improvements in hygiene. Plus, one ofthe most commonly praised project interventions has been the training in the construction of compost bins - whichis seen to have made a positive contribution to production (though benefiting certain types of farms and mostlymen). In all other areas however, the project’s contribution has been modest or non-existent. Rural credit has largelybeen a failure with very few people benefiting and rather more suffering negative effects. The evaluation teamencountered a general sense of dissatisfaction and widespread reports of broken promises.

In Bolivia, PRODESIB aimed to promote the sustainable self-development of the indigenous peoples in Beni throughcapacity building measures at grassroots level. Overall, the project has helped strengthen indigenous organizationsin the land reform process. The most positive changes have been amongst scholarship students who have gainedfrom the increased income earning opportunities following training. In general though, the land titling and trainingprogramme has shown modest results: in terms of raising family income, 49 per cent of beneficiaries said that theirincome had increased slightly, with 39 per cent stating no change. Improvements in food consumption are alsoreported as modest, with 51 per cent of PRODESIB beneficiaries citing small improvements and 33 per cent nochange. In the land titling process there has been little tangible increase in family access to (and use of) forestresources - with the project tending to reinforce community relationships rather than address underlying changesin the existing system.

Table 2 Summary of investment impact 166

(Sample size: 20 projects) High Substantial Modest Negligible N

Overall impact on poverty167 11% 44% 44% - 18

Impact on income 6% 47% 47% - 17

Impact on women 22% 22% 33% 22% 18

Agricultural production and food security 24% 24% 53% - 17

Physical assets - road and irrigation infrastructure 7% 36% 57% - 14

Financial services 20% 7% 73% - 15

Social infrastructure and services - education and skills 25% 42% 17% 17% 12

Social infrastructure and services - health and drinking water 10% 50% 40% - 10

Environment and common property resources 33% 11% 33% 22% 9

Social capital and empowerment 17% 6% 78% - 18

Policies, institutions and regulatory framework - 28% 6% 67% 18

Private sector development 29% 29% 29% 12% 17

Sources: Form 5 ratings; IEE surveys.

2.3 Across the sample as a whole 55 per cent ofprojects are judged to have achieved or areexpected to achieve a satisfactory impact onpoverty (high/substantial) while 44 per centare judged to be falling below expectations.These results are the aggregated outcome ofan immense diversity of performance, butwhat is notable is how closely they comparewith the ARRI for 2002 and 2003, which rated50 per cent of projects high or substantial, and50 per cent modest. More tellingly, however,amongst the closed projects in the sample,where a full ex post assessment of impact ispossible, 50 per cent are judged to have had a less than satisfactory impact on poverty.

3. Domains of impactIncome and household assets

3.1 Overall, 6 per cent of projects are rated high for income, 47 per cent substantial and 47 per cent modest. Plus, according tobeneficiaries, there has generally been amoderate impact on household or familyincomes. Food consumption and cash incomefrom sales show the strongest change, withone half to two thirds of respondentsreporting improvements; income fromemployment is much less marked; as also arebenefits from time savings (see Table 3).

3.2 Limited impact on income is partly the resultof low project coverage and over optimismabout associated employment effects. InBurkina Faso, the PAMER micro-enterprisedevelopment interventions have been moresuccessful in increasing and stabilizingincomes than in the PDRSO integratedproject. Under PAMER, a majority (68 per

cent) say that there have been important orsubstantial increases in income from sales,and, access to financial services have improvedfor all social groups. Seventy eight per cent of respondents say that their income is morestable. Yet, the overwhelming majority ofrespondents reported no changes in incomefrom employment, in household assets, intime saving, in the use of cash crops and inincome due to the animal rearing. In Bolivia,the land titling and training programme ofPRODESIB shows modest results in terms ofraising family income – with 39 per cent of thesample stating that there had been no change.The most positive changes have been amongstscholarship students who benefit from theincreased opportunities following training.For PROSAT, half of all respondents note nochange to income due to participation in theproject, though 70 per cent hoped for positive changes. In Mozambique, a generallymoderate impact on income is seen by bothprojects. In NAFP this is possibly a reflectionof the limited employment impact of theproject, though expectations of the futureimpact are greater. For PAMA, better informa-tion has meant that farmers are now selling at viable prices, but the change in income isnot substantial without a correspondingincrease in production and competitionamongst traders.

3.3 In some cases increases in income have led toimprovements in household assets. Thesampled projects in Peru provide two suchnotable examples. In the MARENASS, a keyresult has been the increase in the numberand value of home and farm assets that was

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

112


Table 3 Income effects of IFAD project interventions

The effect on: % improved % improved % stay % % Totalsubstantially moderately about the deteriorated/ other

same worsened % N

Food consumption 15 50 28 5 0 100 1329

Cash income from sales 13 46 35 5 0 100 1313

Wage earning/employment 9 39 46 4 2 100 1176

Time savings 11 27 39 15 7 100 1213


entirely self-financed (as stimulated bycompetitions and prizes). CORREDOR hasalso gained substantial leverage on familyinvestments, in this case by awardingcontracts for TA to the winners of the compe-titions. In Bangladesh, under SSWRDSP, allbeneficiary types report improvements inhouse fabric and construction, with most nowhaving a tin roof instead of straw. In Guinea(under PD-PEF) there have been improve-ments in living conditions, and the (eventemporary) increase in revenue has played apart, with earnings often invested in (longlasting) housing improvements.

3.4 Elsewhere more moderate impacts were inevidence, and somewhat indicative of a lesssubstantial project impact on income. InArmenia, under NWASP and ASP, beneficiariesnoted improvements in income and foodsecurity, though with little tangible differencein household assets. In Mozambique, underNAFP, the impact on household assets wasconfined to a few individuals who benefitedmost from the project.

Agricultural production, productivity and food security

3.5 Research, extension and organizationalinterventions, often supported by credit orirrigation, to increase crop, home garden,livestock and fishery production forconsumption or sale lie at the historic core of IFAD investments. The sample showsconsiderable variability in impact across this domain, with 48 per cent rated high orsubstantial, and 53 per cent modest (theremaining projects were not rated). Several

examples of projects leading to increasedproduction exist. In Peru, MARENASS hasled to the growth of terracing and horticul-ture, with organic methods and livestockmanagement being very considerable. Asimilar demand-driven TA approach inBolivia (PROSAT) has led to widelywelcomed improvements in food security,income, crop and livestock production. InMozambique, NAFP has increased smallholderfish production and the functioning ofmarkets, with increased incomes for produc-tion, and land-use has increased in all thevisited SSWRDSP (Bangladesh) sub-projects,due to better flood control, drainage andirrigation facilities, plus the introduction ofnew high yielding varieties and high valuecrops. Under AqDP, LFG and panel acqua-culture group members recorded higher fishproduction, with a greater impact on salesthan own consumption (though a third saidthere was no direct gain from the project).80 per cent of beneficiaries eat at least a bitbetter as a result of pond and fishery devel-opment through AqDP.

3.6 The greatest impact on food production hasbeen on that used for consumption, with moremodest changes noted for cash crops andlivestock production. Table 4 shows that overhalf (52 per cent) of beneficiaries interviewedhad observed a high or substantial increasein crop production for own consumption as aresult of the project. The observations forcash crops and livestock are more modest –with the majority, nearly two thirds reportingthat they had seen modest, negligible or nochange in these production domains.

113

Table 4 Beneficiary perspectives of changes in farm production

Change in production of: % high or % modest % no change/ Totalssubstantial or negligible other % N

Food crops 52 26 22 100 1314

Cash crops 37 20 42 100 1286

Livestock for food 38 27 35 100 1259

Livestock for cash 33 27 39 100 1202


3.7 There are also some noticeably poor projectsfailing to make much impact at all. In BurkinaFaso, under PDRSO, the majority of respon-dents noted no (or insignificant) changes inthe production of cash crops, animal rearingfor profit, income from salaries and timesaving. In Guinea, under PD-PEF any increasein yields and change to multiple croppingpatterns have been jeopardized by the returnof leased land to the landowners.

Physical infrastructure3.8 Public good investments in roads, irrigation

and other water infrastructure are some ofthe largest investment that IFAD makes. Aswell as capital works, they typically includestaff and operating costs in establishingarrangements for operation and maintenanceand water-user groups. Of the 20 projectssampled, 14 have road and/or water invest-ments. Overall, the impact of IFAD invest-ments in road and water infrastructureprovision is assessed as modest. Of the 14 projects, only one, MARENASS (Peru) isclearly high impact. Two (PDRSO, BurkinaFaso and PD-PEF, Guinea) are notably poor.Elsewhere four projects typify the implemen-tation, technical, managerial and sustain-ability difficulties typically encountered byirrigation interventions: PFC-ADP (Pakistan),NWASP and ASP (Armenia) and WNRDP(Egypt). They also typify the substantialbenefits for farm households who do achievereliable water supply to their land.

3.9 MARENASS is not a typical publicly-financedinfrastructure investment. Indeed its small,household-level irrigation developmentsresult from a project approach that couplesdemand-driven TA supply with communitycompetitions – leading to a high leverageeffect on household investment. It also hasweaknesses, with a recent study showing sub-optimal irrigation efficiency of some schemes.But overall, the popularity and benefit ofirrigation, its sustainability, and the fact thatit was entirely household-financed make thisa high impact investment in a difficultcontext. The innovative intervention designhas enabled the project to convert irrigationfrom being a hard-to-deliver public good intoa private good.168

3.10 In terms of road investment both projects inMozambique (NAFP and PAMA) show positiveimpacts, with the opening up of fish landingsites to district and provincial consumptioncentres. Similarly, in AqDP (Bangladesh),roads were developed as a supporting activitywithin the project, and highly appreciated bythe communities served. Indeed, lake fish-eries group members have gained substan-tially from improved access to water bodiesfor fisheries, though a third of the non-fishery related community developmentgroup members (mostly poor women)reported losing access to waterbodies. InGuinea, under PAPE-BGN, the borrowerunilaterally changed the project design anddoubled the length of road construction. Thepeople who benefited, particularly women,are strongly enthusiastic, although the align-ment of the extended road through a low-density area was sub-optimal in terms ofcoverage. In the United Republic of Tanzania,under AMSDP, a significant under-estimationof unit costs at design will reduce the volumeof (cofinanced) road construction the projectis able to deliver.

3.11 At the other extreme, the road developmentwithin PDRSO (Burkina Faso) has seen poorcommunity coordination, slow implementa-tion and payments, within a failing projectwhich has required major restructuring. InGuinea, PD-PEF has had very low effective-ness, with small volume of infrastructurebeing completed, and many schemes poorlydesigned, unfinished, silted and not main-tained. Organization of management groupsfor the infrastructure schemes was weak andnon-sustainable.

Financial services3.12 The IEE DRR commended IFAD’s recent

work in establishing state-of-the-art policy forinvestments in rural financial services; for itssuccessful participation in the 2003 donor-peer review process, and for the ‘decisiontools’ publication to support field implemen-tation. The country studies reveal that,despite this, field operations are changingslowly. Of the 15 projects promoting differenttypes of rural finance provision, only four areachieving a substantial or high change in

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

114


access to credit products suitable for therural poor. The impact of the rest is modest,and many very modest indeed.

3.13 There are some notable examples of goodperformance in the rural financial servicessector. In Armenia, under both NWASP andASP, IFAD has supported the development of the ACBA. After initial start-up problems,ACBA is now the third largest (and onlycooperative) bank in the country, plus themain lender to rural areas – where previ-ously the banking sector had focused on lowrisk, high return borrowers from the maincity. Also under ASP, rural financial servicescontinue to be extended through the AnivFoundation – whose borrowers are mostlysmall rural enterprises, in contrast to mostother organizations who focus on medium-sized enterprises. In Peru, MARENASSwomen’s group self-managed rotating fundsare building up steadily through near-fault-less recovery and the women’s own discoverythat high interest rates assist rather thanundermine sustainable credit supply for thepoor. The prospects for RFSP (the UnitedRepublic of Tanzania) are also good, althoughit is too early to assess impact. The projectbenefits from being focused on a singlesector and from competent management,although there are indications that thecurrent drive to increase the number ofparticipating credit cooperatives and theirmembership may be at the expense ofbuilding sufficiently competent governanceand financial management to enable them to retail bank credit.

3.14 Yet, all other credit interventions (in thesample) are failing in one or more respects.Recovery and sustainability have been poor;group leaders have dominated fund accessand management; project demands haveconflicted with bank requirements; and loweffectiveness has led to complete redesign.Under NAFP (Mozambique) for example,financial services have had a limited impactoverall, with NAFP tending to target mostlyboat owners (the rich minority) and sufferingfrom two critical constraints: (i) the cumber-some registration process; and (ii) thetraining in credit. Overall, the majority of

sampled projects display the familiar rangeof weaknesses of traditional project creditschemes and overall impact in terms ofcreating sustainable access is modest. Thediscipline and professionalism required toachieve compliance with IFAD’s new RFSpolicy has yet to be achieved.

3.15 In 2003, the ARRI (paragraph 37) also notedmixed performance in the area of ruralfinance, as it did the previous year. Thereport made three general observations:

n Grassroots, group-based credit and savings institutions have often provedmore successful than official, subsidizedcredit schemes.

n Repayment rates by members of women’sgroups for unspecified small, short-termloans have generally been very high.Repayment of longer-term specified loans to individual farmers has been much lower.

n The need to ensure institutional andfinancial sustainability is often either overlooked when the credit schemes wereestablished, or remains a challenge in anumber of cases.

115

168 The TA plus community-competition methodologieswhich achieved this result are a successful example in avery poor context of the publicly-funded, privately-provided model, driven by consumer choice, which isnow the subject of experiment for public service deliveryin many countries.

The environment and common-property resources

3.16 Nine of the twenty-project sample havesought to improve environmental conserva-tion and the sustainable use of naturalresources. Across these nine projects, threeare rated as high, one substantial, threemodest and two negligible. In Egypt, EDNASPhas had considerable impact including thecreation of 32 environmental-monitoring sites,creation of a model ‘clean and green’ environ-mental village, modest tree plantation, andtraining on recycling, animal hygiene, pesti-cides and rodents. The great majority of bene-ficiaries in WNRDP say the project has beenpositive in terms of access to naturalresources. Like EDNASP, the project istraining on pesticide use and encouragingorganics, and environmental impact assess-ments and soil-testing precede the introduc-

tion of new crops. In Mozambique, NAFP hashad potentially beneficial environmentalimpacts through regulatory reforms toimprove sea-fishery sustainability.

3.17 For every high performing project there areseveral weak performers. In Guinea, bothprojects had environmental objectives butachieved very little, despite widespreadconcern about shrinking fallow-periods,extensive logging, slash-and-burn and theenvironmental impact of the extensive roadconstruction in PD-PAPE. The PD-PAPEdesign did not include environmental analysisor mitigations addressing road constructionthrough fragile areas. In Pakistan, under PFC-ADP, the command area design (over whichIFAD had no control) did not promote low-water intensity rice cultivation and, in afragile area, where waterlogging and salinity

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

116


Box 3 Impacts on health and education

Health is not a major feature of IFAD investments, but half of the project sample has invested in health facilitiesand/or drinking water supply. In Burkina Faso, both projects show general health improvements, with PRDSOmarginally better for improvements in vaccinations and ante- and postnatal healthcare. Under NAFP (Mozambique),social aspects of the programme (mainly improved water supply, and to a lesser extent health and education) bene-fited the generality of the population to a greater extent. In Egypt however, the health picture is more mixed, withante- and postnatal care and immunization scoring relatively highly for EDNASP, but with respondents recording no(or negligible) changes for WNRDP – also, other aspects of healthcare faired less well.

Investments in drinking water supply are highly popular, relatively sustainable and high-impact, even for low-achieving projects – and often as a result of community-selection investment methodologies. In EDNASP (Egypt),access to potable water, including pumping and treatment, has been a significant and popular success. In Armeniaunder NWASP, renovated pipelines have been a significant achievement and sustainability appears promising. InGuinea (despite other failings of the project), the boreholes constructed under PD-PEF have been largely successfuland are appreciated, particularly by women. Similarly, potable water has been a very popular, if under-achieved,component in PD-PAPE (Guinea).

Like health investments, education is not a mainstream IFAD investment line but some progress has been made. Inthe sample, 12 projects have either refurbished schools or developed non-formal literacy and other non-agriculturalskills – with three projects rated as high, five as substantial, two as modest and two as negligible. The CORREDOR(in Peru) is a particularly innovative example; the impact of TA on skills and knowledge was seen as high, with 55 per cent of respondents saying it had increased substantially. Also in Peru, the MARENASS methodology hasboth increased the skills of villagers and created a cadre of about 4,600 yachachics across a wide variety of sectorsand skills who will be able to continue to make a living as non-formal trainers for the future. In Burkina Faso, 86per cent of PDRSO respondents report improvements in adult literacy, whilst in PAMER, which develops rural micro-enterprises, 77 per cent of respondents say skills have improved. Similarly, in PROSAT (Bolivia), beneficiaries rate their increased skill levels very highly.

In Egypt, educational impact is similar across both projects (EDNASP and WNRDP), with respondents seeing amodest to substantial improvement in primary education opportunities, with the reverse being true for secondaryand vocational education services. In PD-PEF (Guinea) literacy classes were organized, although follow-up, effectiveness and impact are low. Adult literacy has been added, post-design, to the training activities designed toimprove market functioning in PAMA (Mozambique).

are already appearing, ex-post mitigations willbe very difficult. Also MSVP, the predecessorto BADP, achieved little against its environ-mental objectives.

3.18 Few projects have seriously grappled with themany challenges and contradictions ofpursuing agricultural development, environ-mental conservation and empowering commu-nities. One notable exception is theMARENASS project, Peru (Box 4).

Social capital and institutional impact3.19 A key element of poverty is a sense of exclu-

sion, of being marginalized or powerless. Bybeing part of an IFAD project, and interactingwith its partners, there is a clear impressionthat most beneficiaries feel more confident ininteracting with public institutions. Accordingto the beneficiary surveys, 59 per cent feltmore confident to speak out and assert theirrights. For example both projects in Peru(MARENASS and CORREDOR) have had asubstantial impact on the self-esteem andpower of community leadership vis-à-vismunicipal authorities. Similarly in BurkinaFaso, under PDRSO, an important majority ofrespondents feel better able to interact withauthorities, though women are less convincedof these changes.

3.20 The impact on social capital is however morethan just a sense of empowerment. In recentyears IFAD’s portfolio has shifted from astrong emphasis on production and produc-tivity to a much broader poverty and ruraldevelopment agenda. As such, its interven-tions are not just about transferring tech-nologies and methods, but involve a muchgreater focus on the way production is organ-ized – and the institutions required for operations and maintenance, marketing,finance and other inputs. This is seen ascritical to improving livelihoods, incomes and security, but also for making sure thatbenefits continue into the foreseeable future.Put simply, it is not enough to build an irri-gation network, rehabilitate a road, providefinancial services or impart new methods offarming; it is also about developing the insti-tutional structures which will enable a moreequitable distribution of the benefits, ensureoperations and maintenance and providefinancial sustainability.

3.21 IFAD interventions have created many newinstitutions but their impact is variable. Thefindings of the IEE country studies reflectthose of the ARRI 2003. In Pakistan, underPFC-ADP, village and women’s group organi-zation was a significant achievement within a

117

Box 4 The challenges of demand-driven natural resource management

A year after start-up, management found it difficult to implement a project with seemingly conflicting goals andmethodologies (i.e. to introduce better resource management practices while also giving full initiative to thecommunities to assert their own priorities). A memorandum from one of IFAD’s Executive Directors to PL Division,clearly demonstrates the point: In other words: how could the communities be induced to want what they oughtto want in order not to work at cross-purposes with these higher-level strategies, while still upholding the principles of bottom-up decision making. In 2000, two years after MARENASS became effective, those concernswere set aside as project management, with support of IFAD’s CPM, elevated the demand-driven goal while puttingthe management of natural resources to the critical vote of the communities.

In many ways MARENASS has worked. Indeed a notable result has been the number and value of home/farm assetsthat were self-financed. In some categories, the leverage effect of contests has been extraordinary, and communi-ties continue to invest in conservation management practices that were relevant to the families’ private holdings:terraces, improved irrigation distraction, herd rotations and fences, etc.

On the down side, the project/communities are not doing as much as had been expected on communal work inthe higher pastures (the heavily degraded rangelands). A recent study shows that the general condition of thesepastures ‘poor’ to ‘very poor’. All attempts to improve conditions by reseeding will be in vain unless institutional issues are also addressed and communities keep their animals off this land. It is not clear whether thereformulation of objectives contributed to this outcome, but questions still remain about whether MARENASS hadthe instruments and incentives to address the high pastures successfully. It seems likely that an alternative interven-tion approach will be required to tackle the acutely-degraded pastures.

difficult project environment, but much ofthat achievement remains dependant oninputs from the sub-contracted NGO. InBangladesh, under SSWRDSP, the develop-ment of Water Management CooperativeAssociations has clearly had a significantimpact as a source of funds with an interest inwater related infrastructure (and later micro-credit operations). It is however too early toassess their sustainability. Under AqDP, theproject supported private sector initiatives forindividual and group based aquaculture inleased private ponds, and also through lakefisheries groups managing larger water bodiesin the form of cooperative businesses. Earlyexamples of the latter have had problems withthe dominance of non-fisher stakeholders. In Pakistan, the Mansehra Village SupportProject (the predecessor to the sampledBADP) formed many groups, but sustain-ability is generally poor. The impact of theBangladesh and Pakistan irrigation and ponddevelopment projects depends in part on thesustainability of group-based water manage-ment arrangements, yet there appears to havebeen little consideration at design whether: (i) it was practicable to expect NGO-led newgroup formation to achieve sustainabilitywithin a normal project timeframe; or, (ii) ofhow the more enduring and increasinglydevolved, even if less motivated andresourced, structures of local governmentmight be deployed.

3.22 Some key lessons include the need to: (i) better analyse institutional and organiza-tional issues; (ii) design projects for thelonger-term, with either longer investmentperiods or appropriate exit strategies; and (iii) build on existing organizations and struc-tures, such as those of local government. InArmenia for example, ASP has taken a delib-erate approach to invest in key institutions foreach component, as it is these organizationsthat will be around long after the projectceases to exit. In several cases the PCU hasbecome intimately (and often uncomfortably)involved in the details of running such organi-zations, particularly during the early stages.The project also promotes these organizationsabove the work of the project. The result isthat beneficiaries are largely unaware of “ASP”

(or the “IFAD project”, “PCU staff ”, etc) butinstead relate to ACBA (to deliver ruralcredit), Aniv Foundation (to deliver smallenterprise finance), and so on.

Sustainability of impact3.23 Sustainability is considered along two main

dimensions. Firstly, whether the stream ofbenefits accrued as a result of the project iscertain (or likely) to continue after closure.And secondly, whether the institutionalchanges induced by the project are likely tocontinue after closure.

3.24 Sustainability of impact is substantial in justunder two-thirds of projects, but in nearly oractually closed the proportion falls to less thanhalf (Table 5). The evaluation of likely sustain-ability depends partially on the views of benefi-ciaries themselves. It is not surprising thatduring the early stages of a project there arerelatively high expectations that the netbenefits accrued as a result of the project willendure.169 These expectations are clearlymodified in the later stages of implementation.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

118


In summary:

n While the impact on poverty is modestoverall, there have been some gains inIFAD’s traditional areas of expertise:agricultural production and foodsecurity, plus a few notable examples of road construction, irrigation worksand financial services.

n Health and education are not signifi-cant IFAD investment streams, yetimpact is reasonable, and community-led water supply seems particularlyeffective and appreciated by beneficiaries.

n The impact on the environment andcommon property resources is mixed;few projects show signs of having seriously grappled with the many challenging issues that exist.

n Support for social capital strengtheninghas been successful in helping to createa ‘sense’ of empowerment amongstproject beneficiaries but wider institu-tional impact is harder to determine.

3.25 In MARENASS (Peru), findings generallysupport the notion that farm practices thatrequire little or no additional cash would bemost likely to survive, while instruments thatrequired cash to continue would suffer most.A high level of sustainability is expected forthe most prominent popular practices, suchas the irrigation systems, stables andorchards. Respondents were slightly lesspositive about the sustainability of method-ologies such as the use of paid TA, competi-tions, prizes and bank accounts. ForCORREDOR, the answers from participantssuggest that they are involved in these newbusinesses for the long haul. It is too early to assess failure rates though the outlook ispositive (with 78 per cent saying they willcontinue). For PRODESIB (Bolivia) some 87 per cent and 100 per cent of respondentsfrom phase 1 and 2 respectively thought thatthe benefits were sustainable. It is probablethat the benefits from land titling will bemaintained as they are firmly embedded inBolivian state structures and are unlikely tobe repealed. In Bangladesh: 70-90 per cent ofthe respondents think that the cooperativesestablished under SSWRDSP are likely orcertain to continue. Under AqDP, 80 per centof members of PAGs and CDGs think theirgroups will continue, yet participants in thelake sites are less optimistic and 66 per centof women of these sites do not expect theirCDGs to continue. It also seems likely thatthe benefits resulting from ASP (Armenia) willcontinue in the longer term. ASP continuesmany of the interventions undertaken underits predecessor (NWASP), for which many

project outputs (irrigation; agriculturalcredit) continue to be utilized.

3.26 But amongst certain projects there are seriousconcerns about sustainability. Two recentlyclosed projects (NAFP, Mozambique and PFC-ADP, Pakistan) have had a substantial impacton poverty but in both cases sustainability ofthe net benefits is seriously in question. InNAFP, there are reservations, especially aboutthe maintenance of rehabilitated roads. InPCF-ADP, the likelihood of project-inducedbenefits continuing after closure is a majorconcern. The NGO will continue with credit,but whether it can sustain other communityneeds is doubtful. Moreover, there are alreadyreports of major rehabilitation required forminors, with much drainage choked ormisused (Box 5). And in the case of Egypt,benefits are likely to continue but farmers canhardly do otherwise. Farmers will go on usingthe techniques they have learned – they canhardly do otherwise in arid lands – but if the project stops then all the linking up willstop too.

119

169 Seven out of ten (70 per cent) of the earlier projects arerated as substantial for sustainability of impact, with 20 per cent not rated.

Table 5 Sustainability of impact and institutional impact

High Substantial Modest Negligible N

Sustainability of impact: The likelihood of project-induced benefits continuing after project closure - 61% 39% - 18

Late and closed projects only - 40% 60% - 10

Sustainability of institutional impact: The likelihood of institutional (state, private, civil) changes induced by the project, continuing after project closure 6% 35% 53% 6% 17

Late and closed projects only - 22% 66% 11% 9

Sources: Form 5 ratings.

Sustainability of institutional impact3.27 The sustainability of the institutional impact

is largely unsatisfactory. Of those projectsthat were rated (i.e. a sample of 17 projects),59 per cent (ten out of seventeen) are ratedas modest or less.170 Of those projects in thelate stages of implementation or closed, 77 per cent (seven out of the nine whichwere rated) are rated as modest or less. Thereare few examples where the institutionalimpact is seen as highly sustainable. In RFSP,

the United Republic of Tanzania, the sustain-ability of the institutional impact is seen ashigh. This is based on an arguably optimisticassessment for an early project, and that thecurrent project performance plus thepositive developments in the national micro-finance environment will continue over thecoming decade. In PROSAT, Bolivia, thegrowth of the market in technical servicesshould be sustainable (with contributionsmade by the groups themselves), providing

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

120


Box 5 The need to plan for sustainability in Pakistan

Four main questions of sustainability were identified in Pakistan’s PF-ACP: (i) Will the investments last? (ii) Will thecommunity organizations survive? (iii) Will the supporting services such as credit survive? (iv) Will land quality besustained? With respect to investments, there is mixed evidence. The survey done for this study suggests substan-tial benefits and therefore high income penalty for members if the investments fall into disrepair. However, the2003 impact study found very few groups that had actually raised money for O&M, notwithstanding a stated will-ingness to do so.

With respect to community organizations, the survey and the observations of the NGO found the communities tobe still mostly active, however, the impact study concluded that continued support for probably another five yearswould be essential for their survival. Yet, other than for rural finance, the funding for broader social facilitationsupport is now minimal and a number of vehicles have been withdrawn. It is difficult to see how this support cancontinue. With respect to the supporting services for credit, while TF is continuing credit for five years, this cannotbe characterized as sustainable microfinance in the longer term.

Sustainability could not be rated in the newly effective BADP, but based on the experience of a forerunner project,expectation of sustainability must be considered modest. Community processes sustainability is uncertain but questionable since the six year Mansehra Village Support Project has left many young community organizations stillvery vulnerable and it is difficult to see how the supporting NGO can sustain them with no explicit project budgetfor old communities while they have to reach out to new communities to meet BADP II targets.

Two issues are key for sustainability. First, the review has not found any appraisal stage exit strategies, not even anindicative one. So-called ‘exit strategies’ are being cobbled together largely by one or two dedicated borrower andNGO staff in the last few months of the projects while, at the same time, the rigid government accountabilitysystem is recalling, for redistribution, all operational facilities such as vehicles, and, in one case, even staplers! Theseare better characterized as ‘escape strategies’ not ‘exit strategies’.

All IFAD project appraisal documents should have a carefully considered exit strategy: not aimed at answering thesupply end question of ‘how can the project disengage with the least disruption’; but focused on the demand endquestion, ‘what community-level indicators should trigger phased disengagement of community-level support ineach community and how should that process be managed and communicated’? In other words, exit should notbe something determined to suit project design and phasing it should be determined by community-level progress.Forcing designers to articulate an exit strategy makes them face important entry questions which are too oftenswept under the carpet.

The second sustainability issue is simply the old question of whether the projects are staying with communityorganizations long enough for them to reach a sufficient stage of maturity, knowing that the borrower is veryunlikely to be able to fund continuing operations at anywhere near the same scale. In both the sample projectssustainability was, and remains, precarious. Sustainability of government line agency services is the essential otherside of the coin. If government line agencies were required to face communities say twice a year to provide infor-mation about how to access project or programme resources and deal with questions and complaints, not onlywould there be improved interaction but the incentive at community level to sustain community processes wouldbe much greater.

there is no strong competition fromstate/NGO in providing free services. Forboth NWASP and ASP (Armenia), the sustain-ability of the institutional impact was seen aspositive, based largely on the continuation ofvillage associations for credit lending byACBA and the state commitment to wateruser associations.

3.28 The reasons given for modest sustainabilityare varied. In Guinea, both PD-PEF andPAPE-BGN function as completelyautonomous interventions and have notcontributed to strengthening public admin-istration (plus, project activities weresubcontracted rather than undertaken inpartnership). For PD-PEF the contrastbetween a well-resourced, transitory projectand the lack of support to the public sector(with scarce resources) raises fundamentalquestions about the nature and impact ofIFAD support. In Burkina Faso, PDRSO hashad minimal impact on private sector devel-opment, and civil society (agricultural input,credit groups, development and post-harvest transformation groups). Relativelyfew current members see benefits frommembership in such groups, and observa-tions suggest that they are not gatheringnew members. Also in Burkina Faso, PAMERhas had an insignificant impact on institu-tions but the sustainability of this is quitegood. In this particular case, the projectbuilt three new bank branches and itappears likely that these will continue afterthe project has closed.

4. Other policy objectives 4.1 Apart from project investments, IFAD also

seeks to perform a catalytic role. Under IFADV, it is clearly stated that IFAD’s role is, “as aleading source of knowledge on the eradica-tion of rural poverty”, and specifically: (i)enhance its participation in policy dialogueand analysis; (ii) take a more structuredapproach to documentation and evaluationof field-based innovations; (iii) step up effortstowards building strategic partnerships”(desk report, paragraph 3.20). IFAD VI setout a major push to link strategy more force-fully to the global consensus around theMDGs and to the poverty and policyperformance of borrowing countries (deskreport, paragraph 3.24). This section looks atIFAD’s contributions beyond project inter-ventions, namely: country programmes,innovations, knowledge sharing, organiza-tional partnerships, and policy dialogue. Theevidence is country-focused (based on theCWPs) and does not include evidence fromIFAD headquarters.

Country programmes4.2 While IFAD has made deliberate attempts

to embrace a more programmatic style, itscountry programmes continue to remainlittle more than a ‘collection of projects’.Indeed, there are disappointing levels ofsynergy between projects, and betweendifferent aid instruments (loans and grants).In only half of the visited countries is thereevidence of established or recently emerginginter-project linkages. In the United Republic of Tanzania, the inter-relationship amongstIFAD projects is partial, with a lack of consis-tency in geographical coverage, varied ratesfor salaries and allowances, and, no estab-lished mechanism for inter-project dialogue.In summary, the portfolio gives the appear-ance of a collection of projects rather than anintegrated programme. Likewise in Pakistan,there does not appear to be a substantialsynergistic relationship between projects

121

In summary:

n The likelihood of project-inducedbenefits continuing after project closureis modest overall, especially where onlylate or closed projects are considered.

n While interventions have generallycreated new institutions, theircontinued existence is undermined byweak attention to sustainability issues.

n Focus should be more on realistic exitstrategies, longer-term support to insti-tutions, and developing projects on theback of existing institutional structures. 170 Three out of twenty are not rated for ‘sustainability of

institutional impact’: AMSDP, the United Republic ofTanzania; BADP, Pakistan; SSWRDSP, Bangladesh.

(except perhaps the TAG for the FederalBank of Cooperatives), and each projectappears fairly self-contained. Much of thesynergy that is evident is across componentswithin projects rather than synergy betweenprojects or TAGs.

4.3 In the few cases where synergy does occur, ithas been used to good effect. Peru is a notableexample. The synergy between MARENASSand CORREDOR is remarkable, with frequentcontact between staff. The package has beenfurther strengthened as well as integrated bythe insistent work of the relevant TAGs, particu-larly the Programme for Strengthening theRegional Capacity for Evaluation (PREVAL)and the Regional Training Programme in Rural Development (PROCASUR) but alsoFIDAMERICA. The CPM has played an impor-tant role in developing and packaging the interventions. In another case, IFAD’s work inMozambique has a good mix of IFAD projects,although its absence from central budgetsupport is conspicuous. It has a balanced port-folio of project support, decentralized cooperation(provincial targeting) and sector budget support.IFAD’s support to the Agricultural Sector PublicExpenditure Programme (ProAgri) (which isproject-cum-sector budget support) has givenIFAD room to benefit from, as well as leveragesome influence over other donors supportingthe agricultural sector.

4.4 In general, grant funds are underutilized.Three key themes emerge from the countryvisits. Firstly, there is a general lack of awarenessamongst project and government staff of suchgrants. This was certainly evident in Guinea,Mozambique and the United Republic of Tanzania(where one TAG was identified in connection toPD-PEF but no more information was foundabout it locally). Similarly in Egypt, project staffand partners were unclear about the criteria forrequesting such grants or how grants areadministered and decided upon.

4.5 Secondly, the potential for synergy often goes unrealized due to weak coordination between loan and grant funding streams. InBangladesh, SSWRSDP has generally beenisolated from TAGs, and grants have notcontributed technologies or interacted with

government agencies involved in loans – norhave project-implementing units sought information or advice from TAG-fundedresearch.171 In Bolivia, PRODESIB TAGs arejudged to have had only a moderate impact onproject development by staff and the director.The PROCASUR grant provided the mostnotable exception however, where the supportto the objective and transparent recruitment ofstaff was highly valued.

4.6 And thirdly, grants and supplementary fundsare used for a wide range of purposes, someof which would be better coming from routineadministrative sources. In Egypt, TAGs havebeen used to facilitate gender work and anIFAD project in Sohag, but otherwise theyhave been granted either to facilitate the start-up of the project or on a wider, more regionalbasis. In Burkina Faso, the use of grants is relatively weak and disappointing. Theregional TAGs appear to have little impact orrelationship with the ongoing projects, andprojects do not seem to use grants foranything other than solving routine adminis-trative funding hitches – none of whichrequires the flexibility of grant funding andcould not have been achieved through routinebudgeting. Similarly for Armenia, grants areminor sources of finance (compared to loanfunding) and have been used to minimaleffect. In some cases, TAGs have been used tofill pressing budget demands (completionreports; expenses between projects).

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

122


In summary:

n IFAD’s existing products and instruments are not used to their fullpotential, and the limited options available place constraints on what can be achieved.

n The use of both grants and loans, andsynergy between projects, can be usedto enhance IFAD’s performance – andyet generally this is not undertaken.

n The almost exclusive reliance on thetraditional project model limits IFAD’spotential, especially in engaging in thenew aid architecture and adding-valueto success.

Innovation4.7 Innovation is described as central to the

achievement of IFAD’s mandate.172 As a rela-tively small IFI in an increasingly congesteddevelopment world, producing rural develop-ment projects does not alone distinguish IFADfrom other international development organiza-tions. The ability to scale-up successful andreplicable innovations provides one way thatIFAD adds value, as well as having a directimpact on poverty.

4.8 IFAD defines innovation in a broad way andwhile it has aspirations to be an innovator, theevidence suggests that IFAD has struggled withthis role. The IEE sample of operations clearlyindicates that while there are a few highly inno-vative projects (as well as others that containinnovative elements), the vast majority ofprojects are not. Indeed on balance, there islittle to distinguish the work of IFAD from thatof other development agencies. And, moreworryingly, IFAD’s contribution to the capture,learning, promotion and replication of innova-tion appears unsystematic and inadequate.

4.9 The term ‘innovation’ provides a useful rhetoricfor IFAD. It is gives both a sense of being at thecutting edge of development whilst ‘meaning allthings to all people’. The current definitionadopted by management, and reflected by OE(in the 2003 ARRI, paragraph 77) provides avery broad approach: the development of improvedand cost-effective ways to address problems or oppor-tunities faced by the rural poor. Moreover, it fails topick up any of the underlying elements aboutwhat enables innovation to happen, thelearning, analysis, and dissemination. Againstsuch a measure, it would seem likely that IFAD(or indeed any organization) would be judgedwell: the ARRI (pages 19-20) states that innova-tion and replicability were rated as substantialin 40 per cent of project evaluations.173

4.10 Innovation can be defined in terms of: (i) creating new technologies or approaches fordevelopment; and (ii) promoting new ideas andways of working. The diffusion of establishedtechnologies and approaches to new places [iii] is a potentially useful role for a develop-ment agency but one that is not normallymeant by innovation. The difference between(ii) and [iii] is that, in the former ‘new’ is relatedto the idea/product, while in the latter it relatesto the situation/place.

4.11 A simple test of ‘national innovation’ and ‘localinnovation’ highlights the difference betweengenuinely promoting new ideas and ways of workingand the diffusion of established technologies andapproaches to new places. At the national levelthere are clearly innovations (e.g. a new type ofmicrofinance organization, a new agriculturaltechnology). What occurs at the local level, to avillage or project area is more commonlyunderstood to be extension or technologytransfer. Table 6 shows that 55 per cent of the

123

171 The rest of the Bangladesh portfolio is less isolated, andAqDP includes an adaptive research component.

172 IFAD V states clear objectives to strengthen IFAD’s role “asa leading source of knowledge on the eradication ofrural poverty”, and specifically: (i) enhance its participationin policy dialogue and analysis; (ii) take a more struc-tured approach to documentation and evaluation of field-based innovations; (iii) step up efforts towards buildingstrategic partnerships. (Desk report, paragraph 3.20).

173 One action in response to the OE evaluation of innovationwas to prepare an IMI into IFAD’s core business, linked toa pledge by the United Kingdom of USD 10 million. Thebroad goal of the IMI is to enhance IFAD’s capacity topromote innovations that will have a positive impact onrural poverty. The three primary results expected from IMIimplementation, which derive directly from its centralobjectives, are: (i) enhanced IFAD innovation culture andcapacity; (ii) improved quality and impact of innovation inthe field; and (iii) improved innovation learning, and theapplication and sharing of such learning. [An EB informa-tion paper was circulated in December 2003 and aprogress report was due in September 2004.]

Table 6 An assessment of innovation

% high % substantial % modest % negligible Sample size

National innovation 15 10 30 45 20

Local innovation 15 40 35 10 20

Source: IEE ratings based on CWP evidence.

project sample can be considered innovative atthe local level, but in terms of wider-scale,national innovation, only 25 per cent areconsidered high or substantial. This wassupported by the view of the OE Evaluation ofInnovation (paragraph 20). The majority ofIFAD’s innovations are not really ‘new’, although they

may be new to the project area involved. Most havealready been tried out (this does not imply ‘tested’) byother partners or by the rural poor themselves.

4.12 Amongst the project sample, the Perucountry programme is an outstandingexample of innovative practice. A series

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

124


Box 6 An exception or model for the future?

A series of IFAD projects undertaken in Peru are considered to be highly innovative, starting with FEAS (Promotionof Technology Transfer to Peasant Communities) and evolving – and improving – to include a further three. The Perucase was recognized as a notable exception by an OE thematic evaluation undertaken in 2002, which otherwisefound that IFAD was falling short in almost all processes necessary to institutionalize an innovative habit. The IEEfindings support this position, finding thirteen ‘innovations’ associated with the Peru programme, several of whichhave since been transferred to other countries in Latin America.

The ‘process’ of innovation has been far from straightforward, although it is possible to discern some sort oftimeline. When FEAS was appraised in 1990, the first IFAD project had long since concluded, and the third was halfway through implementation. Both were considered as belonging to the tradition of supervised credit projects. Thepresent CPM, along with a few consultants with Peru experience, were determined to try something different –partly because they had little confidence in the Ministry of Agriculture, Livestock and Nutrition extension service.

In FEAS they attempted a different approach to extension design (using private Technical Assistance, or TA) andrelaxing the ties to credit (where many beneficiaries would pay for their own investments). An unexpected shockalmost derailed the process, as the Agrarian Bank, which was to execute the project, was abolished with all its ruralbranches. This forced the project team to rewrite the appraisal document, and return to the beneficiaries to deter-mine under what circumstances they agree to a revised design; one that called for their own contributions to financea small share of TA and all the business and farm investments that followed. The fact that the IFAD project did notfinance the investments made it unique, and led to perhaps the most remarkable outcome of the series – theleverage on family investments.

This demand-driven feature – letting the groups decide what to do with the TA – was extended to other projects. MARENASS brought it forward from a sensible strategy to an overarching goal of development program-ming. Funds were transferred from the project to local control, thus enhancing the effectiveness of the demand-driven strategy.

Another successful innovation has been the use of project-supported competitions between families and commu-nities, with project-financed prizes. While the use of competitions and prizes started in Peru in the late 1980s ona European Union rural development project, it was later carried to Bolivia by one of its originators, who thenreturned to Peru in 1994 and led the preparation team for MARENASS. This innovation can at least in part be attributed to IFAD, whose main role was to adopt and raise the prominence of an idea that had alreadyattracted attention.

The idea of concentrating on regional economic zones was not new but CORREDOR was the first in Peru to organizea major programme using the zones as an anchor for its inputs. The government’s strategy now emphasizes corri-dors, the World Bank’s ‘Innovation and Competitiveness for Peruvian Agriculture’ pilot project operates in three corri-dors, USAID’s PRA project operates in 11 ‘mini’ corridors, IADB’s new project uses corridors, and FONCODES(CORREDOR’s sponsor) is switching to a division of funds based on corridors.

IFAD can thus take credit for having initiated or scaled-up a series of remarkable innovations. Indeed, the role of theCPM has been significant, though he himself credits much to the project staff. The CPM is also well connected insocial science circles (for which the cities of Lima and Cusco are noted) and it was they as much as anyone who puta shape to the emerging project pipeline. Within IFAD, it would appear that the CPM operated largely single-handedin designing this innovative programme. There is no evidence that there was at any time a guiding spirit from head-quarters that was helping him shape the emerging overall plan. The best that can be said about headquarters’ rolein the evolution of this plan is that it provided the CPM and his project teams an environment of ‘no objection’ inwhich they could push along. Of course, headquarters’ acceptance of the plan and the use of these consultantsduring the 1990s, and its efficient processing of the loans, were indispensable to any success.

125

of four Peruvian projects (includingMARENASS and CORREDOR) are regardedas highly innovative – an assessmentsupported by the 2002 OE thematic evalua-tion on IFAD’s innovative programming. The IEE has identified some thirteen ‘inno-vations’ with regards to the Peru programme(see Box 6).

4.13 There are also examples of IFAD promotingnew ways of working. In Mozambique, NAFPis considered to be fairly innovative althoughit is not specific components that are thoughtto be innovative but rather the integratedapproach to rural development: in the fish-eries sector, marrying general communitydevelopment (water, health, education) withdirect support for artisanal fishers (fishinginputs, techniques, research, credit, etc) was to many a definite innovation. Overallthough, these innovations have involved only a small number of farmers. In WNRDP(Egypt), several small-scale initiatives (such as using organic crops to gain access toEuropean markets) have been innovative. In Pakistan, the PFC-ADP management teamwas forced to improvise innovative solutionsto an immensely challenging water re-

routing, but there are questions as towhether this approach will be replicated elsewhere in the Pat extension area.

4.14 PMD highlights other initiatives in countriesnot visited by the IEE. Box 7 contains anexample from Nepal.

4.15 Overall evidence from the project sampleshows that there is little to distinguish IFAD’swork from that of other developmentagencies. And in several cases, the work ofIFAD seems outdated and falls behind that of other development agencies.174 Much ofdevelopment is about bringing old technolo-gies and methods to individuals and commu-nities. Indeed while some practices are new to particular locations and villages, the practicesthemselves are often not especially innovative.For example in Guinea, PD-PEF has brought

Box 7 Innovation in Nepal

This project included a new approach to forestry in Nepal: the transfer of 1ha parcels of degraded governmentforest land to poor people on 40 year leases. The original idea came from foresters in Nepal. An IFAD designmission recognized the potential of the idea. An IFAD/Dutch project piloted the idea, modified it, and expanded it.Government replicated it on a much larger scale and introduced the approach as a priority in the PRSP.

n Success of the up scaling of this innovation was influenced by the following driving factors.

n Successful scouting at the early stage of the process. While luck played a role in the identification of the oppor-tunity, credit must go to IFAD for recognizing the potential of the new idea and supporting it financially in spiteof major criticism from other donors.

n Clear advantages to the poor – the technology was simple and affordable for farmers, there was quick impactand hence strong interest from the beneficiaries.

n Strong political commitment for change – strong commitment from government partners in the early stagesof the innovation process; also strong political commitment for up scaling at later stages of the innovationprocess.

n Flexibility and learning – big changes in design were accommodated during implementation (deletion/ introduc-tion of components, introduction of new NGO partners during implementation, continuous adaptation of thetechnology itself).

n Need for strong innovation partners to undertake the R&D – the project benefited from a Dutch grant financedFAO, Technical Centre for Agricultural and Rural Cooperation (plus team) and strong innovative local NGOs.

Source: PMD report of first CPM forum.

174 In the OE Indonesia CPE (paragraph 120) for example, it was noted that while NGOs were using participatory techniques as early as the mid-1980s, IFAD had not fullyintegrated the participatory group approach into itsprojects until 2000.

several novelties for the men and womenwithin the project area: using the bottomland area that was not being used for cropproduction, applying irrigation to the produc-tion of rice, forming a productive groupementor a financial association. This does nothowever mean that the project, as a develop-ment endeavour, has been innovative or isreplicable. In Bangladesh, under SSWRDSP,IFAD’s main innovative role was in the estab-lishment of a process for the formation ofWMCAs, and creating a new formal status forthese organizations. Many of the farm prac-tices/technologies had never been practiced inthe area, although they were not new toBangladesh. The IEE beneficiary survey showsthat 60-75 per cent of respondents fromSSWRDSP thought that the project activitieswere mostly new to people. For AqDP themain innovation was the development ofcommunity organizational arrangements, andmost members found the approaches andtechnologies provided to be new. In PFC-ADP(Pakistan), IFAD came in at a late stage to theproject, and it is difficult to argue that lowerend, on-farm water distribution was an inno-vative element from IFAD – as it was origi-nally to have been supported by AsDB. SinceBADP replicates previous projects, the IFADRRP states that there is “no innovativeelement” – except perhaps a few new designshifts. In the United Republic of Tanzania, half ofthe respondents from RFSP thought that theproject activities were very new to them, andalmost all management and partners inter-viewed felt that it is likely (or very likely) thatother people in the area will adopt the activi-ties currently offered by the project. Sixty sixper cent from AMSDP felt that the projectactivities were mostly or very new – thoughgiven the few meetings undertaken to date, itis difficult to know if they are sufficientlyaware of what the project activities and inter-ventions will be. In summary, the views ofbeneficiaries on innovation amount to anassessment of “innovation for the village”, asopposed to “innovation for the country”.

4.16 In certain country contexts, IFAD is able todescribe virtually any development initiativeas ‘innovative’, regardless of its actual merits.In Egypt for example, given that the majority

of settlers in the area are either new to agri-culture and/or new to newlands agriculture, itis not surprising that 84 per cent of EDNASP(and 90 per cent of WNRDP) respondents feltthat the projects’ activities were either mostlyor entirely new to them. This in itselfhowever, is not a convincing definition ofinnovation and indeed some project partnersargue that EDNASP was not especially innova-tive in its use of agricultural techniques, socialdevelopment, etc. The case is similar forWNRDP. In Armenia, beneficiaries were mostlyfamiliar with the activities of each component.Indeed few new technologies were applied,and irrigation rehabilitation used fairlyorthodox approaches. Instead innovationrelates mostly to the ‘newness’ of activities in apost-Soviet context, plus the extension of suchactivities to rural areas. This includes thecreation of a cooperative bank providing agri-cultural loans, and the establishment of theAniv Foundation to develop small enterprises.Also, in a transition country with an embry-onic civil society, the creation of grass-rootsorganizations (water users associations; villageassociations; horticultural unions; civic actiongroups) was also fairly new to most benefici-aries. The post-conflict, market basedeconomy of Mozambique offers similar oppor-tunities to label development work as innova-tive. For PAMA, around two thirds of thosebeneficiaries surveyed considered the projectactivities to be either mostly or very new tothem. The design of PDIMA (the marketinginformation programme) was confirmed to beunique in that it catered for all crops and allseasons unlike those of the past. The conceptand promotion of local needs assessmentteams, provincial PAMA consultative councils,and focal area reference groups are all new tothe two provinces.

4.17 In summary, there are a few truly innovativeprojects. Even in the most innovative projects,it is difficult to see how IFAD contributes tothe creation, promotion, replication andlesson learning in any systematic way. In Perufor instance, it appears that the CPM (andproject staff) operated in a largely isolatedfashion from IFAD headquarters, with noevidence of an interchange or influence fromheadquarters once the loans were approved.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

126


At best it can be said that headquartersprovided the CPM with an environment of ‘no objection’ in which to operate.

4.18 In an assessment of IFAD’s innovative culture,the results from an independent quantitativestudy (by the University of Brighton) on inno-vation show that, IFAD compares poorly with awide cross-section of organizations that have beenreviewed, using (their methodology) IFAD scored inthe bottom 10 per cent.175 The OE Evaluation ofInnovation (paragraph 3) supports this view:The current approach to innovation is individual-ized, decentralized and unsystematic, and is deter-mined by individual and chance factors rather thanby a well-defined and acceptable sequential process.While innovations do take place, this fragmented,ad hoc style does not lend itself to good use of IFADresources. The OE innovation evaluation alsostated that knowledge management shouldperform a key role in learning from andpromoting innovations, and yet it has notreached its full potential.176

Knowledge sharing4.19 IFAD aspires to be, a leading source of knowledge

on the eradication of rural poverty – and in manycircumstances, “the voice of the poor” inpolicy forums. Yet, there is little evidence thatIFAD is regarded as a leader in its field,indeed often quite the contrary. In Guinea,IFAD has been a follower rather than a leaderwith regards to development trends (thelogical framework approach is still notapplied; the ‘women in development’approach has not given way to a wider genderapproach; and little is being paid to environ-mental issues). IFAD has made a contributionof its own in the area of micro-finance (e.g. the 1996 stocktaking study), but similar initiatives are missing in other much neededfields such as land tenure, promoting peasantorganizations, fighting environmental degra-dation, etc. Some approaches have beenprematurely abandoned without sufficientanalysis of their failure (e.g. the groupementsapproach), and perhaps most damning of all,IFAD has fallen short in an area that it shouldhave specific expertise: testing sustainableagricultural development in different agro-ecological contexts, social groups and withdue consideration of women and youth.

Similarly in Burkina Faso, almost nothing hasbeen achieved in terms of advocacy, catalysisor leadership, with most stakeholders instantlydismissing the questions on IFAD being aleader. Indeed for many observers, IFAD iseven less visible than its presence in terms ofprojects would suggest. In Egypt, partners areclear that IFAD is an advocate for the ruralpoor, but that IFAD simply does not haveenough of a country presence to advocate formuch within the sector. There is no evidencethat IFAD plays a leadership role in Egypt.

4.20 Part of the problem is that there is very littlesystematic capture and sharing of field-basedknowledge. In Pakistan, no evidence wasfound of any systematic process to share IFADknowledge, nor was IFAD regarded (by eitherdonor or borrower) as a current knowledgeleader. Some donors and partners do howeveracknowledge a debt to IFAD’s early focus onrural poverty. In Armenia too, there is limitedevidence that the experience and knowledgeof IFAD is being used in an advocacy, leader-ship or catalytic way within the country. Therapid pace of operations has meant that thesystematic capture and synthesis of knowledgehas not always been achieved, particularlyoutside a small group of individuals. Inter-regional knowledge exchange is especiallyunder-developed. In Bangladesh, both of thesample projects show poor performance interms of sharing knowledge and lessons morewidely beyond the implementing agencies andtheir partners. Overall, neither project hadestablished systems for the rigorous capture ofexperience and lessons. Similar findings werereported in Bolivia, Mozambique and the UnitedRepublic of Tanzania.

127

175 Report on the Evaluation of IFAD’s Capacity as a Promoterof Replicable Innovations in Cooperation with otherPartners, Understanding at Completion Point andExecutive Summary, Report No. 1325, November 2002,Office of Evaluation and Studies.

176 The OE Indonesia CPE also highlights that: In order todevelop programmes at the cutting edge of rural develop-ment and to fulfill its mandate effectively, IFAD must bemore responsive and innovative… Stronger analysis ofknown technologies during project design and criticallearning from IFAD’s own experience (written and oral)would greatly enhance IFAD’s approach, its programmeperformance and its credibility with development partners.

4.21 Where knowledge sharing does occur, it ismostly with regards to building on the experience of past projects.178 Plus, some ofthe more promising examples of knowledgemanagement were reported in Mozambique:(i) knowledge sharing is more structuredbetween IFAD headquarters and individualprojects, particularly at the project inceptionstage; (ii) the IFAD/International LabourOrganization Turin training programme forproject managers and staff was consistentlycited as one of the most valuable processesto transfer knowledge from IFAD headquar-ters to new projects; and (iii) knowledgesharing between IFAD projects at a regionallevel is also programmed, through theIFAD/UNOPS Annual RegionalImplementation Workshops.

Organizational partnerships4.22 Across all CWPs, the general sense is that

IFAD is distant and disconnected from poten-tial partners. In some countries, this lack ofengagement is proving especially serious inthe context of donor harmonization processes.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

128


Box 8 New developments in innovation

Management has responded to concerns about innovation raised in the OE evaluation. Three recent initiatives are geared towards supporting innovation: the 2003 policy for grant financing; the Field Presence Pilot Programme;and most specifically an IMI for which there is financial support in the form of a matching grant from the United Kingdom.

A December 2003 information note to the Executive Board stated the broad goal of the IMI to enhance IFADcapacity to promote innovations that will have a positive impact on rural poverty. The three primary results expectedfrom IMI implementation, which derive directly from its central objectives, are: (i) enhanced IFAD innovation cultureand capacity; (ii) improved quality and impact of innovation in the field; and (iii) improved innovation learning, andthe application and sharing of such learning. The intention is for IFAD to develop a distinctive style of innovationthat is demand-driven, participatory, replicable, policy-directed, mainstreamed and impact-oriented.177

A draft ‘Operational Framework for the Main Phase’ (EB 2004) was in preparation at the time of writing. A centralfeature of the approach is the recognition of the challenge of trying to change the work culture Providing incentives for staff. Promotion of innovation requires an appropriate system of incentives and rewards to motivatestaff. A major element of motivation would be derived from gaining the time and freedom to pursue good ideas,associated with gaining access to IMI funding. This should be reinforced by the professional recognition associatedwith successful innovations. Therefore the following should be introduced:

n non-financial rewards to motivate staff, competency development, results achievement, client feedback andincentives for learning and sharing;

n the release of staff time for participation in immersion opportunities, training and promotion of innovations;

n integration of the innovation support function into job descriptions;

n a system for evaluation of IFAD staff performance in relation to innovation promotion; and,

n a mentoring arrangement between junior and senior staff members.”(paragraph 36)

The IEE welcomes the draft document. In view of the clearly recognized need to strengthen learning and to tacklethe organizational culture, the approach would be strengthened if the text had stronger links to core initiatives suchas the new HR policy, to demonstrate how changes in HR will support innovation and how the IMI might supportthe HR policy. The approach would also benefit from a more detailed presentation of the behavioural changes thatare needed to achieve the outcomes and a set of objective monitorable indicators of change and the outcomes. Alogframe would be a helpful addition.

In summary:

n IFAD’s aspiration to become a, “leadingsource on the eradication of ruralpoverty” is severely undermined by thelimited (and unsystematic) capture offield experience and knowledge.

n This further undermines IFAD’s potentialto be an advocate on behalf of therural poor.

In the United Republic of Tanzania (see Box 9),all donor interviewees and several govern-ment and private sector informants view IFADas almost entirely disconnected from the verysignificant changes underway in government-donor relations in recent years. More seri-ously, IFAD’s traditional modus operandi is seento be regressive and at odds with the new aidcoordination context in the country. Threerecently prepared IFAD projects have facedintense opposition from different partners –over issues ranging from the nature of identi-fication and formulation processes, to moredetailed design issues.

4.23 IFAD’s so-called ‘partnerships’ frequentlyextend little beyond contractual relationships,or a core group of implementing agencies. In Peru, IFAD has no real partnerships. Both

MARENASS and CORREDOR have beenvery influential and developed organizationalrelationships beyond the project structure,but this has resulted from the projects’ repu-tation for innovation and achievement rather

129

177 Information note: IFAD Initiative for MainstreamingInnovation (EB 2003/80/INF.4)

178 In Burkina Faso, PAMER has learnt from a similar projectin Senegal, and uses projects in neighbouring countriesto train and enlighten staff. In Egypt, IFAD has progres-sively learnt from the experiences in Minya and Fayoumand incorporated these into the project design for APIP.In Bolivia, the design documents of both projects reflectlessons learned on other projects, and interviews withproject coordinators and staff confirm their knowledgeof previous experience. Similarly in Bangladesh, thecountry portfolio has been developed from lessons ofpast experience, both of IFAD projects and more gener-ally in Bangladesh.

Box 9 New approaches to aid planning and management challenge IFAD’s traditional business model

The United Republic of Tanzania is now one of several more advanced low income countries, where governmentand donors are moving steadily to replace traditional projectized aid modalities with a domestic process foundedon increased dialogue, shared perspectives and harmonized approaches to policy and organizational reform andfinancing. This process is at different stages in different countries, but trends are very similar: increased localdialogue, leading to unified donor support around government sector reform policies; formal sector-wideapproaches; basket-funding arrangements financing jointly-agreed interventions and harmonized reporting, and,in the most advanced cases, budget support.

The COSOP (October 2003) offers verbal comfort that IFAD programming is articulated with national processes butthe reality is very different. The operational form of the ASDS is the Agriculture Sector Development Programme.The ASDP is a process not a document and it is important to understand the complex and contested nature of thatprocess in the United Republic of Tanzania.

It is well-established that progress on agriculture and rural development sector-wide approaches is much morecomplex and difficult than for those in the social sectors. More ministries are involved, normally with conflictinginterests; defining appropriate public-private functions is much more challenging; key ministries can face down-sizing or abolition rather than the enhanced resources which normally reward health and education ministryinvolvement in sector strategies. Further, in the United Republic of Tanzania, as elsewhere, the ASDP process is itselfembedded in a complex network of other reforms: the civil service reform programme; budget planning, execu-tion and salary reform, and, importantly, the government’s decentralization effort and local government reformprogramme. The inter-relationship between sector strategies and decentralization is notoriously difficult to designand implement in all countries and a push-pull between line ministries and the local government ministry is nowunderway in earnest. The central point – underscored by all donor and central ministry informants – is that in theabsence of decentralized country presence and authority IFAD has no prospect of even understanding day-to-daydevelopments on these issues, let alone of ensuring that its programming is consistent with them. The implicationfrom the COSOP that there is a fixed agriculture strategy from which Rome can somehow read-off appropriateprogramming priorities is a significant error.

Against this background, the reason why key partners see IFAD as not just disengaged, but potentially regressive,is clear. The central issue is IFAD’s traditional programming, identification and formulation modus operandi. IFADdoes not appear to have internalized the nature of the government-donor processes in the United Republic ofTanzania which have emerged over the past half- decade.

than from a determination by the projects toseek out organizational partnerships. InBurkina Faso, none of the partnerships havecontributed in any meaningful way to theobjectives as set out in IFAD V and IFAD VI.Indeed partnerships with governmentservices have not led to sustainable changes,and those with the private sector have not ledto anything other than the completion ofwork as contracted. It should also be notedthat the PDRSO partnership with theNetherlands Development Organisation andFBS were perceived as negative (by thosepartners), as was the failure to relate to theneighbouring complementary GermanTechnical Cooperation project. Likewise, inGuinea, relations with the public sector at thecentral level have been mainly formal or non-existent. At the field level the relationship hasconsisted of subcontracting activities, not inestablishing a real partnership (withexchanges of view and a joint identification ofneeds, concerted planning and execution ofactivities, joint monitoring and analysis). In Egypt, and away from the central imple-menting partner, it is less clear how extensiveand creative IFAD’s relationship with otherpartners is – even with substantial partners in the projects under review.

4.24 There are some instances of relationshipsbeing developed within host ministries –although it is not clear that IFAD has anyparticular policy influence in this context. InEgypt for example, relationships between IFADand the Ministry of Agriculture and LandReclamation are good and there is an acknowl-edged sense of two way dialogue and respectthat allows for supportive mutual criticism.

4.25 In the United Republic of Tanzania andMozambique, liaison officers have contributedto improved relations with key donorpartners. The evidence from Mozambique andthe United Republic of Tanzania is that theappointment of local, relatively junior, under-resourced, liaison officers helps to improvecommunication. But they lack status, have noauthority in relation to identification, formu-lation or implementation and cannot speakon behalf of IFAD. In Mozambique, IFAD issupporting ProAgri, and relations with donor

partners are good, possibly influenced by thefact that donors are currently becoming morecircumspect about the benefits of sector andbasket support to the country. However, priorto the opening of an IFAD Liaison Office inMozambique two years ago, there was not asingle formal mechanism for exchange ofexperiences between IFAD projects and thoseof other donors. Since the opening of theIFAD office, IFAD is now represented inseven permanent forums.

Policy dialogue4.26 There have been some instances of policy

impact. In Bangladesh, the most notable policyimpact (under SSWRDSP) was the project’skey role in convening and facilitating thepreparation of new guidelines in participa-tory water management in Bangladesh –something which is now mandatory for allwater management projects. The project hasalso contributed to the formulation of thenational water policy. RFSP in the UnitedRepublic of Tanzania has played a significantrole in contributing to national policy reformin microfinance provision and cooperativeorganization, while NAFP in Mozambique hasmost some notable inroads on policy, inparticular (i) the adoption of a national fish-eries policy and strategy; (ii) new legislationto legalise fisheries co-management commit-tees; and (iii) a new policy and legislation offishing zones and nets. PAMA Mozambique hasalso demonstrated important policy achieve-ments, namely: (i) draft legislation on regis-tration of farmer associations; and (ii) aposition paper on modification of the agricul-tural concession’s policy.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

130


In summary:

n IFAD field-based partnerships are oftenlittle more than contractual relation-ships, and the organization as a wholecan sometimes appear distant anddisconnected from key agencies.

n Liaison officers have helped to improvesome relationships, but more is neededto improve the connectivity betweenIFAD headquarters and the field.

4.27 In general though, the evidence at country-level shows that IFAD has not engaged muchin policy dialogue. This is closely related toIFAD’s underperformance in its knowledgemanagement and organizational partnerships(see previous sections). In Pakistan, there islittle evidence of significant policy dialogue inrelation to both sample projects. Under PCF-ADP, it is surprising that no attempt was madeto begin to address the issue of inequitableland distribution (and problems of tenantrelationships), given its very large impact onpoverty. Similarly policy issues relating tomaintenance remain largely unresolved andshould have warranted greater attention. ForBADP too, there are significant areas of policydialogue that should have received more

attention, such as: the control of interest ratesto borrowers (in relation to micro-finance),and the implications of the devolution processcurrently underway in Pakistan. Likewise inthe United Republic of Tanzania, despite theproject achievements of RFSP, no governmentor donor informant could cite evidence ofcorporate IFAD impact on national policy,nor IFAD acting as a catalyst/leader in ruralpoverty reduction. The desk reviewconcluded that the United Republic of TanzaniaCOSOP was, “a poor vehicle for communi-cating IFAD country strategy…” and thisseems corroborated by the apparent minimalimpact on government and donor agencies.In Egypt, the IEE found no evidence tosuggest any major impact of IFAD-related

131

Box 10 Policy impact depends on sustainable implementation and enforcement

IFAD’s experience in Mozambique shows that policy change without attention to implementation and enforcement is unlikely to make an impact. A sharp and consistent focus on policy reform through the NampulaArtisanal Fisheries Project contributed to at least five distinct changes to fisheries policy: (i) introduction of a 3-mile trawler exclusion zone to enlarge artisanal fishing grounds; (ii) reduction in import duties/taxes for fishinginputs; (iii) new legislation on minimum mesh sizes to enforce a ban on the use of mosquito nets; (iv) legislation topromote community co-management of the fisheries resources; and (v) waiver on “closed season” restrictions inNampula province to allow year-round fishing by the poor. However, with a few exceptions many of the abovepolicy changes have lacked implementation and enforcement confining their impact largely to paper.

For instance, in order to promote enforcement of the new trawler exclusion zone, NAFP provided a patrol vesseland recurrent resources for surveillance to the Department of Fisheries Administration. However, when the NAFPpipeline dried up at project closure, the government also discontinued trawler surveillance for lack of O & Mresources. The results were obvious: unabated violation of the new law by industrial fishing vessels and continuedconflict between them and artisanal fishers. The large vessels continue encroaching into artisanal fishing groundsdestroying fishing gear and sometimes causing fatal accidents with small-scale fishers at sea.

The experience raises an important question about effectiveness and sustainability of policy change. The government treats the trawler vessel exclusion zone and the surveillance of national waters as matters of toppriority but lacks the means to put in place effective and sustainable systems. At present, government plans toestablish a satellite-based vessel tracking system with funds from the European Community, but the EC facility hasno provision for patrol vessels and the department is already aware it will be an incomplete intervention. It is insuch areas where IFAD could play a value-adding role, assisting older projects to carry forward the institutionaldevelopment process. A project that seeks to achieve policy change is incomplete unless it also supports thecreation of sustainable capacity for policy enforcement within the relevant authorities.

In another example of failure, the waiver on closed season restrictions intended to benefit poor fishers in Nampulawas not supported with a strong exclusion mechanism for the rich. The benefit has since leaked to industrial fishingfirms and fishers from outside the province, and it is feared that it might be leading to more rapid depletion of thefisheries resources in the area. This experience is clearly typical of good intentions with bad results.

The above evidence points to additional roles for IFAD. Firstly, there is a clear need to address issues of policy imple-mentation and enforcement in addition to policy development and change. Second, IFAD should support thecontinuous monitoring and review of the performance of the new policies. The opportunity to review policies isusually availed through government’s usual five-year cycle of planning, normally linked to general elections. Resultsfrom IFAD’s work could feed strategically into this process.

policy initiatives; to aspire to being a leadagency in a sector implies a level of resourcesand engagement that IFAD clearly does nothave. IFAD could have much more impact incooperation during implementation, talkingthrough with partners the changes that areneeded and learning the lessons of workingtogether. Furthermore, the nature ofdialogue with partner organizations in Egyptis largely underdeveloped, and IFAD has anotable absence of tangible instruments fortraining and practice. Similar situations werereported in Armenia, Bolivia, Burkina Fasoand Guinea.

4.28 There remains a huge potential for IFAD to contribute to policy. In Peru, despiteMARENASS being considered a success, it hasnot generated a dialogue on policies for ruraldevelopment either at the local or sector level.This position is beginning to change withPREVAL’s more aggressive publicity campaignincluding the new book, Ten Keys. Some stake-holders felt that IFAD had sufficient experi-ence to exert influence but did not use it (i.e.from a platform above the level of the CPM).In Armenia, both NWASP and ASP havecontributed to changing the institutional land-scape, with the creation of the only coopera-tive bank (and main lender to rural areas) inthe country, plus the Aniv Foundation, whichlends to small entrepreneurs and start-upbusiness (while others have focused more onmedium enterprises).

5. Project performance5.1 There are some high performing projects in

IFAD’s portfolio but many projects are lessthan satisfactory. Table 7 provides a summaryof the main performance ratings generatedby the project evaluations. In terms of rele-vance to corporate objectives and countrydevelopment priorities, IFAD financedprojects are doing well. All the projects evalu-ated sought to reduce poverty, either directlyor indirectly, and poverty reduction is one, ifnot the overarching priority of the majorityof IFAD borrowing countries. Effectiveness issubstantial in two-thirds of cases, but thisleaves one third of projects underachievingagainst development objectives. Efficiency issatisfactory (high/substantial) in only half ofthe projects evaluated and less than fullysatisfactory in the remainder. IFAD financedprojects of course rely heavily on theperformance of the borrower for successfulimplementation. In most of the IEE projectsthe borrower performs reasonably well,although there are clear grounds forimprovement. The performance rating forCIs and IFAD combined is marginally lowerthan that of the borrower, reflecting mainlyproblems with supervision and delayedaction on the part of IFAD to correct under-performing projects. While the sample ischaracterized by a few outstanding projects,the IEE finds that the current operatingmodel fails to lift project performance more generally.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

132


Table 7 Summary of project performance ratings179

(sample size: N = 20 projects except where stated) High Substantial Modest Negligible N

Relevance: The extent to which the project fits country development priorities, IFAD strategy and beneficiary needs. 60% 40% - - 20

Targeting: The extent to which the design targeted the right people with appropriate activities. 10% 55% 35% - 20

Effectiveness: Achievement (or expected achievement) of the development intervention’s major objectives. - 67% 33% - 18

Efficiency: the economical conversion of inputs/ resources into outputs and outcomes. 20% 25% 50% 5% 20

Performance of the borrower: The overall performance of project management and implementers. 25% 35% 40% - 20

Performance of CI and IFAD: The overall performance of IFAD and the supervisors. 10% 45% 45% - 20

Source: IEE ratings (form 5).

5.2 There is a high degree of similarity betweenthe results of the IEE country studies andthose of the OE ARRI 2003. In terms ofproject performance (relevance, effectivenessand efficiency) the aggregate results areremarkably similar, as well as in terms of ruralpoverty impact and sustainability (see Table8). This provides support to the findings ofthe IEE, especially as the ARRI examines adifferent subset of the IFAD portfolio.180 TheARRI draws upon four country programmes,two thematic issues, and one corporate topicevaluated in 2003. Apart from the countrystudies (20 projects, 10 countries), the IEEalso builds upon evidence from a desk reviewof 42 projects across 21 countries.

5.3 Comparing the performance of IFAD withother development organizations is difficultbecause of the lack of readily comparabledata. Many IFIs (ADB, AsDB and IADB) andother development agencies do not disaggre-gate data for the rural sector. The World Bankrural portfolio therefore provides the nearestcomparison to IFAD. For completed projects,World Bank figures show an average 67.5 percent of rural sector projects achieved a satis-factory outcome between 1994 and 2003,compared to 68.9 per cent for the portfolio asa whole.181 In FY02 the figure for the ruralportfolio increased to 81 per cent satisfactoryoutcome, reflecting substantial efforts on thepart of the Bank to improve portfolio qualityin its rural sector operations. The Bank’soutcome rating is a combination of relevance,efficacy (effectiveness) and efficiency.

Averaging the IEE results for all 20 projectsacross the three criteria produces an ‘outcomerating equivalent’ of 70.7 per cent for theperiod, which is close to the Bank’s estimate.When restricted to closed projects only theIFAD figure falls to 61 per cent. The sampleof closed projects is very small and thecomparison should be taken as indicative atbest, but the indication is that, even if IFAD isclaiming to work differently from other organ-izations such as the World Bank, it cannotafford to be complacent about its levels ofproject performance.

Relevance of IFAD investments5.4 Overall, there is a high degree of congruence

between project investments and the countrydevelopment priorities, IFAD strategies andbeneficiary needs.182 This is not particularlysurprising as the development priorities of thecountries concerned are often expressed inbroad terms, and beneficiary expectations arefrequently high. In Mozambique for example,the objectives of PAMA (to promote agricul-tural market linkages) clearly fit with thegovernment strategy of commercializingsmallholder agriculture and its commitment toliberalizing the economy. Indeed the govern-ment has withdrawn direct intervention inagricultural markets, and at a faster pace thanthe growth of the private sector, thus creatinga void within the marketing system into whichPAMA work. In the United Republic of Tanzania,almost all respondents from RFSP andAMSDP reported that the projects are either

133

179 The modal class(es) are shaded. The ‘not rated’ columncovers projects in the early stages of implementation,where it was considered inappropriate to rate effective-ness (plus, impact and sustainability): BADP, Pakistan andAMSDP, the United Republic of Tanzania.

180 Although different projects in both Burkina Faso andGuinea were evaluated by the ARRI and the IEE.

181 This refers to the percentage of satisfactory projects bynumber, not by disbursements. The figure weighted bydisbursement is higher both for the rural portfolio and theBank as a whole.

182 Four projects (PRODESIB, PROSAT, NWASP, ASP) cite a highdegree of congruence with IFAD strategic documents (thestrategic framework, the relevance regional strategy andthe COSOP) though this partially because the projectdesign predates these strategies.

Table 8 Comparison between IEE results and OE’s ARRI 2002 and 2003

% high and IEE ARRI ARRI

substantial sample 2003 2002

Relevance 100 90 80

Effectiveness 67 70 60

Efficiency 45 50 50

Impact on poverty 55 50 50

Sustainability of impact 61 50 40

Source: ARRI of IFAD Operations, 2003.

highly or extremely important to their lives.This is entirely consistent with the aggregateof beneficiary responses (Table 9), yet it seemsto be biased by a high degree of desire-to-please; AMSDP has only just started with fieldoperations in March 2004 and with mostgroups having had only an initial meetingwith the contracted organizers (partneragencies).

5.5 While development priorities are often broad,and beneficiary expectations high, IFAD’sstrategic statements are a weak guide to operational selectivity. The IEE desk review(page 16) notes about the strategic framework2002-2006: The intellectual and policy frame-work is largely permissive, ruling almosteverything in and very little out. Its impact on projects… is hard to discern and it wouldbe hard to rule out ‘fit’ in almost all cases.

The same report finds that the regional strate-gies are performing a critical function, notleast in filling part of the strategy/operationalvacuum left by the strategic framework.Important weaknesses are however noted,including the limited recognition of interna-tional best practice and knowledge aboutgrowth and rural poverty reduction at theregional level, plus a lack of clear indicatorsfor monitoring progress against the regionalobjectives (ibid, pages 16-17).

5.6 COSOPs are not providing the rigorous filterrequired at project identification. The deskreview highlights a number of general weak-nesses in COSOPs (as summarized in Table 10),but most importantly these country strategiesare not adequately limiting and shaping theproject pipeline. In Pakistan for example, theidentification of issues, constraints and

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

134


Table 9 Beneficiary assessment of relevance

Relevance: Of all the help you need to improve your lives, how important is this project to you?

(sample size: 20 projects) Extremely Highly Quite Not really Other Nimportant important important important

Late/closed projects 19% 49% 25% 7% - 701

Early/middle projects 36% 37% 20% 6% 2% 775

All projects 28% 42% 22% 7% 1% 1 476

Source: IEE beneficiary survey (form 4) data.

Table 10 Key findings from the desk review of COSOPs

Criteria Key issues

Relevance n Only 50 per cent of COSOPs rated satisfactory on consistency with national priorities and poverty reduction strategies - although a large number were prepared before the finalization of national PRSPs

n Relatively high share of COSOPs did not provide an adequate description of IFAD’s development niche in the country (with later COSOPs being rated more poorly on this theme)

Effectiveness n COSOPs less good on the analysis of governance and institutional issues.n Improving, but weak clarity and consistency of COSOP objectives and the articulation

of indicators

Efficiency n COSOPs do less well in the rationale for innovation and scaling up, or in leveraging and catalyzing others

n 80 per cent of COSOPs rated unsatisfactory for arrangements for monitoring and evaluating at programme and strategy level

Source: IEE desk review (pages 18-21); Sample of 21 COSOPs

priority directions in the COSOP are gener-ally quite convincing, yet the leap to thethree-project, partially defined, lendingprogramme is abrupt and not connected.There is no explicit application of the givenstrategic filters to the selection, and in none ofthe descriptions of the three potential projectsis the opportunity for policy dialogue orenhanced access to resources explored. InPeru, the COSOP is practically invisible,though IFAD is not (with the CPM being awell respected actor in Peru’s rural develop-ment community).

5.7 At times, weak strategic filters and identifica-tion processes have resulted in IFAD makingquestionable investment decisions. The IEErating of relevance is a measure of the ‘fit’ tonational development priorities, IFADstrategy and beneficiary needs. It does notassess whether the project investment was thebest possible use of resources (that could havebeen designed at the time of the projectformulation). In at least one quarter of theprojects surveyed, there are strong reserva-tions about whether the investment decisionwas correct (see Box 11).184 The reasons forthis are varied, including: (i) Pakistan: strong

135

183 For the Egypt programme as a whole, the CPE (2004, page 5) also notes that there is less relevance to thepoorest.

184 It is problematic to provide exact figures as the reserva-tions are particular to individual projects as well as moregeneral country approaches. The estimate of ‘one quarter’is based on at least five of the twenty projects (25 percent) or three of the ten countries (30 per cent).

Box 11 Reservations about IFAD’s investment decisions

In Pakistan, the beneficiaries of PFC-ADP rate the relevance to their lives as high, yet the project is barely above ‘modest’ in other respects. This is due to concerns about the fit with IFAD’s rural poverty alleviation objective, namely:

n it does not target the poorest areas in the province;

n the project has a poor rate of return in support of the economic growth argument;

n the equity impact of bringing water is skewed by the pattern of land ownership;

n the project does not attempt to address the underlying causes of poverty (skewed land ownership).

The rating for BADP also has qualifications, including the relevance to the target group of agriculture and livestock when these practices currently contribute only 20 per cent of farm income.

In Egypt there is, on the one hand, a clear fit between what the government wants to achieve and where IFAD canmake a contribution: to assist with new lands reclamation and settlement and make viable livelihoods for thedispossessed and unemployed. On the other hand, development of the reclaimed desert areas is not without itscontroversy, and IFAD's involvement in agriculture is not necessarily the highest priority of many beneficiaries(where there is an absence of even basic infrastructure and services). Indeed, there is some debate as to whetherIFAD is working with the poorest in Egypt as a whole - with farmers in Upper Egypt being poorer and an increasingnumber of agencies turning their attention to this region.183 Furthermore, the Ministry of Agriculture and LandReclamation is not a natural partner for participative, bottom-up planning and intervention, plus the more innova-tive aspects of community development and gender work. One way forward could be to introduce the Ministry tothe innovative work of NGOs such as CARE whose action research on small-farmer cash cropping in Upper Egypthas recently been recognized with a World Bank grant.

In summary:

n IFAD investments are broadly congruentwith national priorities, IFAD’s strategiesand beneficiary needs.

n The IFAD strategic framework, and to alesser extent the regional and countrystrategies do not provide the necessaryfilters for selectivity. Plus, there islimited sector work, to provide a strongcase of when and where not to work(i.e. there is no clear basis on which toturn down a project).

n There is no real challenge at identifica-tion such as through a concept noteprocess, which challenges CPMs fromtechnical and cross-cutting areas.

concerns that projects do not fit with IFAD’sobjectives; (ii) the United Republic of Tanzania:IFAD’s traditional project approach runscontrary to the sector-wide approach of thedonor community; and (iii) Egypt: IFAD’sunilateral support of the government under-mines its position to pursue a more desirabledevelopment agenda.

Targeting the rural poor5.8 Under the Agreement Establishing IFAD,

the organization is clearly required to workwith the rural poor: …to introduce, expand orimprove food production systems and to strengthenrelated policies and institutions (Article 2); and,…to increase food production and to improve thenutritional level of the poorest populations…(Article 7). For many this has meant workingwith the ‘poorest of the poor’.

5.9 Effective targeting is also strongly linked tobetter project performance.185 On paper atleast, the IEE DRR rated targeting as mostlysatisfactory (83 per cent). The report alsonotes (paragraph 3.58) that, the best projectformulations provide defined criteria by area andsocial group, which the borrower and projectmanagement can directly operationalize. The less-than satisfactory cases variously fail to state why theproject area was selected; give non-operational defi-nitions (‘the poor’) or fail to note the existence of keygroups (e.g. bonded labourers). In the countrystudies, SSWRDSP (Bangladesh) is one suchexample, where the discussion of target groupsis difficult to follow in the design documents,and it refers to all categories of people fromdestitute women to fishers and medium tolarge farms. There are also examples where

a failure to grasp the underlying causes ofpoverty has undermined the overall impact ofthe investment. In Pakistan (PFC-ADP), therehas been a failure to take account of under-lying issues of land ownership. The originalintentions of selectively supporting poorerdistributaries or reaches was likely to end upas about as regressive as land ownership. InGuinea, PD-PEF identifies the poorest as thosefarming less than three hectares, but no refer-ence is made to the characteristics of the landor the rights to access (ownership or usufruct),yet these are of great importance to deter-mining the benefit that can be drawn fromcultivating a particular plot. Furthermore,because the project focuses on developing thebottomland, it has in practice not focused onthe poorest who cultivate mainly the hillsides.

5.10 Pro-poor project designs do not always leadto on-the-ground success. Results from thecountry studies indicate concerns withtargeting: 65 per cent of the sample are rated high/substantial for targeting overall –in terms of poor areas and poor functionalgroups (smallholders, artisanal fishers) – and 40 per cent rated high/substantial fortargeting poorer people within those areasand groups (see Table 11). This suggests thatthere have been problems with targetingduring implementation.186

5.11 In Bangladesh, SSWRDSP is a jointly financedproject between IFAD and AsDB. The designdocumentation of IFAD differs from that ofAsDB, but there is no evidence that the IFADversion of the targeting – with an emphasison the more vulnerable and extreme poor -

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

136


Table 11 Assessment of targeting187

(Sample size: 42 projects) % % Sample satisfactory unsatisfactory size

Overall targeting (design) 83 17 42

Overall targeting (implementation) 65 35 20

Women (design) 81 19 42

Women (implementation) 58 42 19

Poorer people (implementation) 40 60 20

Source: IEE desk review and Form 5 ratings.

ever influenced the way the project wasimplemented by LGED or supervised byAsDB. In Guinea, the main investment ofrehabilitating the Telimele-Samanko-Sangaredi link (under PAPE-BGN) may beimportant, but its priority is not to connectsmall villages and the productive areas to themain artery. Under both PAPE-BGN and PD-PEF in Guinea, investment in the bottomlandareas is a strategy that will not reach thepoorest. In Pakistan, the area chosen by PFC-ADP is by no means the poorest part ofBalochistan (and plans to target the poorestgroups along watercourses were impractical).The examples continue:n AqDP (Bangladesh): the pond aquaculture

component was redefined during theMTR, and the resultant component nowfocuses primarily on larger ponds. Theseare generally owned by better-off house-holds, thus missing poorer households thatowned small ditches (doba/pagar).

n PDRSO and PAMER (Burkina Faso): In bothprojects, the resultant work has led to animpact on the better-off, as evidenced in the well-being ranking exercises andthose surveyed.

n NWASP and ASP (Armenia): There is anincreasing emphasis by project manage-ment to focus on the more entrepre-neurial/productive poor under ASP (thefollow-on project). Beneficiary surveysshow that 69 per cent of those asked statedthat the majority of beneficiaries underASP were either richer or much richer, ascompared to 49 per cent for NWASP.

n EDNASP and WNRDP (Egypt): Theprojects are intended to target landlessfarmers and unemployed graduates, butthere is some concern as to whether IFADis targeting the poorest in Egypt across thewhole portfolio – with farmers consideredto be poorer in Upper Egypt (in the south).

5.12 Implementation mechanisms are often inade-quate to prevent benefit capture. In manycases it is not appropriate (or even desirable)to target specific groups, though targeting canremain an issue. In BADP (Pakistan) forinstance, the appraisal report and RRPprovide surprisingly little explicit targeting

protocol. In the absence of tighter selectionprocedures, there is an incentive for NGOs tokeep costs down by selecting potentially more‘fruitful’ (and less poor) targets – plus there isno procedure to track assistance given tocommunities and so help ensure that it isfairly distributed. The lessons for implementa-tion are threefold:n Provide an explicit targeting guidance and

procedures for implementing partners

n Design and implement mechanisms thatprevent capture by the richest groups

n Track and monitor the assistance provided, feeding these back into management decisions

5.13 And, it is not always the poorest that are theintended focus of IFAD investments. Projectsin Armenia, Bangladesh and the United Republicof Tanzania demonstrate a purposeful focus onthe more ‘productive’ poor or those whorepresent a better investment opportunity(small entrepreneurs, owners of large fish-ponds, etc). Sometimes this is justified: InPeru, CORREDOR does not work with thepoorest in the region. Percentages of‘extremely poor’ and ‘very poor’ for thefifteen provinces included in CORREDOR are6 per cent and 37 per cent respectively, whichcompares with 27 per cent and 50 per centfor the department overall.188 The project

137

185 A review of targeting by Coady, D., Grosh, M. andHoddinott, J. (2003) found that targeting does work:Typically, the median program provided a quarter moreresources to the poor than random allocations wouldhave. The ten programs with the best incidence deliveredto the poor two to four times their per-capita share ofbenefits. Progressive allocations were possible in allcountry settings, in countries at markedly different incomelevels, and in most types of programs. (World Bank, SocialSafety Nets Primer Notes, Issue 10, 2003).

186 In a recent review of targeting in 122 programmes(projects) in 48 countries, Coady, D. et al (2003: 64-5)concluded that, the quality of implementation matterstremendously to the targeting outcome.

187 The % satisfactory and % unsatisfactory is the aggregateof high/substantial and modest/negligible respectively.

188 In Peru, departments are divided into provinces, which arefurther divided into districts.

aims to strengthen the commercial linkagesbetween populations in the “corridor”, andthe project does not target communities withlimited access to commercial relationshipssuch as due to the lack of reliable roads orgreat distance. This means that while theoutermost belt of isolated communities (wheresevere poverty is practically universal) wasexcluded, to include these would underminethe “linkage” concept of the project (and alsonecessary to keep administrative costs low).

5.14 IFAD is not alone in facing the difficulties oftargeting. A report on the World Bank’s ruralportfolio also highlighted weak targeting inthe rural portfolio, especially to the poorestcountries or regions within countries.189 TheBank programme of assistance to the ruralpoor does however extend beyond the ruralportfolio, and includes sector work, policydialogue, and lending that indirectly affectsrural populations. IFAD’s ability to influencedoes not extend much beyond its lendingprogramme. The combination of weak projectdesign, weak implementation mechanisms anda strategic shift away from a clear focus on thepoorest to a much wider range of rural poorgroups may begin to undermine IFAD’s‘specialist’ position within the rural sector.

Effectiveness of implementation5.15 Two thirds of IFAD financed projects are

expected to achieve their development objec-tives, but one third are lagging behind.Overall, 67 per cent of projects are rated ashighly or substantially effective. This is consis-tent with the 2003 ARRI, where the equiva-lent result is 70 per cent. If effectiveness wereto be narrowly interpreted as a measure of theproject achieving its own objectives (i.e. “howwell has the project done what it set out todo?”) then the figure would probably behigher. This however would mean comparingprojects on a different basis, with someprojects seemingly scoring better just becausetheir objectives are set lower.

5.16 Effectiveness is affected by the quality of aproject’s initial design. During project start-up it is reasonable to expect some changes inthe design and implementation arrange-ments – though better design does lead to

improved effectiveness (see Box 12).Occasionally, reformulation can have a verypositive outcome. In Peru, the MARENASSproject underwent a reformulation of objec-tives initiated by project management a yearafter start-up. This is said to have beennecessary to reconcile the overarching goalsof the project with giving full initiative tocommunities to assert their own priorities.

5.17 Implementation problems are a major cause of poor project effectiveness. Of the 16 projects in the sample where implementa-tion is well underway or completed, half have suffered from major implementationproblems (not including the two Boliviaprojects, which were force majeure inter-rupted). Three projects (two in Guinea, andPDRSO in Burkina Faso) experienced majormutually reinforcing weaknesses: the size ofthe working area, the target population, andthe number of components were too great,and the selection of the project managementteam was not well done. In Egypt, EDNASPwas considered a failing project due to weakmanagement, poor staff recruitment, lack of project infrastructure, slowness incontracting, difficulties in reclaiming land,etc. The project was redesigned following the mid-term review and has improvedsignificantly. In Bolivia, both PRODESIB andPROSAT had major institutional problemsand were re-designed through reorientationmissions. In Pakistan, PCF-ADP underwent asubstantial redesign at MTR. In Mozambique,NAFP had its non-feasible componentscancelled at the MTR stage with new simplerones added.

5.18 Some of the implementation difficulties arebeyond IFAD’s control. Two projects(MARENASS, Peru and RFSP, the UnitedRepublic of Tanzania) have required modifica-tion as a result of the non-appearance ofbudgeted cofinance. The two projects associ-ated with AsDB loans (SSWRDP, Bangladeshand PFC-ADP, Pakistan) have faced difficultiesflowing from both design and implementationarrangements, and largely beyond IFAD’scontrol. The two big marketing projectsrequired modifications in the early stages:PAMA (Mozambique) required modification of

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

138


component design, whilst AMSDP (the UnitedRepublic of Tanzania) has significant mistakes inthe preparation of the (non-IFAD) infrastruc-ture budget as well as major resourcescommitted to a small-group organizationapproach which the appraisal report deemedto be high-risk.

5.19 Redesign is at the MTR, which appearsmore effective at turning around poorperformers than periodic supervision visits.Seven projects in the sample of twenty (35 per cent) were severely delayed untilmajor modification following the respectiveMTRs.191 While it is to be expected thatsome design changes will occur in the firststages of implementation, in many casesthese are not addressed until years later.IFAD prides itself on its flexibility, but a will-ingness to flex is of little value unlesssomeone takes initiative to make it happen.The evidence is that for failing projects, nosuch initiative occurs, from borrower, IFADor the supervisor, until re-design at mid-term and action thereafter.

Efficiency and management arrangements

5.20 IFAD’s economic analysis and planning hasnot kept pace with the changing nature ofinvestments. The DRR highlights concernswith the quality of economic analysis, and the need to use more creative and useful efficiency indicators as well as incorporatepractical design features to promote highreturns and cost-effectiveness. An examplefrom Pakistan clearly illustrates theseconcerns. The PCF-ADP project was uneco-nomic - when the previous AsDB funded R&I

139189 ‘Toward sharpening the focus on rural poverty: A reviewof World Bank experience’, by the Operations EvaluationDepartment, World Bank, 2002.

190 As empirically demonstrated by Wane, W. (2004), TheQuality of Foreign Aid: Country Selectivity or DonorsIncentives?, World Bank Policy Research Working Paper3325, June 2004

191 These were: ADP (Bangladesh), PDRSO (Burkina Faso),EDNASP (Egypt), PD-PEF (Guinea), PAPE-BGN (Guinea),NAFP (Mozambique) and PFC-ADP (Pakistan).

Box 12 Project design issues have impinged on effective implementation.

Design is an essential component of aid quality, and the quality of aid has a tremendous impact on its effectiveness.190 The desk review (paragraphs 3.58-3.88) highlights some key areas where IFAD project design isin need of improvement. These can be summarized as:

n 60 per cent unsatisfactory definition, clarity and practicality of objective and output indicators

n 57 per cent unsatisfactory explanations of systems and responsibilities for monitoring

n 45 per cent unsatisfactory assessment of significant risks, assumptions, mitigating measures

n 43 per cent unsatisfactory analysis of institutional configurations and trends

n 40 per cent unsatisfactory evidence that beneficiaries were directly involved in project design

The CWPs highlight particular instances where weak project design had affected implementation:

n RFSP (the United Republic of Tanzania) has a confusing and overly prescriptive design, and most of the 27 risks identified in the logframe should have been clarified and internalized during formulation and loan negotiations. If no mitigation of risk was possible at design since the project was seeking to pursue a new andlargely untested approach, it may have been preferable for this component to be explicitly described as a pilot initiative.

n PDRSO (Burkina Faso): the logic of the project was weak, and the notion that the community will identify itsown priorities and ways of working is in itself not a development aim. Also, there was a lack of environmentalimpact assessment (of road building) and a high risk of developing valley bottoms (with only weak mecha-nisms for reducing risks as they became apparent).

n BADP (Pakistan): the logic of the intervention deserved more attention, including the fact that 80 per cent ofincome comes from off-farm sources, the problems with public services reaching the poor, and, the projectappraisal documents’ ignorance of the consequences of far reaching devolution in Pakistan.

project is included in the analysis. The projectcould only be economically justified on a ‘sunkcosts’ argument, yet this was never addressedat appraisal. There was no economic rate ofreturn analysis at project completion (thoughAsDB say they will attempt one), and the finaleconomic rate of return would probably below, because: (i) investments took about threeyears longer with a proportionately moremodest reduction in total costs and with aconsequent lag in benefits; (ii) area covered bywatercourses was barely half of that planned;(iii) the impact study farm income incrementsat about 70 per cent were lower than the opti-mistic up to 200 per cent increments assumedin the appraisal report; and (iv) the impactstudy cropping intensity increment may be alittle higher than assumed at appraisal but notenough to outweigh the other negative shifts.

5.21 The long waits for re-orientation action atmid-term undermine effective projectmanagement in the interim and reduce efficiency. Only 45 per cent of the projectsample was rated as having a high orsubstantial level of efficiency, with halfshowing a modest performance. Theseresults are similar to the 2003 ARRI, whichrated 50 per cent as high/substantial.Economic planning is poor, and the effi-ciency of IFAD investments varies hugely.An estimate of cost per beneficiary shows arange from USD 23 to USD 717 per person– with half of the sample in the rangebetween USD 100 and USD 400 per person,a quarter below USD 100 and a quarterabove USD 400 (see Table 12).

Borrower performance5.22 Good examples of project management do exist,

and the use of open competition has been a signifi-cant factor. This is supported by the studies ofPeru, the United Republic of Tanzania andMozambique (PAMA) where the managementteams were hired using open competition.The selection of capable managers and projectstaff translates into improved decision-makingas well as on-the-ground performance. In Peruthe performance by the borrower is rated ashigh for both projects in the sample(MARENASS and CORREDOR). The projectmanagement is distinguished by its small size.The total staff complement for MARENASS isonly 22, all of whom are recruited following anational competition. Together they cover atarget area of 360 of the total 560 communi-ties. Each zonal office has one car, the M&Eagents ride public transport, a few communitypromoters have motorcycles and the rest ridebuses or walk. CORREDOR has a professionalstaff complement of 14, again recruitedthrough national competitions. Other projectmanagement units seen as performing wellinclude: the United Republic of Tanzania (RFSPand AMSDP); Armenia (NWASP and ASP);Mozambique (NAFP and PAMA); Burkina Faso(PAMER) and Bangladesh (SSWRDSP).

5.23 But, in many cases project management is less than satisfactory. In 8 out of 20 projects(40 per cent), the performance of theborrower is rated as modest. Plus five of theprojects in the sample are very new, and therehas not been much time for managementproblems to emerge. The desk review (para-graphs 3.83-3.85) also pointed to concernswith project management, namely the analysisof current capacity and commitment amongstintended implementers. There are two mainissues that have emerged from the countrystudies: (i) that major management problemsare allowed to occur in the first place; and (ii) where less severe management problemsdo exist they persist largely unchecked andfor a number of years.

5.24 Four projects (20 per cent) had major manage-ment problems. These were PD-PEF andPAPE-BGN (in Guinea) plus PRODESIB andPROSAT (in Bolivia). In Guinea, management

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

140


Table 12 Estimates of costs per beneficiary

Cost per beneficiary Frequency

Up to USD 100 5 25%

USD 101-USD 200 2 10%

USD 201-USD 300 5 25%

USD 301-USD 400 3 15%

Over USD 401 5 25%

Totals 20 100%

Source: PPMS data, IFAD.

costs for PD-PEF are enormous, with thenumber of project staff reaching 82 persons at one stage, plus the cost of the project unitrepresenting 26 per cent of the total projectcosts, and the cost of studies and consultationsanother 22 per cent (nearly 50 per cent intotal). For PAPE-BGN, the number of projectstaff in 2002 amounted to 96 technicians and32 fieldworkers, and yet the project unit acts asa subcontracting unit and not as an executingagency. Administrative and management costsare very high at around 58 per cent of the total cost. Both PRODESIB and PROSAT (in Bolivia) suffered from major institutionalproblems on both a large scale with govern-ment reorganization, and with personal rela-tionships and weak capacity. The problemswere partly aggravated by the national politicalsituation of Bolivia, and management hasimproved since reorientation.

5.25 A further five projects (25 per cent) show arange of problems that have been allowed to persist for at least a number of years.192

In PDRSO (Burkina Faso), the project wasfraught with delays creating a long periodbetween project design and implementation.Project management is seen as intenselybureaucratic with a heavy emphasis on finan-cial management and completion of activities.Management lines are also made overlycomplex by the partnership with the BSFresponsible for the health component. InEDNASP (Egypt), the project managementexperienced difficulties in the first two years,including with the delivery of the outcomes.The management team and project directoreventually had to be changed. For AqDP(Bangladesh), several reviews have identifiedweaknesses in the Department of Fisheriesand the Local Government EngineeringDepartment, regarding project accountingand project-specific engineering support.Performance under the first project coordi-nating director was poor, but has improvedunder the second and maintained under the third. In PFC-ADP (Pakistan), there werequite serious project management issuesduring the first three years, with sevenproject directors over the life of the project.The NGO contracted for assisting implemen-tation, turned out to have serious manage-

ment problems of its own and the contractwas terminated. Project management is nowmuch improved.

Supervision and IFAD support5.26 The current arrangements for supervision can

be made to work, but more often than notsupervision underperforms. This suggestsweaknesses in the operating modality.193 Nineof the twenty projects (45 per cent) are ratedas modest for supervision performance (anassessment of both CI and IFAD), and theremaining 55 per cent show concerns that arenot necessarily reflected in a single digitrating. The emerging issues are:

n A clear failure by some CIs in their supervisory role.

n A general tendency by CooperatingInstitutions to focus on fiduciary andadministrative issues at the expense ofdevelopment objectives.194

141

192 This includes PFC-ADP (Pakistan) which was rated as‘substantial’ overall, despite having significant problemsinitially.

193 The evalution of supervision modalities (2003) suggeststhat the IFI model in supervision currently used in IFADsupported projects may not be the most effective modalityto enhance project performance.

194 From a much larger sample, the evaluation of supervisionmodalities (2003) concluded that CIs have consistentlyperformed better on fiduciary aspects whereas implemen-tation support, particularly for IFAD’s specific requirements(and strategic imperatives), lagged behind.

In summary:

n Economic analysis and planning hasnot kept pace with the changing port-folio of investments.

n Institutional analysis during formula-tion is inadequate, with limited assess-ment of project management and insti-tutional capacity. There is a need toplan management arrangementsearlier, and learn from lessons wherecompetition has been used.

n IFAD does not intervene early enoughduring implementation to addressfailing situations. Too much becomesconcentrated in reorientation processesand the MTR.

n The central role of the CPM relationship,where an active CPM can overcome manyof the inadequacies of the supervisionarrangements.

n A lack of project interaction outside theclose CPM-project relationship, with littletechnical, policy, sectoral or cross-cuttingsupport from IFAD.

5.27 The 2003 ARRI found clear differences in theperformance of IFAD and the CI. IFAD’sperformance was rated as modest in two-thirds of the evaluations. Only three OE eval-uations judged IFAD’s performance to begood or very good. Weaknesses were mostlyin areas of design, implementation support,and the creation of effective partnerships forimplementation. In contrast, CI performancewas rated as ‘substantial’ in about two-thirdsof the evaluations. In seven of the tenprojects reviewed, the CI was either theWorld Bank or UNOPS. The findings broadlyconfirm the conclusion on supervision modal-ities: UNOPS and the World Bank generallyshowed a stronger supervision performancethan the regional, smaller CIs.

5.28 Sometimes the CI has clearly failed in itssupervisory role. In Peru, MARENASS wassupervised by the Andean DevelopmentCooperation (CAF). To date, CAF has sentonly a few missions and has not submitted aformal supervision report in four years. InBolivia, both PRODESIB and PROSAT havehad little support and coordination by CAF

(the supervisors) – which are seen as obstruc-tive, bureaucratic and over-controlling.

5.29 In a significant proportion of the samplehowever (40 per cent, or 8 projects), the CIhas performed adequately but failed to tacklecentral development issues. This is sometimesdue to lack of technical capacity, but moreoften than not, because the CI is overlyfocused on administrative and disbursementmatters (Box 13).

5.30 It is therefore important that concerns aboutsupervision are not simply equated with theperformance of the CI, and thus exonerateIFAD.195 CIs are generally performingadequately, and there is evidence that IFAD’ssupervision budget falls well below costsincurred by the World Bank on its ownprojects and this results in cost-cutting tactics.Also, supervision reporting of performancehas not been an active topic for analysis andExecutive Board discussion, and there are keyareas where IFAD needs to take greaterresponsibility, namely: (i) in the selection andmonitoring of supervisors; (ii) taking correc-tive action where CIs are failing in their role;(iii) be more responsive to supervisionmissions and reports;196 (iv) provide addi-tional technical or sectoral support in order toattain development objectives. The evaluationof supervision modalities (2003, recommenda-tion v) also found a need to improve supervi-sion quality assurance in IFAD to facilitatemonitoring of CI performance.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

142


Box 13 Making supervision work within the current system

In Armenia, supervision by both the CI and the CPM is viewed positively by project management and welcomed aspart of the process of improving performance. The proactive and pivotal role of the CPM is a strong contributoryfactor, including:

n An active input during identification and formulation to ensure good project design

n Asserting some influence in ensuring that a capable supervisor is selected

n Using the CI to its strengths, including checking loan conditions, disbursements, and providing fiduciaryexpertise

n Visiting projects at least once a year, and up to three times during the first critical stages

n Developing a good relationship with project management, including frequent communication (such asquick responses to emails/faxes)

n Active participation on MTR missions, including field visits and report writing

5.31 The CPM-project relationship is critical toIFAD’s supervisory role, and while most CPMsare viewed positively, the project usually haslittle interaction with other headquarters staff.In Armenia for instance, the communicationand competence of the CPM(s) and CIs forboth NWASP and ASP are viewed positively bythe project coordinating unit. Beyond thisclose circle however, there are few other IFADstaff actively involved. Similarly in BurkinaFaso, project staff are generally satisfied withthe level of CPM support. There is however alow level of engagement between project staffand IFAD staff at headquarters (such as thoseengaged in policy or technical matters). Inother instances the CPM (and thus IFAD) canappear distant: In Pakistan, supervision byIFAD is seen as fairly remote across bothprojects, with the borrower expressing a pref-erence for closer interaction with IFAD.Likewise in Guinea, IFAD is considered to be adistant and almost absent partner for bothprojects, with its responsibilities delegated to UNOPS.

5.32 One of the responses to IFAD’s ‘arms-length’model of supervision to emerge out of IFADVI consultations is the possibility of extendingIFAD’s field presence. The IEE does notattempt to substitute the on-going study offield presence, but instead it draws togethermuch broader evidence to consider the impli-cations of field presence. Firstly, a rapid anduniversal decentralization of CPMs will notnecessarily address underlying structuralweaknesses in the headquarters-field relation-ship. Frequent contact and support is often

143

195 The 2003 ARRI (paragraphs 91-99), makes a clear distinction between ‘modest’ IFAD performance and‘substantial’ CI performance.

196 In PD-PEF (Guinea) for example, the supervision reports contain repeated comments concerning badmanagement, yet this does not seem to have been sufficient to sound the alarm.

Box 14 Supervision of administrative and financial matters at the expense of development objectives

n AqDP (Bangladesh): Supervision was undertaken by UNOPS which have been reportedly good at super-vising the accounts and auditing the project, but lack the capacity on technical and social monitoring andsupervision – which is seen as essential given the limited social analysis and development capacity of thelead government agency.

n PDSRO (Burkina Faso): Supervision of PDSRO is deemed satisfactory, although the failure of supervision tolead to a redesign in the face of overwhelming evidence is seen as a serious weakness.

n AMSDP (United Republic of Tanzania): The modest rating for AMSDP stems from the finding that althoughproject management argues a sound case for modification in details of design, they feel that the CI is overlywedded to implementation in accordance with the original design and the loan agreement.

n MARENASS (Peru): Project staff and consultants active in the project have been critical of CAF performance,alleging that missions have been overly concentrated on administrative and financial affairs (to the detri-ment of important programming issues).

n PAMER (Burkina Faso): Supervision is well appreciated by project staff, though seen as inadequate by othersdue to the almost exclusive focus on administrative and financial matters.

n EDNASP (Egypt): The World Bank ably undertakes supervision of EDNASP within what they were asked toundertake, but are weaker in aspects to improve the project in development terms.

n PFC-ADP (Pakistan): The supervision achieved a satisfactory two missions per year, yet technical skills are verylimited. The borrower also has concerns about CI supervision, mainly the number of decisions imposed.

n RFSP (United Republic of Tanzania): The RFSP is less concerned to make changes in the design: their concernis principally with the time pressure to achieve FLM targets in the absence of a planned start-up year.Interviews with management show that they see the performance of the CI as adequate or better, thoughthe CI seems to be over-policing appraisal report targets.

most critical,197 and while the CPM-projectrelationship is central to achieving this,project management staff are generally poorlyconnected to IFAD headquarters. There arefor instance, few sectoral or crosscutting chal-lenges to the work of the CPM. This ‘freeagent’ CPM culture can produce good qualityimplementation without field presence, inother cases it fails to perform. The quality ofpoor performing CPMs must be addressed.

5.33 Secondly, the case for field presence needsseems most pertinent where there areadvanced donor-government coordinationmechanisms (in which IFAD should engage),or where projects are failing. The liaisonofficer initiative has been helpful in somerespects, but for some stakeholders this hasbeen lessened by the absence of any realauthority within IFAD. The initiative hasbegun to close the communication gap, butlocally hired liaison officers lack any authorityover identification, formulation or implemen-tation. They are under-resourced and have noauthority to speak for IFAD or action imple-mentation support in the way the decentral-ized CPM in Latin America has achieved inPeru (though with a lack of influence inBolivia). In countries where donor program-ming is increasingly driven by official govern-ment-donor coordination mechanisms, theliaison officer is in a particularly invidiousposition. With the crucial identification decisions made through traditional Rome-to-Minister channels, formulation teams

assembled and tasked from Rome, liaisonofficers are essentially observers. Similarly, if aproject is in difficulty, their role is solely toemail the supervisor or the CPM.

6. Concluding remarks6.1 Whilst the IEE finds examples of high

performing projects, the overall impact isfound to be modest. Improved crop produc-tion for consumption is a significant impact,with corresponding increases in householdincome. The impact of financial services andinvestments in road and irrigation infrastruc-ture has been modest, though with somecritical exceptions. Despite representing aminor share of IFAD investment, health andeducation components have had a substantialimpact (especially in terms of potable watersupply). The impact on social capital andinstitutional development has been well belowexpectations, and sustainability remains a keyconcern. In terms of targeting the poorest,performance has been modest, in part due to problems of poor implementation andbenefit capture.

6.2 IFAD’s lending programme lies at the core ofits business. Yet, despite some stellar projects,there are significant weaknesses at all stages ofthe project cycle. Key issues include:n The lack of serious challenge during the

early stages of project identification, especially given the permissiveness of the strategy and policy context in IFAD.

n Early intervention (plus resources) duringthe critical stages of project start-up.

n Changes to the supervision model toensure close and regular supportthroughout the project cycle.

n Institutional analysis and economic analysisand planning in line with the current port-folio of investments.

n Design, monitoring and improved mechanisms to prevent benefit captureduring implementation.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

144


In summary:

n IFAD needs to better select, monitorand intervene when supervision fallsshort of expectations.

n There is a real gap in the technical andsectoral capacity of supervising CIs.IFAD needs to address this, if develop-ment effectiveness and impact is to beimproved.

n Field presence is unlikely to be apanacea. There are serious structuralweaknesses in how the field relates toheadquarters.

197 In relation to field presence, the evaluation of supervisionmodalities (2003, paragraph 140) also highlighted theimportance of frequent contact and access to local leveladvice/support in the supervision process.

145

Tab

le13

The

cou

ntr

yvi

sit

pro

ject

s

Stag

eat

Proj

ect

Fina

ncin

gIF

AD

Fina

ncin

gBo

ard

Loan

Curr

ent

Regi

onCo

untr

yID

Nam

eA

cron

ymm

id-2

004

Type

Term

sTy

pe(U

SD'0

00)

App

rova

lSi

gnin

gEf

fect

iven

ess

Clos

ing

CI

PNA

rmen

ia10

38N

orth

-wes

tA

gric

.Ser

vice

sN

WA

SPC

IRD

HC

E12

,961

4.12

.97

5.12

.97

14.4

.98

31.1

2.02

IFA

D

PNA

rmen

ia11

77A

gric

ultu

ralS

ervi

ces

ASP

MIR

DH

CE

15,5

1326

.4.0

130

.4.0

126

.9.0

131

.3.0

6U

NO

PS

PIBa

ngla

desh

480

Smal

l-sca

leW

ater

Reso

urce

sSS

WRD

SPC

Irrig

.,wat

eran

dIR

DH

CC

10,4

006.

12.9

515

.12.

9510

.6.9

631

.12.

02A

sDB

PIBa

ngla

desh

1074

Aqu

acul

ture

Dev

elop

men

tA

DP

LA

quac

ultu

reH

CF

19,9

8823

.4.9

820

.5.9

88.

12.9

831

.12.

05U

NO

PS

PLBo

livia

373

Beni

Indi

geno

usPe

ople

PRO

DES

IBM

Land

Tenu

re&

IRD

HC

6.12

.94

20.2

.95

30.1

0.96

31.1

2.04

CA

F

PLBo

livia

1031

Smal

lFar

mer

sTe

ch.A

ssis

tanc

ePR

OSA

TM

TA&

Exte

nsio

nH

CC

8,09

929

.4.9

75.

8.97

30.4

.98

30.6

.05

CA

F

PABu

rkin

aFa

so51

2So

uth

Wes

tRu

ralD

evel

opm

ent

PDRS

OL

IRD

HC

F14

,816

11.9

.96

22.1

.97

12.1

.98

30.0

6.05

BOA

D

PABu

rkin

aFa

so11

03Ru

ralM

icro

ente

rpris

ePA

MER

MSM

Ede

v't

HC

E9,

376

28.4

.99

18.5

.99

14.7

.00

31.3

.08

BOA

D

PNEg

ypt

1014

East

Del

taA

gric

.Ser

vice

sED

NA

SPL

IRD

HC

F25

,000

5.12

.96

30.3

.98

25.1

.99

30.9

.05

IDA

PNEg

ypt

1204

Wes

tN

ouba

riaW

NRD

PE

IRD

IF

18,4

8523

.4.0

229

.5.0

29.

4.03

30.1

2.10

UN

OPS

PAG

uine

a31

3Sm

allh

olde

rD

ev.F

ores

tPD

-PEF

CIR

DH

CE

13,8

202.

12.9

222

.1.9

323

.2.9

430

.6.0

2U

NO

PS

PAG

uine

a47

8N

orth

Low

erG

uine

aPr

ojec

tPD

-PA

PEC

IRD

HC

F15

,200

14.9

.95

18.1

0.95

1.7.

9631

.12.

04U

NO

PS

PFM

ozam

biqu

e33

4A

rtis

anal

Fish

erie

sN

AFP

CSe

aFi

sher

ies

&IR

DH

CF

6,02

515

.9.9

326

.1.9

44.

11.9

431

.12.

02U

NO

PS

PFM

ozam

biqu

e11

09PA

MA

Supp

ort

Proj

ect

PAM

AE

Agr

.Mar

ketin

gH

CE

22,7

838.

12.9

96.

12.0

07.

9.01

31.3

.08

UN

OPS

PIPa

kist

an35

3Pa

tFe

eder

Com

man

dA

rea

PFC

-AD

PC

Irrig

.&IR

DI

C28

,550

19.4

.94

28.9

.94

2.2.

9531

.12.

03A

sDB

PIPa

kist

an11

82N

WFP

Bara

niBA

DP

EIrr

ig.&

IRD

HC

C14

,448

26.4

.01

16.8

.01

9.5.

0331

.12.

09A

sDB

PLPe

ru47

5M

anag

emen

tof

Nat

ural

Reso

urce

MA

REN

ASS

LTA

&Ex

tens

ion

IF

12,2

7814

.9.9

521

.6.9

69.

4.97

30.6

.04

CA

F

PLPe

ru10

44Pu

noC

usco

Cor

ridor

Dev

.C

ORR

EDO

RM

SME

dev'

tO

E18

,923

4.12

.97

7.12

.99

17.1

0.00

30.6

.07

IFA

D

PFTa

nzan

ia11

51Ru

ralF

inan

cial

Serv

ices

RFSP

ERF

Sde

v't

HC

F16

,342

7.12

.00

31.1

.01

12.1

0.01

30.6

.11

UN

OPS

PFTa

nzan

ia11

66Ru

ralM

arke

ting

Serv

ices

AM

SDP

EA

gr.M

arke

ting

HC

F16

,345

6.12

.01

15.1

.02

4.10

.02

30.6

.10

UN

OPS

Stag

e:E

=Ea

rly;

M=

Mid

dle;

L=

Late

;C

=C

lose

dPr

ojec

tTy

pe:

IRD

=In

tegr

ated

Rura

lDev

elop

men

t;TA

=Te

chni

calA

ssis

tanc

e;SM

E=

Smal

l&M

ediu

mEn

terp

rise;

RFS

=Ru

ralF

inan

cial

Serv

ices

Term

s:H

C=

Hig

hly

Con

cess

iona

l;I=

Inte

rmed

iary

;O

=O

rdin

ary

Fina

ncin

gTy

pe:

C=

Initi

ated

byan

othe

rin

stitu

tion

and

co-f

inan

ced

byIF

AD

;E

=IF

AD

initi

ated

and

excl

usiv

ely

finan

ced

byIF

AD

;F

=IF

AD

initi

ated

and

co-f

inan

ced

byan

exte

rnal

part

ner

1. Introduction1. During its deliberations, the Consultation on

the Sixth Replenishment of IFAD’s Resourcesendorsed a proposal to have an independentexternal evaluation (IEE) of IFAD.Accordingly, the Governing Council decidedthat the evaluation should be planned andbegun in 2003 and completed in 2004, intime to allow for full consideration of the IEEreport by the Executive Board, prior to itsrecommendation to the Governing Council on the Consultation on the SeventhReplenishment of IFAD’s Resources. On 9-10 April 2003, the Executive Board consid-ered a report by the chairperson of theEvaluation Committee; endorsed its recom-mendations on the governance and organiza-tion of the IEE and entrusted the Director ofthe Office of Evaluation (OE) with the prepa-ration of detailed terms of reference (TOR)for submission to the steering committee forits review and endorsement198.

2. This document contains the terms of refer-ence for the IEE, prepared in accordancewith guidance provided by the ExecutiveBoard. It is intended to facilitate the selec-tion of a service provider199 through an openand international competitive bidding, inconformity with IFAD’s rules and regula-tions, and to guide the conduct of the IEE.Bidders will have flexibility in proposingtechnical approaches and allocation of

consultant inputs within the parameters ofthe TOR.

2. Objectives of the Evaluation3. The main objective of the IEE is to deter-

mine IFAD’s200 contribution to rural poverty reduction, the results and impact it hasachieved in this area, and the relevance of the organization’s mission and objectives inrelation to international development goalsand the national development strategies ofIFAD borrowing countries. The evaluationis further expected to assess whether andwhat IFAD has learned from past experi-ence and how the Fund’s policies and operations have evolved in response tolessons learned from that experience, andfinally, to offer recommendations on thepolicy directions IFAD should pursue andother steps it should take to improve itsfuture performance.

4. The IEE is required to relate its findings and recommendations credibly to reliableevidence, in accordance with good develop-ment evaluation practice and sound profes-sional methods and criteria. The IEE mustbe independent and external, and recog-nized as such by its client, the ExecutiveBoard, as well as by the international devel-opment community and the general public.It should meet the high-quality standardsrequired to be able to contribute inputs to

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

146

Terms of reference of the IEE

Annex 5

147

the deliberations of the Consultation on theSeventh Replenishment of IFAD’s Resources.

3. Scope and focus5. Given the time and resources available, the

IEE is expected to focus on the assessment ofthe sustainable impact and effectiveness ofIFAD’s development initiatives, includingIFAD-supported projects, policy dialogue,advocacy work, corporate policies and strate-gies. This would include the managementprocesses through which IFAD formulatesdevelopment policies and strategies and itsmanagement of the project cycle in coopera-tion with other partners.

6. The IEE service provider may also extend theevaluation to other aspects of the workings ofIFAD that it considers important and forwhich it has the time and expertise to dealwith beyond the above-mentioned focus of theIEE. The service provider may recommendappropriate follow-up by IFAD if, for anysubject outside its focus, a thorough andcredible evaluation cannot be performedwithin the time and resources available to the service provider.

7. The focus of the evaluation will relate to theimpact, effectiveness and national develop-ment relevance of IFAD-supported projects,programmes, strategies and policies that havebeen initiated or contributed to by IFAD atleast over the last ten years. These include:

(a) completed and ongoing loan projects;

(b) non-lending activities such as technicalassistance grants (TAGs)201, and policydialogue and advocacy work that are notdirectly related to lending operations;

(c) the country strategic opportunities papers(COSOPs) that guide IFAD’s cooperationwith its developing country partners;

(d) the regional strategies202 that guide theoperations of the five regional divisions of IFAD;

(e) the corporate policies, the overall StrategicFramework for IFAD 2002-2006203 and itspredecessor for 1998-2000, and the guidelines and provisions laid down by the various consultations on the replenish-ment of IFAD’s resources;204 and

(f) the contribution to policy dialogue thatIFAD has made over the years through its participation in regional, interna-tional and global development forumsand conferences.

8. In selecting specific countries as well aslending and non-lending operations fordetailed scrutiny, the service provider willensure that the samples used are representa-tive of IFAD operations and free from anybiases that could undermine the independ-ence, impartiality and credibility of the evalua-tion. It is expected that the service providerwill employ stratified, multistage randomsampling methods for this purpose.205

9. The processes employed by IFAD to formulatethe IFAD Strategic Framework, regionalstrategies, COSOPs and corporate policies arealso part of the IEE. So, too, is managementof the project cycle,206 from project identifica-tion (inception) and formulation to approval,implementation and self-evaluation, including

198 For further information, please see the Report of theChairman of the Evaluation Committee on theIndependent External Evaluation of IFAD, available athttp://www.ifad.org/evaluation/iee/chairman.pdf.

199 The term ‘service provider’ indicates a private or publicentity or firm, whereas the term ‘consultant’ indicates anindividual service provider.

200 For an overview of IFAD, its mandate and activities pleaserefer to the IFAD Basic Documents(http://www.ifad.org/pub/basic/index.htm), and to IFAD’sAnnual Report (http://www.ifad.org/pub/ar.htm).

201 For further information please refer to the IFAD Grantssection on the IFAD corporate website underhttp://www.ifad.org/operations/grants/index.htm.

202 Formal regional strategy documents are available from2002. Regional strategies are also discussed annually bythe Executive Board during review of the annual workprogramme and administrative budget. All regionalstrategy documents are available on the IFAD website athttp://www.ifad.org/pub/index.htm for viewing and downloading.

203 The Strategic Framework for IFAD(2002-2006) can beviewed at http://www.ifad.org/pub/index.htm.

204 Such as the IFAD V: Plan of Action (2000-2002). This document is available at http://www.ifad.org/operations/action/index.htm.

205 Annex I lists some of the factors considered important forstratified, multistage random sampling.

206 Annex II gives an illustrative list of processes that maywarrant attention during the IEE.

the process through which IFAD assuresquality and flexibility in project design andimplementation, as well as the selectionprocess and roles of IFAD consultants andcooperating institutions,207 which are ofparticular importance during the projectcycle. The IEE will also pay attention to theprocess through which IFAD ensures theadoption and implementation of recommen-dations generated by its self-evaluation 208

function and by OE.

10. Finally, it is expected that the IEE will takeinto account the national and global contextwithin which IFAD operates and, in particular,the consensus of the development community,the specific potentials and constraints of itsborrowing Member States and the evolvingrole of IFAD’s development partners. This iscritical to ensuring that the relevance ofIFAD’s operations is assessed in terms of: (i) its unique mandate and comparativeadvantage; (ii) the priorities endorsed by theinternational development community (suchas the Millennium Development Goals –MDGs); and (iii) the objectives embedded inpoverty-reduction strategies owned by itsborrowing Member States.

4. Major evaluation questions11. The IEE will address two fundamental ques-

tions: Is IFAD properly focused on its ruraldevelopment mission? Are the skills andresources of IFAD used in the best possibleway, given IFAD’s overarching goal ofsupporting rural development and helpingdeveloping countries eradicate rural poverty?To answer these questions, it will use a varietyof evaluation instruments and techniquesadapted to IFAD’s mandate and in conformitywith good development evaluation practicesand criteria. The following concerns andquestions should be taken as the point ofdeparture for the IEE:(a) What is the development relevance of

IFAD’s policies and programmes? Whatvalue does IFAD add to the internationaldevelopment community, particularly inrelation to rural poverty reduction,improved food security, relevant nationalsector policies, national development andpoverty reduction strategies, such as the

PRSPs, and the international develop-ment goals, e.g. the MDGs?

(b) What has been the sustainable–quantita-tive and qualitative—impact of IFAD-supported projects on the ground? (Asmuch as possible, the IEE is expected toquantify impact and, to that end, empha-size methodologies for quantitative datacollection and analysis of impact.) And towhat extent has IFAD contributed torural poverty reduction? How successfulhas IFAD been in improving agriculturaland rural development policies andstronger institutional capacities in partner countries?

(c) How effectively has IFAD promoted inno-vative approaches in relation to policy,partnerships, project implementation,technology and other aspects of IFAD-assisted operations that are meant toimpact poverty? How has IFAD made useof local knowledge and technology inpromoting innovative approaches? Howhave IFAD’s innovative approaches beenreplicated and scaled up?

(d) How effective has IFAD been inpursuing its objectives? How and to whatextent are IFAD policies adapted to theachievement of these objectives, and how clear, explicit and measurable areIFAD’s objectives?

(e) How efficient has IFAD been in the use oflending and other budget resources andthe deployment of skills? And how selec-tive has it been in the allocation of itsresources and the choice of institutionsand partners? To what extent has IFADpromoted the ownership and partnershipof relevant host country institutions,including those representing the poor?

(f) What explains IFAD’s performance interms of evaluation criteria and questionssuch as those mentioned above?

(g) How can IFAD enhance the impact andsustainability of its development coopera-tion, and contribute more to poverty-reduction efforts? What are the mainrecommendations that IFAD shouldconsider adopting in the near and long term?

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

148

A N N E X 5 : T E R M S O F R E F E R E N C E O F T H E I E E

12. The service provider is expected to opera-tionalize and fine-tune the questions outlinedabove. Within that scope, it is also encouragedto add sub-questions and issues that helpaddress the objectives of the IEE. The outputsof the service provider, however, must be fully compliant with the letter and spirit ofthis TOR.

5. Main tasks and methodology13. The role of OE in the IEE has been defined

by the Executive Board and is outlined belowin section 8 on Governance and Organization.OE, which previously reported directly to thePresident of IFAD, now reports directly to theExecutive Board by virtue of a decision madeby the Board at its Seventy-Eighth Session inApril 2003 and is now truly independent ofIFAD management.209 OE will facilitate thework of the service provider during the evalu-ation in a number of ways, including thefollowing:

(a) At the inception stage, and as often asnecessary, OE210 will brief the serviceprovider on the operations, governingbodies and organizational structure ofIFAD, relevant documents and datasources, and OE’s work programme ofindependent evaluations for 2003-04.

(b) OE will also provide written comments tothe service provider on all its deliverablesin order to facilitate and enhance thecompliance of the evaluation with theagreed TOR and methodology. Thesecomments will address any deviation from the requirements of the TOR, inaddition to methodological issues and anyfactual issues or inaccuracies concerningIFAD that OE may consider relevant tothe IEE. However, OE will neithersupport nor contest IEE findings andrecommendations.

(c) A number of the evaluation reportscontained in OE’s 2003-04 workprogramme are being prepared inde-pendently of IFAD management as aconsequence of the April 2003 Boarddecision211 that OE should report directlyto the Executive Board. The serviceprovider may wish to consider these OEreports as independent evaluations that

augment the field-based independent vali-dation proposed below and add value tothe IEE in general.

(d) Before commencement of the field-basedindependent validation, OE will identify,in consultation with the relevant IFADregional divisions, a national evaluationcounterpart for the service provider ineach selected country.212

14. The IEE is conceived as a process consistingof a number of stages and tasks. The first task(Task 1) of the service provider is to interactwith OE in developing the detailed scope andfinal work plan of the IEE. In specific terms,the service provider will:

(a) take into account the perspectives of theclient, in this case the Executive Board ofIFAD, by reviewing Board minutes andother documents;

(b) develop a common understanding of theTOR among the consultants and the OEDirector and his two senior independentadvisers, and fine-tune and operationalizethe scope, focus, main questions, method-ology and tasks within the TOR frame-work endorsed by the steering committee;

149

207 For further information regarding cooperating institutionsand their role in IFAD operations, please refer to theReport of the Joint Review on Supervision Issues for IFADfinanced Projects at http://www.ifad.org/evaluation/iee/gc-20-l-10.pdf.

208 Please refer to the Progress Report on the Project Portfolio(see http://www.ifad.org/gbdocs/eb/78/e/EB-2003-78-R-16.pdf), and to the stocktaking of monitoring and evalua-tion (M&E) undertaken by OE (M&E systems at projectlevel, available at http://www.ifad.org/gbdocs/eb/ec/e/25/EC-2000-25-W-P-3-REV-1.pdf).

209 At the same session, the Executive Board also approvedthe IFAD Evaluation Policy, which assigns the independentevaluation function within IFAD to OE and codifies theprinciples, policies and main procedures for independentevaluation. The evaluation policy is available athttp://www.ifad.org/gbdocs/eb/78/e/EB-2003-78-R-17-REV-1.pdf

210 In addition to IFAD management and staff as required.211 For more information on OE's work programme for 2003-

04 please refer to OE’s Work Programme and Budget for2003 which can be seen at http://www.ifad.org/gbdocs/eb/ec/e/33/EC-2003-33-WP-3.pdf

212 The national evaluation counterpart is a facilitator selected for the purposes outlined in paragraph 29. Asopposed to the national evaluation teams, the nationalevaluation counterpart is not a member of the serviceprovider’s team.

(c) determine those aspects of the work ofIFAD that the service provider considersimportant but which are not part of thefocus of the evaluation. Within theseaspects of IFAD’s work, the serviceprovider will distinguish between thosethat can be addressed by the IEE, givenits expertise and the time available, andthose that cannot, and for which theservice provider will propose appropriatefollow-up by IFAD in accordance withparagraph 6 of the TOR;

(d) select 20-25 countries through stratified,multistage random sampling (as suggestedin Annex I) for detailed scrutiny, half ofwhich would be selected randomly forconducting independent validationthrough original field research. In thesame way, select 40-50 loan projects,which came into effect between 1994-2002, and 4-6 TAGs in the above-mentioned selected countries;

(e) prepare a list of documents and otherinformation for use in the IEE, withinputs from OE and other IFAD units,including all corporate-level and thematicevaluations and relevant project andcountry programme evaluations preparedby OE;

(f) fine-tune the timetable of the IEE and the consultants’ deployment schedule,including the total amount ofperson/months to be spent in the field;

(g) address any other issues that need to beresolved during the inception phase,including the maximum length of thefinal report; and

(h) summarize the outcome of all thepreceding sub-tasks in one inception report

15. Task 2 is to conduct a desk review, as follows:(a) The desk review will include the country

programmes, loan projects and TAGsidentified in Task 1 and other aspects ofIFAD’s work identified under the scopeand main questions in this TOR. Half thecountry programmes, projects and TAGsselected for desk review will be selectedrandomly for independent field-basedvalidation by the service provider. Before

commencing field work, however, theservice provider will conduct an evalua-bility assessment of the sample and deter-mine whether the data and informationrequired for impact analysis are available.Projects and country programmes that arenot found to be evaluable to a reasonablestandard of evaluability, or lack data,information or effective monitoring andevaluation required for impact assessmentwill be listed and reported as such. Theservice provider will devise an appro-priate approach to evaluability and deter-mine whether or not to replace, throughan additional round of sampling, theprojects and country programmes that arefound to be non-evaluable.

(b) The desk review will consist of two parts.In the first part, the service provider willprepare a preliminary evaluation reporton the IFAD strategic frameworks, replen-ishment consultations, regional strategiesand corporate policies. This will focus ona desk review of the IFAD strategic frame-works, the guidelines and provisions laiddown by the replenishment consultationsthat took place in the period 1994-2003,the regional strategies, and the corporatepolicies that IFAD has developed since1994. The review will include an evalua-tion matrix213 for the strategic frame-works, the replenishment consultationsand the corporate policies along the linesof the matrix developed for the IFAD V:Plan of Action (2000-2002).214 The reviewwill be based on IFAD documents,including OE reports, as well as interviewsand meetings with IFAD managementand staff. Meetings with the Food andAgriculture Organization of the UnitedNations (FAO), the World FoodProgramme (WFP) and a sample of therepresentatives of Member States may alsobe included in this task.

(c) The second part of the desk review willfocus on documents pertaining toCOSOPs and country programmes forselected countries and the selected loanprojects, and to non-lending activities(e.g. TAGs), including their contributionsto advocacy and policy dialogue. This

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

150


will include a review of all evaluationreports prepared by OE on the projectsand non-lending activities selected forinclusion in the IEE. Interviews withrelevant IFAD staff and management willalso be part of this task. Based on thisdesk review, the service provider willprepare a preliminary evaluation reporton IFAD projects, TAGs and other non-lending activities. This will includethe selection of 10-12 countries, 20-25 loan projects and 2-3 TAGs forindependent validation.

16. Task 3 is to obtain independent validationthrough original research in the sample ofcountries, projects, and TAGs selected duringthe completion of Task 1 (this validation willbe augmented by the independent evaluationreports prepared by OE, see paragraph 13[c]). Task 3 will be undertaken in order tovalidate and enrich both parts of the deskreview mentioned above and to identify newfindings that may confirm or refute theconclusions of the desk review. This task alsoincludes consultations and interviews withgovernment representatives, civil society, theprivate sector, development agencies, existingin-country co-ordination groups and therural poor and their organizations. For theindependent validation in the field, theservice provider will organize national evalu-ation teams, as required in section 7.

17. Task 4 is to prepare the draft final report,including the incorporation of commentsreceived on the completion of Task 3.

18. Task 5 is to prepare the final report, reflectingcomments received on the draft final report.A presentation of the final report to the IFADgoverning bodies is also part of this task.

19. Task 6 consists of the submission of bimonthlyprogress reports on the deployment ofconsultants, the completion status of theabove-mentioned tasks, steps taken to solveany management problems that may havecaused deviations from the plan, and thefinancial data required under the serviceprovider’s contract. This task includes submis-sion of progress reports to the Executive

Board according to the timetable given inAnnex III.

20. It is understood that different aspects ofIFAD’s development work (for example,projects, policies and advocacy work) mayrequire different evaluation methodologies.Accordingly, service providers are free toaddress methodological issues in a differenti-ated manner in the technical approach theypropose for desk reviews and field work.Their proposals are expected to be consistentwith good evaluation practice and use a broadrange of techniques such as triangulation,literature review, desk studies, interviews,surveys, focus groups, field investigations, andcriteria such as, for example, those describedby OECD/DAC and OE’s methodologicalframework. The OE evaluation criteria are:impact215 (six domains), sustainability, innova-tion and replicability/scaling up, performanceof the project/policy (relevance, effectivenessand efficiency), and performance of imple-mentation partners (IFAD’s management andoperations, borrowing countries and imple-mentation authorities, cooperating institu-tions, etc). The IEE is expected to determineIFAD’s contribution to rural poverty reduc-tion in quantitative and qualitative terms andto differentiate by gender and other targetgroups the results and impact achieved.

21. In the technical proposals, service providerswill propose how best to undertake the above-mentioned tasks within the available time andresources. They will also provide details of themethodology they propose to use. Themethodology should be participatory and

151

213 Containing objectives, measures, performance criteriaand achievements.

214 The IFAD V: Plan of Action (2000-2002) can be viewed athttp://www.ifad.org/operations/action/index.htm

215 For further information, please refer to OE’sMethodological Framework for Project Evaluation underhttp://www.ifad.org/evaluation/iee/framework.pdf.

should use triangulation techniques and OE’smethodological framework for evaluation asmuch as possible. The evaluation processshould be as independent, objective andimpartial as possible.216

6. Deliverables and milestones22. The deliverables of the IEE process corre-

spond to the main tasks described above.These deliverables will be produced accordingto the following outline plan, which will befinalized at the conclusion of Task 1:

(a) Deliverable 1 is an inception reportcorresponding to Task 1, required withintwo months of the award of the contract.The Director of OE will approve this report.

(b) Deliverable 2 is a desk review asdescribed in Task 2, required within twomonths of the approval of the inceptionreport. The desk review will consist of two parts:

n a review of IFAD strategies, regionalstrategies and corporate policies; and

n a review of IFAD COSOPs, countryprogrammes, projects and non-lending activities.217

(c) Deliverable 3, based on Tasks 1-3, is acomplete report on the evaluation ofIFAD country programmes, projects andnon-lending activities, prepared after thecompletion of independent validationthrough original research in a sample ofthe borrowing Member States. Thisreport will also take into considerationOE’s independent evaluation reportsreferred to in paragraph 13(c). It will besubmitted within six months of thesubmission of Deliverable 2 (that is, eightmonths after the approval of the incep-tion report).

(d) Deliverable 4 is the draft final report, tobe submitted no later than nine monthsafter the approval of the inception report.

(e) Deliverable 5 is the final report, required within 12 months of the awardof the contract.

(f) Deliverable 6 consists of various progressreports required under this TOR, asdescribed in Task 6.

23. Each report will be as concise as possible andsubmitted in English. The language should bereader friendly and direct. It will avoideuphemisms when describing problems andshortcomings, but convey respect for andempathy with the people evaluated by theIEE. The consultants will provide 15 copies ofeach report to OE.

24. The length of the final report, which will betranslated into the other three officiallanguages by the Secretariat of IFAD, and thelength of the executive summary will be deter-mined during the inception phase. A clearrationale for annexes or appendices wouldalso be agreed at that time. The report shouldbe organized in such a way that it becomesclear that it provides answers for the questionsand issues raised in the TOR. A convincingaccount of the reasons why certain issuescould not be addressed should be included.The final report will be distributed widelywithin and outside IFAD to all partners andstakeholders and posted on IFAD’s website.

25. The service provider will submit all the above-mentioned deliverables to the Director of OE.OE will share these deliverables, as describedbelow, with the Executive Board, the steeringcommittee and IFAD management, which willprovide comments as follows:

(a) The steering committee may comment onany aspect of all the deliverables thathelps to ensure that the Evaluation Teamconducts its work in accordance with its TOR.

(b) On Deliverables 1-3, IFAD managementwill provide comments that relate exclu-sively to factual matters.

(c) IFAD management will provide commentson factual matters as well as matters ofjudgment upon receipt of Deliverable 4(the draft final IEE report). Upon receiptof Deliverable 5 (the final report), it will,in addition, provide a managementresponse to the Executive Board thatcontains the management reaction to theIEE and sets forth its views on the feasi-bility or otherwise of the IEE recommen-dations. The management response willexplain why certain recommendations, if

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

152


any, are considered non-feasible and howmanagement would propose to imple-ment the recommendations that it findsacceptable. The management responsewill be included as an appendix to thefinal IEE report.

(d) The Executive Board will receive, discussand comment on the draft final IEEreport (Deliverable 4).

26. In line with the IFAD Evaluation Policy, theservice provider will decide which of thecomments it receives should be incorporatedin the final report. The policy sets forth thegeneral rule, which will also apply to the IEE, that:

(a) The final report must incorporatecomments that point out factual errors or inaccuracies.

(b) Judgements that differ from those of theEvaluation Team may be incorporated bymeans of a note in the report.

7. Team composition and profile27. IFAD expects that the Evaluation Team

fielded by the service provider will consist ofthe following key personnel: a team leader(for 12 months) and two other senior experts(for a total of 17 person-months). The biddersare expected to propose additional personnelon the basis of their approach to the TOR,and to provide individual task responsibilitiesand TOR as part of their technical proposal.Funds have been allocated for a total of 14 person-months for these additionalpersonnel. Thus the service provider’s coreteam will consist of a team leader, two othersenior experts and other personnel (for a total of 43 person-months), all of whom willbe evaluated at the proposal stage.

28. The team that IFAD expects to recruit for theIEE will “meet the high-quality standardsrequired to be able to contribute inputs to thedeliberations of the Consultation on theSeventh Replenishment of IFAD’s Resources,”an expectation that is stated first in paragraph4 of these TOR. The selection of the team willbe based on careful consideration of severalfactors, some of which are considered impor-tant for the core team, some for the Team

Leader and others for the team as a whole.These factors will be elaborated, qualitativelyand quantitatively, in the Request for Proposalthat will be sent to short-listed organizationsand are outlined below:

(a) Key personnel (team leader and twosenior experts). Each of the keypersonnel will have at least 20 years experience of diverse work in the field ofdevelopment, with a considerable part oftheir work in developing countries. Eachof them will have demonstrable leader-ship ability, internationally recognizedachievements, and superior communica-tion ability and conceptual and empiricalanalytical skills. Their evaluation experi-ence will include complex institutionalevaluations, agricultural and pro-poorrural development programmes (particu-larly those supported by internationalfinancial institutions), and the assessmentof global, regional and national develop-ment strategies and issues and develop-ment-oriented corporate policies at thecorporate, country and regional levels.Other experience will include working onpoverty issues, the design and implemen-tation of participatory approaches (partic-ularly social mobilization), experience inthe five IFAD regions, and working withinternational financial institutions, UnitedNations development agencies and civilsociety organizations.

(b) Team leader only. Leadership andcommunication abilities and experiencewith complex institutional evaluations arevital for the Team Leader’s position andconsidered more important for the TeamLeader than for the two Senior Experts.

153

216 These terms may be understood with reference to theIFAD Evaluation Policy available athttp://www.ifad.org/gbdocs/eb/78/e/EB-2003-78-R-17-REV-1.pdf

217 Non-lending activities are defined in paragraph 7(b).

(c) Other personnel of the core team. Othermembers of the core team will have expe-rience in evaluation, participatoryapproaches (particularly social mobiliza-tion), agricultural and rural development,the five IFAD regions and the specificareas of expertise for which they areproposed by the service provider.

(d) The team as a whole (comprising theteam leader, two senior experts, otherpersonnel of the core team and nationalevaluation teams). Overall team composition will be guided, in addition,by the following important and essentialconsiderations:

n The team will include expertise in thequantitative and qualitative methods ofsocial and economic research.

n The team, as a whole, will representregional and gender balance and therequired language expertise. Inaddition, it would be desirable for thecore team to possess diverse regionalexperience and language skills.

n There should be no conflict of interest.That is, the consultants selected shouldnot have any concurrent assignmentswith IFAD; they should not have hadany involvement (either as staffmembers or consultants) in the ProcessRe-Engineering or Strategic ChangeProgrammes undertaken by IFAD, or inany of the earlier externalassessments/reviews/evaluations ofIFAD; or a work history with more than15 per cent of working days with IFAD.

n The service provider and its team mustdemonstrate that they can organize in-country research and nationalresources, including national evaluation teams for the field-based part of the IEE.

29. The service provider will organize nationalevaluation teams in each of the countries inwhich the IEE will carry out independent vali-dation. These teams may be recruited fromamong individuals or national serviceproviders such as consulting firms, non-governmental organizations and academicinstitutions. The service providers are not

expected to identify these teams at theproposal stage, but they will provide lumpsumbudget estimates if invited to submit a costproposal. The service provider will ensurethat the national evaluation team membershave the experience and skills required forthe task assigned to them and have no conflictof interest in terms of the IEE’s TOR. Inaddition, OE will assist the service provider, in consultation with the ProgrammeManagement Department of IFAD, in identi-fying a suitable national evaluation counter-part in each country, who will organize accessto selected IFAD-assisted projects and relevantofficials and other stakeholders.

8. Governance and organization30. The evaluation will be conducted under the

overall supervision of the Director of IFAD’sOffice of Evaluation on behalf of the IFADExecutive Board.

31. A steering committee, composed of represen-tatives of IFAD member countries, will servein an advisory capacity to the OE Director andthe service provider, providing comments ondraft documents and other issues as furtherdescribed below. In specific terms, it isexpected that the steering committee will: (a)review and endorse the TOR; (b) endorse theselection of the Evaluation Team, as recom-mended by the OE Director under proce-dures described below in paragraph 36; and(c) review the IEE reports, as mentioned inparagraph 25 (a).

32. The OE Director will serve as the serviceprovider’s primary counterpart, ensuring thatthe Evaluation Team conducts its work inaccordance with its TOR, and will facilitatethe work of the service provider as specifiedin this TOR. In this capacity, the Director mayseek the advice of the steering committee. TheOE Director will be supported in his task bytwo senior independent advisers of interna-tional standing, who will interact with theservice provider at key stages of the evaluationprocess and review draft interim reports andthe draft final report, as mentioned in para-graph 13 (b). They will provide comments onthe evaluation methods used and processesfollowed, as an input into a report that the OE

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

154


155

Director will make available to the steeringcommittee and the Executive Board, togetherwith the final IEE report. If at any pointduring the evaluation the Director believesthat the service provider is deviating materi-ally from the requirements of its TOR, hemay require corrective measures to be taken.He will inform the steering committee andExecutive Directors/Alternates of any suchactions. He will also provide a status report onthe progress of the evaluation at each Executive Board session during the evaluation.

33. The OE Director will not be responsible,however, for the findings, recommendationsand other contents of the IEE reports. The duties and role of OE in the IEE aredescribed in paragraph 13 above.

34. IFAD will provide adequate office space,photocopying, and telephone connections tothe Evaluation Team while it is based inRome. The service provider will make its ownarrangements from the approved budget fortravel, accommodation, long-distance commu-nication, and in-country visits and work.

35. In accordance with the IFAD EvaluationPolicy, approved by the Executive Board on 9 April 2003, IFAD management will ensurethat IFAD officials and IFAD-assisted projectspromptly provide all documents and otherinformation required and participate andcooperate actively in the IEE. The ProgrammeManagement Department of IFAD will assistthe Evaluation Team in selecting a nationalevaluation counterpart or liaison officer ineach of the countries selected for independentvalidation. Furthermore, IFAD managementand operations will provide comments on theservice provider’s deliverables as described inparagraph 25 (b) and (c) above.

9. Selection procedure36. The OE Director will manage the process for

selecting the external and independentservice provider. The Evaluation Team will berecruited as a unit through an open andcompetitive international bidding process(consisting of a technical and financialproposal and presentations by the three top-ranked bidders) in accordance with the rulesand regulations employed by IFAD for thispurpose. From a short list of at least threequalified candidate service providers, the OEDirector will recommend a selection for thesteering committee’s endorsement.

Annex I

Considerations for constructing a stratified,multistage random sample of projects andcountries

1. For the desk study and subsequent field vali-dation, the IEE will consider a sample ofIFAD projects (stratified by region) that cameinto effect between January 1994 andDecember 2002.

2. The IEE will conduct a detailed desk reviewof 20-25 country programmes encompassing40-50 projects, as well as 4-6 TAGs. Thesecountries and projects will be selected throughstratified, multistage random sampling, asindicated below under paragraphs 3, 4 and 5.TAGs will be selected randomly, taking intoaccount the need to represent different cate-gories of activities financed through TAGs andtheir size. Half of these country programmes,projects and TAGs will be selected randomlyfor independent, field-based validation basedon the sampling considerations given below.

3. The sample will include country programmesfrom the five IFAD regions, namely, Asia andthe Pacific, Western and Central Africa,Eastern and Southern Africa, Latin Americaand the Caribbean, and the Near East andNorth Africa (including Eastern Europe). Thenumber of countries selected in each regionwill reflect the relative importance of thatregion in the total IFAD portfolio of loans for1994-2002 (the weights will be the volume ofloans and/or the number of projects).

4. Once the number of countries per region hasbeen determined, the actual selection ofcountries within the regions will be carriedout randomly, taking into account – to theextent possible – some or all of the followingcriteria: (i) income classification, (ii) size ofcountries’ portfolios in the period 1994-2002(expressed in terms of volume of loansand/or number of projects), (iii) size of coun-tries’ rural populations, (iv) relative size ofthe agricultural sector, (v) IFAD lendingterms to the countries, (vi) human develop-ment index ratings of the United NationsDevelopment Programme.

5. Within each of the selected countries, projectswill be chosen randomly, taking into accountthe need to: (i) represent the main sectors intowhich IFAD projects are classified, (ii) coverdifferent financing modalities and implemen-tation arrangements (such as cofinancing andthe flexible lending mechanism), and (iii)consider implementation difficulties arisingfrom security concerns. To allow for a moremeaningful assessment of results and ensurefull interaction with project management andother stakeholders, a relatively larger propor-tion of the sample will be assigned to projectsin the last year of implementation.

Annex II

Processes to be considered by the IEE

1. Formulation, approval and implementationof:n IFAD strategic frameworks (2002-06 and

1998-2000);n the five regional strategies; andn COSOPs.

2. Identification of the need for corporatepolicies, and their formulation, approval andimplementation.

3. All stages of the project cycle, including:n project identification (inception);n formulation (including selection of project

components and implementing partners,and the role of consultants);

n appraisal;n loan negotiation;n loan approval;n loan effectiveness; andn implementation (including M&E systems,

supervision by IFAD and/or its cooperatinginstitutions, and self-evaluation by theProgramme Management Department).

4. IFAD’s independent evaluation, and theprocess for tracking adoption and implemen-tation of independent evaluation recommen-dations by IFAD management.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

156


Annex III

Timetable for the IEE

15.04.03 Request to the List Convenors to initiate proposal of members of the steeringcommittee (SC)

04-11.05.03 Preparation of the TOR; organization of the entire IEE process

15.05.03 Deadline for the presentation of the members of the SC to OE

18.06.03 Dispatch of letter inviting expressions of interest

Jun./Jul. 2003 Preparation of detailed budget, related Board document for September 2003 andrequest for proposals (RFPs)

10.07.03 Deadline for submission of expressions of interest (EOIs)

15.07.03 Endorsement of the TOR by the SC

11-31.07.03 Screening of EOIs and short-listing for RFPs

01.08.03 Dispatch of RFPs to qualified respondents

08.09.03 Deadline for receipt of technical and financial proposals

08-26.09.03 Ranking of proposals

10-11.09.03 Presentation of the IEE budget proposal for approval and the first progressreport to the Executive Board

06-10.10.03 Top three bidders to make presentations in Rome. Final selection by OE

17.10.03 Approval of selection by Contracts Review Committee

20.10.03 Review and endorsement of selection by the IEE steering committee.

22.10 – 10.11.03 Negotiation and award of contract

24.11.03 Start of work by the contracted Evaluation Team

17-18.12.03 Presentation of the second progress report to the Executive Board

04.04.04 Presentation of the third progress report to the Executive Board

Dec. 04 Completion of the IEE

Jan. 05 Distribution of the IEE report by the IFAD Secretariat to the Executive Board

Feb. 05 Discussion of the IEE report in a special/informal reunion of theExecutive Board

157

Annex IV

Illustrative allocation of level of effort (LOE)

Deliverable and Calendar Time Consultants and LOE Distribution of TimeRome FieldUp to: As little as:

1. Inception report (two months) Up to 3 consultants:i. Team leader (2 months) 100% 0%

ii. Senior consultant No. 1 (1 month) 100% 0%iii. Senior consultant No. 2 (1 month) 100% 0%

2. Desk review (two months) Up to 4 consultants for 2 PM* each:i. Team leader 100% 0%

ii. Senior consultant No. 1 100% 0%iii. Senior consultant No. 2 100% 0%iv. Other personnel No. 1 100% 0%

3. Evaluation report on IFAD Up to 7 consultants for 3-6 PM each:projects and non-lending i. Team leaderactivities (six months) ii. Senior consultant No. 1

iii. Senior consultant No. 2 15% on 85% oniv. Other personnel No. 1 average averagev. Other personnel No. 2

vi. Other personnel No. 3vii. Other personnel No. 4

Plus (for 2-3 months each):n 11 National evaluation counterparts 100% 0%n 11 National evaluation teams 100% 0%

4. Draft final report (one month) Up to 3 consultants for 1 PM each:i. Team leader 100% 0%

ii. Senior consultant No. 1 100% 0%iii. Senior consultant No. 2 100% 0%

5. Final report ( one month) Up to 2 consultants:i. Team leader (1 month) 100% 0%

ii. Senior consultant No. 1 (1 month) 100% 0%

*PM = person-month(s)

Note: For purposes of budgeting, it is assumed that half the total level of effort will

be spent in the field. Standard IFAD travel and DSA rates apply to all work.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

158


1. As requested, the Director of OE receivedregular progress reports on the IEE at allstages of its preparation. This note summa-rizes our joint views on the evaluation process and its outcome.

A unique process2. The President, Executive Board Directors

and members of the steering committeescrupulously respected OE’s independentstatus and the role it played in supervisingthe evaluation; this, together with the excel-lent logistical support provided by OE, wereinstrumental in the successful completion ofthe IEE. The process would not have been socredible had OE been less independent.

3. The governing bodies of the organizationrecommended that an independent, externalevaluation be made of IFAD, and ExecutiveBoard Directors played a prominent role indesigning the entire process. A steeringcommittee endorsed the terms of referenceand consultant selection process; it alsoreviewed the draft reports prepared, whichled to several rounds of multi-stakeholderinteraction. While this unusual andsomewhat cumbersome arrangement added significantly to the costs borne by OE and the consultants, it ensured IFAD’s‘ownership’ of the process and avoided any undermining of the independence of the evaluation.

4. The process of selecting the consulting firm toundertake the IEE was managed by OE in afully transparent manner, which also facili-tated communications among ExecutiveBoard Directors, IFAD management and theconsultants. It also provided logistical supportand information to the evaluators, and moni-tored their compliance with the terms ofreference, agreed IEE methodology, processesand deadlines. Care was taken to ensure thatthe comments of management, the steeringcommittee and senior independent advisorswere reviewed by the consultants. Finally, OEoffered guidance and professional advice tothe evaluators without ‘crossing the line’ orundercutting the integrity of the process.

Compliance with terms of reference5. The scope of the evaluation report is

comprehensive and its analyses are generallysound. Going well beyond an ex post assess-ment of replenishment undertakings, theterms of reference required the consultantsto carry out both an impact assessment and acorporate evaluation. Either task would havebeen demanding in itself: the combination ofboth tasks amounted to an extraordinarilytough challenge in view of the tight deadlinesinvolved, elaborate overview arrangementsand paucity of self-evaluation information.

6. All in all, the process yielded sound andcredible results, especially with regard to

159

The comments of the IEE Senior Independent Advisors

Annex 6

operational aspects. As per the terms of reference, the final report sets out strategicconclusions and offers a number of valuablerecommendations. If used by managementand the Executive Board to guide IFAD’srenewal process, the report could be ofconsiderable value to the institution.

Adequacy of evaluation methods7. In the absence of clear corporate performance

data, agreed metrics and reliable measure-ments, reliance on professional judgment wasinevitable. The consultants had to ‘make do’with a pervasive absence of baseline data and apaucity of verifiable performance indicators atall levels (project, country and corporate)218.Faced with these constraints, they opted for atriangulation method that relied on deskreviews, syntheses of OE and internal auditreports, country visits, extensive face-to-faceinterviews and a staff survey.

8. The consultants did a very good job in termsof collecting and interpreting evidence. Teammanagement was competent and responsiveto advice; the team members’ professionalqualifications were appropriate; the decisionto employ local consultants for country workproved to be sound; the analytical methodswere sensible, transparent and in line withgood development evaluation practice; therandom sampling method was properlyapplied; and good judgment was used inrating project performance and synthesizingthe results.

9. The report would have benefited from: (i) afuller examination of rural development litera-ture; (ii) more searching assessments of IFAD’soperational and budget performance inrelation to other development agencies(benchmarking); and (iii) systematic opinionsurveys of government officials and benefici-aries. On balance, however, the report isadequately grounded in evidence and reflects– in a fair manner – the extensive findings ofthe desk review, ten country reports andproject assessments. It also takes account of theprobing comments furnished by OE and IFADmanagement, and adds considerable value toprior external evaluations of IFAD in terms ofits rigour, independence and originality.

Comments on the diagnostic10. We endorse the evaluation’s central messages.

IFAD’s mission remains central to the devel-opment enterprise. There are still more ruralthan urban poor, progress in reducingpoverty has been lagging, the easy gains fromthe Green Revolution have tailed off, second-generation policy reforms have proved hardto manage, and the volume of aid targeted tothe rural poor has declined. The MDGs willnot be achieved without a more coherent and effective set of action geared to ruralpoverty reduction.

11. Reaching the unreached and amplifying thevoice of the voiceless is IFAD’s unique role inthe development system. For IFAD to reach itspotential, it should develop a new businessmodel that leverages the Fund’s contributionand emphasizes creativity, innovation andoutreach. This will require a change processfocused on improved quality assurance,stronger self-evaluation processes and busi-nesslike partnerships. Every attempt shouldbe made to connect to like-minded partnersand generate cofinancing without compro-mising on basic objectives.

12. Most of all, IFAD should aspire to become alearning organization that helps the rural poorto help themselves. It is not for IFAD todeploy its resources to influence macro-policiesthat should be driven at the national level.Instead, IFAD should concentrate on theactions that governments and their partnersshould take to replicate cost-effective methodsof poverty eradication, leaving to others thetask of probing the frontiers of rural povertyresearch. In support of the MDGs, the organi-zation should build on its principal strength,viz. the legitimacy of its pro-poor, pro-womenand pro-livelihood orientation to rural andagricultural development.

13. IFAD’s strength should be on scaling up inno-vation – i.e. do-how based on the best avail-able know-how. An important method ofpoverty reduction in predominantly agricul-tural and rural countries is the enhancementof the productivity, profitability and sustain-ability of major farming systems. Therefore,IFAD should seek out hard-to-reach rural

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

160

A N N E X 6 : T H E C O M M E N T S O F T H E I E E S E N I O R I N D E P E N D E N T A D V I S O R S

areas inhabited by ecologically and economi-cally disadvantaged communities. Accordinglyit should develop methodologies for engen-dering sustainable livelihood securityprogrammes and specialize on deliverysystems for social and nutritional safety net programmes.

14. In fulfilment of its mandate, IFAD shouldremedy the operational weaknesses high-lighted by ITAD Ltd. At the country level,country strategies should be more focused;the country presence more effective; theinterface with other partners more proactive;and the operational practices less burden-some for developing countries throughharmonization with other partners and align-ment to developing country goals. At theoperational level, quality management needsto be strengthened in terms of real-timetracking of project quality; diversification ofIFAD’s tool kit to emphasize innovation andadaptable lending; and emphasis on innova-tion, creativity and flexibility. At the corporatelevel, human resources policy and budgetmanagement need further strengthening and adaptation.

15. Given its pioneering mandate, IFAD should behighly selective in the choice of its operationsand more rigorous in the arrangements that itmakes for tracking the development results ofits operations. But the Performance-Based

Allocation System should not discriminateagainst fragile, conflict-prone and conflict-affected countries with a large number ofendemic difficulties, including infrastructuredeficiencies and governance weaknesses. IFADhas much to contribute in the enhancement ofhuman security in difficult operating environ-ments. Well-defined project objectives andperformance and output indicators should beput in place at the time of project preparation.

16. Finally, we agree that the Executive Boardshould play a more active role in prioritysetting and policy oversight. Managementshould concentrate on adapting the organization to its core mandate and onreaching out to partners. This will requiregoal-oriented partnerships designed toamplify IFAD’s pro-poor, pro-women andpro-livelihood impact. Transaction costs willneed to be reduced and this calls for aseamless commitment to shared values, partic-ipatory policy development and humanresources initiatives of the kind recentlylaunched by the President.

17. A global organization dedicated to findingsolutions to the complex dilemmas faced bythe rural poor in developing countries isbadly needed. Now more than ever, the worldneeds a centre of excellence dedicated torural poverty reduction. If IFAD did not exist,it would have to be invented.

161

218 These gaps, having been identified, should be filled toguide corporate management and policy development.

Professor M. S. Swaminathan Dr Robert Picciotto

Rome, March 2005

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

162

IFAD management’s response to the IEE

Annex 7

The challenge ofachieving the MDGs:The centrality ofrural poverty reduction

On 18 January 2005, thethree Rome-based UNagencies launched the reportof the UN Millennium Projectin Rome as part of a globalprocess. The challenge isimmense: an estimated 1.1 billion people live on lessthan one dollar a day. Ofthese women and men, over 800 million – or about75 per cent – live in ruralareas. For now and manyyears to come, the principalchallenge to achieving theMDGs lies in the rural areasof the world where themajority of poor peopledepend on agriculture,forestry, fisheries and relatedactivities for their livelihoods.Increasing incomes for ruralpoor people, throughincreased productivity,production and employmentopportunities, remains thecentral development issue.

During the past year, IFAD has undergone its most comprehensiveevaluation ever. The Independent External Evaluation (IEE) wasprobably the most ambitious evaluation of its kind for a UN agency,breaking new ground in addressing institutional performance interms of impact. This evaluation also set new directions by encour-aging an open and transparent process that shared each draft reportwith the Executive Board members and staff.

We welcome the evaluation and agree with the main thrusts of itsconclusions and recommendations. The evaluation confirms conclu-sions we have drawn and the relevance and direction of changesalready under way. We intend to build on the evaluation by designingand implementing a comprehensive plan of action. This documentsets out proposed building blocks of that plan.

The context This evaluation comes at a critical time, when the internationalcommunity is taking stock of its progress in achieving the MillenniumDevelopment Goals (MDGs) and considering its approaches to eradi-cating poverty. Recognizing the links between security and develop-ment, a strong partnership is emerging to defeat poverty. The call isfor a comprehensive approach, where developing country actions,official development assistance (ODA), debt relief, a development-enhancing trade regime and direct foreign investments all cometogether in the cause of poverty eradication. Today, ODA isincreasing, debt-relief efforts are gaining momentum and the DohaDevelopment Round is addressing the links between poverty andworld trade. Increasingly, countries are realizing that rural povertyeradication is essential to achieving the MDGs.

Our missionThis is the context in which the evaluation confirms the relevance ofIFAD’s mandate. The IEE found that IFAD is “The only internationalorganization established to focus exclusively on the situation of therural poor” and that “The problem that IFAD was created to tacklecontinues, and is likely to continue, in the foreseeable future”.

163

In the international architecture for poverty eradication, IFAD’s missionand responsibilities remain essential. While there appears to be growinginterest and engagement in the social and infrastructural aspects ofrural development, IFAD is one of the few organizations that supportsthe productive activities of the rural poor, both in agriculture as well asin related off-farm activities.

We will work with others to develop and implement effective, coordi-nated and harmonized projects and programmes, bring more materialresources to rural poor people and, equally important, bring under-standing and knowledge to bear – knowledge that is developed withpoor people and their institutions.

The Fund’s target group includes the rural poor in low income coun-tries, particularly in sub-Saharan Africa, but also in parts of Asia andLatin America, in which they account for a large part of the popula-tion, sometimes even the majority. But there are significant groups ofrural poor in other developing countries as well, who need support tohelp them to work their way out of poverty. Supporting programmesfor the latter often provides insights and lessons which can proveuseful in low income countries.

Since 2002, IFAD’s work programme has risen by 28 percent from just under US$400 to US$500 million in 2005. IFAD projects attractco-financing of over one United States dollar for each dollar of IFADlending. Thus, in 2005, the total investment costs of IFAD-supportedprojects and programmes are likely to total over USD 1 billion.Recognizing the need to serve more rural people and to have greater impact, we are committed to continuing this growth and areproposing a significant increase in the Seventh Replenishment. Theinternational community is gearing up to increase the resources itprovides to poverty eradication, and this provides a growing opportu-nity for IFAD to play an important part. As the IEE points out, thiswill require new ways of working and more resources.

Our evolving roleA rapidly changing global economy and political environment todayplace a premium on innovation – on new answers to old and newproblems. We concur with the IEE conclusion that, in a generalcontext of rapid and continuous change, IFAD must pursue innova-tion with its partners – not for its own sake, but to generate new solutions to these challenges and to promote their replication andscaling up.

At IFAD, we are committed to exploring innovative and effective solutions to the challenges faced by rural poor people. We will support our development partners as they apply these solutions on a large scale within coordinated, country-led processes.

As the number of partners engaged in rural poverty eradication hope-fully increases, it becomes all the more important that we contributeour field-based experience and knowledge to enriching country-led

IFAD and other inter-national financialinstitutions

The IEE states: “In terms ofperformance IFAD appears tobe on a par with other IFIs”.With a relatively small staffand with operationalarrangements that leavemany key activities in thehands of third parties, theoperational effectiveness of IFAD projects andprogrammes has beencomparable to that of otherIFIs. IFAD has already begunimplementing a broad rangeof reforms to enhanceresults, impact and sustainability. The IEE pointsto the need to completeongoing key reformprocesses, in areas such as human resource management. It also calls on IFAD to develop newcapabilities for acceleratingrural poverty reduction in response to a growingglobal concern about thebasic vulnerabilities of ruralpoor people – so clearlydemonstrated by thetsunami in Asia.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

164

A N N E X 7 : I FA D M A N A G E M E N T ’ S R E S P O N S E T O T H E I E E

Managementresponse: The driving force

The driving force behindIFAD management’sproposals is a commitmentto increasing the development effectiveness of the rural poverty eradication effort. It willinvolve developing strongermechanisms for managingfor results, setting objec-tives and measuringperformance, and creatingnew instruments and relations to achieve impact.This will include new andimproved ways ofmanaging staff resources,developing and managingpartnerships, and workingwith governments.

policy dialogue, harmonization and the scaling up of proven solutions. We need to ensure that decisions are made with fullawareness of the constraints and opportunities that domestic liberalization and external globalization bring to rural poor people.

Enhanced performanceThe emergence of a global focus on poverty has opened newopportunities for influence, and has necessitated the rapid generation and communication of experience and knowledge. At the same time, strong subscription to country leadership and to the development and implementation of country poverty strategies has led to systematic decentralization of developmentassistance and to new forms of financing and programming. Weagree with the evaluation’s conclusion that to get better results,IFAD must re-configure itself in response to these changes.

The area of human resources management is a key element ofongoing changes at IFAD. In other areas, proposed changes gobeyond technical refinement and streamlining, to involve, in thewords of the IEE, a new “business model”. We fully agree on the direction and need for change, and these are elaborated in the section on building blocks.

IFAD’s broad membership, and the role all members play in thefunding and governance of the institution, shapes IFAD’s opera-tions in many ways, and accounts for some of the differencesbetween IFAD and similar institutions. Sustainable change in IFADwill require the support of all its members. Recognizing the needfor consultation, dialogue and consensus, IFAD management’sresponse to the evaluation identifies the building blocks that willform the framework of the Way Forward for the SeventhReplenishment and lead to a comprehensive plan of action for further strengthening the institution.

Building blocks for changeThe evaluation calls for IFAD to work on six key areas to enhance its effectiveness:– clarify IFAD’s strategic niche– develop a new business model– adopt smarter ways to skills and learning– address causes of low impact– provide direction for development effectiveness– manage change

IFAD’s response to the evaluation addresses these areas bydefining six building blocks for change to cover the issues identified. These responses are highlighted next, followed by amore detailed framework that sets out specific actions and results.

165

1. Clarifying IFAD’s role and focusThis involves the clear articulation of IFAD’s role and focus within thecurrent international architecture. It includes:– strengthening opportunities for the rural poor who have

productive capability– supporting country leadership and harmonization– strengthening the effectiveness of national poverty reduction

programmes by promoting the scaling up of innovative responses tokey issues

– raising globally-linked local development issues in international fora – raising the overall level and impact of assistance to rural poverty

reduction and seeking new and more effective ways to eradicate poverty

This role and focus will be articulated in the new Strategic Framework.

2. Develop a new operating modelThe model will ensure that IFAD’s operations are organized on acountry-programme basis by effective country teams. It will changeIFAD’s participation in supervision and enable a more flexible systemof country presence. These changes will be underpinned by enhancedstaff training and recruitment. IFAD will shift from quality control toquality assurance, introduce a new accountability framework, developstronger self-evaluation systems and improve knowledge management.A stronger Policy Division will inject knowledge about local constraintsand opportunities into global policy dialogue. The change in supervi-sion modalities will require a change of the articles of agreement establishing IFAD.

3. Strengthen knowledge management andleverage external institutional skills

IFAD will build staff capacity for developing skills and managing knowledge. A strengthened knowledge management system willleverage external institutional skills and use IFAD’s operations as abasis for knowledge development in an explicit, focused and moni-tored fashion. IFAD will adopt an inward and outward-oriented,network-based approach to knowledge management, mobilizing alimited number of partnerships with regional and country networksas well as international centres of excellence. External expertise and knowledge shared through structured and long-term relation-ships will strengthen IFAD’s internal development and quality-assurance processes.

4. Strengthen the culture of results and performance

An accountability framework will be developed to embed quality assurance into operations. This framework will be integrated into corporate planning, budgeting and monitoring and will incorporateRIMS, PBAS, and inputs from the Office of Evaluation and the Office of Internal Audit, based on the three-year medium term plan. It willenhance quantitative performance reporting and analysis, andstrengthen self-evaluation systems. IFAD will mobilize external

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

166


Six building blocks of change

IFAD’s management agreeswith the importance of thesix areas identified by the IEEfor future emphasis andperformance enhancement,and commits to a plan ofaction that will:n clarify IFAD’s role

and focusn develop a new

operating modeln strengthen knowledge

management and leverageexternal institutional skills

n strengthen the culture ofresults and performance

n enhance leadership fordevelopment effectiveness

n manage change

quality-assurance expertise for performance assessments. IFAD’s keypartners will contribute to the institution’s corporate performancereviews. In implementing all these, improved management of staffperformance will be a key priority.

5. Enhance leadership for development effectiveness

Management will consult with the Executive Board on ways toimprove the Board’s ability to lead and support IFAD’s quest toenhance development effectiveness. The Executive Board willconsider and approve a new Strategic Framework. Management willpresent to the Executive Board a three-year medium-term plan anda review of impact and effectiveness. The Executive Board would beprovided with more time to review development effectiveness.

6. Manage changeThe President will manage the change process, with individual andcollective accountabilities assigned to the senior management team. A change management strategy will be part of the costed action plan.Management will retain an external quality-assurance group to adviseon action plan implementation. Management will also review budg-etary allocations against action plan objectives and priorities, and report to the Executive Board on plan implementation and performance.

Taken together, these proposals represent a comprehensive responseto the challenges identified by the evaluation. They largely incorporatethe recommendations of the IEE, and in some areas go beyond them.

IFAD’s management did not agree with the IEE in two areas and has elected to defer two other areas for further discussion. Withregard to the IEE’s recommendation on the need for a managingdirector, we believe the intended results would be better achieved by strengthened senior management oversight and direction, moreeffective planning and reporting and enhanced opportunities for the Executive Board to monitor progress. With regard to ExecutiveBoard approval of projects, we recommend that the Executive Boardcontinue to approve projects. We defer comment on the develop-ment of new financial instruments and more flexible approaches to serving fragile states pending fuller consultation with theExecutive Board.

Overall, we intend to build on the IEE and the experience of ourpartners to make IFAD an institution characterized by effectiveness,collaboration and transparency of operations and governance withthe capacity to deliver a growing programme of work. A higher-levelSeventh Replenishment will enable IFAD to play this role. We expectthe action plan to be developed with institutional costs fully recog-nized. Implementation of the action plan should commence in 2006and be completed during the Seventh Replenishment period from2007 to 2009.

167

Building block

Clarify IFAD’s role and focus

IEE recommendation: Clarify IFAD’s strategic niche

1

Results

n IFAD focuses on rural poor people who have

productive capability, in particular women as well

as marginalized groups such as indigenous

peoples and ethnic minorities. The Fund applies

selectivity at the country and sector levels to

improve access of rural poor people to assets,

technology, markets and rural institutions

including financial ones.

n IFAD-funded operations work with rural poor

people and other partners to identify new,

more effective ways to catalyse the scaling

up and replication of innovations through

communication, policy dialogue and resource

mobilization partnerships that are linked to

national poverty reduction strategies. This

involves mobilizing greater investment resources

and increasing the capacities of national institu-

tions and processes.

n IFAD partnerships build on country ownership,

including support to national groups and institu-

tions as leaders of local development. These

efforts provide, through participation

in development assistance coordination and

harmonization processes, new perspectives and

solutions for poverty eradication that are based

on experience and results.

n IFAD’s Strategic Framework focuses on raising

the level and effectiveness of national and inter-

national efforts to enable rural poor

people to overcome poverty, helping countries

to achieve the MDGs.

Actions

n Include a comprehensive statement of IFAD’s

development objectives and modalities in the

Seventh Replenishment Consultation Report.

n Produce a new Strategic Framework, to also

guide the medium-term plan and the annual

work plan and budgets for 2007-2009.

n Prepare updated regional strategies that are

guided by the Strategic Framework.

n Develop and implement a strategy to enhance

impact of IFAD’s projects and programmes.

n Develop policy partnerships to help bring a

pro-rural poor perspective to global forums

and processes.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

168


Building block

Develop a new operating model

IEE recommendation: Develop a new business model

2

Results

n Country programmes, as more than a compilation

of projects, are the core of IFAD’s country activities.

Developed through consultations with governments,

other national partners, and development assistance

coordination processes, country programmes establish:

– major activities and processes to be supported by

loans and grants

– objectives for innovating and scaling up, in

conjunction with PRSPs, SWAps and other processes

– objectives for influencing policy and developing

institutions

– national and international partnerships for

supporting activities

– frameworks for performance measurement and

accountability that reflect PBAS findings on key

institutional and policy issues in rural development,

RIMS and project portfolio risk assessments.

n Country teams are IFAD’s interface with countries. They

work in partnership with national players and represen-

tatives of development assistance institutions. At IFAD,

planning and follow-up involve cross-unit collaboration.

n Enhanced country presence engages IFAD in national

processes, supporting the development and manage-

ment of partnerships, policy dialogue, implementation,

harmonization and knowledge management. The form

of country presence varies according to the scope of

IFAD operations, range of possible local partnerships,

costs and benefits.

n Supervision is reorganized for greater

development effectiveness.

n CPMs and others are recruited and trained for policy

dialogue, partnership development, implementation

support and team-building – and are motivated

with incentives.

n Global dialogue on poverty eradication is enriched

by perspectives and issues identified in country

development and assistance processes.

Actions

n Make a strategic, results-based COSOP

the core instrument for managing

country programmes.

n Revise the project cycle and undertake

country programme reviews, encom-

passing assessment of project and non-

project activities against objectives.

n Include performance of country teams

in individual performance evaluations.

n Reassess staff competencies across IFAD

in the development of a new operating

model, and train staff accordingly.

n Develop and implement a human

resources strategy to enable CPMs to

acquire competencies required for policy

dialogue, managing partnerships and

conducting team-based operations.

n Present an approach on enhancing

country presence to the Seventh

Replenishment Consultation.

n Amend IFAD’s Articles of Agreement at

the Governing Council session in 2006 to

enable IFAD to pursue a broader range of

supervision options, including national

and direct supervision.

n Produce a supervision policy in 2006

that defines supervision modalities

and selection criteria.

n Subject supervision processes to annual

performance reviews and internal quality

assurance, and integrate supervision

findings into knowledge management

and performance management systems.

169

Building block

Strengthen knowledge management and leverageexternal institutional skills

IEE recommendation: Adopt smarter ways to skills and learning

3

Results

n A knowledge management strategy is

integrated into country operations, as

well as in regional and country strategies.

Knowledge management reflects local

conditions and is included in unit and

individual performance measurements.

n Knowledge management is strengthened

and focused: IFAD plays a vigorous role in

developing and disseminating knowledge –

and in using its in-country programmes for

innovation, scaling up and influencing policy.

n IFAD stimulates knowledge development

and dissemination by external centres

of excellence, focusing on key rural

poverty issues.

n Loan and grant programmes have

knowledge objectives, specific partnerships

and measurable performance benchmarks.

These programmes flow from IFAD’s

Strategic Framework, regional strategies and

COSOPs and include engagement in policy

processes, innovation, and the building

of local assets and institutions.

Actions

n Revise planning and review processes to ensure

that all activities supported by IFAD have specific

knowledge objectives and concrete mechanisms

for capturing and sharing knowledge at

appropriate levels.

n Implement an inward and outward-oriented,

network-based approach to knowledge manage-

ment through investments in regional networks of

institutions (such as project-based knowledge

networks, farmers’ associations, research centres

and key NGOs) to deepen knowledge and enhance

exchange among country and regional rural poverty

eradication practitioners.

n Invest in knowledge partnerships with international

centres of excellence in key strategic areas. They

will maintain thematic information inventories,

supervise research activities, and manage exchanges

with national and regional networks.

n Mobilize institutional knowledge partnerships

selectively to support IFAD’s own internal

processes, including quality assessments, review

processes, and production of major policy and

programme materials.

n Incorporate knowledge management and innova-

tion objectives into staff performance assessments

and incentive systems and provide necessary time

and training.

n Develop the Rural Poverty Portal and systems for

knowledge dissemination through strengthened

internal capture and sharing of learning.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

170


Building block

Strengthen the culture of results and performance

IEE recommendation: Address causes of low impact

4

Results

n A comprehensive system to embed

quality and impact assurance is imple-

mented. The system is based on a

clear accountability framework and is

informed by medium-term planning and

monitoring, RIMS, PBAS, and the inputs

of the Office of Evaluation.

n IFAD’s country-level partners focus

on results for the rural poor.

n There is enhanced Executive Board

accountability for development effective-

ness, made possible by improved moni-

toring and reporting.

n Staff members are accountable for

results and rewarded for performance.

Actions

n Fully implement the human resources management

reforms currently under way, linking pay and

promotion to performance within a transparent

human resource system based on careful planning,

monitoring and reporting.

n Link strategy to action by consolidating a planning,

budgeting and monitoring system that focuses work

on clear results, impact and performance targets

through annual and three-year plans.

n Establish capability for quantitative monitoring of

corporate performance, establishing key performance

indicators relative to strategic objectives, such as

innovation and scaling up, in collaboration with

partners and centres of excellence.

n Establish a system of internal quality assurance for

programmes at entry and during implementation,

including the quality of supervision.

n Engage an external quality assessment group to assist

IFAD’s management in reviewing qualitative perform-

ance systems.

n Include IFAD’s key partners in reviews of performance.

n Fully deploy the project Results and Impact

Management System (RIMS).

n Fully implement a system of self-evaluation based

on strengthened programme reviews and project

reporting processes.

n Collaborate with national and international partners to

develop local capacities for participatory monitoring of

national programmes for rural poverty eradication.

171

Building block

Enhance leadership for development effectiveness

IEE recommendation: Provide direction for development effectiveness

5

Results

n Executive Board provides more effective over-

sight of IFAD’s efforts to improve direct

and indirect development effectiveness.

n Executive Board approves projects in a new

format, including approving their objectives

and funding levels.

Actions

n Obtain Executive Board endorsement of

the new Strategic Framework.

n Produce benchmarks for evaluation and

improvement of IFAD performance.

n Submit to the Executive Board a three-year,

medium-term plan as a framework for

determining annual work plan and budgets.

n Develop reports on effectiveness for the

Executive Board.

n Increase the time available for the Executive

Board to review development effectiveness

targets and performance.

n Develop a new project approval format that

seeks Executive Board approval of project

objectives, funding level and results frameworks.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

172


Building block

Manage change

IEE recommendation: Manage change

6

Results

n Major changes for increased development

effectiveness are strategically managed by being

carefully planned, financed, implemented,

monitored and evaluated.

Actions

n Submit a Way Forward document to the

Seventh Replenishment Consultation that sets

out the principal methods for enhancing IFAD’s

development effectiveness within the context

of the MDGs.

n Develop from the Seventh Replenishment

Consultation an action plan, which includes:

– major changes required in IFAD

– measurable change indicators and milestones

– costs of individual changes and financial

implications of the overall change programme

– a strategy to manage the change agenda

n Retain an external quality consultant to support

IFAD’s management in implementing the

action plan.

n Produce a zero-based budget for 2007

reflecting changed institutional priorities and

structures in the core budget mechanism.

n Report to IFAD’s governing bodies on achieve-

ments against change targets.

Report on the IndependentExternal Evaluation of IFAD(agenda item 3)

1. The report on the Independent ExternalEvaluation (IEE) of IFAD (document EB2005/84/R.2), as submitted by the Director ofthe Office of Evaluation (OE), was initiallyreviewed at an informal meeting withMembers of the Consultation on the SeventhReplenishment of IFAD’s Resources.Presentations and introductions were givenon the four parts of the document, followedby a question-and-answer discussion period.The Board subsequently formally consideredthe draft final report at its reconvenedsession, when Directors provided theircomments both to the service provider team – ITAD Ltd. – and to Management on theexternal evaluation and Management’sresponse to it.

2. The OE Director gave a presentation on theIEE process, highlighting its distinguishingfeatures and concluding that the IEE wasamong the first evaluations of its kind formultilateral development organizations. TheOE Director also explained how the gover-nance arrangements laid down by the Boardhad worked in practice and concluded thatthe complex governance had worked remark-ably well. He reported that the evaluationteam had completed a daunting task profes-

sionally, on time and within the budgetapproved by the Board and recorded hisappreciation of the cooperation providedthroughout the process by IFAD Managementand staff, and the value added at variousstages by the steering committee and OE’s two senior independent advisers.

3. The Chair of the IEE Steering Committee,Mme Françoise Mailhot, reported that theIEE was a thorough and rigorous exercisebased on a sound methodology and extensiveconsultation. She recalled that the Board hadassigned only an advisory role to the steeringcommittee, which, therefore, was not chargedwith approving any of the deliverables of theevaluation team. She highlighted that thesteering committee consisted of evaluators aswell as other development experts, includingIFAD Executive Board Directors, and thatseveral Member States participated in itsdeliberations as observers at various stages.She concluded, with satisfaction, that the IEEhad been carried out in an independent andexternal fashion; and that the draft finalreport fully met the terms of reference, waswell grounded in evidence and was of high quality.

173

Excerpt from the Minutes of the Eighty-Fourth Session of the Executive Board

219

Annex 8

219 The Eighty-Fourth Session of the Executive Board was heldin Rome on 18-20 April 2005.

4. The two senior independent advisers,Professor M.S. Swaminathan and Dr RobertPicciotto, presented their assessments of theIEE to the Board and discussed some of itsimplications for the Fund. They commendedthe transparency of the IEE process as well asthe extensive stakeholder participation thathad enhanced IFAD’s ownership of the IEE.They characterized the report as comprehen-sive; its analyses were generally sound andadequately grounded in evidence; and itsrecommendations were valuable for guidingIFAD’s renewal process. They drew attentionto a number of trends and concerns aroundthe world that are relevant to IFAD, includingthe feminization of agriculture and ruralpoverty, the problems of the poor in fragilestates and the challenges of enhancing agricul-tural productivity and skills to help the ruralpoor. They also drew attention to the chal-lenges faced by the Fund in the evolvingglobal environment and reiterated the highrelevance of IFAD’s mandate in it.

5. Representatives of the evaluation team from ITAD Ltd. presented an overview ofthe draft final report of the IEE. Theyoutlined the methodology used in the evaluation, and presented in detail itsfindings and recommendations.

6. The President of IFAD introduced theresponse provided by IFAD Management tothe IEE and highlighted the high degree ofconsistency between the IEE recommenda-tions and the ongoing as well as plannedchanges outlined in Management’s response.IFAD endorsed the broad thrusts of the draftfinal report, including its findings andrecommendations. In this respect, thePresident elaborated, in particular, on thebuilding blocks Management had defined toaddress the latter, namely to: (i) clarifyIFAD’s role and focus; (ii) develop a newoperating model; (iii) strengthen knowledgemanagement and leverage external institu-tional skills; (iv) strengthen the culture ofresults and performance; (v) enhance leader-ship for development effectiveness; and (vi)manage change. The President informed thata more detailed action plan specifying thedirections of change and identifying the

nature of resources required, as well asresponsibilities, time lines and performanceindicators would be presented to theExecutive Board in September 2005 as afollow-up to Management’s response. Heinvited the Board’s views and guidance inthis respect so as to assure an approach isadopted that would be synchronized with the replenishment process. As a particularlyimportant example of ongoing changes, theDirector of Human Resources elaborated keyelements of IFAD’s human resources policyand clarified how they would be imple-mented. The President noted, however, thatIFAD Management did not agree with thesuggestion that a managing director beappointed to manage the process of changethat is required. Nor did it agree with therecommendation that most of the loans andgrants should be approved by the Presidentof IFAD without involvement by its Executive Board.

7. The Executive Board welcomed the draft finalreport and commended the OE Director foreffectively supervising the IEE process withinthe established timeframe and budget, as wellas in accordance with the terms of referenceand within the overall provisions laid out bythe Board. It also congratulated ITAD for anextremely useful and high-quality evaluationreport. It further welcomed Management’sresponse to the IEE and endorsed it as theframework for an action plan that would bepresented to the Board in September 2005 for its consideration.

8. The Board acknowledged that the IEE wasindeed the first evaluation of its kindcompared to similar evaluations of othermultilateral organizations. Its focus extendednot only to an assessment of impact andperformance, but also to key corporateprocesses and policies that have a significantbearing on IFAD’s development effectiveness.Its governance was complex but allowedconsiderable participation at important stagesof the process. The IEE was open and trans-parent, and its governance turned out to behighly impressive despite its complexities and might serve as a good example for othersimilar evaluations.

An

Ind

epen

den

tEx

tern

alEv

alu

atio

nof

the

Inte

rnat

iona

lFun

dfo

rA

gric

ultu

ralD

evel

opm

ent

174

A N N E X 8 : E X C E R P T F R O M T H E M I N U T E S O F T H E E I G H T Y- F O U R T H S E S S I O N O F T H E E X E C U T I V E B O A R D

9. The Executive Board found the IEE report to be thorough and rigorous. It presentedconcrete conclusions in direct language, andpragmatic and realistic recommendations forfuture directions. The Board felt, however,that the recommendation for appointing “aperson with broad executive powers andcharged with the task of … driving throughchange to revitalize the Fund” was not justi-fied convincingly by the evaluation team.Moreover, the Board suggested that the finalIEE report specify the role of the Board inmonitoring and providing advice to IFADManagement with regard to the changeprocesses that the Fund should implement inresponse to the IEE. The Board also felt itwould be useful if the final report were toinclude a realistic indication of the pace of theproposed change processes. Finally, the Boardasked that the final IEE report shed morelight on the various notions of targeting thatare currently available within the institutionand which affect the way IFAD approachesthe issue of targeting.

10. The Board agreed that the following specificareas of concern, highlighted in the evalua-tion, needed to be addressed as soon aspossible: articulating IFAD’s specific role andfocus; establishing a new operating model;ensuring that IFAD country strategic opportu-nities papers (COSOPs) truly provide strategic

guidance and a broad framework for all IFADactivities in a given country rather than justserving as an instrument for justifying a set of project interventions; managing change;implementing the IFAD policy for humanresources; dealing with the questions of fieldpresence and strengthening project/programme supervision within the context ofthe evolving operating model; assuring thequality of operations in a culture of learning;engaging in pro-poor policy dialogue;pursuing and scaling up innovative solutionsfor rural poverty reduction; establishingmeaningful partnerships; clarifying IFAD’stargeting approach; enhancing impact andsustainability; and strengthening the role ofthe Executive Board in scrutinizing develop-ment effectiveness.

11. The draft final report will be duly revised byITAD Ltd. to reflect the comments made atthe Board session, in accordance withstandard evaluation practice and, in partic-ular, in line with the provisions of the IFADEvaluation Policy. The relevant extracts of the minutes of the Executive Board discussionon the IEE will be included in the final reportof the Independent External Evaluation ofIFAD, which will then be disclosed andpublished as outlined in document EB 2005/84/R.2.

175

International Fund for Agricultural DevelopmentVia del Serafico, 107 00142 Rome, ItalyTelephone: +39 06 54592048Facsimile: +39 06 54593048e-mail: [email protected] site: www.ifad.org/evaluation