Desk Review Report edited 200604 - International Fund for ...

Independent External Evaluation of IFAD

Desk Review Report

Final Version

Submitted by

ITAD Ltd

15 July 2004

Independent External Evaluation of IFAD Draft Desk Review Report

Page i

Acronyms and Abbreviations

AfDB African Development Bank AFESD Arab Fund for Economic and Social Development AsDB Asian Development Bank BCIE Central American Bank for Economic Integration BOAD West African Development Bank CAF Andean Development Corporation CDB Caribbean Development Bank CI Co-operating Institution COSOP Country Strategic Opportunities Paper CPM Country Portfolio Manager CSO Civil society organizations CWIQ Core Welfare Indicators Questionnaire DS Desk Study ERT External Review Team (External Review of IFAD in 2002) GFATM Global Fund Against AIDS, TB and Malaria GIM General Identification Mission HIPC Highly Indebted Poor Countries IEE Independent External Evaluation of IFAD IFI International Financial Institution LGS Loans and Grants System MDB Multilateral Development Bank MDG Millennium Development Goal MFI Microfinance Institution MIS Management Information System MTR Mid Term Review NGO-ECP NGO Extended Cooperation Programme OE Office of Evaluation OSC Operational Strategy and Policy Guidance Committee PBAS Performance Based Allocation System PCR Project Completion Report PDT Project design team PM Project Manager (in-country) PPMS Project Portfolio Management System PPP Purchasing Power Parity PRSP Poverty Reduction Strategy Paper PSR Project Status Report QA Quality Assurance QAE Quality at Entry Review (a World Bank internal process) QSR Quality of Supervision Review (a World Bank internal process) REA Rapid External Assessment (of IFAD in 1994) RIMS Results-Impact Measurement System RPA Regional Poverty Assessment RRP Report and Recommendation of the President SF IFAD Strategic Framework 2002-2006 SPM Special Programming Mission TAG Technical Assistance Grant TRC Technical Review Committee UNOPS UN Office for Project Services WDR World Development Report (World Bank annual publication) WFP World Food Programme


Page ii


Page iii

Table of Contents

ACRONYMS AND ABBREVIATIONS......................................................................................................................I TABLE OF CONTENTS.........................................................................................................................................III ACKNOWLEDGEMENTS......................................................................................................................................III

SUMMARY.............................................................................................................................................................V

1 INTRODUCTION ........................................................................................................................................1 THE INDEPENDENT EXTERNAL EVALUATION...................................................................................................1 OBJECTIVES OF THE EVALUATION......................................................................................................................1 STRUCTURE OF THE REPORT................................................................................................................................2

2 WORK COMPLETED AND METHODOLOGIES ...........................................................................2

3 EVALUATION OF CORPORATE PERFORMANCE.....................................................................3 PORTFOLIO ANALYSIS..........................................................................................................................................4 CORPORATE STRATEGY DEVELOPMENT .........................................................................................................11 LOANS – PERFORMANCE AND TRENDS IN PROJECT CYCLE MANAGEMENT.................................................23 GRANTS................................................................................................................................................................41

4 EVALUATION OF CORPORATE PROCESSES ........................................................................... 49 POLICY DEVELOPMENT......................................................................................................................................50 KNOWLEDGE MANAGEMENT AND LEARNING................................................................................................51 ORGANIZATIONAL PARTNERSHIPS...................................................................................................................56 HUMAN RESOURCES MANAGEMENT.................................................................................................................60

5 CONCLUSIONS ........................................................................................................................................ 66 INTRODUCTION AND OVERVIEW.......................................................................................................................66 SUMMARY............................................................................................................................................................66 CORPORATE PERFORMANCE..............................................................................................................................68 CORPORATE PROCESSES....................................................................................................................................71 THEMATIC ANALYSIS AFT ER THE COUNTRY VISITS.......................................................................................72

ANNEX 1 BIBLIOGRAPHY .......................................................................................................................... 79

ANNEX 2 LIST OF PEOPLE INTERVIEWED ....................................................................................... 81

ANNEX 3 POVERTY EFFICIENT AID ALLOCATIONS

ANNEX 4 GRANTS

ANNEX 5 COMPLIANCE WITH REPLENISHMENTS V AND VI

ANNEX 6 KNOWLEDGE MANAGEMENT

ANNEX 7 ORGANIZATIONAL PARTNERSHIPS

ANNEX 8 HUMAN RESOURCES AND MANAGEMENT

ANNEX 9 COUNTRY SYNOPSES (DISTRIBUTED SEPARATELY)

Acknowledgements This report was written during the Desk Review Phase of the Independent External Evaluation of IFAD, between January 2004 and 8 April 2004. The principal authors were Chris Barnett, Andy Batkin, Neil Chalmers, Alison Evans, Ian Hill, Derek Poate, John Rowley, and Mary Underwood. Comments from OE and IFAD management are reflected in this final version.


Page iv


Page v

Summary

Introduction and work done

1 This document is the Desk Review Report for the evaluation and responds specifically to Task 2 of the terms of reference. In this phase the team has been based in Rome to gather information from documentary sources and staff at IFAD. Interaction with clients and partners, will take place during the next phase of the work as part of and alongside the country visits. This report should be read as a statement of findings at a point in time, rather than in conclusion of any particular line of enquiry. A key role of this report is to highlight the issues that appear to be important and will be developed in the later stages of the evaluation.

2 The main findings at this stage concern the corporate learning process and performance of policy and operations. Clearly, it is too early to consider recommendations, but the reader will find pointers to issues that the IEE team consider will be important in the final analysis.

3 The main focus of work during the desk phase was to analyse a sample of projects. The terms of reference for the assignment prescribed analysis of a sample of 20 to 25 countries and 40 to 50 projects, half of which will be visited in the next phase. A stratified random sampling scheme was used with probability proportional to numbers of projects in each region. The majority of the time spent by the IEE team was devoted to documentary review of these projects and meetings with CPM and other operations staff.

The IFAD portfolio (page 4)

4 IFAD’s mandate states that eligibility for assistance should be on the ‘..basis of objective economic and social criteria with special emphasis on the needs of the low income countries and their potential for increasing food production, as well as due regard to a fair geographic distribution in the use of such resources’.1 In what is inevitably a rather general analysis at this stage, there seems to be preliminary evidence that IFAD’s portfolio has evolved in line with its original mandate. IFAD project financing is more or less equitably distributed across regions and seems to be oriented towards the better performing countries, with some important exceptions.

Corporate strategy (page 11)

5 Corporate strategy development has been analysed for the period since 1994 to address the evaluation hypothesis that: The adoption of strategic policies during the 1990s has set project activity in a systematic fashion within a broad country and inter-organisational context. Strategic policies adopted since the late 1990s have provided a more systematic framework for project activity, country strategy and partnerships. The hierarchy of strategic objectives outlined in the Strategic Framework, Regional Strategies and COSOPs has helped to build clearer internal consensus and a ‘sense of mission’ within the organisation.

6 Regional Strategies are strong on the quality of analysis, the clarity of objectives and for setting out principles to guide country programming. But they are weak in articulating IFAD’s development niche. Links with in-country processes, such as PRSPs, and articulating ways of catalysing others to take forward replicable ideas also need improvement. However, the biggest weakness of all is the lack of any clear indicators for monitoring progress against Regional objectives.

7 The review of COSOPs finds them satisfactory in terms of corporate alignment, the degree of local ownership and stakeholder engagement, the quality of analysis and targeting 1 IFAD (1976) Agreement Establishing the International Fund for Agricultural Development Article 7 .


Page vi

issues. But while improving in quality, they still largely serve existing project ideas. They tend to be strong on analysis and on the description of targeting issues, but weaker on describing IFAD’s development niche and its catalytic role with other development partners.

Loan projects (page 23)

8 Project Cycle Management (PCM) is the process through which IFAD and borrower governments identify projects; design and formulate them; implement and supervise them; and evaluate lessons from the process in order to improve future performance. IFAD's mandated core-business is to finance projects and programmes, and the quality of its project cycle management matters greatly. The quality of IFAD's PCM is also central to the second fundamental question in the IEE TORs: Are the skills and resources of IFAD used in the best possible way, given IFAD’s overarching goal of supporting rural development and helping countries eradicate rural poverty?

9 IFAD is good at targeting. Beneficiary participation in design has improved sharply, which suggests that the relevance of IFAD investments to rural poverty reduction is high. The poverty analysis of the selected project areas is good overall. However, institutional analysis and the foundation of project design within a clear understanding of the key determinants of the rural growth and poverty reduction dynamic in the project area are less strong.

10 The definition of the project objective, outputs and the logical connection between them is satisfactory overall. But the definition of practical indicators and targets for tracking performance over time is both weak and only slowly improving. The logframe has not been applied sufficiently systematically to have improved designs.

11 There are many innovations claimed for different tiers, particularly for villagers, within IFAD projects. But until definitions are clear and well evidenced, the overall innovative content of the portfolio will be difficult to determine. Focus on cost-effectiveness and potential for replication is weak. Practical design features and targets to ensure they occur are rare. Treatment of sustainability is weak, as is appraisal of the implementation capacity, motivations and support requirements of project managers, partners and monitors.

12 Overall, the quality at entry assessment suggests IFAD projects are both over-designed and under-designed. Appraisal documents, working papers and TRC discussions address many issues in considerable depth for every project. However it is not clear that the core group of design issues – those on which project performance and impact ultimately depend – have been sharply identified and are given most attention by formulators, PDTs, TRCs and appraisers.

13 The Project Status Report process is a commendable attempt to assess project and portfolio status on a common basis. However, it is seriously hindered by the absence of empirical output performance data.

14 IFAD shares many performance standards and concerns with the World Bank. With the exception of the clarity of development objectives, where IFAD appears to have performed better, similar difficulties are experienced across a range of design features: M&E; implementation capacity and institutional analysis; and risk assessment. Performance during implementation is similarly comparable with shared concerns about the quality of supervisory monitoring and the difficulty of remedying problems during implementation.

Grants (page 41)

15 Grants are an important element in IFAD’s portfolio, both for the benefit of clients and to support programme development and management by IFAD. A large number of grants are approved every year, many more transactions than for loans. But consistency in relevance to the underlying objectives of grant funding has been undermined by frequent restatements of policy and the introduction of new instruments. A multiplicity of facilities and modalities has led to a loss of focus, lack of strategic orientation and no prioritisation. With the exception


Page vii

of the agricultural research programme, little evaluative evidence has been used to inform management before the new 2003 grant policy.

16 Good examples can be found of ways in which a variety of grants have complemented loans in support of a programme objective. But these are the exception. There is little evidence of the grants being used in ways that enable IFAD to fulfil core objectives of knowledge sharing, innovation and partnerships.

Policy (page 50)

17 A short review of policy development examines the hypothesis that In response to strategy development during the 1990s IFAD has become a leader and exerted policy influence from pioneering and innovative activities.

18 Three areas where there has been a recent push in preparing policy or operational guidance are the Decision Tools to accompany the Rural Finance Policy, the Guide to Project M&E and the Gender Plan of Action. All three initiatives involved consultative processes across the organisation. All three are quite widely quoted by staff as good examples of operational policy development. Notwithstanding these efforts, large lacunae still exist in IFAD’s internal policy framework. This has partly been exposed by the limited operational added value of the Strategic Framework and the lack of obvious mechanisms for developing policy messages from IFAD’s own rich knowledge base – its project and programme portfolio.

19 The aim of becoming a leader in rural development policy and strategy and to exert policy influence based on pioneering and innovative activities is still largely an aspiration for IFAD. In few cases do documents make it clear just how IFAD intends to pursue its leadership role – whether financia lly or intellectually. Weak knowledge systems for learning from field-based activities and a relatively weak internal policy culture make it difficult to imagine how this leadership role will develop

Knowledge management (page 51)

20 This desk review sets out to test the hypothesis that ‘knowledge sharing through innovation management, learning processes, influencing and scaling-up has improved the effectiveness of IFAD’s programme impact and policy influence’.

21 Knowledge sharing is for a purpose. But there does not appear to be a common understanding of the various formulations of advocacy work: the catalytic role, policy dialogue, innovator in know-how, leader in poverty alleviation strategies, sharing of knowledge to multiply impact, and being a knowledge organisation. There seems to be no shortage of instruments and skills available for good knowledge management. Weaknesses are in planning and monitoring of the use of the instruments. Fundamental problems originate in the poor flow of information from projects and the limited use made to inform strategies and policies.

22 The effectiveness of communications and learning is significantly reduced by the boundaries that exist within the corporate culture: vertical ones, between divisions and departments (the ‘silo’ culture); horizontal ones reflected in the hierarchical structure that divides senior management, professional and general service staff.

Partnerships (page 56)

23 Working in partnership is one means by which IFAD can contribute to achieving its strategic objectives. By partnering with others, IFAD should be more effective in delivering its strategic framework, ensuring programme impact and fulfilling its catalytic role.2 But the widespread and undefined use of the term ‘partnership’ within IFAD, combined with the

2 As stated in the VIth Replenishment document (IFAD 2002: paragraph 72).


Page viii

relatively all-encompassing objectives set out in the Vth and VIth Replenishments, may undervalue the real benefits that can be gained from working in partnership.

24 IFAD’s Strategic Framework is defined in such broad terms, that it is difficult to argue against the relevance of a great many partnerships, even though they may not present the best opportunities for contributing to IFAD’s strategic objectives. While agreements exist between IFAD and many of its so-called ‘partners’, these statements are rarely clear on the shared objectives of the relationship. In several cases there is a limited institution-wide understanding of the benefits of some key partnerships (like GM, GEF, and the Coalition).

25 Evidence of effectiveness and impact is mostly anecdotal and there is limited systematic monitoring of partnership effectiveness against shared objectives. There is no evidence that the partnerships are enhancing IFAD’s impact on rural poverty.

Human resources management (page 60)

26 IFAD’s mode of operation, with a focus on project identification and design; restriction to a single global site; extensive use of consultants to provide technical expertise; and policies to be innovative, act as a catalyst and learn from experience all place demands on personnel management, staff numbers, and corporate culture. The evaluation has focused on these three aspects, to test the hypothesis that development of highly motivated, well-trained and efficient staff has contributed to IFAD’s performance.

27 It is clear that management is working to achieve fundamental reform of human resources management. But the approach chosen is ambitious. At this stage there has not been sufficient progress for the evaluation to reach any judgement about the actual or potential achievements. But the approach taken will be reviewed later during the evaluation.

28 Despite the discipline of a zero-growth budget, there has been an effective growth of 51 percent in staffing across all categories, professional, support and consultants over the years 1994 to 2003. When compared with the loans and grants programme over the same period, a simple but revealing indicator of efficiency, the ratio of person years to new loan approvals has risen. Of course, this does not take account of changes in quality or the growth of policy advocacy. This will be considered further during the country visits. There have been improvements in the gender balance of professional staff and consultants. But women are still more likely to be found in support roles and the main growth at professional level has been within the category of Chapter X, non-permanent appointments.

29 A small sample of staff were interviewed about IFAD’s organizational culture. These enable only tentative conclusions to be drawn at this stage. But the issues raised are vital for the healthy functioning of the Fund: the need for more participatory management, better communication, team building and knowledge sharing; review of the balance between project design and implementation; documentary requirements, technical support and intellectual capital building; and the implementation of the HR strategy. In view of the importance of human resources to IFAD’s role as a knowledge-based organization the evaluation will assess the validity of these views for a wider sample of staff during the remaining period of the evaluation.

Conclusions (page 66)

30 Conclusions at this stage of the work are at best preliminary and tentative. The real test of the Fund is the impact and effectiveness of IFAD’s projects on the ground, working in a participatory way with poor people, often in remote and difficult areas. Chapter 5 sets out initial conclusions structured against the evaluation criteria, evaluation framework and key questions for the evaluation.


Page 1

1 Introduction

The Independent External Evaluation 1.1 During its deliberations, the Consultation on the Sixth Replenishment of IFAD’s Resources endorsed a proposal to have an independent external evaluation (IEE) of IFAD. Accordingly, the Governing Council decided that the evaluation should be planned and begun in 2003 and completed in 2004, in time to allow for full consideration of the IEE Report by the Executive Board, prior to its recommendation to the Governing Council on the Consultation on the Seventh Replenishment of IFAD’s Resources. Following a competitive international tender, ITAD Ltd was contracted to undertake the assignment. This document is the Desk Review Report for the evaluation and responds specifically to Task 2 of the terms of reference.

1.2 The desk review is a phase of the evaluation during which the team has been based in Rome to gather information from documentary sources and staff at IFAD. The wide scope of the terms of reference and demanding requirements for reviews of projects has limited the possibility of interaction with clients and partners, which will take place during the next phase of the work. Although the terms of reference call for a report at the end of the desk phase, this report should be read as a statement of findings at a point in time, rather than in conclusion of any particular line of enquiry. A key role of this report is to highlight the issues that appear to be important and will be developed in the later stages of the evaluation.

Objectives of the evaluation

1.3 The objectives of the evaluation are set out in Section II of the Terms of Reference, and state:

‘The main objective of the IEE is to determine IFAD’s contribution to rural poverty reduction, the results and impact it has achieved in this area, and the relevance of the organization’s mission and objectives in relation to international development goals and the national development strategies of IFAD borrowing countries. The evaluation is further expected to assess whether and what IFAD has learned from past experience and how the Fund’s policies and operations have evolved in response to lessons learned from that experience, and finally, to offer recommendations on the policy directions IFAD should pursue and other steps it should take to improve its future performance.’3

1.4 This objective encapsulates three strands:

• development results and impact in regard to rural poverty reduction;

• the corporate learning process and performance of policy and operations; and,

• forward looking recommendations to improve future performance.

1.5 The work undertaken to date has been to prepare the groundwork for evaluation of development results and impact that will be undertaken during the country visits. The main findings at this stage concern corporate learning and performance of policy and operations. Even here, there are issues that have been identified that require deeper investigation during the next phase of work. Clearly, it is too early to consider recommendations, but the reader will find pointers to issues that the IEE team consider will be important in the final analysis.

3 Annex 1. IFAD (2003) Independent External Evaluation of IFAD, Terms of Reference, Para. 3 Rome


Page 2

Structure of the report

1.6 The report reflects the outline structure presented in Table 11 of the Inception Report, with some small changes. Presentation of findings is organized into two main sections, dealing first, with IFAD’s corporate performance with its strategies, loans and grants, and then examining key corporate processes – policy development, knowledge management, partnerships, and human resources management. These topics are drawn from the evaluation framework developed during the Inception Phase. Chapter 2 summarizes the work that has been done and identifies those aspects that will be followed-up during the next phase. Chapter 3 is the evaluation of corporate performance. It starts with a review of the loans portfolio and draws some tentative conclusions about resource allocation and selectivity. The next section analyses strategy development since 1994. After that comes the analysis from a sample of 42 projects, to identify current performance in loan project identification and formulation. A short treatment deals with the evidence about implementation and supervision that could be gleaned from IFAD headquarters. There is also a summary of available information on project outcomes. The next section reviews grants, with an overview of the grant portfolio and specific analysis of four technical assistance grants. Chapter 4 presents the evaluation of corporate processes: policy development, knowledge management and learning, organizational partnerships and human resources management. Chapter 5 presents conclusions.

1.7 A large body of annexes supports the main report. Annex 1 is a bibliography of the main documents reviewed. Some additional references can be found in the other annexes. Annex 2 is a list of staff interviewed to date. Extended analysis is presented for grants (Annex 4), knowledge management (Annex 6), organizational partnerships (Annex 7) and human resources management (Annex 8). A short note on research evidence about poverty efficiency of aid allocations is reported in Annex 3. Annex 5 contains a summary of compliance against the Vth and VIth replenishments. Annex 9 (distributed separately) is a comprehensive summary of country COSOP and sample project information drawn from the desk review for 21 countries and 42 projects. The primary aim of these country synopses is to act as a resource from the desk review to help inform the later country visits.

2 Work completed and methodologies

2.1 The main focus of work during the desk phase was to analyse a sample of projects. The terms of reference for the assignment prescribed analysis of a sample of 20 to 25 countries and 40 to 50 projects, half of which will be visited in the next phase. The approach to sampling was explained in the Inception Report together with the selected countries and projects. A stratified random sampling scheme was used with probability proportional to numbers of projects in each region. The majority of the time spent by the IEE team was devoted to documentary review of these projects and meetings with CPM and other operations staff. More details are given in Chapter 3 (paragraph 3.48). That effort has meant that other issues such as policy analysis, human resources and management will require more attention during the next phase.

2.2 The data gathering, analysis and interviews all followed the approach set out in the terms of reference. Key elements are:

• Review of design, implementation and evaluation documents about 42 projects from 21 countries using structured checklists

• Interviews with relevant Country Portfolio Managers and other operations management and technical staff based in Rome, in individual and group settings

• Review of a sample of 4 TAGs and databases about the grants portfolio


Page 3

• Review of policy and strategy documents and interviews with key staff to chart the evolving policy environment and examine progress made against the Vth and VIth Replenishments

• Review of knowledge management policy and initiatives, interviews with operations staff and three focus groups

• Partnerships – interviews with focal points for a wide range of arrangements, and review of documentation

• Human resources and management. Interviews with staff in HR division, analysis of statistical data and interviews with a small sample of staff of various grades

• Three open forum meetings available to any staff wishing to participate, to discuss topics of importance to the group.

• Meeting with President Båge In addition, preliminary arrangements have been made for the visits to ten countries. 2.3 The focus within IFAD and the pressure of work from the project reviews has resulted in a number of activities being scheduled for later in the evaluation. Specifically:

• Interviews with consultants. Consultants are a vital element in IFAD’s resources and mode of operation. We plan to interview a sample but have deferred this work for two reasons: to ensure a random sample from a listing provided by HR Division; and to have a clear understanding of our findings from the desk review as a basis for questions.

• Interviews with Directors on the Executive Board. For similar reasons to consultants it was deemed desirable to await a better sense of key issues about IFAD’s governance.

• Human resources and management. As will be seen in Chapter 4, the preliminary findings about staffing levels suggest there is a need to understand the recruitment and disposition of staff and consultants better. Access to data is not straightforward and will require further time and additional inputs during the next phase.

• The extent of policy influence and understanding of IFAD’s niche are important issues for partner organisations. These will be undertaken in conjunction with regional travel during the next phase of work.

• Work so far suggests that it is important for the evaluation to analyse the actual performance associated with re-engineering and resource allocation to the project cycle. The team plans to analyse expenditure under the PDFF but this will require more time to develop the necessary databases.

• Two examples have been given of benchmarking IFAD’s performance: in comparison with the World Bank for project quality at entry (see paragraph 3.106) and against a group of donors for knowledge management (see Annex 6). As more information becomes available about costs and human resources, benchmarking will be extended across other IFIs, to the extent that their published data permits.

• In view of the importance of the re-engineering and strategic change programme to IFADs corporate performance in the past tem years, we plan to explore this work in more detail, in collaboration with the Vice President.

3 Evaluation of corporate performance

3.1 This chapter sets out the results to date of investigations into IFAD’s portfolio of loans and grants. It responds primarily to items 2.1 – Resource Allocation, part of 2.2 Policy and Strategy Development and 2.4 Project cycle management in the evaluation framework. Both these topics support the overarching questions of IFAD’s impact, through policy influence and projects. But because this stage of the work was based on secondary sources of


Page 4

data and interviews with IFAD staff, this report contains few insights into impact. That will come after the next phase of work and country visits. The chapter starts with a review of the loans portfolio and continues with sections on loan projects and grants.

Portfolio analysis 3.2 IFAD’s mandate states that eligibility for assistance should be on the ‘..basis of objective economic and social criteria with special emphasis on the needs of the low income countries and their potential for increasing food production, as well as due regard to a fair geographic distribution in the use of such resources’4 In this section we present an overview of how IFAD’s portfolio has evolved in the light of its mandate, focusing specifically on trends and the composition of lending for the review period 1994-2003. More detailed analysis of the portfolio and of portfolio performance at country and project level will take place alongside the country visits in phase two.

Trends in Project Financing by Source

3.3 Figure 1 summarises the trend and composition of total project financing for the review period. The data show a fairly stable volume of project financing up to 1997 followed by a period of increased volatility. In 1998 total project finance almost doubled over its 1997 level only to fall back significantly in 2000. Much of the volatility seems to be explained by variations in the level of co-financing; although the annual amount of IFAD-only financing also appears more changeable since 1999.

Figure 1 Total Finance by Source – 1994-2003

4 IFAD (1976) Agreement Establishing the International Fund for Agricultural Development Article 7 .


Page 5

3.4 Figure 2 illustrates the share of total project financing by source for all projects 1994-2003. On average about 42 percent of financing for the period came from IFAD loans,5 38 percent from other loans and grants and 29 percent from domestic sources. Around 2 percent comes from sources ‘yet to be determined’ and relates to projects that have only just reached loan effectiveness and have not got all sources of finance completely pinned down. The data suggests that between 1994 and 2003, IFAD reached its target of increasing co-financing to at least 30 percent of its lending programmed,6 although in annual terms the share has fluctuated quite widely.

Figure 2 Total Finance – Shares by Source

Financing by Region and Source

3.5 Tables 1 and 2, and Figures 3 to 5 summarise project financing by region, year and source. Table 1 shows the distribution of IFAD project financing for the entire project loan period (1978-2004) and for the review period. Noteworthy is the stability of regional shares when looked at over the entire period. This suggests that, in line with the IFAD mandate, regional financing has in fact been relatively equitable. This is further confirmed by Table 2 which shows a high degree of consistency in the number of projects financed per region and average project size over the review period.

5 The share of IFAD grants is too small to pick up, mainly because these are disbursed not at project level but at regional or sub-regional level and are not enumerated in the project portfolio data. 6 See Consultation Report on Replenishment VI.


Page 6

Table 1 IFAD Financing by Region 1978-2004

Period PA PF PI Pl PN Total

1978-2004 1120.8 1190.3 2257.3 1132.0 1195.3 6995.6 % 17.5 17.0 32.3 16.2 17.1 100 1994-2003 613.7 690.3 1058.7 658.4 644.3 3665.8 % 16.7 18.8 28.9 17.9 17.6 100 Table 2 Number of projects and project size by region 1994-2003

Region Number of Projects 1994-2003

Average Project Size (US$ million)

PA 55 25,942 PF 57 27,495 PI 65 39,952 PL 51 24,080 PN 50 34,565 3.6 Within regions, however, total project financing has proved more volatile. Almost all regions saw a significant spike in financing in 1998 in line with the aggregate picture, but equally, several saw large falls in 2000 and further spikes in 2001 (Africa I and LAC). Again levels of co-financing appear to be an important driver of variability within regions and across years; although not all regions rely equally heavily on co-financing.

Figure 3 Total Project Financing – Africa I and II 1994-2003


Page 7

Figure 4 Total Project Financing – Asia & Pacific and LAC 1994-2003

3.7 Notable is the relatively small share of co-financing in LAC projects (particularly since 1998), compared with the two Africa regions and even Asia and Pacific. The size of the domestic contribution is correspondingly higher in LAC, but is also relatively high in Asia and the Near East. More in-depth analysis of the portfolio in phase 2 is required to understand the reason for these variations.

Figure 5 Total Project Financing – Near East & North Africa 1994-2003


Page 8

Financing by Economic and Social Criteria

3.8 As well as maintaining a ‘fair geographic distribution’, IFAD’s mandate states the importance of determining eligibility for assistance on the basis of objective economic and social criteria. To assess the extent to which IFAD’s portfolio has evolved in line with what today might be regarded as objective economic and social criteria, we examined the relationship between IFAD project financing and a series of ‘structural’ country-level criteria, including the relative size of the rural population and the size of the agricultural sector in national income. Figure 6 shows the relationship between the share of a country’s population that is rural and the value of IFAD financing for the review period. Except for a number of outliers, which include India, China and Bangladesh, the relationship is barely visible. The same holds for Figure G, which plots the relationship between IFAD financing and agricultural value-added as a percentage share of GDP. One possible explanation is that the principle of maintaining a ‘fair’ geographic spread of financing – which limits not only the number of projects per country but also the average size of projects - is more powerful in determining the pattern of overall financing than any structural criteria. Another explanation is that the criteria are simply too broad to reveal any meaningful relationships, given the relatively small scale of IFAD country operations.

Figure 6 IFAD Financing by the Share of the Rural Population


Page 9

Figure 7 IFAD Financing by Agricultural Value-Added (% GDP)

3.9 A possibly more sophisticated measure of economic and social criteria is the World Bank’s Country Policy and Institutional Assessment index (CPIA). Although the index is restricted to only low-income countries (those receiving IDA), it still covers a large number of IFAD-relevant countries. Figure 8 plots total IFAD financing for 1994-2003 against the five quintiles of the CPIA index. The results shown for 1994 and 2003 suggest that total IFAD project financing in low-income countries is generally oriented towards those with medium to good policy and institutional performance (categories 1&2), and that this has improved over time. There has also been a slight increase in the share of project financing in the poorest performing countries (category 5) which may be due to increased funding for countries emerging out of conflict. Figure 9 shows the level of IFAD financing per project beneficiary household according to each CPIA quintile for 2003. The results show a similar pattern, but less clearly.

3.10 Without more detailed analysis of the portfolio at this stage it would be premature to comment in detail on these results, but recent work by Beynon (2003) (Annex 3) suggests that overall the trend towards more efficient aid allocations in the 1990s was largely due to a combination of falling aid levels and changing policy performance and not due to changes in the pattern of aid allocations by donors. This holds true for the majority of multilateral and bilateral partners, leading to the hypothesis that for IFAD the increased share of project financing going to better performing countries in 2003 is largely due to changes in borrower characteristics rather than deliberate shifts in country allocations. This hypothesis will be assessed in more detail, and in relation to the new Performance Based Allocation System, in the next phase of work.


Page 10

Figure 8 IFAD Financing and the CPIA 1994 and 2003

Figure 9 IFAD Financing per Beneficiary Household and the CPIA 2003

Summary of issues

• In what is inevitably a rather general analysis at this stage, there seems to be preliminary evidence that IFAD’s portfolio has evolved in line with its original mandate. IFAD project financing is more or less equitably distributed across regions


Page 11

and seems to be oriented towards the better performing countries, with some important exceptions. But the fact that a clear picture does not emerge raises fundamental questions about the clarity of IFAD’s niche.

• Total project financing is more volatile however, driven by what appear to be significant variations in the level of co-financing over time and between regions.

• What this analysis has not done is look at the non-lending elements of the portfolio, nor has it attempted to link evidence on portfolio performance with the data on financing. These are two critical tasks for Phase Two.

Corporate Strategy Development

Introduction

3.11 This section presents an analysis of the evolution and impact of corporate strategy development in IFAD. The analysis focuses on the period since 1994. The main hypothesis to be addressed is:7

i. The adoption of strategic policies during the 1990s has set project activity in a systematic fashion within a broad country and inter-organisational context.

3.12 The section starts with a brief review of IFAD’s mandate and comparative advantage in a changing global operating context. It goes on to consider the main features of the two latest Replenishment Rounds (V and VI) and their impact on corporate strategy and policy development. The next section reports results of the desk review of the three main elements of the current corporate strategy framework – the Strategic Framework, Regional Strategies and COSOPs.

IFAD mandate and operating context

3.13 The basic mandate of IFAD is set out in the Agreement Establishing IFAD. Article 2 states that:

‘the objective of the Fund shall be to mobilize additional resources to be made available on concessional terms for agricultural development in developing Member States. In fulfilling this objective the Fund shall provide financing primarily for projects and programmes specifically designed to introduce, expand or improve food production systems and to strengthen related policies and institutions within the framework of national priorities and strategies….’

Article 7 provides further guidance: ‘In allocating its resources the Fund shall be guided by the following priorities:

i. The need to increase food production and to improve the nutritional level of the poorest populations to the poorest food deficit countries;

ii. The potential for increasing food production in other developing countries. Likewise emphasis shall be placed on improving the nutritional level of the poorest populations in these countries and the conditions of their lives.’

7 See Table 7 of the IEE Inception Report


Page 12

3.14 Two earlier external evaluations (Rapid External Assessment (REA) 1994 and External Review (ERT) 2002) have confirmed the continuing relevance of IFAD’s mandate within the global effort of poverty reduction, but at the same time note the challenges faced by the organisation in maintaining a clear and coherent development niche. The most recent evaluation states that ‘Demand for assistance to the large populations of the rural poor will continue for decades to come, leading to the question: will IFAD be able to play the role of premier international financial and development institution in its field – in which its partners look for continued and increased leadership?’(ERT, p28). The view of the ERT was that it could if the organisation remained committed to sustained progress in key areas – developing a more systematic approach to innovation; expanding partnerships and exploring country presence and knowledge management - and had adequate human and financial resources at its disposal. It is currently too early in the IEE to say whether such progress has been fully sustained (although a report on progress against successive replenishment undertakings is provided in Annex 5), but the question itself remains important, in particular, whether IFAD’s interpretation of its mandate continues to provide it with the necessary strategic direction to be an effective partner and leader in the global effort for rural poverty reduction.

3.15 What is clear is that the period since 1978, when IFAD was founded, has seen unprecedented change in the global context for agricultural and rural development. What was once a clearly defined niche for IFAD to be operating in has become a great deal more complex with many more actors and many more ideas about what should be done to bring about sustained growth and rural poverty reduction. The earlier focus on food production has been eclipsed by a whole array of new thinking about the diversity of livelihoods that make up the rural space. The heterogeneity of rural areas themselves, particularly between high potential and low potential, well-connected and weakly connected, and peri-urban or remote areas has become an important stylized fact in the ‘new thinking’ about rural development (Ashley and Maxwell 2001). Equally significant is acknowledgement that the character of rural space is being forever changed by demographic transitions, the spread of market relations, technical change and the gradually shrinking contribution of the agricultural sector to national GDP, export earnings and tax revenue (Maxwell 2003).

3.16 The context for international aid has also changed dramatically with new and multiple pressures bearing down on donors and lenders (IDS 2000). Multilateral institutions have been subject to particular pressures in the context of global protests and efforts to rethink the international aid architecture. In particular, they find themselves responding to simultaneous calls for greater openness and transparency, more effective regional and global leadership, new and more strategic partnerships, more decentralisation, greater operational impact on the ground and greater focus on global development goals; all, very often, within increasingly tight administrative budgets.

Strategy and change processes

3.17 Since 1994, IFAD has responded to these pressures in several ways. In fact as Table 3 suggests, the decade since 1994 has been marked by almost continuous efforts to ‘re-wire’ the organisation to the changing global context. Amongst the most significant in strategy terms has been the introduction of Country Strategy and Opportunities Papers (COSOPs), changes to the project management cycle, the increased focus on impact achievement, the development of the Strategic Framework 2002-6 with its emphasis on policy dialogue, market access and empowerment, and the introduction of Regional Strategies.

3.18 The overall impact of these changes, particularly since the early 2000s, has been a significant improvement in corporate strategic alignment. But has IFAD changed fast enough in recent years and in the right directions? Has it acted responsibly to implement commitments undertaken in Replenishment Rounds? Has the new corporate strategy process with its strategic framework, regional and country strategies produced the desired results in


Page 13

terms of the relevance, effectiveness and impact of what IFAD does? This stage of the IEE begins to address some of these questions in a very preliminary way based on an initial desk review of strategy and change documents, replenishment undertakings and staff interviews.

Table 3: IFAD Strategy and Change Processes 1994-2004

Year Strategy/Process Objectives/Highlights

1994 -1998 Re-engineering Programme Strategic management process, corporate score card, Country Strategic Opportunities Papers (COSOPs), Project Portfolio Management System (PPMS).

1995 IFAD Vision To provide leadership; work through partnerships; be a catalyst to mobilise resources, and develop as a knowledge organisation.

1998-2000 Meeting Challenges in a Changing World: IFAD’ Strategic Framework 1998-2000

Five thrusts identified: (a) Support projects and programmes driven by beneficiary participation; (b) Create an effective portfolio management system that ensures desired field-level results (c) Ensure an effective IFAD presence and impact at the field level (d) Establish and use knowledge networks on rural poverty in order to create a recognized knowledge organisation (e) Develop human resource and management systems that support accountability, teamwork and decentralised decision making and other goals.

2000-2005 Process Re-engineering Programme (Strategic Change Programme)

To intensify IFAD’s impact and its role as a knowledge organisation. IFAD business process architecture; new IT and information systems; human resources. Introduction of log frame, key file in project formulation; strengthening annual portfolio review.

2000-2002 IFAD V Plan of Action

Accompanied Vth Replenishment. Actions designed to bring about impact in four areas: policy and participation; performance and impact, innovation and knowledge management; partnership building

2002-2006 Strategic Framework 2002-6 ‘Enabling the Rural Poor’

Mission: To enable the rural poor to overcome their poverty. Three strategic objectives: (i) to strengthen the capacity of the rural poor and their organisations; (ii) to improve equitable access to productive natural resources and technology; and, (iii) to increase access to financial services and markets. In relation to these objectives the Fund will reinforce its catalytic role both through its field operations and through policy dialogue and advocacy.

2002 Regional Strategies Intended to serve as a framework for IFAD activities at regional level, as a basis for dialogue in the region and as a benchmark against which impact will be assessed.

Box 1 The Strategic Management Process – A Lesson in Corporate Strategy Making.

Following the initiation of the re-engineering process in IFAD in 1994, the Re-engineering Task Force identified strategic management and within that corporate strategy as an important process for fulfilling IFAD’s mandate. In response a Corporate Strategy cell was created to facilitate the process of development of corporate strategy and strategic management. An early outcome was the publication of IFAD’s first Strategic Framework for 1998-2000. Later outputs included the introduction of the Balanced Scorecard 8 as a tool for monitoring corporate performance against strategic objectives (Thrusts). While both the Strategic Framework and the Scorecard had a number of acknowledged strengths, in particular in helping to articulate the strategic agenda more clearly neither really ever got

8 This was about the same time as other IFIs, in particular the World Bank were experimenting with Corporate Score Cards.


Page 14

off the ground in IFAD. Reasons why reflect on the wider SMP which was deemed by staff to be insufficiently participatory, to lack crucial linkages with the budget, and to have weak linkages with other aspects of the re-engineering process, most notably the establishment of integrated IT systems. The Scorecard itself was seen by staff as a reporting device rather than a management tool, as yet another administrative burden running in parallel, instead of in tandem, with other initiatives (such as the 2000-2002 Plan of Action), and too abstract to be meaningful for decentralised performance monitoring. Notable during interviews was how few IFAD staff actually remembered much about the Strategic Framework of 1998-2000 or the Scorecard, both having being eclipsed by the new and much more participatory Strategic Framework of 2002-2006 and plans for a new and more integrated Strategic Planning and Budgeting System.

Replenishment trends

3.19 Before IFAD V, agreements between IFAD and its donors focused heavily on resource concerns, issues of eligibility and general endorsement of IFAD’s ongoing and planned activities. The extensive policy content of IFAD V and even more so of IFAD VI represented a major change.9 IFAD V and VI consultations called for a major intensification of efforts to enhance the poverty impact of IFAD assistance, or as described in IFAD VI ‘to enable the rural poor to overcome their poverty’.

IFAD V

3.20 The main focus of IFAD V was to strengthen the Fund’s role ‘as a leading source of knowledge on the eradication of rural poverty’. The Consultation noted the importance of IFAD maintaining its multilateral character but ‘not going it alone’. It emphasised the overriding importance of strategic partnerships and, because of the Fund’s small size and resource base, its catalytic role in deploying its scarce resources through such partnerships. The Consultation reiterated IFAD’s four major roles in transferring resources: as innovator, as knowledge institution, as catalyst and as leader, and urged IFAD to strengthen its comparative advantage. Specifically IFAD should:

i. enhance its participation in policy dialogue and analysis in relevant areas; ii. take a more structured approach to documentation and evaluation of its field-based

innovations to facilitate their adoption by other partners, and; iii. step up its efforts towards building strategic partnerships that have the potential for

the replication and scaling up of best practices in poverty alleviation (IFAD 2000) 10

3.21 Consultation recommendations focused on enhancing the Fund’s comparative advantage and ability to ‘influence the direction of policies and resource flows to address rural poverty and household food security’ (IFAD 2000). Recommendations were presented as a Plan of Action showing how each would be incorporated into IFAD’s policy framework and management tools. The Plan of Action spanned 2000-2002 and became a crucial tool for senior management in taking forward quality improvements in IFAD performance at project, country and strategy levels.

3.22 Annex 5 provides a summary of progress to date against some of the key commitment areas in the Plan of Action. The evaluation is partial at this stage, relying only on the outcome of the desk reviews. Progress in meeting Plan of Action commitments is largely satisfactory with some preliminary evidence of a positive impact on the portfolio. Results from the desk 9 To the extent of taking IFAD management rather by surprise, some argue. 10 IFAD (2000) Partnerships for Eradicating Rural Poverty. Report of the Consultation to Revie w the Adequacy of the Resources Available to IFAD 2000-2002. GC 24/L.3.


Page 15

review of projects show significant improvements in project design and quality in the latest cohort covering beneficiary engagement in project design and implementation, the quality of analysis and attention to sustainability issues (albeit from a low base). The review of COSOPs, which is reported in detail further on, confirms improvements in the quality of analysis informing COSOPs, greater evidence of local ownership and participation, and greater attention to opportunities for pro-poor policy dialogue in COSOPs formulated since 2001.

3.23 Areas where progress is less certain include linking COSOPs with the strategies of other donors and in-country coordination frameworks, particularly around the PRSP/MDG process. Providing clear definitions of what it means to be innovative 11 and to act ‘as a catalyst’ – which seems in most COSOPs to be entirely limited to efforts to raise co-financing rather than acting as a knowledge sharer or leader in best practice solutions for rural poverty reduction.

IFAD VI

3.24 The Consultation surrounding IFAD VI, which covers the period 2004-2006 represents a major push to link IFAD strategy more forcefully to the global consensus around the MDGs and to the poverty and policy performance of borrowing countries. Critical is the operationalization of the Strategic Framework 2002-2006 and Regional Strategies; the linking of resource allocations to performance through the creation of a performance based allocation system (PBAS); and enhanced selectivity and effectiveness at the operating level through COSOPs; the possibility of greater field presence; a clearer programme approach for country portfolios; and working with efforts to harmonise donor support around national poverty reduction strategies (PRSPs).

3.25 Most of the actions proposed under IFAD VI are still underway although progress is being made in devising the PBAS and plans are for it to come on stream during 2004. A Results-Impact Monitoring System (RIMS) is also in development, which should provide a comprehensive results management system for both new and existing IFAD operations. Common indicators will be introduced into new projects in 2004 and be fully effective for the replenishment period 2004-6.

3.26 Efforts to improve the strategic and operational relevance of COSOPs continue but weaknesses persist in linking the assessment of national and sectoral policy context to the project pipeline and in using the log frame to articulate clear and monitorable programme objectives. This brings concerns that COSOPs are not fulfilling a strategic function over and above defining the project pipeline. Engagement with PRSP processes is also lagging behind that of other donors, in part because of IFAD’s weak field presence but also possibly because of the timing of IFAD’s strategy and project appraisal processes.12 The Gender Action Plan has been prepared and an initial review of progress has been included as the part of the Progress Report on the Portfolio (PRPP). The review suggests major advances in mainstreaming through inclusion of specific sections on gender equality and women’s empowerment in the PSR format, RIMS, PBAS and New Framework for Evaluation, and completion of the Baseline Survey. With the introduction of the new format 75 percent of 2003 PSRs reported specifically on gender differentiated participation in and benefits from

11 The Annual Report on Results and Impact of IFAD Operations (2002) cites features of innovations that work in IFAD projects that include ‘they addressed a need widely shared by the poor’, ‘the cost of adopting them was affordable, financially and socially’, ‘the correct policy environment facilitated them’ 12 At present IFAD country strategy and programming processes are geared around internal Board requirements rather than fitting in with country-led strategy and budgeting processes.


Page 16

project activities (against 31 percent in 2002). IFAD also continues to advocate on gender issues in global fora.

Strategic framework 2002-06

3.27 Unlike its predecessor, the Strategic Framework 2002-6 ‘Enabling the Rural Poor’ was prepared in a participatory manner. Although the drafting time was short (6 weeks) there is a strong sense of ownership by staff across the institution. The mission statement and the three strategic objectives provide staff with a common reference point and a clear basis for communication both internally and externally. With the added introduction of Regional Strategies staff working in PMD note a clearer sense of mission and purpose.

3.28 But the Strategic Framework also lacks in some important respects. Staff argue that it does not provide a meaningful guide to operational selectivity or effectiveness. The intellectual and policy framework is largely permissive, ruling almost everything in and very little out. Its impact on projects, as described later in this chapter, is hard to discern and it would be hard to rule out ‘fit’ in almost all cases. Links with COSOPs are equally opportunistic. Externally the Framework does not specify clear links between the achievement of strategic objectives and progress towards the MDGs 13 and there is no substantive reference to how IFAD plans to engage with PRSPs, or how it sees its role in contributing to global and regional public goods.

3.29 The Strategic Framework has served a crucial purpose in helping to shape IFAD’s corporate identity amongst staff, but a key challenge remains in how to balance the ‘uniqueness of IFAD’s mission’ with the mounting pressures on the multilateral system to be responsive to global agendas, while at the same time adding operational value-added in terms of selectivity and effectiveness.

Regional strategies

3.30 Regional Strategies were introduced in 2001-2 as a critical bridge between the Strategic Framework and COSOPs. Six Regional Strategies were prepared in early 2002 (two covering the Middle East, Europe and North Africa Region). Initially the idea was to base the Regional Strategies on the Regional Poverty Assessments commissioned as background work for the Rural Poverty Report 2001, but at the time of writing the Strategies not all RPAs were complete. However, the assumption is that the Regional Strategies will be read alongside the more comprehensive RPAs.

3.31 Relevance The objective of Regional Strategies is to ‘..serve as a framework for all IFAD activities’, to provide ‘the basis of IFAD’s operations and dialogue in the region’ and ‘as the benchmark against which impact will be assessed.’ The desk review finds that Regional Strategies are performing a critical function, not least in filling part of the strategy/operational vacuum left by the Strategic Framework. The Project Review notes that the Regional Strategy is the most influential internal determinant of project identification post 2002. Forty-eight percent of all projects reviewed and 92 percent of those post 2002 show a close fit with the relevant Regional Strategy.

3.32 The relevance of Regional Strategies in terms of the corporate agenda, and in particular the Strategic Framework, is high although not always described in much detail. Their relevance in terms of the wider global agenda of MDGs is stated but not always clearly articulated. Most draw links between IFAD strategy and MDG 1 and the need to concentrate resources in a few areas where the rural poor are, but few give more detail. Links with

13 See discussion of Consultation on Replenishment VI above.


Page 17

country level PRSP processes or related regional initiatives are poorly spelled out, although the coordinating function of the PRSP is mentioned as a key focus of future IFAD work at country level.

3.33 Regional Strategies are strong on the quality of analysis, the clarity of objectives (although it is less clear that these constitute a good guide to resource allocation) and for setting out principles to guide country programming. In Eastern and Southern Africa, for example, the Regional Strategy sets out a marker for future programming based on larger and longer-term programmes oriented around institutional strengthening, development and reform of the country policy and legislative and strategic framework. Emphasis is given to local ownership in the programme design process and the role of IFAD in being more strategic, providing support and broad guidance rather than the more traditional project appraisal role. The Near East and North Africa Strategy sets out similar principles, emphasising a programmatic rather than traditional project approach. As such Regional Strategies mark a potentially important part of IFADs shift toward upstream lending and policy dialogue.14

3.34 Regional Strategies are weaker in their recognition of other international best practice and knowledge about growth and rural poverty reduction at Regional level. The references to Regional grants and knowledge dissemination activities fall short of any discussion of regional public goods, in particular regional capacity building efforts and technology transfer. Few Regional Strategies expand on IFAD’s catalytic role beyond the prerequisites of leveraging co-financing. Strategic alliances and partnerships are described in some detail but more in aspirational terms than in terms of concrete relationships, the exceptions being references to the Global Mechanism and the Popular Coalition to Eradicate Hunger, both of which IFAD houses. However, the biggest weakness of all is the lack of any clear indicators for monitoring progress against Regional objectives. Only one Regional Strategy (Latin America and the Caribbean) contains any reference to indicators to measure success in implementing the strategy, none of these are specific or quantified and there is no obvious process for following through on monitoring. If a major aim is to use Regional Strategies ‘as the benchmark against which impact will be assessed’ then it is unclear as to how they will succeed.15

Country Strategic Opportunities Papers (COSOPs)

3.35 COSOPs were introduced as part of the Re-engineering Programme in 1995 to provide a medium term strategic framework for country operations. The aim was not to be comprehensive but to focus on IFAD’s specific role, niche and future direction in the country. Over time the guidance on COSOPs has been refined. The Consultation Report for IFAD V states that ‘..COSOPs should ensure that: (a) lending operations have a strategic focus and are not the result of ad hoc decisions on project entry into the pipeline; (b) corporate and regional strategic priorities are adapted to the country polic y context and a synergy is established between the organisation’s strategic thrusts and the government’s policy and planning framework for rural poverty eradication; and (c) opportunities for IFAD’s operations over the medium term are in keeping with its suggested framework’ (IFAD 2000).16

14 Notable is a presentation given by the Eastern and Southern African Regional Directorate at the Governing Council Meeting in February 2004 setting out the case for IFAD further engaging in Sector Wide Approaches (SWAPs) linked to national PRSPs. But the first reference to Sector Wide Approaches is actually to be found in the Tanzania Country Programme Evaluation, dated November 2002. 15 Over time this monitoring may be picked up by the RIMS, but at the time of writing this was unclear. 16 IFAD (2000) Report on the Adequacy of Resources 2000-2


Page 18

3.36 Further guidance appears in 2002.17 The role of the COSOP is described in terms of articulating the relationship between the Fund’s Strategic Framework, the respective regional strategy and the proposed direction of interventions at country level. The aim is to identify strategic opportunities for IFAD project intervention, linking with PRSPs, the UNDAF and other country processes; ‘..it should not be developed to simply serve existing project ideas’.

3.37 Twenty one countries were sampled in total for the IEE, of which 20 currently have COSOPs.18 As part of the desk review all 20 COSOPs were reviewed against a checklist of thirty three questions and a 5 point scale. Consistency of assessment was sought through intensive initial discussions, standardisation of each review question and continual cross-checking during the review process. The ratings were analysed within a custom written data base.

3.38 Relevance Table 4 summarises key results for all COSOPs in the sample.19 Each of the review themes represents an aggregation of checklist questions. The main finding is the high degree of consistency between COSOPs and IFAD corporate strategy (the Strategic Framework) and, where available, Regional Strategy. Ninety percent were rated satisfactory; the 10 percent not rated represent COSOPs either prepared before the current Strategic Framework or Regional Strategy were in place or before there was a clear requirement for COSOPs to link to these. Figure 10 breaks the results down into two cohorts – 1998-2000 and 2001-2004. The most recent COSOPs demonstrate closer fit with corporate and regional strategy in line with corporate expectations and replenishment commitments.

3.39 In terms of demonstrated consistency with national priorities and poverty reduction strategies the COSOPs reviewed perform less well, with just 50 percent being rated satisfactory. However the percentage of not rated COSOPs is relevant here, indicating the large number of COSOPs prepared prior to the finalisation of national PRSPs. Figure 10 confirms that the later cohort COSOPs do better in demonstrating consistency with national poverty reduction priorities, but with still considerable room for improvement. Many, for example, will refer in general to working with PRSP initiatives but do not set out clear or obvious pathways for deeper IFAD engagement.

Table 4 Review of COSOPs – Selected Themes: 1998-2004

Selected Review Themes

Percent Satisfactory

Percent Unsatisfactory

Percent Not rated

Relevance Consistency with IFAD corporate & Regional Strategy

90 0 10

Consistency with national priorities/poverty reduction strategy (PRSP)

50 5 45

Description of IFAD development niche

60 40 0

Effectiveness Opportunities for pro-poor dialogue 65 30 5 Evidence of local ownership/ stakeholder engagement

50 30 20

Quality of upstream diagnosis and analysis

63 24 13

Targeting 75 20 5 Clarity, consistency of goals, objectives and indicators

24 38 38

Efficiency Identification of strategic alliances/ partnerships

70 25 5

17 Procedure for the Review of COSOPs by the Executive Board (2002) 18 Chile, also a sampled country, did not have a COSOP at the time of the review. 19 The results are based on a small sample and should be interpreted accordingly. Data have been aggregated wherever possible to create a critical mass for analysis.


Page 19

Selected Review Themes



Percent Not rated

partnerships Rational for innovation & scaling up 45 33 22 Leveraging and catalysing others 35 30 20

3.40 Perhaps the most striking finding is the relatively high share of COSOPs that did not provide an adequate description of IFAD’s development niche in the country (Table 4). Perhaps even more surprising is the fact that later COSOPs are rated more poorly on this theme than earlier COSOPs. (Figure 10). One possible explanation is that the demands on COSOPs to demonstrate strategic value-added have increased over time and so has the complexity of defining a clear development niche. While Regional Strategies have provided a clearer strategic direction for country programmes as a whole, COSOPs have yet to catch up in identifying a clear strategic role for IFAD at country and project level. Comparisons between COSOPs from the earlier part of the period and those at the end of the decade, in Figure 10, reveal improvements in consistency with IFAD strategy, national priorities and the engagement of stakeholders. But interviews with staff underscored concerns that COSOPs are not yet functioning as strategic documents linking project interventions to country wide programmes and processes. Results in Table 5 confirm a very high congruence between project approaches and COSOPs, but mainly because projects are driving the country strategy rather than the other way around.20

Table 5 Congruence of IFAD projects, COSOPs and Regional Strategies 1998-2003

Review Themes



Percent Not rated

Fit between project approach and COSOP (or equivalent)

83 0 17

Fit between project approach and Regional Strategies

48 4 48

20 See Chapter 3.


Page 20

Figure 10 COSOPs coherence with strategy – percent satisfactory over two time periods

0

10

20

30

40

50

60

70

80

90

100

Consistency with IFADStrategy

Consistency with nationalpriorities/poverty reduction

strategy

Description of IFADdevelopment niche

Evidence of localownership/stakeholder

engagement

Per

cent

Sat

isfa

ctor

y

1998-2000

2001-2004

3.41 Effectiveness The analyses on which COSOPs are based is rated consistently satisfactory (or better) along with the discussion of targeting issues. The analysis of country context, agricultural sector and rural poverty trends tended to perform best. The reviewed COSOPs are less good on the analysis of governance and institutional issues, on some equity issues and in linking the elements of the analysis together. Analysis of equity issues does improve considerably between the two cohorts, however. Figure 10 and 11 also show significant improvement across the two cohorts in the evidence provided of local ownership and stakeholder engagement, a direct result of the IFAD V Plan of Action. There is also clear improvement in the clarity and consistency of COSOP objectives and the articulation of indicators, but from a very low base. The low base relates to the absence of logframes in early COSOPs, while the relatively low percentage of satisfactory COSOPs in the later cohort (42 percent) shows the distance to be travelled in making the logframe a meaningful part of the country strategy process. Poorly articulated indicators, particularly those relating to outcomes and final impacts is a major weakness in COSOPs, with several instances of logframe indicators being left blank or being completed with poorly defined proxies. As in the project reviews, the logframe approach does not appear to be fully internalized amongst staff or consultants responsible for preparing COSOPs.


Page 21

Figure 11 Quality of analysis, targeting and clarity of objectives – percent satisfactory

0

10

20

30

40

50

60

70

80

90

Quality of upstream diagnosis and analysis Targeting Clarity, consistency of goals, objectives andindicators

Per

cent

Sat

isfa

ctor

y

1998-2000

2001-2004

3.42 Tracing back to the main strategic themes of the recent Replenishment rounds, Figure 12 summarises results of the COSOP review against four key dimensions – as a basis for seeking opportunities for pro-poor policy dialogue; as a platform for the identification of strategic alliances and partnerships; the rationale for innovation and scaling up; and as a way of leveraging and catalysing others. In terms of the first two dimensions – pro-poor policy dialogue and partnerships- COSOPs are performing reasonably well, although still around a third of the COSOP’s reviewed provided no satisfactory recognition of opportunities for pro-poor policy dialogue. Many of those that were satisfactory presented little more than a wish list of policy activities rather than clear entry points for dialogue in areas building on IFAD experience and comparative advantage.

3.43 Efficiency The sampled COSOPs do less well in terms of providing a rationale for innovation and scaling up, or in leveraging and catalysing others. Relatively little detail is given to explain innovative activities – much is often stated without being justified. As noted later in this chapter, the term innovation does not seem to imply anything substantive about the portfolio. In terms of catalysing others, there are few clear indications of what this means in COSOPs beyond generating co-financing for projects. Relatively little is said about lesson learning and sharing good practice with others or seeking to catalyse government capacity in response to IFAD experience.

3.44 As a vehicle for the evaluation of IFAD past experience in a country, COSOPs perform well with all COSOPs reviewed being rated satisfactory or better. The analysis of action areas for improving portfolio management is rated satisfactory in about half of the COSOPs. Where a country portfolio issues sheet is available, this is rated more highly. In terms of arrangements for monitoring and evaluation at programme and strategy level the picture is very different with over 80 percent rated unsatisfactory. The data suggest that this does not improve significantly between the two cohorts suggesting continued need for attention and progress


Page 22

Figure 12 Pro-poor policy dialogue, innovation and leveraging others – percent satisfactory

0

10

20

30

40

50

60

70

80

Opportunities for pro-poordialogue

Identification of strategic alliances/partnerships

Rationale for innovation & scalingup

Leveraging and catalysing others

Per

cent

Sat

isfa

ctor

y

1998-200

2001-2004

Summary of issues

In sum, the review of COSOPs finds them:

• Satisfactory in terms of corporate alignment, the degree of local ownership and stakeholder engagement, the quality of analysis and targeting issues.

• There is also evidence of improvement in the clarity and consistency of objectives, although the logframe is still not a fully internalised part of the COSOP.

• In many ways COSOPs are still functioning largely as an aggregation of project ideas rather than as country strategies able to provide selectivity and focus.

• They are strong on analysis but weak in articulating IFAD’s development niche. Links with in-country processes, such as PRSPs, and articulating ways of catalysing others to take forward replicable ideas also need improvement. The vagueness of innovation and poor arrangements for M&E are two highlight the weaknesses.

3.45 Strategic policies adopted since the late 1990s have provided a more systematic framework for project activity, country strategy and partnerships. The hierarchy of strategic objectives outlined in the Strategic Framework, Regional Strategies and COSOPs has helped to build clearer internal consensus and a ‘sense of mission’ within the organisation. Regional Strategies have provided clearer strategic guidance to country programmes and are providing a basis for expanding IFAD’s focus beyond traditional projects to greater engagement with sector programmes and policy dialogue. However, the Strategic Framework lacks clear operational added value. Regional Strategies fall short in terms of guiding selectivity and resource allocation at regional and country level. COSOPs, while improving in quality, still largely serve existing project ideas. They tend to be strong on analysis and on the description of targeting issues, but weaker on describing IFAD’s development niche and its catalytic role with other development partners.


Page 23

Loans – performance and trends in project cycle management

3.46 Project Cycle Management (PCM) is the process through which IFAD and borrower governments identify projects; design and formulate them; implement and supervise them; and evaluate lessons from the process in order to improve future performance. IFAD's mandated core-business is to finance projects and programmes, and the quality of its project cycle management matters greatly. The quality of IFAD's PCM is also central to the second 'fundamental question' in the IEE ToRs: Are the skills and resources of IFAD used in the best possible way, given IFAD’s overarching goal of supporting rural development and helping countries eradicate rural poverty? (ToRs par.11)

3.47 This section describes the desk study findings in relation to how IFAD decides which investments to finance (identification) and how investments are designed (formulation). It also assesses the more limited evidence available concerning project implementation, supervision and the outcome of investments.

Data and methodology

3.48 The sources of information for this section were: intensive study of the formulation and other available documentation for 42 randomly-sampled loan projects reaching effectiveness between January 1994 and December 2003; four TAGs and 20 COSOPs; review of other available Executive Board and policy papers; discussions with IFAD staff individually, in a variety of small group settings, and in three open-forum discussion meetings.

3.49 The documents studied for the sampled loan projects were: the Report and Recommendation of the President (the Board-approved definition of the investment prior to loan signing); the Appraisal Report and Working Papers on which the President's Report is based; recent supervision reports and the summary Project Status Reports; and Mid-Term Review, Project Completion, and Interim Evaluations, and Country Programme Evaluation Reports, where available.

3.50 The formulation documentation was assessed against a checklist of thirty questions. Available information on results was assessed against a fifteen-point checklist.21 Consistency of assessment amongst evaluation team members was sought through intensive initial discussion and standardisation of each checklist question and continual cross-checking during the document review process. Many assessments and ratings were further discussed with the CPMs and other staff concerned. The ratings were analysed within a custom-written database, and consistency was again reviewed as the tabulated results were analysed for this Report. Country Synopsis Papers summarising key findings from the COSOP and the two sampled projects for each of the 21 countries were prepared, and key issues and findings were discussed and clarified with the CPMs concerned. The Country Synopsis Papers are at Annex 9 (distributed separately).

Project cycle management: the recent history

3.51 IFAD has invested considerable effort over the last decade in revising aspects of its project cycle management and changes continue to date. The key developments are summarised in Table 6. The desk study analysis was conscious of this evolution and sought to assess project designs accordingly. For example, logframe presentations only became the norm towards the end of the sample period, and checklist questions were interpreted to accommodate projects with and without logframes.

21 See IEE Inception Report Annex 4, Draft Summary Form 1, Desk Study: Project and Grant Review


Page 24

3.52 Because of the evolution in both PCM and in policy emphases, the desk study sought to capture trends over time. It is important to recognise, however, that the sampling frame was defined by effectiveness date (January 1994 to December 2003) and there is a substantial lag between approval and effectiveness. For the sampled projects, the median time from Board approval to loan effectiveness is 12 months, but with a large variance: from 3 months to 3 years and longer in two cases. Hence the assessment of identification and formulation reflects performance from the early 1990s but not from the most recent projects approved by the Board and yet to reach effectiveness.

Table 6 Key events in the development of operational processes over the last decade

Date Initiative and policy development

October 1993 Report of PMD Task Force on Rationalization of the Project Cycle Cost of project development had risen from an average of US$401,198 in 1982-84 to US$575,359 in 1990-92. – reduced the stages of project design from 5 to 3; initiated the Project Development

Fund; proposed to reduce project development costs to US$450,000 January 1994 New Operational Procedures announced:

- Inception Paper for entry to the project pipeline, replacing General Identification Missions

- Project Development Team (PDT) introduced including CPM, PT and OE, with FAD and OL joining at appraisal stage.

- Benefic iary involvement in formulation emphasised, leading to creation of In-country Resource Groups in some countries.

- Formal TRC review process, with post-appraisal memo indicating compliance - Appraisal to emphasise implementation support

June 1995 Project Cycle Re-engineering Working Group report: - Seeking to reduce costs within zero -growth budget - Comprehensive list of current difficulties, including: PDT performance; technical

quality assurance and implementation support. - Recommendations include: new portfolio management system; COSOP (originally

conceived as a strategy paper, but in practice acting as project pipeline identification, triggering Inception Paper and Memo in the absence of identifications missions); strengthened PDT function and in-country resource group.

- Proposal to further reduce project development costs from the then-current US$433,000 to US$315,990 and release funds for early implementation support. (Senior management did not approve this release.)

July 1995 Re-engineered Project cycle - Pilot Phase including PDT, TRC & OSC stages December 1995 Report of Information Systems Re-engineering Working Group Report January 1998 Mainstreaming of re-engineered project cycle, including Logframe training for 45 staff December 1998 Impact Achievement in the Project Cycle (IAPC) group formed 1998-99 Continuing internal concern that PDT mechanism is not functioning effectively. June 1998 PMD instruction: PDT established as a mechanism running from Inception to Board

approval. June 1999 Pilot thematic groups established (Livestock, rural finance, project design and

implementation, microenterprises.) Late 1990's on OSC (Operational Strategy and Policy Guidance Committee) Meetings increasingly seen

to duplicate previous TRC's, with declin ing senior management attendance. July 2000 IAPC Report recommendations including: strengthened PDT to run from Inception,

through Board approval to project completion, including external expertise; increased direct IFAD implementation support; Key File based on the logframe; Start-up workshop based on participatory logframing; MTR, MTE and IE processes to have ACP/UCP.

October 2000 – continuing Process Re-

Eight processes were reviewed: 1 Human resources; 2 Strategy and Finance; 3 Support services; 4 Information technology; 5 Knowledge management; 6 Impact management; 7 Product development; 8 Partnerships management.


Page 25

Date Initiative and policy development

engineering Programme, renamed Strategic Change Programme

Items 1 to 5 went to the EB in December 2000. Resulted in an allocation of a budget of US$26 million: 1.0 million for design of the processes; 15.5 million for the five Business Cases approved by the EB; 9.5 million remains available, use to be decided

April 2001 ‘Participatory Approaches for an Impact-Oriented Project Cycle’ published, following internal discussions around IAPC Report

June 2001 Project Design Document (PDD) & Key File introduced December 2001 Analysis of project development costs for the PDFF identified average budget allocations

for project development of US$335,000 July 2003 PDD & Key File revised.

Three Thematic Groups formed (rural finance, gender, community-based NRM) to collate information from PDT meetings.

2004 ongoing Discussions underway to replace OSC approval stage with Project Design Assessments (PDA) to go in batches to President's Policy Forum meetings

Thematic Group/Learning approach extended through 'Learning Notes' covering 18 topic areas.

Relevance of the portfolio – project identification

3.53 The first stage in the desk study was to assess what determines how IFAD identifies investment projects. All sampled projects were consistent with both the COSOP and the Regional Strategy where these existed during the project identification and formulation process.22 Five points are important in interpreting this finding. First, the COSOP now plays two roles in the project cycle: it is both a strategy paper and a statement of the project pipeline for Board approval. This dual purpose has inherent contradictions, noted earlier in this report. The mid-1990's reforms to reduce identification and formulation costs led to the replacement of General Identification Missions with an Inception Paper covering one or more projects. Within eighteen months of the introduction of the Inception Paper approach, the Re-engineering Task Force in effect proposed up-grading Inception Papers to the COSOP. In resource terms, therefore, the outcome of efficiency savings has been that the only resources PMD has for identification is COSOP-preparation funds (drawn from the Project Development Financing Facility) currently averaging about $20,000 per country. A cost associated with this change has been the widely held view that COSOPs lack the intellectual depth of the forerunner Special Programming Missions. Further, because bringing about 25 new projects to the Board each year remains a powerful corporate imperative, and inception papers must be approved before expenditure on formulation can be incurred, including the pipeline in the COSOP is an efficient means of speeding up preparation. 23 In sum, since mention in the COSOP is administratively necessary for a project to proceed to formulation, it is unsurprising that, at Board approval, project designs are consistent with the preceding COSOP.

3.54 Second, the concepts and language used in some COSOPs are so broad and indeterminate, that almost any project concept would meet the consistency test. Where a country strategy is presented as a reformulation of the Strategic Framework objectives: '....strengthening the capacity of the rural poor to overcome their poverty....improving access to natural resources...technology and financial services and markets...' it is a challenge to think up a non-consistent project concept. Further, some COSOPs include so many alternative formulations and presentations (Strategic Guidelines, Strategic Thrusts, Goals, Objectives, 22 An analysis of the content and quality of COSOPs appears later in this Chapter, from paragraph 3.35 23 In discussion, PMD staff agree that the practical value of the COSOP to them is that allows them to proceed to project formulation more rapidly.


Page 26

Priorities, Outputs, Corporate Thrusts, Corporate Strategies, Innovations), it is both impossible to discover a unique statement of perceived IFAD strategic opportunities and likely that a project concept will be consistent with at least one of the alternative presentations.

3.55 Third, staff agree that the Strategic Framework 2002-06 does not restrict identification: any investment which would have been consistent with the Mandate would also fit within the three strategic objectives. They also agree that, for more recent projects, the Regional Strategies are more significant in determining investment choice, although there is still a tendency for project to drive the Regional Strategy, as noted between projects and COSOPs. For example, the first 'thrust' of the Eastern and Southern Africa Strategy formalises the post-1995 regional approach of strengthening smallholder market linkages, particularly in areas with higher productive potential. Similarly, the Asia and Pacific Regional Strategy makes the focus on less-favoured areas, indigenous peoples and other marginalized groups explicit. This Report makes no judgement on the relative merits of different regional strategies. The point is that whilst the COSOP is a statement of strategy and pipeline, and the Strategic Framework is entirely permissive, the Regional Strategy is the most influential internal determinant of project identification.

3.56 Fourth, the question of whether IFAD or other agencies are the main identifiers of investments is clear. For the 1994-2003 population of projects, 87 percent were initiated by IFAD, rather than by the co-financing partners, and that share is increasing over time. For projects approved by the Board in the past five years (since April 1999) 95 percent were IFAD-initiated. Project identification is overwhelmingly done by and IFAD and borrower government's and IFAD’s performance in appraising projects identified and prepared by other agencies is a minor element of portfolio performance over the past decade.

3.57 Fifth, an important factor, not explicitly recognised in IFAD documentation, is the extent to which the strategy of building-on-success drives project identification. One third of the sampled projects are second phases, albeit with modifications, of previously successful projects, or were modelled on nearby successes, or are leading to second-phase follow-on projects. A strategy of building-on-success is clearly sound for any organisation, particularly in a field as difficult as development. Such a strategy does call into question the practical response to the objective of being innovative. It is simply a matter of note that such a significant determinant of investment choice does not appear in IFAD's strategy documentation, although it clearly relates to corporate objectives to be a knowledge organisation, as detailed in Annex 6.

Key Issues on Identification

• The most significant current determinants of IFAD's choice of investments are: first, IFAD's own project preferences, developed in discussion with borrower governments, rather than co-financing of projects identified by CI's; second, the March 2002 Regional Strategies, in that they formalise the approaches developed over time within each PMD division; and third, designing new projects built upon past success.

• These three elements of identification make IFAD's own policy clarity, and the

quality of its formulation and implementation processes pivotal in determining project effects and impact.


Page 27

Effectiveness of the portfolio - formulation & quality at entry

3.58 Beneficiary selection and targeting The desk study assessed the quality of projects' social analysis, the definition and evidence for the selection of beneficiaries and project area, and means of targeting. Table 7 summarizes results from the review for key topics. Justification for the selection of project area, or beneficiaries within an area, was satisfactory or better in three quarters of the project sample, with a slight trend improvement over the decade. In the last five years, all but two projects were at least satisfactory, and most include clear and well-evidenced targeting information and justification. The best formulations provide defined criteria by area and social group, which the borrower and project management can directly operationalize. The less-than satisfactory cases variously fail to state why the project area was selected; give non-operational definitions (‘the poor’) or fail to note the existence of key groups (e.g. bonded-labourers).

Table 7 Analysis for formulation24



Percent Not rated

Poverty Analysis: The analysis of the reasons why the beneficiaries and/or the project area are poor:

74 26 0

Institutional Analysis: The analysis of institutional configurations and trends:

55 43 2

The analysis of lessons from previous loans and activities (where they exist):

76 14 10

Participation: The evidence that beneficiaries were directly involved in the project design:

50 40 10

Targeting – Selection: The social analysis and evidence justifying the selection of beneficiaries and/or the project area:

83 17 0

Targeting – Women: The gender analysis, assessment of women’s needs, role and benefits, and incorporation of a clear means to reach women in the project design:

81 19 0

3.59 The assessment of women's needs and role, and the incorporation of a clear means of reaching women through the project is also satisfactory or better in four fifths of the project designs - although not necessarily the same ones as those with good definitions of all beneficiaries. The proportion of well-gendered projects has remained almost constant over the last decade. In the best examples, gender issues are clearly recognised throughout the design, with clear targets for women's involvement. The unsatisfactory cases variously fail to include any gender analysis (in the earlier projects only) or do not specify practical means of reaching women despite noting a clear gender-division of labour in the social analysis. The commonest weakness is the inclusion of aspirations (‘women.....will be targeted ...will have access.... will play an active role ’ etc.) without any design features or targets to ensure the aspiration occurs in practice.

3.60 The desk study assessed the extent to which beneficiaries had been directly involved in shaping project design. IFAD's commitment to participatory approaches appears to have had a marked impact: the proportion of projects where beneficiaries were clearly involved in design, although only 50 percent satisfactory overall, has jumped from 30 percent to 75 percent over the years 1994/98 and 1999/03 of the sample period.

24 The tables in this chapter that report results from the desk review show the percentages of all projects rated satisfactory or higher, unsatisfactory, or not rated. Not rated occurred when a topic was missing completely (such as no indicators) or was held to be available only in other sources not accessible to the team, such as at country level.


Page 28

3.61 In sum, the best designs provide a clear statement of where and who the beneficiaries are; why that category of beneficiary has been selected; how many the project is expected to benefit and how in practice it will do so. Overall, targeting is now well done in the great majority of project formulations.

3.62 Poverty, Institutional and Environmental Analysis The desk study examined the quality of the poverty and institutional analyses, which underpin the project rationale. Three quarters of projects contain a satisfactory poverty analysis or better, in that they explain why the intended beneficiaries are poor. The less satisfactory accounts rely on national indicators alone or present poverty headcount estimates without examining the causes and dynamics of poverty in the area. The quality of poverty analysis has improved sharply over the past decade. In the last five years, all formulations reviewed, bar one, were at least satisfactory and most were impressive.

3.63 The analysis of key institutions relevant to the project – the organisations, agencies, laws, policies, formal and informal rules, which determine the project context - is weak overall. For the whole sample, the institutional analysis was deemed satisfactory in just over half the designs. There has however been a marked trend improvement: two-thirds of projects coming effective over the last five years have a satisfactory account of the institutional context, but few are better than satisfactory. The common weakness in the unsatisfactory accounts is that whilst governmental and other public agencies are mentioned, there is minimal analysis of their capacities, motivations or performance, however nuanced, and sectoral trends and developments and project-relevant political, legal and policy developments and private sector market-driven changes are not well described. The key drivers of growth and poverty reduction (or the absence of them) are not well assessed. The overall quality of institutional analysis is improving over time however.

3.64 Environmental analyses, including possible impacts, risks and mitigations, are satisfactory or better in just under two thirds of the projects reviewed, and there has been modest improvement in environmental appraisal over the decade. The best project designs include empirical data on environmental issues in the project area, with summaries of any previous EIAs; description of pertinent elements in the national regulatory framework; and practical, implementable, mitigation and precautionary measures for each relevant project output. The unsatisfactory treatments do not analyse environmental issues explicitly; make assertions without corroboration; do not address all project components that have environmental implications; or express aspirations without design features to give them reality.

3.65 Economic Analysis, Rationale and Replicability Three-quarters of projects include an economic analysis with a quantified estimate of economic returns. That proportion has remained stable over the past decade. However economic return estimates continue to be based on the traditional crop-budget approach, even where a substantial share of the loan is either financing activities where the end-use is largely unknown ex-ante (social investment funds, MFI-strengthening, marketing systems development etc.) or is financing indirectly-productive processes and capital investments (group formation, training, social infrastructure etc.). Traditional economic analysis is an anachronism for the majority of current project designs and is effectively impossible without introducing ever-more heroic guestimates and potentially misleading assumptions.

3.66 However, no project design reviewed recognises this limitation explicitly and seeks to provide more creative and useful efficiency indicators or design features to promote the key objective of economic effectiveness. Appraisals do not include tables specifying target- or maximum unit-costs for process activities, or practical means of maximising the number of beneficiaries, for example. There are positive moves towards updating the conceptualisation of economic appraisal and cost-effectiveness promotion within IFAD. The Decision Tools for


Page 29

Rural Finance (March 2003) includes practical best-practice methods for use within MFI-support projects, but this appears to be the exception.

3.67 The clear impression from the project formulations reviewed is that both the conceptualisation of project economic analysis, and the incorporation of practical design features to promote high returns and cost-effectiveness, has not kept pace with the changing nature of IFAD investments. Although IFAD loans to institutionally weak borrowers are highly concessionary, with a large effective-grant element, the debt crisis of the past two decades underscores the paramount importance of investments earning a positive return for the borrower country. An up dating of IFAD's approach to economic appraisal, and a rigorous consideration of how higher returns might be promoted in formulations and through loan-agreement undertakings seems appropriate.

3.68 Designing project interventions to be replicable has not been a strong emphasis within IFAD. The Strategic Framework only refers to the replicability of successful innovations as a desirable feature. However, cumulative development impact is strongly enhanced if project interventions are replicated, and cost-effectiveness is a key determinant of whether that occurs.25 Less than a third of the project sample addresses replicability satisfactorily or better, although that share rises to two-fifths for projects reaching effectiveness in the last five years. The majority of project designs do not address replicability of project activities at all. When used, the term normally refers to the potential of a project to spawn a second-phase or similar project elsewhere.

3.69 The desk study also explored the economic rationale for project design through assessing the extent to which projects are founded on clear evidence that other people in a similar situation in the country have successfully escaped poverty through interventions similar to those proposed in the project. Overall, only two-fifths of designs sought to justify project rationale on the basis of proven success elsewhere, and half of these cases referred to preceding IFAD projects. This finding is correlated with the relative weakness of institutional analysis described above. Few formulations offer a thorough presentation of the dynamics and sources of growth and poverty reduction in the project area.

3.70 Innovation and Risk The absence of justification for the project rationale by reference to proven sources of growth and poverty reduction in the project area is also the result of IFAD's demand on formulators to describe projects, or project features, as innovative.26 Three fifths of the sampled projects were assessed as innovative on one criterion or another. However, because such a variety of project elements are labelled as innovative, it is difficult to assess whether the investment itself should be deemed innovative. Use of the flexible lending mechanism; requiring exit strategies and transfer of responsibilities; participatory techniques; technology transfer; private-sector service delivery; MFI viability; civil society service delivery; are described as rendering different projects innovative. Table 8 summarizes analysis of risk, sustainability and innovation.

Table 8 Planning for innovation



Percent Not rated

25 For example, two bamboo treadle pumps are displayed outside the IFAD office. The treadle pump was designed and developed by an NGO in northern Bangladesh in the early 1980's. Its adoption throughout the high water-table areas of the country was the result of the pump's designers aiming for cost-effectiveness and simplicity of manufacture and maintenance, particularly in the design of the pump head. These replicability design features created strong market demand and enabled small workshops across the country to copy the design and start local production. Over one million treadle pumps were estimated to be in use in Bangladesh before the growth of shallow-engine pumping, and treadle pumps are now produced in other countries in Asia and Africa. 26 By definition, the rationale for innovation cannot be that the approach or investment is already a proven success elsewhere.


Page 30

Risk Analysis: The assessment of significant risks and assumptions, and the presentation of mitigating measures:

52 45 3

Innovation: The assessment of the applicability, benefits and costs of new technologies and techniques to be introduced:

62 26 12

3.71 Assessing the significance of innovation is further complicated because the different levels at which innovation might exist are not defined. Innovations might be global (inevitably very rare, and no examples were found in the sample); innovations for the country (e.g. financially-sustainable microfinance organisations); innovations for IFAD, or for IFAD in a particular country (commonly found in the sample). Innovations for villagers (normally described as extension or technology transfer) and in implementation approach (engagement of private or CSO implementers; use of participatory methods) occur very frequently. Until IFAD and formulators become more disciplined in, first, clearly stating the level at which innovation is alleged to occur, and second, providing evidence that the design feature is innovative at that level (i.e. no-one else has done it before), the label ‘innovation’, as currently used in project designs, carries little substantive information about the project or the portfolio as a whole.27

3.72 Further, the implications of introducing authentic innovation in terms of increased borrower and beneficiary risk are not well addressed in project documentation. An authentic country level innovation must be risky because, by definition, it hasn't been tried before and confidence in outcomes must be lower than for tried-and-tested approaches. (The high-risk high-return trade-off exists for IFAD as for any other investor.) In none of the assessed projects does the risk analysis clearly indicate that because the project approach is innovative, it also represents an unusually risky investment. Given the extent to which projects build on success, as noted above, a better understanding of innovation and analysis of risks from experience, should be possible.

3.73 Overall, the quality of risk assessment is not strong in project designs. Only half of the sample was assessed to have a satisfactory or better risk appraisal, and that level has not changed over the past decade. The weaker risk analyses are clearly incomplete, or list universals (political or economic crisis); offer internal not external risks (e.g. project management failure) or are weak on potential mitigations. The fact that the innovative nature of the project does not appear as the central risk in any project tends to corroborate the assessment that the innovative content of most projects is limited to particular implementation features - a participatory approach, use of private implementers etc. – rather than to the project approach as a whole.

Key Issues on project analysis

• IFAD is good at targeting. Beneficiary participation in design has improved sharply. This suggests that the relevance of IFAD investments to rural poverty reduction is high.

• The poverty analysis of the selected project area is good overall. However,

institutional analysis and the foundation of project design within a clear understanding of the key determinants of the rural growth and poverty reduction dynamic in the project area is less strong.

27 The absence of a clear and agreed definition of innovation in project documentation suggests that current proposals to 'mainstream' innovation are premature.


Page 31

• Economic appraisal of project designs has not kept pace with the changing nature of IFAD investments.

• Focus on cost-effectiveness and replicability-potential is weak. Adjectives and

aspirations are common. Practical design features and targets to ensure they occur are rare.

• There are many innovations for different tiers, particularly for villagers, within IFAD

projects. But until definitions are clear (innovation for whom?) and well evidenced, the overall innovative content of the portfolio will be difficult to determine.

• The policy requirement for innovative approaches is not reflected in risk analyses.

Overall risk analysis is not strong.

3.74 Project Definition The desk study reviewed the quality of definition of the project as set out in its objective, outputs, the logic connecting outputs to objective, and indicators for measuring progress over time. For earlier projects, when use of logframe planning was still rare in IFAD, the assessment was based on the text describing objectives and components. For later projects, the assessment considered both the text in appraisal and President's Reports and the appended logframe. Table 9 summarizes results.

Table 9 Measurable objectives



Percent Not rated

Objectives: The clarity, realism and measurability of the project objectives:

74 26 0

Indicators: The definition, clarity and practicality of the objective and output indicators:

26 60 14

Monitoring: The definition and practicality of systems and responsibilities for monitoring activities and outputs:

43 57 0

Sustainability: The assessment of the likelihood of sustainability and the specification of measures to promote sustainability:

40 50 10

3.75 Objectives are clear, realistic and measurable in three-quarters of the sample. This share has remained stable over the decade. The quality of outputs (what project management is expected to achieve) has also been satisfactory in three-quarters of projects for the past five years, but was much more variable in the early 1990's, with only just over half of projects having satisfactorily clear, realistic and measurable outputs. The logical structure of the project, whereby achievement of the outputs by project management would lead to achievement of the objective (provided the risks don't occur) was good across the sample, and all recent projects display a sound logical connection from output/components to objective. However the definition, clarity and practicality of indicators at both objective and output/component level is notably weak overall. Less than one in three projects in the whole sample offer a specification of indicators that is at least satisfactory, although that share rises to two fifths over the last five years.

3.76 The assessment of the quality of project definition in more recent designs is not straightforward, because IFAD has yet to decide whether or not its project planning is logframe-based. Despite being formally introduced in 1998, to date the logframe remains an annex to the project definition set out in the text of formulation, appraisal and President's reports. The logframe is not the basis of the project definition. Staff confirm that a logframe format is normally filled out at the end of formulation, because it is now a required annex, not


Page 32

because it is seen to add any particular value. The first problem with this practice is that it creates two similar but different definitions of the project: one in the text and one in the logframe. In no sampled project with a logframe annex is the text and the logframe exactly the same. The specification of objectives, outputs/components, indicators and risks differs to a greater or lesser extent in every assessed project. Such unnecessary confusion on the basic definition of the project cannot be helpful. Second, the evidence is that the few - but fundamental - logframe concepts and definitions are far from internalised amongst staff or consultants.

3.77 The essence of logframe planning can be stated in four sentences: The outputs state what project management will achieve. The objective is what should happen if all the outputs are achieved and the key risks do not occur. Each output and the objective should have a few practical indicators, with targets, for tracking progress over the life of the project. The logframe should be prepared in discussion with beneficiaries, implementers and technical specialists. Programme staff say they have been trained in logframe methodology, yet that training does not appear to have instilled the essence of logframe planning and thinking. Some staff say they feel the logical framework is ‘inflexible’, but readily accept that the four-sentence definition above allows all flexibility to project management to plan and sequence activities (how the outputs are progressed each year) within the Annual Workplan and Budget.

3.78 The impression from the desk study is that a clear decision is required on whether IFAD wishes to define its investments through logframe planning or not. If IFAD does intend to use logframe planning, a brief set of instructions for all formulators will be required. If IFAD does not, the current practice of adding an alternative semi-logframe presentation of the project should be deleted from the key file and other preparation reports.

3.79 Sustainability The pursuit of sustainability28 is fundamental for development agencies. It is what distinguishes their investments from relief, rehabilitation or welfare interventions. For lending institutions it is particularly important because the flow of benefits is expected to accumulate over time, enabling repayment of the credit received. The Strategic Framework recognises the importance of sustainable impact, sustainable poverty reduction and enabling poor people to sustain change.

3.80 The desk study examined the extent to which formulations analyse the likelihood of sustainability and specify design features to promote it. Just 40 percent of project designs were assessed as satisfactory or better on this criterion (see Table 9). There is a clear trend improvement, however, with almost 60 percent of designs over the last five years showing a satisfactory treatment of sustainability.

3.81 The single largest cause of the 'unsatisfactory' assessment is that the Presidents' and appraisal reports fail to address sustainability at all. In other cases sustainability is included as an aspiration (‘...need to ensure beneficiaries can meet the recurrent cost of...'; '....need to strengthen ministerial responsibility...’) without any design feature to ensure the aspiration occurs. The strength of the more recent rural finance and MFI-strengthening projects, and IFAD's June 2000 Rural Finance Policy on which they are based, is that sustainability is the central objective. These investments are single -mindedly designed to create local MFIs capable of meeting the demand for accessible small-scale credit in perpetuity.

3.82 Implementation Arrangements The desk study reviewed how clearly project management, implementer and partner roles and responsibilities are assessed and defined. The definition of the management structure for the project is satisfactory or better in three quarters of the sample. This share has remained stable over the decade. The unsatisfactory projects do not clearly indicate lines of reporting and responsibility, leave inter-agency co- 28 Sustainability here refers to a sustained flow of benefits over time, rather than the environmental or ecological sustainability of development change.


Page 33

ordination arrangements vague, or provide a confusing account of responsibilities. Most designs include an organogram, but the presentation of some is poor, with vague lines of responsibility and extraneous agencies included. Planning for implementation is summarized in Table 10.

Table 10 Planning for implementation



Percent Not rated

Management: The definition of roles, relationships and responsibilities for project execution and implementation:

76 24 0

Management: The analysis of current capacity and commitment amongst intended executors/implementers, and specification of requisite capacity building measures:

33 52 15

Partners: The definition of the role, responsibilities and expected contribution of other project partners:

45 43 12

3.83 The role of partners is satisfactorily defined in just half of the projects. Much the largest reason for rating projects unsatisfactory is the tendency to list partners which will contribute to the project, but without any description of whether they were consulted by the formulators; whether they have the capacity or motivation to play their allotted role, and what their perceptions and demands are. Some projects now include an 'institutional capabilities' matrix listing strengths and weaknesses of involved and related organisations. However in most cases this appears to add little value: the analyses appear formulaic and generalised and fail to pinpoint which particular strengths or weaknesses of proposed partners are directly pertinent to the success of the project.

3.84 The desk study examined how well project designs assess capacity and commitment amongst intended executors and implementers and how well they specify requisite capacity building measures. There are two main categories of project management arrangement: using existing organisations (central or local government departments, NGOs etc.) or implementing through managers specially hired through advertisement. For the first category, the assessment sought to identify how clearly formula tors had researched and defined management arrangements, capacities, motivations and needs. For the latter category, external hires, the quality of the job specification, recruitment procedure and safeguards was assessed.

3.85 Just over one third of the sample was assessed as providing a satisfactory presentation of the capacity, commitment and needs of intended implementers, although there is a clear trend improvement. Half of projects reaching effectiveness over the last five years have a satisfactory appraisal of intended implementers. As in the assessment of partner roles, the principal reason for unsatisfactory ratings was that two thirds of project documents offer no re-assurance that formulation teams researched existing management capacities thoroughly, or discussed with potential implementers and assessed their capacities, motivations and support needs accordingly.

3.86 A similar picture emerges from the specification of project monitoring responsibilities and arrangements. Just two-fifths of project designs define monitoring arrangements and responsibilities sharply; are clearly based on a sound understanding of real capacities; and specify TA and other support needs. The unsatisfactory preparations always mention an agency as responsible for monitoring, but without any description of whether the nominated agency is in a position to perform its allocated monitoring responsibilities effectively.

3.87 To some extent the absence of assessment of management and performance capacity – for implementers, monitoring responsibilities and partners - may be the result of house-style. Some staff say it's not appropriate to describe local capacities in project documentation.


Page 34

This seems unsatisfactory. First, development projects routinely set out capacity-building and organisational development needs for state and private agencies, based on jointly appraised strengths and weaknesses. Second, the single most important consideration before making any investment is confidence in the capacities and motivations of the people responsible for managing the investment. For this to be left tacit, un-researched, or un-reported to the Board – which has ultimate responsibility for agreeing the investment - seems unacceptable. Third, if formulators don't explicitly state that they met and discussed capacities, motivations and needs with the key agencies responsible for implementation, it inevitably leaves an impression that agencies may have been written-in to designs at a desk.

3.88 The desk study noted that explicit anti-corruption measures do not form part of IFAD's project documentation. Standard paragraphs on procurement procedures are included in the body text of President's reports and in loan agreements. However the Strategic Framework's explicit commitment to good governance has not led to action in the form of specific provisions in project documentation. An Oversight Committee was established in May 2000. The Committee's mandate and guidelines were revised in July 2003, but they remain essentially passive and re-active.

Key issues on project definition, sustainability and management

• The definition of the project objective, outputs and the logical connection between them is satisfactory overall. But the definition of practical indicators and targets for tracking performance over time is both weak and only slowly improving.

• A decision on whether or not logical framework concepts are to be the basis of

project formulation is required.

• Treatment of sustainability is weak. Adjectives and aspirations are not matched by practical design features.

• Appraisal of the implementation capacity, motivations and support requirements of

project managers, partners and monitors is weak overall.

• Anti-corruption measures are not specified in project documentation. 3.89 Quality Assurance Over the last decade, IFAD's quality assurance process for project design has had two elements: the Project Design Team (including the CPM, Technical Advisory Unit and OE staff, with finance and legal staff participating at later stages) and the Technical Review Committee and OSC (Operational Strategy and Policy Guidance Committee) process, whereby formulation documents are reviewed by technical specialists, and a Lead Advisors memo on key issues is prepared for review by the OSC, chaired by the President or Vice-President, before proceeding to appraisal. The detail of these two elements of the quality process has changed over the decade (see Table 6), but the essentials have remained much the same.

3.90 The desk study reviewed the documentary evidence for the impact of this quality assurance process on project design. Overall, the dialogue enabled by the TRC process and its impact on project quality appeared to be satisfactory or better in four fifths of projects sampled. Over the past five years, 90 percent of TRC interactions appeared to have been thoughtful and productive. The formulation-stage issues identified and challenged in Lead Advisor memos cover a wide range of technical, process and project design issues. The project review and subsequent discussions with staff elicited five significant issues in relation to IFAD's quality process.


Page 35

3.91 First, because the IFAD investment portfolio covers such a broad field, it is impossible for the Technical Advisory Unit (PT) to have in-house specialists covering all areas. The Strategic Framework does not restrict eligible investments compared to the Mandate whilst recent growth in sectoral, participatory and process investments has broadened the range of design issues to be addressed. In some areas, IFAD now has clear policy and design checklists to inform quality appraisal. The Rural Finance Policy and accompanying Decision Tools is the leading example.29 The Gender Mainstreaming Plan of Action (April 2003) is a rigorous and impressive initiative, and is leading to a more systematic appraisal of gendering in IFAD policies, projects and loan agreements.30 In Africa, smallholder market-linkage investments have grown, and are now being supported by rigorous research into the fast-changing impact of agri-business investment on outgrower livelihoods. Efforts to disseminate good practice in livestock and range management are underway. At the end of 2003 PT analysed the main issues arising from the last 100 TRCs and produced a set of 18 Guidance Notes for Investment setting out minimum standards, common design issues, good practice examples, and sources of information for PDTs to use. The Notes are at an early stage and their quality is variable, but it marks a positive initiative.

3.92 There have also been setbacks: IFAD recently lost its lead expert in the Strategic Framework priority area of land policy and management. Further, some key policy issues remain undefined. There appears to be little shared understanding of the IFAD target group: some staff see it as all the poor; some, the majority of the poor; some insist it is the poorest of the poor. There appears to be little policy clarity on public -private roles and interface in extension services. The concept of empowerment is used very broadly to encompass political, economic, organisational and intra-household relationships. In-house debate on these issues is healthy, but it is difficult to promote clarity and quality when key concepts used in a growing number of project formulations remain largely undefined and highly flexible.

3.93 Overall, and despite many positive initiatives to enhance quality, it is clearly not possible for IFAD to maintain in-house professional expertise in all sub-sectors of the rural economy. It may be that further technical quality enhancement means that either IFAD must consider restricting the variety of sub-sectors it seeks to invest, build expertise and reputation in, or it will need to buy-in expertise from other specialist organisations.

3.94 Second, despite extensive dialogue during the TRC process, the extent to which TRC/OSC recommendations are explicitly treated during project appraisal is rarely transparent. In some reviewed projects, TRC/OSC points were clearly addressed in the subsequent appraisal report. In others it was unclear if or where points had been tackled. A brief table annex to the appraisal report stating where and how each TRC issue is treated would assist clarity.

3.95 Third, quality assurance is hindered by the absence of names on any project preparation documentation. This undermines transparency, accountability, incentives and quality. Poor attendance at PDT meetings has been a recurrent issue since they were created. It is difficult to discover which consultants were involved in formulation or who is responsible for drafts and revisions. The simple step of requiring the lead CPM, PDT members and consultants to put their names on documents they are responsible for would be a significant contribution to accountability and quality enhancement.

29 The 2003 donor peer-review process effectively endorsed IFAD's approach to Rural Finance as best practice. 30 The December 2003 Baseline Survey Report is a rigorous appraisal of gender quality across country strategies, appraisal reports, grants and implementation phase workplans and supervision reports. This initiative seems likely to result in nest-practice treatment of gender in project design and other IFAD processes in due course.


Page 36

3.96 Fourth, although TRC discussions have clearly been important contributors to design quality, it is less clear that they are sufficiently focused on the critical issues that it is most important to get right at design stage in order to increase the chances of success. The large number of issues raised is itself an indicator that TRCs have not been focused on selected priority issues.

3.97 Fifth, the desk study found widespread concern amongst staff in the Programme and Finance Departments that senior management and Board focus on IFAD's core business of making quality investments in developing countries has declined over recent years. Line staff see the quality of IFAD investments as the essential foundation for impact and for the other strategic goals of knowledge dissemination and policy influence. Staff cite many examples: the progressive decline of OSC project-review meetings since the late-1990's due to diminishing attendance by senior management; diversion of management time onto administrative and other support functions; the creation of task forces who's work vanishes after completion; the share of senior management time devoted to Board-imposed requirements and international engagements.

3.98 This report does not seek to assess how well senior management has allocated its time over recent years. Here, the Report simply notes a significant finding from the desk study, which must be affecting project quality, albeit in unquantifiable ways. All organisational studies point to the pivotal role of senior management in providing leadership and focus. Such widely felt concern, verging on demoralisation, amongst many line staff, justified or not, cannot be helping to promote quality and impact.

Key Issues on quality assurance

• The PDT/TRC/OSC process has been a valuable contributor to enhancing quality at entry.

• Impressive efforts to define and implement best practice are underway in several

sectors. But limited in-house expertise across the whole rural development field will always be a constraint. Either a more restricted portfolio focus towards particular sectors or greater use of external best-practice support will be required in enhancing quality over the future.

• There is extensive staff concern that for some years IFAD has been insufficiently

focused on its core business of making quality investments, which are the foundation of impact and other strategic objectives.

• Overall, the quality at entry assessment suggests IFAD projects are both over-

designed and under-designed. Appraisal documents, working papers and TRC discussions address many issues in considerable depth for every project. However it is not clear that the core group of common and top priority design issues – those on which project performance and impact ultimately depend – have been sharply identified and are given most attention by formulators, PDTs, TRCs and appraisers.

Efficiency in project implementation and supervision

3.99 The main source of evidence on project implementation and supervision will come from the country visit phase, including visits to supervising agencies. The desk study assessed the quality of supervision reports for the sampled projects. Overall, four fifths of supervision


Page 37

reports provide a well-documented and clear account, although mainly in relation to fiduciary and procedural matters.

3.100 There appears to be a pattern in supervision quality, with reports from UNOPS and supervisors engaged under direct-contract providing above-average quality reports overall. Several supervisors are providing substantial implementation support to projects. Overall, the desk study findings tend to corroborate the recent study undertaken by OE.31 In discussion, many staff note the importance of a modest increase in the frequency and intensity of supervisory inputs, particularly for projects in difficulties. It was suggested that with about 200 ongoing projects, a budget envelope rather less than that recently allocated for the Field Presence Pilot would be sufficient to achieve a substantial increase in supervision intensity, principally focused on under-performing projects.

3.101 The desk study assessed the extent to which previous supervision or MTR recommendations had been actioned by project management,32 as an indicator of project management and supervisor effectiveness. Overall, about 60 percent of supervision reports indicate that project management had taken broadly effective action on recommendations made. This finding tends to confirm the view that ensuring sound project management from the outset is pivotal: where management is weak, supervisory recommendations remain un-addressed.

3.102 The standard of mid-term review reports is notably high. Some 90 percent are at least satisfactory, and half of these provide very thorough, well-researched studies. Although the MTR sample is small, mid-term missions appear to be particularly valuable for turning around failing projects. This role appears to be effectively a re-formulation, including replacement of key implementers.

3.103 Supervisors' reports arrive at IFAD at different times, in different languages and on different formats. The Project Status Report (PSR) summarises supervision reports and other sources of information onto a standard two-page format, including performance ratings on standard criteria. The PSRs then become the basis of the Annual Portfolio Review prepared by each regional division. Overall, three-quarters of PSRs offer an accurate reflection of preceding supervision reports or MTRs. The remainder appear either to mis-represent (in a positive direction) or not report significant problems identified in recent supervision reports.

3.104 Two points are pertinent. First, project and portfolio review is hampered by the absence of reliable and systematic output-progress information. The current PSR rating system leaves wide scope for judgement: between project and supervisor and between supervision report and PSR. PSRs are criticised for having too many ratings, no clear way of reviewing progress from outputs to objectives, and a format that does not lend itself to clear statements of the current situation, identifying problems and setting out actions. One or two reliable, checkable monitoring data fields for each output would transform IFAD's capacity to assess project performance against target over time. From the desk review, the great majority of IFAD-project outputs are straightforward, and therefore susceptible to monitoring through a few simple indicators (Groups formed and active; km of road completed; MFI overdues as percent of outstanding; households allocated forest plots; etc.). It is fair to argue that such crude indicators fail to capture the richness which is any development project, but the current paucity of progress information suggests that even crude indicators, for comparison against targets at entry, would be more informative than the current PSR rating system. For the future, current initiatives to define standard component types within a common output

31 Evaluation of Supervision Modalities in IFAD-supported Projects, OE, November 2003. 32 Many supervision reports include a table assessing project action against previous recommendations.


Page 38

monitoring system,33 suggests it may eventually be possible to convert project-level performance data onto a common indicator set for the whole portfolio.

3.105 Second, it is not clear why PSRs do not currently include the financial progress data that is readily obtainable from the Loans and Grants system. It is not possible to map disbursement data by functional category directly onto output performance, but the inclusion of finance data in the project review process would offer another source of significant information. For example, staff say there is a consistent tendency in some regions for projects to vire project budget into the administration and support budget line. Tracking such patterns would offer an important additional indicator of the extent to which projects are succeeding in maximising efficiency and impact.

Key Issues on implementation and supervision

• Overall, current supervision is effective, particularly where project management is strong. The pattern of weak supervision from some CIs identified in the November 2003 OE report is also apparent from the project sample.

• The larger concern is failing projects, where MTR teams appear to be more effective

in turning around poor performers than supervision visits. Increased intensity of supervision and a stronger supervisor remit in relation to low-performers may be appropriate.

• The PSR process is a commendable attempt to assess project and portfolio status on a

common basis. It is seriously hindered by the absence of empirical output performance data however. The use of financial performance data, readily available from current financial systems, would enhance the annual project and portfolio review process.

Quality amongst comparable donors

3.106 The main focus of the IEE during the desk review has been the analysis of IFAD’s portfolio and the project sample. The team will interact with lending partners, cooperating institutions and comparable donors during the country visit phase. However, it is appropriate to contrast the findings in this desk review with published information from the World Bank. The Bank prepares an Annual Report on Portfolio Performance (ARPP) in which trends in lending are summarised together with analysis about the performance of the portfolio. These draw on a variety of instruments that include quality assurance assessment of a sample of new projects (Quality at Entry - QAE) and of projects being supervised (Quality of Supervision - QSR); periodic assessments including performance ratings in a Project Status Report (PSR); and portfolio indicators related to the lending programme (e.g. extent of delay between loan approval and loan effectiveness) and to the country (e.g. macroeconomic performance).

3.107 The ARPP for Fiscal Year 2002 (July 2001-June 2002) draws comparisons across a period of time very similar to the IFAD evaluation. Table 11 highlights key features of the Bank’s analysis with comments about IFAD’s performance.

33 See IFAD (2003) Framework for a Results Management System for IFAD-supported Country Programmes


Page 39

Table 11 Comparative project performance for IFAD and the World Bank

Characteristics World Bank Performance34 IFAD Comparison

Overall quality Stabilised at 85-90 percent satisfactory

No regular measure available. IEE review suggests a lower level of performance, but the methodologies are not the same

Major concerns Clarity of development objectives at design

Objectives clear, realistic and measurable in three-quarters of the sample

Quality of monitoring Absence of key indicator monitoring data and about a quarter of PSRs do not accurately reflect supervision findings

Danger of a return to a lending culture

Widespread staff perceptions of insufficient focus on core business

QAE concerns Deterioration in quality of implementation arrangements

Appraisal of implementation capacity weak overall; institutional analysis not strong

M&E Unsatisfactory and poor definition of indicators, targets and project monitoring responsibilities and capacities

Risk assessment Risk analysis not strong QAE recognition of well-planned elements

Fiduciary; environmental; social development

Targeting and beneficiary participation good and environmental analysis good

QSR findings Not enough attention paid to risky projects

Possible need to vary supervision to give higher intensity for low-performing projects

Difficult to remedy project deficiencies during implementation

MTR used as an entry point for reformulation

Failure to address implementation questions thoroughly, including weak follow-up by managers

PSR process commendable but hindered by absence of output –progress information

3.108 Care must be taken with interpreting the table. Firstly, the World Bank findings relate to a large portfolio of over 2,000 loans covering all sectors and assessed with a long-standing well-honed methodology. In general, ratings were somewhat worse for IDA operations than for IBRD, and rural projects are generally found to under perform the portfolio average. IFAD’s portfolio is entirely rural, and the majority of countries are eligible for IDA terms. That said, the comparison is instructive. IFAD shares many performance standards and concerns with the Bank. With the exception of the clarity of development objectives, where IFAD appears to have performed better, similar difficulties are experienced across a range of design features: M&E; implementation capacity and institutional analysis; and risk assessment. Performance during implementation is similarly comparable with shared concerns about the quality of supervisory monitoring and the difficulty of remedying problems during implementation. The similarity of problems suggest that these reflect the challenges of working in development. It is not possible to draw conclusions about the level of performance from this exercise. It remains to be seen to what extent IFAD’s focus on rural poverty enables it to perform better than other development agencies in this arena.

34 World Bank (2002) Annual Report on Portfolio Performance. Washington


Page 40

Project outcomes

3.109 The desk-study evidence on outcomes from the 42 sampled projects is limited for four reasons. First, the required random sampling methodology left no scope to select projects which are already well into implementation. 35 Second, the Inception Phase decision to extend the sample cut-off date to effectiveness by December 2003 further skewed the sample towards recent projects: eight of the 42 sampled (19 percent) reached effectiveness during 2003. Third, in many projects there are substantial gaps between effectiveness date, the start-up workshop, and activities beginning. It is common for projects to spend at least the first year after start-up on recruitment of staff, procurement and initial surveys. For slow-performers, closure dates are commonly extended, further delaying final reporting. Fourth, the reporting of information on results from the project to IFAD is not systematic: the content and quality of MTRs and PCRs is variable and evaluation coverage is limited. The following assessment is therefore based on 18 projects for which some data are available from project completion, supervision, MTR, evaluation or Project Status Reports. To increase the sample, the available data from two predecessor projects is also included in this analysis.

3.110 Project outputs/components were at least satisfactorily achieved in 12 of the 20 projects, and well-achieved in six of those 12. The available data does not enable a comparison with the planned level of achievement, and the basis for the satisfactory rating in some cases is either against revised targets or refers to new components added during the course of implementation. Some data on beneficiary numbers are available for 17 projects. In at least ten, beneficiary numbers were high and in four of those, considerably higher than in the appraisal estimates. The share of women in the direct-beneficiary total is significantly lower than estimated at entry. In only five projects were women clearly substantial or equal beneficiaries, although the absence of gender analysis of beneficiaries in the various reports or the absence of disaggregated monitoring data is in part the cause of this apparently low share. On balance, however, it appears that IFAD’s growing strength in gender analysis and planning in formulation may not be maintained throughout implementation, and reflected in outcomes.

3.111 Physical and financial assets and services increased at least satisfactorily in 13 of the 20 projects. In the remaining seven, infrastructure provision has increased, but achievement has either been slow or well below the appraisal estimate. Access to social infrastructure (clinics, wells/boreholes, schools and improved access as a result of improved roads) has improved satisfactorily in nine of the 13 projects with social infrastructure components.

3.112 In the 19 projects for which there is data on social capital formation, there were positive developments in at least ten, including new associations of SMEs, savings and credit and self-help groups, increased confidence, literacy, and community cohesion. However evidence on sustainability, or the assessment of its likelihood, is universally lacking. In the other nine cases, either coverage was very low compared to target or the groups created clearly remained project-clients. Eight projects achieved institutional reforms, including: aspects of sea fisheries; pond leasing arrangements; local bank policy, strengthened municipal functioning, development of private SME-development agencies and strengthened capacities in the Ministry of Agriculture.

3.113 Food security appears to have improved significantly in 15 of 19 projects for which there is evidence. Such assessments are difficult from limited information, but it appears that increased yields and production are benefiting both direct project beneficiaries and the wider community via some fall in food prices. In the other four cases, coverage has been low or slow. The evidence is that beneficiary incomes have risen in 12 of the 18 projects for which there is some evidence. However sustainability is difficult to assess, because it is seen to

35 This will also be a significant factor for the impact data to be collected during the field study.


Page 41

depend on assumptions about the sustainability of credit-supply agencies and adequate maintenance of irrigation and other built-infrastructure. In the absence of systematic guidance, MTRs and PCRs are not required to come to a judgement on the likelihood of sustainability of outputs, and – despite its importance – sustainability is barely treated in the available reports. Replication is only treated in reports in terms of prospects for follow-on projects, rather than auto-extension of project-introduced technologies or methodologies. One of the two first-phase projects assessed in this analysis received a Diplome d’Honeur from the head of state.

3.114 On balance, seven of the 15 projects for which there is some data, are likely to achieve a beneficial impact on the environment and natural resource sustainability, although this assessment depends on a possibly optimistic estimate of sustainability of the projects' environmental outputs.

3.115 Of the 16 projects where innovation is mentioned in reports, five have little innovative content. Of the eleven projects with elements of innovation, four appear to be innovations for IFAD in the country and three appear to be innovations in methodology for the project beneficiaries (but common elsewhere). Four projects appear to have elements of innovation for the country as a whole: in credit delivery and credit-in-kind; in new fishing technologies and in project methodology appropriate for tribal areas. In the absence of evidenced treatment of innovation in reporting, this breakdown remains distinctly tentative however.

Key Issues on project outcomes

The field study will be the principal source of evidence on impact. The desk study data set on results was very limited, both by the number of advanced or completed projects and the paucity of available evidence on them. Three points do emerge however.

§ First, and perhaps obviously, satisfactory results depends on project performance: if the project is not achieving, outcomes will be modest.

§ Second, treatment of sustainability in PCRs, MTRs and other reports is negligible. Hard data on sustainability is difficult to obtain, of course, since it depends on good output monitoring data and post-project evaluation, some time after closure. However it is perfectly possible for experienced supervisors, evaluators and consultants to make an informed judgement about the likelihood of sustainability. The fact that such assessments are almost entirely absent – in relation to change in income and food security, built infrastructure, group organisation, institutional change, environmental impact – appears more a result of the absence of a firm requirement that sustainability be explicitly treated in reports, than any intrinsic difficulty in coming to a judgement on the available evidence.

§ Third, the evidence suggests that the most consistently achieved results is increased food security as a result of higher yields and production of crops, livestock and fish, followed by increased access to physical and financial assets, and household incomes.

Grants 3.116 The facility to offer grant financing was created at the formation of the Fund. The agreement establishing IFAD defines the use of loans and grants, and limits grants to ‘not normally exceed one-eighth of the resources committed in any financial year’.36 Grant

36 IFAD (1976) Agreement Establishing the International Fund for Agricultural Development (revised through to 1997). Article 7, 2 (a) & (b).


Page 42

assistance is further defined in the Lending Policies and Criteria, which established a framework for the use of grants to fund preparation activities under the project cycle, in support of research and as an exceptional component of project funding. All three have been applied. An extended treatment of this topic is at Annex 4.

Evolution of grants policy

3.117 The ten-year period since the mid 1990s has seen a large number of initiatives and policy changes. Annex 4 Table 1 sets out the main features in a time-line. Several features emerge from this history. Firstly, there is a move away from grants in support of project development towards a broader view of partnerships, rural dynamics, knowledge dissemination, and complementarity with loans and the country portfolios. But the high frequency with which policy has been restated complicates the message. Secondly, there has been a steady process of innovation and change affecting the ways in which the grants programme is organised and managed in support of the project cycle, with new instruments such as the Project Development Fund (PDF), Special Operations Facility (SOF), Environmental Assessment (EA), and Project Development Financing Facility (PDFF). Each of these places new demands on staff to adapt to changing procedures and mechanisms. Thirdly, there was a protracted process of proposal and consultation from 1998 to 2003 to update the grant policy. This period coincided with several other change processes: Business Process Re-engineering, the Strategic Change Programme, and the development of the Strategic Framework 2002-2006. The delays indicate a prolonged period of uncertainty and management indecision.

3.118 The importance of grants to the development of projects and programmes is clearly evident. The new 2003 policy simplifies strategy, provides clear guidelines on funding priorities, and reinforces the need for grants to complement country programmes. The policy is consistent with and relevant to IFAD’s mandate. But there is very little evaluative evidence used to justify continued levels of spending or to support proposed changes in modalities such as the creation of the PDFF or the absorption of the NGO-ECP window into the small TAG facility.

Summary of issues

• Consistency in the underlying objectives of grant funding has been undermined by frequent restatements of policy and the introduction of new instruments

• The multiplicity of facilities and modalities has led to a loss of focus, lack of strategic orientation and no prioritisation

• With the exception of the agricultural research programme, little evaluative evidence has been used to inform management before the 2003 grant policy

The TAG portfolio

3.119 Analysis of the TAG portfolio is complicated in several ways. Firstly, the profusion of instruments means that a variety of facilities have been used towards similar ends at different times. Secondly, data are available from a multiplicity of sources: the annual budget, records kept by PT, and the Loans and Grants database (LGS). Thirdly, some sources refer to new commitments, others to annual disbursement.

3.120 The total level of TAG approvals grew from US$14 million in 1993 to around US$35 million in 2000 and thereafter. Annex 4 Table 3 presents an analysis of data held by PT for the three classes of Technical Assistance Grants for the period 1994 to 2003: Small TAGs


Page 43

(<US$100,000), TAGs for agricultural research and TAGs for other research and training. These differ in number and value from the figures in the annual report, which include other grants to be discussed later. The data show distribution by year and by global or regional or country focus. The main points to emerge from the data are as follows.

3.121 Small TAGs are the most numerous, totalling 329 with an average value around $50,000. Three-quarters are directed to named countries, most frequently a single country, and occasionally several countries but usually within the same region. The balance is geared towards topics of a regional or global nature. From the information given in the title, one third was given to fund workshops or conferences. No other trends in use are easy to identify. Small proportions have been used for studies (7 percent), impact (4 percent) and about 10 percent for work to do with innovation, partnerships and knowledge sharing.

3.122 Slightly fewer than ten agricultural research grants have been awarded most years, ranging from US$250,000 to US$1.6 million with an average value of $1 million. Awards are generally made to cooperating research institutes and many grants identify specific countries to which the research is expected to be relevant. The regional allocation over the whole period has been 36 percent to Africa, 25 percent to Asia, 22 percent to Near East and North Africa, 7 percent to Latin America and the Caribbean, and 10 percent allocated for cross-regional work. Agricultural research grants are managed by the PT division.

3.123 Table 12 classifies the agricultural research grants by subject and region. The cells show the amount of the grants in thousands of US dollars. A superscript indicates the number of grants awarded if more than one.

Table 12 Distribution of grants for agricultural research

US$ ‘000 Africa Asia NENA LAC Multi-region

Rice (& fish) 2,750 (3) 1,585 (2) Maize, Sorghum, Millet 3,000 (3) 750 2,000 (2) 1,211 Durum 2,400 (2) Grain legumes 2,600 (2) 1,300 Roots & tubers 3,750 (3) 2,800 (3) Underutilised crops 1,410 Bamboo & rattan 2,195 (2) Jute 400 Genetics 800 Tree crops & forestry Agro-forestry & indigenous trees

2,080 (2) 1,000

Coconut 907 Pests & diseases 2,600 (3) 500 1,000 1,000 Soils (+ saline & brackish water)

2,435 (2) 1,000 400

Livestock 1,500 2,000 (2) 1,250 Apiculture & sericulture 2,900 (2) 700 Camel 1,200 Fisheries 650 Animal health 4,258 (5) 875 Strategy development 2,765 (3) Upland poor 3,950 (3) Technology adoption, mixed farming & food security

600 1,500 2,160 2,117 (2)

Socio economics & non-farm 1650 (2) 931


Page 44


Research & HR development 1,670 (2) 3.124 The programme has covered a wide range of topics and settings. Overall, highest priority has been given to ma ize, sorghum and millet, followed closely by roots and tubers and technology adoption. In Africa the highest priority has been animal health, mainly research into trypanosomiasis, followed by roots and tubers, and maize, sorghum and millet. In Asia, research into the upland poor has been the leading recipient. In NENA regional strategy and planning for aquifer management is first, followed by work on durum. Least expenditure has been directed towards Latin America and the Caribbean.

3.125 The programme of grants for Other Research and Training is similar in size to agricultural research, and a similar but slightly smaller average size. Grants under this facility have been allocated to a very wide range of recipients and for diverse purposes. Table 13 summarizes and illustrates their use.

Table 13 Details of Other Research and Training Grants

Unit or Division

Total grants Programmes for grants >=US$1 million

External Affairs $5,525,000 • Implementation of the Convention to Combat Desertification (to UNCCD)

• Community empowerment facility expansion and secretariat operating costs (to PCEHP)

Global Mechanism

$5,000,000 • Grants to support action programmes (to GM/UNCCD)

International Land Coalition

$500,000 • Programmes and Partnership Activities

West & Central Africa

$8,150,000 • Regional environmental management information project for the Congo Basin (to ADIE)

• Market driven initiative for millet and sorghum development (to SG2000)

• Sustainable vegetable production and marketing (to RADORT)

• Rural development HUB – a tool for improved food security and rural poverty alleviation (to UNOPS)

• Regional information network (to UNOPS) PD $1,200,000 • Project completion for PCEHP and a programme to promote

impact orientation during the project cycle East & Southern Africa

$15,811,409 • Agricultural management training programme for Africa (to OAU/STRC)

• Rwanda returnees rehabilitation (to Rwanda) • Strengthening the framework for the provision of investment

capital for the rural poor (to AFRACA) • Farmer field schools (to FAO) • Programme to promote sustainable use of trade in natural

products through SANPROTA (to SAFIRE) • Rural knowledge network pilot project (to FAO) • AFRACA development programme (to AFRACA)

Asia & Pacific $4,570,000 • Livelihoods in uplands and mountains of Hindu Kush Himalayas (to ICIMOD)

• Programme for electronic networking for rural Asia/Pacific (ENRAP) (to IDRC)

Latin America & Caribbean

$16,823,000 • Strengthening regional capacity for M&E of rural poverty alleviation projects PREVAL (to DESCO)

• Regional unit for technical assistance (to RUTA)


Page 45

Unit or Division


• Many projects a little less than $1 million including Institutional and policy support, and poverty reduction in the MERCOSUR region; Regional rural development training (to PROCASUR); Alleviation of rural poverty in the Caribbean (to CA RUTA); FIDAMERICA (to RIMISP); and support of indigenous peoples in the Amazon basin (to CAF)

Near East & North Africa

$14,667,000 • Near East and North Africa Management Training in Agriculture (to NENAMTA)

• Backstopping support to IFAD projects (to ACSAD) • Methodologies and approaches for effective introduction of

PIM (to CIHEAM) • Relief and development in Gaza (to ANERA) • Marine resources management in the Red Sea (to FAO) • To develop a knowledge generation and sharing network (to

IDRC) PT $2,360,000 • Support to micro-finance (to CGAP) 3.126 From the data available in the PT database the grants appear to be relevant to the policy objectives over the period. But there is a lack of any evidence of strategy or prioritisation in the work funded. The guidelines for grant criteria in 2002 stressed that grants should lead to significant measurable impact on IFAD’s rural poor clientele. Most grants pass through intermediaries. Many of the OR&T grants are used for programme development. Some have been used for support to IFAD’s programme implementation. The LAC region has had a substantial programme dealing with M&E and support to implementation. Despite such diverse uses there does not appear to have been any systematic evaluation, nor review of the strategic implications or lessons to be learned for IFAD as a whole.

Other grants

3.127 Technical assistance grants comprise more than just the three sets analysed above: small grants, agricultural research, and other research and training. IFAD’s budget makes special provision for grants to the NGO Extended Cooperation Programme, to the Special Operations Facility and for Environmental Assessment. In addition, some loan projects have grant elements, the Belgian Survival Fund makes grant allocations, and a large number of donors provide supplementary funds linked to loans. Annex 4, Table 6 lists the main components of grant disbursement from IFAD’s Loans and Grants System database.

3.128 Analysis of the LGS database reveals a number of interesting features about the grant programme:

• It is very diverse with more than ten types of facility some of which mix support to final clients with support for programme design and implementation.

• As a percentage of annual loan disbursement, grant spending accounts for around 11 percent, significantly higher than the ceiling on new approvals, mainly because this analysis includes supplementary funds (which are extra-budgetary).

• Technical assistance grants form the largest single category at 47 percent. • Contributions from the Belgian Survival Fund have remained fairly constant in dollar

values; other donors’ supplementary funds used to cofinance loans, and loan contribution grants, have declined over the period. 37

37 The total volume of Supplementary Funds has fluctuated in the range US$8 million to US$12 million over the period. At the same time the proportion given in connection with loans has declined while that used for other forms of programme support has increased.


Page 46

3.129 Comparison of disbursement between the years 1995 and 2003 indicates a number of trends:

• Control of grants has become increasingly centralised, with fewer grants directly accessible by the regional divisions.

• The Belgian Survival Fund has had a significant positive influence on the level of grants going to East and Southern Africa and to NENA.

• The decline in supplementary funds for cofinancing is most marked in Asia, Latin America and NENA. Supplementary funds for Africa have increased.

• The allocation of supplementary funds has reversed from being primarily cofinancing at the beginning of the period, with a small amount for consultancy inputs and thematic work (such as gender, environment) to the opposite at the end of the period with most funds being for consultancy and thematic issues.

• New facilities have developed over the period. The ‘other’ category has increased fourfold, largely through the Global Mechanism.

3.130 Overall the situation with grants is one of complexity and a lack of clarity about how different mechanisms relate to each other. The multiplicity of facilities increases the transaction costs for IFAD, with over 1500 disbursements identified in the LGS in 2003. There is no consolidated regular monitoring analysis of the uses to which grants have been put. The variety of mechanisms is a challenge for CPMs who have to become entrepreneurs in juggling funds. A wider problem is evident for IFAD as a whole to manage the grant process with scattered and incompatible data sources, rules and procedures.

Summary of issues

• Small TAGs are most numerous, averaging US$50,000 and mainly directed towards single countries

• Around 8 to 10 agricultural research and a similar number of Other Research & Training grants have been awarded each year, averaging US$800,000 to US$1 million

• The agricultural research grants have been used for diverse purposes but in accordance with IFAD’s mandate and policies

• The diversity of OR&T grants is very wide, with uses ranging from programme support to relief and development

• The multiplicity of grants creates a complex web of sources, procedures and intended uses that in practice overlap and lack strategic coherence

• There is little or no data about the impact of the grant programme

Evaluation of four TAGS

3.131 Four grants from the Agricultural Research, and Other Research and Training categories were randomly sampled for this evaluation. The relevance, effectiveness and impact of other grants from the small TAGs category will be assessed during visits to sampled countries. Details of the four grants are set out in Table 14. The evaluation examined available documents including the Report and Recommendation to the President (RRP), design documents, progress reports and any dissemination or follow-on material. In addition, the OSC and TRC minutes were also examined. The documents were reviewed against a structured checklist, adapted for relevant questions from the checklist used for all sampled loan projects. Documentation for the EB in support of these grant proposals is deliberately


Page 47

brief but follows a similar structure to a project loan proposal. More details can be found in the design documents prepared by applicants, where these are available. For the four TAGs reviewed, a design document was available for the rice project but not the others. Various progress reports were also viewed and as much dissemination material and final reports as could be located.

Table 14 Details of the sampled TAGs

Year and Type Title

1994 Other research & training

Programme for the establishment of the Near East and North Africa Management Training in Agriculture (NENAMTA)

1997 Other research & training

Global cassava development strategy

1998 Agricultural research

Validation and delivery of new technologies for increasing productivity of flood-prone rice lands in South and South East Asia

1999 Agricultural research

Programme for poverty alleviation and enhanced food availability in West Africa through improved yam technologies

3.132 Relevance All of the grants are judged to be relevant to IFAD’s policies in force when they were approved. The rice project and NENAMTA both explain their relevance to regional strategy. The rice project is particularly good at demonstrating the link with loan projects.

3.133 Analysis Poverty analysis is not well treated in any of the projects. Rice and cassava both review the poverty context but the others have no analysis. Discussion of lessons from previous work is handled well in rice and NENAMTA; the treatment is poor in the cassava project despite prompting during the TRC and there is no coverage at all in the yam project. Institutional analysis is weak or non-existent in all projects, despite all the grants being implemented in multi-organisation settings.

3.134 Targeting and participation Three of the four projects make reference to elements of participation during design and to targeting of benefits. In none are the descriptions very detailed or clear. Reference is made to the planned use of participation during research, and to participation in project design workshops, but exactly who was involved is elusive.

3.135 Objectives None of the projects has a logframe. The rice grant has realistic and well-structured objectives and outputs. Others are lengthy, inconsistent between different sections of the document, and with poor structure. The logic of the intervention is well explained in both the rice and NENAMTA. Neither cassava nor yam has convincing explanations of how the results might be used or the scope for ultimate impact.

3.136 Risks, innovation and sustainability NENAMTA has a reasonable treatment of risks and a good section on the importance of cost recovery. The others have no discussion of risk or sustainability. Innovation is dealt with in the rice and yam projects although the latter is weak on downstream aspects.

3.137 Implementation arrangements All the projects set out the planned arrangements for management of the grant. But none give any analysis of the capacity and commitment of intended implementers. The role of partners is treated satisfactorily in the rice project but less well in the other three.

3.138 Monitoring and evaluation Three of the four projects have little or no discussion of monitoring arrangements; indicators are absent, institutional roles and reporting are weak. The rice project has a brief discussion and includes a plan for evaluation.

3.139 Implementation and impact NENAMTA, the oldest project, has seen the least progress. It was designed as a continuation of the Agricultural Management Training for Africa (AMTA) started by IFAD in the 1980s, but institutional complications created delays and progress has been limited to the development and testing of training material.


Page 48

3.140 The Cassava and Yam projects are not directly related, but should be viewed as part of a wider effort in support of roots and tubers in West Africa. This focus is entirely consistent with IFAD’s original mandate to work with foodstuffs of direct importance to poorer sections of society. The cassava grant has its origins in a large TAG for IITA in the mid-1980s for research on biological control of cassava mealy bug. Successful research led to a cassava multiplication programme being funded as part of a series of World Bank assisted Multi-State Agricultural Development Projects. Evaluations of those projects have linked a large expansion in cassava production in Nigeria to technological change. Subsequently IFAD embarked on a Nigeria Roots and Tubers Expansion Project that has been replicated in Benin, Cameroon and Ghana.

3.141 To date the outputs from the Global Cassava Development Strategy are just reports and the approach has been criticised as being too supply driven; but FAO appears to be continuing support and making progress as the lead agency.

3.142 Yam has been a relatively neglected area and IFAD’s support has helped develop research networks and informal thematic working groups. The apparent successes with release of new varieties, evidence of improved soil management, tools for field diagnosis of nutrient deficiency and low-cost seed health treatments, has led to a funding proposal by IITA for an IFAD grant of US$1.5 million to go to the April 2004 EB. Evaluation results from the first project have been used in preparing the next application. More work is needed to assess farmer benefits and to disseminate research findings.

3.143 The rice project with IRRI ended implementation in December 2003. A short summary of key achievements indicates that most outputs in the form of location-specific technologies were achieved. In addition planned socio-economic studies were accomplished and a number of technologies with potential for scaling up have been identified. Results available at IFAD headquarters do not include evaluation information from the perspective of potential beneficiaries. The project has published thirteen Technical Advisory Notes to disseminate research findings. This is the only one of the four projects with a clear emphasis on research dissemination.

Summary of issues

• The four sample TAGs all appear to be relevant, but most project designs are not well presented. In particular, objectives are mostly weak with no indicators or convincing arrangements for M&E. Despite all the projects working with partners there is no institutional analysis or assessment of capacity.

• NENAMTA has experienced major delays. The other projects have delivered their outputs but there is no independent evidence of effectiveness and impact.

• The experience with cassava shows the scope for programmatic actions that draw together different types of financing towards a common objective

Other evaluative evidence

3.144 In June 2003 the Office of Evaluation published an evaluation of the agricultural research grants programme.38 The main conclusions were:

• IFAD has played a strong advocacy role in redirecting the focus of the CGIAR system towards more poverty-focused research

38 IFAD (2003) Evaluation of IFAD’s Technical Assistance Grants Programme for Agricultural Research. Report No. 1377. Office of Evaluation


Page 49

• TAGs are usually well designed and overall they have been reasonably effective in achieving stated objectives

• Information about poverty impact lacks impact assessment studies • The programme remains relevant to the IFAD poverty mandate • Impact is clearest in terms of institutional capacity-building

Discussion and questions for the country visits

3.145 This review of the grant portfolio has highlighted a number of areas that are important both for grant management and for future policy. Firstly, the evolution of instruments and procedures during the past ten years reflects the importance of the grant programme and the desire by management to develop a coherent link with loans. But the protracted delays in developing the new policy and low attention to evaluation of impact has resulted in a new policy which lacks self-critical reflection. OE’s evaluation of agricultural research grants is cited in the new policy but without building on the critical discussion in that study.

3.146 Secondly, neither the new grant policy nor the policy statements during the period pay sufficient attention to the ways in which the small grants and OR&T grants have been used in support of programme development and implementation. A key recommendation of the 1994 Rapid External Assessment was for IFAD to shift resources from the design phase of the project cycle to implementation support. The grants programme is one way in which the regions have attempted to do that, albeit in the context of a complex and changing array of funding instruments. This role has not been fully explored and does not feature in the new policy.

3.147 Last, there is little systematic assessment of the grants being used to fulfil objectives of knowledge sharing, innovation and partnerships. These three elements describe key features of the way IFAD tries to work and are central to corporate strategy.

3.148 Fieldwork during the country visits will develop the issues identified here in a number of ways. How have the TAGs been linked to the loan programme and used in support of IFAD’s country programme; how effective have they been in achieving stated objectives; and what is the evidence of impact on IFAD’s client group? Picking up on the objectives stated for TAGS in recent years, the field investigations will also examine to what extent the TAG’s have:

• Enhanced IFAD’s catalytic role, extended outreach and helped forge partnerships with country-level institutions at both field and policy levels

• Contributed to rural institutional dynamics for participatory rural development • Helped achieve a high-impact lending portfolio • Enabled knowledge gathering and sharing on rural poverty • Supported agricultural research and technology innovation

4 Evaluation of corporate processes

4.1 This chapter reviews four key areas of corporate process. First is the development of policy. The distinction between corporate policy and strategy is somewhat blurred and the material here needs to be read in conjunction with Chapter 3. This section goes further, however, in examining operational policies to guide loan and grant practice. Next are reviews of knowledge management and the use of partnerships. Last, is a preliminary examination of human resources in IFAD, their management, staff and consultancy complement and corporate culture.


Page 50

Policy Development

Introduction

4.2 This section presents an analysis of the impact of corporate policy in IFAD. It builds on the review of corporate strategy presented in Chapter 3. The section details the evolution of specific operational policy initiatives (since 1994) and their relevance and effectiveness, but the bulk of the work on influencing will be undertaken in the next phase of the evaluation. The analysis focuses on the period since 1994. The main hypothesis to be addressed is:39

i. In response to strategy development during the 1990s IFAD has become a leader and exerted policy influence from pioneering and innovative activities.

Operational policies

Table 15 Policy-Related Initiativesa 1998-2004

Year Initiative

1998 IFAD Framework for Bridging Post-Crisis Recovery and Long Term Development 2000 Rural Finance Policy 2001 Rural Poverty Report 2001 2002 Rural Finance Decision Tools 2003 Rural Enterprise Policy 2003 Mainstreaming a Gender Perspective into IFAD’s Operations: Plan of Action 2003-

2006 2003 Managing Impact for Rural Development – A Guide for Project M&E 2003 IFAD Evaluation Policy 2003 IFAD Policy for Grant Financing 2003 Sourcebook on Pro-Poor Institutional and Organisational Analysis and Change * a All papers presented at the Executive Board/Governing Council except for those marked * 4.3 Three areas where there has been a recent push in preparing policy or operational guidance are the Decision Tools to accompany the Rural Finance Policy, the Guide to Project M&E and the Gender Plan of Action. The latter was required as part of Replenishment VI undertakings. The M&E Guide is part of the overall effort to enhance impact achievement, while the Decision Tools is intended to support greater effectiveness and impact in micro-finance operations (Box 2). All three initiatives involved consultative processes across the organisation. All three are quite widely quoted by staff as good examples of operational policy development.

Box 2 Institutionalising Policy Guidance on Rural Finance

In 2000 IFAD/PT set about preparing a policy paper on rural finance. The process was consultative involving staff from across the geographical divisions. The policy paper itself is centred on three broad but strong messages: (i) the importance of building sustainable rural finance institutions; (ii) the importance of local ownership and (iii) the importance of combining outreach and sustainability. Following the publication of the policy paper, the Executive Board expressed a desire to see a more operational tool developed; one that reflected international best practice in supporting micro-finance. In response, PT set about preparing the Decision Tools in consultation with IFAD staff and external partners (CGAP etc). There were concerns expressed internally that the tool-kit should not be overly normative; it had to reflect important cross-cutting issues, regional and country specificities, and the IFAD project cycle. To be useful, the Decision Tools had to set out key pre-requisites for establishing

39 See Table 7 of the IEE Inception Report


Page 51

sustainable micro-finance institutions without being overly prescriptive. This was a challenge, but the final result is a tool-kit that is widely regarded by staff internally and by the external community, as evidenced by a recent DAC Peer Review of IFAD’s work on micro-finance. The Decision Tools are considered to be a major vehicle for internalising the rural finance policy paper in IFAD. The Peer Review notes some key ways in which this could go further, including the need to develop an agency-wide vision for its new approach to rural finance; clear mechanisms to manage for results, increased internal technical capacity and mechanisms to facilitate learning within and across regions, between the field and Rome and with external partners. 4.4 Notwithstanding recent efforts, large lacunae still exist in IFAD’s internal policy framework. This has partly been exposed by the limited operational value-added of the Strategic Framework and the lack of obvious mechanisms for developing policy messages from IFAD’s own rich knowledge base – its project and programme portfolio. A new Policy Division has been created to facilitate a more systematic process of policy development across the organisation and to support engagement with international development partners. A central Policy Forum has also been established to help identify and provide oversight for the development of policies in IFAD. Current plans are for the Policy Forum to meet twice a month with a variable membership (besides the chair and Policy Division Director) convened to address specific policy topics or themes. The aim is to create a clear institutional space for discussion of policy topics relevant to IFAD, both internally and externally.

4.5 These initiatives need to take root before they can be evaluated. Clearly they reflect a response to the already identified gap in IFAD’s capacity draw on its own knowledge base and articulate operational policies for greater development impact. They are also intended to take forward the Strategic Framework commitment to engage further in policy dialogue as a way of creating conducive policy and institutional environments for rural poverty reduction. But how shared is the understanding of ‘policy dialogue’ in the institution? What is meant by pro-poor policy? Does IFAD have a view? The Review of Regional Strategies and COSOPs would suggest that the intellectual framework for supporting pro-poor policy dialogue is not entirely clear.

4.6 The aim of becoming a leader in rural development policy and strategy and to exert policy influence based on pioneering and innovative activities is still largely an aspiration for IFAD. Recently this has gained more attention, but in few cases do documents make it clear just how IFAD intends to pursue its leadership role – whether financially or intellectually. Weak knowledge systems for learning from field-based activities and a relatively weak internal policy culture make it difficult to imagine how this leadership role will develop without substantially more investment. Getting the right balance between developing in-house capacity for policy development versus being an intelligent customer of existing global knowledge on rural development is likely to be crucial for a small organisation such as IFAD, an issue to be pursued in the next phase.

Knowledge Management and Learning 4.7 IFAD has always required and pursued good information and knowledge management but has not always made corporate statements about this. A major attempt to make knowledge management an explicit part of IFAD’s work was made after the Rapid External Assessment report in 1994. IFAD then pursued it with some energy and drive from senior management from about 1995 to 2002. Some of the methods were effective and some were not well suited to the organisation. The documents of IFAD during this period contain a large number of different formulations of aims and objectives for the organisation. During 2003 the phrase ‘knowledge management’ virtually disappeared from IFAD documents. The many initiatives of the previous three or four years have lapsed but the many declared objectives remain as corporate policy and continue to appear on the public website.


Page 52

4.8 This desk review sets out to test the hypothesis that knowledge sharing through innovation management, learning processes, influencing and scaling-up has improved the effectiveness of IFAD’s programme impact and policy influence. This section provides a review of the declared objectives by IFAD on the purpose and functions of knowledge management; an inventory of methods and instruments for knowledge management in IFAD; an assessment of knowledge management within IFAD against different steps in a model of knowledge management; and a preliminary assessment of knowledge management in IFAD in comparison with a range of other development agencies. Issues are then identified for the subsequent stages of the evaluation. A fuller treatment of this topic is in Annex 6.

A Brief History

4.9 IFAD has always dealt with information and knowledge as an implicit part of the work. There are few descriptions of how information should be managed but the implication is clear even in the earliest statements of the organisation, including the Lending Policies and Criteria. In the early 1990s the Division of Information and Communication was charged with work that would be called knowledge sharing. It described its role as promoting IFAD’s views on key issues of rural poverty, drawing attention to the role of rural women and improving communications work that would support effectiveness of IFAD projects. The main pressure for a more explicit promotion of knowledge management came with the 1994 External Assessment (coincidentally, and for reasons that are not clear from the documentation, at the same time the Division of Information and Communication was disbanded). A sequence of events grew out of that report:

• Post-1994 investment to improve technology and infrastructure in headquarters • 1998 Strategic Framework ‘Meeting Challenges in a Changing World’ identified

knowledge management as one of five strategic thrusts • 1998 Introduction of Corporate Scorecards with learning and knowledge prominent in

one section (never, in fact, put into operation) • IFAD V Action Plan 2000-2002 identifies being a knowledge institution as one of

four major roles • 2000 creation of a Knowledge Management Facilitation and Support Unit • Development of a Knowledge Management Strategy (but subsequently never

formally approved) • May 2000 new policy on improved disclosure of documents • December 2000 each Division included specific plans for knowledge management in

its workplan • 2001 Regional Strategic Plans all make reference to the role of knowledge

management • 2001 an evaluation of innovation was strongly critical of knowledge management in

IFAD • 2002 External Review found slow progress in knowledge management but urged the

Fund to pursue work in this area • 2002 re-creation of a Communications Division • 2003 Office of Evaluation made independent of management

4.10 In attempts to understand the knowledge management functions in IFAD the IEE has assessed the multiplicity of declared objectives and purposes and the draft Knowledge Management Strategy developed in 2001/2. But despite the steam of initiatives listed above, the IEE team has not been able to identify a simple set of objectives that the Fund is genuinely committed to or which are shared by staff interviewed.

4.11 A tour of the IFAD website makes it appear that knowledge management is still a current slogan and focus of work. But in fact, staff consider that the phrase has lost its


Page 53

meaning and prestige in the organisation and the initiatives developed under knowledge management have been allowed to fall into abeyance. There is an apparent drift away from knowledge management as a current term in corporate communications and in staff discourse. Talking to staff at IFAD headquarters adds to the feeling that this is an uncertain time in which senior management has lost confidence in the knowledge management approach and is replacing it with other approaches to the same issues that remain central to the core business of IFAD. But these new initiatives do not seem to be part of an explicit or highly visible approach.

Instruments and initiatives

4.12 One of the difficulties both for promoting and evaluating knowledge management is that the sources, media, targets and flow of information are disparate and diverse. It’s a complex area, with many parallel and overlapping elements. Our review of the means of knowledge management has identified a wide range of components. Table 16 lists these and sets out a simple structure.

Table 16 Elements of knowledge management

Sources Processes Storage Dissemination Targets

Project reporting and supervision

R R

Project M&E R R Other evaluations R Thematic groups R R Project design team R R TRC/OSC R R Core learning partnership R R Agreement at completion point

R

Databases: PPMS; LGS; TAGs; NGO-ECP

R R

Information Resource Centre

R R R

Website – portal R R EKSYST R Mailing R Video-CD Rom-Audio R COSOP formulators R R Project designers – staff and consultants R R Project implementers R R Partners R R Clients R R

4.13 Several points emerge from the groupings in this table. Firstly that the elements include activities – project supervision, TRC meetings; products – evaluation reports, databases, CD-Roms; and people – staff, consultants and partners. Secondly, many of these elements have dual functions. A thematic group is a means of organizing exchange of information both to learn and to disseminate. CPMs are both sources of information and potential targets to learn. A simple model is a useful way of interpreting this complexity. Figure 13 is built up from the component tasks that IFAD sets itself.


Page 54

Figure 13 Our understanding of knowledge sharing – a model

IFAD projects

1

Other sources

2

Identify

3

Store

4

Disseminate

5

IFAD projects

6

Other targets

7

4.14 The model identifies that information needs to be collected from both external and internal sources, assessed for value, held in a storage medium and disseminated to targets that are both internal and external to IFAD. Testing the model against the activities in Table 16 has led the evaluation team to some preliminary findings. Firstly, performance is better for the tasks of storage and dissemination than for data gathering and assessment. The major source of information for IFAD is project performance through project M&E, supervision and evaluations. Analysis of loan and grant projects reported earlier reveal that M&E designs are uneven in quality. Some reports suggest that M&E is very poor and occasionally almost absent. Both reporting and supervision are seen as areas where improvements are needed. The recent reforms to OE and advent of a technical guide to help project monitoring and other information flows are all positive steps to better performance. But it is too early to form a judgement about likely success. It is salutary that although IFAD took a lead among IFIs with a self-critical review of M&E in the late 1980’s, that review failed to stimulate better performance.40

4.15 Secondly, high demands to assess the relevance and merit of new information fall on PT and the IRC. PT is heavily overstretched in trying to service TRCs and provide general support to project and TAG development. Time pressure is a major constraint to sorting and managing new information.

4.16 Core Learning Partnerships (CLP) and the Agreement at Completion Point (ACP) are two important initiatives to bring together people who are stakeholders in the findings of a project evaluation. Both offer potential to improve the effective use of learning. But the few examples of ACP seen by the evaluation team suggest that the process does not work well where there is disagreement between the parties. Several examples demonstrate a modification of language, presumably in order to achieve agreement among the parties, that sufficiently blunts the message that a key finding is in effect misrepresented.41

4.17 Storage and retrieval exhibits mixed strengths and weaknesses. In addition to the IRC a variety of databases are available, ranging from organisation-wide systems such as PPMS and the Loans and Grants System (LGS) to special interest resources such as the (now defunct) NGO-ECP database. The LGS is kept up to date and reflects the most-recently posted financial situation. PPMS is an operations management tool that has never achieved its potential. It is managed by PMD and contains a wealth of information about projects. But modules designed to record the use of consultants and track effort across the project cycle were never implemented. The system is also regarded as being in parallel to the actual work 40 Published in Coleman (1990). The original internal IFAD document has not been located. 41 The 2003 Evaluation Policy restates the objectives and mode of working of the CLP and ACP, but still does not clarify how disagreements are handled (paras. 33 and 46).


Page 55

of operations – data storage rather than a workflow and decision tool. Both the PPMS and LGS suffer from clumsy menu-driven interfaces and provide fairly limited reporting for the infrequent-user. It is notable that a stand-alone system was written to manage grants under the NGO-ECP, rather than use PPMS.

4.18 Dissemination has seen more innovation, with a growing range of products such as Technical Advisory Notes, and Evaluation Profiles and Insights. The intranet provides a rapid service for staff and the 2000 policy on disclosure has brought more documents into the public domain though the external website. Though with the current search facility, some prior knowledge of content, skill and patience are needed to get the best out of the intranet, and archived material is not accessible through the main search facility. Electronic storage and access has been a boon to internal users, but provides only a passive facility to external partners. None of the means of dissemination appear to have been evaluated or even routinely monitored.

IFAD’s learning culture

4.19 Management and dissemination of information work most effectively within small, dedicated groupings, such as the PDTs and thematic groups; or for formal processes such as reporting to the executive board and general council. But many aspects of IFAD’s management culture act in opposition. These are discussed in a later section under human resources and management, but the key elements are: excessive working within departments or divisions and minimal cross-departmental interaction; a preoccupation with grade hierarchies and seniority; top-down communication from senior management with no mixing across boundaries; and a reluctance to be self-critical. The working environment does not foster intellectual challenge to stimulate learning and innovation nor even provide the means for casual meetings.

4.20 Comparison of IFAD with the knowledge sharing experiences in other similar agencies suggests that IFAD has performed to a very similar standard in terms of setting objectives and in many aspects of the approach taken (see Annex 6, Table 2). The changing organizational arrangements have almost certainly undermined some aspects of performance, but the key characteristics that define IFAD’s achievements appear to be the influence of corporate culture. Staff are left with a strong sense of fatigue over calls for better knowledge management and means of working that neither stimulate nor reward learning.

Summary of issues

• Knowledge sharing is for a purpose. But there does not appear to be a common understanding of the various formulations of advocacy work: the catalytic role, policy dialogue, innovator in know-how, leader in poverty allevia tion strategies, sharing of knowledge to multiply impact, and being a knowledge organisation.

• There seems to be no shortage of instruments and skills available for good knowledge management. Weaknesses are in planning and monitoring of the use of the instruments.

• The effectiveness of communications and learning is significantly reduce by the boundaries that exist within the corporate culture: vertical ones, between divisions and departments; horizontal ones reflected in the hierarchical structure that separates senior management, professional and general service staff.

• Questions to pursue during the next phase of the evaluation include the extent to which boundaries and defensiveness prevent open sharing of learning from failure and weakness; and whether those same failings also undermine sharing of good


Page 56

examples?

Organizational Partnerships

4.21 Working in partnership is one means by which IFAD can contribute to achieving its strategic objectives. By partnering with others, IFAD should be more effective in delivering its strategic framework, ensuring programme impact and fulfilling its catalytic role.42 Ultimately, working with the right partners should allow IFAD to better ‘enable the rural poor to overcome their poverty’ (IFAD 2002: 9) 43 – and more so than could otherwise be achieved acting alone.

4.22 The main hypothesis is that, ‘the strategy of working in partnership with others has enabled IFAD to be more effective in delivering its strategic framework and ensuring programme impact and fulfilling its catalytic role’. This hypothesis is based on paragraph 72 of the VIth Replenishment. The Vth and VIth Replenishment documents provide the basis for assessing the relevance, impact, effectiveness and efficiency of IFAD’s partnerships. Evidence is drawn from a series of semi-structured interviews and a review of the documentation available. Due to the sheer number of relationships (across corporate, regional and operational levels), the focus is on a subset of these, including: NGOs and civil society; International Land Coalition (the Coalition); Global Environment Facility (GEF); Global Mechanism (GM); WFP/FAO; AAAID and OPEC; and the Belgium Survival Fund (BSF). NGO relationships will be followed up in the country visits.

4.23 The main issues follow the structure of questions in the Inception Report (Table 7): what are all the relationships that IFAD currently terms partnerships; what is the rationale and relevance for IFAD; to what extent do the partners share a united approach; what evidence is there of effectiveness and impact; and lastly the efficiency of IFAD’s management and coordination. An extended treatment is in Annex 7.

Defining partnerships

4.24 Partnerships can be distinguished from both market-based relations determined by contract, and relationships in the public sector involving hierarchies based on authority. Two common characteristics are key: a sense of mutuality and equality among those involved, including trust between the partners; and a mutual commitment to agreed objectives, reciprocal obligations and accountability. These reflect unity between individuals and organisations, and the direction (or overarching goals) of the partnership.

4.25 For IFAD, the partnerships, and the objectives of working in partnership, are set out in the Vth and VIth Replenishments (see Table 17). They differ in the way they approach partnerships. The Vth Replenishment mainly considers partnerships in the context of using country and regional strategies as an entry point to consolidating existing partnerships and constructing new ones. It states a number of partnership objectives, such as sharing institutional capacity and underpinning macroeconomic reform programmes. The VIth Replenishment provides more of an overview of the areas in which IFAD works in partnership (e.g. multi-stakeholder coordination through UNDAF). The VIth Replenishment is more implicit about the objectives of ‘working in partnership’, and there is considerable overlap with the Vth Replenishment on issues of co-financing and illustrating a catalytic role.

42 As stated in the VIth Replenishment document (IFAD 2002: paragraph 72). 43 IFAD (2002), Strategic Framework for 2002-2006: Enabling the rural poor to overcome their poverty, March 2002, IFAD, Rome


Page 57

Furthermore, it is not made clear whether the VIth Replenishment replaces the 5th, or adds to it, and if so in what ways?

Table 17 Summary of partnership objectives in the Vth and VIth Replenishments Key objectives

1. Replicate and scale up through co-financing 2. Complement or underpin macroeconomic reform programmes 3. Share institutional capacity and comparative advantages V

th

Rep

leni

shm

ent

4. Exchange knowledge on policy and practices 5. Increase multi-stakeholder coordination 6. Effective in creation of sustainable, productive private sector investment in

rural areas 7. Leveraging resources through co-financing

VIt

h R

eple

nish

me

nt

8. Playing a catalytic role beyond the country level 4.26 The Vth and VIth Replenishments cite several very different partnership structures: from the joint programme with WHO, UNICEF and UNDP for the Belgium Survival Fund (BSF), to the plethora of relationships with the private sector and civil society organisations at the operational level. The External Affairs Department (EAD) also uses the term widely, and indeed it provides a cornerstone for EAD’s work programme in 2003.

4.27 The range of relationships which are labelled ‘partnerships’ is both numerous and diverse. At the corporate level, these extend from the hosting of formal secretaria ts (such as GM), to relations with other International Financial Institutions (like the World Bank), to coordination mechanisms of the United Nations such as UNDAF. Within borrower countries, the diversity is just as broad, ranging across macro, meso to micro levels (see Annex 7, for listings and a typology).

4.28 Applying the ‘partnerships’ label so widely suggests a lack of definition and focus in the way it is applied. In publicity terms this may not matter, but with respect to the strategic prioritisation and investment required by such relationships it is less useful. For a relatively small IFI such as IFAD, the broad-brush approach may only serve to undermine the value that could be gained from working in partnership with fewer organisations, but ones that are able really to contribute to IFAD’s strategic objectives.

Relevance to IFAD objectives

4.29 Most of the key partnerships (GM, GEF, the Coalition, NGOs, BSF, OPEC and AAAID) appear relevant to IFAD’s Strategic Framework. The ease with which IFAD partnerships can seem highly relevant at the broad policy level and yet less so for programme operations is also due to the lack of specificity in the IFAD Strategic Framework. The three strategic objectives of IFAD are very broadly defined, covering financial assets and markets, human and social assets, and productive assets and technology. Likewise the Vth and VIth Replenishment documents do not provide a clear and comprehensive strategy for the selection and prioritisation of partnerships. Instead the objectives cover everything from macroeconomic reform, co-financing and scaling-up, to multi-stakeholder coordination, knowledge exchanging and sharing institutional capacity.

Table 18 Initial assessment of relevance against IFAD partnership objectives

Examples of IFAD partnerships IFAD Partnership Objectives (Vth and VIth Replenishment)


Page 58

Rep

licat

ion

and

scal

ing

up

Mac

roec

onom

ic

refo

rm

Inst

itutio

nal

Cap

acity

Kno

wle

dge

exch

ange

Mul

ti st

akeh

olde

r co

ordi

natio

n

Priv

ate

sect

or

inve

stm

ent

Co-

finan

cing

Cat

alyt

ic ro

le

Global Environment Facility ¤ l Global Mechanism l ¤ l l l International Land Coalition ¤ l l l NGOs – corporate and regional levels

¤ l l

NGOs – country level operations l l l AAAID l l l ¤ FAO/ WFP ¤ l l l l

Notes: Contribution to IFAD partnership objectives: l = high; ¤ = partial. 4.30 Beyond a broad policy assessment of relevance, it becomes much more complex (but also potentially more important) to consider the relevance of IFAD’s partners at an operational level. Indeed much of what is considered to be IFAD’s core business (i.e. loan and grant projects), plus its key contribution to ‘enabling the poor to overcome their poverty’, occurs at the country level. For example, the Global Mechanism has strong relevance from a policy perspective but far less so operationally. In contrast, working with NGOs can appear very relevant at the operational level. In the NENA region for example, there is a great deal of state control with highly centralised governments and banks. In such an institutional context, state mechanisms can have a conservative tendency with limited proactive capacity, such as in participation. As project management units tend to be located within a relevant ministry, working in partnership with NGOs can enable IFAD to be more participatory as well as build capacity amongst government staff. Similarly, in Central and Eastern Europe. Overall, however, it does not aid clarity to describe NGOs and other sub-contracted project implementers as ‘IFAD partners’.

4.31 In view of the difficulty in resolving relevance to objectives, a more pragmatic approach might be to prioritise partnerships on the basis of their performance, for which the following questions are key:

• Are there clear objectives for the partnerships? • Can performance be measured against these objectives? • Is there evidence that the partnerships enhance IFAD’s impact?

Partnership unity

4.32 Mutual commitment to agreed objectives is an essential element to successful partnerships. Furthermore, clear, shared objectives are an essential element of being able to monitor and evaluate the effectiveness of a partnership.

4.33 There are many cases of formal agreements between IFAD and its partners, but less clarity over shared objectives to measure the performance of the partnerships. Examples of formal (almost contractual) relationships include most bilateral and other donors, such as the Partnership Agreement with the Government of Italy (GoI and IFAD 2001) and the MoU with the Inter-American Development Bank (GoI, IADB and IFAD 2003). These tend to be legally binding agreements, with limited evidence that they represent more organisation-wide, shared objectives. There are other agreements that offer a better basis for partnership, but still tend to concentrate on legal, procedural or administrative matters (such as for GM, GEF and the


Page 59

Coalition).44 In some cases, the objectives are cited in terms of the constitution of the entity, rather than being expressed as shared objectives for the relationship. 45

4.34 For NGOs, there is a notable absence of a clear policy and corporate or regional level framework for engaging with civil society. One of the goals of the annual IFAD/NGO Consultations was to, ‘provide a framework for strengthening IFAD’s collaboration with NGOs…facilitate the involvement of NGOs as IFAD’s partners and promote mutual cooperation.’ (IFAD 2000).46 This contrasts with the relations between IFAD and NGOs at the operational level. In the Evaluation of the ECP (IFAD 2000: 12),47 it was found that there were: ‘notable examples of very good partnerships, with evidence that ECP projects had indeed served to strengthen relations between IFAD and NGOs…in terms of mutual trust, understanding and working for a common purpose’.

Partnership effectiveness and impact

4.35 Partnerships are a means by which IFAD can enhance its impact, either directly or by performing a catalytic role. There is anecdotal evidence of impact, but little in terms of systematic analysis of partnership impact overall. For example, the evaluation of the ECP grants to NGOs (IFAD 2000, Op. Cit.: 5) focuses only on effectiveness, efficiency and relevance, and not expressly impact. In the evaluation, reasons for processing ECP grants given by CPMs show that there is a tendency to focus more on collaboration with NGOs in the context of loans and project implementation, rather than other issues such as piloting or innovation. The Executive Summary and Agreement at Completion Point (IFAD 2000d: 12) suggests there were notable examples of good partnerships, as well as, ‘remarkable achievements in testing and replicating innovative institutional approaches’.48 Yet in the Main Report (IFAD 2000e: 39) the evidence is less convincing about actual achievements, with some 90 percent of the implementation reports reviewed having inadequate or inappropriate information in terms of achievements – thus making it difficult to systematically consider the achievement of ECP projects.

4.36 Similarly for GM, GEF, the Coalition and AAAID, there is limited systematic measurement of impact, with reference generally made to anecdotal examples. The shortage of evidence arises in part from a lack of monitoring against clear, shared partnership objectives. One of the main mechanisms is through Progress Reports to the President.49 But the information contained in these reports is very variable and tends either to concentrate on the partner (e.g. the Coalition or GM) or be descriptive about the performance of the relationship.

Partnership management and coordination

4.37 Management and coordination arrangements range from formal to highly informal structures, with no single approach across the organisation. NGOs are coordinated and

44 For example, (COP and IFAD 1999) MoU between COP and IFAD for the Global Mechanism 45 For example, the Basic Framework for the Population Coalition (IFAD 2001b). 46 IFAD (2000), Highlights and News: 10th IFAD/NGO Consultation: Tripartite Partnerships for poverty alleviation and food security through programmes, 30 May – 2 June 2000, Pune, India 47 IFAD (2000), Evaluation of the IFAD/NGOs Extended Cooperation Programme: Executive Summary and Agreement at Completion Point , September 2000, IFAD, Rome. 48 Reference is made to two examples: i) from Peru (Grant No. 07), Strengthening Extension and Training Services; ii) from Burkina Faso (Grant No. 34), Savings and Credit with Education (IFAD 2000: 12). 49 For example, IFAD (2003), Progress Report on the Popular Coalition to Eradicate Hunger and Poverty, Governing Council – 25th Session, GC 26/L.11, IFAD, Rome


Page 60

relationships managed by CPMs at country level, but in very diverse ways. While NGOs are valued ‘theoretically’ within IFAD, the management and coordination at operational level has some weaknesses, such as:

• A lack of incentives for CPMs to put NGO related grants (ECPs) into their workplans.

• A lack of instruments through which to engage with NGOs, particularly in countries where loans are suspended (e.g. Togo), governments are weak, and where IFAD needs to work through civil society.

• No specific funds made available for follow-up, such as through the supervision of NGOs

• Limited inclusion of NGOs and civil society as part of a country portfolio review • Lack of initiatives to capitalise from the lessons of working with NGOs

4.38 Communication within IFAD across the range of partnerships varies, but there is a sense that there are pockets of staff concerned with certain partnerships while the rest of IFAD staff have little involvement. For example, beyond senior management, IFAD staff are not particularly well informed of the benefits of the partnerships with GM/GEF and the Coalition. Conversely, experience of NGOs at country level tends to be retained by individual CPMs, with little regional, cross-regional and corporate knowledge sharing.

4.39 For partnerships to function well, they often require a significant investment of time, though the added value of working in this way can be considerable – for innovation, piloting, implementation, etc. But the unstructured ways in which the relationships are managed means that lessons are poorly learned and rarely shared.

Summary of issues

• The widespread and undefined use of the term ‘partnership’ within IFAD, combined with the relatively all-encompassing objectives set out in the Vth and VIth Replenishments, may undervalue the real benefits that can be gained from working in partnership .

• IFAD’s Strategic Framework is defined in such broad terms, that it is difficult to argue against the relevance of a great many partnerships, even though they may not present the best opportunities for contributing to IFAD’s strategic objectives. In the past, IFAD has entered into, and become ‘host’ to, significant partnerships for which questions are now being raised about the real benefits to IFAD.

• While agreements exist between IFAD and many of its so-called ‘partners’, these statements are rarely clear on the shared objectives of the relationship. Furthermore, in several cases there is a limited institution-wide understanding of the benefits of some key partnerships (like GM, GEF, and the Coalition).

• Evidence of effectiveness and impact is mostly anecdotal and there is limited systematic monitoring of partnership effectiveness against shared objectives. There is no evidence that the partnerships are enhancing IFAD’s impact on rural poverty.

Human resources management

4.40 IFAD is a small organisation compared to other IFIs.. As expressed through the annual work programme and budget the Fund has endeavoured to constrain organisational growth so that it would be an efficient provider of development finance. But IFAD’s mode of operation, with a focus on project identification and design; restriction to a single global site; extensive use of consultants to provide technical expertise; and policies to be innovative, act


Page 61

as a catalyst and learn from experience all place demands on personnel management, staff numbers, and corporate culture. The evaluation has focused on these three aspects, to test the hypothesis that development of highly motivated, well-trained and efficient staff has contributed to IFAD’s performance.

Human resources management and strategy

4.41 HR management has suffered from prolonged neglect. Up to the end of the 1990s the HR Division had a mainly transactional role in recruitment and the payment of salaries and benefits to staff and consultants, applying the rules and guidelines of the Personnel Policy Manual and the HR Handbook. There was no human resource development strategy; core competencies for differing jobs or grades were not defined; performance evaluation was not systematically linked to training needs or promotion and managers received little or no training in human resource management. A Review Committee to Modernize HR Policies and Procedures was established in 2001. It identified a significant need for change in HR management within IFAD and drew up an action plan. The main ‘thrusts’ were to: (i) Define a new role of HR within the institutional and Strategic Framework context, while strengthening management HR capabilities through training in human resource management and performance evaluation. (ii) Develop a comprehensive HR strategy, with supporting planning for and monitoring of HR needs. (iii) Manage a sound recruitment process. (iv) Create, manage and monitor a career development system for all staff. (v) Introduce performance-based evaluation for all staff based on defined job families and associated competencies, highlighting exceptional or poor performance, with a clear linkage between individual performance and divisional work plans. (vi) Manage a sound recourse and appeals procedure.

4.42 The plan was approved by the Executive Board in 2002. The Director of Human Resources is now implementing a strategy that is consistent with the ICSC framework for human resource management. This has four main elements:

• Compensation and Benefits • Employment – recruitment, placement and retention, contracts and staff well

being • Career Management – staff development and training, performance management

and mobility • Good Governance – management style, staff representation, administration of

justice 4.43 The strategy strongly emphasizes performance evaluation. All staff will be assessed against the core competencies for specific job families. DFID is supporting the formation of a Development Centre for human resource development in the three Rome-based UN agencies. Assessment against the agreed competencies is being tested in the Centre in three pilot studies, starting in April 2004. Five staff from IFAD at three levels, senior managers, CPMs and P3/P4, will participate.

4.44 IFAD is also participating in a pilot programme to introduce broad banding, which will eliminate divisions between grades so that there is a career continuum for support and professional staff.50 The emphasis on performance is also to be seen in the planned pilot programme of pay for performance, rather than a standard annual within grade increase. It is planned that by 2005 there will be a within-grade increase, together with a performance bonus, and by 2007, increases will be entirely performance based.

Summary of issues

50 Three UN organizations are involved in this pilot, IFAD. UNAIDS, and WFP.


Page 62

• It is clear that management is working to achieve fundamental reform of human resources management.

• The approach is ambitious. At this stage there has not been sufficient progress for the evaluation to reach any judgement about the actual or potential achievements.

• The merits of the approach taken will be reviewed later during the evaluation.

Staffing and consultants

4.45 There are two main categories of staff in IFAD, professionals in Grades P1 – P5 and D1 - D2, and General Service Staff in Grades G1 – G7. In each category there are two types of appointment, regular and temporary. Temporary staff, who may be professional or support staff, are divided into those with fixed term conditions and an appointment for one year or longer, and those without fixed term conditions, appointed for less than 11 months. In addition to staff there are consultants, usually appointed for a fixed term that depends on the specific task to be undertaken.

4.46 Total Staff and Consultant Numbers51 There is some difficulty in reconciling the numbers of staff and consultants provided for in the budget, those derived from the payroll system, and the total number of people estimated to be working in IFAD by the Buildings Department. These figures for December 2003 are summarized below:

Budget Provision Total staff 397 Payroll Total staff 491 Building Department Estimate Total individual

workplaces 650

4.47 The discrepancy between budget and payroll figures can be explained by recruitment of temporary staff. The discrepancy between the payroll total and the number of individuals thought to be working in the IFAD buildings is largely due to the large number of consultants employed, more in number than their person-year equivalents. Hence more workplaces are needed. Overall, the estimates can be reconciled. It is a significant point, however, that the presentation in the Annual Work Programme and Budget, which stresses minimal budget growth over the past decade, does not convey the full extent and diversity of actual staffing.

4.48 The total number of professional and support staff and consultants employed by IFAD in 1994, 1999 and 2003 is summarized in Annex 8, Table 1. It shows that the total number of professional staff increased from 135 to 202, a 50 percent increase. Support staff numbers increased from 169 to 264, an increase of 56 percent. Over the same period, the number of consultant days increased from 33,394 to 48,373 an increase of 45 percent. If consultant days are converted into staff year equivalents there has been an overall increase of 51 percent in personnel over the ten-year period.

4.49 As a simple measure of efficiency, the number and values of loans and grants are shown at the foot of the table. The increase in staff and consultancy effort has not been accompanied by a growth in the volume of loans, though grants have increased somewhat. Overall, IFAD’s manpower efficiency appears to have fallen from 15 staff and consultant years per loan approval in 1994 to 26 in 2002/3. The implication is that the extra staff have been employed on other duties not directly related to loans and grants, or that the quality and

51 Source. Human Resource Department Records


Page 63

content of new grants and loans has changed. But such a dramatic increase in quality is not borne out by the portfolio review in Chapter 3.

4.50 Virtually all Divisions in IFAD have shown an increase in both staff and consultants. The largest expansion has been in the Office of the President, due mainly to the inclusion of the Office of Evaluation and some other units such as the Global Mechanism. Other Departments show an almost evenly distributed increase in professional staff of between 35 percent and 39 percent. External Affairs and PMD show the largest increases in support staff but the greatest increase of 73 percent in consultant use is in External Affairs.

4.51 IFAD makes extensive use of consultants. Staff/consultant ratios based on numbers of staff and equivalent consultant years reveal a fairly constant ratio of total staff numbers to consultants of about 0.40, just under 1 consultant for every two staff. If only professional staff are considered, the ratio is much higher, ranging from 0.85 to 0.95, close to one consultant for every staff member.

4.52 The main reason for the extensive use of consultants is to provide specialist expertize that regular staff do not have. This is clearly justified for various stages of the project cycle, given the wide range of sectoral and sub-sectoral activities included in IFAD operations and PMD has been the single largest user of consultants over the years. The continuity in use of consultants is uncertain and will be explored in the next phase of work.

4.53 The extensive use of consultants in other Departments in IFAD is also widespread. Again it is justified if it is to provide specialist inputs not available in-house. However, in administrative and IT support activities the distinction between regular on-going programs and specific tasks is less well defined and there is a danger that consultants are used as a de facto part of IFAD’s core staffing. Given the data available, it is not possible to make a judgement as to the extent that consultants undertake core staff work.

4.54 Gender Differentiation Analysis of women employed shows that the number of female employees in IFAD increased by 50 percent over the period 1994 – 2003, with the biggest increase of 124 percent in temporary professional staff, though there has been a significant increase of 44 percent in the number of regular women professionals. Similarly, the proportion of women consultants has increased by 50 percent over the period. There has also been an increase in the number of women in P5 or higher posts. In 1994 the proportion of D1 or higher posts held by women was 1 out of 22, in 1999 it was 3 out of 20 and in 2003 it was 5 out of 22 or 5, 15 and 23 percent - a steady, if unspectacular rise. At the P5 level, the proportions are 5 out of 42 in 1994, 8 out of 69 in 1999, and 13 out of 61 in 2003, or 12, 12, and 21 percent. By contrast, there has been little or no change in the percentage of women support staff, though this is at a high level of over 80 percent.

Summary of issues

• There has been an effective growth of 51 percent in staffing across all categories, professional, support and consultants across the years 1994 to 2003.

• When compared with the loans and grants programme over the same period, a simple but revealing indicator of efficiency, the ratio of person years to new loan approvals has risen. Of course, this does not take account of changes in quality or the growth of policy advocacy. This will be considered further during the country visits.

• There have been improvements in the gender balance of professional staff and consultants. But women are still more likely to be found in support roles and the main growth at professional level has been within the category of Chapter X, non-permanent appointments.


Page 64

Corporate culture

4.55 An examination of IFAD’s corporate culture was undertaken to identify staff perceptions of the constraints to achieving IFAD’s primary objective of poverty alleviation. It was not, therefore, an analysis of IFAD’s organization and management but an attempt to understand how corporate priorities, management style and skills, staffing patterns and corporate procedures affect staff morale and their ability to reach their goals. Most significant of all the factors affecting staff has been the seemingly endless programme of institutional and organisational change.

4.56 The introduction of an ambitious change programme over the past nine years reflects senior management’s concern with increasing the relevance, efficiency and effectiveness of IFAD. The change programme has had a very significant impact on IFAD’s work environment. There have been changes in strategic emphasis and policy dialogue, linked to institutional and management reforms, changes in key functional areas, and development of financial and human resource management systems. These changes are reflected in the development of the Strategic Framework 2002 – 2006, six regional strategies, the use of COSOPs to link regional strategies to country programmes and the creation of a new Policy Division, and an independent Office of Evaluation. Other changes underway include the introduction of a Performance Based Allocation System, a Results Management System, and the Business Process Re-Engineering Programme, now known as the Strategic Change Programme, which dates back to 1993. The ambitious programme of HR reform (see para 4.42) also involves very significant change, while changes in the organizational structure with the creation of the Finance and Administration Department and the External Affairs Department have meant changes to the way in which IFAD operates. There have also been important changes in senior management positions. Collectively, these have fostered an environment in which, according to staff attitude surveys, pride in working for IFAD has been undermined by low morale and high stress.

4.57 In view of the fundamental importance of staff performance the evaluation team has interacted with IFAD staff at many levels and has had frank exchanges that reflect the high level of motivation of most staff, but also the constraints and frustrations that affect them in carrying out their work. The observations from those interviews are summarized here and at greater length in Annex 8. The issues presented below are only those that were raised at different times by at least three different staff so they represent common concerns and not merely the opinions of an individual staff member. The language used tries to reflect accurately the language used by staff. This summary does not imply that the IEE team necessarily agree with or endorse all the concerns expressed, but they do have reality for IFAD as they are based on staff perceptions.

4.58 Senior Management Approach Senior management is seen as remote, and although communication has improved recently, there is insufficient vertical communication. A commonly expressed concern is that senior management does not appear to focus on IFAD’s core business of project formulation and implementation. The limited discussion of projects at Board meetings and declining senior management attendance at OSC meetings are seen as evidence of this lack of focus.

4.59 Hierarchy IFAD’s structure is seen as flat and non-hierarchical by senior professionals, with considerable autonomy and authority for operational matters. Junior professionals and most GS staff view IFAD as ‘very hierarchical to feudal,’ exacerbated by strong grade awareness. GS staff feel they lack technical or professional credibility within IFAD, even when they are working in highly specialized fields.

4.60 Management Skills and Team Work. There are concerns about the slow speed of decision-making and a considerable variation in managerial capabilities. Project Design Teams are the clearest example of working teams within IFAD but their effectiveness


Page 65

depends to a large extent on the CPM. Teams developed across Divisions or Departments are rare, though institutional working groups come together for specific purposes and some of the thematic groups have functioned effectively.

4.61 Knowledge Sharing . Although knowledge management is held to be a priority within IFAD, few staff feel there is adequate sharing of knowledge and experience, even within a division. There is a marked separation of divisions, referred to as the ‘silo culture’. Knowledge sharing is also constrained by a lack of open dialogue with a perception that staff are overly sensitive to criticism, with defensive attitudes even to constructive criticism.

4.62 The Change Agenda. There is an acknowledgement that the development of the SCP was based on a fairly participatory process, but a perception that it does not focus on IFAD’s core business and therefore has little positive impact on those engaged in that core business. There is a degree of cynicism that change programmes will be effective and a concern that introduced changes never come to closure. In addition, new initiatives are commonly seen to lack clarity of objectives, do not have adequate procedural guidelines, add to the workload and are not adequately resourced. This all adds up to feelings of change fatigue.

4.63 Operational Procedures. Staff suggest that these have not evolved to meet IFAD’s changing priorities or been linked to the lessons of experience. For example, experience of the need for re-structuring of projects at MTR has not led to changes in project design or phasing. In addition, there is a continuing focus of time and money on project design and little evidence that the Re-engineering led to any changes in the budget allocation to implementation, as opposed to design. Key elements include:

• The need to streamline the review and approval process and improve the quality and rigour of reviews

• Simplification of documents to improve their utility for implementation

• Increasing demands for detailed and high quality analytical work on poverty, institutions, economics and finance, incentives for beneficiaries, technical issues, and policies are not being met. Systematic problem analysis is generally weak and needs greater emphasis. The Technical Division is not able to provide the full range of expertize required to meet the varying demands yet there is little evidence of growth of enduring technical partnerships with external sources of niche expertise. The PDT is at present the de facto technical team for many projects, but their effectiveness depends on the CPM.

4.64 Intellectual Framework. There is no institutiona l focus to provide intellectual understanding or rigorous analysis of IFADs priorities and programmes. The newly established Policy Division could provide this focus, but there is some concern as to whether it will have the capacity to examine fundamental problems in addressing the country specific needs of target groups, in the way that Special Programming Missions, ended in the mid 1990s, were able to do. COSOPs are not seen as providing an adequate replacement, while Regional Economists are unable to fulfil SPM functions.

4.65 Human Resources. There is widespread concern over the use of temporary staff on a long-term basis and the use of consultants or short-term temporary staff to undertake work normally done by regular staff. This is seen as an inappropria te solution to manpower constraints. By contrast, dependence on specialist consultants for project work provides the required range of expertise and an important part of a CPM’s function should be to mobilize and manage this expertise. Another issue of concern at all levels of the organization is the lack of a corporate policy on non-performing staff.

4.66 Innovation and Risk. Although IFAD stresses the importance of innovation a definition of innovation was only produced in 2001. As noted in Chapter 3, staff hold differing views as to what constitutes innovation, from an initiative that has never been tried before, to a well-known initiative that is tried for the first time in an IFAD project in a


Page 66

particular country. Innovative programmes are by definition risky, but staff feel there is no clear management policy or guidance as to whether IFAD’s objective is relatively risk free resource transfer or the development of new concepts and programmes that may be inherently risky. There is also an acceptance that there is little capacity in IFAD for realistic risk analysis. Finally, for some, there is a perception that management is not concerned with risk.

Summary of issues

• The opinions of staff about IFAD’s organizational culture enable only tentative conclusions to be drawn at this stage. But the issues raised are vital for the healthy functioning of the Fund: the need for more participatory management, better communication, team building and knowledge sharing; review of the balance between project design and implementation; documentary requirements, technical support and intellectual capital building; and the implementation of the HR strategy.

• Human resources are fundamental to IFAD’s role as a knowledge-based organization, so issues that impact on the way staff perceive corporate culture affects their ability to achieve their objectives and are of crucial importance.

• It is important, therefore, to assess the validity of these views for a wider sample of staff during the remaining period of the evaluation.

5 Conclusions

Introduction and overview 5.1 This chapter sets out the conclusions that can be drawn from the desk review phase. It is in three parts. First an overview that highlights the evaluation criteria of relevance, effectiveness and efficiency. Impact is excluded because information on impact, both from policy influence and loan and grant projects, will be gathered during the next phase of the evaluation. Next is a treatment of conclusions structured according to the evaluation framework set out in the Inception Report. Last is a preliminary response to the questions posed in the terms of reference for the evaluation.

5.2 IFAD is an organisation that has successfully established processes and procedures to deliver a steady stream of 20 to 25 projects each year, focussing on the needs of the rural poor. This orientation presents IFAD with a challenge. Trying to work with the poor, often in remote or difficult locations, and working in a participatory way with beneficiaries must be one of the hardest challenges faced by any IFI: the most difficult target group, in the most difficult areas, with the most difficult approach. The test of IFAD’s performance will only come when information on impact is brought together and analysed. The preliminary conclusions in this evaluation need to be read in the context of this operating environment and with awareness that the picture so far is one drawn from a headquarters perspective alone.

Summary

Relevance

5.3 Analysis of the portfolio since 1994 and scrutiny of the sample of 21 countries and 42 projects shows that the current portfolio is in line with IFAD’s original mandate. A preliminary assessment of resource allocation suggests that distribution of loans has been geographically equitable. There is slight evidence that loans have been focussed on countries with sound policy and institutional environments, but that is a tentative finding. No other


Page 67

trends in selectivity have been identified and further assessment must await the country visits. The advent of corporate, regional and country strategies has brought clear benefits internally, with consensus around objectives. The sampled projects are well aligned with IFAD’s strategies, but this is thought to reflect the permissive nature of the strategies. Alignment with national objectives will be examined during the country visits.

5.4 Identification of loan projects appears to be driven primarily by dialogue with national policy-makers, followed by the regional strategies and then what appears to be an active policy of building on successful experience elsewhere. The Strategic Framework and Regional Strategies offer little added value to operations. They bring no guidance on selectivity and resource allocation.

5.5 Grants for agricultural research conform to IFAD’s mandate and reflect regional priorities and policy objectives, although there is a wide scatter of projects around these objectives. Other technical assistance grants appear diverse and less coherent towards country programme objectives. A complex mixture of sources of funds, instruments and modalities has resulted in the grant portfolio lacking any clear strategic relevance.

Effectiveness

5.6 Loan projects, and to a lesser extent grants, display a range of good design features. Targeting, poverty analysis, environmental analysis, beneficiary participation and local ownership are done well. Overall, objectives are well specified (and would be further improved by whole-hearted adoption of the logical framework approach), but there are major weaknesses in important areas. Institutional ana lysis is poor, as is treatment of risk and sustainability, assessment of management capacity of implementers and partners is weak, and the specification of performance indicators and monitoring arrangements is inadequate. A significant feature of the portfolio is the number of projects that require reformulation at the mid-term review.

5.7 The quality of project designs is mixed, but a preliminary comparison suggests that IFAD is grappling with many of the same challenges identified by the World Bank. Focus and selectivity are key issues.

Efficiency

5.8 The strengths in IFAD’s operations appear in aspects of design as noted above. Measures of efficiency are in the main less satisfactory. Firstly, IFAD has adopted some very demanding objectives: to be innovative, a leader in rural poverty reduction, and a catalyst for rural interventions. Yet none of these areas is characterized by clear statements of policy, strategy and implementation. The interpretations of what is innovative are so diverse in documents and among staff that they are meaningless. Efforts to develop a learning and knowledge management culture have foundered in lack of drive and support from senior management. The corporate culture actively mitigates against knowledge sharing.

5.9 Second, the basic requirement for successful learning (and management), a flow of good monitoring information from projects, is not being met. Despite being a forerunner among IFIs in taking a critical look at project M&E arrangements in the late 1980’s performance has not improved. Poor monitoring undermines effective supervision. The Project Status Report system is a good innovation to capture performance information, but the underpinnings are weak and there are significant examples of PSRs being less than faithful to the underlying supervision report. The quality assurance system also has a sound structure, but there has been a perceived lowering of interest by senior management. The Technical Review Committee process is not delivering objective peer review and the resources of PT are overstretched.


Page 68

5.10 Third, the overall efficiency of IFAD’s project delivery is falling. Total staff and consultancy manpower resources have risen by 51 percent since 1994, whilst the delivery of loans and grants has been largely unchanged.

5.11 One possible reason for the fall in efficiency has been the acquisition of resources for the Re-engineering and Strategic Change Processes. The impact of Re-engineering and the current performance of the Strategic Change initiative are both topics for further study under the evaluation.

Corporate performance

Policy influence and project impact - [Evaluation framework 1.1 and 1.2]

5.12 The work done so far has enabled the team to make a detailed assessment of policy and strategy development, which is summarised below. But the main focus of impact concerns the influence IFAD has at global and regional levels, and through the programmes of loans, grants and policy dialogue, at country level. All these aspects will be explored during the next phase of work in meetings with partners and in visits to sampled countries.

5.13 Findings from the desk review indicate that the aim of becoming a leader in rural development policy and strategy, and to exert policy influence based on pioneering and innovative activities is still largely an aspiration for IFAD. Recently this has gained more attention, but in few cases do strategy documents make it clear just how IFAD intends to pursue its leadership role – whether financially or intellectually. Weak knowledge systems for learning from field-based activities and a relatively weak internal policy culture make it difficult to imagine how this leadership role will develop without becoming much more selective. A relatively small organization can only offer leadership through developing a reputation for analytic and implementation excellence in selected niches.

5.14 Similarly, for project impact. The desk review has focused on evidence available at IFAD headquarters - mostly aspects of project cycle management and the quality of project designs. Available evidence suggests that the most consistently achieved results are increased food security as a result of higher yields and production of crops, livestock and fish, followed by increased access to physical and financial assets, and household incomes.

Portfolio analysis and resource allocation – [Evaluation framework 2.1]

5.15 The preliminary evidence is that IFAD’s portfolio has evolved in line with its original mandate. IFAD project financing is more or less equitably distributed across regions and seems to be oriented towards the better performing countries, with some important exceptions. Total project financing is more volatile however, driven by what appear to be significant variations in the level of co-financing over time and between regions.

Strategy development – [Evaluation framework 2.2]

5.16 In terms of relevance, the Strategic Framework, regional strategies and COSOPs all demonstrate a clear interpretation of the policy framework set during the Vth and VIth replenishments and are internally consistent with each other. Evaluation of COSOP fit with country circumstances is a matter for the country visits.

5.17 They have been effective to the extent that the strategic policies adopted since the late 1990s have provided a more systematic framework for project activity, country strategy and partnerships. The hierarchy of strategic objectives outlined in the Strategic Framework, Regional Strategies and COSOPs has helped to build clearer internal consensus and a ‘sense


Page 69

of mission’ within the organisation. Regional Strategies have provided clearer strategic guidance to country programmes and are providing a basis for expanding IFAD’s focus beyond traditional projects to greater engagement with sector programmes and policy dialogue.

5.18 Evidence of the efficiency with which strategies have been operationalized and been used to guide resource allocation is less positive. The Strategic Framework lacks clear operational added value. Regional Strategies fall short in terms of guiding selectivity and resource allocation at regional and country level. COSOPs, while improving in quality, still largely serve existing project ideas. They tend to be strong on analysis and on the description of targeting issues, but weaker on describing IFAD’s development niche and its catalytic role with other development partners.

Loans project cycle management – [Evaluation framework 2.4]

5.19 As prescribed in the terms of reference for the IEE, the greater part of effort during the desk review was spent in assessing the quality of project designs and evidence of supervision and results. The resulting data on 42 projects provides an interesting benchmark of project cycle management efficiency for IFAD. Overall, the analysis reveals a portfolio of projects that are improving in terms of design quality over the ten years studied.

5.20 Analysis IFAD is good at targeting. Beneficiary participation in design has improved sharply, which suggests that the relevance of IFAD investments to rural poverty reduction may be high – a point of importance for the country visits. Poverty analysis of the selected project area is good overall. However, institutional analysis and the foundation of project design within a clear understanding of the key determinants of the rural growth and poverty reduction dynamic in the project area are less strong.

5.21 Economic appraisal of project designs has not kept pace with the changing nature of IFAD investments. Focus on cost-effectiveness and potential for replication is weak. Aspirations are common, but practical design features and targets to ensure they occur are rare.

5.22 The innovative content of the portfolio is difficult to determine. There are many innovations for different tiers, particularly for villagers, within IFAD projects. But there are widely different interpretations of innovation and a lack of clear policy guidance. It is notable that risks are not well analyzed in project designs, despite the apparent focus on innovation.

5.23 Project definition, sustainability and management The definition of the project objective, outputs and the logical connection between them is satisfactory overall. But the definition of practical indicators and targets for tracking performance over time is both weak and only slowly improving. Introduction of the logframe has been half-hearted and it has not brought the expected benefits to project design and supervision. Although the nature and scope of project partners and implementers are described at length, appraisal of the implementation capacity, motivations and support requirements of project managers, partners and monitors is weak overall. Treatment of sustainability is also weak, with poor analysis of conditions for success.

5.24 Quality assurance The PDT/TRC/OSC process has been a valuable contributor to enhancing quality at entry. However, there is evidence of a decline in the effectiveness of the OSC reviews and some disquiet that project design teams are insufficiently rigorous to stimulate challenging peer review. Impressive efforts have been made to define and implement best practice in several sectors. But limited in-house expertise across the whole rural development field will always be a constraint. Either a more restricted portfolio focus towards particular sectors or greater use of external best-practice support will be required in enhancing quality over the future.


Page 70

5.25 Faced with a policy in the mid 1990s to constrain growth in the budget, one target was the cost of project development. Changes to procedures brought costs down, but task forces argued for a shift of resources away from formulation into early implementation support. Many would argue with that request. Experience by the World Bank’s Operations Evaluation Department is that it’s easier to correct a bad project during design than during implementation. Perhaps the consensus for early implementation support reflects IFAD’s frustration at not having resources for supervision. Either way, operations has ended up with the worst of both worlds, a reduced budget for project development and no funds for implementation support.

5.26 Overall, the quality at entry assessment suggests IFAD projects are both over-designed and under-designed. Appraisal documents, working papers and TRC discussions address many issues in considerable depth for every project. However it is not clear that the core group of common and top priority design issues – those on which project performance and impact ultimately depend – have been sharply identified and are given most attention by formulators, PDTs, TRCs and appraisers.

5.27 Implementation and supervision Current supervision is effective, particularly where project management is strong. The pattern of weak supervision from some CIs identified in the November 2003 OE report also appears from the project sample. The larger concern is failing projects, where MTR teams appear to be more effective in turning around poor performers than supervision visits. The PSR process is a commendable attempt to assess project and portfolio status on a common basis. It is seriously hindered by the absence of empirical output performance data however. The use of financial performance data, readily available from current financial systems, would enhance the annual project and portfolio review process.

5.28 A preliminary benchmarking comparison shows that IFAD shares many performance standards and concerns with the World Bank. With the exception of the clarity of development objectives, where IFAD appears to have performed better, similar difficulties are experienced across a range of design features: M&E; implementation capacity and institutional analysis; and risk assessment. Performance during implementation is similarly comparable with shared concerns about the quality of supervisory monitoring and the difficulty of remedying problems during implementation. The similarity of problems suggest that these reflect the challenges of working in development.

Grants – [Evaluation framework 2.4]

5.29 Grants are an important element in the range of products and services IFAD can offer. But the underlying purpose of grant funding has been undermined by frequent restatements of policy and the introduction of new instruments. A multiplicity of facilities and modalities has led to a loss of focus, lack of strategic orientation and no prioritisation. Grant project documents show strong similarities to loans in design performance, especially regarding clarity of objectives and means for monitoring. Despite all the projects working with partners there is no institutional analysis or assessment of capacity. With the exception of the agricultural research programme, little evaluative evidence has been used to inform management before the 2003 grant policy. But the experience of grant support for roots and tuber development in West Africa demonstrate how effective grants projects can be when used to complement loan projects and other grant mechanisms.


Page 71

Corporate processes

Policy development– [Evaluation framework 2.2]

5.30 A number of new policies have been developed in recent years, in part responding to the Strategic Framework commitment to engage further in policy dialogue as a way of creating conducive policy and institutional environments for rural poverty reduction. But the review of Regional Strategies and COSOPs would suggest that the intellectual framework for supporting pro-poor policy dialogue is not entirely clear.

5.31 The aim of becoming a leader in rural development policy and strategy and to exert policy influence based on pioneering and innovative activities is still largely an aspiration for IFAD. Recently this has gained more attention, but in few cases do documents make it clear just how IFAD intends to pursue its leadership role – whether financially or intellectually. Weak knowledge systems for learning from field-based activities and a relatively weak internal policy culture make it difficult to imagine how this leadership role will develop.

Knowledge Management – [Evaluation Framework 2.5]

5.32 Knowledge sharing has been an implicit feature of IFAD since the Fund was established. Following the external review in 1994 knowledge management was made an explicit objective. But it has suffered from too many varied attempts to chart policy. There does not appear to be a common understanding of the various formulations of advocacy work: the catalytic role, policy dialogue, innovator in know-how, leader in poverty alleviation strategies, sharing of knowledge to multiply impact, and being a knowledge organisation.

5.33 It is a complex area for management. It builds on the array of products, processes and people that create and manage country programmes, but seeks to find means for them to share knowledge and learn from each other. There is no shortage of instruments and skills available but the weaknesses are in planning and monitoring of the use of those instruments. Vertical boundaries (the ‘silo’ culture), and a non-participatory management style inhibit the effectiveness of learning. There is clear evidence that an independent evaluation function is strengthening critical assessment and knowledge. However, the substantive issues raised by project, country and thematic evaluations – and by the management response to findings – is then obscured by the ACP/UCP process. This diminishes the learning impact of independent evaluation.

5.34 The preliminary assessment is that it is unlikely that knowledge management has improved IFAD’s effectiveness.

Partnerships – [Evaluation Framework 2.3]

5.35 The term ‘partnership’ is widespread within IFAD and no distinction is drawn between different types of working relationship. This casual terminology, combined with the all-encompassing objectives of partnerships set out in the Vth and VIth Replenishments, undervalues the real benefits that can be gained from working in partnership towards common goals.

5.36 IFAD’s Strategic Framework is defined in such broad terms, that it is difficult to argue against the relevance of a great many partnerships, even though they may not present the best opportunities for contributing to IFAD’s strategic objectives. Indeed the IEE has not seen any analysis showing how or why a partnership would add to IFAD’s effectiveness. In the past, the Fund has entered into, and become ‘host’ to, significant partnerships for which questions are now being raised about the real benefits to IFAD.


Page 72

5.37 While agreements exist between IFAD and many of its so-called ‘partners’, these statements are not necessarily very clear on the shared objectives of the relationship. Furthermore, in several cases there is a limited institution-wide understanding of the benefits of some key partnerships (like the Global Mechanism, Global Environmental Facility, and the International Land Coalition).

5.38 Evidence of effectiveness and impact is mostly anecdotal and there is limited systematic monitoring of partnership effectiveness against shared objectives. Whilst government, NGOs and others clearly contribute to project implementation, there is no evidence that long-term focused organizational partnerships are enhancing IFAD’s impact on rural poverty.

Human Resource Management – [Evaluation Framework 2.6]

5.39 Human resources management has been a neglected area at IFAD. It is clear that priorities have changed and management is working to achieve fundamental reform. But the approach is ambitious. At this stage there has not been sufficient progress for the evaluation to reach any judgement about the actual or potential achievements but progress will be reviewed later during the evaluation.

5.40 Human resources are fundamental to IFAD’s role as a knowledge-based organization, so elements of HR that impact on the way staff perceive corporate culture affect their ability to achieve their objectives and are of crucial importance. The opinions of staff enable only very tentative conclusions to be drawn at this stage. But the issues raised are vital for the healthy functioning of the Fund: the need for more participatory management, better communication, team building and knowledge sharing; review of the balance between project design and implementation; documentary requirements, technical support and intellectual capital building; and the implementation of the HR strategy. Thus the effectiveness of current HR performance is under challenge.

5.41 A simple analysis of efficiency also raises some key questions. Despite a zero-growth budget for the last ten years there has been a de facto growth of 51 percent in staffing across all categories, professional, support and consultants across the years 1994 to 2003. When compared with the loans and grants programme over the same period, a simple but revealing indicator of efficiency, the ratio of person years to new loan approvals has risen. Analysis of the loans and grants quality of design suggests that the extra effort has not resulted in substantial changes in the quality of the portfolio. Some of the time may be devoted to policy development and advocacy. But many staff contend there has been a loss of focus by senior management away from IFAD’s core business. These issues will be considered further during the country visits.

5.42 There have been improvements in the gender balance of professional staff and consultants. But women are still more likely to be found in support roles and the main growth at professional level has been within the category of Chapter X, non-permanent appointments.

Preliminary analysis after the country visits 5.43 After the country visits have been completed the team will undertake a synthesis of findings in order to provide a consolidated analysis from all twenty projects. Other work that will be taking place in parallel to the country visits includes further analysis of human resources management, and an exploration of governance issues through meetings with members of the Executive Board. These strands will be brought together in an initial report entitled “Findings, themes and preliminary conclusions”, which will replace the proposal in the Inception Report, to draft thematic papers.


Page 73


Page 74

Table 19 Preliminary response to key evaluation questions in the Terms of Reference

Evaluation Questions from the

Terms of Reference52

Evaluation Findings at the Desk Review

§ What is the development relevance of IFAD’s policies and programmes? [a]

Analysis of the portfolio since 1994 and scrutiny of the sample of 21 countries and 42 projects shows that the IFAD’s current operations are in line with its mandate. Over time, however, as trends in rural space have become more complex and ideas about how to achieve rural poverty reduction more contested, IFAD’s development niche has become harder to define. Continuous efforts to ‘re-wire’ the organisation to the changing global context have proved important. The introduction of the Strategic Framework, Regional Strategies and COSOPs along with changes to the project management cycle have resulted in significantly improved corporate strategic alignment and some key changes to the quality of the portfolio. But as the aid context becomes more complex and the pressures on the multilateral system increase, the need for IFAD to become more focused and selective is clear. At the present time there appears to be little to guide IFAD in how to do this. The internal strategy environment is still largely permissive, providing only limited direction to operations at country and project level. There is as yet no clear guide to selectivity in resource allocation or in the use of instruments. There is no evidence that long term organisational partnerships are enhancing IFAD’s impact on rural poverty. These are all issues affecting the Fund’s development relevance and will be considered in more detail during the next phase of the evaluation.

§ What value does IFAD add to the international development community, particularly in relation to rural poverty reduction, improved food security, relevant national sector policies, national development and poverty reduction strategies, such as the PRSPs, and the international development goals, e.g. the MDGs? [a]

IFAD’s focus on rural poverty reduction gives it a special role in the international community. It is too early in the evaluation to demonstrate how effectively it has been able to use this specialised role to add value to national and international debates about poverty reduction, but two observations from the desk review are instructive. While COSOPs and Regional Strategies emphasise the importance of engaging with national development and poverty reduction strategies, such as PRSPs, few provide a clear sense of how IFAD intends to engage and with what intended impact on the national strategy process. There is recognition of the importance of the PRSP as a ‘coordinating framework’ at country level, but only limited acknowledgement of how PRSP principles may change the way IFAD is engaging at country level. The Strategic Framework 2002-2006 does not make clear links with the MDGs. This has subsequently been rectified and monitoring of IFAD’s contribution to achieving the MDGs is an important part of the RIMS. However, where IFAD can add specific value as opposed to being broadly aligned with the MDGs is not clearly articulated.

§ How successful has IFAD been in improving agricultural and rural

Addressing this question is reserved until after the next phase of the evaluation, however some initial insights from the desk review are instructive. Regional Strategies and COSOPs emphasise the importance of engaging with policy

52 Terms of Reference for the Independent external Evaluation of IFAD Paragraph 11. Subparagraphs noted by letter against each question.


Page 75




development policies and stronger institutional capacities in partner countries? [b]

on agricultural and rural development, particularly in the areas of legal and regulatory frameworks, sectoral and sub-sectoral policies and supporting the advocacy capacity of grass roots organisations to engage on policy issues. Emphasis on institutional capacity strengthening – beyond that of grass roots organisations - is less clearly articulated. Institutional and governance analysis tends to be weak in the sampled COSOPs and projects. Projects are still regarded as the main platform for policy engagement, although some Regional Strategies point to the importance of moving further into programmatic approaches to gain influence on rural and agricultural policy debates.

§ How have IFAD’s innovative approaches been replicated and scaled up? [c]

Addressing this question is reserved until after the next phase of the evaluation.

§ How effective has IFAD been in pursuing its objectives? [d]

The introduction of corporate, regional and country strategy documents has brought clear benefits internally and consensus around broad objectives. The sampled projects and COSOPs are well aligned with these objectives overall, although there are major weaknesses in important areas. Institutional analysis is poor, as is the treatment of risk and sustainability. Innovation is poorly articulated as is the role of IFAD operations in catalysing others – beyond the limited objective of leveraging co-financing.

§ How and to what extent are IFAD policies adapted to the achievement of these objectives, and how clear, explicit and measurable are IFAD’s objectives? [d]

Internal policy development has been limited compared with other IFIs. Recent policy initiatives, including the Decision Tools on microfinance, the M&E guide and the gender action plan are all intended to support key areas of IFAD operational work and are in line with overarching objectives. However, this leaves significant operational areas still unguided by formal policy statements. The new Policy Division is intended to facilitate a more systematic policy development process both as a guide to operations and to engagement with development partners. The Strategic Framework specifies key areas for monitoring progress towards strategic objectives but lacks any kind of monitoring framework as do the Regional Strategies. COSOPs show an improvement in the articulation of objectives and indicators but the logframe approach is still not a meaningful part of the country strategy process. Efforts to construct a results -impact monitoring system (RIMS) may go some way to addressing weaknesses in monitoring of objectives. While at the very early stages, progress will be assessed as part of the next phase of the evaluation.

§ What has been the sustainable–quantitative and qualitative—impact of IFAD-supported projects on the ground? (As much as possible, the IEE is expected to quantify impact and, to that end, emphasize methodologies for quantitative data collection and analysis of impact.) And to

Field studies have yet to be undertaken, but an initial review of project achievement in 20 projects with relevant data finds that the most consistently achieved results are increased food security as a result of higher yields and production of crops, livestock and fish, followed by increased access to physical and financial assets and household incomes. This suggests that IFAD projects have made some contribution to improving rural livelihoods, although the scale and depth of that contribution is impossible to enumerate in full outside of the field visits. The treatment of sustainability is negligible in most PCRs and MTRs, making it impossible to assess the sustainability of any of these outcomes at this stage.


Page 76




what extent has IFAD contributed to rural poverty reduction? [b] § How has IFAD made use of local

knowledge and technology in promoting innovative approaches? [c]


§ To what extent has IFAD promoted the ownership and partnership of relevant host country institutions, including those representing the poor? [e]


§ How efficient has IFAD been in the use of lending and other budget resources and the deployment of skills? [e]

The efficiency of resource use is still under consideration but preliminary findings on the use of human resources during the review period points to a significant growth in staff across the organisation despite a zero-growth budget. However the analysis of loans and grants quality suggests that there have only been modest changes in the quality of the portfolio, leading to the finding that more staff has not necessarily been good for IFAD’s core business. However, without a more comprehensive look at resourcing, especially in the policy and strategy area, it is premature to draw any firm conclusions from this data.

§ And how selective has it been in the allocation of its resources and the choice of institutions and partners? [e]

The analysis of selectivity is in its early stages but there is at least some evidence that IFAD has oriented its lending resources towards borrowers with moderate to good policy and institutional performance, although this only applies to those countries classified as low-income. Preliminary analysis of the relationship between IFAD lending and other structural criteria for selectivity at country level failed to produce significant results. The clearest driver of resource allocation at the regional level appears to be the desire to maintain geographical equity, in line with the IFAD mandate. In terms of the choice of institutions and partners, it appears that relatively little selectivity has taken place. The term partnership is used very loosely in IFAD terminology, while the Strategic Framework presents little guide as to how different kinds of partnerships are expected to contribute to IFAD strategic objectives. In the past the Fund has entered into, and become ‘host’ to numerous partnerships for which questions are now being raised about the real benefits to IFAD. Evidence of effectiveness and impact is mostly anecdotal and there is limited monitoring of partnership effectiveness against shared objectives. There is as yet no evidence that partnerships are enhancing IFAD’s impact on rural poverty.

§ How effectively has IFAD promoted innovative approaches in relation to policy, partnerships, project implementation,

Innovation is a key dimension of IFAD’s stated comparative advantage, however the desk review finds a lack of clarity around the meaning and monitoring of innovation at all levels of the institution. Review of the sampled projects finds that the innovative content of the portfolio is hard to determine. There are many stated innovations


Page 77




technology and other aspects of IFAD-assisted operations that are meant to impact poverty? [c]

for different tiers within IFAD projects, but these involve widely different interpretations of the term innovation. The same holds for the review of COSOPs. There is also a lack of clear policy guidance on innovative approaches. It is notable that risks are not well analysed in project designs or in COSOPs, despite the intended focus on innovation.

§ What explains IFAD’s performance in terms of evaluation criteria and questions such as those mentioned above? [f]

Concluding questions

§ How can IFAD enhance the impact and sustainability of its development cooperation, and contribute more to poverty-reduction efforts? [g]


§ What are the main recommendations that IFAD should consider adopting in the near and long term?[g]



Page 78

Independent External Evaluation of IFAD Desk Review Report Annex 1

Page 79

Annex 1 Bibliography

COP and IFAD (1999) MoU between COP and IFAD for the Global Mechanism Beynon, Jonathan (2003) Poverty efficient aid allocations – Collier/Dollar revisited.

Economics and Statistics Analysis Unit Working Paper 2. ODI, London CENTRIM and IDS (2001) IFAD’s capabilities as a promoter of replicable innovations.

Brighton, Sussex Coleman, G (1990) ‘Problems in project-level monitoring and evaluation: evidence from one

major agency,’ Journal of Agricultural Economics, 41, pp 149-161 Cooke, Rodney & 16 others (2000) Report of the working group on impact achievement

through the project cycle (IAPC) (mimeo) Dabelstein, Niels, Victor Hugo Morales Melendez, Adnan Bashir Khan, Lothar Caviezel,

Osvaldo Feinstein, Gabriel Lombin, Robert van den Berg (2002) External Review of the Results and Impact of IFAD Operations. Copenhagen

Head, Ivan L., Ali Ahmed Attiga, Martin Enrique Pineiro, Yves Rovani and Djibril Sene (1994) The Challenge of Rural Poverty: The Role of IFAD. Report of the Rapid External Assessment of IFAD. Rome

IFAD (1976) Agreement Establishing the International Fund for Agricultural Development (revised through to 1997)

IFAD (1977) By-laws for the Conduct of Business of IFAD (Revised 1997). Rome IFAD (1977) Financial Regulations of IFAD (Revised 1997). Rome IFAD (1977) Rules of Procedure of the Governing Council (Revised 1997). Rome IFAD (1978) Lending Policies and Criteria (3rd Revision, 1998). Rome IFAD (1993) Report of the PMD Task Force on the rationalization of the project cycle –

Draft (mimeo) IFAD (1994 to 2003) Programme of work and administrative budget for IFAD (Annual).

Rome IFAD (1998) General conditions for Agricultural Development Financing. Rome IFAD (2000) Detailed design of the process re-engineering programme EB 2000/71/R.9

Rome IFAD (2000) Evaluation of the IFAD/NGOs Extended Cooperation Programme: Executive

Summary and Agreement at Completion Point, IFAD, Rome IFAD (2000) Highlights and News: 10 th IFAD/NGO Consultation: Tripartite Partnerships for

poverty alleviation and food security through programmes, 30 May – 2 June 2000, Pune, India

IFAD (2000) Partnerships for Eradicating Rural Poverty . Report of the Consultation to Review the Adequacy of the Resources Available to IFAD 2000-2002. GC 24/L.3.

IFAD (2000) Rural Finance Policy IFAD (2000) Strengthening IFAD’s support to the development of effective and efficient

monitoring and evaluation systems methodological study (Interim Paper). Evaluation Committee – Twenty-Fifth Session EC 2000/25/W.P.3/Rev.1 Rome

IFAD (2001 Evaluation of IFAD’s capacity to promote replicable innovation Phase I, A preliminary assessment. Office of Evaluation and Studies. Rome

IFAD (2001) Executive Board, Minutes of the Severty -Fourth Session. Rome IFAD (2001) Guidelines. Project Design Document and Key File (Revised 2003). Programme

Management Department. Rome IFAD (2001) Process reengineering programme (Design stage) Impact delivery and

communication process: Operating model and business case Rome IFAD (2001) Strategic Framework for IFAD 2002-2006. Rome IFAD (2002) Procedure for the Review of Country Strategic Opportunities Papers

(COSOPs) by the Executive Board. EB2002/77/R.12 Rome.


Page 80

IFAD (2002) Annual Report 2001. Rome IFAD (2002) Annual Report on Results and Impact of IFAD Operations IFAD (2002) Enabling the Rural Poor to Overcome their Poverty: Report of the Consultation

of the Sixth Replenishment of IFAD's Resources (2004-6) REPL.VI/5/R.2 IFAD (2002) Progress Report on the IFAD V: Plan of Action (2000-2002). Executive Board –

Seventy-Seventh Session EB 2002/77/R.31 Agenda Item 13. Rome IFAD (2002) Progress Report on the Project Portfolio. Executive Board – Seventy-Fifth

Session EB 2002/75/R.12/Rev.1 Agenda Item 6. Rome IFAD (2002) Strategic Framework for 2002-2006: Enabling the rural poor to overcome their

poverty, March 2002, IFAD, Rome IFAD (2003) A methodological framework for project evaluation. Office of Evaluation and

Studies. Rome IFAD (2003) Evaluation of IFAD’s Technical Assistance Grants Programme for Agricultural

Research. Report No. 1377. Office of Evaluation IFAD (2003) Evaluation of Supervision Modalities in IFAD-supported Projects, OE, IFAD (2003) Framework for a results management system for IFAD-supported country

programmes EB 2003/80/R.6 Rome IFAD (2003) IFAD Evaluation Policy. Executive Board – Seventy-Eighth Session EB

2003/78/R.17/Rev.1 Agenda Item 8(c). Rome IFAD (2003) IFAD policy for grant financing EB 2003/80/R.5/Rev.1. Rome IFAD (2003) Independent External Evaluation of IFAD, Terms of Reference, Para. 3 Rome IFAD (2003) Progress Report on the Popular Coalition to Eradicate Hunger and Poverty ,

Governing Council – 25th Session, GC 26/L.11, IFAD, Rome IFAD (2003) Progress report on the process re-engineering programme (Strategic Change

Programme) EB 2003/79/R.36 Rome IFAD (2003) Progress report on the process re-engineering programme (Strategic Change

Programme) GC 26/L.10 IFAD (2003) Progress Report on the Project Portfolio. Executive Board – Seventy-Eighth

Session EB 2003/78/R.16 Agenda Item 7. Rome IFAD (2003) Report of the panel on the performance-based allocation system EB

2003/80/R.3 Rome IFAD (2003) Rural Enterprise Policy IFAD (2004) Consolidated Financial Statements for the years ended 31 December 2003 and

31 December 2003 IFAD (2004) IMT Report. Rome IFAD (2004) Programme of work and administrative budget for IFAD and its Office of

Evaluation. Rome IFAD (2004) Scope and guiding principles of SCP. Rome IFAD (nd) Policy and Resource Trends Likely to Affect IFAD. PMD (2004) PMD Retreat January 2004 Office Memorandum Project cycle reengineering working group (1995) Proposed project development cycle:

Recommendations to the re-engineering task force – draft (mimeo) Watkins, Francis and Csaky, Corinna (2003). Partnerships, Volume 1 - Literature Review.

Edinburgh Resource Centre for DFID Evaluation Department. World Bank (2002) Annual Report on Portfolio Performance. Washington


Page 81

Annex 2 List of people interviewed

Family name Given name Unit

Abdouli Abdelhamid PN Abhyankar Uday OP Angelucci Lucia International Land Coalition Audinet Jean-Philippe OE Aureli Massimo FM Båge Lennart President Baldwin Brian FC Bamba Zoumana EAD/ES Beavogui Mohamed PA Benassi Anna OE Ben-Senia Mohamed PA Bettink Willem PD Bishay Mona OE Bouzar Khalida Global Mechanism Brett Nigel PI/PMD Calcagni Antonia FC Cappai Stefania FC/L Carruthers Jim PD Cassani Robert ER Castellucci Theresa FM Cleveringa Rudolph PT Cong My Huynh FC Cooke Rodney PT Corona Patricia FC Daniel Louise OE de Willebois Ides PF Dellanava Carla FC Demirag UIac PA Di Pietro Daniela FH Dommel Henri PT Elhaut Thomas PD El-Zabri Tawfig PN Faisal Mohammad PF Farmosi Elisabeth PD Farrant Ruth FC Felloni Fabrizio OE Francis David Centrim, Brighton University Gil Pilar PL Giorgi Carina PL Glickman Pablo PL Gracia Charles FC Haidara Hamed PA Hamilton-Peach Julian PT Haralambous Sappho EO Hassani Mohammed PN Hassani Mohammad PN Heinemann Edward PF Hopkins Raul PL Howe Gary PF Kayser Katherine OE Kennedy Sean PT Khadka Shyam PI Kherallah Mylene PN


Page 82

Khimani Beatrice FH Kingsbury David PA Klavins Sandra FC Knopf Ana FD Kuhn Christiane PN Lavizzari Luciano OE Lembo Alessandro PF Lohlein Daniela EO Longo Roberto PF Lubbock Annina PT Manssouri Mohamed PA Marzin Sylvie PA Mathur Shantanu PT McDonald Louise PF McGhie John FC/L McGuire Sandra EC - by Telephone Medrano Rojas Ofelia PA Melina Brigitte FH Micallef Maureen FA Moore Bruce International Land Coalition Muthoo Ashwani OE Nichols Flemming OE Nsimpasi Luyaku Loko PA Okong’o Miriam PF Olsson Gunilla EO Palmeri Chase OE Pedersen Henning PN Pena-Montenegro Raquel PL Pugerup Kare OP Quijandria Benjamin Consultant, ex PL Rahman Abdalla PN Rasha Omar PN Rath Thomas PI Rebuelto Ana Lia PD/PL Renard Nadine PT Rice Theresa PD Rihane Fawzi EAD Rodrigues Doris PL Romagnoli Giuliana FH Rota Antonio PT Roy Phrang ED Ruhio Ladislao PD/PL Ryden Per Global Mechanism Saint Ange Perin PA Saitto Allegra FC Sidahmed Ahmed PT Silveri Paolo OE Slama Abdelmajid PN Sorensen Jens PF Sourang Cheikh OM Stigliano José FM Tounessi Mohamed PA Toure Abdoulaye PA Trupke Hermi PD Ventimiglia Stefano FH Wholey Douglas PT Wieland Ursula PD

Independent External Evaluation of IFAD Desk Review Report Annex 3 – Poverty efficiency of aid allocations

Annex 3 Page 1

Annex 3

Poverty efficiency of aid allocations – IFAD compared

Introduction

1.1 Recent analysis has shown the importance of good policy and institutional performance as a key determinant of aid effectiveness and as a criterion for efficient aid allocations. Improving the poverty-efficiency of aid allocations also requires that aid be better focused on poor countries with large numbers of poor people. Recent work by Beynon (2003)1 reveals how far donors are responding to this evidence through a close look at the relative efficiency of aid allocations over time and between donors. Some of his findings are presented below, with particular reference to trends in IFAD allocations compared to other key multilaterals.

Rising productivity of aid 1.2 Figure 1 shows the marginal effect – in terms of the number of people lifted out of poverty - of an extra $1m of ODA.2 Estimates are given for individual multilateral donors, including IFAD, and a weighted average of all DAC donors and multilateral donors. The figure shows marginal effects doubling overall, indicating a substantial improvement in the productivity of aid during the 1990s. Multilateral donors were and remain marginally more efficient than bilaterals.

1.3 IFAD performed better than the multilateral average, and in 1999/2000 on a comparable level with IDA, AsDB, UNDP and UNICEF. Between 1990/91 and 1999/00 there was an 82 percent increase in the marginal effect of an extra $1m of IFAD assistance, pointing to a significant improvement in the efficiency of IFAD concessional aid over the period.

Figure 1: Numbers lifted out of poverty 1990/91-1999/2000 (per extra $1m aid, marginal effects)

-200

-100

0

100

200

300

400

500

600

DAC Donors AfDF AsDB EC IDA IDB UNDP UNICEF IFAD MulitlateralDonors

1990/91

1999/00

1 Jonathan Beynon (2003) Poverty Efficient Aid Allocations – Collier/Dollar Revisited. Economic Statistics Analysis Unit, ODI. ESAU Working Paper 2. 2 Concessional and/or grant aid only.


Annex 3 Page 2

1.4 While the results are impressive, Beynon finds that, contrary to recent World Bank research (2002), none of the improvement in the efficiency of allocations is attributable to changes in the pattern of aid allocations. Instead almost all the improvement is due to changing aid levels (in particular a fall over the 1990s 3) and changing policy performance in recipient countries.

Poverty Focus 1.5 This finding is further supported by estimates showing that the percentage share of ODA going to low income countries increased only modestly over the 1990s from 57 percent to 63 percent. Multilaterals as a whole remained more focused on low-income countries than bilaterals.

1.6 The share of IFAD concessional assistance going to the poorest countries remained high at around 83 percent for the period, although the ratio of ODA per head going to high versus low poverty countries fell slightly from 1.32 to 1.22 – meaning that high poverty countries received roughly 1.2 times as much per head as low poverty countries in 1999/00.4 This was still well above the average for multilaterals as a whole (0.74 in 1999/00) but below the equivalent ratio for IDA, the AfDF and UNICEF.

Country Policy Environments 1.7 The importance of the policy and institutional environment in determining aid effectiveness is now well established and, as already indicated, donors have been responding. Beynon’s data on the ratio of ODA/per head in good verses poor policy performing countries (Figure 2) suggests that overall donors are becoming more performance-based.

1.8 Estimates for IFAD suggest a fairly significant rise in the ratio of ODA/hd in good versus poor policy environments, although below the ratios for IDA, the EC, the weighted average for bilaterals and multilaterals. Further examination of the composition of IFAD assistance over the period is illustrated in Figures 3 and 4.

3 Lower aid levels yield higher marginal efficiency estimates, given diminishing returns. 4 In Beynon’s analysis, high poverty countries are those with over 50% of the population living on less than $2/day. Low poverty countries are those with below 50% of the population living on less than $2/day.


Annex 3 Page 3

Figure 2: Ratio of ODA/hd to good:poor policy countries 1990/91-1999/2000

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

DAC Donors AfDF AsDB EC IDA IDB UNDP UNICEF IFAD MulitlateralDonors

Rat

io g

oo

d:p

oo

r p

olic

y ($

/hd

)

1990/91

1999/00

1.9 Figure 3 shows the percentage share of IFAD assistance going to countries with good/medium/poor policy environments scored using IDA’s CPIA. The results suggest a gradual decline in ODA levels going to poor performing countries, matched by a substantial increase (in percentage terms) in assistance going to ‘medium’ performers.

Figure 3: ODA % (IFAD) by Country Policy Score

0

10

20

30

40

50

60

70

Good Medium Poor

Country Policy Score (CPIA Terciles)

% O

DA

1990/91

1999/2000

Summary 1.10 Beynon’s analysis suggests that in line with most other multilaterals, and a growing number of bilaterals, IFAD assistance became more efficient during the 1990s. The emphasis on assisting high poverty countries continued throughout the decade, although there was a slight decline in the ratio of assistance going to high versus low poverty countries, possible because of a shift in the number of borrowers receiving concessional assistance. Finally, IFAD assistance is showing signs of being more performance based with a rising ratio of ODA in good versus poor policy performing countries. However the data is patchy and the


Annex 3 Page 4

CPIA index, while probably the best index currently available, does have its limitations. The aggregate data need to be interpreted with care and a closer look at the IFAD portfolio is necessary to understand what lies behind these trends.

Independent External Evaluation of IFAD Desk Review Report Annex 4 – Grants

Annex 4, page1

Annex 4

Technical Assistance Grants & Other Grant Instruments

1 Introduction

1.1 This annex presents an analysis of the use of grants. The main focus is Technical Assistance Grants, but these are reviewed in the context of the various other grant instruments used by IFAD. The section starts by a review of policy and the way it has evolved in the period since 1994. Next, the TAG portfolio is analyzed from IFAD budget documents and records held by PT. Funding under TAGs is then compared with other grant funding. Four TAGs were sampled for desk review. The results of that are described next, followed by a summary of a recent evaluation of agricultural research grants conducted by OE. The last section concludes with issues for follow-up during the country visits and later in the evaluation.

2 Objectives

2.1 The facility to offer grant financing was established at the formation of the Fund. The agreement establishing IFAD defines the use of loans and grants, and limits grants to ‘not normally exceed one-eighth of the resources committed in any financial year’.1 Grant assistance is further defined in the Lending Policies and Criteria:

37. The Fund’s grant assistance, apart from technical assistance, shall be exclusively used for the financing of projects in the absolute poorest food-deficit countries with the most severe development problems. Taking account of the very limited amounts of resources for this type of assistance, the Executive Boards will approve grant financing only for high-priority projects in countries with very severe budgetary constraints; these considerations will apply in particular to those cases where the revenue-generating effects of projects are considered unimportant but where the project still constitutes an essential element of the Fund’s programmes in the country. 38. Technical assistance, particularly for activities listed in paragraph 29, will normally be provided on a grant basis. However, when technical assistance for feasibility studies leads to a loan provided by the Fund, the Executive Board might include the costs of such technical assistance into the loan. In addition, the Fund may provide, in cooperation with other agencies, grants for suitable activities of international, regional and national research institutions. (Further amendments in 1979 and 1989 made changes of detail to the funding of technical assistance.)2

2.2 The framework was therefore created for the use of grants to fund preparation activities under the project cycle, in support of research and as an exceptional component of project funding. All three have been applied.

Evolution of policy

2.3 There has been a steady development of policy and new initiatives since the mid 1990s. Table 1 sets out the main features in a time-line.

1 IFAD (1976) Agreement Establishing the International Fund for Agricultural Development (revised through to 1997). Article 7, 2 (a) & (b). 2 IFAD (1978) Lending Policies and Criteria (revised through to 1998). Paragraphs 37 & 38.


Annex 4, page2

Table 1 Initiatives and policy development in grant funding 1994 – 20043

Year Initiative and policy development

1994 Responding to the TAG Thrust approved by 1991 EB on IPM, soil fertility and water management, and project management staff training. Funding in support of consolidation/follow-up of project Preliminary Development and Testing Phases Additional funding for Special Activities

1995 Grant cap raised from 5% to 7.5% of resources committed Purpose of grants restated to strengthen activities: • Allow IFAD to remain on the front line with tested innovative approaches to rural

poverty eradication • Accelerate programme pipeline build-up • Enhance cost effectiveness of the design cycle • Expedite smoother start-up and implementation of projects

Identification of possible new areas for funding: operational policy research; bridging relief and recovery; broader environmental assessments; technical assistance in projects; preparation for new member countries New Project Development Fund to finance project development excluding appraisal New Special Operations Facility to facilitate early start-up and continuity between design and implementation (for borrowers entitled to most concessional terms, and only for the first IFAD-financed project in a country)

1996 New Environmental Assessment facility 1997 Addition of text about grant purpose ‘allows IFAD to contribute to the build-up of in-country

capacity, especially through training’ to the bullet list from 1995 Planned initiative to enhance cooperation with NGOs

1998 Need to update grant policy – paper expected for April 1998 EB with a proposal to focus design and content on: (a) enhancing the Fund’s catalytic role and extending outreach and forging partnerships (b) rural institutional dynamics for participatory rural development (c) achieving a high-impact lending portfolio (d) knowledge gathering and sharing on rural poverty (e) agricultural research and technology innovation

1999 Board reviewed grant policy submissions and a final paper expected for April 1999 EB Grant to the CCD Support facility for the Global Mechanism Task Force on Small Tags (<US$100,000) established by ED -PD

2000 Final grant strategy expected at the April 2000 EB4 2001 Final grant strategy expected during 2001

Grants identified as an element to innovation and knowledge management on the Corporate Scorecard Ceiling of grants to NGOs raised from US$75,000 to US$100,000 in May Proposal to introduce Project Development Financing Facility (PDFF). A new category of resource use over and above the administrative budget and programme of work. Derived from the re -engineering exercise, intended to finance expenditures for the design and implementation of projects and programmes. Deemed to require a ‘highly decentralized system of resource planning and use, well-defined accountability and an accurate control and assessment system’.

2002 EB to consider a document on grant policy and strategy emphasising investment in three areas: • Technical issues • Organizational issues involving socio-economic mechanisms and incentives • Training and capacity building

3 Sources: IFAD’s Annual Programme of Work and Administrative Budget documents 1994 to 2004; IFAD (2002) Recommendations of the ED-PD Task Force on Guidelines and procedure for the development and clearance of small TAGs (up to USD100,000) in the category of Other research, Training and Others) (mimeo) 4 IFAD (2000) Grant Financing: A New Approach EB 2000/69/R.11 discussed but not approved


Annex 4, page3

Year Initiative and policy development

Guidelines for grant criteria prescribe the following objectives: (a) lead to significant, measurable impact on IFAD’s rural poor clientele (b) have feasible, innovative approaches involving appropriate partners with

established comparative advantage in the research topic (c) have potential for sustainable knowledge dissemination and uptake through

downstream linkages with concerned networks and institutions (d) contain adequate implementation and organizational arrangements,

including procedures and indicators for monitoring (e) provide a justifiable budget, indicating that adequate resources will be

available for efficient utilization and cost-effective implementation 2003 New objectives in the Work Programme and Administrative Budget in support of the

Strategic Framework for IFAD 2002-2006 for grants under the research categories to relate to: (i) IFAD’s target groups and their household food-security strategies (ii) Technologies that build on traditional knowledge systems by improving

productivity and addressing production bottlenecks (iii) Access to productive assets and sustainable and productive management of

such resources (iv) A policy framework that provides the rural poor with an incentive to reach

higher levels of productivity , thereby reducing their dependence on transfers (v) An institutional framework within which formal and informal, public and

private-sector, local and national institutions provide services to the economically vulnerable, according to their comparative advantage

Development of Technical Advisory Notes (TANs) designed to disseminate good practice advice. Internal task force created under the Vice-President to present a policy and strategy paper to the September 2003 EB. IFAD Policy for Grant Financing (EB 2003/80/R.5/Rev.1) approved. Delegated authority to the President raised to US$200,000. Ceiling for the grant programme raised from 7.5% to 10% (to include TAGs, PDFF and grants to NGOs); the entire increase of 2.5% to be applied to country-based initiatives. Development of a PBAS-driven country-specific grant portfolio. Two strategic objectives: (i) promoting pro-poor research on innovative approaches and technological options to enhance field-level impact; and, (ii) building pro-poor capacities of partner institutions, including CBOs and NGOs. NGO-ECP programme to be managed within the Sma ll TAG facility.

2004 New budget presentations to reflect (i) new grant policy with 50:50 split of funding between two grant windows, the global and regional window and the country-specific window; and (ii) activity-based budget presentation according to eight institutional priorities.

2.4 The ten year period has seen a large number of initiatives and policy changes. Several features emerge from this history. Firstly, there is a move away from support to the development of projects in 1995 towards a broader view of partnerships, rural dynamics, knowledge dissemination and complementarity with loans and the country portfolios. But the high frequency with which policy has been restated complicates the message. Secondly, there has been a steady process of innovation and change affecting the ways in which the grants programme is organised and managed in support of the project cycle: the Project Development Fund (PDF), Special Operations Facility (SOF), Environmental Assessment (EA), and Project Development Financing Facility (PDFF). Each of these places new demands on staff to adapt to changing procedures and mechanisms. Thirdly, there was a protracted process of proposal and consultation from 1998 to 2003 to update the grant policy. This period coincided with several other change processes: Business Process Re-engineering, the Strategic Change Programme, and the development of the Strategic Framework 2002-2006. The delays suggest a prolonged period of uncertainty and management indecision.


Annex 4, page4

2.5 The importance of grants to the development of projects and programmes is clearly evident. The new 2003 policy simplifies strategy, provides clear guidelines on funding priorities, and reinforces the need for grants to complement country programmes. The policy is consistent with and relevant to IFAD’s mandate. But there is very little evaluative evidence used to justify continued levels of spending or to support proposed changes in modalities such as the creation of the PDFF or the absorption of the NGO-ECP window into the small TAG facility.

Summary of issues

• Consistency in the underlying objectives of grant funding has been undermined by frequent restatements of policy and the introduction of new instruments

• The multiplicity of facilities and modalities has led to a loss of focus, lack of strategic orientation and no prioritisation

• With the exception of the agricultural research programme, little evaluative evidence has been used to inform management before the 2003 grant policy

3 The TAG portfolio

3.1 Analysis of the TAG portfolio is complicated in several ways. Firstly, the profusion of instruments means that a variety of facilities have been used towards similar ends at different times. Secondly, data are available from a multiplicity of sources: the annual budget, records kept by PT, and the Loans and Grants database (LGS). Thirdly, some sources refer to new commitments, others to annual disbursement.

3.2 Table 2 presents data from IFAD’s Annual Reports and Annual Programmes of Work and Administrative Budgets. The annual totals of grants presented in the two sources are, in most years, of similar order of magnitude, but differ in detail. Data from IFAD’s budgets include allocation among those instruments in use in each year. Thus the total level of TAG approvals grew from US$14.4 million in 1993 to US$36.2 million in 2000. Totals for the subsequent three years exclude allocations to PDFF.

3.3 The ratio of grants to loans, calculated from the Annual Report data is shown in the bottom row. It exceeded the 7.5% ceiling in 1997 and 2001 but was within the policy ceiling over the period as a whole.


Annex 4, page5

Table 2 Overview of the TAG Portfolio5

Programme of Work and Administrative Budget 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003GRANTS US$ '000 Actual Actual Actual Actual Actual Actual Actual Actual Actual Projected Orig ApprPDFF Transfer 13.4TAGs 14.4 16.3 21.6 31.1 33.8 32.5 35.5 36.2 20.1 22.4 20.3

Agric Research 6.0 6.6 12.6 10.6 8.5 8.5 8.5 8.5 5.4 8.6 8.4OR&T - - - 6.6 6.2 9.1 13.7 12.3 9.0 9.6 9.4Accelerated project performance & other implementation 1.5Project Development Fund 3.1 6.4 6.6 9.1 8.1 7.9 8.8 9.2NGO/ECP 0.7 0.7 0.9 1.5 1.9 2.5 2.4 2.6 1.6 2.6 2.5Environmental Assessment 1.5 0.6 0.4 0.8 0.4 0.4 0.4 0.4Special Operations Facility 1.0 1.5 1.9 2.1 1.8 1.8Special activities 3.2 2.8 1.0 6.7 1.9

IFAD Annual ReportsLoan Approvals:

Number of projects 32 28 33 32 30 29 30 27 25 25Amount of loans US$ million 338.8 349.3 391.7 395.1 397.7 407.9 432.7 409.0 403.1 365.9

Grant Approvals:Number 45 71 132 212 258 110 124 120 106 85

Amount US$ million 14.4 16.8 23.6 28.3 35.4 30.2 29.6 32.8 30.8 23.9

RatiosAnnual Report Grants as % of Loans 4.3% 4.8% 6.0% 7.2% 8.9% 7.4% 6.8% 8.0% 7.6% 6.5%

5 Excluding expenditure under the PDFF in 2001, 2002 and 2003


Annex 4, page6

Table 3 Technical Assistance Grants Approved US$

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 Total

Small TAGSNumber 1 7 27 23 28 28 48 61 85 21 329Value US$ 50 000 236 591 1 577 000 1 428 580 1 674 000 1 873 000 1 927 732 2 565 038 4 236 101 1 323 000 16 891 042 100%Average value 50 000 33 799 58 407 62 112 59 786 66 893 40 161 42 050 49 836 63 000 51 341Total value named countries 0 126 591 1 292 000 1 133 580 1 174 000 1 223 000 1 375 732 2 070 038 3 465 977 820 000 12 680 918 75%Total value regional 0 10 000 90 000 235 000 225 000 245 000 310 000 290 000 588 124 315 000 2 308 124 14%Total value global 50 000 100 000 195 000 60 000 275 000 405 000 242 000 205 000 182 000 188 000 1 902 000 11%

Agricultural ResearchNumber 3 5 11 10 8 6 9 7 6 9 74Value US$ 2 511 000 5 548 000 9 565 000 7 967 000 8 265 000 6 325 000 9 865 000 8 553 000 7 150 000 9 100 000 74 849 000 100%Average value 837 000 1 109 600 869 545 796 700 1 033 125 1 054 167 1 096 111 1 221 857 1 191 667 1 011 111 1 011 473Total value Africa 26 973 000 36%Total value Asia 18 567 000 25%Total value NENA 16 476 000 22%Total value LAC 5 161 000 7%Total value cross-regions 7 672 000 10%

Other Research & TrainingNumber 1 10 8 11 9 9 9 8 9 9 83Value US$ 3 000 000 8 090 000 5 166 000 7 533 409 7 980 000 8 610 000 8 174 000 8 303 000 11 350 000 6 400 000 74 606 409 100%Average value 3 000 000 809 000 645 750 684 855 886 667 956 667 908 222 1 037 875 1 261 111 711 111 898 872Total value named countries 3 000 000 1 640 000 2 866 000 5 033 409 610 000 2 730 000 5 324 000 4 353 000 4 600 000 400 000 30 556 409 41%Total value regional 0 3 150 000 2 300 000 2 500 000 5 770 000 3 135 000 1 850 000 2 200 000 5 500 000 5 500 000 31 905 000 43%Total value global 0 3 300 000 0 0 1 600 000 2 745 000 1 000 000 1 750 000 1 250 000 500 000 12 145 000 16%

Total TAGsNumber 5 22 46 44 45 43 66 76 100 39 486Value US$ 5 561 000 13 874 591 16 308 000 16 928 989 17 919 000 16 808 000 19 966 732 19 421 038 22 736 101 16 823 000 166 346 451


Annex 4, page7

3.4 Table 3 presents an analysis of data held by PT for the three classes of Technical Assistance Grants for the period 1994 to 2003: Small TAGs (<US$100,000), TAGs for agricultural research and TAGs for other research and training. These differ in number and value from the figures in the annual report, which include other grants to be discussed later. The data show distribution by year and by global or regional or country focus. The main points to emerge from the data are as follows.

3.5 Small TAGs are the most numerous, totalling 329 with an average value around $50,000. Three-quarters are directed to named countries, most frequently a single country, and occasionally several countries but usually within the same region. The balance is geared towards topics of a regional or global nature. From the information given in the title, one third was given to fund workshops or conferences. No other trends in use are easy to identify. Small proportions have been used for studies (7 percent), impact (4 percent) and about 10 percent for work to do with innovation, partnerships and knowledge sharing.

3.6 Slightly fewer than ten agricultural research grants have been awarded most years, ranging from US$250,000 to US$1.6 million with an average value of $1 million. Awards are generally made to cooperating research institutes and many grants identify specific countries to which the research is expected to be relevant. The regional allocation over the whole period has been 36 percent to Africa, 25 percent to Asia, 22 percent to Near East and North Africa, 7 percent to Latin America and the Caribbean, and 10 percent allocated for cross-regional work. Agricultural research grants are managed by the PT division.

3.7 Table 4 classifies the agricultural research grants by subject and region. The cells show the amount of the grants in thousands of US dollars. A superscript indicates the number of grants awarded if more than one.

Table 4 Distribution of grants for agricultural research


Rice (& fish) 2,750 (3) 1,585 (2) Maize, Sorghum, Millet 3,000 (3) 750 2,000 (2) 1,211 Durum 2,400 (2) Grain legumes 2,600 (2) 1,300 Roots & tubers 3,750 (3) 2,800 (3) Underutilised crops 1,410 Bamboo & rattan 2,195 (2) Jute 400 Genetics 800 Tree crops & forestry Agro-forestry & indigenous trees

2,080 (2) 1,000

Coconut 907 Pests & diseases 2,600 (3) 500 1,000 1,000 Soils (+ saline & brackish water)

2,435 (2) 1,000 400

Livestock 1,500 2,000 (2) 1,250 Apiculture & sericulture 2,900 (2) 700 Camel 1,200 Fisheries 650 Animal health 4,258 (5) 875 Strategy development 2,765 (3) Upland poor 3,950 (3)


Annex 4, page8


Technology adoption, mixed farming & food security

600 1,500 2,160 2,117 (2)

Socio economics & non-farm 1650 (2) 931 Research & HR development 1,670 (2) 3.8 The programme has covered a wide range of topics and settings. Overall, highest priority has been given to maize, sorghum and millet, followed closely by roots and tubers and technology adoption. In Africa the highest priority has been animal health, mainly research into trypanosomiasis, followed by roots and tubers, and maize, sorghum and millet. In Asia, research into the upland poor has been the leading recipient. In NENA regional strategy and planning for aquifer management is first, followed by work on durum. Least expenditure has been directed towards Latin America and the Caribbean.

3.9 The programme of grants for Other Research and Training is similar in size to agricultural research, and a similar but slightly smaller average size. Grants under this facility have been allocated to a very wide range of recipients and for diverse purposes. Table 5 summarizes and illustrates their use.

Table 5 Details of Other Research and Training Grants

Unit or Division


External Affairs $5,525,000 • Implementation of the Convention to Combat Desertification (to UNCCD)

• Community empowerment facility expansion and secretariat operating costs (to PCEHP)

Global Mechanism

$5,000,000 • Grants to support action programmes (to GM/UNCCD)

International Land Coalition

$500,000 • Programmes and Partnership Activities

West & Central Africa

$8,150,000 • Regional environmental management information project for the Congo Basin (to ADIE)

• Market driven initiative for millet and sorghum development (to SG2000)

• Sustainable vegetable production and marketing (to RADORT)

• Rural development HUB – a tool for improved food security and rural poverty alleviation (to UNOPS)

• Regional information network (to UNOPS) PD $1,200,000 • Project completion for PCEHP and a programme to

promote impact orientation during the project cycle East & Southern Africa

$15,811,409 • Agricultural management training programme for Africa (to OAU/STRC)

• Rwanda returnees rehabilitation (to Rwanda) • Strengthening the framework for the provision of

investment capital for the rural poor (to AFRACA) • Farmer field schools (to FAO) • Programme to promote sustainable use of trade in natural

products through SANPROTA (to SAFIRE) • Rural knowledge network pilot project (to FAO) • AFRACA development programme (to AFRACA)

Asia & Pacific $4,570,000 • Livelihoods in uplands and mountains of Hindu Kush Himalayas (to ICIMOD)

• Programme for electronic networking for rural


Annex 4, page9

Unit or Division


Asia/Pacific (ENRAP) (to IDRC) Latin America & Caribbean

$16,823,000 • Strengthening regional capacity for M&E of rural poverty alleviation projects PREVAL (to DESCO)

• Regional unit for technical assistance (to RUTA) • Many projects a little less than $1 million including

Institutional and policy support, and poverty reduction in the MERCOSUR region; Regional rural development training (to PROCASUR); Alleviation of rural poverty in the Caribbean (to CARUTA); FIDAMERICA (to RIMISP); and support of indigenous peoples in the Amazon basin (to CAF)

Near East & North Africa

$14,667,000 • Near East and North Africa Management Training in Agriculture (to NENAMTA)

• Backstopping support to IFAD projects (to ACSAD) • Methodologies and approaches for effective introduction

of PIM (to CIHEAM) • Relief and development in Gaza (to ANERA) • Marine resources management in the Red Sea (to FAO) • To develop a knowledge generation and sharing network

(to IDRC) PT $2,360,000 • Support to micro-finance (to CGAP) 3.10 From the data available in the PT database the grants appear to be relevant to the policy objectives over the period. But there is a lack of any sense of strategy or prioritisation in the work funded. The guidelines for grant criteria in 2002 stressed that grants should lead to significant measurable impact on IFAD’s rural poor clientele. Most grants pass through intermediaries. Many of the OR&T grants are used for programme development. Some have been used for support to IFAD’s programme implementation. The LAC region has had a substantial programme dealing with M&E and support to implementation. Despite such diverse uses there does not appear to have been any systematic evaluation, nor review of the strategic implications or lessons to be learned for IFAD as a whole.

Other grants 3.11 Technical assistance grants comprise more than just the three sets analysed above: small grants, agricultural research, and other research and training. IFAD’s budget makes special provision for grants to the NGO Extended Cooperation Programme, to the Special Operations Facility and for Environmental Assessment. In addition, some loan projects have grant elements, the Belgian Survival Fund makes grant allocations, and a large number of donors provide supplementary funds linked to loans. Table 6 lists the main components of grant disbursement from IFAD’s Loans and Grants System database. Table 7 analyses the same data by division for two years: 1995 and 2003.


Annex 4, page10

Table 66

Disbursement of grants US$

TAGsBegian

Survival Fund

Supplementary Funds

NGO-ECPLoan

Contribution Grants

Special Operations

FacilityOther All Grants All Loans G/L %

1995 6 198 896 7 408 763 9 752 643 706 354 1 173 109 604 336 0 25 844 103 189 667 478 14%1996 10 846 779 8 797 776 5 736 438 856 839 1 028 823 599 237 1 531 554 29 397 446 259 299 872 11%1997 12 063 246 6 711 163 5 101 510 948 460 751 398 1 062 330 0 26 638 107 254 112 239 10%1998 12 394 487 7 111 337 4 628 138 1 140 118 259 832 1 436 244 1 007 578 27 977 734 310 174 312 9%1999 19 079 203 7 432 075 6 152 671 1 233 906 600 784 1 113 237 350 113 35 961 989 285 069 809 13%2000 16 563 977 6 717 854 4 753 688 2 052 013 0 1 407 611 271 313 31 766 456 282 731 180 11%2001 14 789 167 5 300 965 5 255 437 1 204 361 220 349 1 433 988 1 228 124 29 432 391 296 944 896 10%2002 22 076 204 9 652 348 3 704 780 1 634 772 158 217 1 246 217 1 396 509 39 869 047 280 909 962 14%2003 17 033 968 6 886 584 2 805 085 1 887 341 388 590 1 053 926 2 513 120 32 568 614 305 104 535 11%Total $131 045 927 $66 018 865 $47 890 390 $11 664 164 $4 581 102 $9 957 126 $8 298 311 $279 455 887 $2 464 014 283 11%

Disbursement of grants - annual % by grant facilityTAGs BSF Supplementary NGO-ECP LCG SOF Other All Grants

1995 24% 29% 38% 3% 5% 2% 0% 100%1996 37% 30% 20% 3% 3% 2% 5% 100%1997 45% 25% 19% 4% 3% 4% 0% 100%1998 44% 25% 17% 4% 1% 5% 4% 100%1999 53% 21% 17% 3% 2% 3% 1% 100%2000 52% 21% 15% 6% 0% 4% 1% 100%2001 50% 18% 18% 4% 1% 5% 4% 100%2002 55% 24% 9% 4% 0% 3% 4% 100%2003 52% 21% 9% 6% 1% 3% 8% 100%Total 47% 24% 17% 4% 2% 4% 3% 100%

Source: Loans and Grants System

6 Data for Supplementary Funds covers only allocations for cofinancing with loans.


Annex 4, page11

Table 7 Disbursement of grants by division in 1995 and 2003 US$

TAGsBegian Survival

FundSupplementary

FundsNGO-ECP

Loan Contribution

Grants

Special Operations

FacilityOther All Grants

1995 Asia 0 0 3 622 415 0 0 4 620 0 3 627 035E&S Africa 0 5 852 389 73 997 0 674 792 386 651 2 6 987 831LAC 0 0 3 364 552 0 0 0 0 3 364 552NENA 0 1 026 579 2 337 359 0 163 127 8 859 0 3 535 924No division 6 198 896 529 795 354 320 706 354 0 0 1 7 789 366W&C Africa 0 0 0 0 335 189 204 206 0 539 395

$6 198 896 $7 408 763 $9 752 643 $706 354 $1 173 108 $604 336 $3 $25 844 103

2003 Asia 31 648 0 180 000 0 53 687 319 426 0 584 761E&S Africa 0 3 572 026 1 319 665 0 0 261 890 496 706 5 650 287LAC 22 000 0 53 500 0 222 895 80 533 1 378 929NENA 50 672 2 573 374 33 120 0 112 007 34 009 12 332 2 815 514No division 16 882 403 0 626 000 1 887 341 0 0 2 004 082 21 399 826W&C Africa 47 244 741 183 592 800 0 0 358 068 1 1 739 296

$17 033 967 $6 886 583 $2 805 085 $1 887 341 $388 589 $1 053 926 $2 513 122 $32 568 613

Disbursement of grants divisional % by grant facility

TAGs BSFSupplementary

Funds NGO-ECP LCG SOF Other All Grants1995 Asia 0% 0% 100% 0% 0% 0% 0% 14%

E&S Africa 0% 84% 1% 0% 10% 6% 0% 27%LAC 0% 0% 100% 0% 0% 0% 0% 13%NENA 0% 29% 66% 0% 5% 0% 0% 14%No division 80% 7% 5% 9% 0% 0% 0% 30%W&C Africa 0% 0% 0% 0% 62% 38% 0% 2%

100%

2003 Asia 5% 0% 31% 0% 9% 55% 0% 2%E&S Africa 0% 63% 23% 0% 0% 5% 9% 17%LAC 6% 0% 14% 0% 59% 21% 0% 1%NENA 2% 91% 1% 0% 4% 1% 0% 9%No division 79% 0% 3% 9% 0% 0% 9% 66%W&C Africa 3% 43% 34% 0% 0% 21% 0% 5%

Source: Loans and Grants System 100%


Annex 4, page12

3.12 Analysis of the LGS database reveals a number of interesting features about the grant programme:

• It is very diverse with more than ten types of facility some of which mix support to final clients with support for programme design and implementation.

• As a percentage of annual loan disbursement, grant spending accounts for around 11 percent, significantly higher than the ceiling on new approvals, mainly because this analysis includes supplementary funds (which are extra-budgetary).

• Technical assistance grants form the largest single category at 47 percent. • Contributions from the Belgian Survival Fund have remained fairly constant in dollar

values; other donors’ supplementary funds used to cofinance loans, and loan contribution grants have declined over the period.7

3.13 Comparison of disbursement between the years 1995 and 2003 indicates a number of trends:

• Control of grants has become increasingly centralised, with fewer grants directly accessible by the regional divisions.

• The Belgian Survival Fund has had a significant positive influence on the level of grants going to East and Southern Africa and to NENA.

• The decline in supplementary funds for cofinancing is most marked in Asia, Latin America and NENA. Supplementary funds for Africa have increased.

• The allocation of supplementary funds has reversed from being primarily cofinancing at the beginning of the period, with a small amount for consultancy inputs and thematic work (such as gender, environment) to the opposite at the end of the period with most funds being for consultancy and thematic issues.

• New facilities have developed over the period. The ‘other’ category has increased fourfold, largely through the Global Mechanism.

3.14 Overall the situation with grants is one of complexity and a lack of clarity about how different mechanisms relate to each other. The multiplicity of facilities increases the transaction costs for IFAD, with over 1500 disbursements identified in the LGS in 2003. The variety of mechanisms is a challenge for CPMs who have to become entrepreneurs in juggling funds. A wider problem is evident for IFAD as a whole to manage the grant process with scattered and incompatible data sources, rules and procedures.

7 The total volume of Supplementary Funds has fluctuated in the range US$8 million to US$12 million over the period. At the same time the proportion given in connection with loans has declined while that used for other forms of programme support has increased.


Annex 4, page13

Summary of issues

• Small TAGs are most numerous, averaging US$50,000 and mainly directed towards single countries

• Around 8 to 10 agricultural research and a similar number of other Research & Training grants have been awarded each year, averaging US$800,000 to US$1 million

• The agricultural research grants have been used for diverse purposes but in accordance with IFAD’s mandate and policies

• The diversity of OR&T grants is very wide, with uses ranging from programme support to relief and development

• The multiplicity of grants creates a complex web of sources, procedures and intended uses that in practice overlap and lack strategic coherence

• There is little or no data about the impact of the grant programme

4 Evaluation of four TAGS

4.1 Four grants from the Agricultural Research, and Other Research and Training categories were randomly sampled for this evaluation. The relevance, effectiveness and impact of other grants from the small TAGs category will be assessed during visits to sampled countries. Details of the four grants are set out in Table 8. The evaluation examined available documents including the Report and Recommendation to the President (RRP), design documents, progress reports and any dissemination or follow-on material. In addition, the OSC and TRC minutes were also examined. The documents were reviewed against a structured checklist, adapted for relevant questions from the checklist used for all sampled loan projects. Documentation for the EB in support of these grant proposals is deliberately brief but follows a similar structure to a project loan proposal. More details can be found in the design documents prepared by applicants, where these are available. For the four TAGs reviewed, a design document was available for the rice project but not the others. Various progress reports were also viewed and as much dissemination material and final reports as could be located.


Annex 4, page14

Table 8 Details of the sampled TAGs

Year approved 1994 1997 1998 1999 Title Programme for the establishment of

the Near East and North Africa Management Training in Agriculture (NENAMTA)

Global cassava development strategy

Validation and delivery of new technologies for increasing productivity of flood-prone rice lands in South and South East Asia

Programme for poverty alleviation and enhanced food availability in West Africa through improved yam technologies

Country(s) Algeria, Tunisia, Sudan, Egypt, Yemen, Jordan, Morocco, Libya, Lebanon, Syria

Benin, Cameroon, Nigeria, Ghana Bangladesh, Sri Lanka, India, Thailand, Vietnam

Benin, Cote-d’Ivoire, Ghana, Nigeria, Togo

Total cost / IFAD grant

US$5,500,000/ US$3,000,000

US$332,000/ US$200,000

US$1,806,000/ US$1,000,000

US$3,287,000/ US$1,250,000

Co-financiers AfDB, EDI, OPEC, and participating countries

Not specified BRRI, IRRI, NARS in participating countries

IITA, NARS in partic ipating countries

Category OR&T OR&T Agricultural Research Agricultural Research


Annex 4, Page 15

4.2 Relevance All of the grants are judged to be relevant to IFAD’s policies in force when they were approved. The rice project and NENAMTA both explain their relevance to regional strategy. The rice project is particularly good at demonstrating the link with loan projects.

4.3 Analysis Poverty analysis is not well treated in any of the projects. Rice and cassava both review the poverty context but the others have no analysis. Discussion of lessons from previous work is handled well in rice and NENAMTA; the treatment is poor in the cassava project despite prompting during the TRC and there is no coverage at all in the yam project. Institutional analysis is weak or non-existent in all projects, despite all the grants being implemented in multi-organisation settings.

4.4 Targeting and participation Three of the four projects make reference to elements of participation during design and to targeting of benefits. In none are the descriptions very detailed or clear. Reference is made to the planned use of participation during research, and to participation in project design workshops, but exactly who was involved is elusive.

4.5 Objectives None of the projects have a logframe. The rice grant has realistic and well structured objectives and outputs. Others are lengthy, inconsistent between different sections of the document, and with poor structure. The logic of the intervention is well explained in both the rice and NENAMTA. Neither cassava nor yam has convincing explanations of how the results might be used or the scope for ultimate impact.

4.6 Risks, innovation and sustainability NENAMTA has a reasonable treatment of risks and a good section on the importance of cost recovery. The others have no discussion of risk or sustainability. Innovation is dealt with in the rice and yam projects although the latter is weak on downstream aspects.

4.7 Implementation arrangements All the projects set out the planned arrangements for management of the grant. But none give any analysis of the capacity and commitment of intended implementers. The role of partners is treated satisfactorily in the rice project but less well in the other three.

4.8 Monitoring and evaluation Three of the four projects have little or no discussion of monitoring arrangements; indicators are absent, institutional roles and reporting are weak. The rice project has a brief discussion and includes a plan for evaluation.

4.9 Implementation and impact NENAMTA, the oldest project, has seen the least progress. It was designed as a continuation of the Agricultural Management Training for Africa (AMTA) started by IFAD in the 1980s. Start-up was slow, Memoranda of Understanding were signed among the various parties, but then reorganization in Washington led the EDI/WB to withdraw, a low allocation of staff resources in AfDB delayed implementation and the state of emergency in Cote d’Ivoire and resulting relocation of the AfDB all exacerbated the delays. The German Foundation for International Development, Food and Agriculture Development Centre (DSE-ZEL) has replaced the World Bank, but progress has been limited to the development and testing of training material.

4.10 The Cassava and Yam projects are not directly related, but should be viewed as part of a wider effort in support of roots and tubers in West Africa. This focus is entirely consistent with IFAD’s original mandate to work with food stuffs of direct importance to poorer sections of society. The cassava grant has its origins in a large TAG for IITA in the mid-1980s for research on biological control of cassava mealy bug. Successful research led to a cassava multiplication programme being funded as part of a series of World Bank assisted Multi-State Agricultural Development Projects. Evaluations of those projects have linked a large expansion in cassava production in Nigeria to technological change. Subsequently IFAD embarked on a Nigeria Roots and Tubers Expansion Project in 199?.

4.11 At the same time, the then CPM was interested in the wider issues of cassava products, marketing and trade. Brainstorming with other partners including SDC, IDRC and FAO led to the Cassava Development Strategy TAG. That work was in itself innovative being a precursor


Annex 4, Page 16

to the now mainstream concern over issues of trade. The grant was quite small. Another TAG was used to support a validation workshop at FAO, and additional grant funds, mainly PDF, funded cassava country case studies, including countries in regions other than West Africa. In 1998/99 the NGO-ECP facility was used to enable the Leventis Foundation in Nigeria to expand intake at its farmer school, specifically linked to IFAD project areas. Expansion of IFAD’s loan projects for roots and tubers has continued to now include Cameroon, Ghana and Benin.

4.12 To date the outputs from the Global Cassava Development Strategy are just reports and the approach has been criticised as being too supply driven; but FAO appears to be continuing support and making progress as the lead agency.

4.13 The yam project should be viewed in the same context of support for roots and tubers. Yam has been a relatively neglected area and IFAD’s support has helped develop research networks and informal thematic working groups. The apparent successes with release of new varieties, evidence of improved soil management, tools for field diagnosis of nutrient deficiency and low-cost seed health treatments, has led to a funding proposal by IITA for an IFAD grant of US$1.5 million to go to the April 2004 EB. Evaluation results from the first project have been used in preparing the next application. More work is needed to assess farmer benefits and to disseminate research findings.

4.14 The rice project with IRRI ended implementation in December 2003. A short summary of key achievements indicates that most outputs in the form of location-specific technologies were achieved. In addition planned socio-economic studies were accomplished and a number of technologies with potential for scaling up have been identified. Results available at IFAD headquarters do not include evaluation information from the perspective of potential beneficiaries. The project has published thirteen Technical Advisory Notes to disseminate research findings. This is the only one of the four projects with a clear emphasis on research dissemination.

Summary of issues

• The four sample TAGs all appear to be relevant, but most project designs are not well presented. In particular, objectives are mostly weak with no indicators or convincing arrangements for M&E. Despite all the projects working with partners there is no institutional analysis or assessment of capacity.

• NENAMTA has experienced major delays. The other projects have delivered their outputs but there is no independent evidence of effectiveness and impact.

• The experience with cassava shows the scope for programmatic actions that draw together different types of financing towards a common objective

5 Other evaluative evidence

5.1 In June 2003 the Office of Evaluation published an evaluation of the agricultural research grants programme.8 The methodology and findings from that study are summarised in Box 1.

Box 1 Evaluation of IFAD’s Technical Assistance Grants Programme for Agricultural Research

8 IFAD (2003) Evaluation of IFAD’s Technical Assistance Grants Programme for Agricultural Research . Report No. 1377. Office of Evaluation


Annex 4, Page 17

The study was major piece of work with both desk and field-based activities in four stages: i) a desk review of 42 grant programmes involving 67 TAGs out of 199

approved between 1979 and 2001 ii) discussion with IFAD staff iii) a formal survey of recipient institutions (25 responded out of 31) iv) field visits to nine grant recipients and some cooperating NARs The main conclusions were: Overall

• IFAD has played a strong advocacy role in redirecting the focus of the CGIAR system towards more poverty-focused research

• TAGs are usually well designed and overall they have been reasonably effective in achieving stated objectives

• Information about poverty impact lacks impact assessment studies • The programme remains relevant to the IFAD poverty mandate • Impact is clearest in terms of institutional capacity-building

On programme policy, strategy and procedures • The AR/TAG programme needs a clear strategy, priorities and better focus. • The consistency between resource allocation in the AR/TAG programme and

that of IFAD loans should be increased. • Programme procedures need strengthening for selection rigour and efficiency • Reporting should be appropriate to IFAD concerns – reports are not consistent

and veer between publicity handouts and technical dissertations On the research funded

• AR/TAG grants show wide variation in quality • Some TAGs are moving beyond technology validation to technology

dissemination – which raises strategic issues • There is a danger of research achievements and impact being undermined by

the narrow time frame • The programme has made a good contribution to capacity-building,

particularly in participatory methodologies and poverty oriented research at the national level

• Too little is known about the poverty impact of individual TAGs or clusters of TAGs

6 Discussion

6.1 This review of the grant portfolio has highlighted a number of areas that are important both for grant management and for future policy. Firstly, the evolution of instruments and procedures during the past ten years reflects the importance of the grant programme and the desire by management to develop a coherent link with loans. But the protracted delays in developing the new policy and low attention to evaluation of impact has resulted in a new policy which lacks self-critical reflection. OE’s evaluation of agricultural research grants is cited in the new policy but without building on the discussion in that study.

6.2 Secondly, neither the new grant policy nor the policy statements during the period pay sufficient attention to the ways in with the small grants and OR&T grants have been used in


Annex 4, Page 18

support of programme development and implementation. A key recommendation of the 1994 Rapid External Assessment was for IFAD to shift resources from the design phase of the project cycle to implementation support. The grants programme is one way in which the regions have attempted to do that, albeit in the context of a complex and changing array of funding instruments. This role has not been fully explored and does not feature in the new policy.

6.3 Last, there is little evidence of the grants being used in ways that enable IFAD to fulfil objectives of knowledge sharing, innovation and partnerships. These three elements describe key features of the way IFAD tries to work and are central to corporate strategy.

Questions for the country visits: How have the TAGs linked to the loan programme and been used in support of IFAD’s country programme? How effective have they been in achieving stated objectives? What is the evidence of impact on IFAD’s client group? To what extent have the TAG’s:

• Enhanced IFAD’s catalytic role, extended outreach and helped forge partnerships with country-level institutions at both field and policy levels

• Contributed to rural institutional dynamics for participatory rural development • Helped achieve a high-impact lending portfolio • Enabled knowledge gathering and sharing on rural poverty • Supported agricultural research and technology innovation

Independent External Evaluation of IFAD Desk Review Report Annex 5 – Compliance with Replenishments V and VI

Annex 5 page 1

Annex 5 IFAD Replenishment Round Undertakings and IEE Assessment of Compliance/Progress To Date

Repl. Round

Recommendations/Actions Output/Results Target Date

Compliance/Progress to date

V IFAD V: Plan of Action 2000-2 A. General Policy Objectives 1. Ensure that COSOPs bring out the

national and sectoral policy issues relevant to programme success with conclusions on project proposals reflecting such analysis

(a) More comprehensive analysis of enabling policy environment to improve prospects for programme and project success

Dec 00 (a) Review of 20 COSOPs shows generally satisfactory agricultural sector and rural poverty analysis, and generally good links between the identification of opportunities and constraints and agric/poverty analysis. Links made to national policy priorities are made but often in only vague terms. Limited, if any, detail in COSOPs on PRSP policy frameworks.

2. Adopt an approach that harmonises the CDF and UNDAF, bearing in mind IFAD’s specific mandate, and ensure that national policy and institutional environment in prospective recipient countries is taken fully into account in deciding the level and form of assistance.

(b) Over the next 3 years, gradual achievement of effective linkage between COSOPs and programming UNDAF and CDF (in selected countries) (c) Complementary to this, increasing impact on poverty through participation in the design of poverty-reduction strategies with recipient countries.

Ongoing approach Ongoing approach

(b) Linkage between COSOPs and UNDAF not made explicit in many cases. CDF largely incorporated into PRSP approach in most countries. The latest cohort of COSOPs attempt more explicit linkages with PRSPs, but discussion of IFAD strategy in relation to other donors’ strategies and alignment/harmonisation around PRSP remains weak. (c) IFAD participation in PRSP processes weakly discussed in COSOPs. Many COSOPs were formulated before the PRSP got underway. Of those that were not, few provide a satisfactory description of IFAD engagement or policy influence.

3. Ensure in collaboration with relevant government agencies, the fullest participation of prospective beneficiaries and other stakeholders in project design and implementation.

(a) Increase the extent of beneficiary and stakeholder participation with projects that are better managed and that respond better to beneficiary and stakeholder needs and sense of ownership.

April 01/ Ongoing

(a) 43% of projects reviewed showed unsatisfactory evidence of beneficiary engagement. However many of these were earlier cohort projects (mid 1990s). Amongst the latest cohort 75% showed satisfactory evidence of beneficiary engagement.


Annex 5 page 2

Repl. Round



V IFAD V: Plan of Action 45% of 1998-2000 COSOPs showed satisfactory performance in consulting with key stakeholders in formulation. This increased to 75% amongst 2001-04 COSOPs.

4. Give appropriate weight to borrowers’ implementation performance in determining resource allocations.

(a) Decisions on future allocations determine increasingly by performance assessment of portfolio, taking into account opportunities to address institutional weaknesses.

Sept-December 01

(a) Recent data on poverty-efficient aid allocations by Beynon (2003) suggests IFAD concessional assistance is moving in the direction of better performing countries, but not so much because aid allocations have shifted but because of improvements in borrower performance. IFAD portfolio data don’t reveal a lot about how responsive allocations are to borrower performance, but CPIA scores suggest that the bulk of project investments are going to countries with better policy and institutional performance; or at least are not going disproportionately to the worse performers.

B. Objectives Relating to Specific Approaches and Impact

1. Improve the effectiveness of the Fund’s approach to the task of poverty alleviation through an intensified search for new and innovative solutions in key areas. These include the environment; household food security; grassroots organisations and capacity building; rural financing services and gender.

(a) Increase in number of projects reflecting new approaches to major problems in key areas. (b) Evidence in project results of sustainable improvements in livelihoods of beneficiaries.

Annual reporting As of Sept. 00

(a) IFAD Annual Progress Report 2002 suggest 50% of projects evaluated provided evidence of successful innovation in some aspects. IEE Review finds that 75% of projects describe themselves as innovative in one criterion or another but that the lack of a clear, consistent definition of ‘innovation’ is a problem. The label of innovation appears to carry little substantive information about the project or the wider portfolio. (b) IFAD APR 2002 finds 60% of projects as unlikely to be sustainable. The IEE review finds similarly with some signs of improvement in recent years. Sustainability remains largely an aspiration rather than


Annex 5 page 3

Repl. Round



V IFAD V: Plan of Action remains largely an aspiration rather than something that has to be designed into projects.

2. Build on the Fund’s comparative advantage by enhancing its policy dialogue and analysis in relevant areas and by sharpening its focus on areas that can act as catalysts for wider application.

(a) IFAD’s role as a catalyst and knowledge centre enhanced and increasingly exploited by stakeholders and others involved in development. (b) Rural development policies improved through IFAD’s influence and in countries where IFAD operates, a policy environment beneficial to the rural poor emerging or further developed. (c) IFAD’s capacity in policy dialogue and project design improved.

Dec 00-02 Ongoing Ongoing

(a) COSOP review suggests less than satisfactory performance in identifying opportunities for ‘acting as a catalyst’ – partly due to the profile of the COSOPs under review and partly due to lack of clarity as to what ‘acting as a catalyst’ means. In most cases being a catalyst is interpreted as leveraging co-financing. There are signs of improvement. The 2001-2004 cohort of COSOPs is better at describing IFAD’s catalytic role than the earlier cohort. (b) COUNTRY EVALUATIONS (c) COSOP review suggests satisfactory performance in identifying/recognising opportunities for pro-poor dialogue as part of country strategy, but often links between this and project portfolio unclear. Opportunities for policy dialogue often presented as a wish list rather than a clearly identified set of entry points into policy discussions at sectoral or national level. Project still regarded as the primary vehicle for IFAD policy dialogue.

3. Improve impact assessment (a) With new and improved practices agreed and in place, and a system for regular assessment of IFAD’s success in promoting innovations and their replicability, IFAD internationally recognised as a sound institution with a durable and effective impact on poverty alleviation.

April 01/Sept. 01

COUNTRY EVALUATIONS


Annex 5 page 4

Repl. Round



V IFAD V: Plan of Action 4. Document innovative features in a

standard format, and devise and implement a strategy for knowledge management and sharing lessons learned with other stakeholders.

(b) IFAD’s role as a centre of excellence in the field of rural poverty alleviation more widely recognised. Extent and frequency of information gathered, shared and disseminated markedly increased over the next 3 -5 years.

April 01/02 PARTNER REVIEWS

5. Direct its programme of assistance to middle-income countries in which there are clear opportunities for innovative projects and for leveraging institutional and policy reorientation in favour of the rural poor, together with mobilization of more domestic resources.

(a) Over the 2000-02 period, IFAD’s programme in middle-income countries concentrated on innovative approaches and on greater leverage, both in resources and policy development.

Ongoing

6. Use grant resources to further the core objectives of the Fund, in particular innovative policy and initiatives, institutional solutions and knowledge sharing.

(a) Refined policy agreed on use of grant resources to further the core objectives of the Fund.

Sept. 00/ In consultat-ion with Board

Evolution of instruments and procedures shows importance of the grant programme but protracted delays in developing new policy and low attention to evaluation of impact has resulted in a new policy lacking critical self-reflection. Inadequate attention to small grants and OR&T grants. Limited evidence that grants are being used to pursue wider corporate objectives such as knowledge sharing, innovation and partnerships.

7. Explore the scope for increasing the financing available from non-donor resources.

(a) Agreed approach to possible new ways of increasing non-donor finance.

Dec 00/April 01

PORTFOLIO REVIEW

C. Complementarity and Replication Objectives

1. Forge more strategic partnerships and expand the Fund’s collaboration and co-financing with others donors. COSOPs should provide the framework for such cooperation. The objectives are to improve mutual learning and lesson sharing; share institutional capacity; and strengthen the potential for replication and expansion of best practices in poverty alleviation.

(a) IFAD’s efforts at increasing co-financing reach at least 30% of its annual lending programme, with an increased and measurable volume of further resources, national or external leveraged for poverty reduction. (b) Significant improvement in both quantity and quality of technical and financial cooperation with other donors shown in

April 01 & annual reporting thereafter April 01 onwards

(a) Review of portfolio shows co-financing at 30% in 2003/4. COSOP reviews indicate growing attention to cofinancing issues at level of country engagement.


Annex 5 page 5

Repl. Round



V IFAD V: Plan of Action alleviation.

cooperation with other donors shown in programming, henceforth leading to more sustainable, expandable and replicable poverty alleviation.

D. Objectives Relating to Use of Resources

1. Consider the distribution of annual lending by region, including demands from new countries and post-crisis recovery situations.

(a) Agreed distribution of lending programme by region

April 01 & annual reports thereafter

PORTFOLIO REVIEW

2. Concentrate its resources in poor countries and ensure that their share, on highly concessional terms, should be no less than 67%.

(b) Continued focus on poor countries ensured.

PORTFOLIO REVIEW

3. Draw up criteria to determine the circumstances under which loans on highly concessional terms might go to other countries, with a proposed ceiling on the proportion of such funds.

(c) Criteria and ceiling for highly concessional loans to other countries agreed, including the role of Board in approving such exceptional cases.

?

Repl. Round



VI

2004-06

Operationalising the Strategic Framework and the Regional Strategies

Performance-Based Allocation System 1. The PBAS in IFAD will be developed through a consultative process with Members and presented to the Executive Board. 2. Country performance assessments under PBAS will be initiated. 3. Allocation decisions will be governed by PBAS.

1. (a) Revised document REPL.VI/4/R.5 (b) Informal seminars (c) Technical discussions with a special informal panel (d) Executive Board approval 2. PBAS implemented 3. Ongoing process

(a) Dec 02 (b) EB 03 (c) As necessary (d) Sept 03 2. 2004 3. 2005

PBAS in preparation following work of EB working group.

COSOPs 1. The conclusions of the EB review of the pilot experience of COSOPs will be

1. EB review 2. Country strategies

1. Dec 02 2. Ongoing

Evidence of improvement based on pilot experience. Latest cohort showing stronger/clearer structure, objectives and


Annex 5 page 6

Repl. Round



VI

2004-06


pilot experience of COSOPs will be internalized. 2. IFAD will use the COSOP process to advocate a policy and institutional transformation agenda for effective rural poverty reduction, in partnership with other organisations and ensuring country ownership of the agenda where applicable, in line with the PRSP. 3. IFAD will participate in PRSP processes, giving priority to those countries in which the potential for effective rural poverty reduction is high, and the potential for partnerships with other organisations in the context of the PRSP is most promising.

3. Country strategies Ongoing 3. Ongoing

stronger/clearer structure, objectives and articulation of IFAD’s niche, good quality ‘upstream’ analysis including gender analysis, rationale for project interventions etc. Weaknesses exist in linking assessment of national & sectoral policy context to project pipeline, ensuring local ownership, treatment of governance/institutional issues & equity, use of logframe; identification of opportunities for scaling up and M&E at programme and strategy level. COSOP review suggests less than satisfactory discussion of IFAD participation in PRS processes

Field Presence and In-Country Capacity 1. IFAD will further analyse the issue of enhancing its field presence and in-country capacity. In consultation with List convenors, the Secretariat will convene a proposal on the process and submit to the EB.

1. EB paper

1. Dec 02

Board paper received. Piloting beginning in a number of countries. MORE IN PHASE II

Working in Partnership 1. A Strategy for achieving greater involvement of private sector participants in IFAD programmes, through cofinanicng and other forms of partnership consistent with IFAD’s mission, will be provided to the 80th Session of the EB in Dec. 2003. 2. Opportunities for partnerships with private sector enterprises in the financing of projects should also be pursued more systematically as part of IFAD’s cofinancing and partnership strategy

1. EB paper 2. COSOPs 3. Regional & country strategies

1. Dec 03 2. Ongoing 3. Ongoing

COSOP reviews points to some increase in mention of possible private sector partners but, overall, still lagging well behind corporate aspiration. Only one COSOP reviewed included strong articulation of possible private sector partnerships. Some reference to mul ti-stakeholder processes but relatively little on coordination efforts and IFAD’s proposed engagement. Recognition of the importance of the PRSP as a coordination


Annex 5 page 7

Repl. Round



VI

2004-06


cofinancing and partnership strategy objectives. 3. IFAD will continue to engage actively in multi-stakeholder coordination processes, develop programme based partnerships and pursue cofinancing to leverage its programme of work.

importance of the PRSP as a coordination mechanism not matched by detail on potential IFAD engagement.

Gender Mainstreaming 1. An action plan for gender mainstreaming for 2003-5 will identify actions to operationalise strategy framework principles as they relate to gender mainstreaming and the empowerment of rural women. Annual progress reports will be provided to the EB.

1. Annual portfolio report

2003 & annually

Action plan prepared. Review of progress included as part of Progress Report on the Project Portfolio (PRPP). Good in principle but the reporting rather scant as a result of being folded into PRPP. Regional contributions variable.

Monitoring Results and Impact

Results and Impact 1. A consolidated overview of IFAD’s impact and development effectiveness and a summary of lessons learned from a group of completed projects evaluated during the course of the given year will be presented in annual reports.

1. Annual report to EB

Dec 03 & annually

Independent Evaluation Function 1. An evaluation policy for IFAD will be formulated 2. Management will prepare a detailed proposal on strengthening the independence and effectiveness of IFAD’s evaluation function.

1. Policy document 2. EB paper

1. 2003 2. April 03

Evaluation policy presented to Board. Proposals for strengthening the independence of OE implemented. Strengthening of Evaluation Function underway.

Independent External Evaluation of IFAD Desk Review Report Annex 6 – Knowledge Management

Annex 6 Page 1

Annex 6

Knowledge Management

“The fundamental cure for poverty is not money but knowledge”1

1 Introduction

Overview of this annex

1.1 IFAD has always required and pursued good information and knowledge management but has not always made corporate explicit statements about this. A major attempt to make knowledge management an explicit part of IFAD work was made after the Rapid External Evaluation report in 1994.

1.2 IFAD pursued the issues of knowledge management with some energy and drive from senior management from about 1995 to 2002. Some of the methods pursued were effective and some were not well suited to the organisation. The documents of IFAD during this period contain a large number of different formulations of aims and objectives in knowledge management for the organisation.

1.3 During 2003 the phrase “knowledge management” virtually disappeared from IFAD documents. The many initiatives of the previous three or four years appear to have been allowed to lapse but the many declared objectives remain as corporate policy and continue to appear on the public website.

1.4 There is a great deal of good knowledge sharing between different members of staff and between different programmes although most successful exchanges appear often to be based on informal mechanisms. There are significant efforts at sharing IFAD information externally.

1.5 Overall, knowledge management remains weak. There are major problems with the collection of learning from project level work and there are difficulties in improving knowledge management within IFAD, which appear to be more a result of the institutional culture than a result of poor policy or corporate level intent.

1.6 The new Policy and Communications Divisions are still settling into their roles and it is not clear how they will contribute to knowledge management within the organisation. However, it is clear that a major new drive in knowledge management is being developed and a draft strategy document is being discussed while this report is being drafted. The IEE will not be able to comment on the strategy in the Desk Review stage of this evaluation. The new communications strategy will face some of the same difficulties that the previous Knowledge Management strategy faced in 2002.

1 Sir William Arthur Lewis and quoted in: EB 2003/80/R.4 19 November 2003, Agenda Item 4, IFAD, Executive Board – Eightieth Session, Rome, 17-18 December 2003, FIELD PRESENCE PILOT PROGRAMME. (Para 6). 6


Annex 6 Page 2

Evaluation methodology

People met

1.7 During the desk study our work has been based on discussions with individuals. Most of the interviews that were carried out for the purpose of assessing sample projects also included some elements of knowledge management. A number of interviews were carried out specifically for the purpose of understanding the situation and history of knowledge management within IFAD.

Meetings held

1.8 The IEE held a number of open group meetings to which staff were invited in as open a way as we could design. The IEE also invited three sets of individuals drawn at random from staff lists in order to discuss different issues of knowledge management in a focus group setting.

Project and programme assessments

1.9 The IEE team analysed programmes and projects that emerged from the random sampling process. The desk study involved interviews and examinations of documents, which led to complex scoring of the COSOPs and projects on a range of different criteria. These criteria include a range of questions that relate to learning and collecting information. The aggregate scores from this exercise have been used to assess some aspects of knowledge management.

Structure of the report

1.10 This report sets out to test the hypothesis that knowledge sharing through innovation management, learning processes, influencing and scaling-up has improved the effectiveness of IFAD’s programme impact and policy influence. It attempts to provide:

• a review of the public statements by IFAD on the purpose and functions of knowledge management,

• an inventory of methods and instruments for knowledge management in IFAD,

• an assessment of knowledge management within IFAD against different steps in a model of knowledge management,

• an assessment of knowledge management in IFAD in comparison with a range of other agencies,

• a list of issues in knowledge management that the IEE will continue to work on in general and focus on during the country visits.

2 Recent policy and experience with knowledge management

2.1 This section covers the findings from the desk study of the positions adopted by IFAD on knowledge management. It looks at IFAD’s presentations of itself initially as a small organisation needing to influence others; then, as striving to be a knowledge organisation and knowledge centre; and last, at the plethora of declared aims in knowledge management and the uncertainty of the current situation.


Annex 6 Page 3

A Brief History

The early statements of need to influence

2.2 IFAD has always dealt with information and knowledge and this has tended to be an implicit part of the work. There are few descriptions of how information should be managed but the implication is clear even in the earliest statements of intent of the organisation. For example, the Agreement Establishing IFAD2 which gives as part of the objectives “to strengthen related policies and institutions within the framework of national priorities and strategies,… ”.

2.3 Lending Policies and Criteria 3 states that “IFAD’s investment priorities should be guided by the consideration of sharing know-how with other international financial institutions (IFls), so as to stimulate them to invest in rural development with a view to alleviating mass poverty”. These statements do not explicitly state the need to collect and use information to win influence but it is implicit.

2.4 The Division of Information and Communication was charged with work that would be called knowledge sharing. It described its work as promoting IFAD’s views on key issues of rural poverty, drawing attention to the role of rural women and improving communications work that would support effectiveness of IFAD projects. These objectives might have been written at any time in the history of IFAD.4

2.5 All corporate strategic statements refer to the need to use information to inform and influence other agencies. One example from 1999 is typical, “Exchanging knowledge on policy and practices. IFAD will continue to work with selected international agencies and civil-society and non-governmental organizations (CSOs and NGOs) to establish networks for the analysis and exchange of experiences and practices on vital cross-cutting factors that have an impact on poverty”.5 Again there is an implicit need to collect good information from project level experiences.

Becoming a knowledge organisation

2.6 There was an important change in making knowledge management an explicit aim of the organisation in 1994 when the Rapid External Assessment reported, “The assessment team sees the Fund emerging as a “knowledge organisation” a knowledge creator, a knowledge storehouse and a knowledge disseminator”(p25). The phrases knowledge organisation and knowledge management were probably new but the concepts were not new and IFAD staff may have felt that they were being urged to do what they already did without having a name for it.

2.7 The recommendations of the REA appear to have been accepted by senior management. However, immediately after the evaluation there was relatively little direct action on knowledge management. Most activities focus on technological solutions and an important investment was made in improving the infrastructure in headquarters offices.

2.8 In 1998 there appears to have been a renewed focus on the human and management aspects of knowledge management and this is reflected in a range of initiatives. The Strategic

2 Article 2, Agreement Establishing the IFAD (1977 and last amended 1997) 3 Para 5, Lending Policies and Criteria (1977 and last amended 1997) 4 In fact, this comes from: Information and Communications Activities Carried Out in 1992 and Work Plan for 1993. EB 92/47/INF.6, 4 December 1992. 5 Executive Board – Sixty-Seventh Session, Rome, 8-9 September 1999, REVIEW OF THE PROGRAMME OF WORK AND BUDGET OF IFAD FOR 2000.


Annex 6 Page 4

Framework for 1998-2000 “Meeting Challenges in a Changing World” puts knowledge management in the centre of IFAD’s concerns and installs it as one of five Strategic Thrusts. ‘Thrust D, Establish and use knowledge networks on rural poverty in order to create a recognised knowledge organisation.’ As part of this the public document states “Central to IFAD’s efforts to re-engineer its internal processes and strengthen linkages with other organisations has been an emphasis on generating analysing and disseminating knowledge about poverty .”

2.9 The Meeting Challenges initiatives included the introduction of Corporate Scorecards, which are divided into four sections including a section on Innovation and Learning. There are three objectives in this section:

• Learn from experience, synthesise knowledge and share with others.

• Capture and share IFAD’s field knowledge with emphasis on innovative aspects

• Improve impact assessment.

2.10 These statements help to make more explicit the need to improve information collection from the project level experience. The Scorecard never seems to have been used and the IEE were unable to find a Scorecard in its original condition.

2.11 The Strategic Framework was replaced in its time by the IFAD V Action Plan 2000-2002 and again knowledge management remains central. The Fund is described as having four major roles.

“As an innovator in the development of effective rural poverty -eradication instruments, models and know-how at the grass-roots level, IFAD seeks new and effective ways to address the constraints faced by its beneficiaries in a diversity of local contexts. In so doing, it consolidates, refines and enhances its ‘intellectual capital’. As a knowledge institution, IFAD builds on its role as an innovator through a process of mutual learning and lesson-sharing with other stakeholders active in this field. As a catalyst, IFAD extends the outreach of its poverty -eradication efforts through strategic partnerships with other donors, governments, non-governmental organizations (NGOs) and civil-society organizations working with the poor at the grass-roots level. Such partnerships build on cross-fertilization of ideas and approaches, taking into account the programmes and efforts of other donors. The Fund’s catalytic role also implies facilitating the replication and scaling up of national and local initiatives, as well as influencing policy and practices in favour of the poor. As a leader in effective, sustainable poverty-alleviation strategies, IFAD demonstrates its achievements qualitatively and quantitatively.”

2.12 This is the stage at which innovation is formally and explicitly linked with knowledge management. It is clear that there is little point in developing innovative practice if this is not shared with other agencies. The Strategy requires that knowledge management operational guidelines are prepared that will facilitate documentation of innovations and sharing of lessons learned.


Annex 6 Page 5

2.13 The initiative set up a Knowledge Management Facilitation and Support Unit, which was to prepare a Knowledge Management strategy. Fairly rapidly a number of Thematic Groups were established (for example: Diagnostic Tools, Rangeland Management, Rural Micro-enterprise, and Rural Finance). Work on Gender was more advanced and a specific sub-site was created on the IFAD website.

2.14 The Knowledge Management Strategy was never formally approved although it went through several versions and was subject to comment and discussion.

2.15 In May 2000, IFAD developed a policy on disclosure of documents, which would have a positive impact on learning from IFAD’s work.6 Notice that the goal of becoming a knowledge organisation is taken as a central aim. The Knowledge Management strategy is still under discussion but will reach three different targets; other donors, beneficiaries and the general public.

11. Management is also aware that, in accordance with IFAD’s goal of becoming a knowledge organization and in the interests of increased knowledge-sharing, a disclosure policy is not sufficient. The Fund is in the process of defining an independent strategy on knowledge management, which will, inter alia, have the following objectives:

• improve the quality, efficiency and potential for replication of development programmes and projects through enhanced coordination and sharing of "lessons learned" among donors;

• encourage beneficiary participation and project ownership through increased knowledge of development activities in neighbouring or similar regions;

• promote public debate on the benefits of sustainable development activities. 2.16 In December 2000, the plans for the following year for each Division contain a specific paragraph on work on knowledge management (EB 2000/71/R7/Rev.1). The Knowledge Management Facilitation and Support Unit put forward a business case which was approved alongside similar proposals from the seven other processes that required re-engineering (EB 2000/71/R.9).

2.17 The Regional Strategic Plans that were developed and published in 2001 all make reference to the role of knowledge management in their country programmes. For example: in Near East and North Africa, knowledge management is one of the “pillars”; in West and Central Africa its is one of the “implementation modalities”; in East and Southern Africa, better knowledge is one of the Thrusts and knowledge sharing is an “implementation approach”.

2.18 In 2001 a major evaluation on IFAD’s capacity to support innovation produced a devastating report, which criticised many aspects of IFAD’s functioning that hampered the promotion of innovation and the sharing of information on innovation. This review is poorly reported in other IFAD documents and the negative observations are disguised as interesting findings. This report deserves serious attention because it suggests that there are fundamental problems with knowledge management although the focus of the study was on the ability to promote innovation. The processes of knowledge management and promotion of innovation

6 EB 2000/69/R.13/Rev.1 4 May 2000, IFAD, Executive Board – Sixty-Ninth Session, Rome, 3-4 May 2000, IFAD POLICY ON THE DISCLOSURE OF DOCUMENTS.


Annex 6 Page 6

overlap strongly and the capacities required for both areas of work are similar and complementary. The special case of the ACP of this report is considered below.

2.19 The report is uncompromising is some of its assertions. For example: there is no clear mechanism for learning (12), only rarely are lessons learned (14), blockages to the promotion of innovation listed in paragraph 55, include: lack of direction, capability, absence of learning and sharing of knowledge, inadequate structure and processes and decision making. Of course there are good features that are mentioned also but the scale of the criticism requires a considerable response from management. Also in paragraph 55 is the mention of an unhelpful institutional culture.

2.20 The External Review reporting in June 2002 found that progress in knowledge management has been slow. “The translation into reality of its goal to become a ‘knowledge organization on rural poverty’ (with a well-functioning knowledge management system), which was formulated in 1995, has not come easily to IFAD.” This is put in gentle terms but the language cannot hide the fact that the work proposed in the 2000 plan has not been achieved. Some work that should have been completed by the spring of 2002 was not available to the review and the ERT manages to suggest that “a less hesitant approach might have been indicated” as recognition that some of the drive had been lost.

2.21 Nevertheless the ERT urged the Fund to pursue work in this area, “IFAD should sharpen its knowledge management to facilitate the dissemination, both internally and externally, of lessons learned from its experience and to increase its visibility”7 (p x). There are many other references to the need to continue the work and there are some specific suggestions and concludes (page 28 of 28):

The ERT believes that knowledge management needs additional attention, with monitorable indicators of progress in the three areas indicated and involvement of the Fund’s management. This attention should cover the full range of activities from knowledge generation to its dissemination – so as to make IFAD the premier institution influencing the thinking and activities for overcoming rural poverty.

Restructuring Policy and Communications 2.22 IFAD disbanded its Communications Division in 1994 at about the same time as the Rapid External Assessment made its recommendations about the need to become a knowledge organisation. The IEE has not been able to find written justification for the dissolution of Communications as a separate division. It would be possible to defend the decision on the basis that each separate section of the organisation has to take responsibility for communications and can be serviced by a central division that would manage the production and dissemination of communication materials.

2.23 In fact the recreation in 2002 of a Communications Division at the same time as the creation of a Policy Division within the Department of External Affairs suggests that senior management now want to move in the other direction of creating specialist functions in one part of the organisation with a remit to service all the other parts. The fact that both Policy and Communications are both new and finding their way means that it is more possible for them to develop their collaboration and the precise sharing of functions. It would be more difficult if one were long-established and being joined by a new office.

2.24 The Communications Division is developing its own strategy at the present time (March 2004) and the IEE will not be able describe the new arrangements in this report. The Division is assessing possibilities for collaboration with the Information and Resources

7 External review of the Results and Impact of IFAD Operations, Report of the External Review Team, June 2002.


Annex 6 Page 7

Centre and the Office of the Secretary. The successful configuration of different tasks will be very helpful. Currently it would not be obvious whether a particular communications product would come from the IRC or the Communications Division.

2.25 The Communications Division and the Office of Evaluation also need to find clear distinction of their roles. The OE produces a large number of communications products and it must retain its ability to produce reports independently of any influence from any other part of IFAD. However, since most of the products go straight into the public domain there is no reason why they should be managed by a separate chain of command. Negotiations between OE and the Division of Communications might lead to an effective separation of OE’s two roles: the fearless criticism of the organisation; and, the promotion of learning outside IFAD from evaluation of IFAD programme work.

Current Objectives of Knowledge Management

Different Statements of Intent

2.26 The IFAD web site leads directly to a sub-site called knowledge management and the purposes are explained in this paragraph (emphasis added).

Effective knowledge management can help IFAD to realise its mandate of fighting rural poverty:

• knowing more about the rural poor will help us to serve them better; • sharing our knowledge with others will multiply its impact; • learning from our partners will improve our operations; • knowing more about our internal processes will make us more efficient in all

our activities. 2.27 These purposes originate in different ways in different documents over the years with a peak of discussions in 2001-2002.

2.28 The need to base IFAD information work on its experiences in its projects is not firmly fixed in statements of intent but is alluded to and the Policy Task Force is clear that IFAD is not a “policy institution per se” but that its policy work should be “embedded in IFAD’s operational experience”. Also its work will include “examining IFAD’s grassroots experiences in the light of emerging concerns of the international community (website under Policy Division).

2.29 Similarly there is a diversity of possible targets for information and learning that IFAD comes to hold. The beneficiaries themselves are targets and IFAD should help “increasing access to knowledge so that poor people can grasp opportunities and overcome obstacles,”8.

2.30 Country level influence is important to IFAD and international level influencing work is mentioned in many places. Sometimes this is described as a catalytic effect but the precise meaning of this appears to be the influencing of other donor agencies as well as national government policy making and project implementation practice. The general public are also described as potential targets for IFAD information and knowledge. In many documents these aims are described in terms of the 1994 evaluation, that “IFAD become a knowledge centre on rural poverty alleviation” (Program of work for 2001 OE, EC 2000/26/W.P.2/Rev 1), that

8 REPL.VI/4/R.3 26 September 2002, IFAD, Consultation on the Sixth Replenishment of IFAD’s Resources – Fourth Session, Rome, 9-10 October 2002, CRITERIA AND PRINCIPLES FOR THE DEVELOPMENT AND OPERATION OF A PERFORMANCE-BASED ALLOCATION SYSTEM IN IFAD


Annex 6 Page 8

it “become an authoritative reference point on policies related to rural poverty reduction and development” (Policy Task Force 11/11/2002).

2.31 In our attempts to assess the knowledge management functions in IFAD we have assessed the multiplicity of declared objectives and purposes and the draft Knowledge Management Strategy developed in 2001/2. The IEE team has not been able to identify a simple set of objectives that the Fund can be said to be committed to in knowledge management. This is made more difficult by the apparent drift away from knowledge management as a current term in corporate communications and in staff discourse.

An uncertain position

2.32 A tour of the IFAD website makes it appear that knowledge management is still a current slogan and focus of work. But in fact, it seems as if the phrase has lost its meaning and prestige in the organisation and the initiatives developed under knowledge management have been allowed to fall into abeyance. Talking to staff at IFAD headquarters adds to the feeling that this is an uncertain time in which senior management has lost confidence in the knowledge management approach and is replacing it with other approaches to the same issues that remain central to the core business of IFAD. The new initiatives do not seem to be part of an explicit or highly visible approach.

Summary of issues

• What should the IEE evaluate in knowledge management? If IFAD should be evaluated against what it claimed to be doing then:

o Is there a common model of knowledge management especially now that the phrase has less currency?

o Is there a common understanding of the various formulations of advocacy work: the catalytic role, policy dialogue, innovator in know-how, leader in poverty alleviation strategies, sharing of knowledge to multiply impact and being a knowledge organisation?

• How best to drive knowledge management in IFAD? Can it be done largely from one Division and what advantages and difficulties would this create?

3 Instruments and Initiatives

3.1 This section is a partial inventory of methods and instruments for knowledge management that the IEE has come across during the desk study. The purpose is to provide an overview of the types of method and range of skills available within the organisation in order to base an assessment of knowledge management on both practical and cultural strengths and weaknesses.

The Website - Portal

3.2 The IFAD portal has become an essential element of knowledge management both for sharing within the organisation and for the export of information to external users. The site is well regarded by users. The intranet is the most important tool mentioned by staff in meetings called by the IEE on knowledge management. Of course, staff put immense value on face to face contacts but the use of email and access to information via the intranet is crucial to their work.


Annex 6 Page 9

3.3 However, for an outsider it is extremely difficult to find a particular document from the starting point of the dates of meetings of management boards whose functions are not obvious. The members of the IEE team are good test cases since we approached the sites as naïve intelligent and necessarily determined observers. After a few weeks it became clear which papers would be presented to the different bodies and we came to know which sessions would contain which papers. It was even then, sometimes difficult to identify the right document from its title. It is easy to imagine less determined visitors giving up the struggle and leaving the site without the information that they came to seek. The site can only provide access to documents that were produced since the mid-1990s and this might be a weakness where a greater historical perspective was required.

3.4 The search engine on the webdocs site can be of great service but tends to be unspecific and deluge the searcher with virtually unlimited numbers of documents. The search engine on the public site seems ineffective and unable to find documents which are clearly available. It is not able to search on document numbers which may be a serious weakness. This weakness is aggravated by the tendency for documents to go through many versions so that searching by title is not reliable. External search engines also cannot identify documents within the IFAD site.

3.5 The anonymity of documents works against easy access both electronically and conventionally. It is argued elsewhere that human resources development would benefit from the naming of authors but it would also be helpful for access to information. It is common in many organisations to refer to reports by the author and searching for, say, project evaluations, can be frustrating even when they are known to exist. How much more frustrating might it be for visitors who do not know at the outset what is likely to be available?

3.6 The web page devoted to explaining IFAD structures and functions to consultants is excellent and a good use of a website. Its targets are almost universally able to access the information and will have similar needs for information. The electronic presentation should reduce the amount of staff time spent on briefings on administrative functions.

Information Resource Centre 3.7 The document service of the IRC has been a fantastic resource to the IEE and again we represent a good test case of the usefulness of the functions of the Centre. The changes in format and use of different electronic storage mechanisms make it difficult for an outsider to get quick access to older documents. The staff remain vital to the usefulness of the Centre and their workloads make them unable to support all requests for help.

3.8 The Topic of the Month is an interesting product that should be an additional tool for thematic groups or information “evangelists”. That is an enthusiast with a message might be able to post important ideas that could lead to important learning. In fact, the tool seems very little used and the IEE only found out about it by accident despite many interviews on tools that people find helpful in their work.

OE products 3.9 The Office of Evaluation has a particular and specific role in the collection and management of information and learning. In the paper on strengthening the evaluation function this is acknowledged very specifically.9 Paragraph 53 states that “OE must generate

9 REPL./VI/4/R.4 26 September 2002, IFAD, Consultation on the Sixth Replenishment of IFAD’s Resources – Fourth Session, Rome, 9-10 October 2002, STRENGTHENING THE EFFECTIVENESS OF THE EVALUATION FUNCTION AT IFAD, A DISCUSSION PAPER ON IFAD IN THE LIGHT OF INTERNATIONAL EXPERIENCE


Annex 6 Page 10

a great deal of knowledge”. One of the key reasons for this is the lamentable quality of monitoring and evaluation in country projects (para 52) and the weaknesses in making use of the knowledge that is available (para 56).

EKSYST 3.10 The Evaluation Knowledge System provides a mode l approach to finding IFAD documents in that it is organised along the lines that might interest a visitor. Although a visitor would not understand the difference between the different products (e.g. an evaluation profile or an evaluation insight?) they can search by region or by country or by theme from drop down menus.

3.11 The products include evaluations at different levels (Corporate, Country and Project) and Thematic focus. The Profiles and Insights and Agreement at Completion Point provide short versions of main findings that would guide a reader to decide if they wanted more detailed information.

3.12 The products have changed over time and Lessons Learned based on regional working appear to have disappeared. Only the Insights and the Profiles have explicit stated purposes and target readerships. No work appears to have been done to assess the impact or satisfaction levels of the target constituencies.

3.13 The general observation that IFAD does not evaluate its communications work assumes a slightly sharper irony when applied to the products of the Office of Evaluation. Again a wide range of products, some of which appear to be excellent, are broadcast without attempts to collect feedback from target readers or indeed to check on the readership.

3.14 An analysis of potential communications targets and approaches has been carried out and a great deal of work has been put into assessments of how different evaluation products should be produced to reach the different constituencies.

Monitoring and Evaluation at Project Level 3.15 The weaknesses of monitoring and evaluation is a continuous refrain in IFAD documents (see Box 1) and the lament can be found in corporate statements (as in the previous paragraph and most progress reports and the ARRI, see for example para 112 of the ARRI of 2002 in which M&E is described kindly as “generally weak”) and in regional statements and in specific project reports.

Box 1 Specification of M&E Design

The following text is taken from an IFAD document dating from the late 1980s. From inspection of the 42 projects sampled for the IEE, it holds true today.

The specification of the M&E component of projects in preparation and appraisal reports has been uneven in quality and inconsistent in content… The institutional position of M&E activities has frequently been left vague, without clarifying the various responsibilities towards the project, the government, the co-operating agency and (the funding agency). Activities (planning, baseline studies, evaluation surveys, provision for special studies of problems, participatory evaluation, mid-term and completion evaluation) were included or omitted without clear justification … Methodology is sometime elaborated in detail in working papers or annexes … while in other cases it is only passingly mentioned. Technical assistance in establishing M& is sometime provided for and sometimes not. Costs are rarely identified in detail and provisions have sometimes been omitted … The variations between projects seem to be largely due ton the different priorities and skills of the responsible

10

10 Reported in Coleman, Gilroy (1990)


Annex 6 Page 11

mission members rather than adaptations to different projects or country needs.10

3.16 The IEE team assessments of 42 sample projects reinforces the image of very poor collection and analysis of monitoring information. Of the 42 examined only 6 are scored above the mid-range, none is given a top score and 24 are weak or very weak. The specific use of the logical framework appears even worse. The definition of indicators would be a major component of good monitoring but 25 projects are scored as weak or very weak in definition of indicators and only 2 projects are assessed as being above the mid-point (i.e. 4 on a 5-point scale). The generally weak nature of monitoring will be an important feature of the country level visits. If the weaknesses are confirmed it will be a major blow to an organisation that claims to learn and use learning from its field operations.

3.17 Weaknesses at project level are compounded by inconsistent supervision reporting and variable content and quality of completion reports. The move to make OE independent of management from April 2003 is a step towards remedying this situation, but the culture of weak M&E has become pervasive over the years. The net result is that the major potential source of factual information to underpin learning and policy influence is ineffective.

3.18 Staff in IFAD are highly aware of weaknesses in M&E and a number of initiatives to improve the situation have been followed over the last few years. The New Approach to Evaluation promoted by the OE should improve the quality of evaluations, which may feedback into improved monitoring. The Guide for Project M&E is a thorough and comprehensive approach to M&E. It will need to be introduced with considerable support to project management teams with training and feedback made available. Most development aid agencies struggle with M&E and it is common to find initiatives that focus on fewer and fewer key observations in order to improve the probability of getting at least the core observations of the minimum necessary information. It seems clear that greater efforts will continue to be required by IFAD to improve M&E.

Core Learning Partnerships

3.19 The Core Learning Partnerships (CLP) are an important initiative that brings together the different actors who might reasonably be considered to be stakeholders in the findings of a project evaluation. As such this should be a useful technique and a major improvement on the approach of evaluators reporting simply to management. The simple evaluator - manager reporting method works poorly when it becomes adversarial, which is common, and the CLP arrangement should make it harder for different sides to an evaluation to emerge and engage in unhelpful antagonism or opposition.

Agreement at Completion Point 3.20 The ACP is a key tool for the CLP and should allow the effective use of learning from evaluations and by linking it to the management response it should make it less likely that evaluation reports could be accepted and then ignored. The ACP in fact only works well when there is room for non-agreement as well as agreement and where the essential distinction between the evaluation findings and the management responses is maintained and if necessary reinforced. It is possible for the distinction to become blurred so that the ACP in fact presents a bland version of the evaluation findings and therefore requires only an anodyne response from management.

3.21 Thus whilst the ACP has the potential of obliging a confrontation of findings and responses it can be manipulated to render evaluations less useful.

3.22 One example will suffice to make this point. An interesting case may be the ACP to the evaluation of IFAD capacities for promoting innovation. One key finding of the evaluation


Annex 6 Page 12

team was that there was no common understanding of innovation and of its processes (paragraph 12). This is an important observation and a management response would be to work on common understanding and develop methods of rewarding the presentation of innovation. The ACP presents the findings as “IFAD staff has diverse understandings of innovation and its process.” (paragraph 4) which is an accurate representation of a different point in the evaluation report and misses the key point. The ACP then goes on to recommend a new definition of innovation and presents it as a decision of senior management.

“Innovation for IFAD is the development of improved and cost effective ways to address problems/opportunities faced by the rural poor through the projects and programmes it supports. These encompass institutional and technological approaches, as well as pro-poor policies and partnerships. IFAD directly supports innovation and, together with its partners, facilitates its processes and promotes its replication and scaling up.” (Recommendations, page 3 of the ACP).

3.23 The overall effect of this process is that a key finding is misrepresented and a recommendation to management is ignored. The proof of the weakness of this response would be obtained if IFAD staff were still using a wide range of different understandings of innovation and if they were unable to refer to this hastily formulated definition.

3.24 The IEE has come across similar but less extreme cases of modification of language between the evaluation report and the ACP and it raises the question of how evaluations should be shared within IFAD and made public. It is possible to make the case that evaluations should remain in-house whilst the culture of taking criticism is strengthened and only important lessons are broadcast. The knowledge that the ACP becomes public may tempt staff to soften the language before the lessons have been assimilated by the institution. The converse situation that the document would remain internal may allow more direct and harsh lessons to be learned although it appears to be denying the principle of open access to IFAD documentation.

Inter - Department Instruments

Thematic Groups

3.25 Thematic groups were formally started under the Knowledge Management initiative and a number of groups were formed in 1999. A number of these were well regarded and served a purpose which was time-bound and then ceased to meet. The Gender group had considerable respect for promoting the gender policy which is still seen as a good policy product. The rural finance group continues to function and the leader has broad intellectual respect from colleagues and the group concerns issues of importance to most programmes. The livestock group also continues to function and had produced a CD-ROM of information and useful sources. Again the convenor is respected especially for their intense work and commitment and the products whilst serving a narrow band of interest are considered worthwhile.

3.26 It may be that the thematic groups tended to replicate existing informal groupings when they were first started and a slight sense of artificiality lead to negative feelings about the groups even though the declared purposes were seen as very important.

Project Development Teams

3.27 PDTs are seen as a key method of sharing professional expertise. The best PDTs are clearly very good examples of interdivisional sharing. However, there is some disquiet among staff that the PDTs depend a great deal on the personality of the CPM who is convening the


Annex 6 Page 13

team. The CPM can determine the frequency of meetings, the duration of meetings, the intensity of discussion and to some extent the agenda of each meeting. IFAD staff repeat the common observation that those who might most benefit from external advice are the least likely to seek it.

TRCs/OSCs

3.28 The formal meetings to record the findings, concerns and recommendations of groups examining and supporting project propositions can commonly be much less rich and interesting than the discussions and exchanges in the meetings. The usefulness of the TRCs seems to be only in the record of the discussions and these are not referred back to in the supervision reports and summary reports. The OSCs seem to be less important in terms of project management guidance but are treated as a formality that needs to be observed. The decline of the OSC is a key issue dealt with in the quality assurance section of the project review.

The Management of TAGs

3.29 The management of TAGs can be used to promote information sharing between CPMs where TAGs with a regional remit are made the responsibility of particular CPMs they are obliged to collaborate with their colleagues in order to make a success of the TAG.

Report and Recommendation of the President

3.30 The RRP is the visible result of project planning in IFAD that serves to release funding for project initiatives. The authors are obliged since 1997 to mention learning from earlier project work and they do this with some regularity. The IEE study of 42 projects, assessed11 only 6 projects to have been weak in this area and 18 were considered to be good. However, the IEE team also assessed project proposals for their assessment of learning from other situations and other agencies and here the results are less positive. Only 6 project designs were considered good against this criterion and 15 were weak. It may be that the formal requirement to mention learning was observed whilst the purpose of the requirement, that the formulation benefit from other experience, was not being genuinely fulfilled. A review of the observations that constitute learning from other IFAD projects supports this interpretation.

3.31 The formal requirement to consider learning is probably necessary and it will be interesting during the country visits to see how much the learning identified at design was available and useful to the project management during implementation.

COSOP 3.32 The COSOP also requires the authors to identify learning from IFAD work at regional level. This effectively means learning from other projects in different countries. Again the IEE assessments show a strong conformity to the requirement and only one COSOP was found to be weak in this area and fifteen of the twenty that were studied were good. The formal focus on examining learning is a necessary step in valuing learning but it is not clear at this stage if the completion of requirements for successful documents in the Rome also provides some genuine exploration and exploitation of learning in the management of projects.

11 The team studied RRPs but the overall assessment is based on an assessment of a range of appraisal and preparatory documents.


Annex 6 Page 14

Mailing

3.33 The newly formed Communications Division manages large mailing lists and sends large numbers of documents to readers all over the world. No follow up has been done so far on the degree of satisfaction of the readerships. The products include IFAD documents and other sources.

Video, CD- ROM, Audio

3.34 IFAD produces a curious range of CD-ROMs including the findings of the Livestock and Rangeland thematic group and an annual compilation of governing body documents. There has been no assessment of the readership or perceived usefulness of these products. Videos are relatively new mechanisms and a sound studio is being developed to assist the production of audio cassettes that might be used by local radio stations. All these products are likely to have a value but at present the IEE has not been able to trace a case history from project level need to product creation and impact or usefulness.

Working on the basis of personal connections. 3.35 In the report on Supervision Modalities in November 200312 an important observation is made on learning methods. “The main learning linkage continues to be an informal and loose process. No formal mechanism was put in place to capture learning from cooperating institutions and IFAD supervision experiences” (39, 4). Recommendation 6 (page 12) is to strengthen the learning loop but the document warns that “much more needs to be done”. The paper is the first that we read that acknowledges that this will be difficult “given staff workload” (page 8).

3.36 These points echo what the IEE has heard directly from staff. “… each of us has a personal set of links.” said one long-serving member of staff and this was not only said to us but demonstrated in terms of recommendations for people to talk to but also in problem solving during our investigations. Finding documents that are not available through the portal (and for some that are) was often achieved by following a chain of contacts to someone who could lay their hands on a copy.

The pressure of time is a huge burden and a major impediment to effective learning. Everyone is too busy to give time to non-urgent work. This was the most common complaint heard during the desk study and was an element in exercises during group meetings. Many members of staff felt unable to come to meetings that the IEE called even in case where they were personally invited.

Summary of issues

• There seems to be no shortage of instruments and skills available for good knowledge management. Weaknesses appear to be in planning and monitoring of the use of the instruments.

• Will the new strategy from the Communications Division provide a unifying force linking the different activities with IFAD purposes?

• Where will support come from to develop assessments of needs and monitoring of effectiveness of the different instruments?

12 IFAD (2003) Evaluation of Supervision Modalities in IFAD Supported Projects. Office of Evaluation. Rome


Annex 6 Page 15

4 A model of knowledge management

4.1 In our view, knowledge sharing is best understood as a number of discrete steps that can apply to a range of sources of learning and a range of targets for learning. This model is built up from the component tasks that IFAD sets itself (Figure 1). The IEE team has assembled it in the absence of a model for knowledge sharing in the IFAD documentation that we have studied. The beginning and end of the model process are simply the input and output of learning. The central steps are those described in the website “How is “knowledge management” implemented”.13 The model may be useful if it allows a separate assessment of different elements of knowledge management since it is likely, as in any organisation, that there are different strengths and weaknesses in different stages of the process.

Figure 1 Our understanding of knowledge sharing – a model

IFAD projects

1

Other sources

2

Identify

3

Store

4

Disseminate

5

IFAD projects

6

Other targets

7

4.2 Three examples of increasing complexity may make the diagram clearer.

• One CPM tells another of an event on a recent project visit. In the simplest case of knowledge sharing a project manager will tell another of something they have learned and the process is the passage of learning direct from 1 to 6 with the most minimal treatment in 3, 4 and 5.

• Project evaluation ACP is posted on the website. A project evaluation brings together different actors in 1 and 3 and then the key lessons are developed in the ACP process. Professional staff work on the design of the document and post it on the site after it has been through the appropriate approval processes. The document is mentioned to colleagues who access it (6) or to interested actors outside IFAD (7).

• The Livestock Thematic Group produces its CD-ROM. In a more complex case a range of project experiences may be collected in 1 and a detailed scanning of the environment in 2 before a signif icant effort deployed by professional staff in 3, 4 and 5 until a CR-ROM of lessons learned and important other sources is developed and distributed to staff (6) and other practitioners (7).

4.3 The IEE examination during the desk study of the different steps in the knowledge management model leads us to the comments set out in Table 1. The scoring is intended as a spur and guide to discussion and is not a definitive view of the IEE.

13 http://www.ifad.org/governance/ifad/km.htm


Annex 6 Page 16

Table 1 Key steps in the model

Stage Activities and comment

Preliminary assessment 1 = very weak 5 = very strong

1. Collection of information from IFAD project level work.

Monitoring and evaluation designed in project formulation. New approach to evaluation and new M&E Guide may lead to improvements if management support is available.

Project based M&E teams carry out work and supervision missions collect findings.

1 - Very poor. Weak monitoring must be scored harshly because it is so important.

2. Scanning for likely interesting learning outside IFAD.

IRC staff functions Thematic groups Individuals explore their professional domains and connect to personal networks.

Technical department has formal links to other agencies. Research with other agencies via TAG funded initiatives.

2 – Good work of limited scope, not perceived as a high priority and therefore massive time constraints on all staff.

3. Identification of important or highly relevant materials

IRC staff, professional staff assess and promote according to their judgement.

Defensive culture and boundaries may prevent learning reaching central processes.

2 – Staff aware of the need to scan for important learning but massive time constraints prevent investment in this work.

4. Storage and retrieval

IRC staff, Communications Division staff, G/S staff input data to PPMS LGS etc. Much effort devoted to this but largely seen as of low importance to management

Some weaknesses include: inappropriate software in many versions and late input and short electronic history.

3.- Once key information has been identified and collected. Staff manage well despite a dispersion of activities and resources. Time pressures again are important.

5. Dissemination

IRC staff, Communications Division staff

Office of the Secretary for translations and formal documents.

4 – A wide range of skills available and many methods are used. Assessments of need and monitoring of results would lead to a top score.

6. Access by IFAD staff

Intranet portal and personal networks. The limits of personal networks and a defensive culture prevents sharing of some lessons.

IRC services and personal knowledge of IRC staff. Webdocs system has reduced some access but staff may find a way around.

4 - Staff can find what has been collected either directly or via contacts. 2 - IFAD project managers may have more difficulties depending on electronic access.

7. Access by other actors:

Community level

Other projects Other local

donors

Host governments

To be assessed in country visits

Southern public Access open web-site


Annex 6 Page 17

Stage Activities and comment

Preliminary assessment 1 = very weak 5 = very strong

Northern public Access open web-site, mailings, …

Monitoring of reach or impact would improve scores.

3 - Easy access for computer capable. However, not easy to know what is available or to search for governing body documents by topic.

Other targets (to be identified)

Summary of issues

§§ Is monitoring and learning from project level work as bad as it is reported to be? If so, what does this mean for IFAD’s claim to be a learning organisation and promoting change on the basis of its own experience?

§§ Do boundaries and defensiveness prevent open sharing of learning from failure and weakness? Do these cultural forces also prevent sharing of good examples?

§§ Is the burden of overwork damaging the sharing of those bits of learning that is interesting but not urgently needed by the control functions of the organisation?

All these elements will be open to continuous examination in the following phases of the IEE.

5 Learning Culture

What works well in IFAD? 5.1 Information and learning are shared within IFAD and the most effective learning on professional issues occurs through personal contacts both formal as arranged in groupings like the Project Development Teams and informally according to the personal networks of individual staff members.

5.2 It is also clear that information required by the Executive Board and Governing Council is made available in style format and content that is demanded. These demands override any other work priorities.

5.3 IFAD is good at producing reports and a wide range of information products. Interna l reports are produced for a range of management purposes and it seems that the control functions and nominal supervision roles are fulfilled correctly.

Aspects of culture that work against good KM 5.4 The very effectiveness of the control functions within IFAD work against the effectiveness of knowledge sharing. This need not be the case and it is possible to imagine changes that would lead to improvements in knowledge sharing without reducing the effectiveness of control functions.

5.5 IFAD appears to suffer from a large number of important boundaries that reduce the effectiveness of communications and learning. These boundaries are referred to in the report on innovation14 and also discussed in this report in the section on Corporate Culture. There

14 IFAD’s Capacity as a Promoter of Replicable Innovations in Cooperation with other Partners, Understanding at Completion Point and Executive Summary, November 2002, Report No. 1325


Annex 6 Page 18

are important vertical boundaries that make it more difficult to communicate between divisions and departments. These boundaries are overcome by some of the measures and practices described above but remain important.

5.6 There are also very important horizontal boundaries that separate senior management from professional staff and professional staff from general service staff. Although the organisational structure is relatively flat the awareness of hierarchy is intense. The IEE has had a number of experiences where it was unable to carry out its own work because of an inadequate understanding of the influence of hierarchy. The suggestion that IEE members would meet senior staff before speaking with their subordinates betrays a degree of sensitivity that is not conducive to easy exchanges of information and ideas.

5.7 Boundaries can be broken down and their influence can be reduced. This normally requires demonstration and leadership from the highest levels of the organisation.

5.8 The greatest achievement in knowledge management that an organisation should aspire to, may be the celebration of learning from failure. It is hard to imagine this happening in the current institutional culture of IFAD. The IEE has experience of defensive behaviour in interviews around projects that are considered to have been weak. The value of sharing learning from failure is recognised in the policy statement on disclosure of documents but not to our knowledge, elsewhere. 15

12. Sharing information about what worked and what failed will raise the quality of development assistance as a whole. In defining a knowledge-management strategy, IFAD will support the global effort to achieve sustainable development. In addition, the potential will be greater for cross-fertilization and the replication of successful projects or components. With better access to information, moreover, development partners and beneficiaries will be able to draw on examples of other projects in order to reinforce ownership and sustainability.

5.9 A very radical change in culture would be required to achieve genuine acceptance of learning from failure; new heroes of failure would need to be honoured and new corporate narratives (mythologies) would need to be promoted by senior management before such changes can start. The strong boundaries and differentiation that can be held in place by senior management can be dismantled by the same.

What other agencies have done to promote KM

5.10 It is probably not realistic to expect UN agencies to adopt behaviours that are reported from leaders in innovation and learning like 3M or Apple. Apple’s use of learning “evangelists” may not be easily accepted in multilateral donor agencies. Nevertheless IFAD could possibly be fairly compared with similar donor agencies and might learn from their experiences.

5.11 A recent evaluation of Knowledge Management by the World Bank included a comparative review of knowledge management at other development agencies. Table 2 summarises the findings, which provide a useful benchmark for IFAD.

15 EB 2000/69/R.13/Rev.1 4 May 2000, IFAD, Executive Board – Sixty-Ninth Session, Rome, 3-4 May 2000, IFAD POLICY ON THE DISCLOSURE OF DOCUMENTS. Paragraph 12, emphasis added.


Annex 6 Page 19

Table 2 Summary of Knowledge Sharing in Four Agencies16

Topic DFID JICA Sida EBRD

Preferred terminology

Knowledge sharing Knowledge management No terms Information management and knowledge sharing

Knowledge sharing & agency strategy

Emphasis on influencing rather than knowledge sharing

Implicit emphasis on support to the creation of local rather than global knowledge. Established a knowledge management programme for internal knowledge sharing in 2001 Swedish Agency for Research Cooperation (SAREC) has an explicit knowledge policy relating to Southern universities

“Knowledge is our most important resource. During the next five years we shall implement an investment programme for the long-term renewal of knowledge and skills in our partner countries, of our Swedish partners and at Sida.” In practice the language of knowledge is far less evident than those of “learning” and “capacity development”.

No clear knowledge vision within its mission statement or core documents.

Intended outcomes and results of knowledge sharing programmes

Lead to practical improvements in the efficiency and effectiveness of staff and the organisation as a whole.

Primary objective of ensuring better knowledge sharing within the organisation

Improvements in knowledge are central to the challenge of working as a development cooperation agency; the emphasis is on shared knowledge generation (learning) as the key activity.

Designed to provide the necessary information to allow staff to “work smarter”;… through the acquisition of external information and its presentation in a timely fashion to professional staff.

Organisational structures and resources

Since 2001 has a clear structure for internal knowledge management: a small unit with its own manager; a departmental head over a Knowledge and Information Services (KIS) Team; and a Deputy Director-General for Knowledge Sharing and Special initiatives. External learning less explicit, but a de facto role by the Knowledge Policy Unit; plus work of research managers in several Advisory

No unit explicitly responsible, but the Unit for Organisational Learning does similar activities. The Methods Development Unit arguably has a key role in external knowledge sharing. SAREC has a specific role noted above.

Initiatives coordinated by Planning and Evaluation Department; Staff are appointed to be core members of thematic groups.

Two units: Information technology Department, with responsibilities more than just infrastructural. The Bank Information Centre (BIC) is more networked into external knowledge sharing, this mandate appears not to be universally accepted.

16 Abstracted from King, Kenneth and Simon McGrath (2003) Knowledge Sharing in Development Agencies: Lessons from Four Cases. World Bank Operations Evaluation Department, Washington


Annex 6 Page 20


groups Staff incentives for involvement in knowledge sharing

Proposal to include a knowledge sharing component in staff appraisal; no other specific incentive

Plan for core members of thematic groups to have knowledge sharing as part of staff appraisal; no other specific incentive

No specific incentive No specific incentive

Internally focused knowledge sharing activities

Approximately 70 informal communities of practice Intranet identified as most important element of knowledge sharing (Insight), plus work to improve effectiveness of digital working Project for electronic documentation Provision of informal office space for knowledge sharing Explicit encouragement of departmental and cross-departmental seminars Use of ICT to increase knowledge sharing between country offices and headquarters (plus dedicated satellite space)

Formal organisation of communities of practice with plans for 5 networks and 20 teams (2002). Staff allocated formally. Intranet Existence of unplanned office space for knowledge sharing Use of ICT to increase knowledge sharing between country offices and headquarters

Little formal promotion of communities of practice Intranet, but little explicitly to develop Project for electronic documentation Provis ion of informal office space for knowledge sharing Use of ICT to increase knowledge sharing between country offices and headquarters

Intranet Existence of unplanned office space for knowledge sharing BIC acts as a single core help desk for staff Use of ICT to increase knowledge sharing between country offices and headquarters

Externally-focused knowledge sharing activities

Emphasis on influencing so much internal discourse on developing staff skills and knowledge to promote DFID’s policies and practices Uses research programmes to support national research In favour of untying TA Support for global public goods – e.g. CGIAR Support to knowledge networks and websites: Nature website; GDNet; id21; Livelihoods connect e-learning initiatives: Imfundo project on teacher training in Africa

Opposed to ‘influence’ and stresses mutual knowledge construction Strong promoter of TA, but focused at middle level practitioner posts, emphasising practitioner knowledge and people-to-people transfer of skills and technology High emphasis on provision of training in Japan CGIAR and non-CGIAR support e-learning initiatives: J-Net (not linked to World Bank Global Distance Learning Network) Strong South-South and regional cooperation support

Reluctant to ‘interfere’ in national policy-making Promotion of mutual knowledge creation over knowledge transfer CGIAR support via SAREC, plus long term support to research institutions Twinning


Annex 6 Page 21


Distance learning: British Council links to World Bank Global Distance Learning Network

Processes for determining needs/ demand

Little evidence of actions being based on needs or demand


Little evidence of needs but provision for M&E to give feedback on needs and demand by Unit for Organisational Learning


M&E indicators and evaluation findings

‘Snap shot’ study in 2002 reported on internal knowledge sharing, but based on a two-thirds response to a sample of 35 out of 2400 staff.

No arrangements yet. Annual Human Resources Report includes departmental learning activities, and notes problem of mapping and measuring informal learning. Since 2000 also an annual staff survey on organisational effectiveness.

BIC keeps all records of requests for information; annual staff survey reports satisfaction with the service – 40% response to a survey of 66% of staff.

Partner feedback None None None None


Annex 6 Page 22

5.12 From what the IEE team currently understand of IFAD, it would attract the following comments based on the same categories used in the King and McGrath study (table 3).

Table 3 Knowledge sharing characteristics in IFAD

Topic IFAD

Preferred terminology Knowledge Management was current until 2002 but has not been used greatly since and has not been replaced. Knowledge Sharing seems the most acceptable usage among staff.

Knowledge sharing & agency strategy

A major element of strategic statements at all levels and in aims of operational projects.

Intended outcomes and results of knowledge sharing programmes

Knowledge is described as important for: improving project management, influencing other project level work, influencing policy, empowering beneficiaries and establishing IFAD as a prestigious source of information.

Organisational structures and resources

The Information and Communications Division was disbanded in 1994. Information Resources Centre has key storage and retrieval functions and limited dissemination roles. A Knowledge Management Facilitation Unit ran for several years but was allowed to lapse. Communications Division was created in 2002 and may take on lead role across the corporation.

Staff incentives for involvement in knowledge sharing

No specific incentives (although the need was acknowledged).17

Internally focused knowledge sharing activities

Intranet of considerable importance. Some internal groups are established as Thematic Groups, are formally acknowledged and obtain funding.

Externally-focused knowledge sharing activities

Influencing policy in host countries, influencing other donors and acting as a “catalyst” are all declared as strategic purposes. Technical knowledge in shared research programmes. Public access to documents via website and policy on disclosure. Documents mailed to individuals and institutions.

Processes for determining needs/ demand

Many unfocussed calls for improvements at corporate level. COSOPs and RRPs do not normally identify information and policy targets.

M&E indicators and evaluation findings

No routine monitoring of usefulness. No feedback on publications or analysis of readership. Website hits can be assessed. Anecdotal evidence of impact from staff.

Partner feedback Nothing observed in Desk Study. A key element for assessment in next phases of IEE.

5.13 Generally IFAD scores well for taking the issues of knowledge management seriously and devoting attention to the questions. It also manages well by virtue of being relatively small and with many long-serving staff it is possible to build personal networks relatively easily.

5.14 One clear observation that emerges from the King and McGrath study is the usefulness of common spaces for people to meet in without any consideration of territoriality like the cafeteria and restaurant. It appears to be good practice to provide staff with spaces which do not belong to any group or section that can be used for informal meetings. The value of these spaces is further enhanced by senior management demonstrating their own use of the opportunities for informal chats with staff. 17 REPL.VI/3/R.6 19 June 2002 Agenda Item 5 IFAD Consultation on the Sixth Replenishment of IFAD’s Resources – Third Session Rome, 2-3 July 2002 ENHANCING IFAD’S FIELD PRESENCE Para 17.


Annex 6 Page 23

5.15 Knowledge management mostly concerns people and how they behave. Information systems are important, documents must be written and filed, electronic communication methods must be obtain and used and all the techniques of communication should be explored and exploited. These technical and procedural measures are important but at bottom the success of knowledge management depends on people and IFAD’s ability to get the best out of staff and partners in terms of their approach to sharing information and knowledge.

Summary of issues

Internal Issues

• What can be done to raise the profile of knowledge management without risking the backlash associated with old initiatives and initiative fatigue?

• What can be done in a small multilateral agency to raise the importance of personal relations in knowledge management without damaging the hierarchical control functions; how to reduce boundaries whilst protecting the dignity of post holders?

External Issue

• How to improve the sensitivity of IFAD knowledge management functions to the needs and views of outside constituencies?

6 Issues in knowledge management

Further work required

6.1 To check assertions on knowledge management in this report. To review new knowledge management strategy documents. The IEE will need to follow the development of the strategy of the new Communications Division over the coming months. Some greater understanding of information flows within the headquarters offices will be valuable.

Issues for country visits

6.2 To assess project staff learning from other projects, especially that mentioned in planning documents. The country visits will provide an opportunity to assess the use made of learning that was identified in project design. All projects mention some useful learning from other IFAD projects and it will be instructive to see how this insistence on a formal mention of learning translates into a learning approach in project management.

6.3 To assess strengths and weaknesses in routine project monitoring and assess implications of findings. The weaknesses in routine monitoring and evaluation in country programmes needs to be better understood. The country visits will perhaps identify examples of good practice and effective monitoring that would be helpful in promoting better practice.

6.4 To assess strengths and weaknesses in country level influence and advocacy. The country visits will also identify local examples of influence and effectiveness of the use of information and learning from IFAD programme work. The IEE will be looking for examples among other development project management staff and among donor offices.

6.5 To assess work done to improve access to information of project beneficiaries. The use of information by beneficiaries will also be an important element of the interviews to be carried out during the country visits. This is one of the key contributions of the IEE which is


Annex 6 Page 24

unlike earlier evaluations. The views of beneficiaries on a range of issues will be collected including the questions of information needs and learning mechanisms.

Information from other donors 6.6 To assess value to staff of other development agencies of IFAD as a source of information and learning. The value and importance of information received from IFAD by other donors will be sought in international settings and in other national situations.

The role of consultants 6.7 To assess roles played by consultants in knowledge management in IFAD. To assess consultants’ appreciations of IFAD as a source of learning. The IEE will follow up discussions with a wide range of consultants who have worked for IFAD over the years and assess their roles in knowledge management for and by IFAD. We will also benefit from their understanding and experience of obtaining information from IFAD and presenting information and learning to the agency.

Independent External Evaluation of IFAD Desk Review Report Annex 7 – Organizational Partnerships

Annex 7 Page 1

Annex 7

Organizational Partnerships

1 Introduction

1.1 Working in partnership is one means by which IFAD can contribute to achieving its strategic objectives. By partnering with others, IFAD should be more effective in delivering its strategic framework, ensuring programme impact and fulfilling its catalytic role.1 Ultimately, working with the right partners, in the best possible way, should allow IFAD to better “enable the rural poor to overcome their poverty” (IFAD 2002b: 9) – and more so than could otherwise be achieved apart. Over recent years, IFAD has devoted some effort to explaining and publicising the importance of partnerships in its operations and impact, with whole chapters in the Annual Report (2000a: Chapter 4) and the Rural Poverty Report (2001a: Chapter 6) devoted to partnerships, coalitions and alliances.2

1.2 This paper attempts to understand the extent to which IFAD is entering into relevant partnerships to meet its development goals, and whether these partnerships are being effective, and managed efficiently (see: IEE of IFAD Inception Report, Table 7). To begin with the paper provides an overview of the literature on this subject, in order to demonstrate the ‘added value’ of working in partnership (over and above other types of relationships). The rest of the paper is then based on evidence from a range of semi-structured interviews with IFAD staff, alongside a review of the documentation.

1.3 At this stage of the Interdependent External Evaluation (IEE), the primary purpose of this paper is to provide a ‘think piece’ on key issues - some of which will be followed up in subsequent work during the country visits. The main issues are presented here using the structure of questions outlined in the Inception Report (Table 7). The first set of issues are concerned with the plethora of relationships that IFAD currently terms as ‘partnerships’. This leads into a more in-depth consideration of why it is that IFAD should enter into the partnerships it does (relevance), followed by an assessment of the unity that exists between the partners. Subsequent sections go on to consider the effectiveness and impact of the partnerships, and then the efficiency of IFAD’s management and coordination.

1.4 The paper concludes that, while it is difficult (and somewhat problematic) to draw general findings across the sheer multitude and diversity of relationships, there are a number of important issues that need to be considered. Firstly, the widespread application of the ‘partnership tag’ to IFAD relationships, and the relative all-encompassing objectives set out in the 5th and 6th Replenishments, may tend undervalue the real benefits that can be gained from truly working in partnership. Given the limited resources of an organisation like IFAD, it seem to be appropriate to be more strategic about those relationships that offer the best potential and benefits for achieving IFAD’s objectives. Secondly, IFAD’s Strategic Framework is defined in such broad terms, that it is difficult to argue against the relevance of a great many partnerships, even though they may not present the best opportunities for contributing to IFAD’s programme operations and impact. In the past IFAD has entered into (and become ‘host’ to) several significant partnerships for which questions are now being

1 As stated in the 6th Replenishment document (IFAD 2002c: paragraph 72). 2 Also extensive reference is made to partnerships on the IFAD intranet (February 2004), covering: i) multilateral and bilateral organisations (including International Financing Institutions, UN agencies, the International Land Coalition, the Global Mechanism, and the Belgium Survival Fund); ii) partnerships within the borrower countries; iii) partnerships with the poor; and, iv) partnerships with NGOs and the corporate sector. In addition, there have been numerous publications concerned with partnerships including: “IFAD and NGOs: Dynamic partnerships to fight rural poverty” (IFAD 2002a); “ IFAD, OPEC and OIC Member States: A partnership to eradicate poverty” (IFAD 2003b); “IFAD and Italy: A partnership to eradicate poverty” (IFAD 2003a); “ IFAD and the private sector” (IFAD 200X).


Annex 7 Page 2

raised about the real benefits to IFAD corporately and operationally. 3 Thirdly, while agreements exist between IFAD and many of its so-called ‘partners’, these statements are not necessarily very clear on the shared objectives of the relationship. Furthermore, there is a limited, institution-wide understanding of the benefits of several partnerships (GM; GEF; the Coalition). And finally, there is generally limited measurement of achievement against shared objectives, and in most cases the evidence of effectiveness and impact of the partnerships is anecdotal.

Defining partnerships 1.5 The added value of ‘partnerships’ can be a somewhat difficult concept to grasp. It is a term that often applied to a range of inter-organisational relationships and collaborations. Partnerships can be distinguished from both market-based relations determined by contract, and relationships in the public sector involving hierarchies based on authority. In a review of the literature, Watkins and Csaky (2003: 13) find two common themes emerging. The first is a sense of mutuality and equality between those involved, including a sense of trust between the partners working together. The second is a mutual commitment to agreed objectives, reciprocal obligations and accountability, whether informally agreed or laid down in a formal contract. These two themes can be thought of in terms of unity between individuals and organisations, and the direction (or overarching goals) of the partnership shared among its members.

1.6 For IFAD, the partnerships, and the objectives of working in partnership, are set out in the 5th and 6th Replenishments (see Table 1). The 5th and 6th Replenishments differ in the way they approach partnerships. The 5th Replenishment mainly considers partnerships in the context of using country and regional strategies as an entry point to consolidating existing partnerships and constructing new ones. It states a number of partnership objectives, such as sharing institutional capacity and underpinning macroeconomic reform programmes. The 6th Replenishment provides more of an overview of the areas in which IFAD works in partnership (e.g. multi-stakeholder coordination through UNDAF). The 6th Replenishment is more implicit about the objectives of ‘working in partnership’, and there is considerable overlap with the 5th Replenishment on issues of co-financing and illustrating a catalytic role. Furthermore, it is not made clear whether the 6th Replenishment merely replaces the 5th Replenishment, or whether it adds to the 5th, and if so in what ways?

Table 1. Summary of partnership objectives in the 5th and 6th Replenishments

Key objectives 1. Replicate and scale up through co-financing 2. Complement or underpin macroeconomic reform programmes 3. Share institutional capacity and comparative advantages 5th

R

eple

nish

men

t

4. Exchange knowledge on policy and practices 5. Increase multi-stakeholder coordination 6. Effective in creation of sustainable, productive private sector investment in rural areas 7. Leveraging resources through co-financing 6th

R

eple

nis

hmen

t

8. Playing a catalytic role beyond the country level

Methodology

1.7 As previously stated, the primary purpose of this paper is to provide an initial assessment of IFAD’s partnerships and identify key issues, some of which will be followed up in the subsequent stages of the IEE (including the country visits). The main hypothesis is

3 IFAD hosts the secretariat of the International Land Coalition (as well as being a founding member, and acting in a governance capacity at the Assembly of Members and the Coalition Council). IFAD also houses the Global Mechanism secretariat, as well as being a member of its Facilitation Committee (which it chairs in rotation with the World Bank and UNDP). Furthermore, IFAD is an Executing Agency of the Global Environment Facility, and has recently obtained responsibility for managing GEF projects and funding for global issues concerned with land degradation (see IFAD 2004b).


Annex 7 Page 3

that, “the strategy of working in partnership with others has enabled IFAD to be more effective in delivering its strategic framework and ensuring programme impact and fulfilling its catalytic role” (Inception Report, Table 7). This hypothesis is based on paragraph 72 of the 6th Replenishment. Indeed, the 5th and 6th Replenishment documents provide the basis for assessing the relevance, impact, effectiveness and efficiency of IFAD’s partnerships. Evidence is drawn from a series of semi-structured interviews (using a checklist of questions) and a review of the documentation available. Due to the sheer quantity of relationships (across corporate, regional and operational levels), the focus is on a subset of these, including: NGOs and civil society; International Land Coalition (the Coalition); Global Environment Facility (GEF); Global Mechanism (GM); WFP/FAO; AAAID and OPEC; the Belgium Survival Fund (BSF). NGO relationships in particular, will be followed up in the country visits.

Partnership types 1.8 For IFAD, like many bilateral and multilateral agencies, the term partnership is a recently applied phenomenon used to describe a range of pre-existing inter-organisational relationships and alliances. The 5th and 6th Replenishments cite several very different partnership structures: from the joint programme with WHO, UNICEF and UNDP for the Belgium Survival Fund (BSF), to the plethora of relationships with the private sector and civil society organisations at the operational level. The External Affairs Department (EAD) also uses the term widely, and indeed it provides a cornerstone for EAD’s work programme in 2003 (IFAD 2003d: 10-15).4

1.9 The range of relationships which are labelled “partnerships” is both numerous and diverse. At the corporate level, these extend from the hosting of formal secretariats (such as GM), to relations with other International Financial Institutions (like the World Bank), to coordination mechanisms of the United Nations (see Annex 1, for a full listing). Within borrower countries, the diversity is just as broad, ranging across macro, meso to micro levels (see Annex 2, for a full typology).

1.10 By applying the ‘partnerships label’ so widely, this suggests a lack of definition and focus in the way it is applied institution-wide. In publicity terms this may not matter, but with respect to the strategic prioritisation and investment required by such relationships it is less useful. For a relatively small IFI such as IFAD, the broad-brush approach may only serve to undermine the value that could be gained from working in partnership with fewer organisations, but ones that are able to really contribute to IFAD’s strategic objectives. For instance, IFAD has relations with a range of bilateral donors and yet many of these relationships go little beyond resource mobilisation, with Memorandums of Understanding (MoUs) plus a few meetings with a small number of staff (say in EAD). In other cases, more meaningful partnerships occur, and ones which make important contributions to IFAD’s organisational objectives. This paper aims provides a systematic assessment of these relationships, their relevance, impact and effectiveness.

Relevance to IFAD objectives 1.11 In a review of some of the key partnerships (GM, GEF, the Coalition, NGOs, BSF, OPEC and AAAID), most appear relevant to IFAD’s Strategic Framework. Conversely however, it is also difficult to argue that IFAD’s partnerships are not relevant at the broad policy level. For instance the Global Mechanism (GM) is, “designed to rationalise the 4 Following two retreats of EAD staff, the 2003 work programme states, “Expected Output Number 4: Partnerships are managed for the purpose of coalition building and advocacy” (IFAD 2003d: 11). This covers a range of relationships from: i) the work of the Office of UN Affairs and Partnerships, based in New York; ii) the North America Liaison Office (NALO), based in Washington, which includes IFAD participation in ECOSOC and UNCCD; iii) and, the Policy Division (EO) which includes IFAD’s relations with NGOs and civil society organisations (IFAD 2003d: 11-15). The work of EAD also covers tasks of the Resource and Mobilisation Unit (ER) including relationships with bilateral donors, the Belgium Survival Fund, and the private sector (IFAD 2003d: 15-20). See also, the Annual Report 2000 (IFAD 2000a: chapter 4).


Annex 7 Page 4

allocation of existing aid to combat land degradation and mobilise additional funding…” (The Global Mechanism 2003: 4). The GM is not a funding organisation, but one that performs a coordinating, brokering and catalytic role to enable the implementation of the UN Convention to Combat Desertification (UNCCD). As many IFAD projects operate in degraded environments, and as resource degradation can severely affect the productive capacity of poor livelihoods, the GM appears highly relevant to IFAD’s strategic aims. Similarly, the International Land Coalition (the Coalition) is concerned with knowledge exchange and dialogue to create a favourable political-economic context for agrarian reform. The access to (and ownership of) land and water resources by the rural poor is an important element in contributing to IFAD’s strategic objectives of: i) improving equitable access to productive resources and technology; and, ii) strengthening the capacity of the rural poor and their organisations (IFAD 2002b: 9). Furthermore, some would argue that agrarian reform (secure access to land resources) is considered to be a pre-condition for poverty alleviation and increasing household food security. 5 It is also an area in which IFAD has perhaps made a limited contribution in recent years.

Table 2. Initial assessment of relevance against IFAD partnership objectives

IFAD Partnership Objectives (5th and 6th Replenishment)

Examples of IFAD partnerships

Rep

licat

ion

and

scal

ing

up

Mac

roec

onom

ic

refo

rm

Inst

itutio

nal

Cap

acity

Kno

wle

dge

exch

ange

Mul

ti st

akeh

olde

r co

ordi

natio

n

Priv

ate

sect

or

inve

stm

ent

Co-

finan

cing

Cat

alyt

ic ro

le

Global Environment Facility ¤ l Global Mechanism l ¤ l l l International Land Coalition ¤ l l l NGOs – corporate and regional levels ¤ l l NGOs – country level operations l l l AAAID l l l ¤ FAO/ WFP ¤ l l l l Notes: Contribution to IFAD partnership objectives: l = high; ¤ = partial.

1.12 Beyond a broad policy assessment of relevance, it becomes much more complex (but also potentially more important) to consider the relevance of IFAD’s partners at an operational level. Indeed much of what is considered to be IFAD’s core business (i.e. loan projects), plus its key contribution to “enabling the poor to overcome their poverty”, probably occurs at the country level - whether through the implementation projects, innovations or even policy dialogue. For GM, an assessment of relevance of this partnership at the operational level makes a less strong case. The GM works primarily through support to the national planning processes (focusing on combating land degradation), alongside the mobilisation of resources to implement these plans. At present there is relatively limited engagement between GM and IFAD operations.6 For example, in the NENA region the partnership with GM at this level in its relative infancy; there are some cases (such as in Tunisia) of GM assisting IFAD projects to better coordinate with a wider set of ministries

5 As outlined in, “ Making a Difference Through New Ways of Working Together”, The Popular Coalition to Eradicate Hunger and Poverty (now the Internat ional Land Coalition), IFAD intranet, February 2004. 6 As a recent Governing Council document (IFAD 2004a: 3) states, “there is unrealised scope for IFAD and the Coalition to work together to influence global and national policy agendas more effectively”. A recent evaluation of the GM (2003) recommended that there should be stronger linkages with IFAD including: “integrating GM resource mobilisation into the IFAD resources mobilisation department (ER)”, and to, “engage IFAD… to ensure UNCCD incorporation within their lending programs”. Following this evaluation, a Global Mechanism Advisory Group has recently been established to better coordinate and link with IFAD’s operations (IFAD 2004b). The GM Advisory Group will include representatives from EC, ER, NALO, FC, FH and PT, and the secretariat of the Advisory Group will be placed within PMD.


Annex 7 Page 5

through GM’s involvement with the national planning process. There is however limited experience of actual resource mobilisation at the country level. Plus, with the GM having limited capacity to offer technical support, the added value of working with GM (especially above other partners) is far from conclusive.

1.13 In contrast, working with NGOs can appear very relevant at the operational level. In the NENA region for example, there is a great deal of state control with highly centralised governments and banks. In such an institutional context, state mechanisms can have a conservative tendency with limited proactive capacity, such as in participation. As project management units tend to be located within a relevant ministry, working in partnership with NGOs can enable IFAD to be more participatory as well as build capacity amongst government staff. Similarly, in Central and Eastern Europe, the demise of the state following the collapse of the Soviet order has left something of a vacuum – with the collapse of many cooperative arrangements, and the further retrenchment of the state from more remote areas. In this context, NGOs are highly relevant to rural and agricultural lending especially given the centralised nature of much of the commercial banking sector. In other circumstances, NGOs can provide the only opportunity to enable IFAD to reach the rural poor. In Togo, IFAD loan-based operations have been under suspension since January 2001. As such, grants to civil society provides the only way for IFAD to operate in such a dysfunctional context.

1.14 The ease at which IFAD partnerships can seem highly relevant at the broad policy level (and less so for programme operations) is also due to the lack of specificity in the IFAD Strategic Framework. While at the strategic level it is not possible (or even desirable) to be extremely specific (in a programming sense), a strategy should help direct priorities in an organisation. The three strategic objectives of IFAD (IFAD 2002b: 9) are very broadly defined covering financial assets and markets, human and social assets, and, productive assets and technology. Likewise the 5th and 6th Replenishment documents do not provide a clear and comprehensive strategy for the selection and prioritisation of partnerships. Instead the objectives (as presented) cover everything from macroeconomic reform, co-financing and scaling-up, to multi-stakeholder coordination, knowledge exchanging and sharing institutional capacity.

1.15 So, should a relatively small institution like IFAD be engaging in such a wide range of partnerships? Certainly some of the relevance of the current plethora of ‘partnerships’ may be called into question. It is however difficult to be definitive over this issue, and it is perhaps more helpful to consider also issues of partnership unity, effectiveness and impact. A more pragmatic approach might be to prioritise partnerships on the basis of their performance, for which the following questions are key:

• Are there clear objectives for the partnerships? • Can performance be measured against these objectives? • Is there evidence that the partnerships enhance IFAD’s impact?

Partnership unity 1.16 As mentioned previously, partnerships involve activities beyond that which contracts or authority alone would warrant: partnerships are aimed at achieving shared goals based upon close working relationships (Watkins and Csaky 2003). Mutual commitment to agreed objectives in a partnership, shared among its members, are an essential element to successful partnerships. Furthermore, clear, shared objectives are an essential element of being able to monitor and evaluate the effectiveness of a partnership.

1.17 There are many cases of formal agreements between IFAD and its partners, but less clarity over shared objectives to measure the performance of IFAD’s partnerships. Examples of formal (almost contractual) relationships include most bilateral and other donors, such as the Partnership Agreement with the Government of Italy (GoI and IFAD 2001) and the MoU with the Inter-American Development Bank (GoI, IADB and IFAD 2003). These tend to be


Annex 7 Page 6

legally binding agreements, with limited evidence that they represent more organisation-wide, shared objectives. There are other agreements that offer a better basis for partnership, but still tend to concentrate on legal, procedural or administrative matters (such as for GM, GEF and the Coalition).7 In some cases, the objectives are cited in terms of the constitution of the entity, rather than being expressed as shared objectives for the relationship.8 In the case of GM and GEF, there are however recent moves to enhance the relationship and increase its ownership across the organisation with the formation of the Global Mechanism Advisory Committee based in PDM.

1.18 For NGOs, there is a notable absence of a clear policy and corporate or regional level framework for engaging with civil society. One of the goals of the annual IFAD/NGO Consultations was to, “provide a framework for strengthening IFAD’s collaboration with NGOs…facilitate the involvement of NGOs as IFAD’s partners and promote mutual cooperation.” (IFAD 2000c). The 10th (and last) of these conferences was held in Pune, India during May and June 2000. Furthermore, one of the agreed upon recommendations of the Evaluation of the Extended Cooperation Programme (IFAD 2000d: 2) was to develop a strategy for IFAD/ NGO collaboration, which was to include, “the goal and priority objectives of IFAD’s collaboration with NGOs”. Since then little progress has been made, and more recently there has been increased uncertainty with the new Grants Policy (IFAD 2003e) absorbing the previous ECP (NGO) grants - plus the restructuring of the Policy Division.

1.19 This contrasts with the relations between IFAD and NGOs at the operational level. In the Evaluation of the ECP (IFAD 2000d: 12), it was found that there were: “notable examples of very good partnerships, with evidence that ECP projects had indeed served to strengthen relations between IFAD and NGOs…in terms of mutual trust, understanding and working for a common purpose”.

Partnership effectiveness and impact

1.20 Partnerships are a means by which IFAD can enhance its impact, either directly or by performing a catalytic role. There is anecdotal evidence of impact, but little in terms of systematic analysis of partnership impact overall. For example, the evaluation of the ECP grants to NGOs (IFAD 2000d: 5) focuses only on effectiveness, efficiency and relevance, and not expressly impact. In the evaluation, reasons for processing ECP grants given by CPMs show that there is a tendency to focus more on collaboration with NGOs in the context of loans and project implementation, rather than other issues such as piloting or innovation. The Executive Summary and Agreement at Completion Point (IFAD 2000d: 12) suggests that were notable examples of good partnerships, as well as, “remarkable achievements in testing and replicating innovative institutional approaches”.9 Yet in the Main Report (IFAD 2000e: 39) the evidence is less convincing about actual achievements: with some 90% of the implementation reports reviewed having inadequate or inappropriate information in terms of achievements – thus making it difficult to systematically consider the achievement of ECP projects.

1.21 Similarly for GM, GEF, the Coalition and AAAID, there is limited systematic measurement of impact, with reference generally made to anecdotal examples. For the Coalition (IFAD 2003f: 3) examples include: i) the joint design (also with the BSF) of two projects in Nigeria and Uganda; ii) work to achieve desired IFAD and Popular Coalition outcomes from the WSSD through joint events at the Preparatory Committees held in Bali and New York and at the Summit in Johannesburg; iii) contribution to IFAD position papers and publications to the Conference on Financing for Development in Mexico. These do not

7 For example, the MoU between COP and IFAD for the Global Mechanism (COP and IFAD 1999). 8 For example, the Basic Framework for the Population Coalition (IFAD 2001b). 9 Reference is made to two examples: i) from Peru (Grant No. 07), Strengthening Extension and Training Services; ii) from Burkina Faso (Grant No. 34), Savings and Credit with Education (IFAD 2000d: 12).


Annex 7 Page 7

constitute impact in terms of programme operations, although in they may illustrate more of a catalytic role.

1.22 In terms of effectiveness against the four objectives as set out in the 5th Replenishment, the evidence of a direct contribution is variable – and not always substantiated. For example:

• Replication and scaling up through co-financing: The AAAID has been particularly instrumental in four projects in the NENA region (in Sudan, Lebanon, Morocco and Syria). In Lebanon for example, civil war resulted in the loss of much of the dairy herd. IFAD invested in restocking, though in small numbers of 3-4 cows per farmer, plus technical support and six milk collection centres. IFAD was however instrumental in working with AAAID to secure a significant investment in a milk processing plant, which enabled scaling up and longer term security of the main market for small dairy herders. Also, the partnership with GEF is providing increased co-funding to fifteen pipeline GEF/IFAD projects, of which seven also are in partnership with GM. 10 There appears to be less evidence of scaling up through co-financing for others partnerships such as NGOs and the Coalition. Indeed the ECP Evaluation (2000e: 55) notes that ECP projects have been constrained by an inadequate understanding of piloting for up scaling.

• Complementing macroeconomic reform: This is a more difficult objective to substantiate, and tends to mostly involve policy reforms at the national level. The GM is a player in producing national plans for combating land degradation. The Coalition is involved in micro-level agrarian reforms, such as in the Philippines and Indonesia, as well as work similar work in Bolivia, Guatemala and South Africa.

• Sharing institutional capacity. For the Coalition, GM and GEF, (although all very different) the secretariats are hosted by IFAD and share its administrative, publication, legal and financial capacities. Recent evaluations and other documents (Falloux, 2003; IFAD 2004a; 2004b) acknowledge the potential for stronger linkages with IFAD operations.

• Exchanging knowledge. In general, the impression is that there is limited understanding of the work of the Coalition, GM and GEF, and little sharing of experiences, amongst IFAD (and especially operational) staff beyond senior management. Similarly for NGOs, knowledge sharing within regions and especially between regions is weak, with information tending to retained by individual CPMs.11 The ECP Evaluation (2000e: 48) even stated that, “there has been no mechanism to collect, organise, interpret, compare, update and disseminate lessons from ECP”.

1.23 In part, the lack of systematic evidence available for assessing effectiveness is in part due to a lack of monitoring against clear, shared partnership objectives. One of the main mechanisms for monitoring various IFAD-partner relationships, as opposed to partners per se, is through Progress Reports to the President.12 The information contained in these reports varies and can include a host of more administrative matters. There is also a tendency to either concentrate on the partner (e.g. the Coalition or GM) or be descriptive about the performance of the IFAD-partner relationship (i.e. tending to provide anecdotal examples, and little substantial evidence of impact). NGOs at operational levels tend to be monitored as part of the loan project, and for more personality-driven partnerships (such as AAAID) there no recorded measurement.

10 Based on the PT Work Programme 2004: IFAD Pipelin e of GEF Projects (March 2004). 11 For example, the ECP Evaluation (IFAD 2000d: 6) found that, “CPMs played a key role as gatekeepers for the entry of ECP grant proposals and subsequent learning form ECP projects”. 12 For example, IFAD (2003f).


Annex 7 Page 8

1.24 In terms of capacity building, GM and Coalition staff are employed by IFAD (though paid for by their respective secretariats) and are therefore subject to the same financial and administrative procedures. This means that they have access to the same training opportunities as equivalent staff within IFAD. Support and capacity building to other partnerships mostly focuses on NGOs, though with the suspension of ECP and its incorporation into the new Grants Policy, there is some uncertainty on how this support will continue to function.

Partnership management and coordination 1.25 Management and coordination arrangements range from formal to highly informal structures, with no single approach to managing very diverse partnerships across the organisation. The GM and Coalition secretariats are hosted by IFAD, while drawing on wider institutional support, have their own governing structures. More recently, management of the GM/GEF relationship will be undertaken by the newly formed GM Advisory Group, based within PDM. Relations with bilateral and multilateral donors are generally managed more formally by EAD, while the Policy Division (within EAD) is responsible for NGO relations at a corporate level. In terms of management of NGO relations across IFAD, there are designated NGO Focal Points within each Regional Division. This was originally conceived as a link with the Policy Division, and especially the ECP programme. While several NGO Focal Points have undertaken initiatives13, this is largely voluntary, as the role is addit ional to their usual duties. The AAAID relations are handled on a more personal basis within the NENA Region.

1.26 At country level, CPMs and project staff are mostly responsible for the management and coordination of NGOs, and even then mostly in relation to their respective programmes. The way in which NGOs are coordinated and managed by CPMs is said to vary considerably, and while NGOs are valued ‘theoretically’ within IFAD, the management and coordination at operational level has some weaknesses, such as:

• A lack of incentives for CPMs to put NGO related grants (ECPs) into their workplans – to help overcome an essentially loan culture.

• A lack of instruments in which to engage with NGOs, particularly in countries where loans are suspended (e.g. Togo), governments are weak, and where IFAD needs to work through civil society. Also in some cases, NGOs may have political or philosophical objections to the use of loan funding.

• No specific funds made available for follow-up, such as through the specific supervision of NGOs

• Limited inclusion of NGOs and civil society as part of a country portfolio review • Lack of initiatives to capitalise from the lessons of working with NGOs

1.27 Communication within IFAD on the range of partnerships varies, but there is a sense that there are pockets of staff concerned with certain partnerships while the rest of IFAD staff have little involvement. For example, IFAD staff (beyond senior management) are not particularly well informed of the benefits of the partnerships with GM/GEF and the Coalition. Also, experience of NGOs at country level tends to be retained by individual CPMs, with less regional, cross-regional and corporate knowledge sharing. Also, corporate level relations at a policy and advocacy level are currently undergoing change, and NGO Focal Points have few incentives to undertake NGO initiatives or liase with CPMs.

1.28 For partnerships to function well, they often require a significant investment of those involved, though the added value of working in this way can be considerable. Sometimes 13 For example in the West and Central Africa Division, there has been a follow-up to the annual IFAD-NGO conference in Pune, India, with a regional ECP Portfolio Review (Trottier 2003). In Eastern and Southern Africa Division, there has been a successful proposal to support regional knowledge sharing amongst NGOs (IFAD, GOI and GOK 2000). This was initiated by the regional NGO Focal Point.


Annex 7 Page 9

financial resources are less of an issue, especially where significant sums from one partner to another can alter the dynamics of the relationship. The financial costs are still important though, especially where the perceived benefits to IFAD are less certain, such as in the case of the GM and Coalition. For others, like NGOs the relative cost is potentially less, but anecdotal evidence suggests that the benefits can be significant – for innovation, piloting, implementation, etc (especially as IFAD does not have a field presence). For less formal relations, such as with AAAID, the investment is mostly in terms of staff time and opportunities to take the initiative – something that is difficult to quantify in financial terms.

Key Issues

This initial assessment has identified the following key issues:

• The widespread use of the term ‘partnership’ within IFAD, combined with the relatively all-encompassing objectives set out in the 5th and 6th Replenishments, may undervalue the real benefits that can be gained from working in partnership.

• IFAD’s Strategic Framework is defined in such broad terms, that it is difficult to argue against the relevance of a great many of partnerships, even though they may not present the best opportunities for contributing to IFAD’s strategic objectives. In the past, IFAD has entered into, and become ‘host’ to, significant partnerships for which questions are now being raised about the real benefits to IFAD.

• While agreements exist between IFAD and many of its so-called ‘partners’, these statements are not necessarily very clear on the shared objectives of the relationship. Furthermore, in several cases there is a limited institution-wide understanding of the benefits of some key partnerships (like GM, GEF, and the Coalition).

• Evidence of effectiveness and impact is mostly anecdotal and there is limited systematic monitoring of partnership effectiveness against shared objectives.

Issues for follow-up in the country visits 1.29 This paper provides an initial assessment of some of the issues related to IFAD’s partnerships. This evidence is drawn from a desk study plus semi-structured interviews with key partners. As such, there has been little opportunity to explore issues beyond the corporate level, and the country visits provide the opportunity to consider some issues to a greater extent, including:

• Evidence of partners (particularly NGOs) directly contributing to IFAD’s impact, or illustrating a catalytic role.

• Has IFAD promoted host country institutions as partners?

• Evidence of private sector partnerships and their effectiveness and impact.

1.30 There is also a need to further add to the evidence presented in this paper. In particular, it was not possible to visit key staff of the Belgium Survival Fund (BSF), and also to undertake an analysis of the data contained within the ECP/NGO database.


Annex 7 Page 10

References

BSF (1998), The Belgium Survival Fund Joint Programme: Annual Report, IFAD, Italy.

COP and IFAD (1999), Memorandum of Understanding between the conference of the parties of the United Nations convention to combat desertification in those countries experiencing serious drought and/or desertification, particularly in Africa, and the International Fund for Agricultural Development regarding the modalities and administrative operations of the Global Mechanism, April 1999, Conference of the Parties (COP) and IFAD, Rome.

Falloux, F. et al (2003), Independent Evaluation of the Global Mechanism, Final report to the World Bank Development Grant Facility, 26 June 2003, The Evaluation Team.

GoI and IFAD (2001), Partnership Agreement between the Government of Italy and the International Fund for Agricultural Development, 12 November 2001, Rome.

GoI, IADB and IFAD (2003), The Memorandum of Understanding among the Italian Republic, the Inter-American Development Bank and the International Fund for Agricultural Development, March 2003, Rome.

IFAD (2000a), Annual Report, especially pages 102-104 and 106-107, IFAD, Italy.

IFAD (2000b), The Land Poor: essential partners for sustainable management of land resources, IFAD, Italy.

IFAD (2000c), Highlights and News: 10 th IFAD/NGO Consultation: Tripartite Partnerships for poverty alleviation and food security through programmes, 30 May – 2 June 2000, Pune, India.

IFAD (2000d), Evaluation of the IFAD/NGOs Extended Cooperation Programme: Executive Summary and Agreement at Completion Point, September 2000, IFAD, Rome.

IFAD (2000e), Evaluation of the IFAD/NGOs Extended Cooperation Programme, Volume 1: Main Report, September 2000, Report Number 1114, IFAD, Rome.

IFAD (2001a), Rural Poverty Report 2001, Chapter 6, IFAD, Italy.

IFAD (2001b), Presidents Bulletin: The Basic Framework regarding modalities for and administrative operations of the Popular Coalition and its relationship to IFAD, 28 March 2001, PB No. 01/03, IFAD, Rome.

IFAD (2002a), IFAD and NGOs: Dynamic partnerships to fight rural poverty, IFAD, Italy.

IFAD (2002b), Strategic Framework for 2002-2006: Enabling the rural poor to overcome their poverty , March 2002, IFAD, Rome.

IFAD (2002c), Enabling the rural poor to overcome their poverty: report of the consultation on the sixth replenishment of IFAD’s resources 2004-2006, 12 December 2002, IFAD, Rome.

IFAD (2003a), IFAD and Italy: A partnership to eradicate poverty, IFAD, Italy.

IFAD (2003b), IFAD, OPEC and OIC Member States: A partnership to eradicate poverty , IFAD, Italy.

IFAD (2003c), Progress Report on the Popular Coalition to Eradicate Hunger and Poverty , Governing Council – 25th Session, GC 26/L.11, IFAD, Rome.

IFAD (2003d), External Affairs Department (EAD): End of year report 2003, 1st draft for discussion, EAD, IFAD, Rome.

IFAD (2003e), IFAD Policy for Grant Financing, Executive Board 18 th Session, 17-18 December 2003, EB 2003/80/R.5/Rev.1, IFAD, Rome.

IFAD (2003f), Progress Report on the Popular Coalition to Eradicate Hunger and Poverty, Governing Council – 25th Session, GC 26/L.11, IFAD, Rome.


Annex 7 Page 11

Governing Council – 25th Session, GC 26/L.11, IFAD, Rome.

IFAD (2004a), Report on Lessons Learned by the International Land Coalition, Governing Council – 27th Session, GC 27/L.8, IFAD, Rome.

IFAD (2004b), President’s Bulletin: The Global Mechanism, 21 January 2004, PB/04/01, IFAD, Rome.

IFAD (200X), IFAD and the private sector, IFAD, Italy.

IFAD, GOI and GOK (2002), Civil Society in Rural Development and Poverty Alleviation in Eastern and Southern Africa: NGO/IFAD Consultation Workshop, Nairobi, Kenya 6-9 May 2002, IFAD, Government of Italy and Government of Kenya.

ILC (no date), Governance and Operational Framework , International Land Coalition (ILC), Secretariat at IFAD, Rome.

The Global Mechanism (2000), The Global Mechanism (GM) of the UNCCD: Mobilising Resources to Combat Land Degradation and Poverty, The Global Mechanism, UNCCD and IFAD, Italy.

The Global Mechanism (2003), The CCD in Action: The Global Mechanism of the United Nations Convention to Combat Land Degradation (Combating land degradation and poverty in the new millennium), The Global Mechanism, UNCCD and IFAD, Italy.

Trottier, B. (2003), The ECP Portfolio of Africa I Division (Western and Central Africa Division), Draft, December 2002-January 2003, IFAD, Rome.

Watkins, F. and Csaky, C. (2003). Partnerships, Volume 1 - Literature Review, Edinburgh Resource Centre for DFID Evaluation Department.


Annex 7 Page 12

Annex 1: List of partnership types, based on IFAD intranet (February 2003). Level Type of partnership Example(s)

United Nations • Global Environment Facility (GEF) • Global Mechanism (GM) of the UN Convention to Combat

Desertification (UNCCD) • Rome-based UN (WFP and FAO) • United Nations Development Group (UNDG) • United Nations Development Assistance Framework (UNDAF) • Common Country Assessment (CCA) Initiative • Country Strategy Note (CSN) • United Nations System Chief Executives Board (CEB) for

Coordination • Various UN conferences and summits

International Financial Institutions (IFIs)

• World Bank Partnership for Development Initiative • HIPC and Poverty Reduction Strategy Papers • World Bank/ International Monetary Fund (IMF) • Comprehensive Development Framework (CDF) • African Development Bank (AfDB) • Islamic Development Bank (IsDB) • OPEC Fund.

International Organisations

• OECD/DAC (Development Assistance Committee) • Organisation of African Unity (OAU) • Organisation of Islamic Conference (OIC) and Affiliated

Organisations • Islamic Educational, Scientific and Cultural Organization

(ISESCO) • Statistical, Economic and Social Research and Training Centre

for Islamic Countries (SESRTCIC) • League of Arab States (LAS) and Affiliated Institutions

Bilateral and Multilateral Organisations

Other • International Land Coalition (Popular Coalition) • Government of Italy • OPEC and OIC Member States • Belgian Survival Fund

NGOs and civil society

Partnerships with NGOs and the Corporate Sector

Corporate (private) sector

Local (decentralized) governments as implementing partners

Within Borrower Countries

National governments as borrowers and implementing agencies


Annex 7 Page 13

Annex 2: A typology of partnerships, at the country level (IFAD 2001a).

Independent External Evaluation of IFAD Desk Review Report Annex 8 – Human Resources

Annex 8 Page 1

Annex 8

HUMAN RESOURCES

1 Introduction

1.1 IFAD is an organisation of people. The Fund was established with the deliberate intention of constraining organisational growth so that it would be an efficient provider of development finance. Its core objectives and mode of working define the environment within which staff work. A focus on project identification and design; restriction to a single global site; extensive use of consultants to provide technical expertise; and policies to be innovative, act as a catalyst and learn from experience all place demands on personnel management, staff numbers, and corporate culture. This annex is structured around those three themes, to test the hypothesis that development of highly motivated, well-trained and efficient staff has contributed to IFAD’s performance.

2 HR management in IFAD

Personnel management to 2003

2.1 Personnel Policies. Management of personnel has been based on the Personnel Policy Manual, updated most recently in 19911, complemented since April 2000 by a Human Resource Handbook2 that is a compilation and consolidation of all the documents issued on human resource matters. These documents set out the conditions and terms of employment, definitions of positions and their evaluation, recruitment and appointment procedures and guidelines for staff development and training.

2.2 The Role of the HR Division. The HR Division in IFAD has had a mainly transactional role in recruitment and the payment of salaries and benefits to staff and consultants, applying the rules and guidelines of the Personnel Policy Manual and the HR Handbook. There was no human resource development strategy; core competencies for differing jobs or grades were not defined; performance evaluation was not systematically linked to training needs or promotion and managers received little or no training in human resource management.

2.3 Review of the HR Function. This unsatisfactory situation was recognized by senior management and in October 2001 when the Review Committee for Modernizing Human Resource Policies and Procedures started work.. The Committee was mandated to review four distinct areas of human resource policies and procedures: recruitment, career development, performance evaluation and recourse and appeals. The final report of the Review Committee was submitted in December 2001 3. The Committee reviewed the Personnel Policy Manual and the Human Resource Handbook and undertook surveys of staff using staff interviews and focus groups to identify constraints to the HR management system in IFAD. The Committee summarized the comments and feed back on the four priority areas, discussed change management issues, and made recommendations on the introduction of an integrated HR planning and monitoring system and the role of the HR Division.

2.4 The Committee identified a significant need for change in HR management within IFAD and drew up an action plan to bring about the required change. The main ‘thrusts’ of the action plan were to: (i) Define a new role of HR within the institutional and Strategic

1 Personnel Policy Manual 1991 Edition. IFAD. 2 Human Resource Handbook 2000. IFAD 3 Review Committee for Modernizing Human Resources Policies and Procedures. Summary of Findings. December 2001. IFAD.


Annex 8 Page 2

Framework context, while strengthening management HR capabilities through training in human resource management and performance evaluation. (ii) Develop a comprehensive HR strategy, with supporting planning for and monitoring of HR needs. (iii) Manage a sound recruitment process. (iv) Create, manage and monitor a career development system for all staff. (v) Introduce performance-based evaluation for all staff based on defined job families and associated competencies, highlighting exceptional or poor performance, with a clear linkage between individual performance and divisional work plans. (vi) Manage a sound recourse and appeals procedure.

2.5 Recruitment. Issues of recruitment were obviously of particular concern, as shown by the fact that the HR Review Committee’s action plan contained 21 action points related specifically to recruitment. The action points reflect past problems in recruitment and relate mainly to transparency for both internal and external appointments or promotions, adherence to rules of timing and closure, interview procedures, and assessment of candidates against competencies and qualifications for specific job families.

The Proposed HR Strategy 2.6 Management Approval. Senior management discussed and approved the Action Plan, and the President’s Bulletin of 28 February 2002 4 presented the main conclusions of the Review Committee in relation to the new role of HR, strategic HR planning, management and leadership roles, and modern policies and procedures supporting an integrated HR system. Implementation of the Action Plan was approved and senior management committed to review progress.

2.7 Elements of the Strategy. To meet the challenges of the Action Plan the Director of Human Resources is implementing a strategy that is consistent with the ICSC framework for human resource management. This has four main elements:

• Compensation and Benefits • Employment – recruitment, placement and retention, contracts and staff well

being • Career Management – staff development and training, performance management

and mobility • Good Governance – management style, staff representation, administration of

justice 2.8 Competencies. Ten core competencies for IFAD staff have been defined and agreed in January 2004, through an evolving process of discussion within the institution. These core competencies are as follows:

Vision Innovation and risk taking Leadership Leading and managing change Building a successful team

Managing performance/staff development/coaching Interpersonal skills/communication/negotiation/conflict resolution Decision making Planning and organizing/delegating responsibility Acts with integrity, builds trust and confidence

4 President’s Bulletin. Modernising Human Resource Policies and Procedures. 28 February 2002


Annex 8 Page 3

2.9 On going discussions may result in some further amalgamation to result in a final six. Levels of each competency are being defined initially for managers but levels for other grades of staff are to be defined in the near future. These competencies reflect senior management’s emphasis on developing leadership, innovation and risk taking as part of IFAD’s corporate culture.

2.10 Performance Evaluation. The developing HR strategy strongly emphasizes performance evaluation. All staff will be assessed against the core competencies for specific job families. DFID is supporting the formation of a Development Centre for human resource development in the three Rome based UN agencies. Assessment against the agreed competencies is being tested in the Centre in three pilot studies, starting in April 2004. Five staff from IFAD at three levels, senior managers, CPMs and P3/P4, will participate in the pilots.

2.11 Broadbanding and Pay for Performance Pilot. IFAD is participating in a pilot programme to introduce broadbanding, that is eliminating divisions between grades so that there is a career continuum for professional and support staff. The emphasis on performance is also to be seen in the planned pilot programme of pay for performance, rather than a standard annual within grade increase. Three UN organizations are involved in this pilot, IFAD. UNAIDS, and WFP. It is planned that by 2005 there will be a within grade increase, together with a performance bonus, and by 2007, increases will be entirely performance based.

2.12 Structure of the HR Division. The HR Division has been reorganized so that four HR Officers are responsible for co-ordinating with line managers, all aspects of human resource management in one or more Departments within IFAD. A Training and Development Officer provides support to the HR Officers.

3 STAFFING AND CONSULTANTS

Complement 3.1 Categories of Staff. There are two main categories of staff in IFAD, professionals in Grades P1 – P5 and D1 - D2, and General Service Staff in Grades G1 – G7. In each category there are two types of appointment, regular and temporary. Temporary staff, who may be professional or support staff, are divided into those with fixed term conditions and an appointment for one year or longer, and those without fixed term conditions, appointed for less than 11 months. In addition to staff there are consultants, usually appointed for a fixed term that depends on the specific task to be undertaken.

3.2 Total Staff and Consultant Numbers.5 There is some difficulty in reconciling the numbers of staff and consultants provided for in the budget, those derived from the payroll system, and the total number of people estimated to be working in IFAD by the Buildings Department. These figures for December 2003 are summarized below:

Budget Provision Total staff 466 Payroll Total staff 491 Building Department Estimate Total individuals 650 3.3 The discrepancy between budget and payroll figures can be explained by the fact that budget provision for TPOs is for 10 person years. In fact there are 23 TPOs, which gives a payroll total of 489, close to the actual. The discrepancy between the payroll total and the number of individuals thought to be working in the IFAD buildings is largely due to the large

5 Source. Human Resource Department Records


Annex 8 Page 4

number of consultants employed. It can be assumed that consultants working for Departments other than PMD spend most of their time in IFAD, and in 2003 there were 287 individuals. If 50% – 60% of these consultants are working at any one time, it means there are between 143 and 172 consultants in the IFAD buildings. This gives a total of between 634 and 663 individuals in the buildings, close to the Building Department estimate.

3.4 Growth in Staff and Consultant Numbers 1994 – 2003. Table 1 summarizes the total number of professional and support staff and consultants employed by IFAD in 1994, 1999 and 2003. It shows that the total number of professional staff increased from 135 to 202, a 50% increase. Support staff numbers increased from 169 to 264, an increase of 56%. Over the same period, the number of consultant days increased from 33,394 to 48,373 an increase of 45%. If consultant days are converted into staff year equivalents there has been an overall increase of 51% in personnel over the ten year period.

Table 1 IFAD Staff and Consultants. Totals 1994, 1999, 2003

Staff by Category

Gender

1994

1999

2003

% Increase 94 - 03

Prof. Reg. + Extra Budgetary Funds M 98 98 99 1 F 33 56 57 73

Total 131 154 156 19 NSP/ Prof. Chap X M 3 8 20 567

F 1 9 26 2500 Total 4 17 46 1050

GS Reg. + Extra Budgetary Funds M 32 30 33 3 F 137 135 148 8

Total 169 165 181 7 GS Long term temp./ GS Chap X M ND 4 16

F 24 67 Total 28 83

All Professionals M 101 106 119 18 F 34 65 83 144

Total 135 171 202 50 All Support M 32 34 49 53

F 137 159 215 57 Total 169 193 264 56

Total Staff Complement 304 364 466 53

Consultants M 413 419 617 49 F 88 133 228 159

Total 501 552 845 69

Days 33396 37954 48373 45 Equivalent years 128 145 186 45

Total staff and consultant years 432 509 652 51

Number of new loans approved6 28 30 257

Value of new loans 349.3 432.7 365.9 Number of new grants approved 71 124 85

6 Data on loans and grants from IFAD’s Annual Report 7 Loans and grants for 2002


Annex 8 Page 5

Value of new grants 16.8 29.6 23.9 3.5 As a simple measure of efficiency, the number and values of loans and grants are shown at the foot of the table. The increase in staff and consultancy effort has not been accompanied by a growth in the volume of loans, though grants have increased somewhat. Overall, IFAD’s manpower efficiency appears to have fallen from 15 staff and consultant years per loan in 1994 to 26 in 2002/3.

3.6 Staff and Consultant Use By Div ision. Table 2 shows the numbers of staff and consultants in each Department and Division of IFAD and the changes between 1994 and 2003. Comparisons between earlier years and 2003 are not always precise because of organizational changes within IFAD but with guidance from IFAD staff an attempt has been made to get the best equivalence between earlier structures and the present.

Table 2 IFAD Staff and Consultants by Division

Division Professional Staff (No.)

Support Staff (No.)

Consultants (Days)

Department

1994 1999 2003 1994 1999 2003 1994 1999 2003 1994 1999 2003Office of President

OA OA 2 3 5 1 1 2 417 22 OC 2 1 527 OE 0E OE 10 8 11 6 7 10 3903 3911 4547 OL OL 5 6 7 4 3 6 993 859 OM OM 3 13 5 2279 OP OP 1 3 3 5 1418 OP OP OV 2 3 3 4 2 305 Sub-Total 20 23 44 14 15 31 3903 5626 9652

External Affairs CU 1 1 1 1 0 EI EC 7 7 566 1582 ED ED ED 17 18 1 14 11 3 35 2874 208 EF 2 3 183 EP EO 11 5 1482 718 ER 6 2 1000 MS MS ES 9 14 7 17 29 33 195 1766 250 Sub-Total 26 33 35 31 41 54 2278 4640 3941

Finance and Admin. MA MA FA 2 4 4 36 25 28 415 130 777 VC VC FC 7 11 13 29 28 40 471 427 2569 MD MD FD 1 1 1 1 1 2 24 55 749 MP MP FH 3 6 6 11 11 13 84 469 426 MM MM FM 10 11 10 5 17 23 2508 1817 1118 VT VT FT 5 7 4 8 9 10 88 608 BSF BSF 2 2 DL 295 Sub-Total 28 42 38 90 91 116 3885 3506 5639

Program Mgmt. PA/F PA PA 8 11 13 5 7 11 8598 4119 5409 PD PD PD 5 6 7 4 4 4 0 282 164 PF PF 9 13 14 6 8 11 Inc. PA 4496 6885 PI PI PI 13 12 11 6 7 11 5551 4621 5416 PL PL PL 8 9 12 5 7 9 3192 4663 4463 PN PN PN 8 9 13 3 6 10 2498 4416 5287


Annex 8 Page 6

Division Professional Staff (No.)

Support Staff (No.)

Consultants (Days)

Department

1994 1999 2003 1994 1999 2003 1994 1999 2003 1994 1999 2003 PT PT PT 10 13 15 5 7 7 2559 1585 1517 PU PU PU 932 Sub-Total 61 73 85 34 46 63 23330 24182 29141 Total 135 171 202 169 193 264 33396 37954 48373

3.7 Virtually all Divisions in IFAD have shown an increase in both staff and consultants, which reflects the overall 51% increase shown in Table 1. It is not possible to show percentage increases for individual Divisions because of the changing structure. However, Table 3 below shows the increases in each Department. The largest increase has been in the Office of the President, due mainly to the inclusion of Operations Evaluation and some other units such as the Global Mechanism. Other Departments show an almost evenly distributed increase in professional staff of between 35% and 39%. External Affairs and PMD show the largest increases in support staff but the greatest increase of 73% in consultant use is in External Affairs.

Table 3 The percentage increase in staff and consultants in each Department between

1994 and 2003

% Increase 1994 – 2003 Department Professional Staff Support Staff Consultants

Office of President 120 121 147 External Affairs 35 74 73 Finance & Admin 36 29 45 Programme Mgmt. 39 85 25 Total 50 56 45 3.8 It is also interesting to consider whether the proportion of the total complement of staff and consultants within each Department has changed over the years. Table 4 below presents figures showing that PMD was and remains the biggest employer of professional staff and consultants. There has been an increase in the proportion of the total complement working in the Office of the President, again reflecting the additional units reporting directly to the President. The largest proportion of support staff work within the Finance and Administration Department while the proportion of staff in External Affairs has remained about the same. Table 4 Percentage of the total complement of staff and consultants in each

Department in 1994 and 2003

% of Total Complement Professional Staff Support Staff Consultants

Department

1994 2003 1994 2003 1994 2003 Office of the President 15 22 8 12 12 20 External Affairs 19 17 18 20 7 8 Finance and Admin. 21 19 54 44 12 12 Programme Mgmt 45 42 20 24 69 60 Total Complement 1. 135 202 169 264 33396 48373 1. Staff numbers. Consultant Days

3.9 Staff Consultant Ratio. IFAD makes extensive use of consultants. Staff/consultant ratios based on numbers of staff and equivalent consultant years are summarized in Table 5 for the period 1994 – 2003. This shows a fairly constant ratio of total staff numbers to consultants of about 0.40, just under 1 consultant for every two staff. If only professional


Annex 8 Page 7

staff are considered, the ratio is much higher, ranging from 0.85 to 0.95, close to one consultant for every staff member.

Table 5 Staff/consultant ratios 1994, 1999, 2003

Category 1994 1999 2003 All staff 304 364 466 Professional staff 135 171 202 Consultants 128 146 186 Ratio All staff/consultants 0.42 0.40 0.40 Ratio Professional staff/consultants 0.95 0.85 0.92

3.10 The main reason for the extensive use of consultants is to provide specialist expertize that regular staff do not have. This is clearly justified for various stages of the project cycle, given the wide range of sectoral and sub-sectoral activities included in IFAD operations and PMD has been the single largest user of consultants over the years. The use of consultants, however, brings a lack of continuity by its very nature. Although many CPMs use the same consultants for various stages of the project cycle, it is not always possible. Institutional memory therefore suffers.

3.11 The extensive use of consultants in other Departments in IFAD is also widespread. Again it is justified if it is to provide specialist inputs not available in-house. However, in administrative and IT support activities the distinction between regular on-going programs and specific tasks is less well defined and there is a danger that consultants are used as part of IFAD’s core staffing. Given the data available, it is not possible to make a judgement as to the extent that consultants undertake core staff work.

3.12 Continuity. There are no data other than individual personnel records as to how long an individual has been in a particular position. Clearly, many staff have worked in IFAD for many years but equally, many have moved between Departments and Divisions or have been promoted. Even the number of appointments does not provide evidence of the replacement of staff as the appointments may be for additional new staff.

3.13 Gender Differentiation. Table 1 shows the numbers of male and female staff by category of staff and consultants. The proportion of women employed is summarized in Table 6 This shows that the number of female employees in IFAD increased by 50% over the period 1994 – 2003, with the biggest increase of 124% in temporary professional staff, though there has been a significant increase of 44% in the number of regular women professionals. Similarly, the proportion of women consultants has increased by 50% over the period. There has also been an increase in the number of women in P5 or higher posts. In 1994 the proportion of D1 or higher posts held by women was 1 out of 22, in 1999 it was 3 out of 20 and in 2003 it was 5 out of 22 or 5, 15 and 23%. At the P5 level, the proportions are 5 out of 42 in 1994, 8 out of 69 in 1999, and 13 out of 61 in 2003, or 12, 12, and 21%. By contrast, there has been little or no change in the percentage of women support staff, though this is at a high level of over 80%.

Table 6 IFAD Staff and Cons ultants. Percent females 1994, 1999, 2003

Staff by Category

% Increase

% Females

1994 1999 2003 94 - 03

Prof. Reg.+Extra Budgetary Funds 25 36 36 44


Annex 8 Page 8

Staff by Category

% Increase

% Females

1994 1999 2003 94 - 03

NSP/ Prof. Chap X 25 53 56 124 GS Reg. + Extra Budgetary Funds 81 82 82 1 GS Long term temp ./ GS Chap X ND 86 81 All Professionals 25 38 41 64 All Support 81 82 81 0 Consultants 18 24 27 50

4 CORPORATE CULTURE

4.1 An examination of IFAD’s corporate culture was undertaken to identify staff perceptions of the constraints to achieving IFAD’s primary objective of poverty alleviation. It was not, therefore, an analysis of IFAD’s organization and management but an attempt to understand how corporate priorities, management style and skills, staffing patterns and corporate procedures affect the ability of staff to reach their goals and their morale.

Change 4.2 The introduction of an ambitious change programme over the past nine years reflects senior management’s concern with increasing the relevance, efficiency and effectiveness of IFAD and the change programme has had a very significant impact on IFAD’s work environment. There have been changes in strategic emphasis and policy dialogue, linked to institutional and management reforms, changes in key functional areas, and development of financial and human resource management systems. These changes are reflected in the development of the Strategic Framework 2002 – 2006, six regional strategies, the use of COSOPs to link regional strategies to country programmes and the creation of a new Policy Division, and an independent Evaluation Unit. Other changes include the introduction of a Performance Based Allocation System, a Results Management System, and the Business Process Re-Engineering Programme, now known as the Strategic Change Programme. The ambitious programme of HR reform (see para xx) also involves very significant change, while changes in the organizational structure with the creation of the Finance and Administration Department and the External Affairs Department have meant major changes to the way in which IFAD conducts business. There have also been important changes in senior management positions.

4.3 Change and the introduction of new ideas, systems and processes have an enormous impact on staff. Successful change programs depend on three main issues, communication, clarity of objectives and procedures, and closure. Staff perceptions of the relevance and effectiveness of change reflect the extent that management has successfully met these three requirements.

Attitude Surveys 4.4 The last Staff Attitude Survey was undertaken in 1998. Although at that time the more recent change program had not been introduced, it is nevertheless relevant to note a number of important findings. These included a pride in working for IFAD implying a commitment to IFAD’s objectives, team spirit within work units, and the fact that a number of managers assisted innovation and performance planning. At the same time, the survey showed that there was a lack of team work between work units and a very considerable variation in


Annex 8 Page 9

managerial skills. Procedures were seen as burdensome, while morale was low and work stressful.

4.5 The IFAD Staff Association also undertook a Staff Survey in 2002. Although only about 25% of staff responded, the results indicate areas of some concern. Firstly, only 16% of respondents felt that IFAD is making the best use of their professional capacity. Secondly, about half of the respondents felt that their working environment had worsened during the previous two years.

Staff Observations to IEE 4.6 The IEE team has had many opportunities to interact with IFAD staff at many levels and have had frank exchanges that reflect the high level of motivation of most staff, but also the constraints and frustrations that affect staff in carrying out their work. The observations, summarized below, are based on informal discussions during the course of project evaluation, three open forum meetings and a number of more specific one-on-one interviews. The issues presented below are only those that were raised at different times by at least three different staff so they represent common concerns and not merely the prejudices of an individual staff member. The language used in the summary tries to accurately reflect the language used by staff. The issues discussed cannot be considered to be a statistically significant set of data, but they do provide an indication of staff concerns and factors that impinge on the efficiency of IFAD’s work. This summary does not imply that the IEE team necessarily agree with or endorse all the concerns expressed, but they do have reality for IFAD as they are based on staff perceptions.

4.7 Senior Management Approach. Senior management is seen as remote, and although communication has improved recently, there is insufficient vertical communication. This is not only a lack of top-down communication but a lack of bottom-up communication, ie. management doesn’t listen to staff. A commonly expressed concern is that senior management does not appear to focus on IFAD’s core business of project formulation and implementation, appearing to be more concerned with peripheral issues. The limited discussion of projects at Board meetings is seen as evidence of this lack of focus.

4.8 Hierarchy. IFAD’s structure is seen as flat and non-hierarchical by senior professionals, with considerable autonomy and authority for operational matters. Junior professionals and most GS staff view IFAD as very hierarchical to feudal, exacerbated by strong grade awareness. GS staff feel they lack technical or professional credibility within IFAD, even when they are working in highly specialized fields. In addition, many staff see the Strategic Change Programme and other change initiatives as a top-down imposition.

4.9 Management Skills. The conclusions of the 1998 Attitude Survey with regard to staff opinion about managers remain valid, with a concern about the slow speed of decision making and a considerable variation in managerial capabilities. Few managers were considered to have appropriate HRM skills, other than good inter-personal relationships – the nice guy syndrome.

4.10 Team Work. PDTs are the clearest example of working teams within IFAD but their effectiveness depends to a large extent on the CPM. Membership of the PDT, with the exception of the Director and Lead Adviser, is determined by the CPM and may include external consultants and partners. The nature of project-based widely dispersed core business does not readily lead to divisional team building. Teams developed across Divisions or Departments are rare, though institutional working groups come together for specific purposes and some of the thematic groups have functioned effectively. Another major constraint to building successful teams is the separation of professional and GS staff, so that GS staff are either not invited to, or do not attend divisional meetings. One of the most successful initiatives in team building has been involving support staff in field missions. This


Annex 8 Page 10

is not common practice, but those involved report a greater understanding of IFAD’s objectives and a significant improvement in morale.

4.11 Knowledge Sharing. Although knowledge management is a priority within IFAD as discussed in Section xx, few staff feel there is adequate sharing of knowledge and experience, even within a division. There is a marked separation of divisions, referred to as the “silo culture”. Knowledge sharing is also constrained by a lack of open dialogue with a perception that staff are overly sensitive to criticism, with defensive attitudes even to positive criticism. This attitude contrasts unfavourably with previous experience in other knowledge based institutions, such as research and academic organizations. It is also acknowledged that there is a considerable amount of information available through the Resource Center and the Intranet, which is inadequately or insufficiently used because of time constraints. The introduction of the document server is also thought to have reduced staff abilities to browse through documents from other parts of IFAD as access is often restricted.

4.12 The Change Agenda. There is an acknowledgement that the development of the SCP was based on a fairly participatory process, but a perception that it does not focus on IFAD’s core business and therefore has little positive impact on those engaged in that core business. There is a degree of cynicism that change programmes will be effective and a concern that introduced changes never come to closure. In addition, new initiatives are commonly seen to lack clarity of objectives, do not have adequate procedural guidelines, add to the work load and are not adequately resourced. This all adds up to feelings of change fatigue.

4.13 Operational Procedures. Staff suggest that these have not evolved to meet IFAD’s changing priorities or linked to the le ssons of experience. For example, experience of the need for frequent re-structuring of projects at MTR has not led to changes in project design or phasing. In addition, there is a continuing focus of time and money on project design and little evidence that the SCP led to any changes in the budget allocation to implementation, as opposed to design. The need for greater flexibility in project implementation with resources allocated to early implementation is a commonly expressed view. Linked to this, is considerable support for greater decentralization and an IFAD presence in-country. It is recognized that this is under consideration by management ($3m for 15 countries agreed at 12/03 Board?). This could also be linked to greater degree of direct supervision, but supervision by an effective CI, such as UNOPs is supported.

4.14 Review and Approval Process. The need to streamline the review and approval process is widely endorsed. Most projects go through a multi-stage review process, involving as many as six separate reviews, each with their own documentary requirements. Consideration is being given to merging the OSC and TRC reviews and this would be welcomed by staff as a first step in simplifying procedures. There are divergent views about the effectiveness of the TRC, some seeing it as a positive and helpful stage in project appraisal, others regarding it as a negative policing function. At present, peer review, involving staff from a different division is not obligatory but it is thought it would help to ensure quality and transfer experience across divisions. A decline in the role of the OSC is seen as an indicator of lack of management interest in the line functions.

4.15 Documentation. The documentary requirements for most IFAD processes and projects are regarded as over-elaborate. This is reflected by the fact that between 800 – 1700 pages of documents are prepared for each Board meeting. It is also widely considered that many documents meet internal IFAD requirements for design and approval, rather than the needs of implementation.

4.16 Analytical Requirements and Technical Support. There are increasing demands for detailed and high quality analytical work on poverty, institutions, economics and finance, incentives for beneficiaries, technical issues, and policies. It is also accepted that systematic problem analysis is generally weak and needs greater emphasis. The Technical Division is not able to provide the full range of expertize required to meet the varying demands, and it


Annex 8 Page 11

probably should not attempt to do so. At present, CPMs draw on consultants to provide necessary technical support, but it is thought that PT could usefully establish a panel of recognized expert consultants as a resource base. The PDT is at present the de facto technical team for many projects, but their effectiveness depends on the CPM.

4.17 Intellectual Framework. There is no institutional focus to provide intellectual understanding or rigorous analysis of IFADs priorities and programmes. The newly established Policy Division could provide this focus, but there is some concern as to whether it will have the capacity to examine fundamental problems in addressing the country specific needs of target groups, in the way that Special Programming Missions, ended in the mid 1990s, were able to do. COSOPs are not seen as providing an adequate replacement, while Regional Economists work in isolation and are unable to fulfil SPM functions. The knowledge management programme and supplementary budgets can provide some support for intellectual capital building but staff see the need for further focus on systematic work on what IFAD contributes to an understanding of poverty and the development of a range of analytical techniques to meet the needs of present project designs. Partnerships with other agencies, in particular with FAO, are seen as an effective way of meeting these needs.

4.18 Human Resources. Staff are aware of the proposed changes to HR management and the new strategy and the HR Department has kept staff informed of recent developments in the implementation of that strategy. However, there is a widespread concern as to whether the HR Department has the capacity to implement the strategy, based on perceptions of the effectiveness and attitudes of the Department in the past. There is also widespread concern over the use of temporary staff on a long term basis and the use of consultants or short-term temporary staff to undertake work normally done by regular staff. This is seen as an unjust solution to manpower constraints. By contrast, dependence on specialist consultants for project work provides the required range of expertize and an important part of a CPM’s function should be to mobilize and manage this expertize. Another issue of concern at all levels of the organization is the lack of a corporate policy on non-performing staff.

4.19 Innovation and Risk. Although IFAD stresses the importance of innovation a definition of innovation was only produced in 2001. Staff hold differing views as to what constitutes innovation, from an initiative that has never been tried before, to a well known initiative that is tried for the first time in an IFAD project in a particular country. Innovative programmes are by definition risky, but staff feel there is no clear management policy or guidance as to whether IFAD’s objective is relatively risk free resource transfer or the development of new concepts and programmes that may be inherently risky. There is also an acceptance that there is little capacity in IFAD for realistic risk analysis. Finally, for some, there is a perception that management is not concerned with risk.

5 Conclusions

5.1 The internal review of the HR function in IFAD completed in 2001 identified the lack of a human resources development strategy and a number of failings in the way human resources were managed. As a result, a new HR strategy has been developed, pilot initiatives to implement key parts of the strategy are underway or planned, and the HR division has been reorganized. The programme is ambitious. It is too early to reach conclusions about its effectiveness, but it would be useful to make comparisons with initiatives in other UN bodies.

5.2 The most significant findings from the review of staff numbers are that there has been an effective growth of 51 percent across all categories, professional, support and consultants. When compared with the loans and grants programme over the same period efficiency has fallen. Of course, this does not take account of changes in quality or the growth of policy advocacy. These issues will be considered further during the country visits. Second, there have been improvements in the gender balance of professional staff and consultants. But


Annex 8 Page 12

women are still more likely to be found in support roles and the main growth at professional level has been under Chapter X, non-permanent arrangements.

5.3 A number of more tentative conclusions can be drawn from about organizational culture, relating to the need for more participatory management, better communication, team building and knowledge sharing, review of the balance between project design and implementation, documentary requirements, technical support and intellectual capital building, and the implementation of the HR strategy. Human resources are fundamental to IFAD’s role as a knowledge based organization, so issues that impact on the way staff perceive corporate culture affects their ability to achieve their objectives and are of crucial importance. Although every effort was made to confirm that opinions recorded team during the Desk Study phase of the eva luation represented generally held views, it is clear that they may not represent the views of all staff. It is important, therefore, to assess the validity of these views for a wider sample of staff during the remaining period of the evaluation. This could be done through further interviews, focus groups and a questionnaire, ensuring that the study does not duplicate issues to be examined during the forthcoming Staff Attitude Survey.

6 FURTHER WORK

6.1 Given the central importance of human resource management for IFAD as a knowledge based organization it is recommended that work on human resource issues be continued during the third phase of the IEE. The focus of this work would be to:

• Examine the implications for HR management of IFAD’s current Strategic Framework, corporate objectives and proposals for institutional change

• Review the IFAD HR Strategy and associated documentation in relation to: o An analysis of current HR issues and problems o The need for indicators of progress of implementation of the strategy o HR strategies adopted by other development institutions

• Identify the major constraints to successful implementation and highlight any special needs

• Deepen the analysis, as data permits, of the complement of regular and temporary staff and consultants in terms of grades, distribution within IFAD, gender and continuity and their contribution to corporate objectives.

• Undertake a fuller assessment of the reasons for the significant growth in staff and consultant numbers over the decade, the appointments system, how they are financed, assess how the skill profile is changing and the implications for IFAD’s operational efficiency

• Review the observations made by staff and collected during the desk study, on the impact of corporate culture and procedures on their work and morale. Validate these observations through sampling a greater number of staff using a questionnaire or other appropriate methods