Office of Evaluation - IPC Global Platform

Mid-Term Evaluation – Final Report

Mid-Term Evaluation of the FAO-GEF Project: “Transboundary Agro-Ecosystem Management Programme for the Kagera River Basin (Kagera TAMP)” (GCP /RAF/424/GFF)

Office of Evaluation

Food and Agriculture Organization of the United

Nations

August 2013

Food and Agriculture Organization of the United Nations

Office of Evaluation (OED)

This report is available in electronic format at: http://www.fao.org/evaluation

The designations employed and the presentation of material in this information product do not imply the expression of any opinion whatsoever on the part of the Food and Agriculture Organization of the United Nations (FAO) concerning the legal or development status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. The mention of specific companies or products of manufacturers, whether or not these have been patented, does not imply that these have been endorsed or recommended by FAO in preference to others of a similar nature that are not mentioned. The views expressed in this information product are those of the author(s) and do not necessarily reflect the views or policies of FAO. © FAO 2013 FAO encourages the use, reproduction and dissemination of material in this information product. Except where otherwise indicated, material may be copied, downloaded and printed for private study, research and teaching purposes, or for use in non-commercial products or services, provided that appropriate acknowledgement of FAO as the source and copyright holder is given and that FAO’s endorsement of users’ views, products or services is not implied in any way. All requests for translation and adaptation rights, and for resale and other commercial use rights should be made via www.fao.org/contact-us/licence-request or addressed to [email protected].

For further information on this report, please contact:

Director, OED

Viale delle Terme di Caracalla 1, 00153

Rome, Italy

Email: [email protected]

Acknowledgements

This report was prepared by Jean-Joseph Bellamy, Team Leader ([email protected]) and William Critchley, Senior Evaluator ([email protected]). The Evaluation Team would like to express its gratitude and appreciation to all stakeholders interviewed. Their contributions were most appreciated, and facts and opinions they shared played a critical part in this evaluation. The Evaluation Team would also like to extend special thanks to the Project Management Team who supplied key information and key contacts. A special thank you to Sally Bunning, FAO Land and Water Division, Stefan Schlingloff, FAO-Land and Water Division, Joseph Anania, Regional Project Coordinator, Emmanuel Muligirwa, National Project Manager (NPM)–Rwanda, Salvator Ndabirorere, NPM-Burundi, Wilson Bamwerinde, NPM-Uganda, and Fidelis Kaihura, NPM-Tanzania who contributed greatly to the organization of the 2-week fact-finding mission in Burundi, Rwanda, Tanzania and Uganda. We also appreciated the guidance of Luisa Belli, FAO Office of Evaluation, OED during the course of this evaluation.

Composition of the Evaluation Team Evaluation team Jean-Joseph Bellamy, Independent Team Leader William Critchley, Independent Team Member FAO Office of Evaluation Luisa Belli, Evaluation Manager, OED

Mid-Term Evaluation of the Kagera TAMP Project – Final Report

iii

Table of Contents

Acronyms and abbreviations .............................................................................................................................. v

Executive Summary ............................................................................................................................................. vii

1. Introduction ................................................................................................................................................. 1

2. Evaluation Framework ............................................................................................................................. 1

2.1. Purpose and Scope of the Evaluation .............................................................................................................. 1

2.2. Evaluation Approach and Methodology ......................................................................................................... 2

2.2.1. Overall Approach ........................................................................................................................................ 2

2.2.2. Evaluation Instruments .............................................................................................................................. 3

2.2.3. Sequence of Assignment and Schedule ................................................................................................ 4

2.3. Limitations and Constraints ................................................................................................................................ 5

3. Context and Overview of the Project .................................................................................................... 5

4. Evaluation Findings ................................................................................................................................... 6

4.1. Relevance of the project ....................................................................................................................................... 7

4.1.1. Towards UNCCD/GEF Objectives ........................................................................................................ 7

4.1.2. Towards FAO Objectives ......................................................................................................................... 8

4.1.3. Towards Recipient Countries Objectives ............................................................................................. 8

4.1.4. Coherence of Project Concept/Design ............................................................................................... 11

4.2. Efficiency of the project and analysis of the implementation process ............................................. 14

4.2.1. Project Management Approach ............................................................................................................ 14

4.2.2. Financial Planning and Management ................................................................................................. 17

4.2.3. Quality of Technical Assistance / Use of National Capacity ...................................................... 21

4.2.4. Country Ownership / Stakeholder Participation .............................................................................. 22

4.2.5. Monitoring Approach and Progress Reporting ................................................................................ 22

4.3. Effectiveness of the Project .............................................................................................................................. 26

4.3.1. Achievements of Project’s Objectives ............................................................................................... 27

4.3.2. Attainment of Project’s Expected Outcomes ................................................................................... 28

4.3.3. Quality of Technical Content ................................................................................................................ 35

4.3.4. Special achievements .............................................................................................................................. 36

4.3.5. Gender equality ......................................................................................................................................... 36

4.3.6. Capacity Development ........................................................................................................................... 38

4.3.7. The Human- Rights Based Approach (HRBA) ............................................................................... 39

4.3.8. Partnership and alliances ........................................................................................................................ 39

4.3.9. Risks and Assumptions / Risk Mitigation Management ............................................................... 40

4.4. Potential Impacts of Project ............................................................................................................................ 42

4.4.1. Potential Impacts of Project’s Achievements .................................................................................. 42

4.4.2. Potential Global Environmental Benefits (GEBs) .......................................................................... 43

4.4.3. Impacts on Local Environment, Poverty and Other Socio-Economic Issues ......................... 44

4.5. Sustainability of Project Achievements ........................................................................................................ 44


iv

4.5.1. Sustainability of Results Achieved ..................................................................................................... 44

4.5.2. Socio-Economic Risks ............................................................................................................................ 46

4.5.3. Catalytic Role / Replicability of Project Achievements ............................................................... 46

5. Ratings ....................................................................................................................................................... 48

6. Lessons Learned ...................................................................................................................................... 49

7. Conclusions and Recommendations ................................................................................................... 50

7.1. Main Conclusions ................................................................................................................................................ 50

7.1.1. Relevance of the Project ......................................................................................................................... 50

7.1.2. Efficiency of the Project ......................................................................................................................... 51

7.1.3. Effectiveness of the Project ................................................................................................................... 52

7.1.4. Potential Impact and Sustainability of Project Achievements .................................................... 54

7.2. Recommendations ................................................................................................................................................ 54

Annex 1 Evaluation Terms of Reference (TORs) ................................................................................ 61

Annex 2 Evaluation Matrix ......................................................................................................................... 78

Annex 3 List of Documents Consulted .................................................................................................... 85

Annex 4 Discussion Guide .......................................................................................................................... 89

Annex 5 Evaluation Mission Agenda ....................................................................................................... 91

Annex 6 List of People Met ......................................................................................................................... 95

Annex 7 Project Expected Results and Planned Activities ............................................................ 102

Annex 8 Co-financing Table ..................................................................................................................... 106

Annex 9 Full List of Performance Indicators ..................................................................................... 107

List of Tables Table 1: Project Logic Model ................................................................................................................ 12 Table 2: Status of Project Funds Utilization .......................................................................................... 18 Table 3: Commitments + Disbursements Status in each Country ......................................................... 19 Table 4: Co-financing Commitments at the Outset of the Project ......................................................... 20 Table 5: Co-financing Status (June 2012).............................................................................................. 20 Table 6: List of Performance Indicators at Objective and Outcome Levels .......................................... 24 Table 7: Attainment of Project Objectives ............................................................................................. 27 Table 8: List of Kagera TAMP Main Achievements ............................................................................. 29 Table 9: Summary of Activities Accomplished in the Demonstrations at June 2013 ........................... 34 Table 10: List of Risks ........................................................................................................................... 41 Table 11: Project Mid-Term Evaluation Ratings ................................................................................... 48


v

Acronyms and abbreviations BH Budget Holder

CBO Community Based Organization

CCA Common Country Assessment

CD Capacity Development

CEO Chief Executing Officer

DPAE Direction Provinciale de l’Agriculture et de l’Elevage

EAC East African Community

FAO Food and Agriculture Organization

FBA Field Budget Authorization

FFS Farmer Field School

GEB Global Environmental Benefit

GEF Global Environment Facility

HRBA Human Rights Based Approaches

HQ Headquarter

IEM Integrated Environmental Management

K-TAMP Kagera Transboundary Agro-Ecosystem Management Programme for the Kagera Basin

LADA Land Degradation Assessment in Drylands

LDFA Land Degradation Focal Area

LOA Letter Of Agreement

LTO Lead Technical Officer

LTU Lead Technical Unit

LUS Land Use System

LVBC Lake Victoria Basin Commission

LVEMP Lake Victoria Environmental Management Project

M&E Monitoring and Evaluation

MDG Millennium Development Goal

MOU Memorandum of Understanding

MTE Mid-Term Evaluation

NAADS National Agricultural Advisory Service

NAP National Action Programme

NBI Nile Basin Initiative

NELSAP Nile Equatorial Lades Subsidiary Action Programme

NEPAD New Partnership for African Development

NGO Non-Governmental Organization


vi

NPSC National Project Steering Committee

NPM National Project Manager

NSC National Steering Committee

OED Office of Evaluation Division (FAO)

OP Operational Program

PDF (A&B) Project Development Facility

PES Payment for Ecosystem/ Environmental Services

PIF Project Identification Form

PIR Project Implementation Review

PNIA Plan National d’Investissement Agricole

PPP Private-Public Partnership

PSC Project Steering Committee

PY Project Year

QA Questionnaire on Approaches (WOCAT)

QM Questionnaire for Maps (WOCAT)

QT Questionnaire on Technologies (WOCAT)

RAB Rwanda Agriculture Board

RBM Results Based Management

RCU Regional Coordination Unit

RPC Regional Project Coordinator

RPSC Regional Project Steering Committee

SLaM Sustainable Land and Agro-Ecosystem Management

SLM Sustainable Land Management

SMART Specific, Measurable, Attainable, Relevant and Time-bound

SO Strategic Objective

SPAT Strategic Plan for Agricultural Transformation

TAC Technical Advisory Committee

TAMP Transboundary Agro-Ecosystem Management Programme

TOR Terms of Reference

UN United Nations

UNCCD United Nations Convention to Combat Desertification

UNDAF United Nations Development Assistance Framework

UNEG United Nations Evaluation Group

UNEP United Nations Environment Program

USD United States Dollar

WOCAT World Overview of Conservation Approaches and Technologies


vii

Executive Summary

ES1. This report presents the findings of the Mid-Term Evaluation of the GEF-FAO Project “Transboundary Agro-Ecosystem Management Programme For The Kagera River Basin (Kagera-TAMP)” which was conducted between May and July 2013. The purpose of this MTE was to review progress made towards achievement of outcomes in accordance with the project document, and to identify corrective actions where necessary. ES2. The Kagera River Basin is shared by Burundi, Rwanda, Tanzania and Uganda. Maintenance of the flow regime is essential for maintaining water levels of Lake Victoria and outflow to the Nile, while the wetland areas are vital to sustain water quality. The natural resources of the Kagera river basin support the livelihoods of 16.5 million people. Yet the basin’s land and freshwater resource base is threatened by land degradation, declining productive capacity of croplands and rangelands, deforestation and encroachment into wetlands. The overall environment and development goal is to support the adoption of an integrated ecosystems approach for the management of land resources in the Kagera Basin to generate local, national and global benefits. ES3. The evaluation was conducted in accordance with the guidance, rules and procedures established by FAO and GEF. It was undertaken in line with the principles of independence, impartiality, transparency, disclosure, ethicality, partnership, competencies/capacities, credibility and utility. It considered the two GEF evaluation objectives at the project level: (i) to promote accountability for the achievement of GEF objectives; and (ii) to promote learning, feedback and knowledge sharing. Findings were structured around the five internationally accepted evaluation criteria, namely: relevance, efficiency, effectiveness, impact and sustainability. ES4. The exercise provides evidence-based information that is credible, reliable and useful. The findings were triangulated through the concept of “multiple lines of evidence” using varied evaluation tools and gathering information from various stakeholders and different levels of management. The following evaluation instruments were employed: an evaluation matrix, documentation review, an interview guide, and semi-structured interviews. ES5. The approach to the MTE was based on the input of two independent consultants working as a team. During a field mission of two weeks, after jointly meeting the Regional Project Coordinator in Kigali, one visited Burundi and Rwanda, the other Tanzania and Uganda in order to interview key stakeholders and collect evaluative evidence. An initiation briefing had been given at FAO in Rome by the lead technical unit – and on return feedback was shared with the same group. A draft report was circulated amongst all stakeholders, and the evaluators took into consideration a range of comments before finalization. The MTE’s findings are summarized below in the conclusions, from which the recommendations that follow are derived. Conclusions Relevance of the Project ES6. The Kagera TAMP project is relevant to the region. The project is well-aligned with the GEF Land Degradation Focal Area’s strategic objectives as well as FAO’s vision. K-TAMP is a direct response to root causes identified during the design of this project, which revealed several governance-


viii

related and other causes of land degradation. It is embedded in the regional policy context of the Kagera River Basin and also with the enabling environments of the four countries. ES7. A detailed design that extended over a period of almost 10 years between concept and start-up. It took 10 years between the approval of a PDF-A grant in November 1999 to develop the concept, and the start-up of the project in September 2009. Despite this long period the project remained constant in its objectives and proposed methodology and evolved little. ES8. This is a project with very ambitious targets set within a relatively short timeframe. The project consists of an overall goal, an environmental objective as well as a development objective, five outcomes and 13 outputs. It is ambitious considering the allocated resources and the given timeframe. Its demonstration component covers 45 non-contiguous catchments dispersed in 21 administrative units in four countries. The duration is 4.5 years with a core, and very ambitious target, of 100,000 ha under SLM. Efficiency of the Project ES9. The administrative and financial procedures and the financial system in place to record project expenditures do not provide adequate support to project management for an efficient implementation and monitoring of the project. The channeling of financial resources to the project sites is not easy, due to geographical locations in the case of Tanzania and Uganda but also to the centralized nature of the system. Additionally, the FAO financial system records project expenditures by expense category only - such as consultants, training and travel expenses. Financial reports need to be calculated manually by allocating project expenditure to the corresponding outcome. This does not provide project managers with accurate and timely financial information. These limitations hamper the decentralization of the decision-making process to the regional and national level, and the greater empowerment of the RPC and NPMs. ES10. The project is behind schedule and risks not making enough provision for mainstreaming and replicating its achievements. The initial logic was to concentrate the first two years on the demonstrations (outcome 4) and then analyze the lessons learned and best practices, and focus the remaining part of the project on scaling-up through mainstreaming, institutionalizing and replicating achievements; including addressing transboundary issues and replicating best practices demonstrated in the demonstration sites. While there appears to have been a recent impressive pick-up in pace of implementation centered around active and enthusiastic farmer field school groups with vibrant community engagement, there are only 15 months of implementation remaining. It is urgent to start documenting this approach and mainstreaming it into national extension services. ES11. The monitoring and reporting of the project is not adequate to measure progress made at the objective and outcome level – the GEF main reporting requirement. The M&E plan is complex, costs over 6% of the GEF grant, and its cost-effectiveness is questionable. With 49 indicators to monitor it requires considerable effort to produce accurate and timely information for managers. Additionally, most indicators at the output level are simply tracking the delivery of activities. The majority of the indicators selected are specific, easy to measure, attainable and time-bound, but are not fully relevant when measuring how efficiently the project is progressing toward its overall objectives. ES12. In addition, the PIR (a GEF requirement) is the instrument to report, annually, progress made by the project. It includes a section (3) that focuses on how the project is progressing towards achieving


ix

its objectives and outcomes, using a set of 16 indicators that were identified to measure the performance at these levels. PIRs are key reports in documenting progress made by the project towards achieving its high level results. However, these reports could be improved, particularly section 3. Effectiveness of the Project ES13. Progress so far is limited though accelerating, but the outputs are poorly integrated. The project is making progress, though its achievements are still limited compared to the overall targets set. Even though there is currently a noticeable acceleration, the overall achievement to-date of just 5,350 hectares under SLM is a long distance from the (admittedly) over-ambitious final target of 100,000 hectares. The project has delivered outputs but with limited integration among them. For instance, demonstrations are taking place in the four countries with little interconnection; some work has been put in to produce land degradation maps for the countries but with no direct relation to the demonstration sites; WOCAT questionnaires have been completed but with limited added value to the activities implemented. ES14. SLM practices promoted are generally standard, well-known and proven in the region. SLM practices promoted by the project are generally standard, well-known and proven at least in the region. This is not a project that has specifically set out to experiment with and develop new technologies. Nevertheless there could have been more attention given to some technical detail and training in SLM practices. There also appears to be inadequate consideration of the potential of water harvesting other than infiltration ditches, and the role of zero-grazed livestock in the overall conservation cycle is probably undervalued. ES15. LADA-WOCAT methodologies have been piloted but have apparently not yet been fully accepted or proven to add significant value to existing approaches. Kagera-TAMP has commendably set out to pilot and pioneer the use of LADA-WOCAT methodologies. Thus LUS maps have been developed, and QM has been used to develop a series of maps of degradation and sustainable land management status and potential. LADA-local assessments have been carried out in pilot areas in each of the countries. While the current evaluation has not been able to investigate the effectiveness of these activities, there is a sense however that none of these methodologies has, as yet, been fully accepted or proven to add significant value to existing approaches. A foundation however has been laid – but a full assessment of the impact of these methodologies can only be carried out at the termination of the project. ES16. A model is emerging from the demonstrations as a holistic and integrated approach at

the community level to engage groups in sustainable agriculture and livelihoods and soil conservation. There are now “demonstrations” within 45 micro-catchments with 121 farmer field schools involved in these activities. This is the powerhouse of the project – and there is a real sense of vibrancy, and clear demand by communities for assistance with their perceived problems of environmental change and land degradation. The process of engaging communities is excellent, with the involvement of about 3,500 people (47% men and 53% women) through these FFSs. A model is emerging as a holistic and integrated approach at the community level to engage groups in sustainable agriculture and livelihoods and soil conservation; this is being achieved through careful training in FFS methodology – tailored to SLM – and the strategic use of service providers to coach the schools. This model will be a major legacy of the project and has the potential to be a key element in the replicability and sustainability of Kagera TAMP’s achievements.


x

ES17. The project focuses on the individual dimension of capacity development with insufficient emphasis on organizations, and particularly on the enabling environment. Most capacity development activities supported by the project are strongly focused on the “individual” dimension in terms of transfer of knowledge and skill development. This is part of development capacity of a system, but more emphasis needs to be put on organizations (structures and mechanisms) and particularly on the enabling environment (policy, legislation and governance). The success of the project depends on its capacity to scale-up its achievements at the Kagera River Basin level. In order to achieve this result, there is a clear need for not only developing the capacity of people involved in the management of the Kagera River Basin, but also improving the capacity of relevant organizations and developing an adequate enabling environment for the basin including the necessary policy, legislation and governance frameworks. Potential Impact and Sustainability of Project Achievements ES18. Currently, the project is too focused on its demonstrations; it is time to shift focus to scaling-up project achievements in order to improve the long-term sustainability prospects. The success of the project depends closely on the mainstreaming and replication of its achievements. As stated in the project document, “the success of the project over the four and a half year funding period and in the medium to long term across the Kagera River Basin, is highly dependent on the widespread replication of successful outcomes and lessons learned from the target communities and micro-catchments, where on the ground activities will be established and tested in the initial 2 years and subsequently scaled up”. Widespread replication is the most critical success criterion for this project. So far the project has focused on the demonstrations of SLM practices and approaches in selected pilot areas in the four countries. These demonstrations are the first step in catalyzing these practices and approaches, which has been further reinforced by awareness raising activities and training of relevant people involved in the management of these agro-ecosystems. However, achievements planned for the first 24 months have only been realized by the 39th month – and at times of the evaluation field mission there are only 15 more to go. Recommendations ES19. Based on the findings of this final evaluation, the following recommendations are suggested.

Recommendation 1: Three reviews should be conducted before the end of the project:

• Investigate the effectiveness of the LADA-WOCAT methodologies;

• Document the innovative and successful FFS approach used by the project;

• Document the full extent of the area covered by SLM/SLaM under the project.

ES20. The current evaluation has not been able to investigate the effectiveness of the LADA-WOCAT methodologies. However, there is a sense that none of these methodologies has as yet proven to add significant value to existing approaches – though not all activities are completed. Thus this impression should be tested by conducting a review of effectiveness before the end of the project. ES21. A second review should focus on documenting the innovative and successful FFS approach used by the project. This is a non-classical, modified version of the farmer field school model. Thus the original concept of a “seed-to-seed” or “egg-to-egg” model underpinning an ephemeral group has been developed into a longer lasting/semi-permanent group that tracks the benefits of SLM (and associated


xi

economic land-based enterprises) over a number of seasons. The other key difference from the classical model is the way that the FFSs combine participatory research with simple demonstration and learning. ES22. A third review is recommended to document the full extent of the area covered by SLM under demonstrations supported by the project at/ projected for end of the project. It is crucial for the terminal evaluation to have these demonstrations properly documented. This is the core of the project and the basis on which scaling-up should happen. The review should cover extent - and analyse the process also.

Recommendation 2: More focus needs to be given to developing the capacity of related organizations and of an enabling environment to provide the adequate policy, legislation and

governance frameworks.

ES23. So far, most capacity development activities supported by the project have been focused on the “individual” dimension in terms of transfer of knowledge and skill development. This is part of the overall capacity of a system but more emphasis needs to be put on organizations (structures and mechanisms) and particularly on the enabling environment (policy, legislation and governance). The steering committees (regional and national) have a responsibility to guide the project in this respect.

Recommendation 3: A planning exercise should be conducted, including the development/

refinement of:

• A replication/scaling-up strategy where entry points and specified activities are identified;

• A work plan for the entire remaining implementation period with the associated budget;

• A project exit strategy; particularly for disengagement of support in demonstration sites.

ES24. While it is recognized that Kagera-TAMP has plans to enter into a faster tempo of on-the–ground activities, the project indeed needs to shift gears and focus on scaling-up its achievements. At the time of the evaluation field mission, there are 15 months remaining and a budget of approximately USD 2.85M remaining. This review indicates that there are several entry points where SLM could be mainstreamed, such as national extension services, local development planning, land use planning, local bye-laws, and through its partnerships with regional initiatives such as NBI and LVEMPII. The project needs now to focus on lessons learned, increase its visibility nationally and regionally and develop its strategy to replicate its achievements. ES25. A work plan with a budget for the remaining implementation period should be prepared and presented at the next RPSC. Key to on-the-ground achievement and long term sustainability is capitalization on the current momentum combined with better targeting of resources. There are several elements to bring to a closure. These include the need to (a) summarize transboundary studies and make recommendations for greater regional cooperation; (b) publish and handover the LUS and QM maps to the relevant national organizations; (c) finalize involvement in the PES areas; and (d) plan the exit from the pilot sites. ES26. Finally, because there was no planned provision for the post-project period, it is important to now develop an exit strategy to identify which activities should be supported in the pilot sites to ensure sustainability.


xii

Recommendation 4: Specific project activities should be managed as follows:

• No new PES activities be established;

• The LUS and QM maps be handed over through a three step process;

• The potential for an informal GEF partnership be investigated in Uganda

ES27. Initial PES activities took place in Burundi and Rwanda. Considering the limited implementation time left the project should limit its direct engagement in this area. It is recommended to pursue what was started in Burundi and Rwanda and try to partner with Vi-Agroforestry on their soil carbon initiative if possible. ES28. LUS and QM maps have been produced for the four countries. These maps contain about 80 layers of information and have been developed using ArcGIS software. The maps are now ready to be handed over to national partners. The recommendation includes (1) a brief assessment to identify the best organizations to take over these maps; (2) the publishing of these maps in partnerships with relevant national organizations; and (3) the handover through national workshops to showcase these products. It is noted that the publication of these maps with relevant national organizations is already a plan in place. ES29. In Uganda there are two other GEF Land Degradation Focal Area projects currently underway within the Kagera basin – one under UNDP, the other under UNEP. It is well worth investigating whether a GEF “partnership” could be developed at the national level to encourage synergies.

Recommendation 5: The project period should be extended, if the budget permits.

ES30. At the time of this evaluation, it is difficult to assess how much (if any) budget will remain by the end of the project (August 2014), the figures show that 55% of the budget has been consumed so far versus 72% of the time. However, if budget projections indicate funds unspent by August 2014, a no-cost extension of the project is recommended with a firm focus on more mainstreaming, institutionalizing and replicating activities.

Recommendation 6: FAO should decentralize decision-making, including financial approval, in countries and provide accurate and timely financial information by outcome to project

managers.

ES31. The process of utilizing project financial resources is procedurally centralized at HQ with the Budget Holder playing a major role in the decision making to allocate project resources. However, the experience in Burundi – through FBAs - may provide a decentralized solution, which would reinforce the empowerment of project staff and imply a greater role of the RCU in decision-making. The current FAO financial system does not provide managers with accurate and timely financial information: FAO should provide its project managers with financial information by GEF project outcome.

Recommendation 7: Backstopping from FAO HQ should focus more on general progress and

support, and less on technical training sessions at this stage of the project.


xiii

ES32. Much of the support to the project from FAO HQ to-date has taken the form of technical training in various methodologies (LADA-WOCAT; PES etc). However, it is now clear that, with a focus in the last stage of the project period on scaling-up and mainstreaming, there is a need for general backstopping support both at field activity level, and at the national and regional levels. This backstopping should be demand-driven and not too rigorously pre-planned, to allow for responsiveness in the field.

Recommendation 8: The engagement of stakeholders should be urgently improved through

more cross visits for intra-national and regional learning at all levels and through more

frequent RPSC and NPSCs meetings, starting with a RPSC to review the MTE findings and its recommendations

ES33. The project has so far failed to create a true “regional” identity for itself, and appears more like four independent but parallel country programmes. One of the reasons is insufficient regional interaction. A regional project needs to capitalize on its potential for synergy by sharing knowledge and ideas. This can be achieved, at least partially, through cross-learning between partners. Furthermore, within the countries, there is inadequate cross-learning between communities. While budgetary constraints are acknowledged, it is imperative that cross-visits are stepped up. A programme should be drawn up. ES34. There are too few steering committees meetings. NPSCs should meet twice a year, ideally one meeting to review and endorse work plans and a second to review the progress made and address urgent issues. While cost is a constraint, creativity needs to be employed to find ways of bringing members together. For example by making use of opportunities such as other meetings or fora when a quorum of members may (by coincidence) be present. The timing of the RPSC meetings should also take into account the timing of the NPSCs. Following this MTE, a RPSC meeting should be called to review the findings and recommendations and discuss an overall work plan for the remaining implementation period.

Recommendation 9: The set of performance indicators needs to be revised; including the

identification of a few specific capacity indicators and all related targets used to measure the

progress of the project.

ES35. The performance framework (a set of 16 indicators) identified at the outset to monitor the high level results (objectives and outcomes) includes a list of targets by the end of the project that will not be met. It is particularly the case for targets with clear quantified values such as indicator #2 where 100,000 ha will not be achieved (currently about 5,350ha), indicator #7 where 68 communities will not have been identified and agreed priority policy, legal and transboundary issues; and, indicator #13 where SLM practices will not be implemented in 200 pilot communities (currently in 46 communities). These targets were too ambitious from the outset of the project and they need to be revised. ES36. The project is running out of time and instead of struggling to meet these targets, it is recommended that this set of targets is revised realistically, based on achievements and the current trend of implementation. Thus the project should give thought to downsizing its overall target for coverage with SLaM/ SLM from 100,000 to something in the range of 20,000 – 25,000 hectares.


xiv

ES37. In the meantime, Kagera-TAMP is expected to generate a range of global environmental benefits across the Kagera River Basin: this is explicit in its environmental objective and the rationale of GEF-funding. While these are notoriously difficult to assess, more effort should be made to infer these benefits through data being collected on area under SLM. Some data – even simple data at community level – needs to be picked up on biodiversity in rehabilitated landscapes and riverbank. Hydrological data, should be collected from secondary sources even if they are only used as a baseline for the future. ES38. The revision of the set of performance indicators should also include the identification of a few specific indicators to measure how well the project has been developing the capacities needed to address the causes of land degradation and restore ecosystem health. Finally, this recommendation also includes the strengthening of the annual progress reporting in the PIRs, particularly section 3, which is the main measurement tool to assess progress made by the project.


1

1. Introduction

1. This report presents the findings of the Mid-Term Evaluation of the GEF-FAO Project “Transboundary Agro-Ecosystem Management Programme For The Kagera River Basin (Kagera-TAMP)”. This mid-term evaluation was implemented between May and July 2013. 2. This mid-term evaluation report documents the achievements of the project and includes seven chapters. Chapter 1 introduces the report; chapter 2 briefly describes the objective, scope, evaluation users, methodology, and limitations of the evaluation; chapter 3 presents the context and an overview of the project; chapter 4 presents the findings of the evaluation. Conclusions, lessons learned and recommendations are presented in Chapters 5, 6 and 7 respectively and relevant annexes are found at the end of the report.

2. Evaluation Framework

3. This mid-term evaluation (a requirement of FAO and GEF procedures) was initiated by FAO Headquarters as the GEF Implementing Agency. This evaluation provides an in-depth assessment of project achievements and recommendations for the remaining implementation period of the project and also for other similar GEF-financed projects in the region and possibly worldwide.

2.1. Purpose and Scope of the Evaluation

4. The purpose of this mid-term evaluation (MTE), as stated in the Evaluation TORs (see Annex 1), was to review the progress made towards achievement of outcomes in accordance with the full project document and CEO endorsement and to identify corrective actions if necessary. The MTE assessed the project from its concept and design to current and potential results. The evaluation also includes several recommendations for more effectively and efficiently replicating successful project initiatives or for filling gaps not covered yet by the project. The evaluation also provides a basis for learning and accountability for managers and stakeholders. 5. More specifically, the evaluation:

• Reviewed the effectiveness, efficiency and timeliness of project implementation; • Analyzed effectiveness of implementation and partnership arrangements; • Identified issues requiring decisions and remedial actions; • Identified lessons learned about project design, implementation and management; • Highlighted technical achievements and lessons learned; • Analyzed whether the project is on track with respect to achieving the expected results; and • Proposed mid-course corrections and/or adjustments to the Work Plan as necessary.

6. Below is a summary of the elements that were covered by this evaluation. Each element was assessed and those marked with an “*” were rated as per the TOR. These elements are:

• Relevance of Concept and Design o Relevance of the initiative (*) o Clarity, Coherence, and Realism of the logframe and of the design

• Effectiveness of Outputs and Outcomes (*)


2

• Efficiency and Effectiveness of Project Implementation Process (*) o Assessment of project management o Institutional setup o Assessment of financial resources management (*)

• Analysis of the application of the UN common country programming principles, cross-cutting themes, and of the Humanitarian Principles and Minimum Standards in the case

of emergency projects o Analysis of gender mainstreaming for gender equality o Analysis of the Capacity Development dimension in the design, implementation and

results of the project, at individual, organizational and enabling environment levels o Analysis of the adoption of the Human-Rights Based Approach o Analysis of Partnerships and Alliances o Analysis of how environmental impacts were taken into consideration and addressed

• Impact (*) o Overall impact of the project, actual or potential, positive and negative, produced

directly or indirectly, intended or unintended o Overall contribution of the project to FAO Country Programming Frameworks,

Organizational Result/s and Strategic Objectives, as well as to the implementation of the corporate Core Functions

• Sustainability (*) o The prospects for sustaining and upscaling the project's results by the beneficiaries and

the host institutions after the termination of the project 7. As per the TORs, the evaluation also assessed the compliance with the following UN Common Country Programming Principles: Human Rights Based Approaches (HRBA), Right to Food, Decent Work, Gender Equality, Environmental Sustainability, Capacity Development and Results Based Management.

2.2. Evaluation Approach and Methodology

8. The methodology that was used to conduct this mid-term evaluation complies with international criteria and professional norms and standards; including the norms and standards adopted by the UN Evaluation Group (UNEG).

2.2.1. Overall Approach

9. The evaluation was conducted in accordance with the guidance, rules and procedures established by FAO and GEF. It was undertaken in line with GEF principles, which are: independence, impartiality, transparency, disclosure, ethical, partnership, competencies/capacities, credibility and utility. It considered the two GEF evaluation objectives at the project level: (i) promote accountability for the achievement of GEF objectives; including the global environmental benefits; and (ii) promote learning, feedback and knowledge sharing on results and lessons learned among the GEF and its partners. 10. The Evaluation Team developed evaluation tools in accordance with the FAO and GEF policies to ensure an effective project evaluation. The evaluation was conducted and the findings were


3

structured around the GEF five major evaluation criteria; which are also the five internationally accepted evaluation criteria set out by the Development Assistance Committee of the Organization for Economic Co-operation and Development. These are:

• Relevance: an overall assessment of whether the project is in keeping with donors and partner policies, with national and local needs and priorities as well as with its design.

• Efficiency: a measure of the productivity of the project intervention process, i.e. to what degree the outcomes achieved derive from efficient use of financial, human and material resources. In principle, it means comparing outcomes and outputs against inputs

• Effectiveness: a measure of the extent to which formally agreed expected project results (outcomes) have been achieved, or can be expected to be achieved.

• Impacts: the long-term results of the project and include both positive and negative consequences, whether these are foreseen and expected, or not.

• Sustainability: an indication of whether the outcomes (end of project results) and the positive impacts (long term results) are likely to continue after the project ends.

11. In addition to the FAO and GEF guidance for project evaluation, the Evaluation Team applied to this mandate its knowledge of evaluation methodologies and approaches and its expertise in global environmental issues. The Evaluation Team also applied several methodological principles such as (i) Validity of information: multiple measures and sources were sought out to ensure that the results are accurate and valid; (ii) Integrity; and (iii) Respect and anonymity: All participants had the right to provide information in confidence. 12. Finally, the Evaluation Team conducted evaluation activities, which are independent, impartial and rigorous.

2.2.2. Evaluation Instruments

13. The evaluation provides evidence-based information that is credible, reliable and useful. The findings were triangulated through the concept of “multiple lines of evidence” using several evaluation tools and gathering information from different types of stakeholders and different levels of management. To conduct this evaluation the following evaluation instruments were used: 14. Evaluation Matrix: An evaluation matrix was developed based on the evaluation scope presented in the TOR, the project log-frame and the review of key project documents (see Annex 2). This matrix provided overall directions for the evaluation and was used as a basis for interviewing people and reviewing project documents. 15. Documentation Review: The Evaluation Team conducted a documentation review during the field mission and at home offices (see Annex 3). In addition to being a main source of information, documents were also used as preparation for the mission of the Evaluation Team. A list of documents was identified during the start-up phase and further searches were done through the web and contacts. The list of documents to be reviewed was completed during the mission. 16. Interview Guide: Based on the evaluation matrix, an interview guide was developed (see Annex 4) to solicit information from stakeholders. As part of the participatory approach, the Evaluation Team ensured that all parties view this tool as balanced, unbiased, and structured.


4

17. Mission Agenda: An agenda for the mission of the Evaluation Team to Rwanda, Burundi, Uganda and Tanzania was developed during the preparatory phase (see Annex 5). The list of Stakeholders to be interviewed was reviewed, ensuring it represents all project Stakeholders. Then, interviews were planned in advance of the mission with the objective of having a well-organized and planned mission to ensure a broad scan of Stakeholders’ views during the limited time allocated to the mission. 18. Interviews: Stakeholders were interviewed (see Annex 6). The semi-structured interviews were conducted using the interview guide adapted for each interview. All interviews were conducted in person with some follow-up using emails when needed. Confidentiality was guaranteed to the interviewees and the findings were incorporated in the final report. 19. Achievement Rating: The Evaluation Team rated project achievements according to the guidance provided in the TORs. Using the GEF six point scale system, the Evaluation Team rated the list of items provided in the TORs and an overall rating will be given.

2.2.3. Sequence of Assignment and Schedule

20. On the basis of the TORs, the Evaluation Team conducted the assignment in four phases as presented in the table below; including the 2.5 weeks field mission during the period April 28th to May 16th, 2013.

Phase / Task Deliverable (*) Apr. 22

Apr. 29

May 6

May 13

May 20

May 27

June 3

June 10

June 17

June 24

July 1

I. Inception Phase � Collect and review programme documents � Elaborate and submit Inception Report � Prepare mission: agenda and logistic

*

II. Mission / Collect Information � Mission to Rwanda, Burundi, Uganda and

Tanzania for the Evaluation Team � Interview key-stakeholders and conduct

field visits � Further collect of programme related

documents � Mission debriefings at FAO-HQ

III. Analyse Information � In-depth analysis and interpretation of data

collected � Follow-up interviews (if necessary) � Elaborate and submit Draft Evaluation

Report

*

IV. Finalize Evaluation Report � Circulate draft report to client � Integrate comments and submit Final

Report

*


5

2.3. Limitations and Constraints

21. The approach for this mid-term evaluation was based on a planned level of effort of 55 days, including a 2.5 weeks field mission to Burundi, Rwanda, Tanzania and Uganda to interview key stakeholders and collect evaluative evidence. Within the context of these resources, the Evaluation Team was able to conduct a detailed assessment of actual results against expected results and to successfully ascertain whether or not the project has been meeting its main objective - as laid down in the project design document - and whether or not the project initiatives are, or are likely to be, sustainable after completion of the project. It also made recommendations that should be useful for the remaining implementation period and to reinforce the long-term sustainability of project achievements. Finally, it also contained lessons learned and best practices, which could be further taken into consideration during the development and implementation of other similar GEF projects in the region and elsewhere in the world.

3. Context and Overview of the Project

22. The Kagera River Basin is shared by Burundi, Rwanda, Tanzania and Uganda. Maintenance of the Kagera flow regime is vital for maintaining water levels of Lake Victoria (25-30% of lake annual inflow) and outflow to the Nile (7 of 26 billion m3 annual est. outflow), while the riverine wetland areas are vital for allowing deposition of eroded sediments and nutrients and hence maintaining water quality for aquatic life and associated human livelihoods. The natural resources of the Kagera river basin support the livelihoods of some 16.5 million people, the majority being rural and depending directly on farming, herding and fishing activities. However, the resource base and the ecosystems are facing increasing pressures as a result of rapid population growth, agricultural and livestock intensification due to progressive reduction in farm sizes and unsustainable land use and management practices. The basin’s land and freshwater resource base, associated biodiversity and populations whose livelihoods and food security depend on those resources, are threatened by land degradation, declining productive capacity of croplands and rangelands, and deforestation and encroachment into wetlands. 23. Several barriers exist preventing an adequate sustainable management of land resources. These include:

• Limited government support and lack of incentives; • Inadequate policies, laws and regulations and their enforcement; • Poor extension services; • Local government land resources planning capacity remains weak; • Activities remain uncoordinated; • Lack of awareness and understanding of land users and local governments; and • Weak collaboration between the governments, civil society and private sector.

24. The overall long-term environment and development goal of the project is to support the adoption of an integrated ecosystems approach for the management of land resources in the Kagera Basin which will generate local, national and global benefits including: restoration of degraded lands, carbon sequestration and climate change mitigation, agrobiodiversity conservation and sustainable use, protection of international waters and improved agricultural production, food security and rural livelihoods.


6

25. The environmental objective of the project is to address the causes of land degradation and restore ecosystem health and function and, generate a range of global environmental benefits across the Kagera basin through the introduction of adapted agro-ecosystem management approaches. 26. The development objective is to improve the livelihoods and hence contribute to reduced poverty of rural communities in the Kagera Basin through more productive and sustainable resource management practices that are technically feasible and socio-economically viable. 27. In order to reach the above closely inter-related development and environment objectives, the project is expected to achieve the following outcomes:

• Outcome 1: Transboundary coordination, information sharing and monitoring and evaluation mechanisms operational and effective in promoting sustainable, productive agro-ecosystems and restoration of degraded lands.

• Outcome 2: Enabling policy, planning and legislative conditions are in place to support and facilitate the sustainable management of agro-ecosystems and the restoration of degraded land.

• Outcome 3: Capacity and knowledge are enhanced at all levels for the promotion of – and technical support for – sustainable management of land and agro-ecosystems in the basin.

• Outcome 4: Improved land and agro-ecosystem management practices are implemented and benefiting land users for the range of agro-ecosystems in the basin.

• Outcome 5: Project management structures operational and effective. 28. The Kagera TAMP project was planned to be a 4.5 years project with 2 phases: (1) two years to pilot SLM1 practices at catchment and Farmer Field School (FFS) levels in each of the 21 districts; (2) 2 years to scale-up successful practices and approaches through developing and testing enabling policy and planning tools at various levels. The project is funded by the GEF with a grant of USD 6.4M and an estimated total co-financing of about USD 24.5M from the four countries, FAO and other Partners. The project was endorsed by the GEF-CEO in May 2009, became operational in March 2010 after all countries signed the project document and should be completed in August 2014.

4. Evaluation Findings

29. This section presents the findings of this mid-term evaluation, which are based on a desk review of project documents and on interviews with key project informants and project staffs. As described in Section 3.4.1 they are structured around the internationally recognized five major evaluation criteria: Relevance, Efficiency, Effectiveness, Impact and Sustainability.

1 Under the project, the terms SLM (sustainable land management) and SLaM (sustainable land and agro-ecosystem management) are often used interchangeably. This report does not attempt to untangle the two.


7

4.1. Relevance of the project

30. Within the context of the UNCCD implementation, the project seeks to demonstrate an integrated ecosystems approach for the management of land resources in the Kagera Basin, which should generate local, national and global benefits. This section discusses the relevance of the project; including the relevance of its original design.

4.1.1. Towards UNCCD/GEF Objectives

31. As stated in the project document, the Kagera TAMP project was initially designed to be consistent with the objectives of the GEF-3 Operational Program on Sustainable Land Management (OP#15). In rescheduling the project under GEF-4, efforts were made to ensure that the project design was consistent with the objectives of the “Land Degradation Focal Area Strategy and Strategic Programming for GEF-4”. This strategy sought to build a policy and institutional environment conducive to prevention and control of land degradation and effective actions on the ground. It had 2 Strategic Objectives (SOs): (1) to develop an enabling environment that will place Sustainable Land Management (SLM) in the mainstream of development policy and practices at the regional, national, and local levels; and (2) to upscale SLM investments that generate mutual benefits for the global environment and local livelihoods; and it was to be implemented through three strategic programmes: (a) supporting sustainable agriculture and rangeland management; (b) supporting sustainable forest management in production landscapes; and (c) investing in innovative approaches in SLM. 32. The review confirmed that the project is well aligned with this strategy; it contributes to the two strategic objectives presented above. The project contributes to the Strategic Objective (SO)-1 as a catalyst among related institutions in all four countries to overcome barriers to SLM, and develop institutional and human capacity for land use/resources planning and incentive/support mechanisms to promote wider SLM adoption. The studies and demonstrations implemented during the first phase of this project should lead to a better harmonization of the sectoral policies and regulatory frameworks, securing the financing of SLM. Under SO-2 the project demonstrates and seeks to upscale successful, innovative and cost-effective SLM practices and investments that reduce land degradation and deforestation issues, enhance the productivity and the resilience of agricultural systems and generate socio-economic/livelihood benefits for local land users as well as global environmental benefits. At this point, the project particularly contributes to the expected outcome under this SO that is “communities benefit from applying and disseminating SLM practices”. 33. Its interventions are also well aligned with the strategic programmes (a) and (c) presented above. The project’s main focus is to “support sustainable agriculture and rangeland management”. It seeks to restore the health and functioning of the different agro-ecosystems in the Kagera basin through promoting sustainable land and agro-ecosystem management. SLM is promoted to overcome the severe soil erosion and loss of fertility through the use of a landscape approach and integrating ecosystem-based concerns with human land use activities (agriculture, rangeland, forest/tree management). Project interventions address root causes and negative impacts of land degradation on ecosystem stability, functions and services as they affect local people’s livelihoods. The project is also “investing in new and innovative approaches in sustainable land management” that encourage the wide adoption of SLM practices at the community level.


8

34. In addition, as stated in the project document, the Kagera TAMP project is part of the umbrella TerrAfrica/SIP programme for SLM in Sub-Sahara Africa. It is particularly well aligned with the “Activity Line 3 – Investments” of TerrAfrica, which seeks to create a closer cooperation – at the country level - between ministries and departments that are responsible for SLM-related activities and those sections of government responsible for setting national development priorities.

4.1.2. Towards FAO Objectives

35. Following the drafting of the FAO Strategic Framework in 2009, a revised version of this framework as well as the related Medium Term Plan 2014-2017 were last reviewed by the Council of FAO in December 2012. The FAO’s vision is “of a world free of hunger and malnutrition where food and agriculture contributes to improving the living standards of all, especially the poorest, in an economically, socially and environmentally sustainable manner”. The set of core functions in this framework is made up of five (5) strategic objectives (SOs). They represent the main areas of work on which FAO will concentrate its efforts in striving to achieve its Vision and Global Goals. There are:

• Eradicate hunger, food insecurity and malnutrition; • Increase and improve provision of goods and services from agriculture, forestry and fisheries in

a sustainable manner; • Reduce rural poverty; • Enable more inclusive and efficient agricultural and food systems at local, national and

international levels; and • Increase the resilience of livelihoods to threats and crises.

36. Within the context of these five SOs, the project clearly falls under the SO-2 that is to “increase and improve provision of goods and services from agriculture, forestry and fisheries in a sustainable manner”. The Kagera TAMP project is particularly relevant under the organizational outcome of this SO-2 that is “Producers and natural resource managers adopt practices that increase and improve the provision of goods and services in agriculture, forestry and fisheries in a sustainable manner”. Through the demonstration of innovative and cost-effective SLM practices and investments that reduce land degradation and deforestation issues, enhance the productivity and the resilience of agricultural systems and generate socioeconomic/livelihood benefits for local land users, the project contributes to improving frameworks for economically, socially and environmentally sound production systems that embody resource use efficiency, resilience, ecosystem services for agricultural production and accessibility. It also contributes to the development, sharing and adaptation of locally-relevant technologies and approaches with a range of partners, concentrating on addressing the barriers to the adoption at larger scale. Finally, the project supports the development of capacities to access and use evidence to support policy and planning decisions.

4.1.3. Towards Recipient Countries Objectives

37. The project is a response to root causes identified during the design of this project, which revealed several governance-related causes of land degradation; they include:

• Limited government support and lack of incentives for natural resources management; • Inadequate policies, laws and regulations; • Poor extension service; • Weak local government land resources planning capacity; and


9

• Lack of awareness and understanding of land users and local government. 38. At the design stage, Kagera TAMP was well aligned with the regional policy context related to the Kagera River Basin. It included the alignment with the following entities/programmes:

• Nile Basin Initiative (NBI), particularly the Nile Equatorial Lakes Subsidiary Action Programme (NBI-NELSAP) that included a set of investment projects – particularly the Transboundary Integrated Water Resources Management Project of the Kagera River Basin - in poverty-focused development and strengthened cooperation at sub-regional level seeking benefits from cross-border cooperation;

• East African Community (EAC) that provided a framework for extensive political cooperation and integration;

• Its related Lake Victoria Basin Commission (LVBC) managing the entire Lake Victoria Basin area; including the Lake Victoria Environmental Management Project – Phase II (LVEMP-II), which is a transboundary project designed to: (1) improve collaborative management of transboundary natural resources of Lake Victoria Basin; and, (2) reduce environmental stress in the targeted pollution hotspots and selected degraded sub-catchments as a means of improving the livelihoods of communities who depend on the natural resources of the Basin.

• Environment Programme and Action Plan of the New Partnership for African Development (NEPAD); particularly the Comprehensive Africa Agriculture Development Programme as a key entry point for integrating SLM in agriculture and natural resources management but also in mainstreaming SLM in national priorities of poverty eradication, improved food security, accelerated economic growth and development, promotion of women in development and Millennium Development Goals (MDGs).

39. In order to consolidate these alignments, Kagera TAMP has been using the MOU approach. It signed an MOU with NBI in May 2012 to define their collaboration in the Kagera River Basin. In addition to some general clauses to define the collaboration, the instrument defines the scope of cooperation between the two entities and the areas of cooperation, which include:

• Participation of the NBI-NELSAP in the Kagera TAMP basin-wide coordination efforts to facilitate transboundary dialogue, basin-level planning, policy harmonization and coordination of national/sub-national actions;

• Participation of the NBI-NELSAP as Observer in the activities of the Regional Project Steering Committee of the Kagera TAMP;

• Fostering a coordinated Kagera environmental monitoring and information system; • Promoting technical exchange visits and joint capacity building activities in the agriculture,

environment, land and water sectors, and other areas of common interest between the Parties; • Optimize knowledge sharing (including sharing experiences and lessons learnt, TORs, and

study reports) regarding agriculture, livestock, water, land and forestry sectors, natural resources diagnostics, experiences of participatory planning, especially those related to land and natural resources management at a local to a basin scale; and other areas of common interest between FAO and NBI-NELSAP2;

• Advice and support by NBI-NELSAP on hydrological monitoring of micro-catchments, knowledge sharing, and use of existing infrastructure for demonstrating positive effects of sustainable land management measures on reducing erosion and sedimentation;

• Collaboration between FAO and NBI-NELSAP in the review of term of references, technical 2 One important outcome of the collaboration with NBI-NELSAP was that they made available to the project its monograph and its “Kagera Basin Information and GIS Database” containing over 27GB of data that was used in the baseline analysis and LUS mapping.


10

documentation and identification of resource persons/experts and implementation of studies of common interest of the Parties.

40. A similar MOU was also signed with Vi-Agroforestry on October 2012. This instrument focuses on giving priority to issues concerning management of natural resources and agro-ecosystems and to knowledge generation and dissemination including programmes, projects and activities related to the sustainable management of land and agro-ecosystems. Finally, another MOU is in the process of being signed with the LVBC for collaborating in the Kagera River Basin with the LVEMP-II project. A draft was reviewed but it has not been signed yet. 41. The project design is also well aligned with national strategies and action plans that seek to address sustainable management of natural resources, biodiversity conservation, agriculture, forests, desertification and climate change mitigation in the region. Regarding land degradation, all 4 countries ratified the United Nations Convention to Combat Desertification (UNCCD) in the 1990s and subsequently developed their National Action Programmes (NAPs). Furthermore, decision-makers in these countries also recognized that in order to address the issues of land degradation across the basin, which are transboundary in nature, it requires coordination and collaboration among countries and sectors, as well as good coherence among the various national strategies and action plans. To this effect, an initial list of key/critical transboundary issues was identified at a regional meeting in November 2005 with the support of the PDF-B phase of this project. The Regional Project Steering Committee (RPSC) subsequently reviewed these issues at the outset of this project and decisions were made to address some of them with the support of Kagera TAMP. 42. At the national level, the project operates within the context of a set of related national laws, policies, programmes and projects and also institutions. It includes the respective National Environment Action Plans (NEAPs), the Poverty Reduction Strategies and Programmes or equivalent such as the Economic Development and Poverty Reduction Strategy (EDPRS-II) in Rwanda, the “National Agricultural and Livestock Strategies and related plans/programmes such as the Strategic Plan for Agricultural Transformation (SPAT-II)” in Rwanda, the “Plan Directeur de la Recherche et d’une Stratégie des Aménagements des Bassins versants et marais” in Burundi, the “Plan National d’Investissement Agricole (PNIA)” in Burundi. Other related parts of the enabling environment for SLM include land laws, environmental laws, environment/forest code, forest policy, forestry action plan, action plan on soil fertility, policy for the conservation and management of wetland resources, etc. Finally, these enabling environments in the 4 countries also include agriculture extension services such as the Rwanda Agriculture Board (RAB), the Direction Provinciale de l’Agriculture et de l’Elevage (DPAE) in Burundi, the National Agricultural Advisory Service (NAADS) from the Ministry of Agriculture, Animal Industries and Fisheries in Uganda, and the Ministry of Agriculture’s Extension Service in Tanzania, land use planning units, and other organizational entities related to the sustainable use of land. 43. The review of the project strategy and of national contexts indicates a good alignment between the project and the respective enabling environments. However, it is still too early to assess the impacts of the project on these enabling environments (see also Section 4.4). So far, the implementation of the project has focused on the implementation of demonstrations (outcome 4), conducted several studies and produced land degradation maps for the 4 countries. However, the project has not been yet in a position to advise and submit any proposals for mainstreaming SLM into the enabling environments of these countries. The review indicates that there are several entry points where sustainable management of land and agro-ecosystems could be mainstreamed at both the national and regional/local levels.


11

Considering the remaining timeline of the project, it should become one priority within the current and future work plans. It is a critical result to achieve for maximizing the long-term sustainability of project achievements.

4.1.4. Coherence of Project Concept/Design

44. The process to design this project was long and cumbersome. From the documentation reviewed for this evaluation, the following milestones were noted:

• A GEF PDF-A grant of USD 25,000 requested by UNEP was approved in November 1999 to prepare a PDF-B proposal with national endorsements;

• The project entered the GEF Pipeline in June 2003; • A PDF-B grant request was sent jointly by UNEP (Implementing Agency) and FAO (Executing

Agency) to GEF in September 2003 for an amount of USD 700,000 to be funded under GEF-3. The objective of this PDF-B was to produce a full project brief to be prepared through three sub-objectives: (1) Develop collaborative mechanisms; (2) Conduct a transboundary diagnostic analysis (TDA); and (3) Develop a framework and action-oriented implementation strategy for the Transboundary agro-ecosystem management plan (TAMP);

• A project brief (also called project document) was finalized in 2005-2006 with the endorsement of the four countries in March 2003. This project document was still with the participation of UNEP as the GEF Implementing Agency and the FAO as the GEF Executing Agency. The project was to start in May 2007 for 5 years until May 2012; it was to be funded under GEF-3;

• Following the rescheduling of the project under GEF-4 due to lack of funds under GEF-3, a Project Identification Form (PIF) was submitted to GEF in October 2007 with FAO as the GEF Implementing and Executing Agency. This PIF was then revised and approved by GEFSEC in March 2008;

• The project was endorsed by the GEF-CEO on May 5, 2009 with a grant of USD 6,363,700; • The Kagera TAMP project finally became operational in March 2010 for 4.5 years to be

completed in August 2014. 45. Based on the above, it took almost 10 years and USD 725,000 to design the project; from the initial concept of the project to the effective starting date. During this rather long phase, UNEP and FAO teams conducted an extensive review to position the project. It included an extensive analysis of the Kagera River Basin context including its natural resources, land use and socio-economic context. It also included the analysis of land degradation threats; root causes of land degradation and barriers to Sustainable Land Management (SLM). Finally, this analysis included the policy context in the four countries with the identification of national priorities related to SLM. 46. This extensive analysis, added to the review of the baseline – existing projects and programmes supported by governments and donors – and of the GEF alternative (the proposed project), were packaged into a project document that formed the “blue print” for the implementation of this project. The review also found that between the initial project brief (document) produced in 2005-2006 and the final project document endorsed by the GEF CEO in May 2009, not many changes were made. The project strategy stayed more or less the same all along this cumbersome design phase. 47. The review of this project document indicates a good contextual analysis of the project including the review of the baseline and the lessons learned from other relevant initiatives in the area. This information was key to justify the project (the GEF alternative). As per the project document, the


12

conclusion of this contextual analysis was that “land degradation is having a strong negative impact on the structural and functional integrity of the ecosystems, driven largely by changes in land use and management practices in the diverse agricultural ecosystems of the Kagera River Basin. The basin’s increasing ecological vulnerability threatens the livelihoods of the 16.5 million (people) who live in the area today and the ability of the basin to sustain the predicted increases in population over the coming decades. The agro-ecosystem resources of the region have come under increasingly severe pressure in recent years due to natural population increase and returning refugees – accelerating the break-down of traditional agricultural practices (rotations, fallow, shifting cultivation and nomadic livelihoods) and giving rise to food shortages, poverty and economic vulnerability”. 48. However, our analysis of the project strategy reveals that this is a very ambitious project considering the allocated resources divided over four countries and the given timeframe to implement it. The logic model of the project consists of an overall goal, an environmental objective as well as a development objective, five outcomes and 13 outputs. There are presented in the table below (see also Annex 7 for an overview of expected outputs and their related planned activities). Table 1: Project Logic Model

OVERALL GOAL: Adoption of an integrated ecosystems approach for the management of land resources in the Kagera Basin will generate local, national and global benefits including: restoration of degraded lands, carbon sequestration and

climate change mitigation, agro-biodiversity conservation and sustainable use, protection of international waters and improved agricultural production, leading to increased food security and improved rural livelihoods.

PROJECT OBJECTIVES Environmental objective: To address the causes of land degradation and restore ecosystem health and function and generate a range of global environmental benefits across the Kagera basin through the introduction of adapted agro-

ecosystem management approaches.

Development objective: To improve the livelihoods and hence contribute to reduced poverty of rural communities in the Kagera Basin through more productive and sustainable resource management practices that are technically feasible and

socio-economically viable.

Outcome 1: Transboundary coordination, information sharing and monitoring and evaluation mechanisms operational and effective in promoting sustainable, productive agro-ecosystems and restoration of degraded lands.

• A basin-wide coordination mechanism is established to facilitate transboundary dialogue, basin-level policy harmonization and coordination of national/sub-national actions

• An efficient basin-wide knowledge management system is established to support information requirements and decision-making processes at all levels

• Project monitoring and evaluation system and technical reporting supporting Kagera TAMP implementation and decision making in the basin.

Outcome 2: Enabling policy, planning and legislative conditions are in place to support and facilitate the sustainable management of agro-ecosystems and the restoration of degraded land.

• Sustainable management of land and agro-ecosystems (SLaM) mainstreamed in national and district development programmes and basin institutions, enhancing synergy among sector strategies and across the river basin

• Regulatory actions and conflict resolution mechanisms developed and used to promote - or remove existing barriers to - sustainable land and agro-ecosystem management

• A coherent strategic land use planning framework in place (from river basin to district/provincial and community levels) (based on thematic reviews, stakeholder consultations and priority setting) to support SLM efforts by rural communities

Outcome 3: Capacity and knowledge are enhanced at all levels for the promotion of – and technical support for – sustainable management of land and agro-ecosystems in the basin.


13

• Methods and approaches to promote the adoption of SLM practices and agro-ecosystems (pastoral and cropping) developed and validated through demonstrations and study plots and participatory learning and adaptive management processes

• Enhanced quality of services provided to rural communities in the basin for improved agro-ecosystems management, through training by inter-sectoral teams, participatory research and monitoring and building on local knowledge and innovations

Outcome 4: Improved land and agro-ecosystem management practices are implemented and benefiting land users for the range of agro-ecosystems in the basin.

• Participatory land management plans are developed and implemented in targeted communities, micro-catchments and wider land units

• Improved land use and agro-ecosystem management practices are successfully adopted by farmers and herders in targeted communities and replicated in other areas

• Market opportunities and other cost-benefit sharing mechanisms for the provision of environmental services identified, demonstrated and promoted among land users

Outcome 5: Project management structures operational and effective.

• Project management, institutional and administrative structures in place and linked to national and regional decision making structures

• Project monitoring and evaluation system and reporting supporting project management and execution

49. A closer look at this strategy indicates that the project has 4 main directions: a) transboundary coordination; b) enabling environment; c) capacity development; and d) extensive demonstration of improved land management practices in the four riparian countries. Furthermore, the selection of demonstration sites (d) was done at the outset of the project and included project interventions in 45 catchments and in 101 Farmer Field Schools (FFS) located in 21 districts/provinces over 4 countries. Finally, the set targets at the end of the project include, among a long list of indicators (see also Section 4.3.5), the transformation of 100,000 ha toward sustainable agriculture ecosystems, 6% of the population (about 1M people) in the Kagera River Basin to be aware of this new approach and SLM practices implemented over 45,000 ha of land. The review found some incoherence between the project strategy to be implemented and the resources allocated to the project as well as its timeframe. 50. In conclusion, the review indicates three main issues with this design:

• This is a very ambitious project. The scope of the demonstration itself (fourth outcome) is already large; covering four countries (including 2 project official languages) and 21 administrative units. The dispersed locations of too many project sites are a constraint to achieving transboundary objectives. Moreover, each country is at different stage of decentralizing land and natural resource management functions to the district/province and local levels, rendering the mainstreaming of SLM into local development frameworks more complicated.

• A timeframe that is only 4.5 years. It does not allow the project to be implemented sequentially; that is to implement the demonstrations first, then based on lessons learned and best practices from these demonstrations implement the other expected outcomes. There is insufficient time to do that. Instead the project implementation team has to implement project activities in parallel, preventing good synergies – and in some cases good complementarity - among them.

• The project strategy stated in the project document is too detailed. This is illustrated in Annex 7 of this report and also through the performance indicators at the output level (see Section 4.2.5). In addition to the project strategy, the project document also details a long list of activities to be implemented, including a detailed budget. The result is a rather complex “blueprint” for implementing the project that is also so detailed that overtime it acts as a “straightjacket” for the


14

implementation team. As the project is moving into its implementation, it requires more and more adaptive management to adapt the implementation to the ongoing context moving away from the “blueprint” and rendering the implementation more difficult.

4.2. Efficiency of the project and analysis of the implementation process

51. This Section presents findings on the efficiency of the project, which is a measure of the productivity of the project intervention process. It reviews to what degree achievements are derived from an efficient use of financial, human and material resources. It reviews the overall management approach and the use of adaptive management, the financial management of the project, the technical assistance, the delivery mechanisms, the participation of stakeholders and the monitoring approach to measure the project’s progress.

4.2.1. Project Management Approach

52. The efficiency of the management approach to implement the project is moderately satisfactory. It follows FAO project implementation procedures. The project management team applies - when needed - an adaptive management approach to secure project outcomes while maintaining adherence to the overall project design. The detailed project document – including annex 6 - has been used to guide the implementation of the project and track its achievements. Project progress is regularly reported following the monitoring and evaluation (M&E) plan identified at the outset of the project, including the Annual Implementation Reviews (PIRs). Management procedures to procure project assets and equipment and to contract consultants follow the existing FAO rules and procedures. All project transactions are recorded and properly classified and show good internal control mechanisms to manage and control project resources. 53. However, the review found that despite adequate management procedures, the process to use these procedures seems to delay, at times, the implementation of the project and sometimes act as constraints for an effective implementation of field activities. The reasons for these delays were not identified during the field mission of the Evaluators but it is certainly an area that needs to be reviewed by FAO. Key management elements of the project are discussed below: Management Arrangements 54. The management and coordination arrangements for the implementation of the project include:

• FAO is the GEF Implementing Agency and Executing Agency for this project, and the Land and Water division (NRL) at FAO-HQ is the designated Lead Technical Unit and Operational Unit (Budget Holder).

• The Kagera TAMP Regional Coordination Unit (RCU) is hosted and supported by the FAO Representation in Kigali, Rwanda. The RCU is led by a Regional Project Coordinator (RPC), and is responsible for successful cross-country coordination and implementation of the project.

• The Designated National Authorities in the 4 riparian countries of the Kagera River are: o Ministry of Agriculture and Livestock (MINAGRIE) in Burundi; o Ministry of Agriculture and Animal Resources (MINAGRI) in Rwanda; o Division of the Environment, Vice President's Office (DOE/VPO) in Tanzania


15

o Ministry of Agriculture, Animal Industry and Fisheries (MAAIF) in Uganda; • A Regional Project Steering Committee (RPSC) for transboundary and collaborative issues

guides the project across countries. It comprises senior level government officials from each country (designated by the national executing partners/Ministries) and representatives from the major regional programmes, which are relevant for Kagera TAMP.

• A National Project Steering Committee (NPSC) guides the implementation of the project in each country. Members of the NPSC were nominated by participating Ministries and include representatives from Districts, Ministries (or Departments) as well as relevant non-governmental, civil society and private sector organizations. The NPSC reviews and endorses the annual national work-plans and budgets, including guidance for work within the country over the following year. It also oversees the timely implementation of the work-plan, which is implemented under the lead and supervision of the National Project Manager.

• Each of the four project countries has a National Project Manager (NPM) recruited by FAO and acting as fulltime national technical advisor and SLM expert supporting the project implementation in country and collaborating on cross-border activities. Two of the NPMs are based in the capitals, Bujumbura in Burundi, and Kigali in Rwanda and two of the NPMs are based in districts that are within the Kagera river basin but rather remote from the capitals: Bukoba in Tanzania, and Kabale in Uganda.

• The plan also included an ad-hoc Regional Technical Advisory Committee whose members were nominated by the four countries. Additional members can be called upon independently for reviewing project outputs and results, such as technical documents, and to provide specific technical support as and when required.

55. The management arrangements for this type of project are adequate. There is a RCU located in Kigali, Rwanda to coordinate all project activities taken place in the four countries. There are four national PSCs and one regional PSC to guide the project, endorse annual work plans and review the progress. However, despite that these arrangements are adequate, there are not used to their full extend. Based on meetings and observations during the field mission, there is not enough coordination happening at the project level among the four countries and FAO-HQ. As already discussed, the analysis from this review reveals that the project is made up of four juxtaposed sub-projects that are not really integrated, and that, so far, offer little synergetic effect at the regional level. More coordination is needed to address this lack of synergies among the four national demonstrations. 56. The first mechanism to increase this coordination is the PSCs. The review of meetings indicates that, in most cases, they do not meet enough, except in Burundi where 5 NPSC meetings took place so far. Only one NPSC meeting took place in Rwanda (another one is planned in June 2013), 3 meetings took place in Tanzania and 2 meetings in Uganda. These are too few meetings, particularly when considering that the first meeting in the four countries took place in late 2010 at the start of the project. NPSCs should meet twice a year, ideally one meeting to review and endorse work plans and a second meeting to review the progress made, discuss issues and potentially undertake actions to overcome any major obstacles faced by the project. 57. The RPSC is also not meeting as it should. So far, the RPSC met only once in March 2011. It should meet a minimum of once a year to review progress, exchange lessons learned and best practices, endorse annual work plans and discuss greater transboundary coordination at the basin level. It is recommended that it meets at least once and possibly twice a year - depending on the needs - for the remaining implementation period of the project.


16

58. The location of national project offices is also to be considered in the management arrangements. Two project offices are located in the capital: Kigali in Rwanda and Bujumbura in Burundi. The other two are located relatively far from the capital Bukoba in Tanzania (c. 1,500 km from Dar Es Salam) and Kabale in Uganda (c. 450 km from Kampala). The most efficient and effective office location is in Bujumbura where the project is housed within the Ministry of Agriculture. This contributes to the development of a good partnership with the key ministry helping the project management team in the day-to-day implementation of the project and also providing the ministry with a source of expertise on SLM. It will play a positive role for the long-term sustainability of project achievements in Burundi. On the contrary the remote locations of both offices in Tanzania and Uganda from the capitals hamper a good synergy with the national related institutions; though there is no question that they should be located where they are currently, in order to ensure effective implementation of project activities. Utilization of Financial Resources 59. As discussed above, the project is implemented using FAO procedures. In order to provide project financial resources to the field, the project management team uses two main administrative instruments: Field Budget Authorization (FBA) and Letter of Agreements (LOA). FBAs are a share of the overall project budget issued to FAO representations in the 4 countries on an annual basis for implementation of project activities at both the national and local levels. The annual amounts of these FBAs are determined by the annual work plans and budgets prepared and submitted by the NPMs. LOAs are an administrative instrument to formalize contractual arrangements with execution partners; they are an expenditure category in the FAO financial system. Each LOA that is signed with an execution partner includes terms of reference with the list of services to procure and an attached budget. LOAs are usually administered by the FAO Offices based on the FBAs, but can also be administered directly by HQ (like in the case of Tanzania). The administration procedures to manage LOAs are the same at headquarters and in the FAO Offices. LOAs (up to $100,000) can be operated and administered by the FAOR without headquarters involvement. 60. The review found that the efficiency of the administrative/financial process is mixed. In the case of Rwanda and Uganda, the process is viewed as too administrative and too long; particularly in the case of Uganda with the added geographical distance between the project office and the FAO office in Kampala. In Tanzania where the same geographical distance from the capital slows down this kind of process, the project office has been communicating directly with FAO-HQ Budget Holder, with some success, to mobilize project financial resources. 61. Contrastingly, the process to mobilize financial resources in Burundi appears to be more efficient. There seems to be a clearer process that includes the following steps: (i) annual work plans and budgets are drafted by the project team and presented to the NPSC members for their endorsement; (ii) once endorsed by the NPSC, it is sent to the Lead Technical Unit (LTU) at FAO-HQ for approval; (iii) once approved, a Field Budget Authorization (FBA) is prepared and approved by the Budget Holder; (iv) once this FBA is approved, cash is transferred to the FAO office in Bujumbura; (v) finally, when the financial resources are available locally, the FAO office prepares the necessary LOAs, which are approved by the FAO Representative, and then the implementation of project activities can begin. This administrative process does not seem to be different from the overall process described above but the decision-making process seems to be more decentralized in Burundi. As a result, the project team and the FAO office are more empowered in the management of the project. Coupled with a project


17

office based at the ministry of agriculture, the project certainly enjoyed a good visibility and is fully part of the development landscape in Burundi. 62. In addition to the lengthy administrative procedures, one important aspect that is not much taken into account in this administrative process is the sensitivity to seasonality when transferring money. A delay of one month in transferring the money may delay a set of activities for an entire agricultural season. As a project focusing on agro-ecosystems, it is critical that this aspect be incorporated into the overall management of the project for not losing valuable implementation time. Implementation Scheduling 63. Adaptive management has been used to adapt to a changing environment; however, it could have been used more extensively in order to keep the delivery of the project on its original timeline. The project started in March 2010 and should be completed in August 2014. As of the end of May 2013, the project has been implemented for about 39 months (72%) and there are only 15 months (28%) to go before its actual completion date (August 2014). 64. The review of the project timetable and its current achievements indicate that overall the implementation of the project is behind the original implementation plan, due mainly to the slow start-up phase in the first year, which is inherent to many multi-country/multi-sectoral projects. However, there is still enough time to complete the project on time or possibly with a no-cost time extension using the remaining budget. As discussed in Section 4.1.4, it is a very ambitious project, particularly when considering its timeframe. Furthermore, the long-term strategy/sustainability of project achievements relies mostly on the success of outcome 4 - that is the implementation of improved land and agro-ecosystem management practices in the Kagera River Basin including benefits for land users and its replication throughout the Basin. It was noted during this review that the strategy (logical framework) of the project does not emphasize much the replication of results from the demonstration sites implemented under outcome 4. Considering the importance of this aspect for the success of this project, it is recommended to review the original strategy for component 4 and to elaborate a work plan to the end of the project with an emphasis on the replication of project achievements (see Section 8 – Recommendation #3).

4.2.2. Financial Planning and Management

65. Project finances are managed using the FAO financial system and procedures; including the production of financial reports. However, it was noted during this review that the current FAO system has a strong limitation for project managers in that it has limited capacity to store and report financial data by GEF project outcome. Currently, the system captures and reports project expenses by expense categories such as consultants, travel, training expenses, etc. but cannot automatically associate these expenses to one or more GEF outcomes in accordance with the project budget originally defined in the project document3. 66. This limitation renders the management of project finances a difficult task, with limited accuracy and not timely either. In order to produce any meaningful financial report for the project, the 3 It is noted that since the field mission (May 2013), a Results-Based Budget system has been created in the FAO-FPMIS that can financially report up to the output level. The Kagera TAMP Budget Holder has since set up a Results Based Budget by component in the FPMIS.

project management team has to review any expenses entered into the system and, manually, using a separate spreadsheet, allocate each expense to the proper outcome. 67. Nevertheless, the Project Management Team was able to provide the Evaluation Team the following financial information about the project. As of the end of April 2013, the project overall disbursement of the GEF grant is USD 3,523,148, representing about 55% of the GEF budget of US$ 6.364M versus 72% of the total duration of the project (39/54 months). From a timeline the project disbursements are behind (55% vs. 72%). Only 15 months of implementation remain (out of 54) with about 45% of the total budget. In term of project expenditures, the project will have to spend an average of about $189k per month durmonth since its start-up. It is expected that the budget will not be spent entirely at the end of the project. 68. The utilization of funds by project outcome is presented in the following table: Table 2: Status of Project Funds Utilization

Item Budget 2010

Outcome 1 1,766,876 129,945

Outcome 2 423,343 57,515

Outcome 3 1,230,003 188,275

Outcome 4 2,360,683 113,571

Outcome 5 582,795 99,786

Total 6,363,700 589,092

(*) Figures obtained from the Project Management Team

69. As shown in the table above and the left diagram, the analysis by outcome reveals that the allocation of the overall GEF grant differs from outcome to outcome. The largest budget share is allocated to outcome 4 (SLM adoption 1 (transboundary coordination) with 28%, outcome 3 (capacity development) with 19%, then outcome 5 (project management) and 2 (policy and planning) with respectively 9 70. The second diagram also indicates that if 55% of the total GEF grant has been disbursed as of end of April 2013, the disbursements by outcome vary a lot. Outcome 2 budget is almost expended

Mid-Term Evaluation of the Kagera TAMP Project

18

has to review any expenses entered into the system and, manually, using a separate spreadsheet, allocate each expense to the proper outcome.

Nevertheless, the Project Management Team was able to provide the Evaluation Team the tion about the project. As of the end of April 2013, the project overall

disbursement of the GEF grant is USD 3,523,148, representing about 55% of the GEF budget of US$ 6.364M versus 72% of the total duration of the project (39/54 months). From a timeline the project disbursements are behind (55% vs. 72%). Only 15 months of implementation remain (out of 54) with about 45% of the total budget. In term of project expenditures, the project will have to spend an average of about $189k per month during the remaining period as opposed to an average of $90k per

up. It is expected that the budget will not be spent entirely at the end of the project.

The utilization of funds by project outcome is presented in the following table:

: Status of Project Funds Utilization

2010 2011 2012 2013 Total

Spent

129,945 99,422 111,768 80,134 421,269

57,515 112,774 171,921 67,609 409,818

188,275 259,676 294,144 120,299 862,393

113,571 382,002 974,952 66,723 1,537,247

99,786 61,456 76,009 55,170 292,420

589,092 915,330 1,628,794 389,935 3,523,147

(*) Figures obtained from the Project Management Team

shown in the table above and the left diagram, the analysis by outcome reveals that the allocation of the overall GEF grant differs from outcome to outcome. The largest budget share is

to outcome 4 (SLM adoption - demonstrations) with 37% of the budget, followed by outcome 1 (transboundary coordination) with 28%, outcome 3 (capacity development) with 19%, then outcome 5 (project management) and 2 (policy and planning) with respectively 9% and 7%.

The second diagram also indicates that if 55% of the total GEF grant has been disbursed as of end of April 2013, the disbursements by outcome vary a lot. Outcome 2 budget is almost expended

Term Evaluation of the Kagera TAMP Project – Final Report

has to review any expenses entered into the system and, manually, using a

Nevertheless, the Project Management Team was able to provide the Evaluation Team the tion about the project. As of the end of April 2013, the project overall

disbursement of the GEF grant is USD 3,523,148, representing about 55% of the GEF budget of US$ 6.364M versus 72% of the total duration of the project (39/54 months). From a timeline perspective, the project disbursements are behind (55% vs. 72%). Only 15 months of implementation remain (out of 54) with about 45% of the total budget. In term of project expenditures, the project will have to spend

ing the remaining period as opposed to an average of $90k per up. It is expected that the budget will not be spent entirely at the end of the project.

The utilization of funds by project outcome is presented in the following table:

Total

Spent

Budget Remainin

g

421,269 76%

409,818 3%

862,393 30%

1,537,247 35%

292,420 50%

3,523,147 45%

shown in the table above and the left diagram, the analysis by outcome reveals that the allocation of the overall GEF grant differs from outcome to outcome. The largest budget share is

demonstrations) with 37% of the budget, followed by outcome 1 (transboundary coordination) with 28%, outcome 3 (capacity development) with 19%, then outcome

The second diagram also indicates that if 55% of the total GEF grant has been disbursed as of end of April 2013, the disbursements by outcome vary a lot. Outcome 2 budget is almost expended

(97%), followed by outcome 3 and 4 with 70% and 65%, then outcome 5 with 50% and finally with outcome 1 with only 24% of the budget spent. These figures mean from a financial perspective, planned activities for transboundary coordination are well behind (however, activities in policy and planning should almost be completed (97% of the budget spent). Disbursements for activities under other outcomes (3, 4 & 5) seem to be aligned with the overall project timeline. 71. When focusing on the demonstrations being implemented (under outcome 4), the Project Management Team provided the following commitments and disbursements figures:

Table 3: Commitments + Disbursements Status in each Country

(*) Figures obtained from the Budget Holder 72. These figures do not include honorarium and international travel expenses for Project Managers and internationally recruited consultants. They represent mostly the total sum of LOAs that have been established in each country for the implementation of the demonstrations. The diagram shows that 2012 was the peak year in term of disbursements. The table also indicates that Tanzania has, so far, expended about one third more than other countries. 73. The review of these figures indicates that about 67% of total expenditures so far was spent on the demonstrations, representing about 100% of the outcome 4 budget. The Evaluation Team was not able to conduct a more in-depth analysis but the financial shortcomings. As discussed at the beginning of this section, the FAO financial system does not capture financial data by outcome and all reports received are estimations.


19

(97%), followed by outcome 3 and 4 with 70% and 65%, then outcome 5 with 50% and finally with outcome 1 with only 24% of the budget spent. These figures mean from a financial perspective, planned activities for transboundary coordination are well behind (only 24% of the budget spent), however, activities in policy and planning should almost be completed (97% of the budget spent). Disbursements for activities under other outcomes (3, 4 & 5) seem to be aligned with the overall

g on the demonstrations being implemented (under outcome 4), the Project Management Team provided the following commitments and disbursements figures:

: Commitments + Disbursements Status in each Country

(*) Figures obtained from the Budget Holder

These figures do not include honorarium and international travel expenses for Project Managers and internationally recruited consultants. They represent mostly the total sum of LOAs that

been established in each country for the implementation of the demonstrations. The diagram shows that 2012 was the peak year in term of disbursements. The table also indicates that Tanzania has, so far, expended about one third more than other countries.

The review of these figures indicates that about 67% of total expenditures so far was spent on the demonstrations, representing about 100% of the outcome 4 budget. The Evaluation Team was not

depth analysis but the financial reports by outcome presents some shortcomings. As discussed at the beginning of this section, the FAO financial system does not capture financial data by outcome and all reports received are estimations.


(97%), followed by outcome 3 and 4 with 70% and 65%, then outcome 5 with 50% and finally with outcome 1 with only 24% of the budget spent. These figures mean from a financial perspective,

only 24% of the budget spent), however, activities in policy and planning should almost be completed (97% of the budget spent). Disbursements for activities under other outcomes (3, 4 & 5) seem to be aligned with the overall

g on the demonstrations being implemented (under outcome 4), the Project Management Team provided the following commitments and disbursements figures:

These figures do not include honorarium and international travel expenses for Project Managers and internationally recruited consultants. They represent mostly the total sum of LOAs that

been established in each country for the implementation of the demonstrations. The diagram shows that 2012 was the peak year in term of disbursements. The table also indicates that Tanzania has,

The review of these figures indicates that about 67% of total expenditures so far was spent on the demonstrations, representing about 100% of the outcome 4 budget. The Evaluation Team was not

reports by outcome presents some shortcomings. As discussed at the beginning of this section, the FAO financial system does not capture

Co-financing Status 74. During the formulation of t(USD 24.5M) representing almost 80% of the entire “GEF alternative” against 20% for the GEF grant. The breakdown of this amount is given in the table below. Table 4: Co-financing Commitments at the Outset of the Project

(*) Source: project document

75. At a briefing session at FAOthe Evaluation Team (see also Annex 8): Table 5: Co-financing Status (June 2012)

Item Committed in

Prodoc

Gov. Burundi 6,260,000

Gov. Rwanda 6,293,760

Gov. Tanzania 2,463,050

Gov. Uganda 3,707,800

FAO 351,000

Partners/Donors 5,433,600

Total 24,509,210

(*) Source: project document

76. These figures are from June 2012 and would need to be updated. However, they indicate a trend that co-financing commitments are not being fulfilled as planned; only 14% of the commitments have been met as of June 2012. Additionally, discussions on this top


20

During the formulation of the project, a rather large amount of co-financing was identified (USD 24.5M) representing almost 80% of the entire “GEF alternative” against 20% for the GEF grant. The breakdown of this amount is given in the table below.

financing Commitments at the Outset of the Project

At a briefing session at FAO-HQ during this evaluation, the following figures were given to the Evaluation Team (see also Annex 8):

financing Status (June 2012)

As of June

2012

% of

Commitments

234,460 4%

303,600 5%

229,676 9%

1,944,530 52%

242,820 69%

496,212 9%

3,451,298 14%

These figures are from June 2012 and would need to be updated. However, they indicate a financing commitments are not being fulfilled as planned; only 14% of the commitments

have been met as of June 2012. Additionally, discussions on this topic during the field mission indicate


financing was identified (USD 24.5M) representing almost 80% of the entire “GEF alternative” against 20% for the GEF grant.

HQ during this evaluation, the following figures were given to

These figures are from June 2012 and would need to be updated. However, they indicate a financing commitments are not being fulfilled as planned; only 14% of the commitments

ic during the field mission indicate


21

that there are some issues around the definition of co-financing. The Evaluation Team was not able to find more details on how these planned commitments were calculated, including the proportions that are cash and in-kind. Additionally, it seems that some of these numbers were really “associated financing” rather than “co-financing”. 77. Two elements are important when analyzing these figures: (1) most of the co-financing programmes/initiatives identified at the outset of the project are either completed or did not materialize as planned. As discussed in section 4.1.4, this project took a long time between its concept and the start of its implementation. Some of these co-financing commitments became obsolete at the start of the project due to this long delay; (2) Without exact figures, it seems that additional co-financing was identified by the NPMs after the implementation of the project was underway. The case of Burundi is a good example. As of May 2013, the NPM reported a total co-financing so far of USD 877,460. This amount is mostly in-kind (USD 257,500 was cash) contributions from local partners, which partnered with the project to conduct joint activities, and which had not been identified at the outset of the project. 78. Nevertheless, NPMs are keeping track of co-financing in their respective country and it is reported in the annual PIRs. The sum of USD 3.15M was reported as co-financing in the PIR 2011-2012, which is slightly lower that the numbers as of June 2012 presented to the Evaluation Team (see table 9 above). 79. Finally, it is a difficult task to assess the cost-effectiveness of this project. It has spent a total amount so far of USD 3.5M and delivered a set of studies, maps and supported the demonstrations of SLM practices. This task is also rendered difficult by non-accurate and non-timely financial management reports; which make the monitoring of expenses per GEF outcome or output versus the budget stated in the project document an almost impossible exercise. Nevertheless, it was noted that project financial resources were used adequately and that as much as possible, the spending power of each project dollar is effectively stretched through partnership finance under joint project activities.

4.2.3. Quality of Technical Assistance / Use of National Capacity

80. The standard of technical assistance to the Kagera-TAMP is excellent. There are top national experts in charge of the four country programs, all of whom are well qualified in SLM and related topics, as well as being experienced - and clearly motivated to achieve results. At FAO there is a strong team also, able to provide cutting-edge guidance in key topics including LADA-WOCAT methodology, monitoring and evaluation, PES and SLM/ SLaM in general. Specific skills in farmer field school methodology and associated agro-ecosystems analysis have been brought in very effectively through the use of a regionally based consultant. 81. To this impressive pool of knowledge can be added the association with regional agencies and their considerable relevant knowledge – for example the Nile Basin Initiative, Vi-Agroforestry and the LVEMP II project with whom MOUs have been / are in the process of being drawn up. Finally there is another layer of technical assistance closer to the communities themselves: this comprises the “service providers” who are contracted to support the FFS groups and the watershed committees. These agencies, most (but not all) local NGOs, bring their own specific expertise to supplement more general guidance on SLM.


22

4.2.4. Country Ownership / Stakeholder Participation

82. There appears to be good country ownership – through national steering committees, which include the main agencies involved in SLM and the environment. However the scale of actual ‘buy-in’ to the project varies, and the connection between the project ‘nerve-centres’ in each country and the national agencies leaves room for strengthening in some instances. Tanzania and Uganda have the most difficult task in this respect, as their project headquarters lie some hundreds of kilometers from their respective capitals (see 4.2.1). There are examples where opportunities to develop synergistic partnerships with other similar projects - most notably other GEF-SLM projects in Uganda – have not (yet) been taken up. 83. Ironically the national identities are better forged than a regional identity. As is mentioned elsewhere in this evaluation report (see 4.3.2 and 4.2.1), the lack of true integration of efforts hampers the establishment of a regional feeling of ownership. A further complicating factor is that the project has, to a certain extent, a split personality – between the strong centralized executing agency in Rome (FAO) and the Regional Headquarters in Kigali. To the outsider it is hard to reconcile the two, or to establish where the real heart of the project lies. 84. Stakeholder participation is strong, at most levels. While it had been commented in the foregoing that not all national agencies are yet fully on-board, this cannot be attributed to failures within the project system. Certainly regional bodies have been brought into the picture (see the MOUs described above) and all the important national agencies are part of steering committees. District level agencies are part of the implementation thrust, and local NGOs, though their role as service providers, are acknowledged as valuable stakeholders too. At community level there is a very strong feeling of ownership – demonstrably so – and the connection of the land users to the project is obvious to all.

4.2.5. Monitoring Approach and Progress Reporting

85. A Monitoring & Evaluation (M&E) plan was detailed in the project document and was reviewed by a consultant in 20114, including the design of reporting templates and of an SLM impact assessment protocol to monitor the environmental benefits. The objective of this plan is to assist all project participants in assessing project performance and impacts and it takes place at three levels: project execution, project performance, and impact evaluation. Furthermore, this M&E plan is to serve two functions: (1) periodic assessment of project implementation and performance of activities and, (2) evaluation of their results in terms of relevance, effectiveness and impact in promoting the adoption of sustainable land and agro-ecosystem management. A budget of USD 391,600 was allocated to this plan (6.2% of the total project budget). 86. As per this plan, the day-to-day monitoring of implementation progress is the responsibility of the RPC and the NPMs. The RPC and NPMs are also responsible to report regularly to members of Regional and National PSCs, highlighting important issues and constraints for advice and guidance. Other responsibilities are detailed in this M&E plan, including the need for the RPC to advise the FAO LTU, who would duly inform the GEF Secretariat, of any delays or difficulties faced during

4 FAO, GEF, Janie Rioux, December 2011, Monitoring & Evaluation Plan - Transboundary Agro-Ecosystem Management Project in the Kagera Basin : Rwanda, Burundi, Uganda and Tanzania.


23

implementation so that appropriate support or corrective measures can be adopted in a timely and appropriate manner. The M&E plan includes an independent mid-term evaluation (this report) and an independent terminal evaluation to be conducted in close consultation with the participating countries and FAO’s evaluation unit (OED). The mid-term evaluation is to determine progress being made towards achievement of outcomes and to identify corrective actions if necessary. The terminal evaluation is to take place three months prior to completion of the project and to focus on reviewing project impact, analyze sustainability of results and whether the project has achieved the outcomes and the development and environmental objectives. It will also provide recommendations for follow-up actions. 87. In addition to the two evaluation reports, the operating modalities of the M&E plan are as follows:

• Performance Indicators: A set of indicators with their respective baseline and target at end of project as well as their sources of verification were identified and documented in the logframe.

• Project Inception Report: This was to include a detailed first year annual work plan, any monitoring and evaluation requirements to measure project performance during the first year and a detailed narrative on the institutional roles and responsibilities and coordinating action of project partners, progress to date on project establishment and start-up activities, and an update of any changed external conditions that may affect project implementation;

• Monthly Progress Reports: The four NPMs need to report their progress monthly to the RPC; • Quarterly Progress Reports: The NPMs also report quarterly to the RPC and the RPC also

reports quarterly to the LTU/BH; • Quarterly Project Implementation Reports: This is an internal FAO monitoring tool prepared by

the FAO budget holder (BH). It is a review of the project, a review of its approved work plans versus its actual performance, and the identification of corrective action as required;

• Semi-Annual Project Progress Reports: The RPC, with inputs from the NPMs and in collaboration with the National Focal Points, prepares semi-annual progress reports reflecting progress made and identifying any problems/issues;

• Project Implementation Review (PIR): This is an annual monitoring process mandated by the GEF. It is completed by FAO for each year beginning July 1 and ending June 30th;

• Back to Office Reports: This is a FAO requirement for all HQ staff to file a Back to Office report after the return from a field mission.

88. The core of this M&E plan is a set of performance indicators identified at the objective, outcome and output levels. In its entirety, it comprises a set of 49 indicators (see Annex 9). When considering only the performance indicators to measure progress at the objectives and outcomes levels – a GEF requirement in PIR - this set is down to a more manageable list of 16 performance indicators (see Table 10 below).


24

Table 6: List of Performance Indicators at Objective and Outcome Levels

Objectives/Outcomes/Outputs Indicators

Environmental objective: To address the causes of land degradation and restore ecosystem health and function and generate a range of global environmental benefits across the Kagera basin through the introduction of adapted agro-ecosystem management approaches. Development objective: To improve the livelihoods and hence contribute to reduced poverty of rural communities in the Kagera Basin through more productive and sustainable resource management practices that are technically feasible and socio-economically viable.

1. Improved land use systems/ management practices for the range of agro-ecological zones in the basin being tested and adapted (by end PY3) for arable and pastoral systems including measures to reduce pressures on wetlands, riverbanks, forests, protected areas.

2. Transformation of 43,700 ha of land by PY3 and 100,000 ha. by PY5 towards productive and sustainable agricultural ecosystems

3. Potentially 6 percent of today’s basin population (some 1 million people) aware of project activities in target communities, micro-catchments, agro-ecological units through demonstrations and outreach.


4. Transboundary agro-ecosystem management programme (TAMP) to reverse land degradation being implemented and monitored by the 4 riverine countries in 21 districts, reviewed by national and regional PSCs, and project activities & achievements widely shared and available (PY5).

5. Best practices for addressing TB land-related constraints through integrated ecosystems and inter-sectoral approaches mainstreamed in planning and development processes, including. NAPs, and pilot actions implemented to address TB issues in 68 communities (PY3) and replicated in 21 districts (PY5).

6. Regular Government budget allocations to transboundary coordination & collaboration in the Kagera basin increased by 10 percent (PY5)


7. Priority policy, legal and Transboundary issues identified and agreed at community (68), district (21) and river basin levels for SLaM (end PY2) and resulting in supporting policy decisions, regulatory mechanisms and community bye-laws for improved harmonization and application (PY5).

8. At least 2 policy recommendations per country developed that support national policy-decisions and regulatory mechanisms, and 1 per country that support by-laws, etc. at district/ community level.


9. Trained technical staff and policy makers in 21 districts - supporting SLaM planning and implementation and using project information resources in their district and communities (PY5)

10. Community members/local decision makers sensitized on SLaM techniques for pastoral, arable, mixed systems and their on- and offsite impacts and benefits (PYs 1-5)

11. FFS members trained and adopting SLM and promoting upscaling on community territory

12. Training materials on best practices /approaches widely available and SLM demonstrations in place.


13. SLM practices implemented by pilot communities (68 by PY3; 200 by PY5) in demonstrations and farmer plots covering a total of 45,000 ha of land (by PY5) and showing:

a. Effective control of soil erosion (no new visual signs) in all target sites;


25


b. 4 target micro-catchments (PY5) identified and sediment loads monitored(subject to identifying sites where SLM interventions can be applied on a significant area of the catchment and hydrological monitoring can be supported by partner Kagera IWRM, NBI-NELSAP and LVEMP projects);

c. 30 percent increase in vegetation cover (above and below ground biomass) on pilot 23,000 ha arable and 7,500 ha pasture lands where alternatives to slash and burn are applied (PY5)

d. 20 percent increase in soil carbon stores on farmer study plots and sample arable and pasture lands (PY5) inferred on 30,500 ha of land where SLM is practiced/planned.

e. 10 percent increase in production (crop; livestock; other goods) by trained farmers/ herders contributing to livelihoods (income; food security; reduced vulnerability)


14. Execution of project activities and delivery of outputs in accordance with work plan and budget

15. Regional PSC and TAC operational 16. Backstopping by FAO and by Government institutions

Source: Project Document

89. The review of this M&E plan – including the revision done in 2011 - indicates that in its entirety, the plan is complex and the Evaluation Team questions its cost-effectiveness. Tracking 49 indicators – which some of them have multiple targets – requires a lot of effort and time to produce accurate and timely monitoring information for project managers. Additionally, the assessment of these indicators reveals that most indicators at the output level are not really development indicators but rather indicators to track the delivery of project activities. They include indicators such as tracking the number of demonstrations sites, number of FFS study plots, number of FFS Facilitators trained, number of technical staff and policy makers trained, number of participatory land use plans, etc. One can say that these indicators are SMART5, however, despite that they are specific, easy to measure, attainable and time-bound, these indicators are not fully relevant when measuring how well the project is progressing toward its environmental and developmental objectives. The relevance between the number of people trained in SLM techniques and the improvement of land use systems in the Kagera River Basin is limited. Yes, training activities are necessary but many other activities need to take place to improve these practices. Additionally, using this same example, the tracking of the delivery of training do not take into account the quality of this training and if it is sufficient for the trainees to have a better capacity and change their land use management practices. 90. In term of measuring/reporting progress made by the project, the PIRs – a GEF reporting requirement – are focusing on how the project is progressing towards achieving its objectives and outcomes (section 3 of the PIRs6). It uses the set of 16 indicators that were identified to measure the performance of the project at these levels. These indicators are critical for measuring the success of the project. They are used for the PIRs but also by this mid-term evaluation and the terminal evaluation when assessing the real progress made. It is also a fact that the other indicators (49–16 = 33) at the output level are not really used by the project management team to monitor the project on day-to-day basis. They are not part of any monitoring reports reviewed by the Evaluation Team. As for the PIRs,

5 Specific, Measurable, Attainable, Relevant and Time-bound. 6 These PIRs also include a review of the progress in project implementation (section 4) but it is mostly to give the implementation status of the planned activities. This section provides little information on the progress towards the expected results.


26

they are key reports – particularly section 3 that is measuring the progress made against the set targets for the project - documenting the progress made by the project towards achieving its objectives and its outcomes (high level results). The review found that these reports could be improved with more detailed in this section 3 about progress made and how the project is meeting its expected results. It is the main measuring instrument of the project to assess how successful the project is reaching its expected results. 91. Focusing on the set of 16 indicators, the Evaluators found two main limitations:

• Similar to the indicators at the output level, some of them are too focused on project deliverables such as technical staff trained, community members sensitized and SLM practices implemented. They simply measure if yes or no the project delivered its targeted activities but they don’t give much indication that the project is progressing towards its expected outcomes and objectives; they are not relevant enough.

• The current three performance indicators selected to measure the progress made at the objectives level are not comprehensive enough. They do not include any measurement of to what extend the capacity needed to address the causes of land degradation and restore ecosystem health has been developed - nor the extent to which it has been replicated/mainstreamed. There is a need to identify clear targets in this area. Ultimately, this is the “raison d’être” of the project. The project is supposed to demonstrate SLM practices and then these practices adopted and replicated in the four countries; particularly in the Kagera River Basin.

92. Finally, the review of this performance framework (the 16 indicators) indicates that the set targets at the end of the project are very ambitious. As it was already discussed in previous sections, the Evaluation Team concluded that some of these targets will not be achieved by the end of the project. This was also found in the review conducted by the consultant in 2011 where some targets were identified for further discussion with the LTU and to be considered during the MTE. It is particularly the case for targets with clear quantified values such as indicator #2 where 100,000 ha under SlaM will not be achieved (currently about 5,350 ha – see table 4), indicator #7 where it is unlikely that 68 communities will have identified and agreed on priority policy, legal and Transboundary issues (currently about 46); and, indicator #13 where SLM practices will not be implemented in 200 pilot communities (currently in 46 communities). These targets were too ambitious from the outset of the project; they need to be reviewed and revised to be more in line with feasible achievements during the remaining implementation period of the project.

4.3. Effectiveness of the Project

93. This section presents the findings on the effectiveness of the project, which is a measure of the extent to which formally agreed expected project results (outcomes) have been achieved, or will be achieved in the future. It includes an overview of key results achieved to date by the project, followed by the project contribution to capacity development, the review of unexpected project achievements and finally the review of risks management and mitigation measures related to the implementation of the project.


27

4.3.1. Achievements of Project’s Objectives

94. As discussed in Section 4.1.5, the project has two objectives: an environmental objective that is to address the causes of land degradation and restore ecosystem health and function and generate a range of global environmental benefits across the Kagera basin through the introduction of adapted agro-ecosystem management approaches; and a development objective that is to improve the livelihoods and hence contribute to reduced poverty of rural communities in the Kagera Basin through more productive and sustainable resource management practices that are technically feasible and socio-economically viable. 95. The table below presents the key achievements at the objective level, against the three targets set at the outset of the project, to measure the progress of the project. Table 7: Attainment of Project Objectives

Project Objectives Targets at End of Project Key Results


• Improved land use systems/ management practices for the range of agro-ecological zones in the basin being tested and adapted (by end PY3) for arable and pastoral systems including measures to reduce pressures on wetlands, riverbanks, forests, protected areas.

• Transformation of 43,700 ha. of land by PY3 and 100,000 ha by PY5 towards productive and sustainable agricultural ecosystems

• Potentially 6 percent of today’s basin population (some 1 million people) aware of project activities in target communities, micro-catchments, agro-ecological units through demonstrations and outreach...

• SLM activities have been actively demonstrated in 45 micro catchments and through the engagement of about 3,500 people from local communities through 121 FFS

• 5,500 ha of land under sustainable land management; including about (1,000km) of contour ditches

• Nearly 300 km river banks protected by buffer strips

• More than 3,000 community members taking active part in on-the-ground demonstrations through Farmer Field Schools

• An estimated 4% of the basin population is aware of project activities

Source: adapted from PIR 2012, notes from the Project Management Team and notes from mission to Burundi, Rwanda, Tanzania and Uganda.

96. The review of project results achieved so far towards the achievement of project’s objectives reveals that the project is on track albeit at a much slower pace than envisaged in the project document. It is noted, and acknowledged however, that there have been country-specific constraints on delivery (e.g. in Tanzania, District Agricultural Offices were unable to adequately handle funds or activities; in Rwanda, the Rwanda Agriculture Board was found to be non-operational at field level; in Uganda the project was frustrated by the failure of a large NGO to cooperate). Communities benefitting from project interventions are becoming better enabled to address causes of land degradation and restore their ecosystem health and functions, and their livelihoods are being improved. The demonstrations that are underway in the four countries are addressing causes of land degradation and restoring ecosystem health and function through the implementation of improved land use and agro-ecosystem management practices at the community level. It is still too early to fully assess the benefits of these

To the question: “What is the main benefit of these measures?” all communities met in Burundi answered “less water runoff” as the first benefit they see and appreciate.


28

measures, but field observations indicate that the main benefit from these practices is better control of runoff, which limits soil erosion and improves water availability to plants. Soil erosion is also better controlled with the planting of trees and perennial grasses and fodders along the infiltration ditches and other conservation structures; these help hold the soil in place. 97. At the same time, the holistic approach used by the project to address these issues at the community level clearly leads to an improvement in the livelihoods of these rural communities. In addition to the implementation of SLM practices to sustain and eventually to increase soil productivity while simultaneously preventing soil erosion, and in some cases providing fodder for animals, the project approach at the community (micro-catchment) level includes a strong stakeholder engagement process to empower these communities to use these SLM measures. Furthermore, the project is also promoting improved agricultural practices to control pests and increase yields through introducing better crop varieties. It also promotes smallstock such as goats and pigs, as well as fish ponds. These provide manure for soil fertility, but are also economic activities in their own right. Cash crops to generate income for the community include potatoes, cabbage, and eggplants. Other activities include improved cooking stoves to reduce the use of firewood, the fabrication of salt-blocks, and establishing kitchen gardens to produce fresh vegetables for home consumption. In the case of Burundi, where the implementation of these demonstrations is especially well advanced, there is evidence of better livelihoods for communities benefitting from project interventions.

4.3.2. Attainment of Project’s Expected Outcomes

98. The project has five expected outcomes: in brief these are (i) coordination, (ii) enabling policy, planning and legislative conditions, (iii) capacity and knowledge enhancement, (iv) improved land management practices, and (v) project management. The implementation progress towards these expected outcomes has been measured though a set of 13 indicators with their respective baseline values at the outset of the project and target values at the end. Below is a table listing the key results achieved so far by the project against each outcome and their corresponding targets.


29

Table 8: List of Kagera TAMP Main Achievements

Expected Results Targets at End of Project Key Results

Environmental objective: To address the causes of land degradation and restore ecosystem health and function and generate a range of global environmental

benefits across the Kagera basin through the introduction of adapted agro-ecosystem management approaches.

Development objective: To improve the livelihoods and hence contribute to reduced poverty of rural communities in the Kagera Basin through more

productive and sustainable resource management practices that are technically feasible and socio-economically viable.


• Transboundary agro-ecosystem management programme (TAMP) to reverse land degradation being implemented and monitored by the 4 riverine countries in 21 districts, reviewed by national and regional PSCs, and project activities & achievements widely shared and available (PY5).

• All 21 target districts are informed and engaged in identifying project transboundary issues and interventions for sustainable land and agro-ecosystem management

• Conducted assessment of transboundary issues on land degradation and conflict due to livestock management and movements and identification of strategies and option for sustainable and equitable land and livestock management

• A review was done on identification of transboundary agro-ecosystems management issues between Burundi and Rwanda including the situation on deforestation, bush fires, wood charcoal, trans boundary waters and wetlands management as wells as government policies and laws governing those environmental transboundary issues

• Best practices for addressing TB land-related constraints through integrated ecosystems and inter-sectoral approaches mainstreamed in planning and development processes, including. NAPs, and pilot actions implemented to address TB issues in 68 communities (PY3) and replicated in 21 districts (PY5).

• A study was conducted to review transboundary livestock impacts on land degradation and conflict related to livestock management and movements, various strategies on livestock management have been recommended

• WOCAT tools have been used to assess 32 SLM technologies/best practices, documented and are currently being uploaded to the WOCAT database

• Regular Government budget allocations to transboundary coordination & collaboration in the Kagera basin increased by 10 percent (PY5)

• To-date, only the Burundi Government has allocated a cash contribution of about 157,000 USD to the project. The other three countries have considered co-financing of Kagera TAMP in kind, but were yet in a position to make cash commitments


• Priority policy, legal and Transboundary issues identified and agreed at community (68), district (21) and river basin levels for SLaM (end PY2) and resulting in supporting policy decisions, regulatory mechanisms and community bye-laws for improved harmonization and application (PY5).

• Regional experience sharing workshop on land use planning, tenure and PES was successfully organized at Kabale, Uganda in August 2011 with 44 participants from the Kagera region – strategies for policy harmonization were discussed

• A consultant was recruited for about two months (August to October 2012) to carry out an assessment of land and natural resources conflicts in all four Kagera basin countries in which those conflicts impede adoption of sustainable land and agro-ecosystems management. Appropriate recommendations were given and based on that a training will follow suit for government and local authority leaders on land conflict resolution and Participatory Negotiation Territorial Development (PNTD)

• At least 2 policy recommendations per country developed that support national policy-decisions and regulatory mechanisms, and 1 per country

• Initial policy reviews at national and district levels ongoing to identify opportunities for making a transformation towards SLM


30


that support by-laws, etc. at district/ community level.


• Trained technical staff and policy makers in 21 districts - supporting SLaM planning and implementation and using project information resources in their district and communities (PY5)

• Training of technical staff has been taken place in all 21 districts. However the training of policy makers in various SLaM aspects and related policy issues is scheduled in PY4 (2013)

• Community members/local decision makers sensitized on SLaM techniques for pastoral, arable, mixed systems and their on- and offsite impacts and benefits (PYs 1-5)

• Service providers and district project facilitators have started community sensitization process

• FFS members trained and adopting SLM and promoting upscaling on community territory

• 104 SLM/FFS have been established in all four countries with 200 facilitators trained in SLM and Farmer Field School principles and have started farmer field schools in the project catchment sites with just over 4,000 farmers engaged in SLM field activities

• Training materials on best practices /approaches widely available and SLM demonstrations in place.

• Manual on FFS-SLM for FFS being used and validated in English and French

• SLM on the ground being assessed and documented for further use and upscaled to other areas in the entire catchment


• SLM practices implemented by pilot communities (68 by PY3; 200 by PY5) in demonstrations and farmer plots covering a total of 45,000 ha of land (by PY5) and showing: o Effective control of soil erosion (no new

visual signs) in all target sites; o 4 target micro-catchments (PY5 ) identified

and sediment loads monitored (subject to identifying sites where SLM interventions can be applied on a significant area of the catchment and hydrological monitoring can be supported by partner Kagera IWRM, NBI-NELSAP and LVEMP projects);

o 30 percent increase in vegetation cover (above and below ground biomass) on pilot 23,000 ha arable and 7,500 ha pasture lands where alternatives to slash and burn are applied (PY5)

o 20 percent increase in soil carbon stores on farmer study plots and sample arable and pasture lands (PY5) inferred on 30,500 ha of land where SLM is practiced/planned.

• After participatory selection and diagnostics of catchments field activities just started through FFS groups and with community members

• Establishment of soil erosion control structures (bench terraces, water retention ditches, runoff ponds) stabilized with grass has begun in most parts of the project sites

• Target catchments identified and MoU for collaboration signed with Lake Victoria Basin Commission and NBI to monitor soil erosion and sediment load

• SLM activities involving restoration of vegetation cover has begun including planting 900,000 trees in the project micro-catchments (including 50,000 bamboo planted along river banks to protect the buffer zone in Burundi).

• Other SLM activities include Integrated soil management through judicious application of farm yard manure, mineral fertilizers and lime (in some catchments in Rwanda with high acidity ie. low pH)

• Sediment load measurement will start in Y4 in the identified selected micro-catchments in collaboration with LVBC/LVEMPII

• In addition to tree planting and agroforestry, other SLM initiatives for enhancing vegetation cover include protection of natural vegetation by enclosure whereby the vegetation remains intact and allowed to regenerate (mostly practiced in project areas in Tanzania and Uganda)

• To minimize excessive use of vegetation for domestic fuel, communities in the catchments are trained in the use of low cost energy saving stoves

• Establishment of kitchen gardens in micro-catchment communities. The gardens are part


31


o 10 percent increase in production (crop; livestock; other goods) by trained farmers/ herders contributing to livelihoods (income; food security; reduced vulnerability)

of an effort to improve family nutrition by adding vegetables to the diet, while the excess produce can be sold providing a much-needed source of income

• At May 2013 achievement in pilot sites were as follows: o Activities ongoing in 46 communities with 121 farmer field schools established o 3,500 members of FFS (with more women than men – a ratio of 1.15: 1) o 45 micro-catchments under the programme with a total area of 19,000 ha o 5,350 ha already covered by SLM o Buffer strips protecting 283 kilometers of riverbanks o 1.8 million trees – mainly leguminous multipurpose varieties - planted


• Execution of project activities and delivery of outputs in accordance with workplan and budget

• Project staff in place and 4.5 year work plan and annual plans and budgets developed.

• National PSC meetings have been organized in each country to ensure regular advice to the project implementation team and approval of workplan and budgets

• Monthly, quarterly and bi-annually reports are produced by NPMs and RPC to monitor progress

• Project M&E system has been established

• Regional PSC and TAC operational • First Regional PSC meeting held (March 18, 2011), next planned in September 2013. Regional TAC members have been nominated (2 from each country) and composed of ad hoc experts/ advisors to be called upon as and when necessary for peer review, support etc.

• Backstopping by FAO and by Government institutions

• NRL conducted 3 supervisory and technical visits (Regional Policy workshop; LADA & SLM training; Project M&E system review and Project coordination mission in Tanzania and Uganda)

Source: adapted from PIR 2012 and updated data solicited by evaluation team from coordinators in all four countries


32

99. The review of these achievements against the five expected outcomes reveals that the progress is so far limited. Nevertheless it is acknowledged that the speed of implementation has picked-up quickly at this (rather late) stage of the project. The impression emerging from this review indicates a good list of project deliverables but with limited integration among them. Demonstrations are taking place in the four countries with little interconnection; land degradation maps were produced for the four countries but with no direct relation to the demonstration sites; WOCAT questionnaires (QM and QA/QT) have been completed but with limited added value to the activities implemented in the demonstration sites; etc. 100. Based on the findings of the review a brief synopsis of observations is presented below according to the stated outcomes:

i. Transboundary Coordination: A series of useful transboundary studies have been carried out/are planned on various issues (policy/legal; livestock; agro-ecosystems; resource competition; water; etc.) but there is little evidence that these studies have yet been put to use. A synthesized summary of the studies with the main conclusions highlighted might help to achieve greater visibility and impact. Useful partnerships - and the development of MOUs - with other actors in the region have been forged, particularly with the Nile Basin Initiative, Lake Victoria Environmental Management Project Phase II, and Vi-Agroforesty (see also Section 4.1.3). However the overall impression that emerges is of a project in four juxtaposed countries with national programmes that are not integrated, and that offer little synergetic effect at the regional level. There are missed opportunities in terms of cross-visits between countries at all levels (while it is noted that an international visit to Brazil was carried out, and there have been some exchanges between national coordinators). Thus, there is inadequate exchange of ideas or the forging of a sense of common identity or purpose.

ii. Enabling policy, planning and legislative conditions: Here the project targets are daunting, and currently achievements are running some distance behind. Nevertheless, at community level watershed committees are being set up and bye-laws formed and administered. At country level there is potential for the merging of certain initiatives into national programmes: perhaps the integration of a new model of farmer field schools into the national extension services is the best example. There is little evidence yet of policy being formed at a regional or basin level resultant from the project.

iii. Capacity development: This topic is explored in more detail under section 4.3.6 but suffice it to say here that once again the targets specified in the project document are very ambitious and though capacity development is underway, the results are mixed and there is a long road to travel.

iv. Improved land management practices: At the risk of repetition, once again targets are daunting (e.g. 68 communities involved by year 3 and 200 by year 5; 45,000 ha under SLM by year 5; 100,000 ha ‘transformed’ towards productive and sustainable agricultural ecosystems by year 5). While there appears to have been a recent impressive pick-up in pace of implementation centered around active and enthusiastic farmer field schools groups (see discussion of a new ‘model’ in section 4.2.5) and vibrant community engagement, the overall achievements to-date of just

The Bukoora FFS (Uganda) had prepared a fable, written as a ‘news bulletin’:

“In Kanyakwanzi, farmers constructed contour trenches….and the healthy carrot harvest left them in surprise. But in Nyakyato, heavy rains washed soil and property away”. “Floods brought a case against the FFS for having dug trenches to control them. The magistrate ruled in favor of the FFS. The floods had no choice but to continue infiltrating in the trenches”.


33

5,350 hectares under SLM7 are a very long distance from the (admittedly) over-ambitious targets set in the project document. The idea of implementing a payment for ecosystem services model will achieve little or nothing having being introduced too late in a project with too brief a life.

v. Project management: Management matters are discussed in section 4.2. 101. The following paragraphs now summarize the specific achievements of the project, with an analytical comment in each case: these comments, together with the observations in the forgoing section inform the overall conclusions, lessons and recommendations of this evaluation. To summarize, the project has achieved the following: Transboundary studies 102. Three studies have been completed, namely:

a. Transboundary Policy/ Legal Issues b. Transboundary Livestock Issues c. Land and Natural Resource Conflicts d. Two further studies are starting up, these are on Agro-ecosystems and Water.

103. Two further studies are starting up, these are on Agro-ecosystems and Water. While valuable, these last two studies now need to be completed quickly, and an overall synthesis of findings could then make them more accessible and visible. Maps 104. For each country, land use system (LUS) maps have been prepared. LADA-WOCAT’s mapping questionnaire (QM) has then been used to provide up to 80 maps per country, at national level, on various aspects of land degradation (types and causes etc.) as well as level of SLM protection and the potential for SLM in various LUS zones. Questions, however, remain about the value of these maps at national scale – and their custodianship in future. Kagera-TAMP considers that core QM-GIS capabilities are a strong value addition to the project and the region’s development: there is no reason to doubt this. Furthermore it is acknowledged that there are clear plans to take this component further, in terms of institutionalization, in the coming months. WOCAT QA and QT 105. Training was given, in 2012, in the use of WOCAT’s approaches (QA) and technologies (QT) questionnaires. However this training was apparently hurried and had to be repeated in some instances. 25 technologies have been chosen for documentation and (so far) six approaches. There are still doubts about how well the requisite skills have been absorbed, as well as questions about the choice of technologies (in particular) that have been selected. LADA-local 106. Similarly, training in LADA-local (characterization of degradation and potential for SLM at micro-catchment level) was given in 2012. There were 45 participants in the training course. Evidence

7 Source: data solicited by evaluation team from NPMs in all four countries

suggests that this training could have been better if carried out earlier in the project, well before characterization and planning of microundoubtedly of value, does the extra work involved in implementing LADAprocess of participatory appraisal and also run against the principle of ‘Nevertheless, as with the training in mapbuilt for the future. Payment for Environmental (or Ecosystem) Services (PES) 107. Starting with a workshop on PES in DarPES was laid. A review was then carried out of PES in the region in 2010, and a workshop held at Kabale, Uganda, in 2011. It was not until 2012 that PES was introduced as an ‘addthat year, after a roadmap had been drawn up, PES training was given to catchment teams. In Burundi contracts have been awarded to hydcountries – with the possible exception of Rwanda. Demonstrations 108. There are now ‘demonstrations’ within 45 micro-catchments with 121 farmer field schools involved in these activities. This is the powerhouse of the project – and there is a real sense of vibrancy, and clear demand by communities for assistance with their percproblems of environmental change and land degradation. As indicated in the table below currently 5,350 hectares of land have been brought under SLM, as well as 28kilometers of riverbank protected. Table 9: Summary of Activities Accomplished in the Demonstrations

Source: compiled from data solicited by evaluation team from

109. From these demonstrations, a community level to engage communities in sustainable agriculture and livelihoods and soil conservation; this is being achieved through careful training in FFS methodologyevolving from a variation of FFS termed FFSand the strategic use of service providers to coach the schools. The process includes: 8 “Optimal ignorance” is part of the participatory development theory that developed in the 1980s. It stresses the need not to lose time ongathering information that is not essential forfeasibility studies.


34

suggests that this training could have been better if carried out earlier in the project, well before characterization and planning of micro-catchments began. There is another issue here: while undoubtedly of value, does the extra work involved in implementing LADA-local slow down the process of participatory appraisal and also run against the principle of ‘optimal ignoranceNevertheless, as with the training in mapping and WOCAT methodology, LADA

Payment for Environmental (or Ecosystem) Services (PES)

Starting with a workshop on PES in Dar-es-Salaam during 2008 (pre-project) the basis for PES was laid. A review was then carried out of PES in the region in 2010, and a workshop held at Kabale, Uganda, in 2011. It was not until 2012 that PES was introduced as an ‘add-

, after a roadmap had been drawn up, PES training was given to catchment teams. In Burundi contracts have been awarded to hydro-monitoring teams. However little is likely to

with the possible exception of Rwanda.

re are now ‘demonstrations’ within 45 catchments with 121 farmer field schools involved

in these activities. This is the powerhouse of the project and there is a real sense of vibrancy, and clear demand

by communities for assistance with their perceived problems of environmental change and land degradation.

urrently 5,350 hectares SLM, as well as 283

Activities Accomplished in the Demonstrations at June 2013

data solicited by evaluation team from NPMs and RPC in all four countries

e demonstrations, a model is emerging as a holistic and integrated approach at the community level to engage communities in sustainable agriculture and livelihoods and soil conservation; this is being achieved through careful training in FFS methodology, tailored to SLM evolving from a variation of FFS termed FFS-LWM that had evolved in the early 2000s in the region),and the strategic use of service providers to coach the schools. The process includes:

” is part of the participatory development theory that developed in the 1980s. It stresses the need not to lose time ongathering information that is not essential for development purposes - and challenges the convention of detailed investigations and

A hillside (in Tanzania) previously covered with natural forest had been stripped of its natural vegetation and cultivation had caused erosion upslope. The spring that the villagers depended upon was drying up. Supported by K‘Protect the Environment inacted decisively, by outlawing cutting of trees and cultivation. Two years later, in 2013, the spring shows sure signs of recovery and the hillside is green again with trees. Birds and other animals are coming home.


suggests that this training could have been better if carried out earlier in the project, well before s another issue here: while

local slow down the optimal ignorance’8?

ping and WOCAT methodology, LADA-capacity has been

project) the basis for PES was laid. A review was then carried out of PES in the region in 2010, and a workshop held at

-on’ to LADA. Later , after a roadmap had been drawn up, PES training was given to catchment teams. In Burundi

to happen in the other

model is emerging as a holistic and integrated approach at the community level to engage communities in sustainable agriculture and livelihoods and soil

tailored to SLM (and d in the early 2000s in the region),

and the strategic use of service providers to coach the schools. The process includes:

” is part of the participatory development theory that developed in the 1980s. It stresses the need not to lose time on and challenges the convention of detailed investigations and

ide (in Tanzania) previously covered with natural forest had been stripped of its natural vegetation and cultivation had caused erosion upslope. The spring that the villagers depended upon was drying up. Supported by K-TAMP, the Protect the Environment in Kinyamugera’ group

acted decisively, by outlawing cutting of trees and cultivation. Two years later, in 2013, the spring shows sure signs of recovery and the hillside is green again with trees. Birds and other


35

• A participatory process engaging communities from day one using local NGOs/CBOs; • Catchment characterization and planning, including LADA-local, to identify land degradation

issues, status, community needs, SLM responses needed, etc.; • SLM practices to sustain and increase productivity, prevent soil erosion and provide fodder for

animals (while stabilizing SLM structures) and incorporation of agroforestry trees; • The establishment of watershed committees and implementation of self-regulated by-laws; • Improved practices to control pests and increase yields through better varieties and

management; • Promotion of goats and pigs but also fish ponds, etc. as economic activities and as providers of

manure for soil fertility; • Cash crops to generate income for the community such as potatoes, cabbage, eggplants, etc.;

and • Other activities such as improved cooking stoves, and establishment of kitchen gardens, etc.

110. The assessment conducted during this review indicates that this model will be a major part of the achievements of this project and has the potential to be a key element in the replicability and sustainability of the Kagera TAMP’s achievements.

4.3.3. Quality of Technical Content

111. There is no doubt that, without it being specifically mentioned in the project document, Kagera-TAMP has set out to pioneer the use of LADA-WOCAT methodologies. Thus LUS maps have been developed, and QM has been used to develop a series of maps of degradation and sustainable land management status and potential. LADA local assessments have been carried out in pilot areas in each of the countries. While the current evaluation has not been able to investigate the effectiveness of these activities there is a sense that none of these methodologies has as yet been fully accepted or proven to add significant value to existing approaches. However this is an open question – and one of the recommendations of this evaluation is to carry out a review of the effectiveness of these methodologies applied to Kagera TAMP at the end of the project (see Section 8). 112. Sustainable land management practices promoted by the project are generally standard and well-known, at least in the region. This is not a project that has specifically set out to experiment with and develop new technologies. That is commendable, as technology is not the limiting factor in the spread of SLM in the region. Nevertheless there could be more attention given to some technical detail and training in SLM practices. For example while rangeland/pasture rehabilitation is an excellent initiative (seen in Tanzania and Uganda) planting fodder grasses within the rangeland associated with structures9 may not be necessary or cost-effective. There also appears to be inadequate consideration of the potential of water harvesting other than infiltration ditches, and the role of zero-grazed livestock in the overall conservation cycle is probably undervalued. 113. Documentation of SLM through WOCAT’s QT has come up with a rather unbalanced set of technologies and is unclear how these were selected or the purpose of the output (within the database and/or hard copy). Most of the technologies have been already documented under WOCAT (various terraces, hedgerows, mulching, manuring, grass strips, agroforestry) and the added value of the

9 Those structures seen were fanya juu: contour ditches with the soil thrown upslope or fanya chini: contour ditches with soil thrown downslope.


36

documentation is unclear. More interesting would have been the documentation of emerging technologies (and associated approaches), especially rehabilitation of grazing land (as seen in Uganda) and re-establishment of natural forests to protect springs (as witnessed in Tanzania). More time might have been profitably spent searching WOCAT’s database for technical and approach-related ideas rather than simply focusing on adding to it. 114. One further concern is that of the experiments being carried out by the farmer field schools. Since these schools have adopted a research agenda into their approach, based on a development that incorporates AESA (agro-ecological systems assessment) in contrast to the ‘classical’ set-up of FFS, which is very much extension-oriented, that research must be carefully guided. Some evidence from the field suggests that, sometimes, the trials are neither fully replicated, controlled, and supported by research stations, nor simple field comparisons under the jurisdiction of the group. They fall somewhere in between. There is a danger of drawing definitive conclusions from a single season of a quasi-formal experiment – as some communities were heard to do.

4.3.4. Special achievements

115. One of Kagera-TAMP’s main contributions to the region’s development may turn out to be, at least partially, unplanned. While the project had aimed to use farmer field schools as the medium for community organization and learning, these appear to be evolving into semi-permanent organized groups enthusiastically implementing participatory plans for SLM while undertaking targeted research. In some cases these are leading to the establishment of catchment / watershed committees. A viable model is emerging. It is apparent that environmental concerns (including climate change) is felt at community level, and thus communities are an excellent entry point for intervention, and the “model” may be an exciting way forward. 116. In the case of Tanzania the project appears to have uncovered and stimulated a community initiative in spring rehabilitation through the re-establishment of native vegetation. This is already showing impact (spring flow has improved; cleanliness too) and there is a great sense of community pride. There will probably other examples of local SLM/IEM initiatives that can be uncovered and supported. The national programmes should look out for these, and recommend some of the most interesting and important to the regional unit for cross-learning.

4.3.5. Gender equality

117. FAO has a defined policy on gender equality. Its goal is to achieve equality between women and men in sustainable agricultural production and rural development for the elimination of hunger and poverty. Furthermore, it has five objectives to accomplish this goal:

• Women participate equally with men as decision-makers in rural institutions and in shaping laws, policies and programs;

• Women and men have equal access to and control over decent employment and income, land and other productive resources;

• Women and men have equal access to goods and services for agricultural development and to markets;

• Women’s work burden is reduced by 20% through improved technologies, services and infrastructure; and


37

• Percentage of agricultural aid committed to women/gender-equality related projects is increased to 30% of total agricultural aid.

118. The policy recognizes that the major responsibility for achieving this goal and objectives lies with its member countries. However, FAO is accountable for the quality, efficiency and timeliness of its contribution at the output level; including the monitoring of progress toward the achievement of its gender equality goal and objectives. FAO’s contributions toward achieving its gender equality goal and objectives include the generation and communication of gender disaggregated data; the development and sharing of gender equality norms and standards; the incorporation of gender analysis in the formulation, implementation and evaluation of all field programs and projects; the development of internal structures and systems to promote gender equality and ensure equal participation of men and women in decision making in FAO; and other similar actions. 119. There are two strategic directions in its implementation: (1) adopt gender mainstreaming internally in all its work; and (2) carry out programs and projects specifically targeted at women in cases where the gender equality gap is so large that women cannot access opportunities that are available. Furthermore, the policy states that the function of an evaluation is to assess the extent to which FAO addresses gender equality issues in the project. 120. Clearly, Kagera-TAMP falls under the first strategic direction stated above. The review of the project document reveals that there is no gender equality stocktaking exercise and analysis conducted as part of the contextual analysis of the project. Nevertheless, gender sensitivity and gender considerations have clearly been taken into account in the formulation of the project; proposing gender sensitive approaches where needed, including the need to pay attention to gender equality and the reduction of gender constraints in some cases. However, the review also noted that no gender – including sex-disaggregated data – reporting is made through the annual Project Implementation Reviews (PIRs). 121. From the observations made during the mission of the Evaluation Team, Kagera-TAMP has a mixed profile with respect to gender equality. Taking sex ratio as the simplest means of quantifying and assessing “gender equality” the review indicates that:

• At FAO, the project oversight is done by one man (Budget Holder) and one woman (Lead Technical Unit), sharing the decision-making process and most FAO consultants have been women so far;

• In the field, the project scores poorly at senior level with all four national coordinators being men, and the regional coordinator also;

• At grassroots level, however, there is a good distribution of men and women amongst the Farmer Field Schools (FFS) where the breakdown is 47% men 53% women overall . In all countries, there is provision in the constitutions of the farmer field schools for disabled people. Additionally, there is also a provision when constituting a FFS for an equal participation of men and women as FFS members, including the need to have a chair and a deputy chair for each FFS composed of one man and one woman for a gender balanced decision-making process;

• While there are no data to support this, there appeared to be a reasonable split between male and female staff amongst the service providers.


38

4.3.6. Capacity Development

122. The FAO Corporate Strategy on Capacity Development (2010)10 states that FAO’s approach to capacity development (CD) is intended as a principal ‘modus operandi’ underpinning FAO’s programme of work. At the core of this strategy, there is a CD framework that is an analytical tool to assess existing capacities and identify appropriate types of interventions for developing the required capacities in counties. This CD framework encompasses the need for technical capacities in the food and agriculture areas. It also focuses on four functional capacities that would enable countries to sustain the change initiatives supported by FAO: (1) policy and normative capacity; (2) knowledge capacity; (3) partnering capacity; and (4) implementation capacity. Finally, the framework has three dimensions: (i) enabling environment; (ii) organizations; and (iii) individuals. 123. Within this context, the project focuses on capacity development. Its expected outcome #3 is about “enhancing the capacity and knowledge at all levels for the promotion of – and technical support for – sustainable management of land and agro-ecosystems in the basin”. 124. Perhaps the clearest example of capacity built, echoed by most of the respondents, was the training given to farmer field school facilitators. Undoubtedly the consultant employed (from Tanzania) was much appreciated, and the results in the field are a testimony to that training. This training kick-starts a process under which future facilitators are nurtured through the FFS themselves – thus ensuring a degree at least of sustainability. 125. Less effective appears to have been the training given in LADA-WOCAT technology where it was said (by several respondents) that the training courses were inadequate in length and lacking sufficient hands-on exercises to bring the trainees up to the level of confident practitioners. General training in SLM and monitoring and evaluation also seems to have been only superficially covered; however the farmer field schools appear to be using generally appropriate technologies and are certainly keeping records. Certainly training in recording visible local biodiversity would have been – and still can be – a valuable addition to the project’s programme. 126. The use of local service providers to coach communities is an interesting innovation which appears to be working well. But what exactly the impact of these service providers will prove to be is open to judgment. What makes an effective provider? How well qualified are these entities? Do they bring the extra “flavour” of their specific expertise with them? This is an initiative that would be worthwhile evaluating at the end of the project’s term. 127. The review found that most capacity development activities supported by the project are strongly focused on the “individual” dimension in terms of transfer of knowledge and skill development. That is certainly a positive aim; however, at this point in time, more emphasis needs to be put on organizations (structures and mechanisms) and particularly on the enabling environment (policy, legislation and governance). As discussed in section 4.5.3, the success of this project depends on its capacity to scale-up its achievements at the Kagera River Basin level. In order to achieve this result, there is a clear need for developing the capacity of organizations involved in the management of the Kagera River Basin, and also a need to develop an adequate enabling environment for the basin including the necessary policy, legislation and governance frameworks. 10 http://www.fao.org/capacitydevelopment/en/


39

4.3.7. The Human- Rights Based Approach (HRBA)

128. The UN Common Understanding on Human Rights Based Approach (HRBA) was adopted by the UN Development Group (UNDG) in 2003. It is part of the FAO’s five UN Country Programming Principles that were introduced mid-2012 in FAO’s Project Cycle Guidelines, which includes normative principles: Human Rights Based Approaches (HRBA)/Right to Food/ Decent Work; Gender equality, and Environmental sustainability and two enabling principles: Capacity Development (CD) and Results-Based Management (RBM). 129. The purpose of the UN common understanding on HRBA was to ensure that UN agencies, funds and programmes apply a consistent Human Rights-Based Approach to common programming processes at global and regional levels, and especially at the country level in relation to the Common Country Assessment (CCA) and the UN Development Assistance Framework (UNDAF). The statement of this common understanding on HRBA includes three main directions: (1) All programmes of development co-operation, policies and technical assistance should further the realization of human rights as laid down in the Universal Declaration of Human Rights and other international human rights instruments; (2) Human rights standards contained in, and principles derived from, the Universal Declaration of Human Rights and other international human rights instruments guide all development cooperation and programming in all sectors and in all phases of the programming process; and (3) Programmes of development cooperation contribute to the development of the capacities of duty-bearers to meet their obligations and of ‘rights-holders’ to claim their rights. 130. At the time this project was designed, the HRBA guidelines were not in place yet. However, the review found that through the design of this project, a HRBA has been adopted. The dimensions of “right to food” and “decent rural employment concerns” have been integrated into the purpose of this project. The overall goal of the project is the “adoption of an integrated ecosystems approach for the management of land resources in the Kagera Basin will generate local, national and global benefits ….. leading to increased food security and improved rural livelihoods”. Furthermore, its development objective is “to improve the livelihoods and hence contribute to reduced poverty of rural communities in the Kagera Basin through more productive and sustainable resource management practices that are technically feasible and socio-economically viable”. The project seeks to achieve this objective through the adoption by farmers and herders of improved land use and agro-ecosystem management practices, and through market opportunities and other cost-benefit sharing mechanisms for the provision of environmental services.

4.3.8. Partnership and alliances

131. Partnerships and alliances have been part of the implementation strategy of Kagera TAMP since its outset. The project document identified potential linkages to be established by the project with other related programmes and projects, but also with some key organizations. Some are at the global level, such as K-TAMP’s relationships with the WOCAT network and the LADA initiative, where the FAO was already involved. Others are regional. These alliances are contributing to the effectiveness of the programme by providing the opportunity for collaboration, as well as access to lessons learned, best practices and in some cases critical data. Two main instruments are used to formalise the regional linkages: (1) the use of MOUs to define the area(s) of cooperation and the scope of the planned cooperation; (2) the invitation of partners to become members of the RPSC.


40

132. An MOU was signed in May 2012 with NBI to establish collaboration in the Kagera River Basin. The cooperation between the two entities was defined in this MOU, including knowledge sharing where NBI-NELSAP made available to Kagera TAMP its monograph and its “Kagera Basin Information and GIS Database” containing over 27GB of data. This was used in the baseline analysis for the project and for LUS mapping. Another MOU was signed with Vi-Agroforestry on October 2012. This instrument focuses on giving priority to issues concerning management of natural resources and agro-ecosystems and to knowledge generation and dissemination including programmes, projects and activities related to the sustainable management of land and agro-ecosystems. Finally, a third MOU is in the process of being signed with the LVBC for collaborating in the Kagera River Basin with the LVEMP-II project; a draft is being currently reviewed. 133. However, the Evaluation Team noted that the strategy to establish partnerships in the project document was somewhat more ambitious than the existing/soon to be signed MOUs. The project document states that the project will “catalyze inter-sectoral partnerships between institutions in all four countries to overcome barriers to SLM, including enhancement of institutional and human resource capacity for land use/resources planning”. Furthermore, it was envisioned that “relevant government bodies, academic bodies and partner programmes involved in land resources management, agriculture, biodiversity conservation and sustainable ecosystem management would be directly engaged in the project through strategic partnerships based on their comparative strengths”. It was planned that partnerships among concerned sectors, institutions, civil society and service providers would promote sustainable land use/management practices and integrated ecosystems approaches that generate local socio-economic benefits as well as global environmental benefits. Finally, it was also recognized in the project document that a collaborative network and partnership arrangements will need to be established between districts and among countries. The review indicates that so far, during this first implementation phase, the Kagera TAMP has given limited adequate attention to these partnership aspects. Discussions with the project management team indicates that more focus will be given to the anticipated partnership strategy in the second phase of the project; this will form an important basis for replication of the achievements from the demonstrations implemented in the four countries.

4.3.9. Risks and Assumptions / Risk Mitigation Management

134. Potential risks were analyzed at the outset of this project and summarized in the project document. Risks reviewed were seen as potentially affecting the sustainability of project’s achievements. They included:

• Agricultural and Environmental risks: Crop and livestock pest & disease outbreaks both in-country and transboundary, and crop failure due to climatic vagaries would seriously affect the project. They were rated as medium risks and the integrated management of agro-ecosystems promoted by the project was seen as the main mitigation measure.

• Political and institutional risks: High staff mobility and more seriously insecurity or civil strife and refugee movements within and between countries would compromise the project. These risks were rated as medium to high; including low risk of inadequate stakeholder involvement. The main mitigation measures proposed at the outset of the project were for the project implementation team to make efforts to obtain full cooperation of local and national government authorities for inter-sectoral processes and to work closely with all stakeholders to ensure timely achievement of project goals and outputs.

• Human capacity risks: Availability of motivated, competent staff for the posts of regional


41

coordinator and national project managers and for district level facilitators (designated by the government) and timely recruitment of national project managers (NPMs) and regional coordinator will be crucial to the success of the project. These risks were rated as high and it was proposed to mitigate them through the assessment of labor requirements of land management practices and the use of practices that reduce the promotion of drudgery as well as the development of community action plans to include equitable measures.

• Financial and Economic Risks: Severe seasonal price fluctuations, inflation, market failures could restrict community capacity to invest in SLM/SLaM; including large changes in exchange rates. These risks were rated as medium to high and mitigated through the demonstrations of multiple social, economic and environmental benefits of more integrated farming systems.

• Management risks: The project was recognized as complex, involving four governments, many sectors, several regional projects/mechanisms and multiple stakeholders. There was a risk that the implementation could be impeded by inadequate communication, lack of transparency, inappropriate management/organizational procedures, inadequate delegation of responsibility, lack of flexibility to adapt to change and inadequate office space and facilities. These risks were rated as medium and mitigated through efforts to set up the regional and national project management units and recruit personnel rapidly in the host countries, with support by concerned authorities to allocate required office space and designate support staff.

135. In addition to the analysis of these risks, more specific assumptions and risks were identified in the logical framework and documented in Annex 3 of the project document. For each expected result – objectives, outcomes and outputs – assumptions and risks were identified; there are presented in the table below. Table 10: List of Risks

Project Strategy Assumption / Risk

Environmental objective: To address the causes of land degradation and restore ecosystem health and function and generate a range of global environmental benefits across the Kagera basin through the introduction of adapted agro-ecosystem management approaches.

Development objective: To improve the livelihoods and hence contribute to reduced poverty of rural communities in the Kagera Basin through more productive and sustainable resource management practices that are technically feasible and socio-economically viable.

• Strong commitment to address land degradation within the context of sustainable development and poverty alleviation programmes in all four beneficiary countries.

• District offices commit staff and other necessary resources to TAMP implementation

• Absence of serious environmental events (drought leading to food shortage, flooding), crop and livestock (pests and diseases) shocks in project countries.


• Reports and decisions of district, national, river basin policy and planning mechanisms

• Project steering committee reports

• Technical reports and project progress reports

• Field surveys

• National and district financial accounts


• Action plan for the establishment of a supporting policy and legal framework for SLaM across the basin.

• National and regional workshop reports


42

Project Strategy Assumption / Risk


• Project progress reports

• Reports of staff and other stakeholder training workshops

• Targets being monitored by the project and districts


• LAMIS data (RS/GIS) including field monitoring of target areas

• Sample surveys of land degradation, agro-ecological systems analysis and agro-biodiversity in target areas by FFS and technical staff will include LADA-local visual indicators of o Soil properties and erosion backed up by soil C sampling; o Vegetation/litter cover/bare soil/ extent and effect of burning; o Water resources and drought o Inter and intra-species and habitat diversity o Land productivity under different land use types (inputs/

yields/ other NR products e.g. fuel)

• Household surveys in target communities /districts (comparing 360 sample households/ FFS members and controls; analysis of land degradation, poverty; health; food security, vulnerability inter-relations)


• Project steering committee reports

• Technical reports and project progress reports

Source: Project Document

136. Risks are monitored regularly by the project implementation team and reviewed once a year through the PIR process. However, it seems that risks are re-evaluated once a year (low, medium, high) but the mitigation of high risks is not fully documented in the PIRs. As per PIR guidelines, all substantial to high risks should be included in a “top risk mitigation plan” indicating what measure(s)/action(s) will be taken to mitigate these particular risks and who is responsible.

4.4. Potential Impacts of Project

137. This section discusses the progress made so far toward the achievement of strategies and outcomes of the project and the likelihood that project achievements will have a long-term positive impact on the local, regional and global environment, and on livelihoods.

4.4.1. Potential Impacts of Project’s Achievements

138. According to the logframe (see Request for CEO Endorsement, 2009) the project’s benefits and results were intended to include (in paraphrase):

i. A shift to SLM/ SLaM, contributing to reduced poverty, food insecurity and risks; ii. Adoption of SLM/SLaM contributing to climate change adaptation and mitigation through

efficient use of rainwater, enhanced carbon sequestration and more resilient livelihood strategies; and

iii. Reduced erosion and sediment load in rivers, and reduced atmospheric deposition through reduced burning, increased vegetation cover and agrobiodiversity and improved productivity.


43

139. Furthermore environmental results that could be monitored would include: • Sustainable land and agro-ecosystem management practices implemented on 43,700 hectares of

land by PY3 and 100,000 ha. by PY5; • 10 percent increase in production contributing to livelihoods; • 20 percent increase in carbon stores on 30,500 ha of land; • Effective control of soil erosion observed in SLM demonstrations and farmer plots; • Four micro-catchments identified and sediment loads [reduced and] monitored; • Training/upscaling capacity in SLM established through 120,000 community members/local

decision makers sensitized; 4,000 persons trained and adopting /upscaling SLM (3,600 FFS members, 300 technical staff, 200-250 policy makers); training materials and best practices widely available and SLM demonstrations in place and benefits communicated widely; and,

• An enabling environment for regional cooperation in SLM. 140. From the foregoing it is clear that the benefits to be achieved from the project’s direct activities are contingent on (a) the area covered by SLM, (b) capacity built, and (c) an enabling environment established. 141. The cornerstone for benefits and impact, that is the area covered by SLM/SLaM, is running a long way behind schedule – at (currently; June 2013) 5,345 ha. That represents just 12% of the target by PY3. Even with a rapid acceleration in the rate of implementation, it is unlikely that even 25% of the final total can be achieved and this will imply a much lower impact than (over-ambitiously) projected by the end of the project’s life. 142. Nevertheless, ultimate, post-project impact can be awarded a more positive prognosis. That is because the target for capacity built – assuming that quality matches quantity – is beginning to come quite close to that predicted. While the number of community members sensitized has yet to be assessed, the figures for those trained comes quite close, already, to the targets. There is no reason why spread in terms of those aware of SLM (‘sensitized’) should not rise quite quickly. 143. The other element underpinning post-project sustained impact is less convincing: the enabling environment for regional cooperation in SLM has hardly been forged, despite quite strong individual country foundations - particularly through the integration of the emerging community engagement model into national extension services. 144. Monitoring of these various benefits – local or global – has hardly been initiated, though this needs to be a strong focus in the forthcoming year.

4.4.2. Potential Global Environmental Benefits (GEBs)

145. The general points and observation in the above section can be translated into terms of global environmental benefits - as these are drawn more or less directly from the rate at which SLM is implemented. Under the land degradation focal area of the GEF, it can be safely said that local and global benefits come in association, though naturally significant levels of GEBs will take longer to be generated and to have impact. The major GEBs that are envisaged from the project are carbon sequestration (through carbon taken up in soil and vegetation), improved river flow (both the flow regime and water quality), increased biodiversity, and decreased atmospheric pollution from reduced burning. Once again, while current achievements in terms of SLM are well behind target, and unlikely


44

to reach even 25% of the total of 100,000 hectares envisaged by the end of the project, the trend is positive. If the establishment of regional cooperation becomes a reality (see comment above in 4.4.1), and continued support to the process is ensured, then potential GEBs can certainly be translated into real GEBs.

4.4.3. Impacts on Local Environment, Poverty and Other Socio-Economic Issues

146. The impact of the project at the local level can more easily be seen and measured. Where the project concentrates its activities – namely within micro-catchments and communities – there is already a discernable change. This can be observed within these demonstration areas, and is articulated by the communities who are unanimous in their support for the project’s initiatives. Of the 15-20 farmer field schools visited by the Evaluation Team there was a strong local consensus that things were improving (in terms of the environment and livelihoods). Even making allowances for the normal tendency to ‘talk up’ project benefits, this appeared to be a genuine feeling. Without doubt a sense of local ownership, responsibility and ‘can do’ has been kindled. Rehabilitation of grazing lands can be clearly seen, as well as (in one case in Tanzania) the reestablishment of natural forest projecting a spring that has become rejuvenated. Protected riverbanks (through ‘buffer strips’) have become mini-hotspots of biological diversity. These multiple local co-benefits on the environment and on society are happening: it may be at a small scale, but the trend is positive.

4.5. Sustainability of Project Achievements

147. This section discusses the potential for the long-term sustainability of project achievements. It is an indication of whether outcomes (end of programme results) and positive impacts (long-term results) are likely to continue after the project ends.

4.5.1. Sustainability of Results Achieved

148. The prospects for the long-term sustainability of project achievements are moderately likely (ML). SLM practices demonstrated in the Kagera River Basin are being implemented in selected communities and they should be sustainable over the long-term. They will continue to provide benefits to these local communities in the Basin and overall improve their livelihoods. The benefits from the project-supported approach include higher crop yields, less rainwater runoff, better income generation activities, increased food security and community strengthening. However, despite good sustainability prospects for the demonstrations, the success of this project ultimately depends on its capacity to scale-up its achievements at the Kagera River Basin level. Project achievements need to be mainstreamed and institutionalized in national policies, strategies and programmes but also in local planning processes at the regional and local levels. So far, the project is too focused on the demonstration element. It is time that this focus shifts to scaling-up the achievements; the project is already in its second half and it is critical that activities are taking place to promote and assure the scaling-up of these achievements. 149. The strategy for the sustainability of project achievements identified at the outset of the project includes three main axes:


45

Institutional Sustainability 150. It was expected that project achievements would be sustained due to the commitments and priorities of the countries to the conservation and sustainable management of the shared natural resources of the basin in the medium and long term, and to the improvements in the livelihoods of local people. Additionally, regional co-operation to address transboundary issues was also seen as a contributor to the institutional sustainability. Finally, it was anticipated that the generation of socio-economic benefits as a result of improved land use systems/practices and resulting sustained ecosystem function would help ensure the wider uptake of improved practices in the target districts and across the basin. 151. It is true that project supported activities should contribute to these priorities and that information and awareness raising activities should enable a larger population to be aware of opportunities and potential benefits of SLM and to replicate relevant actions in their communities. However, there are moderate risks that affect this dimension of sustainability such as the lack of visibility of the project and the slow advancement of the transboundary coordination activities as well as the mainstreaming of SLM activities into the respective policy and legislation frameworks of the four countries. Environmental Sustainability 152. From an environmental perspective, it was anticipated that, through the adoption of improved agro-ecosystem management practices, the benefits generated in terms of ecosystem function, and as a result the improved livelihoods and food security, should ensure the sustained uptake of these practices by land users and consequently stopping the causes of land degradation and restoring ecosystem health and functions. 153. At the demonstration sites level there are no particular risks affecting this dimension of sustainability; moreover the enthusiasm surrounding the project-supported activities in the communities is almost a guarantee that these practices will be sustained over the long term. The review of these demonstrations indicated that communities are already benefiting from these practices in term of protecting their ecosystem and of harvesting higher crop yields. Financial Sustainability 154. The financial dimension of sustainability was viewed as critical. It was recognized that the success and the wider scaling-up and adoption of improved diversified ecosystems and management practices will depend on secure funding by districts, through national and regional agricultural and environmental development programmes supported by the donor community. It was anticipated that through the mainstreaming of project achievements into major national development programmes, as well as into district and community planning processes, it would ensure the institutionalization of regular support from governments (financial and human resources) and local communities (in kind and cash) as well as from transboundary investment programmes such as LVEMP and NELSAP. 155. Considering the assumptions made under this dimension of sustainability and the progress made so far, the financial sustainability of project achievements is uncertain. It is true that by the promotion of sustainable use of resources, land users should benefit from increasing economic returns through sustained productivity, payments for environmental services, opportunities for neglected


46

biodiversity, etc. However, at this point in time, it is less certain that the project will be able to ensure regular support from governments. The project is still focusing on the implementation of demonstrations and it may run out of time to institutionalize and mainstream its achievements into national development programmes and into community planning processes, and obtain government support after the project ends.

4.5.2. Socio-Economic Risks

156. As discussed in other sections of this report, the demonstrations implemented under outcome 4 have had positive impacts on the communities in the Kagera River Basin. Observations during the field mission of the Evaluation Team indicate that the livelihoods of these communities has improved. Furthermore, there is good potential for more socio-economic impacts in the medium and long-term. As discussed in section 4.3.2, a model is emerging at the community level and within the demonstration sites there is a good local ownership of this approach, which should contribute greatly to the long-term sustainability of these demonstrations. The review indicates that there are no particular socio-economic risks that could threaten the sustainability of project achievements. 157. However, as mentioned above, there is a certain political risk that project achievements may not be scaled-up nor replicated throughout the Kagera River Basin. So far, the project lacks clear visibility in each country and there is limited uptake of project achievements by related government institutions. As the project is moving into its second phase, it is time to start assessing the results from these demonstrations, increase the visibility of these results in the region and promote the institutionalization of these achievements.

4.5.3. Catalytic Role / Replicability of Project Achievements

158. The environmental objective of the project is “to address the causes of land degradation and restore ecosystem health and function and generate a range of global environmental benefits across the Kagera basin through the introduction of adapted agro-ecosystem management approaches”. As stated in the project document, “the success of the project over the four and a half year funding period and in the medium to long term across the Kagera River Basin, is highly dependent on the widespread replication of successful outcomes and lessons learned from the target communities and micro-catchments, where on the ground activities will be established and tested in the initial 2 years and subsequently scaled up”. 159. Widespread replication is the most critical success factor for this project. So far the project has focused on the development of SLM practices and approaches, which have been demonstrated in selected pilot areas in the four countries. These demonstrations are the first step to catalyze these practices and approaches, which is further reinforced by awareness raising activities and training of relevant people involved in the management of these agro-ecosystems. Information on lessons learned have started to be disseminated but more is needed in the near future to contribute to the widespread replication of project achievements. 160. From these demonstrations, some replication has started to happen at the local level. This is the case in Burundi where the FFS in Rumuri – after seeing the results in other FFSs - started on its own to dig 5 km of contour ditches to capture rainwater runoff. In Uganda, FFS groups are forming


47

spontaneously – independent of project help. However, despite the fact that these – and other - localized replications are excellent, this is still very limited from a Kagera River Basin point of view. The project needs to develop a replication/scaling-up strategy whereby entry points to replicate these achievements are identified in the four countries and activities to promote replication and scaling-up implemented. The project management team has at its disposal excellent initial results from these demonstrations in four countries and in varied ecosystems. The project needs now to shift its focus to the lessons learned from these demonstrations, increase its visibility nationally and regionally and start developing its strategy to replicate and scale-up its achievements; this includes scaling-up through its partnerships with regional initiatives such as NBI and LVEMPII. 161. Finally, the Evaluation Team noted that the project document has no provision for the post-project period. No exit strategy is planned, particularly under outcome 4 where communities are engaged in project-supported activities. It is still early enough to develop an exit strategy to properly withdraw project support and, at the same time, maximize the sustainability of achievements.


48

5. Ratings

162. Below is a table summarizing the ratings as requested in the TORs. It includes all the required performance criteria rated as per the rating scales presented in Annex D of the TORs. Table 11: Project Mid-Term Evaluation Ratings

Evaluation Criteria Rating Comments Ref. Section

Achievement of objectives MS • On track with Environmental and Development objectives, but slower than envisaged.

4.2.1

Attainment of outputs and activities

MS • Progress so far limited, but speed of implementation recently picked-up.

4.3.2

Progress towards meeting GEF-4 focal area priorities/objectives

S • Contributing especially to LDFA’s SO-2 outcome

“communities benefit from applying and disseminating SLM practices”.

4.1.1

Cost-effectiveness S • Difficult to assess but financial resources are flowing, and appear to be effectively used.

4.2.2

Impact MS • Impact is ultimately contingent on extent of SLM/ SLaM: currently running well behind projections.

4.4.1

Risk and Risk management S • Risks monitored regularly by project, but some extra risks are worthy of attention.

4.3.9

Sustainability ML • Institutional sustainability is the key; at this stage there is potential but much yet to be realized.

4.5.1

Stakeholder participation HS • The most satisfactory aspect of all: at community level there is clear, popular, engagement.

4.2.4

Country ownership S • National steering committees well-formed and evidence of ownership emerging.

4.2.4

Implementation approach MS • FAO implementation procedures in place but not always conducive to effective management.

4.3.1

Financial planning MS

• Financial system unable to report financial data by project outcome. It needs a manual allocation of each expense to the proper outcome; it is a constraint for efficient financial and management planning.

4.2.2

Replicability MS • Demonstrations have laid the foundation but little evidence as yet of potential for replication

4.5.3

Monitoring and evaluation MU • Plan too complex, with doubtful cost-effectiveness. Basic data is available in field - but reporting poor.

4.2.5


49

6. Lessons Learned

163. Based on the review of project documents, interviews and meetings with key informants, and the analysis of this information, the Evaluation Team collated several lessons learned in the context of K-TAMP. Several are reconfirmation of existing lessons; many have wide applicability. They are as follows:

• Participation in national processes through national agencies leads to better visibility, improved ownership at that level, and helps in mainstreaming and upscaling;

• SLM projects in Sub-Saharan Africa are consistently overambitious and thus consistently ‘underachieve’: K-TAMP confirms this lesson;

• Projects built around the typical 3 (or 4) year cycle do not have time to achieve full momentum (because of inevitable delays in establishing mechanisms) before they close. K-TAMP confirms that a minimum of 5 years is required;

• There is a danger that projects like Kagera-TAMP suffer by being too ‘cluttered’ with testing a whole range of add-on methodologies: this takes attention and resources away from the core business of SLM/SLaM;

• Training in new methodologies (LADA-local; LADA-WOCAT QM, WOCAT QA/QT; PES etc) must come very early in a project’s life and have direct applicability, or risk being viewed as a burden;

• WOCAT QA and QT must be well-targeted to add value to project or to the international database;

• Strong engagement of communities from the outset - based on their environmental and socio-economic ‘felt-needs’ – can lead to enthusiastic action and a firm sense of ownership;

• Farmer field schools can be modified to accommodate SLM/SLaM by being inter alia longer term and focusing on productive aspects of land management;

• A simple model built around participatory processes, incorporating PRA/ LADA-local, modified FFS, local service providers and income-generating enterprises can unlock communities’ potential to implement SLM;

• Watershed committees, developing and administrating their own bye-laws, are key to local environmental improvements;

• A lack of cross-visits at all levels directly detracts from the sense and purpose of ‘regionality’ in a multi-country project;

• Simple backstopping feedback to country programmes is crucial to appreciate what is happening in the field, to provide positive feedback, and to help correct and realign processes where necessary;

• Opportunities to join forces – through MOUs etc. – with regional bodies and associated projects/programmes (including GEF projects) pays dividends within regional projects in building up synergies and strengthening sustainability.


50

7. Conclusions and Recommendations

7.1. Main Conclusions

7.1.1. Relevance of the Project

164. The Kagera TAMP project is relevant for the region. It directly addresses root causes of land degradation. Well-aligned with the GEF Land Degradation Focal Area’s strategic objectives, the project is a direct response to root causes identified during the design of this project, which revealed several governance-related causes of land degradation. In order to be effective in addressing these root causes, the project was well-aligned with the regional policy context related to the Kagera River Basin, particularly with the NBI-NELSAP, the LVBC and its LVEMP-II project and NEPAD and also with the enabling environments of the four countries. 165. However, it is noted that the project has not been yet in a position to advise and submit any proposals for mainstreaming SLM into the enabling environments of these countries. There are several potential entry points where sustainable management of land and agro-ecosystems could be mainstreamed at both the national and regional/local levels and it should become a priority for the remaining implementation period to maximize the long-term sustainability of project achievements. It includes the possible institutionalization of the overall approach used by the project at the community level within the extension services in each country and within the regional and local development structures. 166. A detailed design that extended over a period of almost 10 years between the concept and the start-up of the project. It took 10 years between the approval of a PDF-A grant in November 1999 to develop the concept, and the start-up of the project in September 2009. It included an extensive PDF-B phase with a grant of USD 700,000 to produce a full project brief. During these years, an extensive review was conducted to position and justify the project. A project brief was finalized in 2005-2006 but due to the lack of funds under GEF-3, it had to be re-submitted to the next GEF financial cycle under GEF-4 in 2007. The project was finally endorsed by the GEF-CEO in May 2009. Despite this long period, the project remained constant in its objectives and proposed methodology and evolved little between 2006 and 2009. 167. A project with very ambitious targets and a relatively short timeframe. The logic model of the project consists of an overall goal, an environmental objective as well as a development objective, five outcomes and 13 outputs. It has four main directions: a) transboundary coordination; b) enabling environment; c) capacity development; and d) extensive demonstration of improved land management practices in the four riparian countries. However, this is a very ambitious project considering the allocated resources and the given timeframe to implement it. Its demonstration component covers 45 non-contiguous catchments dispersed in 21 administrative units in four countries. The duration is 4.5 years with a core, and very ambitious targets, of 100,000 ha under new SLaM.


51

7.1.2. Efficiency of the Project

168. The administrative and financial procedures and the financial system in place to record

project expenditures do not provide adequate support to project management for an efficient implementation and monitoring of the project. The channeling of financial resources to the project sites is not easy, due to geographical locations in the case of Tanzania and Uganda but also to the centralized nature of the system. Additionally, the FAO financial system records project expenditures by expense category only - such as consultants, training and travel expenses. It is not capable of disaggregating this financial information by GEF outcome or output. Therefore, all financial management reports need to be calculated manually by allocating project expenditure to the corresponding outcome. Overtime, it does not provide project managers with accurate and timely financial information. The limitations of these financial and administrative procedures hamper the decentralization of the project decision-making process to the regional and national level and the greater empowerment of the project staff: RPC and NPMs. 169. The project is behind schedule and runs the risk of not making enough provision for mainstreaming and replicating its achievements. The initial logic was to focus the first two years on the demonstrations (outcome 4) and then analyze the lessons learned and best practices and focus the remaining part of the project on scaling-up through mainstreaming, institutionalizing and replicating project achievements; including addressing transboundary issues and replicating best practices demonstrated in the pilot sites. 170. While there appears to have been a recent impressive pick-up in pace of implementation of the demonstrations centered around active and enthusiastic farmer field schools groups and vibrant community engagement, there are only 15 months of implementation remaining. Despite the fact that these demonstrations are not completed yet, there is already an emerging model of a holistic and integrated approach at the community level to engage communities in sustainable agriculture and livelihoods and soil conservation. It is time to start documenting this approach and mainstreaming it in national extension services (or equivalent) in each country. 171. The monitoring and reporting of the project is not adequate to measure progress made at the objective and outcome level – the GEF main reporting requirement. The M&E plan is complex, costs over 6% of the GEF grant, and its cost-effectiveness is questionable. With 49 indicators to monitor – some of which have multiple targets – it requires a lot of effort and time to produce accurate and timely monitoring information for project managers. Additionally, the assessment of these indicators reveals that most indicators at the output level are simply tracking the delivery of project activities, such as the number of demonstrations sites, FFS study plots, FFS Facilitators trained, technical staff and policy makers trained, number of participatory land use plans, etc. 172. Most indicators selected are specific, easy to measure, attainable and time-bound, but are not fully relevant when measuring how well the project is progressing toward its environmental and developmental objectives. They are too focused on project deliverables, simply measuring if the project has delivered its targeted activities or not. In addition, the three performance indicators selected to measure the progress made at the objectives level are not comprehensive enough. They do not include any measurement of to what extent the capacity needed to address the causes of land degradation and restore ecosystem health has been developed - nor the extent to which it has been


52

replicated/mainstreamed. This is the “raison d’être” of the project and progress should be measured at this level. 173. In addition, the PIR – a GEF reporting requirement – is the instrument to report, annually, progress made by the project. It includes section 3 that focuses on how the project is progressing towards achieving its objectives and outcomes, using a set of 16 indicators that were identified by the project to measure the performance of the project at these levels. PIRs are key reports documenting the progress made by the project towards achieving its high level results. However, these reports could be improved, particularly section 3 with more details about the progress made and how the project is meeting its expected results.

7.1.3. Effectiveness of the Project

174. Progress so far is limited though accelerating, but the outputs are poorly integrated. The project is making progress, though its achievements are still limited when compared to the targets set for the end of the project. The project has been delivering outputs but with limited integration among them. For instance, demonstrations are taking place in the four countries with little interconnection; some work has been put in to produce land degradation maps for the four countries but with no direct relation to the demonstration sites; WOCAT questionnaires have been completed but with limited added value to the activities implemented in the demonstration sites. The assessment in the four directions of the project reveals the following:

• Transboundary Coordination: A series of useful transboundary studies have been carried out/are planned on various issues (policy/legal; livestock; agro-ecosystems; resource competition; water; etc.) but there is little evidence that these studies have yet been put to use.

• Enabling policy, planning and legislative conditions: Currently, achievements are running some distance behind the set targets. There is little evidence yet of policy being formed at a regional or basin level resultant from the project.

• Capacity development: The targets specified in the project document are very ambitious and though capacity development is underway, the results are mixed and there is a long road to travel.

• Improved land management practices: Targets are daunting under this outcome with the involvement of 200 communities, 45,000 ha under SLM/SLaM and 100,000 ha ‘transformed’ towards productive and sustainable agricultural ecosystems by the end of the project. While there appears to have been a recent impressive pick-up in pace of implementation centered around active and enthusiastic farmer field schools groups and vibrant community engagement, the overall achievements to-date of just 5,350 hectares under SLM are a very long distance from the (admittedly) over-ambitious targets set in the project document. The idea of implementing a payment for ecosystem services model will achieve little or nothing having being introduced too late in a project with too brief a life.

175. SLM practices promoted by the project are generally standard, well-known and proven in the region. SLM practices promoted by the project are generally standard, well-known and proven, at least in the region. This is not a project that has specifically set out to experiment with and develop new technologies. Nevertheless there could have been more attention given to some technical detail and training in SLM practices. There also appears to be inadequate consideration of the potential of water harvesting other than infiltration ditches, and the role of zero-grazed livestock in the overall conservation cycle is probably undervalued.


53

176. The use of LADA-WOCAT methodologies has been piloted but have apparently not yet been fully accepted or proven to add significant value to existing approaches. Kagera-TAMP has commendably set out to pioneer the use of LADA-WOCAT methodologies. Thus LUS maps have been developed, and QM has been used to develop a series of maps of degradation and sustainable land management status and potential. LADA-local assessments have been carried out in pilot areas in each of the countries. While the current evaluation has not been able to investigate the effectiveness of these activities there is a sense that none of these methodologies has, as yet, been fully accepted or proven to add significant value to existing approaches. A foundation however has been laid – and a full assessment of the impact of these methodologies can only be carried out at the termination of the project. 177. Finally, the documentation of SLM practices through WOCAT’s QT has come up with a rather unbalanced set of technologies and is unclear how these were selected or the purpose of the output. Most of the technologies have been already documented under WOCAT (various terraces, hedgerows, mulching, manuring, grass strips, agroforestry) and the added value of the documentation is not evident. More interesting would have been the documentation of emerging technologies (and associated approaches), especially rehabilitation of grazing land (as seen in Uganda) and re-establishment of natural forests to protect springs (as witnessed in Tanzania). 178. A model is emerging from the demonstrations as a holistic and integrated approach at

the community level to engage groups in sustainable agriculture and livelihoods and soil conservation. There are now “demonstrations” within 45 micro-catchments with 121 farmer field schools involved in these activities. This is the powerhouse of the project – and there is a real sense of vibrancy, and clear demand by communities for assistance with their perceived problems of environmental change and land degradation. The process of engaging communities is excellent, with the involvement of about 3,500 people (47% men and 53% women) through these 121 FFSs. 179. A model is emerging as a holistic and integrated approach at the community level to engage groups in sustainable agriculture and livelihoods and soil conservation; this is being achieved through careful training in FFS methodology – tailored to SLM – and the strategic use of service providers to coach the schools. The process includes:

• A participatory process engaging communities from day one using local NGOs/CBOs; • Catchment characterization and planning, including LADA-local, to identify issues, status,

community needs, etc.; • SLM practices to sustain and increase productivity, prevent soil erosion and provide fodder for

animals (while stabilizing SLM structures) and incorporation of agroforestry trees; • The establishment of watershed committees and implementation of self-regulated bye-laws; • Improved practices to control pests and increase yields through better varieties and

management; • Promotion of goats and pigs but also fish ponds, etc. as economic activities and as providers of

manure for soil fertility; • Cash crops to generate income for the community such as potatoes, cabbage, eggplants, etc.;

and • Other activities such as improved cooking stoves, and establishment of kitchen gardens, etc.


54

180. This model will be a major legacy of this project and has the potential to be a key element in the replicability and sustainability of the Kagera TAMP’s achievements. 181. The project focuses on the “individual” dimension of capacity development with insufficient emphasis on organizations and particularly on the enabling environment. Most capacity development activities supported by the project are strongly focused on the “individual” dimension in terms of transfer of knowledge and skill development. This is part of development capacity of a system, but more emphasis needs to be put on organizations (structures and mechanisms) and particularly on the enabling environment (policy, legislation and governance). The success of the project depends on its capacity to scale-up its achievements at the Kagera River Basin level. In order to achieve this result, there is a clear need for not only developing the capacity of people involved in the management of the Kagera River Basin, but also developing the capacity of relevant organizations and to develop an adequate enabling environment for the basin including the necessary policy, legislation and governance frameworks.

7.1.4. Potential Impact and Sustainability of Project Achievements

182. Currently, the project is too focused on its demonstrations; it is time to shift focus to scaling-up project achievements in order to improve the long-term sustainability prospects. The

success of the project depends closely on the mainstreaming and replication of its achievements. As stated in the project document, “the success of the project over the four and a half year funding period and in the medium to long term across the Kagera River Basin, is highly dependent on the widespread replication of successful outcomes and lessons learned from the target communities and micro-catchments, where on the ground activities will be established and tested in the initial 2 years and subsequently scaled up”. Widespread replication is the most critical success criterion for this project. 183. So far the project has focused on the demonstrations of SLM practices and approaches in selected pilot areas in the four countries. These demonstrations are the first step in catalyzing these practices and approaches, which is further reinforced by awareness raising activities and training of relevant people involved in the management of these agro-ecosystems. However, instead of 24 months as per plan, the implementation is already in its 39th month and has only 15 more to go. 184. Some replication has started to happen at the local level. However, this replication approach is very limited from a Kagera River Basin point of view; so far it is minor replications locally around the selected sites. It is time for the project to develop a replication/scaling-up strategy where entry points to replicate these achievements are identified in the four countries and activities to promote replication and scaling-up implemented. The project needs now to shift its focus to the lessons learned from these demonstrations, increase its visibility nationally and regionally and start developing its strategy to replicate and scale-up its achievements; including scaling-up through its partnerships with regional initiatives such as NBI and LVEMPII.

7.2. Recommendations

185. Based on the findings of this final evaluation, the following recommendations are suggested. They are in no particular order of hierarchy.


55

Recommendation 1: It is recommended that three reviews are conducted before the terminal

evaluation/end of the project:

• Investigate the effectiveness of the LADA-WOCAT methodologies;

• Document the innovative and successful FFS approach used by the project;

• Document the full extent of the area covered by SLM/SLaM under demonstrations supported by the project.

Issues to address 186. The current evaluation has not been able to investigate the effectiveness of the LADA-WOCAT methodologies. However, there is a sense that none of these methodologies has as yet proven to add significant value to existing approaches – but not all activities are completed. Thus this impression should be tested by conducting a full review of the effectiveness of these methodologies before the end of the project. 187. A second review should focus on documenting the innovative and successful FFS approach used by the project. This is a non-classical, modified version of the farmer field school model. Thus the original concept of a one-season “seed-to-seed” or “egg-to-egg” model underpinning an ephemeral group has been developed into a longer lasting/semi-permanent group that tracks the benefits of SLM (and associated economic land-based enterprises) over a number of seasons. The other key difference from the classical model is the way that the FFSs under Kagera-TAMP combine participatory research with simple demonstration and learning. 188. A third review is recommended to document the full extent of the area covered by SLM/SLaM under demonstrations supported by the project, which should be achieved before the terminal evaluation. There is no time to conduct such a review in the context of an evaluation but it is crucial information for the terminal evaluation to have these demonstrations properly documented. It is the core of the project and the basis on which the scaling-up of project achievements should happen. The review should cover the extent of these demonstrations with a particularly emphasis on the process: engagement of stakeholders, decision-making process, role of service providers, etc.

Recommendation 2: It is recommended that more focus is given to developing the capacity of

related organizations (structures and mechanisms) and of an enabling environment to provide

the adequate policy, legislation and governance frameworks.

Issues to address 189. So far, most capacity development activities supported by the project have been strongly focused on the “individual” dimension in terms of transfer of knowledge and skill development. This is part of the overall capacity of a system but more emphasis needs to be put on organizations (structures and mechanisms) and particularly on the enabling environment (policy, legislation and governance). The steering committees (regional and national) have a responsibility to guide the project in this respect.

Recommendation 3: It is recommended that a planning exercise is conducted, including the

development/ refinement of:


56

• A replication/scaling-up strategy where entry points to replicate these achievements are identified in the four countries as well as activities to promote replication and scaling-up;

• A work plan for the entire remaining implementation period (15 months) with the associated budget;

• A project exit strategy; particularly for proper disengagement of project support in the selected pilot sites.

Issues to address 190. While it is recognized that Kagera-TAMP has plans to enter into a faster tempo of on-the–ground activities, the project indeed needs to shift gears and focus on scaling-up its achievements. There are 15 months remaining and a budget of approximately USD 2.85M remaining. The review indicates that there are several entry points where the sustainable management of land and agro-ecosystems could be mainstreamed such as the national extension services in each country, local development planning, land use planning, local bye-laws, etc. Considering the remaining timeline of the project, it should become a priority. The project needs now to shift its focus to the lessons learned from these demonstrations, increase its visibility nationally and regionally and start developing its strategy to replicate and scale-up its achievements; including scaling-up through its partnerships with regional initiatives such as NBI and LVEMPII. 191. In addition to scaling-up project achievements, there are also several elements to bring to a closure such as the need to summarize all transboundary studies conducted under outcome 1 and make recommendations for a greater regional cooperation; the need to publish and handover the LUS and QM maps to the relevant national organizations; the need to finalize the project involvement in the PES area; and finally the need to plan the exit from the pilot sites. 192. A work plan with a budget for the remaining implementation period should be prepared and presented at the next RPSC. It is important to remember that ultimately the success of the project depends on its capacity to scale-up its achievements. The quicker and the more resources the project will have for this end phase, the more successful the project will be. 193. Finally, considering that no planned provision for the post-project period was included in the project document, it is recommended to develop an exit strategy to identify a range of activities to be supported in the pilot sites emphasizing sustainability of these demonstration sites over the coming 15 months.

Recommendation 4: It is recommended that specific project activities are managed as follows:

• No new PES activities are developed;

• The LUS and QM maps are handed over through a three step process;

• The potential for an informal GEF partnership be investigated in Uganda

Issues to address 194. Initial PES screening activities took place in Burundi and Rwanda on water quality. Considering the limited implementation time left and the need to focus on scaling-up project


57

achievements, the project should limit its direct engagement in this area. It is recommended to pursue what was started in Burundi and Rwanda and try to partner with VI-Agroforestry on their carbon initiative if the four countries are interested in joining. 195. LUS and QM maps were produced for the four countries. These maps contain about 80 layers of information and are developed using the ArcGIS software. The development of these maps included a series of workshops – GIS workshops and workshops with broader group of stakeholders - accompanied by fieldwork to complete questionnaires. These maps are now ready to be published and handed over to national partners in the four countries. The recommendation includes (1) a brief assessment to identify the best organizations to take over these database/maps and their capacity needs; (2) the publishing of some of these maps in partnerships with the relevant national organizations; and (3) the handover through national workshops to showcase these products. It was noted that these steps and the publication of these maps with the relevant national organizations was already a plan in place. 196. In Uganda there are two other GEF Land Degradation Focal Area projects currently underway within the Kagera basin – one under UNDP, the other under UNEP. It is well worth investigating whether a GEF “partnership” could be developed at the national level to encourage synergies. This partnership concept could be applied regionally if any other GEF LDFA projects come on-line. The same is true in Rwanda where two other relevant GEF projects are ongoing/about to start.

Recommendation 5: It is recommended that the time of the project is extended, if the budget

permits.

Issues to address 197. At this point in time, it is difficult to assess how much budget will remain by the end of the project (August 2014), the figures show that 55% of the budget has been consumed so far versus 72% of the time. Considering that to spend the entire budget by August 2014, the project would need to expend twice as much on a monthly basis ($189k vs. $90k per month), it is doubtful that the budget will be expended by August 2014. If there is any budget left unspent by the end of the project, a no-cost extension of the project is recommended with a firm focus on mainstreaming, institutionalizing and replicating activities. The duration of this no-cost extension will depend on how much budget is remaining. Once the work plan for the remaining period and its associated budget are completed, the project management team should be in a position to assess how much budget remains to finance this no-cost extension.

Recommendation 6: It is recommended that FAO decentralizes decision-making – including

financial approval - in countries and provide accurate and timely financial information by

outcome to project managers.

Issues to address 198. The process of utilizing project financial resources is procedurally centralized at HQ with the Budget Holder playing a major role in the decision making to allocate project resources. However, the experience in Burundi – through FBAs - may provide a decentralized solution, which would reinforce the empowerment of project staff and imply a greater role of the RCU in decision-making.


58

199. In addition, the FAO financial system records project expenditures only by expense categories, such as consultants, travel, training expenses, etc. Currently it cannot automatically disaggregate this financial information by GEF outcome or output. Therefore, all financial management reports need to be done manually by allocating project expenditure to the corresponding outcome. Over time, it does not provide project managers with accurate and timely financial information. It is recommended that FAO provide its project managers with financial information by project outcome11.

Recommendation 7: It is recommended that backstopping from FAO HQ focuses more on

general progress and support, and less on technical training sessions at this stage of the

project.

Issues to address 200. Much of the support to the project from FAO HQ to-date has taken the form of technical training in various methodologies (LADA-WOCAT; PES etc.) and some management support to review progress. However, it is now clear that, with a focus in the last stage of the project period on scaling-up and mainstreaming, there is a need for more general backstopping support both at field activity level, and at the national and regional levels. This backstopping should be demand-driven and not too rigorously pre-planned, to allow for responsiveness in the field.

Recommendation 8: It is recommended that the engagement of stakeholders is urgently

improved through more cross visits for intra-national and regional learning at all levels and

through more frequent RPSC and NPSCs meetings, starting with a RPSC to review the MTE

findings and its recommendations

Issues to address 201. The project has so far failed to create a true “regional” identity for itself, and appears more like four independent but parallel country programmes. One of the reasons is insufficient regional interaction. Some interaction activities were conducted such as an international visit to Brazil was carried out, and there have been some exchanges between national coordinators; however, these interactions are visibly not enough and more are needed. A regional project needs to capitalize on its potential for synergy by sharing knowledge and ideas. This can be achieved, at least partially, through cross-learning between partners. Furthermore, within the countries, there is inadequate cross-learning between communities. While budgetary constraints are acknowledged, it is imperative that cross-visits, at all levels, within and between countries, are stepped up. A programme of what is possible/ practical should be drawn up. 202. There are too few steering committees meetings, particularly considering the fact that the first meeting in the four countries took place at the start of the project in late 2010. NPSCs should meet twice a year, ideally one meeting to review and endorse work plans and a second to review the progress made, review issues and possibly undertake actions to overcome any major issues faced by the project. While cost is obviously a consideration and a constraint, creativity needs to be employed to find ways of bringing members together. For example by making use of opportunities such as other meetings or 11 As noted in Section 4.3.2, a Results-Based Budget system has been recently created in the FAO-FPMIS that can financially report up to the output level. The Kagera TAMP Budget Holder has since set up a Results Based Budget by component in the FPMIS, which should address this financial reporting issue.


59

fora when a quorum of members may (by coincidence) be present. The timing of the RPSC meetings should also take into account the timing of the NPSCs. Following this MTE, the first recommended step is to organize a RPSC meeting to review the MTE findings and recommendations and discuss an overall work plan for the remaining implementation period.

Recommendation 9: It is recommended that the set of performance indicators is revised;

including the identification of a few capacity indicators to measure the capacity developed by

the project and all related targets used to measure the progress of the project.

Issues to address 203. The performance framework (set of 16 indicators) identified at the outset of the project to monitor the high level results (objectives and outcomes) includes a list of targets by the end of the project that are too ambitious and that will not be met. 204. It is particularly the case for targets with clear quantified values such as indicator #2 where 100,000 ha will not be achieved (currently about 5,350 ha), indicator #7 where 68 communities will not have identified and agreed on priority policy, legal and transboundary issues; and, indicator #13 where SLM practices will not be implemented in 200 pilot communities (currently in 46 communities). 205. The project is running out of time and instead of trying to expand the demonstration and meet some of these targets, it is recommended that this set of targets is revised based on achievements to-date and the current trend of implementation within the pilot sites. The revision should identify target values that could be met during the remaining implementation period of the project. Thus the project should give thought to downsizing its overall target for coverage with SLaM/ SLM from 100,000 to something in the region of 20,000 – 25,000 hectares after discussions with the national project managers about what is truly feasible. 206. In the meantime, Kagera-TAMP is expected to generate a range of global environmental benefits across the Kagera River Basin: this is explicit in its environmental objective and the rationale of GEF-funding. While these benefits are notoriously difficult to assess, more effort should be made to infer these benefits through data being collected on area under SLM/ SLaM. Some data – even simple data at community level – needs to be picked up on biodiversity in rehabilitated landscapes and riverbank. Hydrological data, as far as possible, should be collected from secondary sources even if they are only used as a baseline for the future. It is recommended to use the GEF tracking tool: Land Degradation Focal Area – Portfolio Monitoring and Assessment Tool when revising these indicators and their related targets. 207. The revision of the set of performance indicators should also include the identification of a few capacity indicators to measure how well the project has been developing the capacities needed to address the causes of land degradation and restore ecosystem health. 208. Finally, this recommendation also includes the strengthening of the annual progress reporting in the PIRs, particularly section 3 of these reports, which is the main measurement tool to assess progress made by the project.

Mid-Term Evaluation of the Kagera TAMP Project – Annexes

60

Annexes

Annex 1 Evaluation Terms of Reference (TORs) Annex 2 Evaluation Matrix Annex 3 List of Documents Consulted Annex 4 Discussion Guide Annex 5 Evaluation Mission Agenda Annex 6 List of People Met Annex 7 Project Expected Results and Planned Activities Annex 8 Co-financing Table Annex 9 Full List of Performance Indicators


61

Annex 1 Evaluation Terms of Reference (TORs)

1. Background of the Project 1.1 Programme Overview 1. The 4 countries sharing the transboundary Kagera River Basin, notably Tanzania UR, Uganda, Rwanda and Burundi, endorsed the project in 2006 document and thereby declared their willingness to work together to develop a coordinated programme of action to support the adoption of an integrated agro-ecosystems approach for the management of land resources in the Kagera Basin. Subsequently, the full size project document was developed under the PDF phases A and B in 2006-08, and endorsed by the GEF CEO in May 2009. GCP/RAF/424/GFF became operational in FAO after all four countries had signed the project document by April 2010. 2. The full size Kagera TAMP project is envisaged as a 4.5 year long project with 2 phases, during the first two years to pilot SLM practices at catchment and Farmer Field School (FFS) levels in each of the 21 districts and during the subsequent 2 years to scale up successful practices and approaches through developing and testing enabling policy and planning tools at various levels. 3. The project is funded by the Global Environment Facility (GEF), with estimated co-funding contributions from the four project countries through partnership arrangements and from FAO and other Programmes according to the Project Financing Plan below. The project grant from GEF amounts to 6.363.700 USD (excluding 10% Implementing Agency fee). In the land degradation portfolio the project grant is aimed at leveraging 3-4 times the amount in cofinancing with a view to greater sustainability.

Financing Plan USD

|Project Development PDF A 25,000 PDF B 700,000 FAO (in cash and kind) 200,000 Governments (in kind) 205,000 UNEP 10,000 Sub-Total PDF 1,140,000 Full size Project

GEF grant (including IA fee) 7,000,000 GEF grant (excluding IA fee) 6,363,700 Project Co-financing FAO (in kind ) 351,000 Government of Burundi - Districts - Govt/Partner programmes

6,260,000 860,000

5,400,000 Government of Rwanda - Districts - Govt/Partner programmes

6,293,760 768,000

5,525,760 Government of Tanzania (U.R.) - Districts

2,463,050 418,650


62

Financing Plan USD

- Govt/Partner programmes 2,044,400 Government of Uganda - Districts - Govt/Partner programmes

3,707,800 260,800

3,447,000 Other Partner Programmes and donors 5,433,600 Sub-Total Co-financing 24,509,210

Total Project Cost 32,012,910

4. The Food and Agriculture Organisation of the United Nations (FAO) is the GEF Implementing Agency and Executing Agency for this project under the Land Degradation GEF Focal Area of GEF-4 Strategic Programs LD SP-1, Supporting Sustainable Agriculture and Rangeland Management, and LD SP-3, Investing in New and Innovative Approaches to Sustainable Land Management. The project is executed in close consultation and collaboration with the designated National Authorities in the four riparian countries of the Kagera river, which are:

• Ministry of Agriculture and Animal Resources (MINAGRI) in Rwanda; • Ministry of Agriculture, Animal Industry and Fisheries (MAAIF) in Uganda; • Ministry of Agriculture and Livestock (MINAGRIE) in Burundi; • Division of the Environment, Vice President's Office (DOE/VPO) in the United Republic of

Tanzania in close collaboration with the Ministry of Agriculture, Food and Cooperatives (MAFC).

5. The project is also part of the larger TerrAfrica/SIP for SLM in Sub-Saharan Africa. 6. There are 4 components to the project with a substantial share of the resources for SLM on the ground through contracting and improve a capacity of local service providers. 7. Component 1: Transboundary coordination, information sharing and monitoring and evaluation mechanisms

• Component 2: Enabling policy, planning and legislative conditions • Component 3: Capacity and knowledge for the promotion of and technical support for

Sustainable land and agro-ecosystem management across the basin • Component 4: Improved land and agro-ecosystem management practices implemented and

benefiting land users in all agro-ecosystems in the basin 8. A 5th component covers the project management and operational structures which support the effective implementation across all four above-mentioned components. 9. The implementation of these four components is expected to mobilize collaboration and cofunding support from a range of partners from governmental technical and academic bodies, NGO/ CSO partners and partner projects to support investments in SLM in the Kagera basin. The agreed incremental costs associated with the project implementation are expected to lead to measurable impacts in transboundary land and water management and in terms of sustainable cropping, livestock and forest systems and livelihoods of the populations that depend on them.

Financing of Project Components GEF

Financing %

Co-

Financing %


63

Financing of Project Components GEF

Financing %

Co-Financing

%

Component 1: Transboundary coordination, information sharing and monitoring and evaluation mechanisms 1,766,873 28% 2,316,520 9% Component 2: Enabling policy, planning and legislative conditions 423,342 7% 1,273,320 5% Component 3: Capacity and knowledge for the promotion of and technical support for Sustainable land and agro-ecosystem management across the basin 1,230,003 19% 3,636,520 15% Component 4: Improved land and agro-ecosystem management practices implemented and benefiting land users in all agro-ecosystems in the basin 2,360,682 37% 15,682,850 64% Component 5: Project management structures operational and effective 582,800 9% 1,600,000 7%

Total Project Cost 6,363,700 24,509,210 10. By the end of 2012, about half of the funds provided by the GEF grant has been delivered (3,13 million USD expenditures and unliquidated obligations).

Financial Delivery 2010 2011 2012 Total

Commitments & Expenditures (in US Dollars) 589,091 915,330 1,629,323 3,133,744

1.2 Project objectives 11. The overall long-term environment and development goal of the project is to support the adoption of an integrated ecosystems approach for the management of land resources in the Kagera Basin which will generate local, national and global benefits including: restoration of degraded lands, carbon sequestration and climate change mitigation, agro-biodiversity conservation and sustainable use, protection of international waters and improved agricultural production, food security and rural livelihoods. 12. The environmental objective of the project is to address the causes of land degradation and restore ecosystem health and function and generate a range of global environmental benefits across the Kagera basin through the introduction of adapted agro-ecosystem management approaches. 13. The development objective is to improve the livelihoods and hence contribute to reduced poverty of rural communities in the Kagera Basin through more productive and sustainable resource management practices that are technically feasible and socio-economically viable. 14. In order to reach the above closely inter-related development and environment objectives through its five components, the project is expected to achieve the following outcomes:

Outcome 1: Transboundary coordination, information sharing and monitoring and evaluation mechanisms operational and effective in promoting sustainable, productive agro-ecosystems and restoration of degraded lands. Outcome 2: Enabling policy, planning and legislative conditions are in place to support and


64

facilitate the sustainable management of agro-ecosystems and the restoration of degraded land. Outcome 3: Capacity and knowledge are enhanced at all levels for the promotion of – and technical support for – sustainable management of land and agro-ecosystems in the basin. Outcome 4: Improved land and agro-ecosystem management practices are implemented and benefiting land users for the range of agro-ecosystems in the basin. Outcome 5: Project management structures operational and effective.

15. The attached log-frame of the project provides an overview of the activities and expected outputs and targets. 1.3 Kagera TAMP execution and management structure

16. The Food and Agriculture Organisation of the United Nations (FAO) is the GEF Implementing Agency and Executing Agency for this project, and the Land and Water division (NRL) at FAO HQ is the designated Lead Technical Unit and Operational Unit (Budget Holder). 17. The Kagera TAMP Regional Coordination Unit (RCU) is hosted and supported by the FAO Representation in Kigali, Rwanda, and is led by the Regional Project Coordinator (RPC), and is responsible for successful cross country coordination and implementation of the project. 18. The project is guided for transboundary and collaborative issues across the countries by a Regional Project Steering Committee (RPSC) comprising senior level government officials from each country (designated by the national executing partners/Ministries) and representatives from the major regional programmes which are relevant for Kagera TAMP. The regional project steering committee (RPSC) for the full project was set up and first met on 18th March 2011 (at the end of the project development phase a regional SC had reviewed and endorsed the project including with representatives of the Government of Burundi which was not involved in the PDF due to insecurity. 19. The implementation of the project in each country is guided by the National Project Steering Committee (NPSC). Members of the NPSC were nominated by participating Ministries and include representatives from Districts, Ministries (or Departments) as well as relevant non-governmental, civil society and private sector organizations. The NPSC considers and endorses the annual national workplan and budget, including specifications for work within the country over the next year, and supports the timely undertaking of the workplan through activities of the National Project Manager, consultants, and letters of agreement with service providers. The 4 national project steering committees first met respectively in Rwanda and Burundi in October 2010, in Uganda in November 2010 and in Tanzania in February 2011 (postponed by Government). They reviewed the project log-frame, agreed on their TOR and met regularly and provided advice on implementation and progress. 20. Each of the four project countries has a National Project Manager recruited by FAO and acting as fulltime national technical advisor and SLM expert supporting the project implementation in country and collaboration on cross-border activities. Two of the NPMs are based in the capitals, Kigali, Rwanda and Bujumbura, Burundi, and two of the NPMs are based in districts that are within the Kagera river basin but rather remote from the capitals which influences their tasks and relations with Government bodies and donors. 21. There is also an ad-hoc Regional Technical Advisory Committee (RTAC) whose members were nominated by the 4 countries. Additional members can be called upon independently for review


65

of project outputs and results, such as technical documents, and to provide specific technical support as and when required. 2. Purpose of the Evaluation 22. In accordance with the Project Document, an independent Mid-term Evaluation will be undertaken “at the beginning of the third year of project implementation” The project was endorsed by GEF in 2009 but only became operational with FAO in April 2010 after the last country had signed the project agreement and the recruitment of project personnel (RPC and NPMs) was initiated in the following months. The start-up phase of the project, including the first steering committee meetings and national and regional level, therefore experienced delays until the end of 2010 to become fully operational in all countries. Therefore also in terms of financial delivery the “mid-term” was reached only by the end of 2012. The MTE should therefore now review the progress made towards achievement of outcomes in accordance with the full project document and CEO endorsement and identify corrective actions if necessary. 23. The Terms of Reference for this Mid-term Evaluation were prepared by the Land and Water Division in close consultation with FAO Office of Evaluation (OED) and the FAO GEF Coordination Unit within FAO Investment Centre (TCID) in accordance with the evaluation policies and procedures of FAO and the GEF. 24. The presentation of MTE findings should be made to the Kagera TAMP National and subsequently Regional PSCs. 3. Evaluation Framework and Scope

3.1 Scope 25. The independent Mid-Term Evaluation will evaluate the first three years of project implementation. will be undertaken at the beginning of the third year of project implementation. It will determine progress being made towards the achievement of project outcomes and will identify corrective actions if necessary. It will, inter alia:

• review the effectiveness, efficiency and timeliness of project implementation; • analyze effectiveness of implementation and partnership arrangements; • identify issues requiring decisions and remedial actions; • identify lessons learned about project design, implementation and management; • highlight technical achievements and lessons learned; • analyze whether the project is on track with respect to achieving the expected results; and • propose any mid-course corrections and/or adjustments to the Work Plan as necessary.

26. The MTE It will assess the project from its concept and design to current and potential results. 3.2 Evaluation criteria

27. The project will be critically assessed through the internationally accepted evaluation criteria, i.e. relevance, efficiency, effectiveness, impact and sustainability. In line with the new FAO project


66

cycle, the evaluation will assess compliance with the following UN Common Country Programming Principles: Human Rights Based Approaches (HRBA)/ Right to Food/ Decent Work; Gender equality, Environmental sustainability, Capacity Development and Results Based Management. 3.3 Evaluation issues

I. Relevance of concept and design

a. Relevance of the initiative to: • the GEF-5 land degradation focal area strategy under which the project was approved; • Kagera TAMP countries development priorities and needs for to the sustainable management of

land and agro-ecosystems of the Kagera river basin; • needs of the population; • FAO Global Goals, Strategic Objective F “Sustainable management of land, water and genetic

resources and improved responses to environmental challenges affecting food and agriculture” and Organizational Results FO1 and Core Functions;

• UNDAF; • FAO Country Programming Framework (if applicable); • Other aid programmes in the sector

b. Clarity, coherence and realism of the Logical Framework of the project and of its design,

including: i) logic of causal relationship between inputs, activities, expected outputs, outcomes and

impact (against specific and development objectives); ii) validity of indicators, assumptions and risks; iii) approach and methodology; iv) resources (human and financial) and duration; v) stakeholder and beneficiary identification and analysis; vi) institutional set-up and management arrangements. vii) ambition in project design and scale versus current status of project implementation

(punctual and partially dispersed activities)

II. Effectiveness of outputs and outcomes

a. Overall effectiveness of the Kagera TAMP project, actual or potential, in attaining its

intermediate/specific objectives, i.e. at completion of year 3 of implementation; b. Description and analysis of the outputs produced, in terms of quantity, quality and timeliness; c. Description and analysis of the outcomes achieved, expected and unexpected, their robustness

and expectations for further uptake and diffusion (see list of outcomes and outputs, Annex 3 of this TOR).

d. Use made by the project of FAO’s normative and knowledge products and actual and potential contribution of the project to the normative and knowledge function of the Organization. In particular, the team will assess the use made by the Kagera project of specific normative tools and methods and technical documents developed by the Natural Resources and Environment Department and other technical Divisions and if appropriate decentralized offices and wider TerrAfrica/SIP programme.


67

28. The following key events and main achievements to date that should be assessed in view of their contribution to the achievements of the expected project outcomes:

e. Adoption of LADA-WOCAT tools to develop LUS maps as a basis for the conduct of LD and SLM assessment and mapping and the identification of key land degradation issues and hotspots as well as bright spots in terms of SLM for catchment selection and information and priority setting by decision makers);

f. Identification and selection of micro-catchments in all project sites and target districts. g. Catchment characterization: conduct of natural resources and livelihoods diagnostic assessment

of catchments including: participatory socio-economic analysis and bio-physical assessment and mapping. This includes the training of staff and partners in land degradation and SLM assessment (using LADA local methods and tools and WOCAT tools -QT; QA; QM)

h. Establishment of Farmer Field Schools (FFS) as a participatory learning and empowerment approach for Sustainable Land and Agro-ecosystems Management and links with community and catchment level interventions;

i. Identification and engaging of service providers (local NGOs and CBOs) for technical support in promoting adoption of sustainable land management practices

j. Efficiency and Assessment of land and natural resources conflicts in the transboundary Kagera river basin and its range of agro-ecosystems and identified actions for addressing the conflicts

k. Organization of various workshops for capacity development and policy and planning support for project implementation:

- Project launch in each country;

- Regional technical workshop (Kigali in March; 2011);

- Kabale regional workshop (planning of SLM with its supporting pillars including land planning policies, tenure and PES) August –September 2011

l. Policy review and land planning, legislation and policy and programmatic and financial support towards SLM implementation

m. Project steering committee meetings at national and regional level and the extent of support and guidance from PSC members;

n. Establishment of project M&E system and its effectiveness in monitoring progress and impacts

o. effectiveness of service providers (local NGOs and CBOs) for technical support in sustainable land management

p. Effectiveness of a roadmap for Payment for Environmental services (PES) schemes as an option to incentivate SLM

29. The team will also identify potential products by the Programme for wider diffusion of lessons learned.

III. Efficiency and effectiveness of project implementation process

a. Assessment of project management: � Quality, realism and focus of work plans; � Assessment of inputs delivery, causes and consequences of delays and of any remedial

measure taken, if any; � Gaps and delays if any between planned and achieved outputs, the causes and

consequences of delays and assessment of any remedial measures taken;


68

� Gaps and delays if any between planned and achieved outcomes, the causes and consequences of delays and assessment of any remedial measures taken;

� Effectiveness of internal monitoring and review processes; and feed-back loop into improved management and operations; this will also include information provided by the project through GEF Tracking Tools

� Staff management; � Efficiency and effectiveness of coordination and steering bodies, e.g. RPSC and NPSCs; � Quality and quantity of administrative and technical support by FAO to the project,

including the Lead Technical Unit, the Budget Holder and project Task Force; � Quality and quantity of technical support by the RCU to the National project managers

and their SLM implementation teams. � Development and implementation of an exit strategy;

b. Institutional Setup: � Administrative and technical support by FAO HQ, regional, sub-regional and country

office, as appropriate; � Institutional set-up, internal review processes, coordination and steering bodies; � Inputs and support by the Government/s and resource partner/s.

c. Assessment of financial resources management, including:

� Adequacy of budget allocations (GEF and cofunding) to achieve outputs and promote outcomes;

� Coherence and soundness of Budget Revisions in matching budget adjustments to implementation needs and project objectives;

� Rate of delivery and budget balance at the time of the evaluation.

IV. Analysis of the application of the UN common country programming principles, cross-

cutting themes, and of the Humanitarian Principles and Minimum Standards in the case

of emergency projects

d. Analysis of gender mainstreaming for gender equality. This will include: � extent to which gender equality considerations were reflected in project objectives and

design to address the needs, priorities and constraints of both women and men, and in the identification of beneficiaries;

� extent to which gender equality considerations were taken into account in project implementation and management;

� extent to which gender relations and equality have been or will be affected by the project.12

e. Analysis of the Capacity Development dimension in the design, implementation and results of

the project, at individual, organizational and enabling environment levels.13 This will include CD on both technical and soft-skills, i.e. planning, budgeting, partnering and negotiating. Including the training of staff and partners in land degradation assessment (LADA local) and SLM assessment using WOCAT tools; WOCAT tools (QT; QA; QM); the efficiency and

12 See: http://typo3.fao.org/fileadmin/templates/gender/docs/FAO_FinalGender_Policy_2012.pdf 13 See: http://www.fao.org/capacitydevelopment/en/


69

effectiveness of the workshops organized (i.e. Project launch in each country; Regional technical workshop - Kigali in March; 2011-;Kabale regional workshop - planning of SLM with its supporting pillars including land planning policies, tenure and PES)

f. Analysis of the adoption of the Human-Rights Based Approach, namely:

� the integration of the Right to Food dimension and principles, in the design, implementation and results of the project;

� the integration of decent rural employment concerns in the design, implementation and results of the project.

g. Analysis of Partnerships and Alliances, namely:

� how they were planned in the project design and developed through implementation; � their focus and strength; and � their effect on project results and sustainability.14 � effectiveness of partnerships with key institutions and organizations (MOUs with Nile

Basin Initiative; Vi-Agroforestry, LVBC/LVEMP)

h. Analysis of how environmental impacts were taken into consideration and addressed, following the steps and criteria contained in the FAO Environmental Impact Assessment guidelines.

i. In the case of emergency projects, analysis of the extent to which the programme has

effectively adhered to the principles promoted in the Humanitarian Charter and to the Minimum Standards as defined in the Sphere handbook.15

V. Impact

j. Overall impact of the project, actual or potential, positive and negative, produced directly or

indirectly, intended or unintended; and

k. Overall contribution of the project to FAO Country Programming Frameworks, Organizational Result/s and Strategic Objectives, as well as to the implementation of the corporate Core Functions.

VI. Sustainability

30. The prospects for sustaining and up-scaling the project's results by the beneficiaries and the host institutions after the termination of the project. The assessment of sustainability will include, as appropriate:

� Institutional, technical, social and economic sustainability of proposed technologies, innovations and/or processes;

� Expectation of institutional uptake and mainstreaming of the newly acquired capacities, or diffusion beyond the beneficiaries or the project; at national and regional level

� Environmental sustainability: the project’s contribution to sustainable natural resource management, in terms of maintenance and/or regeneration of the natural resource base.

14 See: http://www.fao.org/partnerships/partners-home/en/ 15 In the Humanitarian charter, humanitarian agencies jointed expressed their conviction that all people affected by disaster or conflict have a right to receive protection and assistance to ensure the basic conditions for life with dignity (http://www.spherehandbook.org/)


70

The assessment of environmental sustainability of the Programme will require specific focus on analyzing on costs and benefits of SLM interventions in the short and long term and the extent to which they are shared and supported by various actors.

� Assessment of land and natural resources conflicts in transboundary agro-ecosystem management project Kagera basin and recommendations for addressing the conflicts

� The catalytic role of the programme in supporting the creation of an enabling environment with a view to achieve sustainable global environmental benefits.

� The availability of cofinancing to effectively upscale piloted project activities at basin level;

31. Based on the above analysis, the evaluation will draw specific conclusions and formulate recommendations for any necessary further action by Government, FAO and/or other parties to ensure sustainable land and agroecosystem management and livelihoods, including any need for follow-up action. The evaluation will draw attention to specific good practices and lessons of interest to other similar activities. Any proposal for further assistance should include specification of major objectives and outputs and indicative inputs required. 4. Evaluation methodology

4.1 Approach and tools 32. The evaluation will adhere to the UNEG Norms & Standards16. It will adopt a consultative and transparent approach with internal and external stakeholders throughout the evaluation process. Triangulation of evidence and information gathered will underpin its validation and analysis and will support conclusions and recommendations. 33. The evaluation will make use of the following methods and review of existing reports, semi-structured interviews with key informants, stakeholders and participants, supported by check lists and/or interview protocols; direct observation during field visits; surveys and questionnaires. 34. Particular attention will be devoted to ensure that women and other under-privileged groups will be consulted in adequate manner. Insofar as possible and appropriate, interaction will also take place with non-participants to canvass their opinions. The Sustainable Livelihoods Framework;17 the Strengths, Weaknesses, Opportunities and Threats (SWOT) framework can be used for assessment of project results.18 35. The evaluation team will visit Burundi, Rwanda, Uganda and UR Tanzania Tanzania which share the Kagera basin and will have a (de)briefing at FAO headquarters in Rome (Italy). In each of the four riparian countries of the Kagera river basin country, the evaluation team will visit:

16 http://www.uneval.org/normsandstandards; both GEF and FAO evaluation units are members of UNEG and subscribe to its Norms and Standards 17 The Sustainable Livelihoods Framework identifies five different capitals (human, social, natural, financial, and physical), each

including different assets. It helps in improving understanding of livelihoods, in particular of the poor. For more information, among others: http://www.livelihoods.org/info/guidance_sheets_pdfs/section2.pdf

18 SWOT is a widely used strategic planning tool, useful also in the assessment of development interventions, to canvass their strengths and weaknesses, as well as future perspectives. It is particularly used in focus groups, but it can be adapted to individual interviews as well.


71

• locations that hosts the national coordination units, namely in Bujumbura/Burundi, Kabale/Uganda, Bukoba/Tanzania and Kigali/Rwanda which is also the regional project coordinating unit

• up to 2 target districts to capture a broad range of field activities that will enable the team to capture the variability of the context in which the project operates as well as the specific challenges and progress. The selection criteria for locations and partners to visit will include the state of progress, (e.g. more and less advanced activities).

• The capitals (Location of Government authorities); • Kigali/Rwanda which hosts the Regional Project Coordinating Unit.

36. For cost, time, and language considerations it is suggested that the evaluation team divides for the country visits (head of mission/francophone speaker to Burundi + Rwanda; other member to Uganda + Tanzania). Within resources available, and after consultation with OED and project management, the team will independently decide which locations, outputs and outcomes to assess in detail. 37. The evaluation will include the following activities:

• A desk review of the project document, outputs, monitoring reports (e.g. project inception report, steering committee reports and reports from other relevant meetings; project implementation reports (PIR); quarterly and six-monthly progress reports), and other internal documents including consultant and financial reports;

• A review of specific products including the content of the project website, annual work plans, publications and other materials and reports;

• Interviews with staff and national institutions and national/regional counterparts involved in project implementation including the Regional Project Coordinator, the National Project Managers, the FAO Representations; the Lead Technical Unit and Budget Holder, task force members of other involved technical units of FAO;

• In the visited countries, the team will also meet other actors who are active working on the Kagera TAMP project, notably counterpart institutions and service providers and partner projects to assess actual and potential areas of collaboration and partnership;

38. The briefings and debriefing will include the Government, the partners, the FAO Representation and other relevant actors. 39. The evaluation team will discuss in detail with the key stakeholders of the project and will take into account their perspectives and opinions. Key stakeholders will include:

• Government representatives from the executing partner institutions at national and subnational level;

• the partners and service providers involved in collaborative arrangements through MOUs and LOAs and other potential partners as described in cofinancing plan;

• Participants in communities, including farmers organizations and local leaders. • FAO Representatives in the participating countries; and • the Lead Technical and Operational Unit as well as the GEF Coordination Unit at FAO HQ.

40. The evaluation team will maintain close liaison with the FAO Office of Evaluation (OED), FAO offices at regional and country level and the RCU as appropriate, and all key stakeholders. Although the mission is free to discuss with the authorities concerned anything relevant to its


72

assignment, it is not authorized to make any commitment on behalf of the Government, the donor or FAO. 41. At the end of the mission, the team will present its preliminary conclusions and recommendations to the RPC unit in FAO-Kigali, the LTU (NRL), FAO-GEF Unit and OED in FAO HQ. 42. The Terms of Reference of the evaluation and the final draft report will be circulated to project management before finalisation; comments and suggestions will be incorporated as deemed appropriate by the evaluation team. 5. Roles and responsibilities 43. Should other stakeholders engage in the evaluation process beyond participating in meetings, discussions and information gathering, their roles and responsibilities should be added here. Additional tasks can also be added to those proposed below. 44. FAO Budget Holder (BH), the Lead Technical Officer (LTO) and RPC unit of the project to be evaluated are responsible for initiating the evaluation process, drafting the first version of the Terms of Reference, and supporting the evaluation team during its work. They are required to participate in meetings with the team, make available information and documentation as necessary, and comment on the draft final terms of reference and report. Involvement of different members of the project Task Force will depend on respective roles and participation in the project. 45. The BH is also responsible for leading and coordinating the preparation of the FAO Management Response and the Follow-up Report to the evaluation, fully supported in this task by the LTO and PTF. OED guidelines for the Management Response and the Follow-up Report provide necessary details on this process. 46. FAO Office of Evaluation assists the BH and LTO in drafting the ToR, in the identification of the consultants and in the organization of the team’s work; it is responsible for the finalization of the ToR and of the team composition; it shall brief the evaluation team on the evaluation methodology and process and will review the final draft report for Quality Assurance purposes in terms of presentation, compliance with the ToR and timely delivery, quality, clarity and soundness of evidence provided and of the analysis supporting conclusions and recommendations. The Office of Evaluation has also a responsibility in following up with the BH for the timely preparation of the Management Response and the Follow-up to the MR. 47. The Evaluation Team is responsible for conducting the evaluation, applying the methodology as appropriate and for producing the evaluation report. All team members, including the Team Leader, will participate in briefing and debriefing meetings, discussions, field visits, and will contribute to the evaluation with written inputs for the final draft and final report. 48. The Team Leader guides and coordinates the team members in their specific work, discusses their findings, conclusions and recommendations and prepares the final draft and the final report, consolidating the inputs from the team members with his/her own.


73

49. The Evaluation team will be free to expand the scope, criteria, questions and issues listed above, as well as develop its own evaluation tools and framework, within time and resources available. 50. The mission is fully responsible for its report which may not reflect the views of the Government or of FAO. An evaluation report is not subject to technical clearance by FAO although OED is responsible for Quality Assurance of all evaluation reports. As a contribution to the OED Knowledge Management System:the Team Leader will be responsible for completing the OED quantitative project performance questionnaire, at the same time as the final evaluation report; 51. OED will ask all team members to complete an anonymous and confidential questionnaire to get their feedback on the evaluation process. 6. Evaluation team 52. The team will include competence and skills in evaluation and relevant technical topics (see below|); to the extent possible it will be balanced in terms of geographical and gender representation to ensure diversity and complementarity of perspectives. 53. Mission members will have had no previous direct involvement in the formulation, implementation or backstopping of the project. All will sign the Declaration of Interest form of the FAO Office of Evaluation. 54. The evaluation team will comprise the best available mix of skills that are required to assess the project, namely:

• Demonstrated experience in the evaluation of large/complex, regional technical assistance projects

• Familiarity with the objectives of the GEF Land degradation portfolio • Understanding of governance, political, economic and institutional issues associated with

transboundary land, water and agroecosystem management issues in the Southern and Eastern Africa region.

• Watershed management and participatory land use planning from local to national scales • Gender equality and HRBA; • Conduct of evaluations.

55. In so far as possible, the team will be regionally and gender balanced. 56. All team members will have a University Degree and a minimum of 15 years of professional experience, or equivalent level of competence, in their respective areas of specialization. Insofar as possible, all will be fluent in English with at least 1 member of the team fluent in French. Individual Terms of reference will be developed referring to this ToR, upon recruitment of each team member. 7. Evaluation deliverables 57. The evaluation report will illustrate the evidence found that responds to the evaluation issues, questions and criteria listed in the ToR. It will include an executive summary. Supporting data and analysis should be annexed to the report when considered important to complement the main report.


74

58. The recommendations will be addressed to the different stakeholders and prioritized: they will be evidence-based, relevant, focused, clearly formulated and actionable. 59. The evaluation team will agree on the outline of the report early in the evaluation process, based on the template provided in Annex I of this ToR. The report will be prepared in English, with numbered paragraphs, following OED template for report writing. 60. Translations in other languages of the Organization, if required, will be FAO’s responsibility. 61. The team leader bears responsibility for submitting the final draft report to OED within 4 weeks from the conclusion of the mission, which will provide comments within one week. The revised report will be circulated to other FAO stakeholders, who within two additional weeks will submit to the team comments and suggestions that the team will include as appropriate in the final report within one week. 62. Annexes to the evaluation report will include, though not limited to, the following as relevant:

� Terms of reference for the evaluation; � Profile of team members; � List of documents reviewed; � List of institutions and stakeholders interviewed by the evaluation team; � List of project outputs; � Evaluation tools. � Itinerary of the evaluation team mission; � Data collection instruments (e.g. copies of questionnaires, surveys – if applicable)

8. Ratings 63. In order to facilitate comparison with routine reporting to GEF and contribute to the GEF programme leaning process (LD portfolio), the evaluation will rate the success of the project on the GEF six-point scale system: Highly Satisfactory (HS), Satisfactory (S), Marginally Satisfactory (MS), Marginally Unsatisfactory (MU), Unsatisfactory (U), and Highly Unsatisfactory (HU). 64. Each of the items listed below should be rated separately, with comments and then an overall rating given.

� Achievement of objectives � Attainment of outputs and activities � Progress towards meeting GEF-4 focal area priorities/objectives � Cost-effectiveness � Impact � Risk and Risk management � Sustainability19 � Stakeholder participation � Country ownership

19 Sustainability will be assessed in terms of Likelihood: Likely (L): There are no risks affecting this dimension of sustainability. Moderately Likely (ML). There are moderate risks that affect this dimension of sustainability. Moderately Unlikely (MU): There are significant risks that affect this dimension of sustainability Unlikely (U): There are severe risks that affect this dimension of sustainability


75

� Implementation approach � Financial planning � Replicability � Monitoring and evaluation.

9. Evaluation timetable 65. The timetable below applies to the whole evaluation process. Number of days allocated to team members will vary according to responsibility.

Activity Date No. of days

Cumulati

ve total days

Finalization of TOR (including country consultations) Finalization of MTE team composition and mission time schedule Mission leader 2nd Evaluater Remarks

Background reading (home base); Preparat. of the Evaluation matrix

Background reading (home base);

2 From home to Rome

From home to Rome

Sun 28-Apr 1

Briefing at FAO HQ, Rome, Italy


Mon 29-Apr 1

Briefing at FAO HQ, Rome, Italy; Travel Kigali, Rwanda


Tue

Tue 30-Apr 1 Desk work Desk work Wed 1 May 1 Briefing with Regional Coordinator, Kigali, Rwanda

Briefing with Regional Coordinator, Kigali, Rwanda and visit regional partners (NBI, VI, IFDC?)

Project strategy and regional considerations

Thu 2-May 1 Assessment Rwanda

Travel Uganda; Assessment Uganda- Kabale

Fri 3-May 1 Assessment Rwanda

Assessment Uganda

Sat 04-May 1

Travel Burundi Desk work

Desk work Sun 05-May 1

Assessment Burundi

Assessment Uganda

Mon 06 May 1

Assessment Assessment Tue 07-May 1


76


Cumulative total

days

Burundi Uganda (Kampala and Entebbe)

Assessment Burundi

Travel to Tanzania (met by NPM at border) and desk work

Wed 08-May 1 Travel Rwanda; Desk work

Assessment, Tanzania (Bukoba and Missenyi)

Thu 09-May 1 Rwanda Assessment,

Tanzania (Karagwe and Ngara) and travel to Kigali

Fri 10-May 1 Debr. with Regional Coordinator, Kigali,; Travel Dar-Es-Salaam

Debr. with Regional Coordinator, Kigali,; Travel Dar-Es-Salaam

Sat 11-May 1

Sun 12 May 1 Assessment, Tanzania, Dar-Es-Salaam; Travel Rome

Assessment, Tanzania, Dar-Es-Salaam; Travel Rome

Mon 13-May 1 Debriefing FAO HQ, Rome, Italy

Debriefing FAO HQ, Rome, Italy

Tue 14-May 1 If possible and required Debriefing with NRL/ travel home

If possible and required Debriefing with NRL/ travel home

Wed 15-May 1 Finalize report and submission (WAE at home base)

Finalizes section(s) under his responsibility

5 days

5 Comments by OED

Within 7 days

Circulation of final draft

2 days work, within 3 days 2

Comments to team on final draft

Within two weeks

Finalization of report

2 days, within one week 2

RCU presents preliminary/final


77


Cumulative total

days

Evaluation report to the RPSC Kigali and NPMs to NPSCs

10. Annexes to the TOR Annex 1, Annotated Report Outline Annex 2, FAO Global Goals, Strategic Objectives and Organization results Annex 3, Project key outputs and outcomes, Annex 4; Project log-frame


78

Annex 2 Evaluation Matrix

The evaluation matrix below served as a general guide for the evaluation. It provided directions for the evaluation; particularly the collect of relevant data. It was used as a basis for interviewing people and reviewing programme documents. It also provided a basis for structuring the evaluation report as a whole.

Evaluated component

Sub-Question Indicators Sources Data Collection

Method

Evaluation criteria: Relevance - How did the Project relate to the main objectives of GEF, UNCCD and to the environment and development priorities of Uganda, Tanzania, Rwanda and Burundi?

Is the Project relevant to GEF and UNCCD objectives?

� How does the Project support the related strategic priorities of the GEF and of the UNCCD?

� Were GEF criteria for Project identification adequate in view of actual needs?

� Level of coherence between project objectives and those of the GEF and of the UNCCD

� Extent to which the project is actually implemented in line with incremental cost argument

� Project documents

� GEF policies and strategies

� GEF web site

� Documents analyses

� Interviews with government officials and other partners

Is the Project relevant to FAO objectives?

� How does the Project support the objectives of FAO in this sector?

� Existence of a clear relationship between project objectives and regional programme objectives of FAO


� FAO strategies and programme



Is the Project relevant to Recipient Countries’ development objectives?

� How does the Project support the development objectives of Recipient Countries?

� How country-driven is the Project?

� Does the Project adequately take into account national realities, both in terms of institutional framework and programming, in its design and its implementation?

� To what extent were national partners involved in the design of the Project?

� Degree to which the project support national environmental and development objectives

� Degree of coherence between the project and nationals priorities, policies and strategies

� Appreciation from national stakeholders with respect to adequacy of project design and implementation to national realities and existing capacities?

� Level of involvement of Government officials and other partners into the project

� Coherence between needs expressed by national stakeholders and FAO-GEF criteria


� National policies, strategies and programmes

� Key government officials and other partners



Does the Project address the

� How does the Project support the needs of target beneficiaries?

� Is the implementation of the Project been inclusive of all relevant Stakeholders?

� Strength of the link between project expected results and the needs of target beneficiaries

� Degree of involvement and inclusiveness of beneficiaries and stakeholders in project design and

� Beneficiaries and stakeholders

� Needs assessment studies


� Document analysis

� Interviews with beneficiaries and stakeholders


79

Evaluated component


Method

needs of target beneficiaries?

� Are local beneficiaries and stakeholders adequately involved in Project design and implementation?

implementation

Is the Project internally coherent in its design?

� Is there a direct and strong link between project expected results (log frame) and the Project design (in terms of Project components, choice of partners, structure, delivery mechanism, scope, budget, use of resources etc.)?

� Is the length of the Project conducive to achieve project outcomes?

� Level of coherence between project expected results and project design internal logic

� Level of coherence between project design and project implementation approach

� Program and project documents

� Key project stakeholders


� Key Interviews

How is the Project relevant in light of other donors?

� With regards to Recipient Countries, does the Project remain relevant in terms of areas of focus and targeting of key activities?

� How do GEF-funds help to fill gaps (or give additional stimulus) that are crucial but are not covered by other donors?

� Degree to which program was coherent and complementary to other donor programming in Recipient Countries

� List of programs and funds in which the future developments, ideas and partnerships of the project are eligible?

� Other Donors’ policies and programming documents

� Other Donor representatives



� Interviews with other Donors

Future directions for similar Projects

� What lessons have been learnt and what changes could have been made to the Project in order to strengthen the alignment between the project and the Partners’ priorities and areas of focus?

� How could the project better target and address priorities and development challenges of targeted beneficiaries?

� Data collected throughout evaluation

� Data analysis

Evaluation criteria: Effectiveness – To what extent are the expected outcomes of the Project being achieved?

How is the Project effective in achieving its expected outcomes?

� Is the project being effective in achieving its expected outcomes?

o Transboundary coordination, information sharing and monitoring and evaluation mechanisms operational and effective in promoting sustainable, productive agro-ecosystems and restoration of degraded lands.

o Enabling policy, planning and legislative conditions are in place to support and facilitate the sustainable management of agro-ecosystems and the restoration of degraded land.

o Capacity and knowledge are enhanced at all levels for the promotion of – and technical support for – sustainable management of land and agro-ecosystems in the basin.

� New methodologies, skills and knowledge

� Change in capacity for information management: Knowledge acquisition and sharing; Effective data gathering, methods and procedures for reporting.

� Change in capacity for awareness raising o Stakeholder involvement and government

awareness o Change in local stakeholder behavior

� Change in capacity in policy making and planning o Policy reform for sustainable management of agro-

ecosystems and the restoration of degraded land o Legislation/regulation change to improve the

sustainable management of agro-ecosystems and the restoration of degraded land

o Development of national and local strategies and


� Key stakeholders including FAO, Project Team, Representatives of Gov. and other Partners

� Research findings

� Documents analysis

� Meetings with main Project Partners

� Interviews with project beneficiaries


80

Evaluated component


Method

o Improved land and agro-ecosystem management practices are implemented and benefiting land users for the range of agro-ecosystems in the basin.

plans supporting sustainable management of agro-ecosystems and the restoration of degraded land

� Change in capacity in implementation and enforcement o Design and implementation of risk assessments o Implementation of national and local strategies

and action plans through adequate institutional frameworks and their maintenance

o Monitoring, evaluation and promotion of pilots

� Change in capacity in mobilizing resources o Leverage of resources o Human resources o Appropriate practices o Mobilization of advisory services

How is risk and risk mitigation being managed?

� How well are risks and assumptions being managed?

� What is the quality of risk mitigation strategies developed? Are these sufficient?

� Are there clear strategies for risk mitigation related with long-term sustainability of the project?

� Completeness of risk identification and assumptions during project planning

� Quality of existing information systems in place to identify emerging risks and other issues?

� Quality of risk mitigations strategies developed and followed

� Project documents and evaluations

� FAO, Project Staff and Project Partners


� Interviews


� What lessons have been learnt for the project to achieve its outcomes?

� What changes could have been made (if any) to the design of the project in order to improve the achievement of the project’s expected results?

� How could the project be more effective in achieving its results?


� Data analysis

Evaluation criteria: Efficiency - How efficiently is the Project implemented?

Is Project support channeled in an efficient way?

� Is adaptive management used or needed to ensure efficient resource use?

� Are the project logical framework and work plans and any changes made to them used as management tools during implementation?

� Are accounting and financial systems in place adequate for project management and be able to produce accurate and timely financial information?

� Are progress reports produced accurately, timely and responded to reporting requirements including adaptive management changes?

� Availability and quality of financial and progress reports

� Timeliness and adequacy of reporting provided

� Level of discrepancy between planned and utilized financial expenditures

� Planned vs. actual funds leveraged

� Cost in view of results achieved compared to costs of similar projects from other organizations

� Adequacy of project choices in view of existing context, infrastructure and cost

� Quality of RBM reporting (progress reporting,


� FAO, Representatives of Gov. and Project Staff

� Beneficiaries and Project partners


� Key Interviews


81

Evaluated component


Method

� Is project implementation as cost effective as originally proposed (planned vs. actual)

� Is the leveraging of funds (co-financing) happening as planned?

� Are financial resources utilized efficiently? Could financial resources be used more efficiently?

� How is RBM used during project implementation?

� Are there an institutionalized or informal feedback or dissemination mechanisms to ensure that findings, lessons learned and recommendations pertaining to project design and implementation effectiveness are shared among project stakeholders, FAO and GEF Staff and other relevant organizations for ongoing project adjustment and improvement?

� Does the project mainstream gender considerations into its implementation?

monitoring and evaluation)

� Occurrence of change in project design/ implementation approach (i.e. restructuring) when needed to improve project efficiency

� Existence, quality and use of M&E, feedback and dissemination mechanism to share findings, lessons learned and recommendation on effectiveness of project design.

� Cost associated with delivery mechanism and management structure compare to alternatives

� Gender disaggregated data in project documents

How efficient are partnership arrangements for the Project?

� To what extent partnerships/linkages between institutions/organizations are encouraged and supported?

� Which partnerships/linkages are facilitated? Which one can be considered sustainable?

� What is the level of efficiency of cooperation and collaboration arrangements? (between local actors, FAO/GEF and relevant government entities)

� Which methods are successful or not and why?

� Specific activities conducted to support the development of cooperative arrangements between partners,

� Examples of supported partnerships

� Evidence that particular partnerships/linkages will be sustained

� Types/quality of partnership cooperation methods utilized


� Project Partners

� Beneficiaries


� Interviews

Does the Project efficiently utilize local capacity in implementation?

� Is an appropriate balance struck between utilization of international expertise as well as local capacity?

� Does the Project take into account local capacity in design and implementation of the project?

� Is there an effective collaboration with scientific institutions with competence in SLM?

� Proportion of total expertise utilized taken from Recipient Countries

� Number/quality of analyses done to assess local capacity potential and absorptive capacity


� FAO, Project Team and Project partners

� Beneficiaries


� Interviews


� What lessons can be learnt from the project on efficiency?

� How could the project have more efficiently addressed its key priorities (in terms of management structures and procedures, partnerships arrangements etc.…)?


� Data analysis


82

Evaluated component


Method

� What changes could have been made (if any) to the project in order to improve its efficiency?

Evaluation criteria: Impacts - What are the potential and realized impacts of activities carried out in the context of the Project?

How is the Project effective in achieving its long-term objectives?

� Will achievements contribute to its goal that is to support the adoption of an integrated ecosystems approach for the management of land resources in the Kagera Basin, which will generate local, national and global benefits?

� Will the project achieve its environmental objective that is to address the causes of land degradation and restore ecosystem health and function and generate a range of global environmental benefits across the Kagera basin through the introduction of adapted agro-ecosystem management approaches?

� Will the project achieve its development objective that is to improve the livelihoods and hence contribute to reduced poverty of rural communities in the Kagera Basin through more productive and sustainable resource management practices that are technically feasible and socio-economically viable?

� Change in capacity: o To pool/mobilize resources o For related policy making and strategic planning, o For implementation of related laws and strategies

through adequate institutional frameworks and their maintenance,

� Change in use and implementation of sustainable alternatives

� Change to the quantity and strength of barriers such as change in o Institutions in charge of adopting an integrated

ecosystems approach for land management o Land management/monitoring system o Methodology for land management o Organization of land users o Policy and legislation governing land in the Kagera

Basin o Land management infrastructures o Livelihood of land users in the Kagera Basin


� Key Stakeholders

� Research findings; if available

� Documents analysis

� Meetings with FAO, Project Team and project Partners

� Interviews with project beneficiaries and other stakeholders

How is the Project impacting the local environment?

� What are the impacts or likely impacts of the project? o On the local environment; o On poverty; and, o On other socio-economic issues.

� Provide specific examples of impacts at those three levels, as relevant


� Key Stakeholders

� Research findings

� Data analysis

� Interviews with key stakeholders

Future directions for the Project

� How could the project build on its successes and learn from its weaknesses in order to enhance the potential for impact of ongoing and future initiatives?


� Data analysis

Evaluation criteria: Sustainability - Are the initiatives and results of the Project allowing for continued benefits?

Are sustainability issues

� Are sustainability issues integrated into the design and implementation of the project?

� Evidence/Quality of sustainability strategy

� Evidence/Quality of steps taken to address sustainability


� FAO, project staff and project Partners


� Interviews


83

Evaluated component


Method

adequately integrated in Project design?

� Beneficiaries

Financial Sustainability

� Does the project adequately address financial and economic sustainability issues?

� Are the recurrent costs after project completion sustainable?

� Level and source of future financial support to be provided to relevant sectors and activities after Project end?

� Evidence of commitments from international partners, governments or other stakeholders to financially support relevant sectors of activities after Project end

� Level of recurrent costs after completion of project and funding sources for those recurrent costs



� Beneficiaries


� Interviews

Organizations arrangements and continuation of activities

� Are results of efforts made during the project implementation period well assimilated by organizations and their internal systems and procedures?

� Is there evidence that project partners will continue their activities beyond project support?

� What degree is there of local ownership of initiatives and results?

� Were appropriate ‘champions’ being identified and/or supported?

� Degree to which project activities and results have been taken over by local counterparts or institutions/organizations

� Level of financial support to be provided to relevant sectors and activities by in-country actors after project end

� Number/quality of champions identified



� Beneficiaries


� Interviews

Enabling Environment

� Are laws, policies and frameworks addressed through the project, in order to address sustainability of key initiatives and reforms?

� Are the necessary related capacities for lawmaking and enforcement built?

� What is the level of political commitment to build on the results of the project?

� Efforts to support the development of relevant laws and policies

� State of enforcement and law making capacity

� Evidences of commitment by the political class through speeches, enactment of laws and resource allocation to priorities



� Beneficiaries


� Interviews

Institutional and individual capacity building

� Is the capacity in place at the regional, national and local levels adequate to ensure sustainability of results achieved to date?

� Elements in place in those different management functions, at the appropriate levels (regional, national and local) in terms of adequate structures, strategies, systems, skills, incentives and interrelationships with other key actors


� FAO, Project staff and project Partners

� Beneficiaries � Capacity assessments

available, if any

� Interviews � Documentation review

Social and � Does the project contribute to key building blocks for social and political sustainability?

� Example of contributions to sustainable political and social change in support of SLM reform


� Interviews

� Documentation review


84

Evaluated component


Method

political sustainability

� Does the project contribute to local Stakeholders’ acceptance of the new practices?


� Beneficiaries

Replication � Are project activities and results replicated elsewhere and/or scaled up?

� What is the project contribution to replication or scaling up of innovative practices or mechanisms that support sustainable management of agro-ecosystems and the restoration of degraded land?

� Number/quality of replicated initiatives

� Number/quality of replicated innovative initiatives

� Volume of additional investment leveraged

� Other donor programming documents

� Beneficiaries



� Interviews

Challenges to sustainability of the Project

� What are the main challenges that may hinder sustainability of efforts?

� Have any of these been addressed through project management?

� What could be the possible measures to further contribute to the sustainability of efforts achieved with the project?

� Challenges in view of building blocks of sustainability as presented above

� Recent changes which may present new challenges to the Project


� Beneficiaries



� Interviews

Future directions for the Project

� Which areas/arrangements under the project show the strongest potential for lasting long-term results?

� What are the key challenges and obstacles to the sustainability of results of the project initiatives that must be directly and quickly addressed?

� How can the experience and good project practices influence the strategies for sustainable management of agro-ecosystems and the restoration of degraded land in other Basins?

� Are national decision-making institutions (Parliament, Government etc.) in Recipient Countries ready to improve their sustainable management of agro-ecosystems and the restoration of degraded land?


� Data analysis


85

Annex 3 List of Documents Consulted

Anania Joseph, May 1, 2013, Kagera TAMP Synoptic View: Implementation Status and Next steps

Achieng’ Ogola Syprose, Land and Natural Resources Conflicts in Transboundary Agro-ecosystem Management - Project Kagera Basin

Bernadette Neves, September 2011, Recommendations to identify and promote PES as cost-benefit sharing mechanisms for SLM adoption by the Kagera TAMP

Bigirimana Gilbert, août 2011, Planification et tenure foncière au Burundi

Centre for Sustainable Development and Geo-Information Studies (CSDGIS), August 2012, Diagnostic analysis, land degradation assessment, mapping and land use planning and management in selected micro-catchments of Kayonza and Nyagatare districts in Rwanda under Kagera TAMP framework

Duveskog Deborah, October 2010, Final Report - Deborah Duveskog, REOA Regional FFS Support Officer - Summary of inputs provided under TCE International Consultant contract 65 days WAE; 26 July- 31 October, 2010

Duveskog Deborah, September 2010, Mission Report - Rwanda, Uganda, Tanzania (Kagera River Basin)

FAO, FAO Approaches to Capacity Development in Programming: Processes and Tools – LM2: Learning Module 2

FAO, Octobre 2012, Cadre de Programmation Pays (CPP Burundi) - 2012 – 2016

FAO, FAO Good Learning Practices for Effective Capacity Development – LM3: Learning Module 3

FAO, GEF, Project Document: Kagera TAMP

FAO, GEF, Various communications/exchange letters for approving Kagera TAMP

FAO, GEF, Participatory Watershed Planning – Kabale - Rwamate Watershed – Kagarama Parish, Bubare Sub-County

FAO, GEF, 6 avril 2013, Etat d’avancement des activités du projet TAMP Kagera

FAO, IIRR, Discovery-based learning in land and water management - A practical guide for farmer field schools

FAO, LEI, CTA, April 2006, Farmer field schools on land and water management in Africa - Proceedings of an international workshop in Jinja, Uganda, 24–29 April 2006

FAO-Kagera TAMP, April 29, 2013, Briefing of the Mid-Term Review Mission Rome, 29 April 2013 - Progress and Achievements of the Kagera TAMP

FAO-Kagera TAMP, October 2012, Terms of Reference for Report on Identification of Priority Transboundary Conflicts and Policy/Legal Issues (to Address LD, BD, CS, CC and a Proposal of Initial Actions and Areas tor the Tamp Project

FAO-Kagera TAMP, 2013, Terms of Reference for a Regional Consultant for identification of trans boundary agro ecosystems management issues between Burundi and Rwanda

FAO-Kagera TAMP, August 2011, Regional Technical Workshop on Land Use planning and Management held on 29th – 31st August, 2011 at White Horse Inn, Kabale-Uganda - Workshop Proceedings

FAO-Kagera TAMP, Baseline Catchment Reports (multiple reports)

FAO-Kagera TAMP, WOCAT QA and QT Summaries (multiple reports)

FAO-Kagera TAMP, Site Characterization Reports for Tanzania (several reports)


86

FAO-Kagera TAMP, April/May 2012, Participatory Watershed Management Training Uganda

FAO-Kagera TAMP, Activity Timeline – Country Updates (4)

FAO-Kagera TAMP, Various communication products on Kagera TAMP

FAO-Kagera TAMP, QM Maps – Validated Data Presentations for BU, RW, TZ and UG

FAO-Kagera TAMP, LUS Database Presentations for BU, RW, TZ and UG

FAO-Kagera TAMP, Maps for the 4 countries

FAO-Kagera TAMP, Monographies et Caractérisation de bassins versants in Burundi (multiple reports)

FAO-Kagera TAMP, Project Implementation Review (PIR) (2 reports: 2010-2011 and 2011-2012)

FAO-Kagera TAMP, November 2012, Land Degradation Assessment and Sustainable Land Management Plan in Butare Micro catchment, Ngoma Sector, Rulindo District - Butare Micro-catchment Assessment Report

FAO-NRL, November 2, 2012, IFAD – SDC Grant Follow-up of the 2012 Land and Water Days at FAO “Improving the quality and impact of land and water projects” - Cross-Pollination of Watershed Management Projects: A Learning Path in Brazil and Rwanda - Phase 1: Brazil 8 – 23 September (various reports on the Brazil exchange)

GEF, March 2008, Revised Project Identification Form (PIF): Kagera TAMP

GEF, March 2011, Land Degradation Focal Area - Portfolio Monitoring and Assessment Tool - Guidelines

GEF, Land Degradation Focal Area Strategy and Strategic Programming for GEF-4

GEF, October 7, 2011, Guidelines for Project Financing

GEF, April 7, 2013, Cofinancing

GEF, June 17, 2009, Co-Financing Data Related to GEF Operations

HLPE4, June 2012, Social protection for food security - The High Level Panel of Experts on Food Security and Nutrition

Janie Rioux, December 2011, Monitoring and Evaluation Plan for Kagera TAMP

Julianus Thomas, February 2013, Sustainable Land Management Farmer Field Schools’ Activities of the Kagera Trans-boundary Agro-Ecosystem Management Project (TAMP) - Refresher Training Course and FFS Follow up and On-Field Trainings For Rwanda - Interim Report

Julianus Thomas, Report for Development of Comprehensive Project SLM FFS program, Curriculum and Implementation of Training of Trainers courses in Uganda, Tanzania and Rwanda

Kagera TAMP, Burundi: Rapport mensuels et trimestriels de progrès (several)

Kagera TAMP, Rwanda: Monthly and Quarterly Progress Reports (several)

Kagera TAMP, Uganda: Monthly and Quarterly Progress Reports (several)

Kagera TAMP, Tanzania: Monthly and Quarterly Progress Reports (several)

Kagera TAMP, Co-financing Letters (several)

Kagera TAMP, LOAs and MOUs (copies of agreements with Service Providers and copies of MOU signed with partners)

Kagera TAMP, Procès verbaux des reunions du comité de pilotage (Burundi) (several)

Kagera TAMP, Juillet 2011, État des lieux des agroécosystèmes et systèmes d’utilisation des terres dans les zones du programme de gestion transfrontalière des agro-écosystèmes du bassin de la Kagera au Burundi

Kagera TAMP, Plans d’action communautaire de micro bassin versants (several)


87

Kagera TAMP, Kagera TAMP workplan with follow -up activities for 2013/14

Kagera TAMP, Semi-annually Progress Reports (July-December 2010 to Jan-June 2012)

Kakuuto Community Development Project (KACODEP), KACODEP - KAGERA –TAMP (FAO) 1st Phase Report

Kizima Jonas B., May 2013, Transboundary Issues of Land Degradation and Conflict Related to Livestock Management and Movements and Identification of Strategies and Option for Sustainable and Equitable Land and Livestock Management

LTS, December 10, 2012, Feasibility Study for an Integrated Watershed Management Programme for the Kagera River Basin

Mashinga Theobald N., August 2010, Draft Report: Assessment of Baseline Information in Six Districts of Kagera Tamp in Rwanda

Ministère de l’Agriculture et de l’Élevage (Burundi), Juin 2011, Plan National d’Investissement Agricole (PNIA) - 2012 - 2017

Ministère de l’Agriculture et de l’Élevage (Burundi), Juillet 2008, Stratégie Agricole National – 2008-2015

Ministry of Agriculture and Animal Resources (Rwanda), Strategic Plan for the Transformation of Agriculture in Rwanda Phase III – PSTA III, Draft I

Ministry of Agriculture and Animal Resources (Rwanda), February 2009, Strategic Plan for the Transformation of Agriculture in Rwanda – Phase II (PSTA II) - Final Report

Muheto Ruzika N., Report on Identification of Priority Transboundary Conflicts and Policy/Legal Issues (to Address LD, BD, CS, CC and a Proposal of Initial Actions and Areas tor the Tamp Project

NBI, BRL, July 15, 2008, Kagera River Basin Monograph – Basin Development Report

NBI, August 2009, Harmonize National Reports to Assess, Review and Design of a Sustainable Hydrometric Network for Kagera River Basin

Ndayishimiye Jean Marie, Réussites du PDDAA au Burundi

NEPAD, The TerrAfrica/GEF Strategic Investment Programme for Sustainable Land Management in Sub-Saharan Africa (SIP)

Neves Bernadette, April 23, 2013, Kagera TAMP Component 4 - Adoption of improved land use systems/practices generating improved livelihoods and environmental services - 4.3 Identify and promote market opportunities and other cost/benefit sharing mechanism such as PES – Update

Neves Bernadette, Payment for Environmental Services in the Kagera TAMP

Republic of Rwanda, April 9, 2013, Economic Development And Poverty Reduction Strategy - 2013–2018 - Shaping our Development

République du Burundi, 12 octobre 2011, Cadre stratégique de croissance et de lutte contre la pauvreté - deuxième generation (CSLP II)

Rwanda Agriculture Board (RAB), Soil Erosion Control Baseline in Rwanda – Year 2012

Rwanda Agriculture Board (RAB), April 10, 2013, Strategic Plan (2013-2017) for Rwanda Agriculture Board - Extending Innovations to Farmers

Sendegeya Marcien, Sinzinkayo Joram, Havyarimana Déo, September 2012, Analyse Biophysique des Bassins Versants Nyakibari, Magamba, Gihehe, Rusi, Rwibikara et Kibogoye

Sendegeya Marcien, Sinzinkayo Joram, Havyarimana Déo, September 2012 and April 2013, Analyse Biophysique des Bassins Versants Nyakibari, Magamba, Gihehe, Rusi, Rwibikara et Kibogoye (2 reports)


88

UN Rwanda, April 2013, United Nations Development Assistance Plan – 2013-2018

UN, The Human Rights Based Approach to Development Cooperation: Towards a Common Understanding Among UN Agencies

UNEP, GEF, PDF-B Block B Grant Request

UNEP, GEF, PDF-A

Universalia, July 2010, Country Let Evaluation of Delivering as One in Rwanda – Draft Final Report

Various, August 2011, Presentations for the Regional Workshop on Land Planning and Management - 29-31 August 2011

_____, Sanga Bee Keepers – Project Technical Report, Kagera TAMP – November 2011-December 2012

_____, Progress Report on SLM/FFS Field Implementation by District

_____, GEF Project Concept Review Criteria

_____, Draft Report Kabale workshop (notes Sally), Kagera TAMP Regional Technical Workshop on Land Planning and Management

_____, Kabale Workshop Evaluation

_____, Validation Workshop for Land Degradation Assessment and Micro-catchment characterization of micro-catchments in the districts of Bugesera an, Kayonza, Kirehe and Nyagatare under Kagera TAMP - Proceedings and Recommendations

Websites Consulted:

FAO Website http://www.fao.org/home/en/

Kagera TAMP Website http://www.fao.org/nr/kagera/en/

GEF Website: http://www.thegef.org/gef/

http://www.icp-confluence-sadc.org/projects/transboundary-agro-ecosystem-management-programme-kagera-river-basin

http://hrbaportal.org


89

Annex 4 Discussion Guide

Note: This was only a discussion guide for the Evaluator; it is a simplified version of the evaluation matrix. All questions were not asked to each meeting/interview; it was a reminder for the Evaluators on the type of information required to complete the evaluation exercise and a guide to prepare the semi-structured interviews.

I. RELEVANCE - How does the Project relate to the main objectives of the GEF, UNCCD and to the environment and development priorities of Recipient Countries? I.1. Is the project relevant to the GEF and UNCCD objectives? I.2. Is the project relevant to FAO objectives? I.3. Is the project relevant to Recipient Countries’ development objectives? I.4. Does the project address the needs of target beneficiaries? I.5. Is the project internally coherent in its design? I.6. How is the project relevant in light of other donors? Future directions for similar projects I.7. What lessons have been learnt and what changes could have been made to the project in order to

strengthen the alignment between the project and the Partners’ priorities and areas of focus? I.8. How could the project better target and address priorities and development challenges of targeted

beneficiaries? II. EFFECTIVENESS – To what extent are the expected outcomes of the project being achieved? II.1. How is the Project effective in achieving its expected outcomes?

o Transboundary coordination, information sharing and monitoring and evaluation mechanisms operational and effective in promoting sustainable, productive agro-ecosystems and restoration of degraded lands.

o Enabling policy, planning and legislative conditions are in place to support and facilitate the sustainable management of agro-ecosystems and the restoration of degraded land.

o Capacity and knowledge are enhanced at all levels for the promotion of – and technical support for – sustainable management of land and agro-ecosystems in the basin.

o Improved land and agro-ecosystem management practices are implemented and benefiting land users for the range of agro-ecosystems in the basin.

II.2. How is risk and risk mitigation being managed? Future directions for similar projects II.3. What lessons have been learnt for the project to achieve its outcomes? II.4. What changes could have been made (if any) to the design of the project in order to improve the

achievement of project’ expected results? II.5. How could the project be more effective in achieving its results? III. EFFICIENCY - How efficiently is the project implemented? III.1. Is adaptive management used or needed to ensure efficient resource use? III.2. Are the project logical framework and work plans and any changes made to them used as management

tools during implementation? III.3. Are accounting and financial systems in place adequate for project management and producing accurate

and timely financial information? III.4. Are progress reports produced accurately, timely and responding to reporting requirements including

adaptive management changes? III.5. Is project implementation as cost effective as originally proposed (planned vs. actual) III.6. Is the leveraging of funds (co-financing) happening as planned?


90

III.7. Are financial resources utilized efficiently? Could financial resources be used more efficiently? III.8. How is RBM used during project implementation? III.9. Are there an institutionalized or informal feedback or dissemination mechanism to ensure that findings,

lessons learned and recommendations pertaining to project design and implementation effectiveness were shared among project stakeholders, FAO and GEF Staff and other relevant organizations for ongoing project adjustment and improvement?

III.10. Does the project mainstream gender considerations into its implementation? III.11. To what extent are partnerships/ linkages between institutions/ organizations encouraged and supported? III.12. Which partnerships/linkages are facilitated? Which one can be considered sustainable? III.13. What is the level of efficiency of cooperation and collaboration arrangements? (between local actors,

FAO/GEF and relevant government entities) III.14. Is an appropriate balance struck between utilization of international expertise as well as local capacity? III.15. Did the project take into account local capacity in design and implementation of the project? Future directions for the project III.16. What lessons can be learnt from the project on efficiency? III.17. How could the project have more efficiently addressed its key priorities (in terms of management

structures and procedures, partnerships arrangements etc…)? IV. IMPACTS - What are the potential and realized impacts of activities carried out in the context of the project? IV.1. Will the project achieve its goal that is “to support the adoption of an integrated ecosystems approach for

the management of land resources in the Kagera Basin which will generate local, national and global benefits?

IV.2. Will the project achieve its environmental objective that is “to address the causes of land degradation and restore ecosystem health and function and generate a range of global environmental benefits across the Kagera basin through the introduction of adapted agro-ecosystem management approaches”?

IV.3. Will the project achieve its development objective that is “to improve the livelihoods and hence contribute to reduced poverty of rural communities in the Kagera Basin through more productive and sustainable resource management practices that are technically feasible and socio-economically viable”?

IV.4. How is the project impacting the local environment such as impacts or likely impacts on the local environment; on poverty; and, on other socio-economic issues?

Future directions for the project IV.5. How could the project build on its apparent successes and learn from its weaknesses in order to enhance

the potential for impact of ongoing and future initiatives? V. SUSTAINABILITY - Are the initiatives and results of the project allowing for continued benefits? V.1. Are sustainability issues adequately integrated in project design? V.2. Does the project adequately address financial and economic sustainability issues? V.3. Is there evidence that project partners will continue their activities beyond project support? V.4. Are laws, policies and frameworks being addressed through the project, in order to address sustainability

of key initiatives and reforms? V.5. Is the capacity in place at the national and local levels adequate to ensure sustainability of results

achieved to date? V.6. Does the project contribute to key building blocks for social and political sustainability? V.7. Are project activities and results being replicated elsewhere and/or scaled up? V.8. What are the main challenges that may hinder sustainability of efforts? Future directions for the project V.9. Which areas/arrangements under the project show the strongest potential for lasting long-term results? V.10. What are the key challenges and obstacles to the sustainability of results of project initiatives that must be

directly and quickly addressed?


91

Annex 5 Evaluation Mission Agenda

The timetable below applies to the whole evaluation process. Number of days allocated to team members will vary according to responsibility.


Cumulative total days

Finalization of TOR (including country consultations) Finalization of MTE team composition and mission time schedule Mission leader 2nd Evaluator Remarks

Background reading (home base); Preparat. of the Evaluation matrix

Background reading (home base);

2 From home to Rome From home to Rome Sun 28-Apr 1 Briefing at FAO HQ, Rome, Italy


Mon 29-Apr 1



Tue

Tue 30-Apr 1 Desk work Desk work Wed 1 May 1 Briefing with Regional Coordinator, Kigali, Rwanda

Briefing with Regional Coordinator, Kigali, Rwanda and visit regional partners (NBI, VI, IFDC?)

Project strategy and regional considerations

Thu 2-May 1 Assessment Rwanda Travel Uganda;

Assessment Uganda- Kabale

Fri 3-May 1 Assessment Rwanda Assessment Uganda

Sat 04-May 1 Travel Burundi Desk work

Desk work Sun 05-May 1

Assessment Burundi Assessment Uganda Mon 06 May 1 Assessment Burundi Assessment Uganda

(Kampala and Entebbe)

Tue 07-May 1 Assessment Burundi Travel to Tanzania

(met by NPM at border) and desk work

Wed 08-May 1 Travel Rwanda; Desk work Assessment, Tanzania

(Bukoba and Missenyi)

Thu 09-May 1 Rwanda Assessment, Tanzania

(Karagwe and Ngara) and travel to Kigali

Fri 10-May 1 Debr. with Regional Coordinator, Kigali,; Travel Dar-Es-Salaam

Debr. with Regional Coordinator, Kigali,; Travel Dar-Es-Salaam

Sat 11-May 1

Sun 12 May 1


92



Mon 13-May 1 Debriefing FAO HQ, Rome, Italy

Debriefing FAO HQ, Rome, Italy

Tue 14-May 1 If possible and required Debriefing with NRL/ travel home

If possible and required Debriefing with NRL/ travel home

Wed 15-May 1 Finalize report and submission (WAE at home base)

Finalizes section(s) under his responsibility

5 days

5 Comments by OED within 7 days

Circulation of final draft 2 days work, within 3 days 2

Comments to team on final draft

within two weeks

Finalization of report 2 days, within one week 2

RCU presents preliminary/final Evaluation report to the RPSC Kigali and NPMs to NPSCs

RWANDA - Programme Date Activity Remarks

30/Apr/13 (Tue) Arrival of MTE team (Jean-Jo Bellamy & William Critchley) 1/May/13 (Wed) Desk work 2/May/13 (Thu) - Courtesy call (FAO Rep)

- Briefing with Regional Project Coordinator

- Meeting with partners (MINAGRI, MINIRENA, NBI, LVEMP II, REMA, Vi-Agroforestry)

- Both consultants - Country perspectives/brief - Project strategy and regional

considerations

3/May/13 (Fri) Field Visit (Visit to Karambo, Butare Meet with Rulindo mayor)

- Jean-Jo & Project team + District Authorities

- William travels to Uganda

4/May/13 (Sat) Rwanda assessment (Visit more field sites Kamonyi/Bugesera-Gakindo?)

5/May/13 (Sun) Travel to Burundi Jean-Jo Bellamy 9/May/13 (Thu) Travel to Rwanda + Desk Work Jean-Jo Bellamy 10/May/13 (Fri) Assessment Rwanda (Pending issues and questions) 11/May/13 (Sat) Briefing & Travel to Tanzania Jean-Jo Bellamy


93

Burundi - Programme

Date Activités

05.05.2013 Arrivée de M. Jean-Jo Bellamy

06.05.2013 • Rencontre avec les partenaires • Rencontre avec FAOR Burundi • Rencontre avec Madame la Ministre de l’Agriculture de l’Elevage • Debrieng sur la sécurité au PNUD • Rencontre avec les partenaires • Séance de travail avec le NPM Kagera Burundi

07.05.2013 Visite des réalisations dans la province de Mwaro et Gitega

8.00 Départ de bujumbura vers Nyakirwa

8.45 • Arrivée à Nyakirwa • Visite des exploitations des cultures fourragères • Discussions avec les partenaires. • Rencontre avec le Gouverneur de la province de Mwaro

10.30 Départ vers Nyakibari et Magamba

12.00 • Echanges avec le FFS Nyakibari

13.00 • Echanges avec les FFS Biraturaba et Vyizigiro de Magama

15.00 Départ vers Gishubi en province de Gitega

16.00 • Echanges et visites des réalisations du FFS Twunguranubumenyi de Bucana • Discussions avec les partenaires et les élus locaux

17.30 Départ vers Gitega

08.05.2013 • Rencontre avec des experts de l’IGEBU chargés des mesures hydrologiques • Visites des FFS Gasunu et Mirama de la province de Gitega • Visites du FFS Ruvyironza en commune Giheta en province de Gitega, • Retour sur Bujumbura

09.05.2013 Dernier jour de la mission

8.00 Rencontre avec le Représentant de la FAO et départ à l’aéroport

Uganda - Programme

Date District visited Comment


94

Date District visited Comment Thursday 2 May Arrival from Kigali /* Friday 3 May Kabale Meet the District Leadership,

visit Catchment Saturday 4 May Mbarara or

Kiruhura Visit the Rubagano catchment in Mbarara, or visit Kiruhura district, Sanga watershed (rangeland management)

Sunday 5 May Travel to Kampala Monday 6 May Kampala FAOR, NPC, MAAIF, Tuesday 7 May Kampala NEMA, MLWE, TERRAFRICA focal point Wednesday 8 May Travel to Mutukula border station */ Depending on the completion of the mission briefing in Kigali, it is suggested that the team sleeps in Kabale on Thursday so that one can meet the district authorities early on Friday morning. If this is not possible it is suggested to come very early to be at the border by 9 a.m. (Uganda time) so that the mission can meet the district official of Kabale, the host district of the project.

Tanzania - Mission Programme 8 May Travel Masaka - Bukoba (meet at Mutukula border station)

9 May Visit Karonge micro-catchment, Kabirizi FEC and if possible Butulage micro-catchment

10 May Visit Kihanga/Katera micro-catchment and if time allows Bujuruga micro-catchment and back to Bukoba

11 May Travel Bukoba - Mwanza (by car, if no flight available on 12 May)

12 May Flight Mwanza - Dar es Salaam

13 May Visit FAO Representation Dept of Environment, Vice President’s Office and if time allows visit NLUPC

13 May Return flight


95

Annex 6 List of People Met

Title Name Institution Position

Ms. Bunning, Sally FAO Lead Technical Officer

Mr. Schlingloff, Stefan FAO Budget Holder/Project Operations

Ms. Verbeke, Isabelle FAO Communication

Mr. Faures, Jean Marc FAO Water management and climate change adaptation

Mr. Groppo, Paolo FAO Participatory and Negotiated Territorial Development (PNDT)

Ms. Neves, Bernardete FAO PES

Ms. Piechowiak, Iwona FAO WOCAT QA, QT

Ms. Rioux, Janie FAO M&E, LADA-Local

Ms. Cooney, Barbara FAO FAO/GEF Coordinator

Ms. Petri, Monica FAO LUS and WOCAT-QM

Ms. Wolter, Petra FAO Fouta Djallon Highlands Integrated Natural Resources Management Project

Ms. Belli, Luisa FAO Office of Evaluation

RWANDA

Mr. Joseph Anania Bizima Project Regional Project Coordinator

Mr. Emmanuel Muligirwa Project National Project Manager (Rwanda)

Mr. Julianus Thomas Project FFS Master Trainer

Mr. Attaher Maiga FAO-Rwanda FAO Representative - Rwanda

Mr. Laurent Gashugi FAO-Rwanda Assistant Representative Programme - Rwanda

Mr. John Veerkamp IFDC Country Representative and Chief of Party

Mr. Emmanuel Olet NBI Program Officer Water Resources Development


96


Mr. Rukundo Jean Marie Vi-Agroforesty Field Operation Unit head

Mr. Justus Kangwagye Rulindo District Mayor / President RALGA

Mr. Prosper Mulindwa Rulindo District Vice Mayor – Economic planning

Mr. Fidel Uwimpaye OPEDSA Director

Mr. Raphael Rurangwa MINAGRI Director General of planning and Programme Coordination

Ms. Caroline Kayonga MINIRENA Permanent Secretary

Ms. Annette Sylvie Muhayimana

REMA Project Coordinator LVEMPII

Dr. Rose Mukankomeje REMA Director General / GEF Focal Point

Nyirarubomboza Micro-catchment

Mr. Fidel Uwimpaye OPEDSA – Service Provider Director

Mr. NSHIMIYUMUREMYI Viateur

Nyirarubomboza micro catchment

FFS Facilitator

FFS: Duteraninkuna 30 members

FFS: Nyirabumboza II 41 members

Gakindo Micro-catchment

Mr. Didier Butoyi AVODI – Service Provider Agronomist

Mr. BIZUMUREMYI Alexandre

Gakindo micro catchment FFS Facilitator

FFS: Dufatanye Gakindo 30 members

FFS: Gakindo II 30 members

BURUNDI

Mr. Salvator Ndabirorere Project National Project Manager (Burundi)

Mr. Elie Buzoya Project Consultant


97


Mr. Mohamed Hama Garba FAO-Burundi Représentant au Burundi

Ms. Odette Kayitesi Ministère de l’Agriculture et de l’Elevage

Ministre

Mr. Epimaque Murangerantwari

Ministère de l’Eau, de l’Environnement, de l’Aménagement du Territoire et de l’Urbanisme

Secrétaire Permanent / Point Focal Opérationnel GEF au Burundi

Mr. Massimo Giovanola FAO-Burundi Chargé des Opérations au Burundi

Mr. Cyrille Mbonihankuye ISABU Chercheur

Ms. Gloriose Habonayo ISABU Chercheur, spécialiste dans la fertilisation des sols

Mr. Willy Ntamagara MINAGRIE Conseiller, / Point Focal du projet TAMP Kagera

Mr. Sébastien Bukuru ENVIRON- PROPRE (ONG)

Réprésentant Légal

Aloys Rurantije IGEBU-Gitega Directeur de l’Hydrométéorologie

Site Nyakirwa : Cultures fourragères, blocs à lécher, ensilage

Mr. Aloys Nijimbere ISABU Chercheur /Responsable de la Station Zootechnique Mahwa

Mr. Elias Minani ISABU Chercheur /Agrostologue

Mr. Innocent Masabo Agri-éleveur multiplicateur des cultures fourragères diversifiées

Mr. Innocent Nkeshimana ABAVUDUKANAKIVI Président de l’Association (Agri-éleveur)

ABAVUDUKANAKIVI de Nyakirwa

25 Membres de l’Association

Mr. François Ninteretse DPAE Mwaro Directeur Provincial

Mr. Joseph Yengayenge DPAE Mwaro Responsable de la formation / Facilitateur Provincial du Projet TAMP Kagera dans la Province de Mwaro

Site Nyakibari : FFS TWITEZIMBERE

Mr. Sébastien Bukuru Prestataire de services Bassins versants Nyakibari, Magamba et Tye

Mr. Emmanuel Kantungeko FFS TWITEZIMBERE de Nyakibari

Agronome de zone kayokwe, Animateur


98


Mr. Marc Ndayisenga FFS TWITEZIMBERE de Nyakibari

Président

FFS TWITEZIMBERE 30 Membres

Echanges avec les FFS BIRATURABA et VYIZIGIRO de Magamba

Mr. Richard Mbonyingingo FFS Biraturaba et FFS Vyizigiro

Animateur

FFS Biraturaba et FFS Vyizigiro

60 membres

Commune de Gishubi, province de Gitega

Mr. Herménégilde Manyange

DPAE Gitega Directeur Provinciale / Président du Comité de Pilotage National du projet TAMP Kagera

Mr. Donatien Karumbete DPAE Gitega Chef de Service Formation / Facilitateur provincial du projet TAMP Kagera

Mr. Serges Barampanze FFS TWUNGURANUBUMENYI

Animateur

FFS TWUNGURANUBUMENYI

25 Membres

Région de Gitega

Mr. Guy Ngenzebuhoro FFS de Gasunu et de Mirama

Animateur

FFS de Gasunu et de Mirama

50 membres


Chef de Zone Gasunu


Chef de colline Mirama

Province de Karusi

Isaac Nimpagaritse DPAE Karusi Responsable de la Production Végétale / Facilitateur Provincial du projet TAMP Kagera

Joël Rukiga FFS TUGWANYINKUKURA de Rusi

Animateur

Kaboneka Désiré Prestataire de service/ bassin versant de Rusi

FFS TUGWANYINKUKURA de Rusi

30 membres


99


Yolande Ntirandekura FFS TUGWANYINKUKURA de Buhiga

Animateur

Kaboneka Désiré Prestataire de service/ bassin versant de Rusi

FFS TUGWANYINKUKURA de Buhiga

30 membres

TANZANIA

Mr. Fidelis Kaihura Project National Project Manager (Tanzania)

Ms. Diana Tempelman FAO-Tanzania Representative in Tanzania

Mr. Richard Muyungi Ministry of Environment Ag. GEF NFP, Kagera TAMP Tanzania NFP and Chair NPSC

Mr. Nassoro Mnambira Kagera Region Regional Administrative Secretary

Mr. Eldom Anyosisye Kagera Region Assistant Administrative secretary and member of NPSC

Mr. Innocent Ndyetabura Agricultural Research Institute Maruku, Bukoba

Ag. Officer In charge

Ms. Bertha Munyaga Bukoba district District Project Facilitator

Mr. Savera Mutayabarwa Karonge Micro-catchment Committee

Chair

Mr. Josephat B Kakuru Kyerwa (previously Karagwe district)

Ag. District Commissioner and District Administrative Secretary

Mr. Ernest Kalisa Kyerwa District Councillor Isingiro ward,

Mr. Jafari Kyobya Karonge Micro-catchment Site facilitator

Ms. Julitha Nkuba Butulage micro-catchment Site facilitator and Manager, Kabirizi Farmer Extension Centre

Mr. Eustad Shumbusho Kolping Services (Service provider)

Representative

Mr. Mathias Seth ARI Maruku Project Facilitator

Mr. Francis Katemana Karonge Micro-catchment Committee

Member

Mr. Grace Baguma Vi-Agroforestry Project, Kaisho

Zonal Coordinator

Ms. Edina Lutakinikwa Kihanga/Katera micro-catchment

Site facilitator


100


Mr. Focus Matayo Kihanga Village Village Chair

Mr. Thomas Kiruba Isingiro Ward Extension Officer

Mr. Chrisant Katunzi Kihanga Village Executive Officer

Mr. Paul Mwanakunu

Kihanga/Katera Micro-catchment Committee and Katera Pasture Improvement FFS

Chair

Mr. Richard Muyungi Ag. GEF NFP, Kagera TAMP Tanzania NFP and Chair NPSC

District of Bukoba FFS Twelinde FFS Nitubasa FFS Byonabyaitu

30 members

District of Karagwe

FFS Mazingira FFS Kinymugera FFS Uoto wa Asili Siina FFS Mkombozi

15 members

FFS Amani FFS Emilembe

70 members

UGANDA

Mr. Wilson Bamwerinde Project National Project Manager (Uganda)

Mr. Alhaji Jallow FAO-Uganda Representative in Uganda

Ms. Maria De Silva FAO-Uganda Operations Officer

Mr. Sunday Mutabazi Farm Development (MAAIF)

Commissioner

Mr. Alex Lwakuba MAAIF Commissioner

Mr. Kamala Grace MAAIF Senior agriculture officer

Ms. Jeniffer Tumushabe. Kabale DLG Director of Production

Mr. Christopher Bukenya NAADS Deputy Executive Director

Mr. Mutesi Osham Kacodep Facilitator

Mr. Kiyingi Jamil Rakai DLG Natural Resources Officer

Mr. Murangira Kenneth Mbarara FFS facilitator


101


Mr. Kashangirwe Sarapio Kabingo Watershed, Mbarara

Service provider

Mr. Buzuri Jonathan Rubagano Service provider

Mr. Byamukama Rubarita Mbarara FFS facilitator

Ms. Tumuhairwe Honest Mukirwa, Kabale Service provider

Mr. Edison Hillman Kabale District Facilitator

Mr. Twinorusa Sam Kabale FFS facilitator

Mr. Lubega Emmanuel Rakai district Service Provider

Mr. Loy Zikampereza Kabale district Acting district chairperson

Ms. Peace Rutarimara Sector committee of production and marketing, Kabale district

Chairperson

District of Kabale

FFS Hamurambi FFS Kagarama Bahingyi

50 members

FFS Bukoora FFS Kanyakwanzi FFS Kagugo

40 members

District of Mbarara

FFS Rubangano (a) FFS Rubangano (b) FFS Kakore FFS Rubangano Bataka

70 members


102

Annex 7 Project Expected Results and Planned Activities

Expected Results Financial

resources Activities


Output 1.1: A basin-wide coordination mechanism is established to facilitate transboundary dialogue, basin-level policy harmonization and coordination of national/sub-national actions

1,766,873

• National-level workshops among stakeholders and decision-makers leading to the development of policy, institutional and legal mechanisms for enhanced inter-sectoral cooperation to address the priority transboundary issues identified, resolve conflicts and promote sustainable land and agro-ecosystems management (SLaM)

• Appropriate, affordable institutional mechanisms developed for sustained regional cooperation and support across the basin for SLaM

• Regional workshop held to finalize and agree on required policy, legal and institutional mechanisms and tools and implementation arrangement across the basin

• A broad public information and awareness-raising campaign • National and transboundary mechanisms established and functioning for coordinated and harmonized policy and legal

approaches and decision making Output 1.2: An efficient basin-wide knowledge management system is established to support information requirements and decision-making processes at all levels

• An environmental monitoring and information system for SLaM (SLaM-IS) • A pilot district level GIS capacity developed in each country • District/Community information centers • Project information and communication system • Linkages with relevant networks

Output 1.3: Project monitoring and evaluation system and technical reporting supporting Kagera TAMP implementation and decision making in the basin.

• Development of the project participatory M&E system • Training in participatory M&E • Independent mid term (year 3) review and a final (year 5) project evaluation


Output 2.1: Sustainable management of land and agro-ecosystems (SLAM) mainstreamed in national and district development programmes and basin institutions, enhancing synergy among sector

423,342

• Mechanisms and approaches for improved synergy and harmonization among sectoral plans • SLaM Mainstreaming into policy and planning processes • Inter-sectoral workshops to review results and lessons learnt • Knowledge and expertise shared with districts and communities


103



strategies and across the river basin Output 2.2: Regulatory actions and conflict resolution mechanisms developed and used to promote - or remove existing barriers to - sustainable land and agro-ecosystem management

• Increased effectiveness of implementation of enhanced policies, laws and by-laws for SLaM • Monitoring of policy/legal application/enforcement and conflict resolution capacities • Experiences and lessons–learnt on the above regulatory and legal issues shared among stakeholders, and appropriate

mechanisms developed for their application at a wider scale

Output 2.3: A coherent strategic land use planning framework in place (from river basin to district/provincial and community levels) (based on thematic reviews, stakeholder consultations and priority setting) to support SLM efforts by rural communities

• District consultations and priority setting for inter-sectoral processes among planners (land use; financial) and technical advisors

• Status and trends of land degradation on croplands assessed, and cost-benefit options for improved management and restoration of degraded lands identified

• Status and trends of pasture and rangelands assessed and cost-benefit options identified and a coherent strategy developed for improved range Management

• Status, trends and opportunities for better protection and management of wetlands across the basin identified • Awareness raising consultations at district level with technical specialists and target community leaders leading to the

identification of prioritized concerted actions for inclusion in district and community action plans for the conservation and sustainable use of agricultural biodiversity and associated livelihood benefits

• Status and trends of energy use and needs at community level and across the basin, and actions identified and implemented to meet energy requirements and ensure the maintenance and regeneration of trees and forest resources

• Dialogue with local stakeholders on risk of crop and livestock pest and disease transmission leading to mechanisms identified to better manage cross-border movements

• District officers and local government staff enabled to develop and implement inter-sectoral actions and plans for agriculture and natural resources management

• Communities and districts supported (human and financial resources) to implement the above action plans and strategies

Outcome 3: Capacity and knowledge are enhanced at all levels for the promotion of – and technical support for – sustainable management of land and agro-

ecosystems in the basin.

Output 3.1: Methods and approaches to promote the adoption of SLM practices and agro-ecosystems (pastoral and cropping) developed and validated through demonstrations and study plots and participatory learning and adaptive

1,230,003

• District consultations and agreements • Pilot micro-catchments and communities, representing the range of agro-ecosystems and target land units and contexts • Knowledge base created in target areas • Training methods and materials in beneficiary districts developed with support of subject matter specialists/trainers to

support integrated agro-ecosystems approaches and FFS approaches • Varied information/materials produced and disseminated • Development of effective extension, scaling-up, income generation and marketing strategies • Community awareness/ training sessions held on the effects of current practices on-farm and on ecosystem services

and opportunities identified for reducing/ preventing negative impacts and generating benefits through integrated agro-


104



management processes ecosystem approaches and longer term management strategies • Intervention areas and sites identified and agreed upon with communities and districts for demonstrations/study plots

to test and locally adapt technologies building on local experiences and innovation Output 3.2: Enhanced quality of services provided to rural communities in the basin for improved agro-ecosystems management, through training by inter-sectoral teams, participatory research and monitoring and building on local knowledge and innovations

• Practical training workshops held to develop the knowledge and build capacity of service providers and community leaders (male and female) on integrated agro-ecosystems approaches and the benefits of agricultural biodiversity

• Training of trainers held on participatory learning-action-research approaches for working with local land users • Short courses and exchange visits conducted for sharing knowledge and experiences among service providers and

local innovators across the basin • Linkages established between communities and farmer groups with private sector suppliers and researchers for

improved access to inputs and training in their use • Collaboration between researchers, service providers and land users/farmers/common interest groups promoting

diversified farming systems that are productive and sustainable in the short and long term • Raised awareness of the importance of sustainable land management for ensuring reliable and good quality water

supply and community-level opportunities identified and supported Outcome 4: Improved land and agro-ecosystem management practices are implemented and benefiting land users for the range of agro-ecosystems in the

basin.

Output 4.1: Participatory land management plans are developed and implemented in targeted communities, micro-catchments and wider land units

2,360,682

• Training conducted and participatory land use plans developed for targeted community territories (68), micro-catchments (46), and specific land /agro-ecological units (10)

• Capacity built for implementation and monitoring of action plans • Stakeholder review conducted of pilot results and experiences from year 1-3 leading to promotion and wider

application of successful planning and management tools, processes and interventions

Output 4.2: Improved land use and agro-ecosystem management practices are successfully adopted by farmers and herders in targeted communities and replicated in other areas

• Target communities and land users sensitized on agro-ecosystems approaches • Required back-up support provided for the uptake and adoption by farmers and herders and communities of improved

land use and management practices on-farm and on common property lands • Locally adapted training and technical support for community adoption of diversified land use systems, and improved

management practices and participatory monitoring • Community-level inventory and rapid assessment conducted by all target communities on status of and threats to

agricultural biodiversity • Land users, farmer groups and communities across the target micro-catchments, adopting and generating benefits

Output 4.3: Market opportunities and other cost-benefit sharing mechanisms for the provision of environmental services identified, demonstrated and promoted among land users

• Mechanisms identified and supported for reduced risks, improved farmer income/benefits and reduced costs (labor, energy) and equitable sharing of costs and benefits (monetary and non-monetary) of sustainable agro ecosystem management

• Review and testing of possible incentive measures • Promotion of improved farmer/community organization, empowerment and business management for agro-ecosystem

management • Review conducted of constraints to adoption of diversified systems and problems and needs identified for added value

and improved marketing of local agro-environmental products


105




Output 5.1: Project management, institutional and administrative structures in place and linked to national and regional decision making structures

582,800

• Project management structures established and functioning effectively • Project staff recruited and managing activities at regional and national levels • Adequate office premises and equipment and support services provided • Project coordination mechanisms established and functioning • Resource mobilization strategy and funding plan developed, regularly updated and shared with partners

Output 5.2: Project monitoring and evaluation system and reporting supporting project management and execution

• Regular monitoring and reporting • Conduct of a project management and performance review


106

Annex 8 Co-financing Table

CO-FINANCING

Co financing FAO Financing Government Partner Agencies Total

(Type/Source) (mill US$) (mill US$) (mill US$) (mill US$)

Planned Actual Planned Actual Planned Actual Planned Actual

Grant

Loans / Concessions

* In-kind Support 0.351 0.243 18.724 2.712 5.434 0.496 24.509 3.451

* Other

TOTAL 0.351 0.243 18.724 2.712 5.434 0.496 24.509 3.451

(*) Source: Project document, PIR 2011-2012 and notes from the Project Team.


107

Annex 9 Full List of Performance Indicators



1. Improved land use systems/ management practices for the range of agro-ecological zones in the basin being tested and adapted (by end PY3) for arable and pastoral systems including measures to reduce pressures on wetlands, riverbanks, forests, protected areas.

2. Transformation of 43,700 ha. of land by PY3 and 100,000 ha. by PY5 towards productive and sustainable agricultural ecosystems

3. Potentially 6 percent of today’s basin population (some 1 million people) aware of project activities in target communities, micro-catchments, agro-ecological units through demonstrations and outreach.


4. Transboundary agro-ecosystem management programme (TAMP) to reverse land degradation being implemented and monitored by the 4 riverine countries in 21 districts, reviewed by national and regional PSCs, and project activities & achievements widely shared and available (PY5).

5. Best practices for addressing TB land-related constraints through integrated ecosystems and inter-sectoral approaches mainstreamed in planning and development processes, including. NAPs, and pilot actions implemented to address TB issues in 68 communities (PY3) and replicated in 21 districts (PY5).

6. Regular Government budget allocations to transboundary coordination & collaboration in the Kagera basin increased by 10 percent (PY5)

Output 1.1: A basin-wide coordination mechanism is established to facilitate transboundary dialogue, basin-level policy harmonization and coordination of national/sub-national actions

7. Sustainable coordination mechanism for SLaM agreed upon among the 4 countries (eventually as part of wider NBI and EAC mechanisms) and reflected in a memorandum of understanding.

8. Recommendations to harmonize policies, laws and regulations and address transboundary issues in the river basin developed by an ad-hoc basin-wide task force with stakeholders (PY3) and mechanisms in place for their implementation in 21 districts (by PY5).

9. Transboundary SLM action plans in development/ in place with budget allocations and institutional support.

Output 1.2: An efficient basin-wide knowledge management system is established to support information requirements and decision-making processes at all levels

10. TAMP knowledge management system established and functioning at all levels (PY2) including:

a. Kagera environmental monitoring and information system (SLaM-IS) supported by a GIS and RS tools (PY1-5).

b. Pilot district level GISs developed and operational - 1/country (by PY3).

c. Community information centers set up and servicing stakeholders in target communities (PY2).

Output 1.3: Project monitoring and evaluation system and technical reporting supporting Kagera TAMP implementation and decision making

11. M & E system established and functioning 12. Project management and district partners trained in data collection

and participatory M&E (by end PY 1)


108


in the basin


13. Priority policy, legal and Transboundary issues identified and agreed at community (68), district (21) and river basin levels for SLaM (end PY2) and resulting in supporting policy decisions, regulatory mechanisms and community bye-laws for improved harmonization and application (PY5).

14. At least 2 policy recommendations per country developed that support national policy-decisions and regulatory mechanisms, and 1 per country that support by-laws, etc. at district/ community level.

Output 2.1: Sustainable management of land and agro-ecosystems (SLAM) mainstreamed in national and district development programmes and basin institutions, enhancing synergy among sector strategies and across the river basin

15. SLaM considerations/actions integrated in annual district development plans and budgets (21),

16. SLM practices/ approaches mainstreamed into river basin and national agriculture and NR sector action plans (e.g. biennial) and a set of results based indicators used to monitor how they contribute to NAPs (4) and NBSAPs (4) (by PY4-5).

17. Successful and diverse experiences of inter-sectoral processes and systems approaches for SLaM documented annually in 21 districts and the river basin reports and case studies/findings made available for decision making by PSC members (PY4-5)

Output 2.2: Regulatory actions and conflict resolution mechanisms developed and used to promote - or remove existing barriers to - sustainable land and agro-ecosystem management

18. Locally adapted bye-laws developed and agreed at community level (24 cases/ country) (PY3) and implemented (PY5)

19. Best practices for effective policy and legal application/enforcement disseminated in the basin (PY 2-5).

Output 2.3: A coherent strategic land use planning framework in place (from river basin to district/provincial and community levels) (based on thematic reviews, stakeholder consultations and priority setting) to support SLM efforts by rural communities

20. National and local government staff trained in land use planning (at least 42 district level; 64 community level) (PY1-5)

21. Land use policy being effectively applied/ enforced in 68 communities by PY5.

22. Participatory strategies and action plans developed for SLaM in 21 districts across the basin (PY1-3)

a. improved pasture and rangelands management (at least 15 areas; 7,500ha)

b. transboundary livestock movements (5 borders) c. conservation and sustainable use of wetlands (at least 9 areas;

6,000 ha), d. conservation and sustainable use of agro-biodiversity (68

communities) e. sustained energy supply (68 communities)


23. Trained technical staff and policy makers in 21 districts - supporting SLaM planning and implementation and using project information resources in their district and communities (PY5)

24. Community members/local decision makers sensitized on SLaM techniques for pastoral, arable, mixed systems and their on- and offsite impacts and benefits (PYs 1-5)

25. FFS members trained and adopting SLM and promoting upscaling on community territory

26. Training materials on best practices /approaches widely available and SLM demonstrations in place.


109


Output 3.1: Methods and approaches to promote the adoption of SLM practices and agro-ecosystems (pastoral and cropping) developed and validated through demonstrations and study plots and participatory learning and adaptive management processes

27. Demonstration sites (68) and FFS study plots (136) identified and agreed upon (end PY1), established (end PY2) and FFS study plots scaled-up x 3 (PY4-5)

28. Training materials developed and used in training in 21 districts 29. Advocacy and training materials disseminated and used in 21

districts and 68 communities (PY3), available from community information centers and districts as and when required in the basin (PY 5)

Output 3.2: Enhanced quality of services provided to rural communities in the basin for improved agro-ecosystems management, through training by inter-sectoral teams, participatory research and monitoring and building on local knowledge and innovations

30. FFS facilitators/extensionists (150); district staff (4 x 21), community leaders (150) and partner NGO staff (42) trained in PLAR /FFS approaches (PY 2+) and best practices for SLaM.

31. Target communities (68) benefiting from improved access to service providers competent in SLaM (planning; intersectoral/ systems approaches) and SLM support

32. 300 technical staff and 200-250 policy makers (15/districts) trained to support SLaM planning and implementation and using project information resources in their district and communities (PY5)

33. 120,000 community members/local decision makers sensitized on SLaM techniques for pastoral, arable, mixed systems and their on and off-site impacts and benefits (PYs 1-5)


34. SLM practices implemented by pilot communities (68 by PY3; 200 by PY5) in demonstrations and farmer plots covering a total of 45,000 ha of land (by PY5) and showing:

f. Effective control of soil erosion (no new visual signs) in all target sites;

g. 4 target micro-catchments (PY5 ) identified and sediment loads monitored(subject to identifying sites where SLM interventions can be applied on a significant area of the catchment and hydrological monitoring can be supported by partner Kagera IWRM, NBI-NELSAP and LVEMP projects);

h. 30 percent increase in vegetation cover (above and below ground biomass) on pilot 23,000 ha arable and 7,500 ha pasture lands where alternatives to slash and burn are applied (PY5)

i. 20 percent increase in soil carbon stores on farmer study plots and sample arable and pasture lands (PY5) inferred on 30,500 ha of land where SLM is practiced/planned.

j. 10 percent increase in production (crop; livestock; other goods) by trained farmers/ herders contributing to livelihoods (income; food security; reduced vulnerability)

Output 4.1: Participatory land management plans are developed and implemented in targeted communities, micro-catchments and wider land units

35. 100 participatory land use plans and action plans developed (PY2) and being implemented (PY2-4) and replicated x 2 (PY5)

a. community action plans (68) b. micro-catchments (46); c. pasture/ range areas (15); d. target wetlands (10); e. riverbanks (1000km)

36. Capacity built for implementation and monitoring of community action plans (PY1-5) in 136 communities.

Output 4.2: Improved land use and agro-ecosystem management practices are successfully adopted by farmers and herders in targeted communities and replicated in other areas.

37. 136 communities implementing SLaM (PY5) 38. Wide adoption of improved agricultural systems, management

practices including biodiversity conservation by members of 72 farmer/herder groups (PY3) and replicated x 3 (PY5)

39. 1,800 farmers trained and adopting /upscaling SLM through FFS approaches (PY3) and a further 1,800 farmers by PY5

40. Local-level indicators of benefits of SLaM (income, household food


110


security, reduced risk) confirmed by all target farmer groups and a sample 10 percent of the target population (100,000 persons) (by PY5)

Output 4.3: Market opportunities and other cost-benefit sharing mechanisms for the provision of environmental services identified, demonstrated and promoted among land users.

41. Incentive and benefit sharing mechanisms (monetary; non-monetary) identified and supporting adoption of SLaM and biodiversity conservation, including payments for environmental services (PES), products added-value and marketing in 34 communities (PY 1-5)

42. Incentive/ support mechanisms reaching vulnerable groups (tenant farmers, youth, HIV/AIDS widows/orphans; female headed households) 15 percent of target population (PY5)


43. Execution of project activities and delivery of outputs in accordance with workplan and budget

44. Regional PSC and TAC operational 45. Backstopping by FAO and by Government institutions

Output 5.1: Project management, institutional and administrative structures in place and linked to national and regional decision making structures.

46. Regional project coordinator and national project managers recruited and execution of activities and delivery of outputs in accordance with workplan and budget

47. Regional PSC and TAC operational and providing guidance and decision making

48. Backstopping provided by FAO Lead technical unit and project task force and by Government institutions

49. Constructive recommendations by mid term evaluation to address key problems identified

Output 5.2: Project monitoring and evaluation system and reporting supporting project management and execution.