Evaluation of the Bank’s Country Strategy and Program in ...idev.afdb.org/sites/default/files/documents/files...AFDB’S STRATEGIC RESPONSE FOR MALAWI (2005-2016) 5 4.1 Overall Strategy

Evaluation of the Bank’s Country Strategy and Program in the Republic of Malawi (2005-2016)

Summary Report September 2018

ii

ACKNOWLEDGEMENTS

Task Manager Khaled Ibn Waleed Samir Hussein

Task Team Members Mirianaud Oswald Agbadome

Bilal Bagayoko

Consultants Lattanzio Advisory

External Peer Reviewer James C. McDAVID

Knowledge Management Officer

Jayne Musumba

Special Thanks All Bank staff and Country Team members in particular, for their contribution and good cooperation during the evaluation phases. Staff of development partners, representatives and external stakeholders, clients and partners and private sector clients for their time for interviews, to arrange site visits, and to comment on evaluation findings.

Division Manager Madhusoodhanan Mampuzhasseril (OIC)

Evaluator General Karen Rot-Munstermann (Acting)

iii

CONTENTS

ACKNOWLEDGEMENTS ii

ACRONYMS AND ABBREVIATIONS iv

INTRODUCTION 2

THE DEVELOPMENT ASSISTANCE CONTEXT OF MALAWI 2

MALAWI GOVERNMENT DEVELOPMENT STRATEGIES 4

AFDB’S STRATEGIC RESPONSE FOR MALAWI (2005-2016) 5

4.1 Overall Strategy 5

4.2 AfDB Portfolio 7

EVALUATION FINDINGS 10

5.1 Relevance 10

5.2 Effectiveness 13

5.3 Sustainability 14

5.4 Crosscutting issues 18

5.5 Efficiency 19

5.6 Knowledge and Policy Advice 20

5.7 Partnerships, Harmonization and Leverage 23

5.8 Managing for Results 25

5.9 Lessons Learned and Knowledge Management 27

LESSONS LEARNED AND RECOMMENDATIONS 29

6.1 Lessons Learned 29

6.2 Recommendations 29

Annex A: Evaluation Methodology 31

Annex B: Evaluation Matrix 34

Annex C: Project List 41

Annex D: Sector Level and Overall Performance 44

Annex E: Malawi – Comparative Socioeconomic Indicators 47

Bibliography 48

iv

ACRONYMS AND ABBREVIATIONS

ADF African Development Fund AfDB African Development Bank AISP Agricultural Infrastructure Services Project CABS Common Approach to Budget Support CARLA Climate Adaptation for Rural Livelihoods and Agriculture CSF Community Storage Facilities CSP Country Strategy Paper

DCAFS Donor Committee on Agriculture and Food Security DfID Department for International Development (UK) DPs Development Partners ESIA Environmental and Social Impact Assessment ESMP Environmental and Social Management Plan ESW Economic and Social Work EU European Union EQ Evaluation Question GBS General Budget Support GDP Gross Domestic Product GoM Government of Malawi GPRSG Governance and Poverty Reduction Support Grant ICSP Interim Country Strategy Paper IDEV Independent Development Evaluation function of the AfDB JC Judgment Criteria JIU Joint Infrastructure Unit KM Knowledge Management LED Local Economic Development M & E Monitoring and Evaluation MCs Market Centers MDGs Millennium Development Goals MGDS Malawi Growth and Development Strategy MTR Mid-Term Review NIP National Irrigation Policy NWDP National Water Development Project OECD Organization for Economic Cooperation and Development OSBP one-stop-border-posts PBO Program Based Operation or Policy Based Operation PBS Protection of Basic Services PCR Project Completion Report PD Policy Dialogue PFEMRP Public Financial and Economic Management Reform Program PFM Public Finance Management PIU Project Implementation Unit PSD Private Sector Development RBM Results Based Management REC Regional Economic Communities RFSSP Restoration of Fiscal Stability and Social Protection RISP Regional Integration Strategy Paper RWSSI Rural Water and Sanitation Supply Initiative SCPMP/SCMP Smallholder Crop Production and Marketing Project SIVAP Smallholder Irrigation and Value Addition Project SVIP Shire Valley Irrigation Project SWAp Sector-wide Approach to Programing UK United Kingdom WATSAN Water and Sanitation WB World Bank

EXECUTIVE SUMMARY

Introduction

This report summarizes the findings, lessons and recommendations of the independent evaluation of the African Development Bank’s (AfDB or, the Bank) Country Strategy and Program in the Republic of Malawi during 2005-2016. The main objective of the evaluation is to inform the next Country Strategy Paper (CSP) and to contribute to both accountability and learning in the Bank as well as in countries where the Bank’s program is similar to that of Malawi. The evaluation exercise focused on both strategic and individual interventions in all sectors (Social, Transport, Water and Sanitation or WATSAN, Agriculture, Multisector – Financial Governance, and Private Sector Development) where the Bank was active during the review period.

Evaluation Findings

Relevance

Relevance was evaluated by assessing the extent to which the Bank’s country strategies and operations were aligned to Malawi’s development needs, strategies and priorities as well as the needs of ultimate beneficiaries. The relevance of Bank strategy and assistance to Malawi was Satisfactory.

Bank strategies are aligned with the priorities of the Government of Malawi (GoM) and continued to adopt to changes in national development plans as they evolved over time. The Bank’s interventions at sector level are directly related to one or more of the Government’s strategic areas of focus.

However, it should be noted that the CSP has not clearly detailed specific beneficiaries even though the project documents contain references to beneficiaries, and the Bank’s projects are aligned with their needs.

Effectiveness

Effectiveness was evaluated by assessing the extent to which the Bank’s interventions in Malawi achieved their expected development results (outputs and outcomes) against targets

set in project approval documents. Additionally, effectiveness was evaluated also by ascertaining the extent to which the Bank interventions benefited targeted beneficiaries. This performance dimension was Satisfactory.

Project Completion Reports (PCR) that were available for the portfolio under review reflect a satisfactory level of performance in terms of achievement in both outputs and outcomes. However, it should be noted that the PCRs take into consideration a post-Midterm Review (MTR) definition of expected results, which according to the majority of PCRs, had considerably reduced expected output and outcome targets.

In the agriculture sector, on the two projects that were closed during the review period, expected effects were clearly stated and monitored, and achievement levels were clearly stated. In the transport sector, two projects which were evaluated in-depth had delivered expected outcomes and physical outputs were well achieved in the sector.

In the transport sector, project progress reports and two PCRs were scrutinized. All Bank supported interventions for provision (or rehabilitation/upgrading) of road transport infrastructure delivered expected outputs (in some cases at additional time and cost; and in some cases at lower than estimated cost). Overall, the physical outputs of the interventions were well achieved in the sector.

For the WATSAN sector, the sector targets were set out in general terms in the CSPs and responded to Malawi Gov’t Development Strategy (MGDS) targets. All sector interventions specify expected results (outputs, outcomes and impacts) in response to higher-level strategies. Monitoring indicators are defined and logical linkage formulated, although the translation of outputs into higher level outcomes and impacts was not as clearly outlined.

Sustainability Sustainability was evaluated by assessing the extent to which the benefits accrued and gained from Bank interventions continued to exist or were likely to continue to exist after project

vi

completion. This performance dimension was Unsatisfactory.

None of the completed projects under review have reached a post-closure period of 5 years, which makes assessment of actual sustainability challenging. The evidence collected from the field shows that some benefits gained from past interventions (in particular the soft components such as training and community-led management) were already at risk of not being sustained in the immediate years after completion. Benefits from past interventions, most notably infrastructure, have continued to exist (for longer than 5 years), but longer-term sustainability (i.e. for the expected project life) could not be ascertained.

Better sustainability practices were found in the agriculture and transport sector, compared to the social sector. At the same time, sustainability was not guaranteed in any of these three sectors. In the agriculture sector, e.g. the Smallholder Crop Production and Marketing Project (SCPMP) was rated unsatisfactory on three parameters: i) financial sustainability; ii) ownership and sustainability of partnerships; and iii) environmental and social sustainability. Only institutional sustainability and strengthening of capacities was rated satisfactory, with a score of three. In contrast, the Climate Adaptation for Rural Livelihoods and Agriculture (CARLA) project was rated satisfactory on all similar parameters, with a highly satisfactory rating of 4 for environmental and social sustainability. In the social sector, the evaluation concludes that the Local Economic Development (LED), Competitiveness, and Job Creation may not be sustainable under the present aid architecture (i.e. unless considerable investment continues to flow to those strategies over a long period).

Crosscutting Issues

Crosscutting issues are strongly considered at the strategic level by the Bank and were evaluated by assessing the extent to which the Bank’s objectives on inclusiveness across gender, regions, and environmental sustainability been mainstreamed into Bank interventions. This performance dimension was Satisfactory.

1 The same issues are included in the Bank’s results framework.

In both the Malawi CSP and the Southern Africa Regional Integration Strategy Paper (RISP); gender issues were addressed (including disparities in access to services, decision-making structures, economic opportunities, cultural biases, budgetary allocation and monitoring of gender actions).1

The Bank carried out a country Gender Profile in 2005, which highlighted the main constraints to women empowerment. Therefore, project documents, mainly appraisal reports, have a section dedicated to gender issues, which are expected to be directly addressed by the project. Bank projects in Malawi during the period under review had a gender informed design, which specifically incorporated gender related elements.

For the social sector (education and health), it is noted that specific targets for gender are included for every intervention. In particular, for the education sector, the analysis shows that women tend to be less represented in the secondary cycle with negative consequences on employment. As for the transport sector, although Bank strategies do not identify transport sector-specific gender issues, there is little doubt that the transport projects would support women, men, children and the elderly equally via improved social and economic mobility and improved access to markets and health centers.

Investments in the WATSAN sector usually generate great inherent impacts on women and girls at multiple levels – access to adequate quantities of potable water within a reasonable distance, water carrying (which is predominantly a task of women and girls), sanitation and hygiene with direct impacts on family health, irrigation for agriculture (an activity with high female participation).

There is evidence that women participated in the agriculture sector projects. For instance, the SCPMP completion report notes that the project was designed to promote gender mainstreaming and specifically targeted female-headed households to improve their livelihoods through membership in Farmers’ Organizations.

vii

The Bank programs are expected to cover all regions, but there is limited rollout to the remote and poorest districts either by the GoM or other donors. All interventions cover more than one district and are implemented in rural areas. However, inclusive coverage of regions where poverty is most prevalent were the most problematic (as is quantification of Bank sector support contribution to poverty alleviation). Unless the poorest and most vulnerable are explicitly targeted, they are unlikely to benefit from the Bank support for big infrastructure (road corridors) or health and education projects.

In accordance with the 1996 Environmental Management Act, all sector projects in Malawi should undertake an Environmental Impact Assessment. Therefore, the Bank has made efforts to mainstream environmental safeguards into project preparation and implementation. Although in most cases good engineering practices should minimize adverse environmental and social impacts, there is only limited capacity in applying sound environmental and social mitigation measures.

At the CSP level, environmental and social issues in Malawi are concerned with describing the current situation rather than formulating explicit strategies to address these issues. However, it is clearly stated that environmental protection and mitigation of adverse social impacts are a component of Bank support. RISP goes further noting that the Bank will take leadership in promoting climate friendly infrastructure and reduced carbon emissions, 2 making specific reference to Malawi’s (and some other southern African countries) problems regarding the effects of increasing climatic variability on infrastructure, food security and development.

Environmental and Social Impact Assessments (ESIAs) and Environmental and Social Managements Plans (ESMPs) are produced for all sector interventions, although, a lack of reporting on the implementation of ESMPs is

2 No targets have been set for reduced or avoided emissions or reduction in carbon footprint (at project of program levels) except in general terms (e.g. by reference to MDG 7: CO2 emissions). 3 It should be noted that some social mitigation measures (e.g. compensation for land, houses, crops) might be

noted. These ESIAs and ESMPs are prepared in compliance with national legislation and AfDB environmental and social policies (which, in turn, are fully compliant with international best practices). All AfDB projects are categorized at an early stage in preparation and this categorization defines the expected nature of environmental (and social) impact and the mandatory mitigation treatment.3 Climate change is mentioned in such studies, but no proactive resilience measures are noted in the projects scrutinized as well there are no targets for carbon footprint reduction.

Efficiency

Efficiency was evaluated by assessing the extent to which the resources and inputs provided were converted to results at a rate that reflects the assumptions at appraisal. Timeliness in implementation was an integral part of this assessment. This performance dimension was Unsatisfactory.

The Bank has generally been responsive and flexible in ensuring satisfactory project performance by allowing a reallocation of resources across the Public Finance Management (PFM) of the project’s main components, and using unused resources to fund follow-up activities within the ‘multisector.’ Completed agriculture projects (SCPMP, CARLA) had PCR results that gave a resource use efficiency rating of three, which is satisfactory.

The Bank’s interventions are associated with delays largely due to noncompliance with the Bank’s operational standards (although slow Bank response to Government requests for ‘no objection’ has also been noted). Added to this are the country capacity constraints, causing implementation delays.

For non-budget support projects, disbursement delays were caused mainly by the administrative procedures of GoM - this is a general problem for almost all Bank operations in Malawi. The non-Bank managers find the Bank’s finance-related processes cumbersome. Finally, it may be noted

frustrated due to late disbursement of funds for Government (payment of such costs is responsibility of Government and a component of GoM commitment to the project).

viii

that the disbursements predicted for private sector development through commercial banks did not materialize.

Efforts were made by the Bank to use national systems, without much success. For a country at Malawi’s level of development, difficulties in the use of national systems are predictable. Project Implementation Units (PIU) are very common and the Bank’s systems were used such that there were no considerable delays in first disbursements, which took place largely as planned.

Knowledge and Policy Advice

This performance dimension was evaluated by assessing the extent to which the Bank was engaged in and influenced Policy Dialogue (PD) through provision of relevant advices which were then incorporated into policy decisions. The appropriateness and adequacy of the analytical work in support of its interventions, positioning and policy advice were assessed. Knowledge and policy advice dimension was Satisfactory.

The Bank engaged regularly in dialogue with the government. However, such dialogues had no clear predefined objectives or targets. It was also mostly collective in nature and executed through various donor groups. Interviews with donors stressed the fact that the PD undertaken by the Bank was very useful and was always done in a spirit of collaboration, cooperation and support. Examples in General Budget Support (GBS), Health, education, transport and agriculture were noted, with many expressing a desire for the Bank to take a leadership role, given its ‘preferred relationship’ with the GoM, which considers the AfDB as ‘its Bank.’

The Bank was consulted during the preparation of all Government MGDSs and the Government took on board some of the advices given by the Bank. The GoM’s decision to change its policy on farm subsidies and a subsequent move to a more private sector model was largely the result of the Bank’s PD with the Ministries responsible for agriculture, trade, irrigation and related sectors. Other examples include health policy on maternal care and content of MGDSs. The Bank also fully participated in the Common Approach

4 In the area of PFM, the Bank and WB went as far as conducting a joint appraisal mission in 2013 to define their

to Budget Support (CABS) process and structure. However, it did not significantly influence policy change. There is no major example to show that CABS was influential in getting any difficult reforms off the ground.

Partnership, Harmonization and Leverage

Partnership, Harmonization and Leverage were assessed by evaluating the extent to which the Bank’s interventions and processes were harmonized with those of other donors to avoid duplication, and the extent to which the Bank’s interventions and resources were influencing other stakeholders to become involved in the sectors that were the focus of country strategies. This performance dimension was Satisfactory.

The Bank and other donors have long followed a coordinated approach to their development assistance in Malawi. In key sectors, sector-wide approaches have allowed government and donors to work towards shared strategic objectives. While donors have continued to follow their own programing cycle, 4 this aid architecture has been conducive to a relatively high level of shared (rather than joint) strategic planning. In most sectors, donors broadly share the same analysis and objectives about key reforms.

The Bank has been an active participant in several donor platforms such as the Donor Committee on Agriculture and Food Security (DCAFS) for agriculture, and the Public Financial and Economic Management Reform Program (PFEMRP) for PFM. While some de facto form on the division of labor among donors has been in place, donor ambitions to move towards joint undertaking, managed jointly, have only been met partly. There are limited examples of co-funded projects, such as the Shire Valley Irrigation Project (SVIP), which is implemented together with the World Bank (WB), and the National Water Development Project (NWDP) in the WATSAN sector, which is co-financed by AusAid. In the transport sector, the Joint Infrastructure Unit (JIU) was discontinued in 2012 after only two years of operations. The JIU was a local initiative by AfDB in Malawi for closer cooperation between AfDB and WB for joint

respective contribution to strengthening PFM institutions. This example, however, remains isolated.

ix

management of the infrastructure support portfolios (WATSAN and transport) of the two Banks.

Budget support has limited shared undertaking with the multitude of projects in some sectors, like PFM - where many donors intervene, and has continued to weigh heavily on the government as donors stick rigidly to their own procedures and there is no common reporting format. Even in the case of CABS, the Bank and other donors occasionally came up with their own specific requirements.

Managing for Results

This performance dimension evaluated the extent to which the Bank was successful in focusing on results in its management of day-to-day operations, and the extent of its support to national systems that focus on results. Managing for results was Unsatisfactory. The Bank’s strategy and project documents contain results that are well stated in near- Results Based Management (RBM) terms or expected effects, but there are no intermediate results, a chain of execution or implementation in the key documentation. Bank monitoring focus on outputs rather than outcomes. Analysis of transport and WATSAN sectors shows that CSPs and individual projects have a Logical Framework Matrix which specifies results indicators at activity, output, outcome and impact levels, but not a detailed project path analysis or Theory of Change logic including assumptions and risks. In the agriculture sector, the Bank has often successfully implemented management systems that focus on process-based results and facilitate learning from experiences. The designs of both the SCPMP and CARLA projects include a Result-Based Logical Framework, which demonstrates the linkage between inputs, outputs, outcomes and impacts, and risk mitigation measures and key assumptions.

5 The Evaluation considers only documented evidence as being “identified.” 6 Iinterpreted as being documented to show that there has been an influence.

In terms of GBS and PFM, recent changes to the Program Based Operation (PBO) guidelines are designed to improve the extent to which these types of interventions are results-based. PBOs had causal chains that were poorly defined and, confused different levels of results (outputs and outcomes) with the corresponding indicators. The experiences from the transport and WATSAN sectors show that the Bank has broadly supported development of national capacities and management systems, but that focus has been largely on process and program/project implementation rather than outcomes, or overall capability to design and implement regulatory/ policy frameworks.

Knowledge Management

Knowledge Management was evaluated by assessing the extent to which the Bank learned from its experience. This was done by examining whether the factors affecting performance and results of the Bank interventions were identified5 and informed 6 its actions and strategies. Moreover, the issue of incorporating lessons from experience to inform future strategies and programs is also looked at. This performance dimension was Satisfactory.

The Bank has mechanisms in place to picture and record positive and negative factors affecting project performance. Lessons from experience are usually recorded in CSPs, MTRs, PAR7s, PCRs and BTORs. However, there is no knowledge management system in place that makes storing, sorting, searching and retrieving such lessons possible and accessible via data mining or thematic searches. The utilization of the lessons included in the documents mentioned above is dependent on the availability of such documents or lessons. Notwithstanding, the Task Managers appear to have strong knowledge and experience of available knowledge in their respective fields. Cross-sectoral and generic lessons (pertinent to Bank-wide processes such as procurement and quality-at-entry), are repeated in consecutive reports though the underlying issues persist.

7 Public Administration Reform

x

Lessons Learned

The following lessons are identified from the Bank’s program in Malawi:

1. Clear definition of beneficiaries. When the intervention beneficiaries are described in overly generic terms, they tend to be seen as a numerical target to be reached, and therefore the supervision and management of the projects tend to become input- and process focused, rather than outcome focused. Beneficiaries need to be as astutely described as possible so that the effects of the targeted ‘benefits’ can be monitored. Specific baselines are to be established upfront and constantly updated with valid information.

2. Results-based monitoring system. There is

no benefit accrued by the Bank in generating results chains that are only partially managed. If outcomes are the core objectives of investments, then monitoring and supervision of processes and indicators must be undertaken at all levels, including outcomes. With commitment to RBM, it is necessary to have a monitoring system that generates decisions focused on outcomes, and a solid analysis of results chains before project approval.

3. Facilitative conditions for policy dialogue.

The Bank is well regarded in terms of its non-lending activities and produced results. The policy and program dialogue that has taken place over the years has also been well

appreciated as a driver of change. Given that the GoM requires a considerable amount of support in order to accelerate its development, an important lesson is that the Bank has been able to influence and advantage policy when it has put in place the conditions that facilitated and supported dialogue. Increasing the quality and strategic nature of the dialogue will likely improve the level of influence. These conditions include developing a knowledge trust, promoting evidence-based decision making, developing the capability of institutions and agencies to design change strategies and monitor progress.

4. Timely decisions on design change. Major

changes in project design, if required, should be introduced by taking into account not only achievable outputs but also continued project viability and relevance; this should not be postponed to the MTR if identified early during implementation.

5. Fiduciary safeguards and budget support. In

a highly aid-dependent country, periodic withdrawal of budget support constitutes an external shock that aggravates macroeconomic instability and can cause unintended damage to growth and to poverty alleviation efforts. It is essential to strike a balance between promoting country ownership and using additional fiduciary safeguards that can prevent the sudden withdrawal of budget support.

Recommendations

Based on the foregoing findings and lessons learned, the evaluation proposes the following recommendations:

1. Enhance capacity of the Bank’s officers and managers in Results-based Logical Framework and the use of Results-based Management tools, including theory-based design, results-based contracting, outcome-focused supervision and management of interventions. Each intervention should have clear identification of its target beneficiary characteristics and design the subsequent

xi

content and required management systems. This includes a baseline for monitoring, the focus of the M&E8 system, reference points for the RBM system, indicators for outcome measurement, contexts for project implementation, basis for the Theory of Change analysis, and basis for assumptions and risks.

2. Enhance the proficiency of the Bank staff to effectively operate in the business ecosystems in which key sector actors operate, and design interventions that are holistic enough to stimulate the private sector’s investment and actions. This is essential since most GoM development strategies rely on the operation of its private sector. As most of the business ecosystems are international in scope, the Bank actions in Malawi need to be international to some extent as well, with country programs developing ‘regional at least’ markets in a number of countries.

3. Since the Bank and Development Partners (DP) are pressing for effective and efficient PFM

systems, the Bank should establish a clear timeline within which all required performance standards are met and proven. The GoM has been implementing PFM reforms for over 20 years, and therefore the Bank should identify what it should do if the systems cease to function during a specific period. The Bank could consider generating contingency supports to overcome critical weaknesses without compromising transparency and mutual accountability and achieve development effectiveness.

4. The Bank should redesign its Knowledge Management (KM) practices to address the need for access to information and develop explicit and implicit knowledge in its Operations Divisions to improve decision making capabilities of the Bank and its clients. The Bank should build effective learning organizations by making learning routine as well as stimulating cultural change and innovation. It is therefore important that the Bank invest in modern KM technologies, develop its capacity and capability for data mining and use KM as an integral part of any implementation strategy.

5. The Bank should ensure sustainability of its interventions by taking concerted actions

throughout the project cycle to ensure that impediments to sustainability are addressed and managed. From design through implementation, critical assumptions should be analyzed, and steps taken to mitigate risks. In situations where sustainability cannot be reasonably ensured, it is advisable not to approve such interventions in the future and ongoing interventions should be adjusted so that sustainability can be assured.

6. The Bank should take adequate measures to ensure that its interventions and strategies provide

equality and equity between genders and that the interventions provide the means to manage, track and report on these objectives. Gender as a crosscutting issue should be included in project designs and mainstreamed into implementation. Beyond simply including women as a fraction of the participants in activities, they should be part of the management, and their needs - as being different from that of men - fully acknowledged. Project designs need to be clearly structured so that results are defined in terms of outcomes that bear on women and girls.

8 Monitoring and Evaluation

2

INTRODUCTION

This report summarizes the findings, lessons and recommendations of the independent evaluation of the African Development Bank’s (AfDB or Bank) Country Strategy and Program in the Republic of Malawi during 2005-2016. The main objective of the evaluation is to inform the next Country Strategy Paper (CSP) and to contribute to both accountability and learning in the Bank and in countries where the Bank’s program is similar to that in Malawi.

The evaluation focused on both the strategic and individual interventions levels in key sectors (Social, Transport, WATSAN, Agriculture, and Multisector–Financial Governance and Private Sector Development) where the Bank is active. In addition, the evaluation looked at the achievement of development results by answering evaluation questions based on four of the five evaluation criteria suggested by the Development Assistance Committee of the Organization for Economic Cooperation and Development (OECD-DAC), those of relevance, effectiveness, efficiency, sustainability and impact, with impact not a required criterion for this evaluation.

Additional AfDB-specific criteria (viz. the management of crosscutting issues, knowledge management and policy improvement, partnerships and harmonization, management by results and lessons learned) were added in order to cover the design and management-related elements and show the benefit and the strategic positioning of the Bank in Malawi.

The evaluation methodology is based on the use of an evaluation matrix detailing the evaluation questions, judgment criteria (JC) and indicators as the building blocks for the analysis leading to conclusions and recommendations. The methodology focusses on results achieved and demonstrated, a Theory of Change approach to intervention design and management, and a sector-by-sector analysis followed by a roll-up of sector level findings to the program level, evidence-based research, contribution analysis and iterative feedback with key stakeholders to strengthen the validation bonds. The evaluation process adopted a systematic approach that uses different building blocks to answer the evaluation questions and to formulate key conclusions and recommendations. The methodology applied for this evaluation is based on the evaluation criteria developed by OECD-DAC. The detailed evaluation methodology and limitations are presented in Annex A. This report is laid out in six main sections: i) Malawi Government development strategies 2005-2016; ii) the development assistance context of Malawi; iii) the Bank’s strategic support to Malawi during 2005-2016; iv) evaluation findings; and vi) lessons learned with recommendations.

THE DEVELOPMENT ASSISTANCE CONTEXT OF MALAWI

The Republic of Malawi is a landlocked country in South-East Africa, sharing borders with Tanzania, Zambia and Mozambique. Divided into 28 districts and three regions (north, central and south), Malawi has a total surface area of 118,484 sq. km and an estimated population of 16.4 million in 2015,9 with 67.0% of the population being under 24 years of age, only 5.7% over 55 years and a population growth rate of 2.8%.10 The capital is Lilongwe (population 905,00011) and the other major urban areas are Blantyre, Mzuzu and Zomba. Although approximately 80% of the population lives in rural areas, the annual urban growth rate is about 3.8%, thus making urbanization a real challenge.

9 African Development Bank, https://www.afdb.org/en/countries/southern-africa/malawi/, accessed January 18th 2017 10 Ibid. 11 CIA World Factbook https://www.cia.gov/library/publications/the-world-factbook/geos/mi.html, accessed January 20th 2017

3

The tables below provide an overview of the Official Development Assistance (ODA) that the country has received during the 2007-2015 period and the sectors of destination for that support. It may be noted that half of the ODA was directed at education or health.

Table 1: Destination of Bilateral Aid 2007-1512

Sector Percentage

Health and Population 36%

Other social infrastructure 10%

Economic Infrastructure 9%

Production 17%

Education 13%

Multisector 5%

Program Assistance 3%

Humanitarian Aid 7%

Action relating to Defense Negligible

Other and unallocated 1%

The African Development Fund (ADF) is the seventh largest ODA partner, a long way behind the USA, International Development Association partners, the United Kingdom and the Global Fund, among others.

Table 2: Main donors and their disbursements in US$ over the last three years (2014-2016)

Main Donors Actual Disbursements (US$ million)

World Bank 372.4

Department for International Development 216.0

Global Fund (HIV/AIDS & Malaria) 189.3

African Development Bank 183.5

Norway 171.4

European Union 155.2

Centers for Disease Control and Prevention 90.2

Source: Malawi Aid Management Platform

Table 3: Government of Malawi Receipts 2013-2016

2013 2014 2015

Net ODA (USD million) 1,132.7 931.4 1,049.4

Net ODA/GNI (%) 21.1 15.8 16.5

Gross ODA (USD million) 1,176.1 1,030.6 1,104.0

Bilateral Share (Gross ODA) (%) 56.1 54.4 53.7

Net Private Flows (USD million) 444.6 35.5 0.8

Total Net Receipts 1,577.8 966.0 1,049.3

Source: OECD.org/dac/financing-sustainable –development /development-finance-data/aid-at-a-glance.htm

The population of Malawi stands at 18.1 million in 2016, up from 13 million in 2005 and 11.4 million in 2000. The ratio of net ODA received as a percentage of Gross National Income (GNI) is a key indicator of the level of dependence of the country on external grant financing. The following table provides an overview of ODA/GNI ratio over selected years:

12 Source: OECD.org/dac/financing-sustainable –development /development-finance-data/aid-at-a-glance.htm

4

Table 4: Evolution of net ODA received as a percentage of GNI-Malawi

Year 1960 1980 1990 1994 2000 2005 2008 2012 2015

ODA/GNI Ratio 2.6 12.4 27.2 41.4 26.3 15.9 17.5 19.8 16.9

Source: http://data.worldbank.org/indicator/DT.ODA.ODAT.GN.ZS?locations=MW

MALAWI GOVERNMENT DEVELOPMENT STRATEGIES

In its Vision 2020, the GoM is committed to making the country ‘secure, democratically mature, environmentally sustainable, self-reliant with equal opportunities for and active participation by all, having social services, vibrant cultural and religious values, being a technologically driven middle-income country.’ To tackle the development challenges under the evaluation period, three national policy documents were in place. The documents defined the overarching strategies of the country in order to accelerate the attainment of its long-term development aspirations and the MDGs. The main objectives of the three strategies are presented in the table below.

Table 5: Malawi Government’s strategies and priority areas

Strategy Objectives and Priority Areas

2002-2005 Malawi Poverty Reduction Strategy & Plan (MPRSP)

The objectives of the MPRSP 2002-2005 included:

a united, secure and democratically mature nation with full participation of the population;

access to quality health and education for all;

sustainable growth and development with a per capita income of USD 1,000 by 2020;

a technology-based, manufacturing-led economy generating 25% of Gross Domestic Product (GDP);

a fair and equitable distribution of wealth;

a sustainably managed environment; and

food security for all.

This strategy had a three-pronged approach:

emphasize smallholder agriculture to raise the productivity and income of the rural poor;

promote private sector growth to expand off-farm employment through a number of measures including reduction of transport costs; and

expand social services.

2005-2011 Malawi Growth and Development Strategy I (MGDS I)

Overall MGDS I priorities:

The overriding philosophy of the MGDS I was “poverty reduction through sustainable economic growth and infrastructure development.” The MGDS I identified six key priority areas, which defined the direction the country intends to take in the next five years to achieve economic growth and wealth creation because, it is noted, these are “critical for immediate improvements in the economic wellbeing of Malawians.” These priority areas were:

agriculture and food security;

irrigation and water development;

transport infrastructure development;

energy generation and supply;

integrated rural development; and

prevention and management of nutrition disorders, HIV and AIDS.

5

Strategy Objectives and Priority Areas

2011-2016 MGDS II13

Overall MGDS II priorities

The MGDS II was structured somewhat differently compared to its predecessor with the top level of analysis being the “theme,” and only the second being “priority,” thus providing a more focused approach. The MGDS II identified six thematic areas, namely;

Sustainable Economic Growth;

Social Development;

Nutrition;

Population;

Social Support and Disaster Risk Management;

Infrastructure Development;

Governance; and

Gender and Capacity Development. The MGDS II pointed to nine key priority areas from the themes. Much as what was done in MGDS I, these were considered “necessary to achieve rapid economic growth and improvement in the well-being of Malawians” within the implementation period (quotation from MGDS II) The key priority areas were:

Agriculture and Food Security;

Energy, Industrial Development, Mining, and Tourism;

Transport Infrastructure and Nsanje World Inland Port;

Education, Science and Technology;

Public Health, Sanitation, Malaria and HIV and AIDS Management;

Integrated Rural Development;

Green Belt Irrigation and Water Development;

Child Development, Youth Development and Empowerment; and

Climate Change, Natural Resources and Environmental Management.

AFDB’S STRATEGIC RESPONSE FOR MALAWI (2005-2016)

4.1 Overall Strategy

The AfDB’s cooperation with Malawi dates back to 1969. The Bank’s Malawi Field Office was established in 2007 as a result of the Bank’s decentralization. According to government officials, PIU coordinators and the Bank staff, the establishment of the field office has contributed to the enhancement of the Bank’s portfolio management, country dialogue and aid coordination.

The AfDB country strategies largely corresponded to the two principal MGDSs. The CSP 2005-2009 and ICSP14 2011-2012 corresponded to MGDS I (2005-2011) with a planning construct that began to take the direction of the GoM in its MGDS II into account. The CSP (2013-2017) corresponded to MGDS II (2011-2016).

For the period under review in this evaluation, the Bank’s assistance encompasses three strategic cycles, consisting of two CSPs (2005-2009 and 2013-2017) and an ICSP (2011-2012). In addition, 13 It may be noted that the priorities in MGDS II that are the same as those in MGDS I are italicized. 14 Interim Country Strategy Paper

6

Malawi was included in the RISP for Southern Africa 2011-2015. The CSPs provide a multiannual framework and a guide for the Bank’s interventions in order to support the country’s development agenda and priorities and operate in synergy with them. The RISP aims at providing support to member countries and Regional Economic Communities (RECs) in designing and implementing measures to tackle the regional infrastructure gap and accelerate economic integration.

The 2005-2009 CSP had three core principles: i) support the implementation of the new MGDS to achieve its goals and help eventually lead to the attainment of Malawi’s Vision 2020 goals and MDGs; ii) provide a clear focus on results and outcomes to improve the rate of return on the Bank Group’s interventions; and iii) improve the coordination of Bank Group activities with government and other donors. The Bank Group’s interventions were to continue by providing project- and policy-based support. The AfDB strategy was thus structured around three pillars: 1. Expanding rural infrastructure (increased levels of irrigation infrastructure and power supply to

support agriculture, rural development and Private Sector Development or, PSD); 2. Developing human capital and institutional capacity (enhanced skills through better education,

improved health service delivery and strengthened institutional capacity); and 3. Improved policy and institutional framework for Governance (support of policy and institutional

reforms for good governance).

The preparation of an Interim Country Strategy and Program was agreed with the GoM in order for the Bank to align with the timeframe for the preparation of MGDS II, which was expected in 2012. This allowed the Bank to effectively align its next strategy from 2013 onwards to the country’s new poverty reduction strategy, i.e. MGDS II. The ICSP 2011-2012 focused on two pillars: 1. Improving infrastructure to help remove infrastructure bottlenecks hindering Malawi’s

competitiveness and to open access to regional markets by reducing transport costs and improving trade competitiveness; and

2. Accelerating PSD to help create an enabling environment for business growth and innovation focusing on the ‘missing middle’ in the private sector supporting value chain integration, improving business infrastructure and access to finance.

The CSP 2013-2017 responded to GoM priority focus areas identified in the MGDS II and the Economic Recovery Plan 2012, focusing on areas of sustainable economic growth, infrastructure development and improved governance under two pillars, namely: 1. Address infrastructure bottlenecks to competitiveness and growth in order to address

infrastructure constraints to service delivery, private sector growth and competitiveness supporting interventions in transport, energy, water supply and irrigation (including addressing regulatory and market challenges in infrastructure services and Malawi’s regional integration agenda); and

2. Support actions to expand private sector investment and trade including policy reform and actions to improve the economic and business environment (such as improved PFM, trade and regional integration and improved financial services for Small and Medium Enterprise.

This third CSP is unique in that it specifically added a focus on regional strategies. It notes, for instance, that “...to ease challenges posed by Malawi’s landlocked position, the Bank will scale up support to regional infrastructure to deepen the country’s integration with its neighbors.”

7

Figure 1: Evolution of the Pillars of the Bank’s CSPs for Malawi 2005-2016

The Southern Africa RISP 2011-2015 has a primary focus on enhancing and advancing development of key corridors with the objective of fostering and sustaining economic growth and social integration by means of increased intraregional trade, greater legal, institutional and regulatory harmonization and lower production cost structure. There are two strategic support pillars as follows:

1. Regional infrastructure focusing on environment and climate friendly infrastructure programs in transport, energy and Information and Communication Technology; and

2. Capacity building in support of infrastructure focusing on supporting the CES Tripartite Arrangement as well as developing joint frameworks to improve regional coordination by means of strengthening institutional capacity of RECs and national implementation units for implementation of infrastructure programs and trade facilitation activities.

As shown in Figure 1 above in all three strategic cycles, there has been a focus on infrastructure: transport at the national and regional levels, energy, water supply, irrigation and communication. In the first one cycle, the other major area of intervention was human capital and institutional capacity development, with a shift to PSD, investment and trade from 2011 onwards. The importance given to regional integration increased in the same period with the approval of the RISP. In addition to the projects, the Bank’s strategy in Malawi includes a range of non-lending activities, mainly PD, knowledge and analytical work and donor coordination, aid harmonization and alignment.

4.2 AfDB Portfolio

Lending Operations The analysis covers all projects implemented Bank in Malawi during the period 2006-2016. Over the evaluation period, the committed funds amounted to UA 428 million for a total of 36 projects, (Annex C), and the corresponding disbursements amount to UA 275 million. The total contribution per year amounts to an average of UA 36 million; however, from 2010 the funds have sharply decreased

8

reaching UA 3 million in 2011 (Figure 2). The disbursed amount covers 64% of the total committed amount, however, for the period 2006-2011, disbursement almost equaled commitments while, from 2012, there is a normal decrease due to ongoing and approved projects. Notably in 2016, the disbursed amount is only UA 30,688 corresponding to one project in the Agriculture sector.

Figure 2: Amounts (UA million) committed and disbursed by year in Malawi (2006-2016)

Source: AfDB SAP Data

In terms of sectoral distribution, the AfDB contribution is distributed among all major sectors with a particular focus on the social sector which accounts for 24% of total funds with the exception of the power sector which received less than 1% of the total committed amount.

Figure 3: AfDB commitments by sector in Malawi (2006-2016)

Source: AfDB data

Table 6 below provides data on the status of the projects. At the time of the evaluation, there were 16 closed and completed projects, and 15 projects were ongoing and 5 at the approval level.

Table 6: AfDB Commitments by status and sector

Status Commitment (UA million)

Sectors Total

Agriculture Transport Power WATSAN Social Multisector

Approved 28.0 3 1 1 5

Ongoing 208.4 3 2 1 2 5 2 15

Completed 158.1 2 2 2 1 4 11

Closed 33.6 2 2 1 1 5

Total 428.2 10 5 1 5 8 7 36

0

10

20

30

40

50

60

70

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Mill

ion

s

Netloan (UAC) Disbursement (UAC)

Social24%

Transport21%

Agriculture22%

Multisector 17%

Watsan16%

Powerless than 1%

9

Non-lending operations In addition to lending activities, the Bank also conducted, within the framework of CSPs, non-lending activities - i.e. Economic and Sector Work (ESW), and other studies and initiatives supporting the implementation of CSPs. These facilitated high level PD with the GoM and provided analytical evidence for future programing cycles. These products are developed in collaboration with various operational and thematic departments of the Bank, often in partnership with other donors. For analytical purposes, ESW and other non-lending activities can be grouped into three main categories. 1. In-depth studies and sectoral reviews specifically aimed at helping in building the

information/knowledge base to underpin the Bank’s lending operations and to inform the PD process. It includes both country studies and regional studies with case studies on Malawi relevant to this evaluation;

2. Periodical publications and bulletins, e.g. the African Economic Outlook. These contributions typically compile secondary data that are analyzed in greater detail in other documents.

3. Other factsheets and brief overviews, such as summary brochures of the Bank’s interventions in a given sector, or brief chapters in general publications.

Box 1: List of ESW conducted during the evaluation period

2012 Private Sector Profile and the Joint Country Analysis 2012 Evaluation of Public Financial Management Reform 2012 African Economic Outlook 2011 Policy brief on Economic Management and evaluation 2011 Public Expenditure and Financial Accountability Assessment (PEFA) 2010 Education and Employment in Malawi 2009 Country Economic Memorandum (CEM) 2009 Skills for Private Sector Development 2009 Case study on Malawi for enhancing Linkages with vertical Health funds 2005 Multi-sector Country Gender Profile 2005 Agricultural Sector Review

In addition to financing instruments and ESW, the Bank actively engaged in PD with various Ministries, Departments and Agencies, both bilaterally and in the framework of thematic working groups. Benefiting from country presence since 2007, the Bank’s visibility and quality of dialogue have improved. The main issues for PD highlighted in CSP 2013-2017 included:

Portfolio Management: dialogue with GoM to address the portfolio management and performance constraints.

Economic Governance and Business Climate: PD to improve the business environment. The main issues include improving the regulatory and institutional environment for key services, deepening financial services, improving institutional capacity for trade promotion and export diversification, and maintaining a good macroeconomic environment.

Urban Development and Population growth: In view of the high population growth and urbanization rates, the Bank engaged with other partners, to articulate measures to meet these challenges.

Joint Assistance Strategies: The ongoing Joint Country Analysis being undertaken by DPs in Malawi was an opportune platform to move beyond discussing co-financing arrangements and joint implementation units towards developing joint assistance strategies with other

10

donors. The Bank explored this during the CSP period and discussed the strategy with various donors.

Regional – National Program Interfaces Malawi is a landlocked country with high transport costs and international journey times—only very recently has Malawi become a credible transit country (from Zambia to ports in Mozambique). In acknowledgement of this situation, the CSP 2013-2017 (and SA RISP 2011-2015) emphasized a regional focus to strengthen regional operations and movements by way of hard and soft infrastructure (in transport and energy) to reduce transit times and costs, and enhance connectivity to facilitate increased regional and international trade. This support identified rehabilitation of sections of the Nacala Road Development Corridor and provision of two one-stop-border-posts (OSBPs) connecting Malawi, Mozambique and Zambia; the development of the Mtwara Corridor connecting Malawi, Tanzania, Zambia and Mozambique; and supporting the Sena rail line connecting Malawi and Mozambique. At the time of this writing, implementation progress has been limited—the corridor road links are well advanced and a Bank feasibility study has been undertaken for restoration of the washed-out Sena rail line. However, construction of the OSBPs has not yet started as well, support to the Mtwara corridor has not advanced beyond the concept level. The Bank’s strategy assumes that investment in transport infrastructure under national programs will provide the building blocks for regional connectivity (with associated support to facilitation issues such as border crossing procedures) but such national programs respond to national rather than wider regional priorities such that efficient implementation of cross-border regional programs is at risk as neighboring countries do not share a common vision of priorities. For example, Mozambique does not share Malawi’s sense of urgency for the development of OSBPs, the Sena rail line or the Shire/Zambezi Waterway. The Bank’s regional strategy is not effective in providing complementarity coordination with the national strategies for Malawi.

EVALUATION FINDINGS

The performance of the Bank’s program in Malawi in each of the evaluation criteria assessed is rated using a four-point scale. The ratings awarded in this evaluation are at a program level (overall/synthesis) but rely on the information and analysis contained in the sector reports, and ultimately from individual interventions (see Annex D). The four-point rating scale is briefly defined in the table below:

Table 7: Rating scale and description

Rating value: Represents: Brief description:

4 Highly Satisfactory Expected performance was exceeded

3 Satisfactory Expected performance levels were met

2 Unsatisfactory Most performance levels were met but a few were not

1 Highly Unsatisfactory Most expected performance levels were not met

5.1 Relevance

This performance dimension assessed the extent to which the Bank’s country strategies and operations were aligned with Malawi’s development needs, strategies and priorities as well as the needs of ultimate beneficiaries. The relevance of the Bank strategy and assistance to Malawi was rated Satisfactory.

11

The Bank strategic areas were well aligned with GoM priorities. Overall, key national priority sectors were covered by the Bank’s strategies even though over a time span different from the MDGs. For instance, in terms of priority areas, the transport sector was well covered by the Bank starting from the ICSP 2011-2012 under Pillar I “Improving Infrastructure”, (Figure 4) while it was already a priority for the government in the MGDS I (2005-2009). By contrast, the Bank strategy included within the priority areas, PSD and good governance interventions, were considered by the GoM as overall thematic areas for development but not included under priority areas for intervention. Therefore, the Bank’s strategic decisions in good governance, which were addressed in all of the Bank strategies, were in line with existing diagnosis and analysis of the context, which showed that Malawi needed to promote good governance to achieve its poverty reduction goals. The Bank therefore filled in key strategic gaps and helped to further define, in practical terms, GoM strategies and priorities. This is particularly relevant in regional development in the transport sector, where the regional actions required for the development of the transport in Malawi were essentially defined through the interaction of the Bank and RECs such as Southern African Development Community.

In each Appraisal Report, the Bank has tied the justification and design of its interventions directly to one or more of these policies and strategies. This was done by specifying how the Bank’s policies and strategies related to GoM sector strategies. In many cases, however, the justifications were not tied to GoM’s expected outcomes but to overall policy and strategy statements (e.g. HEST and LED social projects). Some GoM officials also believe that while the Bank’s project designs are relevant, some thought should be given to bigger but fewer projects, making the bigger scale more relevant to the achievement of the outcomes and therefore to the needs of the beneficiaries.

Similarly, the Bank’s provision of GBS within the CABS framework has broadly reflected government’s own priorities, as laid down in the MGDS and other policy documents; however there were at times some deviations or inconsistencies in the choice of performance indicators (in multisector projects), weakening the linkage between the Bank and the GoM.

The interventions at sector level were aligned with the country strategies. The analysis of project documents throughout the sectors indicates that the operations contain analyses that compare country strategies to results framework of interventions thus clearly showing that the Bank is aligned at the sector level with the strategies for Malawi. That is to say that, for instance, the Education V project was aligned to the education strategy and the Health SWAp project was aligned to the National Health Policy and a Program of Works. The analysis also shows that at a higher level, the Bank interventions were also in line with the strategies laid down in the CSPs.

Nevertheless, the documentation that defined interventions do not compare the intervention result framework to that of the CSP. In part, this is because the CSPs are not specific enough in their operations descriptions to do that. However, a majority of the PARs refer to a coherence between the pillars-thrust of the CSPs and the objectives/targets of the project (establishing a high level alignment).

Of particular interest are the considerations for the PSD pillar. Although the Bank is including PSD in its strategies, it has made little progress in developing a PSD portfolio. As a result, there has been a slight disconnect between the Bank’s past and present portfolio and the goals and objectives stated in its CSP.

Specific beneficiaries were rarely identified in CSPs and RISP though there were references in project documents. Analysis shows that by defining the “beneficiary” at a high level and by profiling beneficiaries into a very small number of types, the Bank’s operational designs contained targets for beneficiaries, even if these were not characterized or broken down further in terms of expected effects within sub-samples. The beneficiaries defined in the CSPs tend to be generic, as one would expect from a document that is designed to show strategic alignment and only point to intentions for interventions by sector.

12

Figure 4: AfDB CSPs and Malawi MGDS priority areas

13

The Bank’s particular concern in ensuring that its operations are aligned with the needs of the population can be demonstrated by the Bank’s decision to put particular emphasis on social sectors, while designing its PBOs in 2007, and then again in 2012 (with Restoration of Fiscal Stability and Social Protection (RFSSP), 2015 (Protection of Basic Services – PBS), and in 2016 with Food Crisis Response Budget Support (FCRBS). On two occasions, the Bank used the Crisis Response Budget Support modality—first to protect the population from the adverse social impact of the Kwacha devaluation in May 2012 (RFSSP, 2012), then to contribute to the Government’s Emergency Response Plan to mitigate the food crisis (FCRBSP, 2016).

5.2 Effectiveness

Effectiveness is evaluated by assessing the extent to which Bank’s interventions in Malawi achieved their expected development results (outputs and outcomes) against targets set in project approval documents. Additionally, effectiveness is also evaluated by ascertaining the extent to which Bank interventions benefited targeted beneficiaries. This performance dimension is rated Satisfactory. The portfolio representing the Bank program in Malawi during the timeframe under review, was made up of 36 projects, for which a total of 11 PCRs were available. Bank PCRs reflect a satisfactory level of performance both for output and outcome level achievements. However, it may be noted that the PCRs take into consideration post- MTR definition of expected results. For the majority of PCRs, the MTRs have reduced the expected output and outcome targets considerably. In the agriculture sector, on the two sector projects that were closed during the review period, expected effects were clearly stated and monitored, and achievement levels were stated. For the SCPMP, there was a 54% overall achievement for irrigation development, a 102% achievement for farmer support program and 100% achievement for project coordination and management. About 85% of the schemes and 74% of the Market Centers (MCs) and Community Storage Facilities (CSFs) were only completed after Project Year 4 leaving little time for farming activities. 24 out of 26 CSFs and 13 out of 14 MCs were developed. The remaining two CSFs and 1 MC were among the contracts, which were terminated. Only 59% (23 out of targeted 39) of Water Users’ Associations were formed and registered. In the transport sector, project progress reports and two PCRs were scrutinized. All Bank support interventions for provision (or rehabilitation/upgrading) of road transport infrastructure have delivered expected outputs (in some cases at additional time and cost, and in some cases at lower than estimated cost). Overall, the physical outputs of the interventions were well achieved in the sector. The beneficiaries (private sector firms needing transport, consumers of transported products, etc.) were not monitored to see if their needs were met. The information on “meeting needs” is limited to the opinions of government officials and the consultations with key stakeholders such as the Chambers of Commerce. At a higher level the Bank monitors CSP implementation (e.g. MTR, Supervision Missions) and measures progress of implementation of MGDSs (e.g. IMF Joint Staff Appraisals) but weak institutional capacity of line ministries and agencies involved in transport jeopardizes ongoing and real-time M&E, policy analysis and eventually decision making. In addition, the sector beneficiaries are mostly considered to be those who need to use roads to make a living (e.g. transport firms) while the other beneficiaries are not sufficiently taken into account. For the WATSAN sector, sector targets for outcomes and impacts were set in general terms in CSPs, responding to the MGDSs targets. All sector interventions specified expected results (outputs, outcomes and impacts) in response to higher level strategies. Monitoring indicators were defined and logical linkage presented in the Logical Framework even though the translation of outputs into higher level outcomes and impacts was not clearly discussed. However, while there was some monitoring of direct project-related outcomes there was little or no monitoring of the achievement of target

14

impacts, as no ex-post project evaluations have been carried out. It is thus difficult to accurately state to what extent the Bank interventions have achieved the expected outcomes and impacts or development objectives. The Bank interventions have benefitted target group members (though not necessarily to the extent expected at project design). Monitoring systems were established for individual projects (although baseline information against which change could be measured was not always available or of good quality). Monitoring systems did not deal with ultimate beneficiaries in large part because most monitoring was restricted to activities and outputs.

The Bank’s interventions partly benefited the target group in the agriculture sector. The SCPMP had issues in identification of genuinely committed beneficiaries and therefore had a lower (72%) achievement rate of progress towards target. The issue of ‘commitment to participate in a project’ or ‘being shortlisted as a targeted beneficiary’ was identified in the field as among the extremely important first steps in project implementation, as, if the wrong people are ‘targeted’ or ‘recruited’ into a project, the whole project process starts off on a wrong trajectory and implementation becomes a constant struggle, resulting in low achievement rates. Analysis shows that during the project planning phase, there was insufficient clarity in defining the beneficiaries, with clear selection criteria. SCPMP had important weaknesses in knowing exactly who to support, and the evaluation team found this to be a serious weakness in design and implementation. For the CARLA project, the number of beneficiary farmers surpassed the original target, as at the end of the project period. There was positive response from the communities due to the felt needs arising from frequent disasters occurring among the targeted communities. Participation of women farmers in both the SCPMP and CARLA projects was very good. The target of 45% was achieved in the SCPMP while the CARLA achieved 53% against a 60% target. At the higher level, and very much the same as for the transport sector, the Bank monitored CSP implementation (e.g. MTR, Supervision Missions) and measured progress of implementation of MGDS (e.g. IMF Joint Staff Appraisals). However, weak institutional capacity in WATSAN-related ministries jeopardized the analysis of the effects that interventions were having on beneficiaries.

5.3 Sustainability

Sustainability is evaluated by assessing the extent to which the benefits from Bank interventions continued to exist, or were likely to continue to exist, after project completion. This performance dimension is rated Unsatisfactory.

Few of the completed projects under review have reached a post-closure period of 5 years, which makes actual sustainability assessment challenging. Some benefits gained from past interventions (in particular the soft components) were already at risk of not being sustained in the immediate years after completion. Benefits from past interventions, most notably infrastructure, have continued to flow (longer than 5 years), but longer-term sustainability (i.e. for the life of the project envisaged at design) could not be ascertained.

Better sustainability practices were found in the agriculture and transport sectors, compared to the social sector. At the same time, sustainability is not guaranteed in any of these sectors. In the agriculture sector, for example, the SCPMP was rated unsatisfactory on three parameters including i) financial sustainability; ii) ownership and sustainability of partnerships; and iii) environmental and social sustainability. Only institutional sustainability and strengthening of capacities was rated satisfactory, with a score of 3. In contrast, the CARLA project was rated satisfactory on all similar parameters, with a highly satisfactory rating of 4 for environmental and social sustainability. In the social sector, the evaluation concludes that the LED, competitiveness, and job creation may not be

15

sustainable under the present aid architecture (i.e. unless considerable investment continues to flow to those types of strategies over a long period).

It is a requirement that sustainability considerations be included at the design stage. The projects in agriculture and infrastructure in particular included sustainability plans, with a clear definition of roles and responsibilities.

In the agriculture sector, plans included clear definition of accountability for sustainability. This included the GoM paying for recurrent costs, the Water Users’ Association paying for costs using user fees as well as management of schemes, all at the same time as the Dwangwa Cane Growers Trust in Dwangwa and village management committees were to manage and maintain market and irrigation infrastructure in Dwangwa, Blantyre and Shire Valley Agriculture Development Divisions.

In the transport sector, plans are in place for the maintenance of roads both at the national level and in the preparation of individual projects. There is also clear definition of responsibilities for sustainability (maintenance and operation) in the short term, with the contractor responsible for the ‘defects liability period’ immediately following completion of construction (usually one year), and thereafter being the responsibility of the Roads Authority (with funding raised mainly by fuel levies accruing to the Roads Fund Administration).

However, the management of sustainability has not been institutionalized, follow-up actions have been limited, and the execution of sustainability commitment, including financial, has remained weak. More specifically, the sustainability plans developed at design stage have been based on a set of intertwined, unproven, assumptions about government commitment, government capacity / internal sustainability, community participation, recurrent costs, and the external environment. These are discussed below.

Government commitment

Most projects implicitly equate sustainability with government commitment, which is inaccurate. For example, the Competitiveness and Job Creation project, Health SWAp, and Education V projects note that sustainability was to be ensured by the Ministry. Yet the Ministries involved had a very hard time in financing operations, e.g. this was a considerable concern that justified the PBS project. Interviewees in the Health sector, for example, noted that the GoM had not budgeted sufficient resources for ongoing maintenance and operations of the maternal care facilities and programs. One of the sites visited during the field mission had no running water for the maternal care unit even though the water tower was built, the result of a lack of budget for a simple water pump. There is hence a constant assumption that the government institutions involved will not only honor their commitment but also have access to sufficient resources to fulfil these commitments (which may include resources from the Government budgets and/or setting of tariffs/collection of revenues).15

Similarly, PBO’s full alignment of government priorities aims to ensure strong government ownership. In practice, however, progress with selected government actions has been at risk of reversal in Malawi, as the country went through three changes in government and commitment to PFM reforms has remained cyclical. There is also a perception amongst donors that government actions in PFM are mostly driven by donor requests. In addition, when GoM policy involves devolution of responsibilities to lower level administrative bodies in sectors such as WATSAN and agriculture, this rarely comes with the provision of commensurate resources.

Government capacity / internal sustainability

The Bank has assumed that using and supporting established government structures to implement the projects would be adequate to ensure sufficient follow-up in terms of both commitment and

15 It may be noted that the CARLA project learned a lot from the SCPMP project, and therefore had better internal sustainability support systems. For the CARLA project, government had incorporated activities on CARLA in its national budget, with possibility of inclusion into subsequent budgets.

16

capacity. In practice, the soft component of the interventions has been insufficient and/or not adequately followed up with complementary measures to ensure that the required skills were institutionalized. For example, the SCMP project was implemented within the established structures in the Department of Irrigation and the District Councils. While field extension staff and district engineers were adequately trained, long lasting capacity building of mainstream civil service staff was not addressed at design stage and the hand-over between project specific staff and the mainstream civil service was in some cases not coordinated. Similarly, in the health sector, the Ministry of Health and Districts have not taken the training of people sufficiently into account, therefore not providing qualified employees as expected.

PBOs are also designed so that the government can fully use and, in so doing, strengthen, its PFM systems. According to the Bank, this inherent feature of PBOs ensures, in principle, strong institutional/internal sustainability. The Bank’s decision to move towards ring-fenced budget support operation has had an impact on sustainability; however, with PBS, while funding still goes to the Treasury, the Bank operation no longer relies on the country’s financial reporting and procurement systems. These additional safeguards mean that PBS bypasses the country’s procurement and financial reporting systems, with key informants criticizing the lack of knowledge sharing and/or skills transfer between the hired audit company and Ministries, Departments and Agencies’ procurement unit. However, PBS was never designed to be sustainable in the first place, and the project shows how precarious the situation is in Malawi for long term sustainability of any program.

The inadequate quality of training and equipment financed by the Bank in the social sector was also likely to have a negative impact on government capacity for follow-up actions. More specifically, interviews and studies show that some of the programs supported by the Bank have received just enough funding to remain active, but at a level of quality that is insufficient to meet the statement of outcomes. For example, teachers do not receive better training, and new texts are not forthcoming within the Ministry of Education Science and Technology budget allocation; maintenance of maternal health clinics is substandard and some facilities have no water or power; professors of universities and technical colleges do not get upgraded and equipment maintenance and replacement is substandard (i.e. poorly planned and funds not available and/or improvement not done as required).

Similarly, the Bank’s decision not to set up a PIU but select the PFM Division as the body responsible for coordinating and monitoring its Institutional Support Project in PFM scores well in terms of sustainability. The Bank’s Institutional Support Project is also likely to generate sustainable results when its focus is on strengthening local training institutions. Elsewhere, the Bank’s approach to training remains overall poorly defined and is unlikely to provide results, let alone sustainable results.

Community participation

Where communities were given management and operational responsibilities, their level of ownership still remained weak. Reasons put forward included: i) underutilization of the projects’ infrastructure and services (some being barely functional at completion, for example with the SCPMP project, most of the schemes were not fully operational at completion); ii) the mindset of local communities, as evidenced through undue reliance on the government; and iii) the lack of adequately timed and resourced capacity building activities (in the agriculture sector, for example, SCPMP, farmer training on leadership, marketing and technical issues was conducted before the infrastructure was developed). From a financial point of view, this has led, among other factors, to lower than expected users’ fee contributions, putting sustainability at risk. From a physical point of view, this has led to a premature deterioration of the projects’ community managed infrastructure.

In the agriculture sector, community participation was compromised in the Agriculture Infrastructure Support Project (AISP). Inefficiencies and continued delays in implementing the infrastructure

17

component16 meant that the time left for operationalization - and with it, capacity building - was likely to be reduced significantly. In the project sites where irrigation and solar infrastructure were installed, there was a lack of a coherent plan for the beneficiaries to fully utilize the irrigation scheme. The mission further observed that the farmer cooperative lacked the capability to initiate and manage a market-facing business strategy.

Recurrent costs

The cost of operating and maintaining the project’s infrastructure was consistently under-estimated. For example, at appraisal, beneficiaries of operation and maintenance commitments in the SCPMP were estimated at USD 12,360 per year (about 0.14% of capital costs or about USD 3/ha) but, actual costs were closer to USD 30-60/ha/year/plot (including fuel for schemes that were using motorized pumps). The HEST budgets for replacement of computers and software were linked to operating budgets that have traditionally been based on ‘replacement of components’ and do not cover repairs, broken-down equipment, technical advances, etc. In addition, sustainable benefits further hinged on the provision of good quality infrastructure built to appropriate specifications and hence requiring low, yet regular, maintenance work. This was not alway the case, as was seen in the transport sector.

External environment

Depending on the intervention, environmental sustainability has only partly been taken into account. Projects in the WATSAN sector, have only limited reference to climate and biodiversity despite wider resource management being an important issue in Malawi (as many water supply systems are gravity-fed piped systems which have fallen into disuse due to lack of maintenance and, in some cases, reduced surface water runoff). Continuing availability of sufficient water is assumed instead of being managed proactively under a sustainability thrust.

Other external factors, some of which are linked to the policy environment, have also been overlooked. For example, the limited market linkages partly explain the lack of sustainable results under the SCPMP; in the transport sector, axle load control was not been adequate, which contributed to prematurely deteriorating infrastructure. Overall, although the impact of external shocks (for example political instability, balance of payment issues, flood) on sustainability has generally been recognized, they have not been dealt with. In the transport sector, it was also assumed that there will be a rapid response to emergencies (e.g. flood damage), but this did not materialize.

The policy environment also remained problematic, as in the WATSAN sector, which remains characterized by inefficient systems, demand outstripping supply, ineffective community-based management systems, unregulated WATSAN services, slow and unbalanced decentralization of responsibilities, inadequate budgets, and institutional weakness.

Similarly, the long-term sustainability of PBOs’ key benefits, i.e. improved macroeconomic stability, strengthened PFM systems, and improved public service delivery, has been affected by economic and external shocks.

Finally, the issue of sustainability is not adequately monitored. An absence of ex-post evaluations or other follow up makes for limited assessment on the effectiveness of actions aimed at ensuring sustainability, the degree of fulfilment of commitments, and actual performance of the systems delivered by Bank support. As a result, there is usually no long-term follow-up of service levels and condition of Bank-supported infrastructure.

16 Most schemes had to be redesigned due to changes in technologies and sites due to land disputes and an underestimation of the costs of certain items at appraisal stage.

18

5.4 Crosscutting issues

This dimension is evaluated by assessing the extent to which the Bank’s objectives on inclusiveness across gender, regions, and environmental sustainability have been mainstreamed in Bank interventions. This performance dimension is rated Satisfactory.

Gender issues are strongly considered at the strategic level by the Bank. In both the Malawi CSP and the RISP, gender issues were taken into consideration (e.g. disparities in access to services, decision making structures, economic opportunities, cultural biases, budgetary allocation and monitoring of gender actions).17

The Bank carried out a country Gender Profile in 2005, which highlighted the main constraints for women empowerment. Therefore, project documents, mainly appraisal reports, have a section dedicated to gender issues, which are expected to be directly impacted or addressed by the project. Bank projects in Malawi during the time-frame under review had a gender-informed design, which specifically incorporated: i) sector-specific gender analysis; ii) a gender-equality-related outcome statement; iii) a gender-equality-related baseline; iv) specific activities to address gender gaps; and v) adequate financial and human resources to implement the activities. Moreover, most of the monitoring indicators are gender disaggregated.

For the social sector (education and health), it is noted that specific gender targets are specified for every intervention. In particular, for the education sector, where the indicators show that women tend to be less represented in the secondary cycle with negative consequences on employment - although disaggregated data on unemployment and salary rate are not available. As for the transport sector, although Bank strategies do not identify transport sector-specific gender issues, the transport projects would support women, men, children and the elderly equally via improved social and economic mobility and improved access to markets and health centers.

Usually the WATSAN sector is one with the greatest inherent impacts upon women and girls at multiple levels – access to adequate quantities of potable water within a reasonable distance, water carrying (which is predominantly a task of women and girls), hygiene and sanitation with direct impacts upon family health, irrigation for agriculture (an activity with high female participation).

As for the agriculture sector, there is evidence that women participated in the projects. For instance, the SCPMP PCR further notes that the project was designed to promote gender mainstreaming and specifically targeted female headed households to improve their livelihoods through membership in Farmers’ Organizations. The envisaged participation of women farmers (45%) was achieved, but not so for Government women staff, where only 10% was achieved against a target of 33%. In the CARLA project, women participation was 53% against a target of 60%.

The Bank programs appear to cover all regions, but there is no evidence of a further rollout to other GoM districts, nor other donors once the intervention ends. All interventions cover more than one district and are widely implemented in rural areas. However, inclusive coverage of regions where poverty is most prevalent is highly problematic. The poorest and/or most vulnerable have not been explicitly targeted in any of the projects and the poorest people predominantly living in rural areas are unlikely to be direct beneficiaries unless the rural (and urban) poor can enjoy greater direct benefits of Bank interventions such as big infrastructure (road corridors) or health and education projects which targeted rural areas within the beneficiaries.

Environment. In accordance with the 1996 Environmental Management Act, all sector projects in Malawi should undertake an Environmental Impact Assessment. Therefore, the Bank has made efforts to mainstream environmental assessment into project preparation and implementation but, although in most cases good engineering practices should minimize adverse environmental and social impacts,

17The same issues are included in the Bank results framework.

19

there is a reported capacity deficit in the application of sound environmental and social mitigation measures.

At the CSP level, most reference to environmental and social issues in Malawi is concerned with describing the current situation rather than setting out explicit strategies addressing these issues. However, it is clearly stated that environmental protection and mitigation of adverse social impacts is a component of Bank support. RISP goes further noting that the Bank will take leadership in promotion of climate friendly infrastructure and reduced carbon emissions, 18 making specific reference to Malawi’s (and some other Southern African countries’) problems regarding the effects of increasing climatic variability on infrastructure, food security and development.

ESIAs and ESMPs are produced for all sector interventions, but a lack of reporting in the implementation of ESMPs during the course of a project is noted. These ESIAs and ESMPs are prepared in compliance with national legislation and in conformity with AfDB environmental and social policies (which, in turn, are fully compliant with international best practices). All AfDB projects are categorized at an early stage in preparation and this categorization defines the expected nature of environmental (and social) impact and the mandatory mitigation treatment.19 Climate change is mentioned in such studies, but no proactive resilience measures have been noted in scrutinized projects and there are no targets on carbon footprint reduction.

In the agriculture sector, the Bank’s interventions demonstrated a clear link with green growth strategies and plans. For instance, the two agriculture projects SCPMP and CARLA (completed projects) as well as ongoing projects (AISP and Smallholder Irrigation and Value Addition Project (SIVAP)), were designed with the intention of supporting resilience capabilities and managing natural assets. New projects in the agriculture sector had climate-informed designs that included satisfactory actions to mitigate against impacts of climate change, climate variability, extreme weather events and biodiversity loss.

Despite having very good environmental management plans, not all projects (e.g. SCPMP in the agriculture sector) were managed in a manner that would ensure environmental sustainability and/or support the transition to green growth. On the other hand, environmental management plans were included in all stages of project planning and contracting. Indeed, all examined projects considered such issues and sanitary conditions were manifestly due to such support. Community sensitization has included environmental (and hygiene) best practices and catchment protection.

As regards to social aspects, beyond compensation (for land, resettlement, etc.), Bank sector support has included good practices such as community consultation and collaboration in rural areas (9 districts) and informal urban settlements (in Blantyre). The support projects scrutinized were all responsive to expressed demand and such community support was expected to contribute to ownership and management of the facilities. However, longer-term continuity of such involvement and motivation is not proven.20

5.5 Efficiency

This performance dimension is evaluated by assessing the extent to which the resources and inputs provided through the Bank’s support are converted into results at a rate that reflects assumptions at

18 No targets have been set for reduced or avoided emissions or reduction in carbon footprint (at project of program levels) except in general terms (e.g. by reference to MDG 7: CO2 emissions). 19 It should be noted that some social mitigation measures (e.g. compensation for land, houses, crops) might be frustrated due to late disbursement of funds for Government (payment of such costs is responsibility of Government and a component of GoM commitment to the project). 20 There are examples (in Malawi and elsewhere) of reducing motivation (and resources) and technical back up over time resulting in reduced contributions to maintenance and spares.

20

project appraisal. Moreover, whether the Bank’s interventions were implemented in a timely manner is also assessed. This performance dimension was rated Unsatisfactory.

The Bank was responsive and flexible to ensure satisfactory project performance, notably by allowing a reallocation of resources across the PFM project’s main components, and using unused resources to fund follow-up activities within the ‘multisector.’ Completed agriculture projects (SCPMP and CARLA) had PCR results that gave a resource use efficiency rating of 3, which is satisfactory. There was reference to ‘increasing the size of operations in order to reduce transaction costs’ (RBCSP21 2005-2009) and ‘the Bank needs to focus on a few sectors with an increased average size of operations by leveraging resources from other donor partners. This was expected to contribute to the reduction of transaction costs for both the Government and the Bank (2005-2009 CSP Midterm Review).

The Bank’s operations have incurred delays mainly due to noncompliance with its own operational standards (although slow Bank response to Government requests for ‘no objection’ has also been noted). The delays are mainly procedural and reflected in delays in decision making, disbursement, government commitment, and capacity constraints for implementation.

Long-term predictability of budget support disbursement has been problematic. The Bank did not provide budget support during 2011-2012, 2013-2014 and 2014-2015, and the amount fluctuated widely from year to year for the years when budget support was provided. The main constraint to ensuring the Bank’s predictable provision of budget support has been linked to the Malawi context, with the GoM first going off-track with IMF arrangements in 2009, then the ‘Cashgate’ scandal prompting a budget support withdrawal from all CABS DPs in 2013. As a result, the Bank’s budget support grant resources under ADF XI was reduced from UA 20.3 million to UA 11.5 million in 2010 to a slippage in the Country Policy and Institutional Assessment rating.22 This meant that the provisional budget support amounts announced to the government as part of CABS, could not be achieved.

For non-budget support projects, wherever there was a problem with disbursements, it was related to GoM administration of the disbursement process. This is a general problem across almost all Bank Operations in Malawi, with non-Bank managers finding it difficult to manage the finance-related processes of the AfDB. Finally, the disbursements predicted for private sector development through the commercial banks did not materialize.

Efforts were made by the Bank to use national systems but with little success, meaning that PIUs are very common and the Bank’s systems are used. Overall, for a country with the level of development of Malawi, the tardiness of completion and the difficulty of implementation, the problems encountered in the use of national systems are predictable. In terms of time to first disbursement, most first disbursements took place more or less as planned.

5.6 Knowledge and Policy Advice

This performance dimension was evaluated by assessing the extent to which the Bank was actively engaged in and influenced PD through the provision of relevant advice that was incorporated into policy decisions, and the extent to which the Bank provided appropriate and adequate analytical work in support of its interventions, positioning and policy advice. This performance dimension was Satisfactory.

Even though detailed plans for engaging in PD was not available, the Bank has been substantially involved in PD in Malawi. For example, the PBS appraisal report describes the many strategic topics of the accompanying PD for the intervention. It is collective in nature and is largely executed through various donor groups. Although there is no mention of specific value-added by the Bank or of comparative advantage, and no targets or objectives set for the dialogue, the interviews have revealed

21 Results Based Country Strategy Paper 22 Malawi’s PBA allocation was further reduced from UA 103 million under ADF XII to UA 68.39 million under ADF XIII.

21

that the Bank had a long history of PD and other non-lending activities in Malawi. These were strategic in that the Bank was consulted extensively for the preparation of all MGDSs and that some of its advices were taken on board, especially in irrigation and agriculture. The Bank was always part of the Heads of Cooperation discussions leading to taking strategic positions with the GoM (including those on farm subsidies in which the Bank played a leadership and determinant role in policy change). The Bank was instrumental in prioritizing, through dialogue, the maternal health component of the health strategic plan for its support, and in finding solutions to help Malawi in the immediate aftermath of ‘Cashgate.’ These examples point to a strategic perspective to policy change. Each Task Manager works on policy issues and the Country Manager was often mentioned by interviewees as having considerable legitimacy in the eyes of the GoM as regards to decisions on strategic options, including those in agriculture and power.

The Bank has adopted a strategy of engaging in PD through association with other donors. For instance, the Bank is part of various donor groups through which it had sector or strategic discussions with the GoM. While acknowledging the work done by the Bank, the country stakeholders, including other donors, thought that it could play a larger role. It was often noted that other donors do much more preapproval analysis of interventions, so as they have a better idea of what has to be modified in terms of policy and can incorporate policy change requirements in the design. The Bank officials feel that they also need this opportunity to improve on PD and to participate in deeper level consultations and analysis.

The Malawi Country Office undertakes PD through a variety of means, including participation in technical and donor working groups, participation in high-level forums with the GoM, one-on-one meetings with GoM officials and ministers, donor-country meetings, and political dialogue meetings. In some of these meetings, the GoM reports on modifications it has made to regulatory and policy frameworks as part of the dialogue in those fora. In terms of the ‘perception of usefulness of PD by cooperation partners,’ various project reports describe processes and inputs and activities, not PD (with very few exceptions). Interviews with donors revealed that the PD undertaken by the Bank was very useful and was always done in a spirit of collaboration, cooperation and support. Examples in GBS, health, education, transport and agriculture were cited. Many had wished that the Bank would assume a larger leadership role, given its ‘preferred relationship’ with the GoM, which considers the AfDB as ‘their own Bank.’ Evidence from consultations shows that the GoM is particularly satisfied and even encouraged with the relationship that it has with the Bank, largely in the area of PD. The PD in question has had specific effects, for example, the GoM decision to change its policy on farm subsidies and a subsequent move to a private sector model was largely the result of the Bank’s PD with the Ministries responsible for agriculture, trade, irrigation and allied sectors. Other examples include health policy on maternal care and content of MDGSs. Interviews undertaken have shed light on the way the GoM uses PD strategically, indicating that, in essence, it could improve. It uses PD as a component of project management and as a way of getting issues out into the open, but rarely uses it as a formative process, and seldom asks to start a dialogue on a particularly difficult topic. In specific sectors the experience has been that PD is not documented well. For example, there is little explicit reference to WATSAN PD in Bank documentation – only RISP and 2013-2017 CSP MTR have reference to WATSAN sector PD, which may indicate increasing Bank engagement in PD but such reference is more concerned with activities (and coordination and harmonization, participation in Sector Working Groups, PFM donor groups, joint committee, Group on Financial and Economic Management, Donor Coordination Group and so on) rather than required policy change as such. On the other hand, the Bank’s pan-African WATSAN Initiative, Rural Water and Sanitation Supply Initiative (RWSSI), might be expected to provide a platform for such high-level PD but there is no reference to PD in recent RWSSI progress reports. This situation is also recorded in the 2015 IDEV report (WATSAN in Africa). It is almost as though the Bank separates ‘dialogue’ from ‘policy’ in this sector. GoM’s

22

capability to use PD as a strategic tool is not established – there is little such references in MGDS apart from the GoM’s intention to “lead the dialogue with donors…and seek to ensure that aid flows are predictable”. However, institutional inertia suggests not only GoM’s limited capacity to harness such dialogue but also casts doubt on the effectiveness of PD as a whole. In the agriculture sector, the ‘Troika’ has produced policy papers that have greatly facilitated PD. Some of the recent policies that the DCAFS has influenced include the shift of the focus of the Farm Input Subsidy Program to work through private sector, the drafting of the Seed Policy and the legislation of the National Agriculture Policy. Cooperating partners have a high opinion of the PD, but perceive the Bank’s attendance at DCAFS as being sporadic with only 60% attendance.23 Furthermore, the Bank turning down an opportunity to chair the DCAFS in 2017 was perceived as a lost opportunity, which would have given it greater advantage and visibility, and could have influenced the GoM. The GoM has used PD to lobby for increased support from DCAFS members, and this has resulted in a noticeable shift in the support that it has received from donor partners. For instance, the support for the irrigation sector has come after many years of lobbying, with the GoM’s interest to invest in irrigation going back to the 1940s.24

In the multisector domain, the Bank fully participated in the CABS process and structure. However, it may be noted that while CABS provided an effective platform for dialogue, it did not significantly influence policy change. For example, there is no single example to show that CABS was influential in getting some difficult reforms off the ground. Some government and donor officials have questioned whether the scope of PD was not too broad and whether CABS was the right platform to discuss non-PFM related issues. At the same time, an independent review carried out in 2013 indicates that “line ministries felt marginalized in the CABS process (headed by the Ministry of Finance) and that many did not see the direct benefits of GBS support.” The same review concludes that CABS had been effective in strengthening donors’ relationship and dialogue with government, but that the dialogue had become diluted away from key policy issues.

As indicated by a donor representative during the country mission, several compromises were made along the way, as the government continued to meet the majority of its Performance Assessment Framework indicators and targets. The CABS mechanisms also allowed little opportunity for follow-up with the aide memoires not carrying a list of agreed recommendations and/or actions. While all parties committed to a mutual accountability principle, there was no effective mechanism for holding government and donors to account.

Overall, policy influence was most effective when the IMF arrangements went off-track, budget support was withheld, and the government felt compelled to take remedial actions. PD, for example, intensified after the ‘Cashgate’ scandal with the government agreeing to address donor concerns through the PFM Action Plan and an e-Performance Assessment Framework. This dialogue was mostly carried out through the Troika (with AfDB and European Union as co-chairs) and the Group on Financial and Economic Management25, involving donors and Ministry of Finance, rather than during the CABS reviews.

A significant number of new economic and sector papers were produced by the Bank over the years. Even the other donors referred to some of the themes or topics covered in the papers as having been central to their programing. Unfortunately, the mechanisms to finance the generation of these documents are unwieldy, based on competition, or are embedded within projects and so are long and complex to be written contractually. Embedding financing as part of an intervention means that the funds cannot easily be used upfront for ex-ante analysis, and the timing will be delayed if the analysis

23 Discussions with DCAFS members during Field Mission in April 2017 24 Discussions with Ministry of Agriculture staff during Field Mission in April 2017 25 GFEM

23

is required. The Bank’s internal research facility is not used as it could be as it is not perceived as being client-oriented or responsive to urgent or even operational needs.

GoM and sector experts in donor offices appreciated the quality and depth of analysis and the way the advice was offered. They also appreciated the value of the Bank Task Managers in resolving project management issues. This is important in the Malawi context because interviewees noted that the various departments, agencies and ministries of the GoM had very limited capability to generate evidence-based analytical research in any domain, with the possible exception of phytosanitary or agricultural research.

There is limited reference to ESW in PARs and PCRs in the transport sector (although project preparation studies such as feasibility, detailed design and ESIAs could arguably be considered as ESW). Interestingly ESW is classed with dialogue activities hinting that ESW is intended to be used as a facilitation for such dialogue although such linkage/leverage is not articulated as such. The CSP 2013-2017 is more explicit about the ESW to be conducted – mainly feasibility and other studies – while RISP indicates that ESWs will be grant-funded.

Perceptions of the value of ESW advice in that sector were limited (mainly to project identification, feasibility and design studies) with some reference to Quality at Entry issues, which were, overall, not considered serious. However, GoM transport sector capacity to take advantage of such ESW outputs in order to generate analysis for policy development is limited. In that sector, as in WATSAN, the Bank has provided reasonably adequate analytical work in support of its interventions (although these interventions continue to be subject to over-optimistic assumptions). The positioning of the Bank has been dictated by its higher-level policies and initiatives (e.g. RWSSI) and there is little evidence to show to what extent the Bank’s dialogue activities are informed by such ESW; the perceived value of such advice and analysis (to the Bank, GoM and other sector stakeholders) has been confirmed by the stakeholders.

The Agricultural sector has seen a significant number of analytical and policy options papers prepared through the Bank’s non-lending facility. These include: i) a Feasibility Study on the Establishment of an Agriculture Cooperative Bank,; ii) a Regional Comparative Fertilizer Subsidy Study; and iii) a SVIP feasibility study. The Bank contributed to the following sector papers through the existing Bank staff: i) the Malawi Country Economic Memorandum: Seizing Opportunities for Growth through Trade; ii) the Background Note 2: Agriculture Growth and Diversification; iii) Background Note 3: Value Chain Analysis; and iv) Background Note 4: Agribusiness.

Field research shows that the Bank’s contribution to PD through these analytical works is appreciated, especially since the Bank also supports the implementation of the policies. The GoM is able to generate evidence-based analysis for policies relevant to the Bank through the work of the National Statistical Office, which has the necessary capabilities, with additional external expertise brought in when necessary.26

In terms of analytical work related to PFM and GBS, the Bank has regularly conducted or contributed to these analyses. Some analytical work was used to inform interventions, positioning and policy advice. In addition to the regular Fiduciary Risk and Country Procurement Analysis, and the yearly African Economic Outlook, the Bank published several economic and sector work relevant to financial and economic governance.

5.7 Partnerships, Harmonization and Leverage

This performance dimension was assessed by evaluating the extent to which the Bank’s interventions and processes were harmonized with those of other donors to avoid duplication in addition to the

26 Discussions with the staff of the Ministry of Agriculture during Field Mission in April 2017

24

extent to which the Bank’s interventions and resources influenced other stakeholders to get involved in sectors that are the focus of country strategies. This assessment dimension was Satisfactory.

The Bank and other donors have long followed a coordinated approach to their development assistance in Malawi. In key sectors, sector-wide approaches (SWAps) have allowed government and donors to work towards shared strategic objectives. While donors have continued to follow their own programing cycle,27 this aid architecture has been conducive to a relatively high level of shared (rather than joint) strategic planning. For example, in most sectors, donors broadly share the same analysis and objectives on key reforms.

The Health SWAp and agriculture sectors have entailed regular Joint Sector Reviews. There was also a high level of harmonization among budget support donors until 2013, with all CABS donors broadly following common procedures for monitoring and reporting. In addition, the Bank has regularly participated in joint studies such as the SVIP feasibility study and sector papers such as the i) Malawi Country Economic Memorandum: Seizing Opportunities for Growth Through Trade; Background Note 2; Agriculture Growth and Diversification, ii) Background Note 3: Value Chain Analysis, and iii) Background Note 4: Agribusiness. As noted earlier, the Bank also conducted a joint fiduciary risk analysis with DfID28 in 2009. It may be noted that while Joint Sector Reviews have been commonly used, the Bank (like many other DP) has continued to use its own monitoring and reporting procedures in parallel.

The Bank has been an active participant in several donor platforms such as the DCAFS (agriculture) and the PFEMRP (for PFM). While some de facto division of labor amongst donors has been in place, donor ambitions to move towards joint undertaking as managed jointly have been only partly met. There are limited examples of co-funded interventions, such as the SVIP, which is implemented together with the WB, and the NWDP in the WATSAN sector, which is co-financed by AusAid. In the transport sector, the JIU was discontinued in 2012 after only 2 years’ operations. The JIU was a local initiative by AfDB in Malawi for closer cooperation between AfDB and WB for joint management of the infrastructure support portfolios of the two Banks (i.e. WATSAN and transport). However, this initiative was short-lived owing to the lack of formal agreement ‘signed off’ by WB and AfDB, the absence of a mutual acceptance of ‘proprietary’ administrative, fiduciary, procurement or safeguard/governance systems, and limited support from the two Bank’s headquarters. In PFM, attempts to launch a Multi-Donor Trust Fund were only partly successful, as only a few donors joined in, owing to legal reasons (like the AfDB, which did consider it), or, because some projects were already ongoing.

As a result of the above, the level of complementarity between donors has not been optimal. In principle, the Bank takes into account other existing donor interventions during appraisal to minimize the risk of overlap. For example, the complementarity between the SCPMP and existing donor interventions in the sector was noted to be high during appraisal, 29 yet in practice, there is no description of how these various projects are made to be complementary. Complementarity is neither monitored nor measured during project implementation, and PCRs do not refer to the interfaces between the interventions. Evidence collected during the field visit nonetheless tends to indicate a higher level of complementarity in some sectors, in particular agriculture.

Because of the limited success in shared undertakings, the multitude of projects in some sectors, like PFM, where many donors intervene, has continued to weigh heavily on the government, as donors stick rigidly to their own procedures and there is no common reporting format. Even in the case of

27 In the area of PFM, the AfDB and WB went as far as conducting a joint appraisal mission in 2013 to define their respective contribution to strengthening PFM institutions. This example, however, remains isolated. 28 United Kingdom Department for International Development 29 One notable exception concerned the LED project, where various donors did not coordinate, and no sense of complementarity was achieved.

25

CABS, the Bank together with other donors were occasionally seen to come up with their own specific requirements.

Going forward, recent developments have shaken the donor landscape, bringing new challenges for future harmonization. The multi-sector and transport sectors illustrate these challenges well:

In the last 5-6 years, donors’ response to the context and their readiness to accept (fiduciary) risk has diverged, with bilateral donors no longer providing budget support.30 This has in effect ended the CABS arrangements. Understandably, some bilateral donors are uncomfortable with the AfDB (and now WB)’s decision to resume budget support, which they say is sending wrong signals to government that the prerequisites for resuming budget support have been met. It has hence become crucial for all donors to share the same messages to the government, whatever their choice of aid modality.

In the transport sector, the number of traditional donors has reduced with the withdrawal of the European Union and some EU Member States from the sector. In this sector, new donors follow different approaches and do not normally take part in sector donor fora.

The Bank’s influence on traditional donors also appears to have been relatively limited, although this varies across sectors. While the Bank’s decision to resume budget support with PBS may have helped the WB’s subsequent decision, this was not directly confirmed by key stakeholders interviewed during the field visit. In the governance sector, the Bank has also been relatively passive in getting other stakeholders involved in its PFM work. More specifically, the main visit highlighted the lack of a sufficiently inclusive and open consultation process over the amendment of the Public Procurement Bill, which the Bank initially supported by a standalone Technical Assistance. The use of a regional envelope31 may explain this relatively ad hoc approach to Technical Assistance.

Agriculture provides a more positive example of the Bank’s influence. In this sector, the Bank was able to leverage the SCPMP to get financing for the CARLA project from the GEF. As a reliable and consistent partner in supporting irrigation (including the use of solar power), the Bank’s interventions are also inspiring other donors to support similar interventions,32 thus serving as pilot projects that other donors can learn from.

5.8 Managing for Results

This performance dimension evaluated the extent to which the Bank succeeded in focusing on results in its management of day-to-day operations as well as the extent of its support to national systems that focus on results. This performance dimension was Unsatisfactory. The Bank’s strategy and project documents contain outcomes and results that are well stated in near-RBM terms (i.e. in terms of the effects expected) but there are no intermediate results, nor chain of execution or implementation in the key documentation. The verification of whether Bank interventions were managed by results requires an analysis of intervention management practices, including procurement, contracting, supervision, etc. The documentation does not support the premise that these were results-based, and interviews indicate that the Task Managers do not believe they have RBM-ready systems (this applies to most interventions with the partial exception of CARLA and SCPMP).

30 Some donors like DfID have also stopped using country systems altogether. This has contributed to weakening the SWAps previously developed in key sectors. 31 The Bank first supported the proposed amendments since the late 2000s, when it first provided TA, with direct involvement of the Bank procurement officer – as part of Common Market for Eastern and Southern Africa (COMESA) Enhancing Procurement Reforms and Capacity Project grant. 32 Discussions with Ministry of Agriculture staff during field visit in April 2017.

26

The Bank’s Monitoring systems do not focus on results, the evidence points to instances where lower-level results (outputs) were monitored, but not the entire results chain to outcome level. No mention of the extent to which RBM must be used has been found in the M&E sections, neither of Appraisal Reports, nor in the profiles and descriptions of the PIUs. Analysis and interviews on the ground indicate that monitoring systems are clearly not geared towards RBM. The reporting systems of the Bank are not RBM-based with most being process or input management-based and some output-based. Where there was a reference to results, it generally quantified achieved targets, rather than described the effects in terms of the original expected outcomes.

In terms of the sector level findings, the Transport and WATSAN sector analysis showed that all sector support documents (CSPs and individual support projects) have a Logical Framework Matrix which specifies results indicators at output (and activity), outcome and impact levels, but not a detailed project path analysis or Theory of Change logic including assumptions and risks. To that extent, Bank support can be said to be only partially results-based (as already mentioned above). However, in practice monitoring of such support is largely confined to activities and outputs, with limited coverage of outcomes and virtually no coverage of impacts. Thus, although action is taken during the course of implementation based upon mainly activity monitoring (e.g. MTR of CSP, progress reports of transport and WATSAN projects) the benefits of such RBM is limited by not supervising for results, nor in proactively managing assumptions and timing, nor extending management to higher level results (e.g. not putting in place any form of baselines on benefit analysis over the long term). In the Agriculture sector, evidence points to the finding that the Bank has often successfully implemented management systems that focus on process-based results and facilitate learning from experiences. The designs of both the SCPMP and CARLA projects include a Result-Based Logical Framework, which demonstrates the linkage between inputs, outputs, outcomes and impacts, and includes a risk mitigation assessment and key assumptions. Monitoring systems and mechanisms were based on mid-level results, which enabled stakeholders to keep track on implementation progress in order to achieve the intended objectives. Reporting mechanisms were component-based and structured according to evaluation criteria, such that the supervision missions have ToRs, and the reports (Back-to-Office report, aide memoire) are based on implementation progress, but lack focus on what the ultimate outcomes should be. These reports include a ‘Status of Project Implementation/Key Indicators of Progress’, which keeps track of the progress made against project results/targets.33 In terms of GBS and PFM, recent changes to the PBO guidelines are designed to improve the extent to which these types of interventions are results-based. PBOs evaluated in this mandate displayed the following characteristics:

The causal chains were poorly defined and confused different levels of results (output and outcomes) and their corresponding indicators.

The choice of indicators was largely driven from PAF. The Bank chose its own indicators in some occasions (often to ensure timely payment), which created some misunderstanding and delays due to additional reporting.

PAF was too broad, placed a heavy emphasis on process and input-based targets, and was not consistent with or directly linked to other government planning and M&E systems, which in turn remained weak (with often contradictory baseline values). When results indicators were used (e.g. pupil/teacher ratio), they did not sufficiently take into account the time it took the government to achieve the agreed targets.

In terms of the experience of the Transport and WATSAN sectors, evidence points to the finding that the Bank broadly supported the development of national capacities and management systems, but

33 SCPMP and CARLA Supervision Missions and Back-to-Office Reports.

27

that focus has been largely on process and program/project implementation rather than outcomes, or overall capability to design and implement regulatory or policy frameworks; results have been assumed to be developed from such enhanced capacities. Transport and WATSAN sector projects include capacity strengthening activities as a component of implementation (e.g. pump maintenance, district capacity for water supply operation and maintenance, road maintenance) but such capacity is usually related to the operation of project infrastructure and systems after the end of the project period. However, the evidence clearly points to important capacity constraints that will continue with multiple examples of both Transport and WATSAN sector infrastructures rapidly falling into disuse due to lack of sustainability. Most donors in the sector have RBM systems, although limited information is available for Non-Governmental Organizations and Civil Society Organizations active in the WATSAN sector. Most public bodies involved in WATSAN management are seriously under-resourced such that effectiveness of results management due to capability weaknesses or gaps is constrained.

In the Agriculture sector, the Bank has supported the development of national capacities and management systems, but a great deal remains to be done, at all levels of the capability chain. Results-based capacity development efforts for GoM line ministries and agencies in the sector are related to input and output organization and combination, not strategic outcome improvement. The evidence shows that most of the actions are not based on knowledge of what will, or won’t, work but on ideas for structuring and processing economic elements (from agricultural inputs to cooperatives and market councils, etc.). Very little analysis of cause and effects is drawn up and much of the strategic direction is based on isomorphic analysis (i.e. if it works there it will work here). The Bank’s supervisory and midterm analyses determined that it had been successful in its capacity building of GoM staff, such that both the SCPMP and CARLA projects had a satisfactory rating of 3 on institutional sustainability and strengthening of capacities. The evaluation findings do not support this level of appreciation, believing that it is a reflection of the level of execution of activities and not one of sustainable capability.

In the Multisector domain, evidence shows that the Bank’s specific objectives on procurement and domestic resource mobilization were to improve capacity in revenue administration and customs service delivery, leading to increased revenues, enhanced competition, efficiency and controls in procurement. In practice, insufficient attention has been given to managing and monitoring any of the results in those areas and the entire revenue and procurement systems are still unable to perform satisfactorily.

The PFM Division reports quarterly to the Bank on progress-to-date. These reports, however, focus mostly on activities. The choice of performance indicators at output level does not seek to capture the positive effect that training should have on procurement performance; while indicators at outcome level are too broad and/or largely depend on policy actions that are outside the scope of its interventions.34 The quality of the deliverables has not sufficiently received attention either and serious issues remain; consistency is lacking and the systems are dependent on the skills and knowledge of people in a context of rapid turnover.

5.9 Lessons Learned and Knowledge Management

This performance dimension is evaluated via assessing the extent to which the Bank learns from its experience. This is done by looking at the whether the factors affecting performance and results of Bank interventions are identified35 and inform36 Bank actions and strategies. Moreover, the issue of

34 This includes taxes on international trade as a percentage of GDP and number of contracts awarded on basis of open competition as a percentage of all contracts awarded in a given year. 35 The Evaluation considers only documented evidence as being “identified.” 36 Interpreted as being documented to show that there has been an influence.

28

incorporating lessons from Bank experience to inform future strategies and programs is also looked at. This performance dimension is rated Satisfactory.

The Bank has means in place to capture and record positive and negative factors that influence project performance. Lessons from previous interventions have been included in the CSPs under review, MTRs, PARs, PCRs as well as Back-to-Office reports of supervision missions. The narrative in these reports included description of achievement or failure to achieve results. Some reports also make recommendations for mitigation of negative effects and accentuation of positive aspects at sector level and/or for the portfolio as a whole.

An assessment on the extent to which such findings and lessons were incorporated into the design of new CSPs and interventions shows that use of KM depends upon the perceptions of potential users of the KM system, its availability and accessibility, and the definition and scope the said KM system.37 Experienced Task Managers have a depth of knowledge about the sectors they managed although their access to wider available information is limited – the Bank KM system is rudimentary relying on access to whole documents, which is dependent on the knowledge of the existence of these documents, rather than to data-mining or thematic search in a KM database. Beyond publication of some multi-national case studies, there is little active dissemination of lessons learned or experiences by the Bank to ensure better application of such experiences in subsequent support. Current use of the existing Bank KM system does not frame communities of practice nor facilitate system functionality or collaborative knowledge broking.

In terms of completed operations with a timely PCR, all available PCRs, except 3 reports, were prepared within 12 months following closure date.38 Government generated PCRs for Bank projects are not available and GoM contacts advised the evaluation that these are not normally prepared. Notwithstanding, PCRs contain elaborate sections on lessons learned, although lessons are stored in raw format (generally text) in the documents in which they were originally produced.39 Therefore, accessing information, findings, conclusions and recommendations from that data is tedious and depends upon knowledge of the existence of the specific intervention. At the strategy level, analysis of lessons learned is not always indepth or integrated into future support strategies and interventions.

Cross-sectoral, i.e. generic lessons (e.g. procurement and decision-making delays, quality at entry) that are continuously repeated in consecutive support programs, show that the lessons identified were not always learned and that the underlying issues are not resolved. This takes into account that some issues are beyond the Bank’s capacity to resolve (or avoid), remediation or mitigation being dependent upon other actors (e.g. delivery of GoM or community commitments). The Bank’s support has taken such political economy contexts (multi-variable cause and effect analysis) into account by identifying issues of government commitment (and capacity) to undertake policy and institutional reforms; these were treated as a factor which would influence results and effectiveness of the Bank support (positively and negatively) thus prompting the Bank to belatedly launch PFM ISPs.

37 ‘Knowledge management’ suggests use and application of such lessons learned and findings, not only a record of this information. Sources define a KM system as any kind of IT (or hard copy library) system that stores and permits location, retrieval and application of knowledge. 38 Exceptions are: Education V – 15 months after closure date; NCRP (Nacala Corridor Road Development Program) I – 19 months; NWDP – 16 months 39 Recently there has been improvement in this respect, by IDEV, via the creation and population of the Evaluation Results Database (EVRD)

29

LESSONS LEARNED AND RECOMMENDATIONS

6.1 Lessons Learned

The following lessons are identified from the Bank’s program in Malawi:

1. Clear definition of beneficiaries. When the beneficiaries of the interventions are described in overly generic terms, they tend to be seen as a numerical target to be reached, and therefore the supervision and management of the projects tend to become input- and process focused, rather than outcome focused. Beneficiaries need to be as astutely described as possible so that the effects of the targeted ‘benefits’ can be monitored. Specific baselines are to be established upfront and constantly updated with valid information.

2. Results-based monitoring system. There is no benefit that will accrue to the Bank in generating

results chains that would only partially be managed. If outcomes are the core objectives of investments, then monitoring and supervision of processes and indicators must be undertaken at all levels, including outcomes. With commitment to results-based management, it is necessary to have a monitoring system that generates decisions focused on outcomes, and a solid analysis of results chains before project approval.

3. Facilitative conditions for PD. The Bank is well regarded in terms of its non-lending activities and

the results they have produced. The policy and program dialogue that has taken place over the years has been well appreciated as a driver of change. Given that the GoM requires a considerable amount of support in order to accelerate its development, an important lesson is that the Bank has been able to influence and advantage policy when it puts in place conditions that facilitate and support dialogue. Increasing the quality and strategic nature of the dialogue will likely improve the level of influence. These conditions include developing a knowledge trust, promoting evidence-based decision making, developing the capability of institutions and agencies to design change strategies and monitor progress.

4. Timely decisions on design change. Major changes in project design, if required, should be

introduced by taking into account not only achievable outputs but also continued project viability and relevance, and this should not be postponed to the MTR if identified early during implementation.

5. Fiduciary safeguards and budget support. In a highly aid-dependent country, periodic withdrawal

of budget support constitutes an external shock that aggravates macroeconomic instability and can cause unintended damage to growth and to poverty alleviation efforts. It is essential to strike a balance between promoting country ownership and using additional fiduciary safeguards that can prevent the sudden withdrawal of budget support.

6.2 Recommendations

Based on the foregoing findings and lessons learned, the evaluation proposes the following recommendations:

1. Enhance capacity of Bank’s officers and managers in Results-based Logical Framework and the use of Results-based Management tools, including theory-based design, results-based contracting, and outcome-focused supervision and management of interventions. Each intervention should have clear identification of the characteristics of its target beneficiaries and design the content and required management systems. This includes the baseline for monitoring, the focus of the M&E system, reference points of the RBM system, indicators for outcome

30

measurement, contexts for project implementation, basis for the Theory of Change analysis, and the basis for assumptions and risks.

2. Enhance Bank staff proficiency to effectively operate in the business ecosystems in which key sector actors operate, and design interventions that are holistic enough to stimulate the private sector’s investment and actions. This is essential since most GoM development strategies rely on the operation of its private sector. As most of the business ecosystems are to some extent international in scope, the Bank actions in Malawi need to be international, with country programs developing ‘regional at least’ markets in a number of countries.

3. Since the Bank and DP are pressing for effective and efficient PFM systems, the Bank should

establish a clear timeline within which all required performance standards are met and proven. The GoM has been implementing PFM reforms for over 20 years, and therefore the Bank should identify what it should do if the systems cease to function during a specific period. The Bank could consider generating contingency supports to overcome critical weaknesses without compromising transparency and mutual accountability, and achieve development effectiveness.

4. The Bank should redesign its KM practices to address the need for access to information, and develop explicit and implicit knowledge in its Operations Divisions to improve decision making capabilities of the Bank and its clients. The Bank should build effective learning organizations by making learning routine and stimulate cultural change and innovation. It is therefore important that the Bank invest in modern KM technologies, develop its capacity and capability for data mining, and use KM as an integral part of any implementation strategy.

5. The Bank should ensure sustainability of its interventions by taking concerted actions

throughout the project cycle to ensure that impediments to sustainability are addressed and managed. Beginning at design and continuing through implementation, the critical assumptions should be analyzed and steps taken to mitigate risks. In situations where sustainability cannot be reasonably ensured, it is advisable not to approve such interventions in the future and ongoing interventions should be adjusted such that sustainability can be assured.

6. The Bank should take adequate measures to ensure that its interventions and strategies provide

equality and equity between genders and that the interventions provide a means to manage, track and report on these objectives. Gender as a crosscutting issue should be included in project designs and mainstreamed into implementation, going beyond simply including women as a fraction of the participants in activities. They should be part of the management and their needs, as being different from that of men, fully acknowledged. Project designs need to be clearly structured so that results are defined in terms of outcomes that bear on women and girls.

31

Annex A: Evaluation Methodology

The evaluation process adopted a systematic approach that uses different building blocks to construct an answer to the Evaluation Question (EQ) and to formulate key conclusions and forward-looking recommendations. The various phases and subsequent stages coincide with different methodological steps undertaken within the framework of the evaluation:

During the inception phase the Evaluation Team collected the relevant documents and data, gained a clear understanding and overview of the object of the evaluation, mapped AfDB support interventions to Malawi and developed a full methodology.

During the data collection and validation phase, the team completed the data collection at sector and overall levels.

The reporting phase was devoted to constructing answers to the EQs at sector and overall levels and formulating key conclusions and forward-looking recommendations on the basis of the data collected throughout the process.

The methodology applied for this evaluation is based on the evaluation criteria developed by OECD-DAC, but not on impact (not a requirement for this evaluation). In summary, the evaluation methodology is based on the use of an evaluation matrix that uses evaluation questions, judgment criteria (JC) and indicators as the building blocks for the analysis leading to conclusions and recommendations. In terms of epistemological boundaries, the methodology strongly focusses on results achieved and demonstrated, a Theory of Change approach to intervention design and management, a sector by sector analysis followed by a roll-up of sector-level findings to the program level, evidence-based research, contribution analysis and iterative feedback with key stakeholders to strengthen the validation bonds. Five main categories of information were collected and analyzed for the evaluation technical report: Background and context information on the country, the region and the cooperation policy of

the Bank. This information is necessary first for an understanding of the main issues addressed, or that should have been addressed, by the Bank support; and second, for an overview of the country’s situation in relation to these issues, so as to form an initial idea of the possible contribution of Bank support to the observed trends. This work was started during the inception phase and then complemented during the field and desk phase with information for each specific sector.

AfDB cooperation with Malawi in general and at the sector level including a systematic review of strategic and programing documents, alignment with the Government of Malawi (GoM) policies, and analysis of the sector intervention logic, a portfolio of the AfDB funding, etc. This started during the inception phase and has continued throughout the evaluation process.

Portfolio. The Bank portfolio has been reviewed on the basis of additional information collected during the field phase. An overview of each intervention is presented in each sector report. Such analysis supported the identification of lessons and provided elements of the answers relating to each of the JC. The available documentation on these interventions has been screened at the level of the relevant JC.

Case studies: The case studies provide an in-depth analysis of a representative intervention for each sector providing detailed review of the intervention itself, including a Theory of Change and its accompanying assumptions and risks, and an analysis of its evolution and contribution to the sector goals. The case studies were generated through a more in-depth documentation analysis and a thorough interview set in the field. In the documentation, a special effort was made to formally tie the context and management issues of the interventions into the performance actually achieved. In addition, each case study is analyzed to see what contribution

32

it has made to the Malawi program as a whole, including lessons learned. There is no presumption of a typology being used in the selection and use of these case studies as the performance variables are not defined by the Bank except for the overriding variable of supporting the GoM. This handicap does not diminish the validity of the use of the cases; they represent in-depth analyses from which common lessons and findings have been extracted, and from which common lessons learned can be aggregated. It is clear that the overriding factor that distinguishes the case studies from the analysis that was done on other interventions is the extensive use made of the case studies in the analysis of the Country-level Program of the Bank (i.e. at the CSP level).

Interviews: Interviews with key stakeholders (Bank staff, other relevant donors, civil society, Non-Governmental Organizations, beneficiaries, implementing partners) have been carried out for each sector and information has been included in the analysis for each EQ.

The process of data gathering and analysis has been structured in two steps: Identification, collection and processing of information relevant to all JC and Indicators, all

information collected is presented in Volume 2 of the sector reports. Analysis and synthesis of the information at the level of JC and Indicators. For each JC the team

has provided information at sector level on the basis of the data collected and presented in Volume 2 of the sector reports. The team has then analyzed the data across sectors in order to synthesize the information and provide an overall answer for each EQ.

In addition to this Final Technical Report, five sector reports of two volumes each, were prepared during the course of this evaluation. The first volume contained sector context analysis and the responses to the EQ as they applied to the relevant sector. The same Evaluation Matrix used for this report was also used for the sector reports with the exception that sector specific indicators were included in the latter. The evaluation team synthesized and critiqued the analysis of the EQ in each of the sector reports and adapted them to the Evaluation Matrix at the program level. They also actively participated in the development of conclusions and recommendations to the Final Technical Report. Limitations

There were a number of important constraints/limitations in the application of the evaluation methodology:

It was difficult to meet with and interview some of the key GoM officials, Project Implementation Unit managers and Bank Task Managers who had firsthand knowledge of the interventions. Accordingly, many questions that were directed at ministries and agencies 40 remained unanswered. When possible, the evaluation team mitigated the former constraint by directing questions to other individuals who might have the required knowledge. In many cases this was successful (e.g. vocational training strategies, but in others it was not, e.g. health). The Bank task officers were also asked to fill in the information gaps and, where the question was of a content nature, they were invaluable. They could not, of course, respond to policy or organizational capacity development issues.

There has been some difficulty in obtaining project documents due to retirement of key personnel (i.e. the Bank’s Country Manager and the Country Economist). This caused the response to requests for information to come very late and most of the time documentation. However, the evaluation team obtained a good part of information through interviews with the Bank team in the field and through other documents that were available.

The monitoring systems used by the Bank and the GoM do not monitor for outcomes. However this constraint was mitigated via the use of proxy indicators.

40 E.g. Ministry of Education Science and Technology, Ministry of Health, and various agencies and ministries in agriculture and irrigation.

33

For assessing the achievement of objectives, the evaluation considered the objectives included in Project Appraisal Reports (PRAs). In that context, it should be noted that many interventions41 had their levels of expected results significantly reduced during the midterm reviews (MTR), so the ratings provided by this evaluation are by necessity often lower than those provided in PCRs.

Regarding the impact of weather shocks on the economy, the evaluation team took this ‘climate and periodical’ effect into account in assessing the effectiveness of projects, especially, if the project included weather shocks as a critical risk factor to be carefully monitored.

41 This includes Edu V (Support to Secondary Education V), PBS, LED in the social sector alone; SCPMP in agriculture.

34

Annex B: Evaluation Matrix

Evaluation Question Judgment Criteria Indicators/Notes42

EQ 1. To what extent have the Bank’s country strategies and operations been aligned to Malawi’s development needs and its own strategies and priorities?

JC 1.1. The Bank’s country strategies and key operational mechanisms have been aligned with Malawi’s development needs and development strategies and priorities.

Strategy and target correlation between GoM poverty-reduction strategies and development plans and AfDB CSP for Malawi

AfDB operational policies correlate with defined strategies and plans of GoM

AfDB operational policies support development realities of GoM

JC 1.2. AfDB country strategies and key operational mechanisms are aligned with the needs of main (key) ultimate beneficiaries

Beneficiary impact analyses (or equivalent) inform CSP and intervention design

Beneficiaries are well defined in planning documents

M and E systems of AfDB contain means to measure progress on beneficiary needs

JC 1.3. Bank interventions , when viewed at a sector performance level in the country, are aligned with its country strategy and priorities

Intervention design contains analyses that compare country strategies (CSP) to result framework of intervention

New operations rated satisfactory

EQ 2: To what extent has the design and internal management of the Bank’s strategic direction in Malawi reflected a clear, viable and coherent path for achieving development outcomes in a changing national and international context?

JC 2.1. The CSP meets all quality standards of both the Bank and GoM

AfDB guidelines on CSP quality standards exist

Mechanisms for monitoring and judging quality

Correlation between CSPs and Bank guidelines

Correlation between CSPs and policies of the Bank not covered in Guidelines Note: Quality standards were defined in a way that referred to the requirements set out in Bank policies and guidelines. If these are unclear or not comprehensive, these were not in the appropriate documents.

JC 2.2. The Bank has applied selectivity in designing its country portfolio including a focus on areas where it brings added value?

Bank’s own definition of what it considers to be its domains where it has added value

Average CSP rating (from Bank Readiness review)

JC 2.3. The Bank has adapted its approach to the country’s context and development challenges/needs?

Examples of where the Bank has specifically adapted its way of working to take Malawi’s specific context into account.

Examples where GoM believes that the Bank has not been flexible and adaptable.in its approaches

Existence of Country Portfolio Performance Reviews

Follow up of CPPR

42 To facilitate reading, it is understood that each JC that requires a sector analysis will contain specific indicators to that effect.

35

Evaluation Question Judgment Criteria Indicators/Notes42

JC 2.4. Taken as a whole, the Bank’s interventions have been coherent (between interventions) and were well-coordinated internally

Interventions are sufficient to contribute to the realization of GoM objectives

Internal strategies and plans are documented for sector and inter-sector leverage (e.g. rural roads having effects on agriculture, social services, trade etc.?)

Internal strategies are documented for using one intervention to leverage another in the same sector

EQ 3: To what extent has the Bank contributed to the achievement of development objectives and results?

JC 3.1. The Bank’s interventions achieved their expected results (intermediate outcomes and outcomes)

Expected effects from each intervention stated, monitored and achieved

Specific sector targets for Bank defined43: Transport, Power, Agriculture, Education, Governance, Regional integration.

Note: sector-specific indicators will be added to this list. For example, in education the following might be used: - % students that graduate from Technical, Entrepreneurial and Vocational Education

and Training (TEVET) institutions find jobs in their field - % women that graduate from TEVET institutions - TVET training meets expectations of future employers.

JC 3.2. The Bank’s interventions benefited target group members44?45

Monitoring reports from the interventions

JC 3.3. The Bank’s interventions contributed to the achievement of higher-level (outcomes and impacts) development objectives, both intended and unintended.

AfDB “Completed operations rated as satisfactory” index

Ex post evaluations of interventions

EQ 4: Is it likely that the benefits realized as a result of Bank interventions will

JC 4.1 Benefits gained from past interventions (i.e. completed projects have continued to exist in the long-term (five years or more) even though the Bank’s interventions are completed

Existence of intervention and sector sustainability plans

Clear definition as to accountability for sustainability

Follow-up (M and E) actions and execution of sustainability commitments, including financial,

Evidence of internal sustainability support

43 Refer to p. 12-17 of Results Measurement Framework-Indicators, 2013 44 It is understood that the Team interpreted “target group” and “beneficiaries” in a way that does not include the infrastructures, officials, systems and processes that generate service delivery. Teachers, for example, are not targets; students are. Farmers are targets, but the Agriculture ministry is not. The only exception to this interpretation is where the intervention results framework specifically stated otherwise. 45 The difference between beneficiaries and targets in existing documents was not always clear. For the purpose of this evaluation, “Beneficiaries” are any individuals, groups or organisations that will reap some form of benefit from Bank activities in Malawi or in the region. “Targets” are those individuals, groups or organisations whose problems the intervention were specifically designed to resolve. Targets are subsets of beneficiaries and are specifically identified from the outset in the reports generated by the evaluation.

36

Evaluation Question Judgment Criteria Indicators/Notes42

continue to exist once its support is terminated?

Sustainability of climate and biodiversity based on solid science at individual and sector levels

JC 4.2 Benefits gained from ongoing interventions will likely continue to exist once the Bank’s interventions are completed

Existence of intervention and sector sustainability plans

Clear definition as to accountability for sustainability

Follow-up (M and E) and execution of sustainability commitments, including financial.

Existence of sustainability actions in place at least 48 months before end of intervention

Completed Operations with Sustainable Outcomes Rating (COSOR)

Analysis and follow-up of COSOR

EQ 5: To what extent are key cross-cutting policy objectives of the Bank (in terms of inclusiveness46 and a sustainable environment) mainstreamed within the Bank’s interventions?

JC 5.1. The Bank’s interventions have been inclusive (i.e., bringing prosperity by expanding the economic base across the barriers of age and gender.

Because the evaluation dealt with a number of sectors, each with its own set of sector performance indicators, a sampling of the type of indicators that were used for this JC is included below. We assumed that all indicators dealt with the AfDB’s CCO, and not those of the GoM.

Life expectancy

Enrolment in education (gross and net enrolment ratios).47 Graduation ratios

Enrolment and graduation rates in TVET institutions, as a share of total students enrolled in primary or secondary institutions.

Unemployment rates disaggregated, by various classifications such as looking or not for work, age brackets, gender, etc.

Women’s and youth participation in the labor market.

Average salaries by gender and age

Gender-sensitive Country Institutions Index 48 and its component parts,

Access to finance for business

Gender-informed design in results framework

New projects with gender informed design (with sub indicators)

46 ‘Inclusiveness” is defined as in the Bank’s “One Bank Results Measurement Framework 2013-2016” 47 Part of UN Human Development Index 48 Of the OECD

37

Evaluation Question Judgment Criteria Indicators/Notes42

JC 5.2. The Bank’s interventions have been inclusive (i.e., expanding the economic base across all regions where poverty is most prevalent.

Access to basic services including water sources; improved sanitation facilities; education facilities and services;

Improvements in population in GDP per capita; Population living below poverty line; Gini index calculations or equivalent;

Access to power; Access to markets and services through road networks (road density as a proxy)

Share of services in rural versus urban contexts

Percentage of people below poverty line

Country Policy and Institutional Assessments ratings

Access to finance (adults that have accounts in financial institutions);49

JC 5.3. The Bank’s interventions are managed in a manner that will ensure that they are environmentally sustainable and/or support the transition to green growth.

Depending on the sector or the specific cross-cutting issue, the following indicators were retained

Food insecurity: Percentage of population at risk of caloric inadequacy50

Resilience to water shocks 51

Environmental management plans included in all stages of project planning and contracting

Reporting on environmental risk mitigation plans

Environmental monitoring by intervention steering committee (refer to agendas and minutes)

Independent audits or monitoring of environmental plans within interventions

Capability assessments for environmental stewardship (policy) and operational management (interventions)

Documented tie-in between interventions and green growth strategies and plans

Agricultural value-added per agricultural worker as a proxy for green growth

New projects with climate-informed design (with sub-indicators for share of projects that included satisfactory actions to mitigate against impacts of climate change, climate variability, extreme weather events and biodiversity loss

Premiums for Interventions that actually promote green growth

49 Global Findex Database for example 50 Millennium Dev Goal Indicator for goal 1, target c (FAO) 51 FAO Aquastat

38

Evaluation Question Judgment Criteria Indicators/Notes42

EQ 6: To what extent have the Bank’s interventions in Malawi been successful in economically converting resources into expected results in a timely manner?

JC 6.1 Resources and inputs provided to GoM through the Bank’s support are converted to results at a rate that reflects the assumptions in planning documents

Development resources recorded on budget52

Operations no longer at-risk ratio

Operations at risk

Cost of transforming inputs into outputs.

6.2 The Bank’s interventions are implemented in a timely manner (from the perspective of the beneficiary or implementing agency) and in compliance with the Bank’s operational standards

Predictable disbursements

Use of country systems (PFM and procurement)

Time to first disbursement

Disbursement ratio of ongoing portfolio

Time for procurement of goods and works

Time for approving operations

Time from identification to first disbursement

Extent of projects managed entirely from field

Administrative cost per UA 1 million planned

Cost of preparing lending project or grant project

Cost of supporting project planning

Cost of supporting project implementation

JC 6.3 The GoM and intervention partners and stakeholders meet their commitments as planned (ex. delivery of national commitments/contributions and short comings in performance by national partner(s)

Mention of non-compliance in steering and management meetings

Intervention reports and evaluations are positive concerning non-Bank contributions and compliance.

EQ 7: To what extent has the Bank been successful in influencing policy through dialogue supported by the provision of expert advice and analysis?

JC 7.1. The Bank has actively engaged in and influenced PD through the provision of relevant advice that has been incorporated into policy decisions

Strategic nature of PD

Strategy documents showing the Bank’s PD plan

Diversity of PD effort, geared to objectives

Perception of usefulness of PD by cooperation partners

Perception of PD by GoM officials

Examples of effects of PD

GoM capability to use PD as a strategic tool

JC 7.2 The Bank provided appropriate and adequate analytical work in support of its

New economic and sector papers produced using intervention modality

New economic and sector papers produced using existing Bank personnel modality

52 Global Partnership Monitoring Framework Indicator 6

39

Evaluation Question Judgment Criteria Indicators/Notes42

interventions, positioning and policy advice?

Perceived value of advice

Capability of GoM to generate evidence-based analysis for policies relevant to Bank

EQ 8: To what extent are the Bank’s interventions coordinated with other donors and stakeholders in a way that improves efficiency and enables complementarity?

JC 8.1. The Bank’s processes and interventions are harmonized with those of other donors (avoiding duplication, simplifying procedures etc.).

Level of complementarity between donors

Joint evaluations and reviews

Joint strategic planning

Joint undertakings managed jointly

Use of non-Bank processes

JC 8.2. The Bank’s interventions and resources are influencing other stakeholders to become involved in the sectors that are the focus of country strategies

Cases where new (non-traditional) sources of funding have been forthcoming based on opportunities with the Bank

Recent changes in membership in sector management fora,

JC 8.3 The Bank’s participation in Malawi is leveraged by other stakeholders with a view to maximizing development effectiveness at both country and regional levels?

Subsidiarity

Joint financing under Bank management

EQ 9: To what extent has the Bank managed its interventions and programs, as well as its own internal processes, in a way that focusses on results53 ?

JC 9.1. The Bank has successfully implemented management systems that focus on results and allow learning from past experience?

Intervention design is RBM

Monitoring systems focus on results

Reporting mechanisms are results based

Oversight is done at result levels

JC 9.2 The Bank has supported the development of national capacities and management systems that focus on results

Results-based capacity development efforts for GoM that are related to outcome improvement

% of GoM partners that have RBM-based internal systems

EQ 10: Does the Bank’s ongoing analyses of its achievement of

JC 10.1. Factors that have positively or negatively influenced the achievement of the development results supported by

Means in place to capture positive and negative factors and store in a KM system

Evidence of use of KM in the design of CSP and interventions

53 The team focused on result levels that are beyond the mere production of outputs, as the objective of the Bank is to contribute to outcomes, not outputs.

40

Evaluation Question Judgment Criteria Indicators/Notes42

development results inform its planning and operations?

Bank Interventions are identified54 and inform55 Bank actions and strategies

JC 10.2. Lessons learned from the Bank’s experience in Malawi are identified and inform its future strategies and programs.

Completed operations with a timely PCR (measure of the percentage of PCR that were submitted within the 12 months following closure date)

Systems allowing research quality access to Lessons Learned

Systems allowing for capture of knowledge

Justifications for CSP that refer to Lessons Learned

54 The team considered only documented evidence as being “identified” 55 This was interpreted as being documented to show that there has been an influence.

41

Annex C: Project List

Sector Dept.

N. Long name Status Sector Name Year of

commitment Duration Net loan (UA)

Disbursement

Type of service provided

AHAI 1

FOOD CRISIS RESPONSE BUDGET SUPPORT PROGRAM

approved Agriculture MINISTRY OF FINANCE 2016 1 12,000,000 0 grant

AHAI 2

SMALLHOLDER IRRIGATION AND VALUE ADDITION PROJECT (SIVAP/FUN

ongoing Agriculture MINISTRY OF AGRICULTURE & FOOD SECURITY

2013 5 29,306,198 15,718,991 grant

AHAI 2

SMALLHOLDER IRRIGATION AND VALUE ADDITION PROJECT (SIVAP/FUN

ongoing Agriculture MINISTRY OF AGRICULTURE & FOOD SECURITY

2013 5 253,000 194,964 study

PIFD 3

FEASIBILITY STUDY ON THE ESTABLISHMENT OF AN AGRICULTURE COO

ongoing Agriculture MINISTRY OF FINANCE 2016 1 365,000 30,689 study

AHAI 4

HUMANITARIAN EMERGENCY ASSISTANCE TO MITIGATE EFFECTS OF 2015

complete Agriculture MINISTRY OF AGRICULTURE & FOOD SECURITY

2016 0,5 740,056 0 grant

AHAI 5

AGRICULTURE DEVELOPMENT PROGRAM - ISP ongoing Agriculture MINISTRY OF AGRICULTURE & FOOD SECURITY

2009 7 15,000,000 14,532,277 loan

AHFR 6

(SCPMP) SMALLHOLDER CROP PRODUCTION AND MARKETING PROJECT (co-financing loan with CARLA project below)

complete Agriculture MINISTRY OF AGRICULTURE & FOOD SECURITY

2006 8,9 14,882,276 14,882,276 loan

AHFR 7

SHIRE VALLEY IRRIGATION PROJECT FEASIBILITY STUDY

ongoing Agriculture Ministry of Irrigation and Water Develop

2014 4 1,414,629 1,100,586 study

AHAI 8

AGRICULTURAL INFRASTRUCTURE AND YOUTH AGRIBUSINESS PROJECT

approved Agriculture MINISTRY OF AGRICULTURE & FOOD SECURITY

2016 5 16,000,000 0 loan

AHAI 9

HUMANITARIAN EMERGENCY ASSISTANCE TO THE 2005 DROUGHT VICTIM

closed Agriculture WORLD FOOD PROGRAM - WFP- MALAWI

2006 1,6 370,028 0 grant

PECG 10

GEF CARLA CLIMATE ADAPTATION FOR RURAL LIVELIHOODS AND AGRICUlture (co-financing grant with SCPMP project above)

closed Agriculture MINISTRY OF IRRIGATION AND WATER DEVELOP

2012 4,5 2,220,167 2,219,716 loan

PICU 11

MALAWI TRUNK ROAD REHABILITATION: BLANTYRE-ZOMBA

complete Transport NATIONAL ROAD AUTHORITY 2009 5,2 19,034,450 19,034,450 loan

PICU 11

MALAWI TRUNK ROAD REHABILITATION: BLANTYRE-ZOMBA

complete Transport NATIONAL ROAD AUTHORITY 2009 6,0 159,222 159,222 loan

PICU 12

MALAWI: MZUZU-NKHATABAY ROAD REHABILITATION PROJECT

ongoing Transport NATIONAL ROAD AUTHORITY 2013 5 21,890,000 4,991,151 loan

PITD 13

MALAWI NACALA RAIL AND PORT VALUE ADDITION PROJECT

complete Transport Malawi Investment and Centre (MITC) (Private sector financing)

2017 1,0 0 0 loan

PICU 14 NACALA ROAD CORRIDOR (MALAWI) - phase I complete Transport NATIONAL ROAD AUTHORITY 2009 6,0 8,925,658 8,925,658 loan

PICU 15

NACALA ROAD CORRIDOR PROJECT PHASE IV (LIWONDE-MANGOCHI) MA

ongoing Transport NATIONAL ROAD AUTHORITY 2014 4 42,360,000 3,313,755 loan

PESD 16

KOLOMBIDZO HYDRO POWER PROJECT FEASIBILITY STUDY

ongoing Power Ministry of Energy Department of Energy

2013 4 2,000,000 1,496,749 study

42

Sector Dept.

N. Long name Status Sector Name Year of

commitment Duration Net loan (UA)

Disbursement

Type of service provided

AHWS 17

MALAWI RWSSI - RURAL WATER SUPPLY AND SANITATION INITIATIVE

complete WATSAN GOVERNMENT OF MALAWI 2008 5,0 15,200,000 14,746,195 loan

AHWS 17

MALAWI RWSSI - RURAL WATER SUPPLY AND SANITATION INITIATIVE

complete WATSAN GOVERNMENT OF MALAWI 2008 5,0 10,728,000 10,661,911 loan

AHWS 17

MALAWI RWSSI - RURAL WATER SUPPLY AND SANITATION INITIATIVE

complete WATSAN GOVERNMENT OF MALAWI 2008 5,0 2,774,708 2,774,708 grant

AHWS 18

SUSTAINABLE RURAL WATER AND SANITATION INFRASTRUCTURE FOR IMPROVED HEALTH AND LIVELIHOODS

ongoing WATSAN MINISTRY OF IRRIGATION AND WATER DEVELOP

2014 5 15,000,000 2,011,450 loan

AHWS 18

SUSTAINABLE RURAL WATER AND SANITATION INFRASTRUCTURE FOR IMPROVED HEALTH AND LIVELIHOODS

ongoing WATSAN MINISTRY OF IRRIGATION AND WATER DEVELOP

2014 5 2,761,407 720,664 grant

AHWS 18

SUSTAINABLE RURAL WATER AND SANITATION INFRASTRUCTURE FOR IMPROVED HEALTH AND LIVELIHOODS

ongoing WATSAN MINISTRY OF IRRIGATION AND WATER DEVELOP

2014 5 5,000,000 1,628,968 loan

AHWS 19

MZIMBA INTEGRATED URBAN WATER AND SANITATION PROJECT

ongoing WATSAN -- 2015 5 10,989,824 0 loan

AHWS 19

MZIMBA INTEGRATED URBAN WATER AND SANITATION PROJECT

ongoing WATSAN -- 2015 5 3,600,000 122,649 loan

AHWS 20

STRENGTHENING WATER SECTOR MONITORING AND EVALUATION

closed WATSAN MINISTRY OF IRRIGATION AND WATER DEVELOP

2010 3,4 1,494,315 1,494,315 grant

AHWS 21

IMPROVING ACCESS TO WATER & SANITATION FOR URBAN POOR IN BLANTYRE

complete WATSAN CCODE CENTER FOR COMMUNITY ORGANIZATION

2010 5,0 481,897 481,897 grant

AHHD 22 JOBS FOR YOUTH MALAWI approved social Ministry of Labor 2017 4 0 0 n.a.

AHHD 22 JOBS FOR YOUTH MALAWI approved social Ministry of Labor 2017 4 0 0 n.a.

AHHD 23

SUPPORT TO SECONDARY EDUCATION (EDUCATION V) PROJECT

closed social IDA/ADF EDUCATION PROJECTS

2006 6,6 14,621,981 14,621,981 grant

AHHD 24

Support to Higher Education, Science and Technology (HEST)

ongoing social GOVERNMENT OF MALAWI 2012 7 6,500,000 2,477,458 loan

AHHD 24

Support to Higher Education, Science and Technology (HEST)

ongoing social GOVERNMENT OF MALAWI 2012 7 10,950,000 5,165,758 grant

AHHD 24

Support to Higher Education, Science and Technology (HEST)

ongoing social GOVERNMENT OF MALAWI 2012 7 9,050,000 4,439,735 loan

AHHD 25

SUPPORT TO THE HEALTH SECTOR PROGRAM complete social MINISTRY OF HEALTH & POPULATION

2006 8,0 14,668,070 14,668,070 grant

AHHD 26

SUPPORT TO LOCAL ECONOMIC DEVELOPMENT

ongoing social MINISTRY OF LOCAL GOVERNMENT & RURAL DE

2008 8 14,000,000 13,968,722 loan

AHHD 27

SUPPLEMENTARY LOAN LOCAL ECONOMIC DEVLOP

ongoing social MINISTRY OF LOCAL GOVERNMENT & RURAL DE

2011 6 3,162,000 3,136,780 loan

43

Sector Dept.

N. Long name Status Sector Name Year of

commitment Duration Net loan (UA)

Disbursement

Type of service provided

AHHD 28

COMPETITIVENESS AND JOB CREATION SUPPORT PROJECT

ongoing social Ministry of Industry and Trade 2012 6 10,000,000 7,318,945 grant

AHHD 29

PROTECTION OF BASIC SERVICES (Health) budget support (ring-fenced)56

ongoing social MINISTRY OF FINANCE 2015 3 19,000,000 19,000,000 grant

ECGF 30

RESTORATION OF FISCAL STABILITY AND SOCIAL PROTECTION

complete multisector MINISTRY OF FINANCE 2012 1,4 26,000,000 26,000,000 grant

ECGF 31

SUPPLEMENTARY BUDGET SUPPORT TO RFSSP

complete multisector MINISTRY OF FINANCE 2013 0,5 4,000,000 4,000,000 grant

ECGF 32

PUBLIC FINANCE MANAGEMENT INSTITUTIONAL SUPPORT PROJECT

ongoing multisector MINISTRY OF FINANCE 2013 5 2,980,000 2,080,181 grant

ECGF 33

PUBLIC FINANCE MANAGEMENT INSTITUTIONAL SUPPORT PROJECT-PHA

ongoing multisector MINISTRY OF FINANCE 2015 3 1,860,000 358,278 grant

ECGF 34 POVERTY REDUCTION SUPPORT LOAN closed multisector MINISTRY OF FINANCE 2007 1,2 14,890,000 14,890,000 loan

ECGF 35 POVERTY REDUCTION SUPPORT GRANT I complete multisector MINISTRY OF FINANCE 2009 0,7 10,000,000 10,000,000 grant

ECGF 36

GOVERNANCE AND POVERTY REDUCTION SUPPORT GRANT II (GPRSG II)

complete multisector MINISTRY OF FINANCE 2010 1,0 11,546,000 11,546,000 grant

56 Malawi had serious balance of payments problems so the Bank’s FOREX helped. The funds are for direct support of operations of sectors.

44

Annex D: Sector Level and Overall Performance

Evaluation Question Judgment Criteria

Soci

al

Tran

spo

rt

Agr

icu

ltu

re

WA

TSA

N

Mu

ltis

ecto

r

Ove

rall

asse

ssm

en

t

EQ 1. To what extent have the Bank’s country strategies and operations been aligned to Malawi’s development needs and its own strategies and priorities?

1.1 The Bank’s country strategies and key operational mechanisms have been aligned with Malawi’s development needs and development strategies and priorities.

3.1 3.3 3.5 3.0 3.5 Satisfactory

1.2 AfDB country strategies and key operational mechanisms are aligned with the needs of main (key) ultimate beneficiaries.

3.8 2.9 3.6 2.7 3.5 Satisfactory

1.3 Bank interventions, when viewed at a sector performance level in the country, are aligned with its country strategy and priorities.

3.9 3.0 3.7 3.0 2.0 Satisfactory

EQ 2. To what extent has the design and internal management of the Bank’s strategic direction in Malawi reflected a clear, viable and coherent path for achieving development outcomes in a changing national and international context?

2.1 The CSP meets all quality standards of both the Bank and GoM.

3.0 3.0 3.0 2.5 4.0 Satisfactory

2.2 The Bank has applied selectivity in designing its country portfolio including a focus on areas where it brings added value.

3.2 3.0 3.2 3.5 2.0 Satisfactory

2.3 The Bank has adapted its approach to the country’s context and development challenges/needs.

2.5 2.5 3.0 2.5 3.0 Unsatisfactory

2.4 Taken as a whole, the Bank’s interventions have been coherent (between interventions) and were well-coordinated internally.

3.0 3.3 3.0 3.3 3.0 Satisfactory

EQ 3. To what extent has the Bank contributed to the achievement of development objectives and results?

3.1 The Bank’s interventions achieved their expected results (intermediate outcomes and outcomes).

2.8 3.1 2.5 3.5 2.0 Satisfactory

3.2 The Bank’s interventions benefited target group members. 3.3 3.5 2.5 3.0 NA Satisfactory

3.3 The Bank’s interventions contributed to the achievement of higher level (outcomes and impacts) development objectives, both intended and unintended.

3.4 2.5 2.5 2.5 2.0

Unsatisfactory

45

Evaluation Question Judgment Criteria

Soci

al

Tran

spo

rt

Agr

icu

ltu

re

WA

TSA

N

Mu

ltis

ecto

r

Ove

rall

asse

ssm

en

t

EQ 4. Is it likely that the benefits realized as a result of Bank interventions will continue to exist once its support is terminated?

4.1 Benefits gained from past interventions (i.e. completed projects have continued to exist in the long-term (five years or more) even though the Bank’s interventions are completed.

2.3 2.3 2.5 2.2 3.0 Unsatisfactory

4.2 Benefits gained from ongoing interventions will likely continue to exist once the Bank’s interventions are completed.

3.0 2.8 2.5 2.4 2.0 Unsatisfactory

EQ 5. To what extent are key crosscutting policy objectives of the Bank (in terms of inclusiveness and a sustainable environment) mainstreamed within the Bank’s interventions?

5.1 The Bank’s interventions have been inclusive (i.e., bringing prosperity by expanding the economic base across the barriers of age and gender.

3.6 3.0 3.8 4.0 3.0 Satisfactory

5.2 The Bank’s interventions have been inclusive (i.e., expanding the economic base across all regions where poverty is most prevalent.

3.8 3.0 3.0 3.0 NA Satisfactory

5.3 The Bank’s interventions are managed in a manner that will ensure that they are environmentally sustainable and/or support the transition to green growth.

3.0 4.0 3.0 3.0 3.0 Satisfactory

EQ 6. To what extent have the Bank’s interventions in Malawi been successful in economically converting resources into expected results in a timely manner?

6.1 Resources and inputs provided to GoM through the Bank’s support are converted to results at a rate that reflects the assumptions in planning documents.

3.2 2.5 3.0 3.0 2.5 Satisfactory

6.2 The Bank’s interventions are implemented in a timely manner (from the perspective of the beneficiary or implementing agency) and in compliance with the Bank’s operational standards.

3.7 2.8 2.7 2.5 2.5 Unsatisfactory

6.3 The GoM and intervention partners and stakeholders meet their commitments as planned (ex. delivery of national commitments/contributions and short comings in performance by national partner(s).

2.5 2.3 3.0 2.5 3.0 Unsatisfactory

EQ 7. To what extent has the Bank been successful in influencing policy through

7.1 The Bank has actively engaged in and influenced PD through the provision of relevant advice that has been incorporated into policy decisions.

3.7 3.0 3.4 3.0 3.0 Satisfactory

46

Evaluation Question Judgment Criteria

Soci

al

Tran

spo

rt

Agr

icu

ltu

re

WA

TSA

N

Mu

ltis

ecto

r

Ove

rall

asse

ssm

en

t

dialogue supported by the provision of expert advice and analysis?

7.2 The Bank provided appropriate and adequate analytical work in support of its interventions, positioning and policy advice.

3.6 3.3 3.5 3.0 3.0 Satisfactory

EQ 8. To what extent are the Bank’s interventions coordinated with other donors and stakeholders in a way that improves efficiency and enables complementarity?

8.1 The Bank’s processes and interventions are harmonized with those of other donors (avoiding duplication, simplifying procedures etc.).

3.2 3.0 3.0 2.7 3.5 Satisfactory

8.2 The Bank’s interventions and resources are influencing other stakeholders to become involved in the sectors that are the focus of country strategies.

3.0 3.0 3.0 3.0 3.0 Satisfactory

8.3 The Bank’s participation in Malawi is leveraged by other stakeholders with a view to maximizing development effectiveness at both country and regional levels.

3.3 3.0 3.5 3.0 2.0 Satisfactory

EQ 9. To what extent has the Bank managed its interventions as well as its own internal processes, in a way that focusses on results?

9.1 The Bank has successfully implemented management systems that focus on results and allow learning from past experience.

2.4 2.8 3.0 2.5 2.0 Unsatisfactory

9.2 The Bank has supported the development of national capacities and management systems that focus on results.

2.0 3.0 3.0 2.5 2.0 Unsatisfactory

EQ 10. Does the Bank’s ongoing analyses of its achievement of development results inform its planning and operations

10.1 Factors that have positively or negatively influenced the achievement of the development results supported by Bank Interventions are identified and inform Bank actions and strategies.

3.2 2.6 3.0 2.5 3.0 Satisfactory

10.2 Lessons learned from the Bank’s experience in Malawi are identified and inform its future strategies and programs.

3.7 2.8 3.7 3.0 3.0 Satisfactory

47

Annex E: Malawi – Comparative Socioeconomic Indicators

Year Malawi Africa

Develo-

ping

Countries

Develo-

ped

Countries

Basic Indicators

Area ( '000 Km²) 2017 118 30,067 80,386 53,939Total Population (millions) 2017 18.3 1,184.5 5,945.0 1,401.5Urban Population (% of Total) 2017 16.7 39.7 47.0 80.7Population Density (per Km²) 2017 194.1 40.3 78.5 25.4GNI per Capita (US $) 2016 320 2 045 4 226 38 317Labor Force Participation *- Total (%) 2017 81.1 66.3 67.7 72.0Labor Force Participation **- Female (%) 2017 81.5 56.5 53.0 64.5Sex Ratio (per 100 female) 2017 99.8 0.801 0.506 0.792Human Dev elop. Index (Rank among 187 countries) 2015 170 ... ... ...Popul. Liv ing Below $ 1.90 a Day (% of Population) 2010 70.9 39.6 17.0 ...

Demographic Indicators

Population Grow th Rate - Total (%) 2017 3.1 2.6 1.3 0.6Population Grow th Rate - Urban (%) 2017 4.1 3.6 2.6 0.8Population < 15 y ears (%) 2017 44.7 41.0 28.3 17.3Population 15-24 y ears (%) 2017 20.4 3.5 6.2 16.0Population >= 65 y ears (%) 2017 3.4 80.1 54.6 50.5Dependency Ratio (%) 2017 92.9 100.1 102.8 97.4Female Population 15-49 y ears (% of total population) 2017 23.3 24.0 25.8 23.0Life Ex pectancy at Birth - Total (y ears) 2017 65.5 61.2 68.9 79.1Life Ex pectancy at Birth - Female (y ears) 2017 66.4 62.6 70.8 82.1Crude Birth Rate (per 1,000) 2017 37.7 34.8 21.0 11.6Crude Death Rate (per 1,000) 2017 6.9 9.3 7.7 8.8Infant Mortality Rate (per 1,000) 2016 38.9 52.2 35.2 5.8Child Mortality Rate (per 1,000) 2016 55.1 75.5 47.3 6.8Total Fertility Rate (per w oman) 2017 4.9 4.6 2.6 1.7Maternal Mortality Rate (per 100,000) 2015 634.0 411.3 230.0 22.0Women Using Contraception (%) 2017 59.8 35.3 62.1 ...

Health & Nutrition Indicators

Phy sicians (per 100,000 people) 2009 1.8 46.9 118.1 308.0Nurses and midw iv es (per 100,000 people) 2009 33.6 133.4 202.9 857.4Births attended by Trained Health Personnel (%) 2016 89.8 50.6 67.7 ...Access to Safe Water (% of Population) 2015 90.2 71.6 89.1 99.0Access to Sanitation (% of Population) 2015 41.0 51.3 57 69Percent. of Adults (aged 15-49) Liv ing w ith HIV/AIDS 2016 9.2 39.4 60.8 96.3Incidence of Tuberculosis (per 100,000) 2016 159.0 3.8 1.2 ...Child Immunization Against Tuberculosis (%) 2016 86.0 245.9 149.0 22.0Child Immunization Against Measles (%) 2016 81.0 84.1 90.0 ...Underw eight Children (% of children under 5 y ears) 2014 16.7 76.0 82.7 93.9Prev alence of stunding 2014 42.4 20.8 17.0 0.9Prev alence of undernourishment (% of pop.) 2015 25.9 2 621 2 335 3 416Public Ex penditure on Health (as % of GDP) 2014 6.0 2.7 3.1 7.3

Education Indicators

Gross Enrolment Ratio (%)

Primary School - Total 2016 139.3 106.4 109.4 101.3 Primary School - Female 2016 141.3 102.6 107.6 101.1 Secondary School - Total 2016 37.4 54.6 69.0 100.2 Secondary School - Female 2016 35.3 51.4 67.7 99.9Primary School Female Teaching Staff (% of Total) 2015 42.0 45.1 58.1 81.6Adult literacy Rate - Total (%) 2015 62.1 61.8 80.4 99.2Adult literacy Rate - Male (%) 2015 69.8 70.7 85.9 99.3Adult literacy Rate - Female (%) 2015 55.2 53.4 75.2 99.0Percentage of GDP Spent on Education 2016 4.7 5.3 4.3 5.5

Environmental Indicators

Land Use (Arable Land as % of Total Land Area) 2015 40.3 8.6 11.9 9.4Agricultural Land (as % of land area) 2015 61.4 43.2 43.4 30.0Forest (As % of Land Area) 2015 33.4 23.3 28.0 34.5Per Capita CO2 Emissions (metric tons) 2014 0.1 1.1 3.0 11.6

Sources : AfDB Statistics Department Databases; World Bank: World Development Indicators; last update :

UNAIDS; UNSD; WHO, UNICEF, UNDP; Country Reports.

Note : n.a. : Not Applicable ; … : Data Not Available. * Labor force participation rate, total (% of total population ages 15+)

** Labor force participation rate, female (% of female population ages 15+)

MalawiCOMPARATIVE SOCIO-ECONOMIC INDICATORS

May 2018

0

20

40

60

80

100

120

20

00

20

05

20

10

20

11

20

12

20

13

20

14

20

15

20

16

Infant Mortality Rate( Per 1000 )

Mal awi A frica

0

500

1000

1500

2000

2500

20

00

20

05

20

10

20

11

20

12

20

13

20

14

20

15

20

16

GNI Per Capita US $

Mal awi A frica

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

20

00

20

05

20

10

20

12

20

13

20

14

20

15

20

16

20

17

Population Growth Rate (%)

Mal awi A frica

01020304050607080

20

00

20

05

20

10

20

12

20

13

20

14

20

15

20

16

20

17

Life Expectancy at Birth (years)

Mal awi A frica

48

Bibliography

African Development Bank Strategic Documents

African Development Bank, African Development Fund, Malawi 2005-2009 Country Strategy, September 2005 African Development Bank, African Development Fund, Malawi Results Based Country Strategy Paper: 2005-

2009, Mid-Term Review Report, April 2008 African Development Bank Group, African Development Bank Group and Malawi. Thirty years of Fruitful

Cooperation, November 2008 African Development Bank Group, Malawi Interim Country Strategy Paper (ICSP) 2011-2012, March 2011 African Development Bank, African Development Fund, Southern Africa Regional Integration Strategy Paper

2011-2015 African Development Bank Group, Combined Mid-Term Review and Regional Portfolio Performance Review of

The Regional Integration Strategy Paper for West Africa 2011 – 2015, September 2014 African Development Bank Group, Malawi Country Strategy Paper 2013-2017, ORSB DEPARTMENT 2015-2016 Budget Statement, Ministry of Finance 2016-2017, Budget Speech Malawi Country Economic Memorandum: Seizing Opportunities for Growth through Trade; Volume I: Summary

of Main Findings and Recommendations Malawi Country Economic Memorandum: Seizing Opportunities for Growth through Trade; Volume II:

Background Papers - Constraints on Growth in Malawi - Malawi’s Growth and Diversification - Quantitative Value Claim Analysis - Challenges in Malawi’s Agribusiness - Connecting Malawi to Markets to Compete - Infrastructure Investment in Malawi: Insights form a Spatial Analysis - The Energy Constraint - Key Messages Main Assessment Report on Country National Competitive Bidding Procedures, OPRF, Aug. 2011 Critical Assessment Table for use of Borrower’s Procurement System in Bank Financial Operations: Cover Note:

AfDB, May 2016

Bank Corporate documents

African Development Bank, African Development Fund, Evaluation of Paris Declaration Implementation at the African Development Bank, May 2011

African Development Bank Group, The One Bank Results Measurement Framework 2013-2016, December 2013 African Development Bank Group, At the Center of Africa’s Transformation. Strategy for 2013–2022, Executive

summary, 2013 African Development Bank Group, At the Center of Africa’s Transformation. Strategy for 2013–2022, 2013 African Development Bank Group, Annual Development Effectiveness Review 2015. Driving development

through innovation, May 2015 African Development Bank Group, IDEV Annual Report 2015. Strengthening Impact, April 2016 African Development Bank Group, Comprehensive Evaluation of the Development Results of the African

Development Bank Group 2004-2013. Synthesis Report, October 2016 Amendments to the Approved Development and Business Delivery Model (DBDM) October 2016 Operations Evaluation department: “Fostering Regional Integration in Africa: An Evaluation of the Bank’s

Multinational Operations 2000-2010” Bank Publication, 2012 Operations Evaluation department: “Country Assistance Evaluation, Gabon, 1996-2008” Bank Publication, 2011 Operations Evaluation department: “Morocco: Evaluation of Bank Assistance to the Education Sector”” Bank

Publication, 2005

Government of Malawi Strategic Documents

Government of Malawi, Malawi Growth and Development Strategy. From Poverty to Prosperity 2006-2011 Government of Malawi, Malawi Growth and Development Strategy II 2011-2016 Public Finance Management Assessment for Malawi based on PEFA, June 2008

49

Public Finance Management Assessment for Malawi based on PEFA, March 2011 Government of Malawi, Budget Speech 2015/16 Government of Malawi, Draft Training Needs Assessment Report for the Procurement Cluster, March 2011, EU /

Government of Malawi Capacity assessment and development report: Capacity in Public Financial Management and Accountability, 15

August 2008, UNDP/Government of Malawi Malawi Development Assistance Strategy 2006—2011, Government of Malawi, September 2007 PFEM Situation Analysis Report, June 2010, PFEM Unit, Ministry of Finance Revised Priority Public Financial and Economic Management Action Plan, September 2010-July 2011, PFEM

Secretariat/Ministry of Finance, September 2010 Public Financial and Economic Management Reform Program, PFEM RP, July 2011-June 2014, Ministry of

Finance, May 2011 Public Financial Management Strategy and Action Plan 2014-17, Ministry of Finance, June 2014 THE PUBLIC FINANCE MANAGEMENT REFORM PROGRAM, “Restoring Financial Control and Accountability” -

2015, Lilongwe, February 2015, Government of Malawi Draft: A Synopsis of the Revised Public Finance Management Reform Program and a Revised Engagement

Framework with Budget Support DP, Government of Malawi, October 2014 Draft report: Malawi Public Financial Management Strategy and Action Plan, June 2014 Statement by His Excellency, Prof. Arthur Peter Mutharika, President of the Republic Of Malawi at the Launch of

the Public Finance Management Reform Program, April, 2015. Progress Report: Extraordinary Performance Assessment Framework, March 2014. Progress Report: PFM Action Plan, March 2016 Zero draft, PFM Rolling Plan 2017-22, Ministry of Finance, 2017

African Development Bank Sector Documents

SOCIAL SECTOR

Social Sector - Appraisal Reports African Development Fund, Support to the Health Sector Program. Appraisal Report, ONSD, September 2005 African Development Bank Group, Support to Secondary Education (Education V) Project. Appraisal Report,

ONSD 1, February 2006 African Development Bank Group, Support to Local Economic Development Project. Project Appraisal Report,

July 2008 African Development Fund, Supplementary Loan for Support to Local Economic Development Project. Project

Appraisal Report, September 2010 African Development Bank Group, Support to Higher Education, Science and Technology (HEST). Project

Appraisal Report, December 2011 African Development Fund, Malawi Protection of Basic Services. Appraisal Report, OSHD/SARC, April 2015 Social Sector-PCR

African Development Bank Group, Project Completion Report for Malawi: Support to Community Day Secondary Schools (Education IV) Project

African Development Bank Group, Project Completion Report for Public Sector Operations (PCR). Support to the Health Sector Program

Social Sector-Other Intervention Documents

Implementation Progress and Results Reports: various, including PBS and HEST Mid-term Review of Education V, Chimombo, L.J. “Quantity versus Quality: Case Studies in Malawi”, International Review of Education, 2005:51 Kayuni, H., and Tambulasi, R., “Teacher Turnover in Malawi’s Ministry of Education: Realities and Challenges”,

International Education Journal, April 2007 Green LW, Kreuter MW. Health Promotion Planning: An Educational and Environmental Approach. 2nd ed.

Mountain View: Mayfield; 1991. Freudenberg N, Eng E, Flay B, Parcel G, Rogers T, Wallerstein N. Strengthening individual and community capacity

to prevent disease and promote health: In search of relevant theories and principles. Health Education Quarterly. 1995; 22:290–306. [PubMed]

50

Lewis SJK. An overview of health education activities and proposals for strengthening of health education services in Malawi. Lilongwe: Ministry of Health, Malawi; 1981.

Bomba WG. Strengthening of I.E.C & Social Mobilization Unit of Ministry of Health. Lilongwe: Ministry of Health; 1990.

Jiyani E. Paper presented at the peer education national development strategy workshop in Zomba, Malawi from 31st January to 1st February 2000. Lilongwe: Ministry of Gender Youth and Community Services; 2000. Peer education concept paper.

Southern Africa Regional LED Conference paper: “Creating Opportunities for Developmental Local Government through Regional Engagement on LED” March 2014, Lilongwe

AfDB Sector Studies

African Development Bank Fund, Multi-Sector Country Gender Profile, October 2005 African Development Bank Group, Enhancing Linkages with Vertical Health Funds Country Case Study: Malawi,

July 2009 ESMP

Malawi: Support to Higher Education, Science and Technology (HEST) Project. Environmental and Social Management Plan (ESMP)

TRANSPORT SECTOR

Implementation Progress and Results Report (IPR), Mzuzu-Nkhata Bat Road Rehabilitation Project, 10/10/2016 An Evaluation of Environmental Mainstreaming in AfDB Support to the Road Transport Sub-Sector, OPEV, June

2010 The Mchenga Urban Poor Fund, CCODE Inception Report: Data Collection on Nacala Corridor Integrated Development in Southern Africa, JICA, March

2017 Project Completion Report (PCR): Multinational Nacala Road Corridor Development Project: Phase I, 31/10/2016 Project Completion Report (PCR): Malawi Trunk Roads Rehabilitation Project (Blantyre-Zomba), 03/10/2016 Main Assessment Report on Country National Competitive Bidding Procedures, OPRF, Aug. 2011 Combined Mid-Term Review (MTR) of CSP 2013 – 2017 and Country Portfolio Performance Review 2015,

18/10/2016 Critical Assessment Table for use of Borrower’s Procurement System in Bank Financial Operations: Cover Note:

AfDB, May 2016 Completion Report: RBCSP 2005-2010, June 2010 Project Appraisal Report (PAR): Nacala Corridor Project Phase III, Sept. 2012 Multinational Nacala Road Corridor Development Project: Phases I, II, III and IV – Progress on roadworks. Guidelines for Preparation of a PCR to be submitted by the recipient to the AWF, AfDB: AWF Manual 8.1 Water Supply and Sanitation in Africa: Findings, Lessons and Good Practices to Improve Delivery, AfDB, June 2015 Preparation Mission Aide Memoire: Mzuzu-Nkhata Bay Road Rehabilitation Project: May/June 2012 Project Concept Note: Mzuzu-Nkhata Bay Road Rehabilitation Project: June 2012 Counterpart Funding Waiver: Mzuzu-Nkhata Bay Road Rehabilitation Project: July 2012 Project Appraisal Report (PAR): Mzuzu-Nkhata Bay Road Rehabilitation Project: Oct 2012 Aide Memoire: AfDB Transport Sector Supervision Mission, Feb/March 2017 Five Year Strategic and Business Plan, Vol. 1, Roads Authority, 2011-2016 Aide Memoire: AfDB Transport Sector Supervision Mission, Aug 2016 Aide Memoire: AfDB Transport Sector Supervision Mission, Nov 2015 Aide Memoire: AfDB Transport Sector Supervision Mission, Jan 2014 Aide Memoire: AfDB Transport Sector Supervision Mission, Aug 2014 Aide Memoire: Supervision Mission of Malawi Trunk Roads Rehabilitation Project and Launch: Mzuzu-Nkhata

Bay Road Rehabilitation Project, July 2014 Aide Memoire: AfDB Transport Sector Supervision Mission, April 2012 Aide Memoire: Appraisal Mission: Mzuzu-Nkhata Bay Road Rehabilitation Project, Aug. 2012 Aide Memoire: AfDB Transport Sector Supervision Mission, September 2011 Back-to-Office Report (BTOR): Preparation Missions, Nacala Corridor Development Project, Feb/March 2013 Back-to-Office Report (BTOR): Transport Sector Supervision Mission, Sept. 2011 Back-to-Office Report (BTOR): Transport Sector Supervision Mission, Feb 2013 Back-to-Office Report (BTOR): Transport Sector Supervision Mission, April 2012

51

Back-to-Office Report (BTOR): Multinational Road Corridor Development Project Phase IV: SOBP Launch Malawi/Zambia, April 2013

Back-to-Office Report (BTOR): Transport Sector Supervision Mission, July 2013 Back-to-Office Report (BTOR): Transport Sector Supervision Mission, Sept. 2011 Back-to-Office Report (BTOR): Nacala Development Corridor OSBP Program, Project Steering Committee, June

2014 Internal Memorandum: Appraisal of Mzuzu-Nkhata Bay Road Rehabilitation Project, Aug 2012 Project Completion Report (PCR): Strengthening Water Sector M&E Project, MWDI Progress Report No. 7, Oct – Dec 2016, Sustainable Rural Water and Sanitation Infrastructure for Improved

Health and Livelihoods, MAIWD Progress Report Nr. 6, July – Sept 2016, Sustainable Rural Water and Sanitation Infrastructure for Improved

Health and Livelihoods, MAIWD Unlocking Human Development Outcomes through WATSAN Interventions in Africa: An Impact Evaluation from

Geospacial Data in Zambia and Malawi, AfDB National Transport Policy, MTPW Road Authority

Roads Authority, Government of Malawi, Resettlement Action Plan (Rap) Draft for Repugrading of Thensipe- Liwonde- Mangochi Road, vol. II, July 2013

African Development Bank Group, Environmental and Social Impact Assessment & Resettlement Action Plan Summary. Nacala Road Corridor Project Phase IV Project (Nsipe-Liwonde- Mangochi Road Section), August 2013

Roads Authority, Annual Report 2011 Roads Authority, Annual Report 2012 The Roads Fund, Summarized Financial Statements for the year ended 30 June 2013 The Roads Fund Administration, Strategic Plan July 2009- June 2014

WATSAN SECTOR

Project Completion Report (PCR): Improving Access to WATSAN for the Urban Poor in the City of Blantyre Water Supply and Sanitation in Africa: Findings, Lessons and Good Practices to Improve Delivery, AfDB, June 2015 Project Completion Report (PCR): Strengthening Water Sector Monitoring and Evaluation Project, MWDI Progress Report No. 7, Oct – Dec 2016, Sustainable Rural Water and Sanitation Infrastructure for Improved

Health and Livelihoods, MAIWD Progress Report No. 6, July – Sept 2016, Sustainable Rural Water and Sanitation Infrastructure for Improved

Health and Livelihoods, MAIWD Project Completion Report (PCR): Strengthening Water Sector M&E Project, MWDI Unlocking Human Development Outcomes through WATSAN Interventions in Africa: An Impact Evaluation from

Geospacial Data in Zambia and Malawi, AfDB African Development Bank Group, National Water Development Program. Project Appraisal Report, March 2008 African Water Facility, African Development Bank Group, MALAWI Strengthening Water Sector Monitoring and

Evaluation. Project Appraisal Report, August 2009 African Water Facility, African Development Bank, Malawi: Improving Access to Water and Sanitation for the

Urban Poor in the City of Blantyre. Appraisal Report, November 2009 African Development Bank, Progress Report as at 30th June 2011 Rural Water Supply And Sanitation Initiative

(RWSSI) Water And Sanitation Department (OWAS), September 2011 Rural Water Supply and Sanitation Initiative (RWSSI). 2013 Annual Progress Report RWSSI and RWSSI-TF, OWAS,

March 2014 African Development Bank Group, Sustainable Rural Water and Sanitation Infrastructure Project for Improved

Health and Livelihoods. Project Appraisal Report, April 2014 African Development Fund, Mzimba Integrated Urban Water and Sanitation. Appraisal Report, OWAS/GECL,

October 2015 Implementation Progress and Results Report (IPR), Mzuzu-Nkhata Bat Road Rehabilitation Project, 10/10/2016 Appraisal Report: Sustainable Rural Water and Sanitation Infrastructure for Improved Health and Livelihoods,

March 2014 Technical Annexes: Sustainable Rural Water and Sanitation Infrastructure for Improved Health and Livelihoods,

April 2014 Appraisal Report: Mzimba Integrated Urban Water and Sanitation, September 2015

52

Technical Annexes: Mzimba Integrated Urban Water and Sanitation, June 2015 Project Completion Report (PCR): National Water Development Program (NWDP), 15/04/2015 Comments and Responses to PCR, National Water Development Program (NWDP), 01/02/2016 Case Study: Strengthening Water Sector Monitoring and Evaluation Project, March 2015 Quarterly Progress Report: Mzimba Integrated Urban Water and Sanitation, Oct-Dec 2016 The Mchenga Urban Poor Fund, CCODE Completion Report: RBCSP 2005-2010, June 2010

AGRICULTURE SECTOR

African Development Fund, Smallholder Crop Production and Marketing Project (SCPMP). Appraisal Report, May 2006

African Development Bank Group, Agriculture Infrastructure Support Project (AISP). Appraisal Report, June 2009 African Development Bank Group, Climate Adaptation for Rural Livelihood and Agriculture (CARLA). Appraisal

Report, 2010 African Development Bank Group, Shire Valley Irrigation Project (SVIP). Appraisal Report, 2013 African Development Bank Group, Smallholder Irrigation and Value Addition Project (SIVAP). Appraisal Report,

2013 African Development Bank, Feasibility Study on the Establishment of an Agriculture Cooperative Bank; Project

Preparation Financing mechanism (PPF), 2015 African Development Bank, Proposal for a Grant of US$1 million for Emergency Humanitarian Assistance to

Mitigate the Effects of the 2015 Drought, 2016 African Development Bank Group, Agriculture Infrastructure and Youth in Agribusiness Project (AIYAP). Appraisal

Report, 2016 AfDB Sector Studies African Development Bank Group, Agriculture Sector Strategy 2010-2014, 2010 African Development Bank Group, Feed Africa: Strategy for Agricultural Transformation in Africa 2016-2025,

2016 Mid-Term Review African Development Bank Group, Smallholder Crop Production and Marketing Project (SCPMP). Mid-Term

Review Report, 2010 African Development Bank Group, Agriculture Infrastructure Support Project (AISP). Mid-Term Review Report,

2013 African Development Bank Group, Smallholder Crop Production and Marketing Project (SCPMP). Project

Completion Report, September 2013 African Development Bank Group, Climate Adaptation for Rural Livelihood and Agriculture (CARLA). Project

Completion Report, December 2016

MULTISECTOR

Appraisal Report: Poverty Reduction Support Loan, February 2007 Project Completion Report: Poverty Reduction Support Loan, January 2009 Appraisal Report: Poverty Reduction Support Grant I (PRSG I), January 2009 Aide Memoire: Appraisal Mission for the Poverty Reduction Support Grant (PRSG I), December 2008 Project Completion Report: Poverty Reduction Support Grant I (PRSG I), undated Project Concept Note: Governance and Poverty Reduction Support Grant II (GPRSG II), October 2009 Aide Memoire: Governance Poverty Reduction Support Grant II: Appraisal Mission to Malawi, November 2-13,

2009 Appraisal Report: Governance and Poverty Reduction Support Grant II (GPRSG II), March 2010 Aide Memoire: Governance Poverty Reduction Support Grant II: Supervision Mission to Malawi, August 2010 Back-to-Office Report: Governance Poverty Reduction Support Grant II: Supervision Mission to Malawi, August

2010 Back-to-Office Report: Governance Poverty Reduction Support Grant II: Supervision Mission to Malawi, March

2011 Aide Memoire: Governance Poverty Reduction Support Grant II: Completion Report, February –March 2011 Project Completion Report: Governance and Poverty Reduction Support Grant II (GPRSG II), December 2013 Appraisal Report: Restoration of Fiscal Stability and Social Protection, June 2012 Appraisal Report: Supplementary Budget Support to RFSSP, April 2013 Back-to-Office Report, RFSSP Supervision, Project Completion and Budget Support Review Mission, Nov 2013

53

Implementation Progress and Results Report (IPR), Restoration of Fiscal Stability and Social Protection, December 2013

Aide Memoire: PCR Mission for Restoration of Fiscal Stability and Social Protection and Budget support Review, December 2013

Project Completion Report: Restoration of Fiscal Stability and Social Protection and Supplementary Budget Support to RFSSP, December 2013

Appraisal Report: Public Finance Management Institutional Support Project – Phase I, September 2013 Appraisal Report: Public Finance Management Institutional Support Project-Phase 2, June 2015 Aide Memoire: Public Finance Management Institutional Support Project, Supervision and Preparation Mission,

November 2014 Implementation Progress and Results Report: Public Finance Management Institutional Support Project,

Supervision, August 2016 Implementation Progress and Results Report: Public Finance Management Institutional Support Project, Phase

2, Launching, August 2016 Implementation Progress and Results Report: Public Finance Management Institutional Support Project, Phase

2, Launching, December 2015 Quarterly Project Progress Reports: Public Finance Management Institutional Support Project, 2014-present MIPS, Strategic Plan 2016-20. Malawi Country Economic Memorandum: Seizing Opportunities for Growth through Trade; Volume I: Summary

of Main Findings and Recommendations Malawi Country Economic Memorandum: Seizing Opportunities for Growth through Trade; Volume II:

Background Papers Policy Brief: Analysis of Malawi’s Exchange Rate and Case Studies on Exchange Rate Adjustment, July 2011,

Confidential Case study report: Domestic Resource Mobilization for financing development, May 2014 Research Papers: Exchange Rate Misalignment in Malawi: A Monetary Approach (undated). Measuring Potential

Output for Malawi (undated). A Small Macro-Model for Malawi (undated), Reserve Bank of Malawi Study Report: Skills for Private Sector Development, Malawi, December 2009 Inter-office Memorandum: Bank’s legal opinion on the proposed amendments to the Joint Framework

Agreement CABS, December 2007 Procurement and fiduciary risk assessment reports

OTHER DONOR DOCUMENTS

ADB, ADF, DANIDA, SIDA, Joint Evaluation. Evaluation of Public Financial Management Reform in Malawi 2001–2010. Final Country Case Study Report, June 2012

AFD, AUSAid, CIDA et al, Joint Evaluation of General Budget Support 1994–2004, Malawi Country Report, May 2006

AICD, Country Report. Malawi’s Infrastructure: A Continental Perspective, March 2010 CIDA, Malawi Country Program Evaluation 1998-2008. Synthesis Report, July 2010 CMI Brief, the ‘Joyce Banda Effect’. Public Opinion and Voting Behavior in Malawi, Volume 15 No. 6, April 2016 CMI Brief, The Gender Machinery Women in Malawi’s Central Government Administration, Volume 15 No. 11,

June 2016 DFID, Evaluation of DfID Country Program. Country Study: Malawi 2000 - 2005, Evaluation Report Ev661, April

2006 EU, Country Level Evaluation Republic of Malawi. Final Report. Volume 1: Main Report, November 2011 EU, Country Level Evaluation Republic of Malawi. Final Report. Volume 2: Annexes, November 2011 EU DEVCO “Methodological Approach to Blending” Final Report, December 2016, ADE publication for DEVCO FAO (Food and Agriculture Organization), Socio-economic content and the Role of Agriculture; Country Fact

Sheet, March 2015 Global Partnership for Effective Development Co-operation (GPEDC), Monitoring Guide 2015-16, 2015 IFC and World Bank Group, Impact of Efficient Irrigation Technologies on Small Farmers, 2014 JICA, Inception Report: Data Collection on Nacala Corridor Integrated Development in Southern Africa, JICA,

March 2017 UN Women, Planet 50:50 by 2030: Step It Up for Gender Equality World Bank Paper: “Emerging Stronger- Malawi Economic Monitor”, October 2016 World Bank Paper: “Absorbing Shocks, building resilience- Malawi Economic Monitor,” May 2016

54

World Bank Working Paper 182: “The Education System in Malawi,” 2010 World Bank, Malawi Poverty and Vulnerability Assessment, 2007 World Bank, Evaluation Brief: “Overlooked Links in the Results Chain” IEG-World Bank Brief 142011 www.nepad-caadp.org: Introducing CAADP. Website accessed 29th May 2017

OTHER DOCUMENTS

Aide Memoire: Common Approach to Budget Support (CABS) Reviews, October 2010, March 2012, October 2012, March 2013, November 2013, March 2014, August 2014

Minutes: Group on Financial and Economic Management (GFEM) Meeting (incl. Technical Committees), May 2011, December 2011, September 2012, April 2013, June 2013, October 2013

Minutes: PFEM Donor Coordination Group, November 2016, December 2016, January 2017, March 2017 Minutes: PFM Joint Technical Committee Meeting, January 2016, September 2016, February 2017 Letter: GFEM Comments on PFM Reform Program, February 2015, EU. Draft Final Report, Support for the Development of the Public Finance and Economic Management Reform

Program for Malawi, September 2011, ACE / European Commission Brief: Public Financial Management Malawi Donor Coordination Group, Briefing for Heads of Cooperation

Meeting February 2017 PowerPoint presentation: PFM priorities from DP’s perspective, January 2017 Program Document: Public Finance and Economic Management Reform Program (PFEMRP), February 2012,

World Bank Final Report, Review of CABS Mechanism and Its Processes, November 2013 Sarah Holloway, DFID / Mokoro. Draft Mission Report, Independent Monitoring of Progress against the GoM Action Plan to improve PFM, August

2014 Technical report: Malawi Restoring Financial Control and Accountability November 2014, Abdul Khan, Suzanne

Flynn, Guy Anderson, and Amitabh Tripathi, IMF Fiscal Affairs Department Donor Matrix, Development Partner support to assist the Malawi’s PFM, May 2016 Project Appraisal Document: Financial Reporting and Oversight Improvement Project, February, 2013, World

Bank Performance and Learning Review of the Country Assistance Strategy for the Republic Of Malawi for the Period

Fy13-Fy16, World Bank Implementation Completion and Results Report: Financial Management, Transparency, and Accountability

Project (FIMTAP) April 30, 2010, World Bank

AfDB/ DANIDA/SIDA, Evaluation of Public Financial Management Reformin Malawi 2001–2010Final Country

Case Study Report, June 2012 A joint Evaluation of General Budget Support 1994-2004, Evaluation of GBS – Malawi Country Report, May 2006 Initial thoughts on the Procurement and Disposal of Assets Bill, March 2017 IMF Reports. Malawi Confederation of Chambers of Commerce and Industry, “Malawi Business Climate Survey Report”

November 2016 Hausmann, R. et al., “The Atlas of Economic Complexity-Mapping Paths to Prosperity” Harvard Kennedy School

of Government Press 2016.