Evaluation Approach Paper Real-Time Evaluation: ADB’s ...

DIN

Evaluation Approach Paper

Real-Time Evaluation: ADB’s Response to the COVID-19

Pandemic November 2020

Team Leaders: Joanne Asquith, Principal Evaluation Specialist ([email protected]) and Erik

Bloom, Senior Evaluation Specialist ([email protected]) Contact: [email protected]

A. Introduction 1. The Independent Evaluation Department (IED) will conduct an assessment of Asian Development Bank’s (ADB) response to the coronavirus disease (COVID-19) pandemic. The evaluation will be formative, with the goal to help shape ongoing and future support to developing member countries (DMCs). Because the evaluation will be undertaken in real-time, while the pandemic is ongoing and during the implementation of ADB operations, IED is mindful of the pressures that ADB teams and DMC governments are facing. The evaluation will focus on assessing ADB’s support at the implementation stage and identifying early lessons. 2. The purpose of the evaluation is to provide ADB’s Board of Directors and Management with timely evidence on the implementation of ADB’s support to address the health, social, and economic crises brought about by COVID-19. The evaluation will provide emerging lessons on the relevance of ADB’s support, the design and implementation of its operations, and the organization’s preparedness, to inform possible future COVID-19 support. The evaluation will analyze a range of evidence gathered from interviews, data analysis, reports from DMCs, literature and project document review, data mining, and country-level assessments. 3. The evaluation will use established real-time evaluation (RTE) methodology, which is often used during periods of crisis or emergency, specifically to identify early lessons and provide feedback to program managers. The evaluation will be cognizant of the Board of Directors’ changing needs for analysis and evidence and it will use a flexible approach to providing outputs. This includes an interim assessment in December 2020 to coincide with the Board of Director’s review of ADB’s support. This initial assessment would provide analysis on several priority areas.1 The evaluation will also prepare synopsis notes of preliminary evidence and case studies in the first and second quarters of 2021. The evaluation will provide a final formative evaluation report in September 2021. B. The Economic and Social Impact of COVID-19 4. The pandemic. The outbreak of COVID-19 has sparked the most significant global economic and public health crisis of the 21st Century. Since it was identified in December 2019, the virus has spread rapidly across the world. The growth in the fatalities appears to have peaked

1 The preliminary assessment will likely include a review of the financial impact of the pandemic on ADB and on DMCs,

the adequacy of the design and monitoring frameworks, and a review of the process used to provide support. The final content will depend on demand and the availability of data.

Asian Development Bank. 6 ADB Avenue, Mandaluyong City 1550 Metro Manila, Philippines Tel +63 2 632 4444; Fax +63 2 636 2163; [email protected]; www.adb.org/evaluation

2

in April 2020. As of 12 November 2020, there were approximately 52 million reported known cases and 1,300,000 reported deaths. The daily growth rates in cases was 0.8% in October 2020, implying a doubluing period of around 90 days. This represents a continuing slowdown in the growth rate of the number of cases, compared to previous months. 0F

2 The total number of cases in November is spiking in many regions, led particularly by North America, Europe, and Russia as they enter the winter months. However, in the rest of the world, including Asia and the Pacific has slowed (Figure 1).

Figure 1: Global COVID-19 Cases

COVID-19 = coronavirus disease. Source: Johns Hopkins University.

5. COVID-19 has had a major impact on the Asia and Pacific region, although the location of outbreak has varied over time. In February 2020, almost all confirmed cases and fatalities in the world were in the East Asia region, in the People’s Republic of China. By April, the number of new cases in East Asia had dropped significantly and new cases were relatively evenly divided in Central West Asia, Southeast Asia, and South Asia. However, by June, most new cases and fatalities were located in South Asia, which appears to have become a global hotspot. By October and November, the rate of growth was definitely slowing in most DMCs (Figure 2).

2 In August 2020, the daily growth rate in cases was 1.1%, implying a doubllling period of 65 days. In July, the rate of

rate was 1.7% implying a 43-day doubling period. This compares to growth rates of 7.4% in March (10 days) and 4.4% in April (16 days).

3

Figure 2: COVID-19 Confirmed Cases and Fatalities

COVID-19 = coronavirus disease, CW = Central West Asia, EA = East Asia, PA = Pacific, SA = South Asia, SE = Southeast Asia. Source: Johns Hopkins Univeristy.

6. One of the most salient features of the COVID-19 pandemic is the disparity of its impact. Unchecked, the disease has a relatively high viral reproduction rate. However, there are substantial differences in the spread of the virus and its fatality. Several factors seem to contribute to the seriousness of the outbreak, including the public health capacity of the government to quickly identify and isolate cases, the quality of health services to treat victims, demographic conditions, and the public’s cohesion and compliance with risk mitigation guidelines. The virus is also mutating, which may affect both its transmission and fatality rates.3 Currently the issues and the debate is shifting to vaccine and vaccination solutions. 7. Government response. Governments are focusing on interventions to reduce the public health risk while protecting the population from economic and social impacts. On the public health front, they are attempting to prevent or contain the pandemic to reduce infection and the loss of life. Governments have invested in strengthening their health systems, including building new testing systems, isolation wards and intensive care units for the sick, and procuring drugs and personal protective equipment for health and other frontline workers. Other common interventions include mandating mask wearing for the public. To reduce the spread and rate of infection, most countries implemented some form of lockdown that limited international travel and reduced social interactions, with the goal to limit both cross-border and community transmission. 8. All of these actions happen in an enivornment of great uncertainity and a lack of information on the optimal policy. COVID-19 is a new disease and both governments and medical community have been forced to improvise and experiment in an environment of great uncertainity. The last rapidly spreading global pandemic was the outbreak break of H1N1 A influenza (Spanish flu) in 1918.

3 K. Kupferschmidt. 2020. “The pandemic virus is slowly mutating. But is it getting more dangerous?” Science: 14 July

2020.

4

9. Governments in the Asia and Pacific region have delivered wide-ranging policy responses, including policy support packages—mainly income support—amounting to $3.7 trillion, equivalent to about 15% of regional gross domestic product (GDP).4 This includes help for struggling families and businesses to mitigate the adverse impacts of the pandemic and to head off the long-term consequences for growth and development that may otherwise occur. This takes the form of grants, subsidies, and loans. Support for the private sector focuses on protecting businesses from bankruptcy and taking steps to reduce unemployment. Governments have also increased spending on the health sector, both to treat patients with COVID-19 and to strengthen health survelliance systems. Finally, governments are attempting to manage the economic crisis by increasing and expediting spending and reducing taxes. Central banks across the world have provided substantial liquidity and are supporting the financial sector; as a result, both domestic and international lending rates are at record lows. 10. Social and economic impacts. Government actions to stem the impact of COVID-19 have slowed economic activities and consumer demand. The massive decline in international travel has reduced tourism and limited the movement of labor and associated remittances. Stay at home orders and the selected closure of businesses have dampened domestic demand. In addition, trade has been severly affected, in some cases leading to disrupted supply chains.5 This has resulted in the sharpest global economic downturn in more than 80 years. The International Monetary Fund (IMF) is expecting a global recession in 2020, with a growth contraction of -4.4%; by comparison, during the global slowdown in 2009, global economic growth contracted by -0.1%. According to ADB, the Asia and Pacific region is expected to be heavily impacted, with an expected decline of 0.7% in economic growth in 2020. This is the first region-wide recession in more than 30 years. ADB expects a recovery in 2021, although there is much uncertainity. Table 1 shows the growth rate forecasts for 2020 and 2021 from the September 2020 Asian Development Outlook Update report.

Table 1: Projected Growth Rate of Developing Member Countries

()= negative. Note: South Asia includes Pakistan and Afghanistan. Source: ADB. 2020. Asian Development Outlook Update 2020. Manila.

11. Within countries, there is great disparity in the pandemic’s economic impact. This is driven by several factors, including socioeconomic demographics, the structure of the labor market, and access to social assistance and health services. In India and the Philippines, for example, the urban poor often do not have access to formal sector employment or the ability to physically isolate from others. Throughout the world, ethnic minorities, displaced people, and the poor are especially vulnerable to the pandemic’s health and economic risks. The nature of vulnerable populations varies greatly from country to country, depending on the country’s demograohic, economic and social background. While Asia and the Pacific have made substantial progress in reducing poverty, without adequate social protection many families face the risk of falling back into poverty. Poor

4 J. Felipe et al. 2020. ADB COVID-19 Policy Database. Manila: ADB. 5 Restrictions on transport and lower domestic demand may raise the fixed costs associated with trade, leading to more

expensive imported goods, delays in intermediate and capital goods, and lower external demand.

Region 2019 2020 Projection 2021 Forecast

Central Asia 4.9 (2.1) 3.9 East Asia 5.4 1.3 7.0 South Asia 4.3 (6.8) 7.1 Southeast Asia 4.4 (2.7) 5.5 Pacific 3.7 (6.1) 1.3 Developing Asia 5.6 (0.7) 6.8

5

households are especially vulnerable to the pandemic and face the risk of falling deeper into poverty without adequate social protection and insurance. 12. In most countries, prolonged closure of schools and limited access to other essential social services, including health care, will likely have a disproportionate impact on women and single-headed households. This can lead to a reduction in access to basic preventative care, such as child immunization and access to birth control. While women appear to have suffered more economically, the health impact appears to be more severe on men, particularly older men, who have had a higher infection and fatality rate.6 13. As a consequence of school closings, virtually all students were forced to miss at least a few weeks of their education. In many DMCs, students will effectively lose one year of schooling. The long-term consequences of this are not clear but may be severe. School closings may also exacerbate inequality as students from poorer households usually lack access to digital tools to continue with their education at home. 14. The pandemic is having serious implications for the private sector, including enterprises of all sizes in all sectors. Government restrictions on social interactions are barriers to commerce and have the potential to disrupt supply chains. The reduction in household income and global demand has a negative impact on aggregate demand, which is exasperated by disruptions in access to capital markets. Without public intervention, this could lead to increasing bankruptcies and firm closures, which, in turn, place stress on the financial sector as firms are not able to service their loans and access capital markets. These factors may ultimately increase unemployment, leading to a further demand shock. In many economies, small and medium enterprises may especially be at risk, due to limited access to the finance markets and heavier reliance on self-financing. 15. Responding to these negative supply and demand shocks, governments are taking actions to subsidize both households and the private sector. This can partially mitigate the negative economic impact of the pandemic and, by encouraging a reduction in social contact, may reduce the infection rate. However, the economic shock will also reduce tax revenues. Government resources are limited and at some point, deficit spending could become unsustainable. International financial institutions, such as ADB, also face resource limits as a greater percentage of lendable capital is allocated and resources for subsidized financing are eliminated. International financial institutions also will face new risks from nonsovereign partners as many existing partners seeking greater liquidity. Bilateral development organizations may also face additional stress on their own resources as they rely on government budgets that are under stress. 16. The pandemic has highlighted that public health and social and economic wellbeing are intimately related with the environment. The world will continue to face deadly zoonotic diseases unless their root cause are addressed. This will require increased investments in sectors that reduce environmental risks, improve food safety, and provide clean water and sanitation. C. ADB’s Response to COVID-19 17. Since their creation, multilateral development banks (MDBs) have played an important role in responding to economic shocks and disasters triggered by natural hazards through the provision of countercyclical financing and other fast disbursing support. The World Bank’s original mission was reconstruction and development in war-torn Europe. Much of the IMF’s support is focused on

6 See, for example, J. Ng, K. Bakrania, R. Russell, and C. Falkous. 2020. COVID-19 Mortality Rates by Age and Gender:

Why Is the Disease Killing More Men than Women?

6

countercyclical support and economic stabilization. ADB has gained extensive experience in supporting DMCs respond to disasters triggered by natural hazards and epidemics and to address economic shocks.

ADB has over 50 years of experience in the region, dealing with a wide range of economic situations. While each crisis is different, they all contribute to ADB’s capacity to respond quickly and effectively. ADB was active during the 1997–1998 Asian Financial Crisis, providing support to affected DMCs with budget support. Much of this work was done in close collaboration with the World Bank and the IMF. Likewise, during the 2008–2009 global crisis, ADB provided countercyclical budget support to its DMCs. IED’s review of the ADB response showed that the Bank needed to work in a timely fashion to ensure a meaningful contribution to affected economies.7 Asia has also been affected by a number of communicable disease emergencies, including the outbreak of severe acute respiratory syndrome (SARS) in 2003 and avian influenza and/or H1N1 in 2005. In both cases, ADB developed a series of regional and country specific projects to prepare the region for a potential outbreak of a pandemic. In addition, ADB was active in combating HIV and/or AIDS and strengthening preventive health systems, particularly in the Greater Mekong Subregion. 18. In April 2020, ADB’s Board of Directors approved a $20 billion package of support to DMCs, known as the Comprehensive Response to the COVID-19 Pandemic (The ADB Response).8 ADB committed $13.7 billion in new resources and $6.3 billion from reallocated resources for sovereign and nonsovereign operations (Table 2). The ADB Response provides special financing for 15 months, from April 2020 until July 2021.9 ADB will provide an interim review of its response in December 2020 and it may expand and extend it if circumstances so warrant. The ADB Response offers financing to support the prevention and mitigation of negative health, economic, and social consequences of COVID-19.

Table 2: Source of Financing for ADB Response ($ million)

Source of Financing

Sovereign Operations

Nonsovereign Operations

Additional OCR resources 13,000 0 Additional concessional and grant resources 704 0 Reprogramming of planned operations 3,721 1,640 Reallocation from existing operations 366 200 Savings and cancellations of ongoing operations 281 0 Reallocation of existing grant resources 38 0 Total 18,160 1,840 COVID-19 = coronavirus disease, OCR = ordinary capital resources. Source: Asian Development Bank.

19. The ADB Response has adapted several different instruments on the sovereign and nonsovereign side, through which it can expand support for DMCs to address COVID-19 including:

(i) The Countercyclical Support Facility. This facility was established in 2009 to provide budget support to eligible Group C and B DMCs to help finance a fiscal

7 IED. 2012. Special Evaluation Study on Real-Time Evaluation of Asian Development Bank’s Response to the Global

Economic Crisis of 2008–2009. Manila: ADB. 8 ADB. 2020. ADB’s Comprehensive Response to the COVID-19 Pandemic. Manila. 9 The initial period of the ADB Response is 15 months. This can be extended to 24 months, until April 2022. However,

ADB may also request extension or modification of any component of the Response as required as new developments occur.

7

stimulus in the form of countercyclical development expenditures.10 Using this facility, ADB will finance pro-poor countercyclical expenditure programs, which clearly identify measures taken to mitigate impacts of the COVID-19 pandemic on vulnerable groups. Eligibility criteria are widened to include Group A and Group B DMCs to allow the use of concessional and grant resources.11

(ii) Disaster-related financing. ADB has a range of instruments to support DMCs in times of crises. These include: (a) Contingent Disaster Financing (CDF), which provides budget support to DMCs where ADB has had policy dialogue and reforms before a natural hazard occurs, with disbursements made upon satisfaction of pre-agreed disbursement conditions. ADB expanded CDF eligibility criteria to include health related emergencies; (b) the Asian Development Fund’s Disaster Response Facility (ADFDRF), which provides grants and concessional financing to support DMCs to respond to disasters is expanded to include health; and (c) the Asia Pacific Disaster Response Fund, which provides small grants to respond to the immediate needs of DMCs has been adjusted to accept tied contributions.12

(iii) Investment projects. The ADB Response provides guidelines for ADB to carry out investment projects that support government programs to mitigate COVID-19. These include new projects specifically designed for the pandemic, newly added components with a COVID-19 focus in projects under preparation, and additional financing for active projects. ADB can use emergency assistance loans and additional financing to facilitate the process. Many projects focus on health and social protection, as well as those in other sectors that support mitigation efforts.

(iv) Nonsovereign financing. The ADB Response included provisions to support nonsovereign financing, including: (i) increasing the framework limit for the Faster Approach to Small Nonsovereign Transaction (FAST) Program, for high impact support; (ii) modification and expansion of the Trade Finance Program, which provides credit to facilitate trade; (iii) modification and expansion of the Microfinance Risk Participation and Guarantee Program, which supports microfinance; and (iv) enhanced capacity of the Supply Chain Finance Program to provide trade financing. The special funds all operate on a country-level, with ADB approving a country-specific operation to provide support. As with all ADB nonsoveriegn financing, ADB uses market-based terms and does not provide subsidies to the private sector through this window.

(v) Technical assistance (TA). ADB has a variety of funds that provide financing for technical assistance projects. As part of its response, ADB has simplified the processes to prepare new technical assistance projects or to modify existing ones, including reducing or eliminating internal reviews, and speeding approval processes. In addition, technical assistance projects may directly procure medical equipment and supplies or contract with other agencies to meet urgent needs. ADB has also reallocated resources to target more technical assistance to focus on the COVID-response.

10 Using a combination of income, level of development, and creditworthiness ADB divides active DMCs into three

groups: Group A (concessional assistance-only countries): Group B (concessional assistance and regular assistance blend countries): and Group C (regular assistance-only countries). Some DMCs have formally graduated or are no longer active borrowers. See ADB. 2019. Classification and Graduation of Developing Member Countries.Operations Manual. OM Section A1. Manila. ADB for further details.

11 All DMCs are guaranteed a minimum of $20 million. Under new criteria, Group A DMCs may receive up to $100 million in grants and $250 million in concessional lending, while Group B DMCs can access up to $250 million in concessional leding. All lending is subject to an analysis of a DMC’s debt sustainability.

12 Previously ADB accepted only untied contributions but now donr contributions can be tied to COVID-19 support.

8

20. Fast-tracked business process. As part of the response, ADB introduced a number of modifications in its internal procedures to facilitate the approval of support to DMCs. This included changes in processes to speed up the design of new projects and restructuring of existing projects. These changes are mainly relevant for sovereign operations and are built around the concept of the One ADB Team. In this model, the project team includes staff who traditionally provide comments and due diligence. ADB typically used this model for the emergency assistance operations as a way to expedite projects through the internal approval process. By embedding these staff into the project team, less time is spent soliciting comments and more time is spent collaborating. In addition, ADB made other arrangements to expedite administratives requirements. 21. Knowledge solutions. Since its creation, knowledge is an important part of ADB’s support to DMCs. Knowledge has played an important role in ADB’s support for COVID-19. The Economic Research and Regional Cooperation Department has developed a comprehensive database on global economic policy in response to COVID-19 as well as carrying out quarterly economic forecasts for DMCs. ADB’s technical assistance projects support government capacity to implement ADB projects. Likewise, ADB has supported technical assistance (at both the national and regional levels) to prepare for future pandemics. 22. As of 2 November 2020, ADB has approved a total of $14.5 billion in financing for COVID-19, and received $8.7 billion in cofinancing. Table 3 outlines the financing provided under the different COVID-19 instruments. By comparison, in 2019, ADB approved $18.6 billion in sovereign financing, $3.0 billion in nonsovereign financing, and $0.2 billion in technical assistance for all operations. In total, ADB expects to provide around $31 billion in financing in 2020, including much of its original program. Most of ADB’s support for the private sector has come in through its support for revolving funds, particularly the Trade Finance Program.

Table 3: Summary of ADB’s Approved Projects for COVID-19 Response ($ million)

ADB = Asian Development Bank, APRDF = Asia Pacific Disaster Response Fund, COVID-19 = coronavirus disease, CPRO = Comprehensive Response to the COVID-19 Pandemic. a Refers to the value of transactions under short term lending revolving programs: the Microfinance Risk Participation and Guarantee Program (MFP), Supply Chain Finance Program (SCFP), and Trade Finance Program (TFP), and not the programs’ framework limits. Source: Asian Development Bank, 33rd updates of working groups on COVID-19 (24–30 October 2020).

23. ADB plans to play an active role in providing vaccine finance and is preparing to dedicate significant resources to DMCs for this purpose. Initial plans call for ADB to support both the purchase of vaccines and support for countries to improve their capacity to deliver vaccine programs, including support for strengthening vaccine cold chain and storage. For this initiative, ADB will offer a combination of loans and grants, supported by technical assistance.

Item

Approvals of which, committed

ADB Cofinancing Total ADB Cofinancing Total

Sovereign operations 12, 325.94 6,869.77 19,195.71 10,452.43 6,229.77 16,682.20

CPRO 9,680.00 6,481.19 16,161.19 8,272.55 5,941.19 14,213.73 Other projects 2,645.94 388.59 3,034.52 2,179.88 288.59 2,468.47 Nonsovereign

operations

1,981.31 1,779.31 3,760.61 1,907.13 1,779.31 3,682.43

Projects 237.60 25 262.60 163.42 25.00 188.42 Revolving programsa 1,743.71 1,754.31 3,498.01 1,743.71 1,754.31 3,498.01 Other 155.46 7.53 162.99 155.46 7.53 162.99

APDRF 55.97 0 55.97 55.97 0 55.97 Technical Assistance 99.49 7.53 107.02 99.49 7.53 107.02 Total 14,462.71 8,656.61 23,119.31 12,515.02 8,016.61 20,531.63

9

24. Sustainability. Economic risks in the region center on COVID-19. A prolonged pandemic is the main risk to the economic outlook for the region, as it would require protracted caution and containment measures, with consequent disruption to both supply and demand. Protracted weakness caused by COVID-19 could trigger crises in some countries with debt sustainability issues or financial vulnerabilities. Some countries could see fragile banks and debt sustainability severely tested. While several policy measures are been rolled out to mitigate these risks, these risks may impact ADB’s capital adequacy under different lending scenarios. D. Theory of Change 25. The theory of change (TOC) identifies the route through which ADB contributes to mitigating the pandemic’s impact on health, poverty, and economic development (Figure 3). IED prepared the TOC based on the implicit logic behind ADB’s support as articulated in the COVID-19 Board Paper, public announcements, and other ADB strategies, including Strategy 2030. TOC captures ADB’s contribution both during the pandemic and as the world recovers from its negative impacts.

26. The TOC starts with ADB’s financial and human resources, which it uses to create the

inputs in its COVID-19 program. ADB offers a range of products and services, including budget support programs, sovereign and nonsovereign investments, and TA as well as various knowledge solutions. ADB provides this support to DMC governments, private sector clients, and other stakeholders. ADB also provides support at the regional and global levels, through its research, publications, and collaboration with other organizations. 27. There are a number of underlying assumptions behind the TOC regarding how ADB works and how ADB contributes to the region. These are largely based on internal memos and guidelines. The TOC has assumptions regarding (1) what DMCs and private sector clients need from ADB and (2) how ADB will work to provide support to these partners.

1. Assumptions on DMC needs (i) Governments and nonsoveriegn clients need support on an expedited

basis. Countries and clients require a large amount of resources in a relatively short amount of time to address specific needs for liquidity.

(ii) DMCs will prepare well-defined COVID-19 response programs. ADB will work with countries and clients that have specific, well-defined programs for fighting COVID-19 and ADB will provide financial support to specific COVID-19 related projects or provide budget support (liquidity for the private sector) in the face of well-defined needs.

2. Assumptions on how ADB will work

(i) ADB has a stock of knowledge and experience on managing economic and social crises. While financing is important, ADB also brings additional value through its knowledge and experience to DMC governments and the private sector.

(ii) All resources will be used for the COVID-19 response in an “all hands on deck approach.” To achieve a quick and high-quality response, ADB will allocate its internal resources, such as staff, consultants, and its internal budget, where they are most needed.

28. These inputs are expected to result in country level outputs in several areas: (i) support to directly combat COVID-19, including purchasing inputs for testing and surveillance, providing

10

equipment and supplies, vaccinations, and improving relevant infrastructure; (ii) support to promote social and economic priorities, including financing for cash and other transfers for vulnerable individuals, and subsidies to support businesses continuity and limit job losses; (iii) support to promote economic stability, including financing to close budget and trade deficits; and (iv) providing knowledge solutions on topics including COVID-19 policy, sectoral policy, and macroeconomic policy. 29. DMC governments carry out a range of activities and programs to strengthen their health systems and to mitigate the negative social and economic impacts of the pandemic. Government support largely includes some combination of strengthening the health sector, providing subsidies to poorer households, and support for small and medium-sized enterprises. Governments organize their actions through COVID-19 plans, which can take the form of dedicated plan or adjustments to the country’s budget. 30. ADB’s support leads to positive health, and social and economic impacts. This results in a stronger health system, greater economic resilience, and improved social outcomes compared to the counterfactual scenario of no ADB interventions. ADB financing for vaccination purchase and distribution are also likely areas of support. Ultimately, ADB’s interventions lead to greater mitigation and control of the COVID-19 pandemic.

E. Purpose and Scope of the Evaluation 31. The evaluation will assess during implementation the relevance, efficiency, and coherence of ADB’s response to the COVID-19 crisis and early signs of effectiveness, to identify emerging lessons and progress in achieving ADB objectives. It will provide feedback to the Board of Directors and Management on the preparation of ADB’s response and its delivery, to enable them to make evidence-based decisions for ongoing and future support. It may also inform later evaluations. The evaluation will consider the whole of ADB’s response, which includes interventions that are explicitly included in the ADB Approach as well other means of support that ADB has indicated are related to COVID-19. 32. The evaluation focuses on the inputs and output level, identifying the contributions that ADB resources are making in the larger country response. It will look at the processes involved in developing a response as well. As a real time, evaluation, it will also analyze early evidence on the progress that ADB is making with its resources to identify if ADB support is likely to achieve its intended objectives. While it will not have the full set of information that an ex-post evaluation would have, it will provide feedback to make adjustments and to improve future operations.

11

Figure 3: Theory of Change of ADB’s Response to COVID-19

ADB = Asian Development Bank, ADF = Asian Development Fund, COVID-19 = coronavirus disease, DMC = developing member country, OCR = ordinary capital resources, TA = technical assistance. Note: Scope of evaluation excludes assessment of impacts. Source: Independent Evaluation Department.

12

F. Evaluation Questions

33. The overarching question underpinning the evaluation is: How responsive, relevant, and on track towards intended results has ADB’s support been to combat the COVID-19 crisis and what lessons can be immediately applied to ongoing as well as to future operations? 34. The evaluation will address the following subquestions:

(i) How relevant has ADB’s response been to the needs of DMCs to tackle the COVID-19 crisis? This subquestion will analyze ADB’s overall approach to fighting COVID-19, the allocation of budget support and other resources across DMCs and its different programs of support, including sovereign and nonsovereign support. It will consider ADB’s overall support, from an institutional level. It will look at the degree to which ADB balanced its COVID-19-related operations with its planned program of support, as well as how ADB balanced finance (sovereign and nonsovereign) with knowledge solutions. The evaluation will assess the relevance of program design to country needs and context including ADB’s corporate response to the poor and most vulnerable as well as the gender-specific impacts.13 The evaluation will look at the underlying economic and health assumptions behind ADB operations and it will assess the quality of systems for monitoring and evaluation and reporting progress in design and monitoring frameworks (DMFs). The evaluation will also review how COVID-19 has impacted on the debt sustainability of DMCs.

(ii) How efficient and coherent is ADB being in allocating its resources to support the Asia and Pacific Region during the COVID-19 crisis? This will consider how ADB has allocated it own resources and reduced bottlenecks to respond to the needs of the region, including the allocation of budgetary and human resources. It will also analyze how ADB works with governments and other development partners to target priorities, ensure cohesion, and and reduce duplication. A key focus will be the degree to which ADB has collaborated closely with other donor partners especially in areas where ADB does not have a comparative advantage. To the extent possible, given time and resources, the evaluation may be able to track budgetary expenditures.

(iii) How effectively on track towards the achievement of intended results and outcomes are ADB’s pandemic-related interventions? As it will be too early to provide an overall assessment of the effectiveness of ADB’s interventions, the evaluation’s focus will be on immediate, operation-specific results, while exploring the degree to which ADB support provided value addition to government programs to mitigate negative health, social, and economic effects. The assessment will therefore be on the extent to which ADB-supported government programs are achieving or are likely to achieve results. It will also consider the extent to which ADB support played a role in this, with special focus on results from pandemic-related health interventions, social protection measures and health insurance for the vulnerable during the pandemic, support for agriculture sector and food security, and

13 The definition of the vulnerable population will differ markedly from country to country. In all cases, the elderly face a

greatest health risk as do people with pre-existing conditions. In many countries, the young and those without access to reliable internet are greater risk of losing employment and the ability to study. In some countries, migrant workers and displaced people face heightened risk, often because they do not permenant residences. The economic and public health risks of COVID-19 may vary greatly between the urban and rural population as well.

13

support for small and medium sized enterprises that may face economic risks. The evaluation will review the degree to which ADB projects and programs are on track to meet intermediate outcomes, objectives, and providing additionality to DMCs.

(iv) How well prepared was ADB and how quickly has it been adapting to the challenges posed by COVID-19? This question will focus on ADB’s capacity to respond in a timely manner and adapt its approach in the face of the pandemic, especially through the application of the One ADB approach and the support provided by resident missions. It will also consider how well prepared ADB is to measure, track and monitor COVID-19 related operations. Looking forward, the evaluation will focus on ADB’s readiness for future needs associated with COVID-19, including additional waves of the disease, continued economic recovery, universal health coverage, and vaccination financing. It will also identify how the experience has affected its capacity to respond to other emergencies and pandemics. This includes monitoring the extent to which ADB reprogrammed existing pipeline as well as used “new” expenditures. The evaluation will review the pandemic’s impact on ADB’s financial and institutional capacity, including a review of how the pandemic has affected ADB’s credit worthiness and its capacity to respond to future emergencies.

G. Evaluation Methodology 35. Real-Time Evaluation Approach. The evaluation will draw on the accepted methodology for RTEs. The RTE methodology is designed to be used during emergency or crisis settings and is often carried out near to the start of an emergency or recovery response.14 The purpose of an RTE is to provide early and rapid feedback in real-time to those governing and directing the provision of support. As such, they are primarily formative in nature. An RTE can conform to multiple broad approaches concurrently. The evaluation will incorporate elements of a process evaluation, with a focus on how ADB carried out its work given its resources. It will also look at design and emerging results (if any), drawing from techniques used for an outcome evaluation. In both cases, the evaluation will take into account the objectives as understood by ADB at the time the pandemic began. IED will also look at the experience of other organizations to identify good practices and to benchmark ADB’s performance. 36. This RTE of a protracted COVID-19 crisis response will be different from the RTE of the ADB response to the 2008–2009 global financial crisis. The 2012 evaluation was more summative and conducted after the immediate crisis was already over and ADB support completed.15 The present evaluation is more formative in nature. Given the rapidly changing nature of the COVID-19 crisis, collaboration among evaluation and research units plays an important role in sharing information, knowledge, and experience. IED was active in doing this during the 2008–2009 financial crisis, as well as in the current COVID-19 outbreak.16 IED is currently engaged with the COVID-19 Global Evaluation Coalition, which is actively sharing information across multilateral

14 For a brief overview, see INTRAC. 2017. Real-Time Evaluations and Independent Evaluation Group. 2010. High

Impact Evaluations: Exploring the Potential of Real-Time and Prospective Evaluations. Washington, DC: World Bank Group.

15 IED. 2012. Special Evaluation Study on Real-Time Evaluation of Asian Development Bank’s Response to the Global Economic Crisis of 2008–2009. Manila: ADB.

16 IED. 2012. The Implications of a Global Financial Crisis for Asia and the ADB: Lessons from Evaluation. Manila: ADB and IED. 2020. Responding to the Novel Coronavirus Crisis – 13 Lessons from Evaluation. Manila: ADB.

14

and bilateral evaluation departments with a view to undertaking joint evaluation and synthesis work that will go beyond the operations of a single donor.17 37. Activities. The evaluation will draw on qualitative and quantitative data as well as available literature.The evaluation will carry out the following (Appendix 1):

(i) Review of ADB’s COVID-19 Portfolio. The review will analyze all COVID-19 interventions to identify the types of interventions supported; for example, how many are health related, were gender specific indicators included, how many supported social protection, and how many supported small and medium-sized enterprises, what did support consist of, and how was it tailored to meet or respond to different DMCs needs? The review will look at how many interventions contain baselines and appropriate indicators for measuring outcomes. It will also consider the average size of interventions, the type of financing, the main activity of support, and the implementation period of deliverables, and what monitoring and evaluation systems were put in place.

(ii) Review of ongoing ADB, government, and client progress reports related to ADB CPRO and other operations. The evaluation will review ongoing progress reports and updates from ADB. This will help identify the strength of ADB DMFs, the timeliness of documentation, and evidence on outputs and the early results.

(iii) Interviews. The evaluation will interview ADB staff, government officials, development partners, and beneficiaries (including private sector clients of ADB) using a mix of structured and semi-structured questions. Several rounds of interviews may be necessary, with the first round to focus on preliminary review and on issues related to design and preparation. Within ADB, the evaluation aims to interview all team leaders on operations tagged as COVID-19 support, including budget support, public and private sector investment projects, and technical assistance projects.

(iv) Analysis of big data and data mining. The evaluation will undertake innovative social media data mining activities to garner a real-time understanding of public perceptions in relation to the performance of COVID-19 programs in DMCs in general, and the four focus countries for the country level assessments in particular. This approach will also be used to examine potential gaps in the provision of development partner support, to verify the relevance of ADB investments.

(v) Analysis of administrative data. Administrative data will be used to better understand how ADB has allocated its resources, staff (including through short-term assignments) and consultants, and how it has allocated its financial support, commitments and disbursements.

(vi) Quick synopsis notes. The evaluation will undertake assessments of several themes that are relevant for the evaluation. Potential themes include: ADB support for social protection measures targeting affected populations and vulnerable groups; ADB support for small and medium enterprises stimulus programs to support the private sector; the financial impact of COVID-19 on ADB’s finances and its potential for further rounds of support; the role of knowledge and evaluation in the ADB response to COVID-19; ADB support for private sector operations related to COVID-19; COVID-19 related ADB coordination with development partners; lessons from other development partners; and an analysis of ADB support for the DMCs’ health programs related to COVID-19. Since the evaluation will not have

17 The Evaluation Coalition serves as a clearinghouse to discuss and share information on evaluation activities related

to COVID-19. It is organized by the Organizaction for Economic Cooperation and Development.

15

missions, these thematic synthesis notes will be organized by country to make distinct case studies (see Appendix 2).

(vii) Initial assessment. The evaluation will produce a preliminary interim assessment for Board consideration in December 2020. The assessment will highlight key IED findings regarding the strategic approach, ADB’s portfolio (including the nature of the DMFs reporting, and the availability of data), initial findings on the sustainability of ADB’s support, and the targeting of priorities areas and vulnerable groups.

38. Reporting of results. Due to the real-time nature of this evaluation, IED will deliver initial assessment and synopsis notes for the use of the Board and Management, which will be timed according to Board decision making points. IED will use abbreviated procedures to deliver these reports including:

(i) Abbreviated internal circulation period for notes and preliminary review; (ii) Review meeting chaired by IED to consider comments and recommendations by

ADB; and (iii) Case-by-case decisions by IED and the Committee of Development Effectiveness

on the public release of interim reports and notes; and (iv) No formal editing for interim reports.

39. IED will use standard procedures for the submission of the final evaluation report (scheduled for September 2021), including interdepartmental circulation and discussion with management. The final report will be made public. 40. Limitations. The evaluation faces the following limitations (i) field missions are not likely to be possible in the current COVID-19 context, requiring interaction with stakeholders through teleconferencing and videoconferencing, as well as a greater reliance of high-level country-based consultants; (ii) a second wave or further infection outbreaks may impede the evaluation by worsening country conditions economically and socially leading to further round of ADB support; and (iii) it is likely too early to assess outcomes and the country context could be highly uncertain. In addition, as staff are working remotely during the crisis, the evaluation will rely on the cooperation of ADB operational staff and resident missions to help set up meetings, gather data, and organize virtual meetings. As organizing missions and agreeing contacts can be lengthy processes, and due to other limitations, the evaluation proposes to undertake no more than four country level assessments. 41. Resource Requirements and Indicative Timeline. The evaluation will be co-led by a Joanne Asquith (Principal Evaluation Specialist) and Erik Bloom (Senior Evaluation Specialist). The team will also be comprised of Melinda Sutherland (Senior Evaluation Specialist), Noel Gamo (Senior Evaluation Officer), Sergio Villena (Evaluation Officer), Myrna Fortu (Evaluation Analyst), and a team of international and national consultants. The terms of references for the experts to be engaged for this evaluation are provided at Appendix 3. Overall guidance will be provided by the Director, Thematic and Country Division of IED.

16

42. To ensure that the evaluation is as timely as possible, IED will use expedited procedures. The following timeframe is proposed:

Table 4: Indicative Schedule of Evaluation Activities and Milestones Activities and/or Milestone Target Date

Inputs to Board of Directors December 2020

Preparation of Quick Synopsis Notes 1st and 2nd quarter, 2021

Interdepartmental circulation of evaluation July 2021

Heads of Departments Meeting August 2021

Development Effectiveness Committee meeting September 2021 Source: Independent Evaluation Department.

43. Quality Assurance and Peer Review. Linda Adams, Principal Evaluation Specialist and Alexander Wellsteed, Principal Evaluation Specialist will be the commenters. Monika Huppi, Principal Advisor, at the Office of Validation and Evaluation of the Inter-American Bank and Stoyen Tenev, Senior Manager, at the Independent Evaluation Group of the World Bank will serve as external peer reviewers. H. Dissemination 44. The results of the evaluation will be disseminated in ADB and IED regional and online events. The results will also be shared with development partners through the COVID-19 Global Evaluation Coalition. Appendixes: 1. Synthesis Notes 2. Methodology and Indicative Evaluation Framework 3. Terms of References (available upon request) 4. Cost Estimates (available upon request)

cc: Director General; Deputy Director General; Advisor, IEOD; Directors, IESP; IETC; N. Gamo; A. Wellsteed,

IESP; J. Asquith; E. Bloom; M. Sutherland; L. Adams; S. Villena; M. Fortu, IETC.

Appendix 1 17

METHODOLOGY AND INDICATIVE EVALUATION FRAMEWORK

Table A1.1: Evaluation Work Streams Activity Description

Review of ADB’s COVID-19 Portfolio and progress reporting by governments and clients

The portfolio review is central the evaluation. It will provide descriptive data and allow analysis of how ADB’s response is organized. Detailed review of ADB’s support for COVID-19. Coverage includes (i) design of DMF, (ii) procurement details, (iii) gender and inclusiveness details, (iv) use of knowledge in project and/or program development, (v) details on coordination with partners. IED will collect periodic government and client reports on project and program implementation, as indicated in the DMF, to review the quality of data collection and reporting, and to monitor for early results and outcomes. The team will also review progress data from other sources. By December, the evaluation will finish the basic work for all projects, with updates as new projects are developed. Analysis will identify different ADB practices but will not provide ratings. By December 2020, prepare a short review of the status of the portfolio. This work will continue as ADB approves new operations and as new results are available. The final review will cover the complete ADB portfolio and incorporate the latest results available.

Staff and management interviews

Interviews with staff and management on their views regarding design, processing and implementation. IED will develop instruments to ensure consistency in questions. IED staff and consultants will schedule interviews with all team leaders and other key staff. Initial round of interviews will be conducted by November 2020. Aidditional interviews will continue after November to gather further information on implementation and new initiatives. Depending on the availability of resources, final analysis may utilize qualitative tools, such as MAXQDA.

Analysis of big data and data mining

This analysis will provide a better understanding of regional needs as seen by different stakeholders and as indicated by the data. Innovative use of big data to analyze public perceptions and to analyze emerging trends and project-related data. This will include data generated by ERCD as part of the Asian Development Outlook, the COVID-19 Policy Database, and other data sources. Areas of investigations for big data analysis include public awareness and sentiment analysis regarding the COVID-19 pandemic and the role of ADB in mitigating the virus. Data mining can help improve the understanding of the prevalence and incidence rates of the COVID-19, combined with basic epidemiological analysis.

Analysis of administrative data Administrative data include data on the allocation of ADB staff and consultants and the use of ADB budget. The analysis will allow a better understanding of how ADB has allocated its resources, staff and consultants, and how it has allocated its financial support, commitments, and disbursements.

18 Appendix 1

Activity Description

By December 2020, indication on the use of consultants and staff in projects. Administrative data will be used in the quick synopsis notes and in the final evaluation report, which will update any findings in the preliminary assessment.

Country level assessments The assessments will collect feedback from stakeholders and gather data to fill information gaps that are not covered in the desk review to allow for a more detailed review of design elements and understanding of the data related to ADB’s support. Detailed analysis of ADB operations in selected DMCs. IED will contract at least one high-level national consultant and a country assessment consultant in each country to collect and organize information. The Real-Time Methodology Adviser will support the national team in synthesizing and analyzing the country assessment. In some countries, IED will contract a second national consultant to focus on support for the private sector. Other team members, including IED staff and international consultants, will support the preparation of the country-level assessments. By December 2020, IED will have initial findings in at least one DMC. Additional results can be provided in the quick synopsis note as appropriate. The final report will include a single report on the consolidated findings in the country level assessments.

Literature and document review The evaluation will use available published literature, databases, and documents as necessary to complement other sources of data. Review of key elements from national and local programs to combat COVID-19 to identify lessons for ADB. Potential inclusion of additional DMC (Pakistan) as a special country-level assessment, as a desk review using international consultants.

ADB = Asian Development Bank, COVID-19 = coronavirus disease, DMC = developing member country, DMF = design and monitoring framework, ERCD = Economic Research and Regional Cooperation Department, IED = Independent Evaluation Department Source: Independent Evaluation Department.

Appendix 1 19

Table A1.2: Evaluation Matrix Evaluation Question Themes and Topics

How relevant has ADB’s response been to the needs of DMCs to tackle the COVID-19 crisis?

• The role of knowledge in ADB response to COVID-19. Review of how ADB has used knowledge in its response to COVID-19, including its use in development of the response and its sharing of knowledge. In the final evaluation report, review of ADB’s generation and sharing of knowledge to DMCs.

• The role of evaluation in ADB response to COVID-19. Review of past IED evaluations, covering (i) emergency assistance, (ii) financial crises, and (iii) pandemic support to identify key lessons and recommendations to identify how ADB incorporated lessons.

• Analysis of ADB support for health and social protection and support for vulnerable groups. Review of ADB’s approach to vulnerable groups in its response to COVID-19. The composition of the vulnerable population varies greatly from DMC to DMC. In all cases, it includes the elderly and those with preexisting health conditions. In many countries, migrant and informal workers, displaced people, Many young people may find themselves without access to either school or work options and, in all cases, it is important to consider the gender elements of the pandemic. This will require a literature review, a summary of ADB’s approaches, and a review of ADB’s projects. By December 2020, the evaluation will provide a first review of how ABD has approached gender issues and the inclusion of vulnerable groups.

• Adequacy of design and monitoring systems. The evaluation will review how ADB prepared and implemented DMFs at the operation level, using the portfolio review. The evaluation will include an initial review of DMFs by December 2020.

• Appropriateness of instruments and modalities. The evaluation will analyze the how ADB is allocating resources across its various instruments and modalities to support DMCs. Using the country assessments, the evaluation will explore how well governments understood the different options available.

How efficient and coherent is ADB being in allocating its resources to support the Asia and Pacific Region during the COVID-19 crisis?

• Efficiency of processes and arrangements. The portfolio analysis will analyze the process of designing and implementing support, identifying bottlenecks and how ADB addressed and overcame them.

• ADB coordination with development partners. Review how ADB has coordinated with other development partners and identifying good practices in design and implementation. By December 2020, a short analysis on ADB and coordination with development partners. In the final evaluation report, detailed analysis of ADB collaboration

How effectively on track towards the achievement of intended results and outcomes are ADB’s pandemic-related interventions?

• ADB support for private sector. Review how ADB has supported the private sector during the pandemic, through its direct NSO including financing to firms and through revolving programs. support to private sector programs. The evaluation will also review how ADB supports government-led programs to support the private sector, including analyzing modalities, targeting, and early results.

• Identification of outputs. The evaluation will identify the completion of outputs to understand factors in their successful delivery to identify potential lessons for future operations. The

20 Appendix 1

Evaluation Question Themes and Topics

evaluation will also review, to the extent possible, data on outcomes and possible attribution to ADB’s support.

• Quality of DMFs and availability of data. The evaluation will review the realism of data collection and the delivery of data through government reports The evaluation will review DMFs further in the country-level assessments, including exploring alternative data sources.

• Review of early results and progress towards achieving objectives. Using ADB and DMC reports as well as data from other sources, the evaluation will provide an analysis of early results associated with ADB support, including if ADB support has played a role in achieving these early results, and whether the support is likely to lead to desired outcomes later on.

How well prepared was ADB and how quickly has it been adapting to the challenges posed by COVID-19?

• Adequacy of human resources. The evaluation will review how ADB was able to staff its response to COVID-19. This will incorporate both administrative data and interviews. This may serve as a case study of how ADB implemented its Build-Buy-Borrow approach with COVID-19.

• Effectiveness of the One ADB Approach. The evaluation will review how ADB has used the One ADB approach to design and implement its support, identifying its contributions and potential trade-offs associated with simplified procedures.

• Relevance of procedures and tools. The evaluation will review the relevance of ADB’s pre-COVID-19 procedures and identify how they were modified to address the pandemic. By December 2020, the evaluation will prepare a brief overview of the role of new procedures and tools.

• Role of ADB’s health and social protection sectors. Review ADB’s preparedness and capacity in the social sectors prior to the pandemic, and the response of the sector to new demands placed on ADB.

• Financial impact of COVID-19. Review of the impact of COVID-19 on ADB’s balance sheet and (ii) an analysis of debt sustainability of DMCs. By December 2020, prepare a rapid analysis with more detailed analysis in the final report.

ADB = Asian Development Bank, COVID-19 = coronavirus disease, CPRO = Comprehensive Response to the COVID-19 Pandemic, DMC = developing member country, DMF = design and monitoring framework, IED = Independent Evaluation Department, NSO = nonsovereign operation. Source: Independent Evaluation Department.

Appendix 2 21

SYNTHESIS NOTES

1. As described in the evaluation approach paper, the evaluation will have a number of synthesis notes, case studies, and country assessments. They may be circulated as independent documents, included in the final evaluation document, or only used as evidence in the evaluation. The synthesis note will be short (limited to five to ten pages) and narrowly focused.

2. Since the evaluation will not have missions, the evaluation will adopt a flexible approach to collecting country and project level data. Instead of conducting in-depth country visits, the Independent Evaluation Department (IED) will identify several thematic areas of specific interest to the evaluation. The thematic synthesis notes will be organized by country to make distinct case studies. This is similar to the methodology of the case study approach that is commonly in qualitative analysis.1

3. Thematic topics. The synthesis notes will be classified in six themes:

(i) The contribution of ADB support for health. This will focus specifically on how ADB has strengthened the health sector in response to the pandemic. This includes both immediate response (such as the provision of personal protective equipment or testing capacity), support to prevent future public emergencies, and ongoing support of relevance. It will pay particular attention to the role of Asian Development Banks’s (ADB) staffing profile and experience in the health sector and how it contributed to ADB’s operations. It will also examine the early performance of an ongoing ADB regional vaccine project to understand what has worked well and what could be done differently by ADB in the context of a coronavirus disease (COVID-19) vaccine.

(ii) The contribution of ADB support for social protection measures targeting affected populations and vulnerable groups. Social protection is a main focus of many ADB operations, and the note will focus on how ADB has strengthened the social protection networks in developing member countries (DMCs). This will consider both support to existing programs as well as for programs that are introduced specifically to respond to the pandemic. The note will look at how ADB’s previous experience and support for social protection has contributed to its response.

(iii) The contribution of ADB support to the private sector. ADB is providing support both through its sovereign operations and, directly to enterprises, through its nonsovereign operations. The notes will consider how this is being done, both focusing on specific countries as well as on different types of support. The notes will take a particular focus on ADB’s support for small and medium enterprises through both its sovereign lending portfolio (for example, budget support operations) as well as nonsoveriegn operations (for example, support for financial intermediation).

(iv) The contribution of ADB support to economic stabilization. In its initial round of support, much of ADB’s support is in the form of budget support with the implicit objective to stabilize the economy and ensure adequate public financing during the

1 While the evaluation will not carry out academic-level case study research, it will use elements of this method to

ensure consistency and rigor. See, for example, R. Yin. 2017. Case Study Research and Applications: Design and Methods. Sage Publications.

22 Appendix 2

economic slowdown. The notes will focus on the underlying macroeconomic logic behind ADB’s support as well as the likely macroeconomic outcome.

(v) Country responses to COVID-19. These notes will explore cases of successful public health response to COVID-19, defined by reducing the number of new infections and controlling mortality. They will focus on the responses of low- and middle-income countries, in principle including nonmember countries of ADB. While the notes will include the role of ADB (if any) in the positive outcome, the main focus will be on the overall response to develop lessons for future outbreaks. These notes will consider ADB coordination with other development partners as well as lessons from government agencies and development partners.

(vi) Financial impact of COVID-19 on ADB’s finances. This note will explore ADB’s finances in the context of potential for further rounds of support. This will include a study of ADB’s stress tests, recent third-party credit reports, and its overall credit adequacy in the face of future crises. The note will also analyze how COVID-19 operations have affected the financing of different funding windows, such as the Technical Assistance Special Fund, private sector resolving funds, and humanitarian diester response funds.

(vii) Humanitarian disaster response. The notes will consider the use of specific modalities, including emergency assistance loans and budget support. It will play particular attention to the role of ADB’s staffing profile and experience in supporting humanitarian emergencies and disaster responses and how these loans contributed to ADB’s operations. IED is also exploring the lessons in vaccine financing and design, which may be presented separately.

(viii) Innovative ADB approaches. The unexpected outbreak of the pandemic forced ADB, as an institution, to adopt a number of different approaches to ensure its continuity and capacity to operate under stress. In addition, to its headquarters in Manila, ADB has a large number of resident missions and offices both in the region and outside. These notes will explore the steps that ADB has taken to ensure that it and its staff can continue to operate and carry out their functions.

4. Table A1 outlines the initial proposal for synthesis notes, organized by ADB region. This is list is subject to change depending on demand and the availability of resources.

Table A1: Potential Synthesis Notes

Region

Contribution of ADB support on Country Response to

COVID-19

Innovative ADB

Approach Health Social

Protection Private Sector Economic

Stabilization

CWRD Afghanistan

Uzbekistan Pakistan

EARD Mongolia Mongolia PRC

PARD PNG

PNG Fiji SARD Bangladesh

India

SERD Lao PDR Philippines

Philippines Cambodia

Other Vaccine project

Gender Direct COVID-19 response

Regional cooperation

EAL

Support for RM staff

COVID-19 = coronavirus disease, CWRD = Central and West Asia Department, EAL= Emergency Assistance Loan, EARD = East Asia Department, PARD = Pacific Department, PNG = Papua New Guinea, PDR = People’s Democratic Republic, PRC = People’s Republic of China, RM = resident mission, SARD = South Asia Department, SERD = Southeast Asia Department.. Source: Independent Evaluation Department.

Appendix 2 23

5. Methodology. The synthesis notes are not intended as pieces of independent research or evaluations. They will rely primarily on evaluation evidence collected for the evaluation, such as interviews, background literature, and administrative data. For certain notes, the evaluation team will gather additional evidence including Report and Recommendation of the President to the Board of Directors, Project Administration Manuals, back to office reports among others. In some cases, the synthesis notes will require interviews with DMC officials and the evaluation will coordinate with the appropriate ADB staff to coordinate. This may include analysis of social media (Facebook, Twitter, etc.) as well as analysis of other data sources. IED will also carry out a literature review as appropriate and will cooperate with other ongoing IED evaluations. 6. An IED staff member will be responsible for preparing each synthesis note. No travel is expected, but the evaluation may contract consultants in the selected countries as needed. To the extent possible, the evaluation will seek peer review and feedback from ADB staff both to receive comments and to confirm the details. In other cases, IED may also employ consultants to provide to provide work and to review the findings. IED will consult with the ADB Board and Management to decide which notes should be released publicly.