1 APPROACH PAPER Evaluation of the World Bank Group’s Assistance to Islamic Republic of Afghanistan, 2002-2011 Introduction 1. Evaluation Context: The Country Program Evaluation (CPE) will assess the World Bank Group‘s development assistance to Afghanistan covering the ten-year period 2002-11 since the Bank‘s re- engagement in the country. The sociopolitical and economic environment remains volatile with periodic outbreaks of armed conflict. External partners maintain a state of high alert due to the security situation and are severely restricted in their movements limiting the ability to undertake substantial fieldwork. During this period the country has relied extensively on the support of the international development community, including the World Bank Group. During 2002-2011, the World Bank Group provided both emergency assistance and development financing to support development of Afghan institutions and mitigate poverty. While real GDP growth averaged 12 percent a year during the past decade, about 36 percent live below the poverty line. Since 2002 the Bank Group has committed US$2.4 billion in IDA credits and grants and some US$100 million in IFC investments, issued US$78 million in MIGA guarantees, administered larger sums of funding from other donors through trust funds (over US$4 billion). This has been complemented by an extensive program of analytical and advisory activities. 2. Scope of the evaluation: This evaluation includes the period immediately following the Bank Group‘s re-engagement in the country through the present. The Bank Group‘s initial re-engagement was outlined in two TSSs, one covering FY02-03 and a second FY04-05. Two Interim Strategy Notes (ISNs)—one covering FY06-08 and one FY09-11— have organized the Bank Group‘s strategy for Afghanistan into three pillars: (1) Building the capacity of the state and its accountability to its citizens, (2) Promoting growth of the rural economy and improving rural livelihoods, (3) Supporting growth of the formal private sector (including infrastructure). This evaluation will evaluate the three pillars defined by the ISNs. 3. The evaluation will assess the extent to which the outcomes associated with the three pillars of the World Bank Group‘s assistance to Afghanistan since 2002 have been achieved. However, given the high degree of insecurity, the externalities created by the ongoing conflict, and the implementation challenges it has faced in Afghanistan, CPE will place more emphasis on learning from the achievements and limitations of the Bank Group‘s assistance during the period FY02-11. IEG will review individual projects and programs to assess their relevance to, and effectiveness in, the uncertain and fluid political and economic environment prevalent in Afghanistan. Given the large volume of analytical and advisory activities (AAA) and the limited access to the field to assess project performance, the evaluation will supplement the review of the Bank‘s strategy and lending portfolio with an assessment of the Bank Group‘s non-lending activities more than is typically the case in a country program evaluation. The aim of the evaluation is to inform the strategy for the next phase currently being developed by the country team and to derive lessons to improve performance and enhance the Bank‘s effectiveness in fragile states. 4. The central tenet of the World Development Report 1 (2011) is that ―strengthening legitimate institutions and governance to provide citizen security, justice and jobs is crucial to break cycles of violence.‖ Even though all of these were not explicit objectives of the Bank Group‘s strategy, to the extent feasible, the evaluation will generate evidence through the evaluation to assess the relevance of the 1 World Development Report 2011: Conflict, Security and Development . Washington DC: The World Bank.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
APPROACH PAPER
Evaluation of the World Bank Group’s Assistance to
Islamic Republic of Afghanistan, 2002-2011
Introduction
1. Evaluation Context: The Country Program Evaluation (CPE) will assess the World Bank
Group‘s development assistance to Afghanistan covering the ten-year period 2002-11 since the Bank‘s re-
engagement in the country. The sociopolitical and economic environment remains volatile with periodic
outbreaks of armed conflict. External partners maintain a state of high alert due to the security situation
and are severely restricted in their movements limiting the ability to undertake substantial fieldwork.
During this period the country has relied extensively on the support of the international development
community, including the World Bank Group. During 2002-2011, the World Bank Group provided both
emergency assistance and development financing to support development of Afghan institutions and
mitigate poverty. While real GDP growth averaged 12 percent a year during the past decade, about 36
percent live below the poverty line. Since 2002 the Bank Group has committed US$2.4 billion in IDA
credits and grants and some US$100 million in IFC investments, issued US$78 million in MIGA
guarantees, administered larger sums of funding from other donors through trust funds (over US$4
billion). This has been complemented by an extensive program of analytical and advisory activities.
2. Scope of the evaluation: This evaluation includes the period immediately following the Bank
Group‘s re-engagement in the country through the present. The Bank Group‘s initial re-engagement was
outlined in two TSSs, one covering FY02-03 and a second FY04-05. Two Interim Strategy Notes
(ISNs)—one covering FY06-08 and one FY09-11— have organized the Bank Group‘s strategy for
Afghanistan into three pillars: (1) Building the capacity of the state and its accountability to its citizens,
(2) Promoting growth of the rural economy and improving rural livelihoods, (3) Supporting growth of the
formal private sector (including infrastructure). This evaluation will evaluate the three pillars defined by
the ISNs.
3. The evaluation will assess the extent to which the outcomes associated with the three pillars of
the World Bank Group‘s assistance to Afghanistan since 2002 have been achieved. However, given the
high degree of insecurity, the externalities created by the ongoing conflict, and the implementation
challenges it has faced in Afghanistan, CPE will place more emphasis on learning from the achievements
and limitations of the Bank Group‘s assistance during the period FY02-11. IEG will review individual
projects and programs to assess their relevance to, and effectiveness in, the uncertain and fluid political
and economic environment prevalent in Afghanistan. Given the large volume of analytical and advisory
activities (AAA) and the limited access to the field to assess project performance, the evaluation will
supplement the review of the Bank‘s strategy and lending portfolio with an assessment of the Bank
Group‘s non-lending activities more than is typically the case in a country program evaluation. The aim
of the evaluation is to inform the strategy for the next phase currently being developed by the country
team and to derive lessons to improve performance and enhance the Bank‘s effectiveness in fragile states.
4. The central tenet of the World Development Report1 (2011) is that ―strengthening legitimate
institutions and governance to provide citizen security, justice and jobs is crucial to break cycles of
violence.‖ Even though all of these were not explicit objectives of the Bank Group‘s strategy, to the
extent feasible, the evaluation will generate evidence through the evaluation to assess the relevance of the
1 World Development Report 2011: Conflict, Security and Development. Washington DC: The World Bank.
2
World Bank Group‘s support to Afghanistan to this framework and the implications for the future strategy
of the Bank Group. The lessons drawn from the evaluation will be relevant to the future implementation
of the WDR.
5. The CPE will be delivered to CODE at the end of FY12 and inform the forthcoming IEG
evaluation of fragile and conflict affected situations. IEG will also provide inputs based on preliminary
findings for the Board discussion of the Interim Strategy Note (ISN) on Afghanistan, currently expected
in the second half of this fiscal year. To accommodate the timing of the ISN, IEG is conducting a desk
review of the lending portfolio and of the AAA work whose findings based on desk work will be shared
with the country team ahead of the CODE discussion. Additional evaluative work will be undertaken
during field visits to Afghanistan in January-February 2012 to update the findings from the desk review.
The full country program evaluation will be completed and submitted to CODE by June 2012.
Country Context
6. Geography: Afghanistan is a landlocked and mountainous country located in South Asia. It
shares borders with Pakistan and China in the east, Iran in the west, and Turkmenistan, Uzbekistan and
Tajikistan in the north. Afghanistan has a land area of 652,230 square kilometers with a population of
about 29.8 million.2 Due to its unique geographic location—between the Middle East, Central Asia, and
the Indian subcontinent, Afghanistan has long been fought over—despite its rugged land. The political
history of modern Afghanistan began in 1747 with unification of the Pashtun tribes. In the 19th century,
the country served as a buffer between the British and Russian Empires until it won independence from
British control in 1919. Afghanistan became a key Cold War battleground after the Soviet Union sent its
army in 1979 to support Communist rule, leading to a long and destructive war. The USSR withdrew its
troops in 1989 under the pressure of US-supported anti-Communist Mujahedin fighters. After several
years of civil war, the Taliban, a hard-line Islamic movement that grew out of the Afghan refugee camps
in Pakistan, came to power in 1996. Following the September 11, 2001 terrorist attacks in the United
States, military action by the United States, Allied, and the anti-Taliban Northern Alliance overthrew the
Taliban in retaliation for sheltering Osama Bin Laden. Following an initial period of low level
insurgency after the fall of the Taliban regime, its adherents regrouped and conflict intensified since 2006,
particularly in the south and east of the country. As a result, Afghanistan has undergone a very tangible
deterioration in security over the last four years which is reported by the Country team to have impacted
the Bank Group‘s work through increased risks faced by staff, restrictions on movement, further
constraints on supervision, and additional measures to deal with safety. The Afghan Government,
currently led by President Hamid Karzai, has struggled to extend its authority beyond the capital and to
forge national unity.
7. Political developments: The UN-sponsored Bonn Conference in 2001 established a process for
political and economic reconstruction. Following the collapse of the Taliban regime, Afghanistan‘s
constitution was amended to re-establish the country as an Islamic republic with democratic elections for
the National Assembly (parliament) and the presidency. In 2002, Hamid Karzai was chosen by the Loya
Jirga as interim President, largely due to support from foreign backers. Hamid Karzai went on to become
the first democratically elected president of Afghanistan and the National Assembly was inaugurated in
2005. Karzai won a second five-year term as president in the 2009 election, whose results were disputed
and accepted somewhat reluctantly. Despite gains toward building a stable central government,
Afghanistan remains fragile due to continued instability in the south and east. Tensions between the
administration based in the capital, Kabul, and powerful local figures are also apparent in most other parts
of the country. The president remains dependent on support from foreign backers. In the field of security
the Government receives military assistance from the US and others, notably through NATO‘s
2 CIA World Fact Book – Afghanistan, August 2011.
3
International Security Assistance Force (ISAF), both for maintaining security and to build the capacity of
the Afghan National Army. The Government also receives budgetary support from donors, without
which the fiscal situation would be unsustainable. The international community has pledged over US$67
billion at four donor conferences since 2002.3
8. The last international conference on Afghanistan was held in Kabul in June 2011, bringing
together representatives from 50 countries and multilateral organizations for the purpose of discussing the
next stage in Afghanistan‘s reconstruction, with another in Bonn in December 2011. Peace talks, regional
cooperation, the role of neighboring countries in Afghan peace efforts, the security handover and the
continuation of the strategic partnership between Afghanistan and the international community beyond
2014 were the main issues discussed at the summit.4 In July 2011, ISAF completed the first phase of
transition through the handover of security responsibilities to Afghan national security forces in six
locations (Bamiyan province; Parwan province; Herat city; Mehterlam city in eastern Laghman province;
Laskargar city in southern Helmand, and Panjshir province). The transfers are the first phase of a plan to
place the country‘s security under Afghan control in the next three years. International combat troops are
scheduled to leave Afghanistan by the end of 2014.5
9. Afghan economy: Development of the Afghan economy has been constrained by the volatile
political situation. The country‘s economic indicators have improved since the fall of the Taliban regime
in 2001 largely due to the infusion of billions of dollars in international assistance, the recovery of the
agricultural sector, and rapid growth in the services sector. The Afghan economy has been gradually
recovering and progressing toward macroeconomic stability. Real GDP growth has averaged around 12
percent a year during the past decade. Despite this double digit growth, the economy has suffered great
volatility because of its heavy reliance on agriculture, which is subject to weather fluctuations. With 80
percent of the population living in rural areas, the agriculture sector accounts for around one third of the
country‘s economy (excluding the illicit drug sector).6 The importance of agriculture in Afghanistan‘s
economy remains critical, 36 percent of households relying on farming as their mains source of income
and another 6 percent depending on farm wages as their income source7. Real GDP dropped significantly
in 2008 owing to severe drought but bounced back to record growth at 22.5 percent in 2009, reflecting
recovery from the poor harvest of the previous year, following which it was projected to drop back within
the range of the previous years (Figure 1).
10. The Afghan economy is heavily dependent on inflows of foreign aid to finance the government
budget and cover the massive current-account deficit. Dependence on imports makes the country
vulnerable to global price trends, particularly for food and fuel. However, donor commitment has been
strong, and the postwar reconstruction and the return of many skilled and entrepreneurial individuals have
stimulated economic activity. Development aid has focused on core infrastructure, including roads,
airports, schools, the power sector, water supply and healthcare. Dominant economic activities have been
agribusiness, construction, trading and transportation. The degree to which these activities are dependent
upon the currently large presence of the foreign aid community will influence their sustainability. A
number of new economic sectors—telecommunications, banking, food-processing, and mineral (including
natural-gas) extraction—have shown potential.8 Nevertheless, the Afghan economy has been dominated
by the informal sector in agriculture and also in mining, manufacturing, construction and infrastructure.
The investment climate is poor, the private sector is not well developed and export development has been
3 Ibid.
4 Afghanistan Quarterly Country Update #39, The World Bank, July 2011.
5 Afghanistan – World Bank Country Brief, August 2011.
6 Afghanistan – World Bank Country Brief, January 2011.
7 Poverty Status in Afghanistan, The World Bank, 2010.
8 Ibid.
4
modest. The cost of doing business remains high and economic activity is subdued in several areas of
critical importance for growth and sustainability. In particular, no new exports outside the traditional
base (carpets, minerals, and horticulture products) have been developed, owing to low capacity, the high
cost of capital, and lack of electricity.9
Figure 1. Real GDP Growth (percent)
Source: IMF World Economic Outlook, October 2010. *Projections Figure 2. GDP per capita, Real (Constant 2002 US$) vs. Nominal (Current US$)10
Source: WB World Development Indicators, August 2011.
11. The country‘s biggest economic sector—agriculture—is linked to opium production, and the
volatile security situation and the persistence of the drug economy are weakening attempts at broadening
economic development: Afghanistan produces roughly 90 percent of the world‘s opium, and many
influential persons are alleged to have ties to opium trade. Opium production increased from 185 metric
tons in 2001 to 8,200 metric tons in 2007.11
Gross revenues from the drug trade are estimated to be over
one-third of official GDP, with 2.4 million people and 370,000 households directly involved in poppy
cultivation, indirectly affecting the livelihoods of a large part of the population.12
Opium is grown largely
in southern provinces and has become a key source of revenue for the Taliban and other antigovernment
groups inside Afghanistan.
12. Poverty: Despite the economic progress of the past few years, Afghanistan is extremely poor
and highly dependent on foreign aid. Real GDP per capita increased by 75 percent from 2002 to 2009
9 Afghanistan: Article IV Staff Report, IMF, 2008.
10 During 2008-2009, the GDP deflator was extremely volatile, driven largely by the sudden influx of foreign aid,
resulting in the higher than expected increase in real GDP. 11
Ibid. 12
Afghanistan 2009 ISN, The World Bank.
15.1
8.8
16.1
8.2
14.2
3.4
22.5
8.9*
6.8*
0
5
10
15
20
25
2003 2004 2005 2006 2007 2008 2009 2010 2011
174.6 180.1 196.9211.6 227.9 222.0 215.9
306.4174.6 187.0218.3
254.3280.6
344.6405.1
486.0
0
100
200
300
400
500
600
2002 2003 2004 2005 2006 2007 2008 2009
US
D
Real GDP per Capita (Constant 2002 US$) Nominal GDP (current US$) per Capita
5
(Figure 2). However, Afghanistan lags behind other South Asia countries on major social indicators and
living standards are among the lowest in the world: roughly 36 percent live below the poverty line (9
million Afghans are unable to meet their basic needs). More than half of the population is at risk of
falling into poverty. Regional and seasonal differences are important aspects of poverty in Afghanistan,
which has a diverse terrain, climate, and agricultural potential. The three provinces in high mountainous
areas – Bamyan, Daikundi and Badakshan – are often blocked due to heavy snow accumulation in winter
and have poverty rates that are much higher (45 percent) than the national average. Poverty rates are also
higher in the south and east of the country, and are lowest in the southwest. Seasonal surveys have
demonstrated that the poverty rate varied from 42 percent in the lean spring period to 21 percent in the
summer of 2007. Forty percent of Afghan households do not receive any revenue from their principal
income source during winter months, rural households being most exposed to seasonality.13
Poverty data
are not disaggregated by ethnic group but the Kuchi (nomadic pastoralists) are reported to have the
highest incidence of poverty (54 percent). Among the Pashtun tribes those in the east live in high plateaus
or mountain areas and are among the poorest while those in the southwest are not.
13. Much of the Afghan population continues to suffer from shortages of housing, clean water,
electricity, medical care, and jobs. Life expectancy is at 44.6 years,14
compared to an average 59 years for
low-income countries. Only about 30 percent of Afghans have access to electricity;15
27 percent have
access to safe drinking water and 5 percent to adequate sanitation.16
In the field of education, 34.9
percent of children are enrolled and regularly attend primary school; 37.6 percent of the young adult
population (aged 15 to 24) is able to read and write.17
Afghanistan‘s unemployment rate hovers at around
40 percent, but there is little hard data on unemployment.18
Insecurity, weak governance, and the Afghan
Government‘s inability to extend rule of law to all parts of the country pose challenges to future economic
growth.
14. Patterns of exclusion and vulnerability in Afghanistan have a strong gender dimension, women
having limited access and command over productive resources. The female literacy rate is 22 percent for
women compared to 51 percent for men, and the enrollment rate for girls is 31 percent compared to 43
percent for boys. Improvement in educational outcomes is stronger for men than women suggesting a risk
of widening gender gaps. Access to education is similar for children of poor and nonpoor households but
in healthcare there are stark differences in access by poor and nonpoor households.19
World Bank Group Objectives and Activities in Afghanistan in 2002-11
15. Bank Group assistance: The 2002 and 2003 Transitional Support Strategies (TSS) were
designed to support rebuilding of essential governance institutions and capacity with a focus on financial
management and public administration, to initiate emergency reconstruction programs to help restore
livelihoods, essential infrastructure and social services, and to enable private sector development. Since
2006, Bank Group support has shifted from emergency reconstruction activities under the TSSs to
development support guided by Interim Strategy Notes (ISN).The 2006 and 2009 ISNs, aligning with the
Afghanistan National Development Strategy, focused on three strategic pillars:
13
Poverty Status in Afghanistan, The World Bank, 2010. 14
Human Development Report, UNDP, 2010. UNDP‘s Human Development Index ranks Afghanistan 155 out of
169 countries. 15
The Afghan Ministry of Energy and Water, 2010. 16
Afghanistan – World Bank Country Brief, August 2011. 17
Afghanistan – World Bank Country Brief, January 2011. 18
Economist Intelligence Unit: Afghanistan Country Profile 2008. 19
Poverty Status in Afghanistan, The World Bank, 2010.
6
Pillar 1: Building the capacity of the state and its accountability to its citizens;
Pillar 2: Promoting growth of the rural economy and improving rural livelihoods;
Pillar 3: Supporting growth of the formal private sector.
Table 1. Total Net Disbursements of Official Development Assistance and Official Aid, 2002–09
Source: OECD Development Assistance Committee Database as of August 2011 *DAC= Development Assistance Committee of Organisation for Economic Co-operation and Development (OECD)
16. Since 2002,20
the World Bank has committed about US$2.4 billion for 85 development and
emergency reconstruction projects and four budget support operations. This support comprises over
US$1.9 billion in IDA grants and US$436.9 million in IDA credits, financing 89 operations over
FY2002-2011. In August 2011, the Bank had 25 active IBRD/IDA projects in Afghanistan with net
commitments of over US$1 billion.21
The World Bank also administers the Afghanistan Reconstruction
Trust Fund (ARTF) which is supported by 32 donor countries. The ARTF has mobilized over US$4.2
billion since 2002, and serves as one of the main instruments for financing the country‘s recurrent budget
and investment needs.22
As of July 2011,23
IFC had committed US$100.3 million for eight private sector
operations (seven of which are currently active), focusing mainly on the financial and banking sector.
20
Afghanistan became a member of the World Bank in 1955. Shortly after the Soviet invasion in 1979, World Bank
operations were suspended although the Bank continued to provide assistance through the Bank office in Pakistan.
Prior to 1979, the World Bank had supported 21 operations through concessional financing (credits) to Afghanistan
across a wide range of areas including education, roads, and agriculture. Of the original $230 million in credits
approved by the International Development Association (IDA), the Bank‘s concessionary lending arm, $83 million
was disbursed and $147 million was subsequently canceled. Afghanistan had repaid $9.2 million to IDA and was
up-to-date on debt service payments until June 1992 when it stopped making payments. Operations were resumed in
May 2002. 21
Afghanistan – World Bank Country Brief, August 2011. 22
Afghanistan Quarterly Country Update #39, The World Bank, July 2011. 23
Afghanistan became a member of IFC in 1957.
7
Combined World Bank and IFC commitments, including those from Bank-administered trust funds,
totaled US$6.6 billion during the review period. MIGA has issued six guarantees for five projects (three
remain active) in FY2002-2011. World Bank Group institutions have also undertaken 107 separate
analytical and advisory activities (AAA) to assist the Afghan Government.
Figure 3. World Bank Commitments (IDA grants and credits and ARTF) , FY2002-2011
17. World Bank financing: World Bank financing in Afghanistan during the 2002-11 period
focused on support to economic recovery, post-conflict reconstruction and delivery of basic services.
Resources allocated through ARTF and IDA grants and credits financed 89 projects. Bank Group
engagement under the review period can be roughly divided into two phases: in 2002-05, the Bank
strategy focused on the restoration of partnership with the Afghan Government and provision of IDA
support for urgent recovery and technical assistance needs; in 2006-11, strategic directions included
assistance to support the country‘s transition from its emergency status as a conflict-affected country to a
more ―normalized‖ situation and acceleration of progress in state building and service delivery.
18. Total commitments to Afghanistan in FY02-06 reached US$1043.2 million, comprising
US$606.8 million in IDA grants and US$436.4 million in IDA credits.24
However, despite the provision
of support for health, education, microfinance, irrigation and rural livelihoods, macroeconomic and public
financial management in 2002-06, many development challenges remained by the end of the period.
Limited capacity of state institutions, weak systems and procedures, the poor security situation and the
inability of the Government to demonstrate its presence around the country can be expected to continue to
constrain economic development in the foreseeable future.
24
Afghanistan 2006 ISN, The World Bank.
Source: World Bank data as of July 2011 *Pillar 1: Building the capacity of the state and its accountability in provision of services to citizens Pillar 2: Promoting growth of the rural economy and improving rural livelihoods Pillar 3: Supporting growth of the formal private sector ** “Financial Management” is a composite of 35% in Recurrent and Capital Costs support and 1% in Financial Manangement Sector commitment ***: “Other” includes Public Sector Governane (2%), Water (2%), Information Communications Technology (1%) and lesser investments in Social Development, Urban Development, Poverty Reduction, and Social Protextion
Pillar 152%Pillar 2
32%
Pillar 316%
By Pillar* (percent)
Financial Management
36%
Agriculture & Rural Devt
26%
Transport9%
Energy and Mining
5%EDU5%
Economic Policy
4%
FPD4% HNP
4%
Other***7%
By Sector Board**Financial Management
Agriculture & Rural Devt
Transport
Energy and Mining
Education
Economic Policy
Financial & Private Sector Devt
Health, Nutrition & Population
Other***
8
19. In FY07-11,25
the Bank committed more than US$1 billion in IDA grants and credits for lending
activities. The World Bank has been the largest international source of funds (through IDA, ARTF, and
JSDF) for the National Solidarity Program (NSP), a Government flagship program, which finances small-
scale reconstruction and development activities identified by Community Development Councils (CDCs)
across the country. By the end of FY11, the NSP is reported to have reached about 70 percent of the
Afghan rural population located in over 27,000 communities.26
Figure 4. ARTF: Donor Contributions (US$ Million) as of March 2009
Source: Afghanistan 2009 ISN: WB data *Other donors: Italy, Australia, Denmark, Spain, Finland, Saudi Arabia, Kuwait, Ireland, Luxembourg, Korea, Belgium, France, Japan, Russia, UNDP, Poland, India, Portugal, Iran, Switzerland, New Zealand, Bahrain, Turkey.
20. World Bank Administered Trust Funds: The ARTF, the largest contributor to the Afghan
budget, has mobilized funds from 32 donors totaling US$4.2 billion since its establishment in 2002.
More than US$2.29 billion has been disbursed to the Government of Afghanistan to help cover recurrent
costs, such as civil servants‘ salaries, and US$1.9 billion made available for investment projects, of
which 446.84 million are in the active portfolio.27
ARTF investment projects have been aligned with
Government priorities, including agriculture and rural development, private sector development, capacity
development, education, urban development, transport and energy. In 2008, ARTF introduced a policy-
content-based program known as the ARTF Incentive Program (ARTF IP) as a mechanism for
coordinated multi-donor policy dialogue with the Ministry of Finance. The program has allocated
additional (discretionary) funding (around US$70 million per annum) on a performance basis in line with
the Afghan budget cycle, supporting government-led reforms in customs and revenue collection, the
corporatization of public utilities, and the strengthening of the regulatory environment for extractive
industries. The World Bank has facilitated meetings of a donor working group that has negotiated reform
25
As of January 2011. 26
NSP (US$398 million IDA Grant, US$618 million ARTF Grant, and US$15 million JSDF Grant) is active in all
34 provinces and reached over 18 million Afghans by December 2010. Approximately 57,400 community
subprojects have been partially or fully financed, of which some 46,031 subprojects have been completed. About 80
percent of the subprojects involve infrastructure such as irrigation, rural roads, and water supply, all critical for the
recovery of the rural economy, stability, and governance. Afghanistan Quarterly Country Update #39, The World
Bank, July 2011. 27
Afghanistan Quarterly Country Update #39, The World Bank, July 2011.
United Kingdom 24%
United States 17%
Canada 13%
Netherlands 10%
EC/EU 10%
Germany 7%
Norway 5%
Sweden 3%
Others*11%
9
benchmarks with the Afghan Ministry of Finance. The Bank also administers the Japan Social
Development Fund (JSDF), established in 2002. The JSDF has provided a special window for
Afghanistan which operates in a harmonized manner to support the national programs of the Government.
As of May 2011, JSDF‘s total commitment to the country since its establishment reached $81.3 million.28
21. World Bank AAA: The Bank‘s
financial support to Afghanistan has been
supplemented by a program of policy advice,
analytical work and strong coordination with
other donors throughout the review period.
Ninety separate AAA tasks—53 Economic and
Sector Work (ESW) and 37 Technical
Assistance (TA) activities—have been
conducted at a cost of more than US$16 million
during FY2002-11. All TA activities and 50
ESW products were delivered to the client by
the end of FY11. The largest share of AAA
products were on public sector governance
(16%), followed by economic policy (13%),
financial and private sector development (11%),
energy and mining (10%), ICT (8%), and
agriculture (7%). Eleven other sectors including
education, health, environment, poverty, gender
and other infrastructure sectors account for the
remaining 35 percent.
22. In the beginning of the review period,
Bank AAA activities were designed to build the
Bank‘s knowledge base and provide the
analytical underpinnings for the work of the
international community and future Bank assistance. AAA was also intended to support coordination of
donor aid efforts under Government leadership within a comprehensive development framework (CDF).29
Bank strategy documents indicate that in FY2002-06, the Bank focused AAA on quick but practical
technical inputs and advice on specific policy, institutional design and reform issues; in FY2007-11, the
Bank‘s AAA was designed to include a mix of ―just-in-time‖ technical assistance, together with a limited
number of flagship reports to guide the Bank‘s strategy and shift the focus to more immediate outputs
responding to the demands of the Government and donors. In-depth technical work was also conducted
in response to Government requests in key sectors such as public administration and financial
management, private sector development, and infrastructure. The World Bank Institute‘s (WBI) capacity
building programs complemented projects and AAA during FY2002-11. WBI provided training and
technical assistance in the areas of public finance management, urban planning and management, health,
education, poverty reduction, private sector development, and anti-money laundering.
23. Donor coordination: According to the ISNs, the Bank has worked closely with other
multilateral and bilateral agencies and has advocated building capacity and legitimacy of the state by
channeling donor resources through the Government budget to ensure investments are aligned with
28
Ibid. 29
The CDF approach was introduced by the World Bank in 1999, and brings together four principles to improve the
effectiveness of development assistance in reducing poverty: a long-term, holistic framework; results orientation;
country ownership; and country-led partnership.
Figure 5. Distribution of World Bank AAA Products, FY2002-2011
Source: World Bank data as of July 2011 Pillar 1: Building the capacity of the state and its accountability in provision of services to Afghan citizens Pillar 2: Promoting growth of the rural economy and improving rural livelihoods Pillar 3: Supporting growth of the formal private sector
Pillar153%
Pillar28%
Pillar339%
By Pillar* (percent)
10
national priorities. Despite the priority attached by the Government to aid effectiveness, aid management
has been challenging. The presence of around 60 international donors in Afghanistan has sometimes led
to fragmentation of aid efforts. More than two-thirds of international assistance bypasses government
systems, undermining government-led coordination and government‘s accountability for results.30
Following United Nations Security Council Resolution 180631
and recommendations of the June 2008
Paris Conference in Support of Afghanistan, the United Nations Assistance Mission in Afghanistan
(UNAMA) was given the mandate to coordinate donor assistance. Although UNAMA has the overall
lead on donor coordination in the country, the Bank‘s role as administrator of the multi-donor ARTF
provides it with a key coordinating role. The Bank Group also leads or participates in a number of the
UNAMA task forces, for example, the agriculture and sub-national governance task forces.
24. IFC program: IFC committed US$100.3 million32
in eight operations during the review period.
IFC has been active in Afghanistan since 2004, despite political instability and poor security, lack of
reputable local sponsors and low interest from international companies in investing capital which
impeded private sector development. IFC‘s program has included investments and advisory services
through the regional PEP-MENA facility. Addressing private sector recovery needs, IFC has scaled up its
investment activities over the last few years, with its committed portfolio growing from US$8 million in
2004 to US$63 million in 2009.33
In FY04-06, the IFC‘s investment program focused on the financial
sector, including small and medium enterprises (SMEs), microfinance, and the establishment of a
commercial bank and a business hotel. Since 2007, IFC has contributed to the development of the
financial sector through support for small business and trade finance, telecommunications, hospitality,
and healthcare sectors. IFC‘s current investment portfolio totals more than US$90 million in six
companies. In many cases IFC investments are linked with technical assistance to its private sector
clients. IFC‘s advisory services—fifteen products (eight delivered to the client and seven ongoing tasks)
amounting to US$7.5 million during the review period—focused on improving the investment climate
and assisting the Government in establishing regulatory frameworks, increasing access to finance and
strengthening the financial sector. This included capacity building of various financial institutions, and
practical business skills training programs.
25. MIGA program: MIGA has issued six guarantees for five projects in Afghanistan since 2004
but is reported to be facing difficulty in attracting foreign direct investment because of security
difficulties and the crowding out effect of substantial donor grants. The Agency has supported its Bank
Group partners and the donor community by facilitating foreign direct investment, including through the
Afghanistan Investment Guarantee Facility (AIGF), established in December 2004.34
The project has
been designed to stimulate private business activity and attract foreign private investment in a variety of
sectors. In early 2006, MIGA, using its own resources and those of the AIGF, approved its first
guarantees. The total associated foreign investment of the projects under the AIGF is about US$107
million with a self-reported estimate of 850 jobs created as a result.35
MIGA‘s current portfolio of
guarantees, totaling US$78.2 million, consists of five investments in support of the country‘s
agribusiness, financial, infrastructure, and services sectors. MIGA and DfID have conducted an
independent evaluation of the AIGF, providing recommendations to the participating donors whether to
30
Afghanistan 2009 ISN, The World Bank. 31
March 2008, reaffirmed in Resolution 1868 in March 2009. 32
As of July 2011. 33
Afghanistan 2009 ISN, The World Bank. 34
The facility is administered by MIGA and jointly funded by the Government of Afghanistan, the Asian
Development Bank (ADB), IDA, and the UK Department for International Development (DfID). Afghanistan 2009
ISN, The World Bank. 35
Afghanistan Quarterly Country Update, The World Bank, January 2011.
11
extend the facility past its end of 2009 expiration date. At the request of the Government of Afghanistan,
the closing date of AIGF has been extended for two years—to September 30, 2011.
26. Risks: Overall, the World Bank Group‘s involvement in Afghanistan has been characterized by a
need to adapt to the highly unpredictable political and economic environment. Bank Group strategic
documents have traditionally included a chapter on risk management and mitigation measures and sought
to balance rapid results with sustainability, focusing on high-impact activities in key sectors and building
the Government‘s ownership of the reconstruction program. Throughout the past decade, the Bank Group
has identified the major operational risks—such as the fragile peace process and poor security situation,
limited institutional capacity and an undersupply of qualified government staff, fiduciary risks and
corruption of the Government, the opium/criminal economy, and others—and tried to mitigate them,
particularly through AAA products and continued dialogue among the development partners.
27. Previous evaluations: IEG completed ICR reviews for 20 projects, 85 percent of which were
rated as satisfactory or better. Three more ICR reviews are currently under preparation. Two external
evaluations have been carried out of the ARTF (Scanteam 2005, 2008), and reported that the mechanism
was efficient and effective in supporting the recurrent costs of the Government and improving
coordination and harmonization efforts. These findings appear to stand in contrast to similar evaluations
of the World Bank-administered trust funds in Sudan where disbursement was slow due to bureaucratic
procedures which caused serious delays and inefficiencies. The evaluation of funds‘ rules and procedures
appear more suited to medium-term reconstruction and development than post-conflict recovery.36
A 2007
Donors' Committee meeting recommended extending the ARTF to 2020. The 2008 evaluation
recommended a gradual transition towards an integrated, program-based ARTF; the development of an
ARTF financing strategy based on clear criteria; focusing ARTF resources on thematic program areas and
an increased focus on M&E and donor engagement.
28. The National Solidarity Program (NSP) has been evaluated externally and internally since its
launch in 2003. Post-war Reconstruction and Development Unit at the University of York (2006),
Integrity Watch Afghanistan (2007), and Office of the Special Director General for Afghanistan
Reconstruction (2011) assessed the Program‘s impact on development of the country. A randomized
impact evaluation of the NSP37
is currently being conducted by the team of researchers from Harvard
University, MIT, and the Russian New Economic School, with support from the World Bank, FAO, and
the Afghan Ministry of Rural Rehabilitation and Development.
29. A number of bilateral donors and multilateral agencies have evaluated their programs in
Afghanistan during the past decade: for example, FAO (2004); DANIDA (2005); UNDP (2006); CIDA
(2007); DFID (2009); and USAID (2011). In 2011-2012, the Asian Development Bank and NORAD are
36
See Pantuliano, S., Buchanan-Smith, M., Murphy, P. (2007) ‗The Long Road Home. Opportunities and Obstacles
to the Reintegration of IDPs and Refugees Returning to Southern Sudan and the Three Areas. Phase I‘. London:
ODI; Pantuliano, S., Buchanan-Smith, M., Murphy, P., Mosel, I. (2008) ‗The Long Road Home. Opportunities and
Obstacles to the Reintegration of IDPs and Refugees Returning to Southern Sudan and the Three Areas. Phase II.
Conflict, Urbanisation and Land‘. London: ODI; and Scanteam, (2007) ‗Review, Sudan Multi-Donor Trust Funds
Phase 1‘. Draft report, October. 37
The randomized impact evaluation of Phase-II of the National Solidarity Programme (NSP-II) is a multi-year
study designed to quantify changes - across indicators such as economic activity, agricultural production, access to
infrastructure and services, and structures and perceptions of local governance - in 250 ‗treatment communities‘
mobilized by NSP and to compare these changes to those observed in 250 ‗control communities‘ not participating in
NSP. The evaluation spans communities located in Balkh, Baghlan, Daykundi, Ghor, Herat, and Nangarhar
provinces and includes districts mobilized by seven national and international Facilitating Partners (FPs). The
evaluation is being conducted in parallel with the implementation of NSP in the 250 ‗treatment villages‘ and is
designed to report impacts at different stages of the project cycle.
12
undertaking evaluations of their programs, and the Center for Global Development is carrying out an
evaluation of the USAID program in Afghanistan. The IEG team is in communication with ADB,
NORAD and CGD and will, to the extent feasible, draw on the results of these evaluations as resource
materials for the Afghanistan CPE.
Approach and Methodology
30. Scope of the evaluation: The evaluation will assess the outcomes of the World Bank Group‘s
assistance to Afghanistan, as described in the Bank Group‘s strategy documents. This evaluation includes
the period immediately following the Bank Group‘s re-engagement in the country in FY2002 through
FY2011. The Bank Group‘s two ISNs define the three pillars of the Bank Group‘s strategy for
Afghanistan as: (1) Building the capacity of the state and its accountability to its citizens; (2) Promoting
the growth of the rural economy and rural livelihoods; and (3) Supporting growth of the formal private
sector (including infrastructure). This evaluation will assess the extent to which the objectives under these
three pillars have been achieved. Given the high degree of insecurity and the implementation challenges it
has faced in Afghanistan, the CPE will place more emphasis on learning the lessons from the Bank
Group‘s assistance during the period FY02-11 to inform the Bank Group‘s subsequent engagement in
Afghanistan in particular, and engagement with FCS contexts in general. The evaluation will be
undertaken in close coordination with stakeholders and other donors. IEG‘s team has been constituted to
ensure close coordination with other IEG units to evaluate the activities of all three World Bank Group
institutions.
31. The evaluation will be sensitive to the complex and challenging environment in which the
Bank Group has had to operate since re-engagement in 2002. The CPE will account for the political
economy context and the conditions on-the-ground and challenges in which the Bank Group‘s work has
been undertaken, including: conflict affected areas; weak state institutions; the strong presence of other
international actors; and risks of working in fragile states. The World Development Report 2011 finds that
restoring confidence in public actions and transforming the institutions that provide citizen security,
justice and jobs are essential for rebuilding fragile states. This provides a useful framework for assessing
the relevance and effectiveness of World Bank Group assistance to Afghanistan.
32. The CPE will seek to build off and learn from partnerships with other development
partners. There are many development agencies operating in Afghanistan, most of them permanently
represented in the field. The World Bank Group is not the largest of these, but it is one of the most
important in terms of its aid coordination and TF administration functions. The report will look at the
main areas of engagement with other international development partners as part of the overall assessment,
including the role of the Bank Group in administering trust fund resources and providing economic advice
to development partners.
33. The proposed CPE has two main purposes. First, the evaluation will assess the outcomes of the
World Bank Group‘s assistance program in relation to its objectives. The evaluation will look at the
extent to which World Bank Group strategies and projects were results-oriented, that is, sufficiently
specific about results they sought to achieve and founded on appropriate results frameworks. The report
will assess whether inputs were adequate and generated the desired outputs, whether the outputs
contributed to the desired outcomes, and if the outcomes are likely to be sustainable. Where there is a
weak link, the assessment will try to discover what factors contributed to the weakness in the results
chain. Projects conducted in conflict-affected areas, such as Afghanistan, cannot automatically be
assessed against traditional development measures such as economic rates of return, or success in policy
and institutional reforms. The key measure of success will be the Bank Group‘s contributions to building
state institutions and enhancing implementation capacity. The evaluation will make every effort to
understand what can be attributed to Bank Group support (or at least where it can plausibly be determined
13
that the Bank Group‘s efforts contributed to the results).
34. Second, the evaluation will seek to distill findings from the Bank Group‘s ten-year engagement in
Afghanistan to make recommendations for improving design, implementation and management of the
Bank Group‘s work. The CPE will also seek to draw lessons from the Bank Group‘s engagement in
Afghanistan as a fragile state to provide inputs to forthcoming IEG evaluations, including the proposed
evaluation of fragile states.
35. The traditional methodological framework for a CPE—based on assessments of the
outcomes of the Bank Group’s assistance programs—will serve as the overarching framework for
this evaluation. The CPE framework will take cognizance of the fact that in Afghanistan, the Bank
Group has relied on TSSs and ISNs which have a greater degree of flexibility with the objectives than
regular country assistance or partnership strategies. The CPE framework will also take into consideration
the fact that TSSs and ISNs allow emergency financing.38
The CPE will review the extent to which major
strategic objectives of the Bank Group‘s strategic pillars were relevant and achieved. Intermediate
objectives, such as improved delivery of social services or promotion of the rural livelihoods, specified in
the Bank Group‘s strategy documents would be reviewed to gauge their contribution toward higher-order
objectives. For each of the main objectives, the CPE will evaluate the relevance of the objective, the
relevance of the strategy toward meeting the objective, including the balance between lending and non-
lending instruments, as well as the efficacy with which the strategy was implemented, and intended
results achieved. To this end, IEG will review individual projects and programs and AAA to assess their
relevance to the uncertain and often fluid political and economic environment. To the extent feasible, the
CPE will also assess the relative contribution to the results by the Bank Group, other development
partners, and the government. The assessment will take into account the predominance of two major
exogenous factors—the ongoing conflict which, in fact, has intensified since 2006, giving rise to greater
security risks; and the substantial role of the military and donor financing. The CPE will account for any
changes in the Bank Group‘s program (e.g. project restructuring or emergency funding) necessitated by
these exogenous variables. The evaluation will also consider the sustainability of Bank Group projects
and programs in the context of the expecting scaling down of coalition forces in 2014.
36. Evaluation of ARTF. In addition to the ARTF‘s role in donor coordination, the evaluation will
review (a) the extent to which ARTF-financed operations are consistent with the Bank Group‘s strategic
objectives, as described in the TSSs and ISNs; (b) the extent to which the ARTF has managed donor
preferences for earmarking compared to pooling of donor funds; (c) the effectiveness of administrative
arrangements in terms of resource mobilization, reporting requirements, disbursements and project
outcomes.
37. Evaluating Institutional Development. The central tenet of the World Development Report39
(2011) is that ―strengthening legitimate institutions and governance to provide citizen security, justice and
jobs is crucial to break cycles of violence.‖ This is consistent with the OECD framework for capacity
development which recognizes the need to understand the country context to see what is likely to work in
those circumstances and states that priority capacities for development in fragile states should be those
that contribute directly to reducing fragility. In addition to identifying the main capacity gaps in respect of
core functions there is a need to identify those aspects of state performance that are essential to prevent
renewed conflict or collapse.40
The WDR defines these priority areas as security, justice and jobs. To the
38
Projects approved until 2007 relied on OP/BP 8.50 Emergency Recovery Assistance. Projects approved
subsequently are based on OP/BP 8.00 Rapid Response to Crises and Emergencies. 39
World Development Report 2011: Conflict, Security and Development. Washington DC: The World Bank. 40
See ―The Challenge of Capacity Development: Working Towards Good Practice.‖ OECD/DAC Guidelines and
Reference Series, 2005.
14
extent feasible, the evaluation will discuss the extent to which these areas have been address in
Afghanistan either by Bank Group assistance or by other partners. The evaluation is constrained by
limited access to the country and a truncated timeline in order to provide quick results to inform the Bank
Group‘s future strategy. To the extent feasible within those constraints, the evaluation will review the
Bank Group‘s contribution to individual and organizational incentives and toward enhancing the enabling
environment for institutional development in core state functions and service delivery in Afghanistan. The
lessons from these experiences are expected to be relevant to the future strategy of the Bank Group in
Afghanistan as well as to other countries engaged in state building.
38. Cluster AAA Assessment Methodology. The Bank Group‘s 107 AAA tasks have been grouped
into 17 sectoral and thematic clusters, in consultation with the country team. Although not conceived as
programmatic AAA, these clusters represent overlapping or related ESW and/or TA tasks undertaken
over time to enhance the Bank‘s ability to provide advice and development support and the government‘s
ability to develop policies, programs and institutions to meet their strategic objectives. The standard
review of the lending portfolio undertaken for a CPE will be augmented by a review of the results and
Bank performance for all 17 AAA clusters. Results will be assessed by examining evidence of influence
or effects on country policies or programs, institutional development, donor assistance, Bank operations
and policy dialogue, and stakeholders outside the public sector. Bank performance will be assessed by
examining evidence on relevance and timeliness, technical quality, client and stakeholder ownership,
Bank inputs and processes, and dialogue and dissemination. Each cluster will be reviewed by a two-
person panel using a standardized evaluation instrument. Quality control will be undertaken by a seasoned
moderator to ensure consistency of the review across clusters. The findings on each of these criteria at the
cluster level will be discussed in the CPE report with the aim of documenting to what extent the Bank‘s
AAA work can enhance the effectiveness of the Bank‘s financial assistance in similar fragile and conflict-
affected countries.
39. Large scale development assistance in Afghanistan has been channeled through community-
based and community-driven development (CDD) programs. In the absence of state institutions and
capable country systems, international development agencies have relied heavily on CDD programs to
rehabilitate rural infrastructure and provide basic services. The World Bank and other development
partners, including USAID, have conducted surveys of beneficiaries and evaluations of the relevance and
effectiveness of the National Solidarity Program (NSP), which is considered by the Country Unit to be the
flagship of the Bank Group‘s lending portfolio in Afghanistan and had channeled a cumulative total of
over $1.5 billion by FY11. Other agencies have also assessed effectiveness of the CDD approach on local
governance.41
Rather than duplicating this work, this evaluation will build on the findings from previous
evaluations, which will be supplemented by additional information, such as through a beneficiary survey
that adds to the frontier of knowledge on the relevance and effectiveness of CDD programs and service
delivery.
Evaluation Questions
40. The evaluation questions for the Afghanistan CPE fall into two categories. The first question
focuses on the initial period of re-engagement in Afghanistan. The next three questions are objectives-
based, derived from the three pillars of the ISNs. The last three questions are aimed at deriving lessons for
other fragile and conflict-affected situations:
41
See for example, the audit report of NSP by the Special Inspector General for Afghanistan Reconstruction
(SIGAR), March 2011; and Jennifer Brick, ―Investigating the Sustainability of Community Development Councils
in Afghanistan‖. Unpublished report, February 2008. Afghanistan Research and Evaluation Unit, prepared for JICA.
15
To what extent was the assistance of the World Bank Group during the initial years
relevant and commensurate with the country’s urgent needs to build confidence in the
state? This question will be assessed by collecting evidence on the following subsidiary issues
that are most relevant to the years of engagement under the Transitional Support Strategies: the
relevance and alignment of the strategic and policy advice during the early years of re-
engagement in Afghanistan to the country‘s needs to build confidence in state institutions and
demonstrate early results; the timeliness and flexibility in WBG assistance to help the
government demonstrate early results and meet urgent needs; the realism of WBG support in
defining its objectives, sequencing its activities, and tailoring its programs to in-country
institutional capacity and local ownership; the extent to which the WBG‘s program design and
modalities were customized and well suited to the fragile environment facing the country; and the
relevance of WBG interventions in the context of the role of other donors, and the WBG‘s ability
to mobilize support from donor partners.
To what extent was WBG support for governance and public services over the past decade
relevant to the country’s needs and effective in building the capacity of state institutions
and its accountability to its citizens? This question will be assessed by collecting evidence on
the following subsidiary issues: the relevance of the WBG‘s governance objectives to country
conditions, and the extent to which the governance objectives have evolved with the changing
development needs and risks since 2002; the effectiveness of the WBG in strengthening public
institutions; the degree of customization of WBG assistance for provision of public services to the
fragile environment in Afghanistan; the extent to which the Bank‘s focus on central state
institutions and local level Community Development Councils through the National Solidarity
Program could and should have been complemented by attention to intermediary institutions at
provincial and local government levels; the effectiveness of WBG in supporting public services
that reach both male and female beneficiaries; and the effectiveness of country-level and project-
level risk mitigation measures planned and implemented to address the governance risks outlined
in the ISNs. The WBG has provided considerable budget support for recurrent civil service
salaries and other programs from the ARTF, and for public service programs. The evaluation will
also examine the effectiveness of budget support in building state institutions in Afghanistan.
How effective was Bank Group assistance in promoting growth of the rural economy and
improving rural livelihoods? This question will be assessed by collecting evidence on: the
WBG‘s contribution toward strengthening policies and institutions for agricultural growth and
rural employment; the relevance and effectiveness of WBG support for the agricultural sector; the
extent to which the National Solidarity Program (a community driven development program
which is the WBG‘s largest program in Afghanistan) has been able to stimulate local economies
and generate rural employment; the extent to which the Community Development Councils have
been able to leverage and partner with other programs after implementing the initial project
financed by the NSP; the gender impact of the WBG‘s assistance to agriculture or community-
driven development.
How effective were the WBG’s efforts to supporting growth of the formal private sector,
including through infrastructure development? This question will be assessed by collecting
evidence on: the WBG policy and institutional support to the financial and banking sector in
Afghanistan; the effectiveness of the project and program interventions for the private sector, and
their likely impact on job creation; the effectiveness of WBG assistance for development of basic
infrastructure to meet the needs of its citizens and promote private sector activity; the WBG‘s
efforts to address land tenure issues and related effects on investment climate and growth in
Afghanistan.
16
How effective were efforts in harmonization and alignment among donor organizations?
This question will be assessed by collecting evidence on: the WBG‘s contribution to donor
harmonization and alignment; alignment of WBG-financed program activities with, and co-
financing of programs by, other development partners; WBG efforts to strengthen and use
country systems; the Bank's role in managing the risks of multiple, public sector remuneration
and employment Initiatives supported by the government and different donors; the effectiveness
of the WB‘s role as administrator of the ARTF (which has received over $4 billion from 32
donors since 2002); the role of the ARTF in donor coordination and institutional strengthening in
Afghanistan; and the effects of the ARTF on the Bank‘s own program priorities.
What lessons can be derived from Afghanistan for other fragile and conflict-affected states?
This question will be assessed by collecting evidence on: the WBG‘s contribution to establishing
state functions in an environment where state institutions had to be built almost entirely from
scratch; the lessons of WBG experience in mobilizing resources to meet recurrent costs for
rebuilding institutional capacity and ensuring institutional sustainability; innovations in
modalities of aid delivery in Afghanistan; the WBG‘s experience in undertaking strategic
analytical and advisory activities on cross-cutting issues, such as gender, social and
environmental policy, and land policy and promoting policies and programs to support such
cross-cutting issues; the WBG‘s experience to sustain program in a conflict environment with
volatile security conditions; the WBG‘s experience in operating in an environment where
developmental priorities are periodically overshadowed by political and military priorities.
To what extent has the WBG been effective in building institutions that are likely to be
sustainable and resilient to the risks facing the country? This question will be assessed by
collecting evidence on: the degree of broad in-country ownership of policies, programs and
institutions; the degree of dependence on and likely continuation of donor financing at the levels
necessary to sustain key institutions and programs; capacity building of public employees or in-
country non-state actors relative to external technical assistance and consultant support which
may decline significantly beyond 2014; the internal coherence among the different programs
being promoted by different donor organizations.
Analytical Methods and Sources of Information
41. The CPE will rely on a range of analytical methods and sources of information: a literature
review; a desk review of country program documents and project documents; sector and thematic
background papers; interviews with stakeholders and other development partners, and a review of
Advisory and Analytical Services (AAA).
a. Literature Review. The evaluation will incorporate a review of the international development
literature and experience in Afghanistan. It will also draw on extensive Economic and Sector
Work (ESW); Quality Assurance Group (QAG) assessments; Implementation Completion
Reports (ICR) Reviews, and Project Completion Reports (PCRs), as well as other material shared
by development partners.
b. Desk review. The CPE team will undertake a desk review of country-level documents as well as
project and program documents, including ISRs, ICRs, XPSRs, economic and sector work and
advisory services, technical notes and policy briefs.
17
c. Cluster-based Assessment of Advisory and Analytical Activities (AAA). A substantially large
portion of the World Bank‘s support to Afghanistan, 27 percent, has been in the form of
Economic Sector Work and Technical Assistance activities. The CPE team will conduct an AAA
assessment, adapting the methodology previously used by QAG and IEG to undertake country
AAA assessments. The team will adopt a quasi-programmatic approach. In consultation with the
Country Management Unit, all 107 Bank AAA and IFC advisory services have been grouped into
17 clusters. IEG has developed a customized instrument building on previous Bank and IEG
experience of AAA assessments, and will review all 17 clusters of AAA activities as an input into
the CPE.
d. Background Papers. Some sectors and themes have received substantial importance in the
Bank‘s overall development activities. Alongside the desk review, the team will prepare two
background papers on Capacity Building and Basic Service Delivery in Health and Education;
and the Gender Dimensions of World Bank Group support to Afghanistan.
e. Interviews with Stakeholders and other Development Partners. The evaluation team will
employ, to the extent local conditions permit, a wide consultative approach, involving substantial
interaction with various levels of authorities in Afghanistan, Bank and IFC staff and local
stakeholders, including at the community level and with nongovernmental organizations.
Stakeholders would include government officials, private sector actors, project beneficiaries,
members of NGOs, academia and think tanks, and other international development partners,
particularly the EU, the UN agencies, and major bilateral donor agencies.
f. Beneficiary Survey. The evaluation will undertake a beneficiary survey of service delivery
taking advantage of the rapid diffusion of mobile phones in Afghanistan to conduct a customized
SMS-based survey of beneficiaries (building on technologies such as satellite mapping
mechanisms being piloted by WBI, to the extent feasible).
g. Client Survey. Keeping in mind the two-phased approach being adopted for the CPE, the
evaluation will administer a client survey to obtain feedback from in-country clients, including
donors and development partners, as an input to the preparation of the note on interim findings.
h. Comparison of results with other fragile states. This will include, for example, Nepal, Timor-
Leste and West Bank and Gaza.
Evaluation Team
42. The evaluation team will be led by Anis Dani (Task Team Leader, Lead Evaluation Officer,
IEGCC), and consists of Mary Breeding (capacity building), Prem Garg (AAA assessment), Gita Gopal
(gender), Min Joo Kang (private sector), Svetlana Markova (agriculture), Albert Martinez (private sector),
Jack W. Van Holst Pellekaan (agriculture), Rebecca Patterson (urban), Rebecca Patterson (political
economy, infrastructure), Stephen Pirozzi (private sector), Susan Stout (health) and Clay Wescott and
Arianne Wessel (public sector governance). Unurjargal Demberel (IEGPE) will provide data and
research support and Roderick Lopez de Asis will provide research assistance and administrative support
to the task team.
43. Basil Kavalsky and Ian Bannon are the peer reviewers for the evaluation. The report will be
prepared under the direction and guidance of Ali Khadr (Senior Manager, IEGCC).
18
Schedule
44. The report will be submitted to CODE in June 2012. Milestones include:
I. Scoping Mission: March 2011
II. Approach Paper One Stop: October 2011
III. Approach Paper to Bank Management: December 2011
IV. Approach Paper to CODE: December 2011
V. Week of Learning Events with Country Team: November 2011
VI. Note on Interim Findings: January 2011
VII. Main Mission: January-February 2012
VIII. Internal (IEG) draft review: March 2012
IX. Management Comments: April 2012
X. Government Comments: May 2012
XI. Submission to CODE: June 2012
19
Attachment I: Evaluation Design Matrix
Question: Information Source(s) Data Collection Method: Limitations:
To what extent was the
assistance of the World Bank
Group during the initial years
relevant and commensurate
with the country’s urgent
needs to build confidence in
the state?
International development
literature; program
documents; ICRs; public
officials; CSOs; donors;
contractors; local citizens;
country office and project
staff; IEG evaluations of
programs in fragile and
conflict-affected situations
Literature review; desk review;
cluster-based AAA assessment;
interviews; client survey
Preliminary assessment
will need to be verified
during mission
Poor documentation
may limit ability to
assess early phases of
program design
To what extent was WBG
support for governance and
public services over the past
decade relevant to the
country’s needs and effective
in building the capacity of
state institutions and its
accountability to its citizens?
International development
literature; program/project
documents; ISRs; ICRs;
XPSRs; public officials;
CSOs; donors; local citizens;
country office and project
staff
Literature review; desk review of
lending portfolio; cluster-based
AAA assessment; background
papers; interviews; client survey;
beneficiary survey; focus groups;
background paper on capacity
building
Mostly reliable
Some variation by
projects / programs
Constraints on field
visit addressed through
beneficiary/client
surveys & triangulation
of evidence
How effective was Bank Group
assistance in promoting
growth of the rural economy
and improving rural
livelihoods?
Program/project documents;
ISRs; ICRs; XPSRs; external
evaluation reports; public
officials; CSOs; donors;
contractors; local citizens;
country office and project
staff
Desk review of lending and non-
lending portfolio; interviews
with clients and partner
organizations; client survey;
beneficiary survey; focus groups;
case studies; background paper
on gender impacts
Evaluation material
more reliable
Limited field visits may
be possible but
constraints due to
security considerations
How effective were the WBG’s
efforts to supporting growth of
the formal private sector,
including through
infrastructure development?
Program/project documents;
ISRs; ICRs; XPSRs; public
officials; CSOs; donors;
contractors; local citizens;
country office and project
staff
Desk review of lending
portfolio; cluster-based AAA
assessment; background papers;
interviews with clients and donor
partners; client survey; focus
groups
Uneven
documentation
Field visit constraints
How effective were efforts in
harmonization and alignment
among donor organizations?
International development
literature; program
documents; public officials;
CSOs; donor partners;
country office and project
staff
Desk review of lending and non-
lending portfolio; interviews
with public officials and donor
partners; client survey
Affected by quality of
documentation
Staff turnover in
donor agencies can
limit feedback on early
years of re-engagement
What lessons can be derived
from Afghanistan for other
fragile and conflict-affected
states?
Public officials; CSOs;
donors; country office and
project staff; IEG evaluations
of programs in conflict
affected states; published
literature on Afghanistan
Desk review of lending
portfolio; cluster-based AAA
assessment; background papers;
interviews with clients and
partner organizations; client
survey;
Conclusions from ex-
post findings and
structured surveys
more reliable
Interview feedback will
be more subjective
20
Question: Information Source(s) Data Collection Method: Limitations:
To what extent has the WBG
been effective in building
institutions that are likely to be
sustainable and resilient to the
risks facing the country?
Public officials; CSOs; other
donors; partner
organizations; country office
and project staff; Afghan
citizens; evaluative material
from other agencies;
published literature on
Afghanistan
Literature review; IEG
evaluation findings; background
paper on capacity building;
findings on public sector
administration from portfolio
and AAA reviews; triangulation
of data from multiple sources;
interviews with clients and
partner organizations; client
survey; social media.
Heavy reliance on
secondary materials
and interview findings
Prospective evaluation
bears the risks
associated with ex-ante
analysis
21
Attachment II: Afghanistan At a Glance
Key D evelo pment Indicato rs South Low
Afghanistan Asia income
(2009)
Population, mid-year (millions) 29.8 1,545 828
Surface area (thousand sq. km) 652 5,131 17,838
Population growth (%) 2.7 1.5 2.2
Urban population (% of to tal population) 24 29 28
GNI (Atlas method, US$ billions) 9.1 1,534 389
GNI per capita (Atlas method, US$) 310 993 470
GNI per capita (PPP, international $) 860 2,775 1,131
GDP growth (%) 40.8 4.8 6.2
GDP per capita growth (%) 37.1 3.3 3.9
(mo st recent est imate, 2003–2008)
Poverty headcount ratio at $1.25 a day (PPP, %) .. 40 ..
Poverty headcount ratio at $2.00 a day (PPP, %) .. 74 ..
Life expectancy at birth (years) 44 64 57
Infant mortality (per 1,000 live births) 134 56 77
Child malnutrition (% of children under 5) 33 41 28
Adult literacy, male (% of ages 15 and o lder) .. 72 73
Adult literacy, female (% of ages 15 and o lder) .. 50 59
Gross primary enro llment, male (% of age group) 127 110 107
Gross primary enro llment, female (% of age group) 84 105 100
Access to an improved water source (% of population) 48 87 64
Access to improved sanitation facilities (% of population) 37 36 35
N et A id F lo ws 1980 1990 2000 2009 a
(US$ millions)
Net ODA and official aid 33 122 136 4,865
Top 3 donors (in 2007):
United States 2 56 2 2,112
European Commission 0 2 18 349
United Kingdom 1 2 13 322
Aid (% of GNI) 0.9 .. 16.4 45.7
Aid per capita (US$) 2 7 6 168
Lo ng-T erm Eco no mic T rends
Consumer prices (annual % change) .. .. .. -15.0
GDP implicit deflator (annual % change) .. .. .. -15.5
Exchange rate (annual average, local per US$) 44.1 50.6 67.7 49.3