Document of The World Bank Report No.: 70332 PROJECT PERFORMANCE ASSESSMENT REPORT THE MEKONG RIVER COMMISSION WATER UTILIZATION PROJECT (TF-23406) June 28, 2012 Public Sector Evaluation Independent Evaluation Group Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized
40
Embed
World Bank Document · Agreement in 1992, Cambodia requested reactivation to the Committee. In 1995, the original members of the Mekong Committee (Cambodia, Laos, Thailand, and Vietnam)
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Annex A. Basic Data Sheet ............................................................................................... 18
Annex B. Persons Consulted............................................................................................. 21
Annex C. Borrower Comments......................................................................................... 24
Boxes
Box 1: Objectives and Components of the Water Utilization Project (TF-23406) ............. 4
Box 2: The Development of the Decision Support Framework ......................................... 9
Figure 1: The Mekong River Basin and Riparian Countries .............................................. 2
This report was prepared by Lauren Kelly who assessed the project in December 2010. Peer and Panel
Reviews were provided by Kenneth Chomitz (who accompanied the mission) and George T. Keith Pitman,
respectively. Research Assistance was provided by Ms. Tu Chi Nguyen. Romayne Pereira provided
administrative support.
v
Principal Ratings
ICR* ICR Review* PPAR
Outcome Satisfactory n.a. Moderately
Unsatisfactory
Risk to
Development
Outcome
Negligible to low n.a. Significant
Bank Performance Satisfactory n.a. Moderately
Unsatisfactory
Borrower
Performance
Satisfactory n.a. Moderately Satisfactory
* The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible Bank department. The ICR Review is an intermediate IEG product that seeks to independently verify the findings of the ICR. Forthcoming.
IEG Mission: Improving development results through excellence in evaluation.
About this Report
The Independent Evaluation Group assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank’s self-evaluation process and to verify that the Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20-25 percent of the Bank’s lending operations through field work. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons.
To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, and interview Bank staff and other donor agency staff both at headquarters and in local offices as appropriate.
Each PPAR is subject to internal IEG peer review, Panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible Bank department. The PPAR is also sent to the borrower for review. IEG incorporates both Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public.
About the IEG Rating System for Public Sector Evaluations
IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://worldbank.org/ieg).
Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory.
Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable.
Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory.
Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory.
vii
Preface
This is a Project Performance Assessment Report (PPAR) of the Water Utilization
Project (TF-23406), financed by a Global Environment Facility grant of US$11 million
made available to the Mekong River Commission. The grant was approved in February
2000 and closed in June 2008, a year later than planned, at which time US$57,749 was
cancelled. The Mekong River Commission is a river basin organization involving the
governments of Cambodia, Lao PDR, Thailand, and Vietnam that provides assistance and
advice to member countries on the development, utilization, conservation, and
management of the Mekong River Basin water and related resources. China and
Myanmar are observers. Its headquarters are located in Vientiane, Laos, and Phnom
Penh, Cambodia.
This report is based on a review of project documents, including the Implementation
Completion and Results Report, Project Appraisal Document, legal document and project
files, and on discussions with World Bank staff involved in the project. An IEG mission
visited Cambodia, Lao PDR, Thailand, and Vietnam from November 26-December 22,
2010. The IEG team held meetings with the Commission Secretariat – its Chief Executive
Officer and technical staff – in both Vientiane and Phnom Penh. In each member country,
the IEG team met with representatives of the National Mekong Committee, other relevant
Government Officials, donor partners, and NGOs.
Mission coordination was facilitated by Ms. Viktoriya Yevsyeyeva in extensive
cooperation with country counterparts: Ms. Khanh Linh Thi Le, Ms. Hoa Chau Nguyen,
Ms. Nuong Dieu Nguyen, Ms. Dung Thi Thuy Dao, Ms. Phuong Minh Le, and Ms. Khai
Hoan Nguyễn in Viet Nam; Mr. Thalavanh Vongsonephet (Teng) in Lao PDR; Ms. China
Chhun in Cambodia; and Ms. Poonyanuch Chockanapitaksa and Ms. Vachraras
Pasuksuwan in Thailand.
The mission is grateful to Mr. Alessandro (Alex) Nguyen Thanh Nha, for highly
professional translation services that were provided in Vietnam and to Ms. Romayne D.
Pereira for administrative support.
Following standard IEG procedures, copies of the draft PPAR were sent to government
officials and agencies and comments received are attached as Annex C.
ix
Summary
This is a project performance assessment of the Water Utilization Project implemented
by the World Bank and financed with a Global Environment Facility grant of US$11
million made available to the Mekong River Commission. The objective of the project
was to assist the Mekong River Commission (MRC) to establish mechanisms to promote
and improve coordinated and sustainable water management in the Mekong River Basin,
including reasonable and equitable water utilization by the countries of the Lower Basin
(Cambodia, Laos, Thailand, and Vietnam) and protection of the environment, aquatic life,
and the ecological balance of the Basin.
Attempts to coordinate the sustainable development of the lower Mekong Basin began in
1957 with the establishment of the Committee for Coordination of Investigations of the
Lower Mekong Basin (the Mekong Committee), at that time involving Cambodia, Laos,
Thailand, and Vietnam. Relationships in the Lower Mekong Basin shifted with the end of
the Vietnam War and the rise of the Khmer Rouge in Cambodia in the 1970s, resulting in
a restructuring of the Mekong Committee. As Cambodia slipped into the role of a “non-
participating but active” member of the riparian arrangement, an Interim Mekong
Committee was formed by the three remaining countries (Thailand, Vietnam, and Laos) –
an arrangement that would last from 1978–1995. Following the Cambodian Peace
Agreement in 1992, Cambodia requested reactivation to the Committee. In 1995, the
original members of the Mekong Committee (Cambodia, Laos, Thailand, and Vietnam)
signed the Agreement on Cooperation for the Sustainable Development of the Mekong
River Basin.
The Agreement established the Mekong River Commission (MRC), a regional facilitating
and advisory body governed by water and environment ministers of the four Lower
Riparian countries. The aim of the MRC is to ensure that the Mekong water is developed
in the most efficient manner that mutually benefits all Member Countries and minimizes
harmful effects on people and the environment in the Lower Mekong Basin. It is
composed of a Secretariat, with 150 staff members, lodged within the two Secretariat
offices in Phnom Penh, Cambodia and Vientiane, Lao PDR and the four National
Mekong Committees, comprised of Government representatives and technical staff,
located within each of the member countries.
With assistance from the United Nations Development Program, the MRC prepared its
first Strategic plan (1999-2003) that identified five medium-term goals, the first of which
was to assist with the formulation of Rules for Water Utilization and procedures for the
notification and consultation of proposed water uses; maintenance of flows on the
mainstream of the Mekong River, in particular with regard to dry season flows based
upon an analysis of the natural flow regime; water quality criteria, rules, and measures
for the protection of beneficial uses, including the aquatic eco-system; monitoring water
use and diversions in the Mekong Basin; and information exchange and monitoring.
The Water Utilization Project was designed to help the MRC implement the first goal of
its strategic plan through a multi-pronged approach. The project logic included support
for data gathering, basin modeling, and the development of a knowledge base to improve
countries’ understanding of the interaction between the physical and biological features
x
of the Mekong River, in order to facilitate enhanced cooperation and agreement around a
set of water use and quality rules. The project also supplied technical assistance to
facilitate discussions, negotiations and drafting of the water utilization rules by the four
member countries. Agreement on these rules, and an associated set of technical
procedures, was expected to lead to improved and coordinated sustainable management
of the Mekong River and the Mekong Basin systems.
Results. The project partially achieved its objective of assisting the MRC to establish
mechanisms, but by and large they are not yet being used to promote and improve
coordinated and sustainable water management in the Basin. The project also supported
policy and institutional reforms that improved the management effectiveness of the
Mekong River Commission and facilitated some engagement with non-MRC members,
China and Myanmar.
However, consensus was not reached on key technical guidelines, on minimum flow and
quality levels, that are needed to achieve the project objective of ensuring coordinated
and sustainable water management in the Mekong Basin. The premise built into the
Mekong Agreement is that Parties would be able to define clear-cut criteria on water
flows and quality and establish basin standards; and that, with those consensus rules in
hand, they could assess the compliance of any proposed development plan. But no one
party has wanted to pre-commit to criteria that would result in automatic approval or
rejection of a development proposal. Rather, parties to the Agreement have wanted to
run the model supported by the project the other way: by looking at specific development
proposals and then using the outputs supported by the model to decide whether or not to
go forward with the proposal.
Ratings. The Outcome of the project is rated Moderately Unsatisfactory, based on the
assessment of the relevance of the objectives and design, efficacy, and efficiency.
Although the objective of the project was substantially relevant, the relevance of design
was modest since it was based on the unrealistic premise that a fully scientific approach
could replace case-by-case negotiations. Additionally, the project design did not pay
adequate attention to national level adoption and implementation of project outputs, and
did not put in place sufficient mechanisms to extend knowledge generated by the
modeling to decision and policy makers outside of the National Mekong Committees.
Project efficiency, hampered by a lack of donor coordination and poor sequencing, was
also modest. While internal risks have been addressed with regard to ownership and
cooperation within the MRC, significant external risks exist with regard to hydropower
developments both upstream and along the tributaries.
Bank Performance is rated Moderately Unsatisfactory. While the project was well-
aligned with the MRC’s first Strategic Plan, quality at entry was undermined by the way
the project inputs were structured and used. In the initial part of the project, these inputs
could have been better used to design a facilitation framework for country-level decision-
makers; later to facilitate better understanding of the pros and cons of international
obligations of members. Interviews conducted for this review suggested that it may have
been more effective to have first supported workshops at the country through informal
meetings in order to increase awareness about basin dynamics before proceeding to
support international negotiations. Bank supervision was strengthened through a decision
xi
to decentralize task management towards the end of the project – a move that shifted
more support towards capacity building and training at the country level – and that began
preparations for follow-on work with other donors that would address the gaps in country
level implementation that were identified under the project.
Borrower Performance is rated Moderately Satisfactory. All four countries cooperated
with the MRC in implementing the project: the countries assigned qualified staff to
establish a riparian team at the MRC and established a national team within their
respective NMCs. The Mekong River Commission showed strong commitment to the
project objectives and has recently begun to develop an M&E framework to measure
achievement of organizational goals. Human resource capacity of the MRC has been
hampered by staff rotation rules however and more could have been done by the MRC to
disseminate the results of the project both nationally and among civil society across the
riparian member countries.
Lessons
The development of a Decision Support Framework to assist Mekong countries
make more informed decisions about water resource management has
demonstrated the opportunities and limits of bringing scientific and technical
innovation to bear in an environment fraught with political economy constraints.
The premise that scientifically determined acceptable levels of water flow and quality
and related modeling could replace case-by-case, negotiated water resource
management decisions proved unrealistic. Rather, the models have been used to
determine the winners and losers of proposed basin development projects in the
context of negotiations that have focused on country-specific, rather than basin-wide
interests.
Human Resource development at the national level is critical for the effective
implementation of international and regional water resource agreements.
Participating nations require strong water expertise. Donor programs that seek to
support effective implementation of water resource agreements at the international
and regional level should consider how to support skills enhancement and training of
water engineers, hydrological experts, water resource planners, and social and
environmental specialists correspondingly at the national level.
Caroline Heider
Director-General
Evaluation
1
1. Background
1.1 The Mekong River stretches for 4,180 km from its source in the highlands of
Tibet to the South China Sea, Figure 1. The river flows south from China’s Yunnan
province alongside the Myanmar-Laos and the Thai-Laos borders, and then descends
onto the Cambodian flood plain, where it is regulated by the storage of the Tonlé Sap, a
large lake that contracts and expands seasonally from 2,700 km² to 16,000 km² in
response to flooding from the head reaches of the river. As a consequence, the lake
supports one of the most productive natural fisheries in the world and provides 40-60
percent of the protein intake of the Lower Mekong Basin population (White
2002). Below Cambodia, the river splits into several distributaries that form the 39,000
km² Mekong Delta. The proportion of the total annual average flow in the Mekong River
by country varies considerably. Three of the six countries contribute 71 percent of the
5.23 However, China’s construction of the Xiaowan reservoir has occurred outside of
the framework of the Mekong Agreement. Tensions that have ensued highlight the urgent
need to engage China as a full MRC member and full steward of this vital water resource,
especially as its hydropower developments begin to come fully on stream. Mistrust
during a period of extremely low flow conditions in the Lancang-Mekong River in 2010,
for example, prompted China to share hydrological data in what has been described by
the MRC Secretariat as a special measure of cooperation. But the Xiaowan reservoir,
with a storage capacity of roughly 15 billion m³, has just begun and may take up to ten
years to fill.
5.24 Chinese reservoirs could regulate roughly 25 percent of the Chinese portion of the
Mekong River (PAD p. 16). They could be good for water regulation in the basin, if
released properly during droughts and stored during floods. Their storage capacity could
have the potential of increasing dry season flows near Vientiane by 50 percent and into
the Mekong Delta by 20 percent. They could also store part of the flow during floods.
This potentially positive benefit could be magnified under current climate change
projections. But even under normal operating conditions, there are still many unknowns.
How will the change in water quality and flow affect fish migration patterns? How much
nutrient rich sediment will be trapped behind the barrages instead of flowing downstream
to enrich the fields of the Mekong Delta rice farmers?
6. Efficiency
6.1 The efficiency of the project is modest. The GEF-financed project used an
incremental cost analysis in lieu of an economic or financial rate of return calculation.
The incremental cost assessment is based on the assumption that the GEF grant would
help the MRC develop and use water utilization rules and analytical tools necessary for
sustainable management and development of water resources in the Basin in order to
garner substantial specific global environmental benefits. From the point of view of
achieving global environmental benefits, this project was only modestly efficient since
very little progress was made on the implementation of the rules and application of the
tools to achieve this aim.
6.2 The efficiency of the Water Utilization Project was constrained by a lack of
planned donor coordination and sequencing. The project attracted a significant level of
parallel financing. Parallel financing totaled US$5.6 million, including US$4.01 million
from Finland, US$1.0 million from Japan and US$0.59 million from France. While the
leveraged finance is significant, each contribution supported a separate working
arrangement within the MRC. The sequencing of activities, although not entirely due to
project planning, also caused inefficiencies in project implementation. The
implementation of the GEF project, with its focus on facilitating MRC member
agreement around a set of “Rules” and “Procedures,” outpaced other areas of work at the
MRC that were recognized at project design as being critical to achieving MRC’s long-
term goals.
14
7. Ratings
Outcome
7.1 The overall outcome of the Water Utilization Project is rated Moderately
Unsatisfactory. While the project’s objectives were and remain substantially relevant,
project design was only modestly relevant. The project partially achieved its objective of
assisting the MRC to establish mechanisms, but by and large they are not yet being used
to promote and improve coordinated and sustainable water management in the Basin. It
supported the development and negotiation of a set of Rules to help facilitate the
implementation of the Mekong Agreement. However, consensus was not reached on key
technical guidelines, or minimum flow and quality levels, that are needed to achieve the
project objective of ensuring coordinated and sustainable water management in the
Mekong Basin. Project efficiency, hampered by a lack of donor coordination and poor
sequencing, was also modest.
Risk to Development Outcome
7.2 There are significant risks associated with the development outcomes supported
by the project. Foremost among these is a risk of non-cooperation, or the “business as
usual” scenario whereby unilateral or bilateral decisions are taken at the expense of the
common regional good. Chapter 2 of the 1995 Mekong Agreement stipulates that the
regional decision-making process is ”neither a right to veto the use nor unilateral right to
use water by any riparian country without taking into account other riparian’s rights.”
States are obligated to negotiate in good faith – but the liberal interpretation of states’
obligations under the agreement, due in large part to the failure to agree on clear and
binding technical guidelines – has already surfaced as a cause for concern.
7.3 There are also risks associated with sustaining and enhancing the capacity of the
MRC technical teams over team due to a a staff rule attached to the Mekong Agreement
(Article 33) that imposes a limit on the number of years a staff member can be retained.
Although the focus of the rule is meant to direct built capacity to the member countries, it
has left critical gaps in the technical capacity of the Secretariat.
7.4 Risk to Development Outcome is rated Significant.
Bank Performance
7.5 Quality at Entry: Moderately Unsatisfactory. While the project was well-aligned
with the MRC’s first Strategic Plan, quality of entry was undermined by the way the
project inputs were structured and used. In the initial part of the project these inputs could
have been better used to design a facilitation framework for country-level decision-
makers. Once in place, the framework could have been used to facilitate better
understanding of the pros and cons of international obligations of members. Interviews
conducted for this review suggested that it may have been more effective to have first
supported workshops at the country level (with decision-makers, technician, and
members of civil society) through informal meetings in order to increase awareness about
15
basin dynamics before proceeding to support international negotiations. 8 Project
conception paid adequate attention to building capacity within the MRC Secretariat but
not enough within the line ministries of the member countries.
7.6 Bank Supervision: Satisfactory. The Bank provided technical guidance in the
development of hydrological models, monitored the progress of establishing water
utilization rules and coordinated with donors in the provision of project funds. The Bank
participated in donor consultation group meetings and key management meetings
organized by the MRC, during which it provided policy advice. During the last two years
(2007-2008), the Bank task management team had been decentralized to the field and
focus shifted to finalizing the transitional arrangements, increasing capacity building at
the country level, and developing the concept for a follow up operation in order to
consolidate and further the results achieved under the project.
7.7 Overall Bank Performance is rated Moderately Unsatisfactory
Borrower Performance
7.8 Government Performance is rated Moderately Satisfactory. The project
involved the four governments of the member countries as bodies constituting the MRC:
Thailand, Cambodia, Lao PDR, and Vietnam. All four countries cooperated with the
MRC in implementing the project: the countries assigned qualified staff to establish a
riparian team at the Commission, established a national team within their respective
National Mekong Committees, and provided adequate human and financial resources.
However, while all of the Rules have now been adopted, the factor that has most
constrained effective implementation of the project has been the intractability of positions
concerning agreement on the technical guidelines.
7.9 Implementing Agency Performance is rated Moderately Satisfactory. The
Mekong River Commission showed strong commitment to the project objectives.
Leadership was demonstrated by the MRC when it recently commissioned a Strategic
Environmental Assessment to shed more light on transboundary environmental issues
that may be affected by hydropower development planning in the Basin. The MRC has
also recently begun to develop an M&E framework to measure organizational goals.
Human resource capacity of the MRC was modest and, because of staff rotation rules,
trained staff often had to leave the project after a few years. Though largely outside the
control of the project, this did affect capacity. More could have been done by the MRC
to disseminate the results of the project both nationally and among civil society across the
riparian member countries.
7.10 Overall Borrower Performance is rated Moderately Satisfactory.
8. As also discussed in the World Bank’s Final Supervision Mission June 16-18th, 2008.
16
8. Lessons
The development of a Decision Support Framework to assist Mekong countries
make more informed decisions about water resource management has
demonstrated the opportunities and limits of bringing scientific and technical
innovation to bear in an environment fraught with political economy constraints. The premise that water resource management decisions could be based on
scientifically determined acceptable levels of water flow and quality and that
subsequent modeling could be used to evaluate Basin proposals in line with a
minimum set of flow and quality criteria proved to be too constraining. Rather, the
models have been used to determine the winners and losers of proposed basin
development and subsequent negotiations have focused on individual, rather than
collective interests.
Human Resource development at the national level is critical for the effective
implementation of international and regional water resource agreements.
Participating nations require strong water expertise. Donor programs that seek to
support effective implementation of water resource agreements at the international
and regional level should consider how to support skills enhancement and training of
water engineers, hydrological experts, water resource planners, and social and
environmental specialists correspondingly at the national level.
17
References
Cronin, Richard and Timothy Hamlin. 2010. “Mekong Tipping Point: Hydropower
Dams, Human Security and Regional Stability.” Washington, DC: The Henry L. Stimson
Center.
Grimbine, R. Edward and Jianchu Xu. 2011. “Mekong Hydropower Development.”
Science. Vol. 332 no. 6026 pp. 178-179.
Hawkesworth, Nigel, and others. 2007. “Independent Organizational, Financial, and
Institutional Review of the Mekong River Commission Secretariat and the National
Mekong Committees.” Final Report. January.
Kummu, Matti, Juha Sarkkula, Jorma Koponen, and Jussi Nikula. 2006. ”Ecosystem
Management of the Tonle Sap Lake: An Integrated Modeling Approach.” International
Journal of Water Resources Development. 22(3): 497-519.