-
Document of The World Bank
Report No: ICR2936
IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-7458)
ON A
LOAN
IN THE AMOUNT OF US$11.7 MILLION
TO THE
REPUBLIC OF CHILE
FOR THE
INSTITUTIONAL STRENGTHENING OF THE MINISTRY OF PUBLIC WORKS
TECHNICAL ASSISTANT PROJECT
July 28, 2014
Sustainable Development Department Transport Unit Latin America
and the Caribbean Region
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
-
CURRENCY EQUIVALENTS
(Exchange Rate Effective June 11, 2014)
Currency Unit = Chilean Pesos (CLP) 1.00 CLP = US$ 0.001803
US$ 1.00 = CLP 554.48
FISCAL YEAR
ABBREVIATIONS AND ACRONYMS BP Bank Policy IFC International
Finance Corporation M&E Monitoring and Evaluation IMF
International Monetary Fund CPS Country Partnership Strategy IT
Information Technology DPL Development Policy Loan MOF Ministry of
Finance FDI Foreign Direct Investment MOPW Ministry of Public Works
FM Financial Management NCB National Competitive Bidding GDP Gross
Domestic Product OP Operational Procedure GIS Geographic
Information System PAD Project Appraisal Document GNP Gross
National Product PIU Project Imp1ementation Unit HH
Herfindahl-Hirschman Index PDO Project Development Objective IADB
Inter-American Development Bank R&D Research and
Development
IBRD International Bank for Reconstruction and Development SSS
Sole-Sourced Selection
ICB International Competitive Bidding TAL Technical Assistance
Loan IDA International Development Association
Vice President: Jorge Familiar Calderon Country Director: Livia
M. Benavides Sector Manager: Aurelio Menendez Project Team Leader:
Daniel Alberto Benitez ICR Team Leader: Daniel Alberto Benitez
ii
-
CHILE Institutional Strengthening of the Ministry of Public
Works
CONTENTS
Data Sheet A. Basic Information B. Key Dates C. Ratings Summary
D. Sector and Theme Codes E. Bank Staff F. Results Framework
Analysis G. Ratings of Project Performance in ISRs H. Restructuring
I. Disbursement Graph
1. Project Context, Development Objectives and Design
............................................... 1 2. Key Factors
Affecting Implementation and Outcomes
.............................................. 8 3. Assessment of
Outcomes
..........................................................................................
13 4. Assessment of Risk to Development Outcome
......................................................... 18 5.
Assessment of Bank and Borrower Performance
..................................................... 19 6. Lessons
Learned
.......................................................................................................
21 7. Comments on Issues Raised by Borrower/Implementing
Agencies/Partners .......... 22 Annex 1. Project Costs and
Financing
..........................................................................
23 Annex 2. Outputs by Component
.................................................................................
24 Annex 3. Economic and Financial Analysis
.................................................................
34 Annex 4. Bank Lending and Implementation Support/Supervision
Processes ............ 35 Annex 5. Beneficiary Survey Results
...........................................................................
37 Annex 6. Stakeholder Workshop Report and Results
................................................... 38 Annex 7.
Summary of Borrower's ICR and/or Comments on Draft ICR
..................... 39 Annex 8. Comments of Cofinanciers and
Other Partners/Stakeholders ....................... 47 Annex 9.
List of Supporting Documents
......................................................................
48
MAP
iii
-
A. Basic Information
Country: Chile Project Name: Ministry of Public Works
Institutional Strengthening TAL
Project ID: P102931 L/C/TF Number(s): IBRD-74580 ICR Date:
09/19/2013 ICR Type: Core ICR Lending Instrument: TAL Borrower:
REPUBLIC OF CHILE Original Total Commitment:
USD 11.70M Disbursed Amount: USD 6.84M
Revised Amount: USD 11.70M Environmental Category: C
Implementing Agencies: Ministry of Public Works Cofinanciers and
Other External Partners: B. Key Dates
Process Date Process Original Date Revised / Actual Date(s)
Concept Review: 08/10/2006 Effectiveness: 02/27/2008 02/27/2008
Appraisal: 03/13/2007 Restructuring(s): Approval: 06/14/2007
Mid-term Review: 06/08/2011 08/08/2011 Closing: 01/31/2013
01/31/2014 C. Ratings Summary C.1 Performance Rating by ICR
Outcomes: Satisfactory Risk to Development Outcome: Negligible Bank
Performance: Moderately Satisfactory
Borrower Performance: Satisfactory
C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)
Bank Ratings Borrower Ratings
Quality at Entry: Moderately Satisfactory Government:
Satisfactory
Quality of Supervision: Moderately Satisfactory Implementing
Agency/Agencies: Satisfactory
Overall Bank Performance:
Moderately Satisfactory
Overall Borrower Performance: Satisfactory
iv
-
C.3 Quality at Entry and Implementation Performance Indicators
Implementation
Performance Indicators QAG Assessments
(if any) Rating
Potential Problem Project at any time (Yes/No):
No Quality at Entry (QEA):
None
Problem Project at any time (Yes/No):
Yes Quality of Supervision (QSA):
None
DO rating before Closing/Inactive status:
Moderately Satisfactory
D. Sector and Theme Codes
Original Actual Sector Code (as % of total Bank financing)
Central government administration 100 100
Theme Code (as % of total Bank financing) Administrative and
civil service reform 29 29 Legal institutions for a market economy
14 14 Managing for development results 14 14 Other public sector
governance 14 14 Regulation and competition policy 29 29 E. Bank
Staff
Positions At ICR At Approval Vice President: Jorge Familiar
Calderon Pamela Cox Country Director: Livia M. Benavides Axel van
Trotsenburg Sector Manager: Aurelio Menendez Jose Luis Irigoyen
Project Team Leader: Daniel Alberto Benitez Jordan Schwartz ICR
Team Leader: Daniel Alberto Benitez ICR Primary Author: Mirtha
Pokorny F. Results Framework Analysis Project Development
Objectives (from Project Appraisal Document) The Project
Development Objective is to assist the Government in achieving a
sustainable and integrated approach to the planning of
infrastructure; levels of service standards for infrastructure as
well as model contracts and procedures that promote competition and
assign risks appropriately; an integrated project management
process; a
v
-
regulatory framework for all of MOPW’s sectors of responsibility
that increases consumer confidence in infrastructure service
provision; and the legal and organizational separation of the areas
of planning, contracting, execution and regulation of
infrastructure provision. Some of the original PDOs and
intermediate indicators were found difficult or impossible to
measure. Thus in 2009 the Borrower and the Bank agreed to a revised
set of indicators. (See Section 1.3 of the ICR) The new indicators
were reflected in the Aide Memoire dated November 20, 2009 and,
although they were never formally changed, they became the
framework on which the project was monitored. The information below
is based on the agreed revised indicators. a) PDO Indicator(s)
Baseline Value
Original Target Values (from
approval documents)
Formally Revised
Target Values
Actual Value Achieved at
Completion or Target Years
Indicator 1 New planning methodology for infrastructure
functioning and incorporated in the
investment decision of the MOPW Share of projects approved based
on integrated planning methodology
Projects approved based on
the new planning methodology
25% n.a. 93%
Date achieved 2008 2014
Comments (incl. %
achievement)
Fully achieved. This component, one of the main linchpins of the
project, has helped to successfully establish a new paradigm in
MOPW’s planning, introducing a new methodology that integrates
regional needs and captures the synergies of different projects
within a development program. Several consulting contracts
contributed to develop the methodology and apply it to different
programs (e.g. City Ports, Meterx, City Plans, and Systems of
Territorial Information). By 2011 thirteen Regional Infrastructure
Plans and Hydrological Management Plans were developed and
approved. By 2013 the new methodology was fully mainstreamed and
99% of the investments included in the 2014 budget come from
regional plans developed with the new methodology.
vi
-
Baseline Value Original Target
Values (from approval documents)
Formally Revised Target Values
Actual Value Achieved at
Completion or Target Years
Indicator 2
Greater competition in bidding supported by fair risk-sharing
arrangements embedded in contracts and consumer responsive service
targets
More competition in biddings to be reached by (i) improvements
in bidders’ registry (reglamento de obras y consultorías) and (ii)
improved standard bidding documents
Public works and concession contracts
using the new standard bidding
documents
80% of new works and services
contracts based on standard bidding
documents
See description below
Date achieved 2008 2009 2014
Comments (incl. %
achievement)
Partially Achieved. MOPW went through a lengthy and extensive
revision of the regulations governing public works and consulting
services and made proposals that were internally approved in MPOW
to revise the contractors and consultants registries and bidding
documents. The process was slowed down by the Contraloría office,
and is still under revision.
Baseline Value
Original Target Values (from
approval documents)
Formally Revised
Target Values
Actual Value Achieved at
Completion or Target Years
Indicator 3
Greater responsiveness to consumer concerns related to the
levels of service; and greater oversight of performance by public
sector service providers as well as
concessionaires
Number of concession contracts under the jurisdiction of the
MOPW that incorporate (explicitly) service standards
Public works and
concession contracts using explicit service standards
3 contracts (2010), 2 contracts
(2011) and 2 contracts (2012)
See description below
Date achieved 2008 2009 2014
Comments (incl. %
achievement)
Fully achieved. By the end of 2013 MOPW had entered the
following number of contracts with explicit service standards: (a)
five contracts of airport concessions; (b) five concession
contracts of interurban roads with partial definition of service
standards; (c) 111 contracts of performance-based contracts and
ad-measure contracts, of which contracts by service levels comprise
13,371.53 km or 71.7% of the country’s paved network; (d)
concession contracts for the three hospitals (Sotero del Rio,
Santiago Occidente and the Geriatric Hospital).
vii
-
Baseline Value
Original Target Values (from
approval documents)
Formally Revised
Target Values
Actual Value Achieved at
Completion or Target Years
Indicator 4
Reduction in variance between values established at contract
signature and final values at contract execution
Reduction in the variance between costs forecasted when contract
is signed and actual cost of works
Variance of cost when contract is signed and
actual cost of works in 2011 15% (as revised)
More than 32%
Date achieved 2008 2009 2013
Comments (incl. %
achievement)
Fully Achieved. In 2010 MOPW completed the base line built with
contracts implemented in 2005, 2006 and 2007. By 2011 projects
managed under the integrated management system and completed that
year had a cost of 32% below the base line cost. Projects completed
in 2012 had a reduction in cost of 14% with respect to the projects
completed in 2011.
Baseline Value
Original Target Values (from
approval documents)
Formally Revised
Target Values
Actual Value Achieved at
Completion or Target Years
Indicator 5
Normative and organizational structure that provides clear
separation and definition of roles and responsibilities for
planning, contracting, works execution
and oversight Regulatory oversight system in place to be used by
entity independent of planning and
execution divisions of MOPW
Chile does not have a regulatory oversight system
suitable for an entity independent of the planning and execution
divisions of
MOPW
System in place
See description
below
Date achieved 2008 2009 2013
Comments (incl. %
achievement)
Partially Achieved. MOPW developed and implemented oversight
systems for 20 interurban road contracts, six urban road contracts
and two public transport corridors. They also completed and
implemented the necessary forms for contract oversight of
concessioned airports (e.g. information screens, escalators,
bathrooms, etc.), and for contract oversight of rural potable
water. In all cases complaints forms have been completed and being
used for remedial actions. Several web platforms were created for
the oversight efforts and their effectiveness were confirmed during
the reconstruction period after the 2010 earthquake.
b) Intermediate Outcome Indicator(s)
viii
-
Baseline Value
Original Target Values (from
approval documents)
Formally Revised
Target Values
Actual Value Achieved at
Completion or Target Years
Component 1 Strategic and tactical plans approved – number of
Investment Plans (National and Regional) elaborated
No investment plan and regional plans exist Not defined
16 Investment Plans (1 Master
Plan and 15 Regional
Investment Plans)
16 Investment Plans implemented
Date achieved 2008 2009 2013
Comments (incl. %
achievement)
Fully Achieved. A Master Plan and 15 Regional Investment Plans
were developed using the new integrated planning methodology. Two
regional plans were elaborated using a more demanding service
standard (Arica – Parinacota and Los Ríos)
Baseline Value Original Target
Values (from approval
documents)
Formally Revised
Target Values
Actual Value Achieved at
Completion or Target Years
Component 2 Identify, define and adopt service quality standards
in a wide set of work typologies under the responsibility of the
MOPW
No model for quality
standards per type of work exists.
n.a
9 type of works with identified services and
standards being elaborated.
Adopted target: 13 type of works with identified services
and standards being elaborated – see
description below Date achieved 2008 2009 2014
Comments (incl. %
achievement)
Substantially Achieved. By 2013 MOPW had completed the
identification services and relevant quality attributes as well as
standards of service for the following types: (a) airports under
concession; (b) public buildings; (c) roads under concession; (d)
maintenance of paved roads not under concession; (e) rain waters;
(f) hospitals under concession; (g) identification of users and
services of small fisheries. MOPW is well advanced in the
definition of standards for (h) projects of lake, river and
maritime connectivity for isolated communities; (i) rural potable
water; (j) border passes under concession (Paso de los
Libertadores); (k) urban renewal projects (Proyecto Marga Mar); (l)
public assets building; (m) port infrastructure of border coasts;
and (n) irrigation works.
ix
-
Baseline Value
Original Target Values (from
approval documents)
Formally Revised
Target Values
Actual Value Achieved at Completion or Target Years
Component 3 Reduction in the variance of construction time
between the time envisaged when contract is signed and actual
construction time
Variance of time when
contract is signed and actual construction time in 2011
15% (as revised)
19%
Date achieved 2011 2013
Comments (incl. %
achievement)
Fully Achieved. The base line was completed in 2011 using
contracts completed in 2005, 2006 and 2007. Projects completed in
2011 under the integrated management system showed a marginal (1%)
difference in completion time when compared to the base line.
Projects completed in 2012 showed a difference of 19% with respect
to projects completed in 2011.
Baseline Value
Original Target Values (from
approval documents)
Formally Revised Target
Values
Actual Value Achieved at
Completion or Target Years
Component 4 Develop and implement a system for the evaluation of
users’ satisfaction
No model for the evaluation of users’ satisfaction exists.
Develop a model for the evaluation
of users’ satisfaction for 8 types of works
See description below
Date achieved 2008 2009 2012
Comments (incl. %
achievement)
Substantially Achieved. Efforts to develop and implement systems
for evaluating users’ satisfaction started fully in 2009 with the
system developed and tested through surveys in 12 interurban
concessioned roads, three sections of interurban roads under the
administration of the Regional Directorates of Roads, and four
highways. By 2013 MOPW has completed the models for evaluating
users’ satisfaction and the base line for comparisons for six
different types of service: (a) Interurban concessioned roads, (b)
Interurban not concessioned roads, (c) concessioned urban highways,
(d) connectivity of isolated areas’ (d) concessioned airports of
the primary network, and (e) Non-concessioned airports of the
primary network.
x
-
G. Ratings of Project Performance in ISRs
No. Date ISR Archived DO IP Actual
Disbursements (USD millions)
1 09/11/2007 Satisfactory Satisfactory 0.00 2 04/28/2008
Satisfactory Satisfactory 0.00 3 10/30/2008 Satisfactory
Satisfactory 0.00 4 05/06/2009 Satisfactory Satisfactory 0.00
5 11/11/2009 Moderately Unsatisfactory Moderately
Unsatisfactory 0.00
6 04/09/2010 Moderately Unsatisfactory Moderately
Unsatisfactory 0.00
7 11/24/2010 Moderately Unsatisfactory Moderately
Unsatisfactory 1.19
8 05/21/2011 Moderately Unsatisfactory Moderately Satisfactory
1.58
9 06/29/2011 Moderately Satisfactory Moderately Satisfactory
1.58 10 01/30/2012 Moderately Satisfactory Moderately Satisfactory
2.36 11 11/10/2012 Moderately Satisfactory Moderately Satisfactory
3.97 12 06/26/2013 Moderately Satisfactory Moderately Satisfactory
3.97 13 01/04/2014 Moderately Satisfactory Moderately Satisfactory
6.81
H. Restructuring (if any) By mid-2012 the Borrower requested and
the Bank agreed to a one- year extension of the loan with a new
closing date of January 31 2014. The objective of the extension was
to allow further disbursements on long-term consultancies financed
under the loan and also included a few changes in the procurement
rules and the implementation of Anti-corruption Guidelines as
described below. The Procurement and Consultant Guidelines were
updated to reflect the versions dated January 2011, which allows
the use of framework agreements, known as “Convenios Marcos” in
Chile, for the procurement of goods, works and non-consultant
services. In addition, the Bank agreed to raise the maximum limits
for using ChileCompra for consultant services, from US$100,000 to
US$300,000. In addition, the definition of “Convenios Marco” was
amended to include non-consultant services. Finally the
Anti-corruption Guidelines dated October 15, 2006 and revised in
January 2011 was to apply to the Project to be consistent with the
updated version of the Procurement and Consultant Guidelines.
xi
-
Disbursement Profile
xii
-
1 Project Context, Development Objectives and Design
1.1 Context at Appraisal
In the early 1990s, the Government of Chile took stock of its
infrastructure needs and estimated its infrastructure deficit at
US$11 billion with additional losses due to lack of competitiveness
of US$1.7 billion annually.1 In determining how to respond to this
need for massive investments and significant improvements in
service delivery, the Government also recognized that the public
sector did not have the resources to carry out the major works
required, nor could it commit its budgetary resources exclusively
to infrastructure investments given the country's social investment
needs. To help face this challenge, the Government first called on
the private sector to take part in the building, maintenance and
operation of major road works. In the years since its inception,
the concession program has expanded both in terms of sectors and
design. Chile had at the time of appraisal (and still has) what may
be the world's most extensive concession program encompassing
roads, ports, railroads, bridges, tunnels, reservoirs, jails and
canals. The Ministry of Public Works' (MOPW) concessions
portfolio-including roads and highways, bridges, airports, dams,
canals, prisons, and other public edifices-has risen in real terms
from about 450 billion pesos in 1995 to over 1 trillion pesos in
2005 . More strikingly, the share of private sector participation
in investments had increased from 9 percent to 65 percent over the
course of the 1995-2005 decade. In addition to the growing
concessions program, at the time previous to project appraisal the
Government of Chile had announced a significant expansion in the
public financing of infrastructure. The intent was to provide
Chile's regions and export products with greater access to markets
and to further improve the levels of basic services for Chilean
consumers. The administration recognized that both its growing
public investment and concession program would require greater
attention in terms of planning, contract design, project management
and oversight in order for them to maximize their potential as
contributors to higher growth levels. The Chilean administration
had identified specific constraints to the operational and
institutional structure of the MOPW that it believed had to be
addressed in order for Chile to reach new level of efficiency and
accountability in infrastructure investment. Those constraints
included: Planning:
1 Sectors covered by Ministry of Public Works include transport
assets such as inter-urban roads, urban roads, highways, bridges,
and airports; water resource infrastructure such as dams, storm
drainage systems, and canals; rural infrastructure such as small
water systems, ferry landings, and fishing ports; and public
buildings such as jails, hospitals, and schools. Of transport,
water resource and public building contracts, only ports are
concessioned and monitored by a separate agency. There are separate
regulatory agencies for privatized utilities such as water supply,
telecommunications and energy
1
-
• Lack of medium and long-term planning with an integrated
geographic perspective oriented to the competitiveness of Chile's
productive sectors.
• Low level of interaction with other relevant stakeholders in
infrastructure sector and a lack of societal or consumer voice in
project planning.
• Investment programs which are contract-driven rather than
integrated along a supply chain, leading to coordination
difficulties and inefficient expenditures.
Standards, Contracts and Competition:
• Lack of explicit service standards embedded in concessions or
public works due to an historic focus on input-driven rather than
performance-based contracts.
• Uneven levels of competition during bidding processes for
either concessions or public works exacerbated by lack of
information correlating outcomes to contract size, structure,
design elements or bidding procedures
Project Management:
• Low levels of accountability related to project outcomes due
to an atomized approach to project management with different
individuals responsible for each stage of project development.
• Poor availability of information about project costs, contract
management or design and construction issues.
Oversight and Regulation:
• Institutional inability to provide effective oversight of
contracts and concessions according to agree upon levels of
service, let alone cost elements.
• Lack of ability to consider consumer concerns or even to
assure public access to information in the oversight of
services.
Organizational Reform:
• An organizational structure that does not provide for clear
separation of roles and responsibilities among the four major areas
of activities-planning, contract design, project management and
regulation.
Measures to address these challenges made up the core components
of the Institutional Strengthening Program of the Ministry of
Public Works. The Government of Chile requested that this reform
initiative be supported with a Bank Development Policy Loan (DPL)
and, the subject of this report, a Technical Assistance Loan (TAL).
2 The DPL3 intended to achieve explicit short term outputs and
defined a subsequent series of tasks that would require several
years to complete and would be implemented under the TAL. Under the
DPL the following tasks were undertaken: (i) completion of
competitiveness
2 http://www.mop.cl/modernizacion/Paginas/Inicio.aspx 3
Institutional Strengthening of the Ministry of Public Works
Development Policy Loan, 2007. Report No: 39306-CL
2
-
plans for key clusters; (ii) development of sample performance
standards for high and low volume roads, for both concessions and
public works contracts, and (b) review of bidding processes and
recommendations on new procedures; (iii) completion of design for
regulatory accounting system; and (iv) determination of standards
to be regulated for roads and one other sector. Bank's involvement
in this initiative was sought to leverage the Bank's experience in
logistics and territorial planning, in the design and execution of
public-private contracts, in project management and in the
development of regulatory frameworks for infrastructure. In
addition, the Bank offered a comparative advantage in its ability
to draw upon global knowledge, sustain a policy dialogue, and
exploit synergies with other operations and technical support to
Chile including, but not limited to, the Infrastructure for
Territorial Development Project, the TranSantiago Urban Transport
DPL and TAL, and the fee-for-service technical assistance on the
valuation of contingent liabilities in infrastructure - see Annex 9
for a complete list of projects. Finally, both the Ministry of
Finance and the MOPW were attracted by the synergies that could be
gained from having both a DPL and a TAL to finance the key
components of Institutional Strengthening Program. The thematic and
substantive linkages of the two projects were expected to provide
an effective framework for monitoring and evaluating the
implementation of the reform program.
1.2 Original Project Development Objectives (PDO) and Key
Indicators The Project Development Objective was to assist the
Government in achieving a sustainable and integrated approach to
the planning of infrastructure; levels of service standards for
infrastructure as well as model contracts and procedures that
promote competition and assign risks appropriately; an integrated
project management process; a regulatory framework for all of
MOPW's sectors of responsibility that increases consumer confidence
in infrastructure service provision; and the legal and
organizational separation of the areas of planning, contracting,
execution and regulation of infrastructure provision.
Output and Outcome Indicators for monitoring the progress of the
objectives centered on: (a) increased return of infrastructure
projects due to the new planning methodology; (b) greater
competition in bidding supported by fair risk-sharing arrangements
embedded in contracts and consumer responsive service targets; (c)
greater responsiveness to consumer concerns related to the levels
of service; and greater oversight of performance by public sector
service providers as well as concessionaires; (d) reduction in
variance between values established at contract signature and final
values at contract execution; and (e) normative and organizational
structure that provides clear separation and definition of roles
and responsibilities for planning, contracting, works execution and
oversight. The methodology for calculating the baseline and
subsequent indicators would be updated to reflect the availability
and reliability of data as well as the most appropriate approach
for monitoring of change. The baseline for each indicator was to be
determined during the first four months of the project after
effectiveness. The project would also
3
-
monitor progress towards the expected outcomes through the
medium-term indicators for each of the components as seen in the
next paragraph.
1.3 Revised PDO (as approved by original approving authority)
and Key Indicators, and reasons/justification
The PDO remained unchanged during the life of the project. With
respect to the key indicators, they were found to be difficult to
measure and, in some cases, impossible to establish a strong
causality between project and outcome. In May 2009 the PIU and the
Bank agreed on a revised set of indicators which were included in
the Aide Memoire dated November 20, 2009. Although the changes,
summarized in the table below, were never formalized, the new
agreed indicators were the ones used by MOPW and this ICR to assess
the achievements under the project.4 ORIGINAL REVISED (a) PDO
Indicators (a) PDO Indicators Increased return of infrastructure
projects due to the new planning methodology
New planning methodology for infrastructure functioning and
incorporated in the investments decisions of the MOPW
Greater competition in bidding supported by fair risk-sharing
arrangements embedded in contracts and consumer responsive service
targets
Unchanged
Greater responsiveness to consumer concerns related to the
levels of service; and greater oversight of performance by public
sector service providers as well as concessionaires
Unchanged
Reduction in variance between values established at contract
signature and final values at contract execution Unchanged
Normative and organizational structure that provides clear
separation and definition of roles and responsibilities for
planning, contracting, works execution and oversight
Unchanged
(b) Intermediate Indicators b) Intermediate Indicators Strategic
and tactical plans approved (Component 1) Unchanged Share of works
contracts and concessions based on measurable, output-oriented
standards and greater competition in bidding supported by fair
risk-sharing arrangements embedded in contracts and
consumer-responsive service targets. (Component 2)
Identify, define and adopt service quality standards in a wide
set of works typologies under the responsibility of the MOPW.
Reduction in variance between unit values for Reduction in the
variance of construction time
4 The five PDO Indicators are respectively measured by: i) Share
of projects approved based on integrated planning methodology; ii)
More competition in biddings to be reached by (i) improvements in
bidders’ registry (reglamento de obras y consultorias) and (ii)
improved standard bidding documents; iii) Number of concession
contracts under the jurisdiction of the MOPW that incorporate
(explicitly) service standards; iv) Reduction in the variance of
the difference between costs forecasted when contract is signed and
actual cost of works; and v) Regulatory oversight system in place
to be used by entity independent of planning and execution
divisions of MOPW. Note: the Result Framework in the PAD and the
revised Result Framework do not provide a methodological
description on how these five indicators should be measured.
4
-
construction/investment established at contract signature and
final values at project execution. (Component 3)
between the time envisaged when contract is signed and actual
construction time
Greater responsiveness to consumer concerns related to levels of
service; and greater oversight of performance by public sector
service providers as well as concessionaires. (Component 4)
Develop and implement a system for the evaluation of users’
satisfaction
A legal and organizational framework in place that provides for
clear separation of roles and responsibilities. (Component 5)
Unchanged
1.4 Main Beneficiaries The immediate beneficiary of the project
would be the MOPW and its ability to respond more effectively to
the needs of the users of public infrastructure. The main
beneficiary of the project is the Chilean population through the
design and implementation of projects focusing on regional
development and equity, a more efficient use of private capital in
providing public services, and a strong focus on user satisfaction.
Ultimately, those benefits are expected to stimulate economic
activities and spread the benefits of the project to the entire
population in Chile. In addition, improvement of efficiency of
public works management would save public expenditures, which leads
to benefits to all taxpayers.
1.5 Original Components (as approved) The operation had five
components, four of which address the core aspects of MOPW's
business activities: planning; setting standards for contracts, and
conducting bids; managing projects; and regulating or overseeing
("la fiscalización de") both concessions and public works. A fifth
component (Organizational Reform) was to bring coherence to the
program through the development of a legal and organizational
structure that defined the expected new roles and responsibilities
of the MOPW. Each corresponding component as described in the Loan
Agreement is presented below. Section II of the PAD provides a more
detailed description of the project. Part I: Integrated Planning 1.
Design of a Long-Term Planning Methodology for MOPW's
Infrastructure Development:
a) Design of a methodology for the planning process of
infrastructure investments at the strategic level (national level
and long-term planning) as well as at the tactical level (sub-
national level and medium-term planning).
b) Design of an environmental evaluation methodology to be
included in the planning process and provision of training on such
methodology.
c) Design of a strategy of transition towards implementation of
the planning methodology referred to in (a) above, and for the
monitoring of such planning process.
5
-
2. Implementation of a Long-Term Planning Methodology:
a) Development of components of a strategic national
infrastructure plan using the methodology developed under Part
I.1.(a) above.
b) Design and implementation of an information system, database
and geographic information system (GIS).
c) Design of policies for long-term planning in the areas of
land use, resettlement, transport and water resources
management.
d) Implementation of the transition strategy referred to in Part
I.1.(c) above, including implementation of an internal and external
communication plan, and dissemination of the various infrastructure
plans developed under Part I.2.(a) above through workshops, courses
and seminars.
3. Carrying out of studies and training related to the
development of a sustainable and integrated approach to the
planning of infrastructure, as proposed by the Borrower and agreed
to by the Bank. Part II: Standards, Contracts, Competitiveness 1.
Establishment of Service Standards:
a) Establishment of a methodology for setting, evaluating and
updating MOPW's standards of service.
b) Development of a standard-of-service manual that assigns
standards according to type of contract, sub-sector, cost or cost
categories and demand levels.
c) Training of MOPW's and other selected agencies' staff in the
use of service standards.
d) Strengthening the capacity of the MOPW's staff to apply the
service standards referred to in Part II.1.(a) above, to the design
of infrastructure projects.
2. Contracts and Competitive Processes:
a) Analysis of MOPW's current contracts and bid processes as
well as those used in international best practices.
b) Development of a database of MOPW's public works contracts
and concessions. c) Assessment of the variables of contract design
and bidding processes against their
ability to generate competition and investment results. d)
Design of model contracts for each primary type of modality of
contracting of
infrastructure works, and each sub-sector, based on
international best practices. e) Analysis of appropriate risk
allocations in contracts by modality and sub-sector.
3. Carrying out of studies and training related to the
development of service standards for infrastructure as well as
model contracts and procedures that promote competition and assign
risks appropriately, as proposed by the Borrower and agreed to by
the Bank.
6
-
Part III: Project Management 1. Design of an Integrated Project
Management Model:
a) Detailed design of the MOPW's project management model. b)
Establishment of information and software tools to accompany the
project
management model designed under (a) above. 2. Implementation of
a Project Management Model: Carrying out of communications and
capacity building plans to accompany the implementation of the
project management model developed under Part III.1 (a) above. 3.
Carrying out of studies and training related to the development of
an integrated management process in MOPW, as proposed by the
Borrower and agreed to by the Bank. Part IV: Strengthening of
Regulation of Infrastructure Works 1. Development of roles and
regulations for the oversights of public works contracts and
concessions, including:
a) An operations manual to lay out the procedures for such
oversight; and b) Economic and financial models, and their
accompanying regulatory accounting
manuals, for public-private partnerships for selected
infrastructure sectors. 2. Carrying out of studies and training
related to the development of a regulatory framework for MOPW's
sectors of responsibility that increases user confidence in
infrastructure service provision, as proposed by the Borrower and
agreed to by the Bank. Part V: Organizational Reform 1. Norms and
Regulations:
a) Provision of technical assistance for the redrafting of the
bill for the establishment of oversight mechanisms for public
contracts and concessions.
b) Provision of technical assistance for drafting or redrafting,
as the case may be, of other laws and regulations needed for
implementation of the Program.
2. Institutional Development:
a) Definition of alternative arrangements for MOPW's reform in
accordance with the proposed legal changes referred to in Part V.1
above, including:
b) Design of changes to the institutional structure of the MOPW
consistent with any legal or regulatory reforms implemented under
the Program.
c) Design of the work scope and institutional arrangements that
would integrate the management of water resources.
d) Establishment of: (i) a cadaster or registry of existing MOPW
norms; (ii) procedures that regulate the creation, evaluation and
approval of new norms; (iii) procedures for populating,
maintaining, disseminating and updating information systems; and
(iv) a web-based database of the new norms and procedures.
7
-
e) Development of a human resource strategy based on performance
for MOPW. f) Development of an information technology strategy to
support the quality and
efficiency of MOPW's business processes. 3. Carrying out of
studies and training related to the development of an
organizational
separation of the planning, contracting, execution and
regulatory functions of infrastructure provision, as proposed by
the Borrower and agreed to by the Bank.
1.6 Revised Components The project components remained unchanged
throughout the life of the project.
1.7 Other significant changes By mid-2012 the Borrower requested
and the Bank agreed to a one- year extension of the loan with a new
closing date of January 31st, 2014. The objective of the extension
was to allow further disbursements on long-term consultancies
financed under the loan and to include changes in the procurement
rules and the implementation of Anti-Corruption Guidelines as
described below. The Procurement and Consultant Guidelines were
updated to reflect the versions dated January 2011, which allows
the use of framework agreements, known as “Convenios Marcos” in
Chile, for the procurement of goods, works and non-consultant
services. In addition, the Bank agreed to raise the maximum limits
for using ChileCompra for consultant services, from US$100,000 to
US$300,000. In addition, the definition of “Convenios Marco” was
amended to include non-consultant services. Finally the
Anti-Corruption Guidelines dated October 15, 2006 and revised in
January 2011 was to apply to the Project to be consistent with the
updated version of the Procurement and Consultant Guidelines.
2 Key Factors Affecting Implementation and Outcomes
2.1 Project Preparation, Design and Quality at Entry The ICR
finds the quality at entry as moderately satisfactory, since:
Project design was in line with the main pillars of the Bank’s
strategy for Chile. By supporting the implementation of the
Government's Institutional Strengthening Program, the TAL was in
line with the goals and strategic approach set out in the joint
IBRD-IFC Country Partnership Strategy (Report No 38691-CL). The
objective of the CPS was to support Chile's efforts to converge
with OECD income levels and living standards by (i) accelerating
growth and (ii) building a more equitable society. The Bank Group
strategy was based on the maintenance of close engagement with the
authorities, which
8
-
in recent had yielded important results in support of key
Government programs and through flexible, timely support that
helped to advance Chile's development vision A strong policy
dialogue and analytical underpinning contributed to a sound project
design. The objectives and actions of the project were the result
of more than a year of work at MOPW and from consultations with
relevant external stakeholders. The effort was coordinated by a
multidisciplinary team composed of MOPW staff with specific
responsibilities for each technical area of reform. That team
identified the main shortcomings of the MOPW and developed the
proposals for their improvement. The Bank supported the work of
these teams with technical and regulatory advice since the early
stages of the reform’s inception. The result was a very
comprehensive and detailed set of objectives and tasks that
provided an excellent blueprint for a complex institutional reform.
The choice of lending instruments contributed to project
continuity. The TAL was presented to the Bank’s Board of Directors
together with the DPL. The Ministry of Finance and the MOPW, as
already mentioned, were attracted by the synergies that could be
gained from having both loans to finance several of their key
initiatives. First, because the thematic and substantive linkage of
the two projects was expected to provide an effective framework for
monitoring and evaluating the implementation of the Ministry’s
reform program for a period long enough that exceeded one
administration. Second, by having both a DPL and TAL, the political
profile of the Bank-financed projects increased and made possible a
contingent role of the Bank as an honest broker between the
Ministries of Finance and Public Works when the reform suffered
from delays during implementation. The quality at entry would have
deserved a satisfactory rating if it had included a more adequate
results framework. Some of the originally proposed PDO and
intermediate indicators were difficult to measure and unrealistic
in terms of timing. As shown in Section 1.3 preceding and discussed
in more detail in section 2.3 below, some of the indicators had to
be revised to make them more precise, measurable and in some cases,
achievable within the time frame of the project.
2.2 Implementation The project was approved by the Bank's Board
on June 14th, 2007 and it became effective on February 27, 2008.
Implementation progressed steadily throughout the life of the
project, albeit at different pace during different periods. In
particular, internal resistance to the modernization program,
changes in government administrations and staff directly involved
with project implementation, at times lack of adequate funding, and
poor disbursement rates that jeopardized Bank involvement,
contributed to slower
9
-
project implementation.5 However, in spite of its rocky
beginnings, these problems were eventually overcome and the
modernization program, although a work still in progress, has
succeeded in bringing profound changes to MOPW’s view of its role
in a world in which users have a stronger voice and increasing
demands. The following paragraphs summarize the most salient
aspects of the implementation process. Resistance to change. The
launching of the project by its original champion (minister Bitar
during the first Bachelet administration) was immediately faced
with strong opposition from MOPW’s staff. Unions interpreted the
proposed reforms as implying privatizations and the redundancy of
some services. Senior management within MOPW realized that the
reform principles were not properly communicated and slowed down
implementation. The impasse allowed further work with the unions to
convey the message that the objective of the reform was to improve
the efficiency of public spending and that no redundancies were
being considered. Even when the issue of potential redundancies was
put to rest, there was (and, with lesser strength still is) some
institutional inertia to change the vision from a provider of hard
infrastructure to a provider of high quality infrastructure
services in those remaining few areas that are still not fully
mainstreamed. Team leaders have been key in involving MOPW senior
and junior staff (both in Santiago, where the headquarters of the
MOPW are located and in all the country’s Regions) in the
modernization process. They have fostered implementation of the
reform by organizing trainings, strengthening the dialogue and
building coordination stages among areas that had traditionally
worked somewhat in isolation in order to bring everybody on board
of the institutional changes. Changes in administrations and staff.
During the first two years of project implementation the high rate
of turnover of PIU’s coordinators considerably slowed down progress
in the modernization program. Since appraisal to late 2009 the
project had four coordinators, an average of one every six months.
Subsequently, the upcoming elections and expected change of the
administration in March 2010 coupled with a change in priorities to
focus on a stimulus package to increase economic activity,
contributed to further delays. The elections brought about a change
in the coalition in power (the new one being from the center-right)
after more than 20 years of a center-left coalition and the new
government was faced immediately with the devastating earthquake
that hit Chile in February, 2010. After the first few months of the
new administration, the Modernization Program received a strong
endorsement. The tools developed and implemented by the Program
were tested, fully utilized and proved effective in the emergency
reconstruction projects that followed the February earthquake, a
fact that contributed to the renewed strong Government support of
the program. The MOPW undertook several actions to increase the
implementation pace of the modernization efforts, including
providing a stronger institutional support with its transfer from
the Directorate General of Public Works to the Undersecretary of
Public Works and the appointment of a new highly effective project
coordinator.
5 It should be noted that progress on project implementation was
fairly independent from the rate of disbursements during the first
half of execution, since in Chile loan funds are not additional to
the Ministry’s budget.
10
-
Disbursements and Project funding. The first two years of the
Modernization Program were implemented without Bank funding, with
the first disbursement of loan funds taking place in mid-2010. The
poor disbursement rates and the slowing down of project
implementation during 2009 caused the revision of the PDO rating to
MU. Supervision missions identified the problem as a worrisome lack
of communication with the MOF with respect to how budget allocation
mechanisms work in Chile when funds from a Bank loan are available.
Bank funds are not additional to budget allocations and the latter
were many times assigned to areas of the program that were not
eligible for Bank financing. The dismal disbursement rates were the
subject of the mid-term review and the subsequent meetings
involving the Bank’s country director in which the Bank team agreed
with the PIU and MOF that if changes were not made before June 2011
the Project would be canceled. The agreement included to: (1)
appoint a new Project Coordinator, with knowledge of the
Modernization Program and credibility within the organization; (2)
request at least one disbursement before June 2011; (3) define the
new administration’s prioritization areas and launch an activity
(consultancy, workshop) in at least one of these areas; and (4)
provide a stronger commitment to the Modernization Program.
Authorities in the Ministry of Public Works, confirming the
importance that was attributed to the continued presence of the
Bank for the success of the Modernization Program complied
extensively and fast with the agreements made. The actions
undertaken included: (1) a new and highly effective Project
Coordinator, a long term qualified official in the Ministry was
appointed; (2) a new request for retroactive disbursement was
presented to the Bank; and the Minister gave a stronger
institutional support to the Modernization Program by transferring
the management from the Directorate of Public Works to the
Undersecretary of Public Works. In practice, this implies giving
more visibility and power to the PIU. Moreover, the Strategic
Directorate that was created as a high level coordinating unit to
monitor the progress of the Modernization Program started to meet
again once a month, after 15 months of discontinuity. Several
articles about the Modernization Program were published in many
newspapers as a result of interviews with the Undersecretary,
providing clear support to the Modernization Program. Disbursement
issues were satisfactorily addressed to a great extent through the
already discussed loan restructuring. By the beginning of 2013
disbursements reached 34 percent of loan funds and accelerated
considerably during the year. By the loan closing date US$8.24
million were disbursed (70 percent of the loan) and the remaining
US$3.45 million were canceled. Lower than expected budget
allocations also contributed to slow the implementation of the
Program. An estimated front-loaded cost structure resulted
inadequate to finance a program in which the initial stages were
devoted to communicating visions and goals, seeking consensus,
discussing strategies and alternative methodologies, seeking
information from different international experiences and
subsequently develop time consuming terms of reference for complex
studies and/or technical assistances. Since the initial allocations
were not fully utilized, and the MOF relied on past execution rates
to
11
-
define subsequent allocations, funding was decreasing when the
needs increased. Budget allocations during 2009 and 2010 were at a
level that jeopardized the continuation of the program. Again,
these issues were resolved once discussions were held to
restructure the project and place it in the right track.
2.3 Monitoring and Evaluation (M&E) Design, Implementation
and Utilization
As already discussed, the result framework of the TAL included
targets that were not measurable or lacked a clear distinction
between end-of program outcomes monitorable at the time of
completion and outcomes that would materialize in the longer term
as the reform process consolidates. In addition, some of the
indicators for the medium term outcomes were hard to measure, not
very realistic, and misleading.6 These drawbacks of the M&E
framework were identified very early during the project execution
and by 2009 a revised set of more realistic and measurable
indicators were prepared by MOPW and agreed with the Bank. The
revised set of medium term outcomes helped to narrow down and
provided useful milestones for each policy area objectives. The
fine tuning of the indicators to ensure their alignment with the
expected outcomes, the availability of data for their periodic
monitoring, and the establishment of baseline values was carried
out as part of the supervision activities of the TAL. The process
of revision of the M&E and its implementation was useful to
strengthen the dialogue with the counterpart and identify new
opportunities for support. Both the interim semiannual reports
prepared by the MOPW and the supervision missions were particularly
useful to collect data relating to the project implementation. The
missions provided the framework for an in depth follow-up of the
project components’ progress and served to highlight pending issues
to the authorities. In addition the supervision missions gave the
Bank and the MOPW the opportunity to deepen the policy dialogue and
identify areas of joint cooperation that were not identified during
project preparation. Finally, a field visit conducted during
project supervision served to corroborate the process of the reform
implementation in the regions.
6 One PDO indicator was the increase in the rate of return (of
an investment project) due to a better planning methodology. Even
when some positive correlation may occur, the causality between the
new planning methodology and the rate of return could be very
difficult of being proven. Moreover, the indicator does not provide
guidance if the rate or return should be measured ex ante or ex
post (once the project is operative). For instance, some typical
issues in project preparation such as the underestimation of cost
and overestimation of demand can be avoided by the implementation
of the new planning methodology. However, the rate of return may
decrease during the project preparation if cost is higher and
demand is lower. A second indicator was defined to capture the
increase in market competition due to a new contract and process
standardization measured by a Herfindahl index (HHI). While the
contract and process standardization can lead to a more transparent
and simple procurement process, the capacity to affect the market
share of public contractors can be negligible or impossible to
capture. Indeed, the entry of new contractors to the market of
public works, if occurs, can take some time and be affected by
other variables.
12
-
2.4 Safeguard and Fiduciary Compliance Limitations imposed by
the Bank’s procurement rules contributed to the initial substantial
lag in disbursements. The first disbursement took place in October
2010, more than two and a half years after the loan became
effective. Initially, one of the main reasons for disbursement
delays was the role played by Contraloría (auditing body of Chile)
who added at least 6 months to the procurement process of each
consulting work. Eventually, Contraloría agreed to work on a
standard form for Bank-financed consultancies. Subsequently, and as
the pace of implementation improved, the limitations were imposed
by the Bank: i.e. matters related to anticorruption clauses in
contracts and the limit of US$ 100,000 for contracts using the
ChileCompra system.7 The Borrower repeatedly stressed the
importance of revising the procurement procedures and the positive
impact that it would have on disbursements, as several contracts
could not be financed by the loan due to that particular
limitation. On December 12, 2012 the Loan Agreement was revised to
extend the original closing date to January 31st, 2014 and to
update the Procurement and Consultant Guidelines to reflect the
versions dated January 2011, which allows the use of framework
agreements, known as “Convenios Marcos” in Chile, for the
procurement of goods, works and non-consultant services. In
addition, the Bank agreed to raise the maximum limits for using
ChileCompra for consultant services, from US$100,000 to US$300,000.
In addition, the definition of “Convenios Marco” was amended to
include non-consultant services, and the Anti-corruption Guidelines
dated October 15th, 2006 and revised in January 2011 was to apply
to the Project to be consistent with the updated version of the
Procurement and Consultant Guidelines. MOPW had adequate financial
management systems, software, and trained staff. No systemic
problems were identified during implementation. The rating of the
project, however, remained as MS (moderately satisfactory) mainly
because of the high rotation of the financial management staff.
This issue impacted the preparation of financial reports, which
presented problems and needed some corrections to be accepted.
Contraloría, the National Audit Office, also made comments
requesting some changes and improvements in the presentation of the
information. Finally, the correction of some issues took time and a
few issues were repeated in other reports.
2.5 Post-completion Operation/Next Phase The vehicle used for
the implementation of the Institutional Strengthening Program of
the MOPW (Programa de Modernización del MOP) ends at the end of
June 2014. The long-
7 The hiring of consultant firms with estimated cost less than
US$100,000 was to be generally carried out using national
procedures which include advertisement and processing through the
Chilecompra Portal: such procedures - that may include open
competition are not consistent with the short listing procedures as
specified in the Bank's guidelines but in Chile such procedures
ensure transparency and quality service and therefore, the Bank
allowed the direct contracting of the firms thus selected.
13
-
term sustainability of the achievements seems to be guaranteed
by the fact that the program was a mean to improve the functioning
of the different units of MOPW. That is, each unit involved in the
program was the final recipients of the technical assistance and
they are advancing independently with cementing their new role. The
MOPW, however, is interested in continuing with institutional
strengthening in two areas where the Bank might be requested to
provide support in the near future: i) contract management in
infrastructure concessions; and ii) institutional set up of the
water sector. The first topic refers to the need to improve the
oversight function of the concession unit and it was identified as
a priority by the recently appointed management of MOPW. On the
water sector, some progress was achieved during the program but
there are other reforms that required further support and the
timing of the project was not suitable for their
implementation.
3 Assessment of Outcomes
3.1 Relevance of Objectives, Design and Implementation The main
objective of the TAL remains extremely relevant to Chile’s current
agenda. The modernization program of MOPW remains very much in line
with the government’s focus on (i) investment decisions based on
programs fostering equity, territorial integration and regional
development; (ii) a more equitable distribution of public-private
risks and better oversight of the provision of services; and (iii)
the shift towards a performance based infrastructure management as
measured by users’ satisfaction. The recent reelection of President
Bachelet who’s administration was the original champion of the
modernization process, provides further enhancement to the
relevance of the objectives which have been already mainstreamed in
the operations of MOPW. Chile’s continued efforts to converge with
OECD countries performance provide a favorable support for the
reform. Finally, MOPW has an undisputed role in the Chilean economy
as a vehicle for implementation of countercyclical policies. 8 The
need to improve the efficiency and performance of the MOPW becomes
even more important in the light of its increasing role as growth
and employment generator. The objective of the loan also remains
relevant for the achievement of the Bank strategic objectives,
which emphasized the need to continue to focus on the modernization
of the state through a close engagement with the Chilean
authorities. Likewise the project objectives are fully consistent
with those set in the Country Partnership Strategy (CPS) endorsed
by the Board in May 2007 and in February 20109, contributing
directly to objectives of supporting the efficiency of
infrastructure and public services in addition to decentralization
and regional development.
8 Chile has been cited as an example of how to implement
adequate countercyclical economic policies. See for instance the
article titled “Stimulating” published by The Economist on February
19, 2009 (available at
http://www.economist.com/world/americas/displaystory.cfm?story_id=13145570)
9 World Bank 2011. Chile Country Partnership Strategy 2011-2016.
Report No. Report No. 57989-CL
14
-
3.2 Achievement of Project Development Objectives (PDO) PDO
Indicator 1: “New planning methodology for infrastructure
functioning and fully incorporated in the investment decision of
the MOPW” The project fully achieved all the intermediate
institutional targets leading to MOPS’s full adoption of a novel
integrated planning. The new methodology derived from a strategic
long-term view of regional development and was brought down to the
tactical selections of project with an integrated geographic
perspective oriented to the competitiveness of Chile's productive
sectors and the equitable and sustainable access to goods and
services to all the country’s territories. The new methodology, now
being used in 99% of the MOPW’s newly approved projects, includes a
high level of interaction with other relevant stakeholders in the
infrastructure sector and a societal or consumer voice in project
planning. The integration of the investment programs along a supply
chain is expected to substantially decreased coordination
difficulties and inefficient expenditures. All MOPW’s projects
included in Chile’s 2014 budget come form strategic investment
programs evaluated using the new methodology. In 2010, with the new
administration, the Bank was requested to pay increased emphasis on
water resource management under the integrated planning activities.
A large consultancy launched in 2011 produced a manual being
successfully used for the planning and management of rainwater in
large urban centers, thus contributing to the objective of improved
planning in a one of MOPW’s critical priority areas. PDO Indicator
2: “Greater competition in bidding supported by fair risk-sharing
arrangements embedded in contracts and consumer responsive service
targets”. This objective was partially achieved. The objective was
fully achieved with respect to the incorporation of standard of
services in concessions and civil work contracts. Progress in
ensuring greater competition in the bidding process centered in a
series of studies and proposed revisions to reduce the excessive
restrictions that limited the inclusion of participants in the
registry of consultants and construction companies, to increase the
precisions of the bidding documents and to reduce discretionary
decisions. The proposed regulations for public works incorporating
the new proposals were presented to the Contraloría in 2012.
Contraloría raised several comments on the document but stop short
of its approval. Clearly this is the most contentious area were
many strong interests are at stake and decisive political support
is necessary for any meaningful change. Nevertheless, even without
the actual formal revision of the Regulation of Public Works
(Reglamento de Obras Públicas), some progress was made in the
standard bidding documents, such as the handling of the two-
envelope system, the improvement of the polynomial adjustment
method and the dropping of the publication of MOPWs project budget.
Further progress and full approval of a new Reglamento is a
challenge left to the new Bachelet administration.
15
-
PDO Indicator 3: “Greater responsiveness to consumer concerns
related to the levels of service; and greater oversight of
performance by public sector service providers as well as
concessionaires” This objective was fully achieved. This was
probably one of the more complicated objectives in terms of
resistance to a complete change in the conceptual and
methodological model to implement a service- oriented approach to
public works. After a process of identifying service levels and
standards for the design, construction and exploitation of each
type of works, these are being introduced as explicit obligations
in an increasing number of concession contracts for airports,
roads, hospitals, etc., actually surpassing the projections of the
intermediate targets. By the end of 2013 MOPW had entered more than
120 contracts with explicit service standards which, in addition to
several contracts in hospital and airport concessions, include
performance-based and ad-measurement contracts comprising 13,371.53
km or 71.7% of the country’s road network. In addition MOPW
successfully developed and implemented through surveys a system for
the evaluation of user’s satisfaction in at least four types of
services and is continuing with the task of developing the systems
for other services under their responsibilities PDO Indicator 4:
“Reduction in variance between values for construction/investment
established at contract signature and final values at project
execution.” The objective was fully achieved. Reductions in
variance between costs for construction/investment established at
contract signature and final values at project execution were
confirmed by comparing contracts carried out before and after the
full institutionalization of the new project management approach.
The creation of Project Management Offices at the central and
regional levels in 2010 and the full adoption of the “project
manager” figure increased accountability and reduced atomized
decisions. The new organization is being supported by the
development of a solid digital platform to provide rapid
information on project costs, contract management and/or design and
construction issues. The effectiveness of the new project
management system was fully proved during the reconstruction
efforts following the 2010 earthquake, a fact that was instrumental
in securing the new administration support for the project.
Objective 5: “Normative and organizational structure that provides
clear separation and definition of roles and responsibilities for
planning, contracting, works execution and oversight” Partially
achieved. The Concession Law, one of the targets in this component,
was approved and made public as Law 20.410 on January 1st, 2010.
However, the main laws changing MOPW’s organization (the
Superintendence Law and the Organic Law of MOPW) were not pursued
and were not considered a Government priority. Instead, MOPW is
implementing a strong oversight effort through contract oversight
systems that are already in place for concessioned airports (e.g.
information screens, escalators, etc.) and for contract oversight
of rural potable water.
16
-
3.3 Efficiency The project should be considered as highly cost
effective since costs should be exceedingly compensated by improved
capacity reflected in fiscal savings and efficiency gains. The MOPW
manages a budget for 2014 of about US$ 3.3 billion of which about
US$ 2.4 billion are allocated to investments. The cost of the
project, at a total of US$ 29.15 million, represents less than 1%
of the budget of one year, yet it was sufficient to introduce deep
and permanent improvements in efficiency and service levels. The
already confirmed reduction in the costs and execution times of
contracts that are being derived from the adoption of an
“integrated management system” would exceedingly compensate for the
costs of the project. Additional gains, probably more sizable but
more difficult to quantify, should be derived from the selection of
projects that focus on equity and the social benefits that were
previously ignored.
3.4 Justification of Overall Outcome Rating Rating: Satisfactory
The ICR considers the overall outcome of the project as
satisfactory since (a) project objectives remain highly relevant to
the development of public infrastructure in Chile and the
government’s pursue of growth with equitable regional development;
(b) the project demonstrated satisfactory achievements of its DOs,
especially in regard to full institutionalization of integrated
planning, efficient performance based management of projects and
services, a more substantive oversight of the provision of
services, and the inclusion of user satisfaction as a key measure
of performance; and (c) the project passed the test of approval of
administrations covering a wide range of the country’s political
spectrum and is fully embedded in the modus-operandi of MOPW.
Moreover, the project was instrumental in bringing about the
modernization of MOPW. Without the project, and thus the Bank
involvement, it is unlikely that the Modernization Program would
have been kept in the agenda and in the country’s budget. Even when
the financial contribution of the loan was at times minimal, MOPW
insisted that the continuous presence of the project was vital for
the continuation of the reform program. The satisfactory rating,
however, does not coincide with the last ISR’s rating of Moderate
Satisfactory, which unfortunately gives inordinate weight to the
poor disbursement performance of the project rather than to the
achievements of the development objectives and was not revised when
disbursements picked up.
3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty
Impacts, Gender Aspects, and Social Development
The operation built the foundations for generating an indirect
impact on poverty alleviation by inducing greater efficiency,
effectiveness and accountability in the utilization of public
resources. It also broadened participation in public infrastructure
investment, and fostered transparency and accountability in
infrastructure service delivery (principally by enhancing
oversight, and access to information). In addition, the
17
-
project promoted instruments such as strategic environmental
assessments that have already fostered (and will continue to
foster) the integration of environmental aspects into the planning
process. All the above, will help Chile to achieve OECD standards
for public works projects that contribute to sustainable
development. (b) Institutional Change/Strengthening The operation’s
institutional impact was and will be substantial. It established a
new institutional paradigm for the MOPW. This includes a new
planning methodology designed to provide an integrated strategic
vision for all types of infrastructures. The operation introduced
the profile of the project manager which was key to create an
integrated project management approach that covers the full life
cycle of projects. It also designed a Project Management Office
(PMO) to improve the management and oversight of the portfolio of
projects under the responsibility of the MOPW. The operation
further developed an integrated technological platform that is
building a database for monitoring the performance of concessions
and public works. (c) Other Unintended Outcomes and Impacts
(positive or negative) One interesting outcome that was not
included in project design was the establishment of the Public
Works Institute in the MOPW. The reform process required a
substantial amount of training on planning, project management,
contract standards, concessions, information technology, etc. As it
was found that many of the professional being sought in academia
for providing training were members of the MOPW, the government
decided to create the in-house training Institute, a move that is
resulting in the continuous upgrading of the professional skills in
the Ministry. Finally, the relevance of this project for the Bank
agenda should be highlighted as Chile is an example for the Bank in
the area of infrastructure service provision. The Bank has
constantly relied on Chile to illustrate to other countries in
different regions how infrastructure reforms should be conducted,
in particular in the area of attracting private sector financing
and operators through public private partnerships. World Bank staff
and public officials from East Asia and South East Asia have
conducted study tours to learn from the Chilean experience. Thus,
the Bank will benefit significantly from the MOPW modernization
process as it will provide valuable lessons for other countries
that request the assistance of the Bank to follow a similar path of
reforms.
3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder
Workshops
N.A.
4 Assessment of Risk to Development Outcome
Rating: Negligible
18
-
The sustainability of the achievements of the TAL operation is
deemed substantial, given that most of the objectives have been
fully institutionalized and became an integral part of MOPW’s modus
operandi. While to continue to move forward and deepen the
modernization efforts, particularly for the still incomplete legal
and regulatory changes, would clearly require the presence of a
reform-minded management team and strong political support progress
made to date is most likely to endure changes in leadership. There
are several factors that mitigate the risk associated with
potential changes in leadership. In addition to the strong
ownership of the new MOPW’s vision within the institution, risks
are reduced by: i) the already strong support from different
administrations; (ii) the users pressures for reform resulting from
improved performance; iii) the heightened role of public
consultation organisms; and iv) the strong ownership within MOPW’
of its strategic plans. Finally, the risk of cultural resistance
and vested interests within the institution opposing project
implementation, probably the most tangible one during the early
years of project implementation, has been largely contained. A
well-established organizational culture and self-interest inertia
originally resisted institutional change within the MOPW.
Mitigation measures aimed at fostering stakeholders’ participation
and promoting change management served to counter-balance this
resistance to change. The gradual integration of new profiles (such
as the project manager) and the progressive definition of new
institutional bodies for supervision and coordination, also served
to mitigate this risk. Addressing the risks imposed by interests
outside the institution to move forward with the legal and
regulatory changes still necessary to fully complete the
modernization program will require further work, particularly to
strengthen the communication channels between the executive and the
legislative branches of government to gather support for the
modernization of the Ministry’s legal structure.
5 Assessment of Bank and Borrower Performance
5.1 Bank Performance (a) Bank Performance in Ensuring Quality at
Entry Rating: moderately satisfactory The rating for Bank
performance during the preparation phase is moderately
satisfactory. The justification for this rating is as follows:
acknowledging the complexity of the reform, the Bank assigned a
highly qualified team with extensive experience in the relevant
policy areas. The team was able to maintain a close collaboration
with the Minister and his team from the beginning, and responded
promptly and with high levels of flexibility to their requests for
assistance in certain topics (particularly the drafting of the
Concessions and Superintendence of Public Works Law) at the outset
of the reform process. Experts from different parts of the world
participated actively in preparation
19
-
missions and workshops, providing ad-hoc advice directly to the
Minister and his cabinet on specific issues. The team’s appropriate
skill mix helped deliver a well-designed project and provided value
added to the Borrower. The Bank’s performance assisting the
Borrower in identifying and appraising the project to ensure
consistency with the CAS and with government’s priorities, and in a
manner that would ensure attainment of expected outcomes, was also
very valuable. Performance at entry could have been satisfactory if
design of the M&E framework had included more relevant and
measurable indicators. (b) Quality of Supervision Rating:
moderately satisfactory Traditional supervisory activities to
monitor progress toward mid-term outcomes were carried out and
missions traveled to the field more than twice a year, with
additional regular Bank support provided directly by the Project
team. Beyond the focus on regular supervision matters, there was a
strong interest on the part of the Bank in assessing impact of
project activities and, in doing so, maintaining central government
focus in the reform agenda. To ensure a feasible impact analysis,
objective indicators, and the availability of the required data to
monitor their evolution throughout project implementation, the
M&E framework for the project was revised with Chilean
counterparts. In addition the supervision team was very proactive
and successfully intervened to keep the project alive when the
Government support faltered. Supervision missions did play the
productive envisaged role of “honest broker” between MOPW and MOF,
and all of the MOPW officials interviewed for this ICR coincided in
that the presence of the Bank was instrumental in saving the
project when the political support was at its lowest. The less than
satisfactory rating is due to the following: (i) the revision of
the result framework was never formalized. Although a problem of
form more than substance, the change would have improved the rather
confusing presentations of the result framework in the ISRs, which
mixed the original and revised indicators; and (ii) the supervision
team relied heavily on disbursements rather than on achievements to
evaluate project performance. (c) Justification of Rating for
Overall Bank Performance Rating: moderately satisfactory. Based on
the above, the ICR considers the overall Bank performance as
moderately satisfactory.
5.2 Borrower Performance (a) Government Performance Rating:
satisfactory Government performance is rated satisfactory due to
the commitment and involvement in MOPW’s reform initiative, as
evidenced by: (i) the active participation of the Ministry of
Finance from project inception. The positive commitment of the
Ministry of Finance, given the structure of power in Chile, is a
sine qua non condition for the successful
20
-
implementation of any institutional reform in Chile; (ii) the
explicit support from the highest levels of government to the
reform program in public speeches and press releases; and (iii) the
valuable contributions and participation from staff of other public
agencies in multiple initiatives of the reform program (in
particular, involvement of multiple agencies in the seminars
carried out by MOPW under the Integrated Planning policy area). On
the other hand, the poor budget execution of the reform program
during its first two years prompted MOF to reduce budget
allocations at a time in which the program was starting to take
off. These shortages of funds contributed to slowing down the
implementation of the project. (b) Implementing Agency or Agencies
Performance Rating: satisfactory Implementing agency performance is
rated as satisfactory. From the beginning of project
implementation, MOPW has assigned a qualified team to lead the
initiatives under each policy area. These teams have worked hard in
the production of terms of reference, and in the organization of
trainings and other events; strengthening the dialogue and building
coordination stages among Ministry areas that had traditionally
worked in relative isolation. The Project Coordinating Unit has
complied with all of its fiduciary responsibilities in a
satisfactory manner, and played an important role in providing
relevant information to assess the attainment of outputs
established for the ICR. (c) Justification of Rating for Overall
Borrower Performance Rating: satisfactory Based on the above, the
ICR considers the Borrower performance satisfactory.
6 Lessons Learned
• An intensive early period of communication and participation
as part of the process of problem recognition and problem
definition is fundamental for ensuring support to institutional
changes. MOPW underestimated the need for launching an effective
process of communication for gathering staff backing for the
modernization program. Even once the initial concerns for job
stability were addressed, the acceptance of new conceptual and
behavioral norms in a big and powerful institution proved a
difficult task that was eventually overcome by moving from a
“diffusion of communications” to a more participatory approach for
setting common goals.
• The endorsement and strong ownership of the leaders of an
institution are crucial to ensure the continuity and success of
major reforms. After Minister Bitrán, who was the original reform
champion, left office in January 2008, immediate support by his
successor, Minister Bitar, was decisive to providing continuity to
the modernization program. This endorsement was critical at a
time
21
-
when support to the reform was waning within the MOPW due to
weaknesses in the internal communication of the reform’s principles
and objectives among ministry’s stakeholders. The priority of the
program was further and definitely confirmed in 2010 when the
administration changed hands, thus ensuring its continuity.
• The cost curve of institutional reform should take into
account the lead- time necessary for consensus building and task
preparation. The cost curve estimated for the TAL was front-
loaded. During the initial years, however, the needs focused on
internal deliberations on visions, strategies and goals. These
required the help of individual consultants to define future
assistances and develop complex Terms of Reference for more
resource consuming consultancies. Thus, actual need for funds
increased with time (rather than decrease as projected). This
discrepancy between actual and expected costs was at times a source
of project underfunding.
• The Bank should evaluate more closely the procurement
thresholds or consider more flexible procurement rules for
countries with a proven record of controlling corruption. The
procurement limits originally established for the use of
ChileCompra, were at the center of the poor disbursement record of
the project. Its revision and the use of limits more according to
the procurement record of Chile (which probably should have taken
place as soon as the problem was identified), proved instrumental
in turning around disbursement s performance.
7 Comments on Issues Raised by Borrower/Implementing
Agencies/Partners
(a) Borrower/implementing agencies No comment was provided – see
Annex 7. (b) Cofinanciers (c) Other partners and stakeholders (e.g.
NGOs/private sector/civil society)
22
-
Annex 1. Project Costs and Financing
(a) Project Cost by Component (in USD Million equivalent)
Components Appraisal Estimate (USD millions)
Actual/Latest Estimate (USD
millions)
Percentage of Appraisal
1. Integrated Planning 7.59 4.12 -45.7% 2. Standards, Contracts,
and
Competition 4.85
1.22 -74.9%
3. Integrated Project Management 4.00
6.28 +57.1%
4. Regulatory Strengthening (fiscalización) 6.23 1.74 -72.1%
5. Organizational Reform 6.50 9.13 +40.5% 6. Administrative
costs and
contingencies 6.19 6.66 +7.5%
Total Project Cost 35.36 29.15 -17.6%
(b) Financing
Source of Funds Appraisal Estimate
(USD millions)
Actual (USD
millions)
Percentage of Appraisal
Borrower 23.66 20.80 -12.1% International Bank for
Reconstruction and Development 11.70 8.35 -28.6%
23
-
Annex 2. Outputs by Component This annex lists the main goals
achieved in each component. For a more detailed description
including intermediate taken steps, studies, and training, etc.
please consult the Modernization Program progress reports that MOPW
prepared semiannually for the World Bank. COMPONENT 1: INTEGRATED
PLANNING (Appraisal estimated cost: US$ 7.59 million; actual cost:
US$ 4.12 million ) INFRASTRUCTURE INTEGRATED PLANNIG:
Implementation of a full planning process for infrastructure
services, which support the decision making process for investment
by the Ministry of Public Works aligned with the strategic criteria
and the vision of the country. A planning process which also
strengthens competitiveness, which enhances the productive
development of each territory, and ensures its inhabitants equal
access to goods and services in a sustainable and integrated
manner.
OBJECTIVES ACHIEVEMENTS
• Prospective Development o Development of Country Vision
2020 and its relation to infrastructure
o Design of policies for long-term planning concerning: land
use, relocation of settlements, transportation, water resource
management and sustainable territorial development.
o Production cluster’s analysis and its impact on
infrastructure
o Elaboration of studies and delivery of training related to the
development of a sustainable and integrated approach to
infrastructure planning.
• Planning Model and Process o Installation of participatory
Initiatives o Design of methodologies to
promote a participatory planning process for infrastructure
investments
• Transversal Support Systems o Development and
implementation of an information system and a database for a
Territorial Information System “Sistema de
• Set up of the Int