Document of The World Bank FOR OFFICIAL USE ONLY Report No: 63764-CL PROJECT APPRAISAL DOCUMENT ON A PROPOSED LOAN IN THE AMOUNT OF US$40 MILLION TO THE REPUBLIC OF CHILE FOR A TERTIARY EDUCATION FINANCE FOR RESULTS PROJECT III FEBRUARY 9, 2012 Human Development Sector Bolivia, Chile, Ecuador, Peru and Venezuela Country Management Unit Latin American and the Caribbean Regional Office This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized
59
Embed
The World Bank FOR OFFICIAL USE ONLYdocuments.worldbank.org/curated/en/258391468231854219/pdf/637640… · the world bank for official use only report no: 63764-cl project appraisal
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Document of
The World Bank
FOR OFFICIAL USE ONLY
Report No: 63764-CL
PROJECT APPRAISAL DOCUMENT
ON A
PROPOSED LOAN
IN THE AMOUNT OF US$40 MILLION
TO THE
REPUBLIC OF CHILE
FOR A
TERTIARY EDUCATION FINANCE FOR RESULTS PROJECT III
FEBRUARY 9, 2012
Human Development Sector
Bolivia, Chile, Ecuador, Peru and Venezuela Country Management Unit
Latin American and the Caribbean Regional Office
This document has a restricted distribution and may be used by recipients only in the
performance of their official duties. Its contents may not otherwise be disclosed without World
Bank authorization
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
CURRENCY EQUIVALENTS
November 29, 2011
Currency Unit = Chilean Pesos
CLP522 = US$1
CLP1 = US$0.002
FISCAL YEAR
January 1 – December 31
ABBREVIATIONS AND ACRONYMS
AFD Direct Fiscal Grant
APL Adaptable Program Loan
CAE State Guaranteed Student Loan Program, acronym in Spanish
CD Convenios de Desempeño, Performance Agreement
CDI Convenio de Desempeño Institucional, Institutional Performance Agreement
CDF Convenio de Desempeño Mediano, Focused Performance Agreement
CFT Technical Training Centers
CGR Contraloría General de la República, Supreme Audit Institution
CONICYT Comisión Nacional de Investigación Científica y Tecnológica, National Council
for Scientific and Technological Research
CONFECH Conferederación de Estudiantes de Chile, Chile‘s Student Confederation
COREHEG Tertiary Thematic Group
CPS Country Partnership Strategy
CRUCH National Council of Rectors of Chilean Universities
CUT Country‘s main trade union
DAG Departamento de Administración General, General Administration Department
DC Direct contracting
DFI Department of Institutional Financing
DIPLAP Dirección de Planificación y Presupuesto
DIPRES División de Presupuestos, Division of Budget
DIVESUP División de Educación Superior, Division of Higher Education
FA Framework Agreement
FECH Federación de Estudiantes de Chile, Chile‘s Student Federation
FIC Innovation Fund for Competitiveness, acronym in Spanish
FM Financial Management
FMA Financial Management Assessment
GoC Government of Chile
ICB International Competitive Bidding
ICU Institutional Coordination Units
IFR Interim financial report
IP Professional Institute
IPPF Peoples Planning Framework
ISA International Standards on Auditing
MH Ministry of Finance
MINEDUC Ministry of Education
NCB National Competitive Bidding
NPV Net Present Value
OECD Organization for Economic Co-operation and Development
OP/BP Operational Policies and Bank Procedures
ORAF Operational Risk Assessment Framework
PAD Project Appraisal Document
PDO Project Development Objective
PIU Project Implementation Unit
PMI Plan de Mejoramiento Institucional, Institutional Improvement Plan
POA Project Annual Operational Work
POM Project Operational Manual
PSU Prueba de Selección Universitaria (University Selection Test)
R&D Research and development
SEPA Procurement Plan Execution System
SIES Tertiary Education Information System
SIGFE Sistema de Información para la Gestión Financiera del Estado, National
Integrated Financial Management System
SOE Statement of Expenditures
TEI Tertiary Education Institution
TEIs Tertiary Education Institutions
TFP Total factor productivity
UCI Unidades de Coordinación Institucional, Institutional Coordination Units
UCP University-Community Partnerships for Social Action Research
Regional Vice President: Hasan A. Tuluy
Country Director: Susan G. Goldmark
Sector Director: Keith Hansen
Sector Manager: Chingboon Lee
Task Team Leader: Michael Crawford
CHILE
TERTIARY EDUCATION FINANCE FOR RESULTS PROJECT III
Project Implementation Period: Start Date: 06/15/2012 End Date: 10/31/2016
Expected Effectiveness Date: October 31, 2012
Expected Closing Date: October 31, 2016 .
Project Financing Data(US$M)
[ X ] Loan [ ] Grant [ ] Other
[ ] Credit [ ] Guarantee
For Loans/Credits/Others
Total Project Cost : 160.00 Total Bank Financing : 40.00
Government: 120.00 Financing Gap : 0.00 .
Financing Source Amount(US$M)
BORROWER/RECIPIENT 120.00
IBRD 40.00
IDA: New
IDA: Recommitted
Others
Financing Gap
Total 160.00 .
Expected Disbursements (in USD Million)
Fiscal Year 2012 2013 2014 2015 2016
Annual 5 10 10 10 5
Cumulative 5 15 25 35 40 .
Project Development Objective(s)
The objective of the Project is to improve quality and relevance for students in tertiary education by strengthening the link between funding of tertiary
education institutions and accountability for performance. .
Components
Component Name Cost (USD Millions)
1. Performance Agreements
2. Policy Support and Project Management
Total estimated cost: US$155 million; Bank: US$38 million
Total estimated cost: US$5 million; Bank: US$2 million
.
Compliance
Policy
Does the project depart from the CAS in content or in other significant respects? Yes [ ] No [ X ] .
Does the project require any exceptions from Bank policies? Yes [ ] No [ X ]
Have these been approved by Bank management? Yes [ ] No [ ]
Is approval for any policy exception sought from the Board? Yes [ ] No [ X ]
Does the project meet the Regional criteria for readiness for implementation? Yes [ X ] No [ X ] .
Safeguard Policies Triggered by the Project Yes No
Environmental Assessment OP/BP 4.01 X
Natural Habitats OP/BP 4.04 X
Forests OP/BP 4.36 X
Pest Management OP 4.09 X
Physical Cultural Resources OP/BP 4.11 X
Indigenous Peoples OP/BP 4.10 X
Involuntary Resettlement OP/BP 4.12 X
Safety of Dams OP/BP 4.37 X
Projects on International Waterways OP/BP 7.50 X
Projects in Disputed Areas OP/BP 7.60 X .
Legal Covenants
Name Performance Subprojects Recurrent Yes Due Date N/A Frequency Throughout
For the purposes of carrying out Part 1 of the Project, the Borrower, through MINEDUC, shall after having selected a Performance
Subproject in accordance with the criteria, guidelines and procedures set forth in the Operational Manual, and prior to the carrying out of
said Performance Subproject by the pertinent TEI, enter into an agreement with said TEI (―Subproject Agreement‖), for the provision of a
Grant for the implementation of a Subproject, under terms and conditions approved by the Bank and included in the Operational Manual.
Name Safeguards Recurrent Yes Due Date N/A Frequency Throughout
1. The Borrower, through MINEDUC, shall and shall cause the TEIs to: (a) implement the Project in accordance with the IPPF, when
applicable; (b) comply with the procedures detailed in said IPPF for screening, evaluation, implementation and monitoring of Performance
Subprojects; and (c) implement the pertinent indigenous peoples development plans, in accordance with their terms and in a manner
acceptable to the Bank.
2. The Borrower, through MINEDUC, shall, and shall cause the TEIs to ensure that any works to be financed under the Performance
Subprojects do not involve any Resettlement.
Name Special Provisions Recurrent Yes Due Date N/A Frequency Throughout
In addition and without limitation to any other provisions set forth in this Section or the Procurement Guidelines, the following principles of
procurement shall expressly govern all procurement of goods, Non-Consulting Services and works under National Competitive Bidding
procedures:
The lowest evaluated bid shall be selected for contract award.
There shall be no prescribed minimum number of bids to be submitted in order for a contract to be subsequently awarded.
ChileCompra, provided that the bidding documents are acceptable to the Bank.
The Mercado Público Framework Agreements (MPFA) may be used as an alternative to Shopping, as determined by the Bank,
provided that: (A) the specific MPFA does not restrict foreign competition, and is limited to a maximum duration of 3 (three)
years; and (B) the maximum aggregate amount for the use of an specific MPFA by each TEI set in the procurement plan is in no
case higher than the applicable Shopping maximum amount, and shall be agreed with the Bank.
In addition and without limitation to any other provisions set forth in this Section or the Consultant Guidelines, the following principles of
procurement shall expressly govern all employment of consultants:
There shall be no prescribed minimum number of proposals to be submitted in order for a contract to be subsequently awarded.
For small assignments not exceeding US$200,000 equivalent, the procedures of ChileCompra may be followed as a consultants'
selection method acceptable to the Bank. .
Team Composition
Bank Staff
Name Title Specialization Unit
Michael F. Crawford TTL Senior Education Specialist LCSHE
Maria Paulina Mogollón STC Technical Project Design & Safeguards LCSHE
Christopher Sharp STC Consultant LCSHE
Diego Ambasz Operations Officer Operations Officer LCSHE
Francisco Rodriguez Procurement Specialist Procurement LCSPT
Ana Lucía Jimenez Financial Management Specialist Financial Management LCSFM
Mariana Montiel Senior Council Lawyer LEGLA
Maria Elena Paz Gutzalenko ACS Program Assistant LCSHE
Guillermo Toral JPA Operations LCSHE
Non Bank Staff .
Locations
Country First Administrative
Division
Location Planned Actual Comments
.
1
1. STRATEGIC CONTEXT
A. Country Context
1. Chile is an upper middle income country which recently acceded to full membership in
the OECD. It has one of the most politically stable systems in Latin America, strong institutions
and a solid record of economic growth during the last two decades. A center-right administration
took office in March 2010, following on 20 years of continuous center-left governments. Despite
differences in political stance, the current Government is pursuing largely centrist policies with
overall goals similar to those of previous governments: promoting reforms to boost growth,
investing in human capital, and tackling poverty and inequality. Within this context, Chile is
challenged to reverse declining relative growth performance and insufficient innovative capacity
so that it can grow the size and wealth of its middle class. Chile's growth has slowed from an
average of 7.6% during the period of 1986-1997 to 3.9% in the period of 2000-2010.1
2. The Government of Chile (GoC) has launched a development agenda which sets the stage
for achieving high-income status by 2018. The strategic areas that the GoC is emphasizing to
reach this goal are: i) achieving greater competitiveness, including modernization of the state; ii)
improving job creation and job quality; and iii) promoting investment, including in human
capital. The Government has embraced increased investment and increased policy attention to
education at all levels as a key pillar of economic and social progress. While the Government
seeks to implement this agenda, it faces youth who want to see educational opportunity and
educational quality improved immediately and who are impatient for change.
B. Sectoral and Institutional Context
3. Chile‘s tertiary education system is diversified geographically and institutionally. Its
tertiary education institutions fall into three categories: Technical Training Centers (CFTs),
Professional Institutes (IPs) and Universities. Universities are further divided into two categories:
the traditional public and private universities who are members of the National Council of
Rectors of Chilean Universities (CRUCH for its acronym in Spanish), and the newer universities,
typically founded after the 1980s, who do not belong to it. Degree programs in Chile range from
short technical degrees to full doctoral programs. Degree length usually varies by the category
of tertiary education institution (TEI): CFT training typically lasts two years, IP professional
degree courses four years and university degree courses five years. Universities also offer
graduate programs and diplomas, including Master‘s and PhD programs. In 2010, 87 of Chile‘s
approximately two hundred (200) tertiary education institutions were accredited.
4. With a population close to seventeen million people, Chile has almost one million
students in tertiary education. A third of these attend CRUCH universities, a third private
universities, and the remaining third IPs and CFTs. Equity has improved significantly over the
past decade, with the number of students from the lowest two quintiles tripling to reach 25% of
total enrollment.
1 Rodrigo Fuentes & Fabián Gredig & Mauricio Larraín, 2007. "Estimating the Output Gap for Chile," Working
Papers Central Bank of Chile 455, Central Bank of Chile.
2
Institution Type Enrollment (2010)
CRUCH Universities 310,890
Private Universities 323,843
IPs 224,339
CFTs 128,571
Total 987,643
5. The Government finances TEIs via several mechanisms: (i) the Direct Fiscal Grant
(Aporte Fiscal Directo, AFD), which accounts for one-third of tertiary education public
resources and is allocated to CRUCH institutions; (ii) the Indirect Fiscal Grant (Aporte Fiscal
Indirecto, AFI), which accounts for about 5% of public spending and is a type of voucher
provided to the top 27,500 scorers on the university entrance exam; (iii) the competitively-
allocated funds available for R&D and quality improvements; and (iv) the student financial aid
packages (grants and loans) which are transferred to TEIs in students‘ names.
6. While Chile has made commendable progress in expanding access and improving equity,
the system still faces numerous challenges. The OECD/World Bank Review of Tertiary
Education (2009) noted that Chile now requires a ―second generation‖ of reforms that focus on
more complex challenges such as linking funding to performance and accountability, and
improving the quality and relevance of classroom instruction. Low quality has been linked to
the low percentage of PhD holders in Chile‘s professorate; raising this percentage is an objective
of policy and has been for more than a decade.
7. More closely linking funding to performance is a pressing need. In 2010, ~87% of
resources given directly to TEIs were allocated according to historical criteria and not
necessarily to merit.2 The Government can leverage progress on specific challenges like quality
and relevance if it ties incremental funding to progress against agreed indicators. In a complex
system like tertiary education, quality and relevance overlap considerably and their improvement
is multidimensional. By making funding competitive and linking it to accountability, the
Government incentivizes a range of responses. Each response is appropriate to the individual
institution while contributing to improved quality and relevance in the aggregate.
8. Chile‘s challenges for quality relate to (i) how to make curriculum and teaching practices
more rigorous and effective; (ii) how to promote institutional management that both facilitates
and requires high quality teaching and learning; and (iii) how to raise academic readiness for
students—especially first generation students—who may have had mediocre secondary
education and are not prepared to succeed in tertiary-level courses. Improving the quality of
pedagogy schools, which train the nation‘s primary and secondary teachers, is a critical corollary
to this.
9. With respect to relevance, the challenges involve (i) replacing the mandatory theoretical
content in many degree programs with content that promotes the skills and competencies
2 Calculated as funds from the Direct Fiscal Grant (AFD) over funds from MECESUP and the AFD. For year 2010
in Chilean pesos: 146,464,177 / 167,970,490.
3
required by employers; (ii) reaching a critical mass of highly qualified professors who can devise
and deliver curricula that, inter alia, connect to labor market needs; and (iii) shifting policy to
encourage students to seek technical and other non-academic degrees, through shorter programs
with closer connections to employment prospects.
10. Despite tremendous recent progress, concerns for equity remain a key issue. Social
expectations regarding government provisions of tertiary education have risen, tolerance for debt
has dropped, and policy mechanisms are challenged to respond.
11. Finally, the sector requires stronger institutional and regulatory capacity, especially in the
Ministry of Education. The size and complexity of Chilean tertiary education has grown
considerably in the last twenty-five years, creating the need for corresponding changes in the
institutional and regulatory agencies.
12. The Government is pursuing a four-pronged strategy to address the challenges in the
sector: (i) increasing resources for the sector; (ii) maximizing the use of performance-based
funding; (iii) improving institutional and regulatory capacity; and (iv) making student finance
more concessionary to address equity concerns.
13. The Government has recognized the need to invest more heavily in tertiary education, as
part of a plan to improve resources for all levels of education. The proposed budget for tertiary
education for 2012 represents a 15% increase over the previous year. The upward trajectory of
real resources for the sector is expected to continue.
14. The Government‘s strategy for the education sector has sought to maximize the use of
performance-based funding mechanisms to channel additional resources to it. By making
institutions compete for incremental resources, and by linking funding to agreed targets in
priority areas, the Government intends to leverage improvements in the quality and relevance of
education, and in the management of educational institutions.
15. The Government has also recognized the need for improved regulatory capacity. It is
seeking to create an Under-Secretariat for Tertiary Education to allow a higher profile for policy
and more resources to be dedicated to policy formulation and implementation. Within the
Under-Secretariat, it has sought to formalize the creation of the Department of Institutional
Financing (Departamento de Financiación Institucional, DFI), with the explicit mandate of
rationalizing and optimizing direct public funding of TEIs. Finally, the Government seeks to
create a Superintendence for Tertiary Education as an agency that can implement policies to
increase transparency and accountability.
16. As part of the strategy to make continued progress on equity, the Government created the
State Guaranteed Student Loan Program (Crédito con Aval del Estado, CAE) in 2005—300,000
qualified students from lower income families have benefitted to date—expanded existing
scholarship (grant) programs, and created two new ones to serve low-income students. Currently
the Government is seeking to lower the real interest rate on CAE loans from 5.6% to 2%. It has
also sought to change the legal sanctions facing 110,000 borrowers from the University Credit
4
Solidarity Fund (Fondo Solidario de Crédito Universitario, FSCU) who are in default. Such
changes remove punitive sanctions on current and future borrowers in default.
17. Over the past few years, the Government has also sought the World Bank for advice and
resources. The World Bank has supported Chile‘s tertiary education sector since 1997 through
the 1998 – 2005 Higher Education Improvement Project – MECESUP1 (Ln.4404, P055481) and
the 2005-2010 Tertiary Education Finance for Results Project – MECESUP2 (Ln.7317,
P088498). Through these projects, quality improvement was competitively funded, performance
agreements were piloted, and the accreditation system and the tertiary education information
system were created, among others. On the knowledge front, in 2009 the World Bank and the
OECD published a review of the tertiary system, ―Reviews of National Policies for Education:
Tertiary Education in Chile‖. The publication was widely disseminated and discussed in Chile.
In 2010, a subset of the World Bank/OECD team carried out a review of the Becas Chile
Program, ―Human Capital Formation Abroad: A Review of Chile‘s Higher Education
Scholarships Programme‖. In 2011, the World Bank analyzed the effectiveness of the CAE
Program, resulting in ―Chile‘s State-Guaranteed Student Loan Program: Analysis and
Recommendations‖. Throughout this period, the World Bank drew on its deep, evidence-based
tertiary education work in Latin America and beyond, and on the combined knowledge of the
Tertiary Education Thematic Group (Coreheg).
18. The rationale for Bank involvement in Chilean tertiary education stems from the lengthy
and substantive partnership that Chile has forged with the Bank in tertiary education since 1997.
The duration and intensity of this partnership is a testament to the fit between the Bank global
knowledge and skills and Chile‘s needs. It has involved fourteen years of continuous co-
financing and technical assistance on the gamut of policy issues. Along with the OECD, the
Bank is one of the non-Chilean institutions with the deepest knowledge of Chilean tertiary
education. The Bank has also been a conduit for Chile to profit from international best practice
in tertiary education and to share its innovative reforms globally. The Bank is active in every
region of the developing world promoting increased quality, relevance, and access to tertiary
education; its networks and knowledge products create value for clients seeking innovative
reform.
C. Higher Level Objectives to which the Project Contributes
19. By helping the Government implement its plan for increasing quality and relevance at
participating TEIs, the Project would continue to develop Chile‘s human capital. This, in turn,
would contribute to the higher level objectives of creating more and better socially and
economically useful skills for the country and its people.
20. The current World Bank Group's Country Partnership Strategy (CPS), for the Period FY
11-FY16, Report 57989, discussed by the Executive Directors on January 11, 2011, aims to
support the GoC‘s vision of becoming a high-income country by 2018. A key factor in
achieving this vision is improving the quality of education. The CPS focuses its intervention in
three areas: (i) Public Sector Modernization, (ii) Job Creation and Equity Improvement, and (iii)
Sustainable Investment Promotion. This Project advances the second of these by scaling up the
use of performance-agreements that improve the quality of tertiary education and focus it on
priority areas relevant to increased competitiveness and employment.
5
2. PROJECT DEVELOPMENT OBJECTIVES
A. PDO
21. The objective of the Project is to improve quality and relevance for students in tertiary
education by strengthening the link between funding of tertiary education institutions and
accountability for performance.
B. Project Beneficiaries
22. The primary target groups are:
(a) Current and future tertiary education students. Students will benefit from increased
quality of education, more remedial classes and services, shorter degree programs,
more coherent curricula, more online course material, new and better PhD programs
and teachers, and more exchange opportunities with other universities both
domestically and abroad.
(b) Academics and staff. Academic staff will benefit from more investments in their
human capital, an increased ability to conduct research and development with better
equipment, and more exchange opportunities with academics in other universities
both domestically and abroad.
C. PDO Level Results Indicators
23. The proposed MECESUP3 project will seek to: (i) improve faculty qualifications; (ii)
shorten actual time to graduation; (iii) increase retention of first-year students; and (iv) raise
employer perception of quality and relevance of graduates of MECESUP3-supported TEIs, as a
proxy for increased employability and wage premiums. This will be done by increasing the
amount of public funding executed under performance-based agreements.
3. PROJECT DESCRIPTION
A. Project Components
24. The Project would achieve its development objective through the implementation of the
million). Provision of support to TEIs to strengthen the link between funding and accountability
for performance, through the carrying out of Performance Subprojects.
26. Component 2. Policy Support and Project Management (Total estimated cost: US$5
millions; Bank: US$2 millions).
6
a. Provision of policy support and studies needed for the design and implementation of the
tertiary education reforms and potential institutional arrangements.
b. Provision of support to DIVESUP in the administration, monitoring, coordination and
supervision of the Project.
B. Project Financing
Lending Instrument
27. The lending instrument used would be a Specific Investment Loan (SIL) at the
Government‘s request.3 This is appropriate given the flexible nature of the instrument and the
specificity of the investment.
Project Cost and Financing
28. The total cost of the Project is estimated at US$160 million, US$120 million of which
constitutes the GoC‘s contribution.
Project Components Project cost
(US$M)
IBRD Financing
(US$M) % Financing
1. Performance Agreements
2. Policy Support and Project
Management
155
5
38
2
24.52%
40%
Total Project Costs
Interest During Implementation
Front-End Fees
Total Financing Required
160
40
25%
C. Lessons Learned and Reflected in the Project Design
29. The previous MECESUP1 and MECESUP2 provided valuable learning opportunities
regarding tertiary education reforms, leaving lessons that have proved useful in the design of
MECESUP3. The primary lesson from MECESUP2 is that performance agreements are
successful in changing institutional culture. These agreements were public commitments of
individual institutions to seek improvements in exchange for a Government commitment to fund
them. The experience of the MECESUP2 pilot CDs is that the process of taking such a public
decision, signing a contract, and committing an institution to specific changes is a powerful
catalyst to change. Some observers believed that the public commitment trumped the financial
resource as the true engine of change in the MECESUP2 pilot CDs. Once institutions have
publicly committed to a program of reforms, they are motivated to deliver on the promises.
3 The previous MECESUP2 Project constituted the first phase of an Adaptable Program Loan. However, as made
clear by the Implementation Completion and Results Report for that Project (Report No ICR00001840, June 24,
2011, page 10), the Government requested a change to a more adequate lending instrument.
7
30. Competent and stable leadership and team are a key component of success. The General
Coordination of MECESUP already had institutional experience with the MECESUP1 project,
which facilitated implementation of MECESUP2. More importantly, however, the team was
composed of competent and well-prepared professionals committed to Project goals. Little
turnover allowed for continuity and good leadership facilitated Project coordination, execution
and credibility. This same team is in charge of implementing MECESUP3.
31. Realistic timelines for implementation require taking into account the Borrower‘s cycle
for budgetary approval in Congress and the internal approval processes for the various legal
documents needed throughout the life of the Project. In MECESUP2, resources awarded in one
fiscal year could only be disbursed the following year, initially triggering some implementation
delays. The time required for internal approval processes is highly variable, and adequate
flexibility has been included in MECESUP3 project timelines.
32. Execution of performance-based funding requires experienced staff to mediate between
the TEIs implementing the agreements and the Ministry of Education. As implemented, the
performance agreements depended heavily on negotiators hired specifically to mediate between
the TEI Presidents and the Ministry of Education. Without suitable negotiators, national policy
goals would have been difficult to reconcile with the reality of the institutional environments
where they were meant to be implemented. The highly qualified negotiators helped the
agreements gain traction. Generally these were former Presidents or academic Vice-Presidents of
universities themselves, familiar with and well regarded in the academic community. Their past
experience gave them credibility with both parties to the negotiations, and allowed the
negotiations of performance metrics to be productive and grounded in the reality of the situation
of each institution. Further, these negotiators were instrumental in developing specific goals for
each university and the detailed indicators used to measure progress in achieving them. These
negotiators would be an integral component of MECESUP3 CDs.
4. IMPLEMENTATION
A. Institutional and Implementation Arrangements
33. The agency responsible for the implementation of both Components 1 and 2 would be
DIVESUP. The group of people within DIVESUP in charge of implementation, referred to as the
General Coordination of MECESUP, have 13 years of experience working with the World Bank
in the previous MECESUP1 and 2 projects. Stable leadership continues at its helm and
continuity characterizes its team.
34. The General Coordination of MECESUP has four technical directorates (one for each CD
type) and three support units (Administration, Analysis and CD support, and Communications)
to perform its tasks. Each of these has clearly defined tasks in their respective area of
responsibility.
35. Competitive rounds would be held to select those TEIs with which the Ministry of
Education signs performance agreements. All accredited TEIs may compete for funds by
submitting project proposals with clearly defined and measurable objectives, strategies, action
plans, milestones, goals, and performance indicators.
8
36. Proposals would be evaluated by Evaluation Committees, one for each CD type, which in
each case would be led by the technical directorates of the General Coordination of MECESUP
and would include at least five prestigious professionals selected by the Head of DIVESUP. The
Evaluation Committees, on the basis of a closed list of evaluation criteria, would submit their
recommendations for selection to the Selection Committee.
37. The Selection Committee would be responsible for selecting, on the basis of the
recommendations from the Evaluation Committees, those proposals that would be invited to
negotiate and sign a CD. The Selection Committee would be led by the Head of Division of
Higher Education, and would be composed of one professional selected by the Subsecretary of
Education, the General Coordinator of MECESUP, and at least three prestigious professionals
selected by the Head of DIVESUP. Formally, the selection is made by the Minister of Education,
upon recommendation of the Selection Committee. Selected TEIs are invited to fully develop
their proposals, negotiating directly with the Ministry of Education to establish the contractual,
budgetary and technical terms of their agreements.
38. TEIs would carry out the activities as per the CDs and would be responsible for
conducting procurement and financial management in accordance with the stipulations of the
Operational Manual and the requirements of the General Coordination of MECESUP (and of the
Bank). MECESUP2 sponsored the creation of Institutional Coordination Units (ICU) in TEIs
receiving project funds. ICUs had an upstream role of helping TEIs prepare funding proposals
and a downstream role focused on implementation, fiduciary management and monitoring and
evaluation. ICUs would continue in MECESUP3, with additional ICUs sponsored in TEIs that
lack one. There would be significant capacity-building efforts on behalf of the MECESUP team
to strengthen and upgrade CD management.
B. Results Monitoring and Evaluation
39. The majority of the selected MECESUP3 PDO level and intermediate results indicators
(partially stated above and in Annex 1 respectively) are currently tracked by Chile‘s Tertiary
Education Information System (Sistema de Información para la Educación Superior, SIES). The
MECESUP2 project financed the establishment of SIES, which collects, analyzes, tracks and
disseminates a variety of information and statistics related to tertiary education and labor market
outcomes for graduates by institution. These include faculty qualifications, duration of study
across degree programs, retention rates, overall enrolment by institution and degree program,
employment rates and expected salary levels at graduation and over time. TEIs report data
directly to SIES and 99% of institutions are compliant with the system. The SIES statistical
compendium allows for straightforward analysis of the impact of changes in policies on high-
level metrics, hence making monitoring and evaluation for the Project both easy and reliable.
40. At the TEI level, Institutional Coordination Units (ICU) would monitor and evaluate the
progress of performance agreements. In addition to yearly revisions of each performance
agreement to verify progress against goals, midterm reviews of each CD would be conducted
with national and international experts.
9
C. Sustainability
41. The GoC demonstrates a strong commitment to and ownership of the MECESUP
program and its relevant policies. Since the closing of MECESUP2 on December 31, 2010 the
Government has continued to finance the activities of the General Coordination of MECESUP. It
will continue to do so until MECESUP3 is effective. In addition, the Ministry of Education
recently institutionalized MECESUP, absorbing the team, renaming it informally as the
Department of Institutional Financing (DFI), and giving it responsibilities for all public funding
of TEIs. This type of arrangement shows the continuity and permanence of MECESUP activities
for the GoC.
42. The GoC has also addressed factors that are critical to the sustainability of the Project.
(a) Upgrading human capital: DIVESUP continues to upgrade the quality of its human
capital and improve its internal efficiency. Several new PhDs have been brought on
board, including two within the General Coordination of MECESUP, and one in
charge of managing the newly created Center for Higher Education Studies. This
center will further improve internal efficiency by providing opportune evidence and
information for policy decisions.
(b) Increasing the reach of accountability for results: Beyond MECESUP3 funds, the
Government is exploring ways to change the way base funding is allocated to public
universities. The President and Minister of Education have discussed a proposal to
move towards a performance-based system for an additional percentage of base
funding.
(c) Increasing access and equity: The GoC has enhanced access and equity in tertiary
education through student scholarships and loans. CAE has grown exponentially
and now has over 300,000 borrowers (more than a fourth of Chile‘s undergraduate
students), and over 60% of them come from the lowest two income quintiles.
5. KEY RISKS AND MITIGATION MEASURES
A. Risk Ratings Summary
Stakeholder Risk Substantial
Implementing Agency Risk Low
- Capacity Low
- Governance Low
Project Risk Low
- Design Low
- Social and Environmental Moderate
- Program and Donor N/A
- Delivery Monitoring and Sustainability Low
10
Overall Implementation Risk Moderate
B. Overall Risk Rating Summary
43. The overall risk rating of the Project is considered to be moderate, largely driven by the
substantial stakeholder risk. The political context for tertiary education has turned volatile since
student protests in June 2011 resulted in major social and political conflict. The protesters have
won notable concessions from the Government, most which have constituted an acceleration or
intensification of planned policy changes. However, students have articulated demands that seek
fundamental change. The initial call for greater quality and equity has been augmented by calls
for ―a new model‖ where tertiary education is publicly provided (―free‖ for students), of high
quality, and where de facto and de jure profits are prohibited. Nevertheless, to date student
leaders have not rejected the idea that increased investment would be accompanied by greater
accountability for results. In fact, greater investments in quality are a key demand of protesters,
especially in public universities. In this sense, the MECESUP Program is aligned with most
stakeholder demands. A small likelihood exists that either: (i) demands could reverse this
position and reject accountability measures; or (ii) strikes could continue indefinitely, preventing
TEI administrators from carrying out normal Institutional Improvement Plans. For both, the
likelihood is low but the impact would be high. The risks associated with the protests are being
mitigated mainly through Government negotiations with the main Chilean stakeholders, and also
through continuous Bank policy dialogue and demand-driven knowledge services.
6. APPRAISAL SUMMARY
A. Economic and Financial Analyses
44. The economic and financial rationale behind the project is sound. On the one hand,
analyses show there is high return on the investment, and on the other, the Project is not expected
to have any short-term macroeconomic consequences.
45. The expected economic value of the project has been assessed through three distinct net
present value (NPV) calculations, each focusing on an expected outcome associated with
MECESUP3. The economic analysis shows that the Project would have to realize only moderate
progress toward its goals to generate a positive return. If the project funds of US$160M are
disbursed evenly across four years, the NPV threshold for a positive return would be ~US$153M
in 2011 currency. Below, the progress required to generate a positive return is detailed:
(a) A shortening of time to graduation would result in time savings and increased
career earnings for graduates. A reduction of approximately one third of a month in
average time to graduation would result in an NPV of ~US$150M (2011).
(b) An increase in the quality of tertiary education is expected to lead to an increased
premium for wage earners with tertiary education. An increase in the premium of
approximately 0.14% would result in a project NPV of ~US$150 M (2011).
11
(c) An increase in first-year retention would be expected to result in a concomitant
uptick in overall graduation rates. An increase of first year student retention of 4%
would result in an NPV of US$~150M (2011).
46. The progress outlined above details the change within each area that would be required to
independently justify the project. In reality, progress towards all goals is expected, and thus only
moderate progress toward each of the goals would likely lead to full recovery of the NPV of the
disbursed funds, along with a sizeable return.
47. With regards to financial impact, the Project poses no risk to Chile‘s macroeconomic and
fiscal stability. In 2010 Chile‘s GDP was US$256 billion and its foreign direct investment (FDI)
was US$15 billion. This US$160 million project, disbursed over the course of four years,
represents 0.016% of GDP and 0.27% of FDI per year. This suggests that inflationary and
monetary pressures created by the project will be marginal, if existent at all. Moreover, there is
no doubt the Government of Chile will be able to provide counterpart funding and fulfill its debt
servicing responsibilities given its strong copper revenues and its low levels of indebtedness
(8.8% debt to GDP in 2010).
B. Technical
48. The rationale for the selected technical approach is based on both local and international
evidence of its effectiveness. Locally, evidence is based on the pilot performance agreements
conducted under MECESUP2. Internationally, evidence is based on the increasing move towards
results-based financing in the tertiary education sector.
49. The history of the MECESUP1 and MECESUP2 projects is key for understanding the
design of MECESUP3. In 1997, MECESUP was conceived as a project that would competitively
fund innovative subprojects to improve the quality of tertiary education. Mostly, academics
submitted applications directly. TEIs did minimal vetting and prioritizing, mostly because they
lacked the tradition and capacity to do so. In 2005, MECESUP helped promote the creation of an
accreditation and quality assurance system that obliged institutions to analyze themselves and to
designate priorities for improvement. At the same time, the MECESUP2 project sought to
transfer greater responsibility to the institutions so they could determine where and how to
improve. This was done by piloting the use of performance-based agreements (CDs) in four
institutions. MECESUP3 seeks to continue that transfer of responsibility by making CDs a
permanent feature of the Chilean tertiary education funding system. This is important because
performance agreements are the most effective means for incentivizing the difficult micro-level
changes in classroom practice and institutional management that raise quality. Most of the
challenges still present in Chile‘s tertiary education can be mitigated or eliminated through the
constituent activities of the CDs. TEIs in Chile will know they are expected to:
(a) Evaluate themselves routinely and create institutional improvement plans, based
substantially on the feedback from their individual accreditation processes.
(b) Compete for resources to implement positive change, to the extent their goals are
consistent with the country‘s overall policy goals (improved teaching and learning,
12
lower dropout, shorter time to degree, improved human capital and research
capacity, among others).
(c) Account for progress and receive funding upon reaching the agreed goals.
50. Internationally, a recent review of tertiary education systems in 24 OECD countries
highlights trends in the evolution of mechanisms to fund educational institutions. One of the
more significant trends is a marked shift to allocation mechanisms that are more performance-
based. Performance agreements, performance set-asides, competitive funds and payments for
results are all performance-based methods of funding.4
51. To sum up, competitive funding of grants and performance agreements has been found to
be the best way to change tertiary education institutions. Tertiary education must serve a great
diversity of aims. Most modern labor markets have thousands of different careers and highly
specialized human capital needs. Centralized control of tertiary education has a terrible record of
meeting these or other social needs. By contrast, systems wherein tertiary education institutions
seek their own niche and develop specific comparative advantages tend to function best.
Competitive funding for quality improvement allows each TEI to propose a program of
improvement. At the same time, it reserves for the government the prerogative to choose the
portfolio of proposed investment that most closely reflects policy goals. This mechanism—
especially when combined with consequential accreditation processes—has been recognized as
the most effective incentive regime for tertiary education improvement.
C. Financial Management
52. Project financial management will seek to be simple, straight-forward, usable, and easy to
understand for TEIs, DIVESUP, and the World Bank. It will seek to include a reporting system
for beneficiaries and a system for classifying expenditure categories, distinguishing between
those that are procurable and those that are not. It will provide an appropriate degree of
granularity for management to review and make decisions. Past experience has informed the
design of Financial Management (FM) procedures, such that things that worked well have been
built upon and solutions for those that could be improved have been sought. The Operational
Manual will further elaborate on these.
53. As part of project preparation, a Financial Management capacity Assessment to evaluate
adequacy of financial management arrangements under the Project was performed from May 19,
2011 to November 30, 2011. In accordance with the implementation arrangements defined, the
project will be implemented by the Division of Higher Education (DIVESUP) of the Ministry of
Education (MINEDUC). Within DIVESUP, financial management function would be undertaken
by the General Coordination of MECESUP5, with the support of the Ministry‘s General
Administrative Unit (DAG), the Planning and Budgeting Unit (DIPLAP) and selected Chilean
Tertiary Education Institutions (TEIs).
4 Santiago, Tremblay, basru and Arnal (2008) Tertiary Education for the Knowledge Society, Volume 1. OECD.
Available on: http://oecd-conference-teks.iscte.pt/downloads/OECD_vol1.pdf 5 Former UCP under Mecesup1 & 2. The GoC has sought to formalize the General Coordination of Mecesup as the
Department of Institutional Financing. The latter would seek to optimize direct public funding of TEIs.
CHILE: Tertiary Education Finance for Results Project
Results Framework
Project Development Objective (PDO): To improve quality and relevance for students in tertiary education by strengthening the link between funding of tertiary education institutions and
accountability for performance.
PDO Level Results Indicators*
Co
re
Unit of
Measure Baseline
Cumulative Target Values** Frequency
Data Source/
Methodology
Responsibility
for Data
Collection
Description
(indicator
definition etc.) 2012 2013 2014 2015
Indicator One: Number of full-
time-equivalent faculty members
who hold PhDs.
# 4,148 4,300 4,500 4,700 4,900
Annual SIES:
Academics.
Baseline data
from May
2011
SIES PhD professors
hired for ≥39
hours a week at
a TEI.
Indicator Two: Average actual
duration of undergraduate studies
until the degree is awarded.
# of
semesters Univ.
12.9
IPs
9.3
CFTs
7.2
Annual SIES:
Graduates.
Current data
is from 2007.
SIES The formula is
currently being
updated.
Indicator Three: Retention rate:
first-year undergraduate students
that remain at the institution in the
second year.
% 69.5% 70% 72% 74% 76%
Annual SIES:
Enrolment.
Baseline data
from 2010
SIES First-year
students in year
t who continue
studying in year
t+1, divided by
first year
students in year
t, multiplied by
100. (Formula
is being
updated.)
Indicator Four: Employers‘
perception of quality and
relevance of graduates of
MECESUP 3-supported TEIs, as
a proxy for increased
employability and wage
premiums.
Semi-
Annual
Employers
surveys
DESUP Consultancy
services will be
hired to develop
employer
surveys.
The baseline
will consider the
16
likely
distribution of
disciplines and
degree programs
in MECESUP3
and will
anticipate best
ways to survey
likely employers
of graduates
who have
attended TEIs
and programs
benefitting from
CDs. Futuro
Laboral will
help identify
representative
samples of
employers.
INTERMEDIATE RESULTS
Intermediate Result (Component One):
Intermediate Result indicator
One: Number of signed
performance agreements.
# N/A Teacher
Training:
6
Academic
Innovation:
7
Technical
& Profess.:
5
Small
Projects:
14
Teacher
Training:
12
Academic
Innovation:
12
Technical
& Profess.:
10
Small
Projects:
24
Annual DESUP
registers
DESUP
Intermediate Result indicator
Two: Rate of fulfillment of
indicators included in the CDs
% N/A Annual TEI DESUP Average
percentage of
fulfillment of
indicators (sum
of percentage of
17
fulfillment of
CD indicators,
divided by
number of
indicators.) Each
indicator may
vary from 0 to
100%.
Intermediate Result indicator
Three: Direct Project beneficiaries
(number), of which women
(percentage).
# and % Students
Acade-
mics
Students
Acade-
mics
Students
Acade-
mics
Students
Acade-
mics
Students
Acade-
mics
Annual Number of
students and
academics at
TEIs that hold at
least one CD.
Intermediate Result indicator
Four: Number of students
completing remediation classes.
# Annual TEIs DESUP
Intermediate Result indicator
Five: Number of full-time-
equivalent faculty members who
hold PhDs in pedagogy schools.
# Annual SIES:
Academics.
DEMRE PhD professors
at pedagogy
schools hired for
≥39 hours a
week at a TEI.
Intermediate Result indicator Six:
Number of graduates from
domestic PhD programs
# 433 460 480 500 530 Annual SIES:
Graduates,
data from
2010
SIES Domestic PhDs
graduates.
Intermediate Result indicator
Seven: Number of new technical
programs.
# Annual SIES:
Degree
programs
SIES New ―Técnico
de Nivel
Superior‖
programs
Intermediate Result (Component Two):
Intermediate Result indicator
One: Number of policy notes and
studies carried out.
# 0 1 2 3 Annual. Number of
policy notes
and studies.
DESUP Policy notes and
studies delivered
with Project
support.
18
Annex 2: Detailed Project Description
CHILE: Tertiary Education Finance for Results Project
Total: 165,081,057 500,245,628 1,500,736,883 Note: considers graduates from Consejo de Rectores universities, assumes new graduates growing at 5.2% (average 2002 to
2009), initial mean wage in 2009 of M$1,132, grown at 2.5% annually, employment rate of 81.8%, US$ 1.0 = CLP550.
47
Increased earnings from Higher Tertiary Wage Premium
6. A significant focus area of MECESUP3 will be increasing the quality of tertiary
education by improving the faculty qualifications. Additionally, the project anticipates raising
the perception of quality among future employers by increasing relevance and quality of degree
programs. As the quality and relevance of tertiary education increases, the wage premium for
having completed tertiary education is expected to rise, with employers willing to pay increasing
premiums for graduates with well developed and highly relevant skills. The NPV of the project
was calculated in part by examining the value of these incremental earnings.
7. If the wage premium paid to tertiary education graduates increases faster than the
historical average, a portion of that premium can be attributed, in part, to the success of this
project. A number of assumptions were made to assess the potential impact of such a premium
on the earnings of university graduates over the decade from 2013 to 2022. Because the
graduates are expected to benefit from the wage premium over their entire career [30 years
assumed], the assumptions calculate the present value of the wage premium in the year of
graduation; future year premiums are discounted. The rest of the assumptions are detailed below.
8. The same assumptions used for calculating increased wages from shortened study-spans
are used here, as well as a few supplementary assumptions:
In addition to the 2.5% annual growth rate applied to first year wages, through the 30
year employment span, a 1.5% annual lifecycle earnings increase was projected.
As in the PAD, a discount rate of 6% is applied. This rate is used to normalize present
value earnings from 2013-2022 back to USD 2011 and to calculate the 30-year present
value of the wage increase for each graduation cohort.
The wage premium attributable to the project was varied from 0.15%-0.45%.
9. Table 2 shows these assumptions can result in returns ranging from an NPV of $170-510
million in increased earnings for 2013-2022 graduates. A variety of factors can influence the
premium associated with higher education, and given the variety of increases in funding to
higher education and the ongoing transformation of the Chilean economy, it is expected that a
larger premium than 0.45% will be observed, of which only a portion will be attributable to the
project.
48
Table 2. Increased Wage Premiums Associated with Tertiary Education Quality
Note: considers graduates from Consejo de Rectores universities, assumes new graduates growing at 5.2% (average 2002 to
2009), initial mean wage of M$1,132, employment rate of 81.8%, US$ 1.0 = CLP550, assumes 2.5% continued initial salary
growth, with incremental 1.5% lifecycle growth.
* = projected
Increased Earnings from Higher Graduation Rates
10. The final source of returns from the project is associated with increasing retention rates of
students. Tertiary education graduates enjoy a significant wage premium over non-graduates,
earning approximately double what drop-outs can be expected to make. Thus increasing the
graduation rate would have the effect of moving more wage-earners into a higher income group.
11. Reliable data to develop a historical baseline for graduation rates is limited. Accordingly,
the metric targeted by the project is a reduction in first year attrition, an area with more reliable
statistics. This analysis was based on a number of assumptions, including:
CASEN survey data to determine the magnitude of the premium for completion of
tertiary education vs. partial exposure to tertiary education. The premium was
determined, based on 2009 CASEN data, to be ~ $85/month.
Ongoing growth in the premium associated with higher education of ~3.3% per year, in
line with growth in the premium from the 2000-2009 CASEN surveys.
Because the NPV analyses are limited to the near-term decade, and because Chilean
university education on average lasts 7 years, only a handful of graduating classes are
expected to be influenced over the course of the next decade. Specifically, the enrolling
classes of 2012-2015 are expected to graduate between 2019-2022. If the analysis
included more cohorts, the NPV would grow correspondingly.
The first year drop-out rate was based on reported statistics; longer term graduation rates
were derived by extrapolating a steady state graduation rate by excluding first year
enrollees and extrapolating the retention rate necessary to generate the observed number