World Bank Documentdocuments.worldbank.org/curated/pt/501561468284129817/pdf/ICR23740ICR0... · Bantuan Operational Pendidikan/Bantuan Operasional Sekolah Daerah (Operational Assistance

Document of The World Bank

Report No: ICR2374

IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-91895)

ON A

GRANT

IN THE AMOUNT OF €25,618,000 (US$40,493,164 MILLION EQUIVALENT)

TO THE

REPUBLIC OF INDONEISIA

FOR A

BASIC EDUCATION CAPACITY PROJECT

June 27, 2013

Human Development Sector Unit/Education Indonesia Country Department East Asia and Pacific

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS (Exchange Rate Effective April 24, 2013)

Currency Unit = Indonesian Rupiah (IDR)

US$1 = IDR 9,717.5 US$1 = €0.75

€1.0 = US$1.34

FISCAL YEAR January 1 – December 31

ACADEMIC YEAR July 1 – June 30

ABBREVIATIONS AND ACRONYMS

AAA Advisory and Analytical Activity APBD Anggaran Pendapatan Belanja Daerah (Annual District Revenue Plan) APEID Asian Programme of Educational Innovation for Development AusAID Australian Agency for International Development BAPPENAS Badan Perencanaan Pembangunan Nasional (National Development Planning

Agency) Balitbang Badan Penelitian dan Pengembangan (Research and Development Unit) BE-TF Bank Executed-Trust Fund BEC-TF Basic Education Capacity Trust Fund BOP/BOSDA Bantuan Operational Pendidikan/Bantuan Operasional Sekolah Daerah

(Operational Assistance for Education/School Operational Assistance at District or Province level)

BOS Bantuan Operasional Sekolah (School Operational Assistance) BPKP Badan Pengawasan Keuangan dan Pembangunan (Financial and Development

Supervisory Board/Internal Audit Agency) CAS Country Assistance Strategy CDP Capacity Development Plan CPIU Central Project Implementation Unit DAK Dana Alokasi Khusus (Special Allocation Fund) DEO District Education Office DESP Dutch Education Support Program DFO District Financial Office DIPA Daftar Isian Pelaksanaan Anggaran (Approved Budget Document) DPIU District Program Implementation Unit DP2KAD Dinas Pendapatan dan Pengeloaan Kas dan Asset Daerah (Revenue and

Treasury Management of Regional Asset) ECD Early Childhood Development EFA Education for All EPEA Education Public Expenditure Analysis EMIS Education Management Information System ESAs Education Sector Assessments

FM Financial Management GoI Government of Indonesia IDR Indonesian Rupiah IG Inspectorate General INAP The Indonesia National Assessment Program INT Institutional Integrity Unit KPI Key Perfomance Indicator L-BEC Local Government – Basic Education Capacity LG Local Government LGCA Local Governance Capacity Assessment LG-EPEA Local Government-Education Public Expenditure Analysis MDG Millennium Development Goal M&E Monitoring and Evaluation MoEC Ministry of Education and Culture (previously MoNE) MoF Ministry of Finance MoHA Ministry of Home Affairs MoNE Ministry of National Education (now MoEC) MoRA Ministry of Religious Affairs MTEF Medium-Term Expenditure Frame work PAKEM Pembelajaran Aktif Kreatif Efektif dan Menyenangkan (Active, Creative, Joyful,

and Effective Learning) PDO Project Development Objective PEACH Public Expenditure Analysis Capacity Enhancement PIU Project Implementation Unit POM Project Operation Manual PMD Pusat Pemberdayaan Masyarakat (Community Empowerment Center) PP Peraturan Pemerintah (Government Regulation) PPIU District Program Implementation Unit PSC Program Steering Commitee PER Public Expenditure Review PBB Performance Based Budgeting PUSPENDIK Pusat Penilaian Pendidikan (Education Assessment Center) RE-TF Recipient Executed-Trust Fund RENSTRA Rencana Strategis (Five Year Strategic Plan) SOP Standard Operating Procedure SISWA System Improvements through Sector-Wide Approaches SUSENAS Survey Sosial Ekonomi Nasional (National Socioeconomic Survey) SuMS Support My School SWAp Sector Wide Approach SP Service Provider TA Technical Assistance TED Thematic Education Dialog TOR Terms of Reference TP2NK Tim Nasional Percepatan Penanggulangan Kemiskinan (The National Team for

the Acceleration of Poverty Reduction)

TRIMS Tools for Reporting Information and Management System UNESCO United Nations Educational Scientific And Cultural Organization UNICEF United Nations Children’s Fund USAID United States Agency for International Development WAPIK Wahana Aplikasi Praktek Pendidikan yang Baik (Application Mode of Good

Practice in Education)

Vice President: Country Director:

Sector Manager: Project Team Leader:

ICR Team Leader:

Axel van Trotsenburg Stefan G. Koeberle Luis Benveniste Ratna Kesuma Ratna Kesuma

INDONESIA

Basic Education Capacity Project

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 6 3. Assessment of Outcomes .......................................................................................... 12 4. Assessment of Risk to Development Outcome ........................................................ 19 5. Assessment of Bank and Borrower Performance ..................................................... 19 6. Lessons Learned ....................................................................................................... 21 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 22 Annex 1. Project Costs and Financing .......................................................................... 23 Annex 2. Outputs by Component ................................................................................. 24 Annex 3. Economic and Financial Analysis ................................................................. 41 Annex 4. Grant Preparation and Implementation Support/Supervision Processes ....... 47 Annex 5. Beneficiary Survey Results ........................................................................... 49 Annex 6. Stakeholder Workshop Report and Results .................................................. 50 Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR ....................... 51 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 64 Annex 9. List of Supporting Documents ...................................................................... 65 MAP .............................................................................................................................. 66

A. Basic Information

Country: Indonesia Project Name: Basic Education Capacity-Recipient executed

Project ID: P115724 L/C/TF Number(s): TF-91895 ICR Date: 06/27/2013 ICR Type: Core ICR

Lending Instrument: SIL Grantee: Government of Indonesia

Original Total Commitment:

USD 40.49M Disbursed Amount: USD 17.78M

Revised Amount: USD 20.06M Environmental Category: C Implementing Agencies: Ministry of Education and Culture, Directorate General for Management of Basic Education Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: Effectiveness: 06/05/2008 07/10/2008 Appraisal: Restructuring(s): 10/21/2011 Approval: 06/06/2007 Mid-term Review: 05/23/2011 Closing: 04/30/2012 12/31/2012 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Grantee Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory Government: Moderately

Unsatisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies: Moderately Satisfactory

Overall Bank Performance: Moderately Satisfactory Overall Borrower

Performance: Moderately Satisfactory

ii

C.3 Quality at Entry and Implementation Performance Indicators Implementation

Performance Indicators QAG Assessments (if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Unsatisfactory

D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) General education sector 100 100

Theme Code (as % of total Bank financing) Education for all 100 100 E. Bank Staff

Positions At ICR At Approval Vice President: Axel van Trotsenburg James W. Adams Country Director: Stefan G. Koeberle Christopher J. Hoban Sector Manager: Luis Benveniste Eduardo Velez Bustillo Project Team Leader: Ratna Kesuma Mae Chu Chang ICR Team Leader: Ratna Kesuma ICR Primary Author: Halsey L. Beemer Bernardo da Cruz Vasconcellos F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) To support GOI achievement of MDG and EFA goals through good governance in education, improving the delivery of decentralized basic education services by local governments and schools in selected program areas, and by extension in other locations. Revised Project Development Objectives (as approved by original approving authority) Not applicable

iii

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: Increased Primary Net Enrollment Rate (disaggregated by gender and income quintile)

Value (Quantitative or Qualitative)

Total Primary 91.13% Primary Male 91.40% Primary Female 90.85% Primary Poorest Quintile 90.93%



Date achieved 12/31/2007 04/30/2012 12/31/2012 Comments (incl. % achievement)

Primary school net enrollment rates have increased from 2007 to 2012, with total primary and primary females exceeding targets, and primary male and primary poorest quintile falling short of the end-of-project target.

Indicator 2: Increased Primary Gross Enrollment Rate


Total Primary 99.00% Primary Male 99.53% Primary Female 98.43%




Primary school gross enrollment rates have increased from 2007 to 2012 with primary females exceeding target and total primary and primary male falling short of the end-of-project target.

Indicator 3: Increased Junior Secondary Net Enrollment Rate


Total Jr. Sec. 66.47% Jr. Sec. Male 64.37% Jr. Sec. Female 68.68% Jr. Sec. Poorest Quintile 53.03%



Date achieved 12/31/2007 04/30/2012 12/31/2012

Comments (incl. % achievement)

Junior secondary net enrollment rates have increased from 2007 to 2012, especially among the poorest quintile. Total Jr. Sec. and Jr. Sec. female exceeding targets and Jr. Sec. male and poorest quintile falling short of the end-of-project target.

iv

Indicator 4: Increased Primary Completion Rates Value (Quantitative or Qualitative)

96.86% 97.76% 97.06%


Primary completion rates increased during the project but fell short of the end-of-project target.

Indicator 5: Increased Junior Secondary Completion Rates Value (Quantitative or Qualitative)

98.17% 98.61% 98.27%


Junior secondary completion rates increased during the project but fell short of the end-of-project target.

Indicator 6: Increased Transition Rates from Primary to Junior Secondary School Value (Quantitative or Qualitative)

75.58% 82.20% 77.26%


Transition Rates increased during the implementation of the project but fell short of the end-of-project target (based on 2011 data).

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: Number of Education Policy Analyses and Studies Undertaken Value (Quantitative or Qualitative)

N/A 2 19

Date achieved 12/31/2008 4/30/2012 12/31/2012

Comments (incl. % achievement)

Target was exceeded with the completion of 19 studies related to education financing, the higher education law and school based management that were used to impact policy decisions.

Indicator 2: Number of Districts with at least 60% of schools providing BOS education budget information through school announcement boards


16 50 37


Target was not achieved. However, field reports revealed improvement with districts increasingly circulating education data, plans and reports through media outlets such as local newspapers.

v

Indicator 3: Number of Districts provide education budget through local mass media or official announcement boards or on a website


16 35 39


Target was exceeded. This is impressive considering that many of the 50 districts had no initial established local media outlets available to be able to channel information.

Indicator 4: Number of Districts whose Education Unit has a unit budget document with measureable outcomes related to its strategy


43 50 50


Target was achieved.

Indicator 5: Number of Districts with a difference between planned and realized plans (disbursement rates) of less than 10% in the last three fiscal years


31 40 44


Target exceeded by 4 districts. This is a substantial achievement that was supported through the extensive capacity development training on the budgeting process.

Indicator 6: Number of Districts where all of the awards for education bids are published in the local newspapers or official boards or on a website


12 35 48


Target was exceeded. The large increase in the number of districts that achieved this goal is an indication of the increased transparency that districts sought to instill in their education procurement process.

Indicator 7: Number of Districts where fixed assets are inventoried on an annual basis Value (Quantitative or Qualitative)

23 28 36


Target was exceeded. This impressive because it was a new responsibility that was achieved due to capacity training on asset management.

vi

Indicator 8: Number of Districts where internal audit results are filed and action taken to follow up as per recommendation


N/A 26 40


Target was exceeded which demonstrates increased accountability at the district level, a specific goal of the project.

Indicator 9: Number of Districts where the Bupati/Walikota's Accountability Report (including Dinas Pendidikan) is made available to the public


N/A 18 28


Target was exceeded. The fact that MoHA introduced a regulation requiring districts to conduct community based accountability reviews supported achievement of the indicator.

Indicator 10: Number of Districts where enrollment by gender is published annually and is easily accessed through local mass media or official announcement boards, or a website


26 40 40


Target was achieved. This led to improved transparency of data at the local level and their use for education planning.

Indicator 11: Number of Districts where student achievement reports are published annually and easily accessed through local mass media or official announcement boards or an official website


19 35 40


Target was exceeded. These achievements indicate increased transparency at the local level and improved availability of data for use by education administrators.

vii

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 06/30/2009 Satisfactory Satisfactory 2.90

2 05/17/2010 Moderately Unsatisfactory

Moderately Unsatisfactory 4.62

3 05/23/2011 Moderately Satisfactory Moderately Satisfactory 11.56 4 12/21/2011 Moderately Satisfactory Moderately Satisfactory 13.67 5 11/27/2012 Moderately Satisfactory Moderately Satisfactory 17.68

6 12/25/2012 Moderately Unsatisfactory Moderately Satisfactory 19.04

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made DO IP

10/21/2011 MS MS 13.67

Cancellation of project funds at the request of Dutch government and extension of closing date.

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country Context. At project appraisal, Indonesia was in transition from an autocratic, centralized state to a democratic, decentralized one. There had been a move toward public sector and fiscal decentralization since 1999. Economic growth was below 4 percent, poverty reduction remained a challenge, and governance concerns continued to cloud political achievements. Indonesia continued to trail its neighbors in access to quality health, education, and other basic services. Weak governance institutions, poor infrastructure, and shortages of high-level human capital were keeping investors away and undermining service provision. 2. The Government of Indonesia (GoI) had prepared a comprehensive package of reforms covering macroeconomic management, financial sector reform, and policies to help raise investment and reduce poverty. As part of that effort, government agencies developed their own medium-term strategic plan called RENSTRA (Rencana Strategis). At that time, the Ministry of National Education (MoNE - now Ministry of Education and Culture or MoEC) and the Ministry of Religious Affairs (MoRA) produced a five-year medium-term strategic plan for education, or RENSTRA, that focused mainly on three basic education priorities: (i) universal access to good quality basic education; (ii) improvement in basic education quality; and (iii) more effective basic education sector governance in the context of an overarching decentralization reform.

3. Sector Context. The core of GoI’s strategy for quality basic education was derived from its medium-term development plan, aiming at decentralizing education system governance and management, and improving access and quality. GoI’s strategic medium term goals for the mandatory nine year basic education sub-sector included: (i) increased financing for operational budgets; (ii) expanded access to early childhood education, schools for children with disabilities and inclusive schools, and basic education through non-formal education; (iii) special ‘one roof’ schools to provide basic education service in remote areas and sparsely populated islands; (iv) recruitment of educators and education manpower; and (v) improved internal audit function, planning, budgeting and education management. 4. GoI was also committed to achieving the Millennium Development Goals (MDGs) and the Education for All (EFA) targets, and this commitment was embedded in the MoEC and MoRA’s 2005-2009 RENSTRA. The RENSTRA priorities were to support: (i) universal access by reducing school attendance costs to students’ families through the implementation of the government’s free basic education policy and other related strategies; (ii) basic education quality improvements through better facilities and teachers, the implementation of Minimum Service Standards, the rehabilitation and reconstruction of basic education schools and the implementation of a Teacher Reform Law to regulate teacher certification and corresponding salary increase; and (iii) enhanced governance by strengthening the capacity of districts to use their resources more effectively and efficiently given their new responsibility and power to deliver quality education services, and supporting good planning, budgeting, procurement, financial management and accountability practices to minimize leakages due to misuse of funds.

2

5. At the time of project preparation, the GoI had expressed interest in a sector-wide approach (SWAp) to education sector investment – one that would enhance the ability of GoI to efficiently coordinate and maximize the overall effectiveness of donor assistance in achieving Indonesia’s high priority objectives for the education sector. Previous donor assistance had been fragmented, and piecemeal interventions had not been highly effective in producing large, visible impacts on Indonesia’s education sector. Moreover, there was keen interest in finding new ways of providing international assistance that would support the achievement of sector objectives more effectively and comprehensively, disburse faster and meet fiduciary requirements. Although country systems at the time were not sufficiently strong for a sector wide program, the Netherlands, the European Commission (EC) and the World Bank responded by establishing the Basic Education Capacity Trust Fund (BEC-TF) in support of the GoI’s education reform agenda while preparing the ground for a sector-wide investment program in the future. Additional grant funds were also made available by the Dutch Education Support Program (DESP) to fund discrete activities. 6. Project Context. BEC-TF’s five-year financial commitment from the Dutch and the EC amounted to €39 million, which were divided into two parts: €25.60 million was Government-executed and €13.40 million was administered by the Bank. The BEC-TF was also designed to support the 2005-2009 RENSTRA and complement other bilateral and multilateral basic education support programs. Furthermore, these financial commitments were supposed to help the government develop its own sector wide program (the “Education System Improvement through Sector Wide Approaches – SISWA”), under preparation at that time. Although GoI never finalized the program and eventually abandoned the effort, this did not affect the BEC-TF project design, which continued with its original focus on improving good governance, financial management, budgeting and accountability within 50 districts. The BEC-TF project was also designed to provide substantial technical assistance (TA) and respond to government requests for rapid analyses to support GoI’s national School Operational Assistance Program (Bantuan Operasional Sekolah - BOS). The BOS was established in 2005 to provide basic education schools with block grants, which at the time covered 228,000 schools and an estimated 43 million students1. The BEC-TF was also in line with the 2004 – 2007 Country Assistance Strategy (CAS) themes of improving the investment climate and making service delivery responsive to the poor: the BEC-TF supported capacity development at the local level in the areas of finance and management, and improved governance so that local government could provide improved service delivery and be responsive to the needs of the poor, particularly in the education sector. Because of the integrated nature of the trust fund activities and to provide a holistic picture of the project, the ICR team has chosen to report on the totality of both Bank- and recipient-executed portions. Annex 1 “Project Costs and Financing” focuses solely on the recipient-executed portion of the overall trust fund, while Annex 2 differentiates activities by executing agency. 1 The BOS program is regarded as a key pillar of the Government’s Basic Education Policy as it provides funding for school operations and was thus expected to reduce school fees in general while reducing all school fees for the poor.

3

1.2 Original Project Development Objectives (PDO) and Key Indicators 7. The overall project development objective of the program was to contribute to the GoI's achievement of MDG and EFA goals through good governance in education. The purpose of the program was to support GoI in improving the delivery of decentralized basic education services by local governments and schools in selected program areas, and by extension in other locations2. Given the slight variations of PDO wording in various project documents3, the ICR will use as the project development objective “to contribute to the GoI's achievement of MDG and EFA goals through good governance in education” which is found in the grant appraisal document.

8. Progress towards achieving the PDOs was measured using indicators reflecting the national level goals of achieving the MDG and EFA. These were: (i) increased primary net enrollment rate (disaggregated by gender and income quintile); (ii) increased primary gross enrollment rate; (iii) increased junior secondary net enrollment rate; (iv) increased primary completion rates; (v) increased junior secondary completion rates; and (vi) increased transition rates from primary to junior secondary school. 9. The intermediate indicators measured progress achieved within the 50 targeted districts. These were: (i) number of education policy analyses and studies undertaken; (ii) number of districts with at least 60 percent of schools providing BOS education budget information through school announcement boards; (iii) number of districts provide education budget through local mass media, official announcement boards or on a website; (iv) number of districts whose education unit has a unit budget document with measureable outcomes related to its strategy; (v) number of districts with a difference between planned and realized plans (disbursement rates) of less than 10 percent in the last three fiscal years; (vi) number of districts where all of the awards for education bids are published in the local newspapers, on official boards or a website; (vii) number of districts where fixed assets are inventoried on an annual basis; (viii) number of districts where internal audit results are filed and action taken to follow up as per recommendations; (ix) number of districts where the Bupati/Walikota’s accountability report (including Dinas Pendidikan) is made available to the public4; (x) number of districts where enrollment by gender is published annually and is easily accessed through local mass media, official announcement boards, or a website; and (xi) number of districts where student achievements reports are published annually and easily accessed through local mass media, official announcement boards or a website.

2 Grant Appraisal Document, Annex 1: Logical Framework, pg. 26 3 The trust fund agreements with both the Kingdom of the Netherlands and the European Commission (EC) states the objectives as “to support the GoI’s achievement of MDG and EFA goals through better governance in education.” The Grant Agreement between the Bank and the Government of Indonesia states the objective to be “to contribute to the achievement by the recipient of Millennium Development Goals (MDG) and Education for All (EFA) goals through good governance in Indonesia.” In all ISRs the objective is stated as: “To support the GoI achievement of MDG and EFA goals through good governance in education, improving the delivery of decentralized basic education services by local governments and schools in selected program area, and by extension in other locations.” 4 “Bupati/Walikota” refers to Mayor/District Head, and “Dinas Pendidikan” refers to District Education Office.

4

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 10. The PDO and the key indicators were not revised during the project implementation. 1.4 Main Beneficiaries 11. The main project beneficiaries were: (i) national level education policy makers taking part in the Thematic Education Dialogue (TED); (ii) provincial and district education offices in the 50 districts supported by the project; (iii) school principals, school supervisors, and school committee and community members who would benefit from the school-based management support in selected schools in the 50 districts; and (iv) teachers and other educators who would benefit from the new learning and knowledge-sharing systems. 1.5 Original Components 12. Component 1: Policy Analysis and Dialogue included the following sub-components: Sub-component 1.1: Thematic Education Dialogue (TED), which included a forum for key stakeholders to have a dialogue on education sector issues. The objectives of the TED were: (i) to provide strategic and policy recommendations on education sector reforms and development; (ii) to coordinate cooperation among government institutions, and between GoI and international development partners; (iii) to support preparation for a sector-wide approach and for external financing to be delivered through the GoI system; and (iv) to review and evaluate progress on implementation of policies and sector performance; and Sub-component 1.2: Analytical Studies, which included research to support an overall education sector assessment and analysis to support movement toward a sector-wide approach (SWAp). 13. Component 2: Good Governance, Financial Management, Budgeting and Accountability, which focused on strengthening education sector governance and management capacity, particularly at district levels, and to use resources more effectively, efficiently and equitably. It included the following sub-components: Sub-component 2.1: Good Governance & Financial Management, focusing on financial management and good governance activities to be developed for district level capacity development; Sub-component 2.2: Planning and Budgeting, which focused on the use of performance-based budgeting and the Medium Term Expenditure Framework, and to provide capacity development support for results-based planning, budgeting and simplified reporting; and Sub-component 2.3: School Management, which would develop capacity in the focus areas of the project, including financial management, good governance, budgeting and planning, and information management. 14. Component 3: Information Management: This component responded to the need to strengthen systems of information and to develop capacity of key stakeholders to allow for informed decisions on education policy and allocation of resources. The sub-components were: Sub-component 3.1: Management Information Systems and Information-based Decision Making, focusing not only on implementation of Management Information Systems, but also on developing the capacity to properly manage the data, analyze it and make informed decisions based on the results; and sub-component 3.2: Learning and Knowledge, which was to facilitate

5

knowledge sharing and learning about good practices and making these accessible to all districts and schools. 1.6 Revised Components The components were not revised during the project. 1.7 Other significant changes 15. On February 22, 2008, the Administrative Agreements between the Kingdom of the Netherlands, the EC and the Bank were amended following GoI’s decision to abandon the development of the SISWA and to clarify some funding modalities. The amendment entailed: (i) modifying the administrative agreement wording to refer specifically to the 50 districts supported by the project to improve governance, financial management budgeting and accountability at the local level; (ii) align the Description of Operations to focus specifically on strengthening education governance just in the 50 districts as well as clarify the funding modalities to be used for the local government BEC grants (L-BEC grants); and (iii) reallocating funds in line with the project’s focus on the 50 districts.

16. On October 21, 2011, the project went through a level two restructuring of the BEC-TF Grant Agreement. The restructuring covered the following: (i) a €8.7 million reduction of project funds following the Dutch government’s decision to reduce their development assistance to Indonesia; (ii) the cancellation of pilot activities initially covered by the project but subsequently financed by other sources of funds; (iii) the reallocation of grant proceeds due to exchange rate fluctuations, and (iv) the extension of the project closing date from April 30, 2012 to December 31, 2012. The extension was made to enable districts to complete planned 2012 capacity development plans (CDPs) contributing to the project’s Key Performance Indicators (KPIs) within a more realistic timeframe and make up for initial implementation delays. There were no revisions to the PDOs or outcome indicators due to the fact that the Government of the Netherlands and the EC required EFA PDO level indicators to be included in the results framework. 17. As part the BEC-TF restructuring, the two administrative agreements with the donors had to be amended as well. The agreement between the Netherlands and the Bank was modified to reflect: (i) the reduction in the grant as stated above; (ii) the extension of the closing date to December 31, 2012; and (iii) the extension of the trust fund end disbursement date to June 30, 2013. The agreement between the EC and the Bank was modified to reflect the extension of the closing date to December 31, 2012. 18. In October 2012, the GoI requested the cancellation and transfer of €1.1 million from their unallocated disbursement category back to the parent trust fund following lower-than-anticipated disbursements caused by delays in hiring and contracting service providers. In addition, approximately US$3.2 million remained undisbursed as of May 30, 2013, which was in part due to the Euro exchange rate gains over the life of the project. Nevertheless, all project related activities were completed.

6

2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 19. Project Preparation. The project fit squarely within the goals of the FY04-07 CAS goals of improving the investment climate and making the service delivery system responsive to the needs of the poor. BEC-TF focused on the themes of building capacity at the local government level in the areas of finance and management and improved governance so that the local governments could provide improved service delivery in the education sector. 20. The Bank preparation team worked effectively with MoEC (then MoNE) and MoRA in the development of the project. Extensive time was devoted to tailoring project activities around the RENSTRA 5-year strategic plan with a strong focus on capacity building at the local district and school levels, priorities of both MoEC and MoRA. The Bank team also worked with the GoI preparation team in the development of the LGCA (Local Governance Capacity Assessment), which was to identify the capacity development needs at the district level to be addressed by the project. This task was an important aspect of design: it would allow districts to develop a framework for the drafting of their Capacity Development Plans (CDPs) and include tailor-made activities to address the capacity gaps identified in the LGCA baseline report. This was a key departure from the top-down approach used in Indonesia prior to the BEC-TF and reflective of the overall project design that supported a bottom-up approach to improving education at the local level. 21. Project Design. Since the project was to improve local capacity to deliver education services, the Bank team appropriately considered lessons from earlier Bank-supported capacity development programs such as: (i) districts were capable of building capacity given adequate resources and authority to identify needs and carry out programs, and (ii) training of new staff as well as maintaining staff are essential for successful implementation of a project. The design of the components was appropriate to support the government’s decentralization plan and to strengthen district capacity. Component 1 included support for studies on education policy issues such as education financing (including analyses on adequate funding levels to schools), early childhood education (ECD), which led to new guidelines for the ECD sector, and comprehensive skills upgrading for education administrators that fed into national education plans. It also included national and international study visits to learn lessons from other governments in strategic areas to help increase the relevance and quality of the education system. Components 2 and 3 supported and fostered the government’s push for a decentralized education system with increased focus on governance, financial management, and education service delivery at the district level. The project design also supported the MoEC-managed block-grant mechanism at the provincial level and the devolution of the grants to districts. The PDO was appropriate in that it supported the government’s education strategy to achieve the MDGs and EFA, and the PDO level indicators were appropriate to measure the national MDG goals. While the Bank team recognized the difference between sector-wide PDO level indicators and the more circumscribed nature of project activities in 50 districts, as mentioned previously, the PDO and PDO indicators were included at the specific request of the Dutch and EC to link project contributions explicitly to the larger EFA-MDG goals for Indonesia. Intermediate indicators measured progress of project activities in the 50 districts.

7

22. Quality at Entry. Although no Quality at Entry Review was conducted, project preparation did follow the Bank’s comprehensive quality assurance and review process, and led to the government meeting all project readiness conditions by the effectiveness deadline. The readiness conditions included: (i) establishing the Project Steering Committee (PSC) and the Central Project Implementation Unit (CPIU) inside the MoEC, (ii) completing an Operations Manual acceptable to the Bank; and (iii) developing the MoEC L-BEC Grant Manual that specified grant selection, documentation and distribution criteria. As mentioned previously, the Bank team worked closely with the CPIU in developing the terms of reference for the LGCA to help assess the needs for each district and establish project baselines. This was an important project preparation strength which, together with the readiness conditions in place, led to the successful implementation of the BEC-TF project.

23. Based on the analysis above, overall project preparation, design and quality at entry is rated moderately unsatisfactory primarily due to the difference between national-level PDO indicators and the localized nature of project interventions. That said, the project fit within the goals of the CAS, focused on the themes of building capacity at the local government level and was in line with the RENSTRA. Furthermore, project components were designed to support the government’s decentralization plan and strengthen district capacity, as well as help MoEC manage its block-grant mechanism at the provincial level. Although the PDO and PDO-level indicators were sector-wide in nature, the intermediate indicators were appropriate to measure progress in the 50 target districts. Lastly, preparatory work done to establish the LGCA was instrumental for the project’s readiness and subsequent implementation. 2.2 Implementation 24. Project implementation began with effectiveness in June 2008. The Bank and government provided technical assistance to equip districts with the skills necessary to develop their CDPs which, once approved, became the cornerstones of the project (L-BEC grants could only be disbursed against activities outlined in the approved CDPs). Although district CDPs had been developed, L-BEC disbursements were delayed by 18 months because the Ministry of Finance (MoF) had not yet developed the Standard Operation Procedures (SOPs) for the newly-introduced on-granting mechanism5. The absence of guidelines regarding the new mechanism and clear roles and responsibilities of actors at all levels also created confusion, especially at the district level. As a result, overall project disbursements stood at only approximately 10 percent by June 2009 and it was during this period that the overall project was downgraded to moderately unsatisfactory. 25. Recognizing the negative effect these delays had on project implementation, the CPIU, Bank and PSC proactively adjusted the sequencing of activities and rolling out activities not affected by the new on-granting mechanisms instead. For instance, much of the training for districts was originally planned to take place later in the project but was carried out during the

5 After project effectiveness, MoF introduced a policy regulating regional grant channeling, which affected the way L-BEC funds were to be transferred. Districts were now required to follow the newly established “on-granting mechanisms” rather than receiving funds directly from MoEC. However, guidelines to support the roll out of the new policy were delayed, which slowed down overall project implementation and CDP implementation in particular.

8

first two years of project implementation with MoEC funding. These targeted training and socialization sessions focused on the five strategic pillars6 of education governance and management, and were crucial for maintaining project implementation momentum. Initial delays in providing technical assistance to districts on the CDP process were also overcome by the CPIU cancelling the complex procurement process to contract firms and hiring four individual consultants instead. CDP process assistance together with the other training and socialization activities were completed by the time the new MoF on-granting mechanism procedures were fully articulated and L-BEC grants started to flow. This further helped improve overall project performance. 26. Although there were challenges related to the L-BEC on-granting mechanism, the policy analysis and dialogue aspects financed largely by the Bank-executed portion under Component 1 continued to be implemented. These included workshops and study visits to expose key education stakeholders to international experience on governance, financial management, and efficient resource use, as well as analytical studies to support the GoI’s basic education decentralization policy. In addition, the TED, which was the forum for high level discussions on education thematic dialogue under component 1, enabled senior government officials and development partners to meet and discuss key education issues, and provide strategic recommendations on education reform, resourcing and planning. 27. Prior to the October 2011 project restructuring, the PSC met to discuss overall implementation challenges, the €8.7 million funding reduction by the Dutch government and a possible extension of the closing date to allow districts to complete their CDP activities that had been affected by delayed L-BEC disbursements. The MoEC was initially reluctant to extend the project but eventually agreed in light of the unanimous requests from the districts. It was also agreed by all interested parties not to modify the PDO or results framework because of the time needed to make the modifications and get the required approval. In the end, an agreement was reached between the GoI, donors and the Bank to: (i) extend the project closing date; (ii) return funds to the Dutch government; and (iii) reallocate the remaining resources to ensure that project activities could be fully implemented. By project closing, approximately 15 percent of project funds remained undisbursed yet all project activities had been completed, which is explained by significant Euro exchange rate gains. 28. Based on the analysis above, the project implementation is rated moderately satisfactory. This is mainly due to the substantial progress made once the L-BEC grants were disbursed under component 2 and 3, and the number of analytical studies, policy briefs and technical support provided throughout the entire project under component 1. Moreover, the Bank and MoEC were proactive in finding a resolution to the flow of funds and other implementation issues as described above. 6 The five strategic pillars are: (i) Education Service Provision Standards, (ii) Efficient Resource Use, (iii) Transparency and Accountability, (iv) Management Control Systems, and (v) Management Information Systems.

9

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

29. M&E Design and implementation. M&E for the project had several layers. The project results framework was closely aligned with the donor and government’s desire to monitor progress towards achieving the MDG and EFA goals as well as the project goals at the intermediate level. The intermediate indicators were also aligned with the government’s RENSTRA and were appropriate for the project. The data used to monitor progress toward achieving the PDO level indicators were based on established data sources such as the SUSENAS7 and the “Education Statistics in Brief” published by MoEC. The project also funded the development of a number of M&E information systems which included: (i) a “Tool for Reporting and Information Management by Schools (TRIMS)”, which was to provide a systematic approach to gathering and reporting data at the school level; (ii) a complaint handling mechanism at the school and district level, which was supported jointly with DESP; and (iii) the SupportMySchool (SuMS) program that was to deploy an online reporting system for the public and schools to report critical needs. 30. Project support for the LGCA survey8 was also beneficial in a number of ways as it: (i) helped districts assess their capacity development needs; (ii) provided a supplementary and robust source of data which could be used to check data gathered by the districts and reported to the CPIU and the Bank; and (iii) provided the Bank and government with a data set to systematically compare changes in local education governance within the 50 BEC-TF districts between 2009 and 2012. The inclusion of this capacity assessment instrument is judged to be of singular importance for the project and a positive M&E design element. 31. At the time of project design, it was agreed that the July round of SUSENAS data would be used for measuring the PDO level indicators of gross enrollment rates at the primary level and net enrollment rates at both primary and junior secondary levels. However, in mid-2012, the Bank team discovered that the March round of SUSENAS data was the most reliable data source for measuring net and gross enrollment rates over time. Therefore, the baseline values for indicators 1, 2 and 3 reported in the ICR come from the March round of SUSENAS. Moreover, the associated end-of-project targets were re-calculated by applying the same calculation method used by the project preparation team to derive the end of project targets using the July round of SUSENAS. This was done to ensure that the results were comparable over time. The targets for the remaining PDO indicators 4, 5, and 6 were not modified since these were sourced from MoEC’s Education Statistics in Brief 2008-2011. The intermediate indicators were not affected because these were not sourced from the SUSENAS data.

7 SUSENAS is the National Socioeconomic Survey conducted on March and July of every year by the government of Indonesia. 8 Through the BEC-TF, the Local Governance Capacity Assessment and an Indonesia Local Education Governance Index (ILGI) were designed to provide a coherent national capacity development assessment system of performance in education governance and service provision in the decentralized environment. The customized diagnostic tool looked at the five strategic pillars of the BEC-TF: (i) Management Control Systems, (ii) Management Information Systems, (iii) Education Service Provision Standards, (iv) Transparency and Accountability, and (v) Efficient Resource Use.

10

32. M&E Utilization. Results framework data was gathered and reported on after supervision missions by the Bank in the Implementation Status Reports (ISRs). The data was also used to inform discussions at the TED level. The TRIMS was introduced to all schools in 2011 as part of the Education Management Information Systems (EMIS) reforms and 4,993 schools are reporting data. The complaint handling system and associated socialization was rolled out to 497 districts and 33 provinces by 2012. Since the launch of the SuMS in December 2012, there have been over 4,000 users, and MoEC is slowly adopting the system to address the critical needs of schools. There were annual supervision missions by the Bank and CPIU, and results were reported in aides-memoire. In addition, the findings of several policy studies supported under Component 19 were used by the government to support education reforms. The CPIU monitoring and evaluation of the L-BEC grant program implementation at the district level occurred up to three times each year and included representatives from MoEC, MoRA and MoF. The reviews highlighted successes, ongoing challenges and constraints for districts but also enable GoI and Bank teams to provide ongoing monitoring support to districts. 2.4 Safeguard and Fiduciary Compliance 33. Safeguards: Compliance with environmental safeguards is rated satisfactory. BEC-TF was rated a category “C” project based on the fact that no safeguards policies were triggered. 34. Fiduciary Compliance: In October 2009, a financial management (FM) risk assessment was carried out, which found weaknesses in the filing system for supporting. Following the assessment, an FM action plan deemed adequate to bringing the project into compliance with Bank requirements was developed and discussed with MoEC. The steps laid out in the action plan were also meant to complement the FM section included in the operations manual. 35. Financial management is rated moderately satisfactory. Financial management implementation was carefully monitored by Bank financial management and procurement staff who accompanied all six supervision missions. By March 2011, the FM ratings moved to moderately unsatisfactory as the Bank team identified cases of questionable expenditures related to reimbursed costs for a series of workshops. These cases were reported to the Institutional Integrity Unit (INT) and the MoEC which, through its Inspectorate General (IG), conducted an internal audit and determined that there was no evidence of loss to the state. INT also reported to the task team that they were satisfied with how the issue had been handled by the MoEC IG and closed the case. At that time, FM was upgraded to moderately satisfactory. The Bank also moved proactively to hire a Bank-supported financial management specialist to work with MoEC to strengthen their financial management oversight and procedures. Furthermore, the GoI Financial and Development Supervisory Board (BPKP) conducted audits at the national and district levels, which were unqualified and presented on time.

9For example, (i) The Comprehensive Program for Skills Upgrading contributed to GoI’s ‘Skills for the Master Plan’; (ii) an Education Public Expenditure Review highlighted the importance of efficiency in teacher management and improving the quality of teachers; and (iii) rapid response outputs on Formula-based BOSDA and District Performance Based Grants as well as on School Based Management were prepared based on inputs from the project.

11

36. Procurement is rated moderately satisfactory. Procurement was limited to: (i) consultants (service providers and individual consultants); (ii) goods (mostly computers and data processing equipment); (iii) workshops, meetings, training and study tours; and (iv) incremental operating costs. Procurement carried out by the CPIU and the districts was based on guidelines laid out in the BEC-TF Operations and L-BEC Manuals. 37. During implementation, procurement was rated moderately unsatisfactory twice. The first moderately unsatisfactory rating was due to the 18 month delay in developing the on-granting mechanism and the procurement of computers by the CPIU at the central level. Once the L-BEC grants started flowing, however, these delay issues were resolved as districts used L-BEC grants to procure goods at the local level, leading to a moderately satisfactory rating. The second moderately unsatisfactory rating was due to the cumbersome MoEC procurement guidelines that were to be used and which delayed the hiring of the three firms to provide the advisory and training services for districts on the CDP process. The delays resulted in the cancellation of these procurement activities and led the CPIU to hire four individual capacity development and financial management consultants who successfully carry out the advisory and training tasks. As a result, procurement improved and the rating for procurement moved to moderately satisfactory by the end of the project. Furthermore, a 2011 Bank Trust Fund Portfolio Review found that procurement was carried out in compliance with the applicable guidelines and procedures. 2.5 Post-completion Operation/Next Phase 38. There are no plans for a future Bank-supported basic education project. The GoI’s commitment to using 20 percent of its budget for education shows the government’s ability to support the primary and junior secondary education sub-sectors from its own resources. Moreover, 40 out of the 50 districts supported by the BEC-TF had developed post-project sustainability plans focused on improving the delivery of decentralized basic education services and improving student learning outcomes. The majority of districts have allocated funds in their budgets to continue and strengthen the following interventions/activities: (i) the EMIS, including the BEC-TF supported TRIMS; (ii) education service provision standards, including teacher upgrading and competency-based curriculum development; (iii) financial management systems, including performance-based budgeting and planning, asset management, efficient resource use and procurement policies, procedures and controls; (iv) school-based management training, including school committee development and community involvement programs; and (v) reform to district education policies and regulations, including strengthening the BOSDA improvement program10 and creating incentive schemes to encourage the deployment of qualified teachers to underserved provinces/districts in Indonesia. The government also indicated its intention to include lessons learned on transparency, financial management, and budget planning from successfully implemented BEC-TF activities in the BOS operations manual. This is important

10 The BOSDA Improvement Program was initially piloted in 2 provinces and 18 districts. The main objective of the pilot was to support regional governments in their efforts to narrow financing inequalities and improve both the efficiency and effectiveness of education spending. The pilot focused on BOSDA allocations and aimed to: (i) improve current BOSDA allocation criteria to address financing inequalities between schools; and (ii) create incentives for better school performance and encourage more effective, transparent, and participatory school-based management involving all education stakeholders.

12

because the BOS manual is still being used nationwide for the implementation of the BOS program and expected to remain in place.

3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 39. Relevance of Objectives and Design. The project objectives and design were highly relevant. The PDO and the project components were in full alignment with the 2003 CAS framework of contributing to improving governance at the primary and junior secondary education levels. Support to innovations in governance and public financing in the 50 project districts were expected to provide a model for the remaining districts as Indonesia looked to continue its efforts to improve the efficiency of decentralized and democratized service provision. The project was also seen by the GoI as central in taking forward the reforms embedded in their RENSTRA which focused, among other aims, on improving basic education sector governance and increasing the financial and budgetary capacity at the recently decentralized district and school levels. 40. At project closing, the project continued to fit squarely within the pro-jobs engagement area of the current FY13–15 Country Partnership Strategy (CPS), which: (i) supports the strengthening of the overall governance and management of the education system to improve education quality and teacher performance; and (ii) enhance human resource capacity of recent graduates, which are expected to improve economic development results. The government continues to support the education sector RENSTRA as the fundamental goal for ensuring higher quality of management of the primary and junior secondary education. 41. Relevance of implementation. The relevance of implementation is rated substantial. While the 18 month delay in releasing the L-BEC grants had an impact on implementation, there were several positive aspects of project implementation which must be noted: (i) the successful implementation of district level training to increase good governance, financial management, budgeting and accountability capacity at the local level; (ii) the completion of an impressive number of studies, which supported the TED policy discussions; (iii) the execution of local and international workshops and study visits aimed at equipping education stakeholders with the skills sets required to effectively govern and manage the decentralized education system; (iv) the carrying out Education Sector Assessments (ESAs), which informed the RENSTRA; (v) the support for the development and implementation of district level CDPs in the strategic areas of good governance, financial management, budgeting and accountability; (vi) the development and strengthening of information systems (TRIMS, SuMS, complaint handling); and (vii) the establishment of an online knowledge and learning forum (WAPIK). Annex 2 provides more details on activities that were implemented during the life of the project. 3.2 Achievement of Project Development Objectives 42. Assessment of PDO Achievements. The overall rating for the achievement of the PDO is moderately satisfactory. Progress from baseline was observed in the majority of the PDO indicators while ten out of eleven intermediate indicator targets were either met or exceeded.

13

These EFA-MDG PDO indicators that measured impact at the national level, which were included at the request of the donors, showed positive trends for enrollment, completion and transition rates. No direct project attribution can be made to improvements in the PDO indicators related to the BEC-TF interventions. However, they are reflective of the government’s aim of achieving the MDG and EFA goals for Indonesia and were to measure progress of the national BOS program to which the BEC-TF provided technical support. The intermediate indicators, which were strongly tied to the BEC-TF’s interventions, also showed gains during the project period and showed good progress at the district levels in meeting the project development objectives of improving good governance. 43. The PDO Indicators. Progress towards achieving PDOs was to be measured against the six key PDO outcome indicators11. The indicators are reflective of the progress made toward achievement of the PDO at the national level and show positive trends towards reaching the MDG and EFA goals. The achievements are as follows: (i) primary net enrollment rates nationally increased from 91.13 percent in 2007 to 92.52 percent in 2012. The modest improvements in enrollment at the primary level would be expected given that they were already close to universal prior to the launch of BOS. However, data also shows that the government has made progress in reaching the disadvantaged regions: primary net enrollment rates for the poorest wealth quintile increased from 90.93 percent in 2007 to 92.52 percent in 2012. The disaggregated data associated with this indicator shows mixed results with primary female enrollments surpassing targets by 0.45 percentage points and primary male enrollments missing the end-of-project targets by 0.11 percentage points; (ii) primary gross enrollment rates increased from 2007 to 2012 with primary females surpassing the target by 0.33 percentage points and total primary and primary males missing the targets by 0.19 percentage points and 0.08 percentage points respectively; (iii) the increases in net enrollment rates at the junior secondary level were particularly large within the poorest income quintile (rising from 53.03 percent in 2007 to 63.54 percent in 2012). These gains reflect the BOS program’s success in expanding access to poor and disadvantaged children in Indonesia. Significant net enrollment gains at the junior secondary level among both males (rising from 64.37 percent to 68.09 percent) and females (rising from 68.68 percent to 71.95 percent) were also observed over the 2007 to 2012 period; (iv) primary completion rates increased during the life of the project from 96.86 percent to 97.06 percent but were below the end-of-project targets by 0.70 percentage points; (v) junior secondary completion rates increased during the project from 98.17 percent to 98.27 percent, falling short of end-of-project target by 0.40 percentage points; and (vi) transition rates from primary to junior secondary school increased during the implementation of the project from 75.58 percent to 77.26 percent but fell short of the target by 4.94 percentage points (based on latest available 2011 data). 44. Intermediate Outcome Indicators. The intermediate level indicators were closely aligned to the BEC-TF scope and activities and showed significant gains throughout project

11 See paragraph 31 for details on changed baseline and end targets for indicators 1, 2 and 3.

14

implementation. The intermediate indicators are clustered under seven topics and will be discussed below based on the cluster design. 45. Cluster 1. GoI education policy reviewed through policy analysis and studies used to inform education policy. The policy analysis and studies consisted of rapid reviews conducted by the Bank at the request of GoI. The target set for measuring success of this cluster was to produce two studies related to education policy. The project exceeded the target with the completion of 19 studies. These studies are illustrated by the following: (i) an education financing study that reviewed the formula used by the local government school grants (BOSDA) program12 when allocating school resources which is still being used by MoEC; (ii) an analysis of the higher education law that led to MoEC integrating autonomy into the new 2012 higher education draft law; (iii) a school based management study that showed there was limited community participation which led to a requirement that a community representative be on the school committees that review school budgets13; and (iv) an analysis of the regulatory framework for ECD, which has led to new regulations for the sub-sector. In addition to studies, the project supported annual reviews of the BOS operations manual regulations that were updated, reviewed and then used for implementing the national BOS program. This was significant because the BOS operations manual sets the policies and criteria for the annual implementation of the BOS program and utilization of BOS funds14. Another rapid response that made a substantial contribution was a review of the BOS financial mechanisms for providing grants directly to districts. The provision of grants to districts was the primary cause of slow implementation of the BOS program in 2011. The review findings led to a return to the provincial block grant mechanism which streamlined the process and greatly improved on time disbursements. The full list of studies can be found in Annex 2. However, the important outcome of these studies is that they were systematically used by the government to inform their own sector policy dialogue. 46. Cluster 2. Education budget information available at the local level. This cluster of indicators was designed to measure transparency within the system. They were: (i) number of districts providing education budget information through local mass media, official announcement boards, or on a website; and (ii) number of districts with at least 60 percent of schools providing BOS education budget information through school announcement boards. The number of districts providing education budget information through mass media increased from 16 in 2009 to 39 in 2012, exceeding the target of 35. This is impressive considering that many of the 50 districts had no established local media outlets, lacked websites, and at the start of the 12 BOSDA resources are local government funds that are additional to the National School Operational Assistance (BOS) program and are allocated to schools based on a common per student formula. BOSDA also aimed to address the disparities between the higher costs of school operations in peripheral locations compared with those in the district capitals. Eighteen districts and two provinces are now using BOSDA, the more equitable formula-based approach (customized according to local criteria) to provide supplementary funding to schools based on access and performance rather than student enrolment figures. BEC-TF piloted this formula based approach focused on access and equity which was trialed on a ‘demand’ basis. 13 This requirement was made part of the BOS operations manual to strengthen compliance and raise awareness. 14 As an example, discussions about the eligible spending categories outlined in the BOS Operation Manual now include “remedial teaching” rather than lumping it together with sports, art and scouts. Another recent modification has been a clarification of the sub-district roles and the BOS audit procedures. Additionally, there were ICT technical and policy inputs to enhance the 2012 BOS Manual to include information gathering tools for the “safe schools” program, which focused on school safety and rehabilitation of damaged schools and is an outgrowth of a BEC-TF supported activity.

15

project, were not preparing budget documents in accordance with prevailing norms. The number of districts with at least 60 percent of schools providing BOS education budget information through school announcement boards increased from 16 to 37, missing the target of 50. While the target was missed, this is still a substantial improvement. Field reports revealed that districts increasingly circulated education data, plans and reports through local media outlets, such as local newspapers.

47. Cluster 3. Budget and expenditures are well defined, cost-based, and linked to performance indicators. This cluster was measured by the number of districts with education units that had a budget document with measurable outcomes related to its strategy. The target within the 50 districts was achieved. This indicator was designed to measure progress in developing performance-based budgeting at the district level, a newly introduced concept in the education system. This concept underscored the importance of developing well defined, costed budgets, ultimately linked to measureable district outcome targets.

48. Cluster 4. Education budget execution rates at local levels. This cluster was also related to financial management and budget planning, and was measured by the number of districts with a difference between planned and realized programs (disbursement rates) of less than 10 percent in the last three fiscal years. This indicator reflects the emphasis of the project on the use of L-BEC grants to improve the capacity of districts in financial planning, budgeting and execution. The number of districts with improved education budget execution rates increased from 31 to 44, thereby exceeding the target by 4 districts. This is a substantial achievement that was supported by extensive capacity development training. However, had the L-BEC grants been released in a more timely and predictable fashion, districts would have had more time to increase their funds usage rate and thereby narrowing the gap between planned and realized expenditures.

49. Cluster 5. Effective internal information, audit and control systems in place. This cluster of indicators was designed to measure transparency of education expenditures at the local level. The indicators were as follows: (i) the number of districts where all awards for education bids are published in local newspapers, on official announcement boards or a website increased from a base of 12 to 48 by the end of the project, well above the target of 35. This large increase is an indication of the improved transparency that districts sought to instill in their education procurement process; (ii) the number of districts where fixed assets are inventoried on an annual basis increased from a baseline of 23 to 36, well beyond the target of 28; and (iii) the number of districts where internal audit results are filed, and action is taken to follow up on recommendations was met by 40 of the 50 district, thereby surpassing the target of 26. This also demonstrates increased accountability at the district levels, which was a specific goal of the project. 50. Cluster 6. Community-based accountability reviews take place. This cluster was measured by the number of districts where the Bupati/Walikota's accountability report (including Dinas Pendidikan) is made available to the public. The target for this indicator, 18 districts, was exceeded by 10 districts. The fact that MoHA introduced a regulation that required district to conduct community-based accountability reviews is confirmation of the achievement of this indicator.

16

51. Cluster 7. Accurate, reliable, appropriate and timely information available for planning, budgeting and performance assessment. This cluster of indicators was measured by: (i) the number of districts where student achievement (i.e. average national exam scores, graduation rates, dropout rates, survival to grade 5 rates) are published annually, and can be easily accessed through local mass media, official announcement boards or on a website. This indicator increased from a base of 26 districts to 40 districts thereby achieving the target, and (ii) the number of districts where enrollment by gender is published annually, and easily accessed through local mass media, official announcement boards or on a website increased from 19 districts to 40 districts, surpassing the target of 25. These achievements indicate an increase in the transparency at the local level in reporting information on student achievement and enrollment by gender. The collected data is used to calculate the next year budget, which includes the national and local BOS allocations. BEC-TF interventions have also contributed to improved availability of data for use by education administrators and local communities. 52. The substantial progress related to the intermediate indicators can be directly linked to the BEC-TF activities: Transparency, financial management and budget planning at the local level met the targets and have had the intended impact in the 50 districts. The PDO level indicators show positive trends particularly related to increases in enrollments for the poorer income quintile, which indicates the government is having some success in reaching the poor and expanding enrollments. However, the gains in increasing completion rates and transition rates were more modest over the project period. Based on these factors and the overall analysis above, achievement of the development objectives is moderately satisfactory. 3.3 Efficiency 53. Positive Trends in Key Education Indicators and Internal Efficiency Gains. The BEC-TF was designed to support the GoI in achieving the MDG and EFA goals through (i) good governance in education; and (ii) improving the delivery of decentralized basic education services by local governments. The BOS program, which received capacity building and technical assistance from the BEC-TF, has contributed to the positive gains observed within the education sector between 2007 and 2012. Net and gross enrollment rates at the primary and junior secondary levels of education increased between 2007 and 2012. The increases in net enrollment rates at the junior secondary level are particularly large within the poorest income quintile (rising from 53.03 percent in 2007 to 63.54 percent in 2012). Significant internal efficiency gains have also been observed over the 2007 to 2012 period, evidenced by: (i) increased completion rates at the primary and junior secondary levels; (ii) increased transition rates from primary school to junior secondary school; (iii) reduced drop-out rates at the primary and junior secondary levels; and (iv) reduced repetition rates at the primary and junior secondary levels. 54. Unit Costs of Capacity Building Activities funded by BEC-TF. Component 2 sought to improve governance and efficient resource use by providing targeted training in financial management. About 50,105 local education staff benefitted from project-financed training at an average cost of US$62 per person. The unit costs associated with this capacity building activity are comparable to those charged by USAID, AUSAID, and UNICEF for a similar intervention. The project also trained 13,274 local education staff in performance-based budgeting and

17

planning at an average cost of US$65 per beneficiary. Again, the unit costs for this three-day activity are in line with the fees charged by USAID, AUSAID, and UNICEF for an equivalent training program.15 Under Component 3, 2,422 education staff received a three-day course on the use of financial data to improve the efficiency of funds allocation/use at an average cost of US$82 per person. Although no comparable financial data from other donor-funded activities are available for this particular activity, feedback from the districts suggested the training provided was highly relevant and comprehensive. 55. Efficient Resource Use. The BEC-TF capacity development support has contributed to improvements in the efficiency of funds usage among the 50 targeted districts. Over the life of the project, local governments have registered important gains in the processes which determine public resource uses. Results from the Indonesian Local Education Governance Survey (ILEGI) indicate that the number of district education offices providing progress reports on their activity plans have increased during the project. Improvements in the local planning and budgeting processes have also been observed. More specifically, 74 percent of districts are now establishing their own budget priorities and ceilings prior to the provincial/central offices even begin their planning exercise, which is a substantial increase from the 44 percent in 2009 and is a reflection of the positive progress of the education decentralization process. The number of BEC-TF supported districts with improved education budget execution rates have also increased from 31 to 44 over the life of the project. While some of these efficiency gains may have been the result of the central government’s efforts at introducing medium-term expenditure frameworks16, the BEC-TF program’s capacity building activities related to performance-based budgeting, planning, and execution have been instrumental to the success of this process. 56. Reduced Financial Leakage. Prior to the launch of the BEC-TF in 2008, the first Governance and Decentralization Survey (GDS1) found that approximately 70 percent of funds allocated by all levels of Government reached the school. By 2012, nearly 90 percent of the total funds disbursed by all levels of Government were received by schools.17 These lower levels of financial leakage within the education system are in part a result of the GoI’s overall efforts at strengthening external and internal audit systems, as well as the BEC-TF’s increased focus on improving financial management, transparency and accountability at the local level. The BOS Operations Manual, which was updated and revised throughout the life of the project, played a key role in minimizing the misuse of funds at the school level by: (i) clearly stipulating eligible and ineligible expenditures categories; (ii) strengthening financial disclosure and reporting requirements; and (iii) mandating community oversight of funds usage. Given that the BEC-TF provided capacity development and technical assistance to the BOS program, this is an important contribution by the project. It is also important to highlight that all BEC-TF audit reports were unqualified, providing further evidence that no systematic financial issues were identified within the 50 BEC-TF districts.

15 The unit cost of capacity building activities tend to be relatively uniform across the various development agencies operating in Indonesia because the government typically establishes a price ceiling (or “standard price”) which must be complied with by all development agencies. 16 Efforts to ensure better local resource management were embodied in the 2003 state finance law that started implementation in 2006. In particular, the law introduced medium-term expenditure frameworks into local planning and budgeting. This represented a shift from earlier annual input based budgeting with a focus on the expected outputs of planning and budgeting decisions. 17 Regional Independent Monitoring Survey, 2012.

18

57. The ICR rates the efficiency of the project to be substantial. The eighteen month delay slowed implementation of capacity development activities to be funded by the L-BEC grants. However, over this initial period the Bank worked closely with the CPIU to maintain momentum in the project implementation through several activities, such as: (i) local and international workshops and study visits aimed at equipping education stakeholders with the skills sets required to effectively govern and manage the decentralized education systems; and (ii) the support for the development and implementation of district level CDPs in the strategic areas of good governance, financial management, budgeting and accountability. Once the on-granting mechanism was clarified and put into practice, the pace of the implementation of the project increased considerably. 3.4 Justification of Overall Outcome Rating 58. The objective and project design were highly relevant in that they fit squarely into the government’s multi-year strategic plan program. The relevance of implementation is judged to be substantial given that the implementing agencies worked to effectively resolve the implementation challenges related to the 18 month delay in releasing L-BEC grants. The efficiency of the project was substantial considering that improvements in transparency, budget planning and financial management reduced financial leakage and contributed to better resource utilization. Given these results and the detailed findings outlined above, BEC-TF overall outcome as moderately satisfactory.

Original Project Project Relevance Achievement of PDO (Efficacy) Efficiency Overall Rating Substantial Moderately Satisfactory Substantial Moderately Satisfactory

3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development There is no additional information other than what has already been reported. (b) Institutional Change/Strengthening 59. There has been an institutionalization of the BOS operations manual that directs the use of BOS grants, financial reporting and budgeting at the school level. There has also been an institutionalization of the annual reviews of the BOS operations manual so that relevant modifications can be made based on lessons learned during the yearly implementation period. The complaint handling mechanism piloted with BEC-TF support has been widely disseminated across all 50 BEC-TF districts and an improved monitoring and evaluation system has been institutionalized. MoEC monitors the program annually and the results are used to improving the BOS program. The Safe Schools activity, which focused on school safety and rehabilitation of damaged schools, has now been institutionalized by MoEC. (c) Other Unintended Outcomes and Impacts (positive or negative) There are no other unintended outcomes or impacts other than what has been reported.

19

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not applicable. 4. Assessment of Risk to Development Outcome 60. The risk to development outcomes at project closing is rated modest given the GoI’s commitment to the increased budgets for education. In addition, 40 out of the 50 districts have developed post-project sustainability plans and allocated budget resources within their next budget cycle to further improve the delivery of decentralized basic education services. The MoF, in its support of all Indonesian schools through the nationwide BOS program, has now developed an efficient funding flow mechanism which transmits funds from the center to the provinces and then the district and schools. The decision at the MoF level to develop this improved funds flow process was supported by BEC-TF-supported rapid response policy note. Audits at the district levels report reduced levels of financial leakage between the 2008 to 2012 period. Human resources at district levels have benefited from the capacity development training supported by the BEC-TF, and the project outcome indicators already show that management, especially financial management, has improved in the districts. Information collection tools such as TRIMS and SupportMySchool have provided local schools and communities with increased capacity to monitor and evaluate the quality and efficiency of school development programs. Such an involvement, whether by parents or community leaders, is important in assuring transparency and accountability at the school level. The risk for financial management at project preparation was rated high given the complexity and size of the Indonesian education system and the reported challenges with financial reporting and internal and external control systems. Many local governments lacked the financial management technical capacity necessary to manage increased fiscal resources and responsibilities. Through the BEC-TF, capacity development took place and the project enhanced districts’ financial management and reporting skills. Bank and government audits at the end of the project confirmed that such issues as leakage have decreased substantially. The ICR team judges that the project has had a considerable positive impact on the reporting and flow of funds, and rates the current risk as modest. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry 61. Bank performance at entry is judged to be moderately unsatisfactory. The Bank preparation team worked effectively with MoNE and MoRA in the development of the project. Extensive time was devoted to tailoring project activities around the five-year RENSTRA with a strong focus on capacity building at the district and school level, priorities of both MoNE and MoRA. The Bank team worked with the GoI preparation team in the development of the LGCA, in order to identify the capacity needs to be addressed by different project components and sub-components. The thorough preparation process made it possible for the government to satisfactorily comply with the conditions of effectiveness within a short period and for the Bank to declare the BEC-TF effective on June 5, 2008, the date of Grant Agreement signing between the Bank and the Republic of Indonesia. 62. One task team leader was responsible for the entire project preparation cycle and continued her responsibility into the implementation process thus providing continuity to the

20

timely implementation and completion of all the key project preparation activities that were required. Considerable time was spent by the task team in discussions with both the donors and the government in an effort to ensure agreement on the donors’ priorities and the strategic goals of the GoI, the CAS and project development policies of the Bank. Despite these strengths, Bank performance is rated moderately unsatisfactory primarily because the PDO indicators selected were national in scope and supported GoI's achievement of MDG and EFA goals rather than project specific outcomes. On the other hand, the intermediate indictors selected by the preparation team were highly relevant and strongly tied to the program’s interventions/scope. (b) Quality of Supervision 63. The continuity of the task team leader allowed for strong leadership for the project. This was particularly helpful in discussions with MoEC, MoRA, the national planning agency (BAPPENAS) and the MoF related to the development of standard operating procedures (SOPs) for the on-granting mechanism. The Bank was instrumental in supporting the dialogue between the government agencies so that they could find a solution and develop adequate procedures and guidelines. 64. There were six supervision missions which included a mid-term review. There was a total of 54 months elapsing from project effectiveness to the closing which shows that the Bank team carried out, on average, one supervision mission every 9 months, thus maintaining close oversight of project implementation. Bank staff in Jakarta led the supervision efforts and provided ongoing technical support throughout the life of the project. Even though delays occurred, supervision still took place to support district training and capacity development activities. As mentioned before, these activities were brought forward by the CPIU to maintain district involvement and interest in the project following the significant momentum generated during the extensive district selection process. The supervision teams included the appropriate technical experts and appropriate skills mix to ensure compliance with procurement, financial management, and monitoring and evaluation. Mission findings were communicated to the GoI and Bank management in a timely manner, and the task team followed up on their recommendations. The Bank’s supervision missions were instrumental in detecting and resolving issues related to financial management by working with the CPIU and MoEC. 65. In November 2011, the Bank restructured the project to accommodate the cancellation of grant funds due to the Netherlands’ decision to reduce its development assistance to Indonesia. Despite the cancellation, the GoI, with support from the Bank, continued to implement the project with a significant number of capacity development activities that were provided to the 50 BEC-TF districts.

(c) Justification of Rating for Overall Bank Performance Based on the analysis above the overall Bank performance is rated moderately satisfactory.

21

5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory 66. Government performance is rated moderately unsatisfactory. This is based largely on the fact that project implementation was delayed for 18 months because of the lack of standard operating procedures (SOPs) for the MoF’s on-granting mechanism. The delay forced a readjustment in the implementation schedule and ultimately necessitated an extension of the project closing date. The inability of the MoF to develop the necessary operating procedures for the on-granting mechanism led to a downgrading of the project to moderately unsatisfactory in June 2010. However, the GoI and Bank teams worked hard to get the SOPs established which, once developed, allowed L-BEC grants to flow to districts and helped significantly improve project performance for the 2011/12 period. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory. 67. During project preparation, the CPIU’s role focused on coordination, facilitation, mentoring and problem solving. During project execution, the CPIU was in charge of evaluation and reporting as well as coordination between line ministries, the Bank, provinces and districts. Facilitation included the development of guidelines, manuals and modules related to the implementation of activities, action plan preparation, verification of documents, disbursement of grants and preparation of reports. The CPIU also conducted monitoring and evaluation activities of up to three times each year. MoEC, as the implementing agency, and more specifically the CPIU, worked diligently to find a solution to the problem of the lack of SOPs. Furthermore, discussions with the PSC led to bring training and capacity development activities with districts forward. The CPIU carried off these tasks well and were able to maintain project implementation momentum in the face of bureaucratic hurdles placed in front of them. (c) Justification of Rating for Overall Borrower Performance. Taking into consideration the reasons stated above, the recipient performance is rated moderately satisfactory. 6. Lessons Learned 68. Lesson 1. Funds granted against capacity development plans, based on a structured assessment process, are an effective way to meet the specific needs of individual districts. Under the BEC-TF project, districts developed and implemented plans and activities focused on strategic education governance and management improvement needs identified in the baseline LGCA. The iterative L-BEC grant planning, submission and budget acquittal/settlement process necessary for the release of each grant tranche, although time consuming, reinforced transparent and accountable financial management practices. 69. Lesson 2. The use of large scale longitudinal surveys such as the LGCA provides a good capacity development needs assessment tool to a project. Envisioned from the start of the project to be a diagnostic tool and a crucial element to assessing both district level capacity development needs and project outcomes, the LGCA was used in a number of ways as it: (i)

22

helped districts assess their capacity development needs; (ii) provided a supplementary and robust source of data which could be used to check reports gathered by the districts and reported to the CPIU and then to the Bank team, and (iii) provided the Bank and government with a data set to systematically compare changes in local education governance within the 50 BEC-TF districts between 2009 and 2012. 70. Lesson 3. The efficient and timely implementation of project activities is contingent upon establishing and maintaining a clear funds flow mechanism. Prior to the BEC-TF project, funding was provided through the block grant system. However, the MoF decision to move to an on-granting mechanism led to significant delays because of a lack of standard operating procedures due to internal MoF issues. The task team proactively, along with the CPIU, worked with the MoF in coming up with a solution to this impasse, and after an 18 month delay, the project was then able to move forward. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/Implementing agencies See Annex 7 for full report from implementation agency. (b) Co-Financers Not Applicable (c) Other Partners and Stakeholders Not Applicable

23

Annex 1. Project Costs and Financing (a) Project Cost by Category1 (in USD Million equivalent)

Category Description

Appraisal Estimate2

(USD millions)

Actual / Latest Estimate (USD

million)

Percentage of Appraisal

Consultant Service Providers 1.39 1.02 73% Consultant Services-Others 0.64 0.45 70% Goods-Equipment & supplies 0.12 0.11 92% L-BEC Grants 14.06 12.34 88% Workshops, Studies, Surveys, Training and Study Tours 3.12 2.72 87% Incremental Operational Costs 1.20 0.89 74% Designated Account -0.18

Total 20.53 17.35 85%

1The project costs are provided by disbursement category because the project implementation unit did not keep disbursement records by component, only by category. 2This column does not represent the cost at appraisal but rather the net project amounts in US dollars after the trust fund account was reduced by €8.7 million at the request of the Dutch government and €1.1 million was returned prior to project closing December 31, 2013 due to inability to utilize the remaining funds. This is judged to provide a more accurate accounting of the allocations for each project disbursement category. (b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions) Percentage of

Appraisal

Trust Funds 40.83 17.35 40.30%1 Borrower 0.00 0.00 0.00 International Bank for Reconstruction and Development 0.00 0.00 0.00 1The original amount of the trust funds was €25.6 million (US$40.83 million equivalent). The trust fund account was reduced by €8.7 million and €1.1 million was returned prior to project closing December 31, 2013. The project also has an undisbursed balance as of May 30, 2013 in the amount of €2.24 million (US$3.18 million equivalent). Therefore, the percent of disbursements of the net project funds is 85 percent.

24

Annex 2. Outputs by Component 1. The Basic Education Capacity Trust Fund (BEC-TF) was developed to support the Government of Indonesia’s (GoI) Medium-Term Strategic Plan (Rencana Strategis—RENSTRA) and to complement other bilateral and multilateral education programs. The overall objective of the BEC-TF, as stated in the Grant Appraisal Document (GAD), was to contribute to the GoI’s achievement of MDGs and EFA goals through good governance in Indonesia. The purpose of the program was to support GoI in improving the delivery of decentralized basic education services by local governments and schools in selected program areas, and by extension in other locations. Within this context, the BEC-TF sought to strengthen the planning, budgeting, financial management and accountability at the local level in order to reduce leakages, increase the efficiency of fund usage, and improve the delivery of education services. The BEC-TF also envisioned developing a sound monitoring and evaluation system capable of providing relevant data to guide and shape the decision-making process across the entire education system. 2. The following list of outputs reflects a combination of Bank-executed trust fund (BE-TF) and recipient-executed trust fund (RE-TF) outputs. While the ICR analysis is based only on the RE-TF, the team felt that it was important to show the outputs associated with the overall trust fund. The team has identified what outputs are associated with the BE-TF and RE-TF. 3. Component 1: Policy Analysis and Dialogue. Based on the completeness of the work that was supported by the BEC-TF when compared to the prospective outline of activities in the GAD, Component 1 is rated Satisfactory. This component was originally designed to support the development of a Sector-wide Approach for the delivery of education services and to coordinate the development partner’s support for the sector. Two elements required for laying the foundation for SISWA included: (i) a forum that allowed key sector stakeholder to have a dialogue which was the Thematic Education Dialogue (TED) and (ii) in-depth policy analysis which assisted in identifying needs and would inform policy priorities and direction and this was done through analytical studies. 4. Sub-component 1.1: Thematic Education Dialogue (TED). The objectives of the TED were: (i) to provide strategic and policy recommendations on education sector reform and development issues, including resourcing and planning; (ii) to coordinate cooperation among GoI institutions playing significant roles in the education sectors and between GoI and international development partners in the education sector; (iii) to support preparation for a sector wide approach and for external financing to be delivered through the GoI system; and (iv) to review and evaluate progress on implementation of policies and sector performance. 5. TED (BE-TF activity) was responsible for (i) convening education stakeholders to identify the key education sector issues; and (ii) introducing stakeholders to best international practice related to SISWA. This was done through: local and internal training workshops which enabled senior government officials and development partners to discuss key education issues and provide strategic and policy recommendations on education reform and development issues, including resourcing and planning. The discussions at the TED meetings fed into the pipeline of analytical work and collaboration for policy and regulatory change with GoI and other development partners as well as informed the preparation of the 2010-2014 Medium-Term

25

Development. The TED activities invigorated dialogue and debate with stakeholders from Ministry of Finance (MoF), The Planning Agency (BAPPENAS), The Ministry of Education and Culture (MoEC) and the Ministry of Religious Affairs (MoRA) which generated significant momentum between education policy makers and stakeholders at all levels on key GoI identified issues. The analytical work requested were on strategic issues such as: (i) education sector financing that contributed to a change in the BOS fund disbursement and ways to increase efficiency and equity in education budgeting; (ii) basic education provision; (iii) refocusing non-formal education; (iv) higher education; (v) school based management (SBM); (vi) supplementary district funding for schools (BOSDA); (vii) youth skills development, and (viii) senior secondary strategy. The TED requested many activities within Component 1 of the BEC-TF which are summarized below. 6. Local and International Training Workshops. This sub-component under the TED supported workshops on the following topics: (i) building annual assessment systems for tracking sector performance over time; (ii) Performance Assessment System of Education (PASE) to improve sector governance and management; (iii) improving the efficiency, effectiveness and equity of education public spending; and (iv) facilitating access to knowledge and adoption of good education practices.

7. The Comprehensive Program for Skills Upgrading East Asia Conference: Skills Development for Productivity provided the forum for policy makers from participating governments, industry groups, education and training providers and international experts to discuss issues about and approaches to skills development. Findings from the “Revitalizing Public Training Centers in Indonesia: Challenges and the Way Forward” study report, developed with BEC-TF technical analysis support, were presented by the Directorate-General, Training and Productivity Development, MoMT. (BE-TF activity) 8. SupportMySchool (Bantu Sekolahku): Soft-launched in December 2012. TA provided to develop this innovative system which aims to solve service delivery challenges and the need for transparency and accountability through the effective use of ICT and social networking. Discussions underway to integrate TRIMS, complaint handling and Safe Schools into the web-based platform to link complementarities. MoEC has institutionalized and provided financial and human resources to manage the system; continuous improvement is on the basis of user and public feedback. (BE-TF activity) 9. Support to Education Development Program Coordination in Papua: This additional activity followed the Bank decision to hire a locally based coordinator for Papua to facilitate links between existing and planned support programs to the provincial and local RENSTRAs. This activity played a valuable role in briefing ESWG, welcoming incoming missions, facilitating meetings and promoting links between programs and advising on local priorities. (BE-TF activity) 10. World Bank Institute Transforming Tertiary Education for Innovation and Competitiveness Course, Bali: This course was to move the policy dialogue forward on key reform issues, primarily dealing with quality assurance and innovative financing mechanisms,

26

supported by BEC-TF. Generated high levels of interest and follow up from the GoI while also raising Indonesia’s profile in the region and globally. (BE-TF activity) 11. BEC Dissemination Program, Indonesia: Improving Education Governance; Strengthening EMIS for Data based Planning and Budgeting; ICT for Education; Clean Energy Resources to Advance Schooling; Safe Schools and DRR; BOSDA; Public Participation to Improve Service Delivery; Best Practices; Performance Based Budgeting; and SBM Talk show: Developing and Synergizing Local Education Stakeholders to Promote Education Governance. (BE-TF activity)

12. National BOS School Management Team Training design and development, in consultation with other Development Partners. Four modules were developed: School Self-Evaluation, Planning and Budgeting, Financial Management and Character Building. BEC funded the development and production of a DVD which contained all participant and trainer materials as well as supplementary information from MoEC such as Buku Sekolah Elektronik e-textbooks copyrighted to MoEC), HIV-AIDS, Narcotics Awareness and Use Prevention, Disaster Risk Reduction for Safe Schools and Gender. During review, responses to questions about the learning materials indicated that content and materials were highly relevant and a valuable resource for future reference for School Management Teams. The actual training was an intensive 3-day program to support over 650,000 participants from primary and junior secondary schools and madrasahs in 339 Districts and 31 Provinces to maximize the use of BOS and other school resources for improved SBM. However, there were delays in the roll-out of the training program due to the lengthy process of signing the MoU between the GoI and AusAID. As a result the training was conducted after most schools had prepared and submitted 2012 4-year school development plans and annual school activity and budget plans to the district office. (BE-TF activity) 13. Study visits. Under Component 1 study visits were undertaken to learn lessons from other governments in areas that the TED group had identified as important to the further development of the education system. Six of the most relevant study tours were the following: (i) New York, USA to learn about education effectiveness and performance appraisal; (ii) China-Hangzhou for ICT and to attend an educational regional conference; (iii) India-to attend an international seminar on teacher development and management in Udipur; (iv) Hong Kong SAR, China-to attend the East Asian International Conference on Teacher Education Research at the HK Institute of Education Research; (v) Philippines-a study tour to look at the government’s Basic Education Sector Reform Agenda (BESRA) program; and (vi) Thailand-Higher Education Governance and Finance. (BE-TF activity) 14. Development of a Sector-wide Approach Framework. Although the sector-wide approach framework was canceled, the BEC-TF did supported the work of an education specialist, prior to the cancelation, to evaluate existing programs, models and strategies being used for education development at the district level, in order to give an overview of the kind of support which MoEC and development partners should provide in the future. This was a joint exercise with AusAID, the European Commission and the Netherlands as well as the Bank, and was supported by MoEC. The report from this activity was circulated among colleagues in MoEC, BAPPENAS, MoRA and among ESWG partners, and was revised based on discussions

27

at various Education Sector Working Group meetings, a seminar and presentations for the BEC and Bank teams. (BE-TF activity) 15. Rapid Response. Support was provided for: (i) education policy advice; (ii) international fora on school based management; (iii) the national BOS training design and materials development and trailing process; (iv) formula based BOSDA and district performance based grants; and (v) an Education Public Expenditure Review (EPER) which explored the impact on sector performance and efficiency of the constitutional rule of allocating a minimum of 20 percent of the national budget to education. The analysis was done in the context of 10 years of decentralization and the outcome of existing financial mechanisms on the efficiency and equity of education spending at district and school levels. Other rapid response activities were:

i. Basic education decentralization technical assistance and policy development support: Policy advice contributed to the change in policy for BOS fund disbursement in 2012 to reach a new solution following the failure of district disbursement in 2011. The ensuing change resulted in the fastest disbursement of the grant since its introduction in 2005. (BE-TF activity)

ii. Education Financing: Successful meeting brought together representatives of several ministries and the office of the Vice President. Four presentations made by the Bank, the Asian Development Bank, USAID and the Aceh local government led to a rich discussion on ways to increase efficiency in education budgeting. (BE-TF activity)

iii. At national level an Education Public Expenditure Review occurred in close consultation with the Vice President’s and the Minister’s office and, BAPPENAS. Key issues highlighted the importance of efficiency in teacher management, improving the quality of teachers generating ongoing discussions about strategies for the future. (BE-TF activity)

iv. District capacity development training commenced with training in education public expenditure analysis using a highly consultative and participatory approach modeled on the successful Public Expenditure Analysis and Capacity Harmonization approach. Some districts have incorporated ongoing activities in four-year strategic plans. (BE-TF activity)

v. School Management. Regional Ministerial Forum: Benchmarking Education Systems for Results, June 2011 built momentum for three SBM TED meetings, led by the Vice Minister and MoEC. A development partner workshop was conducted to inform a Seminar convened for national, provincial, district and school stakeholders on “Lessons learned from a decade of SBM Implementation in Indonesia”. (BE-TF activity)

vi. A series of international conferences were held in 2011 which focused on benchmarking, analyzing PISA results, lessons learned in SBM practices in Indonesia, and the BOS Manual was updated on how BOS funds could be used to support SBM efforts. standards, competency based training and certification) was produced following analytical technical assistance which has contributed to GoI’s ’Skills for the Master Plan’. (BE-TF activity)

vii. Teacher Reform: Full team of Director Generals from MoNE and MoRA, representatives from MoRA and BAPPENAS and the Vice President’s Office, as well as education leaders from Universities, the Teachers’ Union. Chaired by the Vice Minister with presentations by WB, AusAID and USAID. The development of this activity was to redeploy teachers to underserved parts of Indonesia. (BE-TF activity)

viii. Holistic, Integrated Early Childhood Education provided a platform to: (i) follow up on issues and make recommendations about ECD positioning and policy development in 2013

28

and the development of post-2015 MDGs: (ii) enable Indonesia to share its progress on ECD policy to an international audience and to bring international good practice to an Indonesian audience; and (iii) provide a forum for ‘Special Envoys for ECD Strategy’ aimed at increasing advocacy for budgetary and policy support. MoEC was supported to convene the 15th UNESCO-APEID International Conference. (BE-TF activity)

ix. Inspiring Education: Creativity and Entrepreneurship. It provided the forum to focus on linkages between creativity and entrepreneurship in education, cultivating talents and skills and strengthening linkages between education and the workplace. (BE-TF activity)

x. Public Private Partnerships in Education: Consultations about education sector financing. The Chair of TED suggested revitalizing TED activities by conducting regular meetings to discuss various education issues and to sharpen government policies/programs in education. (BE-TF activity)

xi. MoEC was supported to convene the National Educators Conference: Reshaping Entrepreneurship Education in conjunction with the Sampoerna Foundation and the Mien R. Uno Foundation in December 2012. The forum reinforced the need for entrepreneurship education that will lead to better education quality for Indonesia to increase capacity in the global era. (BE-TF activity)

xii. National Teachers Congress on Education for Sustainable Development: More than 750 teachers from across Indonesia participated in this high profile event opened by the Minister. BEC provided two speakers to lead sessions on Education for Sustainable Development and supported the attendance of 100 teachers and education from BEC and BERMUTU districts. (BE-TF activity)

xiii. BPKP Annual Training for L-BEC grant use: BPKP were requested to audit district grant fund use according to the L-BEC manual. BEC-TF provided training which was conducted annually to ensure that auditors were aware about manual stipulations for grant fund use. (RE-TF activity)

xiv. Disaster Risk Reduction for Safe Schools activities, in consultation with the Global Fund for Disaster Risk Reduction. Outcome: Guidelines for Safe Schools have been produced and implemented during several pilot activities and introduced to all schools during the 2011 BOS national training program. (BE-TF activity)

xv. Clean Energy Resources to Drive Advances in Schooling (CERDAS) which has piloted the installation of renewable energy in two remote schools without electricity. The momentum generated by this pilot has seen it extended by MoEC and the UNESCO supported Green Schools Program. (BE-TF activity)

16. Sub-component 1.2: Analytical Studies. The overall objectives of sub-component 1.2 were to: (i) develop, synthesize and disseminate sector knowledge and information; and (ii) facilitate consensus building on sector issues, priorities, and investment strategies. 17. Education Sector Assessment (ESA): GoI requested assistance in the preparation of the ESA as the main education theme under TED for 2008. This was to form the analytical base for a new strategically pertinent and focused medium term education sector strategy or RENSTRA (2009-2014) to be prepared jointly by MoEC and MoRA with support and guidance from BAPPENAS through: (i) generating, synthesizing and disseminating sector knowledge and information; and (ii) facilitating consensus building on sector issues, priorities, and investment strategies. (BE-TF activity)

29

18. Within the ESA, the Bank supported a number of analytical studies and provided technical assistance and policy development support for the GoI’s basic education decentralization policy. Specifically the project supported policy discussions on the disbursement in a decentralized environment of the BOS program. This policy discussion facility was important because GoI’s decision to decentralize BOS in 2011 caused substantial delays in BOS funds reaching schools. Policy advice contributed to the change in policy for BOS fund disbursement in 2012 to reach a new solution following the failure of district disbursement in 2011. The change resulted in the fastest disbursement of the grants since its introduction in 2005. The BEC team worked closely with the BOS team on the design of a supplementary/variable BOS program to launch in 2011 which included performance requirements and measures to reduce disparity at the local level.

19. Another example of the technical assistance provided by the Bank was a response to the Vice President’s office which asked the Bank to support preparation of a Youth Employment Strategy to focus on skills development. Several presentations made on existing constraints for an effective skills development system in Indonesia, including fragmentation of provision and funding, lack of quality assurance systems and the need to better coordinate policies. As a result of this TA, a comprehensive Skills Upgrading Program was developed in early 2012. This intensified interest in further analytical work and TA linked to skills development agenda in Indonesia. 20. Similarly the Bank provided technical assistance for ICT in Education evidenced by the following: (i) endorsement of Distance Education Ministerial Regulation No. 24 2012 which emphasizes the importance of e-learning; (ii) specific contributions on Open Education Resources incorporated into Higher Education Law; (iii) draft of the ICT Competency Framework for Teachers Ministerial Regulation; (iv) draft of Ministerial Regulation on ICT Governance to replace Ministerial Regulation No. 38 2008 on ICT Management; and (v) initiated dialog focused on ICT based learning in remote areas. 21. The Bank also provided technical assistance on policy development and TA on scholarships for the poor. Proposals focused on maximizing impact and improving targeting when children transition between levels presented to Minister and senior staff. In discussions with GoI, the Bank suggested changes about expanding the program if fuel subsidies were reduced. There is some agreement to use the Unified Poverty List18 to improve targeting. Progress stalled following rejection of the Fuel subsidy Reduction Scheme. 22. A series of workshops were designed to inform the GoI’s RENSTRA. Nine key themes in the education sector identified were: (i) Improving the match between skills demanded by the labor market and produced by the education system; (ii) Non formal education: scoping and

18 The United Poverty List is The unified database on poverty is a database that is being put together by TNP2K (Tim Nasional Pengentasan dan Penanggulangan Kemiskinan (National Poverty Reduction Team, Vice President’s Office) to identify all poor households in Indonesia. The purpose is to improve the targeting of all social assistance programs including scholarships.

30

refocusing; (iii) Compulsory Basic Education: provision, reaching the unreached and expanding coverage; (iv) The National Examinations; (v) Teacher Management: Employment and Deployment; (vi) Teacher Management: Quality and Certification; (vii) Education Sector Financing: Fiscal Reform, Public-private Partnership; (viii) Building skills for increased competitiveness; and (ix) Improving access and quality of basic education and enhancing governance use. These workshops invigorated dialogue and debate and generated significant momentum between education policy makers and stakeholders at all levels on key GoI identified issues. They also informed the preparation of the 2009 - 2014 Medium Term Development Plan. ESA activities used evidence-based approaches which contributed to consensus-building on key development issues. 23. Studies to support capacity development activities. The purpose of the studies was to support the ESA with analytical work. Several national and international experts were recruited to carry out studies and prepare analytical papers and policy notes based on international as well as Indonesian research finding. Among the most relevant policy and analytical studies undertaken by BEC-TF were: (i) Teacher Management; (ii) Quality of Education in Madrasah’s Report Improving the match between skills demanded by the labor market and produced by the education system; (iii) Education, Training and Labor Market Outcomes for Youth in Indonesia: Report and Policy Brief; (iv) Skills Development for Productivity Conference: Policy Dialog, (v) BOS: Making BOS Effective Under Decentralization; (vi) BOSDA: The BOSDA Improvement Program: Enhancing Equity and Performance through Local School Grants Policy Brief; (vii) BOSDA Guidelines; (viii) Higher Education Reform Seminars and Policy Brief; (ix) School Based Management: Education Update Policy Brief; (x) School Financing/Unit Cost calculations: Guidelines and Tools; (xi) Early Childhood Education: Policy Brief; (xii) Trends in Learning Outcomes based on internationally standardized test: Policy Brief; (xiii) Education Public Expenditure Review: Report and Policy Brief; (xiv) Free Basic Education for Poor Households; (xv) Disaster Risk Reduction for Safe Schools: Guidelines and Manual; and (xvi) Clean Energy Resources to Drive Advances in Schooling: Impact Study and Policy Brief. The study reports generated ongoing discussions in a number of stakeholder panels and meetings about strategies for the future and highlighted the importance of efficiency in teacher management, improving the quality of teachers and learning outcomes, improving planning budgeting and financial management with in the education system, strengthening monitoring and assessment systems, and reforming higher education financing mechanisms. 24. The Bank and the MoEC carried out a communication strategy to increase familiarity with BEC-TF achievements so as to ensure the visibility of the program and disseminate its outputs and publications to stakeholders (GoI key Ministries, Provincial, District Education Offices, Schools, NGOs, Donors, Development Partner). The BEC-TF program communication and outreach plan focused on: (i) program visibility: promote awareness of the BEC-TF program and activities among government counterparts, national media and development partners; (ii) issue visibility: promote awareness of BEC-TF supported core principles and strategic areas, and programs and activities (i.e., transparency and accountability, education service provision standards, management control systems, information management systems, efficient resource use) among government counterparts, national and local media, and development partners; (iii) donor visibility: Increase stakeholder awareness of ongoing donors’ efforts to support GoI’s progress in improving education outcomes and governance in the sector; and (iv) capacity

31

building of local and national journalists to cover education governance issues in the print, television and radio media. (BE-TF activity) 25. Component 2: Good Governance, Financial Management, Budgeting and Accountability. (RE-TF activity)There was need to strengthen the education sector governance and management capacity, particularly at the district level to use resources more effectively, efficiently and equitably. Through this component the BEC sought to strengthen planning, budgeting, procurement, financial management and assessment systems to strengthen accountability practices at the district level. The expected outcome of Component 2 has been improved governance and efficient resource use through increased transparency, accountability, improved budget processes and performance base financing, improved financial management and accounting, especially in local governments. The sub components are good governance and financial management, planning and budgeting and school management. 26. This component is rated moderately satisfactory. This rating is based on the fact that the expected training was conducted but there were some reported weaknesses in the impact of the training. Substantial efforts were made in the delivery of capacity development training and technical advice. However, several of the component activities were delivered late in the project lifetime because of the problems in obtaining final agreement related to the on-lending process. These delays led to slower than expected implementation of the new management methods. While there was a strong focus on capacity development in financial management other districts still exhibit capacity constraints that will require on-going training to sustain the improvements. However, it will take more time for districts to fully analyse and use data to inform decision making at the local level. An additional challenge faced by the districts was the difficulty many faced in preparing the necessary documents for withdrawal of the L-BEC grants. Training was provided by MoEC to improve the management capacity of the districts but the problems seemed to persist. 27. Sub-Component 2.1: Good Governance and Financial Management. This sub-component supported the following activities to facilitate the improvement of governance and financial management. The project supported training on Improved Financial Audit Results and Updated Assets Inventory. CDP activities included training for Education Asset Management, as part of the District Finance Office’s district-wide inventory. Training was followed up with practical assignments to register school education assets. As a result the Districts had an updated asset inventory and improved national Audit results from qualified in 2009 to unqualified in 2010 and 2011. Complaint Handling: The BEC-TF supported a number of complaint handling activities, detailed in Component 3 of this annex. However, under this sub-component, this activity supporting the strengthening of complaint handling systems, in particular those associated with the school operational grants (BOS) program to improve transparency and participation for better education governance. This complaint handling mechanism enables all levels of BOS management teams to record, report and respond to any feedback provided by community. 28. Sub-component 2.2: Planning and Budgeting. This sub-component was to support districts as they began to plan for the performance-based budgeting that would be implemented under the medium-term expenditure framework. The activities under this sub-component were

32

expected to take place and the local government level to give the proper tools and skills to assess, determine needs, plan and set targets and properly budget education activities. The activities are described below. 29. Three-year Capacity Development Plans: Districts developed and implemented plans and activities focused on strategic education governance and management improvement needs identified in the baseline LGCA. School management related activities were also included to strengthen the link between the school and district to improve coordination and service delivery. All CDP activities used L-BEC grants to address local needs and, not those determined centrally; targets were monitored on an annual basis. The iterative L-BEC grant planning, submission and budget acquittal/settlement process necessary for the release of each grant tranche, although time consuming, reinforced transparent and accountable financial management practices. Analysis of CDP outputs/outcomes shows overall improvement in education service provision. 30. A series of modules were developed to support local governance capacity development in the five strategic areas of the BEC program, namely, Transparency and Accountability; Education Service Provision Standards; Management Control Systems; Information Management Systems and EMIS; Efficient Resource Use. Each module was accompanied by a trainer’s guide and slides, and is available in hard copy as well as digitally. The approach uses ICARE (Introduction, Connection, Application, Reflection, and Extension) as developed under USAID’s DBE1 program19, and includes large and small group discussions and practical activities within a cycle of continuous improvement. The modules can be delivered at central or provincial levels, or through service providers in districts. (RE-TF activity and a BE-TF activity) 31. Design of Capacity Development Program. The CPIU, working with the task team, carried out needs assessments at the district level in the fields of capacity development, public finance management, procurement, education, local governance, planning and information management. They then developed a framework for the drafting of the CDPs based on the strategic education needs identified by the districts. The BEC team prepared an operations manual for districts and an L-BEC manual that were intended for the use of consultants who were to work directly with local governments in a step by step process. 32. All 50 BEC districts conducted kick-off meetings with related education stakeholders, including local government departments, schools, local parliament, local education board and NGOs to explain the BEC technical assistance program. Following the kick-off meetings, the staff of the district program implementation units (DPIU) and the CDP consultants worked at the district level to help develop CDPs, conduct stakeholder meetings to prioritize the CDP programs, and prepare the L-BEC grant proposals. The BEC provincial program implementation units (PPIU) conducted one coordination meeting for the BEC districts in their provinces. Additional CDP activities included: (i) production of a CDP manual to reflect new changes in education management which was published and distributed to participating BEC districts for use in preparing their 3-year CDP plans; (ii) training the CDP consultants on the use of the manual; (iii) conducting stakeholder meetings to review and prioritize elements within the plan; (iv) development of training modules for the CDP; (v) preparation of materials for the training 19DBE1 is USAID supported Decentralized Basic Education (Program 1).

33

modules; and (vi) stakeholder meetings to review and validate the training models before rolling them out. 33. L-BEC Grants and Manual: In order to provide the rules and regulations for the implementation of the L-BEC grants to local districts, a detailed L-BEC manual was developed. However, following the signature of MOUs by districts with MoEC, a new regulation was issued by the Ministry of Finance (Decree 168 and 169) on a new on-granting and on-lending mechanism, which significantly altered the approach to be taken with BEC, and meant that L-BEC grants could not be transferred to districts in 2008 as planned, even though the local governments had already included them in their own DIPA of budget lines. Under the new regulations, which were discussed with MOF, the CPIU clarified the new approach which required that grants are issued not by MoEC but by MOF. Since this approach has not yet been used in other programs, the L-BEC grants were effectively a pilot program, and as such were viewed as important by MOF. 34. The L-BEC manual was developed, including proposal evaluation criteria, funding flow mechanisms, and reporting requirements that comply to new MOF regulations. Workshops with districts on L-BEC grants and use of manuals were conducted to explain the evaluation criteria, how to prepare and submit proposals, and how to request funds and report on the use of funds, using a simplified reporting format which combines reports needed based on the on-granting system and the project reporting system.

35. L-BEC grants proposal preparation, submission and evaluation. Districts prepared and submitted L-BEC grant proposals which were submitted on time to the CPIU. Based on the on-granting regulation, the proposals included an agreed Annual Plan and Comprehensive Plan in the MOF format, which led to some adjustments of the CDP format. The CPIU reviewed and evaluated 50 L-BEC grant proposals using the established criteria, and approved and endorsed all 50 L-BEC grants. All districts received equal grant amounts, since the delays in setting up the on-granting system led to reduced scope for additional performance based grants, and GoI preferred to select a simpler approach. The release of the L-BEC grants in October 2010 was a major breakthrough and moved the project forward. The L-BEC manual was revised in 2011 and distributed to 50 BEC districts and 9 provinces. The revisions included new guidance related to taxation; management of bank account interest; efficiency savings fund use; eligibility requirements for computer purchasing for schools; and, the required documentation needed to receive L-BEC grants.

36. An education public expenditure assessment capacity development module was developed to guide districts in their planning and budgeting processes. These were prepared by in close consultation with the Public Expenditure Analysis Capacity Enhancement (PEACH) team. It provided guidelines to facilitate local government Education Public Expenditure Analysis (LG-EPEA) technical teams to develop, disseminate, and act on recommendations for improvement that emerge from the internally focused analysis of education expenditure. LG-EPEA was identified as a good practice initiative to assist local governments to better understand their planning and budgeting processes and expenditure structures so as to inform future plans. The EPEA report provided stakeholders with information to better understand the forces that affect budget allocation and spending, the correlation between inputs and outputs, identify

34

existing and emerging financial problems, and develop actions to remedy the problems to achieve maximum public benefit. Guidelines were prepared for local governments to conduct their own public expenditure assessments. 37. BOSDA. BOSDA (BOS Daerah) are local government grants based on a formula that was used to address the disparities between the higher costs of school operations in peripheral locations compared with those in the district capitals. Eighteen districts and one province are now using BOSDA, the more equitable formula-based approach (customized according to local criteria) to provide supplementary funding to schools based on access and performance rather than student enrolment figures. The BOSDA team has established that more than 50 percent of Indonesia’s provinces, districts/cities provide some kind of local BOS grant to schools, or BOSDA. BOSDA allocations vary between 0.1 percent and 5.6 percent of APBD, and very often local governments disburse them based on a per capita amount, mimicking the national BOS program. The aim of this activity was to help local governments to explore ways to encourage them to apply equity and performance based BOSDA allocations, to reward performance and encourage pro-poor spending. Formula-based BOSDA can potentially narrow financing inequalities and improve efficiency as well as effectiveness. (BE-TF activity) 38. Costing School Operations Instrument. Piloting of an approach in three districts resulted in the development of an Excel-based BOSDA costing instrument application to facilitate districts in the calculation of school operational costs to inform BOSDA allocations. Findings from the Bank BOSDA review show that schools receiving BOSDA had higher learning achievement, increased access and, spent more on instruction related activities. Pilot districts are establishing BOSDA taskforces and a communication forum to facilitate central government discussions about the need to develop BOSDA policy and implementation requirements. BOSDA relevant information and the Excel-based formula-based BOSDA instrument application can be accessed through WAPIKweb. (BE-TF activity) 39. Sub-Component 2.3: School Based Management. This sub-component was to focus on the delivery of education services at the school level with the support of the local community. This sub-component produced and distributed several activity briefs that provided best practices and positive results from the SBM activities. A list of these publications, that were published in English and Bahasa, are as follows: (BE-TF activity)

i. Gender mainstreaming checklist and practical manual. This program started with a review of the program design and development of recommendations on how to mainstream gender, which was shared with project teams including local and national consultants and service providers and DPIUs. An education update on mainstreaming gender report was published and disseminated to all BEC districts. It incorporated findings from the survey conducted by the gender advisor. These findings also informed the development of the TRIMS tool, which provides districts with gender disaggregated data.

ii. Making BOS Effective under Decentralization. A policy brief which describes the coverage of the BOS program, annual allocations per student, the approval and implementation of the school budgets and the connection of BOS and the SBM Program, and BOS under decentralization.

35

iii. Making Schools Based Management Work. An education update describing the international evidence on School Based Management, decentralization and student learning outcomes.

iv. Enhancing School-Based Management (SBM) in Indonesia. A policy brief outlining decentralization and school-based management, school committees, challenges and achievements of SBM in Indonesia, and recommendations for the further development of SBM.

v. The Indonesia Early Childhood Education and Development (ECED) Project: Findings and Policy Recommendations. A policy brief providing an overview of and policies the ECED sector in Indonesia, describing the Bank-supported ECED project, and making a number of findings and recommendations on the project.

vi. The Imperative for Connected schools in Indonesia. A policy brief outlining the needs of connected schools in Indonesia, what it will take to connect all school to the internet, the financial implications and necessary legal structure for such interconnectivity.

vii. The BOSDA Improvement Program: Enhancing Equity and Performance through Local School Grants. A policy brief describing the move from BOS to BOSDA, the characteristics of provincial and district-based school grants, the BOSDA Improvement program and the results of a pilot program to implement BOSDA

viii. Bantu Sekolahku (Support My School/SuMS). The objective of the Support my School activity is to develop and deploy an online reporting system that allows members of the public and the school/university community, as well as local government and MoEC personnel, to report on critical needs within schools and universities, so that these needs can be systematically recorded and resolved by the education system (including MoEC, provinces, kabupaten, and schools/universities themselves).

40. Comprehensive study on school-based management: RAND was contracted to do the study. Completed in July 2011 for MoEC review it was published in March 2012. Key findings identify: (i) need to expand capacity of Principals, Teachers and SC members about SBM and clarify roles so as to realize SBM associated benefits; (ii) Increase school staff ability to make managerial and instructional changes through professional development to empower implementation of instructional and curriculum changes; (iii) Develop district capacity to support schools and SBM; and (iv) Expand the role of Supervisor to include monitoring SBM implementation, providing ongoing TA and mentoring to School Management Teams. 41. Strengthening of BOS teams at school level and school committees: An $8.5 million DESP grant was made to support training on revisions to the BOS manual and key SBM concepts (with financial management focus) related to BOS funds management. A national program of cascade training took place in 2009 -- School Management Team (1 day) and Community (1/2 day) socialization/awareness-raising: 200,000 schools (600,000 participants: head teachers, school treasurers, school committee members; parents and community). Lessons learned contributed to the design of the 2011 national 3-day training program for School Management Teams. 42. Component 3: Information Management. This component was designed to strengthen systems of information and to develop capacity of key stakeholders to allow for informed decisions on education policy and allocations of resources. The activities that were supported by

36

Component 3 were, monitoring and evaluation (M&E), complaint handling, and the development of school-based reporting and support instruments, and the establishment of a process for providing on best teach practices. All fit squarely within the project activities outlined in the GAD. Some of the activities were developed at the request of the MoEC, others at the suggestion of the Bank task team. Component 3 is rated moderately satisfactory. This rating is based on that the fact that a substantial level of capacity development training and technical assistance for monitoring and evaluation, complaint handling and school-based reporting was provided. While the data is collected by the districts, they still need to improve their capability to analyze and use the collected information to inform decision making at the local level. 43. Sub-component 3.1: Management information systems and Information-Based Decision Making. This sub-component was to focus on the implementation of management information systems as well as the capacity to manage the data, analyse and make informed decisions based on the results. The outputs in this sub-component are described below. 44. Program monitoring and evaluation workshop and framework. BEC-TF supported the development of a project M&E system to ensure that the input was systematically collected based on the defined indicators contained within the Project Operations Manual and L-BEC-grant manual. The CPIU also prepared a monitoring and evaluation instrument and module to be used in monitoring visits to the LGs. (RE-TF activity) 45. Monitoring and Evaluation. In 2009, at project inception, an approach to M&E was developed by the CPIU and the director of junior secondary education that was supported by BEC-TF technical assistance. In 2011, an approach designed to focus on district education program monitoring and evaluation was developed that integrated BOS ME. The BEC-TF team, in consultation with BOS and BEC CPIUs, used the vehicle for the socialization of the BOS M&E to train BEC districts (based on district demand) in this integrated approach to education program M&E. Training module content was aimed at providing step by step guidance on designing and implementing M&E for education programs for District Education Office (DEO) staff and School Supervisors. The 3-day training provided detailed information about how to design and plan district approaches to M&E. This included development of questionnaires, planning for field monitoring, data analysis, reporting and how to use monitoring results for program improvement. (RE-TF activity) 46. Implementation and Results of the Monitoring and Evaluation process. Following the finalization of the education program specific M&E training module, demand driven training was provided to those DEO’s requesting this training. Demand was not high and districts were more focused on achievement in BEC strategic area training. To address this, the CPIU facilitated a targeted session on the M&E system during the April 2011 CPIU hosted BEC national coordination meeting in Jogjakarta. The aim of the session was to increase understanding about the importance of M&E and district obligation with regard to the BOS M&E approach (developed under DESP) as well as in education program service delivery. While the M&E modules were socialized to all districts, utilization is uneven and data results used to inform policy and budgetary decisions is also uneven. (RE-TF activity)

37

47. Complaint Handling. The complaint handling system supports the Law 14/2008 on Public Information and the implementation of Indonesia’s Open Government Action Plan 2012 within MoEC’s goals of promoting transparency, participation and accountability in the implementation of the BOS program. Complaint handling relies on community participation to provide feedback from beneficiaries in the field (community, schools, districts, and provinces, central). It is also a mechanism to support transparency and participation in education service delivery and effective/efficient governance. (BE-TF activity) 48. The system was socialized to all districts and provinces, followed up by more in-depth training at district level base on request. Advocacy for regulatory support to ensure sustainability resulted in a ministerial regulation for BOS complaint handling. This regulation provides a guide for district level teams about: (i) recording, reporting and resolving complaints received about the BOS program and (ii) the ‘online’ and SMS system which ensures that any feedback to questions, suggestions or complaints are handled efficiently, the issue addressed and complainants receive timely feedback. The system regulates the flow of information so that complaints are directed to the ‘right’ people; complaints are addressed/followed-up; action is documented; and responses publicly accessible. This complaint handling mechanism enables all levels of BOS management teams to record, report and respond to any feedback provided by increased community involvement and communication, resolution of problems at district level with results publicly accessible.

49. Implementation of the complaint handling system. The complaint handling system was nationally socialized to 497 districts and 33 provinces in the 5 regional coordination workshops for BOS program hosted by MoEC and have been institutionalized by MoEC which decided that regulatory functions should be added to 2013 BOS guidelines. This not only simplified the process but ensured that complaint handling would be incorporated into BOS procedures. The requirements provide a guide for district level teams about (i) recording, reporting and resolving complaints received about the BOS program; and (ii) the ‘online’ and ‘SMS complaint handling system. 50. All of key objectives for complaint handling were achieved: (i) informal coordination discussions at central level have been held with MoEC’s Central BOS Management Team, Directorate of Pemberdayaan Masyarakat Desa (PMD) of Ministry of Home Affair (MoHA) and PNPM Generasi. A BEC-TF supported workshop in Surabaya focused on complementarities to improve implementation and socialization of the system at all levels; (ii) a compliant handling training module and ‘on-demand’ training; and (iii) discussions are underway to integrate a web-based complaint handling system for the BOS program which links to SuMS – reinforcing systems complementarities. Complaint handling is considered to be highly sustainable since it is part of Law 14/2008 on Public Information and the implementation of Indonesia’s Open Government Action Plan 2012. Moreover, it was developed in close consultation with GoI and is integrated into web-based BOS transparency and accountability systems. Ten districts in five provinces requested follow-on training following the introduction of Complaint Handling and Resolution Standard Operating Procedures, based on those used for BOS during the January Good Practice workshops. Four districts used L-BEC grants to fund the training; six were funded by BEC-TF. 235 staff from the District Education Office, Sub-District Education Office and school principals was trained.

38

51. Sub-component. 3.2 Learning and Knowledge Sharing Systems. The objective of Learning and Knowledge Sharing was to create ways to share the knowledge and learning about good practices and make these accessible to all districts and schools. The activities are summarized below.

52. Tool for Reporting and Information Management by Schools (TRIMS). The TRIMS is an evidence-based education policy instrument, introduced to all schools nationally in 2011 as part of EMIS reforms to support reliable and timely data collection and reporting. The goal was to: (i) strengthen information systems and develop the capacity of key stakeholders to collect, analyze and use the resulting data for informed decisions on education policy and allocation of resources; and (ii) improve governance (transparency and accountability) by increasing availability and accessibility of education data and information. (BE-TF activity)

53. TRIMS uses a standalone Excel-based application designed to link with national data collection systems. 352 District trainers in 23 BEC districts and three non-BEC districts have been trained and 4,993 schools have submitted valid data to districts using TRIMS. Piloting occurred based on demand requests – several districts actively use the tool for information management, others have also conducted training in schools to improve school data collection. Papua Province has developed a five-year roadmap to implement TRIMS. They were supported to modify the tool according to location (isolated, remote, sub-urban and urban zone) to address the diversity of capacity and capability in the province.

54. On demand, training was provided to schools and districts in the form of technical assistance focused on improving organizational and institutional capacity in EMIS for improved evidence based policy making, planning and budgeting. Districts financed training costs using their own budget or L-BEC grant funds allocated specifically by them in their capacity development plans for EMIS activities. This approach was aimed at creating districts awareness about EMIS from the very beginning. The training materials were developed by the Bank team in consultation with stakeholders and based on feedback from pilot activities.

55. From the assessment to piloting TRIMS, the most challenging part has been convincing district and MOEC partners in the District Provincial Implementation Units of the benefits of an application such as TRIMS, designed to complement existing data collection mechanisms, not to replace. TRIMS has reduced the time it takes to collect and report information from between 2 - 6 months to less than two months. This is a substantial achievement. Following pilots, the Directorate SMP agreed to integrate TRIMS into the School Application Package (PAS). After being socialized to all BEC districts on January 2012, various BEC districts requested additional TRIMS training. There are now 23 BEC districts and 4 non-BEC districts that have allocated budget for 2013 implementation, including its work plan, further, they have established an accessible blog forum: forumstrims.blogspot.com, managed by the districts themselves, which provides a way to access, share and discuss TRIMS related issues.

56. WAPIK Program. To WAPIK was to create ways to share knowledge and learning about good practices in teaching practice, education management and leadership and make these accessible to all districts and schools (building on the work of UNICEF’s ‘Mainstreaming Good

39

Practices Program which supported the collection and packaging of materials and information on Good Practices and Good Practice Programs). WAPIK serves as a repository for educational innovation at various levels of education service delivery. WAPIK evolved from the Good Practices and Information Network – GPIN website which was initiated in 2010. The WAPIK is used to enhance access to information to promote quality education by: (i) increasing access to and building within the education community a platform/medium for sharing good practices; (ii) strengthening the culture of good practice through identifying relevant examples, developing/writing case studies for dissemination through the site; and (iii) establishing a commitment to sharing good practices at the local as well as national level and sustaining the website through GoI and Teacher Training Institute Regional Hubs. (BE-TF activity)

57. The WAPIK team has provided ongoing technical assistance for the development of the site and strategies to engage education practitioners to prepare publications for the site about good practice innovations to which they have contributed or introduced: (i) 15 WAPIK training workshops and dissemination activities for education practitioners and partners have been conducted; (ii) 826 participants have been trained in how to access and use the site and to prepare articles for uploading to the site; and (iii) a series of seminars to expose practitioners to the site and its application/use was held for an additional 290 good practice practitioners.

58. Social Marketing and Information Campaign (SMIC). BOS awareness-raising was supported by the comprehensive SMIC survey funded under the DESP and included the following activities: (i) national public service announcements; (ii) district events; and (iii) school-level interventions. These activities were focused on increasing public knowledge of the BOS national program. However, data from the SMIC survey revealed that that these dissemination activities could be further strengthened. Recognizing the need to further increase parent’s knowledge of the BOS program the government launched another socialization campaign and provided districts with posters containing: (i) objectives of the BOS program (ii) the amount of BOS received by all schools (iii) unit costs for students at the primary and junior secondary levels of education; and (iv) description of the roles and responsibilities of school committees and parents as they relate to the effective implementation of the BOS program. (DESP supported activity)

59. Bantu Sekolahku (Support My School, or SuMS). The objective of the Support My School activity was to develop and deploy an online reporting system that allows members of the public and the school/university community, as well as local government and MoEC personnel, to report on critical needs within schools and universities, so that these needs can be systematically recorded and resolved by the education system (including MoEC, provinces, kabupaten, and schools/universities themselves). (BE-TF activity)

60. This activity, a request from the MoEC, commenced when the Bank engaged in a process of sharing with MoEC international best practices in using ICT and public participation to increase transparency and accountability of the educational bureaucracy in responding to schools’ infrastructural and human resource needs. In particular, the Bank team shared country experiences of the ‘Check My School’ Initiative in the Philippines with a view to its possible replication in Indonesia. Since its launch in December 11, 2012, Bantu Sekolahku has been accessed by 4,293 users and over 1,000 unique needs have been reported. MoEC is slowly

40

beginning to use the SuMS. As of February 17, 2013, needs of schools related to reconstruction and new buildings are those most reported (809), while needs related to students and funding (e.g.: BOS and poor students funding) have been reported the least (45). Schools at SD level have reported most needs (342), while SMKs are among the least to report needs. These statistics indicate clearly that a major challenge remains to accelerate the rate of response by MoEC and the wider education system in meeting reported needs. Thus, a critical focus for 2013 will be to deepen awareness about and use of Bantu Sekolahku by education officials.

61. Cancelled Activities. The Teacher Employment and Deployment activities under the Dutch Education Support Program (DESP) were to be scaled up under SISWA so that BEC-TF districts could pilot activities based on lessons learned. With the cancellation of SISWA this activity was cancelled as it fell under another MoEC Directorate of Teacher and Education Personnel. Likewise, the Student Assessment System – Indonesia National Assessment Program (INAP) for Grades 1, 2, 7, 8, 9 and pilot - were cancelled because INAP is not within the Directorate of Primary and Junior Secondary Education’s mandate but it fell under the responsibility of the National Education Assessment Centre (Pusat Penilaian Pendidikan – PUSPENDIK). It was to be developed under BEC-TF within the context of a national SISWA which was canceled.

41

Annex 3. Economic and Financial Analysis 1. The economic and financial analysis for the Basic Education Capacity Trust Fund (BEC-TF) was not carried out at appraisal due to the fact the trust fund’s major investments did not lend themselves to a standard value for money/efficiency analysis. Thus the economic and financial analysis focuses not only on the efficiency of the funded interventions but more importantly, on the potential impact that these activities have had on the changes in education governance and management across the 50 BEC-TF districts between 2009 and 2012. 2. The overall objective of the BEC-TF was to contribute to the GoI’s achievement of the MDGs and EFA goals through good governance in education. The purpose of the program was to support GoI in improving the delivery of decentralized basic education services by local governments and schools in selected program areas, and by extension in other locations. Within this context, the BEC-TF sought to strengthen the planning, budgeting, financial management and accountability at the local level in order to reduce leakages, increase the efficiency of fund usage, and improve the delivery of education services. The BEC-TF project also envisioned developing a sound monitoring and evaluation system capable of providing relevant data to guide and inform the decision-making process across the entire education system. 3. As mentioned previously, the BEC-TF’s capacity building activities and technical assistance supported national education programs such as the BOS-KITA. Since the BEC-TF focused on building capacity at the district level while also supporting national programs which aimed to impact higher order education outcomes, the ICR’s economic and financial analysis provides an assessment of the following: (i) trends in key education indicators and internal efficiency gains since the 2008 launch of the BEC-TF/BOS-KITA; (ii) unit cost analyses for the BEC-TF’s main capacity building activities under components 2 and 3 for which comparable data from other development partners are available; (iii) analysis of the changes in the quality of education governance within the 50 districts targeted by the BEC-TF between 2007 and 2012; and (iv) district sustainability plans following the completion of all BEC-TF supported interventions. Changes in the quality of education governance and management are evaluated based on the results from the Indonesian Local Education Governance Survey (ILEGS: 2009 and 2012)20. The analysis focuses on five strategic areas covered by the BEC-TF project: Education Service Provision Standards, Efficient Resource Use, Transparency and Accountability, Management Control Systems, and Management Information Systems. 4. Positive Trends in Key Education Indicators and Internal Efficiency Gains. The BEC-TF was designed to support the GoI in achieving the MDG and EFA goals through (i) good governance in education; and (ii) improving the delivery of decentralized basic education services by local governments. The BOS21 national program, which received direct financial

20 Through the BEC-TF, a local government capacity assessment (LGCA) tool and an Indonesian Local Education Governance Index (ILEGI) were designed to provide a coherent national monitoring and evaluation system of performance in education governance and service provision within the 50 BEC-TF districts. 21 In 2005, a school grants program, introduced in the wake of the financial crisis in 1997, was expanded and made available to all primary and junior secondary schools. The program currently covers 228 thousand schools and 43 million students. The national school grants program (Bantuan Operasional Sekolah - BOS) has been an important part of the government’s approach to extending services to and providing protection to poor households. The key objectives of the BOS national program are to: (i)

42

support from the Bank’s BOS-KITA project and capacity building and technical assistance from the BEC-TF program, has contributed to the positive gains observed within the education sector between 2007 and 2012. As Table 1 illustrates, net and gross enrollment rates at the primary and junior secondary levels of education increased between 2007 and 2012. The increases in net enrollment rates at the junior secondary level are particularly large within the poorest income quintile (rising from 53.03 percent in 2007 to 63.54 percent in 2012). Significant internal efficiency gains have also been observed over the 2007 to 2012 period, evidenced by: (i) increased completion rates at the primary and junior secondary levels; (ii) increased transition rates from primary school to junior secondary school; (iii) reduced drop-out rates at the primary and junior secondary levels; and (iv) reduced repetition rates at the primary and junior secondary levels.

5. Component 1—Policy Analysis and Dialogue aimed to: (i) develop, synthesize, and disseminate sector knowledge and information; and (ii) facilitate consensus building on sector issues, priorities, and investment strategies. Although it is not possible to quantify the direct

improve access to education by eliminating school fees for poor students and lowering fees for other students; (ii) enhance the quality of education; and (iii) accelerate the achievement of nine years of mandatory schooling.

TABLE 1: Progress in Achieving Higher Order Education Outcomes since the launch of the BOS-KITA/BEC-TF Higher Order Education Indicators Baseline data Most recent

data Change (percentage

points) Net enrollment rate at primary school (SD) level* 2007 2012

National average (%) 91.13 92.52 1.39 Poorest income quintile (%) 90.93 92.52 1.59

Gross enrollment rate at primary school (SD) level)* 2007 2012 National average (%) 96.56 102.96 6.40 Poorest income quintile (%) 97.92 103.92 6.00

Net enrollment rate at junior secondary school (SMP) level* 2007 2012 National average (%) 66.47 69.96 3.49 Poorest income quintile (%) 53.03 63.54 10.51

Completion rate at primary school (SD) level** 2008 2011 National average (%) 96.86 97.06 0.20

Completion rate at junior secondary school (SMP) level** 2007 2011 National average (%) 98.17 98.27 0.10

Transition rate from primary school (SD) to junior secondary school (SMP)** 2008 2011 National average (%) 75.58 77.26 1.68

Drop-out rates** 2007 2011 Primary school (SD) level (%) 2.37 1.61 -0.76 Junior secondary school (SMP) level (%) 2.88 1.80 -1.08

Repetition rates** 2007 2010 Primary school (SD) level (%) 3.81 3.58 -0.23 Junior secondary school (SMP) level (%) 0.53 0.35 -0.18

Source and notes: World Bank staff calculations. *Calculated figures are based on the March round of SUSENAS. figures are based on MoEC data.

43

economic benefits generated by the outputs produced under this component, the analytical studies carried out under the BEC-TF project informed the GoI’s RENSTRA and provided the government with guidance on a number of strategic areas (education sector financing, teacher management and upgrading, school-based management, skills development for increased competitiveness, etc.). The studies commissioned under this component of the project have also provided the GoI with a sound analytical basis to define and launch capacity building interventions at the district level, while the Bank’s technical assistance supported the development of financial management and data collection instruments currently being used by local governments across the 50 BEC-TF supported districts. The BEC-TF project has supported the development of monitoring and evaluations systems capable of providing relevant data to guide and inform the decision-making process across the education system. Although it is not possible to quantify the economic benefit generated by these studies, the BEC-TF has been instrumental in providing the evidenced-based research needed to inform the government’s efforts at increasing the efficiency and equity of education services. 6. Unit Costs of Capacity Building Activities funded by BEC-TF22. Component 2—Good Governance, Financial Management, Budgeting and Accountability sought to improve governance and efficient resource use by providing targeted training in financial management. The BEC-TF project provided financial management, good governance and accountability training to 50,105 local education staff at an average cost of US$62 per person23. The unit costs associated with this capacity building activity are comparable to those charged by USAID, AusAID, and UNICEF for a similar intervention. The project also trained 13,274 local education staff in performance-based budgeting and planning at an average cost of US$65 per beneficiary24. Again, the unit costs for this three-day activity are in line with the fees charged by USAID, AusAID, and UNICEF for an equivalent training program25. Under Component 3—Information Management, 2,422 education staff received a three-day course on the use of financial data to improve the efficiency of fund allocation/use at an average cost of US$82 per person26. Although no comparable financial data from other donor-funded activities are available for this particular activity, feedback from the districts suggested the training provided was highly relevant and comprehensive. 7. Changes in the Quality of Education Governance. The Indonesian Local Education Governance Survey (ILEGS) is the most comprehensive and robust data source to compare changes in the quality of education governance within the 50 districts targeted by the BEC-TF. The baseline and end-line surveys were conducted in 2009 and 2012, respectively. Key findings from each of the five strategic pillars (Education Service Provision Standards, Efficient Resource

22 Based on the availability of financial data, the analysis covers the main capacity building activities under Component 2 (Good Governance, Financial Management, Budgeting and Accountability) and Component 3 (Information Management) of the BEC-TF project. 23 BEC-TF Financial Activity Report (April, 2013). Unit costs per person were converted to US dollars using the official exchange rate (US$ 1 = IDR 9,715). 24 Ibid. 25 The unit cost of capacity building activities tend to be relatively uniform across the various development agencies operating in Indonesia because the government typically establishes a price ceiling (or “standard price”) which must be complied with by all development agencies. 26 BEC-TF Financial Activity Report (April, 2013). Unit costs per person were converted to US dollars using the official exchange rate (US$ 1 = IDR 9,715).

44

Use, Transparency and Accountability, Management Control Systems, and Management Information Systems) included in the ILEGS are presented below. It is not possible to directly attribute the changes in education governance and management to the BEC-TF program. This is primarily because the system-wide changes observed within the education sector could also be the result of other capacity building activities supported by the GoI and/or other development partners during the same period. Nevertheless, the ILEGS represents the most systematic assessment of the changes observed in education governance and management within the 50 BEC-TF districts. 8. Education Service Provision Standards. Education service provision standards within the 50 districts receiving BEC-TF support have improved from 2009 to 2012. This has been the result of general improvements in access as well as the quality of key personnel in the education system. For example, the proportion of supervisors, school principals and teachers with a four-year university degree has increased over the three years. This is largely the result of the continued implementation of the national teacher certification program. The program has motivated teachers to obtain a university degree in order to be certified and obtain a professional allowance equivalent to their basic pay.

9. Efficient Resource Use. The BEC-TF capacity development support in the areas of good governance, financial management, budgeting and accountability have contributed to improvements in the efficiency of fund usage among the 50 targeted districts. Over the life of the project, local governments have registered important gains in the processes which determine how public resources are used. Results from the Indonesian Local Education Governance Survey (ILEGI) indicate that the number of district education offices providing progress reports on their activity plans have increased during the project. Improvements in the local planning and budgeting processes are also been observed. More specifically, the proportion of districts that set budget priorities and ceilings before sector offices (e.g. the education office) start their own planning exercises increased from 44 to 74 percent between 2009 and 2012. The number of BEC-TF supported districts with improved education budget execution rates have also increased from 31 to 44 over the life of the project. While some of these efficiency gains may have been the result of the central government efforts at introducing medium-term expenditure frameworks27, the BEC-TF program’s capacity building activities related to performance-based budgeting, planning, and execution have also been important in this process. 10. Reduced Financial Leakage. Prior to the launch of the BEC-TF and BOS-KITA projects in 2008, the first Governance and Decentralization Survey (GDS1) found that approximately 70 percent of funds allocated by all levels of Government reached the school. By 2012, nearly 90 percent of the total funds disbursed by all levels of Government were received by schools28. These lower levels of financial leakage within the education system are in part a result of the Government of Indonesia’ overall efforts at strengthening external and internal audit systems, as well as the BOS-KITA/BEC-TF’s increased focus on improving financial 27 Efforts to ensure better local resource management were embodied in the 2003 state finance law that started implementation in 2006. In particular, the law introduced medium term expenditure frameworks into local planning and budgeting. This represented a shift from earlier annual input based budgeting to a medium term planning horizon with a focus on the expected outputs of planning and budgeting decisions. 28 Regional Independent Monitoring Survey, 2012.

45

management, transparency and accountability at the local level. The BOS Operations Manual—which was updated and revised throughout the life of the BOS-KITA project—played a key role in minimizing the misuse of funds at the school level by: (i) clearly stipulating eligible and ineligible expenditures categories; (ii) strengthening financial disclosure and reporting requirements; and (iii) mandating community oversight of fund usage. It is also important to highlight that all BEC-TF Audit Reports were Unqualified, providing further evidence that no systemic financial issues were identified within the 50 BEC-TF districts. 11. Transparency and Accountability. Improvements in transparency and accountability across the 50 BEC-TF districts are also evident between 2009 and 2012. Improvements are seen most clearly in efforts by local governments to encourage greater community participation in decision making and oversight activities. Between 2009 and 2012 the proportion of districts that allowed public participation in parliamentary accountability and audit reporting sessions increased from 14 percent to 52 percent. Similar improvements in access were seen in the number of districts allowing public access to budget sessions in local parliaments. Improvements were also seen in the reporting procedures of local education offices. In 2012, nearly all of the district education officers were producing and making available progress reports on planned activities and budget disbursements.

12. Management Control Systems. While service standards and measures of transparency and accountability have been improving within the 50 districts receiving BEC-TF support, the management control systems component of the ILEGI has declined. Efforts by local governments to systematically document and catalogue good practice seem to have weakened between 2009 and 2012. In 2009, two-thirds of districts made efforts to identify and document good practice, whereas in 2012 less than half of all participating districts systematically carried out this activity. This is of concern given the importance that local solutions play in improving the quality of education and the potential benefit of disseminating best practices across the districts supported by the BEC-TF project.

13. Management Information Systems. Important gains have also been observed in the process of collecting, managing, and analyzing data at the local level between 2009 and 2012. The School Application Package (PAS) and the Tool for Reporting and Information Management (TRIMS)29 have provided schools and local education offices with a template to collect and store relevant data which can subsequently be used to inform and guide the decision-making process at the local level. While the data collection process has improved during the life of the BEC-TF project, additional support must be provided to districts in order to ensure that the gathered information is used to inform the strategic choice of interventions pursued at the local level.

29 The Tool for Reporting and Information Management by Schools (TRIMS)—an evidence-based instrument developed under the BEC-TF program—was introduced to all schools nationally in 2011. TRIMS aims to: (i) strengthen information systems and develop the capacity of key stakeholders to collect, analyze and use the resulting data for informed decisions on education policy and allocation of resources; and (ii) improve governance (transparency and accountability) by increasing availability and accessibility of education data and information.

46

14. Financial Sustainability of BEC-TF supported Activities. By project closing, 40 out of the 50 districts targeted by the BEC-TF project had developed post-project sustainability plans focused on improving the delivery of decentralized basic education services and improving student learning outcomes. The majority of districts have mobilized funds to continue and strengthen the following interventions/activities: (i) EMIS, including the BEC-TF supported TRIMS; (ii) Education Service Provision Standards, including teacher upgrading and competency-based curriculum development; (iii) Financial Management Systems, including performance-based budgeting and planning, asset management, efficient resource use and procurement policies, procedures and controls; (iv) School-Based Management Training, including school committee development and community involvement programs; and (v) Reform to District Education Policies and Regulations, including strengthening the BOSDA Improvement Program30 and creating incentive schemes to encourage the deployment of qualified teachers to underserved provinces/districts in Indonesia.

30 The BOSDA Improvement Program was initially piloted in two provinces and 12 districts. The main objective of the pilot was to support regional governments in their efforts to narrow financing inequalities and improve both the efficiency and effectiveness of education spending. The pilot focused on BOSDA allocations and aimed to: (i) improve current BOSDA allocation criteria to address financing inequalities between schools; and (ii) create incentives for better school performance and encourage more effective, transparent, and participatory school-based management involving all education stakeholders.

47

Annex 4. Grant Preparation and Implementation Support/Supervision Processes (a) Task Team members

Names Title Unit Responsibility/ Specialty

Lending/Grant Preparation Vic Paquao Country Sector Coordinator

EASHE

Mae Chu Chang Education Cluster Leader Steen Anderson Operations Consultant Andy Ragatz Operations Officer Rosfita Roesli Operations Officer Prima Setiawan Operations Officer Ratna Kesuma Operations Officer Susiana Iskandar Sr. Operations Officer

Supervision/ICR Josephine M. Bassinette Manager EACIF Yogana Prasta Operations Adviser EACIF Jana Uno Operations Officer EACIF Budi Permana Procurement Analyst EASR1 Paulus Bagus Tjahjanto Procurement Specialist EAPPR Egi Sutjiati Senior Operations Officer EACIF

Unggul Surprayitno Senior Financial Management Specialist EASFM

I Gusti Ngurah Wijaya Kusuma Financial Management Analyst EASFM Luis Benveniste Sector Manager EASHE Ratna Kesuma Senior Operations Officer EASHE Task Team Leader Yvonne Trethewey Consultant EASHD Benedicta Ridati Sembodo Program Assistant EACIF Samer Al-Samarrai Senior Education Economist EASHE Imam Setiawan Consultant EASHE Prima Setiawan Consultant EASHE Sheila Town Operations Officer EASHE Gedsiri Suhartono E T Consultant EASHE Muhammad Farman Izhar E T consultant EASHE Nur Hidayat Operations Officer EASHE Irene E. Bouwmeester Senior Program Assistant EACIF Wisnu Boediono Consultant EASHE Santoso Consultant EASHE Halsey Beemer Consultant EASHE ICR Author Bernardo da Cruz Vasconcellos Consultant EASHE ICR Author

48

(b) Staff Time and Cost*

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending

Total: N/A N/A

Supervision/ICR

Total: N/A N/A * This activity was mainly funded by a trust fund (TF091264) and no Bank Budget was allocated. Over the life of the project, 647.73 staff weeks were registered and US$1.16 million expensed.

49

Annex 5. Beneficiary Survey Results Not Applicable

50

Annex 6. Stakeholder Workshop Report and Results Not Applicable

51

Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR I. Operation’s Objective

1. The main goal of the Basic Education Capacity Program (BEC-TF) is to support the Government of Indonesia to improve the overall quality of education as set out in the National Education Strategic Plan through three main priorities: improving access, quality and governance of education. 2. In particular the BEC-TF aims to support the government in improving major issues such as: (a) the lack of transparency and accountability in the financial management system; (b) the development of a medium-term framework based financial management; (c) education reform that includes management, planning, and financial management education; (d) implementation of quality basic education; and (e) review of the Regional Education Funding Allocation. 3. The expected output of the program includes:

i. Strengthening the capacity of the thematic dialogue and education stakeholders and other institutions related to education in developing and analyzing policies related to the development and implementation of basic education.

ii. Strengthening the management efficiency and resource utilization through the implementation of transparency and accountability, improving the quality of the education budget allocation process based on performance, improved financial management and accounting, particularly at local government level.

iii. Strengthening the capacity of the management information systems and performance evaluation of existing systems to increase effectiveness in supporting the accuracy and precision of the data and information available to all stakeholders at all levels/tiers.

II. Design 4. There are three components of the BEC program:

- Component 1: Policy Analysis and Dialogue (under the World Bank executed project) - Component 2: Good Governance, Financial Management, Budgeting and Accountability - Component 3: Management of Information.

5. Components 2 and 3 are managed by the Government of Indonesia through the Ministry of Education and Culture’s Directorate of Junior Secondary Education. 6. The GoI-executed BEC program consists of four types of activities:

i. Capacity Building. The program included capacity strengthening activities for targeted grant recipients at district level in education governance and management. This capacity building has taken place through various training activities and workshops, regular coordination meetings and support activities, study visits to good practice districts, international seminars and training, and other related activities.

52

ii. Local Government (L-BEC) Grants. Grants provided to districts to fund the implementation of activities identified in district Capacity Development Plans (CDP).

iii. Facilitation of the Good Practices Pilot Program. The program was intended to facilitate the implementation of a pilot program focused on good practices in several districts and selected schools.

iv. Thematic Dialogue Education at the Central level. The program was linked to supporting Component 1 activities under the World Bank executed project focused on the implementation of thematic dialogue on education policy at central level.

v. L-BEC grants to districts were designed to be used in three ways as funding for (i) Capacity enhancement; (ii) Good practice piloting; and (iii) Knowledge sharing activities.

7. Capacity Enhancement Grants. These grants were awarded to 50 targeted districts to fund strategically identified local capacity building activities in each district listed in CDPs which were developed based on strategic district plans (RENSTRA). In project design, the amount of grant given to each district was Rp 2.5 billion rupiah over a three year period. 8. Pilot Program for Good Practices Grants. Pilot program grants were awarded to districts that successfully implemented the educational capacity building activities stated in CDPs. These grants were awarded to districts and school selected as an incentive to recognize the success achieved. The amount of grants awarded varied according to needs identified by each district. The initial design was focused on good practices linked to BEC’s five strategic programs which were closely related to the project’s Key Performance Indicators (KPI) which aimed to strengthen the programs developed by the district. Some of the selected programs focused on Performance Based Budgeting, Asset management and Complaint Handling. The program took place in six districts and 30 schools. At Central level, the CPIU designed the approach used to support these districts during several workshops convened to design and develop the good practice programs. 9. Knowledge Sharing Grants. These grants were awarded to five government institutions at the central, provincial and district. Several workshops were conducted to develop the design approach, implementation mechanisms and strategies. Knowledge sharing is an important component in the implementation of the BEC program so that knowledge about educational programs that have been successfully implemented by district is shared and districts encouraged to network. 10. BEC-TF District Targeting and Selection. The selection of the 50 participating districts in the project was targeted. This occurred in two phases: Phase 1 involved statistical analysis of 434 districts using criteria related to: Transition Rates at Primary and Junior Secondary levels, National Examination scores, Poverty Index, percent budget allocated by the District for the Education Fund and Education Budget (excluding salaries). Districts were ranked based on this analysis into three classifications -- Low capacity (red); Moderate capacity (yellow); and High capacity (green). 77 districts were short-listed and identified as potential beneficiaries to participate in the BEC project. 11. Phase 2 involved the final selection of 50 districts from the short-list of 77. Criteria for selection related to: District commitment to provide local education budget from the total district budget (the percentage of the education budget, non-salary allocations, the allocation of:

53

education funds, funds for school construction and rehabilitation, funds for the welfare of teachers, fund for the professional development to improve teacher quality, local government education regulations (number of existing regulations and the quality of regulations) and the scope and quality of the education program. Using these criteria, districts were re-categorized as red, yellow or green. 11 districts were categorized as red (low), 20 as yellow (moderate) and 19 as green (high).

III. Implementation 12. Implementing Organization. The Implementing agency in the Ministry of Education and Culture was the BEC-TF program was the Secondary Education Directorate under the Director General of Basic Education. A Central Project Implementation Unit (CPIU) was established to manage all aspects of project implementation. The CPIU was responsible for the coordination and consultation with the Program Steering Committee whose members included Echelon 1 and Program Coordination Management Unit members from Echelon 2 in the Central Planning Agency (Bappenas), Ministry of Finance, Ministry of Religious Affairs and the Ministry of Education and Culture. 13. Role of CPIU. CPIU’s role and function focused on coordination, facilitation, mentoring and problem solving during project and district preparation, implementation, evaluation and reporting, and, coordination between line ministries and the World Bank, provincial and district. Facilitation included the development of guidelines, manuals and modules related to the implementation of activities, facilitation action plan preparation, verification of documents, disbursement of grants and preparation of reports. The CPIU also provided assistance and acted as resource persons to districts during the delays associated with the new On-granting fund mechanism and the implementation of the capacity building activities. The CPIU also conducted monitoring and evaluation activities up to 1-3 times each year. 14. At district level, the District Education Office established a District Program Implementation Unit (DPIU). 15. Role of DPIU. The DPIU was responsible for coordination, implementation and consultation with the District Steering Committee whose members included the district planning agency (Bappeda), Finance Office (DP2KAD), Ministry of Religious Affairs and the District Education Office. It had a role in implementing and monitoring CDPs and ongoing coordination with CPIU, MOF, District Steering Committee and coordination with related district units such as Bappeda, DP2KAD, Ministry of Religious Affairs, etc.). 16. Development of Modules and Guidelines. Education governance and modules and BEC guidelines were developed in close consultation with the World Bank team. The materials were developed through a comprehensive process of consultations with stakeholders through workshops. These included:

- Capacity Development Planning Guidelines - Education Public Expenditure Analysis Training Module - L-BEC Manual Guidelines

54

- Five strategic Area Training Modules: Transparency and Accountability, Management Control Systems, Efficient Resource Use, Management Information Systems, Education Service Provision Standards

- Asset Management Guidelines - Performance Based Budgeting Guidelines - Monitoring and Evaluation Guidelines - Gender Training Module.

17. All BEC districts participated in training conducted by CPIU and World Bank BEC-TF teams. Initial training in CDP planning, EPEA, the five strategic areas and other modules was supported by Service Providers recruited during the 18 month delay to L-BEC funds being released. 18. L-BEC Grant. The total L-BEC grant to each district was for a total of Rp 2.5 billion. These funds were earmarked to implement capacity building activities identified by districts over the expected three years of implementation (stipulated CDPs). At the commencement of the project, the 50 districts signed a Grant Agreement (NPPH) between the Director General of Fiscal Balance, MOF with each district Mayor in November 2009. At the commencement of the project, On-Granting Agreement (NPPH) was signed between the Minister of Finance c.q. Director General of Fiscal Balance, MOF and each district Mayor in November 2009. 19. Given the range of capacity identified during district selection, each district had differing capability to implement activities which impacted on grant absorption capacity. The total L-BEC grant for the 50 districts was Rp 125 billion. Over the three years (2010 – 2012) the total amount in grants disbursed by MOF to district was Rp. 113,312,158,317 (90.65 percent of the total allocated). The total amount absorbed by districts was Rp. 111,714,407,976 (89.37 percent of the total disbursed by MOF). 20. Grant Mechanism. At design, grant disbursement was to be under the Ministry of National Education’s Block Grant mechanism. This mechanism was superseded by the Minister of Finance’s Regulation No. 168/PMK.07/2008 on regional grants and PMK No. 169/PMK.07/2008 which specified new requirements administered through the MOF about procedures for distributing grants to local governments. Based on districts’ proposal approved by Minister of Education and Culture at first, MOF verified completeness and validity of attached document submitted by districts before giving approval to disburse grant to districts. MOF disbursed grant to districts in advance before implementation of activities conducted by districts. However, during final stage of grant disbursement, grant transferred to districts through pre-financing mechanism where districts financed their activities first before reimbursed by MOF. The grant mechanism to districts is detailed in the flow chart below.

55

21. Counterpart Funds. In the implementation of activities the CPIU and districts provided matching funds sourced from national and regional budgets to aid the implementation of operational activities at district level. The amount of matching funds provided by CPIU amounted to Rp 24,461,404,000. These funds were used for coordination, facilitation, study visits, mentoring, monitoring and evaluation. The amount of matching funds provided by each district varied according to the budget capacity of each district – this ranged between 2 percent – 25 percent from the total L-BEC budget. The total value amounted to Rp 15,717,413,443 over the three years of the project. 22. Implementation Strategy to build Basic Education Capacity. The strategy designed to build district capacity in education service delivery included CPIU support to empower districts to conduct and sustain ongoing capacity building activities. Capacity was built through a series of capacity building activities such as: CPIU convened training and workshops; training conducted by other relevant, locally identified training institutions; comparative studies about education programs successfully developed and implemented in other countries and districts. As well, districts were supported to source resource persons to provide additional technical advisory expert support to facilitate the implementation of CDP activities. The content of some of these training and support activities were focused on education governance, management, and improvement in education service delivery. 23. Capacity Development Plans (CDPs). The implementation of capacity building activities in districts commenced with the development of CDPs which were prepared by each district during a consultative process which involved relevant stakeholders (District Secretary, Mayor’s Office, BAPPEDA, Local Parliament, DINAS, Representatives of the Ministry of Religious Affairs, Education Council, Board of Education, School representatives). The CDP developed reflected the analysis undertaken to identify the specific problems faced by each district and their priority needs. Each plan was approved by the District Mayor and supported

56

district strategic plans (which consisted of the district 3 year comprehensive plan and annual plans). 24. Capacity Development Activities districts. The ten highest priority activities identified by districts in their CDPs and implemented during the project were in the form of training, workshops and technical guidance. These are listed below: (1) Education Public Expenditure Analysis (2) Asset Management (3) Developed School Planing and Budgeting (4) Financial Management (5) Education Data Collection on district and School level (6) School-Based Management (7) Stengthening on Curriculum Development (8) Local Regulation/District regulation/Major Regulation on Education (9) Training for Teachers (10) Study Visits to good practice districts: Gorantalo, Jembrana, Sleman. 25. Evaluation of the implementation of these activities, as linked to the five strategic areas of education governance and management, showed that the highest percentage of activities related to Education Management Control Systems (29.9 percent) with Transparency and Accountability (28.9 percent) a close second. Education Service Provision Standards (17.2 percent), Management Information Systems (14.5 percent) and Efficient Resource Use (9.5 percent). 26. District Achievements. Achievements are quite diverse and vary according to the number and type of activities carried out by each district and district contextual considerations (location, date of establishment, economy, etc.). Notable achievements include improved understanding and expertise in financial management at DINAS as well as school levels; Improved understanding and skills in planning and budget documents and reports at both levels; Improved school asset management; Development of the School Planning and Budgeting plans (RKS and RKAS); Development of Education Law and Decrees at district level; Improved documentation and data collection at school level. 27. Grant for Piloting. The Grant pilot program was planned to be given to the district as an incentive for the achievement of the performance of the implementation of capacity building activities. With the 18 month delay to the project, this was cancelled, as there remained only limited time to fully implement district CDPs. Following Management Committee consultations, it was agreed by the Donor, MOEC and the World Bank to cancel this activity. 28. Grant knowledge Sharing. Knowledge sharing grants were planned to be given to Districts that established a good practice learning resource center. This was also cancelled due to the constraints mentioned in 10 above. The Good Practices Information Network developed through a UNICEF initiative had also been expanded under the World Bank executed Trust Fund. This provided a mechanism to collect and share good practices in education with education stakeholders through the WAPIK website. The University of Surabaya was engaged to

57

support the establishment of regional centers to sustain and continuously improve WAPIK through teacher training institutes. As well, the Directorate of Junior Secondary Education has plans to integrate relevant parts of WAPIK into planned good practice initiatives within MOEC. 29. Grant Disbursement. From the total allocation grant provided by donor, CPIU was not able to spend all budget allocated both in total also in each category due to the delay of L-BEC Implementation and no procourement for service provider. The highest procentages of grant disbursement was spent under category L-BEC Grant (12.452.210.84 USD) and the lowest was under category Goods (103.347,40 USD). The disbursement by category is detailed in the Table below.

30. Monitoring and Evaluation. Monitoring and evaluation of the implementation of the grant program L-BEC to the district occurred up to three times each year. M & E teams included representatives from MOEC and MOF. MORA was also invited to participate. Review activities highlighted successes and ongoing challenges and constraints for districts. Review teams provided ongoing support to the district during and in follow-up to monitoring activities. 31. Key Performance Indicator (KPI). Based on the evaluation and data collection from 50 districts done by CPIU and WB, most of the districts have been achieved the KPI both for general indicator and specific indicator. The minimum achievement of the districts was 27 percent between the target and the actual and the maximum was achieved 100 percent. 32. Constraints and Issues. The project faced a number of implementation problems which resulted in significant implementation challenges through the life of the project. These include:

- Grant mechanism changes: Due to the delays associated with the new On-granting Regulation (No. PMK. No. 168 and PMK. 169) Districts did not receive grants until the project half-way point in October 2010. The new regulation meant that grant funds were transferred from the MOEC DIPA to the MOF DIPA. This involved the establishment of new standard operating procedures in MOF which were determined by three Directorates General: Budget, Fiscal Balance and Treasury.

- Districts especially district treasuries have lack of understanding on on-granting mechanism even though awareness activities already conducted by MOF many times. At that time, they only acknowledged and used to understand that grant transferred from

ORIGINAL

EQUIV.

(EUR) (EUR) (USD) (USD) (USD) (USD) (USD) (USD) (USD) (USD)

1 Consultants Services 11.900.000 2.666.000 3.465.800 13.096,86 297.320,32 1.011.442,15 87.385,14 121.657,45 1.530.901,92 1.934.898,08

a. Consultants Services for Cap. Dev 10.558.000 2.173.000 2.824.900 183.617,80 833.129,72 55.841,06 1.072.588,58 1.752.311,42

b. Consultants Services 1.342.000 493.000 640.900 13.096,86 113.702,52 178.312,43 87.385,14 65.816,39 458.313,34 182.586,66

2 Goods (Equipment and Supplies) 1.090.000 89.000 115.700 72.491,92 10.281,99 20.573,49 103.347,40 12.352,60

3 L-BEC Grants 9.850.000 10.835.000 14.085.500 2.751.254,86 5.215.860,96 4.485.095,02 12.452.210,84 1.633.289,16

4 Workshops, studies, surveys, training 2.003.000 2.403.000 3.123.900 319.894,55 1.058.548,50 1.259.390,47 82.299,25 393.450,27 3.113.583,04 10.316,96

5 Incremental Operating Costs 775.000 925.000 1.202.500 122.802,46 699.007,14 48.951,09 55.525,85 926.286,54 276.213,46

25.618.000 16.918.000 21.993.400 405.483,33 1.478.671,28 5.731.376,61 5.455.069,93 5.055.728,59 18.126.329,74 3.867.070,26TOTAL

AMANDMENT

DISBURSED

No CATEGORY

ALLOCATION

TOTAL BalanceFY 2008 FY 2009 FY 2010 FY 2011 FY 2012

58

central government to districts determined by Minister of Finance Decree (PMK) not through On-Granting Agreement. L-BEC grant was the first grant project using on-granting mechanism. Therefore, district treasuries were hard to include L-BEC grant into local budget. Moreover, even though some template of documents required applying for transfer of grant obviously attached in PMK 169 they still got confused. It made districts often failed to fulfill correctness and completeness of document needed. These circumstances made grant disbursement to districts took much longer time.

- Ministry of Finance requirements were not well known in the districts. For example, the mechanism to apply for the transfer of grant funds from the MOF budget to the District budget was not well socialized. This meant that the District Treasury may have not included the grant into district DIPAs and at central level the approval of the annual L-BEC grant budget (DIPA-HPD) was posponed/delayed. In real terms, activity implementation was severely limited to around 17 months: Year 2010: 3 months, Year 2011: 6 months and Year 2012: 8 months.

- MOF documentation requirements for grant withdrawal and disbursement was generally perceived by districts as too complicated and difficult to complete. This was an iterative procedure which had to be completed at the end of each cycle of fund use before the next tranche of funds could be disbursed.

- The process of the submission, verification and disbursement of grants ranged from 1 week up to 3 months depending on the completeness of the documents submitted by the district. This meant that significant, additional CPIU support was needed to facilitate the districts to meet MOF requirements.

- Districts reported that there was lack of coordination between the DPIU and the Treasury Office (DP2KAD). This resulted in incomplete documentation as both did not have a comprehensive understanding of the MOF requirements.

- Lack of direct assistance by the CPIU to support districts to implement CDPs. Service Providers were initially planned to support districts with the technical advisory help needed to fulfill CDPs. When grant funds were released Service Providers were not procured.

- High staff turnover within DPIUs (and districts) meant that during implementation capacity building activities had to be replicated up to 2-5 times.

- The lack of resource personnel with the necessary technical expertise in education management and governance at district level meant that districts did not have the level of technical support to fully implement CDPs.

33. Sustainability Program. CPIU have allocated 2013 budget to support district commitment described in Sustainability Plans. All BEC districts have developed sustainability plans for relevant BEC activities and have allocated budget in draft budget planes in 2013 and beyond. IV. Operational Experiences

34. From design to completion, the BEC project was subjected to approximately 12 different changes from 2008 - 2012. These include:

59

35. Changes in BEC-TF Program Design. The design of the BEC-TF, sponsored by the European Commission and the Dutch government with the World Bank as Trustee was to initiate an Education Sector Wide Approach through a Systems Improvement through a Sector Wide Approach initiative called SISWA. During inception, the SISWA was cancelled and the GoI-executed project changed to one focused on Good Governance, Financial Management, Budgeting and Accountability and improved local government service delivery. This change influenced district perception and loss of confidence in the program.

36. Changed Program Implementation Approach from financial management to education governance and management. The design approach was not explicitly linked to education at district level which was reflected in the Key Performance Indicators in the program document. To ensure a focus on education service delivery improvement within the scope of responsibility of MOEC and the DINAS the KPIs were adjusted. This meant that the original approach which was based on a 5 pillars of financial management was modified so that they became 5 strategic areas of education governance and management. This resulted in significant confusion at district level where DPIU program managers had been socialized into the 5 pillar approach which influenced the activities identified in CDPs. 37. Changes in District Categories Education Capacity into "Red-Yellow-Green". Targeted district selection involved categorization by CPIU based on existing education statistical data and used as the basis for BEC TF intervention activities -- preliminary results based on statistical data were Red: 11 districts, Yellow: 20 districts and Green: 19 districts. As part of project design, a Local Governance Capacity Assessment Survey was also planned to provide district classifications according to the five strategic areas of education governance and management. This survey (which was conducted after project implementation) provided a different result as it was based on the 5 strategic areas -- Red: 20 districts, Yellow: 27 districts and Green: 3 districts. This caused some confusion and disappointment by districts previously categorized under a different mechanism and required additional relationship management support as a result to manage expectations. 38. Changes in GoI Fund Channeling Mechanisms. The initial design occurred at a time when MONE were able to disburse grants to district governments under an ‘old’ block grant mechanism. This was superseded by a new regulation which was administered by MOF (PMK No. 168 on Regional Grants and PMK No.169 on Grant Distribution procedures). The change resulted in an almost 18 month span of time before standard operating procedures were in place which enabled the grants to be disbursed to districts through MOF procedural requirements. BEC-TF became the very first project to be implemented with On-Granting mechanism and lesson to be learned for the new on-granting procedure (which has since been modified based on the lessons learned in the BEC-TF). This further exacerbated loss of confidence in the project by districts and resulted in major input by CPIU and World Bank teams to support districts that had used some of their own budget to implement activities on the basis of expected L-BEC funds. 39. Changes in L-BEC Grant Agreement. At the outset, the legality of awarding grants to local districts conducted by the signing of a Memorandum of Understanding (MOU between the Directorate General of Primary Education, Ministry of Education and Culture at Regent / Mayor and Chairman of the Parliament as a witness). Due to changes in the use of regional

60

grants mechanism, it was necessary to replace the legal document using a on-granting Agreement (NPPH) between the Minister of Finance c.q. Director General of Fiscal Balance, MOF with the Regent / Mayor. 40. During implementation, MOF had to facilitate two addenda to the MOU (NPPH). One related to the cancellation of grant penalties and one the second relating to Pre-financing during the final stages of the project.

41. Changes in L-BEC Grant Manual. In order to facilitate the implementation of the use of grant funds at the district, the CPIU developed guidelines about grant use. This was approved by the World Bank in a No Objection Letter (NOL). The L-BEC Manual became the operations manual for districts for the project. Guidelines were adjusted four times during implementation due to issues that arose and various policy changes. The changes related to the On-granting mechanism; changes from 5 pillars to 5 strategic areas; taxation; re-payments/refunds associated with audit findings through BPKP; and Bank Interest on L-BEC grant funds.

42. Changes in District Technical Assistance. In order to achieve the performance of the BEC program, the implementation of the annual plan and the comprehensive plan detailed in the CDP, CPIU planned to provide guidance to the respective district by providing a consultant/service provider over the 3 year project implementation period (2010, 2011 and 2012). However, this did only took place in 2010 when CPIU sent Consultants to the districts to train them in the 5 strategic area modules over a three month period from October to December 2010. In 2011 and 2012 there was no assistance provided through Service Providers due the lengthy mechanism for procurement of service providers. Districts were encouraged by CPIU and World Bank teams to hire local experts as resource personnel.

43. Staffing changes: central and DPIU staff rotations and promotions. During the life of the project due to normal staff rotations and promotions there were several changes:

(i) The CPIU Program Manager position was rotated during the second half of project implementation.

(ii) DPIU staff rotations and promotions in most districts (up to 2 – 5 occurrences in each district) This resulted in increased workload for CPIU staff as new staff required induction into the BEC project and training and capacity building necessary to implement CDPs developed prior to their commencement.

44. Change of CDP Activities. Three year CDP activities were designed well before grant funds were available. Under the conditions of the grant there was only limited flexibility to adjust the activities to meet current district priorities. However, any changes to the CDP could not take place without prior approval from CPIU which reduced the flexibility of the BEC-TF approach which was meant to be responsive to emerging needs.

45. Changes in Project Budget Allocations by Category. Finance categories in the Project’s Administrative Agreement between the donors and the World Bank (as Trustees) designated the Finance Categories under which BEC-TF resources could be allocated. A budget breakdown by cost category was specified. Any re-allocations across categories required the Bank to consult with Donors when proposed reallocations exceeded, cumulatively: (a) 10 percent

61

of the amount of the Grant Funds allocated to any of the categories of consultants’ services, L-BEC grants; or (b) 20 percent of the amount of Grant Funds allocated to any of the categories of studies, surveys, workshops and in-country training; overseas training and study tours.

46. In practice, from the GoI perspective in the context of the challenges associated with the 18 month L-BEC grant delay this resulted in difficulties when project activities were advanced to sustain district momentum while they were waiting for funds to flow. A high number of activities were conducted by CPIU during this time which meant that funds within categories were expended much earlier than anticipated. This meant that there were only limited funds available for workshops and training when L-BEC grants were disbursed. The Government's proposed changes to the re-allocation of grant categories was lengthy due to the limitations of the Administrative Agreement even though the change required was considered small -- from EUR 775 000 to EUR 2.405 million.

47. Changes in Closing date of program implementation – project extension to December 2012. In accordance with the Grant Agreement (GA) signed between the Government of Indonesia and the World Bank, the BEC program was scheduled to finish at the end of April 2012. However due to the extensive delays to the project the CPIU was supported by the Bank and donors to apply for project extension to December 2012.

48. Changes in Number of Total Grant. Because of the crisis in Europe in 2011, the Dutch government cut their contribution to the BEC-TF grant by EUR 8.9 million. The cut was made to the GoI-executed project which reduced the total grant from EUR 25,618,000 to EUR 16,918,000.

49. GoI had proposed that the cut be proportionally divided between the GoI executed Trust Fund and the World Bank executed Trust Fund. However, following Management Committee meeting discussions with stakeholders it was agreed that the cut would come from a Category of the GoI-executed Trust Fund allocated for Consultant services.

50. Assessment of the outcome of the operation against the agreed objectives. In general, the expected outcome of the program had been achieved by implementing activities training and workshops related with componen 1, 2 and 3 of the BEC program both in central and district level. To achieve of Componen 2, CPIU and 50 districts had been developed training and workshops related with governance and efficient resource use, transparency, accountability, budget processes and performance based financing, financial management and accounting. While for achievement of componen 3, number of training and workshops had been developed related with improving and strengthening of the existing information and performance system to provide accurate data and information. The summary of achievement outcome of Component 1, 2 and 3 in the tabel below.

Componen Program Central Districts Component 1 21 Activities - Component 2 97 Activities 1.056 Activities

-Focus area 1 - 60 Activities -476 Activities -Focus area 2 - 6 Activities -178 Activities

62

V. Evaluation of the Borrower Performance

51. Section IV provides a detailed overview of borrower implementation activities, challenges and performance. 52. In implementing the program, the government was not required to deal directly with donor agencies as the World Bank was the appointed Trustee. However, the government participated in all Management Committee meetings and consulted openly with donors about implementation progress, challenges/problems and opportunities. Technically and substantively, the donors did not intervene but were available to fully support the project during all stages of implementation. 53. Project implemention challenges as a result of the 18 month delay in fund disbursement demanded a flexible and responsive approach to the problems faced by districts and to maintain their motivation and enthusiasm for the project generated through the targeting process. 54. Examples of specific CPIU interventions include: (i) district consultation clinics to mentor and coach districts to address implemention problems and challenges; and, (ii) the series of discussions between MoF, MoRA, BPKP and the World Bank to discuss ways to support districts to address various audit findings and act on/resolve recommendations. This resulted in improved audit findings in the following year. 55. At central and district operational levels, grants provided by donors that are allocated by category limit the way these funds can be utilized to meet emerging needs. Often projects are designed some years before implementation and the dynamics of the context of situation change during that period. The limitations described by rigid budget classifications and reallocation limitations across categories in Administrative Agreement may have been ameliorated by a different approach to the way the budget was allocated across project components. The government proposed a revised budget per category to the World Bank, but the Administrative Agreement limited the amounts per category which could be re-allocated. It is suggested that future projects of this nature should provide funds in the form of block grants which reflect ‘real world/actual’ not perceived requirements. 56. MoEC’s ability to engage effectively with key stakeholders was limited as project budget allocations did not include a dedicated budget to support their involvement, particularly for other necessary Basic Education Directorate staff, and key staff from MoF and MoRA.

VI. Evaluation of World Bank Performance

57. The World Bank as Trustee for the implementation of the GoI-executed BEC program is considered to be cooperative in carrying out activities in the form of co-ordination, facilitation

-Focus area 3 - 31 Activities -402 Activities Component 3 21 activities 212 Actiities

-Focus area 1 -18 Activities -194 Activities -Focus area 2 - 3 Activities - 18 Activities

63

and problem solving the problems and constraints faced by the government. Overall cooperative relationships between the Government and the World Bank went well and smoothly, each has a strong commitment to the implementation and success of the BEC. In its capacity as Trustee, the World Bank’s obligations to the donor were specified in legal agreements.

58. However it was not clear to the Government how the World Bank executed Trust Fund project related to the GoI executed Trust Fund although during the Management Committee meetings detailed progress reports and work plans were provided for both GoI and World Bank Trust Funds. 59. It is suggested that future projects should be in the form of a single grant so that all stakeholders are fully aware of the inter-relationship between projects.

VII. Proposed arrangements for future operations of the project

60. Future design should be the result of comprehensive discussions with education stakeholders and relevant implementing agencies at central, provincial, district, school and community levels and clearly mandated by MOEC.

61. Budget allocations should consider the requirements to enable CPIU as the manager for the project to facilitate wide consultation and involvement of relevant stakeholders. A dedicated budget is required to support other stakeholders (e.g. MORA, MOHA) to be involved so as to provide the necessary support to project activities.

62. Overarching financial management and budget requirements stipulated by MOHA and MOF must be factored into design.

63. Future donor grants should be in the form of a Block Grant with a committed amount that will not be subject to donor budget cuts. Detailed planning and design should occur with education stakeholders and implementing agencies.

64. It is suggested that in future projects of this nature, MOF should provide procedures and mechanisms to simplify the way grants are disbursed and settled to facilitate district financial reporting.

65. The L-BEC and CDP processes served as a catalyst for districts when focusing on education service delivery improvements. The model used should be reviewed and integrated into future district project activities.

64

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not Applicable

65

Annex 9. List of Supporting Documents Republic of Indonesia, Ministry of Education and Culture Bantuan Operasional Sekolah (BOS) National Program – multiple years

Republic of Indonesia, Ministry of Education and Culture, 2013. Basic Education Capacity Trust Fund (BEC-TF) Implementation Completion and Results Report (ICR),

Republic of Indonesia, Ministry of National Education, 2005. Medium Term Strategic Plan (Renstra),

Republic of Indonesia, Ministry of Education and Culture, 2012. Final Report; The Situation of Gender in Institutions and Activities of the Education Departments in 50 Regions/Cities in the BEC-TF program

Republic of Indonesia, Ministry of Education and Culture, 2012. “MoEC Education Statistics in Brief, 2008-2011”

Republic of Indonesia, The National Socioeconomic Survey, Survei Sosial Ekonomi Nasional (SUSENAS survey). March round 2007-2012. Jakarta, Indonesia.

European Commission. “Detailed Feasibility Study”, March-July 2006

Trust Fund Administrative Agreements between the Government of the Netherlands and the European Community (EC) and the World Bank, 2007

World Bank. Aides memoire of missions between June 30, 2009 and December 2012

World Bank, 2003. Country Assistance Strategy for Indonesia (CAS) ID- Report No: 27108. Washington, DC, October 2003

World Bank, 2008. Grant Appraisal Documents (GAD) Versions 1.6 and 1.84. Washington, DC.

World Bank, 2008. Grant Agreement Number TF091895 IND, Washington DC

World Bank, 2011. Project Paper on a Proposed Restructuring. Washington, DC.

World Bank, 2012. Country Partnership Strategy (CPS) for Indonesia for the Period FY13 – 15, December 13, 2012 Report No. 72906-ID.

World Bank, 2012. Public Expenditure Review (2012). Spending more or spending better: Improving education financing in Indonesia. Jakarta, Indonesia

World Bank, 2013. BEC-TF Financial Activity Report

World Bank, 2010 and 2013. Local Governance Capacity Assessment (LGCA) Local Governance Capacity Assessment (LGCA) and an Indonesia Local Education Governance Index (ILGI), 2010 and 2013

World Bank, 2008-2011. Regional Independent Monitoring Surveys. Jakarta, Indonesia.

2121

2828

3131

33333434

1313

1414

1515

1

1919

PAPU

APA

PUA

NEW

GU

INEA

NEW

GU

INEA

A U S T R A L I AA U S T R A L I A

THAILANDTHAILAND

MYANMARMYANMAR

19

21

22

27

30

20

23 26

2524

29

28

31

33

32

34

12

11

13

14

15

8

6 7

3

54

2

1

16

17 18

10

9

Balikpapan

Parepare

Baubau

Tarakan

Raba

Ende

Waingapu

Pematangsiantar

Sorong

Merauke

Timika

FakfakAmahai

JAKARTA

PAPU

AN

EW G

UIN

EA

A U S T R A L I A

SINGAPORE

VIETNAM

THAILAND

MYANMAR

TIMOR-LESTE

BRUNEI

PHILIPPINES

MA L A Y

SI

A

CelebesSea

Java Sea BandaSea

Arafura Sea

SuluSea

PA C I F I CO C E A N

I N D I A NO C E A N

Palu

Ambon

Gorontalo

Jambi

Medan

Kupang

Padang

Manado

Mataram

Bandung

Kendari

Denpasar

Surabaya

Semarang

Bengkulu

Jayapura

Palembang

Samarinda

Tanjung Selor

PontianakPekanbaru

Yogyakarta

BandaAceh

Bandjarmasin

PalangkarayaPangkalpinang

Makassar

Ternate

Bandar Lampung

Serang

Manokwari

Mamuju

Tanjungpinang

AcehAcehBesarBesar

BireunBireun Aceh UtaraAceh Utara

Aceh BaratAceh Barat

Aceh Barat DayaAceh Barat DayaAceh TenggaraAceh Tenggara

Kulon ProgoKulon Progo

BrebesBrebes

BanjarnegaraBanjarnegara

WonosoboWonosoboDemakDemak SragenSragen

BloraBloraRembangRembang

NgawiNgawi BangkalanBangkalanBojonegoroBojonegoro

PalangkarayaPalangkaraya

Polewali MandarPolewali MandarMajeneMajeneMamasaMamasa

KepulauanKepulauanSulaSula

Halmahera SelatanHalmahera SelatanSorongSorongSelatanSelatan

ManokwariManokwari

KaimanaKaimanaNabireNabire

PaniaiPaniai

PegununganPegununganBintangBintang

JayapuraJayapura

TelukTelukWondamaWondama

TernateTernate

Kotawaringin TimurKotawaringin Timur

SeruyanSeruyan

SampangSampang

PurbalinggaPurbalinggaKebumenKebumenPorworejoPorworejo SlemanSleman

WonogiriWonogiri PacitanPacitanTrenggalekTrenggalek

NganjukNganjukJombangJombang

ProbolinggoProbolinggoBondowosoBondowoso

Nagan RayaNagan Raya

LhokseumaweLhokseumawe

JayawijayaJayawijaya

Kota ProbolinggoKota Probolinggo

AcehBesar

Bireun Aceh Utara

Aceh Barat

Aceh Barat DayaAceh Tenggara

Kulon Progo

Brebes

Banjarnegara

WonosoboDemak Sragen

BloraRembang

Ngawi BangkalanBojonegoro

Palangkaraya

Polewali MandarMajeneMamasa

KepulauanSula

Halmahera SelatanSorongSelatan

Manokwari

KaimanaNabire

PaniaiJayawijaya

PegununganBintang

Jayapura

TelukWondama

Ternate

Kotawaringin Timur

Seruyan

Sampang

PurbalinggaKebumenPorworejo Sleman

Wonogiri PacitanTrenggalek

NganjukJombang

Probolinggo

Kota Probolinggo

Bondowoso

Nagan Raya

Lhokseumawe

PAPUA

AruIs.

KaiIs.

TanimbarIs.

Halmahera

Biak

Yapen

Morotai

Misool

Waigeo

PelengObi

Muna

Ceram

Buru

SULAWESISula Is.

Timor

FloresAlor

WetarMoa

Babar

Sumba

SumbawaLombokJAWA

NatunaBesar

Belitung

Madura

SUMATERA

Bangka

Lingga

Nias

Siberut

Enggano

Simeulue

TalaudIs.

Bali

KALIMANTAN

Men t a w

a i I s .

0°

5°

5°

10°

10°

15°

0°

5°

10°

15°

15° 15°

10°

95° 100° 105° 110° 115°

120° 125°

95° 100° 105° 110° 115° 120° 125°

130° 135° 140°

135°

INDONESIA

NANGGROE ACEH DARUSSALAMSUMATERA UTARARIAUSUMATERA BARATJAMBIBENGKULUSUMATERA SELATANLAMPUNG

12345678

PROVINCES:9

10111213141516

BANGKA-BELITUNGBANTEND.K.I. JAKARTAJAWA BARATJAWA TENGAHD.I. YOGYAKARTAJAWA TIMURBALI

NUSA TENGGARA BARATNUSA TENGGARA TIMURKEPULAUAN RIAUKALIMANTAN BARATKALIMANTAN TENGAHKALIMANTAN SELATANKALIMANTAN TIMURKALIMANTAN UTARASULAWESI UTARA

GORONTALOSULAWESI TENGAHSULAWESI BARATSULAWESI SELATANSULAWESI TENGGARAMALUKU UTARAMALUKUPAPUA BARATPAPUA

171819202122232425

262728293031323334

0 200

0 100 200 300 400 Miles

400 Kilometers

IBRD 39718

JUN

E 2013

INDONESIA

BASIC EDUCATION CAPACITYTRUST FUND (BEC-TF)

PROJECT PROVINCES

PROJECT KABUPATENS

CITIES AND TOWNS

PROVINCE CAPITALS

NATIONAL CAPITAL

RIVERS

PROVINCE BOUNDARIES

INTERNATIONAL BOUNDARIES

This map was produced bythe Map Design Unit of TheWorld Bank. The boundaries,colors, denominations andany other information shownon this map do not imply, onthe part of The World BankGroup, any judgment on thelegal status of any territory, or any endorsement oracceptance of suchboundaries.

34

Ternate