Document of The World Bankdocuments.worldbank.org/curated/en/155721502831078013/pdf/ICR395… · DLI Disbursement Linked Indicator DSD Directorate of Staff Development DTE District

Document of

The World Bank

Report No: ICR00003955

IMPLEMENTATION COMPLETION AND RESULTS REPORT

(IDA-51060)

ON A

CREDIT

IN THE AMOUNT OF SDR 225 MILLION

TO THE

ISLAMIC REPUBLIC OF PAKISTAN

FOR A

SECOND PUNJAB EDUCATION SECTOR PROJECT

June 21, 2017

Education Global Practice

South Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective 05/15/2017)

Currency Unit = Pakistan Rupee (PKR)

PKR 1.00 = US$ 0.0095

US$ 1.00 = PKR 104.8

FISCAL YEAR

July 1-June 30

ABBREVIATIONS AND ACRONYMS

ASC Annual School Census

ASER Annual Status of Education Report

CBA Cost-benefit Analysis

CIDA Canadian International Development Agency

CMMF Chief Minister’s Monitoring Force

CPF Country Partnership Framework

CPS Country Partnership Strategy

DFID U.K. Department for International Development

DLI Disbursement Linked Indicator

DSD Directorate of Staff Development

DTE District Teacher Educator

EIRR Economic Internal Rate of Return

EVS Education Voucher Scheme

GER Gross Enrollment Rate

GoPunjab Government of Punjab

ICR Implementation Completion and Results Report

ICT Information and Communication Technology

IPF Investment Policy Financing

ISR Implementation Status and Results Report

LFP Labor Force Participation

M&E Monitoring and Evaluation

MEA Monitoring and Evaluation Assistant

NER Net Enrollment Rate

NPV Net Present Value

NSB Non-salary Budget

PAD Project Appraisal Document

PDO Project Development Objective

PEC Punjab Examination Commission

PEF Punjab Education Foundation

PESP Punjab Education Sector Project

PESRP Punjab Education Sector Reform Program

PMIU Program Monitoring and Implementation Unit

PSLM Pakistan Standards of Living Measurement Survey

SC School Council

SED School Education Department

SPR School Participation Rate

SRC School Report Card

TA Technical Assistance

TPV Third-party Validation

Senior Global Practice Director: Jaime Saavedra Chanduvi

Sector Manager: Keiko Miwa

Project Team Leader: Scherezad Latif

ICR Team Leader: Ali Ansari

COUNTRY

SECOND PUNJAB EDUCATION SECTOR PROJECT

TABLE OF CONTENTS

Data Sheet

A. Basic Information

B. Key Dates

C. Ratings Summary

D. Sector and Theme Codes

E. Bank Staff

F. Results Framework Analysis

G. Ratings of Project Performance in ISRs

H. Restructuring

I. Disbursement Graph

1. Project Context, Development Objectives and Design ........................................................... 1

2. Key Factors Affecting Implementation and Outcomes ........................................................... 4

3. Assessment of Outcomes ........................................................................................................ 8

4. Assessment of Risk to Development Outcome ..................................................................... 15

5. Assessment of Bank and Borrower Performance.................................................................. 16

6. Lessons Learned .................................................................................................................... 19

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ....................... 21

Annex 1. Project Costs and Financing .......................................................................................... 22

Annex 2. Outputs by Component .................................................................................................. 23

Annex 3. Economic and Financial Analysis ................................................................................. 26

Annex 4. Bank Lending and Implementation Support/Supervision Processes ............................. 29

Annex 5. Beneficiary Survey Results ........................................................................................... 30

Annex 6. Stakeholder Workshop Report and Results ................................................................... 31

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................................... 32

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................................... 39

Annex 9. List of Supporting Documents....................................................................................... 40

i

A. Basic Information

Country: Pakistan Project Name: Pakistan: Second Punjab

Education Sector Project

Project ID: P125958 L/C/TF Number(s): IDA-51060

ICR Date: 06/25/2017 ICR Type: Core ICR

Lending Instrument: SIL Borrower: ISLAMIC REPUBLIC

OF PAKISTAN

Original Total

Commitment: XDR 225.00M Disbursed Amount: XDR 225.00M

Revised Amount: XDR 225.00M

Environmental Category: B

Implementing Agencies:

School Education Department

Program Monitoring and Implementation Unit

Cofinanciers and Other External Partners:

B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 11/03/2011 Effectiveness: 05/15/2012 06/06/2012

Appraisal: 01/23/2012 Restructuring(s):

06/04/2013

12/11/2015

06/17/2016

Approval: 04/26/2012 Mid-term Review: 08/25/2014 09/05/2014

Closing: 12/31/2015 12/31/2016

C. Ratings Summary

C.1 Performance Rating by ICR

Outcomes: Moderately Satisfactory

Risk to Development Outcome: Moderate

Bank Performance: Moderately Satisfactory

Borrower Performance: Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)

Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Satisfactory

Quality of Supervision: Satisfactory Implementing

Agency/Agencies: Satisfactory

Overall Bank

Performance: Moderately Satisfactory

Overall Borrower

Performance: Satisfactory

ii

C.3 Quality at Entry and Implementation Performance Indicators

Implementation

Performance Indicators

QAG Assessments (if

any) Rating

Potential Problem Project

at any time (Yes/No): No

Quality at Entry

(QEA): None

Problem Project at any

time (Yes/No): No

Quality of Supervision

(QSA): None

DO rating before

Closing/Inactive status:

Moderately

Satisfactory

D. Sector and Theme Codes

Original Actual

Major Sector/Sector

Public Administration

Public administration - Education 3 3

Education

Primary Education 97 97

Major Theme/Theme/Sub Theme

Human Development and Gender

Education 49 49

Access to Education 49 49

Education Financing 49 49

Public Sector Management

Public Administration 2 2

Transparency, Accountability and Good Governance 2 2

E. Bank Staff

Positions At ICR At Approval

Regional Vice President: Annette Dixon Isabel M. Guerrero

Country Director: Patchamuthu Illangovan Rachid Benmessaoud

Practice Manager: Keiko Miwa Amit Dar

Task Team Leader(s): Scherezad Joya Monami Latif Huma Ali Waheed

ICR Team Leader: Ali Ansari

ICR Primary Author: Ali Ansari

iii

F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) The objective of the project is to support the Government of Punjab's education sector reform

program in order to increase child school participation (at multiple levels) and student

achievement

Revised Project Development Objectives (as approved by original approving authority)

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target

Values (from

approval

documents)

Formally

Revised

Target Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1 : Net Enrollment Rate, primary level, Ages 6-10 years

Value

quantitative or

Qualitative)

70

75

70

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)

Partial achievement. Although the primary level NER target was not achieved,

progress towards increasing school participation was noted for the same cohort.

Please refer to Indicator 18 for more information and the ICR's efficacy section

Indicator 2 : Net Enrollment rate, middle level, ages 11-13 years

Value

quantitative or

Qualitative)

37

44

38

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)

Partial achievement. Although the middle level NER target was not achieved

(14 % achievement), progress towards increasing school participation was noted

for the same cohort. Please refer to Indicator 24 for more information and the

ICR's

Indicator 3 : Net enrollment rate, matriculate level, ages 14-15 years

Value

quantitative or

Qualitative)

25

32

29

Date achieved 06/30/2011 12/31/2015 12/21/2016

iv

Comments

(incl. %

achievement)

Partial achievement. Although the matric level NER targets of the Project was

not achieved (57 % NER achievement), progress towards increasing school

participation was noted.

Indicator 4 : Average share of correct answers, independent test, grade 3

Value

quantitative or

Qualitative)

55%

60%

60%

Date achieved 09/15/2014 12/31/2015 12/21/2016

Comments

(incl. %

achievement)

Target achieved. Target was a 5 percentage point improvement over the

baseline. (100% target achievement). The baseline was established in 2014

Indicator 5 : Net enrollment rate, primary level, ages 6-10, rural girls

Value

quantitative or

Qualitative)

64

69

62

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)

Partial achievement. This indicator was monitored as an "other outcome

indicator". Although NER target was not achieved progress towards increasing

participation for this cohort was noted, please see indicator 47

Indicator 6 : Net enrollment rate, primary level, ages 6-10, rural

Value

quantitative or

Qualitative)

67

71

66

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)



participation for this cohort was noted, please see indicator 48.

Indicator 7 : Net enrollment rate, primary level, ages 6-10 years, asset poor households

Value

quantitative or

Qualitative)

47

49

45

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)




Indicator 8 : Net enrollment rate, primary level, ages 6-10 years. Participation poor districts

Value

quantitative or 55

59

54

v

Qualitative)

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)




Indicator 9 : Net enrollment rate, middle level, ages 11-13 years, rural girls

Value

quantitative or

Qualitative)

29

33

31

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)


indicator". 60.5% achievement. Source for Target Year Values: PSLM 2014-15

Indicator 10 : Net enrollment rate, middle level, ages 11-13 years, rural

Value

quantitative or

Qualitative)

31

39

34

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)


indicator". 35% achievement Source for Target Year Values: PSLM 2014-15

Indicator 11 : Net enrollment rate, middle level, ages 11-13, asset poor households

Value

quantitative or

Qualitative)

13

14

14.66

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)

Target surpassed. This indicator was monitored as an "other outcome indicator".

166% achievement. Source for Target Year Values: PSLM 2014-15

Indicator 12 : Net enrollment rate, middle level, ages 11-13, participation poor districts

Value

quantitative or

Qualitative)

23

25

24.5

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)



Indicator 13 : Net enrollment rate, high level, ages 14-15, rural girls

Value

quantitative or

Qualitative)

19

25

21

vi

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)


indicator". 36% achievement. Source for Target Year Values: PSLM 2014-15

Indicator 14 : Net enrollment rate, high level, ages 14-15, rural

Value

quantitative or

Qualitative)

20

25

23

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)



Indicator 15 : Net enrollment rate, high level, ages 14-5, asset poor households

Value

quantitative or

Qualitative)

7

7

7.5

Date achieved 06/30/2011 12/30/2015 12/21/2016

Comments

(incl. %

achievement)



Indicator 16 : Net enrollment rate, high level, ages 14-15, participation poor districts

Value

quantitative or

Qualitative)

17

21

17

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)

This indicator was monitored as an "other outcome indicator". Target was not

achieved. Source for Target Year Values: PSLM 2014-15

Indicator 17 : School participation rate, ages 6-10 (excluding katchi)

Value

quantitative or

Qualitative)

73

77

70

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)


indicator". Although this SPR target was not achieved progress towards

increasing participation for this cohort was noted, please see indicator 18

Indicator 18 : School participation rate, ages 6-10 (including katchi)

Value

quantitative or

Qualitative)

80.2 (PSLM 2010-11)

82.9 (PSLM-2014-

15)

Date achieved 06/30/2011 12/21/2016

vii

Comments

(incl. %

achievement)

School participation rate for 6-10 year olds increased by 2.7 percentage points

This indicator was included in the ICR to verify the enrollment gains over the

project period. It includes enrollment in katchi grades, public, private, and other

NGO schools

Indicator 19 : School participation rate, ages 6-10 (excluding katchi), rural girls

Value

quantitative or

Qualitative)

65

71

62

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)




Indicator 20 : School participation rate, ages 6-10 (excluding katchi), rural

Value

quantitative or

Qualitative)

69

74

66

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)




Indicator 21 : School participation rate, ages 6-10 (excluding katchi), asset poor households

Value

quantitative or

Qualitative)

48

50

45

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)




Indicator 22 :

School participation rate, ages 6-10 (excluding katchi), participation poor

districts

Value

quantitative or

Qualitative)

57

62

52

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)




Indicator 23 : School participation rate, ages 11-15, all

viii

Value

quantitative or

Qualitative)

70

76

70

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)



increasing participation for this cohort was noted, please see indicators 24 and

25


Value

quantitative or

Qualitative)

75.4 (PSLM 2010-11)

77.7 (PSLM 2014-

15)

Date achieved 06/30/2011 12/21/2016

Comments

(incl. %

achievement)

School participation rate for 11-13 year olds increased by 2.3 percentage points


project period. It includes enrollment in public, private, and

other nongovernment schools.


Value

quantitative or

Qualitative)

61.1 (PSLM 2010-11)

63.5 (PSLM 2014-

15)

Date achieved 06/30/2011 12/21/2016

Comments

(incl. %

achievement)

School participation rate for 11-13 year olds increased by 2.4 percentage points.


project period. It includes enrollment in public, private, and

other nongovernment schools.

Indicator 26 : School participation rate, ages 11-15 rural girls

Value

quantitative or

Qualitative)

58

67

58

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)




Indicator 27 : School participation rate, ages 11-15 rural

Value

quantitative or 65

71

65

ix

Qualitative)

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)




Indicator 28 : School participation rate, ages 11-15, asset poor households

Value

quantitative or

Qualitative)

38

40

39

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)



Indicator 29 : School participation rate, ages 11-15, participation poor districts

Value

quantitative or

Qualitative)

55

60

50

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)

This indicator was monitored as an "other outcome indicator". SPR target for

participation poor districts was not achieved

Indicator 30 : NER, government schools, primary, ages 6-10

Value

quantitative or

Qualitative)

43

48

38

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)

This indicator was monitored as an "other outcome indicator". Target was not


Indicator 31 : NER, government schools, middle, ages 11-13

Value

quantitative or

Qualitative)

24

39

23

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)

This indicator was monitored as an "other outcome indicator". Target not


Indicator 32 : NER, government schools, high, ages 14-15

Value

quantitative or

Qualitative)

16

21

18

x

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)



Indicator 33 : Primary school completion rate, ages 15-19, all

Value

quantitative or

Qualitative)

92

94

91

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)



Indicator 34 : Primary school completion rate, ages 15-19, rural girls

Value

quantitative or

Qualitative)

90

93

89

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)



Indicator 35 : Primary school completion rate, ages 15-19, rural

Value

quantitative or

Qualitative)

90

92

89

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)



Indicator 36 : Primary school completion rate, ages 15-19, asset poor households

Value

quantitative or

Qualitative)

76

78

77

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)

Partial Achievement. This indicator was monitored as an "other outcome


Indicator 37 : Primary school completion rate, ages 15-19, asset poor households

Value

quantitative or

Qualitative)

86

90

85

Date achieved 06/30/2011 12/31/2015 12/21/2016

xi

Comments

(incl. %

achievement)



Indicator 38 : Secondary school completion rate, ages 20-24, all

Value

quantitative or

Qualitative)

50

53

52

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)


indicator" 74% achievement. Source for Target Year Values: PSLM 2014-15

Indicator 39 : Secondary school completion rate, ages 20-24, rural girls

Value

quantitative or

Qualitative)

43

48

48.1

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)



Indicator 40 : Secondary school completion rate, ages 20-24, rural

Value

quantitative or

Qualitative)

41

44

44.2

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)



Indicator 41 : Secondary school completion rate, ages 20-24, asset poor households

Value

quantitative or

Qualitative)

17

21

17.5

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)



Indicator 42 : Secondary school completion rate, ages 20-24, participation poor districts

Value

quantitative or

Qualitative)

41

45

40

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %



xii

achievement)

Indicator 43 : Mean test score, government schools grade 5, all

Value

quantitative or

Qualitative)

158

174

197 (PEC 2015-16)

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)


244% achievement.

Indicator 44 : Mean test score, government schools grade 5, participation poor districts

Value

quantitative or

Qualitative)

173

190

209 (PEC 2015-16)

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)


212% achievement.

Indicator 45 : Mean test score, government schools grade 8, all

Value

quantitative or

Qualitative)

188

207

217 (PEC 2015-16)

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)


153% achievement.

Indicator 46 : Mean test score, government schools grade 8, participation poor districts

Value

quantitative or

Qualitative)

194

213

239

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)

Target surpassed. This indicator was monitored as an "other outcome indicator",

237% achievement.

Indicator 47 : school participation rate (including katchi), ages 6-10, rural girls

Value

quantitative or

Qualitative)

72.23

74.11

Date achieved 06/30/2011 12/21/2016

Comments

(incl. %

achievement)

SPR increased by 1.88 percentage points. This indicator was included in the ICR

to verify the enrollment gains over the project period. It includes enrollment in

katchi grades, public, private, and other NGO schools Source: PSLM

xiii

Indicator 48 : School participation rate (including katchi), ages 6-10, rural

Value

quantitative or

Qualitative)

76.41

78.82

Date achieved 06/30/2011 12/21/2016

Comments

(incl. %

achievement)

school participation increased by 2.41 percentage points. This indicator was

included in the ICR to verify the enrollment gains over the project period. It

includes enrollment in katchi grades, public, private, and other NGO schools

Indicator 49 : School participation rate (including katchi), ages 6-10 , asset poor households

Value

quantitative or

Qualitative)

52.47

56.11

Date achieved 06/30/2011 12/21/2016

Comments

(incl. %

achievement)




Indicator 50 :

School participation rate (including katchi), ages 6-10, participation poor

districts

Value

quantitative or

Qualitative)

64.87

67

Date achieved 06/30/2011 12/21/2016

Comments

(incl. %

achievement)




Indicator 51 : School participation rate, ages 11-15, rural girls

Value

quantitative or

Qualitative)

58.3

59.41

Date achieved 06/30/2011 12/21/2016

Comments

(incl. %

achievement)



includes enrollment in public, private, and other NGO schools. Source: PSLM

Indicator 52 : School participation rate, ages 11-15, rural

Value

quantitative or

Qualitative)

64.74

66.46

Date achieved 06/30/2011 12/21/2016

xiv

Comments

(incl. %

achievement)




Indicator 53 : School participation rate, ages 11-15, asset poor households

Value

quantitative or

Qualitative)

36.28

40.5

Date achieved 06/30/2011 12/21/2016

Comments

(incl. %

achievement)




Indicator 54 : School participation rate, ages 11-15, participation poor districts

Value

quantitative or

Qualitative)

54.97

54.67

Date achieved 06/30/2011 12/21/2016

Comments

(incl. %

achievement)

school participation for this cohort did not increase during the project period.


project period. It includes enrollment in public, private, and other NGO schools.

Source: PSLM

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target

Values (from

approval

documents)

Formally

Revised Target

Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1 : Teacher absence rate

Value

(quantitative

or Qualitative)

16%

15%

6%

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)

1000% achievement. Target surpassed

Indicator 2 : Recurrent non-salary expenditure (RPs billions)

Value

(quantitative

or Qualitative)

8

15

24

Date achieved 06/30/2011 12/31/2015 12/21/2016

xv

Comments

(incl. %

achievement)


Indicator 3 : Share of non-salary expenditures in total recurrent expenditures (percentage)

Value

(quantitative

or Qualitative)

7%

12%

13%

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)


Indicator 4 : Stipends for girls for secondary schooling: number of beneficiaries

Value

(quantitative

or Qualitative)

382,632

440,100

411,000

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)

49% achievement

Indicator 5 : Vouchers for private schooling: number of beneficiaries

Value

(quantitative

or Qualitative)

140,000

200,000

361,000

Date achieved 06/30/2011 12/31/2015 12/21/2016

Comments

(incl. %

achievement)


G. Ratings of Project Performance in ISRs

No. Date ISR

Archived DO IP

Actual Disbursements

(USD millions)

1 11/11/2012 Satisfactory Moderately Satisfactory 45.50

2 05/16/2013 Satisfactory Moderately Satisfactory 46.25

3 12/21/2013 Satisfactory Satisfactory 92.15



6 06/28/2015 Moderately Satisfactory Satisfactory 304.14




xvi

H. Restructuring (if any)

Restructuring

Date(s)

Board

Approved PDO

Change

ISR Ratings at

Restructuring

Amount

Disbursed at

Restructuring

in USD millions

Reason for Restructuring & Key

Changes Made DO IP

06/04/2013 S MS 46.25

The project was restructured in

June 2013, which involved

changes to one of the 10

Disbursement Linked Indicators

(DLIs) agreed with the

Government of Punjab. The

restructuring had no impact on

the PDO.

12/11/2015 MS S 304.14

Extension of project closing

date

06/17/2016 MS S 329.49

Extension of project closing

date

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal

1. Country and provincial context. At the time of project appraisal in 2012, Pakistan was

experiencing adverse fiscal, macroeconomic, political, and security-related challenges that were

directly affecting private and public sector service delivery. Large-scale flooding in 2010 and 2011

further exacerbated socioeconomic conditions in the country and the near-term outlook for the

country was bleak. Prospects for Punjab, Pakistan’s most populous and wealthiest province, were

not much better. The Government of Punjab (GoPunjab) had been experiencing shortfalls in

revenues while increasing expenditure resulting in fiscal deficits.

2. Sector context. In 2012, Pakistan continued to rank poorly in the United Nations

Development Programme (UNDP) Human Development Index, with particularly low education

outcomes. Net enrollment rates (NERs) in Punjab at the primary-, middle-, and high-school levels

were 70 percent, 37 percent, and 25 percent, respectively.1 The age of children, their household’s

socioeconomic status, and location seem to have a strong association with school participation. In

addition to low levels of enrollment, learning outcomes were also poor, as surveys have found low

levels of student achievement across English, Urdu, and mathematics assessments at the grade 3

level.2

3. One of the key features of the education sector in Punjab was that approximately one-third

of children between the ages of 6 and 10 and one-fifth of children between the ages of 11 and 15

attend private schools, driving almost all the gain in school participation rates (SPRs) in the

province between 2004 and 2010.3

4. Rationale for World Bank assistance. Before the second Punjab Education Sector Project

(PESP II), the World Bank had a long-standing engagement in the education sector in Punjab

beginning in 2003 with support to the government’s Punjab Education Sector Reform Program

(PESRP). The World Bank supported the Government’s reform program through four

development policy credits (2004–07) and one Results-based Financing operation (2009–12) in

the form of the PESP. With the introduction of the Education Sector Reform Roadmap in 2011

and the second phase of the Government’s reform agenda, a request was made by the Government

of Pakistan for PESP II.

5. Donor partners. The U.K. Department for International Development (DFID) has also

been actively involved in the sector, and is currently the largest donor in the basic education sector

in Punjab. DFID was a parallel financier in both the GoPunjab’s PESRP I and II and is currently

providing GBP 457 million to the GoPunjab between 2014 and 2018 to support the GoPunjab’s

PESRP II and its successor, the 2018 Education Goals.4 DFID is also supporting the GoPunjab’s

Education Sector Roadmap that consists of a top-down accountability model, which involves

1 Pakistan Standards of Living Measurement Survey (PSLM) 2010–11. 2 Andrabi, Das, Khwaja, Vishwanath, and Zajonc 2007. 3 PSLM 2004–10. 4 PESP III PAD.

2

monitoring select education indicators through regular stock-taking chaired by the Chief Minister,

for accountability and oversight.5

6. PESRP II. The second phase of PESRP (the reform program of the government), which

was supported by this project, was designed to increase child school participation and student

achievement in the province with a particular focus on improving service delivery at the school

level by focusing on improving teacher quality and performance. The teacher quality and

performance initiatives to be implemented under PESRP II included, (i) strengthening the system

of field based advisory support to teachers, (ii) introducing test based recruitment of teachers, (iii)

fixing teacher posts at the school level based on school need and reallocating teachers against

rationalized teaching posts, (iv) formulating and offering school specific non salary budgets based

on school needs, (v) decentralizing administrative and financial management powers to schools

(or school clusters), (vi) tying teacher compensation with school performance, (vii) increasing the

autonomy and capacity of school councils to support and monitor school performance and (viii)

improving regular collection of credible information on school, teacher and student performance

and feeding this information to stakeholders. PESRP II also included reforms that would more

directly promote gains in participation such as (ix) strengthening the design of a tuition

replacement voucher scheme to promote access to and participation in low-cost private schools

and (x) offering attendance tied supplemental cash transfers to promote secondary schooling

among girls in participation-poor districts. These reforms were all directly supported through the

project DLIs. In addition to these reforms, the PESRP II included the strengthening of the Program

Monitoring and Implementation Unit (PMIU), strengthening of the complaints redressal system

for the School Education Department, and strengthening the capacity of the Chief Minister’s

Monitoring Force (CMMF), introducing a new teacher certification and licensing regime and

strengthening the Punjab Examination Commission (PEC) which supported through the project’s

TA component. 6

1.2 Original Project Development Objective

7. The Project Development Objective (PDO) is to support Punjab’s education sector reform

program in order to increase child school participation (at multiple levels) and student

achievement.7

8. Key performance indicators for this project are (a) net enrollment rates at the primary,

middle, and high school levels and (b) average share of correct answers (grade 4 and 5 students).

5 This model is based on the ‘deliverology’ approach which consists of six elements: setting direction; establishing

clear accountabilities and metrics; creating realistic budgets, plans and targets; tracking performance effectively;

holding robust performance dialogues; and ensuring actions, rewards, and consequences.

6 PAD, page 6-7 7 A minor difference in the language of the development objective was observed between the Project Agreement and

the PAD. The PAD refers to the ‘Government of Punjab’ while the Project Agreement refers to ‘Punjab’s’ Education

Reform Program. This has no implications on the meaning of the Development Objective, and the Implementation

Completion and Results Report (ICR) uses the definition stated in the Project Agreement.

3

1.3 Revised PDO (as approved by original approving authority) and Key Indicators and

Results/Justifications

9. The PDO, PDO-level indicators, and intermediate outcome indicators were not formally

revised.

1.4 Main Beneficiaries

10. The project beneficiaries were identified as all students in government schools and low-

cost private schools that were covered by the initiatives under PESRP II.8

1.5 Original Components

11. The project consisted of the following two components, as described in the Project

Agreement and the Project Appraisal Document (PAD):

(a) Component 1: Financing of the Second Punjab Education Sector Reform

Program. Component 1 consisted ten Disbursement Linked Indicators (DLIs) for

each fiscal year which were designed to “help intensify, deepen, or expand the

supported initiatives under PESRP II”.9 This included subcomponents related to (i)

strengthening teacher quality and teaching support, (ii) implementing teachers’

performance bonuses, (iii) strengthening school budget and budget management, (iv)

promoting community and parental engagement in school management through

school councils (SCs), (v) generating and using reliable student test information, (vi)

strengthening the education sector and school performance measurement mechanism,

(vii) creating an enabling environment for improved stakeholders’ oversight of the

education sector, (viii) assisting the implementation of a school vouchers public-

private partnership program by the Punjab Education Foundation (PEF), and (ix)

strengthening the existing girls stipend program.

(b) Component 2: Technical Assistance. This component involved executing capacity-

building activities to strengthen Punjab’s Education Sector Department and agencies

involved in the implementation of the project, which included (i) the carrying out of

the day-to-day coordination, financial management, procurement, environmental and

social management, and monitoring and evaluation (M&E) of project activities; (ii)

supporting the implementation of the reforms under PESRP II; and (iii) verification

of the achievement of DLIs.

1.6 Revised Components

12. There was no change to the agreed components during the three restructurings. However,

the June 2013 restructuring involved changes to one of the ten DLIs agreed with the GoPunjab.

The change was related to the DLI on devolution of financial management powers to schools for

spending school-specific non-salary budgets (NSBs) for school maintenance and operation.

During the course of project implementation, the School Education Department (SED) decided

8 PAD, page 5. 9 PAD, page 8.

4

that instead of devolving financial management powers to clusters of schools, it would be more

effective and efficient to transfer financial management powers directly to the SC bank accounts

of government schools. This change did not result in any change to the PDO, Results Framework,

safeguards category, legal covenants, or implementation and disbursement arrangements.

1.7 Other Significant Changes

13. The project closing date was extended twice, amounting to a total 12-month extension from

the original closing date, to allow the SED to use unspent funds under Component 2. Moreover, a

sample-based learning assessment had been planned to assess the learning outcomes in grades 4

and 5 to track progress toward achieving the PDO related to increasing student achievement.

However, because of the existence of biannual student assessments conducted by DFID’s technical

assistance (TA) firm, the results of these were used to track progress toward achieving the PDO

instead. This change did not require a restructuring of the project.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry

14. Quality at entry. The project rating for the quality at entry is Moderately Satisfactory,

based on the assessment in the subsequent paragraphs. The project was aligned to the Country

Partnership Strategy (CPS), the GoPunjab’s own PESRP I and II as well as the 2018 Education

Goals of the province.

15. Preparation. The project was prepared in close collaboration with development partners,

namely Canadian International Development Agency (CIDA) and DFID, the latter being a co-

financier of the project. The GoPunjab demonstrated strong commitment at the time of preparation

by ensuring close synergy between the project and the government’s own education reform

program PESRP II, and by demonstrating willingness to undertake politically unpopular reforms

(such as merit-based teacher recruitment and teacher rationalization). The project clearly defined

risks associated with the project along with appropriate mitigating measures, as well as safeguards-

and fiduciary-related responsibilities.

16. Design. The choice of lending instrument, a results-based Investment Policy Financing

(IPF), consisting of a results-based component with DLIs and a TA component, was appropriate

given the success of this approach in PESP I. Lessons learned from previous operations in the

sector were incorporated in the design of the program including increasing partnerships with the

private sector for education service delivery, and increasing community outreach and engagement.

The PDO was clearly articulated. However, a detailed analysis in the PAD of the expected impact

of project interventions on school enrollments would have strengthened the justification for the

selection of NERs as the choice of indicator to measure progress toward achieving the school

participation PDO target. Within the context of Punjab, where there is late entry into the schooling

system, the school participation rate would have been a more appropriate indicator to measure

school participation. In addition, the targets for school participation were too ambitious given the

three-year period of the project.

5

2.2 Implementation

17. Overall, project-supported activities were implemented successfully, with all project DLIs

being achieved. The GoPunjab demonstrated strong political commitment to implementing key

reforms as the government withstood political pressure to implement merit-based teacher

recruitment and teacher rationalization. In fact, in many instances the SED surpassed the targets

of the project, by decentralizing NSB management to the school level (a step beyond what the

project had originally envisaged), expanding the education voucher scheme to 361,000

beneficiaries, by expanding field-based mentoring support to all public primary schools in the

province. The project also effectively supported the merit based recruitment of more than 33,000

teachers, implemented teacher rationalization province wide, and strengthened the capacity of

school councils through an ICT based mobilization program. In addition, the project supported

other aspects of the PESRP II which include strengthening the CMMF to ensure that independent

monitors visited more than 90 percent of public schools each month and collected data via tablets

which was uploaded and made available to the general public online. Overall budgetary

allocations to the education sector also increased significantly during this period. The World Bank,

donor partners, and the SED also engaged in regular dialogue, which was critical in addressing

implementation issues on time.

18. There were aspects of project implementation that could have been improved. The project

included an ambitious legal covenant on the establishment of a teacher licensing and certification

system. While the SED was able to make progress in developing a draft teacher certification and

licensing framework and draft law, the framework requires further stakeholder consultation and

revision until a law can be a passed. This is a reform that the SED is continuing to pursue and has

recently established the Punjab Education Standards Development Authority which is mandated

to manage and regulate the certification, registration, and licensing of teachers and education

administrators in the province. With respect to Component 2 of the project, procurement

performance was poor as there were delays in meeting procurement timelines and in paying

contractors. Last, high turnover of PMIU staff also posed a significant challenge, which affected

the ownership of reforms within the PMIU.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

19. M&E design. The M&E design of the project relied on the GoPunjab’s existing education

monitoring systems in addition to third-party assessments. The design of PESP II monitoring

systems included (a) the Annual School Census (ASC) conducted by the SED/PMIU, (b) monthly

monitoring visits conducted by the CMMF 10 under the SED, (c) PESRP II subprograms

monitoring data, and (d) third-party assessments to evaluate implementation progress. The SED’s

monitoring framework had resulted in strong implementation oversight of PESP I and generated

reliable information; therefore, it was appropriate to carry forward the same monitoring framework

for PESP II. However, the choice to use NERs to measure progress toward achieving the school

participation-related PDO may not have been appropriate given the volatility in the indicator which

was also flagged in the PESP ICR. In addition, the results framework was overly complex, and

10 The Chief Minister’s Monitoring Force consists of retired army officials who visit schools unannounced on a

monthly basis to collect information on school infrastructure, facilities, and student and teacher presence, among other

indicators.

6

there was no PDO level indicator to measure the project’s support to the GoPunjab’s reform

program.

20. M&E implementation. The M&E system under PESP II provided strong monitoring and

oversight for the project. A baseline for measuring the student achievement-related PDO indicator

took time to be established; however, the SED was able to conduct a baseline in September 2014

and conduct regular student assessments every six months thereafter. The school census was

conducted annually, and Monitoring and Evaluation Assistants (MEAs) conducted monthly visits

of schools throughout the project period and were able to provide reliable and timely information.

Third-party validations (TPVs) of the monthly monitoring data that were conducted support the

validity of the data with regard to reliability of information collected, data entry, and data cleaning

and merging processes.11 Third-party assessments, as part of the project, were also successfully

conducted to verify the achievement of various reforms discussed in the paragraph below. Impact

evaluations were also conducted for the teacher performance incentives reform and the

supplemental stipends program; however, the results from the supplemental stipends program

evaluation are yet to be finalized.12

21. M&E utilization. The information that was generated by the monitoring systems also fed

into the design of key reforms supported by the project. For instance, the NSB reform used the

ASC data to determine school-specific NSB allocations based on existing enrollment, facilities,

and infrastructure information. Similarly, the SED’s teacher allocation formula also relied on the

school census to determine school-specific teacher needs as well as the school report card (SRC)

which used the data from the ASC to inform community members about overall school quality.

The monthly monitoring data were also used for accountability purposes as part of the Education

Sector Roadmap, which tracks performance against key indicators (such as teacher and student

presence, school functionality, and facilities) at the district level which were reviewed quarterly

during meetings chaired by the Chief Minister. Third-party assessments were actively used to

verify the effectiveness of reforms supported by the project including the NSB reform, the teacher

rationalization exercise, and the SC mobilization exercise based on information and

communication technology (ICT).

2.4 Safeguards and Fiduciary Compliance

22. Environment and social safeguards. Although no construction-related activities were to

be supported by the project, given the potential for some school construction, rehabilitation and/or

upgrading activities that may have taken place under the GoPunjab’s reform program, the project

was rated as a Category B project in accordance with World Bank OP 4.01 (Environmental

Assessment). Overall, safeguard compliance was Satisfactory as the planned activities under the

Environment and Social Management Framework (ESMF) were implemented. The ESMF

developed under PESP I was revised for PESP II and included mitigation measures,

implementation and institutional arrangements, monitoring requirements, and supervision roles

and responsibilities. The Deputy Director (M&E) was designated as the environmental and social

focal point with the overall responsibility for fulfilling the safeguard requirements of the project,

11 Cambridge Education 2014: Monthly Monitoring Technical Report Validations 1–4. 12 Disaggregated data for Punjab Examination Commission (PEC) student achievement indicators were not available

at the time of the ICR and therefore were not included in the Results Framework analysis.

7

while an Environment Officer was to be recruited with the responsibility for implementing various

safeguard compliance activities. At the district level, District Monitoring Officers were responsible

for monitoring and coordinating ESMF implementation while head teachers and SCs were

responsible for implementing the safeguards guidelines during school operations. District Officers

and the Building Department and the District Monitoring Officers, head teachers, and MEAs

received training according to the ESMF framework. 13 An Environment Coordinator was

recruited, and environmental clubs were established in high schools and higher secondary schools

in all 36 districts of the province, and more than 49,000 trees were planted in these districts.

Training material was also distributed to schools, highlighting basic environmental and water

conservation principles.14

23. Financial management. Financial management performance was rated Satisfactory for

most of the project. The PMIU Finance Unit was adequately staffed throughout the project period

as was the PEF Finance Department. Adequate fund allocations were in place including the NSB

allocation for schools and fund allocations for the PEF. There were some delays in the transfer of

non-salary funds to schools particularly in the early phase of the project. Financial reporting in the

form of quarterly budget execution reports for the whole education sector were provided on a

regular basis throughout the course of the project. The project had a sound record of compliance

with the reporting requirements of the World Bank with regard to the interim financial reports and

annual external audit reports. However, complete district-level audit reports were not submitted to

the World Bank on an annual basis as stipulated in the legal covenant, despite numerous requests

from the World Bank and the PMIU to the provincial government.

24. Procurement. Procurement performance was rated Moderately Satisfactory throughout

the project. Under Component 2 of the project, there were delays in meeting planned procurement

timelines and in updating the Procurement Plan to accurately reflect the progress of the project.

Payments to two contracted firms were also significantly delayed, which could have been avoided,

provided there was increased procurement capacity and stronger leadership within the PMIU.

Findings from the Post Procurement Reviews found minor issues relating to poor record keeping,

lack of appropriate documentation, and weak contract management. 15

2.5 Post-completion Operation/Next Phase

25. PESP III was approved by the Board on June 3, 2016, became effective on October 5, 2016,

and is designed to support the implementation of the GoPunjab’s 2018 Education Goals. This

support comes at an opportune time when the provincial government is dedicating more focus and

financing to improve education outcomes in the province. The project is a US$300 million

investment loan that will be implemented over the period FY2016/17–2021/22. The project will

follow a similar design to PESP II, consisting of two main components: a results-based financing

component consisting of nine disbursement-linked results areas and a TA component to support

the implementation of key reform activities. The PDO is to support Punjab Province to improve

school participation, completion, and teaching-learning practices with a particular focus on low-

13 Trainings took place in September and October 2014, and a total of 413 and 104 officials participated in trainings

at the division and provincial levels, respectively (ESMF Training Report 2014). 14 ESMF Status Document 2014, PMIU. 15 Post Procurement Reviews reported deviations in the procurement process relating to documenting comparative

statements, goods received notes, contract negotiations, and contract management.

8

performing districts. PESP III will continue to support some of the reforms supported under PESP

II (such as supporting merit-based teacher recruitment, the NSB reform, and the girls’ stipend

program) and will also support new reforms being introduced by the GoPunjab such as the

expansion of early childhood education and supporting the expansion of public-private

partnerships in education through PEF-supported programs. The project will use the same

implementation and M&E arrangements as PESP II to ensure continuity. This is a clear reflection

of the sustainability of the systems built by both projects—PESP I and PESP II.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation

26. The overall rating for the relevance of the project objective, design, and implementation in

the present context is Modest for the following reasons.

27. Objectives (High). The PDOs are highly relevant to the country’s sectoral needs as well

as the GoPunjab’s own education reform program. The PDO was fully aligned to the Country

Partnership Framework’s (CPF’s) (2011–14) Pillar 4 relating to improving human development,

as well as the current CPF 2015–19, specifically the CPF outcome on increased school enrollment

and adoption of education quality assessment (Outcome 4.3) (Report No. 84645-PK). The PDO

was also aligned with the GoPunjab’s Education Sector Roadmap which commenced in 2011, with

the stated aspiration to ensure that every child in the province is in school and is learning. The

project was fully aligned with GoPunjab’s PESRP II and its successor the 2018 Education Goals.16

In fact, the project was designed to support the same objectives as the GoPunjab’s PESRP II, to

increase school participation and student achievement.17

28. Design and implementation (Modest). The choice of lending instrument, a results-based

IPF which used DLIs, was found appropriate given the previous success of this design in PESP I

and PESP II. The inclusion of a US$10 million TA component as part of the project design proved

valuable in conducting essential TPVs and in providing technical support to the SED in policy

formulation such as the development of a needs-based teacher allocation formula and a school-

specific NSB formula. However, there is a concern with regard to the choice of NER as a PDO

indicator and the overambitious targets for school participation.

29. The choice of using NERs as PDO indicators to measure school participation may not have

been appropriate for various reasons. First, although the official primary school entrance age is 5

years, in Punjab, 18 percent of children begin schooling at the age of 6 or older.18 Given that late

entry into the school system is prevalent in Punjab, an indicator that measures age and grade-

appropriate enrollment, such as NERs, may not be a reliable measure of school participation in the

province. Second, an analysis of both NER and SPR trends indicates that there is greater volatility

and hence lesser predictability in NERs as compared to SPRs, which makes forecasting enrollment

16 The 2018 Education Goals for Punjab consist of four areas (a) ensure high-quality teaching and learning in the

classroom, (b) improve leadership and accountability at all levels, (c) enable a conducive learning environment for

students, and (d) provide support with high-quality school infrastructure. 17 PESP II PAD, page 4. 18 Statistics derived from PSLM 2013–14.

9

trends more difficult.19 In addition, the results framework for the project was overly complex, with

more than 50 PDO and Intermediate Outcome indicators to be tracked under the project.

30. The PDO targets for increasing SPRs were overambitious given the project timeline. The

fact that the project duration, as originally conceived, was three years, the NER PDO targets were

too ambitious.20 The estimated impact of the project on the provincial net enrollments may not

have been achievable given that the interventions that were expected to directly contribute to

increasing NERs were unlikely to have an impact on enrollment in such a short period. The short

duration of the project also made restructuring the project more difficult for the task team- by the

time household survey data documenting the drop in NERs was made available, the majority of

project activities had already been completed. Although the project closing date was extended,

this was only to allow the SED to complete pending TA activities.

31. As indicated above, the SED was able to implement all project activities satisfactorily. The

project was able to take advantage of the appetite for decentralization, by restructuring a

subcomponent of the project related to devolving financial management powers from district

offices to school clusters. The project was able to go a step further than what was original

envisaged and devolve these financial management powers to SCs, allowing for greater

transparency and autonomy by giving schools greater control of their NSBs. The relevance of

implementation is also reflected in the fact that nine out of the ten reforms supported by the project

(all reforms except teacher performance incentives which is currently being redesigned) are still

being implemented by the SED.

3.2 Achievement of Project Development Objectives

32. The PDO is to support Punjab’s education sector reform program in order to increase child

school participation (at multiple levels) and student achievement. There are essentially three

distinct objectives of the PDO which are evaluated to verify overall achievement of the PDO; (i)

to support the PESRP II, (ii) to increase school participation (at multiple levels) and (iii) to increase

student achievement. At project completion, the project had successfully supported the PESRP,

as reflected through the timely achievement of DLIs which were ensconced within the reform

program, and the student achievement-related objective was also achieved. While, the school

participation-related objectives (as measured by NERs) were unmet, there was an increase in

school participation in the province as measured through school participation rates for the same

cohorts. Given that two out of the three objectives were met and some progress was made in

increasing school participation rates (as measured by SPRs), the performance of this project based

on its achievement of the PDO is Substantial.

33. Support to the PESRP (Substantial). The project was able to successfully support the

reforms under PESRP II as demonstrated through achievement of DLI targets: First, the project

was able to effectively strengthen and expand field-based mentoring support to teachers beyond

what was required in the project, that is to all public primary schools in the province. The project

was able to support the merit-based recruitment of 33,832 teachers, and developed a needs based

19 Pakistan Education Indicators Analysis (2017). 20 The NER targets were established by estimating the relationship between NERs and time and multiplying the

forecasted values by 50 percent (the estimated impact of the project) to establish the primary NER target and 25

percent to establish the middle and matric-level NER targets.

10

teacher allocation formula and implemented teacher rationalization in all 36 districts (surpassing

the target of 27 districts). The project was also able to support PESRP’s reform to devolve

administrative and financial management powers to schools (instead of school clusters as

originally envisaged) by providing SCs the authority to plan and utilize non-salary budget

allocations. SC capacity to plan, budget and utilize funds was further strengthened through an ICT

based mobilization program the design and implementation of which was also directly supported

by the project. Additional PESRP II reforms that were successfully supported by the project

include the Education Voucher Scheme (EVS) to provide access to schooling for the poor and

marginalized which exceeded voucher distribution targets and the girls’ stipend program which

provided 411,000 girls with stipends to attend secondary school. A detailed assessment of project

DLIs is provided in annex 2. In addition, as stated earlier, the project also successfully supported

other reform initiatives which were part of the GoPunjab’s PESRP (but not DLIs) by supporting

the strengthening of the CMMF which allowed monitors to survey more than 90 percent of public

schools every month to provide information for accountability at the provincial and district level

and provided support to PEC to administer annual grade 5 and 8 examinations and to disseminate

these results to stakeholders. Overall, the reforms supported through the project, were sustainable

with all reforms either being carried forward by the government or being redesigned. In fact, out

of the ten reforms supported through project DLIs, 9 are being carried forward in the follow-on

project PESP III, while the teacher performance incentive program is being redesigned through

the TA component of the project.

34. Although there was no indicator included in the results framework of the PAD to measure

the project’s overall support to the reform program, the project’s Implementation Status and

Results Reports (ISRs) tracked the achievement of individual reforms (in the form of DLIs) as

intermediate outcome indicators.

35. Increase child school participation (at multiple levels) (Modest). As per the results

framework, child school participation was to be measured by NERs at three levels of schooling:

primary (ages 6 to 10), middle (ages 11–13), and matric (ages 14–15). The project fell short of

meeting school participation-related targets as measured by NERs. However, based on alternate

measures of school participation, it is evident that the share of children enrolled in school in the

same age cohorts actually increased over the project period. As stated earlier, the SPR (including

enrollment in katchi) may be a more appropriate indicator than the NER in the context of Punjab

where late entry to the schooling system is common.21 At the primary level, NERs fluctuated over

the course of the project, with a baseline value of 70 in 2010–11, reaching a high of 74 in 2011–

12 and an end line value of 70 by 2014–15, falling well short of the target of 75. A similar trend

was observed at the middle level, with NERs reaching a high of 40 in 2011–12 and an end line

value of 38 against a target of 44. At the matriculate level, NERs increased by 4 percentage points

21 SPR is calculated as the share of children in a respective age cohort who are enrolled in school regardless of grade

while the NER takes into account both age and grade appropriate enrollment. This differs from the SPR present in the

Results Framework analysis because this measure includes children enrolled in katchi grades, as well as children

enrolled in nongovernmental organizations, madrassas, or other private schools, while the SPR in the Results

Framework of the PAD excludes katchi-enrolled children and only includes children enrolled in either public or private

schools.

11

from 25 to 29 between 2010 and 2014; while, the target for 2014–15 was 32. Nevertheless,

alternate measures of school participation indicate increases in school participation across all age

cohorts: the share of children ages 6–10 who were enrolled in school in Punjab increased from

80.2 percent in 2010–11 to 82.9 percent in 2014-15. For 11–13-year-olds, the percentage of

students enrolled in school also increased from 75.4 percent in 2010–11 to 77.7 percent in 2014–

15, and for 14–15-year-olds, the percentage of children enrolled in school increased from 61.1

percent to 63.5 percent.22 Given that both NERs and SPRs are calculated from sample-based

household surveys which are dependent on up-to-date population census information for

determining the sampling frame and that the last population census in Pakistan was conducted in

1998, there is a need to verify these figures. Administrative enrollment data is also analyzed to

obtain a complete picture about enrollment outcomes in the province over the project period.

36. Other measures of school enrollment in Punjab reflect a more substantial increase in

student enrollment and school completion over the project period. Administrative data collected

from the government’s monthly monitoring data reflect that public school enrollment in grades

katchi through to 12 has actually increased by 10 percent, from 8.9 million students to 9.8 million

students between 2011 and 2014. 23 In addition, the number of public sector students appearing in

grade 10 board exams increased by 44 percent while the percentage of students passing grade 10

exams increased by 42 percent between 2012 and 2016. 24 An increase in enrollment was also

reflected in the interventions directly supported by the project, namely the EVS, which enrolled

an additional 147,000 children into low-cost private schools and the girls’ stipend program, which

saw an increase in beneficiaries from 382,623 to 411,000 beneficiaries between 2012 and 2014. In

addition, the programs under the PEF (which includes the EVS), which were funded by the

GoPunjab through PESRP II, drastically increased overall enrollments from approximately 1.2

million children to 2.1 million children during the project period.

Table 1. NER, SPR, and Enrollment Overview Punjab

Level Baseline

(2010–11)

End Line

(2014–15) Target

Increase

(%) Data Source

Primary NER (Ages

6–10)

70 70 75 0 PSLM Surveys (2010 and

2014)

Middle NER (Ages

11–13)

37 38 44 1

Matric NER (Ages

14–15)

25 29 32 4

Alternate Measures of School Enrollment in Punjab

SPR Ages 6–10 80.2 82.9 n.a. 2.7 PSLM Surveys (2010 and

2014) SPR Ages 11–13 75.4 77.7 n.a. 2.3

SPR Ages 14–15 61.1 63.5 n.a. 2.4

Student enrollment

(katchi–12)

8.88 million 9.77 million n.a. 10 MEA Data (2011–15)

PEF enrollment 1.2 million 2.1 million n.a. 75 PEF administrative data

22 All statistics have been derived from PSLM survey data 2010–11 and 2014–15. This increase in the SPR is also

cross-checked in MICS data 2011–14 which show a 2 percent ,1 percent, and 4 percent increase in enrollment for the

age 6–10, 11–13, and 14–15 cohorts, respectively. 23 MEA Data. Source: http://open.punjab.gov.pk/schools/home/dashboard_home. 24 Source: PMIU, Matric Board Exam Results Punjab (2012–16).

12

37. Increase student achievement (Substantial). The PAD highlights that the majority of

activities supported by the project focus on teacher quality and performance, and that “improved

student learning, in turn, helps retain students in school and attract new children to school.”25 Thus

ensuring student achievement is a critical aspect of the PDO that has the potential to increase

school participation in the medium to long term. Student achievement as monitored in the PDO,

increased by five percentage points between September 2014 and September 2016. 26 The PDO

target for student achievement was a five percentage point improvement over the baseline and this

target was achieved. 27

Table 2. Student Achievement Grade 3 Assessment

Baseline (September 2014) End Line (March 2016) Increase

Average Share of correct

answers, grade 3

55% 60% 5%

Source: DFID biannual Grade 3 assessment.

38. To validate the results of the sample-based assessment, and to assess student achievement

in grades 4,5 and 8, alternate measures of student achievement were also analyzed, namely the

Punjab Examination Commission’s grade 5 and 8 examination data, the Annual Status of

Education Report (ASER), and the Directorate of Staff Development’s monthly student

assessment results.28 PEC data (which are presented in the Results Framework analysis) report an

overall increase in grade 5 and 8 student achievement because mean scores increased by 25 and

15 percent for grades 5 and 8, respectively. The ASER results (which are reported based on student

learning outcomes) indicate mixed results in Urdu and mathematics between 2012 and 2014. The

share of children in grade 3 that could read a story in Urdu remained the same in 2012 and 2014

(approximately 27 percent). However, the share of grade 3 students that could perform 3-digit

division increased by more than 6 percentage points between 2012 and 2014. Likewise, Grade 4

and 5 assessment data of the Directorate of Staff Development (DSD) shows that, on average,

student achievement has increased from an average score of 32 out of 100 in November 2011 to

47 out of 100 by November 2013–14.29 Overall, the general trend from the various data sources

indicate that there was an increase in student achievement over the duration of the project.

25 PAD, page 4 26 Given that DFID was providing parallel financing and was commissioning a grade 3 assessment, the World Bank

team decided to use the results from the DFID assessment to assess gains in student achievement over the life of the

project, however the baseline assessment was not conducted until September 2014. The assessment consisted of a

sample ranging from 231–237 government schools in the province, and covered grade 2 student learning outcomes,

in Urdu, English and mathematics. 27 It is important to note that the PDO in the PAD refers to assessments of grade 4 and grade 5 students, while the

DFID-funded assessment which was used for reporting against this PDO was for grade 3 students. 28 ASER is a privately conducted, provincially representative citizen-led household survey which aims to provide data

on the status of learning and schooling of the official school-going-age population (5–16 years). The DSD’s monthly

assessment takes place in all public primary schools in the province in grades 3, 4, and 5 in English, Urdu, Islamic

Studies, mathematics, science, and social studies. 29 DSD’s Monthly Assessment Data (2016).

13

Table 3. Alternate Measures of Student Achievement - ASER and DSD

ASER 2012 2013 2014

Percentage of grade 3 students who can read a story in Urdu 26.5 31.3 26.8

Percentage of grade 3 students who can perform 3-digit division 13.1 20.7 19.2

2011 2012 2013

DSD Monthly Assessment Mean Score (November) 32 43 47

Note: Reading a story is a class 2 level competency while division is a class 3 level competency as per the

National Curriculum 2007. DSD assessment data for 2014 were not available.

39. There were various project-supported activities that were targeted toward improving

student achievement through improved quality in the classroom. First, the project supported the

merit-based recruitment of teachers, because more than 33,000 teachers were recruited to address

teacher shortages, out of which approximately 60 percent were science and mathematics teachers.

Second, the project incentivized the reallocation of teachers based on school-specific needs to

address teacher shortages in the province. Third, the project also supported field-based

professional development of teachers in primary schools across the province through which

teachers benefitted from field-based mentoring support from teacher educators. These

interventions focused on improving teacher quality and in doing so, contributed to improving

learning outcomes in the classroom that were observed during the project period.

3.3 Efficiency

40. The overall efficiency of PESP II was Substantial. The project had an economic internal

rate of return (EIRR) of 13 percent, and efficiency gains were also evident in the form of timely

achievement of DLIs, efficient provision of funds to schools, and increased efficiency of the

education sector.

41. EIRR. A cost-benefit analysis (CBA) conducted at the time of the ICR reflects a positive

rate of return for the project. The scope of the analysis follows the same approach used at project

appraisal by conducting a CBA of the GoPunjab’s education reform program (PESRP II). 30

Benefits arose from three primary sources: (a) increased primary and secondary school completion

(as a result of increased intake and transition); (b) increased efficiency in the education system (as

reflected by decreased dropout and repetition resulting in fewer years required to produce primary

school graduates); and (b) increased wage premium for school graduates (due to increased

productivity as a result of improved quality of education). Costs were measured as public

expenditures on education (both recurrent non-salary and development) and other household

private expenditures on education as well as the opportunity cost of education, in the form of

forgone wages. The EIRR of PESRP II was 13 percent, indicating that the reform program was

economically viable (refer to annex 3 for more details).31

30 Given that counterpart funding for the project as reflected in the PAD includes all GoPunjab expenses toward

PESRP II, and that the project itself is also supporting the government’s program, the CBA covers the entire PESRP

II. 31 This rate of return is comparable to other education projects in the region, such as the Nepal School Sector Reform

Program which had an EIRR of 18 percent.

14

42. Efficiency in provision of NSB. The NSB reform which allocated school-specific NSBs

to schools, resulted in substantial increases in per student spending: on average, schools spent more

than PKR 1,000 more per student. 32 Not only did this reform result in higher spending at the

school level, it served as a more efficient system to transfer funds from district accounts directly

to school accounts (as opposed to the previous system in which financial management powers were

more centralized) reducing the potential for fund leakages.

43. Efficiency in teacher management. Teacher management also improved over the project

period, as teacher absenteeism decreased from a baseline of 16 percent in 2011 to less than 6

percent in November 2016.

44. Although the project closing date was extended, this was only to ensure that the

GoPunjab had sufficient time to use TA funds to complete TA activities. Project DLIs were

completed within the time frame originally envisaged, aside from the teacher recruitment DLI

which was delayed due to force majeure (court cases); however, this DLI was eventually achieved.

3.4 Justification of Overall Outcome Rating

Rating: Moderately Satisfactory

45. The overall outcome rating of the project is Moderately Satisfactory based on modest

relevance, substantial achievement of the PDO, and substantial efficiency gains achieved under

the project, as discussed above.33

Table 4. Project Rating

Project Relevance Achievement of PDO (Efficacy) Efficiency Overall

Rating Sub dimension Objectives Design and

Implementat

ion

Support to

the PESRP

Increase

school

participation

Increase

student

achievement

Sub dimension

Rating

High Modest Substantial Modest Substantial — —

Overall

Dimension

Rating

Modest Substantial Substantial Moderately

Satisfactory

3.5 Overarching Themes, Other Outcomes, and Impacts

(a) Poverty Impacts, Gender Aspects, and Social Development

46. Poverty and gender. PESP II targeted some aspects of poverty primarily through two

interventions. The EVS offered tuition redemption vouchers to children from poor households

living in urban and peri-urban areas to attend low-cost private schools. The EVS successfully

enrolled more than 360,000 children into school by the end of the project providing children with

free-of-cost education. The Girls Stipend Program was another intervention that offered

32 Cambridge Education (2014): Non-salary budget expenditure review. 33 Given that the June 2013 project restructuring, which involved a change to a DLI, did not result in a change to the

results indicators, a split evaluation is not required.

15

conditional cash stipends to girls in government secondary schools in 16 low literacy districts,

which are also among the most economically disadvantaged districts in the province.

47. Employment. The project also supported merit-based recruitment of teachers, which

resulted in the employment of more than 33,000 teachers.

48. Community engagement. Under two reforms supported through the project, community

engagement played a crucial role. First, the SC mobilization reform leveraged the use of ICT by

reaching out to SC members once a month through scripted telephone calls. As a result, head

teachers reported an increase in council members’ activity and basic knowledge and understanding

about the roles and responsibilities of the SC.34 Through the school-specific NSB reform, SCs

were further empowered by having direct control over the utilization of their NSB. As a result of

these two interventions, SCs were actively engaged and empowered to work toward improving

school-level service delivery.

(b) Institutional Change/Strengthening

49. The various project activities helped institutionalize the use of data to inform policy making

for key reform areas. Specifically, the SED has used its own data on teachers, infrastructure, and

enrollment to determine school-specific teacher needs as well as school-specific NSB allocations.

This approach ensured greater transparency in teacher deployment as well as school financing, by

using data to inform policies.

(c) Other Unintended Outcomes and Impacts (positive or negative)

50. None.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

51. Not applicable.

4. Assessment of Risk to Development Outcome

Rating: Moderate

52. The risk that the development outcomes will not be maintained is Moderate based on the

following assessment:

(a) Country level. At the country level, there is a modest risk that political instability

could arise and result in significant changes in government priorities. In addition,

governance remains weak, and institutions do not possess a strong framework to

ensure transparency and accountability in service delivery. Macroeconomic risks are

substantial due to expansionary fiscal spending and vulnerability to natural disasters.

Security risk is also substantial given the ongoing militant insurgency and military

operations.

34 Cambridge Education (2014) School Council Mobilization Report-study reported that 87 percent of head teachers

surveyed sensed an increase in council members’ motivation while 90 percent of the SC members reported that their

awareness of their SC responsibilities increased because of the program.

16

(b) Sectoral level. At the sectoral level, there are moderate risks that may affect the

sustainability of development outcomes such as ensuring adequate financing for the

basic education sector. In recent years, financing has been increasing steadily in

nominal terms, and education budgets have accounted for approximately 17 percent

of the overall provincial budget; however, suboptimal utilization of the development

budget remains a concern.35

(c) Mitigation of risks. Continued dialogue with all levels of government and analytic

work are expected to help mitigate risks at the country and sectoral levels. The World

Bank is working with development partners and government counterparts to maintain

the momentum for key policy reforms (for example, governance and tax reforms,

power sector, and strengthening social safety nets). In the education sector, the World

Bank is actively engaged with all levels of government and other stakeholders, to

ensure that education remains a key priority on the reform agenda.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry


53. The World Bank performance in ensuring quality at entry is rated Moderately Satisfactory.

The PDOs were strongly relevant given the sector context (both at the time of project preparation

and completion) and were closely aligned with the CPS and the GoPunjab’s own education reform

programs. The task team worked closely with government officials and donors in preparing the

project, and the project used the same results-based design as its predecessor (PESP), linking

disbursements with achievement of targets. The inclusion of a TA component was critical in

providing the SED with technical support in designing and verifying the implementation of

complex reforms, such as teacher rationalization, school-specific NSBs, and ICT-based SC

mobilization. The World Bank team also ensured that the design of the project was in full

compliance with the World Bank’s relevant fiduciary and safeguards policies. All relevant risks

were identified and appropriate mitigation measures were outlined for the project. The project

design also incorporated some lessons learned from previous engagements in the education sector

in Punjab by engaging in close coordination with donors in the sector, using a results-based design,

and ensuring greater precision in formulating DLIs.

54. However, the PAD would have benefited from detailed analysis on the impact of project

interventions toward school enrollments. The choice of the PDO indicator (NER) to measure

school participation could have also been analyzed further. The project could have benefited from

incorporating a key lesson learned from the PESP regarding the selection of PDO indicators. The

ICR for the PESP identified the need for strong analytical work to select PDO indicators, with an

emphasis on measuring the impact of successful achievement of DLIs on the PDO indicators. The

school participation targets, keeping in mind the three-year time period of the project, were

35 Development budget execution rate has ranged from 13 percent to 64 percent between 2010 and 2014 for preprimary

and primary education affairs and services (Education ADP Review 2016).

17

overambitious. The results framework was overly complex with too many indicators which could

have been simplified.

(b) Quality of Bank Supervision

Rating: Satisfactory

55. The quality of World Bank supervision is rated Satisfactory based on the assessment in the

following paragraphs. The World Bank conducted regular supervision missions with a

multidisciplinary team consisting of education, procurement, financial management, and

environment and safeguards specialists. These supervision missions were also conducted in close

coordination with other development partners, including DFID and CIDA. The World Bank team

was well supported by a mix of national consultants with expertise in ICT-based social

mobilization, public-private partnerships in education, and conditional cash stipends. The World

Bank team also engaged in high-level policy dialogue on issues relating to education sector finance

and education sector reform. In addition, implementation issues that arose over the course of the

project (such as the SED’s desire to restructure the school-specific NSB reform DLI) were

addressed on time.

56. There were some delays in establishing a baseline for the student achievement PDO

indicator for the project. Given that DFID was providing parallel financing for the project, the

World Bank team decided against conducting its own independent assessment and instead opted

to use the results of the DFID-funded early grade assessment. As a result, the baseline for the

sample-based student assessment was not established until September 2014, when the DFID grade

3 assessment was finally initiated. There were two main issues that contributed to the decision not

to restructure the project to incorporate a new indicator for school participation and more realistic

targets. These are erratic trend in NERs, and the timeliness of data availability. During the first

year of the Project, the PDO target for primary school participation was almost achieved with

NERs touching 74 (against a target of 75).36 Based on this initial surge, the task team would have

thought they were well placed to achieve the participation related PDO target. In subsequent

rounds of the household survey, the Primary NER was 72 which meant that approximately 50

percent of the target had been achieved, again leaving little cause for concern for restructuring. In

relation to the second issue on the timing of data availability-by the time the end line data (which

reports a decline in the NER back to 70) was publically available (March 2016), the majority of

project activities had been completed and there was little sense in restructuring the Project. The

Project closing date was extended after the majority of Project DLIs had been achieved to allow

the SED to utilize unspent TA funds.

(c) Justification of Rating for Overall Bank Performance


57. The overall World Bank performance is rated Moderately Satisfactory based upon the

Moderately Satisfactory rating at entry and the Satisfactory rating for supervision.

36 PSLM 2011-12

18

5.2 Borrower Performance

(a) Government Performance


58. The GoPunjab’s commitment to the project was strong, as evidenced by the successful

implementation of the project and achievement of all project DLIs. The SED went a step beyond

what was originally envisaged for the school-specific NSB reform, by taking the decision to

provide SCs with the financial powers to execute their own NSBs. Similarly, the SED effectively

recruited teachers through test-based recruitment and withstood political pressure and court cases

to complete the recruitment process. The GoPunjab also successfully contested the court ordered

suspension of the teacher rationalization policy and was able to effectively issue teacher transfer

orders province wide rather than in 27 districts as originally planned despite resistance from

teacher unions. District administrators also increased oversight of schools; in 2011, 55 percent of

schools had been visited by the district administration while in 2015, 96 percent of schools had

been visited. In addition, the provision of basic facilities to schools drastically increased between

2011 and 2015; as a result, 99 percent of schools have access to clean drinking water, 98 percent

have a toilet facility, 96 percent have a boundary wall, and 90 percent have electricity. In addition,

the GoPunjab demonstrated strong commitment to ensuring increased non-salary financing in the

education sector by increasing non-salary expenditure from PKR 9 billion in 2011 to

approximately PKR 14 billion by 2015.37

59. There were some areas in which the GoPunjab’s performance could have been improved.

For example, while the SED took initial steps to develop a teacher licensing and accreditation

framework, the framework still requires consultation and is yet to be finalized keeping in mind the

political economy risk associated with teacher licensing. Greater political support could have

pushed this reform during the life of the project; however, recently the government has decided to

pursue this reform more actively and it is possible that with the right level of political commitment,

the teacher licensing framework could be developed and implemented in the coming years.

(b) Implementing Agency or Agencies Performance


60. The SED/PMIU was identified as the primary implementing agency for PESP II. As

mentioned earlier, the SED/PMIU met all 30 DLIs of the project demonstrating strong

commitment to implementation of the project. All legal covenants, except one relating to teacher

licensing and certification, were complied (or partially complied) with. Project implementation

progress was rated as Moderately Satisfactory in the first two ISRs; however, implementation

progress gained momentum, and all subsequent implementation progress ratings were rated

Satisfactory. The SED also expanded the scope of the continuous professional development

program (beyond what was outlined in the PAD) by providing field-based support to all primary

school teachers and implemented province-wide teacher rationalization rather than limiting it to

27 districts. Throughout the project life, the PMIU had dedicated Deputy Directors responsible for

the monitoring and achievement of specific DLIs, which proved useful in ensuring regular and

adequate follow-up of project activities. The other implementing agencies such as the PEF and

37 PMIU 2016: http://open.punjab.gov.pk/schools/home/dashboard_home.

19

DSD were also able to implement project activities satisfactorily and in the case of the PEF, even

surpassed the project targets for the EVS. However, under Component 2, procurement

performance throughout the project had some room for improvement because there were delays in

meeting planned procurement timelines and in updating the Procurement Plan to accurately reflect

the progress of the project. Payments to two contracted firms were also significantly delayed,

which could have been avoided, provided there was an increased procurement capacity and

stronger leadership within the PMIU. High turnover of Project Directors and Deputy Directors

within the PMIU was also an issue. Between project preparation and project completion, there

were a total of five Project Directors at the PMIU.

(c) Justification of Rating for Overall Borrower Performance


61. The overall performance of the Borrower is rated Satisfactory, based on the preceding

assessment of government performance (Satisfactory) and implementing agency performance

(Satisfactory).

6. Lessons Learned

62. PDO-level indicators and targets must be carefully selected, and the use of SPRs may

be more appropriate to measure school participation in Punjab. There is a need to ensure that

PDO indicators relating to enrollment are carefully analyzed for future projects. An analysis of

primary NERs in the province reflects that there is volatility in the indicator and may not be an

appropriate choice to measure enrollment outcomes in Punjab. SPRs38 may be a more appropriate

choice to measure enrollment outcomes, because there is less volatility in the indicator and greater

predictability. 39 PDO targets should also be set by determining the contribution of each

subcomponent of the project toward the development objective to avoid unrealistic and

overambitious targets. This would also involve making a distinction between the project and its

expected outcomes and the larger reform program of the Government.

63. The use of a results based lending instrument and specifically DLIs help ensure a

focus on outcomes, and the sustainability of reforms. The results-based instrument used in both

PESP I and PESP II, specifically the use of DLIs and clearly defined performance targets, has

contributed to donor harmonization and increased attention to the achievement of results. This also

helped in mainstreaming key education reforms such as the school-specific NSB reform. A

common Results Framework also limited the reporting burden of the Government.

64. Strategically placed TA managed by the SED can help improve the design and

implementation of education reforms. Strategically placed TA managed by the SED played an

important role not only in informing the design of complex reforms but also in providing technical

support to implementing agencies in implementing and verifying the achievement of reforms. It is

critical that the TA component provides focused capacity building in the area of procurement

38 SPRs are measured as the share of children within a respective age cohort who are enrolled in school regardless of

grade level. 39 Pakistan Education Indicators Analysis (2016).

20

specifically in contract management, documentation, and record keeping in order to ensure more

effective implementation of reforms.

65. With the rapid scale up of PPPs in Education, there is a need to review the targeting

of PPP programs and focus on strengthening quality assurance. Rigorous evidence suggests

that low-cost private schools in Punjab have generated enrollment and learning achievement gains

and these efforts need continued support. Rapid scale-up of PEF programs led to increased

enrollment in PEF-supported schools. However, this increase did not proportionately translate to

increases in school participation for the province. There is a need to review the targeting of PEF

programs to ensure that the most underserved areas of the province are being targeted and that

programs are effectively enrolling and retaining out-of-school children. This would include

strengthening existing quality assurance systems and ensuring that adequate capacity within the

PEF is available to implement these programs.

66. District education administrations require further strengthening to improve

education service delivery. In addition to the accountability systems in place at the provincial

level, districts require further strengthening to improve service delivery and improve the

accountability of district staff and schools. Greater clarity from the government is also required on

the role of the newly established District Education Authorities, specifically on how these

structures will coexist with existing governance structures. While district-level accountability

systems are in place in the form of the road map, there is a need to shift from the narrow focus on

performance against a set of indicators, to implementing and owning education reforms. There is

also a need to ensure that future engagements in the sector prioritize poor performing districts

(both with regard to enrollment and learning outcomes), to improve overall outcomes for the

province.

67. While school level monitoring is critical, there may be a need to revisit existing

approaches to monitoring to ensure that monitoring does not adversely impact teaching and

learning activities. Over the last five years, Punjab has managed to establish a robust monitoring

regime which has increased the potential of decision makers to take evidence-based decisions.

However, the emphasis on data collection and monitoring has placed an additional strain on

schools. 40 In many understaffed schools, this can contribute to a reduction in teacher time on tasks,

with head teachers being pulled away from teaching duties to provide information to various

informants. This system needs to be reevaluated to ensure that data collection and monitoring does

not hinder the teaching duties of teachers.

68. There is a need to strengthen the quality of data and to integrate various sources of

data to provide decision makers relevant information. There are various sources of data being

collected by the attached departments. There is a need to strengthen the quality of data in the allied

institutions such as the PEC, DSD, and PEF and to ensure that data can be integrated and fed back

into the design of education reforms. There is also a need for greater transparency in the sharing

of data and a need to integrate the various data sources, to enhance utilization of data and promote

evidence-based decision making particularly at the district and sub-district level.

40 Based on consultations with school teachers, the team was informed that schools in Punjab can experience 3–6 visits

per month from various officials associated with the SED (MEAs, District Teacher Educators [DTEs], Assistant

Education Officers, and District Education Officers, among others).

21

69. Developing and implementing ESMF in operations that do not involve in construction

activities, can still have a positive environmental impact which should be part of future

interventions in the sector. Although the project did not support activities that would have had a

detrimental environmental or social impact, the development and implementation of an

environment and social management framework for the SED had a positive impact on the

environment. Future engagements in the sector should continue to sensitize district officials,

school teachers, and SCs on environmental issues; environment clubs should be established in

additional schools; and tree plantation drives should continue.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners

(a) Implementing Agencies

70. The elements of the ICR were discussed with various stakeholders over the course of the

preparation of the ICR. The government team and the ICR team exchanged views on the project’s

outcomes as well as overall lessons learned from the project. The World Bank team has received

the GoPunjab’s ICR and its key findings, which are presented in annex 7.

(b) Cofinanciers

71. The World Bank ICR team met with representatives from DFID who were part of the

preparation team for PESP II. The representative from DFID expressed the opinion that while

project activities were completed satisfactorily, the targets for school enrollment may have been

overambitious given the nature of activities supported by the project.

(c) Other partners and stakeholders

Not applicable.

22

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in US$, million equivalent)

Components Appraisal Estimate

(US$, millions)

Actual/Latest Estimate

(US$, millions)

Percentage of

Appraisal

Component 1 299.478 220.159 73.50a

Component 2 8.853 8.955 101.15

Total Financing Required 308.331 229.114

Note: a. This is due to the fluctuation in the SDR relative to the dollar.

(b) Financing

Source of Funds Type of

Cofinancing

Appraisal

Estimate

(US$, millions)

Actual/Latest

Estimate

(US$, millions)

Percentage of

Appraisal

Borrower 3,837.70 7,394.00 193.00

CIDA 19.30 0.00 00.00

DFID 200.00 200.00 100.00

IDA 350.00 350.00 100.00

23

Annex 2. Outputs by Component

Component Target (FY2014–15) Achievements Remarks

DLI 1. Field-based

professional development

Primary grades of 100% of the poor schools in all

36 districts actively covered by the strengthened

field-based system for providing advisory support

to teachers at school.

The field-based support was scaled up to

all public primary schools in the province.

DTEs were recruited on a merit

basis (a total of 4,000 DTEs were

providing support to teachers across

the province) and provided with

resources to visit schools on a

monthly basis Monthly assessments of students

conducted by DTEs in all public

primary schools

The field-based support

exceeded the project targets of

only targeting poor schools

and instead expanded support

to all primary schools in the

province.

DLI 2. Teacher

recruitment

All teachers, both subject-specific and general,

recruited to fill confirmed school-specific,

rationalized needs, conditional on clearing a

recruitment test based on teacher standards

approved by the SED.

All teachers recruited during the project

period have been recruited through a test-

based process.

This DLI was met every year;

the last-year DLI achievement

was delayed due to force

majeure (prospective teachers

had registered a court case in

one district, which was

eventually resolved).

DLI 3. Teacher

rationalization

Allocation of sanctioned teaching posts to schools

in accordance with agreed school-specific, needs-

based formula completed and teacher

rationalization and recruitment against school-

specific, needs-based teaching posts implemented

in 27 districts

Transfer orders were issued by all 36

districts, according to the Teacher

Reallocation Policy 2013–14.

The SED decided to

implement this reform

province wide rather than

limit it to just 27 districts.

Implementation of this reform

could have been strengthened

with additional oversight of

district level implementation

by the PMIU/SED as

identified in the TPV

DLI 4. School non-salary

budgets

School-specific NSBs prepared in the

preceding year approved,

disbursed, and openly displayed at schools;

school-specific NSBs for FY2015/16 prepared in

accordance with an agreed funding formula and

submitted for inclusion in FY2015/16 district

budgets for all 36 districts; and an expenditure

review conducted by a third party.

School-specific NSBs were prepared and

disbursed in accordance with the needs-

based formula.

NSB allocation increased over the project

period from PKR 9 billion in 2011 to

PKR 14 billion in 2016–17. A TPV was

also conducted.

The majority of schools

developed school-based action

plans for the utilization of the

NSB with the involvement of

SC members.a

On average, schools spent

PKR 1,000 more per student

than in previous years

24


There were some delays in

releases from the Finance

Department as reported in the

TPV; however, over time,

these issues were minimized

as the reform matured.

DLI 5. Decentralized

resource management

Punjab has transferred the non-salary component

of the education-recurrent budget for primary and

elementary schools in at least 18 districts to the

accounts of the respective SCs within the first

month of each calendar quarter.b

School-specific NSBs were transferred to

primary and elementary schools to SC

accounts

The SED advocated for

greater school level autonomy

in the management of the

NSB; therefore, this DLI

devolved administrative and

financial management powers

to the SC level rather than the

school cluster level, as

originally proposed

DLI 6. Teacher

performance incentives

Continued implementation of adapted teacher

performance bonus program following a new

value-added model in targeted government

primary schools in at least 6 poor districts,

consistent with an agreed impact evaluation design

The program was implemented according

to the design, with the existing improvers

program being implemented in 3 districts

and the adapted improvers program being

implemented in 3 additional districts.

After achieving the final year

DLI, this program is no longer

being implemented. However,

the SED is in the process of

developing a teacher

motivation program to be

informed by a teacher

motivation survey.

DLI 7. School councils Reconstitution and operation of SCs in accordance

with rules and powers in revised school-council

policy, supported by a new capacity development

program, in at least 50% of schools in all districts,

and grant expenditures reviewed by a third party

SCs were reconstituted in 50% of schools

in all 36 districts and ICT-based capacity

building support was provided to the

council members. A third-party

expenditure review was also conducted.

Findings from the TPV

indicate that there was overall

compliance with the SC

policy and that 79% of the

sample SC members surveyed

were contacted once a month

through the ICT-based

capacity-building program.c

DLI 8. Provision and use

of system and school

performance information

District and SRCs, following the agreed format

and contents, prepared and delivered on a quarterly

basis to all district education administrations and

on a semiannual basis to 100% of primary, middle,

and high schools and SCs and openly displayed at

schools.

District and SRCs were prepared and

distributed to district education

administrations and primary, middle, and

high schools.

A TPV found that 75–85% of

focus group participants found

that the SRC was very useful.

Implementation could have

been further strengthened by

ensuring that SRCs were

displayed outside the school

premises rather than within.d

25


DLI 9. Private school

vouchers

Vouchers to attend private schools under a quality-

assurance system expanded to cover additional

disadvantaged children in 36 districts, in

accordance with agreed adapted voucher program,

and consistent with agreed impact evaluation

design.

All targets were achieved each year, with

vouchers being offered in all districts with

a total of 361,000 voucher students

supported by the program.

Voucher beneficiaries

exceeded targets under the

results and monitoring

indicators of the project.

New beneficiary screening

instruments were used; quality

assurance tests were regularly

conducted in partner schools.

DLI 10. Stipends for

secondary school girls

Attendance-tied supplemental stipend benefits

following agreed benefit structure offered to

beneficiary girls in grades 6–10 in all rural

government schools in existing stipends districts.

Intended beneficiary girls in grades 6–10 in 100%

of urban government schools and 50% of rural

government schools receive stipend benefits

through agreed branchless banking options.

Supplemental stipends were provided to

female students in rural government

schools in existing stipend districts, with

branchless banking targets being met.

The SED has decided to adopt

a card-based payment system

for all stipend beneficiaries (in

16 districts) following the

pilot.

Note: a. The TPV reports that 79 percent of primary and 86 percent of elementary schools surveyed developed school-based action plans (Cambridge Education

2013).

b. This is the restructured version of the DLI; the original version stipulated the devolution of administrative and financial management powers to Education

Officers at the school cluster level rather than to individual schools.

c. In addition, 82 percent of SC members surveyed reported that their understanding of school functioning increased as a result of the ICT-based capacity-building

program (Cambridge Education 2014).

d. Cambridge Education (2013).

26

Annex 3. Economic and Financial Analysis

1. This annex summarizes the economic rationale for PESP III by presenting (a) the economic

analysis of the project and (b) the financial analysis of the project.

A. Economic Analysis

2. This economic analysis presents a CBA of PESRP II using the present discounted value

method. Economic costs and benefits are estimated over the four years of the project relative to a

counterfactual. Costs include SED budget expenditure, project disbursements, and private costs

which include direct household expenditure and opportunity costs of schooling over the project

period. Economic benefits comprised (a) the number of additional primary and secondary school

completers in public and PESRP II-supported private schools earning higher wages relative to non-

completers, (b) improved quality and relevance of education resulting in a higher productivity

premium for all primary school completers (both for public schools and private schools), and (c)

lower repetition and dropouts yielding benefits in the form of reduced wastage of public and

private resources (for public schools) relative to the counterfactual (assuming no PESRP II).

3. The following assumptions and data sources were used to estimate the net present value

(NPV) of program benefits, the benefit-to-cost ratio, and the EIRR for PESRP II.

(a) Public school enrollment (grades 1–10) data are from the ASC data from 2011 through

to 2015. Private school enrollment figures are from PEF administrative data. Data on

primary and secondary school completers are calculated from these two sources. The

counterfactual scenario assumes that enrollment remains at 2011 levels over the

project period.

(b) PESRP II program costs include SED non-salary recurrent and development

expenditure over the project period. As with the benefit stream, costs are estimated by

determining the differential impact of PESRP II versus a no PESRP II scenario. As a

result, it is assumed that recurrent salary costs would have been met regardless of the

program intervention, and that non-salary recurrent and development expenditures are

part of the additional investments required to support PESRP II. The total program

expenditure over the four years of the project is US$1.7 billion in nominal terms.

(c) Private per child household outlays on education are calculated using PSLM survey

data 2013–14, by calculating the average annual expenditure per child on education

across respective grade levels (1–5 for primary and 6–10 for secondary). On average,

households spend approximately US$27 per year for children enrolled in primary

grades and US$59 per year for children enrolled in secondary grades. It is assumed

that 24 percent of primary school-age children and 34 percent of secondary school-

age children would have been engaged in paid labor had they not enrolled in school,

based on the labor force participation rates derived from PSLM survey data (2013–

14).

(d) Lifetime labor earnings are estimated using the PSLM survey data (2013–14) to

determine benefits due to increased earnings for both primary and secondary

completers. Lifetime earnings are calculated for four cohorts over a period of 20 years.

27

Age-earning profiles are created for various categories of education: less than primary

education, primary education, middle school education, and secondary education to

calculate wage premiums for primary versus non-primary completers and secondary

versus non-secondary completers. It is assumed that the improvements in the quality

of education that took place during the project period (as evidenced through

improvement in student assessment scores) will result in a 3 percent productivity

premium for primary school completers and a 2 percent productivity premium for

secondary school completers over the existing wage premium (2 percent is assumed

for the low case in the sensitivity analysis, while 4 percent is assumed for the high

case for primary completers, and 1 percent and 3 percent is assumed for secondary

completers for the low and high case respectively). Labor force participation (LFP)

rates are assumed to be 45 percent for primary and 46 percent for secondary

completers in the base case, based on LFP rates calculated using PSLM 2014–15. For

the sensitivity analysis, LFP rates are assumed to be 3 percentage points lower than

the base case (for the low case) and 3 percentage points higher than the base case (for

the high case) for both primary and secondary completers.

(e) It is assumed that due to reduced repetition and dropout rates because of the various

quality-related reforms supported by PESRP II, efficiency benefits are experienced in

the form of savings due to the reduced number of years to produce primary school and

secondary school completers. Based on the comparison of the actual promotion and

repetition rates during the project period, and the counterfactual, which is the

repetition and promotion rates from 2011, it is estimated that PESRP II reduced the

number of student years to produce a primary school graduate by 0.68 student years.

(f) The discount rate assumed for this analysis is 10 percent, while inflation is assumed

to be 7 percent per year (based on a three-year average of the consumer price index).

4. Based on the assumptions above, the NPV of economic benefits under PESRP II is US$404

million, with an estimated EIRR of 13 percent. Table 3.1 presents a summary of the sensitivity

analysis with the NPV of economic benefits, the EIRR, and the benefit to cost ratio under the base,

low, and high case scenarios.41 As reflected in table 3.1, the NPV ranges from US$160 million to

US$666 million while the EIRR ranges from 11 percent to 15 percent under the various scenarios

indicating that PESRP II was economically viable.

Table 3.1. Sensitivity Analysis

NPV (US$, millions) Benefit-to-Cost Ratio Internal Rate of Return (%)

LFP

Base

Case

LFP

Low

Case

LFP

High

Case

LFP

Base

Case

LFP

Low

Case

LFP

High

Case

LFP

Base

Case

LFP

Low

Case

LFP

High

Case

Quality

premium base

case

404 286 523 1.23 1.17 1.3 13 12 14

41 Assumptions relating to the LFP rate and quality premium under the project are varied for the sensitivity analysis

and are specified in the assumptions section above.

28

NPV (US$, millions) Benefit-to-Cost Ratio Internal Rate of Return (%)

LFP

Base

Case

LFP

Low

Case

LFP

High

Case

LFP

Base

Case

LFP

Low

Case

LFP

High

Case

LFP

Base

Case

LFP

Low

Case

LFP

High

Case

Quality

premium low

case

270 160 380 1.16 1.09 1.22 12 11 13

Quality

premium high

case

539 411 666 1.31 1.24 1.39 14 13 15

5. Economic analysis at appraisal. Given that there is limited information provided in the

PESP II PAD regarding the economic analysis at appraisal (the annex for the economic analysis

was not included in the PAD), comparison of the NPV of economic benefits and the EIRR is

difficult. The information provided in the main text in the PAD reports an NPV of economic

benefits of PKR 2,945 per ‘representative child’ (in 2000–01 rupees). The NPV of economic

beneficits per student (in 2000–01 rupees), as calculated for the ICR, is PKR 1,244.

B. Financial Analysis

6. Over the project period, the GoPunjab demonstrated strong commitment to the education

sector by increasing public allocations and expenditure in the education sector as reflected in table

3.2. Since 2012–13, the provincial government’s spending on school education has increased by

30 percent, while non-salary and development expenditure has more than doubled. The project’s

contribution to the overall education sector has ranged from less than 1 percent in the final year of

the project to as high as 8 percent in 2013–14, suggesting that the program had a substantial

contribution to the SED’s reform efforts.

Table 3.2. Public Education Expenditure Trends, GoPunjab (PKR, millions)

2012–

13

2013–

14

2014–

15

2015–

16

SED Salary Expenditure 139,914 146,532 153,183 167,045

SED Non-salary + Development Expenditure 18,170 30,474 32,193 39,224

Total SED Expenditure 158,084 177,006 185,375 206,269

Overall SED Budget Execution Rate 95% 90% 90% 92%

PESP II Disbursement (in PKR) 4,259 14,440 13,539 250

PESP II Contribution to Overall Government Expenditure

(percentage) 3 8 7 0

Note: SED expenditure figures are from the PMIU extracted from Accountant General Data.

29

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/

Specialty

Lending

Javaid Afzal Senior Environmental Specialist GEN03

Khalid Bin Anjum Senior Procurement Specialist GGO06

Umbreen Arif Senior Education Specialist GED06

Anwar Ali Bhatti Financial Analyst SACPK

Zubair Khurshid Bhatti Senior Public Sector Specialist GGO14

Riaz Mahmood Financial Management Analyst GGODR

Amna W. Mir Senior Program Assistant SACPK

Hanid Mukhtar Senior Economist GMFDR

Dhushyanth Raju Senior Economist SARCE

Saeeda Sabah Rashid Sr Public Sector Specialist GGO14

Uzma Sadaf Senior Procurement Specialist GGO06

Martin M. Serrano Senior Counsel LEGES

Elfreda Vincent Temporary GED06

Huma Ali Waheed Education Economist GED01

Supervision/ICR

(b) Staff Time and Cost

Stage of Project Cycle

Staff Time and Cost (Bank Budget Only)

No. of Staff Weeks US$, thousands (including

travel and consultant costs)

Lending

Total: 43 143,914

Supervision/ICR

Total: 167 574,424

30

Annex 5. Beneficiary Survey Results

Not applicable

31

Annex 6. Stakeholder Workshop Report and Results

Not applicable

32

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

1. Assessment of the Operation’s Objectives, Design, Implementation and Operational

Experience

1.1. Project Context

1.1.1. Since 2003, the GoPunjab has been implementing a multifaceted sector reform program,

which mainly supports public education at the primary and secondary levels (up to grade 10),

called the PESRP. Under the PESRP, the GoPunjab has focused on developing administrative

systems (for example, regular measurement of student achievement through universal testing),

improving the provision and quality of inputs (for example, textbook delivery, school

infrastructure development, teachers through merit-based recruitment, school funds through SCs,

and training SCs), and the development of interventions that strengthen incentives and

accountability for service delivery performance (for example, teacher performance pay,

achievement- and enrollment-tied subsidy payments to low-cost private schools).

1.1.2. The PESRP came to an end at the close of this fiscal year (FY2011/2012). Building on the

institutional, administrative, and program foundations laid by the PESRP, the GoPunjab developed

its next medium-term sector reform program for primary and secondary education, PESRP II,

which aimed to better orient and ratchet up efforts to address the remaining deficiencies in

participation and achievement outcomes. To improve outcomes, the main challenge in public

education is to improve teacher quality and performance. The teacher is the main instrument for

student learning in this context, so many of the initiatives can affect the child only through the

teacher. Improved student learning, in turn, helps retain students in school and attract new children

to school. Recognizing that the participation returns from learning gains may take time to

materialize, PESRP II also included initiatives that directly attempt to increase participation at

different levels.

1.2. Project Development Objectives

1..2.1. The overall objective of PESP II was to support the GoPunjab in improving child school

participation (at multiple levels) and student achievement.

1.2.2. The objectives were relevant at the time of approval and remain relevant today. Significant

progress in improving access to schools and infrastructure had been made in the period before this

project but the participation rates and quality of learning remained an issue. Low NERs at all three

levels were symptomatic of the inefficiency and low quality of the education system. The project’s

objectives sought to address the remaining challenges in the primary education sector and to attain

universal primary education, defined as a quality primary education for all relevant children and

an equivalent level of literacy for adults.

1.2.3. The PDOs remained relevant throughout the project and were aligned with the GoPunjab’s

priorities in the education sector. The project objectives were also consistent with the objectives

of the FY2010–13 CPS. PESP II directly contributed to implementing the CPS fourth pillar of

improving human development, by supporting GoPunjab’s education sector reform program

33

which aimed to promote the above stated objectives, through strengthening systems, capacity,

governance, and accountability for improved service delivery performance.

1.2.4. Success of the project in meeting its main objectives was to be measured by the key outcome

indicators: NER at primary, middle, and matriculate levels and average share of correct answers

for English, Urdu, and mathematics tests administered for sample of grade 4 and grade 5 students.

1.2.5. The project also regularly monitored the achievement of DLIs to measure implementation

success of the program.

1.3. Project Design

1.3.1. The relevance of project design is rated Modest. Project activities were designed to be

complementary and synergistic and many activities supported more than one intended outcome.

There was a theoretical link between project inputs and intended outcomes.

1.3.2. Project activities were relevant to achieve the first objective of increased participation at all

levels. Given the improvements in net enrollments between 2007 and 2012 providing stipends and

private school vouchers to improve access to education was highly appropriate.

1.3.3. However, the design for the objective to improve school participation had certain

shortcomings as well. Certain reform programs were designed to be pilot projects and were

discontinued as soon as the pilots were executed. This affected the sustainability of the reforms.42

1.4. Implementation and operation experience

1.4.1. The project’s primary implementing agency was the SED/PMIU working in close

consultation with local education offices. Implementation progress remained satisfactory

throughout the project period. For Component 1, the GoPunjab met all the DLIs with some minor

delays. Implementation on Component 2 (Technical Assistance) remained satisfactory with almost

all the funds used within the project period. The following main factors positively affected the

implementation of PESP II:

Regular and sustained dialogue between the World Bank and SED/PMIU and

stakeholders at the district and school level facilitated steady implementation progress

in most aspects of PESP II. A majority of actions regarding the core components were

taken up in quarterly meetings. In the case of troubleshooting, focal persons from the

World Bank could be contacted effectively.

42 At the time of submission of the borrower’s ICR there were certain aspects of the project that had not been carried

forward (i.e. school report cards, teacher incentives program, and school council mobilization). However, since then

the SED has announced a way forward for each of these reforms, with directives to continue to display school report

cards in schools, to re-design a teacher incentives program, and to develop a school council mobilization strategy for

the coming years.

34

1.4.2. Main factors that slowed down implementation and hindered progress included the

following:

There was limited on-ground presence of the World Bank. While the World Bank

engaged every quarter with the SED through its planned missions, its limited on-

ground presence compared to other development partners such as McKinsey and

DFID limited its involvement in governance issues at the education department.

There was the need for more supervision of the TA by the World Bank. The terms of

references for TA drafted by the SED/PMIU were often not clearly defined and well-

structured and this led to costly procurement of TA services. For instance, the

procurement of Cambridge Education’s services was costlier than it should have been

owing to poorly structured terms of references.

The PMIU lacked the capacity and experience to deal with consultants. This affected

the progress of projects which required TA support.

1.5. Borrower Performance

Overall borrower performance is rated Satisfactory.

1.5.1. Over the project period, the GoPunjab reiterated its commitment and support to education

reform in Punjab. The GoPunjab released funds for core programs as needed, albeit with minor

delays at times. Apart from that, the SED and its PMIU demonstrated strong commitment and

ownership to achieving PESP II development objectives. The PMIU performed a key role in the

preparation and implementation of reform programs supported by the PESP.

1.5.2. Coordination between the SED and its PMIU and the development partners worked well.

The process followed a quarterly review with the World Bank and consultation cycle between

development partners such as McKinsey and DFID, where mutually agreed upon actions were

determined based on strategic areas in need of additional support as identified in Aide Memoires.

A majority of actions were taken up in these reviews.

1.5.3. The GoPunjab’s commitment to the project is also evident by the timely achievement of all

project DLIs and sustainability of reforms and programs under certain DLIs. Apart from the

improvers school bonus program, majority of activities are being carried forward to next phases.

1.6. World Bank Performance

Overall World Bank Performance is rated Satisfactory.

1.6.1. The focal persons at the World Bank actively engaged with the GoPunjab and department

officials in the preparation and implementation of PESP II. The World Bank team continued to

provide the necessary technical advice on important policy and implementation issues through the

project period. The World Bank team reviewed project progress quarterly through its planned

missions and took up necessary actions accordingly.

35

1.6.2. However, there are a few areas where the World Bank’s performance was lacking. First, the

design of the objectives had some shortcomings and gaps. Ideally the achievements of DLIs should

feed into the overall program development objectives. However, in PESP II, this was not the case

and while almost all DLIs were met, the PDO-level objectives were not. Second, in some cases,

the World Bank was solely concentrating on new streams and pilot projects and these pilot projects

were not scaled up or sustained. For instance, the SC mobilization program was discontinued even

after its effective pilot implementation.

2. Assessment of Outcomes

2.1. Achievement of Project Development Objectives

2.1.1. The PDOs were to support the GoPunjab in improving child school participation (at multiple

levels) and student achievement. Achievement of these objectives was to be measured using four

PDO level indicators. These indicators included (1) net enrollment rate (NER), for primary

education, (2) net enrollment rate (NER), for middle level, (3) net enrollment rate (NER), for

matriculate level, and (4) average share of correct answers, English, Urdu, and mathematics tests,

grade 4 and grade 5 students, sample schools. The key reform subprogram under PESP II and their

associated DLIs, linking results to disbursements, were tightly connected with the intermediate

outcome indicator. There was an increase in the NER for middle and matriculate levels while the

NER did not substantially change during the project years. At project completion, three of the four

PDO-level results indicators were not met.

Table 7.1 summarizes the progress on PDOs over the course of project implementation.

Table 7.1. Progress on PDOs

PDOs FY2011

Baseline

FY2013

FY2014

FY2015

FY2015

Target

Data

Source

Net Enrollment

Rate (NER),

primary level,

ages 6–10 years

70 72 72 70 75 PSLM

Surveys

NER, middle

level, ages 11–13

years

37 39 39 38 44 PSLM

Surveys

NER, matriculate

level, ages 14–15

years

25 28 31 29 32 PSLM

Surveys

Average share of

correct answers,

English, Urdu,

and mathematics

tests, Grade 4 and

Grade 5 students,

sample schools

— —

55% aThe baseline was

established in

2014. Target was

a 5 percentage

point

improvement over

the baseline. The

target was met

(100%

achievement).

60% 60%

DFID Bi-

annual

grade 3

assessment

Note: a. Values for NER and gross enrollment rate (GER) (2013, 2014, and 2015) are obtained from the PSLM survey.

The values for NER and GER (baseline and targets) are obtained from approval documents.

36

1.2. Achievement of DLIs of PESP II

PESP II DLIs (Targets)

Overall Achievement:

All 10 of the FY2011/2012 DLIs in PESP II were satisfactorily met by the GoPunjab.

DLI 1: Field-based professional development

DLI status: DLI for field-based professional development program was met during the three program years.

Achievements Comments

1. Primary grades of all primary schools in all 36 districts

covered by the strengthened field-based system for

providing advisory support to teachers at school.

2. DTEs were recruited on a merit basis (a total of 4,000

DTEs were providing support to teachers across the

province).

3. Monthly teacher observations and assessments of

students was conducted by DTEs in all public primary

schools.

1. DLI targets met each year.

2. Directorate of Staff Development’s Continuous

Professional Development program provided field-

based mentoring support to primary school

teachers, which was expanded to all primary

schools in the province.

DLI 2: Teacher recruitment

DLI status: DLI for teacher recruitment program was met during the three program years.

Achievements Comments

1. About 33,832 teachers, both subject-specific and

general, were recruited to fill confirmed school-specific,

rationalized needs after clearing a recruitment test.

1. DLI targets met each year.

2. DLI achievement for FY2014–15 was delayed

due to court cases that were pending, which were

eventually resolved.

DLI 3: Teacher rationalization

DLI status: The DLI for the teacher’s rationalization program was met during the three program years.

Achievements

1. Allocation of sanctioned teaching posts to schools in

accordance with agreed school-specific, needs-based

formula and teacher rationalization and recruitment

against school-specific, needs-based teaching posts were

completed and implemented in all 36 districts, and

implementation review conducted by a third party.

2. A TPV was also conducted.

Comments

1. All DLI targets met each year.

2. Evaluation by a third party revealed that 42% of

head teachers are of the view that sanctioned posts

were not sufficient before and even after the

implementation of the policy.

3. Evaluation by a third party revealed only 31%

head teachers responded that sanctioned posts

before the implementation of the policy was suited

to needs of their schools.

DLI 4: School non-salary budgets

DLI status: The DLI for school NSB program was met during the three program years.

Achievements

1. The NSB was disbursed to all schools in 36 districts

based on an agreed funding formula.


3. In over 90% of schools, the SC was involved in

developing the School Based Action Plan and budget plan.

4. About 97% spent of primary and elementary schools

included at least some of their NSB money.

5. In total 89% of schools, NSB receipts had been spent.

6. Approximately half of NSB spending was on improving

school buildings, classrooms, and facilities.

7. About 57% of schools reported taking specific actions

which they thought would improve student retention in

grades 5 or 8.

8. NSB funding resulted in substantial increases in

spending per student in2013/14 compared to 2012/13: on

average, schools spent over PKR 1,000 more.

Comments

1. All DLI targets met each year.

2. Evaluation by third party revealed that 76%

schools received the exact amount of NSB due to

them.

3. About 50% of schools received less than their

entitlement, 11% received exactly their entitlement

and 39% received more than their entitlement.

37

9. NSB funding resulted in substantial increases in

spending per student, with rural primary schools showing

the largest increase.

DLI 5: Decentralized resource management

DLI status: The DLI for decentralized resource management program was met during the three program years.

Achievements

1. School-specific NSBs were transferred to SC accounts

of primary and elementary schools.

Comments

1. All DLI targets met each year

DLI 6: Teacher performance incentives

DLI status: The DLI for teacher performance incentives program was met during the three program years.

Achievements

1. Improved bonus program was implemented based on

the design in 6 pilot districts.

2. Enrollment increased by, 23% in urban schools and

student exam participation rates by 4% in rural schools.

Comments

1. All DLI targets met each year

2. However, the program does not increase student

exam scores in any year.

3. After achieving the final year DLI, this program

is no longer being implemented.

DLI 7: School councils

DLI status: The DLI for SC program was met during the three program years.

Achievements

1. The School Council Mobilization Program was

implemented in 50% of schools in all 36 districts.

2. ICT-based capacity-building support was provided to

the council members.

3. The program increased overall student enrollment in

primary schools by 3.5% and increased enrollment in

primary schools for girls by 12%.

3. Teacher attendance increased by 1.4 percentage points.

4. Student attendance decreased by 0.5 percentage points.

Comments

1. All DLI targets were met during the activity

period.

2. This activity has been discontinued in Punjab

with respect to ICT-based capacity building;

however, a new strategy is to be formulated this

year.

DLI 8: Provision and use of system and school performance information

DLI status. The DLI for provision and use of system and school performance information program was met

during the three programs years.

Achievements

1. Development and distribution of SRC to all schools and

districts on a six-monthly and quarterly basis.


3. Third-party survey revealed 87% of the users surveyed

considered SRCs to be useful.

4. About 71% head teachers and 61% SC members

expressed that school performance indicators have

improved due to receipt of SRCs.

Comments

1. All DLI targets were met each year.

2. However, survey revealed that only 30%

respondents from the category of students, SC

members, and parents had seen the current SRC.

3. It was also further observed that there was lack of

sharing of information of the SRC on part of head

teachers with other stakeholders and SC members

with local community.

DLI 9: Private school vouchers

DLI status: The DLI for the private school vouchers program was met during the three program years.

Achievements

1. About 361,000 vouchers distributed to children in 36

districts of Punjab.

Comments


DLI 10: Stipends for secondary school girls

DLI status. The DLI for stipends for secondary school girls program was met during the three program years.

Achievements

1. Attendance-tied supplemental stipend benefits

following agreed benefit structure offered to beneficiary

girls in grades 6–10 in rural government schools in

existing 16 stipend districts and phased in consistent with

agreed impact evaluation design.

Comments


38

3. RECOMMENDATIONS

3.1. Clear articulation of linkages in formulating indicators, DLI targets, and protocols is important

to focus efforts on factors that are critical to the achievement of the project objectives. Currently,

there is a narrow focus on the achievement of DLIs with in the education structure in Punjab. This

leads to a disconnect between DLIs and wider PDOs, resulting in a less than the intended impact

on education sector in Punjab.

3.2. It is important for the SED to understand the trajectory toward meeting PDOs. The department

needs to keep connecting DLIs to the wider PDOs.

39

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders

Not applicable

40

Annex 9. List of Supporting Documents

PESP II Aide Memoires (2012, 2013, 2014, 2015)

CPS for Pakistan for FY2011–14: Report No. 65286-PK CPF for Pakistan 2015–2019 Report

No. 84645-PK

PESP II Project Appraisal Document (March 30, 2012)

PESP II ISRs (Seq. No 1–9) (2012–2016)

ESMF Status Document (2016)

Stakeholder Feedback on School Performance Report Cards (Cambridge Education 2013)

Non-Salary Budget Expenditure Review (Cambridge Education 2013)

Review of Implementation of School Council Policy 2013 (Cambridge Education 2014)

Post Procurement Review Documents for PESP II

Legal Agreement and Project Agreement (Credit 5106-PK) (Negotiations package)

41

MAP