Balochistan Education Program End-Line Evaluation Report

Balochistan Education Program End-Line Evaluation Report

Mmm 2 3 APR j

Submitted to Save the Children Netherlands

by

Dr. Sarah Tirmazi

March 10. 2015

Acronyms

ADE Associate Diploma in Education ADEO Assistant District Education Officer AJK Azad Jammu and Kashmir Alif Ailaan An education research non-government organization (NGO) ASER Annual Status of Education Report BEP Balochistan Education Program BEMIS Balochistan Education Management Information System BESP Balochistan Education Sector Plan BoC Bureau of Curriculum CC Child Club C&W Construction and Works Department CFHE Child Focused Health Education Chowkidar Guard CRM Child Rights Movement DAC Development Assistance Committee DBDM Data Based Decision Making DDEO Deputy District Education Officer DDR Disaster Risk Reduction DEO District Education Officer or Office DEMIS District Education Management Information System ECC Early Childhood Care ECD Early Childhood Development ECE Early Childhood Education ED-LINKS Links to Learning; Education Support to Pakistan (USAID) EKN Embassy of the Kingdom of Netherlands ELM Education Leadership and Management EMIS Education Management Information Systems FATA Federally Administered Tribal Areas FGD Focus Group Discussion GB Gilgit-Baltistan GER Gross enrolment ratio GGMS Government Girls Middle School GoB Government of Balochistan HDI Human Development Index H&H Health and Hygiene ICT Islamabad Capitol Territory ICTD Information and Communication Technologies for Development IDO Innovative Development Organization IDSP Institute for Development Studies and Practices

Infra Refers to infrastructure questionnaire in BEP Survey INGO International non-government organization IP Implementing Partner IT Information Technology Katchi Nursery or grade zero KAP Knov/ledge, Attitude and Practices Kll Key Informant Interview KN Kingdom of Netherlands KPK Khyber Pakhtunkhwa LEG Local Education Group, Department of Education, Balochistan M&E Monitoring and evaluation MDTF Multi-DonorTrust Fund (World Bank) MEAL Monitoring, Evaluation, Accountability and Learning Unit at SC MoU Memorandum of Understanding MTB-MLE Mother Tongue Based-Multilingual Education NEP National Education Policy 2009 NER Net enrolment ratio NFC National Finance Commission Award NGO Non-Government Organization OECD Organization for Economic Cooperation and Development OR Operational Research PC Planning Commission PDQ Program Development and Quality PEACe Provincial Education Assessment Center PGEB Promoting Girls Education in Balochistan PITE Provincial Institute for Teacher Education PPAF Pakistan Poverty Alleviation Fund PPIU Policy Planning and Implementation Unit, Department of Education, GoB PSC Public Service Commission PSLM Pakistan Social and Living Standards Measurement PTI Physical training instructor

Parent Teacher School Management Committee QPO Quetta Program Office, Save the Children RCC Releasing Confidence and Creativity RTE Right to Education SC Save the Children, Global SCNL Save the Children, Netherlands SCPK Save the Children, Pakistan SCSPEB Society for Community Strengthening and Promotion of Education or SOCIETY SDCP Sustainable Development Consultants, Pakistan SDPI Sustainable Development Policy Institute (Pakistan) SEHER Society for Empowering Human Resource

SEP School Expansion Plans SHN School Health and Nutrition SIP School Improvennent Plan SNE Statement of new expenditure SO Refers to school observation questionnaire in BEP Survey SOCIETY Society for Community Strengthening and Promotion of Education or SESPEB SOP Standard Operating Procedure SPO Strengthening Participatory Organization SSI Secondary school teacher St. Refers to students' questionnaire in BEP Survey T- Refers to teachers' questionnaire in BEP Survey UNICEF United Nations Children's Fund WATSAN Water and sanitation

MAP OF BALOCHISTAN [BEP focus districts specified)

".UTECSX*»' 'MKTj^u\./

Af«<*"A1cp f

PAK\i

/ f'

TAN

'ek.a- S<4

S 5_

itiiJrim w^yia'4 ll- k^d^. mtj

«1^ de «tf <"V^' »* A* fBrt y rW WuJj fdtl Crovp, Oiy

- .:• • / O

r ./u«ï«boBl» --' isA .'. / MWS/ 'fMè

I V>—<^ScÜrf»

I

iir''.'iJ*r<> ZoraT^'^ ( 'C A .V. ...V. -.

•- - vJF?^ , :;. {/r •"=5^\ \

, . , t^hsnf

^ ,, '/"o^ ^', 0 N" •

• ([PUNJAB \-

^ ^Cdw»., • A • • :•"' / • • w«, -I r" f _Jl.> •'• ., , 'f. •' I; ^ Hoi-SoT " ; 'J Af^ .'' '. i'JZl' , ;

-mA :i Us a^v \lodgati«

ISLAMIC REPUBLIC OF IRAN

f

Ï

Arabian Sea

i INDIA

? r f

Contents

I. Introduction and Background 20

II. Objectives of the Evaluation 26

III. Methodology 27

IV. Findings on Program Objectives 28

V. Relevance 49

VI. Effectiveness 52

VII. Efficiency 62

VIII. Impact 71

IX. Sustainability 86

X. Summary of Conclusions and Recommendations 93

XI. Lessons Learned and Good Practices 103

Tables Table 1. Population of Balochistan and focus districts 20

Table 2. Comparative Education Statistics Pakistan (%) 22

Table 3. Enrolment in focus districts as a percentage of total Balochistan enrolment for that category ..23

Table 4. Rural access 23

Table 5. Rural quality 24

Table 6. Education scores for primary schools 24

Table 7. Education statistics for the three focus districts with reference to the province 25

Table 8. Primary school infrastructure scores for the year 2014 26

Table 9: Children enrolled in ECE Classes 2014 29

Table 10; BEP retention and completion rates 2011-2013 29

Tablell. Number of ECE Centers in Balochistan Schools 30

Table 12: Breakup of Upgraded Schools for Girls by District 30

Table 13: Transition rate of girls - BEP schools 37

Table 14: Progress in Recruitment of ECE Teachers 66

Table 15. BEP Budget and expenditures (€) 67

Table 16. Horizontal distribution of the National Finance Commission Award (NFC) 86

Table 17: Straight Transfers for FY2013-14 & FY2014-15 (Rs. million) 87

TablelB, Donor Collaboration in Balochistan 88

Table 19. Summary of Conclusions and recommendations 93

Diagrams Diagram 1. Incidence of poverty in Pakistan 22

Diagram 2. Which type of infrastructure facility is most important for promoting education according to the teachers? 31

Diagram 3. Provision of utilities in schools 32

Diagram 4. Teachers' opinion about facilities and positive impact on schooling 33

Diagram 5. Availability of School Supplies 34

Diagram 6. Do you feel safe in the school environment with the teacher? 34

Diagram 7. Provision of child friendly school environment 35

Diagram 8. Primary school student teacher ratios 36

Diagram 9. Promotion rates of girls in focus districts overall (%) 37

Diagram 10. What is the most important effect or change that this BEP education project has had on you personally? 38

Diagram 11. Effectiveness of training 38

Diagram 12. is teachers' teaching method participatory (according to school observations)? 39

Diagram 13. How has the teacher organized sitting arrangements for children? 39

Diagram 14. Is the teacher utilizing ECE material and learning support material? ) 40

Diagram 15. Communications with teachers and participation 40

Diagram 16. How does the teacher make the learning process interesting for you? 41

Diagram 17. Observations (primary, middle and high) regarding student learning achievements 42

Diagram 18. Socialization and learning of children in BEP schools 43

Diagram 19. Percentage utilization of the education budget 2010-11 to 2012-13 70

Diagram 20. Children awareness through training 72

Diagram 21. Teachers' opinion of girls' schooling 73

Diagram 22. How to further encourage girls schooling according to teachers 74

Diagram 23. How do teachers feel about ho w to get children to behave? 78

Diagram 24. Bullying in school and teachers' help in countering it 78

Diagram 25. What are the topics which are covered in the (PTSMC) training? 81

Diagram 26. Reported hand washing practices among school children 83

7

Diagram 27. Hand washing practices among adolescent girls 83

Diagram 28. Education Budget as a Percentage of the Provincial Budget 88

Annexes

Annex 1. Persons interviewed 107

Annex 2. References 108

Annex 3. Submissions on BEP by Implementing Partners 110

Annex 4. Quantitative questionnaires 110

Annex 5. Qualitative questionnaires 110

Annex 6. Consolidation and analysis of qualitative questionnaires 110

Annex 7. Survey Data Collection Ill

Annex 8. Procurement orders and dates received for 2014 117

Annex 9. Planned versus actual coverage of quantitative and qualitative data collection 121

Executive Summary

For five years, Save the Children supported the Balochistan Education Program (henceforth called BEP or the Program) which aimed to benefit approximately 56,800 e xisting and 16,200 new children in 340 schools through increased access to a bett er quality of education. Of the total number 73,000 children that the program aimed to reach, about 43,800 were girls. Three Balochistan based partners. Innovative Development Organization (IDO), Society for Empowering Human Resource (SEHER) and Society for Community Strengthening and Promotion of Education (SCSPEB or Society), were selected to i mplement the program in close cooperation with the Department of Education at the provincial and district levels. One o f the mayor achievements is access of quality education to more than 105,000 boys and girls which is 26 percent more than the target. The Program was im plemented in the districts of Quetta, Mastung and Killa Abdu llah and aimed at achieving four objectives: improving access; quality; advocacy with government and implementation thereof, improved management information systems at the district level and enhanced provincial budgets for the education sector; and information on health and hygiene needs at the school level.

The program was funded by the Embassy of the Kingdom of the Netherlands (EKN) for a tota l amount of €10,181,445 for the period December 1, 2009 through November 30, 2014. The original approved budget was €8,800,000, In February 2011, an additional fu nd of €1,361,445 was committed, primarily to implement the health and hygiene component (€380,336), to incorporate an extra implementing partner for the child protection component (€273,462) and for the payment of teachers' salaries (€531,169), all within the program period.

The Balochistan Education Program (BEP) wa s required to be evaluated through an end-line evaluation. The evaluation team comprising the international team leader and the national survey team started the process in accordance with the evaluation design. The evaluation included work on the framework, methodology and work plan, the designing of quantitative and qualitative survey tools for primary data collection, field activities, data analysis and report writing.

The end-line evaluation has reviewed BE P implementation from 2009-2014 against the Organization for Economic Co-operation and Development (OEDC) evaluation areas of relevance, effectiveness, efficiency, impact and sustainability. The evaluation also addresses lessons learned and innovative interventions initiated by the Program to provide a reference point for future interventions.

The methodology comprised use of secondary information which included Save the Children documentation. Government reports and budgets, and documents of some other education experts and organizations. The p rimary data collection included a quantitative survey of BEP and some non-BEP schools, and qualitative key informant interviews and focus group discussions with several stakeholders. These included government officials and implementing partner managers, and members of communities, school management committees and child clubs.

The key findings are th at whereas access and quality have improved, health interventions and planning and budgeting have somewhat mixed results. There has been considerable positive transformation in the awareness levels of children, parents, teachers, and members of the community regarding the right

to education, the inviolability of children's rights and the benefits of education. The teachers have gained considerably from the training and awareness-raising, which is a ppreciated by children, even though there is more to be done in all areas.

The conclusions and resultant recommendations are presented below.

Areas Conclusions Recommendations

Objective 1: To expand and improve the existing infrastructure of the schools, create additional spaces, and to upgrade primary schools to middle and secondary levels to benefit over 73,000 children (of which 16,200 new) in the three districts of Balochistan by end of 2014

The objectives of component 1 are deemed to have been met in a satisfactory manner. The Program has managed to expand and improve the existing infrastructure and surpass by 26 percent the targeted number of children in school that were addressed through the Program. The Program achieved the targets of construction and establishment of new ECE centers, as well as up-gradation of schools fr om primary to middle and middle to higher levels, thus opening up avenues for children to continue in school, especially gi rls. The creation of a safe and friend ly environment in targeted schools has been achieved to quite a large extent, although not fully, with the majority of children agreeing that they felt schools were a clean and safe environment. The teachers were very positive about the impact of new facilities and furniture on schooling as well as availability of water and boundary walls.^ Importantly clean drinking water was fully available in the schools surveyed. Almost 88 percent of the toilets in BEP schools were in use by children a lone, unlike other schools. Especially no teworthy were the disability bars and ramps installed in over a third of the schools. Schools had a good supply of basic classroom materials and an excellent supply of learning materials.^ Children generally considered the school a safe pla ce with a clean environment.

Objective 2; To Improve the quality of

The objectives of component 2 are deemed to have been met in terms of promotion of child friendly methods and the indicators on student-

Future programs should focus more on the software aspects of enhancing

' The impact on parents and children has also been positive, which is highlighted in later chapters on effectiveness and impact. ' The provided supplies were being used to a great extent, as highlighted in a later chapter on effectiveness.

10

education through promoting child friendly and participatory teaching methodologies in all targeted schools of the three districts of Balochistan by end of 2014

teacher ratios, completion and transition, but less so in the areas of development of cognitive skills and overall confidence levels of the children. In terms of some quality indicators the student teacher ratios are satisfactory although higher than the district averages probably due to higher enrolments in BEP schools , especially for girls. The promotion rates have declined over all in Balochistan, whereas the BEP promotion/completion are about 25-35 percent higher, about 85 percent for ECE and primary and, quite high at 95 percent for the middle and high levels. The transition rates, from primary to middle and middle to higher are also grea ter in the BEP schools t han in the districts overall, which shows that the BEP objective of having children continue school to higher levels has been positive. This is an important achievement given the high dropouts rates that persist traditionally at the end of each level of schooling, especially grades 1 to 5. Training has been enhanced through the involvement of PITE in teacher training and the teachers have learned new skills in teaching. Their teaching methods are observed to be participatory although about two thirds are s till following the traditional row seating in classrooms. A solid ma jority is using the learning materials while teaching. Notably, over 75 percent of the teachers find the ECE training quite effective. PITE and other government officials were quite vocal about their appreciation of the teachers' training techniques instilled by the Program, especially for ECE. Conspicuously, a majority of children agree that the teacher makes the learning process enjoyable, mainly due to use of participatory techniques, and to a lesser extent due to use of the learning materials, which needs further examination. Almost all students interviewed expressed that they are able to ask questions of their teachers and that they are encouraged to participate, although this was equally true for BEP and other schools. A

cognitive skills in children, and less on what can became an expensive and time consuming construction program.

> There should be further research into the reasons for children feeling that the lessons are more interesting due to the participatory techniques used and less so due to use of learning materials by the teacher, so that either further emphasis can be placed on the use of participatory techniques or into why the use of materials is not as successful.

11

majority of community members and PTSMC members agreed that the ECE classes had transformed the educational process in their neighborhoods. Student learning outcomes are mixed with relatively poor learning of math concepts, in both BEP and o ther schools. The BEP school children are better equipped in participation in ECE activities and participation in groups, but the other school children did considerably better in terms of concentration skills, willingness to help other children and overall confidence levels which is a debatable result. Discussions with the community members and child clubs members pointed to greater confidence of children participating in the BEP act ivities and schools leading to the need for further research in this area.

Objective 3: To improve educational planning and budgeting systems in the Balochistan province through the development and introduction of evidence based District Education Planning models, as well as the strengthening of Education Management Information Systems by 2014 (increase education budget and strengthen

The objectives of component 3 are deemed to have been partially met since the Program did help to set up the District Education Management Information System (DEMIS) in the three districts.

The links between planning and budgeting were not addressed g reatly; the activity was rescinded since other donors were already working in this area. A lthough staff was trained, there is a lack o f staff in one district which means that in effect the system is not working there. A success of the DEMIS is the scaling up to all districts of Balochistan where data entry points have been set up and data entry Is in process. The enriched EMIS collects data about schools, teachers and for the first time in Pakistan, the Balochistan EMIS is collecting data on individual students. There is a great deal of ownership on the part of Balochistan government. The actual use of data based decision making is n ot emphasized in the work done with the districts, or the use of EMIS in different educational areas such as human resource management or payroll management, although that is e xpected in the future. In terms of other areas of advocacy, the Program was successful

> Future projects should look at how EMIS can be kept ongoing after project funding ends.

> Greater emphasis should be given to client needs and an effort made to integrate data based decision making in the system.

> Future projects should look at the possibility of including other areas into the EMIS system, such as human resource management, including staffing patterns, and payroll management.

> The lessons learned in enabling community to raise their voice regarding making land available from the government should be shared at different forums.

12

management) generating land donations from community and government for the construction of additional rooms and ECE centers, which was a notable achievement since community members were also involved in raising their voice in this regard. The fundraising by community members for schools was also a notable achievement of the Program, given the poverty of the areas worked in.

Objective 4: Improve School Health and Nutrition in 340 Schools of 3 Districts of Balochistan by 2014

The efforts of the BEP team have been satisfactory although the results themselves are mixed. The comparison of base-line and end-line suggests that there has been improvement in key indicators such as positive change in knowledge and practices of safe d rinking water that has been significantly increased. Especially in Quetta; similarly teeth cleaning practice had significantly improved such as cleaning teeth at critical occasions i.e. after breakfast (40% vs 60%) and before sleeping at night (26% vs 47%). There has also been a significant increase in knowledge of washing hands after playing and whenever gets dirty %( 14 Vs 26 and 17 Vs 30 consecutively). However there has been a decline in some indicators like knowledge about critical occasions of hand washing, practice and knowledge about balanced diet. But this data contradicts with the data collected through another tool used in overall BEP ev aluation. The d ata reveals that more than 90% of the children wash their hands be fore and after eating food and using toilets. While the baseline figure was 66%. So, there is a high probability of anomaly during data collection. Save the Children had no access to the field to support enumerators while collecting data during end-line survey in two of the districts (Killa Abdullah and Mastung) due to security reasons. That might have affected quality of the data.

Also the biomedical part of the survey was dropped again due to security reasons that included stool testing for parasites and blood test for Iron deficiency anemia. This also limited to

The end-line survey findings should be complemented by project internal and external review with stakeholders to investigate implementation bottlenecks and explanations for large variations in the base-line and end-line surveys in some of the key indicators.

13

gauge the project success in terms of health improvement. There was no recent relevant secondary data available that could have been linked to the project achievements in program areas.

These findings should be complemented by project internal and external review with stakeholders to investigate any implementation bottlenecks and explanations for large variations in the base-line and end-line surveys in some of the key indicators.

Relevance The Program was and remains very relevant in assessment of the situation of the province in terms of poverty and v ulnerability, and education provision, student learning scores, school infrastructure and gender parity. However, not all Program activities were well thought out. The Program proposal assumed that girls' education is not important for parents in Balochistan, whic h is factually not proven. The issue of teacher absenteeism is not discussed in the proposal although the addressing of this issue was included in the actual implementation through SC and PTSMC interventions. Enabling children to transition from primary to middle and middle to high was and remains very relevant. However, student testing and learning is an area that is not addressed adequately in the proposal or in the actual interventions, although some efforts have been made, for instance in the area of mother tongue, literacy boost and numeracy boost, as discussed in a later chapter. BEP planning and techniques needed honing in the area of quality education and development of cognitive skills. Although PITE was fully involved, the Provincial Educational Assessment Center (PEACe) was not involved in assisting in diagnosing issues in student learning. The Program team made considerable efforts to keep on board the changing government officials and steering committee meetings were held regularly to facilitate Program bottlenecks.

> Program proposals should be stronger in terms of needs of the province and should address the area of where best to focus the funds to get value for money.

> The important and political issue of teacher absenteeism should be addressed in all education programs in the province.

> The existing Provincial Education Assessment System (PEACe) should be strengthened and involved for better quality diagnostic information on student learning outcomes, and subsequent development of learning solutions, and to be able to offer further successes in learning outcomes to other schools in Balochistan.

> Greater focus should be given to software aspects and quality of education.

14

Effectiveness Overall, effectiveness of the Program is deemed quite satisfactory. Tfie baseline survey was undertaken to inform project design but was slow in being realized. It was also not designed for evaluation against tfie initial status of the area. The quality of monitoring is not fully adequate since the logframe has n ot been presented cumulative for each year; that is achievements against activities in the logframe are not available in one document but in biannual and annual reports only for the time under review. There was some effect on adequate monitoring since the SCNL pro gram manager was of ten not able to visit Quetta in the latter half of the program due to security concerns. The capacity of the implementing partners was fairly strong for the most part, and they were further trained by SC in program management and financial management. A weakness was that implementing and supervising construction activities was not their field of expertise. However, perhaps given the restricted capacity in the province it was not possible to commission an implementing partner with more solid expertise in supervision of construction. Once government was more fully involved, the construction aspect moved forward rigorously, which was a lesson learned for the Program. Programming of inputs was sequential and logical except that the construction effort did take up too much time in the first 3 years of the Program. Minimum quality standards were met in terms of monitoring of construction activities. Risk analy sis and meeting of subsequent challenges has been fairly strong and a fair amount of training has been a dministered for disaster risk reduction (DRR) to child clubs and community members, as well as related kits provided. Communities have been strongly mobilized in education delivery. However, they remain dependent on external financing, government or donors, to be able to play an active role in the PTSMCs. Hence, it is a lesson learned for future

The hiring of a construction firm to manage and supervise construction, with IPs acting as facilitators between the firm and the community, should have been undertaken for greater effectiveness and timeliness. Building alliances with other NGOs, INGOs and external donors working in Balochistan should be undertaken to better advocate on education reform and propagate Program learnings. Monitoring of the project should be cumulative for each year

15

implementation that expectations regarding the financial contributions of community members be based on what is realistically possible for poor communities to contribute. The implementing partners have been effective in transparency and information management. Training programs for teachers, PTSMCs and community members were delivered in a timely and effective manner. The partnerships with government partners such as PITE have been strong with the latter incorporating many of the training techniques into their own programs, especially those related to ECE.

Efficiency Program ejficiency is deemed satisfactory especially given the Balochistan situation in terms of security and the costs associated wit h traveling large distances and p roviding services thereof. The procurement policy is well defined as are other financial systems. The Program's coping mechanism in the face of insecurity has been relatively strong. The program has also responded well to changes in the political environment thus providing complementary support for the greater responsibility given to the province due to Article 25-A o f the 18"" A mendment to the Constitution. Budgetary expenditures for the Program are quite efficient, as are financial controls and financial monitoring, at least from the head office perspective. The field office was not able to provide information although it is assumed to be positive since their information feeds into the head office. The procurement policy is well defined and procurements are timely per the information provided for 2014. Information on the quality of the procured items was not provided, although no complaints were raised by PTSMCs or community members either. The financial expenditures for the Government's education budget are also fairly efficient although advocacy by the BEP team was an area that was made redundant midway through the program given that other donors were working on this intervention. One aspect that could have

16

provided further information was a linking up of the baseline survey with a cost-be nefit analysis to better determine value for money for the various activities of the Program.

Impact The impact of the Program has been significant on many fronts and has had notable impact. Girls' enrolment has increased d ramatically, as has the opportunity for girls to continue on to higher levels of education beyond primary and middle, addressing a key weakness in the system thus far. Children's awareness about schooling and their rights has been enhanced greatly, a significant outcome of the Program. Teachers overwhelmingly agree that the Program has encouraged girls' schooling, and that the most important factors have been training and the focus on ECE. The Program's many innovations have had demonstrably good impact such as participatory techniques in teaching; participation by community members, teachers and children in Program implementation; the ECE program of provision of classrooms, materials, teachers and training, as well as other initiatives such as the Literacy Boost, the Numeracy Boost and the Mother Tongue Pilot, which initiated considerable discussion with Government counterparts. Training on peace ed ucation was another major innovation for the provincial context. Although the culture of child abuse has been ingrained in schooling and even in homes, the Program made considerable headway in raising awareness about child rights among community and PTSMC members, and teachers and children. The survey results are q uite significant in these areas. The deep ro oted culture of abuse of children has not been e liminated, but the Program has gone a long way towards enhancing the awareness of teachers, children, community members and members of PTSMCs on the rights of children, including their right to education and to not be subjected to corporal punishment and other forms of abuse. A ma jority

Training on child rights, DRR and other areas should be provided to district education managers since they are key government officials responsible for district education and in a position to bring the learnings to other schools in the province. All the components of a successful ECE program should be thoroughly discussed with all government counterparts and other stakeholders at the initial stages o f program intervention. Efforts should be made that the key area of ECE (and other) teacher recruitment be addressed and implemented up front. Save the Children should continue with efforts to disseminate the solid good practices and learnings of the Program to both national and international audiences.

17

of teachers expressed that they were against corporal punishment in schools, and that it should be banned. Significantly, children approach teachers for assistance when being bullied in schools, and receive a positive response, i n both BEP and other schools. The success or otherwise of the child clubs remains a som ewhat grey area; in the qualitative interviews the members express positively about girls education, child rights, health issues, and others. However, the majority of children said their participation was not voluntary, whereas 11 of the 12 teachers interviewed said that children's participation was voluntary, a clear contradiction. The children were positive about the child friendly construction and attractive classrooms for ECE. Child health has im proved in terms of awareness about good practices and the communication gap between mothers and adolescent girls has been decreased, a concrete achievement. The ECE model has obtained renown and established a solid platform for further realizations in Balochistan, with government especially supportive due to the close involvement of PITE. Save the Children has developed and implemented the blue print which forms the basis for the Government of Balochistan's Education Project which is funded through the Global Partnership for education (GPE), as well as a blueprint for many important components of the Balochistan Education Sector Plan (BESP), especially in the area of ECE.

Sustainability The Program is deemed sustainable based on the many policy influences and the model realized by the ECE program. There is financial space in Balochistan's budgetary situation due to additional funding from the National Finance Commission Award, as well as large proposed d onor projects in education. The education development budget has been fairly substantial for the last two years at 23% of the total provincial budget. The one important area that was unable to be adequately addressed

It is strongly recommended that Save the Children continue education interventions in Balochistan, especially in the ECE subsector, including through provision of technical support to the Policy Planning and Implementation Unit of the

18

was recruitment of teachers due to delays on the side of the government, which makes it difficult to uphold the advances made in terms of development of ECE centers and up-gradation of schools until the posts are fill ed. Until this is done, the upgraded schools will not be functioning efficiently and will not be sustainable. The ro le of all implementing partners as members of the Local Education Group, which provided technical oversight for the development of the Balochistan Education Sector Plan, lends depth to the Program's sustainability. The incorporation of ECE in the Plan as well as acceptance by Provincial Institute for Teacher Education (PITE) of ECE techniques and related training provides further support for sustainability. There has been increased knowledge of implementing partners, all Balochistan NGOs expected to continue work in Balochistan's education system, which is po sitive. The discontinued funding of PTSMCs is another area wher e communities will simply not be able to cover the expenses. However, a formal role for PTSMCs has been defined in new government policy in 2014 concerning the right to education, which lays the groundwork for government funding of the committees.

Department of Education. > The many strong

partnerships developed with PTSMCs, community members, government officials, and implementing partners should be continued and the means found to nurture these partnerships in the future.

> The Government of Balochistan should fill the vacant positions of teachers in order to enable the ECE centers and the upgraded schools to keep functioning and ensure quality and sustainability.

> The Government should review the work of the PTSMCs and consider their funding through provincial resources.

The evaluation ends w ith a present ation of lessons learned, which includes details on follow up actions taken after the midterm evaluation, and reiterates the importance of the good practices of building alliances and working closely with government entities.

19

!. Introduction and Background III (rod uction For five years, Save the Children supported the Balochistan Education Program (henceforth called BEP or the "Program") which had aimed to benefit approximately 56,800 existing and 16,200 new children in 340 schools through increased access to a better quality of education, focusing on about 43,800 girls out of the total 73,000 children. The program was implemented in Balochistan province in the districts of Quetta, Killa Abdulah and Mastung. Three Balochistan based partners. Innovative Development Organization (IDG), Society for Empowering Human Resource (SEHER) and Society for Community Strengthening and Promotion of Education (SCSPEB or "Society"), were selected to implement the program in close cooperation with the Department of Education at the provincial and district levels.

The program was funded by the Embassy of the Kingdom of the Netherlands (EKN) for a total amount of €10,181,445 for the period December 1, 2009 through November 30, 2014. The original approved budget was €8,800,000. In February 2011, an additional f und of €1,361,445 was committed, primarily to implement the health and hygiene (H&H) component (€380,336), to incorporate an extra implementing partner for the child protection component (€273,462) and for the payment of teachers' salaries (€531,169), all within the Program period. The four objectives of the Program dealt with access, quality, planning and budgeting, and health and hygiene.

Background

The basic goal of the Balochistan Education program; "To bring positive changes in the lives of vulnerable and marginalized children, mainly girls, through the provision of quality education in three districts of Balochistan in particular and beyond in general on sustainable basis by 2014." This section provides a framework on the status of poverty and education in Balochistan with respect to the rest of the country and the related special difficulties of educational development endeavors in the province.

The province is a challenging one for educational development given the poverty levels of the population, the large land area, and the difficulty of travelling long distances using inadequate infrastructure. The pop ulation of Balochistan is about 6.5 m illion (based on the 1998 census, the last available) with about 24 percent living in urban areas and the remaining 76 percent in rural areas.^ The area is almost 46 percent of the total land mass of Pakistan with the population only about 5 percent of the total population of the country. In the focus districts, the population figures are presented in table 1 representing Quetta at over 11 percent and the other two districts as relatively small.

Table 1. Population of Balochistan and focus districts Killa Abdullah Mastung Quetta Balochistan

Male 203,324 87,334 412,064 3,507,000 (54.91 %) (53.04 %) (54.22 %) (53.41%)

Female 166945 77,311 347,877 3,059,000(46,59%) (45.08 %) (46.96 %) (45.78 %)

" Pakistan Bureau of Statistics

20

Total 370,278 164,645 759,941 6,565,885

As a % of provincial 5.6% 2.5% 11.57%

Quetta's majority populations are 30 percent Pashto speaking, 28 percent Balochi and 16 percent Punjabi, according to the 1998 census data ." Mastung is 80 percent Balochi and Killa Abdullah is 97 percent Pashto, signifying the different ethnic groups addressed by the Program.

'.'•MiU'rable and inai ginalized Myriad research studies have shown that education can bring positive changes in the lives of the vulnerable and the marginalized. It is basically the poor that the government education systems cater to and poverty in the region and the focus districts plays a m ajor part in the success of development interventions or lack thereof. The definition of poverty can be different based on the research used. One well regarded organization defines it in terms of the following four conditions: ^

1. Lowest Human Development Index (HDI) and food security

2. Low HDI and food security

3. Moderate HDI and food security

4. Good HDI and food security

According to this definition, poverty in the three selected districts varies with Killa Abdullah and Mastung falling in the second category and Quetta in the fourth. Another recognized organization provides an analysis of district and provincial level comparisons, where Balochistan is recorded as the highest poverty province in the country (diagram 1), with Killa Abdullah having the highest poverty intensity and the other two focus districts (Mastung and Quetta) in the low poverty intensity categorization, respectively.®

Gazdar, Haris, 2007, Balochistan Economic Report, p.17 ^ Pakis tan Poverty Alleviation Fund (PPAF), 2013, Balochistan Strategy ... a road map to inclusive socio-economic development for the province, p.12 ® SDPI, October 2012, Clustered Deprivation: District Profile of Poverty in Pakistan, p.22

21

Diagram 1. Incide nce of poverty in Pakistan 0.8

0.7

Source: SDPI, October 2012, Clustered Deprivation: District Profile of Poverty in Pakistan, p. 19

. u.; of education The educational situation relative to the other provinces is r eflected in Table 2, which shows that the least progressive situation educationally is in Balochistan. Only 61 percent of males have attended school and a very lo w 20 percent of females. The population that has completed primary is 48 percent for males and only 13 percent fo r females, underscoring the dire status of education in the province. The gross enrolment ratio and net enrolment ratio are also the lowest for males and females, respectively, compared to the other provinces.

Table 2. Comparative Education Statistics Pakistan (%] Punjab Sindh KPK Balochistan

Male Female Male Female Male Female Male Female

Population that has ever attended school 72 53 72 47 71 35 61 20

Population that completed primary and above 60 44 62 40 57 26 48 13 Gross enrolment ratio (GER) Primary (5-10) 103 92 91 71 100 76 92 54 Net enrolment ratio (NER) Primary (6-10) 73 68 68 55 71 56 68 40

NER Middle (11-13) 38 32 39 32 40 25 34 13

NER Secondary (14-15) 24 21 26 20 23 14 21 4

Literacy Rate 15-r 67 47 71 43 65 28 56 15

22

Source: Government of Ba lochistan, Department of Education, Balochistan Education Sector Plan (BESP), 2013, p. 12 (Pakistan Social and Living Standards Measurement (PSLM) 2010-11 data)

The enrolment of the three focus districts relative to all Balochistan enrolment provides a further picture of the status of education with Quetta enrolment almost 12 percent of the total, with Killa Abdullah at about 5 percent and Mastung at about 3 percent of total Balochistan enrolment, which totals to about a fi fth of the provincial enrolment in the districts being addressed by BEP.

Table 3. Enrolment in focus districts as a percentage of total Balochistan enrolment for that category As a % of

Balochistan Unadmitted

children Primary Middle High Total

Quetta Boys 1.55% 7.96% 15.22% 17.15% 9.28%

Girls 1.10% 14.10% 25.89% 30.19% 15.94%

Total 1.36% 10.38% 19.13% 21.90% 11.88%

Mastung Boys 4.36% 2.82% 2.67% 2.50% 2.82%

Girls 3.22% 3.11% 2.83% 2.53% 3.05%

Total 3.87% 2.93% 2.73% 2.51% 2.91%

Killa Abdullah Boys 1.78% 6.42% 4.43% 3.72% 5.87%

Girls 3.13% 2.71% 1.54% 1.00% 2.49%

Total 2.36% 4.96% 3.37% 2.73% 4.55%

Source: BEMIS census data 2013-14

Rural access and quality education provision The situ ation is especially poor in the rural areas where almost 34 p ercent of all children are out of school, with over 17 percent being girls who are out of school as shown in table 4.

Table 4. Rural access % of children Age 3-5 Age 6-16 Attending paid

tuition (Govt and private schools

in Pre­school

Out of school (all)

Out of school (girls)

Non-state providers

Attending paid tuition (Govt and private schools

National 40.9 21.1 11.3 26.5 10 Azad Jammu and Kashmir 57.7 5.2 2.7 38.6 7.5 Balochistan 18.8 33.8 17.3 14.7 3 FATA 39.3 21.4 13 26.2 6.7 Gilgit - Baltistan 41.3 15.7 9.8 42.6 7.4 Islamabad - ICT 50.9 4.9 2.2 43.9 27.7 Khyber Pakhtunkhwa 44.9 14 8.5 26.7 6.8 Punjab 52.8 15.7 8 35.7 21.6 Sindh 40.8 29.1 15.4 9.8 4.6 Source: Annual Status of Education Report (ASER) Pakistan, 2013, p. 69

23

Table 5 gives in formation on quality indicators nationally showing Balochistan indicators among the lowest. Balochistan children are again lagging in terms of learning of basic reading and arithmetic, reinforcing the need for quality education interventions. Although school age children completing primary, including girls, has been slowly increasing in the province^ this factor shows the continued need for education interventions.

Table 5. Rural quality % Children Class 3 Class 5

Who can read sentence (Urdu / SIndhI / Pashto)

Who can read word (English)

Who can do subtraction

Who can read story (Urdu / SIndhI / Pashto)

Who can read sentence (English)

Who can do division

National 40.6 45.7 39.1 49.8 43.3 43.2 Azad Jammu and Kashmir

60 70 57.9 61.4 57.8 50.5 Balochistan 30 20.5 25.8 48.8 29.1 38.6 FATA 364 52.1 45.5 30.2 27.9 37.4 Gllgit - Baltlstan 45.8 65 49.2 51.1 60.4 50.1 Islamabad - ICT 64.5 65 53.2 61.5 60.4 51.8 Khyber Pakhtunkhwa

36.6 52.4 41.4 39 39.3 37.6 Punjab 52.8 62.3 50.4 65.8 62.1 56.3 SIndh • 33 28.4 24.2 41.2 25.2 29.4 Source: Annual Status of Education Report (ASER) Pakistan, 2013, p. 69

ilciidor parity Although low in absolute and comparative terms, the good news is that the overall learning scores for children in Balochistan are showing improvement in the last couple of years as presented in table 6. However, the score for gender parity remains weak.

Table 6. Education scores for primary schools Rank Province / region Education Enrolment Achievement Attainment Gender Parity

score score score score score

2014 2013 2014 1 1 ICT 83.75 94.02 64.93 79.27 96.77 2 3 Punjab 79.21 98.00 68.45 56.50 93.88 3 2 AJK 77.52 78.75 56.57 78.25 96.51 4 4 GB 72.62 65.67 53.87 95.00 79.69

- - Pakistan 69.70 92.00 52.74 54.00 80.08

' Government of Pakistan, Pak istan Bureau of Sta tistics, Statistics Div ision, May 20 13, Pak istan Social and Li ving Standards Measurement Survey (PSLM) 2011-12, p. 25

24

5 6 Sindh 62.99 81.00 45.06 43.00 81.65

5 5 KPK 62.66 92.00 36.41 64.00 54,98

7 8 Balochistan 58.09 73.00 50.80 43,00 77.32

8 7 FATA 47.23 63.83 31.83 48.00 61.99

Source: Alif Ailaan, 2014, p, 13

Table 7 gives edu cational statistics data for the three focus d istricts and for Balochistan as a whole. Adult literacy is h igher in the three districts than the provincial, average, except for females in Killa Abdullah. The d ifference in data sources may account for larger differences between provincial and district figures. Overall, Killa Abdullah has the weakest education indicators out of the three focus districts, emphasizing its more regressive status.

Table 7. Education statistics forthe three focits districts with reference to the province Percent Quetta Masturig Killah Abdullah Balochistan Percent

Male Female Male Female Male Female Male Female

Adult Literacy rate

83 36 74 37 66 9 56 15

Gross enrolment ratio (GER) Primary*

32 40 69 54 37 13 91 52

Net ehrolrnent ratio (NER) Primary (6-Ï0)

68 57 81 64 39 13 68 40

NER Middle (11-13)

30 19 31 21 18 5 34 13

NER Secondary (14-15)

9 10 12 1 3 2 21 4

Transition rate elementary

52 60 60 • 65 57 18 61 52

Transition rate secondary

71 72 65 77 70 45 74 72

Student to teacher ratio in primary schools

27:1 28:1 43:1 25:1

Primary schools with a single teacher

17% 33% 56% 58%

Source: Government of Balochistan, Department of Education, Balochistan Education Sector Plan (BE SP), 2013, p. 12 (Pakistan Soci al and Living Standards Measurement (PSLM) 2010-11 data) for all Balochistan data and literacy rates; and Alif Allah 2014 for district data; BEMIS 2013-14; and PSLM 2012-13*

25

SI iulo ! iftfi astructijre School infrastructure was another component of BEP given the poor status of school facilities in Balochistan. A well regarded education research organization has computed school infrastructure scores for primary schools. In Balochistan these scores remain comparatively the lowest as shown in table 8.

Table 8. Primary school infi-astructure scores for the year 2014 Rank Province /

Region School Infrastructure Score

Availability Building condition satisfactory

2014 2013

Province / Region

School Infrastructure Score

Electricity Water Toilet Boundary Wall

Building condition satisfactory

1 1 ICT 95.66 98.95 96.86 99.48 98.95 84.04

2 2 Punjab 81.36 71 96 72 86 81.8

3 4 KPK 68.42 52 65 75 73 77,1 - Pakistan 57.68 49 64 58 65 52.4

4 3 Sindh 43.05 39 45 52 56 23.23

5 5 FATA 36.45 39 34 32 52 25.26

6 7 GB 31.57 32 35 30 36 24.84

7 6 Balochistan 28.32 16 50 21 31 23.62

8 8 AJK 23.84 12 27 27 17 36.21 Source: Allf Ailaan, 2014, p. 24

i 111 c n! r 0 f 1" h e e p o r t The report structure is as follows: Chapter one reviews the background of the Program and the poverty and education situation in Balochistan vis a vis the key program objectives; Chapter two reviews the objectives of the evaluation; Chapter three addresses the methodology used; Chapter four reviews progress thus far on the four key objectives of the Program; Chapters five till nine look at findings for the key evaluation areas of relevance, efficiency, effectiveness, impact and sustainability; Chapter ten reviews conclusions and recom mendations; and Chapter eleven closes with a review of lessons learned and good practices.

II. Objectives of the Evaluation BEP was required to be evaluated through an end-line evaluation in the second half of 2014. The evaluation team comprising the international team leader and th e national evaluation team started the process in accordance w ith the evaluation design. The eva luation included work on the methodology and work plan for development of the inception report, the designing of quantitative and q ualitative survey tools for primary data collection, field activities, data analysis and report writing.

The end-line evaluation has reviewed BEP implementation from 2009-2014 against the evaluation areas of relevance, effectiveness, efficiency, impact and sustainability as specified in the Development Assistance Comm ittee (DAC) cr iteria of the Organization for Economic Cooperation and Development

26

(OECD). The evaluation also addresses lessons learned and innovative interventions to provide a reference point for future interventions.

III. Methodology

!,."Jti;nitjtnt:ive data This included a survey of teachers, students, school observations and observations on infrastructure, through individual interviews, or observations, using a standardized survey tool (structured questionnaires for each of these four categories). The language of the questionnaires was English and these we re discussed with the enumerators to give them comfort and facility in their task. The data collection team was selected and trained. The resu lts of this survey were analysed once completed. Annex 7 presents a detailed schedule of visits for the quantitative survey.

QunlitaCive data This included key informant interviews (KM) through the use of semi-structured interviews with Government of Balochistan officials, implementing partners, representatives of Save the Children Pakistan (SC PK), Sav e t he Children Netherlands (SCNL) and the financing partner, the Embassy of the Kingdom of Netherlands (EKN) and the World Bank. The purpose was to seek inputs o n policy issues and project performance on higher level goals, objec tives, and outcomes, on what worked best so far and what can be improved for the future.

Qualitative data was obtained also through conducting focus group discussions (FGDs), using semi-structured questionnaires, with members of parent teacher committees and child club members (in line with SC's Child Safeguarding Protocols). The len gthier discussions with beneficiaries and stakeholders also allowed greater insight into any problem areas that were not captured in the survey. Information collected from FGDs was also used to complement and validate information collected through quantitative techniques. About 15 FG Ds were conducted in each of the three project districts from the different groups for a to tal of about 45 FGD s in the three districts, and the respondents were teacher trainers, child clubs, community members and Parent Teacher School Management Committees (PTSMC) in selected locations. The tables on quantitative and qualitative survey coverage, provides each of the category to be surveyed through appropriate survey t ool and technique. In all, 45 FGD were conducted: 30 with BEP schools and the remaining 15 with non-BEP (control) schools.

The data was collected from three different areas in Quetta with different characteristics as follows:

Category 1: Shaden Zai, Kechi Bagh, Brewery, Qambrani , Kirani which are comparatively lesser settled areas in terms of law and order and quite challenging Category 2: Kuchlak, Baleli, Shiekhmanda, Ko twal, Hanna which are comparatively developed rural parts of the district and more settled Category 3: Satellite Town, Jinnah Road, Ka si Road, Yet Road, Nawa Killi which are within the city limits and more settled / developed and relatively less challenging

27

The survey team took photographs (in the case of children, keep in view the SC child safeguarding protocols) after consent was obtained, during the course of the assignment which served to validate the evaluation exercise. Names, age and family s ituation of each child photographed were provided, subject to consent o f the child/parents. The team also noted down interesting quotes from the stakeholders being interviewed that pertained to the different evaluation areas.

The data verification and quality assurance fo r the quantitative survey and qualitative interviews and FGDs were to have been provided by SC's Monitoring, Evaluation, Accountability and Learning (MEAL) unit, but were not possible. All the field surveys and interviews were conducted by SDC Consultants.

Secondary data collection

Save the Children made available the documents developed by the organization regarding corporate policies/procedures, case studies, and management / diagnostic tools. Some of the key documents consulted are as follows;

i. Relevant background documents such as project descriptions, reports, evaluations, sector studies and assessments

ii. , Official Government of Pakistan documentation related to education and budget documents iii. Education statistics, both national and provincial iv. Education reports by other donors and national research organizations

IV. Findings on Program Objectives The findings are discussed for the four objectives of the Program, access, quality, planning and budgeting, and health and hygiene; and then for the evaluation areas of relevance, effectiveness, efficiency, impact and sustainability. Findings are stated fir st, and conclusions stated at the end of each section or chapter.

Olijuclives of the Program The aim of BEP was to contribute to the following goal: "To bring positive changes in the lives of vulnerable and marginalized children, mainly girls, through the provision of quality education in three districts of Balochistan in particular and beyond in general on a sustainable basis by 2014." This section discusses the Program objectives and progress thus far.

i To expand and improve the existing infrastructure of the schools, create additional spaces, and to upgrade prirhary schools to middle and secondary levels to benefit over 73,000 children (of which 16,200 new) in the three districts of Balochistan by end of 2014.

fjirohueni The Program focused on 340 schools (133 boys' schools and 207 girls' schools) to provide safe access of children to schools and create a c hild friendly learning environment. In the year 2013, the program surpassed the original target of providing access to safe, child friendly learning environment to 73,000

28

children by 26% or 103,984 children, due to focused enrolment campaigns. Strong advocacy with the government for the up-gradation of schools, const ruction of ECE centers, improving of infrastructure and training of teachers for a safe and healthy e nvironment was also undertaken. Enrolment campaigns and mobilization of education department played a piv otal role in increasing the enrollment of girls at schools. In addition up-gradation of 25 primary schools to middle and 9 middle schools to high was also effective in increasing the enrollment of girls of school going age. As a result of enrolment campaigns, a total of 16,452 new girls were enrolled to the program focused schools in 2014 alone including 10,257 girls in the ECE section, 3,797 in th e primary sections, 1,840 girls to the middle sections and 558 girls to the high sections of the focused 340 schools. The number of children enrolled in ECE classes in the focus schools in shown in table 9.

Table 9: Children enrolled in ECE Classes 2014

District Number of Children Enrolled

District Boys Girls Total

Quetta 4,418 8,270 12,688 Mastung 790 1,049 1,839

Killa Abdullah 704 938 1,642

Total ECE 5,912 10,257 16,169

Source: SC, BEP Annual Report 2014, p.13

The student retention and completion rates for a three year period are presented in table 10, for the three different levels addressed in the BEP program, which generally show a gradual increase over the three years.

Table W: BEP retention and completion rates 2011-2013 Retention/ completion 2011 2012 2013 Retention/ completion

Girls Boys Girls Boys Girls Boys

Retention

Primary 83% 80% 87% 82% 87% 81%

Middle 76% 84% 82% 86% 85% 89%

High 89% 88% 91% 89% 94% 90%

Completion

ECE 83% 86% 83% 89% 85% 92%

Primary 77% 78% 82% 86% 85% 90%

Middle 91% 70% 91% 76% 93% 80%

High 96% 87% 97% 94% 96% 96%

Source: SC, BEP, November 14, 2014, Enrolment Trends, pp. 13-16

29

It is useful to also consider the total number of schools with ECE classes in Balochistan (5% of the total) ^

as shown in table 11 versus the 7,200 needed as p roposed in the Balochistan Education Sector Plan. It

should be noted that BEP has provided almost 19 percent of the existing ECE facilities.®

Tablell. Number of ECE Centers in Balochistan Schools Program or Project supported by Number of ECE centers UNICEF 300 Releasing Confidence and Creativity (RCC) project by SCSPEB 100 Balochistan Education Program (SCSPEB and IDO) 105 Institute for Development Studies and Practices (IDSP) and Taraqee Foundation 55 Total 560

Source, GoB, DoE, PPIU, slide number 12

The breakup of upgraded girls' schools by district is presented in table 12. Save the Children directly

undertook the construction for nine of the ten middle schools that were upgraded to the high level. The

remaining one school was constructed by the Construction and Works (C&W) department. Government

of Balochistan with SC providing all the required inputs in this last school.

Table 12: Breakup of Upgraded Schools for Girls by District

10

Type of schools

Districts Total Program target schools

Schools completed till Dec 2013

Total Schools completed till June 2014

Overall Completion rate

Middle to High

Quetta 7* 6 6 86% Middle to High

Mastung 2 2 2 100% Middle to High

Killa Abdullah 1 1 1 100% Total Middle to High* 10 9 9 90%

Primary to Middle

Quetta 17 16 17 100% Primary to Middle

Mastung 5 5 5 100% Primary to Middle

Killa Abdullah 3 3 3 100% Total Primary to Middle 25 24 2S 100% Source: SC, BEP Biannual Report, Jan-June 2014, p.11 *The GGMS Syedabad, Quetta was under construction by C&W, Govt, of Balochistan. Save the Children distributed furniture and equipment for science and computer labs soon after the completion of construction.

Veil facilities and materials

In the BEP s urvey conducted in October-November 2014, many of the teachers surveyed believed that the additional classrooms and upgraded facilities had positively impacted enrolment. Provision of water

was believed to be the most important facility, in the teachers' survey as shown in diagram 2, followed by additional rooms, classroom ventilation and provision of toilets. Boundary walls were given the last

Govt, of Balochistan, Department of Education, PPIU, September 2014, Policy and Right to Education (RTE ): Status of Early Childhood Education in Balochistan

It should be noted, and will be explored further in later sections of this report, that BEP co ntributions have helped in convincing the GoB to make ECE an integral part of the education system of the province.

SC, B EP Biannual Report, Jan- June 2014, p. 11

30

preference in terms of encouraging enrolment, which contradicts the commonly held view regarding the

importance of boundary walls, especially for girls' schools. The observation on boundary walls was true

for both girls' and boys' schools, each of which gave the priority to water as the most important facility

impacting enrolment. It is an important result given the severe water shortages in the province and the

deep water levels which inhibit the boring of wells.

Diagram 2. Which type of infrastructure facility is most important for promoting education according to the teachers?

70% • - - - •

60%

50%

40%

30%

20%

10%

0%

(U ao ro c u O k-v a.

d BEP=78

-;nonBEP=22

Water Supply

46% 59%

iüMlftl Additional

Room 22%

32%

Ventilation

19% 9%

Toilets

12% 0%

Boundary Wall

1% 0%

Source: SC, BEP Survey, Oct-Nov 2014, Question T56

According to the IPs, boundary walls for girls' schools may not be a priority area with regard to the

quality of education, but culturally, it is vital for all areas specified for the use of girls and women to

have boundary walls. Construction of these was undertaken in 40 percent of the 340 BEP schools.

Moreover, according to SCNL staff, during the proposal writing there were several meetings with schools

and communities, and the need was expressed for a boundary wall in every school where it was missing

or needed repair, by both parents and teachers. In urban and semi-urban areas the need for a boundary

wall in girls' schools was quite evident. It may also be that most of the schools surveyed, especially the

BEP schools, already had boundary walls in place, and hence the needs were not significant, in diagram

In this and all other survey diagrams the question number such as T56 in the source section gives the questionnaire reference and question number. For instance, 156 refers to the teachers' questionnaire, question #56. The BEP=78 (etc.) refers to the number of responses for teachers and students, or the number of school observations.

The need for boundary walls is culturally correct and the need was expressed in initial visits to schools for project design. However the baseline survey, also conducted for project design purposes, does not address this area.

31

Notably, the survey found full availability of working water facilities in BEP schools, as well as drinking water, which was not fully the case in other schools. Electric facilities were also more available in BEP focus schools than in other schools as shown in diagram 3.

Diagram 3. Provision of utilities in schools 120

u em ra 4-» c u OJ v a.

100

80

60

40

20

u Water facility Water facility Water clean and

provided working drinkable ! a BEP = 44 100 100 100 1 .Jnon-BEP = 23i 68 50 41 ;

Electricity available

75

65

Source: SC, BEP Survey, Oct-Nov 2014, Question Infra BEP#17; nonBEP#15

Focus group discussions corrobo rated these results. In interviews with child clubs, the children agreed that many of their requirements were being met through the Club, including requests for water wells, and a play area, for instance. In separate FGDs with district managers and community members, additional classrooms are mentioned as a key contribution of BEP, as in FGDs w ith Parent Teacher School Management Committees (PTSMC), which include teachers as m embers of the parent-teacher committees.

The teachers themselves w ere quite positive about the impact of the additional facilities on schooling, such as new rooms, boundary walls, school supplies, fu rniture and clean drinking water, as shown in diagram 4.

32

Diagram 4. Teachers' opinion aboutjacijities and positive [mpact on schooling 100%

QJ 00 ro +-• c v u 01 Q.

Impact of construction of

additional rooms and facilities on enrollment

IBBEP=76 88%

Impact of provision of

school supplies /furniture on

enrollment and attendance

Good teaching and learning

material available in

class?

Construction of boundary results in

increasing girls' enrollment?

86% 72% 89%

Impact of provision of

clean drinking water on

enrollment

89%

Source: SC, BEP Survey, Oct-Nov 2014, Teachers' #39-43

In terms of provision of school supplies, school observations in the BEP survey showed a majority schools (BEP and non-BEP) were provided with basic school supplies such as tab le, chair, and black board (Diagram 5), with Government of Balochistan prov iding to non-BEP schools thro ugh its regular programs. I t is heartening to note that an increased percentage of BEP schools are provided a water cooler and hygiene related items. Importantly, learning materials in BEP schools were observed to be high at 67% versus 25%in non- BEP schools. In the BEP schools the teachers were observed to be using the learning support material most of the time (46%) or some of the time (41%), versus non-BEP schools where i t is being used most of the time (15%) o r some of the time (31%) at lower rates.^^ In terms of extra-curricular activities there is not much difference with BEP schools organizing these 58% of the time and non-BEP schools 55% o f the time." The prov ision of chalk being observed as greater in non-BEP schools seems an odd result. The Program itself did not provide chalk t o any classes except ECE. It is possible that greater amounts of chalk w ere used in BEP schools, perhaps due to greater presence of teachers and more active teaching, thus leading to less being observed in those schools.

BEP survey, question 5046 and 5044 ' BEP 5urvey, 5037

33

Diagram 5. Availability of School Supplies 100% - •

at ns c at u 0^ a.

U7o

j Table | Chair Blackboa rd Cupboard

Stationar .. y/Chalk i

Water I „ , 1 , ; Samawar Bucket

cooler 1

aBEP=43 77% 1 72% 91% 84% 79% i 79% r 70% i 49% 42%

jnonBEP=20 65% : 70% 85% 65% 1

90% j 55% 40% T 20% 20%

Source: BEP Survey, Oct-Nov 2014, Question S062 (a-j)

. i-iniionnicnt

The child friendly environment was promoted for all BEP schools, for boys and girls. The results shown in diagram 6 are for girls. Although girls in both categories of schools, BEP and non-BEP, responded positively about feeling safe in their classrooms, many more non-BEP school girls admitted to having been punished by their teachers.

Diagram 6. Do you feel safe in th e school environment with the teacher? 120%

'i/i a >• 00 c '> n in

'QÓ *0 O 00 n c

V Q.

Do you feel safe in the school environment, with teacher and in

class room

Have you ever been punished by teacher in past 4 years

eBEP=500(appfox) I

,. nonBEP=150 (approx)' _9_5%

96%

39%

60%

Source: BEP school survey, Oct-Nov 2014 (St.52)

Additionally, in terms of toilet facilities, 88 percent of the toilets were used o nly by children in BEP schools versus 25 percent in other (non-BEP) schools also a positive indicator for the Program. The Program has provided a considerably more child friendly environment than other schools, as

34

demonstrated in diagram 7. Especially noteworthy are the disability ramps and bars, which although not observed in all BEP schools, are still a solid example, given that there are 82,000 disabled children in Balochistan, many of them out of school according to the Child Rights M ovement (CRM), Balochistan

15 chapter representatives.

Diagram 7. Provision of child friendly school environment

0^ QO

c O u 0^

CI.

El —, 5? 1 —,

i i. ii

t

i i. ii

i i. ii — i

i. ii — i

i. ii

U BEP=45

.:non-BEP = 23

Children's No

views isolated

taken in school

unvigilant

design area

93 57

Clean and safe I

j enviro nm I ent i

Green

, , , tiwi , iwi

Safe & ! crowded disabled Rounded ' and

and I accessib le ] ' WASH !

84

48 60

48

ventilatio n

64

13

84

52

children- corners of seating grab bars walls space and

ramps

69 36 47 22 " 4 22

Playing areas

78 43

Source: BEP Survey, Infra BEP#3-13; nonBEP#l-ll

Although it was found that 62 percent of the play areas had a plain, levele d surface, almost 37 percent has constru ction debris left over.'^ A ma jor 84 percent of BEP school children stated that the school environment is clean and safe as compared to 48 percent from non-BEP schools.

Conclusions Objective 1:

The objectives of component 1 are deemed to have been met in a satisfactory manner. The Program has managed to expand and improve the existing infrastructure and surpass by 26 percen t the targeted number of children in school that were addressed through the Program. The Program achieved the targets of construction and establishment of new ECE centers, as well as up-gradation of schools from primary to middle and middle to higher levels, t hus opening up avenues for children to continue in school, especially girls. The cre ation of a safe and friendly environment in targeted schools has been achieved to quite a large exte nt, although not fully, with the majority of children agreeing that they felt schools were a clean and safe environment. The teachers were very positive about the impact of new

Dally Balochistan Express, December 5, 2014, "Balochistan has 85,000 disabled children". This figure is about 2.6 percent of the total 3.6 million children in Ba lochistan, of whom on ly 1.3 mill ion are in sch ool, according to the Child Rights Movement (CRM) quoted in the newspaper.

Source: BEP Survey, Infra BEP#21, nonBEP#18 (BEP: 15-31 observations; non-BEP: 12-15 observations

35

facilities and furniture on schooling as well as availability of water and boundary walls. Importantly clean drinking water was fully available in the schools surveyed. Almost 88 percent of the toilets in BEP schools were in use by children alone, unlike other schools. Espec ially no teworthy were the disability bars and r amps installed in over a th ird of the schools. Schools had a good supply o f basic classroom materials and an excellent supply of learning materials.^® Children generally considered the school a sa fe place with a clean environ ment.

: • : : i V I- 2

To improve the quality of education through promoting child friendly and participatory teaching methodologies in all targeted schools of the three districts of Balochistan by end of 2014.

The overall teacher - student ratio in Balochistan is 1:21 (male 1:21 and female 1:26) which is considered a good range and apparently reflects good quality. However, according to experts the actual ground realities are different due to lack of rationalization of teacher demand and supply, and variation of this ratio across urban and rural areas.Another continued worrisome issue is physical presence of teachers in schools, regardless of the student-teacher ratio, a common and serious problem. The average rates for all schools in t he three focus districts over a two year time period is presented in diagram 8.^° The ratios show that the averages in all girls' schools are better than those in the BEP supported girls' schools, probably due to higher than average enrolments in the BEP supported schools.

Diagram 8. Primary school student teacher ratios

1:35 r

ra BC

u ro 01

c OJ

•O 3

Boys (BEMIS) Boys (BEP) ' Girls (BEMIS)

U Student Teacher Ratio 1:32 1:31 1:27

Girls (BEP)

1:34

The impact on parents and children has also been positive, which is highlighted in later chapters on effectiveness and impact.

The provided supplies were being used to a great extent, as highlighted in a later chapter on effectiveness. Strengthening Participatory Organization (SPG), April 2013, Situation Analysis of Education System in

Balochistan, p.14 Data on the middle and high school student-teacher ratio is also available and shows a b etter availability of

teachers than at the primary level.

36

Source: Derived from GoB, BEMIS 2012-13 and BEP Survey 2014

:;inGt!on of students in the three districts In the focus districts, there is an increase in promotion rates for girls in two of the three districts, but a

decline overall for all Balochistan districts, as presented in diagram 9, which points to an educational

decline over the three year time period as d iscussed in the background chapter. The completion rates

for girls in the BEP ta rgeted schools are higher when compared to the district averages: 85 percent for

ECE, 85 percent for primary, 93 percent for middle, and 96 percent for the high level.

Diagram 9. Promotion rates of girls in focus districts overall [%]

80

70 - -

a; OS re 4-* c 01 u k. 01 a.

u • • Killa Abdulla Mastung Quetta

Total for all districts

u 2010-11 Girls' 46 66 60 _ 63

u 2013-14 Girls 45 70 64 61 1

Source: Government of Balochistan (GoB), BEMIS, School Census, 2010-11 and 2013-14

The total retention rate for the reporting period remained at 91%. The retention rate for boys was

recorded at 96% while the retention rate for girl students was 87%.^^ Transition rates for students are

presented in table 13 and are higher for BEP schools than the averages for the entire three districts.

Table 13: Transition rate of girls - BEP schools Primary to middle Middle to Secondary

BEP data for focus schools 63% 77%

Census for the three districts 52% 72%

Source: SC, BEP Annual Report 2014, p.9 and GoB , School Census, 2013-14

lis mi tCE

In terms of development of teachers, a key product of the Program is that teachers have learned

important teaching skills and have also learned in their experience with the PTSMCs, the Child Clubs, which goes beyond the basic training aspects. The impact of training is presented in diagram 10 and

demonstrates that teachers learned from the training in a variety of ways. In terms of ECE training, the

SC, BEP Bi-Annual Report, Jan-June 2014, p.9

37

11 ECE teachers interviewed all concluded that the training had positively changed their training approach."

Diagram 10. What is the most important effect or change that this BEP education project has had on you personally?

Better teaching ability

1% Understanding, community Better" perception Communication

6% 0%

, Sensitization about Health &

Hygiene . 28%

Source: BEP Survey, Oct-Nov 2014 (Question ISO) (Responses from teachers = 72)

Diagram 11 shows that most teachers agreed that ECE t raining helped them, either mo stly or to a gre at extent (total 67%) and its effectiveness (total 76%).

Diagram II. Effectiveness of training

00 m 4-* c Q) u 0) Q.

45% 40% 35% 30% 25% 20% 15% 10%

5% 0% r

i Not at all effective

Ö If yes, how effective was the training (s)

u Did your ECE training proved to be supportive in improving your skills to engage and educate children in ECE

activities

0%

8%

Very litle to some extent

8%

8%

15%

17%

mostly to a great

extent

38% 38%

42% 25%

Source: BEP Survey, Oct-Nov 2014 (Question T15-15) (Responses from teachers = 12-13)

Important skills were defined as skills that are specific to teaching

38

As shown in diagram 12, the teachers were observed to be using participatory teaching methods most of the time (37%) or some of the time (54%), which is a fair majo rity of the teachers.

Diagram 12. is teachers' teaching method participatory [according to school observations]? I

60%

OJ OS ro w c (V t! 01 a

y BEP=41

. J nonBEP=14

Source: BEP Survey, Oct-Nov 2014 (Question S045) (Teachers observed BEP= 39 non-BEP=13)

in terms of seating arrangements, the majority of teachers continue to use rows for seating students, rather than the preferred U-shaped seating arrangements, as shown in diagram 13. This result probably applies more to middle and secondary schools, since the furniture provided to ECE centers included round or square tables for students to sit around in order to promote interactive and participatory learning. Diagram 13. How has the teacher organized sitting arrangements for children?

70%

Source: BEP Survey, Oct-Nov 2014 (Question T-27) (Teachers responses= 9)

39

However, ECE and learning support materials are being used for the most part as shown in diagram 14.

Diagram 14. Is the teacher utilizing ECE material and learning support material? )

50 ; ^ —

45

40

35

& 30

£ 25 u <u a

u Teachers observed=32

Source: BEP Survey, Oct-Nov 2014 (Question 5047)

Very important, a large majority of children in BEP scho ols stated that the teacher makes the learning

process interesting and encourages classroom participation as shown in diagram 15, although the

results obtained were much the same for the non-BEP schools.

Diagram 15. Communications with teachers and participation

98% 97% 96% 95% 94% 93% 92% 91% 90% 89%

OJ OS ra +.» c HI u HI Q.

Are you allowed to ask the teacher questions

in the classroom

Does your teacher encourage you to

actively particpate in school activities

y BEP=800

.nonBEP=430 97%

96%

94%

95%

1 Does the teacher make ;

the learning process I interesting for you |

92% !

92%

Source: BEP Survey, St-37-39

Students were further asked how the teacher makes the process interesting. Diagram 16, and although a

majority said it was through participatory and inactive teaching, the agreement on this from non-BEP

students was even greater, although relatively more BEP supported schools are using visual aids.

40

Diagram 16. How does the teacher make the learning process interesting for you?

QJ QJO ra c 01 u <u a

0% h-1 Participatory/

Interactive Using Visual Aids Others

yBEP=820 ' 84% 15% 2% J nonBEP=400 93% 4% 3%

Source: BEP Survey, Oct-Nov 2014 (St.40)

• i:,aUtative interviews on ECE The Additional Secretary for Education particularly emphasized on the good work done, in institutionalizing ECE. Delays in teacher appointments negatively affected the efforts although posts have been approved and created and there are a number of proposals under consideration of the department for appointment of the teachers, according to the departmental representative. Most notably, "the ECE policy has been drafted and is in the process of being finalized, with ECE a cent ral focus of the approved and under-implemented Balochistan Education Sector Plan", as stated by the Additional Secretary. The d istrict education managers agreed th at the "ECE has been in tegrated in the Education Sector Plan due to provision of good results in the schools and provision of modern techniques of teaching." And "particularly, ECE has been made part of the Balochistan Education Sector Plan, by the Government after recognizing its importance demonstrated under BEP". They also confirmed that the ECE policy is being finalized by the Government.

The community agreed on the importance of ECE: "We felt a great impact on children's enrolment in the pre-primary school due to the construction of the ECE cente r", according to the Killa Abdulla h PTSMC members. The Killa Abdullah child club members agreed about ECE that: "yes it is very beautiful not only by structure but also by education materials even to us and we feel if it is functional it would be helpful for us and our youngsters".

According to IDO, one of the implementing partners, "the ECE centers constructed which have n ewly hired and trained teachers with a lot teaching and learning materials available are an important landmark for the program and an evidence based model for the government to further expand to other parts of the province. The prin t-rich environment and the participatory learning methodologies in the ECE class es have provided a boost to the standard of teaching not usually associated with public schools". The IP representative also felt that the program had greatly improved the facilities and standard of teaching and le arning in grades 1 and 2 in the 105 schools where ECE centers have been constructed. "Children promoted from ECE to grades 1 and 2 w ill not feel any gap in the standards of teaching and learning."

41

• :n-'nq of teachers The training of teachers has been overall appreciated with the Additional Secretary expressing th at "Teachers have been trained and so have th e PITE and BoC staff. It is expected that they will use the training and modules developed and impart a better training to teachers, after being better-equipped". According to the district education managers "the trainings have r esulted in building the capacities of our teachers. The teachers are now well trained and delivering education to the best of their capacity". The DEOs also expressed that additional trainings are needed in administrative management of the schools, performance, classroom environment and overall management. Moreover, they required continuous re-training and refresher courses on child rights and protection, school health and nutrition (SHN), leadership and management, computer trainings and trainings on the use of DEMIS and making data based decisions.

• clfULs learning achievements Most importantly, it is useful to see what children are actually learning. The BEP survey conducted some tests although the numbers observed were few. The first two bars in diagram 17 refer to writing in Urdu and English, the next two to reading in English and Urdu, and the last four to basic ma th functions, where BEP children in primary are doing better in two of the four math functions than their non-BEP counterparts.

Diagram 17. Observations (primary, middle and high] regarding student learning achievements

a; &o re *«» § I

isiif J. U BEP=40 • i nonBEP=19

Slate/ „ , Charts/ , , . Subtracti Multiplic , ^ . Books , „ J Addition • Division Takhti Cards on ation

81% 63%

19% 37%

46% 53%

32% 29%

5% 12%

17% 6%

Source: BEP school survey, Oct-Nov 2014 (S061) (BEP obs. = 39, 42, 41 non-BEP obs. = 19, 19, 17)

In terms of socialization and other kinds of learning, diagram 18 has information only on the average and excellent results for a simp ler presentation, which gives a f air idea of the responses. The highest percentages are for observations of children's concentration, working in a group, helping other children, and girls interacting with other girls.

42

nijgram 18. Socialization and learning of children in BEP schools

60 - -

(U 00 TO 4-* c 01 u 1-<u a.

50

40 r

30

20 -

10

u Average

J Excellent

Are ' Are boys children ' a nd girls willing to equally particpat engaged einECE in ECE activities activities

= 10 =5

30 33

Are Are

children

20 17

children children | willing to, interact able to work in a

concentr group = ate = 10 10

30

60

Source: BEP school survey, Oct-Nov 2014 (5029) (The #s next to each category are the # of observations)

Positively, the learning process taught the children "how to make presentations and instilled confidence"; the children felt that "they were receiving a good education"; and they "enjoyed the learning environment".

Conclusions Objective 2:

The objectives of component 2 are deemed to have been met in terms of promotion of child friendly methods and the indicators on student-teacher ratios, completion and transition, but less so i n the areas of development of cognitive skills and overall confidence levels of the children. In terms of some quality indicators the student teacher ratios are satisfactory although higher than the district averages probably due to higher enrolments in BEP schools, especially for girls. The promotion rates have declined over all in Balochistan, whereas the BEP promotion/completion are about 25-35 percent higher, about 85 percent for ECE and pri mary and, quite high at 95 percent for the middle and high levels. The tra nsition rates, from primary to middle and middle to higher are also gre ater in the BEP schools than in the districts overall, which shows th at the BEP objec tive of having children continue school to higher levels has been positive. This is an important achievement given the high dropouts rates that persist traditionally at the end of each level of schooling, especially grades 1 to 5. Training has been enhanced through the involvement of PITE in teacher training and the teachers have learned new skills in teaching. Their teach ing methods are observed to be participatory although about two thirds are still following the traditional row seating in classrooms. A solid major ity is using the learning materials while teaching. Notably, over 75 percent of the teachers find the ECE training quite effective. PITE and other government officials were quite vocal about their appreciation of the teachers' training techniques

43

instilled by the Program, especially for ECE. Conspicuously, a majority of children agree that the teacher makes the learning process enjoyable, mainly due to use of participatory techniques, and to a lesser extent due to use of the learning materials, which needs further examination. Almost all students interviewed expressed that they are able to ask questions of their teachers and that they are encouraged t o participate, although this was equally true for BEP and other schools. A ma jority of community members and PTSMC members agreed that the ECE classes had transformed the educational process in their neighborhoods. Student learning outcomes are mixed with relatively poor learning of math concepts, in both BEP and other schools. The BEP school children are better equipped in participation in ECE activities and participation in groups, but the other school children did considerably better in terms of concentration skills, willingness to help other children and overall confidence levels which is a debatable r esult. Discussions with the community members and child clubs members pointed to greater confidence of children participating in the BEP activities and schools leading to the need for further research in this area.

OBjfXriVE 3 To improve educational planning and budgeting systems in the Balochistan province through the development and introduction of evidence based Distr ict Education Planning models, as we ll as the strengthening of Education Management Information Systems by 2014 (increase edu cation budget and strengthen management).

The provincial data base has been substantially strengthened with annual statistical reports prepared for the Balochistan Education Management Information System (BEMIS) through district level reports for the three focus districts. Although planning and budgeting was a p art of the objective, including EMIS strengthening, it was later decided to not address this, after consultation with EKN, since other donors were already addressing this aspect. A study had already been conducted in 2011 for tracking of budgetary information to prepare the ground work for this activity."

According to the Additional Secretary " the District Education Management Information Systems have been established and activated, with the support of BEP; however there are gaps in functionality of these DEMIS and HR is one of the issues". The district education managers stated that although training was provided by the IDO on DEMIS additional training would be useful. The Quetta education manager said that "DEMIS plays a pi votal role for monitoring the schools. If the system is p rovided with good manpower, then the data would be reliable and the system would work efficiently. Quetta has recently launched EMIS where the teachers' profiling has been started for the smooth running of the administration". In Killa Abdullah, DEMIS was ineffective due to the absence of technical staff. In Mastung "DEMIS is e ffective in managing education data and the DEMIS cell is keeping the records in safe custody. The biggest challenge is that we are lacking IT skills and ability to effectively manage computers and EMIS (DEMIS) since there are no skilled information technology (IT) personnel available with the District Education Offices".

" BEP, l-SAPS, 2011, Budgetary Tracking in Education Sector in Balochistan

44

The im plementing partner, IDO, also shared that DEMIS cells had been established in three focused districts of Balochistan, are fully operational and the staff in these cells is rece iving continual capacity building training to enable them to provide maximum support to the education managers. However, as already noted, technical staff is insufficient in Killa Abdullah. The IDO trainings for BEMIS were corroborated by the district education managers, who confirmed that training was provides on use of the computer, and on use o f DEMIS in data based decision making (DBDM). According to IDO, the strengthening of the EMIS project was piloted in Quetta (where the DEMIS was set u p under BEP) and has now been up-scaled t o the entire province. The EMIS includes profiles on teachers, schools and students. Further, according to the IP, the government is scaling up EMIS wit h the financial support of UNICEF and the technical support of IDO. The EMI S website www.emis.gob.pk was inaugurated by the Chief Minister of Balochistan.

•iifi'iiciitv

The main constraint faced in the up-gradation of schools in Quetta was the unavailability of land for the construction of additional classrooms. To overcome the issue a series of joint meetings with partners and communities were held with the District Government, Commissioner Quetta Division, local communities and the Minister for Education to petition for the provision of land. The communities were very vocal in sharing their concerns with the Government and managed to mobilize support from the Minister of Education. The overall efforts led to the acquisition of land for five schools to be upgraded. The challenge was met with consistent and issue focused advocacy. Th is was an imp ortant contribution and a major success for BEP's advocacy with government.

Parents have been effectively involved in the process of education. The motivation of parents has led to resource mobilization equivalent to PKR 31,938,200 in cash or kind through acquiring land for construction of new school buildings in some areas, per the interview with IDQ. There were several instances in 2012 and 2013 whereby community members donated their lands for schools.

The proactive involvement of the community and through linkages between government and community has created ownership of the program. Apart from the in-kind and cash contr ibution by the communities and government, the stakeholders were also sensitized and advocated on education issues of their area with senior government officials. The case o f GGHS Scho ol Lore Karez renamed Ghous Bakhsh Raisani school is a case in p oint where the government allocated land for the school. However, the land was occupied by some influential people, which hindered construction of the building. The community members approached the Secretary of Education and Commissioner Quetta, and further involved the Chief M inister. The l atter took swift action and possession of the land was given to the school. The ground breaking ceremony of the school was done by th e Chief Minister and an add itional donation of PKR 5 millio n provided for the construction of the school. Notably, the status of school was elevated from secondary to higher secondary level.

Another example is the primary school in Killa Muhammad Hassni in District Quetta, where the community and education department supported the acquisition of land for the school. Save the

' SC, B EP Bi-annual Report (Jan-June 2012)

45

Children allocated additional funds for the construction of rooms for the school. "Surely, we have touched a few lives the re!" exclaimed the SC staff member.

As shared by the Additional Secretary for Education, "other good initiatives like SHN and child protection are likely to go further". He said that the department has drafted a law for corporal punishment and that child protection and SHN should be a part of other programmes and projects. He further said the department will discuss these measures for inclusion with other partners before the start of new projects. Obtaining the matching grants and land for the project was achieved through the efforts of all the BEP partners.

Conclusions objective 3;

The objectives of component 3 are deemed to have been partially met since the Program did help to set up the District Education Management Information System (DEMIS) in the three districts.

The links between planning and budgeting were not addressed greatly; the activity was rescinded since other donors were already working in this area. Although staff was trained, there is a lac k of staff in one district which means that in effect the system is not working there. A success of the DEMIS is the scaling up to all districts of Balochistan where data entry points have been set up and data entry is in process. The enriched EMIS collects data about schools, teachers and for the first time in Pakistan, the Balochistan EMIS is collecting data on individual students. There is a great deal of ownership on the part of Balochistan government. The actual use of data based decision making is not emphasized in the work done with the districts, or the use o f EMIS in different educational areas such as human resource management or payroll management, although that is expected in the future. In terms of other areas of advocacy, the Program was successf ul generating land donations from community and government for the construction of additional rooms and ECE centers, which was a notable achievement since community members were also involved in raising their voice in this regard. The fundraising by community members for schools was also a notable achievement of the Program, given the poverty of the areas worked in.

OI'TiECTlVI:; 4 Improve School Health and Nutrition in 340 Schools of 3 Districts of Balochistan by 2014

The school h ealth survey has been used as a comparison for a recen t end-line survey for health and hygiene (H&H) which was referred to earlier under the component four discussion.

The intervention was designed to address key health and nutrition problems of primary school children and adolescent girls. The base-line and end-line survey used the same methodological approaches." A baseline was conducted in June 2012 and the e nd-line was conducted in September 2014. Both surveys had equal sample size of 1200 primary school children and 180 adolescent girls of middle and high schools. Two-stage sampling design was used to select c hildren from 58 schools (40 primary and 18

" This section is b ased on the following report: S C, B EP-SHN, November 2014, End-line Survey Draft Report and the reader should refer to that for details.

46

middle and high schools) in t hree districts. Both surveys used the same instruments and procedures to collect data on knoNwIedge, attitude and practices (KAP) regarding health and nutrition of children, iodized salt consumption at home and anthropometry.

Primary School Children The overall end-line assessment indicated a pos itive change in knowledge and practices of safe drinking water. Knowledge about boiled water (11% vs 38%), hand pump (1% vs 2%) and tube well (17% vs 37%), as sa fe sources has significantly improved. However, knowledge regarding government supplied water as safe had significantly declined (50% vs 14). Although there was an increase in percentage of children reporting canal/fountain water open (1% vs 2% ), river water (1% vs 6% ) and rain water (1% vs 2% ) as safe the result was attributed only to Killa Abdullah. The same trend was observed in safe drinking water practices. Utilization of water from own source/tanker, hand pump, tube well and canal/fountain (closed) had increased. A significant decline in utilization of government water (62% vs 57%) was observed.

There was an overall improvement in hand washing practices but a decline i n knowledge about critical occasions of when to do it. Hand washing practices after using toilet, after playing games and when hands get dirty had significantly improved. However, hand washing practices before eating had significantly declined. Moreover, use o f water and soap for hand washing had increased (79% vs 82% ) but there was no change in co rrect hand washing practices. However, knowledge regarding two critical occasions, that is, after using toilet and before eating had significantly declined.

Food hygiene practices and a ttitude to food hygiene had declined as compared t o baseline, while water intake practices in diarrhea had improved (45% vs 48% ). Similarly, latrine utilization had also declined (93% vs 84%) among school children. Knowledge about disease spread by flies had significantly improved although attitude towards disease prevention had declined (66% vs 61%).

Teeth cleaning practices had significantly improved especially cleaning teeth at critical occasions, that is, after breakfast (40% vs 60 %) and before sleeping at night (26% vs 47%). Daily sw eet consumption had declined among children and there was also a decline in different types of sweets consumed. In addition, knowledge about cavities had increased (53% vs 6 0%) while knowledge about tooth ache had declined (48% vs 44%). Knowledge about balanced diet has declined (83% vs 67%) among school children. Similarly intake of balanced d iet at home had also declined (83% vs 67%). Knowledge about iodized salt had im proved but recognition of iodized salt logo had declined. There was also a decline in household utilization of iodized salt at children's homes (77% vs 50%).

Knowledge about malaria had increased f rom 19% to 29% and knowledge about malaria prevention had improved among school children. Nevertheless, knowledge about how malaria actually spread had declined (81% vs 66%) while malaria pre vention practices have improved among school children.

Ailalescent girls There was an overall positive change in knowledge and practices of safe drinking water among adolescent girls, such as safe drinking water sources l ike boiled, hand pump and tube well water. However, knowledge about government supplied water as safe source had declined (11% vs 7% ) and knowledge about unsafe drinking water had declined, both attributed to Killa Abdullah.

47

Hand washing practices at critical occasions were not significantly improved. While there was improvement in hand washing practices after using latrine (65% vs 73%), hand washing before eating (87% vs 80) had declined. There was a significant improvement in hand washing when hands get dirty but a non-significant decline in hand washing practices after playing games. Water and soap uti lization for hand washing had increased (88% vs 92%).

Food hygiene practices among adolescent girls had declined (97% vs 91%). Similarly, water intake practices in diarrhea and latrine utilization had also declined among the surveyed group. Teeth cleaning practices had significantly improved especially cleaning teeth at critical time's i.e after breakfast (40% vs 60%) and before sleeping at night (54% vs 79%). Similarly knowledge about teeth cleaning at critical times had also significantly improved, Daily sweet consumption had declined (49% vs 41%). Consumption of toffees/chocolates had declined but consumption of biscuits and sweets had increased. Knowledge about consequences of daily consumption of sweets had improved. Knowledge about balanced diet had significantly declined (96% vs 8 8%) while knowledge about balanced diet component had significantly improved. Similarly intake of balanced die t at home had also decline d (92% vs 88% ). Knowledge about iodized salt had improved. However, recognition of iodized salt logo had declined.

Overall knowledge about mode of disease spread by flie s was not improved although attitude towards disease prev ention had improved (81% vs 87% ). Knowledge about malaria had declined (70% vs 64% ) while knowledge about malaria prevention had improved among adolescent girls. Similarly, knowledge about how malaria spread had declined (95% vs 90%, while malaria prevention practices have improved.

Percentage of girls who had heard about periods before puberty had increased fr om 54% to 78%. Role of teachers in providing information about period had increased (1% vs 10%). Similarly hygiene practices during periods and sanitary pad utilization had also im proved. Similarly attitude towards daily body cleaning had also improved (40% vs 49%).

Conclusions Objective 4:

The ef forts of the BEP team have been satisfactory although the results themselves are mixed. The comparison of base-line and end-line suggests that there has been improvement in key indicators such as positive change in knowledge and practices of safe drinking water that has been significantly increased. Especially in Quetta; similarly teeth cleaning practice had significantly improved such as cleaning teeth at critical occasions i.e. after breakfast (40% vs 60%) and before sleeping at night (26% vs 47%). There has also been a significant increase in knowledge of washing hands after playing and whenever gets dirty %( 14 Vs 26 and 17 Vs 30 consecutively). However there has been a decline in some indicators like knowledge about critical occasions of hand washing, practice and knowledge about balanced diet. But th is data contradicts with the data collected through another tool used in overal l BEP evaluation. The data reveals that more than 90% of the children wash their hands before and after eating food and using toilets. While the baseline figure was 66%. So, there is a high probability of anomaly during data collection. Save the Children had no access to the field to support enumerators

48

while collecting data during end-line survey in two of the districts (Killa Abdullah and Mastung) due to security reasons. That might have affected quality of the data.

Also the biorhedical part of the survey was dropped again due to security reasons tha t included stool testing for parasites and blood test for Iron deficiency anemia. This also lim ited to gauge the project success in terms of health improvement. There was no recent relevant secondary d ata available that could have been linked to the prpject achievements in program areas.

These findings should be complemented by project internal and external review with stakeholders to investigate any implementation bottlenecks and explanations for large variations in the base-line and end-line surveys in some of the key indicators.

V. Relevance Relevance is addressed in terms of the basic goal of the Balochistan Education program: "To bring positive changes: in the lives of vulnerable and marginalized children, mainly girls, through the provision of quality education in three districts of Balochistan in particular and beyond in general On sustainable basis by 2014."

Initial funding proposal The funding proposal for the Program looked at basic statistics for Balochistan and based the activities on that. For instance, it states that 83% of the schools do not have boundary walls and hence indirectly concludes that this is a req uirement for ,advancing education. The reason boun dary walls are a missing facility in schools across Pakistan is because these can be quite expensive, especially in an area like Balochistan where the school may have a large tract of land, and also because in more rural communities parents may prefer to be able to see their children in the school. In the context of Balochistan, and in other provinces, water is also an important entity for schools and children, as are toilets. Also, the proposal assumes that girls' education is rio t a pr iority for parents and communities; however, work done more than two decades earlier in the province showed that girls' education was a priority for parents.^® An initia l sUrvey of discussions with community on what actual needs were could have advanced community ownership and participation as well as additional focus oh essential needs. Very im portantly, a key rhissing Variable in the analysis as well as in BEP im plenientation thereof, is

For instance, "most of the girls attending boys' schools were in rempte areas of the province, thereby exploding another myth that said people in rural areas were not interested in sending their girls to school." USAID, September 1994, Primary Education Development Program, Final Report, p. 95

49

addressing the issue of teacher absenteeism, which is quite dire in Pakistan and especially Balochistan. This important aspect was included during implementation of the program."

The v alidity of the key objectives of the program has been addressed in the four previous sections dealing with program objectives, and conclusions are stated in each of those sections.

Rfcdevaiice of program design and structure to achieving the intended project goal The program design and str ucture had the overall elements for an all-inclusive education program; the program addressed ECE comprehensively and a ttempted to address the issues of hig h dropouts among girls, especially in th e early grades and then at the fifth and eighth grade levels, where dropouts occur primarily because the re are no school facili ties available for girls. The missing ele ment in the objective dealing with the promotion of quality education is st udent testing to ascertain actual learning in the classrooms as a result of the quality interventions. The program proposal mentions "improved learning outcomes for children in target schools" (p . 12) as an expected outcome. The assumption in program design and actual program implementation is that all the quality inputs will translate into improved learning outcomes. However, this is a major assumption and one commonly made when teacher training and other quality interventions are designed and implemented. Other than PITE, w hich probably does not consider learning outcomes either, the discrepancy could have been addressed thr ough partnership with the Provincial Educational Assessment Center (PEACe), albeit the severe capacity constraints of the latter. The related impact on and inter linkage with curriculum development was also left unaddressed.

Relevance of assistance mechanisms and benef its to the needs of beneficiaries As explained previously, the benefits to the beneficiaries were looked at through the provincial . , framework rather than in discussion with actual community level beneficiaries, although the provincial representatives were engaged. The program was designed before the global restructuring of Save the Children. However, in the first phase of implementation SC became a single entity and suffered staff n changes, as well as some m ajor staff changes in the education section of Save th e Children Pakistan, , j both of which impacted the actual implementation scenario. Initially the plan was to implement in Quetta alone, according to SC Pakista n staff, but based on staff priorities another two districts were added. These were selected based on discussions with education officials which took the needs of the districts into account according to SC staff. Howeve r, the analysis of the needs of these two districts is not discussed in the funding proposal although the sharing of program objectives with UNICEF is mentioned in the context of obtaining assistance for child protection issues. The other two were decided upon once funds were approved by the donor and the government selection criteria were based on

" According to one of the BE P pa rtners, the issue of teacher abse nteeism teach er abse nteeism was inc luded within the scope of P TSMCs roles and responsibilities through the training of PTSMCs on school management and monitoring during implementation. Moreover, SC teams used to visit ea ch s chool at leas t once in a mon th. The teacher attendance registers were checked accordingly and important issues were brought up for the kn owledge of the PT SMC m embers. Further the attend ance of teac hers who wer e on the payroll of Sa ve t he Chi ldren was strictly monitored and verified by the he admaster and E DO before releasing their sa lary and due to thes e efforts absenteeism was quite in control in BEP schools.

50

poverty criteria, out of school children and ethnicity, the latter of special interest to Save the Children. As presented in the introduction, both Mastung and Killa Abdullah have different ethnic groups, whereas Quetta is an amalgam of several ethnicities. Although not reflected in the proposal, SC conducted an in-house districts analysis and dur ing meetings with the provincial government representatives took up district selection as a priority. The selec tion criteria was based on examining the (1) educational statistics i.e. availability of schools with gender break up, total population, presence of other donors to avoid duplication (2) ethnic sensitivity (given the ethnic polarization that has aggravated in Balochistan, the organization wanted to select one Pushtun and one Baloch district) and (3) security situation so that the project team could remain operational and mobile in the field. After taking its time to consider, the Government of Balochistan advised t o choose Mastung from the Baloch and Killa Abdullah from the Pushtun belt as ou r target districts (this is i n addition to Quetta).^®

District Quetta needs are also based on a ten school pilot project implemented in 2008 as a precursor to BEP. According to other sources, literacy rates were highest in Quetta Zarghoon (59%) and Quetta Chiltan (49%) towns", which questions the assumptions for continuing the program in Quetta. According to SCNL staff, it was very strongly recommended in many meetings to start the pilot in Quetta district as there were almost no donors working in that district. Moreover, the needs were high since many families migrate from rural areas to Quetta and there is great need for quality education provision in the less developed areas of Quetta district.

Pro<iram de.sign compliance with government policy guidelines and adjustment thereof The program complied with Government of Pakistan policy guidelines, technically, in the sense that BEP components were within the framework of the National Education Policy (NEP) of 2009. Further, the Program was st ructured to adapt to the changing policy environment due to Constitutional changes placing more educational provision responsibility on the provinces In that objective 2 specifically addressed policy, planning and budgeting. The provincial policy and planning aspects and linking with budgeting were relegated to another development partner once the Program began. The change in policy guidelines also impacted curriculum development which was perhaps wisely not addressed in the Program, given the greater expertise of other development partners in this complex area, and the practical follow up needed for textbook development and provision. There was concern at the secretariat level regarding the high cost construction activities of BEP.

Conclusions on relevance:

The Program was and remains very relevant in assessment of the situation of the province in terms of poverty and vulnerability, and education provision, student learning scores, school infrastructure and gender parity. However, not all Program activities were well thought out. The Program proposal

" Email from E ducation Manager, SC Pakistan to SCNL dated February 3, 2010 and supported by a letter from the then education minister in Balochistan issuing a notice about the three districts dated January 27, 2010 " UNICEF, 2004, MICs Report, p. 30

51

J

assumed that girls' education is not important for parents in Balochistan, which is fac tually not proven. The issue of teacher absenteeism is not discussed in the proposal although the addressing of this issue was included in the actual implementation through SC and PTS MC interven tions. Enabling children to transition from primary to middle and middle to high was and remains very relevant. However, student testing and learning is an area that is not addressed adequately in the proposal or in the actual interventions, although some efforts have been made, for instance in the area of mother tongue, literacy boost and numeracy boost, as discussed in a later ch apter. BEP planning and techniques needed honing in the area of quality education and development of cognitive skills. Although PITE was fully involved, the Provincial Educational Assessment Center (PEACe) was not involved in assisting in diagnosing issues in student learning. The Program team made considerable efforts to keep on board the changing government officials and steering committee meetings were held regularly to facilitate Program bottlenecks.

VI. Effectiveness

Baseline survey A baseline survey for BEP was conducted in 2010-11 for which the TORs were to help design the program.^" According to discussions with staff of SC, the process was not satisfactory and the baseline of limited use since there was data loss due to technical problems as well as oth er issues.Moreover, the actual results were available too late to actual inform program design. Embassy o f the Kingdom of Netherlands representatives offered detailed comments on the report which were perhaps not addressed given the serious issues with the data cited above.^^ The end result was that the baseline was inadequate for use by SC to monitor program impact. A second BEP baseline survey was conduc ted specifically for the school health and nutrition component of BEP, although this was the first baseline for the health component.This report fared better and a formal seminar for sharing the findings of the study was conduct ed in December, 2012 and included participants from government such as the Chief Planning Officer of the Health Department, the Secretary o f Education and the Deputy Secretary of

I Education. 34

Quality of monitoring According to both SC and the implementing partners (IPs), regular meetings were held between IPs and SC Balochistan, quarterly with SC Islamabad and few times a year wi th SCNL. The BEP conducted reviews at various intervals; monthly, quarterly, annually and steering committees meetings which made for a strong control over the project. In addition, there were weekly skype calls b etween the SCNL program

' Asifa Khanum, November 2011, Education and Protection of Children in Pakistan - Baseline Survey Findings ' Save the Children, Balochistan Education Program, Annual Report - January to December 2011, p.6 ' EKN, May 2011, Comments on Baseline Survey ' SC (not specified), date not specified, BEP-SHN Baseline Survey Report - Balochistan • SC, BEP, Annual Report 2012, p.34

52

I

manager and the BEP teams in Islamabad and Quetta. The system o f reporting to SC was generally effective, that is, there were periodic reports going out to SC and SCNL on a regular basis, and follow up meetings to identify and address issues related to program implementation. According to one of the IPs, it "found SC very responsive and helpful in any kind of concerns or issues raised, and they always responded very positively if any issues were raised".The SC MEAL te am also regularly monitored project activities, and the SC Islamabad team was also involved in coordination with the Quetta project teams for ensuring quality.^^ Moreover, other than continuous monitoring, evaluations, monthly / quarterly review meetings and progress sharing, other external factors that influenced the results of BEP are community interest supported by the PTSMCs and appreciation and support by the Education Department. On several occasions t he Secretary of Education and Minister of Education, Balochistan, attended meetings and major events, which is a facet of monitoring. One element of monitoring that is missing is recording of achievements against activities in the logframe cumulatively; the information is available only in biannual and annual report for the period under review.

Security constraints The program manager at SCNL visi ted Pakistan 2-3 tim es a year, but due to security constraints was unable to go to Quetta during the last 2 years, which was according to SC policies for expatriate visitors. SC worked with the IPs very closely at all times on their security which allowed the latter to work on the ground and accomplish almost all the objectives stated in the initial project proposal. The IPs themselves demonstrated very diplomatic protocols. They rented local vehicles for travel and contracted residents for construction in Mastung and Killa Abdullah and that was discussed w ith IPs fro m start to end. The security parameter also changed in the last 6-7 years, and once safe places in Q uetta became security risks for travel.

A serio us security incident did occur when a rocket attack took place at one o f the BEP (ECE) centers on April 25, 2 013." The attacked school was one of the polling stations for the elections that were to take place, and fortunately the damage was li mited and quickly handled by the BEP team. However, the incident highlighted the rather risky environment for both the children and the BEP team.

S::ive the Children staff capacity Staff t urnover in the organization did mar smooth operations to some e xtent. The SC Islamab ad staff underwent changes during the inception and design phases of BEP and later during implementation. SC Balochistan also witnessed a large staff turnover during the program period, including having the long standing BEP program manager leave for another assignment. On a larger scale, the global organization

KPIs with IPs, November 2014 SCSPEB mentioned that its programs Include maintaining a database on monitoring data and visit details. There

was p resumably rigor ous monitoring by e ach IP w hich was complimented by monitoring by the S C MEAL teams which provided feedback for any corrective measures required. " SCNL email to EKN, May 7, 2013; and SC, BEP, May 6, 2013, Report on Rocket Attack

53

underwent restructuring as the various Save the Children programs merge d into one entity. Staff and IP capacity was built up through different training sessions.'®

( a|i ;)city uf the Implementing Partners (IPs) All three implementing partners have been active in Balochistan for some time and are well regarded. Society for Empowering Human Resource (SEHER) has been established since 19 98 and is currently working in 16 Balochistan districts in the areas o f child protection, empowerment (women, legal and youth), education, health, poverty alleviation, advocacy and policy, and responses to emergencies. It has a well-developed management and programmatic framework and was selected for the BEP implementation based on its experience and knowledge of Balochistan socio-economic development, as were the other two IPs. According to SEHER management, the program adopted an efficient approach to utilizing the funds earmarked for capacity building and grants management and standard operating procedures (SOPs) were made part of the subsequent agreements with SC. There was also constant check and monitoring by SC te ams including on financial management, reconciliation of accounts, reviewing internal controls, and audit observations and their follow up.

Innovative Development Organization (IDG) has been similarly working in Balochistan since 1999 in the areas of governance, education, water and sanitation, gender and development, and information and communication technologies for development (ICTD). According to the KM with IDG, since the inception, the Program has helped increase the capacity of staff members of each department in IDG. BEP has also given opportunities to the key staff members to participate in capacity building trainings that has proven fruitful and has enhanced their capacity. According to IDG managemen t, since its inception, the BEP program has increased the capacity of staff members of each and every IDG d epartment. BEP has also given opportunities to the staff members to participate in capacity building trainings that has proved fruitful and has enhanced the capacity of key staff members. In the past IDG and its staff members did not had the experience for management of such a huge grant both in terms of time and cost, but this project has enhanced the capacity of key staff members for management of Grants."

Finally, Society for Community Strengthening and Promotion of Education, Balochistan (Society or SCSPEB) was established in 1992 and has well-regarded education expertise. According to SCSPEB management, the grants have been effectively and efficiently managed. The organization has its internal controls system and conducts external audits on a regular basis. Disbursements were closely monitored by SC and disbursements for each quarter were made after due diligence including verification of expenses and reconcil iation to accounts to the tune of 85% of the previous quarter. SC also conducted program audits through independent external auditors on an annual basis.

According to the SC Quetta office manager, the IPs capacity in construction could have been better since the work was very complicated. In the initial years of the Program, most of the focus was on

" For instance, see S C, BE P, May 10, 2010, Train ing Re port - Orientation and Tra ining on Child Parti cipation, delivered by the SCNL program manager. " Interview with IDG management, November 5, 2014, Quetta

54

I

construction which was done through IDO and Society. The IPs did not necessarily have the capacity for monitoring construction details, and hiring a constr uction firm would have been better, according to a discussion with the BEP project manager. Support for implementation of SEP was provided by the donor. Embassy of the Kingdom of the Netherlands, and Save the Children Netherlands and SC Pakistan which have been observed by the consultant to be coordinated in their efforts.

.Sequential and logical programming of inputs According to the IPs, the objectives of the program were designed to roll out activities in a logical manner. For example, in the first phase schools were constructed and e quipped, then teachers were hired, after which proper training was provided."" However, also according to implementing partner (IP) interviews, a few o utputs such as construction in the far flung areas of Mastung were impeded due to a deteriorating law and order situation and the slack season. The impediments were seemingly overcome through mutual coordination and assistance from the local community according to the IP interview. As compared to Mastung, implementation in Killa Abdullah was faced w ith fewer challenges. The issu e of appointment of teachers remained unresolved till the end of the Program and the lack of a ppointment and placement of teachers, particularly ECE teachers, seriously impeded effectiveness of the Program and its future effectiveness. An important cause of the delay was th e high turnover of Government especially at the senior (secretary) level which impeded decision making. Later, the election process and the delays in installation of a new government impeded teacher recruitment. It was a critica l phase for BEP; teachers were needed to make the classrooms functional, but the delays made this one of the few BEP failures. Although the relationship with government was excellent, the inertia in teacher hiring, due to factors beyond the control of both SC and government functionaries, made the entire process abysmal.

Meeting of minimum quality standards Minimum quality standards were a compone nt of the program activities. For instance, the manual for imparting training to PTSMCs discusses standards in several areas such as education and child protection."^ According to interviews with the IPs, management checks were in place for maintaining quality standards.

In terms of construction, a comprehensiv e Construction Strategy"^ was developed with assistance from SC Islamaba d as physical infrastructure was a major component of the Program."^ The strategy was developed with the participation of all the implementing partners and SC Islamabad office held working sessions with partners for orientation on policies and procurement procedures according to one of the SC biannual reports. The Construction Strategy delineated uniform standards and guidelines for all partners to adhere to in construction activities. A special checklist was develo ped by SEHER to address

Interviews with IPs, November 2014 SEHER, June 2013, Training Manual for Parent Teacher School Management Committee (PTSMCs) SC, BEP, Construction Strategy 2010-2015 SC, BEP Bi-Annual Report; December 2009 to June 2010, p. 5

55

the aspect of child care and protection in this- hard component and find ways of ensuring child participation, in school design and construction. The standard checks used by government in construction activities were in place according to the SC Quetta manager, and construction was done by the implementing partners using local contractors.

Details of water availability or hon-availability were prepared by the BEP team for further action by the Quetta office and the IPs for schools in all th ree districts, although the information is for 2014.'^ Quality assurance tests for construction were also under taken in 2014 and a det ailed report prepared on the type of school, its name and location, the date of the visit, and the quality assurance test performed.'^ A visit report to monitor new facilities was undertaken to look at details on water and sanitation work."^

Risk analysis and overcoming ofchallenges Risk analysis was conducted at different stages during program implementation and the frequent program review meetings generally considered risk mitigation. During project implementation, risk mitigation rhatrices were prepared to help analyze risks and the consequences of these, with mitigating strategies defined in the matrices, which are a part of the SEP annual reports. During one such meeting earlier on in the Program, the key risks identified were security risks in Balochistan, contractor issues, staff capacity building and delays in governance processes, although the mitigating strategies were not discussed."' Risk analysis was also fully incorporated in the log frame analyses.-

In terms of specific micro-elernents of the Program, the construction strategy stated that there should not be any isolated, non-vigilant corner or area in the school premises, which could become a risk or threat for the children. There should also be a clean and safe surrounding environment."^ Another example was the risk assessment of children due to ill health and lack of proper nutrition, and its consequences, which was incorporated into a new component of BEP, school health and nutrition (SHN).=°

Since Balochistan is a disaster prone pro vince it was deemed necessary to prepare the communities for facing disasters, which also had secondary helpful effects in terms of the development of the community, especially children. In order to spread awareness, and to involve the education officers and stakeholders in minimizing the effects of disasters in focused areas, Disiaster Risk Reduction (DRR) training was firs t administered to partner staff in order to disseminate it to the students during their

"" SC, BEP, April 2014, Water Supply Detail (excel sheet) "' sC, BEP, August 2014, Work plan for NHA

BEP, SCSPEB, August,26, 2014, Monitoring Reports of WATSAN and Missing Facilities Works "' SC, BEP Annual Report (January 2011 - December 2011), p. 22

SC, BEP, November 2013, BEP Log frame planning and reporting SC, BEP, Construction Strategy, p. 10 SC, June .2012, SHN Baseline Survey. Also see SC , BE P, 2014, Nu tritional Policy Guidance Note for Balochistan,

which assesses risks of malnutrition

56

visits to the schools.^^ For the children, DRR awareness-raising and tr aining was a basic subcomponent of the Program w ith school based DR R ac tivities." Of the five child club members interviewed, three said they had received training in DRR, and one of those felt more intensive training was needed. The training was on fire risks, and how to protect oneself and others during earthquakes and floods. According to one of the IP managers "training to students and teachers on concepts of DRR, development of DRR plans and provision of DRR kits t o schools have c ontributed in making the school conditions safer, and all 340 schools are better prepared in their response to disasters".

According to the survey, DDR equip ment was received in 76 percent of the surveyed schools, and the equipment received was fire alarms (34 percent), first aid kits (27 percent), megaphones (12 percent), and some other items (7 percent). Sports and ECE kits were also provided, and all kits were observed to being used by teachers most of the time (33 percent) or sometimes (55 percent) with 93 p ercent of teachers finding the training effective. In addition, 53 percent of PTSMCs had developed DRR plans and 95 percent of these found the training effective.

In terms of partnerships, according to the IPs, keeping in close consultation with all relevant stakeholders (government officials, teachers and community members) minimized the risks involved in project implementation." According to IDO management, the major risk involved throughout the project was the very poor law and order situation, especially in district Mastung, where IDO was the only organization working. The risk was managed by working in close coordination with local communities since social mobilization was a key component of the project in minimizing the risks.

According to the IP SEHER management, the deteriorating law and order situation restricted movements of field staff considerably. The partner field staff used alternate means such as three wheel vehicles to access schools and communities instead of cars. Moreover, alternate venues for teachers and school teams were organized to pass on the benefit and by managing these risks. Overall assessment of risk for all projects was done on a monthly basis and is discussed further in the section on financial efficiency.

It is w orthwhile to give an example here which demonstrates the role played by the PTSMCs in risk assessment and mitigation. For instance, the teachers at Government Girls Middle School (GGMS) Shareef Masseza i (Killa Abdullah) were known for arriving late to school leaving the students to wait outside the school u ntil the school gate was unlocked, which created a serious protection risk. The PTSMC members held a meetin g with the teachers to convince them to avoid arriving late to school and they also voluntar ily assigned duties at school opening and closing times to prevent students from being left unattended." The teachers subsequently started arriving at school on time.

" SC, BEP Bi-Annual Report, Jan-June 2012, p. 20 " SC, BEP Narrative Report as sent to EKN, 21-03-2011, p. 18 " Key informant interviews (Klls) interviews with IP managers, November 2014 " SC, BEP Bi-Annual Report, Jan-June 2012, p. 15

57

Comnuiniiy mobilization and education delivery In the model of involving community in education delivery, there are some aspects where such involvement is feasible, such as involving parents in awareness raising and enrolment drives. However, it is not feasible from the point of view of provision of funds by the community, even though some funds have been collected by communities according to Program documents. The Satellite Town, Quetta PTSMC members offered that "we will try our best to contribute ourselves and also encourage other community members to chip in" when asked if the community will continue contributing after the project ends. However, the majority of focus group discussions with BEP and non-BE P PTSMCs made i t very clear that the paucity of funds ranks as an important element in community non-involvement."

Delivery and timeliness of the teacher training programs In terms of the ECE class, significant reductions in school failure, repetition, and absenteeism and dropout rates were found in the vast majority of studies looking at the effects of ECE globa lly. In Pakistan, the National Education Policy (NEP) 2009 states that:

"ECE shall be based on a concept of holistic development of the child; age group shall be recognized as comprising 3 to 5 years, at least one year pre-p rimary education shall be provided by the State; EC E sh all be attached to primary schools provided with additional budget; teachers and assistants for this purpose; and, for ECE teachers, a two-year specialized training shall be a necessary requirement based on ECE revised National Curriculum."

In order to introduce ECE, a natio nal curriculum for ECE wa s developed in the year 2002 and revised in 2007 at the federal level. The Higher Education Commission de veloped a curriculum for elementary school teachers which also catered to the professional needs of ECE teachers. The now d efunct Federal Ministry of Education also developed the Early Learning and Development standards. Currently, in Balochistan E CE is the katchi class, wh ich has a limited concept of ECE, ECE mat erial and usually no classroom and teachers are assigned for this age group. Given the recognized importance of early childhood, BEP too k the initiative to draft an EC E strategy which has compl ied with government ECE policies and was shared with government." The education department has also developed an ECE action plan, according to the strategy document referred to here, to meet the learning needs of this age group^' although the documentation does not say whether the plan was developed under the influence of BEP or includes any BEP role.

Considerable focus was given to the training of teachers in BEP starting ECE, focusing on participatory teaching methods and later on disaster risk and reduction training." Major efforts were made for

" For instance, see PTSMC summary for Killa Abdullah, November 2014 " Save the Children and Children's Global Network Pakistan, 2012, Early Childhood Education (ECE) Strategy for Balochistan " ECE Strategy, Ibid, p. 31 " See Mehnaz Aziz, August 2012, Early Childhood Education (ECE) - Review and Analysis of the Balochistan Education Program for insights in the delivery and quality of teaching for ECE. A notable ECE teach er training program was developed: SC Education, Program Development and Quality Unit, July 2013, Early Childhood Education Training: Five Days Training on Early Childhood Education Teaching Methodologies, Islamabad

58

collaboration with the Provincial Institute for Teacher Education (PITE) in the training of teachers and a strategy developed for that collaboration, with the result that the responsibility of teacher training under BEP was apporti oned to PITE." The rev ival of PITE as a major teacher training entity was rightly considered an achievement for BEP in the mid-term evaluation of BEP.®° As stated in the mid-term evaluation:

"Although the impact is y et to be seen since the training cycle c ompleted only in December 2012, introduction of different types of training initiatives including early childhood education (ECE), health and hygiene (H&H), child focused health education (CFHE), content enrichment in mathematics, science and English on topics of new textbooks can be seen as an important contribution. While engaging PITE for most of its trainings not only increases the acceptance of government officials, it has also the potential of carving a sustainable model for developing and sustaining teaching capital."®^

For purposes of awareness-raising and observation of good training practices, an exposure visit was arranged for Government of Balochistan (GoB) education officials to visit Punjab education facilities, including cluster training and support centers in the districts, which was a key l earning from this visit. In addition, teachers of the BEP focused schools were provided training on different areas such as teaching methodologies to newly hired teachers, training on early childhood education to ECE, grade 1 and grade 2 teachers, and training on child focused teaching to at least 3 teachers in each of the focused schools. After every training program, the master trainers of IPs conducted monitoring visits to measure the effectiveness of the trainings provided, and provided support in case of need. The IPs also organized workshops o f trained teachers facilitated by master trainers to address any issues the teacher faced after training programs, according to interviews with the implementing partners.

The teachers training component was acknowledged by PITE and its faculty and the Department of Education officials acknowledged the timelines, effectiveness and comprehensiveness in content coverage and particularly the training methodology, according to the both PITE officials and IP managers. In addition, cluster based teacher training and mentoring was introduced with the support of EKN." Most importantly, teachers training in ECE teaching concepts and provision of an ECE environment helped with the conversion of katchi to ECE class, a ground breaking endeavor. The effectiveness o f the Program achievements in ECE culm inated most positively: under the BESP, it is planned that 720 ECE centers will be set up each year of the 5 year plan, for a total of 3,600 ECE centers (currently only 5% of schools have ECE facilities).®"

Save the Children and Huma Khan, 2011, Framework for Action: Provincial Institute for Teacher Training, Balochistan

Amima Sayeed, February 2013, BEP Mid Term Evaluation Report, p. 10 Midterm evaluation, p. 11 SC, BEP, April 2012, Report: Exposure Visit of Government Officials to Education Reforms in Punjab

" SOCIETY, November 2014, BEP End Event Ceremony Government of Balochistan, Policy Planning & Implementation Unit (PPIU), September 16, 2014, Policy and Right

to Education (RTE): Status on Early Childhood Education in Balochistan, Presentation

59

11 ;;nsp.ii eiicy and coordination by the IPs For the implementation of BEP, at the programmatic level, Save the Children developed partners' selection criterion and partners were selected accordingly to ensure transparency.^^ According to IP managers, s tatistical year books of focused districts, newsletters, meetings with stakeholders, formal sharing of information of every activity with education department officials, and seminars throughout the project life were an effective and efficient way in managing transparency. The information sharing continued throughout the program implementation, with all implementation partners, communities and PTSMCs. Information was also shared w ith the Government departments at the district level and at the provincial levels, including with PITE. In order to maintain the transparency of training database, only selected personnel such as the director of PITE had the responsibility to authorize changes.®® Iss ues of transparency were also discussed during the government officials' awareness-raising visit to Punjab on how the Punjab reform program manages transparency.

Transparency at the PTSMC leve l was adhered to by ensuring that the signatories of accounts are all present when bank withdrawals are made.®^ The acti vity of opening of the bank accounts was geared towards transparency and accountability of the PTSMCs in implementing the school expansion plans and utilization of funds accordingly. The joint bank account pro cedures were followed according to the bank requirements.®® One research conducted by the IPs in conjunction with MEAL found that knowledge of banking procedures, including the funds transfer process and use of the funds for missing facilities, was ma inly limited to the president of the PTSMC.®® Als o, few members were involved in the school expansion plan (SEP) or were aware of it being used after it had been formulated. However, the BEP survey found that the PTSMC members were clearer about the SEPs and many agreed that they "all worked with consensus due to understanding, worked out the needs, how to prioritize and learned how to work together". Moreover, "with training work became easy and when we worked methodically, the problems got resolved" according to the PTSMC interviews. With regard to bank accounts, about half of those interviewed were aware that there was a bank account, but most were also quite clear that continuing any fun ding after BEP was impossible. As stated by one group and echoed by o thers, "it is almost impossible f or PTSMCs to continue effectively as contribution from community is a far cry; the reason for not contributing is poverty".

At the ground level, school profiles were prepared in which details of all interventions from the start of project were stated with their total cost and these profiles were shared w ith relevant schools and the education department. In schools, visible boards were displayed presenting information on schools interventions and total budgets. In construction, the delay in the envisaged activities were due to the lengthy tender process in o rder to ensure transparency in the scheduled rates owing to which several contractors withdrew from the tenders and the prolonged government process of notification. However,

SC, RNE Education Proposal 2010, p.l9 ®® SC and Huma Khan, PITE report, Huma Khan, p. 8 " SC, B EP Bl-Annual Report, Jan-Jun 2012, p.13

SC, B EP Narrative Report as sent to EKN, p. 7 SC, MEAL, December 2013, Operational Research on Effectiveness of PTSMCs In the BEP Project

60

after receiving the approval from government, the construction activities were expedited.™ One construction monitoring report for 2014 was shared with the evaluation team leader and has been discussed elsewhere in this report. The community and PTSMC members were involved in construction activities including monitoring. As stated by one community group, "the representatives of IDG and SEHER would visit fr equently, would hold meetings, and supervise work in progress, particularly with the PTSMCs. They would consult PTSMCs at every step of construction activities and we were in touch with the PTSMCs on a regular basis".

Partnerships with stakeholders Constructive partnerships have been established with different offices of the Government of Balochistan, as well as the people of the province under the aegis of the Program, which is evident in the positive response from all levels regarding the impact of the Program. As specified in the key informant interviews for this evaluation, the Provincial Institute of Teacher Education considered BEP interventions for professional development better than those of other development partners in terms of the quality of modules, comprehensiveness, adequacy of content and focus on objectives. The training curriculum was considered in line with the 2005 curriculum, thus meeting the expectations of Government counterparts. The ECE training has now become part of the Government's program being considered for accreditation and affiliation by the University of Balochistan. PITE re presentatives agreed that areas such as child protection, health and hygiene, and ECE should be part of all future programs and projects.

BEPs positive contributions were substantiated by all partners including community members, PTSMC members, provincial and district education managers, implementing partners, and GoB teacher education development officials, and key impact has been in the areas o f infrastructure, ECE, Child clubs, teacher trainings and school hygiene and nutrition. According to partners key milestones are increasing enrolments and improving the quality of education services. The community members felt that their children were receiving education in a favorable environment and they themselves felt they are giving more attention to education with girls' education given an especially important focus, according to the Klls with community members.

Conclusions on effectiveness:

Overall, effectiveness of the Program is deemed quite satisfactory. The baseline survey was undertaken to inform project design but was slow in being realized. It was also not designed for evaluation against the initial status of the area. The quality of monitoring is not fully adequate since the l ogframe has not been presented cumulative for each year; that is achievements against activities in the logframe are not available in one document but in biannual and annual reports only for the time under review. There was some effect on adequate monitoring since the SCNL program manager was often not able to visit Quetta in the latter half of the program due to security concerns. The capacity of the implementing partners was fairly strong for the most part, and they were further trained by SC in program management and financial management. A weakness was th at implementing and supervising construction activities was not their field of expertise. However, perhaps given the restricted capacity in the province it was not

SC, B EP Annual Report Jul-Dec 2011, p. 8

61

possible to commission an implementing partner with more solid expertise in supervision of construction. Once government was more fully involved, the construction aspect moved forward rigorously, which was a lesson le arned for the Program. Programming of inputs was sequential and logical except that the construction effort did take up too much time in the first 3 years of the Program. Minimum quality standards were met in terms of monitoring of construction activities. Risk analysis and meeting of subsequent challenges has been fairly strong and a fair amount of training has been administered for disaster risk reduction (DRR) to child clubs and community members, as well as related kits provided. Communities have been strongly mobilized in education delivery. However, they remain dependent on external financing, government or donors, to be able to play an active role in the PTSMCs. Hence, it is a lesson learned for future implementation that expectations regarding the financial contributions of community members be based on what is realistically possible for poor communities to contribute. The implementing partners have been effective in transparency and information management. Training programs for teachers, PTSMCs and community members were delivered in a timely and effective manner. The partnerships with government partners such as P ITE have been strong with the latter incorporating many of the training techniques into their own programs, especially those related to ECE.

VIL . Efficiency

The Program's coping mechanisms The BEP management has encountered many changes at the education secretary level through the life of the Program. This has mea nt briefing incoming secretaries regarding the Program objectives and achievements and bringing them on board as often as needed. In addi tion, there has been an increased demand on provincial education capacity since the induction of the IS'*" Amendment to the Constitution of Pakistan. The Amendm ent designates all education obligations, including the right to education, the language of instruction, curriculum development, and policy and planning as a provincial responsibility.

In Balochistan, as a consequence of the implementation of 18th Constitutional Amendment, the education department was bifurcated into two ministries; secondary and h igher education. To respond to the changes, BEP management organized an orientation session for the new Minister and Secretary of Secondary Education. In addition, with the collaboration of Education Department a dialogue on the implications of 18th Amendment and its implications on educational governance in Balochistan was organized in which expert resource persons, government officials including ministers shared their views and analysis fo r responding to the changed s ituation.The dialogue was attended by more than 60 participants from the Education Department, international non-government organizations (INGOs), civil society and r epresentatives from Pakistan Education Ta sk Force Islamabad. Participants and resource persons shared analytical perspectives on the amendment and challenges for educational governance in Balochistan.

" BEP Report July-Dec 2010, p. 6,12

62

The 18"" Amendment not only made Education a provincial subject as part of increasing provincial autonomy, but also introduced Article 25-A, Right to Education (RTE) which states that: "The State shall provide free and compulsory education to all children of the age o f five to sixteen years in such a manner as may be determined by law." According to the BEP mid-term evaluation, the Program became the first model across Pakistan that addressed the basic parameters and challenges of Right to Education provision by addressing the following."

• Secondary education options and ways of expanding primary schools to cater to students up to sixteen years of age

• Establishment of ECE centers to respond to the learning needs of 5 year olds who otherwise fall out of the ambit of primary education

According to the IP managers, one of the strengths of the project was its close coordination with Government. Every BEP stee ring committee was chaired by the Secretary o f Education or by his/her designees. Other members of the steering committee included assistant commissioners of the three focus districts, the Director of Education, heads of organizations and a represen tative of Embassy of the Kingdom of Nether lands. Whenever insecurity and uncertainty of a high level was foreseen, the project steering committee was very supportive. Throughout the implementation of BEP, t he effective and efficient coordination with government was one of the driving forces in helping achieve program objectives, according to the IP managers.

Regarding delays in approvals for school construction, these were affected by Government procedures, notifications, processes of schools, t ransfer of Government officials (Secretary of Education, Deputy Secretary, and Directors) and changes in the Education Department bureaucracy after 18th Amendment devolution, which the Program managers had to incorporate in the Program." With respect to building the capacity of the teachers, the Provincial Institute of Teacher Education (PITE) has had to play a central role after the devolution of education powers. BEP sponsored the development of a teacher edu cation plan based on the key areas identified in the National Education Policy a nd the concrete steps PITE needed to take in order to implement the plan.'"

i of i n.stniction In terms of language, Balochistan is po pulated by a triba l society of Nomadic and semi-Nomadic tribes. The most important are the Brahvis, Balochis and Pathans who speak their respective languages, w ith the common language being Balochi. The languages used in the focus districts are discussed in the introductory chapter. The BE P sponsored EC E strategy develo pment team held meetings with parents, district officials, teachers, parliamentarians and the Secretary of Education to discuss language issues." The stakeholders were of the strong view that Balochi should be the language of instruction for the ECE BEP. They saw the Program as an opportunity to establish the local language in the early years, given the

" Mid-term evaluation, p. 9 " BEP Annual Report 2011, pp. 15 and 17

SC and Huma Khan, PITE Report, p. 4 " ECE Strategy, p. 29

63

importance placed on teaching in the local language in educational research. The Education Secretary also shared the priorities of the education department with the ECE strategy team, and said that after 18*^ amen dment the province has t he independence to take more appropriate decisions for quality education in the province.^® He added that in the curriculum the language should be given special focus so that children obtain a command of the language along with math and science concepts. He also strongly emphasized on the professional development of teachers, which has been a key B EP area.

in terms of insecurity in the lives of children, BEP has addressed this through its emphasis on child protection. Child p rotection work aims t o strengthen the capacity of all the stakeholders to develop systems and mechanisms that provide meaningful protection for all children in the longer term. The work seeks to address the root causes of child protection failures such as chronic poverty, insecurity, power imbalances, and harmful traditional attitudes."

Financial management and performance of BEP

Pi'ociii'ement Save the Children has a well-defined Procurement Policy whi ch delineates all procurement rules and regulations^® and another manual for the procurement process itself to ensure understanding of all staff involved in procurement activities.^® Furth er, training to prevent procurement fraud is ou tlined in the Fraud Policy which states that "staff in positions of particular fiduciary responsibilities (e.g., legal, finance, procurement, internal audit) shall take training associated with detecting, preventing, reporting, investigating, and addressing fraud relating to these particular areas, as requested."®" Further discussion on fiduciary aspects is further detailed in a later section of this chapter. According to SC documentation, partners need special technical support in the large scale procurement of school supplies, i nvolving advertising of tenders.®^ To as sist wi th partner performance, SC Islamabad office held working sessions with partners for orientation on policies and procurement procedures.®^ In terms of procurement costs, whether in bulk or otherwise, there have been cost increases due to the security situation, as well as the normal high costs associated with the distances in Balochistan, according to the BEP manager in Quetta.

According to a tracking document shared by SC, procurement in 2014 has been quite timely for procurement in the areas of books and educational materials, construction materials, IT e quipment, stationery, visibility materials and miscellaneous.®® The deta ils are provided in Annex 8. Out of the 73 procurements undertaken since June 2014, the date that goods were required is mentioned in 49 percent of the cases. Of these, actual delivery took place within 1-3 days of the required date, except in

" BEP, ECE Analysis, 2012, p. 6 BEP and SEHER, 2011, Training Manual for Child Club (CC) Life Skills, p. 20 SC, Oct 1, 2011, Sav e the Children Employee Manual for Expatriate and Third Country Employees, Chapter XXXV

- Procurement Policy " SC U K Pakistan Program, April 1, 2005, The Procu rement Process; and SC , Oct 2014, Procurement Policy, Save the Children Pakistan Program

SC, Office of General Counsel, 2013, Zero Tolerance Fraud Policy, p.2 BEP Annual Report 2012, p.38 RNE BEP Narrative Report, Dec 2009-June 2010, p.6 SC, SC-PR-04, Procurement Tracker QPO (From 17 June 2014)

64

4 cases where the difference between required date and delivery date was between 14 to 52 days. The 2014 data indicates a well-planned and well-implemented procurement process. The IPs did not comment on any anomalies regarding procurement or problems with the quality of goods purchased, and one IP manager mentioned that goods were purchased per the policies and prescribed SOPs.

recniilinenl. Teacher recruitment was an imp ortant part of the Program. The Government of Balochistan (GoB) and Save the Children agreed in the initial Memorandum of Understanding (MOU) that approximately 330 teachers wou ld be notified by the GoB, of which 105 will be for ECE, 175 for the 25 upgraded middle schools, 50 for the 10 upgraded secondary schools (4 teachers plus 1 head teacher respectively), with support staff, and regularization of the salaries w ithin two and a h alf years.®^ Ho wever, the teachers were to have included PTI, SSTs and drawing teachers, mostly above grade 16, which the Department of Education is not authorized to recruit at its level, and which can only be done through the S&JD or the provincial Public Service Commission (PS C). Recruitment is always fairly politicized in any context, thus leading to issues in implementation of this aspect of the MOU.

The effo rts for teacher recruitment, a critical component in quality education delivery, were ongoing almost from the start of the Program. According to the first GoB led Program Steering Committee held in 2010, it was agreed that approval for hiring will be sought from S&JD and Statements of New Expenditure (SNEs) and a Plan ning Commission Proforma 1 (PCI) will be prepared for the recruitment of all teachers.®^ It was also agreed that hiring in Mastung and Killa Abdullah d istricts would be carried out at village level, and at the Union Council/Ward level for Quetta, and the recruitment would be institution based and non-transferable. It was also agreed i n the second Steering Committee meeting that the project would pay for salaries for two years, while the Education Department finalized the SNEs of the positions to incorporate them in the regular government system after two years.^^

In 2012, the process for the recruitment of teachers for the upgraded schools was in itiated at the start of the academic session.®^ The Education Department placed advertisements in newspapers and applications were collected and lists prepared. However, the employment orders of the successful candidates vyere no t issued at that time. In the meantime, the PTSMCs and teachers of the upgraded schools managed to start middle and secondary school classes w ith the existing staff strength at the start of the academic sessio ns so as to encourage girls' enrolment especially for those girls who were transiting from primary to middle and middle to high school. In schools where there were surplus teachers one of the existing teachers was notified by the Department as an ECE teacher and was

Save the Children and the Government of Balochistan, July 15, 2010, Memorandum of Understanding for the Balochistan Education Program, p.2

BEP, Minutes of the first Project Steering Committee (BEP) held on 23-12-2010. The PC-1 is t he traditional method for requesting funds from the Finance D epartment, and the SNE is required for approval of any new position and its inclusion in the Government payroll.

Balochistan Education Programme (BEP), Meeting Minutes, 2"'^ Project Steering Committee, January 28, 2012 SC, Biannual Report Jan-Jun 2012, p. 9

65

assigned to teach the ECE c lass. In schools where there are no teachers available for the ECE cl ass, the recruitment process for new teachers was initiated.

Lengthy and bureaucratic government procedures for teacher recruitment caused delays in bringing the teachers on board. Frequent strikes in the Directorate of Education also resulted in delays in notifications and other activities which needed approval from the Directorate. For a period of time, an interim government was in place in the Province, and as confirmed by the BEP pro ject manager, the interim government had no power to recruit teachers. In addition, the hiring process was not understood by SC and IP staff. Although the government provided assurance on recruitment through the signed MOU, it was not realistic. Moreover, no teachers could be recruited from the private sector since the GoB would not have regularized those recruited through other processes.

There were some succ esses as various coordination meetings of IP staff with the education department continued to help expedite the induction of teachers for the newly constructed facilities of ECE and up­graded schools.^® In the year 2012, the education department notified 27 teachers and recruited 28 new teachers for ECE classes. On the whole 62 teachers were deputed to the ECE centers until 2013. However, as shared by one of the district education managers, "recruitment was conducted under tremendous pressure".

It was agreed in May 2014 at a meet ing of the IPs with the DoE that:®®

i. Grade 16 and above teaching staff including 28 secondary school teachers (S STs) (G), 24 SSI (Science), 24 physical tr aining instructors (PTIs) and 25 drawing masters in focused schools under Balochistan Education Program (BEP) would not be hired under BEP and the Education Department would fill these positions through posting of already hired (and trained) teachers on the payroll of Education Department.

ii. It was also agreed that the unspent savings out of teacher's salaries and associated trainings allocated for the year 2014 would be utilized by Save th e Children for the provision and improvement of water and sanitation facilities in the BEP focused schools.

In 2014, the list of remaining 43 teachers was endorsed by the education department and the positions were advertised with the hiring presumably in progress. Table 14 provides some c larity to the latest situation regarding ECE teachers,®" although according to the IP managers interviewed, the positions for the teachers have been created recently, but still to be filled. Moreover, proposals are under consideration for absorption of teachers recruited on contract basis under the Aghaz-i-Haqooq-e Balochistan package, an additional development program sponsored by the federal government.

Table 14: Progress in Recruitment of ECE Teachers ECE Teachers' Recruitment 2012 ECE Teachers' ECE Teachers'

Recruitment 2013 Recruitment 2013

BEP Annual Report 2013 ®® Hameedullah Khan (SC) email to SCNL and SCPK dated May 15, 2014. 90 BEP Biannual Report Jan-Jun 2014, p. 13

66

ECE Posts advertized

ECE Teachers Recruited

ECE Teachers Notified

Total Recruited and Notified

ECE Teachers Notified

ECE Teachers Hired

Teacher Hiring in Progress

39 22 24 62 16 0 43 Source: BEP Biannual Report Jan-Jun 2014, p. 13

Imdgets and expenditures Financial managernent and performance under the Program has been quite positive, other than the anomalous issue of teachers' recruitment discussed earlier. Staff costs as a percentage of the total project budget have been well within satisfactory ranges, except fo r 2013 and 2014, when these were somewhat higher, as presented in table 15.®^ Pe r the requirement of the Program, a few positions were added in the 2013 budget after consultation with the donor. Further, the increase in the last two years also reflects a few position upgrades both at the country and field offices.

Overall program budget utilization has b een at 78%, with the partners expending 79% (IDG), 78% (SCSPEB) and 99% (SEHER), respectively, over the life of program. The reasons for less than full utilization are the following:

i. Major costs per taining to close ou t events, audits and en d-line evaluation will be re corded in November 2014, whereas the figures in the table are till mid-October 2014.

ii. For SEHER, spending is 99% as they have completed all their programmatic deliverables and only the salary expenses of November 2014 are pending®^.

iii. IDO and S CSPEB have spent around 80% as some program deliverables are pending, which will be completed in the November 2014.

iv. The final figures will be issued once there is closure for the month of November 2014 in the financial system, which is expected by the third week of December 2014, both for SC and the partners.

Table 15. BEP Budget and expenditures (€) Yearl Year 2 Years Year 4 Year 5 Total

2009-10 2011 2012 2013 2014 2015

Total budget for BEP 1,761,821 2,652,289 3,160,070 2,175,306 2,346,539 12,096,025

Budget Staff Salaries 168,560 197,243 353,589 361,395 338,998 1,419,785

Expenditure on Staff Salaries

143,319 153,621 284,937 325,542 301,316 1,208,735

Salary expenditure /Total Budget

8% 6% 9% 15% 13% 10%

Budget of Partner IDO 535,727 1,449,770 1,299,720 561,515 730,227 4,576,959

Expenditure of Partner IDO

196,107 953,325 1,458,670 371,807 653,786 3,633,695

% Spend 37% 66% 112% 66% 90% 79%

Staff costs at or below 10% of the total budget are usually considered satisfactory. Pending at the time the evaluation fact finding was concluded.

67

Budget of Partner SCSPEB

654,625 730,935 969,630 678,494 553,202 3,586,886

Expenditure of Partner SCSPEB

99,069 547,027 1,013,684 602,399 525,322 2,787,500

% Spend 15% 75% 105% 89% 95% 78%

Budget of Partner SEHER 64,168 125,508 138,850 194,886 191,248 714,660

Expenditure of Partner SEHER

51,918 167,215 153,929 160,884 174,570 708,515

% Spend 81% 133% 111% 83% 91% 99%

Direct Budget SC Program for BEP (other than through partners)

374,828 620,003 708,896 722,362 704,322 3,130,411

Direct Expenditure SC Program for BEP

280,551 285,239 594,803 531,937 515,617 2,208,147

% Spend of direct budget 75% 46% 84% 74% 73% 71%

% spend Total expenditures/Total budget

39% 67% 103% 77% 86% 78%

Source: Save the Children, Pakistan, Finance Department, Novem oer 2014

The e xpenditure rates of SC progra m spending are quite good at 78 percent currently, expected to increase to ICQ percen t by closing of the Program. The partner expenditures have been good at 79 percent (IDO), 78 percent (SCSPEB) and 99 percent (SEHER), also expected to increase to about 100 percent at closing. Staff and program costs are belo w 10 percent but about 13 percent in the last two years due to increased staff strength and increased after consultation with the donor.

F inancial risk and monitoring Financial risk within Save the Children is managed by various departments at different stages per transaction and on an overall basis for each grant. Financial risk management is managed by the finance function, which is composed of two segregated departments; Finance Department and Grants and Compliance Department.^^

The finance role fun ctions with an optimum level of segregation of duties and reports independently to the Country Director. The structure ensures the maximum segregation of duties within each department since the staff required to approve the expenditure is separate from the staff required to process the transactions after they have occurred. Additionally there are separate units for donor compliance and documentary compliance. Separate structures are responsible for accounting of transactions and management of funds for banks and cash, providing a high degree of transparency and risk controls.

There is an accounting system in place and to ensure accuracy, each financial report is reviewed by the country accounting department, the regional accounting team, and Save the Children Netherlands. Each donor report is revi ewed by the Grants Department to ensure that expenses are complete, relevant.

According to the Fina nce Office representative at the Coun try Office, the pro cesses di scussed in this section were followed for the BEP program, and no further information was provided by the Quetta finance office.

68

compliant, and within budget. Through mandatory practice, completed reports are circulated to the management of each grant (also known as the budget holder) to help match the financial report with the programmatic progress report to detect and correct any anomalies.

Save the Children hold regular coordination meetings on the performance of grants, with grants specified as low, medium or high risk, based on financial performance and expenditures.^" The details provided and the staff meetings provide additional transparency to the process of discussion of issues and problem solving thereof.

Interna! and e.vternal controls These include internal and external audits, a high sensitivity and commitment to fraud prevention by senior management and senior management periodic reviews of reports. The in house Internal audit and fraud preventions systems also create sufficient amount of deterrence to prevent fraud. The independently placed internal audit department bears the responsibility of tracking incurred expenses to the field level. To ensure optimum and reliable management of funds. Save th e Children Pakistan deals with a well-respecte d bank. The banking activity is monitored by the country accounting office and the regional accounting office.

SCs detailed in ternal controls are documented in the internationally approved framework for finance^^ and it has been ensured to a great extent that the framework is clearly understood by country office staff. The fraud prevention framework is similarly t ransparent, and is equipped with diverse preventive and d etection controls^®. The overall external audit conducted for the year 2013 concl uded that BEPs financial statement for the year ended December 31, 2013 gave a true and fair view of the completeness and accuracy of all transactions and receipts, and was considered accurate and complete in every respect. It also concluded that the obligations imposed by the grant decision had been met.®^

Efficiencies in government budget utilization The percentage utiliza tion of the budget has been quite high as presented in diagram 19. In some cases utilization exceeds allocations, probably due to re-appropriations during the budgetary year for provincial salary increases.

For instance see Save th e Children Pakistan, Grants Financial Performance Overview, Program Coordination Meeting, October 14-15, 2014; and Save the Children Pakistan, 14*^ Program Co ordination Meeting, October 15, 2014

Save the Children, Finance Manual Save the Children, Office of General Counsel, October 2013, Zero Tolerance Fraud Policy BDO Ibrahim and Company Chartered Accountants, BEP Audit for the year ended December 31, 2013

69

Diagram 19. Percentage utilization of the education budget 2010-11 to 2012-13

u Percentage utilization 2010-11

; 2011-12

u 2012-13

Source: Derived from l-SAPs, 2013, Public Financing of Education, Balochistan chapter

In terms of advocacy with government on planning and budgeting, the comments were not very positive. The A dditional Secretary for Education stated that the budget preparation cycle st arts from February onwards for the new fiscal year and he was not aware o f any advocacy campaigns or technical inputs in this regard to Education Department, Finance Department or Planning and Development Departments by Save the Children. There were no discussions held during the budget (current and development budgets) preparation stage with SC or IPs . However, the Additional Secretary mentioned that the budget is on the rise and there are other projects on girls' education as well including Promoting Girls Education in Balochistan (PGEB).®®

Conclusions on efficiency:

Program efficiency is deemed satisfactory especially given the Balochistan situation in terms of security and the costs associated with traveling large distances and providing services thereof. The procurement policy is w ell defined as are other financial systems. The Program's coping mechanism in the face of insecurity has been r elatively strong. The program has also responded well to changes in the political environment thus providing complementary support for the greater responsibility given to the province due to Article 25-A of the IB**" Amendment t o the Constitution. Budgetary expenditures for the Program are quite efficient, as are financial controls and financial monitoring, at least from the head office perspective. The fi eld office was not able to provide information although it is assumed to be positive since their information feeds into the head office. The procurement policy is well defined and procurements are timely per the information provided for 2014. Information on the quality of the

98 Financed through a World Bank established and administered Multi-Donor Trust Fund (MDTF) set up to respond to the needs resulting from the crisis in the northwestern border areas of Balochistan, Khyber Pakhtunkhwa (KPK) and the Federally Administered Tribal Areas (FATA)

70

procured items was not provided, although no complaints were raised by PTSMCs or community members either. The financial expenditures for the Government's education budget are also fairly efficient although advocacy by the BEP team was an area that was made redundant midway through the program given that other donors were working on this intervention. One aspect that could have provided further information was a li nking up of the baseline survey with a cost-benefit analysis t o better determine value for money for the various activities of the Program.

VOL Impact

i j|l)portiinitit's for girl.s beyond piüniary schooling It is worth reiterating that the goal of the Program was "bringing positive changes in the lives of vulnerable and marginalized children, mainly girls, through the provision of quality education in three districts of Balochistan in particular, and beyond in general, on a sustainable basis by 2014".®® The Balochistan Education Program has increased access for girls to higher levels of education at 34 locations across the targeted districts. In these three districts, 25 primary schools have been upgraded to middle schools whereas 9 middle schools have been upgraded to high schools. As discussed in detail elsewhere in this report, the process for hiring new teachers in the upgraded schools is presumably under progress and w ill be programmed by GoB against the Balochistan Package financing of the federal government, according to the BEP team.^°°

In terms of training 180 girls of up to grade 6 received trainings to develop an understanding on basic operations of computers and the use of software which is v ital for learning and development in the modern era. Moreover, sessions regarding students' health in general were conducted with 110 PTSMCs and 86 sessions with mothers and girls were conducted by teachers regarding the specific health related issues of adolescent girls. A majority of 79 percent of teachers said that BEP interventions have impacted girls' enrolment positively.^®^ Diagram 20 shows the learnings o f children on health issues due to training through BEP, or through other programs, which are fairly positive. The resu lt showing use (and perhaps knowledge) of iodized salt is quite low despite the many campaigns undertaken to advertise its use and benefits over the decades.

®® 5C, Revised Concept Note of BEP submitted to EKN 2010, p. 1 ™ SC, BEP, Biannual report, 2014, p. 6,

BEP school survey, Oct-Nov 2014 (T35) (Teacher responses = 85)

71

Diagram 20. Children awareness through training

0^ ro C s k. u a.

0BEP=2OOO 1

; i-J non-BEP=1000

Do you know hand washing

steps

80%

Do you know how to

prevent cough and cold

Do you know how to keep

away flies

Do you know flies can cause

disease

69% 77%

58% 61% 60% H

63%

54%

Do your parents use

Iodized salt at home

37% 29%

Source: BEP Survey, St. 24 (vi-x)

In terms of parental attitudes, boys were more likely to identify that school dropout is due to child labor,

whereas girls were not able to identify any specific reason, mentioning that a common problem is

usually the nonchalant views of parents regarding girls pursuing education.The FGDs w ith child club

members have revealed that the club formation and activation has helped encourage parents to send

their girls to school. Of the five child clubs interviewed, all five stated that girls' education is very

important and mentioned the right of a girl to an education and its importance for the family and future

generations. Girls from the child clubs started having better interaction with other girls and could make

others better aware of their rights and their role in society. These sentiments were echoed in interviews

with all three government district education managers. The PTSMCs members were also very positive

about girls' education and stated that parents are reluctant to enroll their girls only for primary

education and are more interested in enrolling girls if they have opportunities for middle and higher

education, which supports the BEP thesis and approach and is also reflected in the child clubs life skills

PTSMC training manuals.

A case study reinforced the need for continued education opportunities for girls: the observed school

was successfully upgraded from primary to the only middle level school in the area; the constructed

class rooms and recruited teachers helped in providing quality education to the deprived girls of the area; and the community enthusiastically participated in the up-gradation process, which in turn had

been expected to improve school enrolment.

The BEP construction work afforded girls the opportunity to continue schooling from primary to middle

and from middle to high. The construction work was appreciated by the district education management

who stated that: "The quality of construction is far better than the usual construction undertaken by

BEP, 2013, Final Child Clubs Research Study, p.6 ' SC, September 2012, Success Story of Government Girls' Middle School, BMC colony. Brewery Road, Quetta

72

Communication and Works (C&W) Department of the Government. I and my colleagues are very happy with the new infrastructure being developed / developed under BEP. These SEP infrastructures are also more cost effective in comparison to C&W department", although this last sentiment was not shared by the secretariat level officials. They also shared that the "construction of well-designed up-gradation buildings has very m uch helped in the retention of students, many cases were identified where due to the non-availability of nearby schools girls were made to leave schools by their parents, many of such cases resumed their education once it was stopped after the up-gradation of schools", which confirmed the importance of upgradation.

A relatively low 53 percent of teachers said that parents are motivated to send their child to school due to BEP, as shown in diagram 21. However, they were more positive about the involvement of parents and the PTSMC in education.

Diagram 21. Teachers' opinion of girls' schooling

01 as fa c (U u k_ <u a.

Do you think parents are

motivated to send children to school as a result of BEP

U BEP=75i 53%

Do you think that the level of

involvement of parents with their child's education is

satisfactory

Do you feel that PTSMCs are playing their role effectively

92% 89%

Do you think that the level of

involvement of parents with their child's education is

satisfactory 86%

Source; BEP Survey, Teachers 39-47

An important result is presented in diagram 22 regarding teachers' opinions on how to encourage girls' schooling. Training and ECE are given the most importance by both BEP and non-BEP teachers, whereas provision of facilities and furniture is given less importance. It is interesting since the provision of facilities is the most costly part of any educational development program, and the most time consuming to achieve.

73

Diagram 22. How to further encourage girls schooling according to teachers

0^ to n c dj (J k. 0^ Q.

60%

50%

40%

30%

20%

10%

0% j_ji

IBEP=57

InonBEP= 8

Training

33%

38%

Provisio n of

teaching' material

Greater role of

PTSMCs

7%

0%

4%

0%

-sa Learning Provisio Encoura material | n of I ging

s ' faci lities children ^ ^

ECE

1% 4%

0% 13% 3% 46%

0% 50%

Provisio n of

furnitur e

0%

0%

All of these

__4__ 0%

0%

Source: BEP Survey, Teachers 53

I'gi r r.i innovations Education innovation in training A component in the overall strategy relates to education innovation as a tool for change when "it

guarantees social, economic and cultural opportunities for children and youth, which entails the search

for new, evidence based and participatory educational teaching and management methods".This

component was included in consultation with the Provincial Institute for Teacher Education (PITE),

which was committed to changing the old teaching methods of rote learning with different and modern

interactive teaching methodologies methods. It was believed that these approaches would enable the

development of the critical thinking faculty of children through participatory teaching, and were

introduced through action based research. Some of these new methods and approaches have already

been tried and tested in Aga Khan Foundation's Early Childhood Development (ECD) project and the

primary education projects of United Nations Children's Fund (UNICEF) in Balochistan, which have been

generally well regarded. Education innovation under BEP built on the existing good practices and

introduced new ones. The innovative ECE tra inings have now become part of the Associate Diploma in

Education (ADE) program and are being considered for accreditation and affiliation by the University of

Balochistan according to interviews with the PITE director.

Participatory approach The participatory approach introduced by the implementing partners is another example of innovation

in the project implementation stage. School needs were identified through participation and mutual

consensus of teachers, parents, students and community members and PTSMCs monitored the project

!)/i

SC, RNE Education Proposal, p.7

74

interventions.^"^ The participatory approach was also adopted in the process of school up-gradation and the Director of Schools, d istrict education officers (DEOs), d eputy district education officers (DDEOs), teachers, students and parents were taken on board from the time the base line survey was conducte d till the completion of the work.^°"

The interviews with community and PTSMC members were revealing. The members stated that due to training and participation; "We have acquired confidence as a member and as a parent; it is now possible to reach out to community members and parents making them conscious of checks and balances in bringing up children; parents are now knowledgeable about the rights of children". In addition, based on their own experience the IPs felt that the "program has sensitized the Go vernment on key issues such as ECE, health and hygiene and community participation". Finally, the district management agreed that "community participation was enhanced through building capacity o f PTSMCs and giving them responsibility; and the PTSMC being an important segment of the management be given priority, through participation of good members in the PTSMC; and the government would support those members focusing on the good management" in future.

Mother tongue, literacy boost and numeracy boost A pilot on the mother tongue was one of the first innovative measures undertaken to improve cognitive skills of students. The mother tongue pilot had far reaching consequences and the government also had considerable discussion on language education. The good results obtained in the pilot led to it becoming a policy in th e BESP. Another innovation, the Literacy Boost is S ave the Children's innovative, evidence-based program to support reading skills development in early grade children. The Literacy Boost holistically pursued the goal of literacy in Balochistan by using assessment to identify gaps and measure improvements in the five core reading skills and training teachers to teach the national curriculum with an emphasis on reading skills. The innovation mobilized communities to support children's reading and was introduced in the mother tongue.

Another important innovation was the numeracy boost. In the beginning of the project Save the Children conducted a baseline survey in 18 numeracy boost focused schools and 18 comparison schools to take assess the pre and post situation of the students for numeracy boost and literacy boost programs. The three pronged approach was adapted for the study including assessment, train ing and community actions. The use of Tangerine as a data collect ion and analysis tools was also pilote d during this study. Save th e Children organized o rientation sessions and training for teachers and community learning workers on modules of the research study. This evidence based research pro jects on mother tongue based multi-lingual education, literacy boost and numeracy boost provided a basis for mobilizi ng government to adopt Education Leadership and Management (ELM) teaching methods and incorporate them in the syllabus.

Peace education

BEP, GGPS Muslim Ittehad Colony Success Story, Sep 2012 BEP, Government Girls Middle School (GGMS) BMC Quetta Brewery Road, Success Story, Sep 2012 SC, Literacy Boost Flyer, June 2014

75

A 5 da ys consultative training on Peace Education was conducted for master trainers of IPs, members of PITE, school teachers and BE Ps team members. They were introduced to basic principles of coexistence and role of teachers and policy makers to transmit values such as m utual respect, nonaggression, equality and cooperation in the education system. The topics inc luded acquiring values and knowledge, and developing the attitudes, skills, and behaviors to live in harmony and protecting the natural environment.

Enrolment campaigns The enrolment campaigns organized by the Department of Education with BEP support and the method of conducting these were in themselves an important innovation. The enrollment campaigns consisted of awareness raising activities including walks, meetings with parents and community members, seminars, displays of banners and posters of related messages in the focused schools of the 3 districts. The campaigns were organized with the participation of the PTSMCs, Education Department officials and the 3 implementing partners. The female members of the PTSMCs and focused communities were encouraged to participate in the awareness raising and door-to-door campaigns. The program teams also used banners, posters, messages on FM radio and cable TV to en roll the maximum number of out-of-school children. The banners were displayed at the busiest areas of t owns, in three focused districts and the messages were designed keeping i n mind the comprehension and literacy levels of the target groups.

Inipnct on tlie consciou snes.s of teachers, children and parents A culture of abuse Some documentation portrays a fairly n egative picture about the social, political, economic and cultural environment which actively encourages or tacitly condones violence against children,'"® and which shows a very challenging culture that the Program has had t o work within. In a research study, it was noted that children from both boys' and girls' middle and high school child clubs showed a distinct reluctance to share openly any incidents of abuse or corporal punishment, even without the presence of a teacher in t he room when questioned.'"® According to the referenced report, even when the children were able to recall all that they had le arned in their training sessions on how to identify and report incidents of abuse, there was a sen se of apprehension when they were asked about any examples that they could think of.

It seems the culture of corporal punishment is still very much a part of the psyche of the community and it is likely that the children have unconsciously de veloped an innate sense o f acceptance o f the issue, despite the efforts made by the Program to convince them otherwise. As stated in the baseline report, "the 'culture of silence' against all these insensitivities is ben efiting perpetrators as wel l as moun ting cases of kidnapping, corporeal punishment, physical aggravation, mental provocation, and vulnerability and abuse of children"."" Teachers and PTSMCs of schools are aware that corporal punishment is a very deep-rooted cultural phenomenon, which requires a drastic change in the mindset of both the teachers.

SC, June 2013, PTSMC Training Manual, p.12 SC, Child Club Research Study 2013, p. 7 SEHER and UNICEF, November 2011, Final Report Baseline, p. 4)

76

as well as the par ents of the students. If a few teachers in the school are against co rporal punishment, even their combined efforts to convince the perpetrators otherwise are often unsuccessful."^ As noted by BEP experts, " there is an ov erall cultural acceptance towards corporal punishment, and i t will take steady and gradual work to change t he communal thought process. Simila rly, topics such as sexual abuse are considered a taboo and need to be tackled with care"."^

During quarterly progress reviews the child club members shared their experiences and achievements."^ They also shared the support they got from their parents, communities and teachers in achieving their targets and highlighted the challenges and shared how they were overcome, where possible. Child clubs o f primary sections were unable to document their achievements due to a lack of writing skills. Major challenges that the child clubs in general face are changing the conventional practices that have become part of the culture including corporal punishment and the daily school errands done by children. Younger members complain that they get beaten up by older children when they try to stop other children from fighting during recess and as a result they have to involve the teachers to avoid d irect confrontation, which is actually quite positive. Child clubs in general struggle to get time off from academic activities since initially teachers perceived it as a very time consuming activity. At GGMS Sirki Kala the child club members were told that they were being instilled with rebellion against the administration, which was communicated to the field teams who then organized a meeting with the school teachers to explain the TORs of the child clubs.

The following are some important views from girl students gathered from the BEP survey.""

• Did you voluntarily participate in child clubs? o Only 22% of the girls said yes, which indicates that the concept and the participation may

need further refinement • Do you feel safe in the school environment, with teacher and in class room?

o 93% of the girls said yes which is a positive reflection of BEP efforts • Have you ever been punished by a teacher in past 4 years?

o Only 35% girls said yes althou gh this needs further assessment and correlation with other fact finding regarding corporal punishment versus penalization.

Prooi ain interventions and transformations Some progress was made early in the Program when the education department through campaigning by BEP teams, issued a notification to all schools in Balochistan, calling for immediate elimination of corporal punishment from schools."^ Some additional progress seems to have been made in addressing this very serious cultural issue since there seemed a willingness to discuss it in the FGD with child club

SC, Child Clubs Research Study 2013, p. 26 SC, BEP Biannual Report, Jan-June 2012, p. 37 SC, BEP Biannual Report, Jan-June 2012, p. 19 BEP Survey, Nov 2014, (St 50, 52, 54) BEP, Annual Report 2012, p. 5

77

members. Corporal punishment was mentioned as still happening in 2/5 of the child club interviews and

3/5 said it was not. One group said that parents were not told and no action was taken.

An important result is how teachers get children to behave as shown in diagram 23. These results are an

indication that due to the BEP program there has been a higher level of consciousness-raising of

teachers, a striking and important result especially in their views on corporal punishment.

Diagram 23. How do teachers feel about how to get children to behave?

OJ U) ro c <u u 01 a.

Speak out loudly

Try to attract

them by coercing

them

yBEP=ll' 60% 45%

Identify the group which is

not attentive

and address...

55%

Indicate that they

will be punished

30%

Do you use corporal

punishmen t in school

17%

Do you think that corporal

punishmen t can be useful in

some...

8%

Do you agree that corporal

punishmen t should be

banned

75%

Source: BEP survey, Teachers 19-23

Not surprisingly, children face bullying in school often (11 percent) or sometimes (62 percent). The

results for non-BEP schools are not much different. Whether they inform the teacher and receive

assistance is presented in diagram 24. Even though teachers are not involved often enough, when they

are they are approached to try to help, and can often be successful in that help.

Diagram 24. Bullying in schoo] and teachers'help in countering it

120% • - - -

(U 00 TO

4-» C 01 <J t. 01 a.

100%

80%

60%

40%

20%

0%

HBEP=200

wd nonBEP=100

Did you ever discuss the unpleasant behavior / bullying by class mates

with your teacher

If yes, did he/she help you?

18% 91%

Was the help effective?

20% 97%

54%

75%

78

Source: BEP survey, St. 60-62

Child clubs and change Fostering child participation is a key to combating violation of child rights. Participation raises avwareness in children about child rights and encourages th em to express their point of view. Child clubs create a forum for children to discuss ways o f contributing to the betterment of their school's environment. Teachers and students attended an orientation session about the roles and responsibilities of the child club, prior to the formation of the child clubs. The child club comprises students who show interest by volunteering for membership. The child clubs plan quarterly and contribute through activities, such as raising awareness in their fellow students. The mandate for these child clubs in BEP has evolved from receiving trainings on child rights and identifying child protection issues, organizing school uniform donations for poor students, managing stationary and book banks for poor students, maintaining discipline in schools, to receiving trainings on disaster risk and reduction (DRR) and health and hygiene (H&H)."®

The concept of c hildren's science clubs was attemp ted in Balochistan earlier through the USAID funded ED-LINKS project. However, these science clubs did not succeed and were not considered sustainable. In BEP, there has been an attempt to make the clubs sustainable th rough incorporation in the existing girl guides and boy scouts clubs per the record of the last steering committee meeting."®

In BEP, the child club interviews led to the conclusion that the children had internalized the concept of child rights. They defined it as addressing issues of early marriage, corporal punishment, cleanliness and discipline, sports, preventing or bringing back dro pouts and encourage community members to send their girls to school. They have learned that education is their right and that they should not be punished. They have received training on the rights of children, disabled child rights, health and hygiene and corporal punishment. They stated that: "teachers should not hit on the head, pinch the cheeks, pull at the hair, ask students for push-ups as punishment, wrench ears and braids, or use a stick." M oreover, the teachers give ser ious consideration to issues raised by child club members, for instance on the water availability issue. However, only 18 percent participated voluntarily in the child club (BEP Survey, St. 50). Conversely, out of 12 respondents, 11 teachers said that participation in child clubs was volunt ary (BEP survey, T-25).

The h ealth and hygiene training has also had an impact and they said that "we should wash hands before and after taking food; water should be boiled before drinking; after using bathroom, we should always wash our hands; we should keep the school clean and remove waste; we should cut the nails and brush our teeth." On school environment and improvements, one group was qu ite articulate and said that now there are more children in the school since the school is clean and well painted and classrooms have been constructed. They also raised the need for further improvements such as ins tallation of swings, heaters for the cold winters, a g enerator, and dust bins. The EC E class rooms were admired for

SC, BEP, 2013, Final Child Clubs Research Study USAID, September 2012, ED-LINKS Evaluation Report SC, BEP, April 2014, 4'^ Program Steering Committee Meeting

79

the fact that the children are keen and show interest in learning since the attractive classrooms motivate the children and the attractions include toys, tables and chairs. One gr oup expressed their feelings thus: " If I am the principal, I will get a bigger play area and sport s, have a chow kidar (guard), a high school section, will plant trees, competitions and prizes."

I'arenl teacher school management committees and community members as change agents PTSMCs: A research stud y by Save the Children concluded that as perceived by the BEP design and its objectives. Parent Teacher School Management Committee (PTSMCs) have had a very important role in the project."® The invo lvement was for the school improvement and provision of supplies processes , but also for linking committees with the schools for continuous monitoring and supervision. Another important role of the PTSMC was establishing school linkages w ith communities to make the process sustainable. Active PTSMC were reportedly formed in all 340 BEP ta rgeted schools. The operational research (OR) findings of the study revealed that most of the committee members had basic famil iarity about their roles and responsibilities with committee presidents more aware than other members. During a discussion on their roles and responsibilities, none of the committee members mentioned school financial management and record keeping as one of their functions. Except one, all of the committee members affirmed that they have been trained on different topics by BEP im plementing partners.

The strong c ommunity engagement to improve girls' enrollment and female members' involvement in PTSMCs meetings is confirmed in the findings of the current evaluation. The Mastung PTSMC "works for the school and they play their role in the community; members are conscious that all children are children of the community but it is their primary role to make their future secure"; and they have "became aware of our educational responsibilities after becoming members of PTSMC under the BEP; now all teachers and parents have realized that it is the ir responsibility to meet the requirements and needs of children and for this they need to work together".

The Que tta PTSMC members said th at "good education and health are the fundamental rights of our children; we have to treat them well, and no corporal punishment; after receiving two trainings on child rights, child protection and positive disciplining, we are more aware of our responsibility towards the betterment of our community, particularly with respect to the education of our children."

Training for teachers as well as fo r PTSMC members has created a change in t he thinking. For instance, PTSMC members in Mastung shared that the child rights training has helped them advocate with teachers, which has led to children being more at ease and not so afraid any more. "Training has helped them in their roles as parents and community members and i t has brought a pleasant change in the children and the good effects are seen in their homes. Parents no longer scold o r punish children at home as much as they used to and the same is the case w ith teachers in schools. All this has created self-confidence in the children as well as in the PTSMC members and parents. It is now possible to reach

SC, ME AL Unit, Operational Re search on Effectiveness of Parent Teacher School Mana gement Comm ittees (PSMC) in the BEP Project

80

out to community members and parents making them conscious of check and balances in bringing up children, and parents are now quite knowledgeable about the rights of their children"/^®

Diagram 25 presents information on school observations of PTSMC training topics and the results are an indication of higher order learning by PTSMC members that both child rights and corporal punishment issues have been well infused into their thinking, as corr oborated in the FGDs w ith community and PTSMC members above.

Diagram 25. What are the topics which are covered in the (PTSMC) training?

50

01 as ro c lU u k. « a

40

30

20

10

0 f -- ' ' —'

Child Rights (CR)

Corporal Punishment

(CP)

Positive Dicipline (PD)

DRR Others

HBEP=26! 54 23 8 1 8 8

Source: BEP Survey November 2014 (5066)

The results f rom different backgrounds for Quetta interviews show that the learnings have been fairly similar across the board, which are repeated in the case of Mastung and Killa Saifullah.

The Quetta Category 1 (least settled area o f Quetta) PTSMC fel t that the trainings have he lped them understand their role in the community and that they have obligations to the society. A sense of responsibility prevails and they get respect from their community since they work through consultation with the community. They expressed feeling happy that they are doing it for the children. They admitted that their role in their community has enhanced and people listen to what they say as they now have a better understanding of education and its importance. There is a se nse of responsibility now which was previously missing in the PTSMC members. They said tha t 'they are very happy now and committed to working for children; they have learned how to work together and BEP has sowed th e seed w hich has now developed into a tree".

The Quetta category 2 PTS MC (better settled area of Quetta): A ma jority of the participants indicated that they have received training on child protection and related issues. Their understanding on child right issues was rudimentary but they knew that good education, upbringing and providing good food to

120 The PTSMCs In Klila Abdullah did not share suc h sentiments, which could have been due to different circumstances or due to different interview techniques.

81

the children are basic rights. Quality education for both boys and girls was also ide ntified as a right by the teachers. Parents have also learned that education is th e right of children. It has changed their way of thinking although they said they u nderstand that al lot more,has yet to be done to improve the lives of children. Parents became close to the teachers and interacted with them regarding their children's education with roles and responsibilities distributed among community members. They shared that they learned the importance of the PTSMC role in ensuring good and quality education, and that they can now work in partnership with non-government organizations (NGOs). The overall perception of the community towards girls' education was positive and surprisingly they indicated that the presence of middle or high school for girls is termed as a prerequisite for girls' education. Most of the parents wanted their children, particularly girls, to complete metric (10*^ grade), or at least grade 8. They expressed that "girls have a right to education so that they can become good mothers, doctors, teachers, and give good guidance to others".

The Quetta category 3 PTS MC (the most settled area in Quetta and with the least security concerns); They shared t hat good education and health are t he fundamental rights of children according to this category and they have to be treated well without being meted out corporal punishment. Quality education, right to admission, health and hygiene, mental growth, preventing dropouts are some of the important areas. They have received training on protecting children from corporal punishment and similar training has been given to the teachers. Since awareness has been create d it is possible to keep checks and balances. There is g ood cooperation from the teachers and they check home work of the children and are happy doing that. They said that "the teachers give lectures on showing respect and so do the parents at home, as a result of which the children now show more respect to their elders".

Community: In the current evaluation it was found that even community members were quite positive about the Program such as fro m the Mastung community, who stated that "the PTSMC and ourselves both have endeavored that more and more of our out of school children should get enrolled; our first priority is that our daughters should receive education; we believe that education of daughters is necessary and we now realize that education is necessary for sons as well; we are satisfied and happy with the work done under BEP."

Some of the Quetta community members interviewed said that the Program "has worked hard to increase enrollment in schools parti cularly enrollment of girls." In Killa Abdul lah, some of the PTSMCs interviewees stated that we are "now conscious that education will create awareness among females; there will be positive impact of girls' education on the community; there is success achieved in higher enrolment of girls."

Child Focused Health Education Diagram 26 quite succinctly presents the impact of the Program i nterventions on health and hygiene learning by the children. The surprising finding is that hand washing before eating has declined between the baseline (dark grey) and currently (light grey).

82

Diagram 26. Reported hand washing practices among school children

100% • - -

a Base-line End-line

Source: BEP, Nov 2014, SHN End-line Survey, p. 23

The end-line survey results show that in many cases children have gained knowledge positively in the time between the baseline and the end-line surveys. For instance, their knowledge about how flies spread disease has expanded as about malaria prevention."^ The knowledge of adolescent girls regarding boiling water for safety has increa sed."^ However, results are mixed for hand washing as shown in diagram 27 and somewhat similar to the results presented earlier for all children.

Diagram 27. Hand washing practices among adolescent girls

100% —

0) CIO 2 c

I

a Base-line End-line

80%

60%

40%

20%

__u.. When Hands

Get Dirty* Others*

23% 5%

Source: BEP, November 2014, SHN end-line survey, p. 64

Regard end)}' and safe construction

ng the importance of facilities, interviews with child clubs show that the children agreed that many of their requirements were being met through the Clubs, including a request for a water well and

BEP, Nov 2014, SHN End-line Survey, p. 34 and 47 Ibid,, p. 62

83

a play area, for instance. In separate FGDs w ith district managers and community members, additional classrooms are mentioned as a key contribution of BEP, as in FGDs with Parent Teacher School Management Committees (PTSMCs), which include teachers as members of the parent-teacher committees.

The a ttractive, colorful and child friendly environment of ECE classrooms has a ttracted children and encouraged them to get enrolled in the schools according to BEP experts. The im proved design of ECE classes is a better alternative to the typical katchi class concept and has boosted the parents' confidence that their children can avail better early childhood education in public schools which otherwise was considered to be acquired only through private schools.^" The concept was also a ppreciated by the government stakeholders. For instance. Minister for Secondary Education at the inauguration ceremony of ECE class at GGBMS Railway Housing Society appreciated the design and child friendly environment of the newly constructed ECE class, and advocated expansion of the intervention in other schools of remote districts of the province. The school const ruction strategy highlighted a child friendly where no corner or area in th e school premises should be isolated or unattended, which might become a risk or threat for the child^^" as was also demonstrated in the BEP survey results presented in an earlier section. Interestingly, even some of the non-BEP PTS MCs interviewed agreed that the school environment should be child friendly and free from fear.^"

Kev ch,ir;u:teristics of the Early Childhood Care fECC) model The training of ECE teachers has been a key achievement since the training has shown a great improvement in the teachers based on pre-training and post-training test scores on ECE.^^® Th e main purpose of the training was to make the teachers aware about ECE concepts and how to plan their daily routine and conduct ECE activities which involve children in different activities and which increases the children's knowledge. Teachers were trained on how to build the basic concepts of children by conducting different activities not only for learning but also for enjoying the learning process. The teachers learned about the daily schedule, circle time, energizers, and many other activities.

As noted in the midterm evaluation, one of the widely recognized contributions of BEP was to p ut ECE on the radar of public planning and budgeting, an area long ign ored by policymakers and practitioners alike.SC developed the ECE strategy and the accompanying financial outlay. In the sector planning conducted by the Policy Planning and Im plementation Unit, Department of Education, GoB (P PIU) w ith the support of UNICEF, the ECE componen t was led by BEP partner s. The endeavor culminated in the operational concept being included in the Balochistan Education Sector Plan. At a recent EC E conference in Karachi, the deputy focal person of the PPIU of the Balochistan De partment of Education made a

BEP, Biannual Report Jan-Jun 2012, p 17 BEP, Annual Report 2012, p. 5 and p. 10 Mastung PTSMC_9 and Killa Abdullah_9 SC, BEP, ECE Refresher Report 2014, p. 2 and p. 41 BEP, Midterm Evaluation, p. 10

84

presentation on the history of ECE in the province and its future prospects, a po sitive endeavor arising from the development of the Education Sector Plan and its emphasis on ECE/^®

The Additional Secretary when interviewed particularly emphasized the good work done in institutionalizing ECE. He shared tha t while ECE centers had been established, the issue of appointment of teachers was delayed due to Government's indecision. Both SC and IPs advocated for the hiring of the teachers which was part of the memorandum of understanding with government. The most recent progress is that the posts have been approved and created, according to the implementing partner managers.

(iovei iirnent officials expressing impact performance There was thoughtful concurrence at the secretariat level about BEP's positive contributions. Some concerns were expressed about infrastructure quality and the higher costs associated with infrastructure development). However, the overall assessment was t hat BEP, in comparis on with other programs, offered a wide range of services including infrastructure, ECE, child clubs, teachers trainings, school hygiene and nu trition, and had performed well. At the district management level there was also concurrence that BEPs key milestones had been increasing e nrolments and improving the quality of education services. Finally, the community members, some also PTSMC members, expressed their satisfaction. They felt a big advantage was tha t their children were receiving education in a favorab le environment, with the community members also giving more attention to education, and girls' education being an important focus. As expressed by some Mastung community members, "parents are convinced that education brings sensibility and better understanding, which is a significant change in our society".

Conclusions on Impact:

The impact of the Program has been significant on many fronts and has had notable impact. Girls' enrolment has increased dramatically, as has the opp ortunity for girls to continue on to higher levels of education beyond primary and middle, addressing a key weakness in the system thus far. Children's awareness abou t schooling and their rights has been enhanced greatly, a significant outcome of the Program. Teachers overwhelmingly agree that the Program has encouraged girls' schooling, and that the most important factors have been training and the focus on ECE. The Program's many innovations have had demonstrably good impact such as participatory techniques in teaching; participation by community members, teachers and children in Program implementation; the ECE program of provision of classrooms, materials, teachers and training, as well as othe r initiatives such as the Literacy Boost, th e Numeracy Boost and the Mother Tongue Pilot, which initiated considerable discussion with Government counterparts. Training on peace e ducation was an other major innovation for the provincial context. Although the culture of child abuse has been ingrained in schooling and even in homes, the Program made considerable headway in raising awareness about child rights among community and PTSMC

Govt, of Balo chistan, Department of Education, PPIU, Sep tember 2014, Poli cy and R ight to Education (RTE); Status of Early Childhood Education In Balochistan.

85

members, and teachers and children. The survey results are quite significant in these areas. The deep rooted culture of abuse of children has n ot been eliminated, but the Program has gone a long way towards enhancing the awareness of teachers, children, community members and members of PTSMCs on the rights of children, including their right to education and to not be subjected to corporal punishment and other forms of abuse. A majority of teachers expressed that they were against corporal punishment in schools, and that it should be banned. Significantly, children approach teachers for assistance when being bullied in schools, and receive a positive response, in both BEP and other schools. The success or otherwise of the child clubs remains a somew hat grey area; in the qualitative interviews the members express positivel y about girls education, child rights, health issues, and others. However, the majority of children said their participation was not voluntary, whereas 11 of the 12 teachers interviewed said th at children's participation was vol untary, a clear contradiction. The c hildren were positive about the child friendly construction and attractive classrooms for ECE. Child health has improved in terms of awareness about good practices and the communication gap between mothers and adolescent girls has been decreased , a concrete ach ievement. The ECE model has obtain ed renown and established a solid platform for further realizations in Balochistan, with government especially supportive due to the close involvement of PITE. Save the Children has developed and implemented the blue print which forms the basis for the Government of Balochistan's Education Project which is funded through the Global Partnership for education (GPE), as well as a blueprint for many important components of the Balochistan Education Sector Plan (BESP), especially in the area of ECE.

IX. Sustainability

Slutus of National Finance Coinmission (NFC) financing in Balochistan The tax revenue collected by the federal government is dis tributed between the federation and the provinces and amongst the provinces on the basis of the National Finance Commission (NFC) Award. This share of the province under the NFC Award is k nown as federal divisible pool transfer. Under the 7th National Finance Commission Aw ard, the percentage share o f the provinces in the divisible pool is 57,5% starting FY2011-12 and the federal share is 42.50% . Due t o increases in the nominal size of the federal divisible pool, transfers to Balochistan are expected to increase.The provincial share is divided amongst the provinces in the ratios given in table 16 and was decided on the basis of a multip le criteria based on population, inverse population density, revenue and poverty.

Table 16. Horizontal distribution of the National Finance Commission Award (NFC] Punjab Sindh Khyber Pakhtunkhwa Blaochistan 51.74% 24.55% 14.62% 9.09% Source: Government of Balochistan, Department of Finance, White Paper, Budget 2014-15, p. 12

Government of Balochistan, Department of Finance, White Paper, Budget 2014-15, p. 12

86

In addition there are Straight Transfers, which under Article 161 of the Constitution and the NFC Award include;

i. The net proceeds of the Federal excise duty on natural gas ii. Net proceeds of royalty on crude oil and natural gas assigned to the provinces under the

Constitution

(ins di'velopment Surcharge Prior the announcement of NFC Award in 1991, the Gas Development Surcharge (CDS) was generated only from gas resources emanating from Balochistan. The surcharge was however not paid to the provincial government but utilized from development of other gas fields in the country which deprived the province of Rs.29,000 million. The Government of Balochistan pleaded the case t hat the province should be compensated for its depleted resources and the amount utilized for exploration of new gas fields be paid with interest which by June 2009-10 came to Rs.178,597 million.

Excise dut y on gas, royalty on gas, and surcharge on gas are the most important revenues included in Straight Transfers. The revenues somehow were on a decline till 2003-04, which started taking an upward trend mainly due to increasing well-head prices and payment of arrears. The st raight transfers to the provincial government are reflected below in table 17.

Table 17: Straight Transfers for FY2013-14 & FY2014-15 [Rs. million) Straight transfers Budget estimate

2013-14 Revised budget estimate 2013-14

Budget estimate 2014-15

Royalties, excise duties and surcharges on natural gas ( and so me royalties on crude oil)

14,201 14,480 16,689

Source: Government of Balochistan, Department of Finance, White Paper, Budget 2014-15, p. 15

As shown, straight transfers are expected to increase about 15%, although in nominal terms (that is, not adjusted for inflation, providing the province with additional fiscal space for possible use in the education sector. If inflation of say, 10%, is assumed, the percentage increase falls to 5% in real terms.

(rMvei n inent Budgets Budgetary allocations for education have been increasing annually, due in large part to the significance given to education initiatives by the Government of Balochistan as well as the continued effective advocacy efforts of external donors including Save the Children. The increase in the education budget as a percentage o f the total provincial budget is presented in Diagram 28. Although government budgets are satisfactory, it is as yet uncertain that the all financial requirements of the BEP will be continued; for instance, the all-important ECE class, the funding of PTSMCs, and funding of regularized teaching positions for ECE.

87

Diagram 28. Education Budget as a Percentage of the Provincial Budget

25% -

o; 00 n c s 01 a

20%

15%

10%

5%

0% Current Development Total

2013-14 16% 23% 18%

2014-15 18% 23% 19% i

Source: Government of Balochistan, Finance Department, Budget 2013-14 and 2014-15

D ;; n istan Education Sector Plan and additional funding

Most importantly, BEP has had a major impact on strategizing and planning in the education sector. Save

the Children was an important member of the Local Education Group (LEG), which provided key

technical inputs during the formulation of the Balochistan Education Sector Plan."° A focus on early

childhood education is clearly reflected in the plan, as are the areas of quality education, access and

governance, all areas addressed in the Balochistan Education Program.

In support of the Sector Plan, several donors are interested in working in Balochistan. Table 18 below

gives information on the current development partner supported education projects in the province,

including support to ECE.

TablelS. Donor Collaboration in Balochistan

Development Project Name Implementing Geographic Financial Year Partner

DFAT, Australia

ECE/ECD Program

Idara-e- 'ECE Taleem -o-Program Aagahi (ITA)

Entity

Aga Khan Foundation

ITA

Scope

Gawadar, ADD Killa million Saifullah and Quetta

Commitment

4.75 2010-2015

Area of support

Access to Early Childhood Development

Lesbella, Killa Saifullah Quetta

US$ 1.4 ] 2014-2018 Access for girls to ECE & million P/S in 100 schools; support

& material

Government of Balochistan, Department of Education, PPIU, Balochistan Education Sector Plan 2013-2018

88

UNESCO Supporting Provincial Capacity Building to Realize Girls' Rights to Education

World Bank

World Bank

Global Partnership for Education/ {World Ba^nk lUNICEF

Balochistan Education Support Program

Promoting Girls Education in Balochistan

Balochistan Education Project

Promoting Girls Education in Balochistan

Department of Across the Malala Fund Education province Balochistan (BEF)

Balochistan Education Foundation/ Secondary Education Department

PMU/ Secondary Education Department

PMU/ Secondary Education Department

Province wide

USD 22 jMillion (loan)

Government of Balochistan, Secondary Education Department

12 districts

Province wide

USD Million (grant)

USD Million (grant)

10

34

Upstream work: the whole province

USD Million (grant)

15

2014-2016

2006-2014 Increasing access through community and public private partnerships

2012-2015

2014-2017

2013-2017

Expanding access for girls to primary and secondary education through policy support; and improving quality

Access/ partnerships

Community!

Access, Quality, Community partnerships and Monitoring

Access, Quality, Building and strengthening

Capacity system

Source, GoB, DoE, PPIU, ECE presentation, slide number 15 and 16

Recruitment of teaching staff The delays and general uncertainty about hiring of teachers for key ECE positions and upgraded schools

has been a serious drawback for sustainability and for the continued operations of this extremely

important venture for the children of Balochistan, and has been discussed in detail in an earlier chapter.

iiut eased in knowledge of partner organizations

The implementing partners were interviewed regarding their vision and learning during BEP

implementation. According to IDO, the organization's vision is "a developed and empowered society

with every one enjoying easy access to basic needs and services without any discrimination" and mission is "to undertake an integrated approach leading to the empowerment of marginalized groups and work

for sustainable development by taping interventions in all related sectors". IDO staff believes that

education is the key to playing a major role in a sustainable, developed and empowered society. BEP

played a pivotal role in the institutional capacity building of partner organizations according to IDO

management. Several trainings and exposure visits were conducted for the staff and equipment were

provided to improve productivity. The organization, although with limited resources, replicated the

Balochistan Education Program model in several districts. There was a relatively limited scope and the

89

numbers sent to school were relatively fewer, but the coordination and cordial relations with the department of education will ensure continued efforts on the part of partners to sustain the program.

The second par tner, SEHER, w orks and strives for social protection of the vulnerable segments of the society, particularly the children and child protection is at the center of organizational focus. They believe that BEP's in tervention has greatly contributed to their organizational mission and objectives. There management felt that there could have been a broader group of all major civil society organizations engaged in sharing of experiences of the Program. In any case, the IP's intend to work closely with the network of other civil society organizations in order to replicate the program interventions and for sustainability.

The third partner, SCSPEB's organizational goal is, "to be an ef fective and efficient service organization supporting government, communities and other partners in their efforts for the development (particularly in the province of Balochistan." The mission statement of SCSPEB is, "to create partnerships between government and communities, in all social sectors, for a sustainable development of the province of Balochistan". The organization's management believes that BEP provided the opportunity to further the organizational goal and mission of SCSPEB. Partnerships between government and communities were further strengthened, and the organization provided services for passing on the direct benefit to the children and the communities at large in the three districts. They also felt that BEP played a pivotal role in institutional capacity building of partner organizations. Several tra inings and exposure visits were conducted for the staff and equipment was provided to improve productivity.

Working relationships with key stakeholders and their ownership of the Program The IP's have wo rked hard to assist the government in continued ownership of the Program, primarily by keeping various levels of government informed of BEP activ ities, and by inviting senior education managers such as th e Secretary o f Education to various events. Right from the start, in May 2009 a formal launching ceremony of the Program was held which was attended by the highest level representatives of the federal and provincial governments."^ The advisor t o the Prime Minister on the social sectors was the chief guest which demonstrated the importance and ownership of the Program by the Government of Pakistan. The event witnessed a high level participation from all quarters of the development sector including the provincial education minister, members of the provincial parliament, donors, civil society organizations and the media. This res ulted in the signing of a memorandum of understanding (MoU) between the Government of Balochistan and Save the Children which has institutionalized the working relationship between SC and GoB in a strategic manner. Close coordination with the education department helped in creating ownership of the project activities, and better acceptance amongst government officials. The institutional strengthening and planning of advocacy activities in consultation with the PPIU has made Save the Children a strong technical partner in providing support to that unit."^

SC, BEP, Biannual Report Dec 2009 - Jun 2010, p. 4 SC, BEP, Annual Report 2012, p. 36

90

According to the implementing partners, the Program played a significant role in the capacity building of the education department in Balochistan, and the project has been implemented with continuous consultation with education department. Officials such as D EOs and assistant district education officer (ADEOs) have also re gularly visited schools focused under the project to monitor ongoing activities. Usually the project team accompanied the education officials regarding any issues faced du ring project implementation and their resolution. SC and specially the implementing partners are invited to all policy related meetings conducted by the education department, including a series of consultative sessions on Balochistan Education Sector Plan. The Bureau of Curriculum (BoC) was provided all the training material developed under BEP, and also info rmation on school health and nutrition to assist the government in developing their curriculum.

Also according to the implementing partners, the Government counterparts had ownership of the Program and termed it better in comparison with other programs. PITE was especially appreciative and has included child protection in its training curriculum. Equally importantly, the department of education has created po sitions of ECE teachers. However, there have been frequent changes both at the provincial level (secretariat, BoC, directorates) and at the level of the district education management, which has required additional briefings.

BEP also played an active role in coordinating the various units of the Government that were involved in program implementation such as PITE, BoC, distri ct education offices, the education department and the district education management information system (DEMIS). The program has been able to establish ownership of key stakeholders, government and communities, and in particular the PTSMCs with their increased capacity to deliver actively contributed and provided assistance and oversight to BEP implementation. The Government has also as sured its support for continuing the good work of PTSMCs, for which funds have been earmarked after some tim e. The Balochistan Compuls ory Education Act of 2014 has been notified which specifies a role for PTSMCs."^ Similarly the ECE policy is being developed in line with the ECE model established under BEP. Government institutions like PITE and BoC 's capacity have been strengthened and they are now keen to sustain the professional development of teachers by carrying forward the work done under BEP.

Even at t he community level there seems to be a visible dif ference. The community of Killi Resaldar, in Quetta said that not only do BEP schools have more fac ilities, but "we feel empowered and have taken the ownership of the school wh ich is not the case in ot her schools. We have r egular meetings and our school is improving gradually."

Constructive partnerships have been established with different offices of the Government of Balochistan, as well as the people of the province under the aegis of the Balochistan Education program. This is evident in the positive response from all levels regarding the impact of the Program. As specified in the key informant interview (KM), the Provincial In stitute of Teacher Education (PITE) considered BE P interventions for professional development better than those of other development partners in terms of the quality of modules, comprehensiveness, adequacy of content and focus on objectives. The training curriculum was considered in line with the 2006 curriculum, thus meeting the expectations of

' Government of Balochistan, February 26, 2014, Notification for the Balochistan Compulsory Education Act

91

Government counterparts. The ECE training has now become part of the Government's program and is being considered for accreditation and affiliation by the University of Balochistan. PITE representatives agreed th at areas such as child protection, health and hygiene, and ECE should be part of all future programs and projects.

There was also concurrence at the secretariat level about BEP's positive contributions. Some concerns were expressed about infrastructure quality and the higher costs associated with infrastructure development). However, the overall assessment was that BEP, in comparison with other programs, offered a wide range of services including infrastructure, ECE, child clubs, teachers trainings, school hygiene and nutrition, and had performed well. At the district management level there was also concurrence that BEPs key milestones had been increasing e nrolments and improving the quality of education services. Finally, the community members, some also PTSMC members, expressed their satisfaction. They felt a big advantage was that their children were receiving education in a favorab le environment, with the community members also giving more attention to education, with girls' education being an important focus. As expressed by some Mastung community members, "parents are convinced that education brings sensibility and better understanding, which is a significant change in our society".

Most importantly, BEP has had a major impact on strategizing and planning in the education sector. Save the Children was an important member of the Local Education Group, which provided key technical inputs during the formulation of the Balochistan Education Sector Plan."'' A focus on early childhood education is clearly ref lected in the plan, as are the areas of quality education, access and governance, all areas addressed in the Balochistan Education Program.

'liüiitice.s

The team members at SC and EKN have worked hard to keep government and other stakeholders involved and interested in the Program, especially highlighted by the end result of having BEP team members on the Local Education Group for development of the Education Sector Plan for the Province. The S CNL team members were unable to visit Balochistan too often due to various security concerns, which would have contributed to better alliance building.

Conclusions on sustainability:

The Program is deemed sustainable based on the many policy influences and the model realized by the ECE program. There is financial space i n Balochistan's budgetary situation due to additional funding from the National Finance Commission Award, as well as large propose d donor projects in education. The education development budget has been fairly substantial for the last two years at 23% of the total provincial budget. The one important area that was unable to be adequately addressed was recrui tment of teachers due to delays on the side of the government, which makes it difficult to uphold the advances made in terms of development of ECE centers and up-gradation of schools until the posts are filled. Until this is done, the upgraded schools will not be functioning efficiently and will not be sustainable. The role

' Government of Balochistan, Department of Education, PPIU, Balochistan Education Sector Plan 2013-2018

92

of all implementing partners as members of the Local Education Group, which provided technical oversight for the development of the Balochistan Education Sector Plan, lends depth to the Program's sustainability. The incorporation of ECE in the Plan as well as acceptance by Provincial Institute for Teacher Education (PITE) of ECE techniques and related training provides further support for sustainability. There has been increased knowledge of implementing partners, all Balochistan NGOs expected to continue work in Balochistan's education system, which is positive. The discontinued funding of PTSMCs is anothe r area where communities will simply not be able to cover the expenses. However, a form al role for PTSMCs has been defined in new government policy in 2014 concerning the right to education, which lays the groundwork for government funding of the committees.

X. Summary of Conclusions and Recommendations The Balochistan Education Program has made tremendous strides in advancing education in the three selected districts. The Program has provided an important precedent for future developments as demonstrated through the importance given to similar programs in the Balochistan Education Sector Plan. It is important that Save the Children build on its successes in Balochistan by continuing to be a key player. This can be attained through sustained membership in the Local Education Group, which will probably be continued by the Government of Balochistan for monitoring implementation of the BESP mandated reforms. It is also important that the organization play a strategic role in provision of technical assistance to the Policy Planning and Implementation Unit (PPIU), especially in early childhood education which is the organizations area o f expertise, and in which it has already made tremendous progress. Through successfu l implementation and partnerships, the Program has left a solid fo undation for future reforms and accomplishments.

Table 19. Summaiy of conclusions and recommendations Areas Conclusions Recommendations

Objective 1: To expand and improve the existing infrastructure of the schools, create additional spaces, and to upgrade primary schools to middle and secondary levels to

The objectives of component 1 are deemed to have been met in a satisfactory manner. The Program has managed to expand and improve the existing infrastructure and surpass by 26 percent the targeted number of children in school that were addressed through the Program. The Program achieved the targets of construction and establishment of new ECE centers, as well as up-gradation of schools from primary to middle and middle to higher levels, thus opening up avenues for children to continue in school, especially girls . The creation of a safe and frien dly environment in targeted schools has been achieved to quite a large

93

benefit over 73,000 children (of which 16,200 new) in the three districts of Balochistan by end of 2014

extent, although not fully, with the majority of children agreeing that they felt schools were a clean and safe environment. The teachers were very positive about the impact of new facilities and furniture on schooling as well as availability of water and boundary walls.Importantly clean drinking water was fully available in the schools surveyed. Almost 88 percent of the toilets In BEP schools were in use by children alon e, unlike other schools. Especially no teworthy were the disability bars and ramps installed in over a third of the schools. Schools had a good supply of basic classroom materials and an excellent supply of learning materials."^ Children generally considered the school a safe pla ce with a clean environment.

Objective 2: To improve the quality of education through promoting child friendly and participatory teaching methodologies in all targeted schools of the three districts of Balochistan by end of 2014

The objectives of component 2 are deemed to have been met in terms of promotion of child friendly methods and the indicators on student-teacher ratios, completion and transition, but less so in the areas of development of cognitive skills and overall confidence levels of the children. In terms of some quality indicators the student teacher ratios are satisfactory although higher than the district averages probably due to higher enrolments in BEP schools, especially for girls. The promotion rates have declined over all in Balochistan, whereas the BEP promotion/completion are about 25-35 percent higher, about 85 percent for ECE and primary and, quite high at 95 percent for the middle and high levels. The transition rates, from primary to middle and middle to higher are also gr eater in the BEP schools th an in the districts overall, which shows that the BEP objective of having children continue school to higher levels has been positive. This is an important achievement given the high dropouts rates that persist traditionally at the end of each

Future programs should focus more on the software aspects of enhancing cognitive skills in children, and less on what can became an expensive and time consuming construction program. There should be further research into the reasons for children feeling that the lessons are more interesting due to the participatory techniques used and less so due to use of learning materials by the teacher, so that either further emphasis can be placed on the use of participatory techniques or into why the use of materials is not as successful.

The impact on parents and children has also been positive, which is highlighted in later chapters on effectiveness and impact.

The provided supplies were being used to a great extent, as highlighted in a later chapter on effectiveness.

94

level of schooling, especially grades 1 to 5. Training has been enhanced through the involvement of PITE in teacher training and the teachers have learned new skills in teaching. Their teaching methods are observed to be participatory although about two thirds are s till following the traditional row seating in classrooms. A solid ma jority is using the learning materials while teaching. Notably, over 75 percent of the teachers find the ECE training quite effective. PITE and other government officials were quite vocal about their appreciation of the teachers' training techniques instilled by the Program, especially for ECE. Conspicuously, a majority of children agree that the teacher makes the learning process enjoyable, mainly due to use of participatory techniques, and to a lesser extent due to use of the learning materials, which needs further examination. Almost all students interviewed expressed that they are able to ask questions of their teachers and that they are encouraged to participate, although this was equally true for BEP and other schools. A majority of community members and PTSMC members agreed that the ECE classes had transformed the educational process in their neighborhoods. Student learning outcomes are mixed with relatively poor learning of math concepts, in both BEP and other schools. The BEP school children are better equipped in participation in ECE activ ities and participation in groups, but the other school children did considerably better in terms of concentration skills, willingness to help other children and overall confidence levels which is a debatable result. Discussions with the community members and child clubs members pointed to greater confidence of children participating in the BEP activ ities and schools leading to the need for further research in this area.

Objective 3: To improve

The objectives of component 3 are deemed to have been partially met since the Program did

> Future projects should look at how EMIS can be kept

95

educational planning and budgeting systems in the Balochistan province through the development and introduction of evidence based District Education Planning models, as well as the strengthening of Education Management Information Systems by 2014 (increase education budget and strengthen management)

help to set up the District Education Management Information System (DEMIS) in the three districts.

The links between planning and budgeting were not addressed gre atly; the activity was rescinded since other donors were already working in this area. Although staff was trained, there is a lack of staff in one district which means that in effect the system is not working there. A success of the DEMIS is the scaling up to all districts of Balochistan where data entry points have been set up and data entry is in process. The enriched EMIS collects data about schools, teachers and for the first time in Pakistan, the Balochistan EMIS is collecting data on individual students. There is a great deal of ownership on the part of Balochistan government. The actual use of data based decision making is n ot emphasized in the work done with the districts, or the use of EMIS in different educational areas such as human resource management or payroll management, although that is expected in the future. In terms of other areas of advocacy, the Program was successful generating land donations from community and government for the construction of additional rooms and ECE centers, which was a notable achievement since community members were also involved in raising their voice in this regard. The fundraising by community members for schools was also a notable achievement of the Program, given the poverty of the areas worked in.

>

ongoing after project funding ends. Greater emphasis should be given to client needs and an effort made to integrate data based decision making in the system. Future projects should look at the possibility of including other areas into the EMIS system, such as human resource management, including staffing patterns, and payroll management. The lessons learned in enabling community to raise their voice regarding making land available from the government should be shared at different forums.

Objective 4: Improve School Health and Nutrition in 340 Schools of 3 Districts of Balochistan by 2014

The efforts of the SEP team have been satisfactory although the results themselves are mixed. The comparison of base-line and end-line suggests tha t there has been im provement in key indicators such as positive change in knowledge and practices of safe drinking water that has been significantly increased. Especially in Quetta; similarly teeth cleaning practice had significantly improved such as cleaning teeth at critical occasions i.e. after breakfast (40% vs 60%) and before sleeping at night (26% vs 47%). There has

The end-line survey findings should be complemented by project internal and external review with stakeholders to investigate implementation bottlenecks and explanations for large variations in the base-line and end-line surveys in some of the key indicators.

96

also been a significant increase in knowiedge of washing hands after playing and whenever gets dirty %( 14 Vs 26 and 17 Vs 30 consecutively). However there has been a decline in some indicators like knowledge about critical occasions of hand washing, practice and knowledge about balanced diet. But this data contradicts with the data collected through another tool used in overall BEP e valuation. The data reveals that more than 90% of the children wash their hands b efore and after eating food and using toilets. While the baseline figure was 66%. So, there is a high probability of anomaly during data collection. Save the Children had no access to the field to support enumerators while collecting data during end-line survey in two of the districts (Killa Abdullah and Mastung) due to security reasons. That might have affected quality of the data.

Also the biomedical part of the survey was dropped again due to security reasons that included stool testing for parasites and blood test for Iron deficiency anemia. This also limited to gauge the project success in terms of health improvement. There was no recent relevant secondary data available that could have been linked to the project achievements in program areas.

These findings should be complemented by project internal and external review with stakeholders to investigate any implementation bottlenecks and explanations for large variations in the base-line and end-line surveys in some of the key indicators.

Relevance The Program was and remains very relevant in assessment of the situation of the province in terms of poverty and v ulnerability, and education provision, student learning scores, school infrastructure and gender parity. However, not all Program activities were well thought out. The Program proposal assumed that girls' education is not important for parents in Balochistan, which is

>

Program proposals should be stronger in terms of needs of the province and should address the area of where best to focus the funds to get value for money. The important and political issue of teacher absenteeism

97

factually not proven. The issue of teacher absenteeism is not discussed in the proposal although the addressing of th is issue was included in the actual implementation through SC and PTSMC interventions. Enabling children to transition from primary to middle and middle to high was and remains very relevant. However, student testing and learning is an area t hat is not addressed adequately in the proposal or in the actual interventions, although some efforts have been made, for instance in the area of mother tongue, literacy boost and numeracy boost, as discussed in a later chapter. BEP planning and techniques needed honing in the area of quality education and development of cognitive skills. Although PITE was fully involved, the Provincial Educational Assessment Center (PEACe) was not involved in assisting in diagnosing issues in student learning. The Program team made considerable efforts to keep on board the changing government officials and steering committee meetings were held regularly to facilitate Program bottlenecks.

>

should be addressed in all education programs in the province. The existing Provincial Education Assessment System (PEACe) should be strengthened and involved for better quality diagnostic information on student learning outcomes, and subsequent development of learning solutions, and to be able to offer further successes in learning outcomes to other schools in Balochistan. Greater focus should be given to software aspects and quality of education.

Effectiveness Overall, effectiveness of the Program is deemed quite satisfactory. The baseline survey was undertaken to inform project design but was slow in being realized. It was also not designed for evaluation against the initial status of the area. The quality of monitoring is not fully adequate since the logframe has n ot been presented cumulative for each year; that is achievements against activities in the logframe are not available i n one document but in biannual and annual r eports only for the time under review. There was some effe ct on adequate monitoring since t he SCNL program manager was of ten not able to visit Quetta in the latter half of the program due to security concerns. The capacity of the implementing partners was fairly strong for the most part, and they were further trained by SC in program management and financial management. A weakness was that implementing and supervising construction

The hiring of a construction firm to manage and supervise construction, with IPs acting as facilitators between the firm and the community, should have been undertaken for greater effectiveness and timeliness. Building alliances with other NGOs, INGOs and external donors working In Balochistan should be undertaken to better advocate on education reform and propagate Program learnings. Monitoring of the project should be cumulative for

98

activities was not their field of expertise. However, perhaps given the restricted capacity in the province it was not possible to commission an implementing partner with more solid expertise in supervision of construction. Once government was more fully involved, the construction aspect moved forward rigorously, which was a lesson learned for the Program. Programming of inputs was sequential and logical except that the construction effort did take up too much time in the first 3 years of the Program. Minimum quality standards were met in terms of monitoring of construction activities. Risk analysis and meetin g of subsequent challenges has been fairly strong and a fair amount of training has been a dministered for disaster risk reduction (DRR) to child clubs and community members, as well as related kits provided. Communities have been strongly mobilized in education delivery. However, they remain dependent on external financing, government or donors, to be able to play an active role in the PTSMCs. Hence, it is a lesson le arned for future implementation that expectations regarding the financial contributions of community members be based on what is realistically possible for poor communities to contribute. The implementing partners have been effective in transparency and information management. Training programs for teachers, PTSMCs and community members were delivered in a timely and effective manner. The partnerships with government partners such as PITE have been strong with the latter incorporating many of the training techniques into their own programs, especially those related to ECE.

each year

Efficiency Program efficiency is deemed satisfactory especially given the Balochistan situa tion in terms of security and the costs associated with traveling large distances and pr oviding services thereof. The procurement policy is well defined as are other financial systems. The Program's coping mechanism in the face of insecurity has been

99

relatively strong. The program has also responded well to changes in the political environment thus providing complementary support for the greater responsibility given to the province due to Article 25-A o f the 18'^ Amendment to the Constitution. Budgetary expenditures for the Program are quite efficient, as are financial controls and financial monitoring, at least from the head office perspective. The field office was not able to provide information although it is assumed to be positive since their information feeds into the head office. The procurement policy is well defined and procurements are timely per the information provided for 2014. Information on the quality of the procured items was not provided, although no complaints were raised by PTSMCs or community members either. The financial expenditures for the Government's education budget are also fairly efficient although advocacy by the BEP team was an area that was made redundant midway through the program given that other donors were working on this intervention. One aspect that could have provided further information was a linking up of the baseline survey with a cost-b enefit analysis to better determine value for money for the various activities of the Program.

Impact The impact of the Program has been significant on many fronts and has had notable impact. Girls' enrolment has increased dramatically, as has the opportunity for girls to continue on to higher levels of education beyond primary and middle, addressing a key weakness in the system thus far. Children's awareness about schooling and their rights has been enhanced greatly, a significant outcome of the Program. Teachers overwhelmingly agree that the Program has encouraged girls' schooling, and that the most important factors have been training and the focus on ECE. The Program's many innovations have had demonstrably good impact such as participatory techniques in teaching; participation by

Training on child rights, DRR and other areas should be provided to district education managers since they are key government officials responsible for district education and in a position to bring the learnings to other schools in the province. All the components of a successful ECE program should be thoroughly discussed with all government counterparts

100

community members, teachers and children in Program implementation; the ECE program of provision of classrooms, materials, teachers and training, as well as other initiatives such as the Literacy Boost, the Numeracy Boost and the Mother Tongue Pilot, which initiated considerable discussion with Government counterparts. Training on peace education was another major innovation for the provincial context. Although the culture of child abuse has been ingrained in schooling and even in homes, the Program made considerable headway in raising awareness about child rights among community and PTSMC members, and teachers and children. The survey results are quite significant in these areas. The deep roo ted culture of abuse of children has not been e liminated, but the Program has gone a long way towards enhancing the awareness of teachers, children, community members and members of PTSMCs on the rights of children, including their right to education and to not be subjected to corporal punishment and o ther forms of abuse. A ma jority of teachers expressed that they were against corporal punishment in schools, and that it should be banned. Significantly, children approach teachers for assistance when being bullied in schools, and receive a positive response, in both BEP and other schools. The success or otherwise of the child clubs remains a somew hat grey area; in the qualitative interviews the members express positively about girls education, child rights, health issues, and others. However, the majority of children said their participation was not voluntary, whereas 11 of the 12 teachers interviewed said that children's participation was voluntary, a clear contradiction. The children were positive about the child friendly construction and attractive classrooms f or ECE. Child health has im proved in terms of awareness about good practices and the communication gap between mothers and adolescent girls has been decreased, a concrete

>

and other stakeholders at the initial stages o f program intervention. Efforts should be made that the key area of ECE (and other) teacher recruitment be addressed and implemented up front. Save the Children should continue with efforts to disseminate the solid good practices and learnings of the Program to both national and international audiences.

101

achievement. The E CE model has obtained r enown and established a solid platform for further realizations in Balochistan, with government especially supportive due to the close invo lvement of PITE. Save the Children has developed and implemented the blue print which forms the basis for the Government of Balochistan's Education Project which is funded through the Global Partnership for education (GPE), as well as a blueprint for many important components of the Balochistan Education Sector Plan (BESP), especially in the area of ECE.

Sustainability The Program is deemed sustainable based on the many policy influences and the model realized by the ECE program. There is financial space in Balochistan's budgetary situation due to additional funding from the National Finance Commission Award, as well as large proposed do nor projects in education. The education development budget has been fairly substantial for the last two years at 23% of the total provincial budget. The one important area that was unable to be adequately addressed was r ecruitment of teachers due to delays on the side of the government, which makes it difficult to uphold the advances made in terms of development of ECE centers and up-gradation of schools until the posts are filled. Until this is done, the upgraded schools will not be functioning efficiently and will not be sustainable. The role of all implementing partners as members of th e Local Education Group, which provided technical oversight for the development of the Balochistan Education Sector Plan, lends depth to the Program's sustainability. The incorporation of ECE in the Plan as well as acceptance by Provincial Institute for Teacher Education (PITE) of ECE techniques and related training provides further support for sustainability. There has been increased knowledge of implementing partners, all Balochistan NGOs expected to continue work in Balochistan's education system, w hich is p ositive.

> It is strongly recommended that Save the Children continue education interventions in Balochistan, especially in the ECE subsector, including through provision of technical support to the Policy Planning and Implementation Unit of the Department of Education.

> The many strong partnerships developed with PTSMCs, community members, government officials, and implementing partners should be continued and the means found to nurture these partnerships in the future.

> The Government of Balochistan should fill the vacant positions of teachers in order to enable the ECE centers and the upgraded schools to keep functioning and ensure quality and sustainability.

> The Government should review the work of the

102

The discontinued funding of PTSMCs is another area where communities will simply not be able to cover the expenses. However, a formal role for PTSMCs has been defined in new government policy in 2014 concerning the right to education, which lays the groundwork for government funding of the committees.

PTSMCs and consider their funding through provincial resources.

XL Lessons Learned and Good Practices

fntegt iition of Midterm Evaluation results into the Program The paragra phs headings in this section refer to the issues that were raised in the midterm evaluation (quoted in italics) followed by the progress made against those areas in these last two years o f the Program. The progress demonstrates that the midterm evaluation results were seriously considered and acted upon by the BEP team.

Section 3.1 Linear Approach: Concern that essence of child clubs and discussions on corporal punishment not being explained and more a number crunching exercise. (Also see section 4.2 below)

After the findings of the midterm evaluation the number of child clubs was reduced from 498 to 100 to ensure quality and increased pa rticipation of children. It was also decided that the remaining 100 child clubs would be fo r older children in grade 5 and above in order to disseminate the child rights messages more effectively.

Section 3.2 Teacher appointment: Most of the teachers who have been notif ied by the government are either near retirement or are not as agile and energetic as an Early Childhood level teachers need to be to facilitate a sizeable number of students. What is the status of teacher appointment currently and how did IPs and Quetta office deal with this.

This concern has b een raised with the Government and directives have been issued to post newly recruited staff under the Government of Pakistan's Balochistan Package in BEP focused schools.

Section 3.3 Technical, Leadership and Management Inputs: BEP does not have any vision for teacher education component. The focus is very activity and output driven where the follow-up and support mechanisms have not been figured out. Similarly it is not dear whether all 1100 teachers t o be catered through project will receive another round of training/refresher or not. Also, the quality of technical inputs on part of SC Quetta Program Office (QPO), Country Off ice or SC NL also seems to be minimal in designing the child-club program and its purpose (section 4.2 also)

All teachers were provided with refresher courses along with the on job support. In the second part of the program there was increased focus on quality and learning since there had been more focus was on the construction of schools during the first phase. The country office Program Development and Quality (PDQ) team not only trained the teachers but also helped build the capacity of PITE on early literacy and

103

math so that the knowledge can be further replicated to all the government teachers during the implementation of the BESP. Additionally, RITE was closely involved by the project team in designing and conducting subject based trainings for primary and secondary grade teachers.

SC QPO, Country office and SCNL decided to focus on quality child clubs; as a result the number of child clubs was reduce d and it was decided to work only with older children and extra-curricular activities were added to the child club program.

Section 3.4 Partnership dynamics and power play: How have all parties managed to work together and resolve iss ues that were raised in the mid-term evaluation about covert competition, etc. (Also see section 4.4 below)

A robust co mmunication mechanism was placed where a weekly Skyp e meeting with SCNL and country office and monthly review meetings with SC QP O and partners were regularly conducted. Detailed implementation plans were shared by IPs each month and technical support was provided accordingly. IP's and SC QPO worke d on the ground as a team which was a positive and encouraging experience for the entire team.

Section 3.5 Lack of Monitoring and E valuation: Please explain why monitoring is considered strong and enhanced, especially in these last two years after the mid-term evaluation.

Since the roll-out of the Monitoring, Evaluation, Accountability and Learning (MEAL) approach in Pakistan country program the monitoring and evaluation (M&E) aspects of BEP were also upgraded wit h the addition of accountability and learning functions, respectively. Monitoring visits were undertaken to monitor process and outputs using defined quality standards and feedback from communities and children was documented through complaint response mechanisms. Monitoring and accountability findings were used for course corrections. In addition to strengthening Save the Children MEAL team, all staff including M&E officers and focal points was trained through a series o f trainings and on the job support. After MEAL data was collected during the first two years, SC created an MIS position along with MEAL to effectively use the data to improve program results.

Section 3.6 Conversion of Experience i nto Knowledge and Str ategic insights: With massive inputs and achievements in both strategic and absolute ways. It Is rather curious as to why nothing on BE P is shared with the larger audiences. (Also see section 4.3 below)

Policy dialogues, seminars and coordination meetings were conducted for the purpose of sharing the achievements of BEP with the stakeholders to further reach larger audiences. Supplementary pages and newspaper articles were also published in reputed newspapers, specifically in Balochistan. Each year the Literacy Boost results were shared through policy dialogue with stakeholders. The m other tongue bill was pas sed in Balochistan and the SC sponsored M other Tongue Based-Multilingual Education (MTB-MLE) results played a key role in that. SCNL developed a 4 page Fact Sheet in January 2014 that was widely distributed in the Netherlands and Pakistan. SCNL twice published a full page on the BEP program in one of the prominent national newspapers in The Netherlands, to positive feedback from readers.

Section 4.1 Focus on Teachers' Capacity fo r ECE Impiem entation: It Is highly recommended that SC takes back the charge of ECE Intervention and design a program which is based on NCECE curricula as

104

well as Its own early years' education program. SCSPEB, IDO and PITE may be engaged In the follow up of primary, middle and secondary school teachers who have received already received trainings.

The Balochistan Government has developed the ECE policy frame \work and SC has been actively participating and contributing in the whole process. The E CE log frame , a s ubstantial part of the policy frame work, has been developed by SC on request of the Government and has been included in the BoC.

Section 4.2 Quality of program for children: It Is recommended that SC handholds SEHER In developing a more comprehensive youth engagement program.

To ensure comprehensive engagement of youth, the number of child clubs was reduced to 100 and only middle and high schools were focused. It was not under consideration in the project proposal to focus on a larger youth engagement program. However, for high school children, an additional ICT based training was conducted so that teenagers can use ICT in a positive and productive way.

Section 4.3 High visibility in policy planning circles: If BEP is serious about achieving its overall objectives (not only quantitative targets) and creating models for large scale educational reforms, then It Is Imperative to establish its identity and visibility in the core policy circuits of Government of Balochistan namely PPIU, Secretariat and Directorate. This should be independent of the regular donor coordination meetings or initiatives spearheaded by UNICEF.

Regular quarter ly review and coordination meeting were conducted with government departments. To further strengthen coordination with government stake holders, a delegation comprising of the Education Advisor, Additional Secretary and Director of Education were taken on an exchange visit to the Netherlands. As a result, there are direct and regular contacts with government officials in both countries. The Education Department expended the facility of the SC pro ject proposal for upscaling quality education provision in the Province, based on th e successful results achieved. The proposal for the province, developed through BEP, has been approved by th e GPE with additional funds from EU for the coming 3 years.

Section 4.4 Communication and information generation: BEP does not have a communication strategy or a mechanism through which It generates regular information on program activities especially at school level.

All School Improvement Plans (SIP) were developed by the school management committees and monthly review meetings were regularly conducted with school management where program interventions were shared. Ext ra-curricular activities also contributed to regular information at school level through the years.

.Alliances with stakeholders and flexibility in programming Major effort is needed to build and maintain alliances with government and other development players, which should be a fac tor considered in future development interventions. The close co nsultation with government minimized the risks e ncountered by the Program and allowed for flexibility in adjusting programmatic interventions when needed.

105

1 n n u e II c i n g go ve r n ni e n t Solid work, especially in ECE construction, training and involvement of government trainers has allowed the Program to influence current and future government programs especially in this subsector.

106

Annex 1. Persons interviewed Government of Balochistan Tayyab Lehri, Additional Secretary, Education Department, Balochistan Secretariat, Quetta Gulab Khan, Director School Education, Directorate of School Education, Education Department, Balochistan Secretariat, Quetta Akbar Zehri, Director Provincial Institute of Teachers Education (PITE), E ducation Department, Sariab Road, Quetta Muhammad Hanif, District Officer Education, Quetta Syed Abdul Kaleem, District Education Officer, District Killa Abdullah Muhammad Anwar Shahwani, District Education Officer, District Mastung

Implementing Partners Innovative Development Organization (IDG), Quetta Inam Ul Haq, Director Programs Nouman Sher, Project Coordinator Faisal Baqi, Admin & Finance Manager Society for Empowering Human Resources (SEHER), Quetta Abdul Wadood Khan, Executive Director, Society for Community Strengthening and Promotion of Education, Balochistan (SCSPEB) Irfan Awan, Managing Director Shabir Ahmed, Manager Programs Abrar Ahmed, Program Coordinator (BEP- SCSPEB) .

Donors Embassy of the Kingdom of the Netherlands, Islamabad Saadat Ali, Senior Polic y Advisor, Political Affairs, Human Rights, VVS World Bank, Islamabad Umbreen Arif, Senior Education Specialist, GEDDR

Implementing organization Save the Children, Pakistan and Netherlands Brum Burki, Technical Director, Education Lia van NIeuwenhuijzen, Program Manager / Education Specialist (SC Netherlands) Shahzad S. Mithani, Senior Director, Program Development and Quality Yasir Majeed Waraich, Manager, Monitoring Evaluation Accountability and Learning (MEAL)

107

Annex 2. References Alif Ailaan, 2014, Education Report for Pakistan

Annual Status of Education Report (ASER), Pakistan Report, 2013

Aziz, Mehnaz, August 2012, Early Childhood Education (ECE) - Review and Analysis of the Balochistan Education Program

BDO Ibrahim and Company Chartered Accountants, BEP Audit for the year ended December 31, 2013

Daily Balochistan Express, Decembers, 2014,, "Balochistan has 85,000 disabled children"

Embassy of the Kingdom of Netherlands (EKN), May 2011, Comments on Baseline Survey

Gazdar, Maris, 2007, Balochistan Economic Report

Government of Balochistan, February 26, 2014, N otification for the Balochistan Compulsory Education Act

Government of Balochistan (GoB), Department of Education, School Census, 2010-11 and 2013-14

Government of Balochistan, Department of Education, Balochistan Education Sector Plan (BESP) 2013

Government of Balochistan, Department of Education, PPIU, Balochistan Education Sector Plan 2013-2018

Government of Balochistan, Department of Education, PPIU, September 2014, Policy and Right to Education (RTE): Status of Early Childhood Education in Balochistan

Government of Balochistan, Department of Finance, Budget 2013-14 and 2014-15

Government of Balochistan, Department of Finance, White Paper, Budget 2014-15

Government of Balochistan, Policy Planning & Implementation Unit (PPIU), September 16, 2014, Policy and Right to Education (RTE): Status on Early Childhoods Education in Balochistan. Presentation

Government of Pakistan, Pakistan Bureau of Statistics, Statistics Division, May 2013, Pakistan Social and Living Standards Measurement Survey (PSLM) 2010-11 and 2011-12

Khanum, Asifa, Nove mber 2011, Education and Protection of Children in Pakistan - Baseline Survey Findings

Pakistan Poverty Allev iation Fund, 2013, Balochistan Strategy .. . a road ma p to inclusive socio-economic development for the province

Amima Sayeed, February 2013, Balochistan Education Program, Mid-Term Evaluation Report

Save the Children (SC) and Children's Global Network Pakistan, 2012, Early Childhood Education (ECE) Strategy for Balochistan

SC and Huma Khan, 2011, Framework for Action: Provincial Institute for Teacher Training, Balochistan

SC Education, Program Development and Quality Unit, July 2013, Early Childhood Education Training: Five Days Training on Early Childhood Education Teaching Methodologies, Islamabad

SC Pakistan, 14'^ Program Coordination Meeting, October 15, 2014

SC Pakistan, Finance Department, BEP Budget and Expenditures (excel), November 2014

SC Pakist an, Grants Financial Performanc e Overview, Program Coordination Meeting, October 14-15, 2014

108

SC UK Pakistan Program, April 1, 2005, The Procurement Process

SC, (undated). Finance Manual

SC, 2011, BEP-SHN Baseline Survey Report - Balochistan

SC, B EP and SEHER, 2011, Training Manual for Child Club (CC) Life Skill

SC, BEP Log frame planning and reporting, November 2013

SC, BEP, 2013, Final Child Clubs Research Study

SC, BEP, 2014, Nutritional Policy Guidance Note for Balochistan

SC, BEP, Annual Report 2011

SC, BEP, Annual Report 2012

SC, BEP, Annual Report 2013

SC, BEP, April 2012, Report: Exposure Visit of Government Officials to Education Reforms in Punjab

SC, BEP, Balochistan Education Management Information System (BEMIS) data 2011-12 and 2012-13

SC, BEP, Bi-Annual Report, Dec-Jun 2010

SC, BEP, Bi-Annual Report, Jul-Dec 2011

SC, BEP, Bi-Annual Report, Jan-Jun 2014

SC, BEP, Bi-Annual Report, Jan-June 2012

SC, BEP, Construction Strategy 2010-2015

SC, BEP, ECE Analysis, 2012

SC, BEP, ECE Refresher Report 2014

SC, BEP, November 2013, Logframe planning and reporting

SC, BEP, GGPS Muslim Ittehad Colony Success Story, Sep 2012

SC, BEP, l-SAPs, 2013, Public Financing of Education, Balochistan chapter

SC, BEP, Minutes of Steering Committee Meetings

SC, BEP, Narrative Report as sent to EK N, 21-03-2011

SC, BEP, Revised Concept Note of BEP submitted to EKN 2010

SC, BEP-SHN, November 2014, End-line Survey Draft Report

SC, BEP, June 2013, PTSMC Training Manual

SC, BEP, June 2014, Literacy Boost Flyer

SC, MEAL Unit, Operational Research on Effectiveness of Parent Teacher School Management Committees (PSMC) in the BEP Project

BEP, SCSPEB, August 26, 2014, Monitoring Reports of WATSAN and Missing Facilities Works

SC, Office of General Counsel, October 2013, Zero Tolerance Fraud Policy

SC, RNE Education Proposal 2010

109

SC, Save the Children Employee Manual for Expatriate and Third Country Employees, Chapter XXXV -Procurement Policy, Oct 1, 2011

SC, September 2012, Succ ess Story of Government Girls' Middle School, BMC c olony. Brewery Road, Quetta

SC, BEP, April 2014, Water Supply Detail (excel sheet)

SC, BEP, August 2014, Work Plan for NHA

SDPI, October 2012, Clustered Deprivation: District Profile of Poverty in Pakistan

SEHER, June 2013, Training Manual for Parent Teacher School Management Committee (PTSMCs)

SOCIETY, November 2014, BEP End Event Ceremony

Strengthening Participatory Organization, April 2013, Situation Analysis of Education System in Balochistan

UNICEF, 2004, MICS Report

USAID, September 2012, ED-LINKS Evaluation Report

Annex 3. Submissions on BEP by Implementing Partners Provided in online drop box

Annex 4. Quantitative questionnaires Provided in online drop box

Annex 5. Qualitative questionnaires Provided in online drop box

Annex 6. Consolidation and analysis of qualitative questionnaires Provided in online drop box

110

Annex 7. Survey Data Collection

District Quetta FGDs

Sr. No. BEMIS Code

School Name (Quetta)

Gender Level ORG/1

P DIstrIc

t UC

Schedule/date of Survey

Students q're

Teachers q're

School Observation

Infarastru cture

PTS MC

CI ub

Comm unity

1 3464 GGHS Kllll Kateer

GIRLS High

SCSPE B

Qta KUCHLA

K 21-10-2014

20 2 1 1 0 1 0

2 11142 GGPS kill Attozai

GIRLS Primar

Y 100 Qta Kuchlak 21-10-2014

20 1* 1 1 0 0 0

3 5135 GBMS Kllll Essa Khali

BOYS MIddl e

SCSPE B

Qta KUCHLA

K 22-10-2014 20 2 1 1 1 0 0

4 5030 GBPS kllll Malik A.AII

BOYS Primar

Y 100 Qta Kuchlak

22-10-2014 20 2 1 1 1 1 0

5 5061 GBPS kllll Malik Shah Jahan

BOYS Primar

V 100 Qta Kuchlak

22-10-2014 20 2 1 1 0 0 0

6 3410 GGMS Rlsaldar

GIRLS MIddl e

SCSPE B

Qta SHAOEN ZAI ONE 23-10-2014 20 2 1 1 1 0 1

7 9261 GGMS Amin Abad

GIRLS MIddl e

SCSPE B

Qta SHAOEN

ZA! 23-10-2014 20 2 1 1 1 0 1

8 4847 GBPS Kllll Gulzar

BOYS Primar y

SCSPE B

Qta SHAOEN

ZAI 29-10-2014 20 2 1 1 1 0 0

9 3371 GGHS Sarlab Mill

GIRLS High

SCSPE B

Qta KACHI BAIG 24-10-2014 20 2 1 1 0 0 0

10 14089 GGMMCS kllll Badlnl

GIRLS MIddl e

100 Qta KachI Balg

21-10-2014 20 2 1 1 0 0 0

11 Non BEP Kllll Shah Naws

GIRLS Primar

V Qta

KachI Baig 24-10-2014 20 1 1 1 1 0 1

12 10318 GGMS PIrkanI Abad

GIRLS High lOO Qta

KachI Balg 25-10-2014 20 2 1 1 1 1 0

13 2969 GBMS Kllll BOYS MIddl SCSPE Qta KAICHI 24-10-2014 20 2 1 1 0 0 0

111

Shah Nawaz e B 8AIG

14 4952 GBPS Muslimabad

BOYS Primar

V IDO Ota

KachI Balg 25-10-2014 20 2 1 1 0 0 0

15 3446 GGPS Muslim Abad

GIRLS Primar

Y IDO Qta

Skelkhm anda 27-10-2014 20 2 1 1 0 0 0

16 14607 GGPS Chilian Scheme

GIRLS MIddl e

IDO Ota Balell l-Nov-14 20 2 1 1 0 0 0

17 4813 GBPS kllll Khan Mohd

BOYS Primar

V IDO Qta Kotwai

27-10-2014 20 2 1 1 0 0 0

18 3388 GGPS Tayaraa Road

GIRLS Primar

V

SCSPE B

Qta GOL

MASJID 27-10-2014 20 2 1 1 0 0 0

19 12471 GBPS CGS colbny

BOYS Primar

Y IDO Qta

Satallte town 27-10-2014 20 2 1 1 1 0 0

20 9258 GGMS kllll Keyazal

GIRLS MIddl e

IDO Qta Barori 28-10-2014 20 2 1 1 0 0 1

21 13271 GGPS Kllll Schrab

GIRLS Primar

Y

SCSPE 8

Qta QAM BR

AM 28-10-2014 20 2 1 1 0 0

22 Non BEP New Muslim Abad

GIRLS Primar

Y Qta

25-10-2014 20 1 1 1 0 0 0

23 3385 GGMS Or Banc Road

GIRLS MIddl e

IDO Qta linnah Road 28-10-2014 20 2 1 1 1 0 0

24 3441 GGHS kllll kasi GIRLS

High IDO Qta KasI Road 28-10-2014 20 2 1 1 1 0 0

25 4970 GBMS Yet Road

BOYS MIddl e

IDO Qta Yat Road 31-10-2014 20 2 1 1 0 0 0

26 3334

GGHS Kllll alam Khan( Kareez Mushtarlqa)

GIRLS High

SCSPE B

Qta 42

30-10-2014 20 2 1 1 1 0 1

27 3402 GGPS Sarazawar

GIRLS Primar

V

SCSPE B

Qta 65 30-10-2014 20 1 1 1 0 0 0

28 3358 GGPS Dewar GIRLS Primar IDO Qta KIranI 29-10-2014 20 2 1 1 1 0 0

112

Colony Y

29 4940 GBPS killl Shadezal

BOYS Primar

V 100 Qta 65

31-10-2014 20 2 1 1 0 0 1

30 Non BEP Hana Urak BOYS

High Qta Hannah 31-10-2014 20 1 1 1 0 0 1

31 9621 GGM5 Sidlq abad

GIRLS MIddl e

IDO Qta Pashtoo

n Bag 29-10-2014 20 2 1 1 0 0 0

32 5021 GBPS Srakhula BOYS Primar

y IDO Qta

Zarghoo n 30-10-2014 20 2 1 1 0 0 1

33 Non BEP SIddqa Abad BOYS Primar

V Qta

29-10-2014 20 1 1 1 0 0 0

34 4919 GGP5 Durrani Bagh

GIRLS Primar

V

SCSPE B

Qta 39GOL MASJIO 29-10-2014 20 2 1 1 0 0 0

35 5102 GBP5 Muslim Itehad Colony

BOYS Primar

V 100 Qta

Lowar karez 29-10-2014 20 2 1 1 1 0 0

36 Non BEP TB Santorim BOYS

High Qta Shadezai 28-10-2014 20 1 1 1 1 0 0

37 Non BEP Kakar Colony BOYS Primar

V Qta 39

27-10-2014 20 1 1 1 0 0 0

38 Non BEP

Killl Fathe Bahg Qumbarnl Road

GIRLS Primar

Y Qta

l-Nov-14 20 1 1 1 1 0 1

39 Non BEP Kachl Balg BOYS

High Qta Kachl Baig l-Nov-14 20 1 1 1 1 0 0

40 Non BEP Zarghoon Town

BOYS Primar y

Qta Zarghoo

n 30-10-2014 20 1 1 1 0 0 0

41 Non BEP Nawa Kllli Bar! BOYS Primar

y Qta

Nawa Killl 30-10-2014 20 1 1 1 0 0 0

42 Non BEP Killl Gulam Nabi

BOYS Primar y

Qta 65 31-10-2014 20 1 1 1 0 0 1

43 Non BEP Sandeman BOYS High Qta Yat Road 31-10-2014 22 1 1 1 0 0 0

113

<J4 Non BEP Killi Shekhan BOYS Middl

e Qta 47

30-10-2014 20 1 1 1 0 0 0

45 Non BEP Nawa Killi BOYS Primar

Y Qta

Nawa Killi l-Nov-14 20 1 1 1 0 0 0

46 Non BEP Chaltan Sceem

GIRLS Middl e

Qta 51 l-Nov-14 20 1 1 1 0 0 0

47 4904 Technical School (non-BEP)

BOYS High Qta 17

6-NOV-14 20 1 1 1 0 0 0

Sub Total District Killa Abdullah (Quettal 942 75 47 47 17 3 10

Sr. No. SEMIS Code

School Name Gender Level ORG/1

P Distric

t UC

Schedule/date of Survey

Students q're

Teachers q're

School Observation

Infrastruc ture

PTS MC

CI ub

Comm unity

1 1420 GBPS Tor Khail Saydan

BOYS Primar

V

SCSPE B

K.A Pir All zai 22-10-2014 20 2 1 1 1 0 0

2 22 GBPS Hassan Tahkedar

BOYS Primar

y SCSPE

B K.A

Purana Majaak

22-10-2014 20 2 1 1 0 0 0

3 1478 GBHS Mazai BOYS

High SCSPE

B K.A Mazai 23-10-2014

20 2 1 1 1 0 1

4 Non BEP Mazai Adda BOYS High K.A Mazai 24-10-2014 20 1 1 1 0 0 1

5 Non BEP

Dad Muhmmeed Majk

BOYS Primar

y K.A Mazai 25-10-2014

20 1 1 1 1 0 0

6 12249 GGMS Sharif Masazai

GIRLS Middl e

SCSPE B

K.A Maizai 23-10-2014 20 2 1 1 0 0 0

7 13080 GGMS Abdul Rehman 2ai (Dr. Akbar)

GIRLS Middl e

SCSPE B

K.A Abdul

rehman zai

23-10-2014 20 2 1 1 0 1 0

8 Non BEP Killi Haider Masezai

BOYS Primar

V K.A Mazai 25-10-2014

20 1 1 1 1 0 1

114

District Mastune Sub Total (Killa

Abdullah) 160 13

Sno BEMIS School Name Gender Level ORG Oistric

t UC

Schedule/date of Survey

Students q're

Teachers q're

School Observation

Infrastruc ture

PTS MC

CI ub

Comm unity

1 10467 GGPS Koshkak GIRLS Primar

Y IDO Mstng

Sheikh Wasil

22-10-2014 7 0 1 1 0 0 1

2 Non SEP Arsada Abad GIRLS GIRLS

Mstng Sheikh Wasil

22-10-2014 10 1 1 1 0 0 0

3 11312 GGPS Karez Kalan

GIRLS Primar

V IDO Mstng Sorgaz 23-10-2014

20 2 1 1 1 0 1

4 2996 GBMS Abad BOYS Middl

e IDO Mstng

Ghulam Prinz l-Nov-14 20 2 1 1 0 0 0

5 2895 GBMS Killi Sheikhan

BOYS Middl e

IDO Mstng AliZai 23-10-2014 20 1 1 1 0 0 0

6 2879 GGPS Gazgl GIRLS Primar

Y IDO Mstng

Mastung 11

29-10-2014 20 2 1 1 1 1 0

7 3050 <5GPS Tindlan GIRLS Middl

e IDO Mstng

Mastung 11

29-10-2014 20 2 1 1 1 0 0

8 NB/2852 GBMS Killi pirkano

BOYS Middl

e Mstng Mastung

il 5-NOV-14 20 1 1 1 1 0 1

9 2896 GGPS Sor Gaz

GIRLS Primar

V 100 Mstng Sorgaz 31-10-2014

20 1 1 1 0 0 0

10 2870 GBHS Pring Abad

BOYS High IDO Mstng Sorgaz 30-10-2014

20 2 1 1 0 0 0

11 2876 GGHS Pring Abad

GIRLS High 100 Mstng Sorgaz 30-10-2014

20 2 1 1 0 1 0

12 Non BEP Kllll Noor muhmmed

GIRLS High Mstng Sorgaz 30-10-2014

20 1 1 1 0 0 0

13 Non BEP Ghous Abad GIRLS Middl

e Mstng Sorgaz 31-10-2014

20 1 1 1 0 0 0

Sub Total (Mastune) 237 18 11 4 2

115

TOTAL 1339 106 ^ ' 68 25 6 ': ^16

' Only 1 teacher available - two visits undertaken to confirm (note: few BEP schools only had 1 teacher)

Discussions with other stakeholders Discussion with Education Department (Additional Secretary and Director Schools) held on 5th November 2014

Discussion with IP - IDO held, on the 5th November, 2014

FGDs with three District Education Officers held (6th November 2014) Discussion with SCI (Balochistan) held on 6th November 2014 (Provincial Director, Manager Operations, Project Manager BEP, Infrastructure Coordinator, MEAL Coordinator, Finance & Grants Manager

Discussion held with IP-SOCIETY on 7th Nov., 2014

Discussion with SEFIER, held on 8th Nov., 2014

116

Annex 8. Procurement orders and dates received for 2014

Date of procurement (PO) order

Date goods required Date delivered Difference in required and received dates (in days)

Books and educational materials 07/04/14 Not mentioned n/a

7/11/2014 Not mentioned 19/08/2014

Construction material 7/15/2014 Not mentioned 28/11/2014

IT equipment 06/27/14 Not mentioned 06/08/2014

8/20/2014 Not mentioned N/A

8/25/2014 Not mentioned 18/09/2014

9/17/2014 9/17/2014 19/09/2014 2

9/17/2014 9/18/2014 N/A

9/29/2014 9/30/2014 28/10/2014 28

10/14/2014 10/15/2014 28/10/2014 14

10/22/2014 10/23/2014 30/10/2014 7

11/12/2014 11/17/2014 19/11/2014 2

11/17/2014 11/20/2014 N/A

Stationery 06/18/14 Not mentioned 18/06/2014

06/19/14 06/23/14 20/06/2014 -3

06/24/14 06/25/14 25/06/2014 0

07/04/14 Not mentioned 07/07/2014

7/9/2014 Not mentioned 09/07/2014

117

8/5/2014 Not mentioned 06/08/2014

8/11/2014 Not mentioned 12/08/2014

8/13/2014 Not mentioned 19/08/2014

8/18/2014 Not mentioned 19/08/2014

8/22/2014 .Not mentioned 02/09/2014

8/27/2014 Not mentioned 02/09/2014

8/29/2014 Not mentioned 04/09/2014

8/29/2014 Not mentioned 06/09/2014

8/29/2014 Not mentioned 03/09/2014

9/15/2014 9/17/2014 16/09/2014 -1

9/18/2014 9/18/2014 19/09/2014 1

9/25/2014 9/26/2014 26/09/2014 0

9/26/2014 9/27/2014 26/09/2014 1

10/13/2014 10/18/2014 15/10/2014 -3

10/22/2014 10/23/2014 24/10/2014 1

11/7/2014 11/8/2014 11/11/2014 3

11/10/2014 Not mentioned 11/11/2014

11/12/2014 11/19/2014 20/11/2014 1

11/14/2014 Not mentioned 25/11/2014

11/17/2014 11/20/2014 18/11/2014 2

Visibility materials 8/15/2014 Not mentioned 22/08/2014

8/21/2014 Not mentioned 20/08/2014

118

8/29/2014 Not mentioned 09/09/2014

8/29/2014 Not mentioned 09/09/2014

10/13/2014 10/18/2014 17/10/2014 -1

11/12/2014 11/19/2014 19/11/2014 0

11/12/2014 11/19/2014 19/11/2014 0

11/12/2014 11/19/2014 N/A

Miscellaneous 06/30/14 07/02/14 30/06/2014 2

7/7/2014 Not mentioned 15/07/2014

7/7/2014 Not mentioned 08/07/2014

8/4/2014 Not mentioned 14/11/2014

8/20/2014 Not mentioned 19/11/2014

8/21/2014 Not mentioned 22/08/2014

8/21/2014 Not mentioned 03/09/2014

8/22/2014 Not mentioned N/A

8/25/2014 Not mentioned

8/25/2014 Not mentioned 12/09/2014

8/27/2014 Not mentioned 09/09/2014

8/28/2014 Not mentioned 29/08/2014

8/29/2014 Not mentioned 15/09/2014

9/15/2014 9/17/2014 17/09/2014 0

9/18/2014 9/19/2014 25/09/2014 6

9/18/2014 9/25/2014 25/09/2014 0

119

9/25/2014 9/5/2014 27/10/2014 52

9/26/2014 9/29/2014 03/10/2014 5

9/26/2014 9/29/2014 29/09/2014 0

9/26/2014 9/29/2014 30/09/2014 1

10/22/2014 10/23/2014 11/11/2014 18

10/22/2014 10/23/2014 24/10/2014 1

10/23/2014 10/25/2014 28/10/2014 3

11/12/2014 11/19/2014 18/11/2014 1

11/17/2014 Not mentioned N/A

11/19/2014 11/20/2014 20/11/2014 0

11/19/2014 11/20/2014 20/11/2014 0

Total = 73 Not mentioned = 37 Not mentioned % = 51%

120

Amicx 9. Planned versus actual coverage of quantitative and qualitative data collection

Planned Sample Size (Quantitative) D

istri

ct

Num

ber

of

BEP

scho

ols

1

Sam

pled

# o

f B

EP

scho

ols BEP supported schools

# of

Con

trol

sc

hool

s

Non BEP / Control schools D

istri

ct

Num

ber

of

BEP

scho

ols

1

Sam

pled

# o

f B

EP

scho

ols

Tea

cher

in

terv

iew

s

Chi

ldre

n in

terv

iew

s

Scho

ols

obse

rved

# of

Con

trol

sc

hool

s

Tea

cher

s in

terv

iew

s

Chi

ldre

n in

terv

iew

s

Scho

ols

obse

rved

Quetta 237 31 62 620 31 16 16 320 16 Killa Abd. 39 5 10 100 5 3 3 60 3 Mastung 64 9 18 180 9 4 4 80 4 Total 340 45 90 900 45 23 23 460 23

Actual Coverage / Sample Size Surveyed

Dist

rict

Num

ber

of

BEP

seho

ols

1 i

Sam

pled

# o

f BE

P sc

hool

s BEP supported schools

# of

Con

trol

sc

hool

s

Non BEP / Control schools

Dist

rict

Num

ber

of

BEP

seho

ols

1 i

Sam

pled

# o

f BE

P sc

hool

s

Tea

cher

in

terv

iew

s

Chi

ldre

n in

terv

iew

s

Scho

ols

obse

rved

# of

Con

trol

sc

hool

s

Teac

hers

in

terv

iew

s

Chi

ldre

n in

terv

iew

s

Scho

ols

obse

rved

Quetta 237 31 60 620 31 16 15 322 16 Killa Abdullah 39 5 10 100 5 3 3 60 3 Mastung 64 9 14 167 9 4 4 70 4 Total 340 45 84 907 45 23 22 432 23

Planned Sample Size - Focus Group Discussions (FGDs)

BE? Schools Non-BEP/Control Schools

121

District PTSMCs Children Club

Community Total PTSMCs Community Total

Quetta 10 3 7 20 7 3 10 Killa Abdullah 2 1 1 4 2 1 3

Mastung 3 1 2 6 1 1 2 Total 15 5 10 30 10 5 15

Actual Coverage (FGDs) BEP Schools Non-BEP/Control Schools

District PTSMCs Children Club

Community Total PTSMCs Community Total

Quetta 13 3 7 23 7 4 11 Killa Abdullah 2 1 1 4 2 2 4

Mastung 3 2 2 7 1 1 2 Total 18 6 10 34 10 7 17

122