Improving School Quality in East Africa: Randomized Evaluation of Policies to Create Local Accountability under Free Primary Education in Kenya and Uganda Germano Mwabu University of Nairobi, Kenya Frederick Mugisha Economic Policy Research Centre, Uganda RESEARCH PROPOSAL Presented to PEP-AusAid Policy Impact Evaluation Research Initiative
22
Embed
Improving School Quality in East Africa: Randomized Evaluation of Policies to Create Local Accountability under Free Primary Education in Kenya and Uganda.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Improving School Quality in East Africa:Randomized Evaluation of Policies to Create Local Accountability under Free Primary Education in Kenya and Uganda
Germano MwabuUniversity of Nairobi, Kenya
Frederick MugishaEconomic Policy Research Centre, Uganda
RESEARCH PROPOSALPresented to
PEP-AusAid Policy Impact Evaluation Research Initiative
Overview Randomized controlled trial of institutional interventions in
the Kenyan and Ugandan primary education sector Project will take place in close collaboration with the Ministries
of Education (MoE) in both countries Both countries characterized by large increases in enrollment
following introduction of free primary education and (perceived) decline in educational ‘quality’
Research will focus on two key interventions Hiring local contract teachers to address shortage of
government teachers in Kenya, and funding locally determined projects in Uganda.
Community-Based Monitoring System (CBMS) Interventions correspond to the two challenges in FPE – input
shortages and lack of parental ownership.
Research Question Expansion of the primary education sector under FPE has
led to influx of new students creating shortage of teachers undermined the ability of schools to raise funding locally due to the
abolition of fees weakened community based monitoring of schools. (FACTS)
Main question: does local accountability require local fundraising or is strengthening CBMS sufficient? (question: what is local accountability)
Hypotheses in Detail Hiring additional contract teachers in Kenya will have a
significant effect on student learning achievements. (Either by lowering teacher-pupil ratios or changing the nature of the contract.)
Providing school lunches in Uganda will have a significant effect on student attendance and learning.
Giving parents a financial stake in the school (soliciting co-funding to hire contract teachers) will increase local accountability and community monitoring relative to pure top-down funding.
Informing school management committees of parents’ rights and teachers’ responsibilities and providing them with tools to monitor school performance will improve school management and student performance.
Community-based monitoring systems are particularly effective when parents have a financial stake in the school – i.e., there are complementarities between the interventions.
Description of Interventions School Management Committee Score Cards
completed by SMC on a termly basis, gathering information on…
teacher performance: teacher attendance, preparation of schemes and lesson plans, class room activities measured via direct class room observation and spot checks
financial administration: correspondence between budgeted and actual expenditures, appropriate input purchases, perceived wastage or inefficiency in budget use
School facilities and maintenance: hygiene and sanitation, classroom conditions and repairs
School meals (Uganda only) matching grants for locally funded school lunches Create cooperative decision-making across geographically
proximate schools
Description of Interventions Contract Teachers (Kenya only)
Provide co-funding to SMCs to hire contract teachers Addresses two issues:
Acute teacher shortage due to constraints placed on local fundraising by Free Primary Education
providing for contract teachers will test the importance of local control over teaching staff in creating accountability and quality service provision
Hypothesis: local contract teachers -- hired on a fixed term, renewable contract in which the SMC has control over hiring and firing -- will be more responsive to local demands
Research design: pop & outcomes Population to be studied:
196 Kenyan schools with a total of 3299 students, nationally representative, based on Southern and Eastern African Consortium for Monitoring Education Quality Survey (SACMEQ)
Ugandan sample is representative of rural primary school students in four districts selected to represent poor-performing districts of the 4 regions
Outcome of interest: Student achievement 9 months – 1 year after start of the
intervention: measured by national exams as well as custom-made literacy and numeracy exams (in conjunction with Kenya Institute of Education in Kenya project, Uganda National Examinations Board)
Changes in enrolment and transition to secondary school 9 months after start of the intervention.
Research design: randomization Control schools chosen randomly and simultaneously with the
treatment. In Kenya, use SACMEQ sample stratification so treatment
and control schools have equal representation from urban and rural areas. Successful randomization will be tested using pre-intervention information.
In Uganda, sample from primary data collected at district level. Schools representative of rural sub-counties, with ‘blocking’ of the sample based on pre-intervention test results to ensure representation of low-performing schools.
Use cross-cutting design: treatment group for each intervention assigned orthogonally
Difference-in-differences model exploiting the randomized design of the intervention, comparing mean changes in academic performance between treatment and control schools.
Research design: analytical framework Cross-cutting interventions to test hypotheses about
interactions For instance, is a CBMS program enough to increase accountability,
or is it primarily effective when parents have more control over teachers (through contract &/or fundraising)
If Y is individual exam performance, i indexes individuals and j indexes schools, TEACH and CBMS are treatment dummies then:
The Bx parameter will test the interaction between local ownership and community monitoring.
We plan to go further and test two levels of funding as follows:
ijtjtjtjtXjtCjtTijt wvCBMSTEACHCBMSTEACHY )(
Similarly for Uganda:No meals intervention
Meals intervention
No co-funding (full)
Co-funding(partial)
No SMC intervention 17(no MealsNo SMC),Pure Control
8 8
SMC score-cards
Bottom-up accountability
17 8 X X 8 X X
Top-down accountability
17 8 X X 8 X X
Minimum detectable effects for alternative outcome measures and designs, Uganda
(1) (2) (3)
Design parameters
j=100, n=30, P=.5
j=400, n=30, P=.125
j=100, n=50, P=.5
Controls district, gender
District, gender
district, gender
Outcome English 0.69 (0.36) 0.52 (0.27) 0.68 (0.35)
Math 0.53 (0.34) 0.40 (0.25) 0.52 (0.33)
Avg score 2.46 (0.37) 1.84 (0.28) 2.43 (0.37)
Pass indicator
0.12 (0.30) 0.09 (0.22) 0.12 (0.29)
Note: Minimum detectable effect sizes in parentheses.
Minimum Detectable Effect Sizes for alternative outcome measures and test parameters, Kenya
SAQMEQ
Parameter estimated (assumptions)
Estimated intra-class correlation
MDES (P=.5, j=196, n=28)
MDES (P=.25, j=196, n=28)
Outcome Reading .38 .28 .32
Math .46 .25 .29
Average .45 .27 .32
Data Collection/SourcesKenya:
Admin data:use of Education Management information System, Kenya Certificate of Primary Education exam data, Southern and Eastern African Consortium to Monitor Education Quality (SACMEQ) II and III; the Kenya Integrated Household Budget Survey (KIHBS).
Follow-up to SACMEQ II to be conducted with MoE
Uganda: Admin data: use of Primary Leaving Examination results and
administrative data on school funding and district-level inputs.
Survey of school management and pupils to be collected by EPRC researchers in four districts. Tests prepared by UNEB (National Assessment for Planning in Education) to be administered to a sample of students
Example of MoE Database: nationwide coverage over time
BARINGO
BOMET
BUNGOMA
BUSIA
ELGEYO-MARAK
EMBUGARISSAHOMA BAY
ISIOLOKAKAMEGA
KERICHO
KIAMBU
KILIFI
KIRINYAGAKISII
KISUMU
KITUI
KWALE
LAIKIPIA
LAMU
MACHAKOS
MAKUENI
MANDERA
MARSABIT
MERU
MIGORI
MOMBASA
THIKA
NAIROBI
NAKURU
NANDI
NAROK
MERU SOUTH
NYAMIRANYANDARUANYERI
SAMBURU
SIAYA
TAITA TAVETA
TANA RIVER
TRANS-NZOIA
TURKANA
UASIN GISHU
VIHIGA
WAJIR
WEST POKOT
Change in Log Enrolment(.6,1.2](.4,.6](.2,.4](0,.2](-.2,0](-.4,-.2](-.6,-.4][-1.2,-.6]No data
Values represent the change in the log of public school enrolment
Change in Enrolment since FPE
BARINGO
BOMET
BUNGOMA
BUSIA
ELGEYO-MARAK
EMBUGARISSAHOMA BAY
ISIOLOKAKAMEGA
KERICHO
KIAMBU
KILIFI
KIRINYAGAKISII
KISUMU
KITUI
KWALE
LAIKIPIA
LAMU
MACHAKOS
MAKUENI
MANDERA
MARSABIT
MERU
MIGORI
MOMBASA
THIKA
NAIROBI
NAKURU
NANDI
NAROK
MERU SOUTH
NYAMIRANYANDARUANYERI
SAMBURU
SIAYA
TAITA TAVETA
TANA RIVER
TRANS-NZOIA
TURKANA
UASIN GISHU
VIHIGA
WAJIR
WEST POKOT
Change in KCPE points(5,20](0,5](-5,0][-20,-5]No data
KCPE scores measured by sum of English, Math and Science exams
Change in KCPE Scores since FPE
ConsultationsWhat have we done to include stakeholders in the planning
process? Kenya
Meetings held with Teacher Service Commission, Kenya Institute of Education and MoE (Quality Assurance and Basic Ed Directorates)
Focus groups discussions with teachers and head teachers plus church sponsor
Training workshop with MOE staff to bring them into the project as active participants
Uganda Preliminary meetings with District Education officials in each of
the 4 districts, plus MOE staff in Kampala. School visits to meet with teachers & head teachers in each
district 3-day intervention design and training workshop in Kampala
with MoE staff, UBOS, DEOs., and civil society organizations.
Dissemination Plan Standard research model: get data, do
analysis, try to convince policymakers you’re right
Our approach in this project: Train government officials in methods of analysis Rely as much as possible on administrative data
sources Involve gov officials in the implementation and
analysis Final results are fully owned by policymakers
as their own collaborative product
Project staff
Lead Institutionsa. University of Nairobi, School of Economics, Kenya