1 Incentivizing schooling for learning: Evidence on the impact of alternative targeting approaches 1 Felipe Barrera-Osorio Harvard Graduate School of Education and Deon Filmer World Bank This version: October, 2012 Preliminary version. Please do not circulate. Abstract. Demand-side incentive programs such as scholarships or Conditional Cash Transfer programs have been shown to increase measures of school participation in a number of countries, although impacts on learning outcomes have been harder to identify. We evaluate the impact of a large scale primary school scholarship pilot program in Cambodia and show that the program increases school participation (enrollment and attendance)—despite being targeted to some of the poorest and most remote areas. The program was designed to evaluate the equity and effectiveness implications of two alternative targeting approaches: in some randomly selected schools recipients were targeted on the basis of poverty, in others recipients were targeted on the basis of merit. While we show positive impacts on enrollment and school progression emerging from both targeting approaches, learning impacts are only detectable among merit- based recipients. We present evidence on student effort and household education investment compatible with the asymmetry in learning impacts. While there are some equity implications of a merit-based approach to targeting (the poverty-based approach unsurprisingly identifies a poorer group of recipients), the tradeoff is not particularly stark. Scaling up an approach that targets students with high academic potential—while ensuring that the poorest student are among that set—is likely to be the approach that maximizes both equity and effectiveness objectives. JEL classification codes: I21; I24; I28; O10 Keywords: education; Cambodia; randomization; scholarships; merit-based targeting; poverty-based targeting. 1 We thank Luis Benveniste, Norbert Schady, Beng Simeth, and Tsuyoshi Fukoaka and the members of Primary School Scholarship Team of the Royal Government of Cambodia’s Ministry of Education for valuable input and assistance in carrying out this work. Adela Soliz provided able research assistance. The paper has also benefitted from comments by Muna Meky, Halsey Rogers, and Shwetlena Sabarwal. The authors are, of course, responsible for any errors. This work benefited from funding from the World Bank as well as through the EPDF Trust Fund (TF095245). The findings, interpretations, and conclusions expressed in this paper are those of the authors and do not necessarily represent the views of the World Bank, its Executive Directors, or the governments they represent.
39
Embed
Incentivizing schooling for learning: Evidence on the ... · Incentivizing schooling for learning: Evidence on the impact of alternative targeting approaches1 ... from comments by
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Incentivizing schooling for learning:
Evidence on the impact of alternative targeting approaches1
Felipe Barrera-Osorio
Harvard Graduate School of Education
and
Deon Filmer
World Bank
This version: October, 2012
Preliminary version. Please do not circulate.
Abstract. Demand-side incentive programs such as scholarships or Conditional Cash Transfer programs
have been shown to increase measures of school participation in a number of countries, although impacts
on learning outcomes have been harder to identify. We evaluate the impact of a large scale primary
school scholarship pilot program in Cambodia and show that the program increases school participation
(enrollment and attendance)—despite being targeted to some of the poorest and most remote areas. The
program was designed to evaluate the equity and effectiveness implications of two alternative targeting
approaches: in some randomly selected schools recipients were targeted on the basis of poverty, in others
recipients were targeted on the basis of merit. While we show positive impacts on enrollment and school
progression emerging from both targeting approaches, learning impacts are only detectable among merit-
based recipients. We present evidence on student effort and household education investment compatible
with the asymmetry in learning impacts. While there are some equity implications of a merit-based
approach to targeting (the poverty-based approach unsurprisingly identifies a poorer group of recipients),
the tradeoff is not particularly stark. Scaling up an approach that targets students with high academic
potential—while ensuring that the poorest student are among that set—is likely to be the approach that
maximizes both equity and effectiveness objectives.
1 We thank Luis Benveniste, Norbert Schady, Beng Simeth, and Tsuyoshi Fukoaka and the members of Primary
School Scholarship Team of the Royal Government of Cambodia’s Ministry of Education for valuable input and
assistance in carrying out this work. Adela Soliz provided able research assistance. The paper has also benefitted
from comments by Muna Meky, Halsey Rogers, and Shwetlena Sabarwal. The authors are, of course, responsible
for any errors. This work benefited from funding from the World Bank as well as through the EPDF Trust Fund
(TF095245). The findings, interpretations, and conclusions expressed in this paper are those of the authors and do
not necessarily represent the views of the World Bank, its Executive Directors, or the governments they represent.
2
1. Introduction
There is a steadily growing international evidence-base on the impact of targeted cash transfers on a
range of outcomes. Conditional Cash Transfer (CCT) programs have gained popularity in much of the
developing world and in several countries are now the largest form of social assistance (see the review in
Fiszbein and Schady 2009). These programs, in which cash is transferred to families on the condition that
they comply with a set of conditions (typically that children enroll in—and regularly attend—school,
pregnant women make regular prenatal visits, young children are taken for regular health monitoring and
checkups visits), have been rigorously evaluated in many countries. Scholarship programs, such as the
one we evaluate here, can be thought of as single-child (i.e. individual recipient), single-sector (i.e.
education) CCTs.
Much of the rigorous evidence on the impact of CCTs have been middle-income and in the Latin
America region; the Cambodian evidence joins a much smaller set of evidence focused on low-income
countries, such as Bangladesh and Malawi.2 Moreover, the setting is one in which the primary education
survival rate is low due to high a dropout rate. Indeed, much of the evidence of the impact of CCTs at the
primary level is from countries where baseline enrollments are high, and impacts subsequently small (in
part because there is little room for increase). Establishing the extent to which cash transfers are effective
at this school level in this type of setting is an important contribution of this evaluation.
Despite this large body of evidence of the impact of CCTs on enrollment and attendance, there
has been limited evidence to date on the impact of these programs on test scores, and the few available
studies show mixed results: in the case of Cambodia (Filmer and Schady 2009) and Mexico (Behrman,
Parker and Todd 2005; Behrman, Sengupta, and Todd 2000), no results on achievement tests; in the case
of Kenya (Kremer, Miguel and Thornton 2009) and Malawi (Baird, McIntosh and Özler, 2011), positive
effects. The question of how to turn incentives for schooling into learning remains an open one.
An important feature of the program we evaluate here is that students were selected on the basis
of two alternative approaches. In some schools recipients were selected purely on the basis of poverty
(actually, a proxy thereof), but in other schools recipients were applicants who scored well on an
assessment test—i.e. a merit-based targeting approach. Kremer, Miguel and Thornton (2009) describe a
merit-based approach to targeting scholarships in a secondary school program in Kenya—and show
substantial impacts on both attendance and learning. Targeting students with high academic potential,
through the merit-based selection approach, might be a good approach if it maximizes the probability of
2 See Chaudhury and Parajuli (2008) and Baird, McIntosh and Ozler (2009).
3
having an impact on learning as well as enrollment—incentivizing schooling for learning. But, targeting
high performing students may come at the cost of reaching the poorest, since there is likely to be a
positive relationship between academic success and household economic status. Moreover, some authors
will argue strongly against merit scholarships based on equity considerations (see Orfield, 2002). The
question of whether the efficiency gain (defined narrowly as “getting more learning and enrollment per
dollar transferred”) comes at too great a cost in terms of not reaching the poor is one that this evaluation is
able to investigate.
This evaluation aims at addressing two key questions associated with this pilot program: first,
what is the measurable impact of these primary school scholarships on measures of school participation
and learning? And second, given a choice between targeting recipients based on poverty versus merit,
what are the potential tradeoffs in terms of impacts versus reaching the poorest? The evaluation shows
two main results. First, both targeting approaches cause higher enrollment and attendance rates, but only
the merit-based scholarship shows positive impact on learning, measured as result in test scores. Second,
while there are some equity implications of a merit-based approach to targeting (the poverty-based
approach unsurprisingly identifies a poorer group of recipients), the tradeoff is not particularly stark.
Before addressing the evaluation it is important to emphasize that we focus on a narrow set of
objectives, schooling and learning as measured by test scores. The program aimed to transfer cash to
poor households, which in itself is potentially welfare enhancing—and we do not address that directly.
Moreover, learning is but one objective of schooling—albeit an important one. There are additional
social as well as personal (for example, better health, delayed marriage) impacts of more schooling that
we are not addressing or evaluating here.3
In the next section we describe the setting, program and the evaluation. In section 3 we present
the empirical strategy and results. Finally, in section 4 we discuss the results and we present main
conclusions.
2. Country setting, program design, evaluation design, and data
Country setting
3 As discussed below, we focus below on the fact that the merit scholarship showed impacts on learning, while the
poverty scholarships did not. It is possible that poverty scholarships nevertheless have salutary impacts on other
outcomes through their impact on schooling.
4
Cambodia has a tradition of demand-side incentives intended to raise school enrollment and
attendance rates. While some of these operate at the primary level—such as school feeding programs, or
small scale programs that incentivize attendance for primary school children—the bulk of the programs
are targeted at the lower secondary school level. The largest of these programs have been the so call PB
Scholarships.4 The programs do not operate as simple “fee-waivers”; rather, the families of children
selected for a “scholarship” receive a small cash transfer, conditional on school enrollment, regular
attendance, and satisfactory grade progress.
Two rigorous evaluations of the impact of these programs have shown substantial increase in school
enrollment and attendance as a direct consequence of the programs.5 Recipients are on the order of 20 to
30 percentage points more likely to be enrolled and attending school as a result of the scholarships. The
evaluation of scholarships offered through the CESSP also showed that the scholarships targeted to lower
secondary school students led to more expenditures on education, and to less work for pay among
recipients. There were no negative spillover effects—either to non-recipients in schools or to ineligible
siblings in households. Impacts on learning outcomes were limited, pointing to issues of quality and the
match between students’ skill levels and the instruction they are receiving.
One important finding from previous programs, however, was that their targeting was only mildly
pro-poor. For example for CESSP scholarships, despite the fact that the program was able to reach the
poorest children who applied for the scholarships, the poorest of the poor have already dropped out of
school before grade 6—the point at which they would apply for secondary school scholarships. Figure 1
shows the proportion of children 15 to 19 who have completed each grade, based on nation-wide data:
clearly children from the poorest quintiles are much less likely to make it to 6th grade. This suggests that a
program that targets children at the end of grade 6 is not likely to be pro-poor—and that a program
targeted at poor students, earlier in the schooling cycle—is needed if the goal is to reach the poorest of the
poor.
Based in part on these findings, and on a desire to assess the viability, effectiveness and optimal
design of such a program, the Royal Government of Cambodia included a pilot primary school
4 This program, formerly called PAP12, is operated from the government’s Program Budget, the Japan Fund for
Poverty Reduction (JFPR) Scholarships funded by the Asian Development Bank and UNICEF, the Belgian
Education and Training Trust (BETT) Scholarships, and the Cambodia Education Sector Support Project (CESSP)
Scholarship Program which is funded through a World Bank project. Students who were receiving JFPR and BETT
scholarships, but who were threatened by a cessation of these scholarships because of lack of funds in the projects,
were ultimately covered by the CESSP program. 5 For the JFPR evaluation see Filmer and Schady (2008); for the CESSP evaluation see Filmer and Schady (2009)
and Ferreira, Filmer and Schady (2009).
5
scholarship program as a component of the activities funded by the Fast Track Initiative-Catalytic Fund
(FTI-CF) Grant that it received. The stated goal of the program was to increase schooling by offsetting
the direct and opportunity costs.6 Implicitly, the goal was also to improve learning outcomes through that
additional schooling. This paper reports the results of the impact evaluation of that pilot program.
Program and evaluation design
The basic design of the primary scholarship pilot was to select participating schools; and then
within schools identify scholarship recipients according to a clear and transparent criterion. Once
selected, recipients needed to stay enrolled, attend school regularly, and maintain passing grades in order
to keep the scholarship until they graduate from primary school.7 The program targeted students in
selected schools entering the upper-primary level (Grades 4, 5 and 6). The scholarship amount was set at
US$20 per student, per year.8 The scholarships were intended to be disbursed in two tranches of US$10
over the school year: once towards the beginning of the year, and once towards the middle. In the first
year of the program, scholarships were distributed in one lump sum due to delays in implementation.9
The pilot program was targeted to the three Provinces where average dropout rates between
grades 3 and 6 were highest, as determined by an analysis of Cambodia’s Education Information
Management System (EMIS). These Provinces were Mondulkiri, Ratanakiri and Preah Vihear. In order
to narrow the geographic scope of the program, 7 (of 9) districts in Ratanakiri with the highest dropout
rates were selected for participation, and all districts in the other Provinces were included. Within these
selected districts, all primary schools which offered classes through to grade 6 participated in the
program.
In order to evaluate program impact, 209 schools were randomly assigned to join the program in
its first year, 2008-09 (referred to as Phase 1 schools—104 schools), or in 2009-10, its second year (Phase
2—105 schools), as depicted in Figure 2. The identification of impact is based on the fact that among the
cohort of students studied, Grade 4 —at baseline— students in randomly selected Phase 2 schools were
6 Primary schools are officially non-fee based. Opportunity costs include various forms of child labor which are
relatively common in the areas under study—although typically labor is combined with schooling at the primary
school ages. 7 There is moderate enforcement of the conditionality. Students absent for many days are followed up by school
officials and if they return to school would remain eligible for the scholarship. After a student is absent for too
many days they would be classified as having dropped out and no longer be eligible for the scholarship. 8 CESSP lower-secondary scholarships were in the amounts of $45 and $60—however the evaluation found little
impact on enrollment and attendance of $60 over and above $45 (Filmer and Schady 2011) 9 Scholarships distributions for the cohort of recipients in Phase 1 schools analyzed here took place in July 2009
(US$20), November 2009 (US$10); April 2010 (US$10); November 2010 (US$10); and April 2011 (US$10).
6
not eligible for scholarships. These students therefore serve as a valid counterfactual group—a group that
differs, on average, from the treatment group only in that it did not receive the scholarships. Since these
students in control schools were never exposed to the program (even after the subsequent cohort became
eligible when scholarships were implemented in Phase 2 schools), the two groups of students can be
tracked over time and enrollment, attendance, and other outcomes compared.10
Schools were further randomly allocated to one of two groups in order to evaluate the
effectiveness of the alternative ways of targeting: “poverty-based” and “merit-based” targeting. The first
group of schools (“poverty-based” targeting) used a score, similar to that currently in use in the secondary
school programs (52 Phase 1 and 53 Phase 2 schools) —see Figure 2. All targeted students filled out a
simple form with questions relating to their household and family socio-economic characteristics.11
These forms were scored according to a strict formula based on weights derived from an analysis of
household survey data.12
Scoring of the actual application forms was carried out centrally by a firm
contracted specifically for this purpose, thereby reducing the ability to manipulate the program. Within
each school, the applicants with the highest scores (i.e. “the highest poverty”) were selected to be offered
a scholarship. In a second group of schools (“merit-based” targeting), applicants were ranked based on
scores on a test of learning achievement (52 Phase 1 and 52 Phase 2 schools). The test was adapted from
the Grade 3 National Learning Assessment which was developed under the Cambodia Education Sector
Support Project (CESSP) project.13
All eligible students took the test and, within each school, the
applicants with the highest test scores were selected to be offered a scholarship. Again, scoring the tests
was done centrally in order to minimize the risk of program manipulation. The number of students in each
school type was fixed exogenously, and set to half the number of registered students in the year prior to
the program (as determined by an analysis of EMIS data).14
10
Because scholarship offers are made according to a strict criterion within schools, applicants “just above” and
“just below” the cutoff for eligibility could be studied using a regression discontinuity design (RDD) approach to
evaluate impact. Future work based on this program will exploit that approach, and will be able to use data from the
first and second cohorts of students who applied to the program. 11
Table 1 reports the full set of variables included in the calculation of the score. 12
The weights were determined by estimating a model predicting the probability that a student would drop out of
school during grades 4 to 6—since addressing this dropout was the stated goal of the program. Strictly speaking,
therefore, the score should be referred to a “dropout-risk score”. However, the risk is essentially a set of household
characteristics that capture the socioeconomic status of a household—weighted to capture those elements that
predict dropout best. For convenience and ease of exposition, the score is referred to in this paper, as well as in
program documents, as a “poverty” score. 13
The National Assessment was implemented nationwide in Grade 3 in a sample of schools during the 2005/06
school year (Royal Government of Cambodia 2006). 14
The number of scholarships is not equal to half the number of applicants because (1) the rule was to allocate
scholarships to all applicants who had the cutoff score and ties mean that more applicants would receive a
scholarship offer, or (2) because of changes in enrollment numbers from year-to year.
7
Implementation and data
This study evaluates the impact of the program on first cohort of students—who filled out
application forms when the program began implementation in December 2008/January 2009. At the time,
these students were in the 4th grade.
15 All 4
th grade students in program schools filled out the application
forms as well as took the assessment test—both students in Phase 1 and Phase 2 schools, as well as
students in poverty-targeting and merit-targeting schools (see Figure 2). Recipients received scholarships
disbursements (on condition of remaining in school, attending regularly, and maintaining passing grades)
during the 2008/09, 2009/2010 and 2010/11 school years.
We use three main data sources to evaluate program impact. First, we use the full set of data
collected at the time students applied for the scholarships. That is, we have information on baseline
household characteristics as well as baseline math and Khmer language test scores for all applicants.
Second, we use the official list of students who were offered a scholarship. Third, we use endline data
that were collected specifically for this evaluation. These data are derived from a survey of a random
subsample of students from each program school that was administered at the end of the 2010/11 school
year, when a student who would have stayed in school in the correct grade would have been finishing (or
just finished) grade 6. The survey was administered in households (i.e. not in schools) to the child who
applied for the scholarship, and included a household module administered to their mother, father or other
caregiver. In total 3,618 applicants were interviewed.16
For the bulk of the analysis we use data from
1,377 students in grade 4 at baseline who were offered a poverty- or merit-based scholarship, or would
have been offered a poverty- or merit-based scholarship had they been going to a program school.
The survey asks about a broad range of issues, both in terms of school participation (for example
the “intensity” of school participation though questions relating to time spent in school), other activities
such as labor market participation, as well as various measures of cognitive development and learning
achievement. Importantly, given that these data are collected at the household level the questions can be
asked of both recipients and non-recipients whether they are in school or not. As such, another
contribution of this study is that it avoids the problem of selecting only children who are enrolled and
attending school when analyzing learning achievement data collected at the school (Kremer, Miguel and
15
These students were “supposed” to have filled out application forms prior to the beginning of the school year, i.e.
when they were still in 3rd
grade. Because of delays in effectiveness and implementation of the overall project, the
application process could only be implemented once the students had begun 4th
grade. 16
Attrition was 15%. Analysis of attrition patterns show that the share of attritors is not different by Phase 1/Phase
2 schools, nor is it related to poverty/merit status of the school.
8
Thornton, 2009). Three learning achievement tests were administered: a mathematics test, a Digitspan
test and Ravens Progressive Matrices test. 17
The items on the Mathematics test were drawn from a variety of sources including the baseline
mathematics test; questions from the national grade 6 assessment; questions drawn from publicly released
items from the Trends in International Maths and Science (TIMSS) Grade 4 Assessment. Items were
tested during a pretest and only items with adequate properties were retained for the final test. It is a
multiple choice test, measuring both knowledge and capacity to use this knowledge to solve specific
problems. Presumably, this is the measure of the most immediate academic impact of the intervention,
since exposure of the program can directly affect the ability to solve mathematical problems.
The Digitspan test is a test in which a series of numbers are read to a respondent who is then
asked to repeat the numbers back to the enumerator. The series increases from 2 numbers to a larger and
larger number, until 9 digits. Respondents are also asked to repeat the numbers back in reverse order.
The test is typically interpreted as a measure of short term memory and working memory capacity. In the
Ravens Progressive Matrices test respondents are shown a set of three images each with a pattern that
links to the others. They are then shown a set of potential images, one of which link to the three original
images, and are instructed to tell the enumerator which one “completes” the first three. The test is
typically interpreted as a measure of logical reasoning.
3. Empirical strategy
Empirical Strategy
We estimate the reduced-form of the program impact on enrollment and attendance outcomes; on
test scores; and on potential mechanisms of transmission (school / teacher effort and student / household
effort). The estimation is based on the equation
(1)
denotes the outcome variable for individual i at follow-up (t1); is in indicative of
treatment status; is a vector of controls measured at baseline; and captures unobserved students
17
A vocabulary test was also included. The results of this test were in general terms very imprecise and
tending towards zero—possibly because of problems in translating words and concepts into Khmer. For
the sake of simplicity, we do not report these results.
9
characteristics and idiosyncratic shocks. The controls includes baseline values for gender, number of
minors in the household, indicators for whether the household owns a motorcycle, a car/truck, an
oxen/buffalo, a pig, an ox or buffalo cart; indicators for whether the house has a hard roof, a hard wall, a
hard floor, an automatic toilet, a pit toilet, electricity, piped water; as well as the overall poverty index and
test scores at baseline.
The estimation is done separate for each targeting mechanism. Accordingly, for students in merit-
based targeting schools, is equal one if offered the merit-based scholarship, and zero for untreated
students in the control schools who would have been eligible for merit-based scholarships —based on
their test scores— had they attended a treatment school. An analogous treatment indicator is built for the
poverty-based targeting mechanism. Given that the treatment variable identifies at baseline all individuals
offered the scholarship, the estimation is in practice an intention to treat estimator (ITT). Errors are
clustered at school level, and each estimation includes a district level fixed effects. In order to gain
efficiency, we run seemly unrelated regressions (SUR) in estimating the impact on test scores.
The design of the intervention also allows for directly estimating the effects of the program on
non-treated students. In each treatment school, approximately half of the students were not treated.
Therefore, given the random assignment of schools into treatment, we can compare the non-poor (and
non-treated) students in the poverty treatment schools with non-poor (and non-treated) students in control
schools. Similarly, we can compare students who did not received scholarships in merit-treated schools
with control school students who would not have received merit-scholarships in control schools. In order
to estimate this, we used Equation (1), but replacing with the appropriate group of students. In this
program the scholarship is offered after the baseline test is done, and during the duration of the program,
non-scholarship recipients cannot change their status. Therefore, any effects on non-treated students
emanate from complementarities and interactions between treated and non-treated students during the
academic year.18
Baseline balance and characterization of the study sample
This section presents the general characteristics of the study sample and the validation of the
random assignment by comparing treatment and control students at the baseline. In Table 1, columns (1)
and (2) are based on all students in the control and the treatment schools, whereas columns (3)-(6) only
18
These peer effects are different in nature with the externality effects estimated in Kremer et al (2009).
The authors of that paper estimate the effect of the “promise” of scholarship on students with low scores
pre intervention. Those effects presumably emanate from the effort that all students may exercise in order
to get the scholarship.
10
use information of the treated students in the treatment schools and untreated students in the control
schools who would have been eligible for treatment —based on their poverty index score or on the
baseline test score—had they attended a treatment school. The sample is restricted to Grade 4 students at
baseline, and as described above, the control group of students was not part of the phase-in expansion of
the program. Columns (1), (3), (5) show the means and standard deviations of household and individual
characteristics prior to the intervention. Columns (2), (4) and (6) show the differences and standard errors
at baseline between the treatment and control groups. That is, the values are the coefficient estimates from
a regression of each characteristic on a dummy variable equal to 1 for a treatment school.
Two main features emerge from Table 1. First, treatment and control groups are similar in
observed characteristics. Only a few coefficients in columns (2), (4) and (6) are statistically significant (of
the 48 differences reported, 5 are significant at the 10% level, and among them, 1 is significant at the 1%
level). The results in Table 1 confirm the validity of the random assignment since both control and
treatment groups are similar in their observed characteristics. More important, in means, the poverty
index and the test score are equal between treatment and control groups. Figures 3 and 4 present the
density of poverty index at baseline for treatment and control schools. There is a clear overlap along the
whole distribution. We cannot reject equality of both distributions using a Kolmogorov-Smirnov test.
Second, on average, the recipients who are offered scholarships on the basis of the merit-based
targeting have more assets, a lower poverty index score and better performance on the baseline test than
students in the poverty-based treatment. For instance, the poverty index, which ranges from 0 (wealthiest
family in the sample) to 292 (poorest family in the sample), has a mean of 245.13 for poverty based
students, and 218.2 for merit-based students. Likewise, the baseline test (ranging from 0 to 25) has a
mean of 19.77 for merit-based students and 17.74 for poverty-based students. We return to this issue in
the discussion below where we discuss tradeoffs between the two targeting approaches.
The main finding from Table 1, however, is that it shows a balanced sample between treatment
and control groups at baseline, which is a key determinant of the random assignment approach being a
valid identification strategy.
4. Results
Impacts on enrollment and attendance
The intervention is aimed directly at incentivizing higher enrollment and attendance. In order to
keep the scholarship, the selected students must stay enrolled, attend school regularly, and maintain
11
passing grades until they graduate from primary school (sixth grade). We focus on three enrollment and
attendance proxies: the proportion of students reaching 6th grade, the highest grade completed, and the
hours of school attended in the past seven days.
In order to provide a baseline for assessing the relative magnitude of impacts, Table 2 reports
outcome variables of the students at the follow up in the control group. Between 61% and 64% of the
students reported reaching at least sixth grade and the average grade completion is around 5.4.. The third
outcome variable was constructed from a question that asked students how many hours they attended
school the past seven days, conditional on being enrolled. Depending on the control group, students
reported an average having attended school for about 8.83 hours (poverty) and 9.27 hours (merit) in the
past week.
Table 3 reports the program impacts on the enrollment and attendance proxies. Columns (1) and
(2) present the effects of treatment of the poverty and merit-based interventions, respectively, when
controlling for baseline student characteristics (i.e. the variables through the application forms, reported in
Table 1), poverty index and test scores at baseline, and province fixed effects. As discussed above, the
randomized assignment was successful in that it produced a balanced sample. Controlling for additional
variables in the program impact regressions should therefore only affect the precision of the estimates, not
the magnitude of the estimated effects. Appendix 1 presents the results without controls; as expected, the
results are very similar to the results in Table 3.
Overall, Table 3 shows consistent evidence of positive impacts from the interventions on
enrollment and attendance. The proportion of students reaching grade 6 increased with both treatments,
and the effects are similar in magnitude. The estimated impacts range between 12% point and 17% point
increase from a counterfactual of around 61%-64%. Similarly, the intervention increased the average
highest grade completed in both treatment samples, with effects ranging from 0.332 –in poverty-based
intervention—to 0.187 grades—merit-based—, from a counterfactual of about 5.4 years. These impacts
are similar than those found in the context of the Secondary School scholarships program (where impacts
on enrollment were on the order of 20-25% point increase). These impacts are larger than most
documented in countries elsewhere in the world (Fiszbein and Schady 2011), and should be assessed
against the very small size of the transfer considered (i.e. $US20 per year).
The measure of attendance (number of hours in school in the past seven days) shows positive
estimates, but none of these are statistically significant, with the exception of the estimate without
controls for poverty treatment. Nevertheless, taking the point estimates at face value, the results suggest
12
that the intervention increases attendance by on the order of 2.9 hours per week for poverty-based
treatment and 0.64 for merit-based.
In sum, there is strong evidence that the program increased enrollment and suggestive evidence
that it also improved attendance rates, regardless if the scholarship is based on merit or on poverty
status—the targeting approach did therefore not affect the extent to which the program increased
measured school participation.
Results on test scores
There are two main channels through which the program could impact test scores. First, by
incentivizing enrollment and attendance, students are more exposed to school—and through that
additional schooling acquire more learning. Second, by requiring that the students maintain passing
grades, the program may give students an incentive to study more.
Table 4 presents impacts of the interventions on the three measures of academic and cognitive
achievement: Mathematics test, Digitspan test, and Raven test. (Table 4 has the same structure as Table 3;
it present impact coefficients after controlling for baseline characteristics and district fixed effects;
Appendix A presents results without controls). Given that we use three, likely correlated, measures of test
scores, we estimate the model using Seemingly Unrelated Regressions (SUR) to gain efficiency. All three
measures are standardized using the mean and deviation of the respective control group—impacts can
therefore be interpreted as changes in a standard deviation of the achievement measure.
In contrast to the results on enrollment and attendance, Table 4 reveals different impacts on test
scores between the poverty and the merit treatment group. While the impact estimates show no effects
from treatment for students treated based on their poverty status, there is a clear trend of positive effects
from the intervention for students treated based on merit. All the point estimates for the merit treatment
are positive and statistically significant. For the merit sample when using additional controls, the effect of
the intervention on the math test is 0.170 standard deviations, the effect on the Digitspan test is 0.149
standard deviations, and the effect for the Raven test is 0.178 standard deviations. The results on math
are of particular interest, since math is potentially the measure that is the most sensitive to exposure to
additional schooling. Moreover, households have potentially less ability to substitute for teaching math
than vocabulary or other types of learning. These effects on test scores are similar in magnitude to the
merit-based scholarship program evaluated in Kenya (Kremer, Miguel and Thornton 2009).
13
In sum, the results suggest that the program incentivized both types of students – those from
poorer households and those with higher academic merit—to enroll in additional years of schooling and
have higher attendance. However, only students who received treatment based on merit show any gains in
test scores from the intervention.
Heterogeneity
All in all, it seems that the program incentivized both types of students –those with higher
academic merit and those from poorer households —to enroll and attend additional years of school.
However, only students who received treatment based on merit show any gains in academic achievement
from the intervention. That is, additional schooling (e.g. enrollment and attendance) results in better
learning outcomes (as captured by our four tests) only for those students who had better skills at the
baseline. At the light of this result, it is important to explore the heterogeneity of effects by baseline skill
and poverty levels. These effects are presented in Table 5. For each poverty school (treated and control)
we identify students that were above and below each school’s median baseline test, and then we run
separate regressions to estimate the effect for the two types of students (high baseline achievers and low
baseline achievers). For each follow-up test (math, Digitspan and Raven), Column (1) presents the results
on test scores of the effects of the poverty-targeting mechanism for low achievers at baseline and Column
(2) for high baseline achievers. In an analogous way, for merit-based schools (treatment and controls) we
identify students above and below each school’s median poverty index, and then, we run separate
regressions for the two populations. For each follow-up test, Column (3) presents the impact of merit-
based treatment for those above the school’s median poverty index (non-poor population) and Column (4)
presents the impact of merit-based treatment for students below the school median poverty index (poor
population).19
As Table 5 shows, the merit-based treatment has either similar or larger positive impacts on the
poor population than in the non-poor population. In contrast, the poverty-based treatment does not elicit
positive results from either the high baseline achievers or the low baseline achievers. In other words, the
asymmetry in test results for poverty-based and merit-based targeting mechanisms persists. More
importantly, the poverty-based treatment does not induce better test results among high baseline
performers, whereas the merit-based treatment does induce better test scores among baseline poor
individuals, despite the fact that the conditions of both scholarships—merit and poverty—are the same.
19
An alternative specification is to run the pooled regression with a dummy variable for treatment, a dummy
variable indicating the status at base line, and the interaction term. The results are very similar to the ones presented
in Table 5.
14
Given the results for poor individuals in the merit-based treatment, similar results were expected for the
high achievers in the poverty-based treatment.
Different mechanisms can explain the asymmetry in tests’ results. As argued before, both types of
scholarships provide incentives for students to increase effort. For instance, it is plausible that, due to the
scholarship, students increase hours of studying outside school. Also, families may be motivated to
invest more in education expenditures, textbooks and such and as a result, help the student in conserving
the scholarship. Likewise, the program can impact directly the behavior of the school and teachers. For
instance, under an altruistic model, teachers can increase attention to students with scholarships with the
hope that they can retain the money. Also, it is possible that presents from scholarship winners’ parents to
teacher can induce higher effort. As such, the school can change behavior. Banerjee and Duflo (2006),
while presenting the results of Kremer et al (2009), discuss changes in teacher motivation and higher
control of families. As mentioned before, the follow-up data comes from household interview. As such,
the drawback of household interviews is the lack of school level information. However, household
interviews allow us to gather evidence regarding potential channels that may explain the results, such as,
student effort outside of school and a household’s investment in education.
Students’ effort is captured by the amount of time spent studying, doing homework and taking
private lessons outside of school. The household’s effort is measured by the total amount of education
expenditure by the household and the proportion of this expenditure spent on textbooks. The effects of
treatment on each of these three variables are presented in Table 6. On average, control students spent 3.5
hours per week doing school tasks outside the school. Merit-treated students spent more time doing
academic work outside of school (an increment of 0.579 hours). The household response to scholarships
in terms of education spending and the nature of that spending differs across the scholarship types. On
average, household’s education expenditures are approximately U$17. Households with a merit
scholarship recipient spent U$ 5 more on education, and a higher proportion (1% more) of the
expenditure was on textbooks, than control students’ households.20
In contrast, there were no impacts on
these outcomes in households with poverty targeted recipients. The results suggest that only household
with merit-based students increase effort.
All in all, it seems that families in the merit-based treatment invest more in the education of
students; in contrast to what happen in families with the poverty-based treatment. Also, students in the
merit-based treatment put more effort outside school than the poverty-based students. These findings are
20
For the text book expenditures, the p value of the coefficient is 0.105
15
compatible with a motivation hypothesis: merit-based students are motivated to work more, and their
families are motivated to invest more. Ideally we would like to disentangle the motivation of students and
from the motivation of teachers, however, the data available do not allow us to carry such analyses.
Peer effects on enrollment, attendance and achievement
One concern with a program such as this is that there might be negative spillover effects. If
increased enrollment and attendance leads to classroom overcrowding, then this may hurt the learning
opportunities afforded to other students. At the same time, there might also be positive spillover effects if
scholarships create a positive energy in favor of schooling and learning which could affect all children in
a classroom.21
Moreover, peer effects might differ by the design of the targeting approach: for instance
applicants denied a merit-based scholarship might become discouraged, and eventually perform worse.
Since there will always be limited scope for containing spillover effects, they should be estimated
directly, and factored into the “equity/efficiency” tradeoff in program design.
As discussed above, the design of the intervention allows us to test directly for peer effects. Table
9 presents the coefficient estimates for the indicator variable that is one for those students not treated in
treated schools, and zero for students in control schools with analogous baseline test scores or poverty
index scores. The dependent variables are proportion of students reaching 6th grade, the highest grade
completed, and the hours of school attended. The estimates can be read similarly to those reported in
Table 3, but estimated off of the sample of non-recipients and their counterfactual. There seems to be a
positive peer effect in poverty-treated schools: the three coefficients are positive; and one of them is
significant at the 5% level. Moreover, as expected, the point estimates are lower than the direct effect of
the program (Table 3). Regarding the merit-based treatment, we do not find any evidence of peer effects,
negative or positive. 22
Table 10 is the analogous table for the three measures of test scores. None of the
coefficient estimates are statistically significant. All the coefficients are very close to zero.
In sum, it seems that the poverty-based treatment may induce more attendance from non-treated
peers. No other peer effects are detected, neither positive nor negative, from the program.
5. Conclusions
21
An additional positive spillover might be on younger cohorts who stay in school longer in order to potentially
benefit from scholarships. We don’t have the data available to address this issue. 22
The results therefore dispel the notion that there was discouragement among merit scholarship applicants who did
not receive a scholarship.
16
The fact that some students were able to take better academic advantage from more exposure to
school than others highlights an issue rarely addressed in previous evaluations of conditional cash transfer
programs. Recent evidence on monetary incentives for schooling shows that students are able to change
their behavior on the margins that are under their control—for example enrollment and attendance.
However, these positive effects do not necessarily translate into test score gains. For example, despite the
fact that Mexico’s Oportunidades program—a rigorously evaluated conditional cash transfer program—
induced students to enroll and attend more to school, the program did not induce higher test scores. A
recent set of papers has argued that education systems in developing countries are typically tailored
towards better-off and better-skilled students. Specifically, Glewwe, Kremer and Moulin (2009) show that
only the strongest students at baseline were able to take advantage of textbooks that were provided to
schools in Kenya; Duflo, Dupas and Kremer (2011), while studying the effects of tracking students into
classrooms according to initial achievement (also in Kenya), show that teachers who were assigned to
students at the bottom of the achievement distribution were less likely to teach.
Our findings based on the Cambodian Primary Scholarships Pilot add to this discussion. On the
one hand, additional exposure seems to pay off in terms of test scores to those students who are more
academically ready to take advantage of the opportunity. On the other hand, poorer students who are not
academically prepared are not able to measurably gain in terms of test scores from the additional
schooling. This evidence is compatible with the idea that teachers are not prepared, or do not have the
pedagogical skills, to take on the challenge of reaching the more academically challenged students.
Clearly more work is needed to establish how best to prepare and incentivize teachers to reach these
students—and this would be an important area for Cambodia (and other countries) to generate knowledge.
At the same time, however, this evaluation suggests that for students who are better academically
prepared—including poor students—incentivizing school attendance can pay off in measurable learning
outcomes. This suggests that remedial lessons for students in the early grades, or increasing school
readiness among poorer students, for example through early child development programs, might be
complementary approaches to increasing the impact of schooling, and programs that incentivize
schooling. Indeed, data from Cambodia suggest that children suffer from substantial delays in cognitive
development, which hampers school readiness (Naudeau and others 2011).
The Cambodian program uses two targeting approaches setting up a potential tradeoff between
efficiency—defined as achieving more learning per dollar transferred—versus equity—defined as
reaching the poorest population. Analysis of the socio-economic profile of program applicants and
recipients under the two targeting schemes—and comparing those to the national distribution of socio-
economic characteristics suggests that both targeting approaches are heavily weighted to the poor. The
17
first panel of Figure 5 shows that 50% of those who applied to the program are within the poorest
nationally-benchmarked quintile; fewer than 3% of applicants were from the richest quintile. Clearly the
program was targeted to poor areas and poor schools. Unsurprisingly, targeting the scholarships further
to the poorest from within each school yields an even greater pro-poor distribution of benefits: 85% of
applicants who were in the poorest half in their school (i.e. those targeted by poverty scholarships) were
from the poorest two quintiles of the population—63% were in the poorest quintile (Panel 2). Merit-
based targeting is not as pro-poor—but is still largely able to reach the poorest groups in the population:
76% of applicants who were in the top merit half of their school (i.e. those targeted by merit scholarships)
were from the poorest two quintiles of the population—54% were in the poorest quintile (Panel 7). A
complementary analysis of the within-school correspondence between high/low poverty applicants and
high/low test scoring applicants yields a similar conclusion: wealthier applicants were not necessarily
higher-scoring.23
Given the relatively effective geographic targeting it is unclear whether this result is
generalizable. In other settings (e.g. where there is more heterogeneity in student poverty levels) the
result may not hold. Nevertheless, the results suggest that for this program, the tradeoff between
efficiency and equity was not particularly stark. Scaling up an approach that targets students with high
academic potential—while ensuring that the poorest student are among that set—is likely to be the
approach that maximizes both the equity and effectiveness objectives of the program.
23
See Appendix B for further details.
18
References
Baird, Sarah, Craig McIntosh and Berk Ozler. 2009. “Designing Cost-Effective Cash Transfer Programs
to Boost Schooling Among Young Women in Sub-Saharan Africa.” World Bank Policy Research
Working Paper No. 5090. The World Bank.
Baird, Sarah, Craig McIntosh and Berk Özler, 2011. "Cash or Condition? Evidence from a Cash Transfer
Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 126(4), pages 1709-
1753.
Banerjee, Abhijit, and Esther Duflo. 2006. "Addressing Absence." Journal of Economic Perspectives,
20(1): 117–132.
Behrman, Jere R., Susan W. Parker, and Petra E. Todd. 2005. “Long-TermImpacts of the Oportunidades
Conditional Cash Transfer Program onRural Youth in Mexico.” Discussion Paper 122, Ibero-America
Institute for Economic Research, Göttingen, Germany.
Behrman, Jere R., Piyali Sengupta, and Petra Todd. 2000. “The Impact of PROGRESA on Achievement
Test Scores in the First Year.” Unpublished manuscript, International Food Policy Research Institute,
Washington, DC.
Chaudhury, Nazmul and Dilip Parajuli. 2008. “Conditional Cash Transfers and Female Schooling: The
Impact of the Female School Stipend Programme on Public School Enrolments in Punjab, Pakistan.”
Applied Economics.
Duflo, Esther, Pascaline Dupas and Michael Kremer. 2011. “Peer Effects, Teacher Incentives, and the
Impact of Tracking: Evidence from a Randomized Evaluation in Kenya.” American Economic Review.
101(5): 1739-74.
Ferreira, Francisco H., Deon Filmer and Norbert Schady. 2009. “Own and Sibling Effects of Conditional
Cash Transfer Programs: Theory and Evidence from Cambodia” World Bank Policy Research Working
Paper No. 5001. The World Bank, Washington, DC.
Filmer, Deon and Lant Pritchett. 2001. “Estimating Wealth Effects without Expenditure Data – or Tears:
With an Application to Educational Enrollments in States of India.” Demography. 2001. 38(1):115-132.
Royal Government of Cambodia. 2006. Student Achievement and Education Policy: Results from the
Grade Three Assessment—Final Report. Cambodia Education Sector Support Project—National
Assessment Component. Phnom Penh, Cambodia.
20
Figure 1: Proportion of 15 to 19 year olds who have completed each grade, by quintile.
Source: DHS 2010
0
0.2
0.4
0.6
0.8
1
1 2 3 4 5 6 7 8 9
Poorest quintile
Quintile 2
Quintile 3
Quintile 4
Richest quintile
21
Figure 2. Design of the intervention
209
Schools
Poverty-based
schools
Merit-based
schools
Lottery
Phase 1
292
0
Poverty index
29
Test score
0
Test score
Phase 2
Phase 1
Phase 2
Grade 4
Grade 3
Grade 3
Grade 4
Grade 3
Grade 3
Poverty index
Poverty index
292
0
Grade 4
Grade 4 Test score
Test score
29
0
Grade 5 Grade 6
Grade 4
Grade 4
Grade 4
Grade 5
Grade 4 Grade 5
Grade 5
Grade 5
Grade 6
Grade 6
Grade 5
Grade 5
Grade 6
Grade 5
TIME=0: BASELINE
INFORMATION; LOTTERY
TIME=1:
PHASE-IN
TIME=2:
FOLLOW-UP
= Treated
= No treated
Poverty index
22
Figure 3: Poverty score at baseline, Treatment versus Control
Source: Students at follow-up, using baseline information
0
.00
5.0
1
kd
en
sity p
ov_
scor
0 100 200 300Poverty score at baseline by school-level treatment status
Treatment (both merit and poverty) Control
23
Figure 4: Test scores at baseline, Treatment versus Control
Source: Students at follow-up, using baseline information
0
.02
.04
.06
.08
.1
kd
en
sity k
hm
_m
ath
0 5 10 15 20 25Test score at baseline by school-level treatment status
Treatment (both merit and poverty) Control
24
Figure 5: Distribution of selected populations across nationally benchmarked quintiles
Source: Analysis of Cambodia DHS 2010 and Primary Scholarship Application forms. Quintiles are defined on the basis of an index of household wealth-related variables that are collected in both the DHS 2010 as well as on the scholarship program application forms.
0
10
20
30
40
50
60
70
(1) Program applicants (2) High Poverty (3) High merit
Table 1. Baseline Balance and mean and standard deviation of baseline characteristics
School Level
Student Level
Control Difference
Control Difference Control Difference
with treatment
Poverty with treatment Merit with treatment
(1) (2) (3) (4) (5) (6)
Gender 0.49 0.037
0.52 0.100*** 0.49 -0.03
'(0.50) '(0.02)
'(0.50) '(0.03) '(0.50) '(0.04)
No of minors 1.69 -0.026
1.79 0.075 1.73 -0.097
(1.11) (0.09)
'(1.12) '(0.12) '(1.12) '(0.12)
Own motorcycle 0.42 0.008
0.28 -0.035 0.42 0.003
(0.49) (0.04)
'(0.45) '(0.05) '(0.49) '(0.05)
Own car/truck 0.16 0.017
0.04 0.01 0.13 0.028
(0.37) (0.03)
'(0.19) '(0.03) '(0.34) '(0.04)
Own oxen/buffalo 0.55 0.032
0.39 0.109* 0.53 0.033
(0.50) (0.05)
'(0.49) '(0.06) '(0.50) '(0.06)
Own pig 0.56 0.028
0.43 0.117** 0.55 0.029
(0.50) (0.04)
'(0.50) '(0.06) '(0.50) '(0.05)
Own ox or buffalo cart 0.31 0.02
0.19 0.058 0.29 0.009
(0.46) (0.04)
'(0.40) '(0.05) '(0.45) '(0.05)
Hard roof 0.49 0.064
0.32 0.047 0.48 0.102**
(0.50) (0.04)
'(0.47) '(0.05) '(0.50) '(0.05)
Hard wall 0.54 0.032
0.38 0.045 0.55 0.018
(0.50) (0.04)
'(0.49) '(0.06) '(0.50) '(0.05)
Hard floor 0.85 0.039
0.79 0.037 0.84 0.068*
(0.36) (0.03)
'(0.41) '(0.05) '(0.37) '(0.04)
Have automatic toilet 0.07 -0.02
0.02 -0.01 0.05 0.005
(0.25) (0.02)
'(0.13) '(0.01) '(0.22) '(0.02)
Have pit toilet 0.12 0.018
0.11 0.02 0.13 0.001
(0.32) (0.03)
'(0.32) '(0.03) '(0.34) '(0.04)
Electricity 0.25 0.011
0.16 -0.01 0.23 -0.002
(0.43) (0.04)
'(0.37) '(0.04) '(0.42) '(0.05)
Pipe water 0.06 -0.001
0.03 -0.012 0.06 -0.013
(0.24) (0.02)
'(0.17) '(0.01) '(0.23) '(0.02)
Poverty Index (o to 292) 210.16 -1.609
245.13 -2.924 218.2 -11.771
(60.18) (5.43)
'(32.73) '(5.14) '(51.66) '(8.76)
Test score (0 to 25) 17.47 0.534
17.74 0.888 19.77 0.028
(4.81) (0.52)
'(4.71) '(0.68) '(3.22) '(0.48)
Number of students 940 2448
431 883 474 940
Number of schools 101 204 67 119 67 118
Columns (1), (3), (5): means and standard deviation, control group. Columns (2), (4), (6): difference with treatment, estimated by regressing each variable against corresponding treatment variable; standard error in parenthesis
26
Table 2. Outcome variables at follow-up
Student Level
Poverty-targeting
Merit-targeting
Control Treatment
Control Treatment
(1) (2) (3) (4)
Reach grade 6 0.61 0.8
0.64 0.77
'(0.49) '(0.40)
'(0.48) '(0.42)
Completed grades 5.38 5.73
5.45 5.68
'(1.22) '(0.93)
'(1.23) '(0.94)
Number of hours 8.83 12.29
9.27 10.64
'(12.97) '(14.87)
'(13.22) '(14.38)
Math test 0.02 -0.02
0.16 0.32
'(1.01) '(0.94)
'(1.04) '(1.04)
Digitspan 0.02 0
0.08 0.23
'(0.98) '(1.00)
'(0.99) '(0.97)
Raven Test -0.02 -0.07
0.11 0.21
'(0.98) '(0.92)
'(0.99) '(1.15)
Number of Students 431 452 474 466
Mean and () standard deviation
27
Table 3. Impact on Enrollment and Attendance
Reach Grade Six
Highest Grade Completed
Number of hours in school, last 7 days
(conditional on enrollment)
(1) (2) (1) (2) (1) (2)
Poverty-targeting treatment 0.170***
0.332***
2.865
'(0.04)
'(0.11)
'(1.87)
Merit-targeting treatment
0.120***
0.182*
0.635
'(0.04)
'(0.10)
'(1.55)
Constant 1.764 0.514
8.161** 5.285**
-100.065** 0.575
'(1.42) '(0.83)
'(3.35) '(2.33)
'(45.16) '(28.25)
Control Variables Yes Yes Yes Yes Yes Yes
No. Obs 883 940
831 897
665 713
F() 6.435 4.872
2.271 1.759
1.246 2.026
R2 Adj 0.18 0.155 0.145 0.122 0.131 0.199
Regression coefficient of dependent variable against treatment indicator controlling by Table 1 baseline characteristics and district fixed effects. All estimators using clusters at school level.
28
Table 4. Impact on test scores
Mathematics
Digispan Test
Raven Test
(1) (2) (1) (2) (1) (2)
Poverty-targeting treatment -0.041
-0.059
-0.021
'(0.06)
'(0.07)
'(0.06)
Merit-targeting treatment
0.170***
0.149**
0.178***
'(0.07)
'(0.06)
'(0.07)
Constant -3.204 1.831
-1.977 0.000***
0.000*** 2.055
'(3.26) '(2.19)
'(3.42) '(2.19)
'(0.00) '(2.19)
Control Yes Yes Yes Yes Yes Yes
No. Obs 883 940
883 940
883 940
Chi_ 2 177.525 178.627
112.442 122.093
130.253 169.017
R2 Adj 0.167 0.16 0.113 0.093 0.126 0.152
Regression coefficient of dependent variable against treatment indicator, controlling by Table 1 baseline characteristics and district fixed effects. All estimators using Seemingly Unrelated Regression (SUR) estimation
29
Table 5. Impact on test scores, by baseline test score and poverty index
Regression coefficient of dependent variable against treatment indicator, controlling by Table 1 baseline characteristics and district fixed effects. All estimators using Seemingly Unrelated Regression (SUR) estimation
30
Table 6. Impact on test scores on Non-treated Peers
Mathematics
Digitspan Test
Raven Test
(1) (2) (1) (2) (1) (2)
Poverty-targeting treatment -0.116
-0.08
-0.173*
(0.09)
(0.09)
(0.09)
Merit-targeting treatment
0.033
0.091
0.076
(0.09)
(0.10)
(0.09)
Constant 0.000*** 0.000***
0.000*** -1.202
1.385 2.511
(0.00) (0.00)
(0.00) (0.00)
(2.44) (0.00)
Control Variables Yes Yes Yes Yes Yes Yes
No. Obs 591 503
591 503
591 503
F() 131.4 122.862
94.067 101.866
121.76 92.4
R2 Adj 0.181 0.127 0.137 0.168 0.171 0.155
Regression coefficient controlling by Table 1 baseline characteristics and district fixed effects. All estimators using SUR
31
Table 7. Impact on test scores, by baseline test score and poverty index
Regression coefficient controlling by Table 1 baseline characteristics and district fixed effects. All estimators using clusters at school level.
33
Table 9. Impact on Enrollment and Attendance on Non-treated peers
Reach Grade Six
Highest Grade Completed
Number of hours in school, last 7 days
(conditional on enrollment)
(1) (2) (1) (2) (1) (2)
Poverty-targeting treatment 0.082**
0.058
0.529
'(0.04)
'(0.11)
'(1.82)
Merit-targeting treatment
-0.009
-0.099
0.291
'(0.05)
'(0.12)
'(1.67)
Constant 0.168 2.201**
4.030** 8.599***
12.924 14.213
'(0.83) '(0.94)
'(1.92) '(1.81)
'(29.16) '(28.46)
Control Variables Yes Yes Yes Yes Yes Yes
No. Obs 785 678
732 633
576 486
F() 7.528 6.94
1.603 1.765
2.166 3.195
R2 Adj 0.172 0.183 0.125 0.118 0.191 0.21
Regression coefficient of dependent variable against treatment indicator controlling by Table 1 baseline characteristics and district fixed effects. All estimators using clusters at school level.
34
Table 10. Impact on test scores on Non-treated Peers
Mathematics
Digispan Test
Raven Test
(1) (2) (1) (2) (1) (2)
Poverty-targeting treatment -0.105
-0.009
-0.09
'(0.07)
'(0.07)
'(0.07)
Merit-targeting treatment
-0.061
-0.048
-0.028
'(0.07)
'(0.08)
'(0.08)
Constant 0.000*** 0.000***
-0.922 -0.302
0.359 2.405
'(0.00) '(0.00)
'(1.93) '(0.00)
'(1.91) '(0.00)
Control Variables Yes Yes Yes Yes Yes Yes
No. Obs 785 678
785 678
785 678
F() 143.244 148.59
101.985 101.983
132.628 97.794
R2 Adj 0.153 0.129 0.115 0.131 0.145 0.126
Regression coefficient of dependent variable against treatment indicator, controlling by Table 1 baseline characteristics and district fixed effects. All estimators using Seemingly Unrelated Regression (SUR) estimation
35
Appendix A. Enrollment and test result, without controls
Impact on Enrollment and Attendance
Reach Grade Six
Highest Grade Completed
Number of hours in school, last 7 days
(conditional on enrollment)
(1) (4) (1) (4) (1) (4)
Poverty-targeting treatment 0.186***
0.349***
3.466*
'(0.04)
'(0.11)
'(1.80)
Merit-targeting treatment
0.131***
0.234**
1.374
'(0.05)
'(0.11)
'(2.01)
Constant 0.613*** 0.635***
5.377*** 5.448***
8.829*** 9.270***
'(0.03) '(0.03)
'(0.09) '(0.08)
'(1.15) '(1.14)
Control Variables No No No No No No
No. Obs 883 940
831 897
665 713
F() 18.154 7.753
9.334 4.572
3.691 0.465
R2 Adj 0.042 0.02 0.025 0.011 0.015 0.002
Columns (1) and (3), regression coefficient of dependent variable against treatment indicator without controls; Columns (2) and (4), controlling by Table 1 baseline characteristics and district fixed effects. All estimators using clusters at school level.
Impact on test scores
Mathematics
Digispan Test
Raven Test
(1) (2) (1) (2) (1) (2)
Poverty-targeting treatment -0.035
-0.019
-0.047
'(0.07)
'(0.07)
'(0.06)
Merit-targeting treatment
0.150**
0.147**
0.109
'(0.07)
'(0.06)
'(0.07)
Constant 0.018 0.165***
0.015 0.081*
-0.023 0.106**
'(0.05) '(0.05)
'(0.05) '(0.05)
'(0.05) '(0.05)
Control No No No No No No
No. Obs 883 940
883 940
883 940
Chi_ 2 0.291 4.925
0.085 5.273
0.54 2.426
R2 Adj 0 0.005 0 0.006 0.001 0.003
Regression coefficient of dependent variable against treatment indicator without controls; All estimators using Seemingly Unrelated Regression (SUR) characteristics
36
Appendix B: Equity and efficiency trade-off
Benefit incidence
The primary scholarships pilot was in part motivated by the fact that secondary school
scholarships failed to reach the poorest of the poor as they had dropped out of school before becoming
eligible for those programs. In order to assess the benefit incidence of the primary scholarships pilot, the
data from the applicants and recipients need to be compared to a national survey. The Demographic and
Health Survey (DHS) data collected in 2010 provide a useful comparison point. These data are national
in scope, and include the same set of variables than were collected from program applicants, and the
timing of the survey overlaps with the Primary Scholarships Pilot.
The following procedure was followed to place the DHS and application form data on a common
metric. First, the DHS variables (e.g. ownership of a motorbike, a car, a pig; material of dwelling floor,
walls, roof) were aggregated using principal components (following the approach described in Filmer and
Pritchett 2001, and Filmer and Scott 2011).24
The first principal component from this procedure is
interpreted as a “wealth index” from which quintiles can be derived.
The first three panels of Figure A1 show the percentage distribution across quintiles of the
Cambodian population of 8-15 year olds, of those in rural areas, and of those in the program Provinces.25
Clearly the population in areas served by the program is poorer than the other areas of the country: while
the distribution of 8-15 year olds mirrors the population as a whole (i.e. 20% in each quintile), and the
rural population is slightly more skewed towards the poorer quintiles (with only 12% of the rural
population being in the richest quintile), the population of the program Provinces is heavily concentrated
among the poorest quintiles. Indeed, 30% of 8-15 year olds within program provinces are in the poorest
quintile, and only 9% are in the richest quintile (Panel 3 of Figure A1).
Considering children who have completed grade 3, however, reduces the share in the poorest
quintile—the poorest of the poor do not even make it to grade 3. While 30% of children in the program
24
The procedure was run on the sample of 8 to 15 year olds to reflect the age of those who applied to the program.
The full set of variables used is: number of household members 0-14; ownership of motorbike, car, oxen/buffalo,
pig; dwelling roof made of hard materials; dwelling walls made of hard materials (e.g. concrete); dwelling walls
made of wood; dwelling floor made of hard materials (e.g. concrete); dwelling floor made of wood; flush toilet, pit
latrine; availability of electricity; drinking water from pipe (either in house or yard). 25
The DHS data do not distinguish between Preah Vihear and Stung Treng, and these numbers therefore include
both Provinces, in addition to Ratanakiri and Mondolkiri.
37
provinces are in the poorest quintile, only 20% of those who have attained grade three are in that quintile
(Panels 3 and 4 of Figure A1).
The variables captured in the scholarship application forms can be aggregated using the same
weights derived from the national sample in the principal components procedure described above.
Applicants can therefore be assigned to nationally-benchmarked quintiles, and the distribution of
applicants and recipients can be compared to the national distribution. The program was able to reach the
poorest schools within the program provinces: 50% of all those who applied to the program are within the
poorest nationally-benchmarked quintile; fewer than 3% of applicants were from the richest quintile
(Panel 5 of Figure A1). Unsurprisingly, targeting the scholarships further to the poorest from within each
school yields an even greater pro-poor distribution of benefits: 85% of applicants who were in the poorest
half in their school (i.e. those targeted by poverty scholarships) were from the poorest two quintiles of the
population—63% were in the poorest quintile (Panel 6). Merit-based targeting is not as pro-poor—but is
still largely able to reach the poorest groups in the population: 76% of applicants who were in the top
merit half of their school (i.e. those targeted by merit scholarships) were from the poorest two quintiles of
the population—54% were in the poorest quintile (Panel 7).26
There are two main conclusions that can be drawn from this analysis. First, compared to
targeting schemes in other countries, the benefit incidence of the scholarships pilot is very pro-poor.27
Of
course, the program was implemented in some of the poorest and remote Provinces and Districts;
therefore the universe from which the merit-based recipients were selected was relatively poor. Scaling
up the program would not necessarily achieve a similarly pro-poor benefit incidence as expansion would
mean operating in less poor areas, and therefore the baseline poverty of the population served would be
less severe. Second, the fact that the benefit incidence of the merit-based approach to targeting is largely
pro-poor (and not particularly less pro-poor than the poverty based approach) suggests that the tradeoff
between equity (i.e. pro-poorness of the program) and efficiency (i.e. the impact on learning outcomes)
might not that stark.
Poverty- versus merit-based targeting at the school level
26
A recent review of programs globally reported the share of Conditional Cash Transfer program benefits that were
received by the poorest 20% of the population (World Bank Social Protection Atlas, http://data.worldbank.org/data-
catalog/atlas_social_protection). Globally the average was that 47% of benefits reached the poorest 20%; with a
range from 24% in Bangladesh, to 58% in Panama. These findings suggest that the Cambodian program performs
relatively very well. 27
For a review of the incidence across a variety of programs see World Bank Social Protection Atlas (forthcoming).
The finding that merit-based targeting did not result in an overall regressive scheme is a
reassuring result. Figure A2 shows that within schools, the association between poverty and test scores is
not as close as might have been feared. The horizontal axis of Figure A2 is the relative poverty ranking
of an applicant, where 0 is the 50th percentile (which was the cutoff for scholarship eligibility in poverty-
targeting schools), +1 is the applicant ranked one position higher on the poverty scale, and -1 is the
applicant ranked one position lower. The vertical axis is the relative ranking of an applicant on the merit
scale, again with 0 being the 50th percentile (which was the cutoff for scholarship eligibility in the merit-
targeting schools).
If only wealthier children were to score high on the merit test, and poorer children low, then all
the observations would be in quadrants (A) and (D) of Figure A2 (respectively: Low Poverty/High Merit
and High Poverty/Low Merit). Clearly this is not the case: the observations are roughly equally
distributed across the four quadrants.28
This means that a merit-based targeting approach (which targets
children in quadrants A and B) includes children from both wealthier backgrounds (quadrant A) as well as
children from poorer backgrounds (quadrant B). Analogously, a poverty-based targeting approach
includes both higher scoring (quadrant B) and lower scoring (quadrant D) applicants.
These school-specific rankings are consistent with the benefit incidence analysis. The
equity/efficiency tradeoff between poverty- and merit-based targeting is not particularly stark.
Nevertheless, if the purely merit-based approach is adopted, it must be borne in mind that roughly half of
the recipients will come from better off families.
28
In fact the regression line for this figure has a mildly positive slope: the regression of relative merit ranking versus
relative poverty ranking yields a coefficient of 1.2 with a standard error of 0.17 (significant at the 1% level).
39
Figure A1: Distribution of selected populations across nationally benchmarked quintiles
Source: Analysis of Cambodia DHS 2010 and Primary Scholarship Application forms. Quintiles are defined on the basis of an index of household wealth-related variables that are collected in both the DHS 2010 as well as on the scholarship program application forms.
Figure A2: The association between applicants’ relative poverty and relative merit rankings