Magnet High Schools and Academic Performance in China: A Regression Discontinuity Design Albert PARK, Xinzheng SHI, Chang-tai HSIEH, Xuehui AN HKUST IEMS Working Paper No. 2015-07 February 2015 HKUST IEMS working papers are distributed for discussion and comment purposes. The views expressed in these papers are those of the authors and do not necessarily represent the views of HKUST IEMS. More HKUST IEMS working papers are available at: http://iems.ust.hk/WP
51
Embed
Magnet High Schools and Academic Performance in China ...HKUST IEMS Working Paper No. 2015-07 February 2015 HKUST IEMS working papers are distributed for discussion and comment purposes.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Magnet High Schools and Academic Performance in China:
A Regression Discontinuity Design
Albert PARK, Xinzheng SHI, Chang-tai HSIEH, Xuehui AN
HKUST IEMS Working Paper No. 2015-07
February 2015
HKUST IEMS working papers are distributed for discussion and comment purposes. The views expressed in these papers are those of the authors and do not necessarily represent the views of HKUST IEMS. More HKUST IEMS working papers are available at: http://iems.ust.hk/WP
Magnet High Schools and Academic Performance in China: A Regression Discontinuity Design Albert PARK, Xingzheng SHI, Chang-tai HSIEH, Xuehui AN HKUST IEMS Working Paper No. 2015-07 February 2015
Abstract
This paper investigates the impact of high school quality on students’ educational attainment using a regression discontinuity research design based on entrance examination score thresholds that strictly determine admission to the best high schools. Using data from rural counties in Western China, we find that attending a magnet school significantly increases students’ college entrance examination scores and the probability of being admitted to college.
Author’s contact information
Albert Park HKUST Institute for Emerging Market Studies, Division of Social Science, and Department of Economics The Hong Kong University of Science and Technology T : +852 2358 5981
Magnet High Schools and Academic Performance in China: A Regression Discontinuity Design*
Albert Park, HKUST
Xinzheng Shi, Tsinghua University
Chang-tai Hsieh, Chicago Graduate School of Business
Xuehui An, National Center for Education Development Research, China Ministry of Education
June 2014
Abstract
This paper investigates the impact of high school quality on students’ educational attainment using a regression discontinuity research design based on entrance examination score thresholds that strictly determine admission to the best high schools. Using data from rural counties in Western China, we find that attending a magnet school significantly increases students’ college entrance examination scores and the probability of being admitted to college. Key Words: Magnet high school; Regression discontinuity design; Academic performance JEL Classifications: I21; I28; O53
* Correspondence author: Xinzheng Shi, [email protected]. Other email addresses: [email protected], [email protected], [email protected]. A previous version of this paper was titled “Does School Quality Matter?: Evidence from a Natural Experiment in Rural China.” The authors thank seminar participants at Oxford, the University of Michigan, and Peking University for helpful comments. They acknowledge grants to support data collection from the University of Michigan Population Studies Center, NIH/Fogharty International Center, and the Economics Department of the University of California at Berkeley. IRB approval was obtained. Xinzheng Shi acknowledges the financial support from the National Natural Science Foundation of China (Project ID: 71103108). All remaining errors are ours.
Ability tracking, a practice that originated in developed countries (Figlio and Page,
2002), is now commonly observed in developing countries, including China.1 In China,
middle school graduates often are tracked into magnet or regular high schools based on
their academic ability as measured by high school entrance examination scores. Magnet
high schools teach students the same curriculum as regular schools but typically have
better teachers supported by greater resources, as well as more talented peers.
Studying the effect of magnet high school attendance on educational performance is
important because it helps to answer a fundamental question in the economics of
education: to what extent does attending a better school affect educational attainment?
The answer to this question is of great interest to policymakers in developing countries,
who often must make trade-offs between improvements in educational quality and the
expansion of access to education when budgetary resources are scarce.
Theoretically, the effect of attending magnet schools on students is ambiguous. On
the one hand, grouping students on the basis of their test scores means that magnet
schools have more academically capable students than regular schools, which can
improve students’ performance through peer effects. In addition to direct knowledge
spillovers among students, for teachers, having high-achieving students means less time
spent on discipline and more time spent on knowledge transmission.2 On the other hand,
attending better schools could affect different students differently. As shown in Duflo,
1 Other developing countries in which such a trend can be seen include, but are not limited to, Kenya, Malawi, Colombia, Romania, and India (Duflo, Dupas, and Kremer, 2011; Lucas and Mbiti, forthcoming; de Hoop, 2010; Saavedra, 2009; Pop-Eleches and Urquiola, 2013; Rubinstein and Sekhri, 2010). 2 A recent study by Ding and Lehrer (2007) carried out in a Chinese county produced strong evidence of positive peer effects. Other studies in this arena include Hoxby (2000), Zimmerman (2003), Angrist and Lang (2004), and Duflo, Dupas, and Kremer (2011). Epple and Romano (2011) provide a detailed review.
Dupas, and Kremer (2011), if students are far from the ability level being targeted by
instruction in magnet schools, attending a magnet school could have a negative effect on
academic performance. Students who are relatively poor performers in a better school
may have less confidence and receive less attention than better performers in lower
quality schools. Thus, the effect of magnet schools on the students is not clear and
requires empirical study.
In this paper, we quantify the impact of magnet high school attendance on students’
educational attainment by exploiting the fact that in many of China’s rural counties
admission to the best high schools is strictly determined by entrance examination scores.
In China, many rural counties operate a magnet school system for high schools. Typically,
a rural county has one academically selective magnet high school as well as a number of
regular (less selective) high schools. Nearly all students resident in a given county attend
one of the county’s middle schools. Graduating middle school students must take county-
wide uniform high school entrance examinations, which determine whether they are
eligible to attend the magnet high school, a regular high school, or no high school at all.
In any given county, the magnet school is usually widely viewed to have the best quality
and has the highest entrance examination score cutoff line.3
We compare students in the same county with nearly identical entrance scores who
attend different quality schools because they are just above or just below the cutoff score
for admission to the magnet school. Our main outcome measure is scores on the national
college entrance examination taken at the end of high school. Using information on the
cutoff lines for college admission, we can also examine whether attending a magnet
3 After graduating from middle school, students take the high school entrance examination. Magnet high schools admit students starting from highest score until they fill their admissions quota. The cutoff line is the lowest score among the students admitted.
school affects the probability of qualifying for college admission. Using data from four
counties in Gansu Province in northwest China, we find that for students with entrance
scores near the cutoff line entering a magnet high school significantly increases students’
college entrance examination scores by 0.39 standard deviations and increases students’
probability of qualifying for college by 27.8 percentage points. However, we do not find
evidence of heterogeneous effects with respect to students' gender or age.
Our paper adds to the existing literature using regression discontinuity (RD) design
to examine the impact of attending better quality schools. Among those that examine the
impact of attending selective secondary schools on test scores, two studies of middle
income countries (Jackson (2010) on Trinidad and Tobago, and Pop-Eleches and
Urquiola (2013) on Romania) find positive effects; several studies in the US and UK find
no effects (Abdulkadiroglu, Angrist, and Phatak (2014), Dobbie and Fryer (forthcoming),
Bui, Craig and Imberman (forthcoming), and Clark (2010)); and two studies set in Sub-
Saharan Africa (Lucas and Mbiti (forthcoming) on Kenya, and Ajayi (2014) on Ghana)
also find no impact or mixed impacts.4 A few studies of selective colleges and selective
classes within schools also find mixed results.5
A possible reason for the mixed results of previous studies is that contextual factors
such as the capabilities of schools, teachers, and families of students may play a critical
role in determining the impact of selective schools on academic performance. In addition,
the RD design identifies impacts for students whose ability is near the threshold for
4 de Hoop (2010) does find a positive impact on school participation in Malawi. 5 Saavedra (2009) finds that attending an elite university in Columbia increases college exit examination scores and Rubinstein and Sekhri (2010) find no evidence of better learning in public (more selective) universities compared to private (less selective) universities in India. Duflo, Dupas, and Kremer (2011) find no impacts on test scores of attending selective classes in primary schools in Kenya, and Ma and Shi (2014) find positive impacts of attending magnet classes within a selective high school in China.
qualifying for selective schools, which may differ across different settings. Studies of
elite high schools in New York and Boston examine impacts on students who are quite
accomplished (only about 10% qualify for selective schools). In such settings, regular
high schools and student’s families may have sufficient capacity to enable students to
realize their full potential even if the student does not attend a more selective school. In
Africa and many other developing countries, families are much poorer, parents are much
less educated, schools struggle to effectively teach students, and learning outcomes are
poor. In our context, evidence of large positive impacts on learning in a poor region of
China, akin to a developing country, contrasts sharply with the results found in African
countries and thus makes a valuable contribution to the literature. Although family
resources and parental education also are lacking in our setting, in China and other Asian
countries, families put great emphasis on education and schools perform well, sometimes
spectacularly as seen in Shanghai’s topping the global PISA rankings.
Our paper is also related to other studies that use different strategies to estimate the
impact of different dimensions of school quality on students’ performance.6 On China,
our results are consistent with those of Ding and Lehrer (2007), who find that attending
high schools with higher ability peers and better teachers increases college entrance
examination scores using data from one county in a rich province (Jiangsu).7 Two other
studies exploit natural experiments associated with admissions lotteries to examine the
6 These strategies include comparison with matched control groups (Angrist and Lavy, 2001; Rockoff, 2004; Rivkin, Hanushek and Kain, 2005), randomized trials to examine the impact of specific schooling inputs, educational grants, or teacher incentive schemes (Dee, 2004; Banerjee, et.al., 2007; Glewwe, Kremer and Moulin, 2009; Muralidharan and Sundararaman, 2011; Duflo, Hanna, and Ryan, 2012), and natural experiments that create plausibly exogenous variation in class size (Angrist and Lavy, 1999; Hoxby, 2000) or in the quality of schools attended, e.g., lotteries (Gould, Lavy and Paserman, 2004; Hoxby, Murarka, and Kang, 2009). 7 They do not employ an RD design but control for selectivity by instrumenting for elite school attendance with the estimated probability of such placement as a function of entrance examination scores and other factors.
impact of attending higher quality middle schools in China, finding mixed results (Zhang,
2013; Lai, Sadoulet and de Janvry, 2011).8 Using a randomized trial in primary schools in
China, Li, et. al. (2014) find that pairing high and low achieving students and offering
them group incentives can increase low achiever’s performance.
The rest of the paper is organized as follows. Section 2 introduces the institutional
backgrounds. Section 3 describes the data and variable definitions. Section 4 presents the
methodology for implementing the RD design and describes the empirical specification.
Section 5 describes students’ assignment to different schools in the sample used for
analysis. This section also presents the results of tests of the continuity of covariates.
Section 6 presents the main empirical results. Section 7 extends the analysis in several
directions, and Section 8 concludes.
2. Institutional backgrounds
2.1. Magnet high school system
China’s pre-college education system includes 6 years of primary school, 3 years of
middle school, and 3 years of high school. Nearly all schools are public schools,
especially in poor, rural regions; and public schools enjoy a much stronger quality
reputation than private schools.9 In China, most counties operate a magnet school system
for public high schools. Typically, a county has one academically selective magnet high
school as well as a number of regular (less selective) high schools. Middle school
graduates are obligated to take an entrance examination before they can be admitted to
8 Zhang (2009) finds no impact of attending selective middle schools on high school entrance examination scores, while Lai, Sadoulet and de Janvry (2011) find a positive effect on high school entrance examination scores in a district in Beijing but mainly for lower ability students. 9 We did not find any private schools in the four counties that are examined in this study.
public high schools. A county-wide uniform high school entrance examination is
administered to all middle school graduates by the county Education Bureau. To gain
admission to magnet high schools, students need to achieve examination scores above the
cutoff line set by these schools. After students take the high school entrance examination,
the county assigns students to the magnet high school starting from the highest score until
the school’s admissions quota is filled. The cutoff line thus is the lowest score among the
students admitted. Students who fail to enter magnet high schools can be admitted by
regular high schools depending on whether their scores are higher than the cutoff lines set
by the regular high schools.10 If their scores are below the regular high school cutoff lines,
they can attend vocational high schools (which typically have no cutoff) or exit schooling.
Increasing effort during or prior to the test when expected scores are near the cutoff is not
possible because the cutoffs are set only after the test scores are calculated based on the
entire distribution of scores. Given the importance placed by all parties on high school
placements, county Education Bureaus generally follow strict procedures to ensure the
integrity of the grading of examinations and recording of examination scores, making
manipulation of such scores unlikely.11
Although magnet and regular high schools teach the same curriculum, they differ in
many dimensions. To assess the extent of these differences, we analyze school-level data
10 In 2004, the share of middle school graduates who went to high school in Gansu (not including vocational schools) was 47 percent, compared to 39 percent for all of China based on data from Ministry of Education (2005). In general, middle school graduates can be admitted only to high schools located within the county or district in which they reside. A few elite students may qualify for outstanding high schools in the municipal or provincial capital cities, and some students may attend high schools in other counties or districts if their parents move or have special connections. 11 Required high school tuition and other fees are set by schools with the approval of county Education Bureaus, and may be more expensive for magnet high schools compared to regular high schools. Many schools provide limited scholarships for students from poor families.
on a variety of quality indicators.12 Table 1 presents the results of simple regressions of
school quality indicators on a dummy variable for whether the school is a magnet school
and county-year fixed effects, as well as sample means for regular high schools. All of
the differences are statistically significant. In magnet schools, the share of teachers with
highest quality ranks, which are based on annual teaching evaluations throughout a
teacher’s career, is 0.10 greater than in regular schools, which have a mean share of only
0.07 (column 1).13 Teacher quality ranks have been found to strongly predict differences
in student test scores (Hannum and Park, 2001). The share of teachers with four-year
college education is 0.42 greater in magnet schools than in regular schools, whose share
is only 0.34 (column 2). Class size in magnet schools is greater by about 9 students, or 17
percent (column 3). Magnet schools have 852 more students (or 111 percent) than regular
schools, are larger in area by 52 thousand square meters (208 percent), have 67,720 more
library books (1026 percent), and are 52.6 percentage points more likely to meet national
criteria for adequate school facilities (only 32 percent of regular schools meet this
standard). Thus, magnet schools are superior to regular high schools for a host of
observable quality indicators.
2.2. College admission
In order to be admitted to colleges, Chinese high school graduates are required to
take the nationally standardized College Entrance Examination (CEE). The total CEE
12 Using annual data on schools collected from questionnaires, we measure school quality for each class by 4-year average values of school indicators that span the years that the class attended the school. For example, for students starting high school in September 1997 and graduating in June 2000, we take mean values for the years 1997 to 2000. 13 There are three levels of quality ranks for high school teachers in China: from lowest to highest, a second degree title, first degree title, and advanced title. These titles are awarded primarily on the basis of the educational degrees that teachers have obtained and their number of years of teaching experience. Additionally, there are several requirements regarding their teaching achievements.
score is the main criterion used for college admissions.14 A distinct feature of Chinese
college admission is that colleges are categorized into different tiers and those belonging
to a higher tier are afforded first priority in admitting students. Students submit their
college preferences (4–6 schools in each tier) and favored majors in order of priority, and
are assigned to a university and major based on these preferences and their college
entrance examination score.15 Students then accept the offer or decline, in which case
they will not attend college that year. Many universities have quotas for the number of
students admitted from each province. Given the fixed supply of university openings for
students from each province, there is a minimum cutoff score required for students in
each province to secure a position in a university.
3. Data and variables
The data used in this paper were collected from high schools in rural counties in
Gansu Province in western China during the summer of 2005. Gansu is one of China’s
poorest provinces, with a population of 26 million and GDP per capita of $744 in 2004
which ranked 30th among China’s 31 provinces (National Bureau of Statistics, 2005).
Data was initially collected by graduate students from Northwest Normal University who
approached high schools in a set of randomly selected counties. We were able to obtain
data suitable for analysis in nine counties (and 25 county-years, all for entering classes
14 Applicants to some special programs are screened by additional criteria: some art departments (e.g., audition), military and police schools (political screening and physical exam), and some sports programs (tryout). 15 Most provinces use an admission procedure similar to the Boston Mechanism. In the first round, each college considers only students who list it as their first choice. Students with scores above a threshold score are accepted and the rest are rejected and placed in a pool of candidates for to be considered by the college next on students’ lists of preferences. Only if there are remaining slots after the first round will a college consider admitting students who list it as their second or third choice. Once a college offers admission to a student, the selection process ends and the students are not considered by any other colleges.
from 1997 to 2001).16 These nine counties vary substantially in GDP per capita; on
average they are somewhat poorer than the province as a whole (mean GDP per capita in
2003 was 80% of the provincial as a whole).17 China’s high schools have three grade
levels, so all students in the sample had completed high school and taken college entrance
examinations by the time of the survey.
Given our identification strategy, we focus on counties in which there is strong
evidence that the cutoff lines are actually used to determine admission to the magnet
school. Each county has discretion in how to run its admissions policy, so there is no
guarantee that cutoff lines are strictly enforced in practice in every county. To verify
whether the cutoff line is strictly enforced, for each county we regress an indicator for
entering magnet high school on an indicator for having high school entrance examination
scores higher than the cutoff after controlling for a female dummy, age, middle school
fixed effects, year fixed effects, and a polynomial function of high school entrance
examination score relative to the cutoff. The order of the polynomial function is
determined using the Akaike information criterion (AIC) as in Lee and Lemieux (2010).
According to the results, four of the nine counties strictly enforce the cutoff line for
admission to magnet schools, meaning that having a high school entrance examination
score just above the cutoff line significantly increases the probability of entering the
magnet high school.18 We therefore focus our main analysis on all data available for these
four counties, which includes data on students from 20 high schools in 13 county-years.
16 Gansu has 86 counties. Data was not available for all years in each county due to differences in the quality of record keeping in different schools and counties. 17 Calculated from county data on GDP and population reported in Gansu Bureau of Statistics (2004). 18 Among the 9 counties with suitable data, the four counties that enforce the cutoff line are ranked 1, 2, 5, and 8 in terms of GDP per capita.
Given our sample selection criteria, strictly speaking our estimates capture the
impact of attending magnet schools in counties that strictly enforce entrance examination
score cutoff lines. If enforcement of the cutoff line in a given county is endogenous to the
quality difference between magnet and regular schools, our estimates are likely to be
upper bound estimates for the impact of attending magnet schools in counties that did not
strictly enforce the cutoff lines. However, analysis of the school data does not provide
any evidence that the observable quality differences between magnet and regular high
schools is different in counties that do and do not enforce the cutoff lines. We regress
different measures of school quality on a magnet school dummy and the interaction of the
magnet school dummy and a dummy indicating counties having binding cutoff line after
controlling for county-year fixed effects.19 Results are presented in Appendix Table 1.
None of the coefficients on the interaction terms are statistically significant, which
suggests that the magnet-regular school quality difference does not differ significantly
between counties having binding cutoff lines and those without binding cutoff lines.
We observe the high school entrance examination score for nearly 100 percent of
students in the sample but only for 62 percent of those with high school entrance
examination scores do we have data on the student’s college entrance examination
score.20 Missing data on college entrance examination scores can be due to multiple
reasons: the student could have dropped out or transferred to another school, or decided
not to sit for the college entrance examination; or the school could have kept incomplete
19 As in Section 2.1, we measure school quality for an entering class by the 4-year average values of school indicators that span the years that they attended the school. For example, for students starting high school in September 1997 and graduating in June 2000, we take mean values for the years 1997 to 2000. 20 The sample includes all students beginning high school in each year; students transferring into the high school after the first year are excluded but such transfers are relatively rare. The sample excludes students who take the high school entrance examination but do not attend high school; however, because nearly all students who get into any high school choose to enroll, this is unlikely to create sample selection bias among students whose entrance scores are near the cutoff lines for entering magnet high schools.
records. In one school we visited, college entrance examination scores had been kept only
for those who had scored high enough to enter college. One concern that arises with
missing college examination scores is that our estimates of the impact of attending a
magnet school on college examination scores could suffer from bias caused by
differences in the selectivity of who have college entrance examination scores in magnet
schools and in regular schools. However, our estimate results show that for students
around the cutoff line whether they attend magnet schools or regular schools does not
have a statistically significant impact on their probability of having a college entrance
examination score (see Section 7.1), suggesting that such selection bias is not likely to be
a major concern.21 In this paper, we focus on 5373 students having college entrance
examination scores.
The survey collected school administrative data on students’ gender, birth year,
year of high school entrance, high school entrance examination score, and college
entrance examination score. The survey also collected data from schools on the high
school entrance examination score cutoff line and school characteristics such as teachers’
educational attainment and the availability and quality of different types of school
facilities.
Two treatment variables are defined. The variable magnet is assigned to equal one
if the student actually attended a magnet school. The other treatment variable eligible is
assigned to equal one if the student’s high school entrance score was higher than the high
school entrance examination cutoff line of the magnet school. While magnet more
accurately reflects whether students actually attended better schools, it is subject to
21 In supplementary regressions (not reported), we also find that the relationship between high school examination scores and having the college entrance examination score is not significantly different in magnet and regular high schools.
Another educational attainment variable is an indicator variable for whether the
student is qualified to attend college. We compare the student’s college entrance
examination score with the lowest college admission cutoff lines in Gansu Province in the
year the student took the college entrance examination in order to determine whether the
student qualified for college.22 This measure is highly correlated with whether students
actually attended colleges.23 However, this measure is not subject to selection biases
associated with the student’s decision to actually attend college conditional on his or her
entrance examination score being above the cutoff line. Such decisions could be
influenced by credit constraints, family income and wealth, parental expectations, and
other factors that could be correlated with learning outcomes.
Table 2 gives summary statistics for the variables used in the analysis. Thirty six
percent of students are female and their average age is 15. Fifty four percent of students
attend magnet schools and 53 percent are eligible to attend magnet schools. Among all
students, 30 percent take the liberal arts track. About 50 percent of students have college
entrance examination scores that make them eligible for college entrance.
4. Methodology
We employ an RD design to quantify the impact of school quality on educational
attainment. First developed by Thistlethwaite and Campbell (1960), in recent years there
has been an explosion of interest in applying RD design to a range of empirical questions
22 The cutoff lines from different provinces come from http://www.eol.cn/include/cer.net/gaokao/zhuanti/2006_fenshuxian.shtml#2000. There are different lowest cutoff lines for different types of college entrance examinations. 23 In Gansu Province, the share of students having college entrance examination scores higher than the lowest cutoff lines who enroll in colleges are 94% in 2000, 94% in 2001, 92% in 2002, and 93% in 2003. These numbers are calculated from college entrance examination data files (2000-2003) provided by the Economic and Social Data Center in Tsinghua University.
influence the impact estimates, increasing potential bias. To choose an optimal bandwidth
that balances these factors, we follow the cross-validation procedure suggested by Imbens
and Lemieux (2008). 25 The main idea is to predict outcome values by estimating
nonparametric local linear regressions using a “leave one out” procedure for different
possible bandwidths, and to choose the bandwidth that minimizes the mean square
residuals for each regression specification. The method is carried out separately for
observations on either side of the cutoff line.26
We conduct several tests of the assumptions that underpin the RD specification.
Lee (2008) proposes a direct test of the continuity assumption by checking whether there
are discontinuities in the relationship between the treatment effect and any predetermined
covariates. That is, the following equation can be estimated:
(6)
If is not statistically significant, then the continuity assumption is valid. We test for
three predetermined covariates: gender, age, and the quality of middle school attended,
which is measured by the average high school entrance examination score of students
attending the same middle school in the same year.27
In the RD design, treatment depends on the selection variable S in a deterministic
way. However, in reality, it is likely for treatment assignment to depend on S in a
stochastic manner, which is referred to in the literature as fuzzy RD design. In our main
sample, 11.7 percent of students not in magnet schools have high school entrance
25 Please see Imbens and Lemieux (2008) or Lee and Lemieux (2010) for a detailed description of the cross-validation method. The method includes all of the covariates in the estimated models. 26 Detailed summary statistics on mean square residuals are not reported due to space limitations but are available from the authors upon request. 27 We should note that our measure of middle school quality is not perfect since we only collected information on students attending high schools; therefore, it is an upward biased estimate of the middle school quality.
examination scores above the cutoff line, and 10.9 percent of students in magnet schools
have high school entrance examination scores below the cutoff.28 In this case, the OLS
estimate of in equation (5) using the variable magnet could be subject to selection bias.
This is where the second treatment variable eligible can help avoid the problems
associated with bias caused by fuzzy RD design. The variable eligible itself does not
suffer from fuzziness and so can be used to cleanly estimate an intent-to-treat effect.
However, the impact of eligibility is not of primary interest. Our goal is to estimate the
impact of actually attending better schools. To obtain an unbiased estimate of this effect,
we can use eligible as an instrument for magnet, since eligible strongly predicts magnet
but is not subject to selectivity bias. If we believe that the slope of the relationship
between the outcome variables and the high school entrance examination score differs to
the right and left of the cutoff because the relationship is different in magnet schools and
regular schools, then one can capture this difference by interacting magnet with the
polynomial terms of and instrumenting these interactions with the interactions of
eligible with the same polynomial terms. We note that, conditional on the validity of the
IV, our estimates apply only to students complying with the assignment rule, for whom
we identify a local average treatment effect (LATE).
5. Student assignment and continuity of covariates
28 There are several possible reasons. One is that parents or teachers influence high school placement decisions using personal connections. Many schools even establish explicit systems to allow parents to pay extra fees to enable their children to attend their schools if their children’s test scores are just below the cutoff, although the extent of such practice was limited during the time period covered by the data. In such systems, normally the amount of extra fees is a function of how far the student’s entrance examination score is from the cutoff, with very poor students being excluded altogether because of the school’s concern to maintain its quality reputation. On the contrary, some students having scores higher than the cutoff could decide not to attend magnet high schools because they cannot afford the tuition charged by magnet high schools or they live in remote villages such that the transportation costs are too high.
Students are assigned to magnet schools and regular schools according to their
high school entrance examination score. Figure 1 shows the distribution of students with
different high school entrance scores in magnet and regular schools. In order to pool data
from different county-years, we create a variable that indicates each student’s score
relative to the entrance cutoff score in each county-year, which is shown on the x-axis. Y-
axis shows the share of students enrolled in the magnet school. We plot this share for
students with entrance scores falling in equidistant bins, plotted against the midpoint of
each bin.29 The figure highlights the fact that there is a sharp change in the probability of
treatment close to the cutoff. However, Figure 1 also reveals that, in practice, the cutoff
line is not adhered to in all cases; if it were, then the gap at the cutoff line would be equal
to one. Because of this fuzziness in the implementation of cutoff lines, in the following
analysis, our preferred results come from the regressions using eligible as an instrument
for magnet.
We also conduct regressions to estimate the impact of eligible on magnet. Columns
1 and 2 in Table 3 use the sample consistent with that used in Figure 1, controlling only
for first and second order polynomial functions of students' high school entrance
examination score relative to the cutoff, respectively. Columns 3 and 4 correspond to the
different samples used to explain the two outcome variables: the college entrance
examination score and the probability of qualifying for college. Therefore, these two
columns are also the first stage regression results for the main regressions (shown in
Table 5). A female dummy, age, middle school fixed effects, county-year fixed effects,
29 The bin size width used is 0.3. For this figure and those reported afterwards, in order to ensure that the bin size width does not hide significant outcome differences within bins, we verify that the bin size passes a simple test in which bin dummies and interactions of bin dummies with the running variable are included and the coefficients on the interaction terms are jointly statistically insignificant (Lee and Lemieux, 2010).
and a first order polynomial function of students' high school entrance examination score
relative to the cutoff are controlled for in columns 3 and 4. Table 3 shows that the
coefficient of eligible is statistically significant at the 1% level in all columns. Depending
on the specification, the results imply that having a high school entrance examination
score just above the cutoff increases the probability of entering a magnet high school by
33.1 to 51.7 percentage points. The last row presents F-values for the null hypothesis that
the coefficients of eligible and the interactions of eligible and polynomial terms of
are equal to zero. The F-values are 121.14 (column 3) and 111.59 (column 4).
These results suggest that eligible is an extremely strong predictor of actually enrolling in
a magnet school, justifying its use as an instrument.
Next, we report results for tests that examine whether the three predetermined
covariates jump in a discontinuous fashion at the entrance examination cutoff line. In
Figure 2, the x-axis measures the difference between the high school entrance
examination scores and the cutoff line for each county-year; the y-axis measures
proportion of female students in Panel A, age in Panel B, and middle school quality
(measured by the average high school entrance examination score of students attending
the same middle school in the same year) in Panel C.30 The samples used for Figure 2 are
the same as those used in the regressions in Table 4, which are determined by the cross-
validation method described earlier. It is evident that for all three variables there is no
jump at x=0, the point at which the high school entrance examination score is equal to the
cutoff line. This provides support for the validity of the RD design.
30 We plot the mean values for students with entrance scores falling in equidistant bins, plotted against the midpoint of each bin. The bin size widths used in panels A, B and C are 0.08, 0.08, and 0.016, respectively.
References Abdulkadiroglu, Atila, Joshua D. Angrist, and Parag A. Pathak (2014).“The elite illusion: Achievement effects at Boston and New York exam schools,” Econometrica, Vol. 82, No.1: 137-196. Ajayi, Kehinde F. (2014), “Does School Quality Improve Student Performance? New Evidence from Ghana”, mimeo. Angrist, Joshua D., and Kevin Lang (2004). Does school integration generate peer effects? Evidence from Boston’s Metco Program, American Economic Review, 94(5): 1613-1634. Angrist, Joshua D. and Victor Lavy (1999).“Using Maimonides’ rule to estimate the effect of class size on scholastic achievement,” The Quarterly Journal of Economics, Vol. 114, No. 2, 533- 575. Angrist, Joshua D. and Victor Lavy (2001). “Does teacher training affect pupil learning? Evidence from matched comparisons in Jerusalem public schools,” Journal of Labor Economics, Vol. 19, No. 2, 343-369. Banerjee, Abhijit V., Shawn Cole, Esther Duflo, and Leigh Linden (2007). “Remedying education: Evidence from two randomized experiments in India,” The Quarterly Journal of Economics, Vol. 122, No, 3, 1235-1264. Bui, Sa A., Steven G. Craig, and Scott A. Imberman. (forthcoming). "Is gifted education a bright idea? Assessing the impact of Gifted and Talented programs on Students," American Economic Journal-Economic Policy. Clark, Damon (2010). “Selective Schools and Academic Achievement,” The B.E. Journal of Economic Analysis & Policy, Vol. 10, No. 1 (Advances), Article 9. Dee, Thomas S. (2004). “Teachers, race and student achievement in a randomized experiment,” The Review of Economics and Statistics, Vol. 86, No.1, 195-210. de Hoop, Jacobus. (2010). “Selective Secondary Education and School Participation in Sub-Saharan Africa: Evidence from Malawi,” Discussion Paper 2010-041/2, Tinbergen Institute. Ding, Weili, and Steven F. Lehrer (2007). “Do Peers Affect Student Achievement in China’s Secondary Schools?,” The Review of Economics and Statistics, Vol.89, No. 2,300–312. Dobbie, Will, and Roland Fryer (forthcoming). “The Impact of Attending a School with High-Achieving Peers: Evidence from the New York City Exam Schools,”American Economic Journal: Applied Economics.
Duflo, Esther, Pascaline Dupas, and Michael Kremer (2011).“Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a randomized evaluation in Kenya,” American Economic Review, Vol. 101, No. 5,1739-74. Duflo, Esther, Rema Hanna, and Stephen P. Ryan (2012).“Incentives Work: Getting Teachers to Come to School”, American Economic Review, Vol. 102, No. 4, 1241-1278. Epple, Dennis, and Ricard Romano (2011). Peer effects in education: A survey of the theory and evidence. Handbook of Social Economics, 1(11): 1053-1163. Figlio, David, and Marianne Page (2002). School choice and the distributional effects of ability tracking: Does separation increase inequality?, Journal of Urban Economics, 51: 497-514. Gansu Bureau of Statistics (2004). Gansu Statistical Yearbook 2004 (Beijing: China Statistical Press). Glewwe, Paul, Michael Kremer, and Sylvie Moulin (2009). “Many children left behind? Textbooks and test scores in Kenya,” American Economic Journal: Applied Economics,Vol.1, No. 1, 112-135. Gould, Eric D., Victor Lavy, and M. Daniele Paserman (2004). “Immigrating to opportunity: Estimating the effect of school quality using a natural experiment on Ethiopians in Israel,” The Quarterly Journal of Economics, Vol. 119, No. 2, 489-526. Hahn, Jingyong, Petra Todd, and Wilbert van der Klaauw (2001). “Identification and estimation of treatment effects with a regression-discontinuity design,” Econometrica, Vol. 69, No. 1, 201-209. Hannum Emily and Albert Park (2001). “Do teacher characteristics affect student learning in developing countries?: Evidence from matched teacher-student data from rural China”, mimeo. Hoxby, Caroline M. (2000). “The effects of class size on student achievement: New evidence from population variation,” The Quarterly Journal of Economics, Vol. 115, No. 4, 1239-1285. Hoxby, Caroline M., Sonali Murarka, and Jenny Kang (2009). “How New York City’s Charter Schools Affect Achievement,” Report for the New York City Charter Schools Evaluation Project. Imbens, Guido W., and Thomas Lemieux(2008).“Regression Discontinuity Designs: A Guide to Practice,” Journal of Econometrics, Vol. 142, No. 2, 615–635.
Jackson, C. Kirabo (2010). “Do Students Benefit from Attending Better Schools?: Evidence from Rule-based Student Assignments in Trinidad and Tobago,” The Economic Journal,120: 1399–1429. Karachiwalla, Naureen, and Albert Park (2014). “Promotion Incentives in the Public Sector: Evidence from Chinese Schools”, mimeo. Lai, Fang, Elisabeth Sadoulet, and Alain de Janvry. (2011). “The contributions of school quality and teacher qualifications to student performance: Evidence from a natural experiment in Beijing middle school,” Journal of Human Resources, Vol. 46, No.1, 123-153. Lee, David S. (2008). “Randomized experiments from non-random selection in U.S. House Elections,” Journal of Econometrics, Vol. 142, 675-697. Lee, David S., and Thomas Lemieux (2010).“Regression Discontinuity Design in Economics,” Journal of Economic Literature, 48, 281-355. Lee, David S., and David Card (2008). "Regression discontinuity inference with specification error," Journal of Econometrics, 142: 655-674. Li, Tao, Li Han, Linxiu Zhang, and Scott Rozelle (2014). “Encouraging classroom peer interactions: Evidence from Chinese migrant schools,” Journal of Public Economics, 111: 29-45. Lucas, Adrienne M, and Isaac M. Mbiti (forthcoming). “Effects of school quality on student achievement: Discontinuity evidence from Kenya,” American Economic Journal: Applied Economics. Ma, Mingming, and Xinzheng Shi (2014).“Magnet classes and educational performance: Evidence from China”, Economic Development and Cultural Change, Vol. 62, No. 3, 537-566. Ministry of Education (2005). China Education Statistics Yearbook 2005 (Beijing: People’s Education Press). Muralidharan, Karthik, and Venkatesh Sundararaman (2011). “Teacher Performance Pay: Experimental Evidence from India,” Journal of Political Economy, Vol. 119, No. 1, 39-77. National Bureau of Statistics (2005). Chinese Statistical Yearbook. Pop-Eleches, Christian, and Miguel Urquiola (2013). “Going to a Better School: Effects and Behavioral Responses,” American Economic Review, Vol. 103, No. 4, 1289-1324. Porter, Jack (2003). “Estimation in the regression discontinuity model,” mimeo.
Rivkin, Steven G., Eric A. Hanushek, and John F. Kain (2005).“Teachers, schools, and academic achievement,” Econometrica, Vol. 73, No. 2, 417-458. Rockoff, Jonah E. (2004).“The impact of individual teachers on student achievement: Evidence from panel data ,” American Economic Review, Vol. 94, No. 2, 247-252. Rubinstein, Yona, and Sheetal Sekhri (2010). “Do public colleges in developing countries provide better education than private ones? Evidence from general education sector in India,” mimeo. Saavedra, Juan Esteban. (2009). “The learning and early labor market effects of college quality: A regression discontinuity analysis,” mimeo. Thistlethwaite, Donald L. and Donald T. Campbell (1960).“Regression-discontinuity analysis: an alternative to the ex post facto experiment,” Journal of Educational Psychology, Vol. 51, No. 6, 309-317. Zhang, Hongliang (2013). “The mirage of elite schools: Evidence from lottery-based school admission in China,” mimeo. Zimmerman, David J. (2003). Peer effects in academic outcomes: Evidence from a natural experiment, The Review of Economics and Statistics, 85(1): 9-23.
Note: (1) Dots in the graph are share of students enrolled in magnet high schools in equidistant bins with the width equal to 0.3. The dots are plotted against the midpoint of each bin.
0.2
.4.6
.81
Sh
are
of s
tude
nts
enr
olle
d in
mag
net h
igh
sch
ool
-2 -1 0 1 2High school entrance examination score relative to the cutoff
Note: (1) Middle school quality for a student is measured by average high school entrance examination score of students attending the same middle school in the same year. (2) Dots in Panel A are the mean values of female dummy for students having high school entrance examination score (relative to the cutoff) in equidistant bins with the width equal to 0.08. Dots in Panel B are the mean values of age for students having high school entrance examination score (relative to the cutoff) in equidistant bins with the width equal to 0.08. Dots in Panel C are the mean values of the middle school quality measurement for students having high school entrance examination score (relative to the cutoff) in equidistant bins with the width equal to 0.016. The dots are plotted against the midpoint of each bin. (3) The bandwidth of the neighborhood around the cutoff line in each figure is consistent with that used for the corresponding outcome variable in Table 4, which is chosen by the cross-validation method.
Regular
Magnet
-2-1
01
2M
idd
le s
choo
l qu
ality
-.2 -.1 0 .2 .4High school entrance examination score relative to the cutoff
Note: (1) Dots in Panel A are the mean values of college entrance examination score for students having high school entrance examination score (relative to the cutoff) in equidistant bins with the width equal to 0.06. Dots in Panel B are share of students qualifying for college for students having high school entrance examination score (relative to the cutoff) in equidistant bins with the width equal to 0.06. The dots are plotted against the midpoint of each bin. (2) The bandwidth of the neighborhood around the cutoff line in each figure is consistent with that used for the corresponding outcome variable in Table 5, which is chosen by the cross-validation method.
Table 1 Different characteristics between magnet schools and regular schools
(1) (2) (3) (4) (5) (6) (7)
Ratio of teachers having
advanced title
Ratio of teachers having
education of four year
college
Class size
No. of students
Campus area
(10000 square meters)
No. of books in library (10000 units)
Does equipment
satisfy criteria
Magnet school=1
0.101 0.424 8.883 851.773 5.171 6.772 0.526
(0.047)** (0.094)*** (4.524)* (166.235)*** (0.693)*** (1.070)*** (0.191)**Observations 57 55 58 55 51 46 43 Regular school 0.074 0.341 52.814 767.803 2.523 0.660 0.324 Robust standard errors in parentheses; * significant at 10%; ** significant at 5%; *** significant at 1% Note: (1) All regressions include county-year fixed effects. (2) The row of "Regular school" shows the mean values of the dependent variables for regular schools.
High school entrance examination score 0.098 0.957 5373
Magnet 0.535 0.499 5373
Eligible 0.531 0.499 5373
Taking liberal arts track 0.295 0.456 5373
College entrance examination score -0.085 1.035 5373
Eligible for college 0.497 0.500 5373 Note: (1) Magnet is a dummy variable with one representing attending magnet high school and zero otherwise. Eligible is also a dummy variable with one representing having high school entrance examination scores equal to or higher than the cutoff line of magnet high school and zero otherwise. The definitions apply to all other tables.
Bandwidth used [-2,2] [-2,2] [-1.8, 1.1] [-1.4, 1.1] Observations 4986 4986 4478 4155 R-squared 0.60 0.61 0.68 0.66 IV validity (F-value) 261.28 376.18 121.14 111.59 Standard errors in parentheses are calculated by clustering over county-high school entrance examination score. * significant at 10%; ** significant at 5%; *** significant at 1% Note: (1) In columns 3 and 4, we control for the county-year fixed effects, and middle school fixed effects. (2) In columns 1, 3 and 4, a first order polynomial function of students' high school entrance examination score relative to the cutoff is controlled, respectively. In column 2, a second order polynomial function of students' high school entrance examination score relative to the cutoff is controlled. (3) Bandwidth used in columns 1 and 2 is consistent with that used in Figure 1. Bandwidth in column 3 is consistent with that used in columns 1-3 in Table 5. Bandwidth in column 4 is consistent with that used in columns 4-6 in Table 5.
Age -0.049 -0.050 0.010 0.024 (0.014)*** (0.014)*** (0.014) (0.021)
Bandwidth used [-2.5,1] [-2.5,1] [-1,3.1] [-1,3.1] [-0.2,0.4] [-0.2,0.4] Observations 4648 4648 4174 4174 1388 1388 R-squared 0.05 0.05 0.39 0.39 0.24 0.26 Standard errors in parentheses are calculated by clustering over county-high school entrance examination score. * significant at 10%; ** significant at 5%; *** significant at 1% Note: (1) Middle school quality for a student is measured by the average high school entrance examination score of students attending the same middle school in the same year. (2) In all columns, we control for a first order polynomial function of students' high school entrance examination scores relative to the cutoff, and county-year fixed effects. In columns 1 to 4, we also control for middle school fixed effects. (3) The bandwidths used in this table are chosen by the cross-validation method for outcome variables, respectively.
Bandwidth used [-1.8,1.1] [-1.8,1.1] [-1.8,1.1] [-1.4,1.1] [-1.4,1.1] [-1.4,1.1] Observations 4478 4478 4478 4155 4155 4155 R-squared 0.32 0.31 0.32 0.26 0.26 0.25 Standard errors in parentheses are calculated by clustering over county-high school entrance examination score. * significant at 10%; ** significant at 5%; *** significant at 1% Note: (1) In all regressions, we control for middle school fixed effects, county-year fixed effect, and a first order polynomial function of students' high school entrance examination scores relative to the cutoff. (2) The bandwidths used in this table are chosen by the cross-validation method for outcome variables, respectively.
(0.046) (0.035)* (0.028) (0.016) Age -0.109 -0.031 -0.076 -0.024
(0.029)*** (0.015)** (0.044)* (0.027)
Bandwidth used [-1.8,1.1] [-1.4,1.1] [-1.8,1.1] [-1.4,1.1] Observations 4478 4155 4478 4155 R-squared 0.32 0.25 0.32 0.25 Standard errors in parentheses are calculated by clustering over county-high school entrance examination score. * significant at 10%; ** significant at 5%; *** significant at 1% Note: (1) In all regressions, we control for middle school fixed effects, county-year fixed effects, and a first order polynomial function of students' high school entrance examination score relative to the cutoff. (2) Bandwidth used in columns 1 and 3 is consistent with that used in columns 1-3 in Table 5. Bandwidth used in columns 2 and 4 is consistent with that used in columns 4-6 in Table 5.
Age -0.012 -0.012 -0.008 (0.007)* (0.007)* (0.007)
Bandwidth used [-1.8,1.1] [-1.4,1.1] [-1.5,2.4] Observations 7032 6331 7202 R-squared 0.30 0.29 0.28 Standard errors in parentheses are calculated by clustering over county-high school entrance examination score. * significant at 10%; ** significant at 5%; *** significant at 1% Note: (1) In all regressions, we control for middle school fixed effects, county-year fixed effects, and a first order polynomial function of students' high school entrance examination score relative to the cutoff. (2) Bandwidth in column 1 is consistent with that used in columns 1-3 in Table 5. Bandwidth in column 2 is consistent with that used in columns 4-6 in Table 5. Bandwidth in column 3 is chosen by the cross-validation method for the dependent variable.
Bandwidth used [-1.8,1.1] [-1.4,1.1] [-1.2,2.4] Observations 4478 4155 4395 R-squared 0.11 0.11 0.10 Standard errors in parentheses are calculated by clustering over county-high school entrance examination score. * significant at 10%; ** significant at 5%; *** significant at 1% Note: (1) In all regressions, we control for the middle school fixed effects, county-year fixed effects, and a first order polynomial function of students' high school entrance examination scores relative to the cutoff. (2) Bandwidth used in column 1 is consistent with that used in columns 1-3 in Table 5. Bandwidth used in column 2 is consistent with that used in columns 4-6 in Table 5. Bandwidth used in column 3 is chosen by the method of cross-validation for the dependent variable.
Age -0.117 -0.128 -0.029 -0.035 (0.028)*** (0.030)*** (0.013)** (0.015)**
Bandwidth used [-1.4,1.1] [-1.1,1.1] [-1.8,1.1] [-1.1,1.1] Observations 4155 3591 4478 3591 R-squared 0.29 0.27 0.27 0.23 Standard errors in parentheses are calculated by clustering over county-high school entrance exam score. * significant at 10%; ** significant at 5%; *** significant at 1% Note: (1) In all regression, we control for middle school fixed effects, county-year fixed effect, and a first order polynomial function of students' high school entrance examination scores relative to the cutoff.
Age -0.032 -0.037 -0.043 (0.012)*** (0.013)*** (0.014)***
Bandwidth used [-1.8,1.1] [-1.4,1.1] [-1.2,1] Observations 7032 6331 5765 R-squared 0.21 0.19 0.18 Standard errors in parentheses are calculated by clustering over county-high school entrance exam score. * significant at 10%; ** significant at 5%; *** significant at 1% Note: (1) The dependent variable is an indicator. It is equal to 1 if the student's college entrance examination score is equal to or higher than the lowest cutoff line for entering the college, and it is equal to 0 if the student's college entrance examination score is lower than the cutoff line or the student does not have a college entrance examination score. In other words, students having missing college entrance examination scores are assumed not to be eligible for colleges. (2) In all regressions, we control for middle school fixed effects, county-year fixed effects, and a first order polynomial function of students' high school entrance examination score relative to the cutoff. (3) Bandwidth used in column 1 is consistent with that used in columns 1-3 in Table 5. Bandwidth used in column 2 is consistent with that used in columns 4-6 in Table 5. Bandwidth used in column 3 is chosen by the cross-validation method for the dependent variable.