Educar en el s XXI. UIMP 2013. Does Sorting Students Improve Scores? An Analysis of Class Composition

Does Sorting Students Improve Scores?  Does Sorting Students Improve Scores?  An Analysis of Class Composition

Courtney A. Collins and Li GanJuly 2013

Sorting Students into ClassesSorting Students into Classes Related literature on education inputs  Class size, teacher quality, technology Implementation requires significant spending increases Ex: Florida class size reduction legislation ($24 billion)

Sorting students into existing classrooms Dividing students into classes based on various 

hcharacteristics Reallocation of existing resourcesP i l f i f d Potential performance gains for students

Minimal additional costs

Tracking Effects and Peer EffectsTracking Effects and Peer Effects Heterogeneity in the effect of sorting: do some students gain at the cost of others?gain at the cost of others?

Tracking Effects Efficiency gains for teachers caused by narrower distribution of students

Beneficial for both high ability and low ability students Beneficial for both high ability and low ability students

Peer Effects Students are affected by the quality of their peers Students are affected by the quality of their peers Homogeneous sorting directly impacts peer quality High scoring students benefitg g Low scoring students lose

Estimate overall net effect for high and low ability students Estimate overall net effect for high and low ability students

Sorting Students into ClassesSorting Students into Classes How does sorting students into ability groups affect 

d i f d b t t ?academic performance as measured by test score?

Sorting along different dimensionsg g Test score Gifted and Talented classification Special Education classification Limited English Proficiency classification

3 main research questions How do schools sort? How does the resulting distribution of students affect student achievement?Wh h diff i l ff f diff d ? What are the differential effects for different students?

Existing Literature: TrackingExisting Literature: Tracking Hoffer (1992)

Compares tracked and non tracked schools Compares tracked and non‐tracked schools Tracking is beneficial for high scoring students and hurts low scoring students.

Betts and Shkolnik (2000) Principal‐level survey data Some evidence of small heterogeneous effectsg

Figlio and Page (2002) Create IV for sorting using county‐level instruments Some evidence that tracking actually helps low‐scoring students

Duflo et al. (2011)R d i d t i l i K Randomized trial in Kenya

Schools with a single class are given funds to hire an additional teacher Random assignment (sorted or non‐sorted) St id th t t ki h l b th hi h d l hi Strong evidence that tracking helps both high and low achievers

Existing Literature: Peer EffectsExisting Literature: Peer Effects Sacerdote (2001) Peer effects at college level Peer effects at college level Random assignment of college roommates Peer effects exist for GPA and fraternity membership

Ding and Lehrer (2007) Peer effects for students in China’s secondary schools Peer effects for students in China s secondary schools Students assigned to schools based on observable test scores Students benefit from higher‐performing peers and from smaller variation in peer qualityvariation in peer quality

Lefgren (2004) Peer effects in Chicago Public Schools Student‐level data and IV strategy Evidence of small peer effects Evidence of small peer effects

Texas Education Agency DataTexas Education Agency Data

Public School System Public School System 20 Regions More than 1,000 districtsdistricts

4.7 million students

D t Data Individual student‐level scores for grades 3‐11

Demographic data Students may be tracked across time

Students are linked to a school‐grade, but not to a specific classp

Texas Education Agency DataTexas Education Agency Data

Dallas ISD 2nd largest district in TX (12th largest in US) 200 schools 158,000 students

Data Data Students can be tracked across timeSt d t l l t t d t Student‐level test data

Students linked to both school‐grade and classroomclassroom

Identification of classroom peers

Texas Education Agency DataTexas Education Agency Data Score variables

TAKS (T A t f K l d d Skill ) TAKS score (Texas Assessment of Knowledge and Skills) Math and reading scores for 2004 and 2005 Scores for individual students and classmates

Other student data Race and genderg Free or reduced lunch Gifted and Talented (GT) Special Education Special Education Limited English Proficiency (LEP)

School/classroom variables School/classroom variables Class size Average school score A l Average class score

Do Schools Sort?  Do Schools Sort?  

Do Schools Sort?Do Schools Sort?

Do Schools Sort?Do Schools Sort?

Construction of a Sorting IndexConstruction of a Sorting Index Sorting by test score

t d t i l j h l k student i, classroom j, school k Sorting based on previous score (math and reading) Create a measure of dispersion within the school and p

within the classroom

Higher levels of sort indicate more homogeneous classes.classes.

Lower levels of sort indicate more heterogeneous classes

Empirical ModelEmpirical Model

Basic Model:Basic Model:

sorting indexsorting indexvector of student characteristics

f l h i ivector of classroom characteristics

Test coefficient on sorting index:Test coefficient on sorting index: Positive, significant values indicate that homogeneous sorting improves performancesorting improves performance.

Negative, significant values indicate that homogeneous sorting hurts performancesorting hurts performance.

Endogeneity of SortingEndogeneity of Sorting Sorting may be endogenous because schools choose certain students to be in certain classes.

Example: Error term contains unobserved factors, such as a child’s behavior. Unobserved behavior will impact a student’s classroom 

l d dplacement and sorting index. It will also impact a student’s test score.

unobserved behavior

Possible InstrumentPossible Instrument Use one grade’s sorting index to instrument for 

thanother Create grade‐specific sorting index

U 5th d i i d i f 4th d Use 5th grade sorting index to instrument for 4th grade index

Instrument relevance: A school’s sorting mechanism should be similar across gradesshould be similar across grades.

Instrument exogeneity: The way 5th graders are sorted should not impact performance of current 4th graders.should not impact performance of current 4 graders.

Heterogeneity Across Student GroupsHeterogeneity Across Student Groups Examine differential effects High scorers and low scorers Gifted and Talented classification Special Education classification Limited English Proficiency classification

Differential effects across score distribution Rank students by previous year score Create dummy categories

Effect of Sorting on Math Score (OLS)Effect of Sorting on Math Score (OLS)

V i bl (1) OLS (2) OLSDependent Variable: Math Score

Variables (1) OLS (2) OLS

sort (by score) 0.343*** 0.282***(0 0985) (0 0875)(0.0985) (0.0875)

math score 2004 0.484*** 0.599***(0.0101) (0.00930)

reading score 2004 0.199*** 0.150***(0.0103) (0.00916)

female ‐0.0223 ‐0.0124(0.0146) (0.0130)

lunch ‐0.0478** ‐0.0535***(0.0224) (0.0199)

black ‐0.394*** ‐0.285***(0.0371) (0.0330)

hispanic ‐0.165*** ‐0.107***(0.0363) (0.0323)

Effect of Sorting on Math Score Gain (OLS)Effect of Sorting on Math Score Gain (OLS)

Dependent Variable: Math Score GainVariables (1) OLS (2) OLS

sort (by score) 0 383*** 0 312***

p

sort (by score) 0.383 0.312(0.112) (0.0965)

reading score 2004 ‐0.0449*** ‐0.0348***(0 0103) (0 00893)(0.0103) (0.00893)

female 0.0523*** 0.0422***(0.0166) (0.0143)

lunch ‐0.0510** ‐0.0506**(0.0254) (0.0220)

black ‐0.300*** ‐0.195***(0.0421) (0.0364)

hispanic ‐0.150*** ‐0.0854**(0.0412) (0.0356)(0.0412) (0.0356)

Effect of Sorting on Reading Score (OLS)Effect of Sorting on Reading Score (OLS)

V i bl (1) OLS (2) OLSDependent Variable: Reading Score

Variables (1) OLS (2) OLS

sort (by score) 0.201*** 0.104(0 0742) (0 0654)(0.0742) (0.0654)

math score 2004 0.205*** 0.152***(0.0102) (0.00917)

reading score 2004 0 477*** 0 579***reading score 2004 0.477*** 0.579***(0.0105) (0.00956)

female 0.114*** 0.0858***(0 0149) (0 0132)(0.0149) (0.0132)

lunch ‐0.0923*** ‐0.0669***(0.0227) (0.0202)

black 0 347*** 0 293***black ‐0.347 ‐0.293(0.0376) (0.0333)

hispanic ‐0.204*** ‐0.166***(0 0367) (0 0325)(0.0367) (0.0325)

Effect of Sorting on Reading Score (OLS)Effect of Sorting on Reading Score (OLS)

Dependent Variable: Reading Score GainVariables (1) OLS (2) OLS

sort (by score) 0.295*** 0.164**sort (by score) 0.295 0.164(0.0838) (0.0724)

math score 2004 ‐0.0312*** ‐0.0400***(0 0102) (0 00893)(0.0102) (0.00893)

female 0.0705*** 0.0448***(0.0168) (0.0146)

l h 0 0622** 0 0362lunch ‐0.0622** ‐0.0362(0.0256) (0.0223)

black ‐0.297*** ‐0.244***(0.0425) (0.0369)

hispanic ‐0.143*** ‐0.115***(0.0414) (0.0360)( ) ( )

Effect of Sorting on Math Score (2SLS)Effect of Sorting on Math Score (2SLS)Dependent Variable: Math Score

Variables (1) OLS (2) OLS (3) 2SLS (4) 2SLSVariables (1) OLS (2) OLS (3) 2SLS (4) 2SLS

sort (by score) 0.343*** 0.282*** 0.670*** 0.578***(0 0985) (0 0875) (0 163) (0 145)(0.0985) (0.0875) (0.163) (0.145)

math score 2004 0.484*** 0.599*** 0.484*** 0.598***(0.0101) (0.00930) (0.0101) (0.00936)

reading score 2004 0 199*** 0 150*** 0 199*** 0 152***reading score 2004 0.199 0.150 0.199 0.152(0.0103) (0.00916) (0.0103) (0.00924)

female ‐0.0223 ‐0.0124 ‐0.0194 ‐0.0135(0 0146) (0 0130) (0 0148) (0 0131)(0.0146) (0.0130) (0.0148) (0.0131)

lunch ‐0.0478** ‐0.0535*** ‐0.0454** ‐0.0517**(0.0224) (0.0199) (0.0226) (0.0201)

black ‐0 394*** ‐0 285*** ‐0 401*** ‐0 291***black 0.394 0.285 0.401 0.291(0.0371) (0.0330) (0.0374) (0.0333)

hispanic ‐0.165*** ‐0.107*** ‐0.169*** ‐0.108***(0 0363) (0 0323) (0 0366) (0 0326)(0.0363) (0.0323) (0.0366) (0.0326)

Effect of Sorting on Reading Score (2SLS)Effect of Sorting on Reading Score (2SLS)Dependent Variable: Reading Score

Variables (1) OLS (2) OLS (3) 2SLS (4) 2SLSVariables (1) OLS (2) OLS (3) 2SLS (4) 2SLS

sort (by score) 0.201*** 0.104 0.473*** 0.369***(0 0742) (0 0654) (0 118) (0 104)(0.0742) (0.0654) (0.118) (0.104)

math score 2004 0.205*** 0.152*** 0.204*** 0.152***(0.0102) (0.00917) (0.0103) (0.00924)

reading score 2004 0 477*** 0 579*** 0 478*** 0 580***reading score 2004 0.477*** 0.579*** 0.478*** 0.580***(0.0105) (0.00956) (0.0106) (0.00964)

female 0.114*** 0.0858*** 0.114*** 0.0856***(0 0149) (0 0132) (0 0150) (0 0133)(0.0149) (0.0132) (0.0150) (0.0133)

lunch ‐0.0923*** ‐0.0669*** ‐0.0872*** ‐0.0611***(0.0227) (0.0202) (0.0230) (0.0204)

black ‐0 347*** ‐0 293*** ‐0 343*** ‐0 292***black ‐0.347 ‐0.293 ‐0.343 ‐0.292(0.0376) (0.0333) (0.0380) (0.0337)

hispanic ‐0.204*** ‐0.166*** ‐0.199*** ‐0.165***(0 0367) (0 0325) (0 0371) (0 0328)(0.0367) (0.0325) (0.0371) (0.0328)

Heterogeneity (Math Score)Heterogeneity (Math Score)Dependent Variable: Math Score

Variables (1) OLS (2) OLS (3) 2SLS (4) 2SLSVariables (1) OLS (2) OLS (3) 2SLS (4) 2SLS

sort*(high score) 0.420*** 0.312*** 0.731*** 0.602***(0.0987) (0.0880) (0.163) (0.146)( ) ( ) ( ) ( )

sort*(low score) 0.259*** 0.251*** 0.574*** 0.543***(0.0989) (0.0880) (0.163) (0.146)

math score 2004 0.420*** 0.573*** 0.421*** 0.573***(0.0133) (0.0124) (0.0134) (0.0126)

reading score 2004 0.194*** 0.149*** 0.195*** 0.150***(0.0102) (0.00916) (0.0103) (0.00924)

female 0 0211 0 0121 0 0183 0 0132female ‐0.0211 ‐0.0121 ‐0.0183 ‐0.0132(0.0146) (0.0130) (0.0147) (0.0131)

lunch ‐0.0507** ‐0.0544*** ‐0.0485** ‐0.0526***(0.0223) (0.0199) (0.0225) (0.0201)(0.0223) (0.0199) (0.0225) (0.0201)

black ‐0.393*** ‐0.286*** ‐0.400*** ‐0.292***(0.0370) (0.0330) (0.0373) (0.0333)

hispanic ‐0.163*** ‐0.107*** ‐0.167*** ‐0.108***(0.0362) (0.0323) (0.0365) (0.0326)

Heterogeneity (Reading)Heterogeneity (Reading)

Variables (1) OLS (2) OLS (3) 2SLS (4) 2SLSDependent Variable: Reading Score

Variables (1) OLS (2) OLS (3) 2SLS (4) 2SLS

sort*(high score) 0.270*** 0.149** 0.528*** 0.405***(0.0747) (0.0660) (0.118) (0.104)( ) ( ) ( ) ( )

sort*(low score) 0.120 0.0543 0.385*** 0.313***(0.0750) (0.0662) (0.118) (0.104)

math score 2004 0.202*** 0.151*** 0.202*** 0.151***(0.0102) (0.00916) (0.0103) (0.00923)

reading score 2004 0.410*** 0.536*** 0.414*** 0.538***(0.0146) (0.0133) (0.0148) (0.0134)

female 0 115*** 0 0861*** 0 114*** 0 0859***female 0.115*** 0.0861*** 0.114*** 0.0859***(0.0148) (0.0132) (0.0150) (0.0133)

lunch ‐0.0920*** ‐0.0667*** ‐0.0872*** ‐0.0610***(0.0226) (0.0201) (0.0229) (0.0204)( ) ( ) ( ) ( )

black ‐0.344*** ‐0.292*** ‐0.341*** ‐0.291***(0.0375) (0.0333) (0.0379) (0.0336)

hispanic ‐0.199*** ‐0.163*** ‐0.195*** ‐0.163***(0.0366) (0.0325) (0.0370) (0.0328)

ConclusionsConclusions Variation in the sorting mechanisms of schools Beneficial average effect of sorting Homogeneous sorting increases predicted score of the average student

Heterogeneity Beneficial effects for both high and low scoring students Evidence supports importance of efficiency gains for 

h kteachers in tracking Other Types of Sorting Some evidence of positive effects of sorting by GT status Mixed results for sorting by special education status

Sorting by Other DimensionsSorting by Other Dimensions Schools may consider other observed factors in sorting: Gifted and Talented Special Education Limited English Proficiency

Create additional sorting indices for each variable:

Further ResearchFurther Research Class Size and Sorting Schools with smaller classes have more groups available for sorting, all else equal

l ff f l ff Disentangle sorting effect from class size effect Student and classroom data allow for better identification of class size effectclass size effect

Special Education Inclusion Policies Special Education Inclusion Policies Supplement with data on specific disabilities Determine variation in inclusion policies across schools and Determine variation in inclusion policies across schools and impacts on students

Existing Literature: TrackingExisting Literature: Tracking

Descriptive statistics (Rees et al., 1996)p ( , ) 8th and 10th graders from NELS Most classes are tracked by performance Low income and minority students more likely to be in lower‐level classes

Descriptive statistics (Betts and Shkolnik, 2000) Middle school data from LSAY Middle school data from LSAY Lower ability classes are smaller Lower ability classes more likely to have teachers with less y yeducation and experience

Basic Sorting ExampleBasic Sorting Example 1 school with 12 students and 2 classes School administrators have students’ previous scores

Scores:90 80 70 50 40 3090 80 70 50 40 30

2 options for dividing students into classes: Homogeneous sorting: students with similar scores in same classclass

Heterogeneous sorting: spread students with similar scores across the two classes

Basic Sorting ExampleBasic Sorting Example

Homogeneous Sorting Heterogeneous Sortingg g g g

Class 1 Class 2 Class 1 Class 2

90 5090 50

90 9080 80

80 4080 40

70 7050 50

70 3070 30

40 4030 30

Avg: 80 Avg: 40SD: 8.16 SD: 8.16

Avg: 60 Avg: 60SD: 21.16 SD: 21.16

Do Schools Sort?Do Schools Sort? Create all potential pairs of students within a grade Create difference in student characteristics Regress “same” class on student differences N ti ffi i t i di t h l Negative coefficient indicates more homogeneous classes

Sorting by GT Status (Reading Score)Sorting by GT Status (Reading Score)Dependent Variable: Reading Score

Variables (1) OLS (2) OLS (3) 2SLS (4) 2SLSVariables (1) OLS (2) OLS (3) 2SLS (4) 2SLS

sort*(GT) 0.270*** 0.149** 0.528*** 0.405***(0.0747) (0.0660) (0.118) (0.104)( ) ( ) ( ) ( )

sort*(non‐GT) 0.120 0.0543 0.385*** 0.313***(0.0750) (0.0662) (0.118) (0.104)

math score 2004 0.202*** 0.151*** 0.202*** 0.151***(0.0102) (0.00916) (0.0103) (0.00923)

reading score 2004 0.410*** 0.536*** 0.414*** 0.538***(0.0146) (0.0133) (0.0148) (0.0134)

f l 0 115*** 0 0861*** 0 114*** 0 0859***female 0.115*** 0.0861*** 0.114*** 0.0859***(0.0148) (0.0132) (0.0150) (0.0133)

lunch ‐0.0920*** ‐0.0667*** ‐0.0872*** ‐0.0610***(0.0226) (0.0201) (0.0229) (0.0204)(0.0226) (0.0201) (0.0229) (0.0204)

black ‐0.344*** ‐0.292*** ‐0.341*** ‐0.291***(0.0375) (0.0333) (0.0379) (0.0336)

hispanic ‐0.199*** ‐0.163*** ‐0.195*** ‐0.163***(0.0366) (0.0325) (0.0370) (0.0328)

Sorting by Special Education Status (Reading)Sorting by Special Education Status (Reading)Dependent Variable: Reading Score

Variables (1) OLS (2) OLS (3) 2SLS (4) 2SLS( ) ( ) ( ) ( )

sort*(special ed) ‐0.227 ‐0.402 1.492 1.428(0.912) (0.807) (1.689) (1.523)

sort*(non‐special ed) ‐0.262*** ‐0.245*** ‐0.402*** ‐0.342***(0.0981) (0.0863) (0.148) (0.130)

math score 2004 0.204*** 0.152*** 0.204*** 0.153***(0 0102) (0 00919) (0 0104) (0 00929)(0.0102) (0.00919) (0.0104) (0.00929)

reading score 2004 0.475*** 0.578*** 0.476*** 0.578***(0.0105) (0.00957) (0.0106) (0.00968)

female 0.116*** 0.0863*** 0.115*** 0.0869***female 0.116 0.0863 0.115 0.0869(0.0149) (0.0132) (0.0151) (0.0134)

lunch ‐0.0946*** ‐0.0677*** ‐0.0917*** ‐0.0642***(0.0228) (0.0202) (0.0232) (0.0206)

black ‐0.366*** ‐0.308*** ‐0.377*** ‐0.319***(0.0379) (0.0336) (0.0394) (0.0348)

hispanic ‐0.223*** ‐0.182*** ‐0.234*** ‐0.191***(0 0372) (0 0330) (0 0389) (0 0343)(0.0372) (0.0330) (0.0389) (0.0343)

Sorting by LEP Status (Reading)Sorting by LEP Status (Reading)Dependent Variable: Reading Score

Variables (1) OLS (2) OLS (3) 2SLS (4) 2SLSVariables (1) OLS (2) OLS (3) 2SLS (4) 2SLS

sort*(LEP) 0.106 0.0355 0.198 0.284(0.126) (0.111) (0.250) (0.219)( ) ( ) ( ) ( )

sort*(non‐LEP) 0.142*** 0.0840** 0.0923 ‐0.00163(0.0446) (0.0394) (0.108) (0.0960)

math score 2004 0.205*** 0.152*** 0.210*** 0.156***(0.0103) (0.00929) (0.0106) (0.00954)

reading score 2004 0.476*** 0.578*** 0.476*** 0.577***(0.0106) (0.00968) (0.0109) (0.00993)

f l 0 116*** 0 0854*** 0 115*** 0 0845***female 0.116*** 0.0854*** 0.115*** 0.0845***(0.0151) (0.0133) (0.0155) (0.0137)

lunch ‐0.101*** ‐0.0712*** ‐0.0960*** ‐0.0696***(0.0230) (0.0204) (0.0240) (0.0213)(0.0230) (0.0204) (0.0240) (0.0213)

black ‐0.346*** ‐0.296*** ‐0.317*** ‐0.279***(0.0381) (0.0338) (0.0397) (0.0352)

hispanic ‐0.203*** ‐0.167*** ‐0.195*** ‐0.165***(0.0370) (0.0328) (0.0381) (0.0338)