8/3/2019 NAEP Power Point Version 2011
1/24
NYC ACHIEVEMENT GAINSCOMPARED TO OTHER
LARGE CITIES SINCE 2003Changes in NAEP scores 2003 -2011
Leonie Haimson & Elli Marcus
Class Size Matters
January 2012
www.classsizematters.org
8/3/2019 NAEP Power Point Version 2011
2/24
NAEP Scores: Why are they important? The National Assessment of Educational Progress (NAEP) is largest
continuing assessment of the knowledge and abilities of American students.
NAEP assessments are given by the federal govt. every two years tostatistical samples of students, change little over time & are low-stakes, andso can be used as a reliable metric to compare achievement trends among
states and urban districts.
The Trial Urban District Assessment (TUDA) has been given in 10 large citiesincl. NYC since 2003 in four categories: reading and math in 4th and 8th
grades.
What follows is an analysis of the changes in NYC NAEP scores since 2003,when Bloombergs educational policies were first implemented, compared tochanges in scores in the 9 other cities, plus large cities in general (w/ at least250,000 inhabitants).
8/3/2019 NAEP Power Point Version 2011
3/24
How did we compare trends among the
large urban districts? Since overall scores can change depending on changes instudent population, we compared changes in scores since2003 for six major NYC subgroups (white, black, Hispanic,
Asian, free lunch and non-free lunch students) compared totheir peers in other large cities.
Only major subgroups whose results we did not compare werestudents with disabilities and English language learners, sincerates of identification and exclusion from testing differ widelyamong the ten cities.
Our comparisons give insights into where NYC standsnationally, and allows us to assess the reality of DOEs claimsof great improvement.
Thesecomparisons giveinsightintowhereNYC stands nationally andprovides arobustexaminationoftheDOEs claims o
8/3/2019 NAEP Power Point Version 2011
4/24
When 2011 NAEP scores were released this fall,
NYC DOE claimed great progress *
Claim: NYC students have improved significantly onthree of the four math and reading tests between 2003and 2011.
Reality: This is true in nearly every city tested since
2003.
Claim: .since 2003, the gap between black and whitestudents in New York City has narrowed on all fourexams, and on all four since 2009.
Reality: There has been no statistically significantnarrowing of the achievement gap between any of theracial/ethnic groups in NYC in any subject testedsince 2003.
*Source: NYC DOE Press release , December 7, 2011
8/3/2019 NAEP Power Point Version 2011
5/24
DOEs other unfounded claims of progress Claim: On all four tests, low-income students in NYC now outperform their
peers across the nation, and thats a reason to be proud, said Chief
Academic Officer Shael Polakow-Suransky.
Reality: In 2003, NYC low-income students already outperformed their
peers nationwide in all four categories tested, and since then havemade fewer gains than peers in several other cities.
Claim: By the gold standard for measuring academic progress, our students
have made impressive gains since 2003especially compared to their peers
across New York State and the nation, said Chancellor Walcott.
Reality: When measured across subgroups, NYC students have made
less academic progress since 2003, compared to their peers, in every
other city except one.
*Source: NYC DOE Press release , December 7, 2011
8/3/2019 NAEP Power Point Version 2011
6/24
NYC comes in 2ndto lastamong all 10 cities + large city
category when NAEP score gains are averaged across 6
subgroups*
1
4.3
7.9 8.8 8.9
10.3 10.4 10.912.4 12.9
15.3
0
2
4
6
810
12
14
16
18
Cleveland NYC Charlotte large city Chicago SD Houston DC LA Boston Atlanta
*Subgroups include white, Hispanic, Black, Asian, free-lunch &
non-free lunch
Test score gains since 2003, averaged across all four categories:
reading & math in 4th & 8th grades
8/3/2019 NAEP Power Point Version 2011
7/24
Scores by subgroup: In NYC, Black students scores
rose less than
their peers in
most other cities In 4th grade reading, NYC black students dropped from
tied for 3rd to 4th place among all cities since 2003.
In 8th grade reading, NYC blacks were tied for 2nd anddropped to 3rd.
In 4th grade math, NYC blacks dropped from 3rd to 4th
place.
in 8th grade math, NYC blacks went from 3rd to tied for
4th place.
8/3/2019 NAEP Power Point Version 2011
8/24
NYC scores by subgroup: Black Students4
th
and 8th
grade reading and math gains in average scale scores since 2003
-4
6 68 8 9 9 9
10
2023
change in 4th grade readingscores 2003-2011
13
67 7 8 8
10 10 10
14
Change in 4th grade mathscores 2003-2011
-5 -4
1 2 23 3 4
69
12
change in 8th grade readingscores 2003-2011
04
9 9 1012 12
15 15
21 21
change in 8th grade mathscores 2003-2011
8/3/2019 NAEP Power Point Version 2011
9/24
Subgroup: White students fell sharply behind their
peers in
other large cities since 2003, especially i
n8
th
grade reading & math
In 4th grade reading, NYC white student scores droppedfrom 5th to 7th place.
In 4th grade math, NYC white students dropped from 5th
place to 8th place.
In 8th grade reading, NYC white students dropped from
tied for 2nd to 7th place, and came in last in score gains.
In 8th grade math, NYC white student scores droppedfrom 4th to 8th place and came in last in score gains.
8/3/2019 NAEP Power Point Version 2011
10/24
NYC scores by subgroup: White Students
1 1 14 5 6
7 8 8 9
16
change in 4th grade readingscores 2003-2011
1
5 5 6 67 8
10
13
Change in 8th gr readingscores
2003-2011 white students
-1
24 5
7 810 11 11
15
21
change in 4th grade mathscores 2003-2011
38 10 10
1114 16 16
18 20
change in 8th gr mathscores
2003-2011 white students
8/3/2019 NAEP Power Point Version 2011
11/24
Subgroup: Hispanic Students fell sharply behind
peers since 2003 In 4th grade reading, NYC Hispanic students dropped from
1st place among large cities to tied for 4th.
In 4th grade math, NYC Hispanic students dropped fromthird place to sixth place among other large cities.
In 8th grade reading, NYC Hispanic students dropped from
2nd to 5th place, with a net negative change in scores.
In 8th grade math, NYC Hispanic students came in last
place in score gains, falling from third place to 7th place.
8/3/2019 NAEP Power Point Version 2011
12/24
NYC scores by subgroup: Hispanic Students
-5
4 5 6 6 67
1013
17
28
Change in 4th grade readingscores 2003-2011
-8
-1 0
6 7 78
12 13
change in 8th grade readingscores 2003-11
1
79 10
11 1215 15
1719
change in 8th grade mathscores 2003-2011
-2
6 7 7 9 9 1013
18 19
Change in 4th grade mathscores
2003-2011
8/3/2019 NAEP Power Point Version 2011
13/24
Subgroup: Asian Students were the only NYCgroup to make substantial gains compared to peers in
other cities.
4th grade reading, NYC Asian student scores droppedfrom first place to second place, and placed fourth inoverall score improvement among large cities.
In 4th grade math, Asian student scores dropped fromsecond place to third place among large cities.
In 8th grade reading, NYC Asian student scores movedup from third place to second place among large cities.
In 8th grade math, NYC Asian student scored moved upfrom third place to second place
8/3/2019 NAEP Power Point Version 2011
14/24
Subgroup: Asian Students
1 23 3
7
15
large city SD NYC Boston LA Charlotte
Change in 4th gradereading scores 2003-2011
Asian students
-4
67
910
12
Chicago Boston SD NYC large city LA
change in 8th grade readingscores 2003-2011 Asian students
3 46
10 10
16
large city NYC Charlotte LA SD Boston
change in 4th grade mathscores 2003-2011 Asian
students
10 11
15 15
18 1920
Chicago Charlotte SD large city NYC Boston LA
change in 8th grade math scores2003-2011 Asian students
8/3/2019 NAEP Power Point Version 2011
15/24
Changes in demographics: Asian student pop rising faster in
NYC than elsewhere; otherwise progress on NAEPS would have
been even smaller
0 0
31
3
6
5
8
1
8
52
3
68
19
0
5
10
15
20
Atlanta Boston Chicago DC Houston LA large city NYC
Asians as % of total students tested4th grade reading
2002
2011
0
8
4 31
2
67
12
18
1
8
56
23
68
19
15
02468
101214161820
Asians as % of total students tested4th grade math
2003
2011
8/3/2019 NAEP Power Point Version 2011
16/24
NYC scores by subgroup: Free Lunch
students had only middling gains In 4th grade reading, NYC free lunch student scores remained
in 1st place but placed behind five other large cities in gainssince 2003.
In 4th grade math, NYC free lunch student scores dropped fromsecond place to third place, and placed fifth in score gainsamong large cities.
In 8th grade reading, NYC free lunch student scores remained
in 1st place but placed behind three other large cities in scoregains.
In 8th grade math, NYC free lunch student scores dropped from1st place to 3rd place.
8/3/2019 NAEP Power Point Version 2011
17/24
Subgroup: free lunch
-2
6 6 7 7 78 8 8
1112
change in 4th grade readingscores 2003-2011
-4
0
23
5 56
78
11
change in 8th grade readingscores 2003-2011
1
6 79 9 9 10 10
11 12
16
change in 4th grade math scores2003-2011
3
9 11 1113 14 15
17 17 1921
change in 8th grade math scores2003-2011
8/3/2019 NAEP Power Point Version 2011
18/24
NYC non-free lunch students made the smallest gains of
any city in every category; and dropped sharply at 8th grade
In 4th grade reading, NYC non-free lunch students fell from 1st place to 2nd
place.
In 4th grade math, NYC non-free lunch students fell from 2nd place to 3rd
place.
In 8th grade reading, NYC non-free lunch student scores dropped 11 points
the only city where scores dropped and fell from 1st place to 8th place.
In 8th grade math, NYC non-free lunch students dropped seven points the
only city where scores dropped -- and fell sharply from 1st to 8th place
In 8th grade reading and math, basic and proficient levels of non-free lunch
also dropped sharply.
8/3/2019 NAEP Power Point Version 2011
19/24
Subgroup: non-free lunch
1 36
914 15 16 16
18
28
change in 4th reading scores2003-2011
-11
5 68 10 10
11 11
19 24
change in 8th grade readingscores 2003-2011
58 10
14 16 1618 19
29
change in 4th grade math scores2003-2011
-7
1014 14 15
20 2124
27
38
change in 8th grade math scores2003-2011
8/3/2019 NAEP Power Point Version 2011
20/24
NYC is ONLY city where proficiency levels in 8th grade
reading and math have dropped for non-free lunch students
82
49
75
41
0
10
20
30
40
50
60
70
80
90
at or abovebasic
at or aboveproficient
8th grade math for non-free lunch students
2003
2011
87
48
79
34
0
10
20
30
40
50
60
70
80
90
100
at or abovebasic
at or aboveproficient
8th grade reading for non-free lunch students
2003
2011
8/3/2019 NAEP Power Point Version 2011
21/24
All other cities made gains in 8th grade proficiency in reading &
math fornon-free lunch students, while in NYC they dropped
-7
7 711
14 15 16
2429
37
-8
10
15 14
22
16
24 2325
30
-20
-10
0
10
20
30
40
NYC Chicago Charlotte large city Boston Houston SD DC Atlanta LA
change in % non-free lunch students at or above basic & proficient
in 8th grade math 2003-2011
diff basic
diff proficient
-8
25 6
710 10
12
19 21
-14
13
8
139
1511
1719 22
-20
-15
-10
-5
0
5
10
15
2025
NYC Chicago Charlotte Boston large city SD Houston DC Atlanta LA
change in % non-free lunch students at or above basic &proficient in 8th grade reading 2003-2011
diff basic
diff proficient
8/3/2019 NAEP Power Point Version 2011
22/24
Summary of findings: When analyzing subgroup performance, NYCs relative progress since
2003 compared to other large cities has been mediocre to poor.
NYC came in 2nd to last in NAEP gains among 10 cities and large citycategory tested since 2003 when averaged across six subgroups.
All NYC subgroups fell in ranking, compared to peers in other large cities,with White, Hispanic and non-free lunch students dropping most sharply.
White students made the smallest gains compared to their peers in othercities in both 8th grade reading and math; Hispanics in 8th grade math.
Asian students were only NYC subgroup to advance in ranking in anysubject or grade;
NYC was only city in which non-free lunch students scored lower in 2011than in 2003, in both 8th grade reading and math, and their proficiencylevels also dropped sharply.
8/3/2019 NAEP Power Point Version 2011
23/24
What about mayoral control?Two districts under mayoral control made least progress & on
average, cities with elected school boards have done better
1
4.3
7.98.8 8.9
10.3 10.410.9
12.4 12.9
15.3
0
2
4
6
8
10
12
14
16
18
Cleveland NYC Charlotte large city Chicago SD Houston DC LA Boston Atlanta
Cities with mayoral control since 2003 or earlier in red;
DC has had mayoral control since 2007.
8/3/2019 NAEP Power Point Version 2011
24/24
What else do these results suggest?
The administrations aggressive free-market strategies of
high-stakes accountability, school report cards, fair
student funding, principal empowerment, and the closing
of more than one hundred schools & the opening of morethan 400 new schools & charters, while allowing class
sizes to increase sharply, have not worked to increase
achievement compared to cities elsewhere.
In fact, the relative positions of white, Hispanic and non-
free lunch students in NYC have all dropped substantially,
with the declines especially sharp at the 8th grade level.