1
Effects of Race to the Top on Students’ Science Achievement and Closing the
achievement Gap in Science
Kitchka Petrova, [email protected]
Patrice Iatarola, [email protected]
Anastasia Semykina [email protected]
Florida State University
Paper prepared for the AEFP 42nd Conference
Washington, DC
March 18th, 2017
This working paper may not be cited or distributed without explicit permission from the authors
2
Introduction
Preparing K-12 students to have strong background in Science Technology, Engineering
and Mathematics (STEM) and inspiring them to pursue STEM careers defined President
Obama’s administration’s efforts to address the issues with K-12 students’ science proficiency
and the existing science achievement gap (Holdren, Lander & Varmus, 2010). RTTT was one of
the educational initiatives of the US Department of Education that was designed to stimulate
innovations in education and improve the educational outcomes of US students by incentivizing
states to make changes in four areas: adopting rigorous standards, building longitudinal state data
systems, ensuring schools have effective teachers and principals and turning around the lowest
achieving schools. The states were encouraged to include STEM component in their applications
and all states that received RTTT funds had such components. The states had to present sound
plans, geared towards offering rigorous studies in STEM, cooperating with industry, universities
and informal science education establishments, improving STEM teaching, preparing more
students for advanced careers in STEM, and addressing the needs of underrepresented groups
(U.S. Department of Education, 2009). RTTT also coincided with other recommendations of the
federal government such as “Educate to Innovate” Campaign for Excellence in Science,
Technology & Math (STEM) Education that aspired to improve STEM education within the next
ten years (The White House, 2009) and the release of the President Obama’s Council of Advisors
on Science and Technology report “Prepare and Inspire: K-12 education in STEM for America’s
future” that contained specific recommendations for improvements by preparing students to have
strong background in STEM and inspiring them to pursue STEM careers (Holdren et al., 2010).
The last report also addressed the existing achievement gap in science and its consequences for
minority and students from low socioeconomic background who would not have the
opportunities to pursue careers in STEM fields that are well paid and highly respected.
3
Furthermore, the report specifically highlighted that due to the lack of minority and women in
STEM fields, these underrepresented groups cannot contribute with their innovative ideas to the
development of STEM fields (Holdren et al., 2010).
These recent events demonstrate how much interest is there to improve STEM education and
one would expect discussions on a national level about the role of initiatives such as RTTT in the
process, because RTTT also targeted improvement in STEM. However, such discussion has been
absent in the education policy research community, a fact that is puzzling considering the goals
of RTTT.
Therefore, investigating how large investments in education such as RTTT contribute to
improving STEM outcomes for students and science education is needed. Our study addresses the
following research questions:
1. Did RTTT improve science achievement?
2. Did subgroups of students, defined by race/ethnicity, English language learner,
disability or income status experience different outcomes of RTTT in terms of science
achievement?
This study reports on the effectiveness of RTTT in improving students’ science
achievement. By evaluating the RTTT effects on science education this study fills the gap in the
literature if large scale education investments, based on a competitive selection process help
improve science education. It also sheds light on how successful RTTT was to decrease the
achievement gap in science for minority, students with disabilities, ELL and students with low
socioeconomic status. It also demonstrates if K-12 public education, when supported with funds
4
is contributing to meeting the National Science Foundation’ s strategic goal to “support
development of a diverse STEM workforce” (National Science Foundation, 2014).
Literature Review
RTTT and student outcomes. Due to the recent end of RTTT program there is a dearth of
peer review articles discussing its effects on student achievement. Scholars have investigated the
political climate of implementing the RTTT policies (Howell, 2015, McGuinn, 2011) or the
legality of the program (Barnes, 2011,). Others have examined issues related to the application
and implementation processes of the program (Scott, 2011) and the awardee states’ capacity to
implement RTTT (Nowicki, 2015). The U.S. Department of Education reported on the impact of
RTTT on students’ graduation rates, enrollment in higher education institutions and participation
in Advanced Placement (AP) classes. (U.S. Department of education, 2015). In the final report,
released in October, 2016 the effect of RTTT on student achievement was reported as
inconclusive and descriptive statistics of changes in the students’ achievement in language arts
and mathematics were reported, but not in science (Dragoset, Thomas, Herrmann, Deke, James-
Burdumy et al., 2016).The evaluators of the Tennessee’s RTTT’s STEM professional
development (PD) programs found out that a significant growth in effectiveness and attitudes of
science and math teachers (Johnson, 2014), but there was no discussion how this growth affected
students’ science and math achievement.
RTTT and closing the achievement gap in science. A goal of RTTT was to decrease the
achievement gap between subgroups of students in reading/language arts and mathematics (US
Department of Education, 2009). Aside from these two subjects’ achievement gap there is also a
science achievement gap that has been explored by several scholars. (Abedi & Gándara, 2006;
Anderman, 1998; Bacherach, Baumeister & Furr, 2003; Cawley, Hayden, Cade & Baker-
5
Kroczynski, 2002). For example, in 2015 4th graders who were English Language Learners
(ELL), Black and Hispanic, had a disability, or were eligible for free and reduced lunch scored
27-37 points below their peers who did not belong to these subgroups when assessed for NAEP
science (National Center for Educational Statistics, n.d ). Large body of knowledge explains that
the racial, ethnic, ELL, disability and socioeconomic status are associated with the achievement
gap in science. Often such children attend schools where they are not usually challenged with
rigorous curriculum or not taught by qualified science teachers (Lieberman & Hoody, 1998;
Ruby, 2006; Lee, Maerten‐Rivera, Penfield, LeRoy & Secada, 2008, Quinn & Cooc, 2015).
Unfortunately, no studies have examined yet the impact of RTTT on science achievement of
these subgroups of students. One report discussed the approaches RTTT states employed to
improve the outcomes for English Language Learners (ELL) and their performance in math and
reading/language arts (Dragoset, James-Burdumy, Hallgren, Perez-Johnson, Herrmann, et al.,
2015), but there was no reference to the ELL students’ science achievement. The achievement
gap in science and it being persistent makes it even more important for us to understand how
RTTT may or may not contribute to improving educational outcomes in science and reducing the
achievement gap. Further investigation of the individual state programs would help to determine
what initiatives were successful, so that they could be used as models for implementation in
other states.
In this study, we use the education production function (EPF) as a theoretical framework
for analyzing the relationship between the educational inputs (RTTT funds and students’
background) and the educational outputs (student achievement scores in science) (Bowles, 1970;
Ferguson, 1991; Hanushek, 1986; MacPhail-Wilcox, & King, 1986; Todd & Wolpin, 2003). The
conceptual framework of RTTT with the study’s logic model is outlined in Figure 1.
6
Fig. 1. Race to the Top Conceptual Framework with the study’s logic model
INPUTS INPUTS OUTPUTS Long-term OUTCOMES
Federal Government
Transparent
Competitive
Selection Process
RTTT grant
Resources
Students’
background
Initiatives,
specific for
each RTTT
state
RTTT
resources
Expected
Increased Student
achievement in
science
Closing the
achievement gap in
science
Excellence in
Science
Education
Competent
STEM Force
Closing the
achievement
gap in Science
Education
Diverse
STEM force
Innovative
Science
Teaching
Learning
Developing 21st
century skills
Phase I - DE, TN
Phase II- DC, FL, GA, HI,
MA, MD, NC, NY, OH, RI
Phase III- AZ, CO, IL, KY,
LA, NJ, PA
Treatment states
Non awardees States Applicants:
AL, AR, CA, CT, IA, ID, IN,
KS, ME, MI, MS, MN, MO,
MT, NE, NH, NM, NV, OK,
OR, SC, SD, UT, VA, WA, WI,
WV, WY
Non awardee states Non-
applicants: TX, ND, AK, VT
Comparison states
Economic and political conditions in each state 2009-2014/2015
RTTT Framework
7
Methodology
Data
The data for this project came from the National Center for Educational Statistics
(NCES) database – the National Assessment of Educational Progress (NAEP). The NAEP
provides data on the independent assessment of student science achievement across the U.S. with
science score data for 4th (2009 and 2015) and 8th (2009, 2011 and 2015) grade students in public
schools. The study’s focus is on 4th and 8th graders’ science achievement, because if students are
not engaged in quality science learning in elementary and middle school that will affect their
future science achievement and interests to pursue careers in STEM disciplines. Publicly
available NAEP data are used for this study and in the future, upon receiving access to the
restricted NAEP data we will refine these analyses. Using the NAEP science achievement data
for 4th and 8th graders in public schools in all 50 states and District of Columbia (DC) and
additional variables from Common Core Data (CCD), an Independently Pooled Cross Sections
(IPCS) data sets (4th and 8th grade) was assembled. This data set contained random samples of
independent observations (science scores of 4th and 8th graders for each state), collected in
different time periods (2009, 2011, 2015). As such, IPCS data are very useful to evaluate the
effect of policies (Wooldridge, 2010) and the described data provide independent assessment
about the changes in students’ science achievement before and after the RTTT. The NAEP also
reports the scores for subgroups of the ELL students, students with disability, students from low
socioeconomic status and different minorities (Black and Hispanics) and these average scores
will be used to estimate the effects of RTTT on the achievement gap in science. The average
national sample size per tested subject, per grade, per year is between 110,000 to 115,000
students from more than 6,000 schools from across the US. Common Core Data (CCD) data and
8
American Community Survey provided information on the, the spending per pupil per state for
2009, 2011 and 2015. Since the data for 2015 spending has not been released yet we are using
the spending per pupil from 2014. Additionally, the second model included dummy variables to
account for the adoption of Common Core State Standards (CCSS) in mathematics and language
arts and the Next Generation Science Standards (NGSS) in the specific states to be able to
control for the influence of such educational initiatives on student achievement.
Data Analysis
The research questions are answered using a difference in differences estimation
technique (Wooldridge, 2010).
Yit= β1+ β2 d2015t + δ (d2015t* dRTTTi) + β3State Dummy variablesi+ uit, , ( t=1,2)
Yit – science scores for 4th or 8th graders for state i at time period t where t=1 for 2009 (phase I
and phase II) OR (t=1 for 2011 for phase III); t=2 for 2015 (phase I, II and III);
β1 – average science scores for the reference state in period t=1;
β2 – change in average science scores for all 50 states and DC between periods 1 and 2 (t=1 &
t=2);
d2015t is a dummy variable for period 2 which is 2015 for all phases. (It is set for 1 for 2015
and 0 for the year before RTTT (2009 for phases I and II; 2011 for phase III). It is important to
include this second-time period dummy variable to account for any general changes that take
place over time. By doing this we ensure that the estimated effect of RTTT is not confused with
the effect of other factors that have changed between the two-time periods for all studied states
and DC);
9
dRTTTi is a dummy treatment variable and it equals 1 for all states that received RTTT grant
and 0 for the states that did not received grants);
β3 – shows the differences in student achievement by state due to various unobservable
factors; State Dummy variables- There will be 50 dummy variables and DC will be a reference
group. By including these dummy variables, we account for the differences between the students
who attend public schools in the different states, the differences between their teachers’
preparation or different state science standards;
δ – shows the effect of the RTTT on students’ science achievement and answers the research
question; uit – error.
The dependent variable is the average student science achievement in state i, at the time
before (t=1) RTTT took place and after (t=2). RTTT was announced in July, 2009 and awards
were allocated to states in three phases (March, 2010, September 2010 and December 2011)
(U.S. Department of Education, 2009). The test allows to determine student achievement for
overall science and for life science, physical science and earth science which gave us an
opportunity to explore if there is some more specific effect depending on the type of science.
Key independent variable is the RTTT award for the specific state. Having states that received
awards and states that did not allows for identifying treatment and comparison groups. Model 1
estimates the effect of RTTT on science achievement without control variables, while Model 2
contains control variables such as spending per pupil per state and adoptions of CCSS and
NGSS, two important educational initiatives that might have influenced the outcomes for the
students.
10
The complete list of variables used for the analyses for both 4th and 8th grade is included
in Table 1.
Table 1. List of Variable Names, Descriptions, and Labels
RQ1. Did RTTT improve science achievement?
RQ2. Were different subgroups of students, defined by race/ethnicity, English language
learner, disability or income status experienced different outcomes of RTTT?
Variable name Description NAEP Label
NAEP sci overall Average sci scores in each state SRPUV
NAEP Physical sci score Average life sci score in each state SRPS1
NAEP Earth sci score Average earth sci score in each state SRPS2
NAEP Life sci score Average life sci score in each state SRPS3
National Public school students Charter, but not BIE and DOD schools TOTAL
Public schools Students attending public schools only SCHTYP2
Science scores of
ELL and non-ELL*
Classified by school LEP
Science Scores of Students with
disabilities*
Classified by SD, IEP, or 504 Plan IEP
Science Scores of students in
the National School Lunch
program
Eligibility based on school records SLUNCH3
Science scores by race and
ethnicity
White, Black, Hispanic, Asian/Pacific
Islander, American Indian/Alaska
Native, Two or more races
SDRACE
Note: BIE = Bureau of Indian Education, DOD = Department of Defense, ELL = English
Language Learners, Sci = Science.
Results
Fourth Grade Science Achievement
Examining the average science scores per state for 4th grade shows that there is a large
variability between states. The average score is 151 points, but Mississippi (MS) has the lowest
average score – 133 points and New Hampshire (NH) has the highest – 163 points in 2009. In
2015 the average science score per state is 155 and the states where students scored lowest and
highest are the same as in 2009 – MS – 135 points and NH – 167 points. The differences in
11
average science scores of subgroups of students in the states are substantial – black 4th graders
scored 31-34 points lower than their white peers; Hispanic 4th graders scored 23-27 points lower
than their white peers; ELL students scored 33-34 points lower than non-ELL students; students
with disabilities scored 24-26 points lower than students not identified as having disabilities;
students who qualified for free and reduced price lunch also scored 25-26 points lower than
students who are not part of the National School Lunch Program (NSLP). These trends of
science achievement of subgroups of students are observed at national and state level. Table 2
contains the summary statistics of 4th graders’ science achievement at a national level only.
The results of the analyses for 4th grade science achievement in the states that received
RTTT grant are presented in Table 3. The column, labeled Model 1 represents the estimated
coefficient of the effect of RTTT on student science achievement for the overall science scores,
and for life, earth and physical sciences. In addition to that, the estimated coefficients of the
effect of RTTT on the science performance of different subgroups of students (ELL, non-ELL,
Black, Hispanic, White, students with and without disabilities and students who qualify for free
and reduced lunch program and the ones who do not ) are also presented in this column. These
results are the estimates of single regressions that did not contain any control variables. Our
results show there is no statistically significant effect of RTTT on the overall science
performance of 4th graders and the different subgroups of 4th graders, except we report
statistically significant positive effect of RTTT on the science achievement of students who
qualify for free and reduced lunch at p<.05 level. To account for differences between the states in
terms of spending per pupil and important educational initiatives such as the adoption of CCSS
and NGSS we use these three controls in the second model. The statistically significant positive
effect of RTTT on science achievement of 4th graders who qualify for free and reduced lunch is
12
Table 2.
Summary Statistics for Fourth Grade for Years 2009
Science Score Science Score by Student Subgroup
Overall Earth Physical Life Black Hispanic White ELL NELL WD WOD FRL NFRL $/pupil
4th Grade
2009
#Obs. 48 48 48 48 41 45 48 39 48 48 48 47 47 52
Mean 151 151 151 151 128 135 162 119 153 130 154 137 163 10748
STD 7 8 7 7 7 8 4 11 7 11 7 7 5 2612
Min 133 132 133 135 116 121 150 98 133 95 135 122 149 2612
Max 163 165 163 163 141 153 172 143 164 148 167 151 172 18126
2015
#Obs. 48 48 48 48 40 45 48 41 48 48 48 47 47 52
Mean 155 156 154 154 133 141 164 124 157 132 158 143 168 11508
STD 6 7 6 5 7 7 4 12 6 10 6 6 4 3293
Min 140 140 141 143 119 128 152 94 141 101 143 128 156 6500
Max 166 168 165 165 148 163 175 150 167 147 170 152 177 20610 ELL = English Language Learners, NELL = Non-English Language Learners, WD = Students with Disabilities, WOD = Students without
Disabilities, FRL = Students on Free/Reduced Lunch, NFRL = Students not Eligible for Free/Reduced Lunch
13
Table 3. Estimated Effects of Race to the Top on 4th grade students’ Science Achievement
4th grade science #
States
N Model 1 Model
2
Science overall 49 92 1.100
(1.172)
1.848*
(1.036)
Life science 49 92 1.148
(1.147)
1.510
(1.097)
Earth Science 49 92 0.807
(1.106)
1.809*
(.902)
Physical Science 49 92 2.064
(1.235)
1.719
(1.136)
Black 43 77 1.833
(1.794)
3.234*
(1.849)
Hispanic 46 86 1.573
(1.389)
2.159
(1.438)
White 49 92 .505
(1.057)
1.143
(1.011)
English Language
Learners (ELL) 43 76
-2.615
(2.519)
-6.753***
(2.715)
Non-ELL 49 92 1.245
(1.102)
2.024**
(1.005)
Students with
Disabilities 49 92
1.433
(1.804)
3.000**
(1.741)
Students without
Disabilities 49 92
0.879
(1.127)
1.506
(1.033)
Students eligible for
Free and Reduced
Lunch
49 92 2.283**
(1.049)
2.610**
(1.055)
Students not eligible
for Free and Reduced
Lunch
49 92 0.663
(1.433)
1.243
(1.212)
Note: Robust standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1. Model 2 controlled for state spending
per pupil, state adoption of Common Core State Standards, and state adoption of Next Generation Science
Standards.
14
also observed in Model 2, where the control variables are added. In the second model, we
observed positive and statistically significant effect of RTTT on the overall 4th grade students
‘science scores, their earth science scores and black and non-ELL students’ science scores at
p<.1 level. Additionally, we report negative statistically significant effect of RTTT on ELL
students’ science achievement and positive statistically significant effect of RTTT on science
scores of students with disabilities at level p<.05.
Eight grade Science Achievement
The descriptive summary statistics of the science scores of eight graders is presented in
Table 4. 8th graders were tested in 2009, 2011 and 2015, because the decision was made to align
NAEP with TIMSS in 2011. The average science scores per state for 8th graders also show that
there is a large variability between states. The average score is 151 points, but Mississippi (MS)
has the lowest average score – 132 points and North Dakota(ND) and Montana(MT) have the
highest – 162 points in 2009. In 20111, MS still has the lowest score (137) and ND the highest –
164 points. In 2015 the average science score per state is 155 and MS still with the lowest (140
points) and UT with the highest average score (166 points). The differences in average science
scores of subgroups of students in the states are also substantial – black 8th graders scored 33-35
points lower than their white peers; Hispanic 4th graders scored 22-25 points lower than their
white peers; ELL students scored 43-46 points lower than non-ELL students; students with
disabilities scored 31-33 points lower than students not identified as having disabilities; students
who qualified for free and reduced price lunch also scored 24-25 points lower than students who
are not part of the NSLP. Similarly, to the 4th graders the same trends are observed for the 8th
grade subgroups of students’ science performance at national and state level. Summary statistics
of eight graders’ science achievement at a national level only is presented in Table 4.
15
In addition to these differences in student science performance across the different states
there are also vast differences in the spending per pupil in each state, with the state spending the
most per pupil being almost three times higher than the lowest spending state. This discrepancy
in spending is observed in 2009, 2011 and 2014. The decisions to adopt the CCSS and NGSS are
also vastly different. There were incentives to the states to adopt the CCSS while the adoption of
NGSS is without any incentives. CCSS was originally adopted by 48 states, but later some of the
states choose to withdraw. NGSS is rather new and many states have not decided on adopting
them yet. As of November, 2016 the following states have adopted the NGSS – Arkansas,
California, Connecticut, Delaware, Hawaii, Illinois, Iowa, Kansas, Kentucky, Maryland, Nevada,
New Jersey, Oregon, Rhode Island, Vermont, Washington, and DC. The results of the analyses
for 8th grade science achievement and the RTTT effects are presented in Table 5. RTTT had
three phases – April, 2010, September, 2010 and December, 2011. We estimate the overall effect
of RTTT for the three phases. The column, labeled Model 1 represents the estimated
coefficients and robust standard errors of the effect of RTTT on 8th grade student science
achievement for overall science scores and for life, earth and physical sciences and on the
science performance of different subgroups of students (ELL, non-ELL, Black, Hispanic, White,
students with and without disabilities and students who qualify for free and reduced lunch
program and the ones who do not). These results are the estimates of single regressions that did
not contain any control variables. The results show we have no statistically significant effect of
RTTT on the overall performance of 8th graders and most of the different subgroups of 8th
graders except we report statistically significant positive
16
Table 4.
Summary Statistics for Eight Grade for Years 2009, 2011, and 2015
Science Score Science Score by Student Subgroup
Overall Earth Physical Life Black Hispanic White ELL NELL WD WOD FRL NFRL $/pupil
8th Grade
2009
#Obs. 48 48 48 48 41 43 48 31 48 48 48 47 47 52
Mean 151 151 150 151 127 135 160 109 152 123 154 136 161 10748
STD 8 8 8 8 7 8 5 11 7 11 8 8 5 2612
Min 132 131 132 134 111 119 146 86 132 97 135 122 147 6356
Max 162 165 162 163 144 155 170 136 164 140 166 151 169 18126
2011
#Obs. 53 53 53 53 41 46 53 28 53 53 53 51 51 52
Mean 152 152 152 153 128 138 163 109 154 125 156 140 164 11037
STD 9 9 8 9 7 8 5 13 9 12 9 7 5 2982
Min 112 110 117 113 107 116 150 75 113 74 119 127 151 6212
Max 164 166 164 164 149 158 179 133 165 143 168 153 171 19076
2015
#Obs. 48 48 48 48 39 45 48 29 48 48 48 47 47 52
Mean 155 154 154 156 131 142 164 111 157 126 159 142 166 11508
STD 6 7 6 6 7 6 4 12 6 9 6 6 4 3293
Min 140 137 138 143 118 127 150 88 140 102 143 129 154 6500
Max 166 165 167 168 151 162 173 136 169 144 170 152 175 20610
ELL = English Language Learners, NELL = Non-English Language Learners, WD = Students with Disabilities, WOD = Students
without Disabilities, FRL = Students on Free/Reduced Lunch, NFRL = Students not Eligible for Free/Reduced Lunch
17
Table 5. Estimated Effects of Race to the Top on 8th grade students’ Science Achievement
8th grade science #
States
N Model 1 Model
2
Science overall 50 143 .426
(.587)
.697
(.543)
Life science 50 143 .582
(.610)
.776
(.537)
Earth Science 50 143 1.266**
(.565)
1.511***
(.492)
Physical Science 50 143 -.339
(.673)
-.164
(.661)
Black 49 115 -.718
(2.384)
-1.571
(2.863)
Hispanic 48 128 .307
(1.113)
.575
(1.190)
White 50 plus
DC 143
.187
(.697)
.202
(.577)
English Language
Learners (ELL) 35 85
-3.338
(2.557)
-1.322
(2.690)
Non-ELL 50 143 .796
(.601)
1.049*
(.531)
Students with
Disabilities 50 143
0.085
(1.630)
.237
(1.603)
Students without
Disabilities
50 plus
DC 143
0.271
(.182)
.905*
(.476)
Students eligible for
Free and Reduced
Lunch
50 plus
DC 143
1.574**
(.693)
1.654**
(.698)
Students not eligible
for Free and Reduced
Lunch
50 plus
DC 142
0.831
(.725)
1.105*
(.596)
Note: Robust standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1. Model 2 controlled for state spending
per pupil, state adoption of Common Core State Standards, and state adoption of Next Generation Science
Standards.
effect of RTTT on 8th graders’ earth science scores at p<.05 level and statistically significant
positive effect on 8th graders who qualify for free and reduced lunch science scores at p<.05
level. The column, labeled Model 2 represents the estimated coefficients /robust standard errors
when control variables were added to the model. Adding the control variables increased the
18
estimated coefficient for earth science and it was also statistically significant at p<.05 level. The
estimated coefficient of the RTTT effects on students who qualify for free and reduced lunch
slightly increased and stayed positive statistically significant at p<.05 level. The RTTT effect on
8th graders’ earth science scores became statistically significant at p<.001 level and the RTTT
effects on 8th grade non-ELL students’, students with disabilities and students who do not qualify
for free and reduced lunch and their science scores became statistically significant at p<.1 level
compared to being statistically non-significant in model 1.
Our results show RTTT had some positive and some negative effects on public school
students’ science achievement. To some extent our results agree with the findings included in
the RTTT final report that concluded that there might be positive, negative or no effects of RTTT
on student outcomes in math and reading/ language arts, but the evaluation report refrained from
any conclusions related to causal effect of RTTT on student outcome (Dragoset et al., 2016)
Since no science scores were analyzed as part of the RTTT evaluation we do not have other
studies to compare our results to now.
Discussion
In this study, we sought to find out if RTTT had any effects on science achievement of 4th
and 8th graders in public schools and if different subgroups of students who underperform in
science experienced different effects of RTTT program or if RTTT has contributed to closing the
achievement gap in science. While, we do not report about positive, statistically significant effect
of RTTT on student science achievement across the grades, we do have positive statistically
significant effect of RTTT on 8th graders’ achievement in earth science for all students at p<.05
and for 4th graders’ achievement in earth science at p<.1 level. In terms of RTTT closing the
achievement gap, we report that 4th and 8th graders who are eligible for free and reduced lunch
19
experienced statistically significant positive effect on their overall science scores, which is
encouraging. The ELL in 4th grade experienced statistically significant negative effect of RTTT
on their overall science scores, a finding that should be explored further. The statistically
significant positive effect of RTTT on 4th graders’ who belong to the subgroups of black students
and students with disabilities are optimistic.
Limitations
Estimating causal effect of RTTT on science achievement of elementary and middle
school students requires multilayered approach. Here we presented results that are at state level
and based on comparison of the performances of the states before and after the RTTT. Even
though NAEP aims to provide a national assessment of student progress we still have states that
choose not participate or participate selectively at specific year. This attrition at a state level
might be affecting our estimates. For example, DC and Alaska participated in the NAEP testing
only for 8th graders in 2011. Similarly, some states do not have representative sample of special
subgroups of students and that might also interfere with our estimates.
Conclusion
The importance of providing equal learning opportunities for all students is undisputed.
In this study, we found significant positive effects of RTTT on 8th graders’ performance in earth
science and on the overall science achievement of 4th and 8th graders’, who are eligible for free
and reduced lunch. While some of our results are, inconclusive and require further investigation,
we could say that the large investments in education, such as RTTT could have an impact on
student outcome. Our study was motivated by the lack of information about the effects of RTTT
on students’ science achievement and closing the achievement gap in science. We conclude with
20
how important is to improve science education for all students and to remediate the implications
of the persisting achievement gap in science for the students who are minority, ELL, have
disabilities or are from low socioeconomic background for their future opportunities to pursue
careers in STEM and contribute to these fields with innovative ideas.
21
References
Abedi, J., & Gándara, P. (2006). Performance of English Language Learners as a Subgroup in
Large‐Scale Assessment: Interaction of Research and Policy. Educational Measurement:
Issues and Practice, 25(4), 36-46
Anderman, E. M. (1998). The middle school experience effects on the math and science
achievement of adolescents with LD. Journal of Learning Disabilities, 31(2), 128-138.
Bacharach, V. R., Baumeister, A. A., & Furr, R. M. (2003). Racial and gender science
achievement gaps in secondary education. The Journal of genetic psychology, 164(1),
115-126.
Barnes, C. R. (2011). Race to the Top Only Benefits Big Government. JL & Educ., 40, 393.
Bowles, S. (1970). Towards an educational production function. In Education, income, and
human capital (pp. 11-70). NBER.
Cawley, J., Hayden, S., Cade, E., & Baker-Kroczynski, S. (2002). Including students with
disabilities into the general education science classroom. Exceptional Children, 68(4), 423-
435.
Dragoset, L., James-Burdumy, S., Hallgren, K., Perez-Johnson, I., Herrmann, M., Tuttle, C., ... &
Graczewski, C. (2015). Usage of Policies and Practices Promoted by Race to the
Top (No. 4b296d4310f843acad1f80b5aadd54d2). Mathematica Policy Research.
Retrieved from https://ies.ed.gov/ncee/pubs/20154018/pdf/20154018.pdf
Dragoset, L., Thomas, J., Herrmann, M., Deke, J., James-Burdumy, S., Graczewski, C., Boyle,
A., Tanenbaum, C., Giffin, J., & Upton, R. (2016). Race to the Top: Implementation and
22
Relationship to Student Outcomes (NCEE 2017-4001). Washington, DC: National Center
for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S.
Department of Education. Retrieved on Nov7th, 2016 from https://www.mathematica-
mpr.com/download-media?MediaItemId={75821048-6E9E-456B-A13D-
731A64BE45FD}
Ferguson, R. F. (1991). Paying for public education: New evidence on how and why money
matters. Harv. J. on Legis., 28, 465.
Hanushek, E. A. (1986). The economics of schooling: Production and efficiency in public
schools. Journal of economic literature, 24(3), 1141-1177.
Holdren, J. P., Lander, E. S., & Varmus, H. (2010). Prepare and inspire: K-12 education in
science, technology, engineering, and math (STEM) for America's future. Executive
Report). Washington, DC: President's Council of Advisors on Science and Technology.
Howell, W. G. (2015). Results of President Obama's Race to the Top. Education Next, 15(4)
Johnson, C. (2014).THEC STEM Professional Development Program: Final Evaluation.
Tennessee Consortium on Research, Evaluation and Development. Vanderbilt
University’s Peabody College. Nashville, TN 37203
Lee, O., Maerten‐Rivera, J., Penfield, R. D., LeRoy, K., & Secada, W. G. (2008). Science
achievement of English language learners in urban elementary schools: Results of a first‐
year professional development intervention. Journal of Research in Science
Teaching, 45(1), 31-52.
23
Lieberman, G. A., & Hoody, L. L. (1998). Closing the Achievement Gap: Using the
Environment as an Integrating Context for Learning. Results of a Nationwide Study.
MacPhail-Wilcox, B., & King, R. A. (1986). Production functions revisited in the context of
educational reform. Journal of Education Finance, 12(2), 191-222.
McGuinn, P. (2012). Stimulating reform: Race to the Top, competitive grants and the Obama
education agenda. Educational Policy, 26(1), 136-159.
National Science Foundation Strategic Plan for 2014-2018. (2014). NSF. Arlington, VA.
Nowicki, J. M. (2015). Race to the Top: Education Could Better Support Grantees and Help
Them Address Capacity Challenges. Report to the Ranking Member, Subcommittee on
Labor, Health and Human Services, Education and Related Agencies, Committee on
Appropriations, House of Representatives. GAO-15-295. US Government Accountability
Office.
Quinn, D. M., & Cooc, N. (2015). Science Achievement Gaps by Gender and Race/Ethnicity in
Elementary and Middle School Trends and Predictors. Educational Researcher, 44(6),
336-346.
Ruby, A. (2006). Improving science achievement at high‐poverty urban middle schools. Science
Education, 90(6), 1005-1027.
Scott, G. A. (2011). Race to the Top: Reform Efforts Are Under Way and Information Sharing
Could Be Improved. Report to Congressional Committees. GAO-11-658. US Government
Accountability Office.
24
The White House. Office of the Press Secretary. (2009). President Obama Launches” Educate to
Innovate” Campaign for Excellence in Science, Technology, Engineering& Math (Stem)
Education. Washington, DC: The White House. Retrieved from
https://www.whitehouse.gov/the-press-office/president-obama-launches-educate-
innovate-campaign-excellence-science-technology-en
Todd, P. E., & Wolpin, K. I. (2003). On the specification and estimation of the production
function for cognitive achievement. The Economic Journal,113(485), F3-F33.
U.S. Department of Education (2009). Race to the top Program Executive Summary.
Washington, DC 20202. Author: Retrieved: August 4th, 2016 from
https://www2.ed.gov/programs/racetothetop/executive-summary.pdf
U.S. Department of Education (2015). Fundamental change. Innovation in American schools
under the race to the top. U.S. Department of Education. Washington, DC.
Wooldridge, J. M. (2010). Econometric analysis of cross section and panel data. MIT press
423-435.