A study of the effect of sustained, whole-school professional development on student achievement in science

JOURNAL OF RESEARCH IN SCIENCE TEACHING VOL. 00, NO. 0, PP. 1–12 (2007)

A Study of the Effect of Sustained, Whole-School Professional Developmenton Student Achievement in Science

Carla C. Johnson,1 Jane Butler Kahle,2 Jamison D. Fargo3

1University of Toledo, 2015 Gillham Hall, Mail Stop 924, Toledo, Ohio 43606

2School of Education and Allied Professions, McGuffey Hall, Miami University,

Oxford, Ohio 45056

3Utah State University, 2810 Old Main Hill, Logan, Utah 84322-2810

Received 26 February 2006; Accepted 15 June 2006

Abstract: This longitudinal study of middle school science teachers explored the relationship, if any,

between teacher participation in whole-school, sustained, collaborative professional development and

student achievement in science. Eleven teachers from Glendale Middle School participated in the Discovery

Model Schools Initiative 2-week summer institute, followed by monthly release day professional

development sessions focused on implementing instruction outlined in the National Science Education

Standards. Student achievement was assessed using the Discovery Inquiry Test in Science. The same

students completed the test in grades 6–8. Students of teachers at Glendale Middle School significantly

outperformed students at the control school. Findings in this study revealed the positive impact that whole-

school, sustained, collaborative professional development programs have on student achievement,

indicating that programs of this nature could be a means to narrowing or eliminating achievement gaps

in science. � 2006 Wiley Periodicals, Inc. J Res Sci Teach 00: 1–12, 2007

In the current era of accountability, science teachers across the United States are standing in

the shadows of No Child Left Behind (NCLB), which will require science testing in 2007. Science

teaching and achievement have not been scrutinized yet under NCLB, as reading and mathematics

have been the major focus. As a result, school level emphasis on science has diminished,

especially in terms of the time allocated to teaching science. However, because nationwide

accountability for science achievement is now imminent, there is a great need to identify strategies

for improving students’ performance in science.

Contract grant sponsor: Ohio Board of Regents (Discovery Model School Initiative; principal investigator: Terry L.

McCollum).

Correspondence to: C.C. Johnson, University of Toledo, 2015 Gillham Hall, Mail Stop 924, Toledo, OH 43606;

E-mail: [email protected]

DOI 10.1002/tea.20149

Published online in Wiley InterScience (www.interscience.wiley.com).

� 2006 Wiley Periodicals, Inc.

This study is focused on the efforts of one school to improve science teaching and learning

through having all science teachers participate in a science and mathematics professional

development program that immerses teachers in experiencing science, developing and

implementing science curricula, and examining current instructional practices. In addition,

science teachers participated in collaborative grade- and school-level groups to implement

standards-based instruction. The intervention was developed and implemented by the Discovery

Institute, which began in 1991 as one of the ten original Statewide Systemic Initiatives and

continues with support from the State of Ohio. The focus of Discovery Institute’s teacher

professional development was on individual teachers and principals as change agents in the

schools. Individual teachers from various schools attended 6-week summer institutes where they

experienced inquiry-based learning and were involved in standards-based instruction while also

engaged in strengthening their content knowledge. As Discovery evolved over the years, its focus

changed from providing sustained professional development for individual teachers to

collaborating with schools to provide professional development for all science and/or

mathematics teachers in a school through its Model School Initiative. The Discovery intervention

included all components that have been identified as part of effective professional development

defined by current research in this area (Loucks-Horsley et al., 2003). In this study, the efficacy of

the Discovery Model School Initiative on student achievement in science is described and

assessed.

Review of the Literature

Current research in science education indicates that an effective professional development

program can have a positive impact on student attitudes and increased teacher use of standards-

based instructional practices (Supovitz & Turner, 2000). However, research linking professional

development with increased student achievement is inconclusive, primarily due to the dearth of

longitudinal studies in this area (Huffman & Thomas, 2003; Shymansky, Yore, & Anderson,

2004). Professional development programs that are successful in developing standards-based

instructional environments and in increasing student achievement are needed to meet the demands

of NCLB, as well as to ensure that all students attain scientific literacy, as defined by the National

Science Education Standards (NSES; National Research Council, 1996). Federal funding

agencies, such as the National Science Foundation and the U.S. Department of Education, award

millions of dollars for professional development each year without requiring data to support

student achievement gains—the indirect goal of most professional development efforts

(Frechtling, Sharp, Carey, & Vaden-Kiernan, 1995). This situation is due to the difficulty of

isolating and measuring ‘‘the specific effects of professional development on student

achievement’’ (Shymansky et al., 2004, p. 772).

However, recent studies have shown that students of science teachers who participate in

professional development activities designed to increase the use of standards-based instructional

practices demonstrate increased achievement, as measured by pre/post-unit assessments (Marx

et al., 2004; Rivet & Krajcik, 2004). Findings from several studies demonstrated improved state

assessment scores across years by students whose teachers have participated in professional

development (Czerniak et al., 2005; Schneider, Krajcik, Marx, & Soloway, 2002). One problem is

that most of these studies did not compare the same students over time, but different groups of

students each year. In summary, the current research base failed to demonstrate a clear relationship

between professional development and student performance. As Shymansky et al. (2004) argued,

‘‘We have no choice as professional educators but to continue to study those connections, lest we

2 JOHNSON, KAHLE, AND FARGO

Journal of Research in Science Teaching. DOI 10.1002/tea

admit that we are only stabbing in the dark with our professional-development promises and

practices’’ (p. 788).

Other research in mathematics and science indicates that increased student achievement is

directly related to the length as well as the type of professional development that teachers

experience (Cohen & Hill, 2000; Loucks-Horsley et al., 2003). Supovitz and Turner (2000) also

found that at least 80 hours of professional development are needed before a statistically

significant relationship can be identified between professional development experiences and

changes in teaching practices (e.g., increases in the type of instruction promulgated by the NSES,

referred to as ‘‘standards-based’’ practices). However, many professional development

opportunities are not sustained.

McChesney (1998) has argued that, in order to obtain significant improvements in student

learning, change must be established at the school level. Unfortunately, Anderson (2002)

contended that a systemic design of professional development is rare, because of the difficulty in

getting all teachers in a content area or in a school to participate. He pointed out that funding for

systemic reform is a problem at district and state levels (Anderson, 2002). When whole-school,

collaborative, sustained professional development programs are offered, most teachers will

improve their instructional practices (Hart & Lee, 2003; Johnson, 2006). Further research is

needed on whole-school professional development programs to determine the impact on

instructional practices and student achievement in science. The present study examines the

efficacy of sustained professional development at the school level to improve student achievement

in science.

Background

Purpose of the Study

The findings reported in this investigation are a part of a 3-year longitudinal study that

examined whole-school, collaborative, sustained professional development in science and the

impact of this experience on instructional practices, teacher beliefs, and student achievement. The

specific research question explored was as follows: ‘‘What relationship, if any, exists between

teacher participation in whole-school, sustained professional development and student

achievement in science?’’

Setting

The study focused on two suburban middle schools in Ohio (pseudonyms are used for both).

One school, Glendale Middle School, was involved in the Discovery Model School Initiative for

3 years. A second school, Central Middle School, was not involved in the program. There were 11

science teachers at Glendale Middle School and 6 science teachers at Central Middle School. All

of the science teachers at both schools participated in this study. Central Middle School was

chosen as the comparison school because its student body closely matched that of Glendale. Both

schools consisted of grades 6 through 8 and enrolled between 750 and 900 students during a given

year of the study.

The majority of students at both schools were white, although African-American students

were the largest minority group at both schools. A smaller minority of students identified

themselves as Asian, Hispanic, or multiracial. The percentage of economically disadvantaged

students at the two schools in this study ranged from 28% to 36% (Table 1).

PROFESSIONAL DEVELOPMENT ON STUDENT ACHIEVEMENT IN SCIENCE 3

Journal of Research in Science Teaching. DOI 10.1002/tea

The study followed reform efforts at Glendale Middle School for 3 years, from fall 2002 to

summer 2005. The Discovery Model School Initiative involved science and mathematics teachers

in professional development experiences at Glendale Middle School. However, this research study

focused only on the science teachers at Glendale.

The Discovery Model School Initiative required whole-school involvement, including

representation of the administrative staff in the sustained, collaborative professional development

experience. It was an intensive program, engaging participants in 80 hours of professional

development during the first summer, followed by 36 hours across each of the 3 academic years,

for a total of 198 hours. The Discovery Model School Initiative emphasized standards-based

instructional practices. Standards-based instruction refers to the instructional strategies, described

by the NSES, which focus on inquiry as the central mode for teaching science. The program

initially provides professional development for a team of science and mathematics teachers and

administrators during the summer. The team then provides professional development during the

school year for all the science and/or mathematics teachers in the school. Both during the summer

and throughout the school year, each team has a university faculty member who serves as a coach.

Methods

Professional Development

The Discovery Model School Initiative team from Glendale Middle School attended a 2-week

intensive summer institute with teams from other schools. The summer institute was held at a local

university. Each school team worked with a science education faculty coach from the university.

Team members were immersed in standards-based instruction, which focused on process skills in the

context of content, examining current instructional strategies through a collaborative process, and

developing or modifying curriculum to better meet the needs of students. Teachers were introduced to

the NSES and inquiry teaching, and they were provided time to experience inquiry and to reflect on

how to use inquiry in their lessons. In addition, during the summer session, the Glendale science team

developed a professional development plan for the academic year, which focused on implementing

standards-based instruction. Their professional development plan was implemented over each of

3 years with support from the faculty coach. However, the professional development was led by the

team of teachers and administrators at the school. Eleven science teachers at Glendale participated in

the program and formed an ongoing support network, holding monthly meetings during the school

day (Johnson, 2006). Prior to the Discovery Model School Initiative, the science teachers at Glendale

did not have a common planning time and met only at monthly departmental meetings focused on

administrative tasks (e.g., schedules, ordering supplies, etc.).

During Year 1 of the Discovery Model School Initiative, science teachers learned about the

NSES (NRC, 1996) and inquiry through the monthly release sessions. Grade-level teachers

collaborated to modify existing curriculum and to design new inquiry lessons, which were

Table 1

Demographic Data for Students at Glendale and Central Middle School 2004–2005

SchoolAfrican-

American Asian Hispanic Multiracial White LEPEconomicallyDisadvantaged

GMS 18% 3% 2% 4% 73% 3% 28%CMS 20% 2% 2% 4% 74% 2% 36%

GMS, Glendale Middle School; CMS, Central Middle School.

4 JOHNSON, KAHLE, AND FARGO

Journal of Research in Science Teaching. DOI 10.1002/tea

implemented during Year 1. After implementation, teachers would discuss experiences, including

what worked, what needed to be revised, and how they would use the lesson in the following year.

In Year 2 of the program, the Glendale teachers focused their professional development

efforts on learning about Ohio’s new science content standards and aligning their instruction to

those standards at each grade level. This exercise was frustrating for teachers, as they gave up

topics they had taught for several years in order to include new topics and present new content

included in the state standards for their grade level. This task provided another opportunity for

collaboration. The focus of the school-based professional development in Year 2 was to examine

each grade’s science curriculum, to share effective investigations, and to modify others to meet the

new state standards (Johnson, 2006). The professional development in Year 3 was a continuation

of Year 2, as the science teachers continued to try new investigations and to modify others during

their monthly meetings. In addition, authentic assessment was a focus of the professional

development activities in Year 3.

Teachers at Glendale Middle School increased their use of standards-based instructional

practices while participating in the 3-year program. Classroom observations were conducted

several times each year using the Horizon Research Local Systemic Change (LSC) Classroom

Observation Protocol. Teachers at Glendale improved in design, implementation, content

knowledge, and classroom culture components of standards-based instruction identified on the

LSC Classroom Observation Protocol (Johnson, 2006).

Research Design

This study consisted of a comparison of students of teachers who had participated in the

professional development (intervention) versus students in a comparable school whose teachers

had not participated (comparison). Therefore, the study is best described as a posttest-only, quasi-

experimental, control-group, interrupted-time-series design. Achievement was assessed by the

Discovery Inquiry Test (DIT) in science (described in what follows). Students at Glendale and

Central Middle Schools completed the DIT in grades 6 (Year 1), 7 (Year 2), and 8 (Year 3). A total

of 17 teachers across the two schools participated in the study. However, for the purpose of data

collection and analysis, only the teachers who were teaching science to the group of students each

year (i.e., Year 1, grade 6 teachers only; Year 2, grade 7 teachers only; Year 3, grade 8 teachers

only) were included as participants in this portion of the larger study for that particular year. The

DIT assessment was administered to students during March of each year. At Glendale, individual

students were followed over the 3 years of the study, whereas at Central scores from students at the

appropriate grade level were collected each year.

Sample

The DIT was administered (same instrument across years) to 282 students at Glendale Middle

School (intervention) in Year 1 (grade 6), 288 in Year 2 (grade 7), and 289 in Year 3 (grade 8). One

hundred eighty-five students from Glendale completed the DIT in all 3 years of the study. Data for all

students from Glendale who completed the DIT during at least one of the years of the study were

included in the analyses. At Central Middle School (comparison school), a total of 217 students

completed the DITin Year 1, 210 in Year 2, and 215 in Year 3. Although student identifiers were used to

follow students at Glendale over time, the study could not track individual students at Central.

Therefore, only year-by-year, cross-sectional comparisons could be made between the control and

intervention schools, although change in achievement could be analyzed across the 3 years at Glendale.

PROFESSIONAL DEVELOPMENT ON STUDENT ACHIEVEMENT IN SCIENCE 5

Journal of Research in Science Teaching. DOI 10.1002/tea

Instruments

The DIT in science assessment was developed in 1994 by members of the Ohio Statewide

Systemic Initiative academic leadership teams, university science faculty, and other Ohio

teachers. Items were derived from the National Assessment of Educational Progress 1990 and

1992 public release items and focused on measuring students’ ability to analyze and interpret data,

to extrapolate from one situation to another, and to utilize conceptual understanding (Kahle,

Meece, & Scantlebury, 2000). As shown in Table 2, the DIT includes 29 items, 11 focusing on life

science, 8 on physical science, 6 on earth and space science, and 4 on the nature of science. The

DIT has been validated by a national and international expert panel of science educators, and its

internal consistency reliability is high (Cronbach’s alpha coefficient¼ 0.94). Because the DIT is

not linked to a particular curriculum and it assesses both content and process skills, it was

considered appropriate for this study.

Data Analysis

The majority of students identified themselves as either Caucasian or African-American

(86% in Year 1, 83% in Year 2, and 86% in Year 3). Varying by year, the remaining students

identified themselves as either American Indian/Alaskan Native (2–3%), Asian/Pacific Islander

(2–5%), Hispanic (2%), and Other (7–8%). To assess any effect of race/ethnicity on outcomes in

the statistical analyses, a dichotomous race/ethnicity variable was formed: white and minority.

Chi-square analyses were used to evaluate any differences in the proportions of males and females

or white and minority students, as distributed across the intervention and control groups in each

year of the study. McNemar tests for correlated proportions were conducted to determine if the

proportions of students by gender or race/ethnicity varied significantly across the 3 years within

the intervention and comparison groups.

To determine the effect of teacher participation in the Discovery Model School Initiative on

student achievement, a cross-sectional multiple regression analysis that adjusted for

cluster sampling was conducted for each year of the study using procedures currently

implemented in the Mplus version 3.13 statistical modeling program (Muthen & Muthen,

2004). This particular method of analysis was selected because of the importance of adjusting the

standard errors and Chi-square tests of model fit for the non-independence among students due to

cluster sampling within classrooms. Predictors in each model included study group

membership (intervention/comparison), race/ethnic identity (white/minority), gender (male/

female), and interactions between (a) study group membership and race/ethnicity and (b) study

group membership and gender. Student classroom membership formed the cluster variable and

was treated as an unestimated nuisance variable. Such time-by-time regression models were used

Table 2

Description of Discovery Inquiry Test (DIT)

Content

NAEP Process Code

Knowing Science Solving Problems Conducting Inquiry Totals

Life science 2 3 6 11Physical science 4 2 2 8Earth/space science 2 4 0 6Nature of science 1 3 0 4Totals 9 12 8 29

6 JOHNSON, KAHLE, AND FARGO

Journal of Research in Science Teaching. DOI 10.1002/tea

because the identity of the participants in the control group was unknown and, therefore, the effect

of group membership could not be modeled over time (Diggle, Heagerty, Liang, & Zeger, 2002).

Results

Demographic Characteristics

A summary of the demographic characteristics of the students and teachers from Glendale

and Central Middle Schools is presented in Table 3. A series of Chi-square tests indicated that

there were no significant differences (p > 0.05) between the intervention and comparison groups

in terms of: (a) the proportions of male and female students in any given year of the study, and (b)

the proportions of white and minority students in Years 1 and 2. However, there were significantly

more white students in the control (80%) versus the intervention (68%) condition in Year 3

[w2(1)¼ 8.61, p¼ 0.003]. However, a measure of effect size, f¼ 0.13, indicated that this

association was not strong. A series of McNemar tests for correlated proportions indicated that the

proportions of student gender and race/ethnicity did not vary significantly from year to year

(p > 0.05).

Year-by-Year Test Score Comparisons

A series of three multiple regression analyses were conducted to examine the effect of study

group (intervention/comparison), race/ethnic identity (white/minority), and gender (male/

female) on test scores for each year of the study, adjusting for non-independence due to cluster

(i.e., classroom) sampling. The mean number of students sampled within classrooms was 13.61.

See Figure 1 for adjusted test score means and Table 3 for unadjusted test score statistics by year

and study group.

The analysis for Year 1 indicated a significant main effect for race/ethnicity, with minority

students scoring, on average, 0.83 point lower than white students (b¼ 0.83, p< 0.05). However,

there were no significant differences in student achievement between Glendale (intervention) and

Central (comparison) or between male and female students (p > 0.05), indicating that students in

Table 3

Demographic Characteristics and Student Test Scores for Middle School Students Included in the

Intervention and Control Conditions

Year 1: Grade 6Teachers and Students

Year 2: Grade 7Teachers and Students

Year 3: Grade 8Teachers and Students

Control Intervention Control Intervention Control Intervention

Teachers (n) 2 5 2 3 2 3Students (n) 217 282 210 212 215 210Student gender (% M) 51 53 47 53 50 54Student race/ethnicity (%)

White 78 73 75 69 80 68African-American 10 11 10 13 9 16Other 12 16 15 18 11 16

Student test scoresMean 8.28 8.93 8.17 12.28 8.16 13.28Median 8.00 9.00 8.00 13.00 8.00 14.00SD 3.96 3.13 3.02 3.16 3.19 3.36Minimum 1.00 1.00 1.00 4.00 1.00 5.00Maximum 17.00 18.00 15.00 20.00 16.00 21.00

PROFESSIONAL DEVELOPMENT ON STUDENT ACHIEVEMENT IN SCIENCE 7

Journal of Research in Science Teaching. DOI 10.1002/tea

both schools were relatively similar after the first year of the study in terms of science content

knowledge and process skills. There were no significant interactions present.

In Year 2, there were significant main effects for both study group (b¼ 3.89, p< 0.0001) and

race/ethnicity (b¼ 0.55, p< 0.05), but not for gender (p > 0.05). Examination of the adjusted test

score means for the schools indicates that students at Glendale (M¼ 11.94) scored significantly

higher on the DIT than students at Central (M¼ 8.04). Again, no significant interactions were

present.

In Year 3, there was an even greater increase in student science achievement at Glendale. Both

the main effects for study group (b¼ 5.19, p< 0.001) and for race/ethnicity (b¼ 0.91, p< 0.01)

were significant. The main effect for gender was nonsignificant (p > 0.05). Examination of the

adjusted test score means for Glendale and Central Middle Schools in Year 3 indicated higher

achievement for students at Glendale (M¼ 13.32) than for their peers at Central (M¼ 8.13). No

significant interactions were present.

A comparison of grade 6 state assessment scores for students at Glendale and Central prior to

and during the 3 years of this study indicates a similar trend, in that Central’s scores were similar

across years, and Glendale experienced an increase in Year 1, along with higher scores than

Central in Years 2 and 3 of the study (Table 4). In Ohio, only grades 6 and 9 science assessments are

given annually, so there are no data for achievement in grades 7 or 8 other than the DIT data

presented in this study. DIT assessment data in Year 1 was from students in grade 6, in Year 2

students in grade 7, and in Year 3 students in grade 8. Therefore, only grade 6 science assessment

results were directly related to the grade 6 teachers in this study and serve to show the pattern of

achievement for grade 6 students at both schools across years.

Table 4

Percentage of Students Proficient in Science: Grade 6 State Assessment

2001–02a 2002–03 2003–04 2004–05

Glendale MS 78 85 78 80Central MS 61 62 65 64

aThe 2001–02 school year was prior to this study.

Figure 1. Adjusted mean test scores (�2 SE) for each year of the study stratified by study group

membership.

8 JOHNSON, KAHLE, AND FARGO

Journal of Research in Science Teaching. DOI 10.1002/tea

As mentioned previously, no significant interactions were found between study group

(intervention versus comparison) and race/ethnicity or gender. This suggests that any differences in

test scores between male and female or white and minority students did not vary by study group. In

addition, for each year of the study, simple follow-up tests were conducted to determine whether test

scores of white and minority students differed significantly within each school. Results indicated

that DIT scores for white and minority students only differed significantly in the comparison school

during Years 1 and 3 (p< 0.05), whereas scores did not differ significantly during any year in the

intervention school (see Table 5 and Figure 2). For instance, in the comparison school, average

minority student scores were lower than white student scores by 1.57 and 1.34 points for Year 1 and

Year 3, respectively. Thus, although a significant difference in test scores was observed for the entire

sample as a function of race/ethnicity (average difference¼ 0.83, 0.55, and 0.91 points for each year

of the study), when examined within school/group, this difference was only statistically significant

for the control school during Years 1 and 3.

Both white and minority students at Glendale Middle School increased their science

achievement, as measured by the DIT, over the 3 years of the Discovery Model School Initiative (see

Figure 2). At Glendale in Year 1, white students’ mean score on the DIT was 9.07, whereas the

minority students’ mean score was 8.61. In Year 2, the mean for white students was 12.23 and 11.67

for minority students. In Year 3, white students’ mean score for Glendale was 13.35, compared with

Table 5

White and Minority Student DIT Scores

Year 1: Grade 6 Year 2: Grade 7 Year 3: Grade 8

Control Intervention Control Intervention Control Intervention

White (n) 158 192 155 193 157 191Minority (n) 45 72 51 85 39 88Student test

scoresa

White 8.86 (4.05) 9.07 (3.09) 8.33 (2.98) 12.23 (3.21) 8.34 (3.29) 13.35 (3.36)Minority 7.29 (3.40) 8.61 (3.06) 7.69 (3.15) 11.67 (3.19) 7.00 (2.83) 12.60 (3.13)

Note: Student test scores expressed as mean (standard deviation).

Figure 2. Mean achievement scores (�2 SE) for each year of the study stratified by study group and race/

ethnicity.

PROFESSIONAL DEVELOPMENT ON STUDENT ACHIEVEMENT IN SCIENCE 9

Journal of Research in Science Teaching. DOI 10.1002/tea

12.60 for minority students (see Table 5). On the other hand, at Central Middle School, scores

for the white and minority students stayed approximately the same in all 3 years of the study

(see Figure 2).

In summary, we found that there was a relationship between student science achievement and

teacher participation in whole-school, sustained, collaborative professional development. There

were no significant differences in achievement scores on the DIT between the two study groups in

March of the first year of the intervention. However, both white and minority students at Glendale

Middle School (intervention), when compared with similar students at Central Middle School

(comparison), had significantly higher mean scores on the DIT in Years 2 and 3. The increase in

achievement by minority students is particularly notable. Although both white and minority mean

scores rose across the years at Glendale, neither group of students at Central had significant

changes over the 3 years. In Year 3, there were significant main effects between study groups and

for race/ethnicity. Students of teachers who had participated in the professional development

scored significantly higher on the DIT than their counterparts, whose teachers had not participated,

in Years 2 and 3.

Discussion

This study was designed to assess the effect of a sustained, whole-school professional

development program on student achievement. Existing research studies have demonstrated that

student achievement in science may be affected in the short term (pre- or posttesting) through

effective professional development experiences. However, there have been few attempts to

conduct a longitudinal study of the effects of sustained professional development on student

achievement. Concurrently, there is a growing need for research to investigate the effect of

sustained versus short-term (less than 40 hours) professional development in order to determine

whether the increased investment results in measurably enhanced student learning in science.

Findings from this study suggest that sustained professional development that involves all the

science teachers in a school enhances the school’s student achievement in science.

During the first year of the study, students from Glendale Middle School and Central Middle

School scored similarly on the Discovery Inquiry test in science. Only in Years 2 and 3 were

significant differences in achievement found between students at the two schools. During the first

year of the Model School Initiative, teachers were introduced to the NSES, learned how to manage

an investigative science classroom, and began to examine and revise their current practices in

order to use standards-based instruction. By the end of Year 2, teachers had completed 100 hours of

professional development. At this time, significant changes in student achievement were realized

and increased use of effective instructional practices were documented for teachers at Glendale

Middle School (Johnson, in press). Through a more conducive learning environment created by

teachers using effective instructional strategies learned or enhanced through the professional

development program, student achievement at Glendale Middle School was positively impacted,

as student performance on the DIT improved each year. As discussed in the literature (Johnson,

2006; Supovitz & Turner, 2000), changes in instructional practice, associated with participation in

a professional development experience, require time to be transferred effectively into practice.

The findings provide evidence that teacher participation in effective, sustained, professional

development and their subsequent use of standards-based instructional strategies have a positive

impact on their students’ performance in science. Both white and minority students at Glendale

Middle School increased their scores on the DIT assessment over the 3 years of the study.

However, neither white nor minority students in the grades tested at the comparison school

(Central Middle School) significantly increased their scores. Similarly, scores on the state’s

10 JOHNSON, KAHLE, AND FARGO

Journal of Research in Science Teaching. DOI 10.1002/tea

proficiency test in science were higher for the intervention school than for the comparison school

during the period of professional development.

Summary and Implications

The results of this quasi-experimental study indicate a relationship between whole-school,

sustained professional development and student learning. At Glendale Middle School, students

were followed across 3 years of science classes and participated in the Discovery Model School

Initiative, whereas a comparable sampling of students at each grade level from Central Middle

School was used for comparison. The study suggests that the duration and structure of professional

development is linked to increased student achievement in science. Whole-school involvement

ensures that students will have continuous opportunities to experience standards-based

instruction. Whole-school, sustained professional development provides the opportunity for

collaboration of teachers over time, which creates a community of learners and enables

professional growth, even outside the realm of the program. In Year 4, when funding for the

program ceased, teachers continued to collaborate extensively on their own time. A shared vision

of continuous improvement of practice and desire to improve student learning emerged as the

result of the Discovery Model School Initiative experience for these teachers. There has been no

turnover in science teaching staff in the past 6 years, which speaks to the power of professional

development, collaboration, and camaraderie to retain teachers.

In this study, students’ repeated involvement in improved instruction resulted in significant

achievement gains by both majority and minority students in Years 2 and 3. Supovitz and Turner

(2000) found that the duration of professional development is vital, arguing that ‘‘dramatic results

emerged when experiences were deeper and more sustained’’ (p. 975). If the ultimate goal of

professional development is to improve student learning, the findings of this study suggest that

sustained activities are important in spite of increased costs.

Sustained professional development also improved the performance of all students, as both

majority and minority student achievement on the DIT increased each year at Glendale Middle

School. Findings from this study add to the current knowledge base on the ability of professional

development experiences, focused on increasing content knowledge and use of standards-based

teaching practices, to ‘‘reduce inequities in achievement patterns’’ for minority students (Kahle

et al., 2000, p. 1035).

The findings also suggest that a more rigorous study is needed. It is suggested that researchers

consider a study that includes assessing changes in teaching practices (by observation and

interviews), following the same group of student longitudinally at both the intervention and

comparison schools, and using several assessments of science achievement.

References

Anderson, R.D. (2002). Reforming science teaching: What research says about inquiry.

Journal of Science Teacher Education, 13, 1–12.

Cohen, D.K., & Hill, H.C. (2000). Instructional policy and classroom performance: The

mathematics reform in California. Teachers College Record, 102, 294–343.

Czerniak, C.M., Beltyukova, S, Struble, J., Haney, J.J., & Lumpe, A.T. (2005). Do you see

what I see? The relationship between a professional development model and student achievement.

In R.E. Yager (Ed.), Exemplary science in grades 5–8: Standards-based success stories (pp. 13–

43). Arlington, VA: NSTA Press.

PROFESSIONAL DEVELOPMENT ON STUDENT ACHIEVEMENT IN SCIENCE 11

Journal of Research in Science Teaching. DOI 10.1002/tea

Diggle, P.J., Heagerty, P., Liang, K.Y., & Zeger, S.L. (2002). Analysis of longitudinal data

( 2nd ed.) Oxford: Oxford University Press.

Frechtling, J., Sharp, L., Carey, N., & Vaden-Kiernan, N. (1995). Teacher enhancement

programs: A perspective on the last four decades. Washington, DC: National Science Foundation.

Hart, J.E., & Lee, O. (2003). Teacher professional development to improve the science and

literacy achievement of English language learners. Bilingual Research Journal, 27, 475–501.

Huffman, D., & Thomas, K. (2003). Relationship between professional development,

teachers’ instructional practices, and the achievement of students in science and mathematics.

School Science and Mathematics, 103, 378–387.

Johnson, C.C. (2006). Effective professional development and change in practice: Barriers

teachers encounter and implications for reform. School Science and Mathematics. 106, 1–12.

Johnson, C.C. (in press). Whole school collaborative professional development and science

teacher change: Signs of success. Journal of Science Teacher Education.

Kahle, J.B., Meece, J., & Scantlebury, K. (2000). Urban African-American middle school

science students: Does standards-based teaching make a difference? Journal of Research in

Science Teaching, 37, 1019–1041.

Loucks-Horsley, S., Love, N., Stiles, K.E., Mundry, S., & Hewson, P.W. (2003). Designing

professional development for teachers of mathematics and science. Thousand Oaks, CA: Corwin

Press.

Marx, R.W., Blumenfeld, P.C., Krajcik, J.S., Fishman, B., Soloway, E., Geier, R., & Tali Tal,

R. (2004). Inquiry-based science in the middle grades: Assessment of learning in urban systemic

reform. Journal of Research in Science Teaching, 41, 1063–1080.

McChesney, J. (1998). Whole-school reform. ERIC Digest, 124 (ERIC Document

Reproduction Service No. ED 427 388).

Muthen, L.K., & Muthen, B.O. (2004). Mplus user’s guide (3rd ed.). Retrieved February 17,

2006, from the Mplus website: http:/ /www.statmodel.com/ugexcerpts.shtml

National Research Council. (1996). National science education standards. Washington, DC:

National Academy Press.

Rivet, A.E., & Krajcik, J.S. (2004). Achieving standards in urban systemic reform: An

example of a sixth grade project-based science curriculum. Journal of Research in Science

Teaching, 41, 669–692.

Schneider, R.M., Krajcik, J., Marx, R.W., & Soloway, E. (2002). Performance of students in

project-based science classrooms on a national measure of science achievement. Journal of

Research in Science Teaching, 39, 410–422.

Shymansky, J.A., Yore, L.D., & Anderson, J.O. (2004). Impact of a school district’s science

reform efforts on the achievement and attitudes of third and fourth grade students. Journal of

Research in Science Teaching, 41, 771–790.

Supovitz, J.A., & Turner, H.M. (2000). The effects of professional development on science

teaching practices and classroom culture. Journal of Research in Science Teaching, 37, 963–980.

12 JOHNSON, KAHLE, AND FARGO

Journal of Research in Science Teaching. DOI 10.1002/tea