Summary of research on the effectiveness of math professional development approaches Russell Gersten, Mary Jo Taylor, Tran D. Keys, Eric Rolfhus, and Rebecca Newman-Gonchar Instructional Research Group This comprehensive literature review identified 643 studies of professional development interventions related to math in grades K–12. Thirty-two of the studies used a research design for assessing the effectiveness of math professional development approaches, and five of those met What Works Clearinghouse evidence standards. Of the five, only two found positive effects on student math proficiency. This study used a systematic process modeled after the What Works Clearinghouse (WWC) study review process to answer the question: What does the causal research say are effective math professional development interventions for K–12 teachers aimed at improving student achievement? The study iden- tified and screened 910 research studies in a comprehensive literature search for effectiveness studies 1 of math professional development approaches. (See appendix A for details of the search, screening, and review process.) Of these 910 studies, 643 examined professional development approaches related to math in grades K–12 and were conducted in the United States. Of the 643 studies, 32 focused primarily on math professional development provided to teachers and used a research design for examining effectiveness (see appen- dix B for a list of the 32 studies). Five of those were determined to have met WWC evidence standards (version 2.1) either with or without reservations (appendix C). 2 And of those five, only two found posi- tive effects on students’ math proficiency. Thus, there is very limited causal evidence to guide districts and schools in selecting a math professional development approach or to support developers’ claims about their approaches. The limited research on effectiveness means that schools and districts cannot use evidence of effectiveness alone to narrow their choice. Instead, they must use their best judgment until more causal evidence becomes available. U.S. Department of Education At Florida State University
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Summary of research on the effectiveness of math professional development approaches
Russell Gersten, Mary Jo Taylor, Tran D. Keys, Eric Rolfhus, and Rebecca Newman-Gonchar
Instructional Research Group
This comprehensive literature review identified 643 studies of professional development
interventions related to math in grades K–12. Thirty-two of the studies used a research
design for assessing the effectiveness of math professional development approaches,
and five of those met What Works Clearinghouse evidence standards. Of the five, only
two found positive effects on student math proficiency.
This study used a systematic process modeled after the What Works Clearinghouse (WWC) study review process to answer the question: What does the causal research say are effective math professional development interventions for K–12 teachers aimed at improving student achievement? The study iden-tified and screened 910 research studies in a comprehensive literature search for effectiveness studies1 of math professional development approaches. (See appendix A for details of the search, screening, and review process.)
Of these 910 studies, 643 examined professional development approaches related to math in grades K–12 and were conducted in the United States. Of the 643 studies, 32 focused primarily on math professional development provided to teachers and used a research design for examining effectiveness (see appen-dix B for a list of the 32 studies). Five of those were determined to have met WWC evidence standards (version 2.1) either with or without reservations (appendix C).2 And of those five, only two found posi-tive effects on students’ math proficiency.
Thus, there is very limited causal evidence to guide districts and schools in selecting a math professional development approach or to support developers’ claims about their approaches. The limited research on effectiveness means that schools and districts cannot use evidence of effectiveness alone to narrow their choice. Instead, they must use their best judgment until more causal evidence becomes available.
U . S . D e p a r t m e n t o f E d u c a t i o n At Florida State University
2
The effects of five professional development approaches on student math proficiency
Of the five math professional development approaches that had effectiveness studies that met WWC stan-dards, two had statistically significant positive effects, one had limited effects, and two had no discernible effect (table 1).3
Table 1. Summary of findings for the five effectiveness studies of math professional development approaches that met What Works Clearinghouse evidence standards
Findings of the effectiveness study Professional development approach
Statistically significant positive effectsa • Intensive math content courses accompanied by follow-up workshops (Sample McMeeking, Orsi, & Cobb, 2012).
• Lesson study focused on linear (measurement) model of fractions (Perry & Lewis, 2011).
Limited effectsb • Cognitively Guided Instruction (Carpenter, Fennema, Peterson, Chiang, & Loef, 1989; Jacobs, Franke, Carpenter, Levi, & Battey, 2007).c
No discernible effectd • America’s Choice (Garet et al., 2010, 2011).• Pearson Achievement Solutions (Garet et al., 2010, 2011).
a. Implies that the researchers are confident that there is a real, causal relationship between the professional development ap-proach and any subsequent changes in student performance and that the probability of observing such a result by chance is very slim. For more information, see U.S. Department of Education (2011) and WWC Glossary of Terms (http://ies.ed.gov/ncee/wwc/Glossary.aspx).
b. The effects were limited to knowledge of the equal sign. None of the measures of broader mathematics proficiency were significant.
c. The findings described differ from those reported in Yoon, Duncan, Lee, Scarloss, & Shapley (2007), which evaluated studies of professional development from 1986 to 2006 using standards similar to but less rigorous than those of the WWC standards (personal communication with Kwang Suk Yoon).
d. Implies that there was no evidence that the professional development approach had an effect on student math proficiency. For more information, see U.S. Department of Education (2011) and WWC Glossary of Terms (http://ies.ed.gov/ncee/wwc/Glossary.aspx).
Source: Authors’ literature review (see appendix A).
Statistically significant positive effects on student math proficiency
Intensive math content courses accompanied by follow-up workshops (Sample McMeeking, Orsi, & Cobb, 2012). This approach, stemming from a Mathematics Science Partnership grant funded by the National Science Foundation, combines math content courses and workshops. Teacher participants selected and enrolled in one or two university summer courses in math (lasting two to three weeks). The courses were geared toward the needs of middle school and upper elementary school teachers (for example, math modeling, algebraic patterns and functions, geometry). Each course focused 80 percent on math content, with time spent on math principles and math problems, and 20 percent on pedagogy. Fall follow-up workshops, held on four Saturdays, focused on designing lessons using the content from the summer courses. The fall courses were taught by university instructors, and the follow-up sessions were directed by both university instructors and district personnel approved by the university. This intensive professional development effort resulted in significant improvement in student math achievement as measured by the statewide assessment. The effect was found only for teachers who enrolled in two full math courses during the summer; there was no effect for teachers who enrolled in only one course.
Lesson study focused on linear (measurement) model of fractions (Perry & Lewis, 2011). In the lesson study approach small groups of teachers observed and analyzed fractions lessons that they planned collabora-tively. The lesson study groups met 12–14 times over five months during the school year. Unlike the previ-ous approach, where university instructors or district personnel provided intensive training and follow-up,
3
teachers took turns leading the group, following the lesson study cycle outlined in the intervention mate-rials. Instructors and consultants provided the intervention materials (including a fractions toolkit that included materials to help students learn how to represent fractions on a number line4) and were available to answer teachers’ questions as they led their teacher study groups. Implementation was thus similar to actual practice in a school or district. This study resulted in a significant increase of fractions knowledge on a test at the end of the year in grades 2, 3, and 5 but not in grade 4.
Limited effects on student math proficiency
Cognitively Guided Instruction (Carpenter et al., 1989; Jacobs et al., 2007). Cognitively Guided Instruction focused on helping teachers understand how students think about math, what informal knowledge teachers bring to the classroom, and how algebraic principles can be linked to the arithmetic taught in elementary school. Carpenter et al. (1989) examined the effectiveness of Cognitively Guided Instruction provided in a four-week summer workshop. Jacobs et al. (2007) looked at an approach in which teachers met once a month, receiving onsite support from a facilitator one-half day a week. In both cases researchers from local universities conducted the professional development. Neither study provided the information needed to calculate effect sizes using WWC standards (version 2.1).5 Jacobs et al. (2007) found a statistically signif-icant positive effect on student performance on a test focused on one specific, fairly limited topic: under-standing the equals sign for simple addition problems.
No discernible effect on student math proficiency
America’s Choice (Garet et al., 2010, 2011). America’s Choice asks teachers to solve sets of math problems both individually and in small groups, use precise definitions, give short oral presentations explaining how they solved the problems, and receive feedback on their approach. Teachers present their solutions and discuss the most common student misconceptions associated with the topic. This approach was adapted to conform to the requirements of a national research study on rational numbers for grade 7 students. The adaptations included retooling the scope and sequence of the professional development approach. On a test of student knowledge of rational numbers, no statistically significant effect was observed at the end of either year 1 or year 2.
Pearson Achievement Solutions (Garet et al., 2010, 2011). Each professional development segment of Pearson Achievement Solutions (also referred to as Lesson Lab) focuses on a single problem or task. Each task is designed to elicit multiple approaches, which are intended to fuel extended discussions on the core ideas, common student approaches, and potential misconceptions associated with the task. The tasks were open-ended, and facilitators used their expertise to structure the discussions and determine whether to extend a professional development segment to address teacher responses. Extensive time was devoted to lesson planning, which was supplemented by videos. As in the America’s Choice study, this approach was adapted to conform to the requirements of a national research study on rational numbers for grade 7 students. The adaptations included retooling the scope and sequence of the professional development approach. On a test of student knowledge of rational numbers, no statistically significant effect was observed at the end of either year 1 or year 2.
Implications of the findings
Until more causal evidence becomes available, schools and districts must supplement the limited evidence of effectiveness with their best judgment. Schools and districts should be encouraged to rigorously evaluate professional development approaches themselves and, when possible, to report the findings publicly to build up the knowledge base on the topic.
4
Appendix A. The search, screening, and review process
The research question that guided the systematic review of evidence outlined below was: What does the causal research say are effective math professional development interventions for K–12 teachers aimed at improving student achievement?
Study eligibility criteria
To be included in this review, studies had to meet four relevancy criteria:• Topic. Each study included an intervention, program, or product focused on providing math profes-
sional development to teachers. The professional development had to focus on improving teacher content knowledge and instruction in math in order to improve student learning in math.
• Time. Math professional development studies had to be published between January 2006 and July 2012 or be identified in Yoon et al. (2007) as having met standards similar to earlier What Works Clearinghouse (WWC) evidence standards.6
• Sample. The sample had to include math teachers and their students in grades K–12 in the United States.
• Study design. Only studies that used a randomized controlled trial or a quasi-experimental design with a comparison group were included.
Reviewing studies using What Works Clearinghouse evidence standards
Studies were reviewed using the WWC Procedures and Standards Handbook (version 2.1) for group design studies (U.S. Department of Education, 2011) . Although each study that met the screening criteria was reviewed by a WWC-certified reviewer, this report is not a WWC review but was modeled after the WWC approach to reviewing causal evidence. Studies that the first reviewer determined to have met standards were reviewed independently by a second reviewer. A senior reviewer double-checked each completed review to ensure its accuracy and to reconcile any differences between the reviewers. The summary of pro-grams described in this report includes only professional development approaches with studies that review-ers determined met WWC evidence standards (version 2.1), with or without reservations.
Screening the research studies
Initially, 910 research studies were located through a comprehensive literature search (figure A1). Studies were then screened using a three-phase process:
• 643 studies met Phase I screening, which means that they:• Included an intervention related to math.• Were published between January 2006 and July 2012 or were identified in Yoon et al. (2007) as
having met standards similar to earlier WWC evidence standards.• Examined students in grades K–12.• Were conducted in the United States.
• 47 studies met Phase II screening, which means that they:• Included an intervention, program, or product focused primarily on providing math profession-
al development to teachers.• Used a relevant design (randomized controlled trial or quasi-experimental design).
• 32 studies met Phase III screening, which means the nature of the math professional development approach was relevant to this brief. (15 studies from Phase II were excluded because the profes-sional development approach was either generic professional development that did not focus on math—for example, school reform focused on strategic plans—or professional development that
5
was associated with implementing a specific curriculum.) These 32 studies were reviewed using WWC evidence standards (version 2.1).
• 5 studies met WWC evidence standards (version 2.1), with or without reservations. One two-year study was discussed in two reports, each focused on one year of the project (Garet et al., 2010, 2011). This was treated as one study, but two professional development approaches were involved. Impacts were reported separately for each approach. One professional development approach (Cog-nitively Guided Instruction) was included in two studies.
• 2 studies (representing two professional development approaches) met WWC evidence stan-dards (version 2.1), with or without reservations, and showed positive impacts on student math proficiency.
Figure A1. Screening funnel
910 studies found in literature screen
643 met Phase I screening
47 met Phase II screening
32 met Phase III screening
5 met What Works Clearinghouse evidence standards, with or without reservations
2 showed positive effects on student math proficiency
Source: Authors.
6
Appendix B. Thirty-two final studies reviewed using What Works Clearinghouse standards
Of the 910 studies identified in the literature search, 643 examined interventions related to math in grades K–12 and were conducted in the United States. Of the 643 studies, the 32 studies listed in this appendix focused primarily on math professional development provided to teachers and used a research design for examining effectiveness. Five of those were determined to have met What Works Clearinghouse evidence standards (version 2.1) either with or without reservations; these are identified with an asterisk.
Balfanz, R., Mac Iver, D., & Byrnes, V. (2006). The implementation and impact of evidence-based math-ematics reforms in high-poverty middle schools: A multi-site, multi-year study. Journal for Research in Mathematics Education, 37(1), 33–64. http://eric.ed.gov/?id=EJ765472
Bell, C., Wilson, S., Higgins, T., & McCoach, D. (2010). Measuring the effects of professional development on teacher knowledge: The case of developing mathematical ideas. Journal for Research in Mathematics Education, 41(5), 479–512. http://eric.ed.gov/?id=EJ902109
Blank, R., Smithson, J., Porter, A., Nunnaley, D., & Osthoff, E. (2006). Improving instruction through school-wide professional development: Effects of the data-on-enacted-curriculum model. ERS Spectrum, 24(2), 9–23. http://eric.ed.gov/?id=EJ795681
Boston, M., & Smith, M. (2009). Transforming secondary mathematics teaching: Increasing the cognitive demands of instructional tasks used in teachers’ classrooms. Journal for Research in Mathematics Educa-tion, 40(2), 119–156. http://eric.ed.gov/?id=EJ833624
Broyles, M. (2009). The effect of teacher participation in the Gateway Institute of Algebra on student academic achievement. (Doctoral dissertation, Union University, 2009). ProQuest UMI No. 3323764.
Campbell, P., & Malkus, N. (2011). The impact of elementary mathematics coaches on student achieve-ment. Elementary School Journal, 111(3), 430–454. http://eric.ed.gov/?id=EJ963689
*Carpenter, T., Fennema, E., Peterson, P., Chiang, C-P, & Loef, M. (1989). Using knowledge of children’s mathematics thinking in classroom teaching: An experimental study. American Educational Research Journal, 26(4), 499–531. http://eric.ed.gov/?id=ED292683
Dominguez, P., Nicholls, C., & Storandt, B. (2006). Experimental methods and results in a study of PBS TeacherLine math courses. Syracuse, NY: Hezel Associates. http://eric.ed.gov/?id=ED510045
Figueroa, M. (2008). Examination of a new method to teach elementary students mathematics. (Doctoral dissertation, Northern Arizona University, 2008). ProQuest UMI No. 3318482.
*† Garet, M., Wayne, A., Stancavage, F., Taylor, J., Eaton, M., Walters, K., et al. (2011). Middle school mathematics professional development impact study: Findings after the second year of implementation (NCEE 2011–4024). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. http://eric.ed.gov/?id=ED519923
*† Garet, M., Wayne, A., Stancavage, F., Taylor, J., Walters, K., Song, M., et al. (2010). Middle school mathematics professional development impact study: Findings after the first year of implementation (NCEE
7
2010–4009). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. http://eric.ed.gov/?id=ED509306
Harvey-Buschel, P. (2009). A quantitative examination of factors that impact technology integration in urban public secondary mathematics classrooms. (Doctoral dissertation, Bowie State University, 2009). ProQuest UMI No. 3365812. http://eric.ed.gov/?id=ED512800
Heller, J. I., Curtis, D. A., Rabe-Hesketh, S., & Verboncoeur, C. (2007). The effects of “Math Pathways and Pitfalls” on students’ mathematics achievement: National Science Foundation final report. Wash-ington, DC: U.S. Department of Education, National Center for Education Statistics. http://eric.ed.gov/?id=ED498258
*Jacobs, V., Franke, M., Carpenter, T., Levi, L., & Battey, D. (2007). Professional development focused on children’s algebraic reasoning in elementary school. Journal for Research in Mathematics Education, 38(3), 258–288. http://eric.ed.gov/?id=EJ765495
Laumakis, P., & Herman, M. (2008). The effect of a calculator training workshop for high school teachers on their students’ performance on Florida state-wide assessments. International Journal for Technology in Mathematics Education, 15(3), 87–93. http://eric.ed.gov/?id=EJ837650
McBride Martin, A. (2008). The effects of professional development to create standards-based curriculum on student achievement in fourth and fifth grade mathematics classrooms. (Doctoral dissertation, Indiana State University, 2008). ProQuest UMI No. 3299648.
McIntosh, T. (2010). Math matters: A study of a professional development program for a group of K–8 teachers in the Alum Rock School District. (Doctoral dissertation, Fielding Graduate University, 2010). ProQuest UMI No. 3371266.
*Perry, R. R., & Lewis, C. C. (2011). Improving the mathematical content base of lesson study. Summary of results. Retrieved January 24, 2013, from http://www.lessonresearch.net/IESAbstract10.pdf
Phelan, J., Choi, K., Niemi, D., Vendlinski, T., Baker, E., & Herman, J. (2012). The effects of POWERSOURCE© assessments on middle-school students’ math performance. Assessment in Educa-tion: Principles, Policy & Practice, 19(2), 211–230. http://eric.ed.gov/?id=EJ962954
Phelan, J., Choi, K., Vendlinski, T., Baker, E., & Herman, J. (2011). Differential improvement in student understanding of mathematical principles following formative assessment intervention. Journal of Educational Research, 104(5), 330–339. http://eric.ed.gov/?id=EJ934894
Phelan, J., Vendlinski, T., Choi, K., Dai, Y., Herman, J., & Baker, E. L. (2011). The development and impact of POWERSOURCE©: Year 5 (CRESST Report No. 792). Los Angeles: University of California, Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing. http://eric.ed.gov/?id=ED520532
Ribeiro, J. (2009). How does a co-learner delivery model in professional development affect teachers’ self- efficacy in teaching mathematics and specialized mathematics knowledge for teaching? (Doctoral dis-sertation, Johnson & Wales University University, 2009). ProQuest UMI No. 3387318. http://eric.ed.gov/?id=ED528371
8
Ross, J., & Bruce, C. (2007). Professional development effects on teacher efficacy: Results of a randomized field trial. The Journal of Educational Research, 101(1), 50–60. http://eric.ed.gov/?id=EJ776274
Sample McMeeking, L., Cobb, R. B., & Basile, C. (2010). Evaluating long-term complex professional development: Using a variation of the cohort control design. Evaluation & Research in Education, 23(4), 273–285. http://eric.ed.gov/?id=EJ903678
*Sample McMeeking, L., Orsi, R., & Cobb, R. B. (2012). Effects of a teacher professional development program on the mathematics achievement of middle school students. Journal for Research in Mathe-matics Education, 43(2), 159–181. http://eric.ed.gov/?id=EJ978876
Santagata, R., Kersting, N., Givvin, K., & Stigler, J. (2011). Problem implementation as a lever for change: An experimental study of the effects of a professional development program on students’ mathematics learning. Journal of Research on Educational Effectiveness, 4(1), 1–24. http://eric.ed.gov/?id=EJ911538
Saxe, G., Gearhart, M., & Nasir, N. (2001). Enhancing students’ understanding of mathematics: A study of three contrasting approaches to professional support. Journal of Mathematics Teacher Education, 4(1), 55–79. http://eric.ed.gov/?id=EJ622169
Smithson, J., & Blank, R. (2006). Indicators of quality of teacher professional development and instructional change using data from surveys of enacted curriculum: Findings from NSF MSP-RETA project. Washing-ton, DC: Council of Chief State School Officers.
Stone, J., Alfeld, C., & Pearson, D. (2008). Rigor and relevance: Enhancing high school students’ math skills through career and technical education. American Educational Research Journal, 45(3), 767–795. http://eric.ed.gov/?id=EJ807311
Stone, J., Alfeld, C., Pearson, D., Lewis, M., & Jensen, S. (2006). Building academic skills in context: Testing the value of enhanced math learning in CTE. Columbus, OH: National Dissemination Center for Career and Technical Education, Ohio State University. http://eric.ed.gov/?id=ED493604
Topping, K., Miller, D., Murray, P., & Conlin, N. (2011). Implementation integrity in peer tutoring of mathematics. Educational Psychology, 31(5), 575–593. http://eric.ed.gov/?id=EJ932919
Vendlinski, T. P., & Phelan, J. (2011). Using key conceptual ideas to improve teacher use of formative assess-ment data (CRESST Report No. 794). Los Angeles: University of California, Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing. http://eric.ed.gov/?id=ED520526
Williams, L. (2010). Building connections between sociomathematical norms and cognitive demand to improve the quality of whole class mathematics conversations. (Doctoral dissertation, University of Wisconsin- Milwaukee, 2010). ProQuest UMI No. 3437973. http://eric.ed.gov/?id=ED523425
† Garet et al. (2010) and Garet et al. (2011) are together considered one study because the same two-year study appears in both reports. Each report focuses on one year of the project. However, because two different professional development approaches were involved in this study, the impacts for each approach are reported separately in this report.
9
Appendix C. Research basis for the five studies meeting What Works Clearinghouse evidence standards
Tabl
e C
1.
Inte
nsiv
e m
ath
cont
ent
cour
ses
acco
mpa
nied
by
follo
w-u
p w
orks
hops
: po
tent
ially
pos
itiv
e ef
fect
s (p
rofe
ssio
nal
deve
lopm
ent
appr
oach
mor
e ef
fect
ive
for
teac
hers
enr
olle
d in
tw
o m
ath
cour
ses
than
for
tho
se e
nrol
led
in ju
st o
ne)
Sta
tist
ical
ly
sign
ifica
nt a
t p
< .0
5?A
ctua
l im
plem
enta
tion
Nat
ure
of t
he
coun
terf
actu
alS
tude
nt o
utco
me
mea
sure
sEff
ect
size
(H
edge
s’s g)
aFu
ll ci
tati
onR
esea
rch
desi
gnA
naly
sis
sam
ple
Sam
ple
McM
eeki
ng,
L.,
Ors
i, R
., &
Cob
b,
R.
B.
(201
2).
Effe
cts
of a
tea
cher
pr
ofes
sion
al
deve
lopm
ent
prog
ram
on
the
mat
hem
atic
s ac
hiev
emen
t of
m
iddl
e sc
hool
st
uden
ts.
Jour
nal
for
Res
earc
h in
M
athe
mat
ics
Educ
atio
n, 4
3(2
),
15
9–1
81
.
Qua
si-e
xper
imen
tal
desi
gnb
23
tea
cher
s an
d 2,3
19
stu
dent
s in
gr
ades
5–8
fro
m
four
dis
tric
ts
Part
icip
ants
too
k ei
ther
one
or
two
univ
ersi
ty c
ours
es
Two
to t
hree
wee
ks
per
cour
se in
the
su
mm
er; fo
ur
half-
day
follo
w-
up s
essi
ons
on
Sat
urda
ys d
urin
g th
e sc
hool
yea
r
No
part
icip
atio
n in
th
e co
nten
t-ba
sed
mat
h co
urse
s or
fo
llow
-up
sess
ions
Col
orad
o S
tude
nt
Asse
ssm
ent
Prog
ram
pro
ficie
ncy
rate
s (f
or t
each
ers
who
com
plet
ed o
nly
one
sum
mer
mat
h co
urse
)
0.0
9N
o
Col
orad
o S
tude
nt
Asse
ssm
ent
Prog
ram
pro
ficie
ncy
rate
s (f
or t
each
ers
who
com
plet
ed
two
sum
mer
mat
h co
urse
s)
0.2
0Ye
s
a. B
olde
d va
lues
are
sub
stan
tive
ly im
port
ant;
eff
ect
size
s of
0.2
5 o
r gr
eate
r ar
e co
nsid
ered
sub
stan
tive
ly im
port
ant,
reg
ardl
ess
of s
tatis
tica
l sig
nific
ance
, ac
cord
ing
to W
hat
Wor
ks C
lear
ingh
ouse
(W
WC
) ev
iden
ce s
tand
ards
(ve
rsio
n 2
.1).
b. M
eets
WW
C e
vide
nce
stan
dard
s (v
ersi
on 2
.1) w
ith
rese
rvat
ions
.
Sou
rce:
Aut
hors
’ lit
erat
ure
revi
ew (s
ee a
ppen
dix
A).
10
Tabl
e C
2.
Less
on s
tudy
foc
used
on
linea
r (m
easu
rem
ent)
mod
el o
f fr
acti
ons:
pot
enti
ally
pos
itiv
e ef
fect
s (o
ne s
tudy
wit
h si
gnifi
cant
impa
ct o
n st
uden
t ac
hiev
emen
t)
Sta
tist
ical
ly
sign
ifica
nt a
t p
< .0
5?A
ctua
l im
plem
enta
tion
Nat
ure
of t
he
coun
terf
actu
alS
tude
nt o
utco
me
mea
sure
sEff
ect
size
(H
edge
s’s g)
aFu
ll ci
tati
onR
esea
rch
desi
gnA
naly
sis
sam
ple
Perr
y, R
. R
., &
Le
wis
, C
. C
. (2
01
1).
Impr
ovin
g th
e m
athe
mat
ical
con
tent
ba
se o
f le
sson
stu
dy
sum
mar
y of
res
ults
. R
etriev
ed J
anua
ry 2
4,
201
3,
from
htt
p://
ww
w.le
sson
rese
arch
.ne
t/IE
SAbs
trac
t10.
pdfb
Ran
dom
ized
co
ntro
lled
tria
l21
3 t
each
ers
and
1,0
59
stu
dent
s in
gr
ades
2–5
fro
m
27
dis
tric
ts
Less
on s
tudy
on
frac
tion
s (w
ith
frac
tion
s re
sour
ce
kit
and
less
on s
tudy
to
ols)
Actu
al d
urat
ion
(ran
ging
fro
m 7
to
42 h
ours
) an
d sc
hedu
ling
varied
by
site
Rec
omm
ende
d du
ration
: 1
2–1
4
mee
tings
ove
r fiv
e m
onth
s of
the
sch
ool
year
Cou
nter
fact
ual 1
:
Less
on s
tudy
on
loca
lly c
hose
n to
pic
(with
less
on s
tudy
too
ls).
No
frac
tion
s to
olki
t. T
each
ers
wer
e to
ld t
o ch
oose
a t
opic
ot
her
than
fra
ctio
ns f
or t
heir
less
on s
tudy
ses
sion
s, b
ut
teac
hers
stil
l tau
ght
frac
tion
s to
the
ir st
uden
ts.
Frac
tion
s kn
owle
dge—
a te
st d
evel
oped
by
rese
arch
ers
0.8
4Ye
sc
Cou
nter
fact
ual 2
: B
usin
ess
as u
sual
(lo
cally
cho
sen
prof
essi
onal
lear
ning
oth
er
than
less
on s
tudy
on
loca
lly
chos
en t
opic
). T
each
ers
wer
e to
ld t
o ch
oose
a t
opic
ot
her
than
fra
ctio
ns f
or t
heir
prof
essi
onal
lear
ning
wor
k, b
ut
teac
hers
stil
l tau
ght
frac
tion
s to
the
ir st
uden
ts.
Teac
hers
att
ende
d a
wid
e ra
nge
of a
ctiv
itie
s, in
clud
ing
self-
stud
y an
d ac
tivi
ties
off
ered
lo
cally
or
at c
onfe
renc
es.
Frac
tion
s kn
owle
dge—
a te
st d
evel
oped
by
rese
arch
ers
0.7
1Ye
sc
a. B
olde
d va
lues
are
sub
stan
tive
ly im
port
ant;
eff
ect
size
s of
0.2
5 o
r gr
eate
r ar
e co
nsid
ered
sub
stan
tive
ly im
port
ant,
reg
ardl
ess
of s
tatis
tica
l sig
nific
ance
, ac
cord
ing
to W
hat
Wor
ks C
lear
ing-
hous
e (W
WC)
evid
ence
sta
ndar
ds (ve
rsio
n 2
.1).
b. T
his
docu
men
t is
a p
ublic
ly a
vaila
ble
sum
mar
y of
the
stu
dy; ho
wev
er, m
ost
of t
he d
ata
wer
e ob
tain
ed t
hrou
gh a
n au
thor
que
ry.
The
auth
ors
expe
ct t
o pu
blis
h th
eir
rese
arch
in a
pee
r- re
view
ed jo
urna
l in
the
near
fut
ure.
c. In
sub
grou
p an
alys
es, re
sear
cher
s re
port
ed t
hat
resu
lts
wer
e st
atis
tica
lly s
igni
fican
t in
gra
des
2, 3, an
d 5 b
ut n
ot in
gra
de 4
.
Sou
rce:
Aut
hors
’ lit
erat
ure
revi
ew (s
ee a
ppen
dix
A).
11
Tabl
e C
3.
Cog
niti
vely
Gui
ded
Inst
ruct
ion:
lim
ited
eff
ects
(on
e st
udy
had
no s
igni
fican
t im
pact
on
any
aspe
ct o
f m
ath
profi
cien
cy;
the
othe
r st
udy
had
a si
gnifi
cant
impa
ct o
n a
brie
f m
easu
re o
f co
ncep
ts o
f eq
ualit
y bu
t no
impa
ct f
or a
ny o
ther
asp
ects
of
mat
h)
Sta
tist
ical
ly
sign
ifica
nt a
t p
< .0
5?A
ctua
l im
plem
enta
tion
Nat
ure
of t
he
coun
terf
actu
alFu
ll ci
tati
onR
esea
rch
desi
gnA
naly
sis
sam
ple
Stu
dent
out
com
e m
easu
res
Car
pent
er,
T.,
Fenn
ema,
E.,
Pete
rson
, P.
, C
hian
g, C
. P.
, &
Loe
f, M
. (1
98
9).
Usi
ng k
now
ledg
e of
ch
ildre
n’s
mat
hem
atic
s th
inki
ng in
cla
ssro
om
teac
hing
: An
expe
rim
enta
l stu
dy.
Am
eric
an E
duca
tiona
l R
esea
rch
Jour
nal,
26
(4), 4
99
–531
.
Ran
dom
ized
co
ntro
lled
tria
l4
0 t
each
ers
and
appr
oxim
atel
y 4
80
gra
de 1
st
uden
ts f
rom
24
sch
ools
Four
-wee
k su
mm
er
wor
ksho
p, f
our
days
pe
r w
eek,
five
hou
rs
per
day
Two
two-h
our
wor
ksho
ps
(one
in S
epte
mbe
r an
d on
e in
Feb
ruar
y) f
ocus
ed o
n so
lvin
g no
nrou
tine
prob
lem
s us
ing
variou
s he
uris
tics
. Th
e co
unte
rfac
tual
dif
fere
d fr
om
Cog
nitive
ly G
uide
d In
stru
ctio
n in
ter
ms
of f
ocus
, du
ration
, ex
tent
of co
vera
ge, an
d pu
rpos
e. A
ccor
ding
to
the
auth
ors,
“Th
e go
al w
as t
o pr
ovid
e co
ntro
l tea
cher
s w
ith
som
e se
nse
of p
artici
pation
in
the
pro
ject
and
giv
e th
em
som
e im
med
iate
rew
ard
for
thei
r pa
rtic
ipat
ion”
(p.
507
).
Iow
a Te
st o
f B
asic
Ski
lls L
evel
7 C
ompu
tation
No
Iow
a Te
st o
f B
asic
Ski
lls L
evel
7 P
robl
em S
olvi
ngN
o
Prob
lem
sol
ving
sub
test
: si
mpl
e ad
dition
/su
btra
ctio
n w
ord
prob
lem
No
Prob
lem
sol
ving
sub
test
: co
mpl
ex a
dditio
n/su
btra
ctio
n w
ord
prob
lem
No
Prob
lem
sol
ving
sub
test
: ad
vanc
ed w
ord
prob
lem
No
Prob
lem
sol
ving
inte
rvie
w (a
mea
sure
of
stud
ents
’ st
rate
gy u
se)
No
Num
ber
fact
s in
terv
iew
(a m
easu
re o
f st
uden
ts’
stra
tegy
use
)N
o
Jaco
bs,
V.,
Fran
ke,
M., C
arpe
nter
, T.
, Le
vi,
L.,
& B
atte
y, D
. (2
007
). P
rofe
ssio
nal
deve
lopm
ent
focu
sed
on c
hild
ren’
s al
gebr
aic
reas
onin
g in
ele
men
tary
sch
ool.
Jour
nal f
or R
esea
rch
in
Mat
hem
atic
s Ed
ucat
ion,
3
8(3
), 2
58
–28
8.
Ran
dom
ized
co
ntro
lled
tria
l1
80
tea
cher
s an
d 6
06
st
uden
ts in
gr
ades
1–5
fro
m
19
sch
ools
in
one
dist
rict
A 4.5
-hou
r S
atur
day
intr
oduc
tory
mee
ting
with
teac
hers
fro
m
acro
ss t
he d
istr
ict;
1
.5-h
our
afte
r-sc
hool
wor
ksho
ps
appr
oxim
atel
y on
ce a
mon
th e
ight
tim
es o
ver
the
year
; ad
dition
ally
, a
half-
day
per
wee
k of
on
site
coa
chin
g.
Bus
ines
s as
usu
al; te
ache
rs in
th
e co
ntro
l con
dition
rec
eive
d pr
ofes
sion
al d
evel
opm
ent
the
year
aft
er t
he s
tudy
.
Equa
lity
subt
est
(gra
de 1
)Ye
s
Equa
lity
subt
est
(gra
des
2/3
)Ye
s
Equa
lity
subt
est
(gra
des
4/5
)Ye
s
Com
puta
tion
sub
test
(gr
ade
1)N
o
Com
puta
tion
sub
test
(gr
ades
2/3
)aN
o
Com
puta
tion
sub
test
(gr
ades
4/5
)aN
o
Sol
ving
equ
atio
ns s
ubte
st (
grad
es 2
/3)a
No
Sol
ving
equ
atio
ns s
ubte
st (
grad
es 4
/5)a
No
Not
e: S
tude
nt-le
vel e
ffec
t si
zes
wer
e no
t ca
lcul
ated
bec
ause
the
aut
hors
rep
orte
d on
ly c
lass
room
-leve
l dat
a an
d w
ere
not
able
to
prov
ide
addi
tion
al d
ata
whe
n re
ques
ted.
a. A
utho
rs r
epor
ted
resu
lts
as n
onsi
gnifi
cant
but
did
not
rep
ort
effe
ct s
izes
or
rele
vant
dat
a fo
r ca
lcul
atin
g ef
fect
siz
es a
t th
e le
vel o
f as
sign
men
t (s
choo
l lev
el)
usin
g W
hat
Wor
ks C
lear
ing-
hous
e st
anda
rds
(ver
sion
2.1
). H
owev
er, be
caus
e th
e au
thor
s us
ed li
bera
l mea
ns o
f ca
lcul
atin
g st
atis
tica
l sig
nific
ance
, no
ne o
f th
e ef
fect
s w
ould
be
stat
istica
lly s
igni
fican
t us
ing
the
scho
ol a
s th
e un
it of
ana
lysi
s.
Sou
rce:
Aut
hors
’ lit
erat
ure
revi
ew (s
ee a
ppen
dix
A).
12
Tabl
e C
4.
Am
eric
a’s
Cho
ice:
no
disc
erni
ble
effe
cts
on s
tude
nts’
acq
uisi
tion
of m
ath
Sta
tist
ical
ly
sign
ifica
nt a
t p
< .0
5?N
atur
e of
the
co
unte
rfac
tual
Stu
dent
out
com
e m
easu
res
Eff
ect
size
(H
edge
s’s g)
Full
cita
tion
Res
earc
h de
sign
Ana
lysi
s sa
mpl
eA
ctua
l im
plem
enta
tion
Gar
et,
M., W
ayne
, A., S
tanc
avag
e,
F., Ta
ylor
, J.
, W
alte
rs, K
., S
ong,
M
., e
t al
. (2
010). M
iddl
e sc
hool
m
athe
mat
ics
prof
essi
onal
de
velo
pmen
t im
pact
stu
dy:
Find
ings
aft
er t
he fi
rst
year
of
impl
emen
tatio
n (N
CEE
2010
–4
00
9). W
ashi
ngto
n, D
C: U
.S.
Dep
artm
ent
of E
duca
tion
, In
stitut
e fo
r Ed
ucat
ion
Sci
ence
s, N
atio
nal
Cen
ter
for
Educ
atio
n Ev
alua
tion
an
d R
egio
nal A
ssis
tanc
e.
Gar
et,
M., W
ayne
, A., S
tanc
avag
e,
F.,
Tayl
or,
J.,
Eat
on,
M., W
alte
rs,
K., e
t al
. (2
01
1).
Mid
dle
scho
ol
mat
hem
atic
s pr
ofes
sion
al
deve
lopm
ent
impa
ct s
tudy
: Fi
ndin
gs a
fter
the
sec
ond
year
of
impl
emen
tatio
n (N
CEE
201
1–4
024)
. W
ashi
ngto
n, D
C: U
.S.
Dep
artm
ent
of E
duca
tion
, In
stitut
e fo
r Ed
ucat
ion
Sci
ence
s, N
atio
nal
Cen
ter
for
Educ
atio
n Ev
alua
tion
an
d R
egio
nal A
ssis
tanc
e.
Ran
dom
ized
co
ntro
lled
tria
lYe
ar 1
:9
9 t
each
ers
and
2,3
85 g
rade
7
stud
ents
fro
m
40 s
choo
ls in
12
dist
rict
s
Thre
e-d
ay s
umm
er in
stitut
e,
five
one-d
ay s
emin
ars
during
the
sch
ool y
ear,
and
10 d
ays
of in
tens
ive
in-s
choo
l coa
chin
g. T
he in
-sc
hool
coa
chin
g se
ssio
ns
occu
rred
imm
edia
tely
aft
er
each
sem
inar
and
wer
e sc
hedu
led
(to
the
exte
nt
poss
ible
) to
alig
n w
ith
period
s in
whi
ch r
atio
nal
num
ber
topi
cs w
ere
bein
g co
vere
d in
the
dis
tric
ts’
grad
e 7 m
ath
curr
icul
um.
The
tota
l int
ende
d do
sage
of pr
ofes
sion
al
deve
lopm
ent
(incl
udin
g in
stitut
es, se
min
ars,
an
d co
achi
ng) in
rat
iona
l nu
mbe
r to
pics
was
68
hour
s.
Bus
ines
s as
usu
al;
grad
e 7 t
each
ers
in c
ontr
ol s
choo
ls
rece
ived
pro
fess
iona
l de
velo
pmen
t of
fere
d by
the
dis
tric
t or
st
ate.
Year
1:
Nor
thw
est
Eval
uation
As
soci
atio
n to
tal
scor
e—
a te
st o
f st
uden
t kn
owle
dge
of r
atio
nal n
umbe
r to
pics
0.0
2N
o
Year
2:
Nor
thw
est
Eval
uation
As
soci
atio
n to
tal
scor
e—
a te
st o
f st
uden
t kn
owle
dge
of r
atio
nal n
umbe
r to
pics
0.0
3N
oYe
ar 2
:1
55 t
each
ers
and
3,9
71 g
rade
7 s
tude
nts
from
4
0 s
choo
ls in
6
dist
rict
s
Sou
rce:
Aut
hors
’ lit
erat
ure
revi
ew (s
ee a
ppen
dix
A).
13
Tabl
e C
5.
Pea
rson
Ach
ieve
men
t S
olut
ions
: no
dis
cern
ible
eff
ects
on
stud
ents
’ ac
quis
itio
n of
mat
h
Sta
tist
ical
ly
sign
ifica
nt a
t p
< .0
5?N
atur
e of
the
co
unte
rfac
tual
Stu
dent
out
com
e m
easu
res
Eff
ect
size
(H
edge
s’s g)
Full
cita
tion
Res
earc
h de
sign
Ana
lysi
s sa
mpl
eA
ctua
l im
plem
enta
tion
Gar
et,
M., W
ayne
, A., S
tanc
avag
e,
F., Ta
ylor
, J.
, W
alte
rs, K
., S
ong,
M
., e
t al
. (2
010). M
iddl
e sc
hool
m
athe
mat
ics
prof
essi
onal
de
velo
pmen
t im
pact
stu
dy:
Find
ings
aft
er t
he fi
rst
year
of
impl
emen
tatio
n (N
CEE
2010
–4
00
9). W
ashi
ngto
n, D
C: U
.S.
Dep
artm
ent
of E
duca
tion
, In
stitut
e of
Edu
cation
Sci
ence
s, N
atio
nal
Cen
ter
for
Educ
atio
n Ev
alua
tion
an
d R
egio
nal A
ssis
tanc
e.
Gar
et,
M., W
ayne
, A., S
tanc
avag
e,
F.,
Tayl
or,
J.,
Eat
on,
M., W
alte
rs,
K., e
t al
. (2
01
1).
Mid
dle
scho
ol
mat
hem
atic
s pr
ofes
sion
al
deve
lopm
ent
impa
ct s
tudy
: Fi
ndin
gs a
fter
the
sec
ond
year
of
impl
emen
tatio
n (N
CEE
201
1–4
024)
. W
ashi
ngto
n, D
C: U
.S.
Dep
artm
ent
of E
duca
tion
, In
stitut
e of
Edu
cation
Sci
ence
s, N
atio
nal
Cen
ter
for
Educ
atio
n Ev
alua
tion
an
d R
egio
nal A
ssis
tanc
e.
Ran
dom
ized
co
ntro
lled
tria
lYe
ar 1
:8
9 t
each
ers
and
1,8
26 g
rade
7
stud
ents
fro
m
37 s
choo
ls in
12
dist
rict
s
Thre
e-d
ay s
umm
er in
stitut
e,
five
one-d
ay s
emin
ars
during
the
sch
ool y
ear,
and
10 d
ays
of in
tens
ive
in-s
choo
l coa
chin
g. T
he in
-sc
hool
coa
chin
g se
ssio
ns
occu
rred
imm
edia
tely
af
ter
each
sem
inar
and
w
ere
sche
dule
d (t
o th
e ex
tent
pos
sibl
e) t
o al
ign
with
period
s in
whi
ch
ration
al n
umbe
r to
pics
w
ere
bein
g co
vere
d in
the
di
strict
s’ g
rade
7 m
ath
curr
icul
a. T
he t
otal
inte
nded
do
sage
of pr
ofes
sion
al
deve
lopm
ent
(incl
udin
g in
stitut
es, se
min
ars,
an
d co
achi
ng) in
rat
iona
l nu
mbe
r to
pics
was
68
hour
s.
Bus
ines
s as
usu
al;
grad
e 7 t
each
ers
in c
ontr
ol s
choo
ls
rece
ived
pro
fess
iona
l de
velo
pmen
t of
fere
d by
the
dis
tric
t or
st
ate.
Year
1:
Nor
thw
est
Eval
uation
As
soci
atio
n to
tal
scor
e—
a te
st o
f st
uden
t kn
owle
dge
of r
atio
nal n
umbe
r to
pics
0.0
5N
o
Year
2:
Nor
thw
est
Eval
uation
As
soci
atio
n to
tal
scor
e—
a te
st o
f st
uden
t kn
owle
dge
of r
atio
nal n
umbe
r to
pics
0.0
3N
oYe
ar 2
:1
23 t
each
ers
and
2,6
89 g
rade
7 s
tude
nts
from
3
6 s
choo
ls in
6
dist
rict
s
Sou
rce:
Aut
hors
’ lit
erat
ure
revi
ew (s
ee a
ppen
dix
A).
14
Notes
1. Effectiveness studies determine whether there is a causal relationship between the approach and the outcomes; in this case, effectiveness studies examine whether mathematics professional development approaches cause improvements in student mathematics proficiency.
2. These studies focused only on students in grades 1–8.3. There are many professional development programs and approaches beyond the five presented here, and
not all five have positive findings. These five are highlighted because the research evidence for these approaches is of the highest quality in terms of determining the professional development approach’s effect. For this WWC-style review, the review team determined that these five studies met WWC evidence standards (version 2.1) with or without reservations. Effect on student achievement is just one piece of evidence to consider when deciding whether to adopt an approach.
4. Earlier case study research suggests that focusing on concrete materials that are also used in the class-room is a hallmark of effective professional development (Garet, Porter, Desimone, Birman, & Yoon, 2001; Gersten, Chard, & Baker, 2000; Huberman & Miles, 1984).
5. This finding differs from those reported in Yoon et al. (2007), which evaluated studies of professional development from 1986 to 2006 using standards similar to but less extensive in some dimensions of research quality than an earlier version of the WWC standards (personal communication with Kwang Suk Yoon).
6. Yoon et al. (2007) evaluated studies of professional development from 1986 to 2006, using standards similar to, but less extensive in some dimensions of research quality than, an earlier version of the WWC standards (personal communication with Kwang Suk Yoon).
15
References
Carpenter, T., Fennema, E., Peterson, P., Chiang, C-P, & Loef, M. (1989). Using knowledge of children’s mathematics thinking in classroom teaching: An experimental study. American Educational Research Journal, 26(4), 499–531. http://eric.ed.gov/?id=EJ406340
Garet, M., Porter, A., Desimone, L., Birman, B., & Yoon, K. S. (2001). What makes professional devel-opment effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915–945. http://eric.ed.gov/?id=EJ648260
Garet, M., Wayne, A., Stancavage, F., Taylor, J., Eaton, M., Walters, K., et al. (2011). Middle school math-ematics professional development impact study: Findings after the second year of implementation (NCEE 2011–4024). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. http://eric.ed.gov/?id=ED519922
Garet, M., Wayne, A., Stancavage, F., Taylor, J., Walters, K., Song, M., et al. (2010). Middle school math-ematics professional development impact study: Findings after the first year of implementation (NCEE 2010–4009). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. http://eric.ed.gov/?id=ED509306
Gersten, R., Chard, D., & Baker, S. (2000). Factors that enhance sustained use of research-based instruc-tional practices: A historical perspective on relevant research. Journal of Learning Disabilities, 33, 445–457. http://eric.ed.gov/?id=EJ614660
Huberman, A. M., & Miles, M. (1984). Innovation up close. New York: Plenum.
Jacobs, V., Franke, M., Carpenter, T., Levi, L., & Battey, D. (2007). Professional development focused on children’s algebraic reasoning in elementary school. Journal for Research in Mathematics Education, 38(3), 258–288. http://eric.ed.gov/?id=EJ765495
Perry, R. R., & Lewis, C. C. (2011). Improving the mathematical content base of lesson study summary of results. Retrieved January 24, 2013, from http://www.lessonresearch.net/IESAbstract10.pdf
Sample McMeeking, L., Orsi, R., & Cobb, R. B. (2012). Effects of a teacher professional development program on the mathematics achievement of middle school students. Journal for Research in Mathe-matics Education, 43(2), 159–181. http://eric.ed.gov/?id=EJ978876
U.S. Department of Education. (2011). What Works Clearinghouse: Procedures and standards handbook (version 2.1). Washington, DC: Institute of Education Sciences. Retrieved June 26, 2013, from http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf
Yoon, K. S., Duncan, T., Lee, S. W. Y., Scarloss, B., & Shapley, K. (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues & Answers Report, REL 2007–033). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. http://eric.ed.gov/?id=ED498548
REL 2014–010
The National Center for Education Evaluation and Regional Assistance (NCEE) conducts unbi-ased large-scale evaluations of education programs and practices supported by federal funds; provides research-based technical assistance to educators and policymakers; and supports the synthesis and the widespread dissemination of the results of research and evaluation through-out the United States.
January 2014
This report was prepared for the Institute of Education Sciences (IES) under Contract ED-IES-C-12-0011 by Regional Educational Laboratory Southeast administered by Florida State University. The content of the publication does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.
This REL report is in the public domain. While permission to reprint this publication is not neces-sary, it should be cited as:
Gersten, R., Taylor, M. J., Keys, T. D., Rolfhus, E., & Newman-Gonchar, R. (2014). Summary of research on the effectiveness of math professional development approaches. (REL 2014–010). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast. Retrieved from http://ies.ed.gov/ncee/edlabs.
This report is available on the Regional Educational Laboratory website at http://ies.ed.gov/ncee/edlabs.
Related Documents
![The effectiveness of continuous quality improvement for ......the CQI intervention that have previously been identi-fied as important [20, 21, 38], including its scope; inclu-sion](https://static.cupdf.com/doc/110x72/60a8e0a1bc7951684f49a39f/the-effectiveness-of-continuous-quality-improvement-for-the-cqi-intervention.jpg)
