Hopkins PSMG Oct 2005 1 An Incomplete Model for Trial Designs for Moving to Scale: The Whole Day Program C Hendricks Brown Sheppard Kellam Jeanne Poduska Amy Windham John Reid Carla Ford Natalie Keegan Supported by R01 DA15409-02 PI: Kellam Ro1 MH40859-17 PI: Brown Baltimore City Public School System
29
Embed
Hopkins PSMG Oct 20051 An Incomplete Model for Trial Designs for Moving to Scale: The Whole Day Program C Hendricks Brown Sheppard Kellam Jeanne Poduska.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Hopkins PSMG Oct 2005 1
An Incomplete Model for Trial Designs for Moving to Scale:
The Whole Day ProgramC Hendricks BrownSheppard KellamJeanne PoduskaAmy Windham
John ReidCarla Ford
Natalie Keegan
Supported by R01 DA15409-02 PI: KellamRo1 MH40859-17 PI: Brown
Baltimore City Public School System
Hopkins PSMG Oct 2005 2
Premise
The Whole Day Program is where the school would want to be in 5 years, systemwide.
What specific steps should be examined along the way?
Hopkins PSMG Oct 2005 3
Primary Question
Effective?Who benefits, for how long?Who doesn’t benefit, who is harmed?
Efficacy Trial12 Schools24 Classrooms600 Students3 Time Periods
Hopkins PSMG Oct 2005 4
Differential impact on spring of first grade reading achievement by
baseline reading (fall of first grade): control against two interventions
Control
Classroom / Parent Interventions
Brown: Statistical Power
Hopkins PSMG Oct 2005 5
Presuming Effectiveness
SustainabililtyDo WD teachers continue to sustain the same
effects on children that they have in following years?
12 WD Teachers vs Themselves & 12 Wait-Listed Controls
Scalability Can program be successfully implemented
outside the original 12 schools throughout the school system?
Hopkins PSMG Oct 2005 6
Scale to What?
• From 2 To all 4 of the School Areas
• From 12 To 20 To 135 Schools
• From 24 To 40 To 400 First-Grade Classrooms
Hopkins PSMG Oct 2005 7
An Evidence-Based Decision Tree
Efficacious especially w/ High-Risk Youth?
Sustainable?
Feasible to Scale?
Y
Y
Go to Scale!
Y
N
N
N
Hopkins PSMG Oct 2005 8
The Real Decision Tree for BCPSS
Something Different
Hopkins PSMG Oct 2005 9
Person, Place, and Time
• Child• Classroom / Teacher (1st Grade)• School• Area (Region of Baltimore)
• Time – Scale(s): 10 Secs, Min, Quarter, Year• Child : Track from 1st through 3rd Grade• Teacher: Track over 3 years (cohorts)
Hopkins PSMG Oct 2005 10
Randomization
• 20 Schools in 2 Areas randomly assigned to One of 12 schools where trial begins
One of 8 schools that are “wait-listed”
• Children randomly assigned to classes within school
• Teachers/Classes randomly assigned to WD or wait-list
Hopkins PSMG Oct 2005 11
Baltimore Whole Day TrialAssignment of Classrooms/Teachers in First YearHalf the Teachers are Wait-Listed for WD to 3rd
Year
Replicate Same Design
w/ SC and and WD
BOTH in 12 Different Schools
Brown: Statistical Power
Hopkins PSMG Oct 2005 12
Effectiveness Evaluation
Compare 3 years of Outcomes for Children in the 12 WD classes to those in 12 Standard classes -- 1st Cohort
Examine Level of Program Implementation in the 12 WD and 12 Standard Classrooms in 1st Cohort
Hopkins PSMG Oct 2005 13
Classroom Observations by Independent Observers
3 times per year WD and SC classroomsAM-Reading InstructionPM-Other subjects
Teacher PracticesTimed ObservationsChecklist Global Ratings of Quality
Student BehaviorOn-Task/Off-taskRatings of Disruptive, Aggressive, and Shy Behavior
Hopkins PSMG Oct 2005 14
Classroom Observation Data Collection FormTeacher Practices and Student Behavior
Actual Form
Hopkins PSMG Oct 2005 15
Domains of Reading Instruction
• Word Work – phonemic awareness, alphabetic instruction, letter recognition, work work
Compare Teacher Implementation (and Child Outcomes) for 12 WD Teachers in 1st through 3rdYear for First Grade Classes
Hopkins PSMG Oct 2005 24
Sustainability Hypotheses and N’s & Years
• Teacher practices over time will be related to the quality of WD integration at the level of the School Building Team.
12 20 135
2• Teacher practices over time will be related to
the quality of WD integration at the level of the School Area over time.
2 4
Hopkins PSMG Oct 2005 25
Scalability Design
Change in fidelity (Teacher practices) and impact (Child Outcomes) in succeeding years after initial training
12 24 135
Hopkins PSMG Oct 2005 26
Combined RCT and Implementation Trial
Trial After a Trial Design
Full Implementation Control Waiting
Effectiveness Trial
Yr1 Yr2
Yr3
Moving to Scale
Full Implementation Control
Varying Levels of Implementation Due to Sustainability and Scalability
Hopkins PSMG Oct 2005 27
How to Go From 24 to 400 Classes?
A Dynamic Wait-Listed type of design
Randomly Assign WHEN schools and classes obtain training, coaching, and full implementationContinue to assess intervention fidelityPaired-down measure of Teacher Practices