Scaling Up Innovations Dean L. Fixsen, Ph.D. & Karen A. Blase, Ph.D. University of North Carolina - Chapel Hill Rob Horner, Ph.D. University of Oregon George Sugai, Ph.D. University of Connecticut OR Superintendents’ Forum April 2010 Scaling Up Effectiveness
58
Embed
Scaling Up Innovations Dean L. Fixsen, Ph.D. & Karen A. Blase, Ph.D. University of North Carolina - Chapel Hill Rob Horner, Ph.D. University of Oregon.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Scaling Up Innovations
Dean L. Fixsen, Ph.D. & Karen A. Blase, Ph.D.
University of North Carolina - Chapel Hill
Rob Horner, Ph.D.
University of Oregon
George Sugai, Ph.D.
University of Connecticut
OR Superintendents’ Forum April 2010
Scaling Up Effectiveness
OR Public Schools
Students: 565,000
Schools: 1,800
School Districts: 196
Counties: 36
Budget: $8.5 Billion
“Bottom 20%”
113,000
360
39
36
Challenges
Follow Through Programs
Figure 1: This figure shows the average effects of nine Follow Through models on measures of basic skills (word knowledge, spelling, language, and math computation), cognitive-conceptual skills (reading comprehension, math concepts, and math problem solving) and self-concept. This figure is
adapted from Engelmann, S. and Carnine, D. (1982), Theory of Instruction: Principles and applications. New York: Irvington Press.
Hattie (2009) recently reported a meta-analysis of 816 meta-analyses
52,649 research studies in education involving over 83 million students, teachers, staff, parents, and others.
"It is what teachers get the students to do in the class that emerged as the strongest component of the accomplished teachers' repertoire."
Hattie (2009)
Achieving Student Benefits
Increasing opportunities to respond and the amount/ accuracy of feedback is an important correlate of student achievement
The feedback to the teachers about what students can and cannot do is more powerful than feedback to the student
This requires a change in the conception of what it means to be a teacher – not a solo performer
Hattie (2009)
Achieving Student Benefits
We now know a lot about WHAT to do to educate students
We can improve education for students – on purpose!
Achieving Student Benefits
Science “to” Service
SCIENCE SERVICEGAPIMPLEMENTATION
Challenges
Science to Service Gap
What is known is not what is adopted to help students
Implementation Gap
What is adopted is not used fully and effectively in practice
Achieving Student Benefits
Good Intentions
Actual SupportsYears 1-3
Outcomes
Every Teacher Trained
Fewer than 50% of the teachers received some training
Fewer than 10% of the schools used the CSR as intended
Must be “worth the effort” to scale up (e.g. EBISS)
Eventually want to see educationally and socially significant changes in student outcomes across the State
Know HOW
Effective NOT Effective
Effective
NOT Effective
IMPLEMENTATION
INT
ER
VE
NT
ION Student Benefits
Highly variable, often ineffective, sometimes harmful to students,
families, and adults
(Institute of Medicine, 2000; 2001; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999)
Poor Outcomes
Implementation
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
OUTCOMES(% of Participants who Demonstrate Knowledge, Demonstrate
new Skills in a Training Setting, and Use new Skills in the Classroom)
TRAININGCOMPONENTS
KnowledgeSkill
DemonstrationUse in the Classroom
Theory and Discussion
10%
5% 0%
..+Demonstration in Training
30%20%
0%
…+ Practice & Feedback in Training
60% 60% 5%
…+ Coaching in Classroom
95% 95% 95%
Joyce and Showers, 2002
Staff Coaching
Support Implementation
Students cannot benefit from education practices they do not experience
Support implementation practices within schools and districts
Capacity Building Scaling up = at least 60% of
the students who could benefit from an innovation have access to that innovation
Achieve significant benefits to students and society
Letting it happen Recipients are accountable
Helping it happen Recipients are accountable
Making it happen Purposeful use of implementation
practices and science
Implementation teams are accountable
Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004
Capacity Building
Letting it happen
Recipients are accountable
Helping it happen
Recipients are accountable
Making it happen
Implementation Teams are accountable: THEY DO THE WORK (Heart of Scaling)
Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004
Capacity Building
Implementation Team
Minimum of three people (four or five preferred) to promote effective, efficient, and sustainable implementation, organization change, and system transformation work
Tolerate turnover; teams are sustainable even when the players come and go
Implementation Team
A group that knows the innovations very well (formal and craft knowledge)
A group that knows implementation very well (formal and craft knowledge)
A group that knows improvement cycles to make intervention and implementation methods more effective and efficient over time
N = 565,000All Students & All Students & FamiliesFamilies
All Students & All Students & FamiliesFamilies
School Teachers School Teachers and Staffand Staff
School Teachers School Teachers and Staffand Staff
StateStateDepartment Department LeadershipLeadership
StateStateDepartment Department LeadershipLeadership
1 for each School(N = 1,800 School Teams)
School School Implementation Implementation
Team (N=4)Team (N=4)
School School Implementation Implementation
Team (N=4)Team (N=4)N = 7,200
N = 6
N = 288
Implementation-Skilled Workforce
N =
76 <
0.1%
$$
N = 70
Re
-Pu
rpo
se
““District” District” Implementation Implementation Teams (N=4)Teams (N=4)
““District” District” Implementation Implementation Teams (N=4)Teams (N=4)
1 for every group of 25 Schools (N = 72 “District” Teams)
Regional Regional Implementation Implementation Teams (N=5)Teams (N=5)
Regional Regional Implementation Implementation Teams (N=5)Teams (N=5)
1 for every group of 5 “Districts” (N = 14 Regional Teams)
1 for every 10 RITs(N = 1 State Teams)
State State Transformation Transformation
Team (N=6)Team (N=6)
State State Transformation Transformation
Team (N=6)Team (N=6)
Intensive Development
Saturation
State Capacity Development
0
2
4
6
8
10
12
14
16
1 2 3 4 5 6 7 8 9 10 11 12
Years
Reg
iona
l Im
pl. T
eam
s RITs
STTs
Intensive Development
Oregon Districts and Student Enrollment
ODE Report Card 2008-2009
Scale Up
To scale up interventions we must first scale up implementation capacity
Building implementation capacity is essential to maximizing the statewide use of EBPs and other innovations
Oregon needs
About 14 Regional Implementation Teams (and support staff)
One State Transformation Team
Annual cost about $8 million
About $40,000 per year per district (or $14 per year per student)
Scale Up
System Change
Innovative practices do not fare well in existing organizational structures and systems
An infrastructure for implementation does not exist
Organizational and system changes are essential to successful use of innovations
Legacy Systems A legacy system is a system or
application that continues to be used despite its poor competitiveness and compatibility with modern equivalents
Difficult to integrate new systems into legacy systems because it is a difficult and time intensive process to understand current system functionalities
Legacy methods create a huge conversion challenge for implementation teams
Ashok R. B. Samuel (2009)
EXISTING SYSTEM
EFFECTIVE INNOVATIONS
ARE CHANGED TO
FIT THE SYSTEM
EXISTING SYSTEM IS
CHANGED TO SUPPORT
THE EFFECTIVENESS OF
THE INNOVATION
EFFECTIVE INNOVATION
System Change
Compliance and Crises, Urgent, Time Sensitive!!• Services not meeting Standards• Deal with urgent and high profile issues
Best PracticesImplemented Fully With Good Outcomes
System Supports & Stability • Regulatory roles• Basic Data Systems• Financing and Fiscal Accountability• Accreditation/ Licensing Standards• HR rules and regulations• Safety Standards• Work with Legislature• Inclusion of Stakeholders
System Supports & Stability
Mandates, System Supports,
Foundational Polices & Regulations
Leadership Responsibilities and Leverage PointsLeadership Responsibilities and Leverage Points
Thanks to Tom Bellamy
Implementation Team
Management Team
TeachersInnovations
Students
Po
licy En
abled
P
ractice (PE
P)
Pra
ctic
e In
form
ed
Po
licy
(P
IP)
Sys
tem
C
han
ge
“Ext
ern
al”
Sys
tem
Ch
ang
e S
up
po
rt Adaptive Challenges
•RFP methods
• IHE curricula
• Salaries
• Funding
• Credentialing
• Licensing
• Time/ scheduling
• Union contracts
• Duplication
• Fragmentation
• Hiring criteria
• Federal/ State laws
SYSTEM ALIGNMENT
State Department
Districts
Schools
Teachers/ Staff
Effective Practices
AL
IGN
ME
NT
Federal Departments
Imp
lem
enta
tio
n T
eam
s
FORM SUPPORTS FUNCTION
The End in Mind With the purposeful use of
implementation science, we can:
Make statewide use of good instruction, evidence-based practices, and other innovations…
To produce increasingly effective outcomes for all students…
For the next 50 years.
Implementation Science
Global Implementation Conference 2011 www.implementationconference.org
Integrate the science, practice, policy of implementation, organization change, and system transformation
Call for Applications
Do you know of an organization/coalition currently implementing an evidence based practice in their local community?
Is the implementation of this program/innovation producing beneficial outcomes to the community?
Are they a role model for moving the evidence-based practice from science to service?
If you answered “YES” to all of the above, check out: http://www.samhsa.gov/scienceandservice
Nominate your own organization or someone you know!!
For More InformationState Implementation and Scaling up of Evidence-
based Practices (SISEP)
Dean Fixsen, Karen Blase, Rob Horner, George Sugai
www.scalingup.org
“Resources” Tab
Concept paper
Annotated bibliography
Data on scaling up
Scaling up Briefs
Evidence-based
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).