Dean L. Fixsen, Karen A. Blase, Michelle A. Duda, Sandra F. Naoom, Melissa Van Dyke National Implementation Research Network Implementation and System.

Dean L. Fixsen, Karen A. Blase,

Michelle A. Duda, Sandra F. Naoom,

Melissa Van DykeNational Implementation Research Network

Implementation and System Change

NHSC 2008

Copyright © Dean L. Fixsen and Karen A. Blase, 2008

NCLB Act & IDEA 2004

Implementation of scientifically based research

Ensure that school personnel have the skills and knowledge necessary to improve the academic achievement and functional performance of children, including the use of scientifically based instructional practices, to the maximum extent possible;


High School Education

15 million high school students

1.2 million high school teachers

20,000 high schools

3,143 counties

60 states & U.S. jurisdictions


Research to Practice

RESEARCH PRACTICEGAPIMPLEMENTATION

Improved Student Outcomes

Improved Education Systems


“In theory there is no difference between theory and practice; in practice, there is.”

Yogi Berra

Thanks to Joanne Cashman, The National Association of State Directors of Special Education


Research to PracticeResearch to Practice Gap

What is known is not what is adopted to help students, families, and communities

Implementation Gap What is adopted is not used with fidelity

and good outcomes for students.

What is used with fidelity is not sustained for a useful period of time.

What is sustained is not used on a scale sufficient to impact all high school students.


0

100

200

300

400

500

600

700

800

900

≤1972 1974 1976 1978 1980 1982

0

50

100

150

200

250

300

CumulativeCouples

CumulativeHomes

Fixsen, Blase, Timbers, & Wolf (2001)

Teaching–Family Model


Implementation Reviews

Human service prevention and treatment program literature (e.g. substance abuse, MH, justice, education, health)

Literature re: advanced manufacturing technologies, business, management, agriculture, engineering

Successful practices on a national scale (e.g. SW-PBS, SFA, MST, FFT, NFP, SE, IDDT, DBT, MI, PMTO, Incredible Years)


Insufficient Methods

Implementation What Works Clearinghouse

Pretty small

Implementation What Does Not Work Clearinghouse

Very large


Insufficient MethodsExcellent experimental evidence for

what does not work

Diffusion/dissemination of information by itself does not lead to successful implementation (research literature, mailings, promulgation of practice guidelines)

Training alone, no matter how well done, does not lead to successful implementation.


Insufficient MethodsExcellent evidence for what does

not work

Implementation by edict/ accountability by itself does not work

Implementation by “following the money” by itself does not work

Implementation without changing supporting roles and functions does not work Paul Nutt (2002). Why Decisions Fail


What Works

Effective intervention practices

+

Effective implementation practices

=

Good outcomes for students


Implement Innovations

Effective NOT Effective

Effective

NOT Effective

IMPLEMENTATION

INT

ER

VE

NT

ION Student Benefits


Implementation

An effective intervention is one thing

Implementation of an effective intervention is a very

different thing


Implementation

Letting it happen

Recipients are accountable

Helping it happen


Making it happen

Purposeful use of implementation practices and science

Implementation teams are accountable

Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004


Sustainable Benefits: Goal

Start with the end in mind

What will it take to:

make statewide use of high school innovations

that produce increasingly effective outcomes

for the next 50 years?


Sustainable Benefits: What

Rigorous curriculum and instruction

Assessment and accountability

Teacher quality/ PD

Student and family supports

Stakeholder engagement

Leadership and governance

Organization and structure

Resources for sustainability


Sustainable Benefits: How

Choosing an innovation

Implementation drivers

Implementation stages

Implementation teams

System improvement


Choosing an Innovation

Core intervention components

Clearly described (who/what)

Fully operationalized (do/say)

Practical measure of fidelity **

Field tested (recursive revision)

Contextualized (org./systems fit)

Effective (worth all the effort)


Sustainable Benefits





System improvement


Implementation Drivers

Reliably produce

predictable outcomes

for students, families,

and communities


INTEGRATED & COMPENSATORY

CONSULTATION & COACHING

CONSULTATION & COACHING

STAFF PERFORMANCE EVALUATION

STAFF PERFORMANCE EVALUATION

FACILITATIVE ADMINISTRATIVE

SUPPORTS

FACILITATIVE ADMINISTRATIVE

SUPPORTS

RECRUITMENT AND SELECTION

RECRUITMENT AND SELECTION

PRESERVICE TRAINING

PRESERVICE TRAINING

SYSTEMSINTERVENTIONS

SYSTEMSINTERVENTIONS

Implementation Drivers

DECISION SUPPORT DATA SYSTEMS

DECISION SUPPORT DATA SYSTEMS

OUTCOMES(% of Participants who Demonstrate Knowledge, Demonstrate

new Skills in a Training Setting, and Use new Skills in the Classroom)

TRAININGCOMPONENTS

KnowledgeSkill

DemonstrationUse in the Classroom

Theory and Discussion

10%

5% 0%

..+Demonstration in Training

30%20%

0%

…+ Practice & Feedback in Training

60% 60% 5%

…+ Coaching in Classroom

95% 95% 95%

Joyce and Showers, 2002

Integrated & Compensatory







System improvement


Stages of Implementation

Exploration

Installation

Initial Implementation

Full Implementation

Innovation

Sustainability

Implementation occurs in stages:

Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

2 – 4 Years


Stages of Implementation

Exploration

Installation

Initial Implementation

Full Implementation

Innovation

Sustainability

Implementation occurs in stages:

Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

2 – 4 Years

Intervention Outcomes

0% 100%

Implementation Outcomes







System improvement


Implementation

Letting it happen


Helping it happen


Making it happen

Purposeful use of implementation practices and science

Implementation teams are accountable

Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004


Implementation Team

A group that knows the innovation very well (formal and craft knowledge)

A group that knows how to implement that innovation with fidelity and good effect

A group that accumulates data & experiential knowledge -- more effective and efficient over time (information economics, K. Arrow)


Implementation Team

School

Management (leadership, policy)

Administration (HR, structure)

Supervision (nature, content)

Teachers & Staff

State and Community Context

District

Imp

lem

enta

tio

n T

eam

Simultaneous, Multi-Level Interventions


Implementation Team

Implementation Team

Prepare Communities

Prepare schools faculty, staff

Work with Researchers

Assure Implementation

Prepare Districts Assure Student Benefits







System improvement


A Sobering Observation

"All organizations [and systems] are designed, intentionally or unwittingly, to achieve precisely the results they get." R. Spencer Darling

Business Expert


System Change

To scale up interventions we must first scale up implementation capacity

Building implementation capacity is essential to maximizing the use of EBPs and other innovations

Large scale, real time change


System Change

ImplementInnovations

SustainableInfrastructure

Initiate & ManageChange

Management Group

Implementation Team #1(Up to 50 Schools)

Po

licy En

abled

P

ractice (PE

P)

Pra

ctic

e In

form

ed

Po

licy

(P

IP)

Sys

tem

C

han

ge

State T

ransfo

rmatio

n T

eam (S

TT

)


System Change

Transformation ZonesFocus on one thing – do it well

Amend the usual rules

Establish the first operating example of an innovation & system change

Manage change, reduce risks

A zone may be a region, a school, a part of a system

Depends on where you choose to start


Transformation Zone

Management Group and Practice Group meet monthly in order to bring about system change (urgent, real time)

– The first Implementation Team begins the implementation process in 5 – 10 schools (Transformation Zone)

– Repeat the process in subsequent sets of 5 – 10 schools

– CAPACITY DEVELOPMENT GOAL: Maximize opportunities for Implementation Team members to learn


Implementation Cost

Implementation Costs & Savings(Inflation Adjusted)

60

70

80

90

100

110

120

1 Yr Pre During PostYear 1

PostYear 2

PostYear 3

Ch

ang

e in

Bu

dg

et (

Per

cen

t)


Systems Change

State Department

Districts

Schools

Teachers/ Staff

Effective Practices

AL

IGN

ME

NT

Federal Departments

Imp

lem

enta

tio

n T

eam

s

FORM SUPPORTS FUNCTION


Scale Up Website

New OSEP Center (Fixsen, Blase, Horner, Sugai)

State Implementation and Scaling up of Evidence-based Practices (SISEP)

www.scalingup.org


Thank YouWe thank the following for their support

Annie E. Casey Foundation (EBPs and cultural competence)

William T. Grant Foundation (implementation literature review)

Substance Abuse and Mental Health Services Administration (implementation strategies grants; NREPP reviews; SOC analyses of implementation; national implementation awards)

Centers for Disease Control & Prevention (implementation research contract)

National Institute of Mental Health (research and training grants)

Juvenile Justice and Delinquency Prevention (program development and evaluation grants

Agency for Children and Families (child welfare leadership development contract)

Office of Special Education Programs (implementation capacity development center contract)


For More Information

Dean L. Fixsen

813-974-4446

[email protected]

Karen A. Blase

813-974-4463

[email protected]

National Implementation Research Network

At the Louis de la Parte Florida Mental Health Institute

University of South Florida

http://nirn.fmhi.usf.edu


For More Information

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

Download all or part of the monograph at:http://nirn.fmhi.usf.edu/resources/publications/Monograph/index.cfm

Implementation Research: A Synthesis of the Literature

Dean L. Fixsen, Karen A. Blase, Michelle A. Duda, Sandra F. Naoom, Melissa Van Dyke National Implementation Research Network Implementation and System.

Documents

copyright dean

small implementation

large slide

communities implementation

high school education

high school students

teachingfamily model

based research