Circe Stumbo, West Wind Education Policy Inc. and CCSSO CCSSO/SCEE National Summit on Educator Effectiveness April 10, 2013 Implementation Science: Closing.

Circe Stumbo, West Wind Education Policy Inc. and CCSSO

CCSSO/SCEE National Summit on

Educator Effectiveness

April 10, 2013

Implementation Science:

Closing Gaps Between Policy and Practice

Acknowledgements

Special thanks to Dean Fixsen, Karen Blase, and the National Implementation Research Network for their generosity in sharing their findings and their passion for all things implementation. This PowerPoint slide deck is built off their original work, with permission.

2

Sixth Year of Working with SEAs

State Implementation & Scaling-up of Evidence-based Practices Center

scalingup.org

3

http://www.scalingup.org/

Implementation Science`

A policy is one thing

Implementation of a policy is a very different thing

Students cannot benefit from a policy they do not experience

An Implementation Failure labeled as an Intervention Failure

Example: $500 million for Homebuilders (Family Support Services) 1993-1998

Funding only for interventions

No fidelity criteria insisted upon by the developers

National evaluation = not effective

Chapin Hall Center for Children, 2002

An Implementation Failure labeled as an Intervention Failure

Example: $500 million for Homebuilders (Family Support Services) 1993-1998

Funding only for interventions

No fidelity criteria insisted upon by the developers

National evaluation = not effective But, over 25% was spent on in-office

interventions with parents or children(< 0 fidelity)

Chapin Hall Center for Children, 2002

Longitudinal Studies of CSR Programs

Source: Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006

Evidence-Base For Effectiveness

Every Teacher Trained

Every Teacher Continually Supported

Actual SupportsYears 1-3

Fewer than 50% of the teachers received some training

Fewer than 25% of those teachers received support

Fewer than 10% of the schools used the CSR as intended

Vast majority of students did not benefit

OutcomesYears 4-5

Formula for Success

8

Socially Significant Outcomes

Effective Policies/

Interventions

Effective Implementation

Methods

Enabling Contexts

Formula for Success

9

WHY: Socially

Significant Outcomes

Effective Policies/

Interventions


Methods

Enabling Contexts

(Vision!)

Formula for Success

10

WHY: Socially


WHAT: Effective Policies/

Interventions


Methods

Enabling Contexts

Formula for Success

11

WHY: Socially



Interventions


Methods

WHERE: Enabling Contexts

Formula for Success

12

WHY: Socially



Interventions

HOW & WHO: Effective

Implementation Methods

WHERE: Enabling Contexts

Table Discussions

Share a situation you have experienced/ observed where one of the variables in the formula was zero

Implementation Science

Excellent evidence for what does not work

Implementation without changing supporting roles and functions

Implementation by edict

Diffusion/dissemination of information by itself

Implementation by “following the money”

Training alone, no matter how well done

Paul Nutt (2002). Why Decisions Fail

What does work? An Implementation Framework

Implementation Teams

Implementation Drivers

Improvement Cycles

Implementation Stages

http://sisep.fpg.unc.edu/learning-zone/science-of-implementation/implementation-frameworks



An Implementation Framework



Improvement Cycles





IMPLEMENTATION

Implementation Science

Effective NOT Effective

Effective

NOTEffectiveIN

TE

RV

EN

TIO

N Student Benefits

(Institute of Medicine, 2000; 2001; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999)

Poor outcomes

Inconsistent, not sustainable, poor outcomes

Poor outcomes; sometimes

harmful

IMPLEMENTATION


Effective

INT

ER

VE

NT

ION

80%, 3 Yrs 14%, 17 Yrs

Impl. Team NO Impl. Team

NOTEffective

Balas & Boren, 2000 Green, 2008

Fixsen, Blase, Timbers, & Wolf, 2001

Implementation Team

A group that knows:

the intervention/policy

implementation

improvement cycles

Simultaneous, Multi-Level Interventions

School

Management (leadership, policy)

Administration (HR, structure)

Supervision (nature, content)

Teacher

Region

District

Imp

lem

enta

tio

n T

eam

State

Partners

Iowa’s Collaborating for Kids (C4K)

22

Students & Families

Students & Families

Building Teachers and Staff


Building Leadership and Implementation

Team


Team

District Leadership and Implementation

Teams


Teams

Regional Implementation

Teams


Teams

State Transformation

Team


Team

StateDepartment of

Education Leadership

StateDepartment of


Cascading Logic Model

An Implementation Framework



Improvement Cycles





Improvement Cycles

Practice-Policy Communication Loops

Plan-Do-Study-Act cycles

Usability Testing

25

Students & Families

Students & Families




Team


Team


Teams


Teams


Teams


Teams


Team


Team

StateDepartment of


StateDepartment of


Polic

y Ena

bled

Pract

ice

(PEP)

Pract

ice

Info

rmed

Polic

y (P

IP)

Practice/Policy Communication

Loops

Students & Families

Students & Families




Team


Team


Teams


Teams


Teams


Teams


Team


Team

StateDepartment of


StateDepartment of


Polic

y Ena

bled

Pract

ice

(PEP)

Pract

ice

Info

rmed

Polic

y (P

IP)


Loops

Students & Families

Students & Families




Team


Team


Teams


Teams


Teams


Teams


Team


Team

StateDepartment of


StateDepartment of


Polic

y Ena

bled

Pract

ice

(PEP)

Pract

ice

Info

rmed

Polic

y (P

IP)


Loops

Students & Families

Students & Families




Team


Team


Teams


Teams


Teams


Teams


Team


Team

StateDepartment of


StateDepartment of


Feed

back

Loop

s

Feed

back

Loop

s

North Carolina Example

30

Students & Families

Students & Families




Team


Team


Teams


Teams


Teams


Teams


Team


Team

StateDepartment of


StateDepartment of


Authentic Work Informs the

PEP-PIP Cycle

Plan-

Do-

Study

-Act

Plan-

Do-

Study

-Act

Students & Families

Students & Families




Team


Team


Teams


Teams


Teams


Teams


Team


Team

StateDepartment of


StateDepartment of


Authentic Work Informs the

PEP-PIP Cycle

Usabi

lity

Studi

es

Usabi

lity

Studi

es

Table Discussions

How do teams at each level of your system interact?

What work are the teams doing with each other?

What feedback loops exist between teams?

How might you increase interaction among teams?

Have/might you incorporate improvement cycles into pilots and policy innovations?

33

Growing Implementation Capacity Using

Transformation Zones

Implementation Team

Minimum of 3 people (4-5 preferred)

Tolerate turnover; sustainable

State Transformation Team

First School Implementation Team

N = 10 Schools

Growing Implementation Capacity

(Phase 1)

Ext

ern

al S

up

po

rt &

2 F

TE

s

First Regional Implementation Team

N = 9 Staff

Invest up front to build

capacity


First School Implementation Team

N = 10 Schools


(Phase 2)

Ext

ern

al S

up

po

rt &

2 F

TE

s


N = 3 Staff

Subsequent Regional Implementation Team (3 Staff)



First School Implementation Team, Expanded

N = 20 Schools


(Phase 2)

Ext

ern

al S

up

po

rt &

2 F

TE

s


N = 3 Staff


Subsequent School Implementation Teams

N=20 Schools


N=20 Schools


N=20 Schools




N = 20 Schools


(Phase 2)

Ext

ern

al S

up

po

rt &

2 F

TE

s


N = 3 Staff



N=20 Schools


N=20 Schools


N=20 Schools




N = 20 Schools


(Phase 2)

Ext

ern

al S

up

po

rt &

2 F

TE

s


N = 3 Staff



N=20 Schools


N=20 Schools


N=20 Schools




N = 20 Schools


(Phase 2)

Ext

ern

al S

up

po

rt &

2 F

TE

s


N = 3 Staff



N=20 Schools


N=20 Schools


N=20 Schools


STAFF

Table Discussions

What responsibility will your state take to ensure implementation of your policies?

How can you grow implementation capacity within your state?

How can you rethink staffing in order to invest in the early stages of implementation?

42

Poll

One key take-away

OR

What questions do you have?

43

Possibilities to consider….

44

SCEE webinars

SCEE Topical Meetings

SCEE Discussion Groups

GIC

Individual state follow-up

Thank you

45

Circe Stumbo, West Wind Education Policy Inc. and CCSSO CCSSO/SCEE National Summit on Educator Effectiveness April 10, 2013 Implementation Science: Closing.

Documents

implementation failure

things implementation

org3 implementation

contexts formula

significant outcomeswhat

contexts vision

contexts stakeholders

west wind education