Rob Horner University of Oregon . Implementation of Evidence-based practices School-wide behavior support Scaling evidence-based practices.

Scaling Up Effective Practices in Education: Doing What Works

Rob HornerUniversity of Oregon

www.pbis.org

GoalsImplementation of Evidence-based

practices

School-wide behavior support

Scaling evidence-based practices

School-wide PBIS Build a continuum of

supports that begins with the whole school and extends to intensive, wraparound support for individual students and their families.

What is School-wide Positive Behavior Interventions and Support?

School-wide PBIS is:

A systems approach for establishing the social culture and behavioral supports needed for a school to be an effective learning environment for all students.

Evidence-based features of SWPBIS Prevention

Define and teach positive social expectations

Acknowledge positive behavior

Arrange consistent consequences for problem behavior

On-going collection and use of data for decision-making

Continuum of intensive, individual intervention supports.

Implementation of the systems that support effective practices

School-wide Systems Use the core messages of the multi-tiered

prevention approach:

Invest in prevention first

Always build multiple levels of support

Adapt procedures to fit the local culture

SWPBIS is about making schools more effective for ALL children:

A commitment to building schools that are responsive to the culture, disabilities, learning needs, and behavior support needs of each child will begin with establishing a predictable, consistent, positive and safe social culture

SWPBIS in 13,331 schools 8/10’Al

abam

a

Alas

ka

Ariz

ona

Arka

nsas

Calif

orni

a

Colo

rado

*

Conn

ectic

ut

Del

awar

e

Flor

ida*

Geo

rgia

Haw

aii

Idah

o

Illin

ois

Indi

ana

Iow

a*

Kans

as*

Kent

ucky

Loui

sian

a*

Mai

ne

Mar

ylan

d*

Mas

sach

usett

s

Mic

higa

n

Min

neso

ta

Mis

siss

ippi

Mis

sour

i*

Mon

tana

*

Neb

rask

a

Nev

ada

New

Ham

pshi

re

New

Jers

ey*

New

Mex

ico

New

Yor

k

Nor

th C

arol

ina*

Nor

th D

akot

a*

Ohi

o

Okl

ahom

a

Ore

gon*

Penn

sylv

ania

Rhod

e Is

land

Sout

h Ca

rolin

a*

Sout

h D

akot

a

Tenn

esse

e

Texa

s

Uta

h*

Verm

ont

Virg

inia

Was

hing

ton

Stat

e

Was

hing

ton

DC

Wes

t Vir

gini

a

Wis

cons

in

Wyo

min

g

0

200

400

600

800

1000

1200

1400

Oregon: (49% of all Schools in Oregon)

Illinois

FindingsSWPBIS is possible (at all grade

levels)

SWPBIS is associated with: 20-60% reduction in problem behavior

Increases in academic performance

Improved self-assessment of faculty effectiveness

Average Major Discipline Referrals per 100 Students by Cohort

Cohort 1 (n=15) Cohort 2 (n=19) Cohort 3 (n=34) Cohort 40

20

40

60

80

100

120

140

160

180

2004-2005 2005-2006 2006-2007 2007-2008

Cohort 1 Cohort 2 Cohort 3 Cohort 40%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2003-04 2004-05 2005-06 2006-07 2007-08 2008-09

Percent of Students meeting DIBELS Spring Benchmarkfor Cohorts 1 - 4 (Combined Grades)

5,943 studentsassessed

5,943 studentsassessed

8,330 studentsassessed

8,330 studentsassessed

16,078 studentsassessed

16,078 studentsassessed

32,257 studentsassessed

32,257 studentsassessed

Spring ’09: 62,608 students assessed in cohorts 1 - 4

Spring ’09: 62,608 students assessed in cohorts 1 - 4

Percent of Students at DIBELS Intensive Level across year by Cohort

Cohort 1 Cohort 2 Cohort 3 Cohort 40%

5%

10%

15%

20%

25%

30%

2003-04 2004-05 2005-06 2006-07 2007-08 2008-09

Pe

rce

nt

of

Stu

de

nts

at

DIB

EL

S I

nte

nsi

ve I

nte

rve

n-

tion

Le

vel

Bully Prevention (Scott Ross)** In Oregon

Instead of focusing on removing students who bully, remove the rewards for bullying.

12

0

2

4

6

8

10

0

2

4

6

8

10

Baseline Acquisition Full BP-PBS Implementation

0

2

4

6

8

10

0

2

4

6

8

10

0

2

4

6

8

10

Num

ber

of I

ncid

ents

of

Bul

lyin

g B

ehav

ior

School Days0

2

4

6

8

10

School 1

Rob

Bruce

Cindy

Scott

Anne

Ken

School 2

School 3

3.14 1.88 .8872%

BP-PBS, Scott Ross 13

Conditional Probabilities of Victim Responses to Problem Behavior

0%

10%

20%

30%

40%

50%

"Sto

p"

"Wa

lk"

Po

sitiv

e R

esp

on

se(l

au

gh

ing

/ch

ee

rin

g)

Ne

ga

tive

Re

spo

nse

(cry

ing

/fig

htin

gb

ack

)

No

Re

spo

nse

Pro

bab

ilit

y o

f R

esp

on

seBaseline

BP-PBS

28% increase 19% decrease

BP-PBS, Scott Ross 14

Conditional Probabilities of Bystander Responses to Problem Behavior

0%

10%

20%

30%

40%

50%

"Sto

p"

"Wa

lk"

Po

sitiv

e R

esp

on

se(l

au

gh

ing

/ch

ee

rin

g)

Ne

ga

tive

Re

spo

nse

(cry

ing

/fig

htin

gb

ack

)

No

Re

spo

nse

Pro

bab

ilit

y o

f R

esp

on

seBaseline

BP-PBS

21% increase

22% decrease

Lessons LearnedInvesting in a school-wide social

culture benefits the adults as well as the students.

Scott Ross

Lessons LearnedInvest in Evidence-based

practices

• Oregon Department of Education

• Promoting Educational Effectiveness

• Standard, Emerging, Scale Worthy

• Ineffective practices

• Effective, but costly practices

Lessons LearnedNever stop doing what already

works

• Build from a commitment to student outcomes.

Lessons LearnedAlways look for the smallest

change that will produce the largest effect

• Iterative cycles of change

• View comprehensive models with skepticism

Lessons LearnedWhen adopting something

new, define what you will STOP doing to create the resources for the new activity.

• All current activities can be defended. The issue is priority.

The typical school has over a dozen “programs” to address the social behavior of students. Typically none are implemented

with fidelity and few with effect.

Glenn Latham

~80% of Students

~15%

~5%

ESTABLISHING CONTINUUM of SWPBS

SECONDARY PREVENTION• Check in/out• Targeted social skills

instruction• Peer-based supports• Social skills club•

TERTIARY PREVENTION• Function-based support• Wraparound• Person-centered planning• •

PRIMARY PREVENTION• Teach SW expectations• Proactive SW discipline• Positive reinforcement• Effective instruction• Parent engagement•

SECONDARY PREVENTION• • • • •

TERTIARY PREVENTION• • • • •

PRIMARY PREVENTION• • • • • •

Linking Academic and BehaviorEffective School-wide Academic Support

Effective School-wide Behavior Support

More Minutes In School

Increase in Minutes Academically Engaged

Improved Academic Outcomes

Implementation Select effective practices

Build the training, coaching and evaluation capacity to support those practices

Implement with clear policy, support and evaluation.

State Leadership

Regional Implementation Capacity

District infra-structure

School fidelity

Using data for decision-making

Policy Practice Feedback Loops

Policy

Practice

Structure

Procedure

Policy

Practice

Fee

dbac

k

Pol

icy

Ena

bled

Pra

ctic

es(P

EP

)

Pra

ctic

e In

form

ed P

olic

y(P

IP)

Fixsen & Blase, 2009

PDSA CyclesShewhart (1924); Deming (1948); Six-Sigma (1990)

Plan – Develop specific things to do

Do – Do them (make sure)

Study – See what happens

Act – Make adjustments

Cycle – Do over and over again until the goal is reached (again)

Stages of Implementation

Exploration

Installation

Initial Implementation

Full Implementation

Innovation

Sustainability

Implementation occurs in stages:

Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

2 – 4 Years

Core Implementation

Components

Core Implementation

Components

© Fixsen & Blase, 2008

Successful Student Outcomes

Com

pete

ncy

Driv

ers

Com

pete

ncy

Driv

ers O

rganization Drivers

Organization D

rivers

LeadershipLeadership

Program/Initiative/Framework (e.g. RtI)

How:

What:

Why:

Capacity to provide direction/vision of process

Staff capacity to support students/families with the selected practices

Institutional capacity to support staff in implementing practices with fidelity

© Fixsen & Blase, 2008

Performance Assessment (Fidelity)

Coaching

Training

Selection

Systems Intervention

Facilitative Administration

Decision Support Data System

Core Implementation

DriversCom

pete

ncy

Driv

ers

Com

pete

ncy

Driv

ers O

rganization Drivers

Organization D

rivers

LeadershipLeadership

Adaptive Technical

Successful Student Outcomes

Program/Initiative/Framework (e.g. RtI)

Lessons LearnedMeasure FIDELITY as well as

impact

• Are we doing what we said we would do?

• Is what we are doing benefiting students?

Iowa Checklist 01-05, PK-6 % Fully & Partially Implemented

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

05-A

ug-0

3

05-N

ov-0

3

23-F

eb-0

4

22-J

an-0

4

01-F

eb-0

5

02-J

un-0

5

12-A

ug-0

4

24-N

ov-0

4

01-M

ar-0

5

12-S

ep-0

2

31-O

ct-0

2

28-F

eb-0

3

21-A

pr-0

3

01-S

ep-0

3

05-N

ov-0

3

05-A

ug-0

3

11-S

ep-0

3

07-N

ov-0

3

06-F

eb-0

4

01-S

ep-0

3

01-N

ov-0

3

01-M

ar-0

4

03-A

ug-0

4

08-N

ov-0

4

08-M

ar-0

5

03-J

un-0

5

1 1 1 2 2 2 3 3 3 4 4 4 4 4 4 5 5 5 5 6 6 6 7 7 7 7

Start Up Full Implementation Start Up Part Implementation

50%

60%

70%

80%

90%

100%

Fully Implementing (n=272) Partially Implementing (n=25)

87% 81.78%

9%12%

4% 7%

% o

f Stu

dent

s w

ith O

DR

s

Triangle Data by Fidelity Results Only

Mean Percentage of Students Statewidewith Majors 2009-10

Middle Schools

High Schools

Out of School Suspension per 100 Students Enrolled

National Medians .22 .50 .68 .42

Implementing at ScaleScaling up involves

implementation of effective practices at a level that the practice becomes the “regular way of operating.”

Implementing at ScaleMaking SWPBIS the norm not just an initiative

District Policy stating value of student social behavior

School Improvement Plan lists social behavior of students as a core goal.

Annual Orientation: School-wide academic and behavior supports

Hiring: Competence in school-wide behavior support is an expected competence

Annual Evaluations: Include competence in school-wide behavior support as part of annual evaluations

Professional Development focused on school-wide systems of behavior support

Develop data systems for decision-making at the school level (literacy, behavior, math, writing).

Racial Disproportionality In School DisciplineElementaryMiddleHigh Schools

Preventing DisproportionalityEstablish School-wide expectations

Establish a predictable, consistent, positive social culture

Deliver Effective instructionEvidence-based, Multi-tiered intensity

Provide Faculty orientationRole of culture, Expectations of school

Data collection and continuous review

ODR rates, disaggregated by race/ethnicity

Summary Fiscal constraints create opportunities

Efficient Improvement through integration and collaboration

Implement practices that are evidence-based

Implement practices with the systems needed for sustainability and impact.

Emphasize measuring for improvement, not just “accountability” or “compliance”

Are we doing what we said we would do?

Are practices benefiting students?