Lessons Learned from OSEP’s Model Demos: Creating Change to Promote Children’s Success 1.

1

Lessons Learned from OSEP’s Model Demos: Creating Change to Promote Children’s

Success

Lessons Learned from Model Demonstration Projects

Mary Wagner, Ph.D., Principal InvestigatorPhyl Levine, Ph.D., Director

SRI International

OSEP Project Directors’ ConferenceJuly 19, 2010 Washington, DC

Model Demonstration Project Cohorts

• Cohort 1: Progress monitoring/RtI for struggling readers K-5– Lehigh University, University of Pittsburgh– University of of Minnesota, Minneapolis Public Schools– University of Oregon

• Cohort 2: Tertiary behavior interventions K-8– University of Kansas, Illinois PBIS Network– University of Oregon– University of Washington

• Cohort 3: Early childhood language development– Orelena Hawks Puckett Institute – University of Kansas – Vanderbilt University, Florida State University

• Cohort 4: Writing proficiency in high school– University of Kansas– University of Hawaii-Manoa

The Model Demonstration Coordination Center (MDCC) was launched in 2005 to:

• Identify characteristics of an effective implementation/evaluation/refinement process that moves a practice from early testing to being ready for wider adoption

• Coordinate the evaluation of each cohort of Model Demonstration Project’s (MDPs) and synthesize and analyze their findings to maximize the strength of evidence produced

MDCC activities:• Facilitate a collaborative partnership with the MDPs

to create opportunities for learning• Contribute and/or broker methodological expertise• Develop a data system to assemble

MPD-provided data• Conduct cross-MDP and cross-cohort

analyses• Communicate implementation and

evaluation findings to promote adeeper understanding of the modeldemonstration process and its results

Framework for

understanding model

implementation and outcomes

Characteristics of the “source” in progress monitoring models*

SourceThe Model

Core intervention componentsMeasurement model and proceduresHow data are used to:▪ Monitor student progress

▪ Adapt instruction for strugglingreaders

▪ Inform eligibility determination forspecial education and developmentand monitoring of IEP goals

*Using cohort 1 as an example

Characteristics of progress monitoring “purveyors”

PurveyorThe MDP Grantee

Core implementation components▪ Introducing model to schools/teachers

▪ Formal professional development

▪ Ongoing coaching

▪ MDP staff selection and staffingstrategy

Characteristics of the progress monitoring “destination” organizations

DestinationParticipating Schools and Teachers

Characteristics of participating schools,classrooms, and teachers

Implementation outcomes―changes in:▪ Staff knowledge, attitudes, and actions

▪ Organizational structures, processesand culture

▪ External relationships

Sustained implementation

Influences on destination organizations and their implementation

• District-level (e.g., other initiatives, superintendent turnover, history with grantee university)

• State level (e.g., RtI initiatives, testing requirements)

• Other factors (e.g., union power/influence)

Intervention outcomes of progress monitoring models Student Outcomes

Increased:Oral reading fluencySAT-10 scoresPercentage reaching

benchmarksPercentage proficient on

state tests

Systems OutcomesFewer students referred

to special educationfor reading disabilities

Increased use of studentPM data in determin-ing special educationeligibility and settingand monitoring IEPgoals

Feedback on progress monitoring model implementation and effectiveness

• Reflect on lessons learned within cohorts

• Fidelity data

• Social validity data

Analysis and reporting

• Describe variations in each component of the conceptual framework for the three MDPs in a cohort

• Generate hypotheses from implementation/ innovation research regarding how variations may shape implementation experiences

• Hold hypotheses up to implementation experiences and outcomes– Across MDPs in a cohort– Across cohorts

• Derive principles regarding an effective and efficient model demonstration process

14

Cohort I:The University of Minnesota: Teri

Wallace Minneapolis Public Schools: Douglas

Marston

Lehigh University: Edward Shapiro and The University of Pittsburgh: Naomi

Zigmond

The University of Oregon: Gerald Tindal

15

Project MP3: Monitoring Progress In Pennsylvania Pupils

Lehigh University

16

What Makes RTI(I) Work?

• Consensus• Infrastructure• Implementation

17

Lessons Learned: What Made It Work?Consensus

• District committed to sustainability from outset

• District support at highest level of administration

• Willingness of schools to own the process

18

Lessons Learned: What Made It Work?Infrastructure Support

• Professional Development Specialist in district• District redirected resources to support implementation• District partnered with Project to secure new instructional

resources• Schools had similar context• Each school had identified principal leadership, from the

district perspective• Presence of well established core reading program• Willingness to modify schedules• Willingness to seek needed professional development• Inclusion of parent advisory component

19

Lessons Learned: What Made It Work?Implementation Support

• Universal screening measures already in place for several years

• Use of data, use of progress monitoring, not evident despite universal screening

• Schools modified professional development schedule to meet project specifications

• Schedules included recognized time for core and grade level meetings

20

Process Outcomes

• Implementation fidelity high for model in all schools• Data use and data based decisions high in all schools• District expanded the project to all elementary buildings

in district within 2 years of project ending• Maintained presence of parent advisory group to process• Maintained process despite change in building principal

at one building• Staff able to assume data management task after support

for a year• Model fit within the statewide initiatives

21

Some Not So Good Outcomes

• Despite strong implementation in all buildings, one building had much poorer student outcomes

• Instructional leadership of building principal level becomes crucial

22

DPM

University of Minnesota and Minneapolis Public Schools

http://progressmonitoring.net/ - see link to video

23

Context

• District had long history of commitment to using data for instructional decision-making though not implemented fully in all schools

• Implementation school’s leadership was committed to RTI, Reading Initiatives, Professional Development, etc.

• Schools had numerous and varied initiatives needing alignment

• Teachers’ knowledge of RTI and its various components varied across the schools

24

Implementation - Tools

• System Supports– Data Management System– Time – Master Calendar

• Professional Development• Leadership and Changing Roles

25

Process Outcomes

• Procedures were developed to assess teachers’ fidelity of implementation related to interventions, progress monitoring procedures, and data review meetings

• Professional development strategies were targeted based on the needs shown in student data. For example, PLCs focused on Tier 2 and 3 interventions needed to meet student needs. Coaching responded to needs identified through fidelity checks, and more…

26

Process Outcomes, continued

• Data meetings met to review progress of all students and focused on students receiving Tier 2 and 3 interventions. A RTI coordinator was hired to facilitate these meetings, organize data, etc. Originally supported through grant dollars, this position remains despite district cuts.

• Teacher attitudes were measured to determine their view of the components of RTI. They were supportive and positive.

27

Teacher Testimonial

I thought I was an “OK” teacher and having been through the whole process of learning more about instruction, understanding the developmental aspects of reading, knowing what to do because I have good assessment data – I feel like a great teacher.

28

Lessons Learned from Model Demos:

Creating Change to Promote Children’s

Success

Cohort II

Tertiary Intervention: The K-I Center

Lucille Eber Co-Principal Investigator

Illinois PBIS Network

Wayne Sailor

Principal Investigator University of Kansas

OSEP Project Director’s Conference

July 19-21, 2010

29

Cohort II- cont.

Intensive Positive Behavior Support (IPBS) The University of Oregon: Cynthia Anderson

Scaling the Pyramid: A Model of TertiaryIntervention Services to Students withChallenging BehaviorUniversity of WA: Carol Davis Ilene Schwartz I

30

K-I Center Leadership

• Jamie Bezdek• Amy McCart• Holly Sweeney

• Kimberli Breen• Kelly Hyde• Sheri Luecking• Diane McDonald• Jen Rose

31

Implementation Outcomes: Change in Knowledge/Skills of Adults

• Role of Building Administrators with behavior support

• Change in role of Special Education personnel

• Role of District Administrators in guiding the systems change process

32

Example: Principal Role(no longer a “Special Education issue”)

Administrators being taught the system features, the data/tools, and the practices well enough to guide/lead any “corrections” needed within FBA/BIP and wraparound plans.

Administrator Training Skill set example:

If an individual behavior intervention plan

is not working, what should a principal look for or ask?

33

Administrator Role Example

School personnel should not be able to choose NOT to provide students with evidence-based interventions.

Academic ‘analogy’:

Personnel are expected to provide evidence-based academic instruction; same expectations need to be established for behavior support.

34

Implementation Outcomes: Change in Organizational Structure/Culture

• Systems need to establish competency and confidence with fidelity of behavior interventions within general education.

• All staff “owning” success of ALL students

• Shift in system habits: from “test/place” to interventions and progress monitoring

35

Implementation Outcomes: Changes in External Relationships

• District Leadership Teams

• District External Coaches

• Special Education Directors/Organizations

• Community Partners

• Families

36

Implementation Outcomes: Sustainability of Model

Replication of Tertiary Demos Moving Faster…

IL Phases of Implementation: Secondary Phase I (n=8 Replication Schools)

0%10%20%30%40%50%60%70%80%90%

Team meets

regularly

Sec/Ter.

tracking tool

used

Students are

referred for

tier 2

interventions

DPR used 70% success

rate for simple

tier 2

interventions

Sch

oo

ls w

ith It

em

in P

lac

e

Fa ll 2008 Spring 2009

38

Implementation Outcomes: Sustainability of Model

• Policy changes are addressed

– transition support to ensure behavioral success

– Job Description and supervision changes

• Training and technical assistance for behavior skill sets are routinely scheduled in districts

• Use of data by district teams (risk ratio, LRE, etc) becomes ‘business as usual’.

39

Policy Examples

• Summer FTE of School Social Workers to support some students w/Tier 3 plans

• Transition planning support expected as students change grade levels

• Job Descriptions include expectations for evidence-based behavior support

40

Training Examples

• Administrator role with behavior support at all three tiers

• Changing role of special education personnel in facilitating teams/plans

• Teacher expectations with behavior support at all three tiers

• Using RtI ‘framework’ with family engagement

41

Data Examples

Review data by ethnicity and disability routinely at building and district levels

– Discipline data

– Academic data

– Restrictiveness of placement data

Students with IEPs Served in Separate Placements

102122

7.48

4.91 4.58

9.01

0

2

4

6

8

10

FY07 FY08

Ra

tio

90

100

110

120

130

# o

f St

ud

ents

Students w/IEPs in separate placementsDistrict Ratio State Target

43

Lessons Learned from Model Demos:

Creating Change to Promote Children’s Success

Cohort 3Early Childhood Language

Interventions

OSEP Project Director’s Conference Washington DCJuly 19-21, 2010

44

The Center on Everyday Child Language Learning

Orelena Hawks Puckett Institute Carl Dunst

Carol Trivette

University of Kansas Dale Walker

Jane Atwater Kathy Bigelow

Vanderbilt University Florida State University

Ann KaiserJuliann Woods

45

Common Features of Cohort 3Process

Models of naturalistic evidence-based language interventions for children ages birth through 5

Implement the model in 3 sites representing typical settings

Enroll children in Part C programs and follow them into Part B programs

Provide professional development to providers implementing the model

Assess impacts using some common measures

46

Coaches build the capacity of parents and early educators to promote communication

Parents/teachers use evidence-based practices

Build on strengths and collaborate with parents about strategies and routines Embed intervention into home and school routines Collaborate with Part C and Part B during transitions

47

Parents are their child’s first communication partners

Teachers and service providers support child communication across settings

KTTP Communication teams include families, providers across agencies, and communication coaches

Communication coaches facilitate parent and service providers learning

Coordinated transition and continuity in communication intervention between Parts C and B is lead by the parent in the communication team

48

Identify children’s interests and everyday activities that are suited for learning communication skills Increase child participation in these interest-based everyday activities

Embed instructional practices for supporting and strengthening communication in the contexts of activities

Part C providers facilitate parent learning and use of strategies

Approach fits within existing programs

Strengthen parents’ abilities to enhance the transition to Part B

49

Strategies to Change in External Relationships

Use the team that includes parents to collaborate with the Part B preschool services during transition

Empower parents to lead the team to support the transition to Part B preschool services

Empower parents to promote continuity for children from Part C to Part B preschool services

50

Process Reflections on TransitionsProjects differ in the people they are trying to include in

the transition from Part C to Part B Projects all include parents in the transition but the

emphasis of the role of the parent variesProjects differ in terms how direct their involvement is

in the transition process

These differences lead to different transition challenges: joining an already formed team, development of parent leadership, development of transition skills in parents