Team-Initiated Problem Solving (TIPS) Overview Presented by: Dale Cusumano, Ph.D., NCSP Research Associate…

Team-Initiated Problem Solving (TIPS)

OverviewPresented by:

Dale Cusumano, Ph.D., NCSPResearch Associate

University of North Carolina at Charlotte

Presented to: Seventh Annual Response to Intervention Best Practices Institute

October 15, 2013

90,000 public schools in the United States

Each school has 1+ teams to address

challenges and build solutions

Each team meets at least monthly

On average there are 5 people on each

810,000 hours of meetings

4,050,000 hours of personal

time annually

Problem

Solution

Organizing for an Effective Problem Solving Conversation

Use Data

Bob Algozzine, Kate Algozzine, and Dale CusumanoUNC at Charlotte

Rob Horner, Anne Todd, and Steve NewtonUniversity of Oregon

Outcomes

To what extent do teams follow problem solving steps and include

critical components?• Include a behavioral definition of

target behavior• Have a direct measure of the target

behavior prior to intervention• Include a step-by-step intervention

plan• Graph intervention results• Compare pre-intervention and post-

intervention performance• Develop a hypothesized reason for

the problem• Gather evidence that the

intervention was implemented as designed

Behavioral definition of target behavior, baseline

data, step-by-step plan, or graphic or numeric data

comparison from pre to post intervention were not

included (Flugum & Reschly, 1994)

Lowest implementation for hypotheses for problem,

treatment fidelity, and pre and post data comparison(Telzow, McNamara, & Hollinger,

Team members rated implementation as higher than observers with observers rating identifying antecedents and consequences for

behavior, identifying data to monitor progress, scheduling a follow up meeting as unmet

(Telzow, McNamara, & Hollinger, 2000)

People aren’t tired from solving problems – they are tired from solving the same problem over and over.

6Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual.

Efficient

Effective

Logical

Thorough

Team-Initiated Problem Solving II(TIPS II)

Decisions are more likely to be effective and

efficient when they are based on data

Quality of decision-making depends most on the first step (defining the

problem to be solved)

Data help us ask the right question…they do not

provide the answers: Use data to identify problems, refine the problems, and define the questions that

lead to solutions

Data help place the “problem” in the context rather than in students

Main Ideas

Build “decision systems” not “data systems”• Problem solving is driven by

decision systems, which are fueled by data

Use data in “decision layers”

(a) Is there a problem? (b) Localize the problem

(location, problem behavior, students, time of day), and

(c) Get specific

Do not drown in data

Be efficient

It is OK to be doing well!

More Main Ideas

What do we need?

• A clear model with steps for problem solving

• Access to the right information at the right time in the right format

• A formal process that a group of people can use to build and implement solutions.

10Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports,

University of Oregon, unpublished training manual.

Eight Keys to Effective Meetings

1. Organization (team roles, meeting process, agenda)2. Data (right information at right time in right format)3. Separate (a) Review of On-going Problem Solving (b)

Administrative Logistics and (c) New Problem Solving4. Define Problems with Precision5. Build Comprehensive Solutions that “fit”6. Add “Action Plans” for all solutions7. Review Fidelity and Impact regularly8. Adapt Solutions in response to data.

Implement Solution with High Integrity

Identify Goal for Change

Identify Problemwith

Precision

Monitor Impactof Solution and

Compare against Goal

Make SummativeEvaluationDecision

MeetingFoundations

Team-Initiated Problem Solving (TIPS II) Model

IdentifySolution and

CreateImplementation

Plan with Contextual Fit

Collect and Use

TIPS ModelTIPS Training

• One full day team training• Two coached meetings

Team Meeting• Use of electronic meeting minute system• Formal roles (facilitator, recorder, data analyst)• Specific expectations (before meeting, during meeting, after meeting)• Access and use of data• Projected meeting minutes

Research tool to measure effectiveness of TIPS Training• DORA (decision, observation, recording and analysis)• Measures “Meeting Foundations” & “Thoroughness of Problem Solving”

Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished manual.

TIPS I Study: Todd et al., 2011

OctNov Dec Ja

nFeb Mar Apr

OctNov Dec Ja

nFeb Mar Apr

OctNov Dec Ja

nFeb Mar Apr

OctNov Dec Ja

nFeb Mar Apr

10%20%30%40%50%60%70%80%90%

School A

School B

School C

School D

Baseline Coaching TIPS%

Solid = SW PBIS meetings using SWISOpen = progress monitoring meeting using DIBELS

Journal of Applied School Psychology

Pre Post/Exp0

80Exp Control

N = 17 N = 17TIPS Control TIPS Control

s of D

Newton et al., (2012):Effects of TIPS Training on Team Decision-making

Pre TIPS Training Post-TIPS Training

Problem-Solving Meeting

Foundations

Structure of meetings lays foundation for efficiency & effectiveness

Identify Goal for Change

Precision

MeetingFoundations

Plan with Contextual Fit

Collect and Use

PREDICTABILITYPARTICIPATION

ACCOUNTABILITYCOMMUNICATION

ACCOUNTABILITYCOMMUNICATIONRoles & Responsibilities

The process a

team uses is

important

FacilitatorRecorder

Data analystActive Member

Organization

Agenda, old business, new business, action plan for

decisionsWhat happens BEFORE a

meetingWhat happens DURING a

meetingWhat happens AFTER a

meeting

Foundations of an Effective and Efficient Meeting

Who is Responsible?Action Person Responsible

Reserve Room

Recruit items for Agenda

Review data prior to the meeting

Reserve projector and computer for meetingKeep discussion focused

Record Topics and Decisions on agenda/minutesEnsure that problems are defined with precisionEnsure that solutions have action plans

Provide “drill down” data during discussionEnd on time

Prepare minutes and send to all members

Start and end times

Agenda is used

TIPS Meeting Minutes and Problem-Solving Action Plan FormToday’s Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:

Next Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst: Team Members (bold are present today________________________________________________________________

Information for Team, or Issue for Team to Address

Discussion/Decision/Task (if applicable) Who? By When?

Administrative/General Information and Issues

Implementation and Evaluation

Precise Problem Statement, based on review of data

(What, When, Where, Who, Why)

Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction,

Safety)Who? By When?

Goal, Timeline, Decision Rule, & Updates

Problem-Solving Action Plan

Agenda for NEXT Meeting

1. 2.3.

Implementation and EvaluationPrecise Problem Statement, based on

review of data(What, When, Where, Who, Why)

Solution Actions (Prevent, Teach, Prompt, Reward,

Correction, Extinction, Adaptations, Safety)

Who? By When? Goal with Timeline

Fidelity of Imp measure

Effectiveness of Solution/Plan

Not started Partially Imp Imp Fidelity Done

Goal Met Better Same Worse

Agenda for Today:1. 3. 5.2. 4. 6.Previously Defined Problems/Solutions (Update)

TIPS Meeting Minutes and Problem-Solving Action Plan FormToday’s Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:

Next Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst: Team Members (bold are present today________________________________________________________________

Information for Team, or Issue for Team to Address

Discussion/Decision/Task (if applicable) Who? By When?

Administrative/General Information and Issues

Implementation and Evaluation

Precise Problem Statement, based on review of data

(What, When, Where, Who, Why)

Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction,

Safety)Who? By When?

Goal, Timeline, Decision Rule, & Updates

Problem-Solving Action Plan

Agenda for NEXT Meeting

1. 2. ‘3.

Implementation and EvaluationPrecise Problem Statement, based on

review of data(What, When, Where, Who, Why)

Solution Actions (Prevent, Teach, Prompt, Reward,

Correction, Extinction, Adaptations, Safety)

By When? Goal with Timeline

Fidelity of Imp measure

Effectiveness of Solution/Plan

Not started Partially Imp Imp Fidelity Done

Goal Met Better Same Worse

Agenda for Today:1. 3. 5.2. 4. 6.

Previously Defined Problems/Solutions (Update)

Where in the Form would you place:

1. Planning for next PTA meeting?

2. There have been five fights on the playground in the past 3 weeks.

3. Update on CICO implementation

4. Increasing gang recruitment as an agenda topic for today.

5. Next meeting report on lunch-room status.

Start and end times

Agenda is used

Meeting schedule

PARTICIPATION

Engaged in Topic

Decision Makers

COMMUNICATIONNorms for the meeting

Access to Meeting Minutes

ACCOUNTABILITY

Prepared for meetings

Are solutions working?

Documentation of decisions

Differences in lives of students

ACCOUNTABILITYCOMMUNICATION

IdentifyGoal for Change

Precision

MeetingFoundations

Identify Solution and

Plan withContextual Fit

Collect and Use

Problem = A difference between what is expected and what is observed that a team feels is worthy of addressing.

Start with Primary Problem Statements

Look at the Big Picture. Then use data to refine the problem to a Precise Problem

Statement.

Office discipline referrals for 3rd graders are above national medians for schools our size.

Morphing Data into Useful Information

Develop Primary Problem Statements• Look first at your patterns (tell the story)

• Level, Trend• Peaks• Match data to current perceptions

• Compare your data• With national median or percentiles• With last year• With what your faculty/students/ families want

Start with Primary Problem Statements

Look at the Big Picture. Then use data to refine the problem to a Precise Problem

Statement.

Move to Precise Problem Statements

Office discipline referrals for 3rd graders are above national medians for schools our size.

Referrals for defiance among third grade students from 11:30-12:30 in the cafeteria are increasing over time. It is believed that this is happening because students want

to avoid silent reading that happens after lunch.

What When

WhoWhy

Precision Components for Behavior Problem Statements

Precision Components for Academic Problem Statements

Examples: Primary to Precise

Gang-like behavior is increasing.

Our fourth graders cannot comprehend when reading!

• Bullying (verbal and physical aggression) on the playground is increasing during “first recess,” is being done mostly by four 4th grade boys, and seems to be maintained by social praise from the bystander peer group.

• Forty-seven percent of 4th grade students did not meet reading comprehension targets on AIMSweb Maze benchmark assessments when 80% of students at a grade level should meet this target. It appears that weak vocabulary skills are lowering students’ comprehension skills.

Examples: Primary to Precise

Carly is having reading difficulties.

• Carly is reading 20 words correctly per minute (goal is 60), skips or guesses at words she doesn’t know, mostly during language arts.

• Carly can not decode and struggles to read words containing R controlled vowels, digraphs, & long vowels.

• Jack’s math CBM scores fell at the 10th percentile as compared to national norms. Jack is not fluent in his knowledge of basic math facts and often does not attend to addition or subtraction signs on written math problems.

Jack cannot add or subtract.

Precise or Primary Statement?

Minor disrespect and disruption are increasing over time, and are most likely during the last 15 minutes of our block periods when students are engaged in independent seat work. This pattern is most common in 7th and 8th grades, involves many students, and appears to be maintained peer attention.

Precise or Primary Statement?

James is instructional in reading content that falls two grades below his current placement. His reading fluency scores fall below the 25th %ile and his rate of improvement is predicting that he will not meet end of the year goals for his grade. He has difficulty decoding unfamiliar words and does not track accurately from left to right when reading.

The boys in third grade are performing well below the 3rd grade girls in reading.

Three 5th grade students are having great difficulty expressing their thoughts in writing. Their fine motor skills are well developed but writing test scores are well below target. Since they can verbally share well-constructed and grammatically correct stories, it is thought that their spelling skills are hindering their writing skills with average weekly spelling test scores of C’s and D’s obtained.

Next, We Need Goals…

We need measurable and realistic SMART goals.

• SMART Goals• Specific • Measurable • Achievable• Relevant• Timely

Precision Problem StatementMinor disrespect and disruption are increasing over time, and are most likely during the last 15 minutes of our block periods when students are engaged in independent seat work. This pattern is most common in 7th and 8th grades, involves many students, and appears to be maintained by peer attention.

Precision GoalThe rate of disrespect and disruption among 7th and 8th graders (as measured by SWIS) will decrease to 2 per week by the 15th of next month (~.40 per day per month) and decrease to one or fewer per week (<.20 per day per month)by the end of the following month to be maintained across the remainder of the school year.

Examples

Our average Major and Minor ODRs per school day per month are higher than national median for a school of our enrollment size for all months except June. We have peaks in frequency of problems in Nov, Dec, Jan, and March with an increasing trend from September to March.

What When

WhoWhy

Precision Components for Behavior Problem Statements

Precision Components for Academic Problem Statements

SWIS Big 4 for October 1, 2011 through December 31, 2011

Defiance

ClassroomPlaygroun

11:30-12:15

Avoid Work

Possible Problem Precision Statement

Average referrals per day per month for All Referrals & Minors are above the 75th percentile for 5 of 6 months with an increasing trend. Defiance (what) in classrooms and on the playground (where) for a group of 15 fifth graders (who) between 11:30 and12:15 (when) is the most frequently reported behavior problem and occurs as a means of avoiding work (why).

How does this work for academics?

80% meeting

expectation

15% need more

5% need the most

Is there a problem? Academic Reference Points

50th Percentil

e on national norms

Low Risk indicator

DIBELS Universal Screening

Primary Problem StatementOur DIBELS Distribution summary shows that 49% of our kindergarten students at Adams Elementary fall in the strategic and intensive range. We have over 50% of our students requiring strategic and intensive supports for ISF, LNF.

Primary Goal At least 80% of our Kinders will be in Benchmark range at Winter Universal Screening Time

1 2 3 4 50%

Percentage of Students Meeting Expectation in Reading Comprehension (AIMSweb Maze)

FallWinterSpringGoal

Grade Level

Only 62% of 4th graders and 65% of 5th graders are meeting expectations in reading comprehension at the winter

benchmark; although a slight improvement from fall to winter is noted, it is below the goal of having 80% of students meet

expectation.

For today, we will focus on 4th

graders

Four 4th graders have the lowest scores on measures of reading comprehension. Other

students are below expectation but their scores are within + 1 correct responses (CR) of the target (14 CR). Scores for these four students fall in the

At Risk range, which is below the 10th percentile on national norms.

Low Risk > 14 CR

All close

to goal of 14 CR

Skill Developm

Functional

Drivers

Three 4th grade students are performing well below expectation comprehending written text that is presented at grade level (scores on Maze measures fall below the 10th percentile on national norms). These students also have weak reading

fluency skills that fall well below expectation, which is thought to be the reason for their low reading comprehension skills. A

fourth student (Sally) also is performing well below expectation in reading comprehension, but her reading fluency skills are in expected ranges. Weak vocabulary skills may be lowering her

comprehension skills.

Let us look at reading fluency as a possible reason for these

four students’ low reading comprehension.

Harris Word list for Sally = 2nd grade

Precision

MeetingFoundations

Team-Initiated Problem Solving (TIPS-II) Model

Identify Solution and

Collect and Use

Using Precision Problem Statements to Build Solutions, Action & Evaluation

PlansSolutions

Prevention: How can we avoid the problem context? Teaching: How can we define, teach, and monitor what we want? Recognition: How can we build in systematic reward for desired behavior?Extinction: How can we prevent problem behavior from being rewarded?Consequences: What are efficient, consistent consequences for problem behavior?

Action PlanWho will do each task & when will it be completed?

Evaluation

How will we collect and what data will we use to evaluate: • Implementat

ion fidelity?• Impact on

student outcomes?

Prevent “Trigger”

Define & Teach

Reward/Reinforce

Withhold Reward

Corrective consequence

Safety62

Example for Precise Behavior Problem Statement6th and 7th graders are engaging in inappropriate language, harassment, disrespect and aggression in two classrooms at 9:45 and 12:45 to get peer and adult attention and to escape the work. There are 175 total instances of problem behavior in 6th and 7th grade classrooms, for 2010-11 school year.

Re-review 6th and 7th graders classroom expectations/Respecting others, daily

Focus on Respect, Re-teach stop-walk routine

Set up “Daily Double”: Class period without problem behavior occurrence receives extra 2 mins, at end of period to talk.Provide specific feedback for using stop-walk routineEnsure staff use routine for responding to a report when student comes to talk.Use School Defined Process

Prevent “Trigger”

Define & Teach

Reward/Reinforce

Withhold Reward

Corrective consequence

Safety63

Example for Precise Academic Problem StatementFour 4th grade students are performing below expectation in reading comprehension. For three of these students, their performance is being impacted by low reading fluency skills. For one student, weak vocabulary skills are keeping her from comprehending written text.

Provide instructional level materials for independent reading activities.

Participation in a Repeated Readings intervention group.Self monitoring of reading fluency skills with gains tied to rewards.Self monitoring of known and unknown vocabulary words and accuracy rates.

Participation in a Drill Sandwich vocabulary practice.

Every solution/task needs an action plan• Who will do the task?• When will the task be completed?

Action Planning

Precision

MeetingFoundations

Collect and Use

Evaluation Planning

Evaluation Plan for monitoring fidelity of implementation AND impact on student behavior

• Evaluate fidelity of implementation compared to the goal

• Define how, when, criteria• Evaluate effect of solutions on student behavior

(impact) as compared to the goal• Define data to be used, how often and criteria

• Data analyst with data summaries and data access

Fidelity of ImplementationMeasure the degree in which the intervention was implemented as defined/expected

• Use percent/absolute value/ rate/scale as metric• Strive for 80% fidelity of implementation as measured

weekly (bi-weekly) on scale of 1-5Make easy for staff to record data

• Fidelity Check Board: X on number line• Fist of five• Fidelity check basket• Direct observation

Are we implementing the plan?

1 2 3 4 5 No Yes

Other Data for Monitoring Fidelity of Implementation

Fidelity Check RoutineWe do what we say we will do and we do it

with 80% fidelityEstablish a fidelity check routine that relates to Implementation

A 1-5 scale is used for all questions, with up to three questions per weekAt staff meeting, use fist of five while asking questionsIn staff room, create number line poster with questions

Did you provide instructional level reading to students?

1 2 3 4 5No Yes Did you acknowledge 5 students, not in

your classroom, daily? 1 2 3 4

Evaluation PlanningEvery problem needs to be monitored and evaluated

• Fidelity of Implementation• Effectiveness of Implementation

Impact of SolutionFor Behavior Example

Did we make a difference?

Impact of SolutionFor Academic Example

Evaluate the Problem: Did we make a difference?

1 2 3 4 50%

Percentage of Students Meeting Expectation in Reading Comprehension (AIMSweb Maze)

FallWinterSpringGoal

Grade Level

Evaluate the Problem: Did we make a difference?

✔ ✔

PrecisionMake Summative

EvaluationDecision

Precision

MeetingFoundations

Collect and Use

TIPS Team

Training

Readiness

Team Membershi

Team Data Access

Team Commitme

Coaching Commitme

Team Membership1. Representation needed for meeting their purpose 2. Inclusion and presence of administrator with authority to

make decisions

Team Data Access3. Data available for problem solving & decision-making

before and during the meeting4. Consistent process & procedures for documenting &

entering data exists5. Team member is fluent in generating basic and custom

reports from data set(s) being used

TIPS Team Training Readiness10 readiness guidelines

TIPS Team Training Readiness10 readiness guidelinesTeam Commitment

6. Implementation of TIPS Meeting Foundations7. Team & coach attendance at TIPS Team Training

• ** one full day or two half day team trainings8. Application of the TIPS model through the school year &

annual TIPS boosters

Coaching Commitment9. Team has access to a coach who knows the TIPS system

& who is available before, during, & after meetings to support fidelity of implementation

10. Commitment to attend team training and provide coaching before, during and after the meetings

Where are you in the implementation process? Adapted from Fixsen & Blase, 2005

• We think we know what we need so we are planning to move forward (evidence-based)

Exploration & Adoption

• Let’s make sure we’re ready to implement (capacity infrastructure)

Installation

• Let’s give it a try & evaluate (demonstration)Initial Implementation

• That worked, let’s do it for real (investment)Full Implementation

• Let’s make it our way of doing business (institutionalized use)Sustainability & Continuous Regeneration

Putting “it” in place.

Should we do it?

Now make it better

Initiative is way of work

TIPS II: Contacts and ReferencesUniversity of

OregonRob Horner

robh@uoregon.edu

Anne Toddawt@uoregon.edu

Steve Newtonjnewton@uoregon.edu

UNC at CharlotteBob Algozzinerfalgozz@uncc.edu

Kate Algozzinekmalgozz@uncc.edu

Dale Cusumanodcusuman@uncc.edu

Team-Initiated Problem Solving (TIPS) Overview Presented by: Dale Cusumano, Ph.D., NCSP Research Associate…

Documents

By: Kinsley Godwin and Nicki Cusumano

Susan Davies Gfroerer, NCSP - University of Dayton

Cusumano Presentation

©David Yoffie & Michael Cusumano. Do not reprint or ... ·...

Lori Anselmo, MA, CAGS, NCSP Lauren Schnare Mahoney, EdS,...

INDIAN NATIONAL CYBER SECURITY POLICY (NCSP-2013)

Recovering from Reading Recovery: A long-term program...

Eisner v Cusumano Constr., Inc. - NYCOURTS.GOV -...

Rate of Improvement Calculation and Decision Making Caitlin....

Michael Cusumano - Strategy Rules

All in the Family · or Angelo’s Pizza in Berlin and Sal....

120116 Cusumano Presentation

Foglio1 Beneficiari · 145 cusumano borina contributo...

Vietnam NCSP 2006 Verified

ncsp mcnp 2017 · MCNP Progress for NCSP LA-UR-17-21840 2.....

MICHAEL A. CUSUMANO - MIT Sloan · 2020-05-28 · 1 MICHAEL...