Measuring outcomes that are functional and …thearcalliance.org/.../10/Measuring-Outcomes-that-are-Functional...Measuring Outcomes that are Functional and Meaningful ... Anecdotal

Measuring Outcomes that are Functional and Meaningful

Chester County Early Intervention

Birth to Three August 2015

Outcomes

2

ECTA Center has identified 6 criteria for quality outcomes

Click below to take you to the ECTA Center Website

6 Criteria for Outcomes

1. The outcome is necessary and functional for the child’s and family’s life.

2. The outcome reflects real-life contextualized settings.

3. The outcome integrates developmental domains and is discipline-free.

3

6 Criteria for Outcomes

4. The outcome is jargon-free, clear and simple.

5. The outcome emphasizes the positive, not the negative.

6. The outcome uses active words rather than passive words.

4

ectacenter.org/~pdfs/pubs/rating-

ifsp.pdf

When the child’s contextual

information is available, the following

IFSP outcome criteria can also be

evaluated: •The outcome is based on the family’s

priorities and concerns.

•The outcomes described both the child’s

strengths and needs based on

information from the initial evaluation and

ongoing assessment.

The outcome

uses active

words rather

than passive ones.

The outcome

emphasizes

the positive,

not the

negative.

The outcome

is jargon-free,

clear and

simple.

The outcome

crosses

developmental

domains and is

discipline-free.

The outcome

reflects real-

life

contextualized

settings.

Place Card Here

Criteria Defining

High Quality, Participation-Based

IFSP Outcomes

Writing functional outcomes

The outcome is

necessary and

functional for

the child’s and

family’s life.

5

The OUTCOME

is necessary

and functional

for the child’s

and family’s life.

The OUTCOME

is discipline-free.

The OUTCOME

reflects real-life

contextualized

settings (e.g.,

not test items).

The wording of

the OUTCOME is

jargon-free,

clear and simple.

The wording of

the OUTCOME

emphasizes the

positive.

The OUTCOME

avoids the use

of passive words

(e.g., tolerate,

receive, improve,

maintain).

Criteria for Rating

IFSP Outcomes

When the child’s contextual information

is available, the following IFSP outcome

criteria can also be evaluated: •The outcome is based on the family’s

priorities and concerns.

•The outcomes described both the child’s

strengths and needs based on information

from the initial evaluation or ongoing

assessment.

IFSP Outcome Card 1

Nolan will play with toys

with his sister during bath

time.

6

Your Turn To Evaluate Outcomes

• Use the attached examples provided to determine if outcomes meet the criteria

7

8

1) Start with parents’/caregivers’ priorities about child’s learning/development and/or family’s needs

2) Consider what’s working and what’s challenging in everyday routines and activities

3) Consider how the child’s developmental skills, needs and disability influence the child’s learning and participation in everyday routines and activities

Using Information to Develop Outcomes/Goals

Meaningful Conversation With Families

9

Click below to hear about having meaningful conversation with families when creating Outcomes

WATCH VIDEO

Video Courtesy of Ken-Crest Services

Meaningful Conversations With Families

10

• Use Open-Ended Questions, avoid yes/no questions

• Allow parent/caregiver time to respond

• Follow-up with another question to identify parent/caregivers priority

• Avoid providing answers

Helpful Resources

Click on the icon for additional resources

regarding having conversations with

families

Measuring Progress on Outcomes

• What do we need to measure?

• How do we involve families in the discussion?

• Ways to collect information

• Documenting

11

Measuring Progress Ensure when choosing a progress measurement

that you answer the following questions:

• What will be measured?

• How will progress be measured?

• When will progress be measured?

• Who will collect progress measurement?

• What is the agreed upon frequency in which progress will be reported to parents/caregiver

12

Data Collection

13

Two Types of Data can be Collected

• Quantitative- Numerical representation of behavior(s)

• Qualitative – Measurement of desired behavior using descriptive and essential characteristics

14

15

SOME DATA COLLECTION TOOLS

16

Observation:

Seeing the child in a variety of naturally occurring routines and multiple settings, if appropriate, provides valuable insight into typical performance.

EXAMPLE: Dustin was observed both at home and at the child care center eating breakfast. At home he ate with verbal prompts, while at the child care, he needed hand over hand assistance.

17

Observation:

ADVANTAGES

• Gives a good idea of typical behaviors in different routines

• Gives a good idea of typical behaviors in different environments within the same routine

DISADVANTAGES

• Possibility of reporter bias

• Doesn’t typically yield concrete data

18

Photographs, Videotapes or Audiotapes:

Taken over time, showing the child engaged in activity that demonstrates the work toward the desired outcome.

EXAMPLE: James was videotaped in March crawling, and in August taking his first steps.

19

Photographs, Videotapes or Audiotapes:

ADVANTAGES

• Concrete record

• Lasting record

• Family friendly and meaningful

• Objective

DISADVANTAGES

• Need equipment

• Can be expensive

20

Rating Scales :

A list of behaviors or score on a scale; such as satisfaction scale, goal attainment scale, etc.

Day 5 25 40 50 75 90 100

Mon

Tues

Wed

EXAMPLE: percent of food he fed himself at lunch

21

Rating Scales

ADVANTAGES

• Easy to determine

• Easy to score

• More informal than tests

DISADVANTAGES

• Bias in giving credit

22

Anecdotal Records:

A brief, factual account of unanticipated behavior used to record information about a single event involving a child (the event being selected because it focuses on certain skills or behaviors).

EXAMPLE: Billy asked his mother for “more” milk by signing “more” at lunch. (Billy at kitchen table in high chair, 11:30 a.m. 10/05/03)

23

Anecdotal Records:

ADVANTAGES

• Quick

• Easy to do

• Interesting information

• Focus on behaviors

• Information in context

DISADVANTAGES

• Not complete picture

• Might miss details

• Bias of reporter

• Not representative

• Other staff needed

24

Frequency Sampling:

How often a child engages in certain kinds of behaviors. Need to determine when to observe, how long, in what settings and exactly what behaviors are to be observed.

EXAMPLE:

Skill – Initiating contacts Nonverbal Verbal Nonverbal Verbal

with sister with sister with mother with mother

/// / //// / //

25

Frequency Sampling:

ADVANTAGES

• Skills we value

• Uses a chart with periods of time

DISADVANTAGES

• May not be as meaningful for children with complex needs

• Demonstration of time or activities

26

Event Recording/Sampling:

Counting the number of times a targeted behavior occurs, indicating both the number of opportunities the child has to exhibit the skill and the number of times it occurs.

EXAMPLE:

Skill: Mary will drink her milk from a cup and put the cup on the table.

SCHEDULE DRINK THROW COMMENTS

Morning Yes No No problems!

Lunch No Yes Refused drink and threw

once

27

Event Recording/Sampling:

ADVANTAGES

• Very accurate information

• Objective

• Used in a variety of settings and times

DISADVANTAGES

• Takes event out of context

• May not offer antecedent events

• Lacks details

28

Time Sampling:

Observing behavior (or absence of behavior) at

fixed time intervals.

EXAMPLE:

Skill: Manipulates a toy during play

Evaluate at 30, 60, 90 second intervals.

Circle “Y” for yes or “N” for no

Activity 30 sec. 60 sec. 90 sec. Comments

Manipulates toy Y/N Y/N Y/N Jack-in-the-box

Manipulates toy Y/N Y/N Y/N Shape sorter

29

Time Sampling:

ADVANTAGES

• Efficient

• Objective

DISADVANTAGES

• Out of context

• No descriptions or details

30

Permanent Products:

Counting or saving and comparing observable, concrete results or products.

EXAMPLE: The number of scribbles made with a crayon on paper.

31

Permanent Products:

ADVANTAGES

• Observer not needed

• Can be counted/ analyzed after the fact

• Adds to information about the child

DISADVANTAGES

• Limited use

32

Representing Data to document progress

VISUAL PRESENTATION PROVIDES A DRAMATIC

VIEW OF PROGRESS-TO-DATE

33

Day 5 25 40 50 75 90 100

Mon

Tues

Wed

Thurs

Fri

Sat

Sun

EXAMPLE: percent of food he fed himself at lunch

Percent of food Jason ate by himself at lunch

34

Practice and Discussion

• Think of a child and family that you are currently working with. Create an Outcome that meets the 6 ECTA Criteria for Good Outcomes and indicate how you will measure progress (What? How? When? Who? What is the frequency in which progress will be reported to parents?)

35

Key Considerations

Interventionists need to familiarize themselves with the outcome and how data is to be collected regarding progress monitoring once assigned to provide services.

If the data collection measurement is not working, discuss and contact the SC to make appropriate changes.

Ensure that you review/analyze the data collected for quarterly progress and that you use the results to recommend any changes. Include a baseline and the results at the end of the quarter.

Please indicate the strategies that are working

and strategies that were not successful. 36

Chester County Procedures

• Quarterly progress reports are to be completed by each team member providing services to the child.

• The IFSP Team will determine who on the team that is responsible for reporting on the specific progress monitoring results on each Outcome in their quarterly progress report.

37

Chester County Procedures • Quarterly progress reports are to be completed and

submitted using the Formatta Software. • 3, 6, and 9 Month quarterly progress reports

(reviewed and signed by the Agency Reviewer) are to be submitted via e-mail to the assigned SC two weeks prior to the calculated quarterly due date based upon the current IFSP Dates.

• 12 Month quarterly progress reports are to follow the same procedures above with the

exception of due date. They are to be submitted via e-mailto the assigned SC two weeks prior to the scheduled annual evaluation meeting.

38