PROGRESS MONITORING FOR DATA-BASED DECISIONS June 27, 2007 FSSM Summer Institute.

PROGRESS MONITORING

FOR DATA-BASED DECISIONS

June 27, 2007 FSSM Summer Institute

2

Progress Monitoring: Learning Goals for Today

• Determine student’s current level of performance (Baseline data) CBM

• Identify learning goal (norms) • Implement Research-Based Interventions• Continue to measure and monitor students’

performance on a regular basis• Graph the results• Compare expected progress to actual rate• Adjust instruction based on the data

3

What IS Progress Monitoring

Find the in your handout packet.

Work with your TEAM to complete this.

Be ready to share a specific example of Progress Monitoring in your school.

Handout PM 1

4

Progress Monitoring

is a scientifically-based practice that is used to assess students’ academic performance and evaluate the effectiveness of instruction – for an individual student or an entire class.”

National Center on Progress Monitoring

www.studentprogress.org

5

A Scientific Base Supports One Form of Progress Monitoring:

Curriculum-Based Measurement

CBM

6

Curriculum-Based Measurement (CBM) . . .

• Result of 20-30 years of research

• Used in schools across the country

• Uses short “probes” for frequent PM

• Demonstrates strong reliability and validity

• Is sensitive to small gains in progress

• Helps teachers plan instruction

7

Uses of CBMBenchmark

All Students – 3 times a year – F, W, SStrategic Monitor

Monthly check up for students with moderate skill deficits who are receiving supplemental instruction in small group (3-5)

Progress MonitorMonitor once or twice weekly to measure student’s response to more intensive interventions (individual or group of 2). Graph the data for on-going decision making.

8

2006

2007

School Calendar Year (2006-2007):

Benchmarking

(Tier 1)

2-weeks during:September 1 to October 15

January 1 to February 1

May 1 to June 1

9

For Teachers: Classroom ReportBenchmark Data

AIMSweb.com

10

10th %ile10th %ile

90th %ile90th %ile

75th %ile75th %ile

50th %ile50th %ile

25th %ile25th %ile

Student is above the90 %ile and is well above average.

Student is above the90 %ile and is well above average.

TargetTarget

____________________________________Box and Whisker Charts

AIMSweb

11

AIMS web / Harcourt w/ permission 6/07

12

Decision Making RulesStudents who score below the 25th percentile on

general outcome benchmark screening receive targeted intervention, and progress is monitored

on a monthly basis. Progress for those below the 10th percentile is monitored at least weekly, and they receive intensive intervention.

Eight data points over at least 4 weeks are required to determine a trend line.

Change or modify the intervention when the data points are below the aim line (goal line) for 4 consecutive data points, or when the trend line is not on course to meet the goal line.

13AIMSweb/Harcourt w/ permission 6/07

14AIMSweb/Harcourt w/ permission 6/07

15AIMSweb/Harcourt w/ permission 6/07

16

How Does Student Progress Monitoring Work?

• Determine student’s current level of performance (Baseline data)

• Identify learning goal (local norms or accepted research standards)

• Use research based intervention(s) targeting the problem

• Continue to measure performance on a regular basis (CBM probes at same level)

• Graph the results• Compare expected progress to actual rate• Adjust instruction based on the data

17

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Weeks of Primary Prevention

WIF

: C

orr

ec

tly

Re

ad

Wo

rds

Pe

r M

inu

teCalculating Slope: First draw a

Trend Line – Tukey MethodStep 1: Divide the data points into three equal sections by drawing two vertical lines. (If the points divide unevenly, group them approximately.)

Step 2: In the first and third sections, find the median data-point and median instructional week. Locate the place on the graph where the two values intersect and mark with an “X.”

Step 3: Draw a line through the two Xs, extending to the margins of the graph. This represents the trend-line or line of improvement.

www.studentprogress.orgPM 2

18

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Weeks of Primary Prevention

WIF

: C

orr

ec

tly

Re

ad

Wo

rds

Pe

r M

inu

te

Step 1: Divide the data points into three equal sections by drawing two vertical lines. (If the points divide unevenly, group them approximately.)

Step 2: In the first and third sections, find the median data-point and median instructional week. Locate the place on the graph where the two values intersect and mark with an “X.”

Step 3: Draw a line through the two Xs, extending to the margins of the graph. This represents the trend-line or line of improvement.

19

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Weeks of Primary Prevention

WIF

: C

orr

ec

tly

Re

ad

Wo

rds

Pe

r M

inu

te

Step 1: Divide the data points into three equal sections by drawing two vertical lines. (If the points divide unevenly, group them approximately.)

Step 2: In the first and third sections, find the median data-point and median instructional week. Locate the place on the graph where the two values intersect and mark with an “X.”

Step 3: Draw a line through the two Xs, extending to the margins of the graph. This represents the trend-line or line of improvement.

X

X

20

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Weeks of Primary Prevention

WIF

: C

orr

ec

tly

Re

ad

Wo

rds

Pe

r M

inu

te

Step 1: Divide the data points into three equal sections by drawing two vertical lines. (If the points divide unevenly, group them approximately.)

Step 2: In the first and third sections, find the median data-point and median instructional week. Locate the place on the graph where the two values intersect and mark with an “X.”

Step 3: Draw a line through the two Xs, extending to the margins of the graph. This represents the trend-line or line of improvement.

X

X

21

Baseline Data & GoalTOM

• Second grade student, fall assessment, oral reading fluency, second grade probes

• Baseline Data: day 1 =11, day 2 = 13, day 3 =12

• Plot these points on your graph

PM 3

22

Baseline Data & Goal Line

• Baseline Data: 11, 13, 12

• Find the median, or middle number when numbers are rank ordered: 11, 12, 13

• Median = 12 words read correctly per min.

• Average Peer = ? Use norms table, Handout PM 4

• Rate of Improvement = ? Words per wk.

• Use norms table, Handout PM 4

23

Baseline Data & Goal

• Baseline Data: 11, 13, 12

• Median = 12 words read correctly per min.

• Average Peer = 55 wrc per minute

• Average Rate of Improvement = 1.1 words per week.

• Set a realistic but ambitious goal, such as:

• Goal = 2 words per minute more per week

24

Baseline Data & Goal• Goal = 2 words per minute more per week• Select # wks for monitoring: Weeks =12• Calculate: 2 wds./wk. increase, times 12

wks. = 24 words total increase• Add the expected 24 word increase to

baseline (12). Goal= 36 wrc in 12 weeks• Mark the goal point (36) on the last data

collection day, Thursday, at the end of 12 weeks of intervention.

• Connect baseline data median(12) to the goal point (36) = Goal Line (Aim Line)

25

GOAL LINETom's Oral Reading Fluency

0

5

10

15

20

25

30

35

40

Sep-06 Oct-06 Nov-06 Dec-06

Dates

WRC

26

Intervention Data Points

9/16/2006 14

9/18/2006 15

9/23/2006 13

9/25/2006 17

9/30/2006 16

10/2/2006 15

10/7/2006 16

10/9/2006 15

Enter this data on your graph and draw a trend line.

27

Tom's Oral Reading Fluency

0

5

10

15

20

25

30

35

40

9/5 9/15 9/25 10/5 10/15 10/25 11/4 11/14 11/24 12/4 12/14

dates

wrc

Goal Line

Intervention 1 data pts

Linear (Intervention 1 data pts)

28

Data-Based Decision Making

• Were there 4 consecutive data points below the goal line?

• Are there 8 data points for a trend line?

• Is the trend line on course to meet the goal line?

• Is this intervention effective?

• Decision?

29

Intervention #2 Data Points10/14/2006 17

10/16/2006 19

10/21/2006 21

10/23/2006 26

10/28/2006 28

10/30/2006 26

11/4/2006 30

11/6/2006 32

Enter this data on your graph and draw a trend line.

30

Tom's Oral Reading Fluency

0

5

10

15

20

25

30

35

40

9/5 9/15 9/25 10/5 10/15 10/25 11/4 11/14 11/24 12/4 12/14

dates

wrc Goal Line

Intervention 1 data pts

Intervention 2 data pts.

Linear (Intervention 1 data pts)

Linear (Intervention 2 data pts.)

Intervention 1Intervention 2

31

Data-Based Decision Making• Draw a vertical line to show each

intervention change.

• Label “Intervention 1”, “Intervention 2”, etc.

• Keep records of specific intervention protocols and fidelity of implementation.

• What is your decision based on Intervention 2 data?

32

Why is Fidelity Important?

• To demonstrate that measurable changes in behavior are related to systematic & controlled changes in the environment (intervention)

• Without objective & documented evidence that the intervention was implemented as planned, we can’t conclude that inadequate response to intervention was due to a poor intervention or insufficient intensity (ie: inadequate response may be due to poor instruction.)

• Likewise, success can’t be attributed to the intervention if we don’t know how it was implemented

33

3rd Grade Winter R-CBM – Mia - Baseline

Tues. 34 WRC Wed. 40 Thurs. 36

Enter the data.

Mark MEDIAN with X PM 5

34

Graphing: Make Mia’s Goal Line

Set a Reasonable but Ambitious Goal

What is the average Rate of Improvement (ROI) for third grade? – norms – PM 4

Would a gain of 15 words per week be ambitious? Reasonable?

Would a gain of 2 words per week be ambitions? Reasonable?

Calculate Mia’s goal at the end of 12 weeks.

Graph the GOAL LINE (Aim Line)

35

Mia’s Problem Statement

“When given a third grade oral reading fluency probe, Mia reads 36 words correctly in one minute. Her average third grade peer reads 98 words correctly in one minute.”

Intervention: Repeated Readings 2:30 – 2:50 M/W/F

Implemented by: Classroom teacherProgress Monitor: T/Th R-CBM (3rd)

36

Mia’s Progress Monitoring Data

Mia’s R-CBM scores (Tues./Thurs.)

Week 1: 34, 36Week 2: 42, 36Week 3: 36, 38Week 4: 38, 38

Graph the data.

PM 5

37

Make a Trend Line

Use Mia’s data: Weeks 1-4

(Don’t include the baseline data)

Use the Tukey Method to draw a trend line.

Compare Mia’s Trend Line & Goal Line.

What’s your decision about the intervention?

38

To Monitor Student Progress• Determine student’s current level (Baseline)• Identify learning Goal (local/research norms)• Research based Intervention(s) target

problem• Implement with Fidelity• Continue to measure performance on a

regular basis (CBM probes at same level)• Graph the results• Compare expected progress to actual rate• Adjust instruction based on the data

39

Where Do We Go From Here? RtI / SLD Eligibility Determination?

Students who score below the 25th percentile on general outcome benchmark screening receive targeted intervention, and progress is monitored on a monthly basis. Progress for those below the 10th percentile is monitored at least weekly, and they receive intensive intervention.

Eight data points over at least 4 weeks are required to determine a trend line.

Change or modify the intervention when the data points are below the aim line (goal line) for 4 consecutive data points or when the trend line is not on course to meet the goal line. At least 2 intervention changes (ie: 3 interventions) are required before students may be referred for evaluation due to suspected disability.

40

Special Education Regulations Apply at Time of Referral

• Letter to parent informing of referral• Evaluation decisions made by the Evaluation

Team at the Review of Existing Data/Evaluation Plan Meeting.

• Parents are given written notice and must sign consent for the evaluation. “Specialized Instruction” is listed under “other” in the area of functioning (Ex: “Specialized Instruction, 225 minutes per week, special ed resource room.”)

• Specialized instruction must be very focused and targeted at the problem as operationally defined in the problem statement.

41

• CBM is used to collect data in all academic areas, administered frequently (2X/wk)

• The Evaluation must be comprehensive and address all areas of concern.

• Consider norm referenced achievement assessment in addition to CBM (if not indicated in the student’s records). Level of performance must be far below & rate of improvement far below average peers.

• Use cognitive assessment if broad & pervasive concerns with student’s functioning across areas.

• However, do not calculate a discrepancy between IQ and norm referenced achievement scores. This information is not relevant to determining eligibility or interventions.

See Handouts PM 7 & PM 8 for more specific procedues.

42

Progress Monitoring & IEP Goals

Formula for Good Goals Using CBMGiven a (specific grade level and subject

area) probe,STUDENT will (increase/decrease/maintain)

his/her ability to (state skill addressed) by (observable behavior)

from (insert baseline)to (insert goal)

for (insert monitoring period)

43

References• Deno, S. (2003). Developments in Curriculum-

Based Measurement. Journal of Special Education (37) (3), 184-192.

• Fuchs, L. & Fuchs, D. (2002). Curriculum-based measurement: Describing competence, enhancing outcomes, evaluating treatment effects, and identifying treatment nonresponders. Peabody Journal of Education, 77, 64-84.

• Hosp, M. & Hosp, J. (2003). Curriculum-based measurement for reading, math, and spelling: How to do it and why. Preventing School Failure,

48 (1), 10-17.