Predictive Validity Evidence for DIBELS: Correlation to WASL Reading Scores Jack B. Monpas-Huber, Ph.D. Director of Assessment & Student Information.

Predictive Validity Evidence for DIBELS:Correlation to WASL Reading Scores

Jack B. Monpas-Huber, Ph.D.Director of Assessment & Student Information

Validity Evidence for DIBELS

Primary purpose of universal screeners (like DIBELS, easyCBM) is to predict future performance

This prompts a search for evidence of predictive validity, which means correlations with criterion measures (such as state assessments)

Previous studies have found strong correlations between DIBELS and state assessments:

Following are correlations between DIBELS ORF and WASL scores (Grades 3-6) based on 2008-09 Shoreline data

Study State Grade CorrelationsWilson, 2005 Arizona 3 0.74Vander Meer, Lentz, & Stollar, 2005 Ohio 4 0.65Shaw & Shaw, 2002 Colorado 3 0.73Buck & Torgeson, 2003 Florida 3 0.73Barger, 2003 North Carolina 3 0.73Good, Simmons, and Kame'enui, 2001 Oregon 3 0.67Stage & Jacobsen, 2001 Washington 4 0.44

Met WASL Reading Standard At Risk Some Risk Low Risk Total

No 45 24 21 9061.6% 25.3% 5.2% 15.8%

Yes 28 71 380 47938.4% 74.7% 94.8% 84.2%

Total 73 95 401 569100.0% 100.0% 100.0% 100.0%

Fall ORF Score

DIBELS ORF and WASL Reading Standard2008-09 Grade 3

Nearly all students who meet fall benchmark later met state proficiency standard

Students who score below fall benchmark at some risk of not meeting state proficiency standard

DIBELS ORF and WASL Reading Standard2008-09 Grade 4

Met WASL Reading Standard At Risk Some Risk Low Risk TotalNo 37 18 19 74

52.1% 25.0% 5.0% 14.2%Yes 34 54 358 446

47.9% 75.0% 95.0% 85.8%Total 71 72 377 520

100.0% 100.0% 100.0% 100.0%

Fall ORF Benchmark

DIBELS ORF and WASL Reading Standard2008-09 Grade 5

Met WASL Reading Standard At Risk Some Risk Low Risk TotalNo 30 16 24 70

69.8% 31.4% 8.2% 18.0%Yes 13 35 270 318

30.2% 68.6% 91.8% 82.0%Total 43 51 294 388

100.0% 100.0% 100.0% 100.0%

Fall ORF Benchmark

DIBELS ORF and WASL Reading Standard2008-09 Grade 6

Met WASL Reading Standard At Risk Some Risk Low Risk TotalNo 23 20 33 76

85.2% 31.7% 8.2% 15.4%Yes 4 43 369 416

14.8% 68.3% 91.8% 84.6%Total 27 63 402 492

100.0% 100.0% 100.0% 100.0%

Fall ORF Benchmark

What is correlation? What’s a “good” one?

Correlation:…is a measure of how consistently two measures sort the same examinees…is a measure of linear relationship between two measures…ranges from zero (no relationship at all) to one (perfect prediction)

Typical correlation between WASL scores is in this range

A stronger correlation is evidence that two measures are measuring the same thing. We’d expect this of two sets of WASL reading scores.

A strong correlation of two different reading measures (WASL and DIBELS) is evidence that both are measuring a lot of the same thing (“reading”) even though each is measuring different reading skills

DIBELS-WASL correlations in this range

Grade N r3 574 0.5714 537 0.5315 418 0.5866 512 0.563

Highest ORF Score

DIBELS ORF and WASL Reading StandardCorrelations

Shoreline Public Schools - 2008-09

R2 = 0.3263r = 0.571

200

250

300

350

400

450

500

550

600

0 25 50 75 100 125 150 175 200 225 250

2008-09 Grade 3 Highest Oral Reading Fluency Score

2009 Gr3 WASL Reading Scale Score

DIBELS ORF and WASL Reading Standard2008-09 Grade 3

What is the ORF threshold for meeting WASL standard?

DIBELS ORF and WASL Reading Standard2008-09 Grade 4

R2 = 0.2818

200

250

300

350

400

450

500

550

600

0 25 50 75 100 125 150 175 200 225 250 275 300

2009 Grade 4 - Highest ORF Score

2009 Grade 4 WASL Reading Scale Score

DIBELS ORF and WASL Reading Standard2008-09 Grade 5

R2 = 0.3436

200

250

300

350

400

450

500

550

600

0 25 50 75 100 125 150 175 200 225 250

2008-09 Highest ORF Score

2009 Grade 5 WASL Reading Scale Score

DIBELS ORF and WASL Reading Standard2008-09 Grade 6

R2 = 0.3167

200

250

300

350

400

450

500

550

600

0 25 50 75 100 125 150 175 200 225 250

2008-09 Gr6 Highest ORF Score

2008-09 Gr6 WASL Reading Scale Score

-0.10 0.45 1.00 1.55 2.10-10.30

46.35

103.00

159.65

216.30

TIME

ORF

TITLE = 0

TITLE = 1

What’s Working?: DIBELS ORF Growth Curves to Study Effectiveness of Reading Programs/Interventions

Fall Winter Spring

Do students in Title schools grow faster? Slower?

Does poverty and/or reading instruction predict faster growth (controlling for where kids start)?

What does predict rate of learning?

2008-09 Third Grade Cohort (random sample)

-0.10 0.45 1.00 1.55 2.101.95

57.23

112.50

167.78

223.05

TIME

ORF

TITLE = 0

TITLE = 1

What’s Working?: DIBELS ORF Growth Curves to Study Effectiveness of Reading Programs/Interventions

What about missing data?

It limits our view of kids’ learning.

Must statistically “impute” probable growth

Fall Winter Spring

2008-09 Third Grade Cohort (random sample)

-0.10 0.45 1.00 1.55 2.1016.95

72.23

127.50

182.78

238.05

TIME

ORF

TITLE = 0

TITLE = 1

What’s Working?: DIBELS ORF Growth Curves to Study Effectiveness of Reading Programs/Interventions

Fall Winter Spring

2008-09 Third Grade Cohort (random sample)