Graphing, Graphing, Calculating, and Calculating, and Interpreting Rate of Interpreting Rate of Improvement Improvement Caitlin S. Flinn, Ed.S., Caitlin S. Flinn, Ed.S., N.C.S.P. N.C.S.P. Andrew E. McCrea, M.S., Andrew E. McCrea, M.S., N.C.S.P. N.C.S.P. NASP Convention NASP Convention March 3, 2010 March 3, 2010
115
Embed
Graphing, Calculating, and Interpreting Rate of Improvement Caitlin S. Flinn, Ed.S., N.C.S.P. Andrew E. McCrea, M.S., N.C.S.P. NASP Convention March 3,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Graphing, Calculating, and Graphing, Calculating, and Interpreting Rate of Interpreting Rate of
ImprovementImprovement
Caitlin S. Flinn, Ed.S., N.C.S.P.Caitlin S. Flinn, Ed.S., N.C.S.P.
Andrew E. McCrea, M.S., N.C.S.P.Andrew E. McCrea, M.S., N.C.S.P.
NASP ConventionNASP Convention
March 3, 2010March 3, 2010
ObjectivesObjectives
There needs to be a standardized There needs to be a standardized procedure for calculating RoI procedure for calculating RoI
We’re proposing a method using Simple We’re proposing a method using Simple Linear RegressionLinear Regression
OverviewOverview
Importance of RoIImportance of RoI RoI ResearchRoI Research A Need for A Need for
ConsistencyConsistency Calculating RoICalculating RoI
Decision MakingDecision Making Grounding the DataGrounding the Data Interpreting GrowthInterpreting Growth
Individual StudentIndividual Student Student GroupsStudent Groups
ConsiderationsConsiderations ResourcesResources
Importance of GraphsImportance of Graphs
Vogel, Dickson, & Lehman, 1990Vogel, Dickson, & Lehman, 1990Speeches that included visuals, especially in Speeches that included visuals, especially in
color, improved:color, improved:Immediate recall by 8.5%Immediate recall by 8.5%Delayed recall (3 days) by 10.1%Delayed recall (3 days) by 10.1%
Importance of GraphsImportance of Graphs
““Seeing is believing.”Seeing is believing.”Useful for communicating large amounts Useful for communicating large amounts
of information quicklyof information quickly ““A picture is worth a thousand words.”A picture is worth a thousand words.”Transcends language barriers (Karwowski, Transcends language barriers (Karwowski,
2006)2006)Responsibility for accurate graphical Responsibility for accurate graphical
representations of datarepresentations of data
Skills Typically GraphedSkills Typically Graphed ReadingReading
Oral Reading Fluency (ORF)Oral Reading Fluency (ORF) Word Use Fluency (WUF)Word Use Fluency (WUF) Reading ComprehensionReading Comprehension
(PSF)(PSF) Nonsense Word Fluency (NWF)Nonsense Word Fluency (NWF)
SpellingSpelling Written Expression Written Expression BehaviorBehavior
MathMath Math ComputationMath Computation Math FactsMath Facts Early NumeracyEarly Numeracy
Oral CountingOral Counting Missing NumberMissing Number Number IdentificationNumber Identification Quantity Quantity
DiscriminationDiscrimination
Importance of RoIImportance of RoI
Multi-tiered modelMulti-tiered modelProgress monitoringProgress monitoringData for decision-makingData for decision-makingGoal setting (Shapiro, 2008)Goal setting (Shapiro, 2008)
Importance of RoIImportance of RoI
Visual inspection of slopeVisual inspection of slopeMultiple interpretationsMultiple interpretations Instructional servicesInstructional servicesNeed for explicit guidelinesNeed for explicit guidelines
General outcome measuresGeneral outcome measuresShortShortStandardizedStandardizedRepeatableRepeatableSensitive to changeSensitive to change
RoI ResearchRoI Research
Fuchs & Fuchs, 1998Fuchs & Fuchs, 1998Hallmark components of Response to Hallmark components of Response to
InterventionInterventionOngoing formative assessmentOngoing formative assessmentIdentifying non-responding studentsIdentifying non-responding studentsTreatment fidelity of instructionTreatment fidelity of instruction
Dual discrepancy modelDual discrepancy modelOne standard deviation from typically performing One standard deviation from typically performing
peers in level and peers in level and raterate
RoI ResearchRoI Research
Ardoin & Christ, 2008Ardoin & Christ, 2008Slope for benchmarks (3x per year)Slope for benchmarks (3x per year)More growth from fall to winter than winter to More growth from fall to winter than winter to
springspringMight be helpful to use RoI for fall to winterMight be helpful to use RoI for fall to winterAnd a separate RoI for winter to springAnd a separate RoI for winter to spring
1.5 to 2.0 times typical slope to close gap1.5 to 2.0 times typical slope to close gap
ExampleExampleBob is below benchmark on ORFBob is below benchmark on ORFTypical slope is 1 wcpm per week growthTypical slope is 1 wcpm per week growthBob would need slope of 1.5 to 2 to close gap Bob would need slope of 1.5 to 2 to close gap
in a reasonable amount of timein a reasonable amount of time
RoI ResearchRoI Research
Deno, Fuchs, Marston, & Shin, 2001Deno, Fuchs, Marston, & Shin, 2001Slope of frequently non-responsive children Slope of frequently non-responsive children
approximated slope of children already approximated slope of children already identified as having a specific learning identified as having a specific learning disabilitydisability
RoI ResearchRoI Research
Algebraic term: Slope of a lineAlgebraic term: Slope of a lineVertical change over the horizontal changeVertical change over the horizontal changeRise over runRise over runm = (ym = (y2 2 - y- y11) / (x) / (x2 2 - x- x11))Describes the steepness of a line (Gall & Gall, Describes the steepness of a line (Gall & Gall,
2007)2007)
RoI ResearchRoI Research
Finding a student’s RoI = finding the slope Finding a student’s RoI = finding the slope of a lineof a lineUsing two data points on that lineUsing two data points on that line
Finding the line itselfFinding the line itselfLinear regressionLinear regressionOrdinary Least SquaresOrdinary Least Squares
RoI ResearchRoI Research
Gall & Gall, 2007Gall & Gall, 200710 data points are a minimum requirement for 10 data points are a minimum requirement for
a reliable trendlinea reliable trendlineHow does that affect the frequency of How does that affect the frequency of
Using RoI for instructional decisions is not Using RoI for instructional decisions is not a perfect processa perfect process
Research is currently looking to address Research is currently looking to address sources of error:sources of error:Christ, 2006 – standard error of measurement Christ, 2006 – standard error of measurement
for slopefor slopeArdoin & Christ, 2009 – passage difficulty and Ardoin & Christ, 2009 – passage difficulty and
variabilityvariabilityJenkin, Graff, & Miglioretti, 2009 – frequency Jenkin, Graff, & Miglioretti, 2009 – frequency
of progress monitoringof progress monitoring
RoI ResearchRoI Research
Questions yet to be empirically answeredQuestions yet to be empirically answeredWhat parameters of RoI indicate a lack of RtI?What parameters of RoI indicate a lack of RtI?How does standard error of measurement How does standard error of measurement
play into using RoI for instructional decision play into using RoI for instructional decision making?making?
How does RoI vary between standard protocol How does RoI vary between standard protocol interventions?interventions?
How does this apply to non-English speaking How does this apply to non-English speaking populations?populations?
How is RoI Calculated? How is RoI Calculated? Which way is best?Which way is best?
Multiple Methods for Multiple Methods for Calculating GrowthCalculating Growth
““Eye ball” ApproachEye ball” Approach Last point minus First point ApproachLast point minus First point Approach Split Middle ApproachSplit Middle Approach Linear Regression ApproachLinear Regression Approach
Eye Ball
8
10
7
17
14
11
19
14
0
2
4
6
8
10
12
14
16
18
20
1 2 3 4 5 6 7 8
Last minus First
8
10
7
17
14
11
19
14
0
2
4
6
8
10
12
14
16
18
20
1 2 3 4 5 6 7 8
14 - 8 = 6; 6/ 8 weeks = 0.75 words per week
Split Middle
10.5
14.5
0
2
4
6
8
10
12
14
16
18
1 2 3 4 5 6 7 8 9 10
14.5-10.5 = 4; 4/8 weeks = 0.5 words per week
Linear Regression
8
10
7
17
14
11
19
14
y = 1.1429x + 7.3571
0
2
4
6
8
10
12
14
16
18
20
1 2 3 4 5 6 7 8
1.1 Words Per Week
RoI Consistency?RoI Consistency?
Eye BallEye Ball ??????
Last minus FirstLast minus First 0.750.75
Split Middle*Split Middle* 0.500.50
Linear Linear RegressionRegression
1.101.10
RoI Consistency?RoI Consistency?
If we are not all using the same model to If we are not all using the same model to compute RoI, we continue to have the same compute RoI, we continue to have the same problems as past models, where under one problems as past models, where under one approach a student meets SLD criteria, but approach a student meets SLD criteria, but under a different approach, the student does under a different approach, the student does not. not.
Hypothetically, if the RoI cut-off was 0.65 or Hypothetically, if the RoI cut-off was 0.65 or 0.95, different approaches would come to 0.95, different approaches would come to different conclusions on the same student. different conclusions on the same student.
Technical AdequacyTechnical Adequacy
Without a consensus on how to compute Without a consensus on how to compute RoI, we risk falling short of having RoI, we risk falling short of having technical adequacy within our model. technical adequacy within our model.
So, Which RoI Method is Best?So, Which RoI Method is Best?
Literature shows that Linear Literature shows that Linear Regression is Best PracticeRegression is Best Practice
Student’s daily test scores…were entered into a Student’s daily test scores…were entered into a computer program…The data analysis program computer program…The data analysis program generated generated slopes of improvementslopes of improvement for each level for each level using and using and Ordinary-Least SquaresOrdinary-Least Squares procedure procedure (Hayes, 1973) and the line of best fit. (Hayes, 1973) and the line of best fit.
This This procedure has been demonstrated to procedure has been demonstrated to represent CBM achievement data validly within represent CBM achievement data validly within individual treatment phasesindividual treatment phases (Marston, 1988; (Marston, 1988; Shinn, Good, & Stein, in press; Stein, 1987).Shinn, Good, & Stein, in press; Stein, 1987).
Growth (RoI) Research Growth (RoI) Research using Linear Regressionusing Linear Regression
Christ, T. J. (2006). Short-term estimates of growth using Christ, T. J. (2006). Short-term estimates of growth using curriculum based measurement of oral reading fluency: curriculum based measurement of oral reading fluency: Estimating standard error of the slope to construct confidence Estimating standard error of the slope to construct confidence intervals. intervals. School Psychology ReviewSchool Psychology Review, 35, 128-133., 35, 128-133.
Deno, S. L., Fuchs, L. S., Marston, D., & Shin, J. (2001). Using Deno, S. L., Fuchs, L. S., Marston, D., & Shin, J. (2001). Using curriculum based measurement to establish growth standards curriculum based measurement to establish growth standards for students with learning disabilities. for students with learning disabilities. School Psychology School Psychology ReviewReview, 30, 507-524., 30, 507-524.
Good, R. H. (1990). Forecasting accuracy of slope estimates for Good, R. H. (1990). Forecasting accuracy of slope estimates for reading curriculum based measurement: Empirical evidence. reading curriculum based measurement: Empirical evidence. Behavioral AssessmentBehavioral Assessment, 12, 179-193., 12, 179-193.
Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L. & Germann, G. Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L. & Germann, G. (1993). Formative evaluation of academic progress: How much (1993). Formative evaluation of academic progress: How much growth can we expect? growth can we expect? School Psychology ReviewSchool Psychology Review, 22, 27-48., 22, 27-48.
Growth (RoI) ResearchGrowth (RoI) Researchusing Linear Regressionusing Linear Regression
Jenkins, J. R., Graff, J. J., & Miglioretti, D.L. (2009). Jenkins, J. R., Graff, J. J., & Miglioretti, D.L. (2009). Estimating reading growth using intermittent CBM Estimating reading growth using intermittent CBM progress monitoring. progress monitoring. Exceptional ChildrenExceptional Children, 75, 151-163., 75, 151-163.
Shinn, M. R., Gleason, M. M., & Tindal, G. (1989). Shinn, M. R., Gleason, M. M., & Tindal, G. (1989). Varying the difficulty of testing materials: Implications for Varying the difficulty of testing materials: Implications for curriculum-based measurement. curriculum-based measurement. The Journal of Special The Journal of Special EducationEducation, 23, 223-233., 23, 223-233.
Shinn, M. R., Good, R. H., & Stein, S. (1989). Shinn, M. R., Good, R. H., & Stein, S. (1989). Summarizing trend in student achievement: A Summarizing trend in student achievement: A comparison of methods. comparison of methods. School Psychology ReviewSchool Psychology Review, 18, , 18, 356-370.356-370.
So, Why Are There So Many So, Why Are There So Many Other RoI Models?Other RoI Models?
Ease of application Ease of application How many of us want to calculate OLS How many of us want to calculate OLS
Linear Regression formulas (or even Linear Regression formulas (or even remember how)?remember how)?
An Easy and An Easy and Applicable SolutionApplicable Solution
Get Out Your Laptops!Get Out Your Laptops!
Or Kindly Look Over Your Or Kindly Look Over Your Neighbor’s Shoulder!Neighbor’s Shoulder!
I loveROI
Open Microsoft ExcelOpen Microsoft Excel
Microsoft Office 2003 for PCsMicrosoft Office 2003 for PCsMicrosoft Office 2007 for PCsMicrosoft Office 2007 for PCsMicrosoft Office for MacsMicrosoft Office for Macs
Graphing RoIGraphing RoI
For Individual StudentsFor Individual Students
Setting Up Your SpreadsheetSetting Up Your Spreadsheet
In cell B2, type In cell B2, type School WeekSchool Week In cell C2, type In cell C2, type BenchmarkBenchmark In cell D2, type WPM (or Student Scores)In cell D2, type WPM (or Student Scores)
Labeling School WeeksLabeling School Weeks
In cell B3, type In cell B3, type 11 Continue entering numbersContinue entering numbers
through through 3636 in column B in column BWeek Week 3636 will be in cell B38 will be in cell B38
Entering BenchmarksEntering Benchmarks
In cell C3, type the fall benchmark In cell C3, type the fall benchmark 7777 In cell C20, type the winter benchmark In cell C20, type the winter benchmark 9292 In cell C38, type the spring benchmark In cell C38, type the spring benchmark
110110
Entering Student Data PointsEntering Student Data Points
Student data points are entered between Student data points are entered between cells D3 and D38.cells D3 and D38.
Type the student’s score next to the Type the student’s score next to the corresponding week that it was corresponding week that it was administered.administered.
Entering Student Data PointsEntering Student Data Points
If a student was not assessed during a If a student was not assessed during a certain week, leave that cell blankcertain week, leave that cell blank
Do not enter a score of Zero (0) it will be Do not enter a score of Zero (0) it will be calculated into the trendline and calculated into the trendline and interpreted as the student having read interpreted as the student having read zero words correct per minute during that zero words correct per minute during that week. week.
Creating a GraphCreating a Graph
Highlight the Highlight the data in Columnsdata in ColumnsC and DC and D Include cells Include cells C2 and D2 through C2 and D2 through cells C38 and D38cells C38 and D38 Include any Include any blank cellsblank cells
Excel 2003/MacsExcel 2003/MacsChoose Choose CustomCustom and check box next to and check box next to
Display equation on chartDisplay equation on chart
Excel 2007Excel 2007
Adding a TrendlineAdding a Trendline
Clicking on the equation highlights a box Clicking on the equation highlights a box around itaround it
Clicking on the box allows you to move it Clicking on the box allows you to move it to a place where you can see it betterto a place where you can see it better
Adding a TrendlineAdding a Trendline
You can repeat the same procedure to You can repeat the same procedure to have a trendline for the benchmark data have a trendline for the benchmark data pointspoints
Suggestion: label the trendline Suggestion: label the trendline Expected Expected ROIROI
Move this equation under the firstMove this equation under the first
Individual Student GraphIndividual Student Graph
Diego's Rate of Improvement
y = 1.6317x + 50.928
y = 0.9434x + 75.704
0
20
40
60
80
100
120
1 4 7 10 13 16 19 22 25 28 31 34
School Week
Wo
rds
Per
Min
ute
Benchmark (3rd)
Diego's Scores (3rd)
Diego's ROI
Goal Slope
Individual Student GraphIndividual Student Graph
The equation indicates the slope, or rate of The equation indicates the slope, or rate of improvement. improvement.
The number, or coefficient,The number, or coefficient, beforebefore "x" is "x" is the average improvement, which in this the average improvement, which in this case is the average number of words per case is the average number of words per minute per week gained by the student. minute per week gained by the student.
Individual Student GraphIndividual Student Graph
The rate of improvement, or trendline, is The rate of improvement, or trendline, is calculated using a linear regression, a calculated using a linear regression, a simple equation of least squares. simple equation of least squares.
To add additional progress To add additional progress monitoring/benchmark scores once you’ve monitoring/benchmark scores once you’ve already created a graph, enter additional already created a graph, enter additional scores in Column D in the corresponding scores in Column D in the corresponding school week.school week.
Individual Student GraphIndividual Student Graph
Remember to leave cells blank for the Remember to leave cells blank for the weeks that no score was obtained. weeks that no score was obtained.
The graph will incorporate that score into The graph will incorporate that score into the set of data points and into the the set of data points and into the trendline.trendline.
Individual Student GraphIndividual Student Graph
The slope can change depending on The slope can change depending on which week (where) you put the which week (where) you put the benchmark scores on your chart. benchmark scores on your chart.
Enter benchmark scores based on when Enter benchmark scores based on when your school administers their benchmark your school administers their benchmark assessments for the most accurate assessments for the most accurate depiction of expected student progress.depiction of expected student progress.
Type Type RoIRoI in cell B39 below the last week in cell B39 below the last week of schoolof school
Calculate Expected SlopeCalculate Expected Slope
Click on cell C39Click on cell C39Put your cursor at the top next to the Put your cursor at the top next to the fxfxType =SLOPE(C3:C38,B3:B38)Type =SLOPE(C3:C38,B3:B38)Hit Enter/ReturnHit Enter/Return
Calculate Actual SlopeCalculate Actual Slope
Click on cell D39Click on cell D39Put your cursor at the top next to the Put your cursor at the top next to the fxfxType =SLOPE(D3:D38,B3:B38)Type =SLOPE(D3:D38,B3:B38)Hit Enter/ReturnHit Enter/Return
ROI as a Decision ToolROI as a Decision Tool
within a Problem-Solving Modelwithin a Problem-Solving Model
StepsSteps
1.1. Gather the dataGather the data
2.2. Ground the dataGround the data
3.3. Interpret the dataInterpret the data
4.4. Figure out how to fit Best Practice into Figure out how to fit Best Practice into Public EducationPublic Education
*Best Practices in Setting Progress Monitoring Goals for Academic Skill Improvement (Shapiro, 2008)*Best Practices in Setting Progress Monitoring Goals for Academic Skill Improvement (Shapiro, 2008)
Needed GrowthNeeded Growth
Difference between student’s BOY (or Difference between student’s BOY (or MOY) score and benchmark score at MOY MOY) score and benchmark score at MOY (or EOY).(or EOY).
Example: MOY ORF = 10, EOY Example: MOY ORF = 10, EOY benchmark is 40, 18 weeks of instruction benchmark is 40, 18 weeks of instruction (40-10/18=1.67). Student must gain 1.67 (40-10/18=1.67). Student must gain 1.67 wcpm per week to make EOY benchmark.wcpm per week to make EOY benchmark.
Expected GrowthExpected Growth
Difference between two benchmarks.Difference between two benchmarks.Example: MOY benchmark is 20, EOY Example: MOY benchmark is 20, EOY
benchmark is 40, expected growth (40-benchmark is 40, expected growth (40-20)/18 weeks of instruction = 1.11 wcpm 20)/18 weeks of instruction = 1.11 wcpm per week.per week.
Tigard-Tualatin School District (www.ttsd.k12.or.us)Tigard-Tualatin School District (www.ttsd.k12.or.us)
Looking at Percent of Looking at Percent of Expected GrowthExpected Growth
Tier ITier I Tier IITier II Tier IIITier III
Greater than Greater than 150%150%
Between Between 110% & 150%110% & 150%
Possible LDPossible LD
Between 95% Between 95% & 110%& 110%
Likely LDLikely LD
Between 80% Between 80% & 95%& 95%
May Need May Need MoreMore
May Need May Need MoreMore
Likely LDLikely LD
Below 80%Below 80% Needs MoreNeeds More Needs MoreNeeds More Likely LDLikely LD
Making Decisions: Best Practice Making Decisions: Best Practice
Research has yet to establish a blue print Research has yet to establish a blue print for ‘grounding’ student RoI data. for ‘grounding’ student RoI data.
At this point, teams should consider At this point, teams should consider multiple comparisons when planning and multiple comparisons when planning and making decisions. making decisions.
Making Decisions: Lessons Making Decisions: Lessons From the FieldFrom the Field
When tracking on grade level, consider an When tracking on grade level, consider an RoI that is 100% of RoI that is 100% of expectedexpected growth as a growth as a minimum requirement, consider an RoI minimum requirement, consider an RoI that is at or above the that is at or above the neededneeded as optimal. as optimal.
So, 100% of expected and on par with So, 100% of expected and on par with needed become the limits of the range needed become the limits of the range within a student should be achieving.within a student should be achieving.
What do we do when we do not What do we do when we do not get the growth we want?get the growth we want?
When to make a change in instruction and When to make a change in instruction and intervention?intervention?
When to consider SLD?When to consider SLD?
When to make a change in When to make a change in instruction and intervention?instruction and intervention?
Enough data points (6 to 10)?Enough data points (6 to 10)?Less than 100% of expected growth.Less than 100% of expected growth.Not on track to make benchmark (needed Not on track to make benchmark (needed
growth).growth).Not on track to reach individual goal.Not on track to reach individual goal.
When to consider SLD?When to consider SLD?
Continued inadequate response despite: Continued inadequate response despite: Fidelity with Tier I instruction and Tier Fidelity with Tier I instruction and Tier
II/III intervention.II/III intervention. Multiple attempts at intervention.Multiple attempts at intervention. Individualized Problem-Solving Individualized Problem-Solving
approach. approach.
Three Levels of ExamplesThree Levels of Examples
Whole ClassWhole ClassSmall GroupSmall Group Individual Student Individual Student
Making growth?Making growth?How much (65% of expected growth).How much (65% of expected growth).Atypical growth across the year (last 3 Atypical growth across the year (last 3
data points). data points). Continue? Make a change? Need more Continue? Make a change? Need more
data?data?
RoI and Behavior?RoI and Behavior?
Percent of Time Engaged in Appropriate Behavior
y = 2x + 22
y = 3.9x + 19.8
y = 7.2143x - 1.5
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Per
cen
t
Baseline Condition 1 Condition 2 Linear (Baseline) Linear (Condition 1) Linear (Condition 2) Linear (Condition 2)
Step 4: Figure out how to fit Step 4: Figure out how to fit Best Practice into Public Best Practice into Public
EducationEducation
Things to ConsiderThings to Consider
Who is At-Risk and needs progress Who is At-Risk and needs progress monitoring?monitoring?
Who will collect, score, enter the data?Who will collect, score, enter the data?Who will monitor student growth, when, Who will monitor student growth, when,
and how often?and how often?What changes should be made to What changes should be made to
instruction & intervention? instruction & intervention? What about monitoring off of grade level?What about monitoring off of grade level?
Who is At-Risk and needs Who is At-Risk and needs progress monitoring?progress monitoring?
Below level on universal screeningBelow level on universal screening
Entering 4Entering 4thth Grade Example Grade Example
DORF DORF (110)(110)
ISIP ISIP TRWM TRWM
(55)(55)
4Sight 4Sight (1235)(1235)
PSSA PSSA (1235)(1235)
Student AStudent A 115115 5858 12551255 12321232
Student BStudent B 8585 4848 12161216 11261126
Student CStudent C 7272 3535 10561056 10481048
Who will collect, score, and Who will collect, score, and enter the data?enter the data?
Using MBSP for math, teachers can Using MBSP for math, teachers can administer probes to whole class.administer probes to whole class.
DORF probes must be administered one-DORF probes must be administered one-on-one, and creativity pays off (train and on-one, and creativity pays off (train and use art, music, library, etc. specialists).use art, music, library, etc. specialists).
Schedule for progress monitoring math Schedule for progress monitoring math and reading every-other week.and reading every-other week.
Week 1Week 1 Week 2Week 2
ReadingReading MathMath ReadingReading MathMath
11stst XX XX
22ndnd XX XX
33rdrd XX XX
44thth XX XX
55thth XX XX
Who will monitor student Who will monitor student growth, when, and how often?growth, when, and how often?
Best Practices in Data-Analysis Teaming Best Practices in Data-Analysis Teaming (Kovaleski & Pedersen, 2008)(Kovaleski & Pedersen, 2008)
Chambersburg Area School District Elementary Chambersburg Area School District Elementary Response to Intervention Manual (McCrea et. Response to Intervention Manual (McCrea et. al., 2008)al., 2008)
Derry Township School District Response to Derry Township School District Response to Intervention Model Intervention Model (http://www.hershey.k12.pa.us/56039310111408/lib/56039310111408/_files/Microsoft_Word_-(http://www.hershey.k12.pa.us/56039310111408/lib/56039310111408/_files/Microsoft_Word_-_Response_to_Intervention_Overview_of_Hershey_Elementary_Model.pdf)_Response_to_Intervention_Overview_of_Hershey_Elementary_Model.pdf)
What changes should be made What changes should be made to instruction & intervention?to instruction & intervention?
Ensure treatment fidelity!!!!!!!!Ensure treatment fidelity!!!!!!!! Increase instructional time (active and Increase instructional time (active and
engaged)engaged)Decrease group sizeDecrease group sizeGather additional, diagnostic, informationGather additional, diagnostic, informationChange the intervention Change the intervention
When Instructional Level is Not When Instructional Level is Not the Same as Grade Levelthe Same as Grade Level
Understand needed and expected RoI Understand needed and expected RoI within broader context: within broader context:
Needed growthNeeded growth will only get student will only get student to nextto next level by next benchmark (as opposed to level by next benchmark (as opposed to on on levellevel).).
100% of expected growth100% of expected growth may not be an may not be an acceptable minimum (not enough growth b/c acceptable minimum (not enough growth b/c level is so low).level is so low).
Grounding RoI When Monitoring Grounding RoI When Monitoring Off of Grade Level: Two OptionsOff of Grade Level: Two OptionsBest Practices in Setting Progress Best Practices in Setting Progress
Monitoring Goals for Academic Skill Monitoring Goals for Academic Skill Improvement (Shapiro, 2008).Improvement (Shapiro, 2008).
Tigard-Tualatin SD Chart: 150% instead of Tigard-Tualatin SD Chart: 150% instead of 100% as minimum RoI requirement???100% as minimum RoI requirement???
Ardoin, S. P., & Christ, T. J. (2009). Curriculum-Ardoin, S. P., & Christ, T. J. (2009). Curriculum-based measurement of oral reading: Standard based measurement of oral reading: Standard errors associated with progress monitoring errors associated with progress monitoring outcomes from DIBELS, AIMSweb, and an outcomes from DIBELS, AIMSweb, and an experimental passage set. experimental passage set. School Psychology School Psychology Review, 38(2),Review, 38(2), 266-283. 266-283.
Ardoin, S. P. & Christ, T. J. (2008). Evaluating Ardoin, S. P. & Christ, T. J. (2008). Evaluating curriculum-based measurement slope estimates curriculum-based measurement slope estimates using triannual universal screenings. using triannual universal screenings. School School Psychology Review, 37Psychology Review, 37(1), 109-125. (1), 109-125.
ReferencesReferences
Christ, T. J. (2006). Short-term estimates of Christ, T. J. (2006). Short-term estimates of growth using curriculum-based measurement growth using curriculum-based measurement of oral reading fluency: Estimating standard of oral reading fluency: Estimating standard error of the slope to construct confidence error of the slope to construct confidence intervals. intervals. School Psychology Review, 35School Psychology Review, 35(1), (1), 128-133.128-133.
Deno, S. L. (1985). Curriculum-based Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. measurement: The emerging alternative. Exceptional Children, 52,Exceptional Children, 52, 219-232. 219-232.
ReferencesReferences
Deno, S. L., Fuchs, L.S., Marston, D., & Deno, S. L., Fuchs, L.S., Marston, D., & Shin, J. (2001). Using curriculum-based Shin, J. (2001). Using curriculum-based measurement to establish growth measurement to establish growth standards for students with learning standards for students with learning disabilities. disabilities. School Psychology Review, School Psychology Review, 3030, 507-524., 507-524.
Flinn, C. S. (2008). Graphing rate of Flinn, C. S. (2008). Graphing rate of improvement for individual students. improvement for individual students. InSight, 28(3InSight, 28(3), 10-12.), 10-12.
ReferencesReferences
Fuchs, L. S., & Fuchs, D. (1998). Treatment Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unifying concept for reconceptualizing validity: A unifying concept for reconceptualizing the identification of learning disabilities. the identification of learning disabilities. Learning Learning Disabilities Research and Practice, 13Disabilities Research and Practice, 13, 204-219., 204-219.
Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L., & Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L., & Germann, G. (1993). Formative evaluation of Germann, G. (1993). Formative evaluation of academic progress: How much growth can we academic progress: How much growth can we expect? expect? School Psychology Review, 22School Psychology Review, 22, 27-48., 27-48.
ReferencesReferences
Gall, M.D., & Gall, J.P. (2007). Gall, M.D., & Gall, J.P. (2007). Educational Educational research: An introductionresearch: An introduction (8th ed.). New (8th ed.). New York: Pearson.York: Pearson.
Jenkins, J. R., Graff, J. J., & Miglioretti, D.L. Jenkins, J. R., Graff, J. J., & Miglioretti, D.L. (2009). (2009). Estimating reading growth using Estimating reading growth using intermittent CBM progress monitoring. intermittent CBM progress monitoring. Exceptional Children, 75Exceptional Children, 75, 151-163., 151-163.
ReferencesReferences
Karwowski, W. (2006). Karwowski, W. (2006). International International encyclopedia of ergonomics and human encyclopedia of ergonomics and human factorsfactors. Boca Raton, FL: Taylor & Francis . Boca Raton, FL: Taylor & Francis Group, LLC.Group, LLC.
Shapiro, E. S. (2008). Best practices in setting Shapiro, E. S. (2008). Best practices in setting progress monitoring goals for academic skill progress monitoring goals for academic skill improvement. In A. Thomas and J. Grimes improvement. In A. Thomas and J. Grimes (Eds.), (Eds.), Best practices in school psychology V Best practices in school psychology V (Vol. 2, pp. 141-157). Bethesda, MD: National (Vol. 2, pp. 141-157). Bethesda, MD: National Association of School Psychologists.Association of School Psychologists.
ReferencesReferences
Vogel, D. R., Dickson, G. W., & Lehman, J. Vogel, D. R., Dickson, G. W., & Lehman, J. A. (1990). Persuasion and the role of A. (1990). Persuasion and the role of visual presentation support. The UM/3M visual presentation support. The UM/3M study. In M. Antonoff (Ed.), study. In M. Antonoff (Ed.), Presentations Presentations that persuade. Personal Computing, 14that persuade. Personal Computing, 14..