Presented at OSEP Project Directors Meeting Washington, DC July, 2012 Child Outcomes for Early Intervention and Preschool Special Education: The National Picture Lynne Kahn ECO at FPG Kathy Hebbeler ECO at SRI
Feb 22, 2016
Presented at OSEP Project Directors Meeting
Washington, DCJuly, 2012
Child Outcomes for Early Intervention and Preschool Special
Education: The National Picture
Lynne KahnECO at FPG
Kathy HebbelerECO at SRI
Overview
• National numbers and the quality of the state data
• The findings for 2010-11 • Findings for the last 3 years• State variation and data quality
2Early Childhood Outcomes Center
3Early Childhood Outcomes Center
OSEP Reporting Requirements: Child Outcomes
• Positive social emotional skills (including positive social relationships)
• Acquisition and use of knowledge and skills (including early language/ communication [and early literacy])
• Use of appropriate behaviors to meet their needs
If you have not seen it….
4Early Childhood Outcomes Center
www.the-eco-center.org
State Approaches to Outcomes Data, 2010-11
Approach Part C (56 states/jurisdictions)
Preschool(59 states/jurisdictions)
COS* 7 pt. scale 43/56 (77%) 36/59 (61%)
One tool statewide 7/56 (13%) 9/59 (15%)
Publishers’ online analysis
3/56 (5%) 6/59 (10%)
Other 5/56 (9%) 8/59 (14%)
*Child Outcomes Summary Rating
• States are in various stages with regard to building their child outcomes measurement systems
• Some phrased in• Some changed approaches
6Early Childhood Outcomes Center
Child Outcomes: What we know
3 Methods Methods for Calculating National Estimates
1. States with the highest quality data weighted by child count to represent all states*
2. All states weighted by child count3. All states averaged (each state weighted
as 1)
7Early Childhood Outcomes Center
*The data we will be presenting for the national picture
Identifying the States with Highly Questionable Data
Criteria for eliminating states:• Not reporting data on enough children• Odd patterns in the data• Review of method reported suggested the
data were of questionable quality
8Early Childhood Outcomes Center
Continued increase in the number of states included
9Early Childhood Outcomes Center
08-09 09-10 10-11
Pt. C 19 29 39
Part B Preschool 15 33 36
Number of States that Met Criteria for Inclusion in the National Analysis
• And now the data….
10Early Childhood Outcomes Center
And now the data….
11Early Childhood Outcomes Center
OSEP Reporting Categories
Percentage of children who: a. Did not improve functioningb. Improved functioning, but not sufficient to move nearer
to functioning comparable to same-aged peers c. Improved functioning to a level nearer to same-aged
peers but did not reach itd. Improved functioning to reach a level comparable to
same-aged peerse. Maintained functioning at a level comparable to same-
aged peers
3 outcomes x 5 “measures” = 15 numbers
Illustration of 5 Possible Paths
0
10
20
30
40
50
60
70
1 6 11 16 21 26 31 36 41 46 51 56Age in Months
Sco
re
Maintained functioning comparable to age peersAchieved functioning comparable to age peersMoved nearer functioning comparable to age peersMade progress; no change in trajectoryDid not make progress
13*Based on 39 states with highest quality data
14*Based on 36 States with the highest quality data.
The Summary Statements
1. Of those children who entered the program below age expectations in each outcome, the percent who substantially increased their rate of growth by the time they turned 3 [6] years of age or exited the program.
2. The percent of children who were functioning within age expectations in each outcome by the time they turned 3 [6] years of age or exited the program.
15Early Childhood Outcomes Center
16*Based on 39 states with highest quality data
17*Based on 36 States with the highest quality data.
Part C: Trends over Time
18Early Childhood Outcomes Center
Part C: Trends over Time
19Early Childhood Outcomes Center
Part B Preschool: Trends over Time
20Early Childhood Outcomes Center
21Early Childhood Outcomes Center
Part B Preschool: Trends over Time
Looking for Patterns
22Early Childhood Outcomes Center
Early Childhood Outcomes Center 23
24Early Childhood Outcomes Center
%
Early Childhood Outcomes Center 25
Building the national capacity to report and use child outcomes data is a
long term endeavor.
26Early Childhood Outcomes Center
Part C: States continue to report on more children
Total Number of Children States Included in the Child Outcomes Data
Number of children reported
Number of States and Jurisdictions
2007-08(N=56)
2008-09(N=56)
2009-10(N=54)
2010-11(N=54)
99 or less 13 6 5 4
100-499 25 16 9 10
500-999 6 13 10 7
1,000-1,999 9 11 10 13
2,000+ 3 10 20 22
27Early Childhood Outcomes Center
28Early Childhood Outcomes Center
Early Childhood Outcomes Center 29
Total Number of Children Included in Progress Data
Number of children reported
Number of States and Jurisdictions
FFY 2007(N=58)
FFY 2008(N=58)
FFY 2009(N=58)
FFY 2010 (N=59)
10 or fewer 1 1 1 1
10-99 11 7 6 7
100–499 14 6 4 2
500-999 10 12 10 8
1000–1999 8 7 9 5
2000–2999 5 10 11 12
3000–4999 5 11 9 13
5000-8999 3 3 7 9
9000+ 1 1 1 2
Part B Preschool: States continue to report on more children
Early Childhood Outcomes Center 30
31Early Childhood Outcomes Center
%
Early Childhood Outcomes Center 32
Conclusions
• The data show good outcomes for many children who attend Part C and Part B, Preschool.
• The data quality continues to get better.• The data will be a powerful tool for
program improvement when– The data are consistently good at the local level– States and local programs consistently use it.
33Early Childhood Outcomes Center
For more information….
www.the-eco-center.org
34Early Childhood Outcomes Center
35Early Childhood Outcomes Center
Identifying States with Quality Data
36Early Childhood Outcomes Center
Calculating Missing Data for Part C
Proxy for missing data =Number with data for C3/
Exiting Data (618)
37Early Childhood Outcomes Center
• Do not expect this number to be 100%
• ..but we don’t expect it to be 10% either
Percentage of Exiters Reported by Part C Continues to Increase
38Early Childhood Outcomes Center
Calculating Missing Data for 619
Proxy for missing data =Number with data for B7/
Child count
39Early Childhood Outcomes Center
• Do not expect this number to be 100%
• ..but we don’t expect it to be 10% either
Percentage of Child Count Reported by Part B Preschool Continues to Increase
40Early Childhood Outcomes Center
08-09 09-10 10-1102468
1012141618
Pt. B Preschool: % of Child Count Reported in Child Outcomes Data
<10%
10-20%
21-30%
31-40%
41-50%
>50%Num
ber o
f Sta
tes
Many states reporting a low percentage for “a” but still have a few outliers.
Part CPart B
Preschool<2% 32 302 to <5% 11 115 to <7% 5 3
7 to <13% 0 5
28 to 49% 3 0
41Early Childhood Outcomes Center
Number of States by % Reported in “a,” 2010-11
Early Childhood Outcomes Center 42
Reason Part C state was excludedNumber
excludedPercent(n=51)
State is sampling 3 6No outcomes data reported 0 0Reported outcomes data on less than 28% of reported exiters
3 6
Had at least one outcome with category a greater than 10% or category e greater than 65%
4 8
Reported outcomes data on less than 28% of reported exitersANDHad at least one outcome with category a greater than 10% or category e greater than 65%
1 2
Questionable data quality based on review of SPP/APR and knowledge gained through TA
1 2
States included in the analysis 39 76
Early Childhood Outcomes Center 43
Reason Part B state was excludedNumber
excludedPercent (n=51)
State is sampling 4 8No outcomes data reported 1 2Reported outcomes data on less than 12% of child count 2 4
Had at least one outcome with category a greater than 10% or category e greater than 65% 4 8
Reported outcomes data on less than 12% of child countANDHad at least one outcome with category a greater than 10% or category e greater than 65%
0 0
Questionable data quality based on review of SPP/APR and knowledge gained through TA 2 4
No child count data available 1 2Reported Summary statements but no progress categories 1 2
States included in the analysis 36 71
Producing National Estimates Using the States with the Highest Quality Data
1. Divide the states into those with large, medium, and small child counts.
2. Identify the states that appear to have the best data.
3. Weight by child count and calculate the overall OSEP percentages using the “best data” states in each category.
4. Use the OSEP percents and total child count for each category to get the national estimates.
44Early Childhood Outcomes Center