How to Explain the Numbers: Helping Staff, Parents, and Other Stakeholders Understand the Results of the NCSEAM Surveys for Part C and 619 Batya Elbaum, NCSEAM Pam Roush, West Virginia Part C OSEP National Early Childhood Meeting Arlington, VA, December 2007
45
Embed
Batya Elbaum, NCSEAM Pam Roush, West Virginia Part C OSEP National Early Childhood Meeting
How to Explain the Numbers: Helping Staff, Parents, and Other Stakeholders Understand the Results of the NCSEAM Surveys for Part C and 619. Batya Elbaum, NCSEAM Pam Roush, West Virginia Part C OSEP National Early Childhood Meeting Arlington, VA, December 2007. Purpose of the session. - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
How to Explain the Numbers:Helping Staff, Parents, and Other
Stakeholders Understand the Results of the NCSEAM Surveys for
Part C and 619
Batya Elbaum, NCSEAMPam Roush, West Virginia Part COSEP National Early Childhood
MeetingArlington, VA, December 2007
Purpose of the session To provide participants with
strategies for explaining the measures and percentages that come from the NCSEAM rating scales addressing Indicators C4 and B8.
Part C Indicator #4
“Percent of families participating in Part C who report that early intervention services have helped the family a) know their rights, b) effectively communicate their children’s needs, and c) help their children develop and learn.”
Part B Indicator #8
“Percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities.”
WV Part C Indicator 4 4A – Know rights
2006 76.5% 2007 73.8%
4B – Communicate 2006 72.8% 2007 71.0%
4C – Help child 2006 86.3% 2007 78.7%
Change in mean measure by region
Region 2006
N 2006 Mean
2007 N
2007 Mean
Change in Mean
1000 Part C Impact on Family Measures 62 716.41 44 705.32 -11.09
1100 Part C Impact on Family Measures 64 702.91 44 715.36 +12.45
1200 Part C Impact on Family Measures 50 665.87 47 670.60 +4.73
1300 Part C Impact on Family Measures 93 717.19 58 688.85 -28.33
1400 Part C Impact on Family Measures 28 755.98 24 637.00 -118.98
1500 Part C Impact on Family Measures 25 615.88 25 627.71 +11.84
1600 Part C Impact on Family Measures 48 695.17 33 651.53 -43.63
1700 Part C Impact on Family Measures 90 706.44 53 681.74 -24.69
Percent on 4A by Region in 2006 and 2007
2006 2007
Region N 4A % N 4A %
1000 62 72.58 44 70.45
1100 64 71.87 44 86.36
1200 50 72.00 47 74.47
1300 93 .78.49 58 77.59
1400 28 92.86 24 58.33
1500 25 60.00 25 64.00
1600 48 79.17 33 63.64
1700 90 81.11 53 79.25
Why use a measurement analysis?
Why use a measurement analysis?
Use of a measurement framework ensures that measures will mean the same thing, regardless of how many items, or which specific items, are administered.
Why use a measurement analysis?
We can’t assume that all survey items are equally agreeable.
A measurement analysis gives us a measure of each item’s overall agreeability.
Order of Impact on Family items from least to most agreeable
Item Calibration
Item Stem: Over the past year, Early Intervention services have helped me and/or my family:
678 Participate in typical activities for children and families in my community.
656 Know about services in the community.
640 Know where to go for support to meet my family's needs.
625 Keep up friendships for my child and family.
609 Know where to go for support to meet my child's needs.
584 Be more effective in managing my child's behavior.
576 Make changes in family routines that will benefit my child with special needs.
576 Do activities that are good for my child even in times of stress
570 Improve my family's quality of life.
565 Feel that I can get the services and supports that my child and family need.
563 Get the services that my child and family need.
562 Feel that my family will be accepted and welcomed in the community
559 Feel more confident in my skills as a parent.
559 Feel that my child will be accepted and welcomed in the community.
556 Communicate more effectively with the people who work with my child and family.
553 Understand how the Early Intervention system works.
546 Understand the roles of the people who work with my child and family.
539 Know about my child's and family's rights concerning Early Intervention services.
534 Be able to evaluate how much progress my child is making.
516 Understand my child's special needs.
498 Feel that my efforts are helping my child.
498 Do things with and for my child that are good for my child's development.
Why use a measurement analysis?
Use of a measurement framework allows us to test whether all the items are measuring the same thing.
Measuring the 3 subindicators
NCSEAM’s measurement analysis of pilot data from thousands of families showed that items that are related to the three OSEP outcomes all fit into a single scale of family outcomes.
Measuring the 3 subindicators
Families appear to achieve these outcomes in a very consistent order. Families who report that EI helped them
know their rights also report that EI also helped them help their child develop and learn.
Families who report that EI helped them effectively communicate their children’s needs also report that EI also helped them know their rights and help their child develop and learn.
Location of key items related to Indicator C4
Indicator 4c: Help their children develop and learn. IFS items: “Understand my child's special needs.” [516] “Do things with and for my child that are good for my child's development.” [498]
Indicator 4b: Effectively communicate their children’s needs. IFS Item: “Communicate more effectively with the people who work with my child and family.” [556]
Indicator 4a: Know their rights.IFS Item: “Know about my child's and family's rights concerning Early Intervention services.” [539]
556
539
516
Why use a measurement analysis?
We can’t assume that the response choices (very strongly disagree, strongly disagree, disagree, agree, strongly agree, very strongly agree) indicated by equidistant circles or numbers on a page are really equidistant.
Spacing of response choices
◘ ◘ ◘ ◘ ◘ ◘ 1 2 3 4 5 6
Spacing of response choices
◘ ◘ ◘ ◘ ◘ ◘ 1 2 3 4 5 6 1K 5K 10K ½ Mar. Marathon Ultra
The logic of distributions
Example: Distribution of height
Distribution of Measures of Reading Achievement - 1
X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X >5’ 5’0 5’2 5’4 5’6 5’8 5’10 6’0 6’2 6’4 6’6 6’8 6’10 7’ >7’
Distribution of Measures of Reading Achievement - 2
X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X >5’ 5’0 5’2 5’4 5’6 5’8 5’10 6’0 6’2 6’4 6’6 6’8 6’10 7’ >7’
West Virginia results 2006
West Virginia results 2007
WV without extreme cases 2006
WV without extreme cases 2007
Instrumentation and approach to data analysis make a difference
Choice of instruments: Part B NCSEAM K-12 and 619 9 NCSEAM K-12 21 Customized NCSEAM survey 11 State-developed or adapted 18 ECO Family Outcomes Survey 1
States’ reported baseline data onIndicator 8B
Choice of instruments: Part C NCSEAM Survey 25 ECO Family Outcomes Survey 18 State-developed or adapted 10 Combination 1
Analysis of states’ baseline data:Indicator C4
Sub-Indicator a. Know their
rights b. Communicate children’s needs
c. Help children develop and learn
Mean 79% of families 78% of families 85% of families
Range 45% - 99% 42% - 99% 53% - 99%
Analysis of states’ baseline data:Indicator C4
Figure 2: Baseline by Survey Tool and Criteria for Positive Response
0
10
20
30
40
50
60
70
80
90
100
ECO 3 & above
N=2
ECO5 & above
N=16
NCSEAM alternate analysis
N=6
NCSEAM standard analysis
N=17
State Survey
N=11
Tool and Analysis
% a
% b
% c
APR Family Outcome Results – Part C States Using NCSEAM Survey and RASCH analysisAPRs submitted February 2007