Jan 01, 2016
Linking Early Intervention Quality Practices With Child and Family Outcomes
Technical Assistance for Local Early Intervention SystemsInfant & Toddler Connection of Virginia1Technical Assistance Includes:Education about the process for measuring family outcomesReview of local system resultsData analysisIdentification of Improvement strategiesInfant & Toddler Connection of Virginia2Our technical assistance for improving family outcomes includes these components.Education about the Process:Check Your KnowledgeWhat are the Family Outcomes?How are the results for the Family Outcomes Indicator derived for the Annual Performance Report to OSEP?What do the results measure? Who can impact the family outcomes?What factors impact the family survey results?
Infant & Toddler Connection of Virginia3I am going to highlight our work with one particular local system in 2011 for this presentation.
Technical Assistance was provided during a site visit that included the local system manager, service coordinators and service providers. A simple pre-test indicated that there was a big knowledge gap regarding family outcomes, including the relationship of the survey to the results and even a lack of information about the questions on the survey.Education about the ProcessNCSEAM Survey administered annuallySurvey includes two rating scalesImpact on Families (22 items)Family Centered Services (25 items)Use of Part C standards (cut-scores)4a 5394b 5564c - 516
Infant & Toddler Connection of Virginia4As part of the education process, we explained that wee use the NCSEAM survey to measure family outcomes. This survey includes two rating scales, one of which measure impact on families and the other measures the quality of family-centered services provided to families. While both scales provide valuable information, only the impact on families scale is used for calculating the states results for the family outcomes. Deriving a percent from a continuous distribution requires application of a standard, or cut-score. Virginia uses the Part C standards recommended by a nationally representative stakeholder group convened by NCSEAM. The recommended standards, established based on item content expressed in the scale, were as follows: for Indicator 4a, know their rights, a measure of 539; for Indicator 4b, effectively communicate their childrens needs, a measure of 556; and for Indicator 4c, help their children develop and learn, a measure of 516.
The survey administered by the State of Virginia included two rating scales developed and validated by the National Center for Special Education Accountability Monitoring (NCSEAM). The 22-item Impact on Family Scale (IFS) measures the extent to which early intervention helped families achieve positive outcomes, including the three outcomes specified in Indicator #4. The 25-item Family-Centered Services Scale (FCSS) measures the quality of family-centered services provided to families.
For each scale, the analysis produces a measure for each survey respondent. Individual measures can range from 0 to 1,000. For the IFS, each familys measure reflects the extent to which the family perceives that early intervention has helped them achieve positive family outcomes. The IFS measures of all respondents were averaged to yield a mean measure reflecting the overall performance of the state in regard to the impact of early intervention on family outcomes.
Relationship of Survey to Outcome ResultsReview of the survey itself to understand what parents are being askedSurvey is mailed to all familiesDepending on response, a 2nd mailing and phone call follow up may be doneInfant & Toddler Connection of Virginia5So we started with a review of the survey itself (handout) including the information I mentioned earlier about the two scales. A number of the service coordinators and service providers did not know what families were being asked on the survey.
Translating Survey Results to Outcome ResultsDetailed Analysis Reporthttp://infantva.org/documents/sup-FamilySurvey-2011.pdfSummary Explanation http://infantva.org/documents/sup-FamilySurvey-2011-expl.pdf
Infant & Toddler Connection of Virginia6Virginia contracts with Old Dominion University to administer and analyze the survey results. ODU prepares a detailed analysis report about the results which is posted on our website. A summary explanation is provided by Infant & Toddler Connection of Virginia staff and is also posted on the website.Review of Local Results: Percent Meeting StandardInfant & Toddler Connection of Virginia7Next we reviewed with the local system their results. This slide shows the significant slippage that occurred in that system from 2009 to 2010.
2009 23 surveys were returned2010 39 surveys were returnedComparing 2009 LS Results to Targets and State ResultsInfant & Toddler Connection of Virginia8These next two slides show the relationship of the local results to the target and the state results. In 2009, the state results exceeded the targets, and the local system results were slightly below the target.Comparing 2010 LS Results to Targets and State ResultsInfant & Toddler Connection of Virginia9In 2010, the state results again exceeded the target, but the local system were dramatically below the targets. Local Results:Percent of TargetInfant & Toddler Connection of Virginia10This slide shows the results as a percent of the state targets. You can see that though the results for indicator 4c were not dramatically different from 2009 to 2010, the local systems percent of the target is quite different for 4a and 4b for 2009 and 2010.Data Analysis Included:Review of current results in comparison to prior years; in comparison to target and in comparison to state resultsReview of responses for each question on the Family Centered Practices Scale and the Impact on Families ScaleUse of Local Contributing Factor Tool [RRCP/NECTAC/DAC (6-5-09)] adapted for TA with local system (handout)
Infant & Toddler Connection of Virginia11We proceeded to data analysis.Factors Identified as PotentiallyImpacting ResultsChange in Local Lead AgencyNew local system leadershipNew service coordinatorsNew direct service providersIncreased child count106 in 2009; 140 in 2010Statewide system transformationInfant & Toddler Connection of Virginia12During the March local system TA visit, we brainstormed issues that could be impacting the results. There were some very significant challenges for this local system including..Improvement Strategies2007 -2011Assuring that all providers and service coordinators know what is on the surveyLocal analysis of results of both scales; focus in on family centered practices and key elements where responses were not as positive as expectedUse terminology contained in surveyReminders to staff based on survey questions
Infant & Toddler Connection of Virginia13Improvement Planning Tool Introduced in 2011Relationship of Quality practices to Child and Family OutcomeMeasurement Results (draft 2/4/11)
Infant & Toddler Connection of Virginia14The 2nd tool, the focus of this session had just become available. It provided very concrete information not only about quality practices, but also about the relationship of specific practices to the child and family outcomes.Infant & Toddler Connection of Virginia15
The tool was reviewed briefly during the session with a request that each individual review it in more depth after the meeting.Next Steps: Local Follow Up PlanReview of ToolsFurther analysisStaff discussion and self-reflection Improvement Planning and ImplementationInfant & Toddler Connection of Virginia16The plan after the site visit included a continuation of the discussion, analysis and planning to be led by the local system manager and to include staff (SC and providers) during staff meetings.
Infant & Toddler Connection of Virginia17
" ... So then I thought, 'What the heck, let him have the darn ball of yarn.' Well, . . little did I know ... " " ... So then I thought, 'What the heck, let him have the darn ball of yarn.' Well, . . little did I know ... " " ... So then I thought, 'What the heck, let him have the darn ball of yarn.' Well, . . little did I know ... " So the tools and information were provided to the local systemActions Taken by LocalityRevision of Intake FormStaff discussion/work on effective communication with familiesContinuing work to improve IFSP outcome developmentParticipation with child/family in activities/routines (store, swimming, etc)
Infant & Toddler Connection of Virginia18The local system manager reported back in August that they utilized parts of this form to re-think how we do/present things. In regards to Practice #2, they revised our intake form in an attempt to get a better picture of the family and what their concerns and needs are.They also focused on Practice #5. They have been focusing on IFSP process for the past 6 months, from whom should attend to how to write outcomes, to post IFSP follow up. The team approach to the IFSP has improved. Providers really do want to take an active role in the IFSP process, which is reassuring to the family because they feel comfortable with them because theyve done the assessment. They continue to work on communication and reflection skills, to do a better job listening to and being responsive to familiesThe SM reported that local system providers are great about embracing #8. Providers go to grocery stores, swimming pools, schools, etc.
2011 Results: Percent Meeting StandardInfant & Toddler Connection of Virginia19The 2011 results are dramatically different from 2010. This chart shows the target, the state actual results and above both of those, the local system results. 2009 2011 Results:Percent Meeting