Top Banner

of 19

Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes

Jan 28, 2016

ReportDownload

Documents

beyla

Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes. July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of using changes in growth rates to measure child outcomes in Part C” Rosanne Griff-Cabelli, Delaware Part C rosanne.griff-cabelli@state.de.us. - PowerPoint PPT Presentation

  • Delaware Birth to Three Early Intervention System Evaluation:

    Child OutcomesJuly 15, 2004Conference Call Series: Measuring Child OutcomesExamples of using changes in growth rates to measure child outcomes in Part CRosanne Griff-Cabelli, Delaware Part Crosanne.griff-cabelli@state.de.us

  • Delaware Birth to Three Child Outcomes EvaluationPrimary purposes:1. To document changes in the children served by Part C Programs2. To collect data, systematically so that programs can communicate their impact on children and families to various audiences.

  • Impetus for Delawares Child Outcome EvaluationThe State Budget Office, as part of the Interagency Management Resource Committee (IRMC), requested child change data from federal and state funded programs. Part Cs evaluation was linked through the IRMC to early childhood programs, such as Head Start and Part B 619. The IRMC Annual Report includes Part C evaluation results, distributed to Department Secretaries and Legislative Joint Finance Committee. Part C formed an Ongoing Program Evaluation Committee (OPEC) to coordinate, review and update components of child outcome evaluation. Results are shared annually at ICC and Part C regional staff meetings.

  • Outcome Areas: Focus and ScopeOutcome Measures:

    Describe changes in eligible childrens development and functioning over time, while active in Part C programs by: Developmental Domains Motor Cognitive Play and Functional Changes

  • Outcome Areas: Focus and ScopeDemographic Variables:

    Describe socio-economic factors of familyDescribe demographic information such as gender, race/ethnicity, county of residence, age child started program, primary language spoken

  • Design/MethodPopulation to Sample:

    Children selected are Part C eligible, have their first MDA completed, and not older than two years and six monthsRandom sample of every 15th child in New Castle County and every 10th in Kent and Sussex CountiesChildren will have at least two points of evaluation, approximately one year apartNo service coordinator will have more than three children actively involved in Child Change at any given point

  • Design/MethodMeasurement Instruments:

    Bayley Scales of Infant Development II, administered as part of MDAFewell Play Assessment Scale (PAS) with adaptations for children with more severe disabilitiesDemographic Information Form, administered by service coordinatorsHOME [Home Observation for Measurement of the Environment] Inventory Birth to Three, administered by service coordinators

  • Design/MethodAll assessments are administered by Part C program staff.The PAS is a valuable addition to the battery of assessments conducted with children.The PAS can be implemented by Part C program personnel but additional resources are needed for annual training.A contract is in place with the University of Delaware, Center for Disabilities Studies for data analysis and reports.

    Sheet1

    Delaware Part C Continuous Improvement Monitoring and Evaluation Timeline

    Evaluation Strategy2001-022002-032003-042004-052005-062006-07

    Family Focus GroupFallSpringFall

    Family SurveySpringFallFall

    Service Coordinator Focus GroupSpringSpringSpring

    Provider Focus GroupSpringSpringSpring

    Provider SurveyFallSpring

    Child OutcomesOngoing

    Sheet2

  • Analysis Using Proportional Change Index (PCI)Delawares Part C Child Change Evaluation is analyzed through a contract with the University of Delaware, Center for Disabilities Studies. The statistical procedure used is PCI.

    The PCI is a method devised by Mark Wolery of the University of Kentucky that compares childrens rate of development at pretesting to their rate of development during intervention.1

    The following link is to a paper on measuring developmental change. It refers to the PCI and its strengths and drawbacks.http://www.aare.edu.au/94pap/konzd94260.txt

    1Mark Wolery, Exceptional Children, Volume 50, Number 2, page 168, 1983

  • Analysis Using Proportional Change Index (PCI) Design/Method

    Developmental Gain / Pretest Developmental AgeTime in Intervention / Pretest Chronological Age

    Divide second rate of development by first rate of development to calculate PCI.

    Children who continue to develop at the same rate during intervention as they did prior to intervention will have a PCI of 1.0Children whose rates of development are slower during intervention will receive a PCI of less than 1.0Children whose rates of development accelerated during intervention will receive a PCI greater than 1.0

  • Analysis Using Proportional Change Index (PCI) Design/Method

    Developmental Gain / Pretest Developmental AgeTime in Intervention / Pretest Chronological Age

    Divide second rate of development by first rate of development to calculate PCI.

    Children who continue to develop at the same rate during intervention as they did prior to intervention will have a PCI of 1.0Children whose rates of development are slower during intervention will receive a PCI of less than 1.0Children whose rates of development accelerated during intervention will receive a PCI greater than 1.0

  • Analysis Using Proportional Change Index (PCI)Strengths of the PCI It is not a measure solely of the actual number of months gained in intervention, but takes into account the number of months in intervention and the childs rate of development at pretesting.

    Comparisons can be made between children at various levels of delay, chronological age, and developmental age. Drawbacks of the PCI Assumes childrens rates of development at pretesting would be stable in the absence of intervention.Larger sample and three measures for each child would allow use of Growth Curve Analysis.

  • Major Findings (sample in child change as of January 2004)The Children and Families:

    43 children tracked 22 Northern CDW 21 Southern CDW 23 male; 20 female Disabilities: 8 VLBW 11 developmental delay 24 VLBW and developmental delay Ethnicity 15 African American, 18 Caucasian, 6 Hispanic, 4 bi-racial

  • Major Findings (as of January 2004)The Children and Families:

    Family SES15 families below poverty level20 families above poverty level8 unknownEducation Level of Mothers2 less than 8th grade4 less than high school7 high school graduate/GED15 some college or technical school9 college degree2 graduate degree4 unknown

  • Major Findings (as of January 2004)Play Skills Development:

    32 children with two PAS scores; 11 with three scores Time between PASs (M=12.9 months) Play skill changes (M=11.2 months; this is a .87 rate of development) Expected rate of development (M=.72 or 8.64 months in 12 months time) 1.21 times the rate of development expected for this group of children There was no significant difference in rates of development for girls or boys or children living in poverty and those not living in poverty

  • Major Findings (as of January 2004)Mental Skills Development:

    32 children with 2 Bayley Mental scores; 11 with three scores Time between Bayleys (M=12.3 months) Mental score changes (M=10.6 months; this is a .86 rate of development) Expected rate of development (M=.67 or 8.0 months in 12 months time) 1.28 times the rate of development expected for this group of children There remains no significant difference in rates of cognitive development for girls and boys or children living in poverty and those not living in poverty

  • Major Findings (as of January 2004)Motor Skills Development:

    32 children with 2 Bayley Motor scores; 11 with three scores Time between Bayleys (M=12.3) Motor score changes (M=10.3 months; this is a .84 rate of development) Expected rate of development (M=.69 or 8.3 months in 12 months time) 1.22 times the rate of development expected for this group of children No differences in rates of development for children living in poverty and those not; no differences between boys and girls

  • Major Findings (as of January 2004)Additional Findings:

    Of the 43 children, 36 had rates of development greater than 1.0 for at least a one year period of time in both motor and mental skills Seven children had rates of development equal or greater to 1.50 for at least a one year period of time in at least one domain of development

    Seven children had rates of development below 1.0 for at least a one year period of time After at least one year of services, only two children had rates of development lower than that which they entered the program (usually attributed to a progressive or degenerative disorder)

  • Lessons LearnedEarly intervention seems to be having a positive effect on childrens play, and mental and motor skill development across demographic groups. Child Change assessments are conducted systematically and program staff are able to implement the Bayley assessments as part of MDA annual timelines. Child Change results are reported in context of Birth to Three Evaluation Plan.

    RecommendationsReport the trend of childrens outcomes in three year intervals so that a long-term trend can be determined; the cohort should always be between 40-60 children.Efforts are in place to increase the number of males in the evaluation to 55-60% (currently at 54%).Continued efforts need to be in place to encourage families to participate in the evaluation since a sampling method is used.