YOU ARE DOWNLOADING DOCUMENT

Please tick the box to continue:

Transcript
Page 1: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Delaware Birth to Three Early Intervention System Evaluation:

Child Outcomes

July 15, 2004

Conference Call Series: Measuring Child Outcomes

“Examples of using changes in growth rates to measure child outcomes in Part C”

Rosanne Griff-Cabelli, Delaware Part C

[email protected]

Page 2: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Delaware Birth to Three Child Outcomes Evaluation

Primary purposes:

1. To document changes in the children served by Part C Programs

2. To collect data, systematically so that programs can communicate their impact on children and families to various audiences.

Page 3: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

• The State Budget Office, as part of the Interagency Management Resource Committee (IRMC), requested child change data from federal and state funded programs.

• Part C’s evaluation was linked through the IRMC to early childhood programs, such as Head Start and Part B 619.

• The IRMC Annual Report includes Part C evaluation results, distributed to Department Secretaries and Legislative Joint Finance Committee.

• Part C formed an Ongoing Program Evaluation Committee (OPEC) to coordinate, review and update components of child outcome evaluation.

• Results are shared annually at ICC and Part C regional staff meetings.

Impetus for Delaware’s Child Outcome Evaluation

Page 4: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Outcome Areas: Focus and ScopeOutcome Measures:

• Describe changes in eligible children’s development and functioning over time, while active in Part C programs by:

Developmental Domains

• Motor• Cognitive

Play and Functional Changes

Page 5: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Outcome Areas: Focus and Scope

Demographic Variables:

• Describe socio-economic factors of family

• Describe demographic information such as gender, race/ethnicity, county of residence, age child started program, primary language spoken

Page 6: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Design/Method

Population to Sample:

• Children selected are Part C eligible, have their first MDA completed, and not older than two years and six months

• Random sample of every 15th child in New Castle County and every 10th in Kent and Sussex Counties

• Children will have at least two points of evaluation, approximately one year apart

• No service coordinator will have more than three children actively involved in Child Change at any given point

Page 7: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Design/Method

Measurement Instruments:

• Bayley Scales of Infant Development II, administered as part of MDA

• Fewell Play Assessment Scale (PAS) with adaptations for children with more severe disabilities

• Demographic Information Form, administered by service coordinators

• HOME [Home Observation for Measurement of the Environment] Inventory – Birth to Three, administered by service coordinators

Page 8: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Design/Method• All assessments are administered by Part C program staff.• The PAS is a valuable addition to the battery of assessments

conducted with children.• The PAS can be implemented by Part C program personnel but

additional resources are needed for annual training.• A contract is in place with the University of Delaware, Center for

Disabilities Studies for data analysis and reports.

Evaluation Strategy 2001-02 2002-03 2003-04 2004-05 2005-06 2006-07Family Focus Group Fall Spring FallFamily Survey Spring Fall FallService Coordinator Focus Group Spring Spring SpringProvider Focus Group Spring Spring SpringProvider Survey Fall SpringChild Outcomes

Delaware Part C Continuous Improvement Monitoring and Evaluation Timeline

Ongoing

Page 9: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Analysis Using Proportional Change Index (PCI)• Delaware’s Part C Child Change Evaluation is analyzed through a

contract with the University of Delaware, Center for Disabilities Studies. The statistical procedure used is PCI.

• The PCI is a method devised by Mark Wolery of the University of Kentucky that “compares children’s rate of development at pretesting to their rate of development during intervention.”1

• The following link is to a paper on measuring developmental change. It refers to the PCI and its strengths and drawbacks.

http://www.aare.edu.au/94pap/konzd94260.txt

1Mark Wolery, Exceptional Children, Volume 50, Number 2, page 168, 1983

Page 10: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Developmental Gain / Pretest Developmental AgeTime in Intervention / Pretest Chronological Age

Divide second rate of development by first rate of development to calculate PCI.

– Children who continue to develop at the same rate during intervention as they did prior to intervention will have a PCI of 1.0

– Children whose rates of development are slower during intervention will receive a PCI of less than 1.0

– Children whose rates of development accelerated during intervention will receive a PCI greater than 1.0

Analysis Using Proportional Change Index (PCI) Design/Method

Page 11: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Developmental Gain / Pretest Developmental AgeTime in Intervention / Pretest Chronological Age

Divide second rate of development by first rate of development to calculate PCI.

– Children who continue to develop at the same rate during intervention as they did prior to intervention will have a PCI of 1.0

– Children whose rates of development are slower during intervention will receive a PCI of less than 1.0

– Children whose rates of development accelerated during intervention will receive a PCI greater than 1.0

Analysis Using Proportional Change Index (PCI) Design/Method

Page 12: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Strengths of the PCI

• It is not a measure solely of the actual number of months gained in intervention, but takes into account the number of months in intervention and the child’s rate of development at pretesting.

• Comparisons can be made between children at various levels of delay, chronological age, and developmental age.

Drawbacks of the PCI

• Assumes children’s rates of development at pretesting would be stable in the absence of intervention.

• Larger sample and three measures for each child would allow use of Growth Curve Analysis.

Analysis Using Proportional Change Index (PCI)

Page 13: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Major Findings (sample in child change as of January 2004)

The Children and Families: 43 children tracked43 children tracked

22 Northern CDW22 Northern CDW

21 Southern CDW21 Southern CDW

23 male; 20 female23 male; 20 female

Disabilities:Disabilities: 8 VLBW8 VLBW

11 developmental delay11 developmental delay

24 VLBW and developmental delay24 VLBW and developmental delay

EthnicityEthnicity 15 African American, 18 Caucasian, 15 African American, 18 Caucasian,

6 Hispanic, 4 bi-racial 6 Hispanic, 4 bi-racial

Page 14: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Major Findings (as of January 2004)

The Children and Families: Family SESFamily SES

15 families below poverty level15 families below poverty level 20 families above poverty level20 families above poverty level 8 unknown8 unknown

Education Level of MothersEducation Level of Mothers 2 less than 8th grade2 less than 8th grade 4 less than high school4 less than high school 7 high school graduate/GED7 high school graduate/GED 15 some college or technical school15 some college or technical school 9 college degree9 college degree 2 graduate degree2 graduate degree 4 unknown4 unknown

Page 15: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Major Findings (as of January 2004)

Play Skills Development:

32 children with two PAS scores; 11 with three scores32 children with two PAS scores; 11 with three scores

Time between PASs (M=12.9 months)Time between PASs (M=12.9 months)

Play skill changes (M=11.2 months; this is a .87 rate of Play skill changes (M=11.2 months; this is a .87 rate of development)development)

Expected rate of development (M=.72 or 8.64 months in Expected rate of development (M=.72 or 8.64 months in 12 months time)12 months time)

1.21 times the rate of development expected for this group 1.21 times the rate of development expected for this group of childrenof children

There was no significant difference in rates of There was no significant difference in rates of development for girls or boys development for girls or boys or children living in poverty or children living in poverty and those not living in povertyand those not living in poverty

Page 16: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Major Findings (as of January 2004)

Mental Skills Development:32 children with 2 Bayley Mental scores; 11 with three 32 children with 2 Bayley Mental scores; 11 with three scoresscores

Time between Bayleys (M=12.3 months)Time between Bayleys (M=12.3 months)

Mental score changes (M=10.6 months; this is a .86 Mental score changes (M=10.6 months; this is a .86 rate of development)rate of development)

Expected rate of development (M=.67 or 8.0 months in Expected rate of development (M=.67 or 8.0 months in 12 months time)12 months time)

1.28 times the rate of development 1.28 times the rate of development expected for this expected for this group of childrengroup of children

There remains no significant difference in rates of There remains no significant difference in rates of cognitive development for girls and boys or children living cognitive development for girls and boys or children living in poverty and those in poverty and those not living in povertynot living in poverty

Page 17: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Major Findings (as of January 2004)

Motor Skills Development:

32 children with 2 Bayley Motor scores; 11 with three scores32 children with 2 Bayley Motor scores; 11 with three scores

Time between Bayleys (M=12.3)Time between Bayleys (M=12.3)

Motor score changes (M=10.3 months; this is a .84 rate of Motor score changes (M=10.3 months; this is a .84 rate of development)development)

Expected rate of development (M=.69 or 8.3 months in 12 Expected rate of development (M=.69 or 8.3 months in 12 months time)months time)

1.22 times the rate of development expected for this 1.22 times the rate of development expected for this group of childrengroup of children

No differences in rates of development for children living No differences in rates of development for children living in poverty and those not; no differences between boys and in poverty and those not; no differences between boys and girlsgirls

Page 18: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Major Findings (as of January 2004)

Additional Findings: Of the 43 children, 36 had rates of development greater Of the 43 children, 36 had rates of development greater than 1.0 for at least a one year period of time in than 1.0 for at least a one year period of time in both motor and mental skillsboth motor and mental skills

Seven children had rates of development equal or greater to Seven children had rates of development equal or greater to 1.50 for at least a one year period of time in at least one 1.50 for at least a one year period of time in at least one domain of developmentdomain of development

Seven children had rates of development below 1.0 for at Seven children had rates of development below 1.0 for at least a one year period of timeleast a one year period of time

After at least one year of services, only two children had After at least one year of services, only two children had rates of development lower than that which they entered the rates of development lower than that which they entered the program (usually attributed to a progressive or degenerative program (usually attributed to a progressive or degenerative disorder)disorder)

Page 19: Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of.

Lessons Learned• Early intervention seems to be having a positive effect on children’s play,

and mental and motor skill development across demographic groups.

• Child Change assessments are conducted systematically and program staff are able to implement the Bayley assessments as part of MDA annual timelines.

• Child Change results are reported in context of Birth to Three Evaluation Plan.

• Recommendations– Report the trend of children’s outcomes in three year intervals so that

a long-term trend can be determined; the cohort should always be between 40-60 children.

– Efforts are in place to increase the number of males in the evaluation to 55-60% (currently at 54%).

– Continued efforts need to be in place to encourage families to participate in the evaluation since a sampling method is used.


Related Documents