Fostering a Permanent Home : A Mixed Methods Evaluation of the ZERO TO THREE Court Teams for Maltreated Infants and Toddlers Initiative Kimberly L. McCombs-Thornton,

Fostering a Permanent Home:

A Mixed Methods Evaluation of the ZERO TO THREE Court Teams for Maltreated Infants and Toddlers Initiative

Kimberly L. McCombs-Thornton, PhDUniversity of North Carolina at Chapel HillDecember  10 , 2011

Outline

• Background and significance

• ZTT Court Teams program

• Quantitative study

• Qualitative study

• Limitations and Discussion

2

Background and Significance

• Infants experience highest rate of victimization; toddlers second  – Infants (< age 1):   20.6 per 1000   – Toddlers (age 1):   11.9 per 1000– Toddlers (age  2):   11.3 per 1000

• Developmental needs of young children are time sensitive                 

• The caregiver is important for the child’s healthy development 

• Infants and toddlers in child welfare are at risk for– Attachment disorders – Poor physical health 

3

Background and Significance

• The Adoption and Safe Families Act (ASFA) of 1997 seeks to expedite the time to reach a permanent, stable home for children in child welfare

• A permanent home may include:– family reunification– legal guardianship– or traditional adoption

• More children appear to be reaching permanency since ASFA’s enactment, though the need is still great– Adoptions rose 64.5% in the three years following ASFA’s enactment  – Represents only 28% of children eligible for adoption during these years  

4

ZTT Court Teams Project

Judge

Community Coordinator

Court Team-DHS case worker-CASA-Attorneys-Providers-Etc.

Monthly Case Reviews

Referral to Child-focused Services

Child-Parent Psychotherapy

Evidence-Based Parent Education

ZTT National Office Activities

-Training and TA-Resource materials-M and E

Core Components

Decrease in Time to Permanency

Reduction in Recurrence of Maltreatment

Improved Child Well-being

LOCAL  ACTORS ACTIVITIES LONG-TERM  OUTCOMES

TARGET GROUP: CHILDREN ENTERING FOSTER CARE BEFORE AGE 3

5

Previous Evaluation

• JBA completed evaluation in 2009– Process oriented– 95% of closed cases reached permanency– 55% of those reached permanency within one year

• No comparison                 Problem of causal                              group                            inference

6

Study AimsAim 1:  Determine the effect of the ZTT Court Teams project on time to permanency. 

Aim 2:  Assess the influence of the ZTT Court Teams program on how children exit the foster care system.  

 Aim 3:  Examine successful and unsuccessful cases to understand how program components and/or client characteristics contribute to time to permanency. 

7

Causal Inference

Q:  How do we know outcomes are due to the program?

 A:  Need a comparison group

      Basic Advanced

Pre/post Secondary Matched Randomized Test Data as Comparison Experiment Comparison Group (w/ control group)

8

Quantitative Data

ZTT “Treatment” Group  

Dataset:

Court Teams MIS   

Data Source: CPS documents, observations in court, etc.  

What collected: Child background, service needs and usage, visitation, placements, case status  

Study sample: All children in first 4 ZTT sites who entered by end of 12/2009  

Number in study:

298 children   

Follow up period: One year or more for 94% of cases 

9

Quantitative Data

ZTT “Treatment” Group  

NSCAW Comparison Group 

Dataset: Court Teams MIS  National Survey of Child and Adolescent Well-Being (NSCAW)  

Data Source: CPS documents, observations in court, etc.  

Interview with CPS case worker 

What collected: Child background, service needs and usage, visitation, placements, case status  

Family risks, child living environments, services needed and received, child behavior, etc. 

Study sample: All children in first 4 ZTT sites who entered by end of 12/2009  

Children who entered child welfare supervised out of home placement before the age of 3  

Number in study:

298 children  511 children 

Follow up period: One year or more for 94% of cases 

One year or more for all cases 10

Case Characteristics ZTT

(n=298) NSCAW (n=511)

Child Characteristics

Age of child at first out of home placement Mean (months) Infant

9.3 (1.0) 67% (.04)

11.0 (.99) 57% (.04)

Child gender Male

50% (.02)

46% (.04)

Race/Ethnicity African American Caucasian Latino/a

37% (.10) 29% (.10) 14% (.10)

24% (.04) 41% (.05) 18% (.05)

Parent Characteristics Substance Abuse 72% (.06) 59% (.05) Severe Mental Health Needs 17% (.03) 21% (.04) Poverty 95% (.02) 81% (.03) Lack of Employment

35% (.15) 47% (.05)

11

Propensity Score

Propensity score =  association between covariates and group membership                      

12

Selection into ZTT Court Teams

How child selected within a site:- Child under age of 3 at time of entry into child welfare 

system- Either all young children assigned to Court Teams judge or 

randomly assigned

13

14

Predictors of Time to Permanency

Reasons for Removal

Community Characteristics

Child Characteristics

ParentCharacteristics

Aim 1

Determine the effect of the ZTT Court Teams project on time to permanency.

15

Aim 1

2 Outcome Measures:– Time to “move in” permanency– Time to “official” permanency

Methods:– Time to event analysis with propensity score weights– Test for proportional hazards– Diagnostics

16

Aim 1

Time to “move in” permanency

17

Permanency ZTT (n=298)

NSCAW (n=511)

Move In

Median (SE) 168 (10.5) 210 (25.6)

Mean (SE) 343 (32.1) 398 (23.4)

ZTT cases move into what becomes the permanent home 1.5 to 2 months faster on average

Aim 1Time to “move in” permanency- Kaplan Meier

- Time to Event AnalysisParametric model (Gompertz):  HR = 1.06  (CI .67, 1.65)   

p=.81

0.00

0.25

0.50

0.75

1.00

0 500 1000 1500 2000Days until move into what becomes the permanent home

ZTT Court Teams NSCAW

Kaplan-Meier survival estimates: Time to Move in Permanency

18

Aim 1

Time to “official” permanency

19

Permanency ZTT (n=298)

NSCAW (n=511)

Official

Median (SE) 383 (10.8) 762 (26.8)

Mean (SE) 466 (20.5) 825 (18.5)

ZTT cases exit foster care one year earlier on average

Aim 1Time to “official” permanency- Kaplan Meier

0.00

0.25

0.50

0.75

1.00

0 500 1000 1500 2000Days until officially discharged from foster care

ZTT Court Teams NSCAW

Kaplan-Meier survival estimates: Time to Official Permanency

20

Aim 1

Time to “official” permanency

- Time to Event Analysis

    ZTT cases exit foster care nearly three times as fast as the

comparison group

21

Parametric model (Weibull):  HR = 2.67   (CI 1.65, 4.31)   p=.000

Aim 2

Assess the influence of the ZTT Court Teams program on how children exit the 

foster care system.  

22

23

Aim 2

4 ways to exit foster care:

- Reunification

- Adoption

- Relative custodianship

- Non-relative legal guardian

fastest

slowest

Aim 2Type of

Foster Care ExitZTT Court Teams

(n=298)NSCAW sample

(n=511)

pct se pct se

Reunification 37.6% (.047) 29.3% (.042)

Adoption 15.4% (.059) 40.7% (.045)

Relative custodian 24.8% (.085) 8.4% (.033)

Non-relative guardian 3.0% (.017) 1.6% (.007)

Still in foster care at end of study period

19.1% (.068) 20.1% (.036)

24

Aim 2Days until Exit Foster Care, by Type of Exit

25

Type of Exit from Foster Care

ZTT (n=298)

NSCAW (n=511)

Reunification Median 309 547 Mean 340 587

Adoption Median 464 764

Mean 496 800 Relative custodian Median 351 450

Mean 363 487 Non-relative guardian Median 481 878

Mean 467 780  

8 months faster on average

10 months faster on average

3-4 months faster on average

10-13 months faster on average

Patterns hold when use propensity scores in a competing risks analysis

Aim 3

Examine successful and unsuccessful cases to understand how program components 

and/or client characteristics contribute to time to permanency. 

26

Aim 3

Examine successful and unsuccessful cases to understand how program components 

and/or client characteristics contribute to time to permanency. 

27

Aim 3Methods

•  One-on-one open-ended phone interviews with each     community coordinator;  2 interviews each 

•  Approach orientated to unique cases– discuss top and    bottom decile in each site  (Patton, 1990)

•  Community coordinator’s perception of how each     component of the program model contributed  time to    permanency for each case

•  Record and transcribed calls

•  Coded using Atlas.ti

28

Aim 3

Parents comply with service plan?

Yes

Reunification

“[The mother] is absolutely amazing that, what is she, 20 years old, she had two children with special needs that really require a lot of doctors visits. So she was able to show all the professionals in the system that she was able to follow through with all of these things. And at the same time, she was dealing with her own victimization issues and poor relationship choices…..The children were [ultimately] reunified with the mother.”

29

Aim 3

Parents comply with service plan?

Yes No

Reunification Terminate/Surrender/SuspendParental Rights

Adoption Legal Guardian

“The mother would come into court every time positive for cocaine, refused to get any help….. She didn’t approach it [the service plan]. She just didn’t do anything….This child was only placed in one foster home… [who] adopted the child.” 30

Aim 3

Parents comply with service plan?

Yes NoSomewhat

Reunification Terminate/Surrender/SuspendParental Rights

Adoption Legal Guardian

“An inpatient drug treatment program was recommended for her. She went to one and she completed it. And then when she came out of course they recommended outpatient care and AA meetings and so forth. She started taking those and then slacked off. So that was a discussion in every hearing. Was she or was she not in compliance with the aftercare recommendation….”

31

Aim 3

Parents comply with service plan?

Yes NoSomewhat

Reunification Terminate/Surrender/SuspendParental Rights

Adoption Legal Guardian

Parent attitude/willingness

SOCIAL SUPPORT INFLUENCES

Family, friends, foster parents, and others to provide: transportation, housing, visitation supervision, financial support, care for children, emotional support, etc.

Judge:- monitor case progress- verbal support when parents comply- verbal warning when parents not complying- timekeeper

Service providers:- attempt to meet needs, teach

skills, stabilize condition

Department of Social Services:- design service plan- assistance seeking services- visitation supervision

Parent behavior :- substance abuse issues- mental health issues- relationship issues

Parent personal resources:- education/job skills- poverty- transportation

32

Aim 3

Role of Judge in Time to Permanency:

• Motivate the parents

“[The judge] was very supportive and really wanted the children with their mother. And you know basically would encourage her and would actually praise her and tell her she was doing a good job and tell her to keep it up…assuring her we were going in the right direction… [The judge] is very good about praising when you’ve made progress on your service plan and you’re doing what you’re supposed to do.”

“[The judge] became increasingly frustrated… and really confronted the mom on how the professionals were working harder than she was and her inability to take ownership for her part that she played in this. And confronting mom on not understanding the impact it had on the children.” 33

Aim 3

Role of Judge in Time to Permanency:

• Motivate social support network

• Motivate the case workers

• Role as timekeeper

34

Aim 3

Role of Monthly Case Review in Time to Permanency:

• Keeping all on task“Everybody stayed on task because they knew we were gonna be staffing and we were going to be in court. So there was no room for making, for example, making a referral a week before we go to court because we were always going to court.”

“Usually CPS cases, the hearings are every 90 days. But with this, the parents know that they have to be in court every month. It gets them motivated to get on the ball so they don’t have to go to the judge in 30 days to explain to the judge why they haven’t done what they are supposed to have done 30 days prior. “

• Monitor parental compliance

35

Aim 3

Fast v. Slow case comparison

• Program is consistent across the cases-  All have monthly case reviews- Judge has equal contact with cases

36

Aim 3Activity to Strengthen Results  Reliability  Validity 

     Within a Site  

Used same questions 5 to 6 times in one interview re: fastest cases  

Used same questions 5 to 6 times in a second  interview on a different day re: slowest cases  

Used same questions for fast and slow cases  

Validation interviews – with one other professional in each site  

Negative case analysis to search for examples contrary to findings  

Across Sites     Asked same questions across the sites a total of 46 times    

Compared findings to the James Bell Associates Court Teams evaluation  

Mixed methods – quantitative analysis  

37

Conclusion• The ZTT Court Teams program reduces time to official permanency, but not time to move in permanency

• More kids are exiting foster care through reunification and less through adoption

• Though – ZTT Court Teams cases are exiting faster regardless of type of exit

• The program operates consistently for both “fast” and “slow” cases

• The judge and the monthly case review components appear to be most linked to time to permanency

38

Limitations and Next Steps

Quantitative study• Propensity score subject to omitted variable bias

• Only considers child’s first episode in child welfare

• Assess effect of number of court hearings

• Are all types of exits equal?

• Next step – cost effectiveness study

39

Limitations and Next Steps

Qualitative study• Only one person coded and analyzed data

• Sorted by time to move in permanency

• Collected before completed quantitative analysis

• Site differences – fidelity to the model?

40

Reflections

Judge Constance CohenPolk County, Iowa

Judge Ernestine GrayOrleans Parish, Louisiana

Judge Michael McPhailForrest County, Mississippi

41