1 “Just the facts, ma’am”: In search of FACT fidelity Joe Morrissey University of North Carolina at Chapel Hill Festschrift for Gary Bond IUPUI, Indianapolis,

1

“Just the facts, ma’am”: In search of FACT fidelity

Joe MorrisseyUniversity of North Carolina at Chapel Hill

Festschrift for Gary BondIUPUI, Indianapolis, INSeptember 23, 2009

Funding from the NIMH and MacArthur Foundation’s Mental Health Policy Research Network is gratefully acknowledged.

2

“Just the facts, ma’am”

Joe Friday’s signature directive in 1950-60s TV docu-drama about LAPD

Searching for FACT fidelity has been a lot like detective work . . . cajoling findings from unruly data

3

Some facts about FACT Forensic adaptations of ACT (FACT) are one of

the latest efforts to keep persons with severe mental illness out of jail

Despite rapid dissemination, current evidence about FACT’s public safety and mental health effects is weak

Today, I’d like to add to that evidence base and discuss future prospects for FACT fidelity

Throughout this work Gary Bond has been an inspiration . . .

4

Gary Bond’s First Principles of Mental Health Services Research

#1. “Any untested service intervention should not be demonstrated because it will fail.”

#2. “A social experiment is a contrivance that when applied to a group of people leads to a scientific publication.”

5

Mentally ill & jails

People with mental illness in the criminal justice system have become the new frontier for community mental health interventions

Now, more than 1 million jail bookings of people with SMI each year; SMI prevalence: 14.5% male & 31.0% female detainees (Steadman et al., 2009)

Relative risk of persons with SMI being jailed is 150% greater than being hospitalized (Morrissey et al., 2007)

6

Current ACT evidence

Bond & colleagues (2001): 8 of 10 trials, usual care equal-to-or-better-than ACT on arrests & jail use

Calsyn & colleagues (2005): ACT no-better-than usual care on range of CJ outcomes

Chandler & Spicer (2007): IDDT no-better-than usual care on range of CJ outcomes

7

Significant ACT Outcomes in 25 RCTs pre-2000 [n of trials, %]*

Outcome Better No Difference Worse

Psych hosp use 17 (74%) 8 (26%) 0

Housing stability 8 (67%) 3 (25%) 1 (8%)

Symptoms 7 (44%) 9 (56%) 0

Quality of life 7 (56%) 6 (42%) 0

Social adjustment 3 (23%) 10 (77%) 0

Jails/arrest 2 (20%) 7 (70%) 1 (10%)

Substance use 2 (33%) 7 (70%) 0

Vocational functioning

3 (37%) 5 (63%) 0

*Source: Bond, Drake, Mueser & Lattimer, 2001

8

ACT as an intervention platform

Bond & colleagues (2001) review also showed weak effects of ACT on substance abuse & vocational functioning outcomes

But when ACT teams were retrofitted to address these issues (via IDDT & supported employment) subsequent RCTs showed positive effects

Would same thing happen if ACT was retrofitted to prevent arrests and incarcerations?

9

ACT to FACT AdaptationsDACT Core Items

Mobile/comprehen-sive services/team-based

Psychiatrist on team 1-10 S-to-C ratio 24/7 crisis response Time unlimited

FACT Same Same Same Same Same

Prior arrests CJ referrals CJ partners Court sanctions

10

Other Differences

ACT Target: SMI @ risk of

hospitalization Goal: Prevent

hospitalization & sustain community living

Vocational, AOD staffing on team

DACT fidelity standards

FACT Target: SMI @ risk of

arrest/ jail detention Goal: Prevent re-

incarceration

Less vocational, linked AOD services

Probation officers as team members

No clinical model or fidelity standards

11

Current FACT Evidence Separate pre-post studies (no control groups)

with small samples, FACT associated with fewer jail days, arrests, hospital days, and hospitalizations (Lamberti et al., 2001; Weissman et al., 2004; McCoy et al., 2004)

Still no published reports based on rigorous comparison group data clearly showing FACT can improve both mental health & public safety outcomes

12

Our efforts. . .1. Birmingham study (2004-05)

abortive effort to retrofit the ‘first’ RCT (so we thought) on a SAMHSA jail diversion site

2. FACT survey and site visits (2005-06) surveyed 30 ACT & CJ programs, visited 12 FACT programs to document operating characteristics and sustainability

3. Mentally Ill Offender Crime Reduction Grant (MIOCRG) program (2006-09) discovered 20+ RCTs and opportunity to re-analyze data from several counties in California

13

MIOCRG initiative California Board of Corrections: 30 county,

$80.5 million program 2000-04, Sheriffs Assn. and MH Association bill sponsors

Goals: identify what works most effectively in reducing recidivism among mentally ill offenders

Local evaluations with random assignment; individual data on ~8,000 enrollees reported to BOC for cross-site evaluation

BOC Report: ACT-like programs improved CJ and MH outcomes, but major sampling and statistical problems not addressed

14

MIOCRG re-analysis saga1. Find Calif. counties w true FACT models2. Get local evaluators to share data3. Get CA-DMH to agree to create linked, de-

identified services data files Get IRB approvals at UNC-CH & CA-DMH

4. Get county MHAs to approve re-analysis plan & send study IDs to DMH for record linkage

5. Obtain & link data across counties with common prospective cohort format & with common variables

6. Run individual site & pooled analyses

15

CountySmal

l Rati

o

24/7

Psychia-trist

on Team

Daily

Mtgs.

Shared

Cases

In-house

Services

Adaptations

RCT

Sample Sizes

(TX/CTRL)

PO* MH

Court

Forensic Assertive Community Treatment

Marin 70/25

San Joaquin

101/47

Stanislaus 72/62

Forensic Intensive Case Management

San Mateo 79/81

Butte 137/98

Solano 44/26

* Police or probation officer on the team

Found six MIOCRG sites that allowed for FACT v. FICM comparisons

16

Best laid plans go awry . . .

1. CA-BOC failed to implement a true experimental study; we had access to the MIOCRG data for all 30 sites but we couldn’t make sense of it

2. Ended up working with three sites with same evaluator, but even then, the CA-BOC design led to incomplete data and we were unable to fix that for 2 of 3 FACT sites

3. So, we resorted to administrative data to assess impact of FACT at 1 site

17

Setting and design

Mid-size city FACT program (2000-03) enrolled consumers

from county jail; probation officers on team DACT scores of 4.5 and 4.6 Retained MIOCRG randomized groups for our

analyses: FACT v. treatment as usual (TAU) Followed both groups in administrative data

12 mos. pre and 12 & 24 months post

18

Administrative data elementsMental Health

Service utilization # psych hosp days # crisis contacts # outpatient visits

Costs

Public Safety

Jail use & arrests # bookings # felony/misdemeanor

charges & convictions # jail days

Costs

19

Study sample

Participants had histories of frequent jail use But they also had a lot of mental health

services use in baseline period Random assignment to FACT and TAU worked

to produce two equivalent groups (age was only significant difference but it didn’t matter in multivariable analyses)

20

Sample & randomization resultsBaseline Characteristics

FACT (N= 72) TAU (N= 62)

Demographic % Caucasian % African Am % Hispanic % Male Age: mean (sd)

618

2260

38.8 (10.9)*

658

2158

34.4 (8.9)

Clinical % Psychotic dx % Affective dx % Comorbid SA

612977

722464

Criminal Involvement % Any booking % Any felony charge % Any conviction

965765

956667

Mental health services % Any hospital use % Any outpatient use # hospital days # outpatient visits

5888

13.7 (19.6)29.8 (35.7)

4784

7.1 (14.1)26.6 (35.1)

*p<.05

21

Results

Compared to TAU participants: FACT participants had fewer bookings

(p<.01) and jail days (p<.05) in each year. FACT participants had more outpatient

visits (p<.001) but fewer days of hospitalization (p<.05) and incurred lower overall costs for the county jail and the county mental health service system.

22

Conclusion

A forensically-oriented, high-fidelity ACT (FACT) team can alter the criminal justice involvement of offenders with serious mental illness, reduce their time spent in inpatient psychiatric settings, while providing more appropriate and less costly outpatient services.

23

Implications for FACT fidelity

Single RCT is never definitive, but it helps to elevate the evidence base

FACT works, but it is expensive and it should be carefully targeted to those most in need, not everyone who ends up in jail

More needs to be done to specify and test a clinical model for FACT; then, fidelity issues become meaningful

Some feel criminogenic needs should be targeted via a CBT add-on to FACT; further research needed here

24

An interplanetary traveler’s advice “In doing meaningful

services research, try to . . . Fill what’s empty. Empty what’s full. And scratch where it itches.”

“The really important thing is . . . not to stop questioning.”

“The best is yet to come.”

25

Acknowledgements . . .

All personal attributions herein are apocryphal . . .they have been gleaned, stolen, modified, invented, and filched from various sources to fit the occasion!