Using Research/Evaluation Questions to Define Data Collection and Findings: Findings from the FY 2004 KTOS Follow-up Study Robert Walker, Allison Mateyoke-Scrivener,

Using Using Research/Evaluation Research/Evaluation

Questions to Define Data Questions to Define Data Collection and Findings:Collection and Findings:

Findings from the FY Findings from the FY 2004 KTOS Follow-up 2004 KTOS Follow-up

StudyStudyRobert Walker, Allison Mateyoke-Robert Walker, Allison Mateyoke-

Scrivener, & Jennifer ColeScrivener, & Jennifer Cole

University of Kentucky University of Kentucky

Center on Drug and Alcohol Center on Drug and Alcohol ResearchResearch

Question 1: What are the Question 1: What are the demographic demographic

characteristics of KTOS characteristics of KTOS sample?sample?

Almost any study asks for these dataAlmost any study asks for these data As with all research questions and As with all research questions and

decisions about measures, you must decisions about measures, you must know how fine a detail is needed – e.g., know how fine a detail is needed – e.g., on ethnicity.on ethnicity. Does your survey discriminate between Does your survey discriminate between

race and ethnicity?race and ethnicity? Can subjects have more than one race?Can subjects have more than one race? More than one ethnicity?More than one ethnicity?

DemographicsDemographics

Intake question Breakdown N=888

Gender Male 62.4%

Mean age 33.8

Race WhiteBlackOther

86.7%11.5%1.8%

Ethnicity: Hispanic /Latino1

0.9%

Marital status Never marriedMarriedDivorcedSeparated/Widowed

38.6%21.0%27.8%12.6%

Current living arrangement

HousedInstitutionalizedOther

87.9%8.7%3.4%

Percent of sample who live with:

Dependent child(ren) Anyone with an alcohol

problemAnyone with a drug

problem

31.4%8.8%8.3%

Question 2: Policy Q #1 is Question 2: Policy Q #1 is about abstinenceabout abstinence

The study must address the The study must address the sophistication level of the users of sophistication level of the users of the data.the data.

Current political climates have little Current political climates have little tolerance for harm reduction ways of tolerance for harm reduction ways of understanding treatment outcomes.understanding treatment outcomes.

Moralistic views of substance abuse Moralistic views of substance abuse call for abstinence, not reduced use.call for abstinence, not reduced use.

Increase in Percent of Clients with Alcohol Abstinence

During Preceding 30 Days(n=888)

Remained abstinent

Newly abstinent

Total % alcoholabstinent at

follow-upN Valid % N Valid %

Alcohol Male (n = 549)

Female (n = 333)

Total (N = 882)

233187420

42.4%56.2%47.6%

13982

221

25.3%24.6%25.1%

67.7%80.8%72.7%

Alcohol to

Intoxication

Male (n = 551)

Female (n = 332)

Total (N = 883)

344245589

62.4%73.8%66.7%

13060

190

23.6%18.1%21.5%

86.0%91.9%88.2%

How to present findings How to present findings that highlight changethat highlight change

Raw numbers help, but often do not help Raw numbers help, but often do not help the reader understand the effect of the reader understand the effect of seemingly modest differences in seemingly modest differences in numbers.numbers.

Findings should include ways of showing Findings should include ways of showing the significance of change both the significance of change both statistically and through the use of statistically and through the use of different ways to think about change different ways to think about change values.values.

Rate of Percent Increase in Rate of Percent Increase in Alcohol Abstinent ClientsAlcohol Abstinent Clients

(n=888)(n=888)

59.7%**

37.8%**43.8%**

24.5%**

52.7%**

32.2%**

010203040506070

Alcohol Alcohol to intoxication

Males Females Total

Increase in Percent of Clients who Reported Abstinence from Illegal Drugs in the Past 30 Days

(Excluding Alcohol)Remained abstinent

Newly abstinent Total % abstinent at follow-

upN Valid %

of the total

sample

N Valid % of the total sample

Male (n = 553) 330 59.7% 154 27.8% 87.5%

Female (n = 332)

158 47.6% 143 43.1% 90.7%

Total (N = 885) 488 55.1% 297 33.6% 88.7%61.0%**

90.6%**

46.6%**

0 20 40 60 80 100

Males

Female

Total

Males Female Total

aSignificance established using z test for proportions.*p < .01. **p < .001

Question 3: Are there Question 3: Are there differences by drug type?differences by drug type?

Measures in KTOS instrument continue to Measures in KTOS instrument continue to drill down to greater detail on specific drill down to greater detail on specific drugs.drugs.

Differences in drug use patterns may Differences in drug use patterns may suggest differences in the rates of suggest differences in the rates of becoming abstinent as well.becoming abstinent as well.

Taking each drug separately may help Taking each drug separately may help discriminate which drugs appear more discriminate which drugs appear more difficult for recovery.difficult for recovery.

Remained abstinent

Newly abstinent Total % abstinen

t at follow-

up

N Valid % of the total

sample

N Valid % of the total

sample

Male (n = 552) 469 85.0% 66 12.0% 97.0%

Female (n = 330) 251 76.1% 70 21.2% 97.3%

Total (N = 882) 720 81.6% 136 15.4% 97.0%18.9%**

27.9%**

14.1%**

0 5 10 15 20 25 30 35

Males

Females

Total

aSignificance established using z test for proportions.*p < .01. **p < .001

Increase in Percent of Clients who Report Past 30 Day Tranquilizer Abstinence and Rate of Change

Increase in Percent of Clients who Report Past 30 Day

Opiate Abstinence and Rate of ChangeRemained abstinent

Newly abstinent Total % abstinent at follow-

upN Valid %

of the total

sample

N Valid % of the total

sample

Male (n = 551) 478 86.8% 53 9.6% 96.4%

Female (n = 331) 252 76.1% 68 20.5% 96.6%

Total (N = 882) 730 82.8% 121 13.7% 96.5%16.6%**

26.9%**

11.1%**

0 5 10 15 20 25 30 35

Males

Females

Total

aSignificance established using z test for proportions.*p < .01. **p < .001

Increase in Percent of Clients who Report Stimulant Abstinence from Intake to

Follow-up and Rate of Change

Remained abstinent

Newly abstinent

N Valid % of the total

sample

N Valid % of the total sample

Male (n = 553) 527 95.3% 19 3.4%

Female (n = 332)

305 91.9% 23 6.9%

Total (N = 885) 832 94.0% 42 4.7%98.7%**

98.8%**

98.7%**

80 85 90 95 100

Males

Females

Total

aSignificance established using z test for proportions.*p < .01. **p < .001

Sub-question: For those who Sub-question: For those who continue to use, are there any continue to use, are there any

reductions in frequencies?reductions in frequencies? Validated instruments such as the ASI use a composite Validated instruments such as the ASI use a composite

scores of past 30 days use to examine level of drug use.scores of past 30 days use to examine level of drug use. However, none of these instruments control for time in However, none of these instruments control for time in

jail or hospital.jail or hospital. For example, consider two clients, both of whom report For example, consider two clients, both of whom report

using marijuana for 15 of the past 30 days.using marijuana for 15 of the past 30 days. 1 client was in jail for 15 days in the past 30 days1 client was in jail for 15 days in the past 30 days The other client had been on the street for 30 of the past 30 The other client had been on the street for 30 of the past 30

daysdays Using the ASI, both clients would be evaluated the Using the ASI, both clients would be evaluated the

samesame In KTOS, we use a proportion of days to estimate use In KTOS, we use a proportion of days to estimate use

levelslevels Client 1 would use 100%Client 1 would use 100% Client 2 would use 50%Client 2 would use 50%

Percent Reduction in Proportion of Percent Reduction in Proportion of Past 30 Days that Substances were Past 30 Days that Substances were

Used (n=888)Used (n=888)

85.7%

52.3%

72.4%

92.0%

81.6%85.5%

0

20

40

60

80

100

Alcohol** Marijuana** Tranquilizers** Opiates** Cocaine** Stimulants*

Perc

ent

of C

lien

ts

Question 4: Does Question 4: Does Treatment Result in Treatment Result in

Increased Employment? Increased Employment? The Role of Functionality The Role of Functionality

VariablesVariables Apart from drug use changes, Apart from drug use changes,

functionality measures can be used to functionality measures can be used to partially confirm clinical results.partially confirm clinical results.

Given extensive research on substance Given extensive research on substance use and employment, the equations use and employment, the equations should be:should be: Decrease in drug use = Increased Decrease in drug use = Increased

employmentemployment Increased employment consistent with Increased employment consistent with

decreased drug usedecreased drug use

30.5%29.4%

10.9%

40.3% 40.8%

16.1%

38.7%44.6%

10.0%

54.6%

29.2%

15.2%

5

15

2535

45

55

65

Full-time Part-time Anyemployment

Unemployed Disabled Proportion ofdays worked

in last 30

Intake Follow-up

Percent of Clients in Each Employment Category

Rate of Change in Rate of Change in EmploymentEmployment

(n=888)(n=888)

↑51.7%**

↓8.3%**

↑35.5%**

↓28.4%**

↓5.6%

0 10 20 30 40 50 60

Full time

Part time

Any employment

Unemployed

Disabled

aSignificance established using z test for proportions.*p < .01. **p < .001

Question 5: Are there any Question 5: Are there any changes in arrests after changes in arrests after

treatment? Another treatment? Another Functionality VariableFunctionality Variable

As with employment, the equation As with employment, the equation typically is:typically is: Decreases in drug use = Decreases in Decreases in drug use = Decreases in

criminal activity and therefore, criminal activity and therefore, decreases in arrests.decreases in arrests.

Likewise, decreases in arrests tend to Likewise, decreases in arrests tend to confirm that decreases in substance use confirm that decreases in substance use have occurred.have occurred.

Percent of Clients Arrested in the Past 12 Months

61.0%59.5%

27.4%23.2% 27.6%

5.8%8.5%25.3%

0

20

40

60

80

100

Arrested on anycharge

Arrestedspecifically ondrug charge

Arrest on DUIcharge

Spent at least 1night in jail

Intake Follow-up

Rate of change in arrestsRate of change in arrests

↓57.5%**

↓54.8%**

↓78.8%**

↓63.4%**

40 60 80 100

Spent at least 1 night in jail

Arrest on DUI charge

Arrested specifically on drug charge

Arrested on any charge

Percent of Clients

Significance established using z test for proportions.*p < .01. **p < .001

Question 6: Are there changes Question 6: Are there changes in mental health symptoms in mental health symptoms

after treatment?after treatment? Given robust associations of mental Given robust associations of mental

health problems associated with health problems associated with substance use, the equation should substance use, the equation should be:be: Decreases in substance use should = Decreases in substance use should =

Decreases in mental health problems.Decreases in mental health problems. Likewise, decreases in mental health Likewise, decreases in mental health

problems should be consistent with problems should be consistent with decreased mental health problems.decreased mental health problems.

Changes in Mental Health Ratings(n=888)

Past 12 months Intake Follow-up

Serious depression 42.1% 35.4%

Serious anxiety 48.7% 41.1%

Trouble understanding/concentrating

41.3% 36.4%

Trouble controlling violent behavior

12.2% 9.9%

Suicidal thoughts 12.1% 7.4%

Attempted suicide 4.8% 2.0%

Prescribed psychiatric medications

31.7% 26.0%

Rate of change in MH Rate of change in MH symptomssymptoms

Significance established using z test for proportions.*p < .01. **p < .001

↓15.9%*

↓15.6%*

↓32.8%

↓11.9%

↓18.9%

↓38.8%*

↓58.3%*

↓18.0%*

0 10 20 30 40 50 60 70

Depression

Anxiety

Hallucinations

Trouble concentrating

Trouble controlling violent behavior

Suicidal thoughts

Attempted suicide

Prescribed psychiatric medication

Question 7: Are there any Question 7: Are there any cost offsets to society from cost offsets to society from

treatment?treatment? Substance abuse treatment, more Substance abuse treatment, more

than any other behavioral treatment, than any other behavioral treatment, has had to show that it saves has had to show that it saves taxpayers money.taxpayers money.

The changes in employment and The changes in employment and arrests offer the clearest behaviors arrests offer the clearest behaviors to measure in terms of cost changes to measure in terms of cost changes from intake to follow-up.from intake to follow-up.

Total Cost of Publicly-funded Treatment Services

$1,884,689

$2,252,359

$367,670

Substance abuse treatment Other MH Treatment Total Service Cost (n=888)

Past 12-month Arrest at Baseline and Follow-up and Estimated Victim

Costs of Crime (n = 888)Arrests

by type of crime

Estimated cost

per arrest

Past 12

month arrests

at intake

Cost of crimes at

intake

Past 12 month arrests

at follow-up

Cost of crimes

at follow-up

Reduction in cost

Drug $3,580 291 $1,041,780

98 $350,840 $690,940

Property

$15,418 377 $5,812,586

127 $1,958,086

$3,854,500

Violence

$35,745 166 $5,933,670

56 $2,001,720

$3,931,950

DUI $25,225 167 $4,212,575

56 $1,412,600

$2,799,975

Total 1001 $17,000,611

337 $5,723,246

$11,277,365

Reduction in Nights in Jail and Related Costs

Jail time and costs At intake

At follow-up

Reduction

Overall number of nights spend in jail in the past 12 months

32,723 9,410 23,313 (71.2%)

Annualized total estimated cost of jail stays at $38.03 per night

$891,047

$256,234 $634,813

(28.8%)

Increased Employment and Earnings (n=888)

Employment variable Baseline Follow-up

Increase

Number of clients working full or part-time

356 485 129

Annualized – days worked in the past 30 days for follow-up sample

83,508 125,496 41,988

Total annualized hours of paid work

668,064 1,003,968

335,904

Annualized total estimated labor value at $7.00 per hour times total hours

$4,676,448

$7,027,776

$2,351,328

Estimated tax value (@6%) of labor income

$280,586 $421,667

$141,080

ConclusionConclusion• The reductions in self-reported arrests for

Kentucky clients• Combined with estimates for reduced victim

costs of their crimes• Reduced costs of jail time and -• Increased earnings and tax revenues• Suggest a cost offset benefit for Kentucky

taxpayers estimated at a ratio of 5.35 to 1.

• In other words, Kentucky saved $5.35 for every dollar spent on treatment during 2004.