Performance and Outcome Measures

Performance and Outcome Measures

Why is Drug Court Data Important to Me?

• Courts that were evaluated and used the results to make changes to their court had 85% greater reductions in recidivism.

• These same courts had 100% higher cost savings than courts who did NOT use evaluations to modify their programs.

Source: NPC Research – Top 10 Drug Court Best Practices and More! What Works? New Findings from the Latest Research http://www.npcresearch.com/Files/Conference%20presentations/2012_Best_Practices_Top_10.pdf

“Performance is your reality.Forget everything else.”

- Harold S. Geneen

Performance Measurement Quotes

“It doesn't matter how valid your excuses, they will never change your performance.”

- Unknown

Performance Measurement Quotes

“Where performance is measured,

performance improves. Where performance is

measured andreported, the rate of

improvement accelerates.”

- Thomas S. Monson

Trends in Performance &

Outcome Measures

• Ensure the court is targeting prison-bound offenders (legislative intent)

• Drug courts show the greatest benefits for high risk offenders - those with more prior felony convictions, with relatively more severe antisocial backgrounds or treatment-resistant histories.*

• These high-risk offenders are the most in need of the intensive supervision services as embodied in the 10 Key Components of drug courts.*

Target Population

* Targeting the Right Participants for Adult Drug Courts: Part One of a Two-Part Series – Douglas B. Marlowe, JD, PhD

FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%

10%

20%

30%

40%

50%46%

41%

28% 30% 29%26%

23% 25%28% 29%

22%

FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120

25

50

75

52 51

3943

5348

42

5053

FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

66% 64% 66%64% 60% 62% 63% 66%

FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

83%

72%

85%92%

84%89% 87% 90%

82%90% 92%

FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%

10%

20%

30%

40%

50%

11%

19%

33%

19%

10%

18%23% 25%

23%

29%

40%

FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

8%

24%29%

78%

63%56% 58%

55%48%

FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%

20%

40%

60%

80%

100%

120%

11%

34%

52%

80% 83% 80%

108% 112%

86%

101%96%

FY06 FY07 FY08 FY09 FY10 FY11 FY120

0.5

1

1.5

2

2.5

3

3.5

4

0.30.57

0.98 1.17

1.86

2.61

3.26

FY06 FY07 FY08 FY09 FY10 FY11 FY120

3

6

9

12

7.6

9.68.8 8.8

9.310.4 10.8

FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%

2%

4%

6%

8%

10%8.7%

7.6%

4.8%

3.4%

4.9%5.9%

6.8%

4.5%5.2%

CurrentPerformance &

OutcomeMeasures

Male Female Total0%

25%

50%

26%

43%

31%

2%

8%4%

92.3

%

81.4

%

87.1

%

Entry Graduation % Decrease

Male Female Total$0

$500

$1,000

$1,500

$2,000

$886

$388

$826

$1,630

$955

$1,54783.9% 146.3% 87.2%

Entry Graduation % Increase

Male Female Total0%

25%

50%

25%

32%27%

18%22% 19%

29.0

%

33.0

%

30.4

%

Entry Graduation % Decrease

Male Female Total0%

25%

50%

75%

100%

38% 40% 39%

55%68%

60%44

.7%

70.0

%

53.8

%

Entry Graduation % Increase

Percent of Admissions by Drug of Choice and Fiscal Year

FY2002 FY2003 FY2004 FY2005 FY2006 FY2007 FY2008 FY2009 FY2010 FY2011 FY20120%

5%

10%

15%

20%

25%

30%

35%

40%

Alcohol Cannabis CocaineMethamphetamine Prescription Other

METH Busters

Meth abusers will not do well in Drug Court.

10%

25%

50%

75%

100%

70% 73%

60%66%

Methamphetamine AlcoholCannabis Total

Meth Total0%

25%

50%

41%

31%

3% 4%

92.7

%

87.1

%

Entry Graduation % Decrease

Meth Total0%

25%

50%

29% 27%

19% 19%35

.9%

30.4

%

Entry Graduation % Decrease

Meth Total0%

25%

50%

75%

100%

30%39%

57% 60%

90.0

%

53.8

%

Entry Graduation % Increase

0%

25%

50%

75%

100% 93%

36%

90%82%

22%27%

87%

39% 39%

88%

28%

55%

Methamphetamine Alcohol Cannabis Total

Meth abusers will not do well in Drug Court.

Performance & Outcome MeasuresThrough

AdministrativeData Matching

• Sustainable and Ongoing • Provides Long-Term Outcome Measures• Empirically-Based & Objective• Less Resource Intensive• No Sampling or Response Rate Issues• Incremental for Populations & Time

Frames• Unobtrusive• Allows Examination of Overlap Between

Agency Populations

Matching ProcessData

Extraction

External agency staff extract appropriate data from databases

Data transferred to

ODMHSAS staff

Data Cleaning

ODMHSAS staff clean database to remove suffixes from names, bad characters

Data Matching

External agency records are matched to themselves to identify duplicates

Manual review of names and other

fields

DataMatching

External agency data are matched to ODMHSAS data based on Name, DOB, Sex, SSN

DataAnalysis

Analysis assists in identification of overlap, differences between groups, and possible outcome measures

Manual analysis to

verify accuracy of

matching process

ODMHSAS staff reviews

results, information provided to

external agency staff

for review

DataDistribution

Results are provided to appropriate stakeholder groups

• ODMHSAS – State & Medicaid funded Substance Abuse and Mental

Health Treatment Services• OSBI – Arrests• DOC – Incarcerations, Probation &

Parole• OESC – Wage & Unemployment

Benefits• OSDH – Mortality Data• DHS – TANF, Food Stamps, Child

Welfare • County Jails – Arrest and Booking

Information

Drug Court Graduates Successful Standard Probation Offenders

Released Inmates0%

20%

40%

60%

23.5%

38.2%

54.3%

2004 2005 2006 20070%

2%

4%

6%

8%

10%

8.2%

6.6%

8.7%

6.0%The rate for all

2,094 graduates represented in

this chart is 7.1%.

N = 293

N = 437

N = 484

N = 880

Drug Court Graduates Released Inmates0%

10%

20%

30%

6.0%

24.0%

Matching Drug Court Participant Data

with Oklahoma Employment Security Commission • Purpose

– Research/Cost-Benefit Analysis– Validation/Verification of Drug Court Coordinator

Data in DC WEBS• Research Questions

– What percentage of participants are employed before, during and after drug court?

– What is the income of participants before, during and after drug court?

– How much income is earned through wages from drug court graduates?

– How much tax revenue is generated from drug court graduates?

1 Year Prior to Admission

At Admission 1 Year Post Admission

At Graduation - approx 20

months Post Admission

2 Years Post Admission

3 Years Post Admission

0.0%

25.0%

50.0%

75.0%

100.0%

58.2%

45.5%

71.0% 75.7% 77.8% 80.0%

At Admission At Graduation 0.0%

25.0%

50.0%

75.0%

100.0%

50.6%

73.7%

45.5%

75.7%

Coordinator OESC

1 Year Prior to Admission

At Admission 1 Year Post Admission

At Graduation - approx 20 months Post Admission

2 Years Post Admission

3 Years Post Admission

$0

$5,000

$10,000

$15,000

$20,000

$10,926

$13,332 $13,735

$16,388$17,439 $18,498

At Admission At Graduation $0

$5,000

$10,000

$15,000

$20,000

$9,131

$16,386

$13,332

$16,388

Coordinator OESC

1 Year Post Admission

2 Years Post Admission

3 Years Post Admission

$0

$3,000,000

$6,000,000

$9,000,000

$6,537,853

$8,405,269 $8,545,757

$23.5 million in total wages

were earned over a three year pe-riod after admis-sion to

drug court.

1 Year Post Admission

2 Years Post Admission

3 Years Post Admission

$0

$200,000

$400,000

$600,000

$373,631

$489,514 $499,533

$1.4 million in total tax revenue was

expected to be generated over

a three year period after

admission to drug court.

Series1

-$40,000,000

-$30,000,000

-$20,000,000

-$10,000,000

$0

$10,000,000

$20,000,000

$30,000,000

-$38,190,000

-$5,583,333

$23,488,879(Wages)

$482,400

$1,362,678(Taxes)

Cost of DOC Wages 3 Years Post

DOC Admis-sion

Cost of Drug Court

Wages & Taxes 3

Years Post Drug Court Admission

Drug Court Data Collection

Importance of Data Collection

How ODMHSAS Utilizes DC WEBS Data1. Provides drug court team with data about

• performance and outcome measures compared to other courts and the state

• demographics• offense types• drug of choice• violations/sanctions and incentives

2. Demonstrates to the legislature and other stakeholders the effectiveness of drug courts

3. Determines court’s funding4. Program monitoring 5. Potential grant funding

Drug Court Outcomes & Funding• Percent of Participants without Prior Felony

Sentences– Percent of admissions with no priors (the lower the

better)• Completion Rate

– Percent of discharges that are grads• Employment

– Percent of graduates that are unemployed at graduation (the lower the better)

• Percent without H.S. Diploma/GED– Percent of graduates that do not have a H.S.

Diploma/GED at graduation (the lower the better)• Percent of Children Living with Participant

– Percent of the participants’ children that are living with them at graduation (the higher the better)

Percent of participants without priors over current FY state averageOR

0 of 4 outcomes better than/at state average (employment, children living with parents, education, or completion rate)

Percent of participants without priors less than current FY state average

AND2 of 4 outcomes data better than/at state average (employment,

children living with parents, education, or completion rate)

Select the outcomes category for the drug court

YTD Outcomes (for completion rate last 2

fiscal years’ data is used)

Percent of participants without priors less than current FY state average

AND1 of 4 outcomes data better than/at state average (employment,

children living with parents, education, or completion rate)

Percent of participants without priors over current FY state averageAND

0 of 4 outcomes data better than/at state average (employment, children living with parents, education, or completion rate)

Active Counts & Funding• Run monthly on the 3rd of each month• Only records that have been updated in the

last 60 days are counted as active• The monthly active count includes all

participants that have been active at least one day during the month– Ex. Participant is pled in on the last day of the

month– Ex. Participant graduates on the first day of the

month– BOTH are counted as active for the month

Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun30

35

40

45

50

55

60

65

70

75

80

67

43

Below Tier Within Tier Above Tier Active

Participants in red will not be counted as active for funding purposes.

Performing Your Own Evaluation

Why is Drug Court Data Important to Me?

• Courts that were evaluated and used the results to make changes to their court had 85% greater reductions in recidivism.

• These same courts had 100% higher cost savings than courts who did NOT use evaluations to modify their programs.

Source: NPC Research – Top 10 Drug Court Best Practices and More! What Works? New Findings from the Latest Research http://www.npcresearch.com/Files/Conference%20presentations/2012_Best_Practices_Top_10.pdf

Using Your DC WEBS Data

• Show community your court’s successes

• Track your own program’s progress• Apply for grants• Your court’s state funding is

dependent on data!

Target Population

• Demographics Report

• Criminal Justice Report

Compare Demographics to the general criminal justice population to make sure your court isn’t missing out on including certain groups.

• Ensure that you are targeting prison-bound offenders (legislative intent)

• “Research reveals that it is these high-risk offenders who are most in need of the intensive supervision services embodied in the 10 Key Components of drug courts.”*

• “Drug Courts have been shown to produce the greatest benefits for offenders who have relatively more severe antisocial backgrounds or treatment-resistant histories.”*

Target Population

* Targeting the Right Participants for Adult Drug Courts: Part One of a Two-Part Series – Douglas B. Marlowe, JD, PhD

Performance & Outcomes

• Outcomes List Report

• Performance and Outcome Report

Available

upon request

Another way to check target population.

Check performance measures.

Check outcome measures.

Your court’s outcomes are also sent out quarterly.

Goals and Plans• Develop goals for your court to improve

target population, performance and outcomes.– Good goals for outcomes for adult drug courts

is the state totals because they are what are used for funding.

• Identify and implement strategies to attain your goals– Your field reps are good resources for ideas

• Check your data periodically to see if you have reached your goals

Program Assistance

Effective Data Reporting and

Collection

Outcome Reports to Assist in Program

MonitoringProgram Monitoring

Improved Outcomes

Time Is Up!.

Contact Information:

David Wright, Ph. [email protected]

Nancy [email protected]

Questions or problems with CDC entry or billing contact:Local Number: 405-521-6444Toll Free [email protected]