Defining Alcoholism Treatment Episodes from Mental Health Care Utilization Records Kurt D. Stromberg, M. S. Melanie M. Wall, Ph.D. Sandra Pothoff, Ph.D. Robert L. Kane, M.D. From the University of Minnesota School of Public Health, Division of Biostatistics (KS, MW), Division of Health Services Research and Policy (RK), Carlson School of Management (SP) Corresponding author: Melanie M. Wall, Ph.D. University of Minnesota School of Public Health Mayo Mail Code 303 420 Delaware St. SE Minneapolis, MN 55455 612-625-2138 612-626-0660 (fax) [email protected]This work was supported by a grant from the National Institute of Alcohol Abuse and Addiction (No. 1 R01 AA11781). The opinions are soley those of the authors and do not reflect official government positions.
41
Embed
Defining Alcoholism Treatment Episodes from Mental Health ... · Web viewDefining Alcoholism Treatment Episodes from Mental Health Care Utilization Records Word count of Abstract:
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Defining Alcoholism Treatment Episodes from Mental Health Care Utilization Records
Kurt D. Stromberg, M. S. Melanie M. Wall, Ph.D.Sandra Pothoff, Ph.D.Robert L. Kane, M.D.
From the University of Minnesota School of Public Health, Division of Biostatistics (KS, MW),Division of Health Services Research and Policy (RK), Carlson School of Management (SP)
Corresponding author: Melanie M. Wall, Ph.D.University of Minnesota School of Public Health
This work was supported by a grant from the National Institute of Alcohol Abuse and Addiction (No. 1 R01 AA11781). The opinions are soley those of the authors and do not reflect official government positions.
Brief title: Defining Alcoholism Treatment EpisodesNumber of words: 3,838
COMPLETE AUTHOR INFORMATION
Corresponding author: Melanie M. Wall, Ph.D.University of Minnesota School of Public Health
The diversity in encounter profiles over the entire study period among patients suggests that
any method to create episodes of care from utilization records must be flexible and allow the
researcher to explore different conditions for constructing episodes of care. For example, the
outpatient encounter profile for nine randomly selected alcoholism patients with at least one
outpatient encounter is shown in Figure 1. Outpatient encounter profiles of patients in panels A,
B, C, and E have very prominent regions of high alcoholism treatment utilization, while profiles in
panels D and F have two pronounced areas of utilization that the researcher may or may not want to
combine into one episode of treatment (Fig. 1). Patients shown in panels G, H, and I appear to
have very few outpatient encounters and may not have made a serious commitment to outpatient
alcoholism treatment (Fig. 1). Thus, a suitable mechanism for constructing episodes of care should
allow the user to explore different definitions of alcoholism treatment and evaluate the performance
of each definition.
6
Stromberg
-----------------Figure 1 here -----------------------------------
In this study, an algorithm is developed to choose the beginning and end of a treatment
episode according to three parameters inputs: 1. the minimum number of OP encounters required to
constitute an OP episode, 2. the minimum number of IP encounters required to constitute an IP
episode, and 3. the length of the clear zone (i.e. cluster of months with no encounters) for a
particular episode definition. The algorithm indicates whether each patient is treated or not
according to the specified inputs and it designates when the individual's treatment started and
ended, how many encounters it included, and whether it was an OP or IP treatment
episode. The algorithm is flexible to allow multiple treatment episodes across time within an
individual.
-----------------Table 2 here -----------------------------------
To illustrate the algorithm, Table 2 shows the outpatient alcohol encounters for three
patients for a portion of the study period between January 1995 (month 49) and April 1996 (month
64). Thus, for example, if the minimum outpatient treatment episode is defined as 3 outpatient
encounters prior to an OP-clear zone of 3 months, then patient 1 would have a single episode of OP
treatment lasting from month 50 to month 54 and containing 6 outpatient alcohol encounters. The
second patient would have a single outpatient episode lasting from month 53 to month 57
containing 56 outpatient alcohol encounters. The third patient would have two separate OP
episodes, one from month 51 to 54 containing 9 OP encounters and the second from month 61 to
7
Stromberg
62 containing 7 OP encounters. However, if the minimum outpatient episode is defined as 3
outpatient alcohol encounters prior to an OP-clear zone of only 2 months then the first patient will
now have two outpatient treatment episodes: the first during month 50 containing 3 OP encounters
and the second from month 53 to 54 containing 3 OP encounters (because of the gap of 2 zero
months between months 50 and 53). Under this second minimum OP episode definition the second
and third patients' OP episodes remain unchanged.
Impact of episode definition on number of patients with episode
The treatment episode is defined by three different parameters that can vary. Specifically,
the minimum number of IP encounters required for an IP episode was varied from 2 to 6, the
minimum number of OP encounters required for an OP episode of treatment ranged from 2 to 6,
and the length of the clear zone (IP and OP) necessary to end an episode of treatment ranged from 1
to 6 months. A minimum of 2 encounters was used because clinicians indicated that 1 OP
encounter usually means a patient was assessed but not treated, and 1 day of IP care typically
means the patient likely received only detox. Hence, 5 5 6 = 150 different episode definitions
are considered.
We first consider how each of these parameters impacts the total percentage of patients
receiving at least one episode of alcoholism treatment (IP or OP). Clearly the more restrictive
episode definitions will tend to result in fewer patients treated, however, we want to investigate
how much influence each of the parameters has on changing the percent of patients considered
treated. An ANOVA is used to quantify the variability associated with each factor in the episode
definition. The proportion of variation explained by each parameter in the episode definition is the
value of R2 associated with each component in the ANOVA. A parameter with a large R2 implies
8
Stromberg
that it has a substantial influence on the outcome (i.e. proportion of patients with at least one
episode of alcohol treatment).
Validation of episode definition
Several methods to validate the episode of care methodology were employed in this study
using the subset of patients who completed a baseline questionnaire. The baseline questionnaire
provided information regarding reasons patients sought alcoholism treatment, patient motivation
level, and whether patients received previous alcoholism treatment. This information provides a
mechanism to measure both convergent and criterion validity.
An episode definition with high convergent validity should be highly associated with
patient information known to be correlated with commitment to alcoholism treatment. For
example, patients wishing to achieve abstinence from alcohol generally have been shown to have
higher commitment to alcoholism treatment programs than those wishing only to control alcohol
use [10,11]. Thus, episode definitions with high convergent validity should indicate a strong
association between the probability of patients receiving either type of episode (IP or OP) and
whether patients sought to achieve abstinence from alcohol consumption. On the patient baseline
questionnaire, patients indicated whether they sought alcoholism treatment for legal reasons, health
reasons, to achieve abstinence, or control alcohol use. Odds ratios provide a measure of the
association between each of the reasons patients sought treatment and the probability a patient
received at least one episode of alcoholism treatment (IP or OP) under each of the different episode
definitions.
Patients also rated their motivation for completing a course of alcoholism treatment as poor,
fair, good, or excellent on the baseline questionnaire. Patients with high motivation for completing
9
Stromberg
a course of alcoholism treatment upon entry into a treatment program are more likely to stay
engaged in alcoholism treatment [12]. Thus, episode definitions with high convergent validity
should also show an increase in the likelihood of having at least one episode of alcoholism
treatment (IP or OP) as patient motivation level increases. The odds ratio of having at least one
treatment episode of either type for each motivation level (using poor motivation as the reference
group) are summarized across the different definitions.
The criterion validity of each episode definition considered was established by examining
whether treatment episodes are identified when treatments are known to have occurred. During the
baseline interview, patients were asked to report if they had previous alcoholism treatment. Based
on patient responses clinicians determined whether patients had received past treatment. Clinician
decisions on whether treatment occurred or not was considered the most accurate way of
determining treatment history. Baseline interviews did not start until 1993 (month 29 of study) and
continued until 1997 (month 85), while alcoholism utilization records were available from 1991-
1998. Thus, both positive and negative predictivity [13] provide measurements of the criterion
validity of each episode definition. Specifically, positive predictivity can be measured for each
episode definition by determining the proportion of patients who report a previous episode of
treatment when the treatment episode algorithm identifies an episode of treatment prior to baseline.
Likewise, negative predictivity is measured by determining the proportion of patients not reporting
a previous episode of treatment when the treatment episode algorithm does not indicate an episode
of treatment prior to baseline. Thus, treatment episode definitions with high criterion validity
should have both high positive and high negative predictivity.
10
Stromberg
Results
Impact of episode definition on number of patients with episode
Table 3 shows the total percentage of patients receiving at least one treatment episode of
any type (IP or OP) for a subset of 25 treatment definitions when the clear zone is held fixed at
three months. As the number of encounters required for an episode of treatment increases, the
percentage of patients receiving at least one treatment episode of any type decreases (Table 3). For
example, with a clear zone of 3 months, when only 2 IP or 2 OP encounters are required for an
episode of IP or OP treatment respectively, 68.49% of the patients would be considered treated.
However, when 6 IP or 6 OP encounters are required for an episode of IP or OP treatment
respectively, only 43.59% of the patients are considered treated. Furthermore, the decrease in the
percentage of patients with at least one alcohol treatment episode decreases faster when the number
of OP encounters increases than when the number of IP encounters increases (Table 3).
-----------------------Table 3 here ------------------
The standard deviation in the percentage of patients receiving at least one episode of either
type of treatment (IP or OP) among all the 150 episode definitions is 6.8%. An
ANOVA was used to quantify the amount of this variability explained by each of the parameters in
the definition. The proportion of variation explained by both the minimum number of OP
encounters required for an OP episode and the minimum number of IP encounters required for an
episode of IP treatment is extremely high (R2=0.995), while the clear zone accounts for less than
0.5% of the variability. This suggested that the clear zone contributes very little to the overall
variability found in the % of individuals treated so it could be fixed. In order to choose an
11
Stromberg
appropriate length of the clear zone, a residual plot from the reduced ANOVA without clear zone
included was examined. This plot shows that there is a relationship between the residuals and clear
zone length but that it is best centered around zero when clear zone is three months (Fig. 2). Thus,
in subsequent analyses, the clear zone was held fixed at three months reducing the number of
episode definitions from 150 to 25.
-----------------------Table 4 here ------------------
-----------------------Figure 2 here ------------------
Validation of episode definition
Optimal treatment episode definitions should be strongly related to validation variables.
Thus, the next step in selecting good episode definitions was to measure both the convergent and
criterion validity for each of the 25 episode definitions considered. The 25 episode definitions
arose from fixing the clear zone length at 3 months, allowing the number of OP encounters
required for an episode of OP treatment to range from 2 to 6, and allowing the number of IP
encounters required for an episode of IP treatment to range from 2 to 6.
-------------------------Figure 3 here ---------------------
Figure 3 shows the marginal odds ratio (OR) of a patient receiving at least one episode of
alcoholism treatment (IP or OP) associated with four possible reasons patients sought alcoholism
treatment for all 25 episode definitions considered. The average ORs across the 25 episode
12
Stromberg
definitions are 1.15 for legal reasons, 1.12 for health reasons, 1.78 for abstinence reasons, and 0.80
for control reasons. Thus, patients seeking treatment to achieve abstinence from alcohol
consumption were on average more likely (OR = 1.78) to receive at least one episode of alcoholism
treatment (IP or OP) than those patients not seeking treatment to achieve abstinence. Likewise,
patients indicating they were seeking alcoholism treatment only to control alcohol use were on
average less likely (OR = 0.80) to receive an episode of treatment across all 25 episode definitions
considered. Furthermore there was a strong association between patient motivation level and the
probability of receiving a treatment episode. Specifically, under all definitions patients with
excellent motivation had higher probability of being treated followed by good motivation,
followed by fair motivation.
None of the point estimates of the ORs for any of the four reasons patients sought
alcoholism treatment varied substantially among the 25 different episode definitions considered.
Consequently there is no clear winner based only on Figure 3 in terms of convergent validity (i.e.
clearly larger OR for all variables), but closer inspection of the general trends finds that the
definitions with 2 or 6 OP encounters are never best for any of the four variables. Consequently,
definitions with 3,4, or 5 OP encounters have slightly better convergent validity. Furthermore, the
definitions requiring only 2 or 3 IP encounters have slightly larger OR for all four variables and
thus better convergent validity.
----------------Figure 4 here --------------------
Calculation of the positive and negative predictivity of previous alcoholism treatment for
each of the 25 different episode definitions served as a method to compare the criterion validity for
13
Stromberg
each of the 25 treatment episodes. An episode definition with high criterion validity should have
both a positive and negative predictivity close to one. The positive predictivity for the 25 episode
definitions ranges from 0.66 to 0.79 and generally increases as the restrictiveness of the episode
definitions increase (Fig. 4A). The positive predictivity increases most sharply when the minimum
number of OP encounters required for an episode of OP treatment increases from 2 to 3 and then
remains relatively constant. Negative predictivity is nearly constant across the 25 episode
definitions and only ranges from 0.729 to 0.740 (Fig. 4B).
Discussion
Episode definitions based on utilization data facilitate the comparison of health outcomes
across clinical sites and across time since definitions of treatment may vary spatially and
temporally. Previous researchers have often studied the effectiveness of alcoholism treatment
within only one treatment center (e.g. [14]) or within the context of narrowly defined alcoholism
treatment regimes (e.g. [5]). Furthermore, alcoholism treatment has changed over time from a
higher reliance on inpatient care programs to a greater tendency to place patients in outpatient
treatment programs to contain rising costs [15,16].
This research describes a methodology to test statistically a number of different definitions
of an episode of treatment for alcoholism. The results show that 1. the definition of an episode is
insensitive to the number of months required for a clear zone of no encounters, with the ANOVA
residuals centered closest to 0 when the clear zone is 3 months, 2. convergent validity, while
similar for all definitions, is slightly better for definitions with 3-5 minimum OP encounters or 2-3
minimum IP encounters, 3. criterion validity of positive predictive value increases the most when
the minimum number of OP encounters increases from 2 to 3. Based on these results, the
14
Stromberg
definition of an episode of treatment for our subsequent research on assessing the impact of
alcoholism treatment on medical care utilization was set to a minimum of 3 IP or OP encounters
with a clear zone of 3 months.
The episode of care methodology developed in this paper allows more general inferences
regarding health outcomes to be made to larger populations of patients than may be obtained by
following specific cohorts of patients over time. Furthermore, computer based automation of
episode construction, together with the relatively inexpensive cost of obtaining utilization records
means that episodes of care can be constructed for many patients over long periods of time. The
relative ease in which episodes of care can be created with different definitions enables researchers
to begin with many candidate episode definitions and select subsets of definitions based upon the
importance of the factors composing the episode definition. The ANOVA model used in this study
provided a mechanism to quantify the relative importance of each factor in the episode definition
and subsequently greatly reduced the set of episode definitions that needed to be further examined
by fixing the clear zone length to three months. Moreover, the mechanism to evaluate both the
convergent and criterion validity of each episode definition allowed for the selection of a specific
episode definitions which can then be used for further analysis.
For alcoholism research, the episode of care provides an ideal tool for studying treatment
outcomes across clinical setting, clinical management region, or comparing pre-/post-episode
behaviors. The alcoholism treatment episode provides the exact starting time, stopping time, and
measures the intensity of alcoholism treatment regardless of the actual treatment program. For
example, this methodology could enable a better estimation of the cost offset associated with
treating alcoholism. Previous investigations of the cost offset hypothesis have often focussed on
identifying health care savings in particular cohorts of patients [16,17] where inferences may not
15
Stromberg
have wider applicability. Other cost offset studies have focussed on large cohorts of alcoholics, but
have not adequately described the period of time in which alcoholism treatment occurred. These
studies instead focussed on comparing costs before and after a single index case of alcoholism
treatment [18,19]. Use of the episode of care methodology in cost offset analyses could establish a
better criterion in which to compare pre-treatment with post-treatment health care costs across a
diverse patient population.
We chose to study only episodes consisting of all IP encounters (IP episodes) or OP
encounters (OP episodes) and did not consider episodes composed of both IP and OP encounters.
This enabled us to detect differences in both forms of treatment. Furthermore, construction of IP
and OP episodes of alcohol treatment separately could facilitate future comparisons between the
effectiveness of IP versus OP treatment, currently a research area of some debate [16,20].
However, the methodology reported here could easily be adapted to construct mixed episodes
containing both OP and IP encounters.
16
Stromberg
REFERENCES:
1. Hornbrook, M. C., Hurtado, A. V., and Johnson, R. E. . Health care episodes: definition, measurement, and use. Med. Care Rev. 1985; 42(2): 163-218.
2. Rosen, A. K., Houchens, R. L., Gibson, T. B., Mayer-Oakes, A. Developing episodes of care for adult asthma patients: a cautionary tale. Amer J. Med. Quality. 1998; 13(1): 25-35.
3. Mehta, S. S., Suzuki, S. S., Glick, H. A., Schulman, K. A. Determinining an episode of care using claims data. Diabetes Care. 1999;7(22) 1110-1115.
4. Kessler, L. G., Steinwachs, D. M., Hankin, J. R. Episodes of psychiatric utilization. Med. Care. 1980; 18(12): 1219-1227.
5. Trent, L. K. Evaluation of a four- versus six-week length of stay in the Navy's alcohol treatment program. J. Stud. Alcohol. 1998;59(3): 270-279.
6. Holder, H. D. and Blose, J. O. The reduction of health care costs associated with alcoholism treatment: A 14-year longitudinal study. Journal of Studies on Alcohol. 1992;53 293-302.
7. Lestina, D. C., Miller T. R., Smith, G. S. Creating injury episodes using medical claims data. Journal of Trauma: Injury, Infection, and Critical Care. 1998;45(3): 565-569.
8. Mitchell, J. B. et al. Using medicare claims for outcomes research. Medical Care. 1994;32(7): JS38-JS51.
9. Schulman, K. A., Yabroff, K. R., Kong, J., Gold, K. F., Rubenstein, L. E., Epstein, A. J., Glick, H. A claims approach to defining an episode of care. Health Services Research. 1999;34(2): 603-621.
10. Pachman, J. S., Foy, D. W., and Van Erd, M. Goal choice of alcoholics: a comparison of those who choose total abstinence vs. those who choose responsible, controlled drinking. J. Clin. Pyschol. 1978;34(3): 781-783.
11. Pendery, M. L., Maltzman, I. M., and West, L. J. Controlled drinking by alcoholics? New findings and a reevaluation of a major affirmative study. Science. 1982;217(4555): 169-175.
12. Walitzer, K. S., Dermen, K. H., and Conners, G. J. Strategies for preparing clients for treatment. A review. Behav. Modif. 1999; 23(1): 129-151.
13. Le, C. T. Applied Categorical Data Analysis. John Wiley and Sons, Inc. New York; 1998.
17
Stromberg
14. Holder, H. D. The cost offsets of alcoholism treatment. Recent Developments in Alcoholism. 1998;14: 361-374.
15. Schuckit, M. A., 1998. Penny-wise, ton-foolish? The recent movement to abolish inpatient alcohol and drug treatment. J. Stud. Alcohol. 1998;59: 5-6.
16. Walsh, D. C. et al. A randomized trial of treatment options for alcohol-abusing workiers. NEJM. 1991;325(11) 775-782.
17. Weisner, C., Mertens, J., Parthasarathy, S., Moore, C., Hunkeler, E. M., Hu, T., and Selby, J. V. The outcome and cost of alcohol and drug treatment in an HMO: day hospital versus traditional outpatient regimes. Health Services Research. 2000;35(4): 791-812.
18. Goodman, A. C., Tilford, J. M., Hankin, J. R.,Holder, H. D., and Nishiura, E. Alcoholism treatment offseteffects: an insurance perspective. Medical Care Research and Review. 2000; 57(1) 51-75.
19. Holder, H. D. and Blose, J. O. Alcoholism treatment and total health care utilization and costs. A four-year longitudinal analysis of federal employees. JAMA. 1986;256(11) 1456-1460.
20. Parthasarathy, S., Weisner, C., Hu, T., and Moore, C. 2001. Association of outpatient alcohol and drug treatment with health care utilization and cost: revisiting the offset hypothesis. J. Stud. Alcohol. 2001;62(1): 89-97.
18
Stromberg
Table 1: Demographic characteristics of patients in alcoholism care database.
All Subset withbaseline data
n 88,188 8,080
age(mean±sd) 40.0±11.9 40.8±9.8sex(%)F 33.2 33.3M 66.8 66.7region(%)W 15.5 6.2S 18.3 7.6MW 32.3 38.2NE 33.9 48.0alcoholism encounters>1 IP (%)1 28.9 20.2>1 OP(%)2 75.5 76.81Percentage of patients with at least 1 inpatient alcohol encounter.2Percentage of patients with at least 1 outpatient alcohol encounter.
19
Stromberg
Table 2: Sample of outpatient alcohol encounters for three patients from study month 49 (January, 1995) to study month 64 (April, 1996).
Table 3: Percentage of patients receiving at least one episode of IP or OP alcoholism treatment under different definitions. Clear zone length held fixed at three months.
Title:(S-PLUS Graphics)Creator:S-PLUSPreview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Figure 1: Outpatient encounter profile of nine randomly selected alcohol patients having at least one outpatient alcohol encounter.
22
Stromberg
Title:(S-PLUS Graphics)Creator:S-PLUSPreview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
23
Figu
re 2
: S
tand
ardi
zed
resi
dual
s vs.
clea
r zon
e le
ngth
afte
r ad
just
ing
for t
he m
inim
um n
umbe
r of O
P an
d IP
enc
ount
ers
requ
ired
for a
n O
P or
IP tr
eatm
ent e
piso
de, r
espe
ctiv
ely.
Stromberg
Title:(S-PLUS Graphics)Creator:S-PLUSPreview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
24
Figu
re 3
: Mar
gina
l odd
s rat
io o
f rec
eivi
ng a
t lea
st o
ne e
piso
de o
f alc
ohol
ism
tre
atm
ent (
IP o
r OP)
ass
ocia
ted
with
four
reas
ons p
atie
nts s
ough
t tre
atm
ent.
Leg
al
reas
ons (
A),
heal
th re
ason
s (B
), ac
hiev
e ab
stin
ence
(C),
cont
rol a
lcoh
ol u
se (D
).
Verti
cal l
ines
are
95%
poi
ntw
ise
conf
iden
ce in
terv
als f
or th
e od
ds ra
tio a
t eac
h of
ht
e 25
epi
sode
def
initi
ons c
onsi
dere
d. C
lear
zon
e le
ngth
fixe
d at
thre
e m
onth
s.
Stromberg
Title:(S-PLUS Graphics)Creator:S-PLUSPreview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.