Identifying Meaningful Patient Subgroups via Clustering - Sensitivity Graphics Bob Obenchain Commercial Information Sciences Outcomes Research, US Medical.

Identifying Meaningful Patient Subgroups

via Clustering - Sensitivity Graphics

Bob ObenchainCommercial Information Sciences

Outcomes Research, US Medical Division

Rosenbaum PR. Observational Studies, Second Edition. New York: Springer-Verlag. 2002, page 5.

In its essence, to “adjust for age” is to compare smokers and nonsmokers of the same age.

…differences … in age-adjusted mortality rates cannot be attributed to differences in age.

Adjustments of this sort, for age or other variables, are central to the analysis of observational data.

Cochran WG, Cox G.Cochran WG, Cox G. Experimental Designs 1957Experimental Designs 1957

• BlockingBlocking• RandomizationRandomization• ReplicationReplication

• Epidemiology (case-control & cohort) studies• Post-stratification and re-weighting in surveys• Stratified, dynamic randomization to improve

balance on predictors of outcome• “Full Matching” on Propensity Score estimates • Econometric Instrumental Variables (LATEs)• Marginal Structural Models (InvProbWgt 1/PS)• Unsupervised Propensity Scoring: Nested

(Treatment within Cluster) ANOVA …with LOA, LTD and Error components

Forms of Local Controlfor Human Studies

“Local” Terminology:

•Subgroups of Patients

•Subclasses…

•Strata…

•Clusters…

Source Degrees-of- Freedom Interpretation

Clusters (Subgroups)

C = Number of Clusters

Local Average Treatment Effects (LATEs) are

Cluster MeansTreatment

within Cluster

Number of “Informative” Clusters C

Local Treatment Differences (LTDs)

Error Number of Patients 2C Uncertainty

Although a NESTED model can be (technically) WRONG, it is sufficiently versatile to almost

always be USEFUL as the number of “clusters” increases.

Nested ANOVA

In general, subgroups of patients can be

considered meaningful only if patients are much

more similar within subgroups than between

subgroups.

Notation for Variables

y = observed outcome variable(s)x = observed baseline covariate(s)t = observed treatment assignment

(usually non-random)z = unobserved explanatory

variable(s)

When making head-to-head treatment comparisons,

subgroups of patients remain meaningful only if the

observed distributions of within subgroup differences in outcome due to treatment also

differ among subgroups.

When making head-to-head treatment comparisons,

subgroups of patients remain meaningful only if the

observed distributions of within subgroup differences in outcome due to treatment also

differ among subgroups.

Meaningful subgroups can only contain smaller patient subgroups that

are not meaningful.

Overshooting!!!

16 Clusters (each containing both treated

and untreated patients) in a two dimensional X-space

Different Possible LTD Distributions of Y Outcomes will be Illustrated here in the Bottom Half of the Following Slides...

These clusters may NOT be “meaningful” when the resulting distribution of

treatment differences in Y looks quite peaked:

Main Effect with Little Noise

16 Clusters (each containing both treated

and untreated patients) in a two dimensional X-space

0 Y-outcome-difference

However, these clusters are “meaningful” when the resulting distribution of

treatment differences in Y is different from the

corresponding distribution using RANDOM subgroups:

16 Clusters (each containing both treated

and untreated patients) in a two dimensional X-space

0 Y-outcome-difference

Artificial LTD Distributions fromRandom Subgroupings…

• Relatively Flat?, Smooth? or Unimodal?• Maximum Uncertainty and Bias because least

relevant comparisons are included !!!

LTD Distributions from SubgroupsRelatively Well-Matched in X-space…

• Shifted Mean? Skewness?• Distinct Local Modes?• Lower Local Variability? …Meaningful!

These clusters may be meaningful when the

distribution of treatment differences in Y looks something like this:

16 Clusters (each containing both treated

and untreated patients) in a two dimensional X-space

0 Y-outcome-difference

These 7 clusters are meaningful when the

distribution of treatment differences reveals a “local

mode” attributable to “adjacent” clusters:

16 Clusters (each containing both treated

and untreated patients) in a two dimensional X-space

0 Y-outcome-difference

Clusters could fail to remain meaningful if the “local mode” in the distribution of treatment

differences corresponds to outcomes from widely

dispersed clusters:

16 Clusters (each containing both treated

and untreated patients) in a two dimensional X-space

0 Y-outcome-difference

Subgroup-based approaches used in Observational Human Studies face “reality”:

1. (Non-parametric) Nested ANOVA models (treatment within subgroup) are “robust.”

2. Sometimes, it’s just not possible to make clearly fair comparisons!

Traditional Covariate Adjustment methods used in Randomized Clinical Trials (i.e. Least Square Means) make very strong assumptions:

1. Their (complex?) parametric models are correct.

2. Factors should compete for (causal) credit.

Global / Marginal Inference…

Difference of Overall Averages…one average for each treatment group or a simple “contrast” (single degree-of-freedom)

Local / Conditional Inference…

Distribution of Local Differences…one treatment difference within each informative subgroup of similar patients

What is a Treatment Effect?

Mortality Rates: Simpson’s Paradox

Mild Severe

W-Class 1% 6%Local 3% 9% Difference: 2% 3%

Total

4.4%3.8%

Mild Severe Total

W-Class 3/327 41/678 44/1005Local 8/258 3/33 11/291Total 11/585 44/711

Disease severity is a confounder here in the sense that it is associated with both outcome (mortality) and treatment choice (hospital.)

+0.6%

i.e. not Generalized Linear Modelsand their Nonlinear extensions.

The “statistical methodology” engine ideal for making fair treatment comparisons is:

Cluster Analysis(Unsupervised Learning)

plus Nested ANOVA

Is there statistical “theory” suggesting use of clustering to identify treatment

effects?

Fundamental PS TheoremJoint distribution of x and t given p:

Pr( x, t | p ) = Pr( x | p ) Pr( t | x, p ) = Pr( x | p ) Pr( t | x ) = Pr( x | p ) times p or (1p) = Pr( x | p ) Pr( t | p )

...i.e x and t are conditionally independent given the propensity for new, p = Pr( t = 1 | x ).

Conditioning (patient matching) on estimated Propensity Scores

implies both…

Balance: local X-covariate distributions must be the same for both treatments

and

Imbalance: Unequal local treatment

fractions unless Pr( t | p ) = p = 1p = 0.5

Pr( x, t | p ) = Pr( x | p ) Pr( t | p )The unknown true propensity

score (in non-randomized studies) is the “most coarse”

possible balancing score.

The known X-vector itself is the“most detailed” balancing score…

Pr( x, t ) = Pr( x ) Pr( t | x )

The known X-vector itself is the“most detailed” balancing score…

Pr( x, t ) = Pr( x ) Pr( t | x )

Pr( x, t | p ) = Pr( x | p ) Pr( t | p )The unknown true propensity score (in non-

randomized studies) is the “most coarse”possible balancing score.

Conditioning upon Cluster Membership is intuitively somewhere between the two PS extremes in the limit as

individual clusters become numerous, small and compact…

The known X-vector itself is the“most detailed” balancing score…

Pr( x, t ) = Pr( x ) Pr( t | x )

Pr( x, t | C ) Pr( x | C ) Pr( t | x, C ) constant Pr( t | C )

Pr( x, t | p ) = Pr( x | p ) Pr( t | p )The unknown true propensity score (in non-

randomized studies) is the “most coarse”possible balancing score.

Start by Clustering Patients in X-Space

Divisive Coefficient = 0.98

3clusters

21clusters

In Observational Human Studies, fraction treated (imbalance) varies even more from subgroup to subgroup due to:

3. Treatment Selection Biases

In Randomized Experiments, fraction treated (imbalance) will vary from subgroup to subgroup due to:

1. Bad Luck (Murphy’s Law)

2. Small Subgroups

Source Degrees-of- Freedom Interpretation

Clusters (Subgroups)

C = Number of Clusters

Local Average Treatment Effects (LATEs) are

Cluster MeansTreatment

within Cluster

Number of “Informative” Clusters C

Local Treatment Differences (LTDs)

Error Number of Patients 2C Uncertainty

Although a NESTED model can be (technically) WRONG, it is sufficiently versatile to almost

always be USEFUL as the number of “clusters” increases.

Nested ANOVA

0 = Number untreated patients in th cluster > 0in i

Nested ANOVA Treatment Difference within ith Cluster:

(1 )iPS

1 = Number treated patients in th cluster > 0in i

1 0

outcomefor a treated patient outcomefor an untreated patient

i in n

1 0 1 1/i i i i iPS n n n n

PSi Local TreatmentImbalance!

Local Control: A Subgroup / Sensitivity Graphics

approach to Robustness

• Replace covariate adjustment based upon a global model with inference based upon local clustering (sub-grouping) of patients in X-covariate space.

• Explore sensitivity by increasing the number of clusters, intentionally over-shooting, then recombining.

• Also vary distance metric and clustering method while employing computationally intensive algorithms and interactive graphical displays.

Urgent need for “Up Front” Sensitivity Analyses

Common Threads:• Many possible answers !!!• No single approach nor set of

assumptions is clearly most appropriate.

Survey of 1314 Whickham Women

y = 20 year mortality (yes or no) in 1995 follow-up study of a survey made in 1972-1974

x = age decade (20, 30, 40, 50, 60, 70 or 80) at the time of the initial survey

t = smoker or non-smoker at the time of the initial survey

Appleton DR, French JM, Vanderpump MPJ. “Ignoring a Covariate: An Example of Simpson’s Paradox” Amer. Statist. 1996; 50: 340-341.

20 Year Mortality

Rate Difference

Overall Mean and +/-Two Sigma Limits for the Distribution of

LTD Differences:

Mortality Rate of Smokers minus

that of Nonsmokers.

Number of Clusters

0.0

25.0

50.0

75.0

100.0

Y

0 20 40 60 80 100

Age

Simpson’s Paradox At Work:

Percentage of Smokers by Age Decade…and 20 Year Mortality Percentages for Smokers and Non-Smokers

LTD Distribution of Heteroskedastic Estimates…

-0.2 -0.1 0 .1 .2 .3

49 Informative Clusters for 20 Year Mortality of 1314 Wickham Women: Smokers minus Nonsmokers

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Cu

m P

rob

-0.2 -0.1 0 .1 .2 .3

CDF

KEY QUESTIONS:

• Is this distribution mostly just noise around some central value?• How many local modes might this distribution really have?• Do the Xs predict the most likely LTD for some (or all) patients?

0.00000

0.10000

0.20000

0.30000

0.40000

Y

-6.000 -4.000 -2.000 .000 2.000 4.000 6.000

X

2 3

1

|12|/

Mixture Joint Density:

• Continuing evolution in methodologies for and attitudes about analyses of non-randomized human studies

• Postpone decisions whenever available data are insufficient to provide high confidence

• Statistical methods CAN work better-and-better in the dense data limit

Future “Needs”

Hansen BB. “Full matching in an observational study of coaching for the SAT.” JASA 2004; 99: 609-618.

Fraley C, Raftery AE. “Model based clustering, discriminant analysis and density estimation.” JASA 2002; 97: 611-631.

Imbens GW, Angrist JD. “Identification and Estimation of Local Average Treatment Effects.” Econometrica 1994; 62: 467-475.

McClellan M, McNeil BJ, Newhouse JP. “Does More Intensive Treatment of Myocardial Infarction in the Elderly Reduce Mortality?: Analysis Using Instrumental Variables.” JAMA 1994; 272: 859-866.

References

McEntegart D. “The Pursuit of Balance Using Stratified and Dynamic Randomization Techniques: An Overview.” Drug Information Journal 2003; 37: 293-308.

Obenchain RL. “Unsupervised Propensity Scoring: NN and IV Plots.” 2004 Proceedings of the ASA.

Obenchain RL. “Unsupervised and Supervised Propensity Scoring in R: the USPS package” March 2005. http://www.math.iupui.edu/~indyasa/download.htm.

Rosenbaum PR, Rubin RB. “The Central Role of the Propensity Score in Observational Studies for Causal Effects.” Biometrika 1983; 70: 41-55.

Rosenbaum PR. Observational Studies, Second Edition. 2002. New York: Springer-Verlag.

References …concluded

Backup

Current GuidelineInitiatives…

• Randomized Clinical Trials

– CONSORT: www.consort-statement.org

• Observational & Non-randomized Studies

– STROBE: www.strobe-statement.org

– TREND: www.trend-statement.org

The Propensity Scoreof a Patient with Baseline Characteristic Vector x :

PS = Pr( t | x )is a vector of conditional

probabilities that sum to 1.

The length of the PS vector is the total (finite) number of different treatments.

Propensity Scoresfor only 2 Treatments:t = 1 (new) or 0 (standard)

p = Propensity for New Treatment = Pr( t = 1 | X ) = E( t | X ) = a scalar valued function of X only

X = vector of baseline covariate values for patient

PS = (p, 1p)

0 Y-outcome-difference

0 Y-outcome-difference

0 Y-outcome-difference