Accounting for Clinical Heterogeneity in Comparative Effectiveness Research How Can One Examine a Trial for Heterogeneity of Treatment Effect (HTE)? The Example of the BARI trial for CABG vs PTCA September 28, 2010 Carlos Weiss, MD, MHS
Mar 15, 2016
Accounting for Clinical Heterogeneity in Comparative Effectiveness Research
How Can One Examine a Trial for Heterogeneity of Treatment Effect (HTE)?
The Example of the BARI trial for CABG vs PTCA
September 28, 2010
Carlos Weiss, MD, MHS
AHRQ DEcIDE Project: Methods to Study the Heterogeneity of Treatment
Effects in Comparative Effectiveness Research
PI: Ravi Varadhan, PhDCo-I: Jodi Segal, MD, MPH; Cynthia Boyd, MD, MPH;
Al Wu, MD, MPH
Consultant: David Kent, MD, MPHTechnical Experts: Curt Furberg, MD, PhD; Bruce
Psaty, MD, PhDTask Order Officer: Parivash Nourjah, PhD
N=1,829
BARI Clinical Question
Population targeted: “Multivessel disease” with severe angina or ischemia
Intervention: PTCA (a form of PCI)Comparator: CABGOutcome: 5-yr Mortality
Questions to Audience - Set 1
What are sources of HTE?
How would pre-specification of analyses affect interpretation of results?
BARI Design for HTE
Protocol pre-specified 4 subgroup analyses:• angina severity
BARI Design for HTE
Protocol pre-specified 4 subgroup analyses:• angina severity• left ventricular function• number of diseased vessels• complex lesions
BARI Clinical Question: Sources of HTE in CABG vs PTCA
BARI Clinical Question: Sources of HTE in PTCA v CABG
• Patients– baseline risk– competing risks– risk of treatment harms– treatment responsiveness
>>Ideas drawn from Kravitz, Duan & Braslow, 2004, Milbank Quarterly
BARI Clinical Question: Sources of HTE in PTCA v CABG
• Patients – baseline risk– competing risks– risk of treatment harms– treatment responsiveness
• Treatment• Providers• Environments
PATIENTS
PROVIDERS ENVIRONMENTS
TREATMENT
BARI Results
5-yr Mortality:
Overall, no clinically significant nor statistically significant difference
CABG,+ treated diabetes
PTCA,+ treated diabetes
PTCA ,- treated diabetesCABG,- treated diabetes
Questions to Audience - Set 2
When should one be worried that a subgroup result is an error (Type I or Type II) ?
What can be done to lower error probabilities?
Proposed General Approach to Examining a Trial for HTE
1. HTE hypotheses pre-specified?2. Design and measurement quality?3. Modeling pre-specified?
Proposed General Approach to Examining a Trial for HTE
1. HTE hypotheses pre-specified?2. Design and measurement quality?3. Modeling pre-specified?4. If No to 1, 2 or 3:
Validation study available?
Proposed General Approach to Examining a Trial for HTE
1. HTE hypotheses pre-specified?2. Design and measurement quality?3. Modeling pre-specified?4. If No to 1, 2 or 3:
Validation study available?5.a. If frequentist, test of interaction performed?6.b. If Bayesian, pre-specified priors and
variance acceptable?
Extra Slides
What is Heterogeneity of Treatment Effect?
Non-random variability in the direction or magnitude of a treatment effect
Ris
k if
Trea
ted,
eve
nts
per 1
00 p
-y
5
10
Risk if UnTreated or “Baseline Risk”, events per 100 p-y105
Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale
Ris
k if
Trea
ted,
eve
nts
per 1
00 p
-y
5
10─ Average Absolute Treatment Effect (ARR)
Risk if UnTreated or “Baseline Risk”, events per 100 p-y105
AR
Open circles - HTE absentClosed circles - HTE present
Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale
Ris
k if
Trea
ted,
eve
nts
per 1
00 p
-y
5
10
Risk if UnTreated or “Baseline Risk”, events per 100 p-y105
- - Average Relative Treatment Effect (RRR)
Δy/Δx = RR
Open circles - HTE presentClosed circles - HTE absent
Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale