Alternatives to Randomized Trials for Estimating Treatment Efficacy (or Harm) Thomas B. Newman, MD, MPH Professor of Epidemiology and Biostatistics and.

Alternatives to Randomized Trials for Estimating Treatment Efficacy (or Harm) Thomas B. Newman, MD, MPH Professor of Epidemiology and Biostatistics and Pediatrics, UCSF AltToRcts31Oct06

Lecture Outline n Prelude Announcements Double gold standard bias n Background n Instrumental variables and natural experiments n Measuring additional unrelated variables to estimate bias n Propensity scores n Illustration using phototherapy for jaundice

Announcements n Handouts: Chapter 11, problems, this presentation n Exam question contest: questions due Thurs, 11/9/06 by e-mail to TN Real examples only, recent articles strongly preferred Include answer We may change it n Take-home final will be handed out 11/16, discussed in class 11/30

Double Gold Standard Bias Revisited n Test: ultrasound n Disease: Intussusception n 2 different gold standards. Air contrast enema Observation/follow-up n As long is there is no spontaneous resolution the two gold standards give the same answer n Spontaneous resolution gives + BE and F/U

Suppose spontaneous resolution occurs n In q cases with + U/S n In r cases with U/S n BE will be + for q and r n F/U will be for q and r

Effect of double gold-standard

Alternatives to Randomized Trials for Estimating Treatment Efficacy (or Harm) Thomas B. Newman, MD, MPH Professor of Epidemiology and Biostatistics and Pediatrics, UCSF AltToRcts31Oct06

Background n Why do RCTs? Assemble comparable groups (avoid confounding) Allow blinding (to avoid placebo effect, cointerventions, and bias in measuring outcome variable) n Observational studies May be able to assemble comparable groups or use statistical adjustment Wont be blinded

Why is it hard to assemble comparable groups without randomizing? n People who get treated differ from those who dont n Important differences are with respect risk of the outcome Treated people often at higher risk (confounding by indication for treatment). Treated people may be at lower risk (selection bias)

Pre-test n Observational studies can never establish causation. Proof of causation requires randomized trials. n How many have heard this? n How many agree?

Do you believe there is a causal relationship between n Acetaminophen overdose or mushroom poisoning and liver failure? n Wearing glasses for refractive errors and improved vision? n Infiltrate of IV calcium infusion and skin sloughing? n Receipt of fluids and recovery from dehydration? n Land mine explosions and limb injuries?

Post-test n Observational studies can never establish causation. Proof of causality requires randomized trials. n How many agree?

When is causal inference from observational studies easy? n Outcomes not related to indications for treatment rarely if ever occurs spontaneously highly localized in time or space n Treatment well-understood biologically very rapidly acting

When its hard: n Outcomes are related to indications or selection for treatment, are delayed, non specific, or not well understood Learning disabilities in children treated with anticonvulsants Suicide in users of antidepressants Mortality after surgery for gastroesophageal reflux in children

Natural Experiments and Instrumental Variables n Find a time or place where receipt of treatment was unlikely to be related to prognosis E.g., time-series analyses where something changed (e.g. new intervention became available) n Instrumental variables (IV): measurable factors that influence probability of treatment that are not otherwise associated with outcome

Use of large databases n Allows use of (weak) surrogate measures for actual predictor n Biased towards null n Achieve statistical significance with large sample size n Algebraically reverse bias towards null (with various assumptions)

Delayed Effects of the Military Draft on Mortality n Origin of study: Agent Orange concern n Design: Randomized natural experiment using the draft lottery n Data source: computerized death certificate registries, CA and PA n Predictor variable of interest: military service Hearst N, Newman TB, Hulley SB. NEJM 1986; 314:620-24

Why not compare outcomes according to the predictor variable of interest? n Biased comparison those who serve in the military start out healthier n Healthy warrior effect

Delayed Effects of the Military Draft on Mortality n The instrumental variable measured: draft lottery number below cutoff (based on date of birth) n IV associated with predictor variable of interest, not independently associated with outcome

BUT: Having an eligible number was a poor measure of military service:

Results

RCT as an Instrumental Variable: Health effects of exclusive breast feeding n Cant do RCT of exclusive breast-feeding n Can do RCT of breast-feeding PROMOTION n Assignment to BF promotion group should be associated with exclusive breast feeding, but not independently associated with outcome n Need very large sample size n Algebraic correction

Promotion of Breastfeeding Intervention Trial (PROBIT) n Cluster-randomized trial at 31 sites in Belarus n Subjects 17,046 term singleton infants >2500g initially breastfed n Intervention: WHO/UNICEF Baby Friendly Hospital Initiative n Outcomes: BF @ 3,6,9,12 months and allergic, gastrointestinal and respiratory disease n F/U to 12 months on 16,491 (96.7%) Kramer MS, et al. JAMA 2001;285:413-20.

PROBIT, RQ #1 n Does a Baby Friendly Hospital increase exclusive breastfeeding? Predictor = Group assignment Outcome = Exclusive breast feeding Intention-to-treat (ITT) analysis is fine n Exclusive BF at 3 months (rounded) 40% vs 5%; P < 0.001

Probit RQ#2 n Does exclusive breastfeeding reduce the risk of eczema in the infant? n If the only effect of intervention related to eczema is increasing exclusive BF, then Predictor = Group assignment Outcome = Eczema ITT analysis: biased towards null; informative if study positive n Eczema 3.3% vs 6.3%; adjusted OR = 0.54 (95% CI 0.31-.95 based on GLIMMIX; P = 0.03)

PROBIT, RQ#3 n How much does exclusive breastfeeding reduce the risk of eczema in the infant? (What is the NNEBF*? ) Predictor = Group assignment Outcome = Eczema ITT wont work -- too much misclassification. (Gives the number needed to be exposed to the intervention, not the NNEBF.) *Number Needed Exclusively to Breast Feed

Algebraic correction n If all of the difference in eczema is due to the difference in exclusive breast feeding, it can be shown that the ARR is

NNEBF and caveat n Since ARR = 8.6%, NNEBF to prevent 1 case of eczema is about 1/.086 = 12 n Caveats: Results are for the effect of breastfeeding in response to the intervention Assumes the only effect of the Baby Friendly Hospital is via difference in exclusive breastfeeding n Similarly, effects of draft lottery only apply to those who served as a result of the lottery.

Summary/other examples n If variables known NOT to be associated with outcome are associated with treatment of interest, consider this approach. n Generalizes to manynatural experiments. E.g., an intervention is intermittently available, or only available to certain groups. -- different outcome by day of the week, etc.

More natural experiments: n Costs of discontinuity of care: increased laboratory test ordering in patients transferred to a different team the next morning* n Effect of ER Copay: rate of appendicitis perforation unchanged after increase in copay.** n Aircraft cabin air recirculation and symptoms of the common cold: no difference by type of air recirculation in aircraft *** * Lofgren, RO. J Gen Intern Med. 1990;5:501-5 **Hsu J, et al. Presented at Bay Area Clinical Research Symposium 10/17/03 *** Zitter JN et al. JAMA 2002;288:483-6

Unrelated variables to estimate bias or confounding n Measure an outcome that WOULD be affected by bias, but not by intervention (and see if it is) n Measure a predictor that WOULD cause the same bias as the predictor of interest (and see if it does)

Observational study of screening sigmoidoscopy n Possible bias: patients who agree to sigmoidoscopy are likely to be different n Solution: measure an outcome that would be similarly affected by bias n Results: Decreased deaths from cancers within the reach of the sigmoidoscope (OR= 0.41) No effect on deaths from more proximal cancers (OR= 0.96). Selby et al, NEJM 1992;326:653-7

Effect of British breathalyser crackdown n Abrupt drop in accidents occurring during weekend nights (when pubs are open) n Measure an outcome that would be affected by bias: accidents during other times n Result: No change in accidents occurring during other hours See Cook and Campbell: Quasi- Experimentation.Boston:Houghton Mifflin, p. 219

Calcium Channel Blockers (CCB) and AMI n Population based case-control study at Group Health n Progressive increase in risk of AMI with higher doses of CCB (P

Alternatives to Randomized Trials for Estimating Treatment Efficacy (or Harm) Thomas B. Newman, MD, MPH Professor of Epidemiology and Biostatistics and.

Documents

n people

n outcomes

intussusception n

n land

n infiltrate

r n fu

announcements n handouts

natural experiments