Introduction to Randomized Clinical Trials Stephen Bent Associate Professor of Medicine University of California, San Francisco.

Introduction toIntroduction toRandomized Clinical TrialsRandomized Clinical Trials

Stephen BentStephen Bent

Associate Professor of MedicineAssociate Professor of Medicine

University of California, San University of California, San FranciscoFrancisco

Randomized TrialsRandomized Trials

–RationaleRationale–Basic designsBasic designs–ParticipantsParticipants–InterventionIntervention–BlindingBlinding–OutcomesOutcomes–AdherenceAdherence–Follow-upFollow-up

Take Home MessagesTake Home Messages

•1. RCT’s really only do one thing1. RCT’s really only do one thing–But, it’s a very important thing.But, it’s a very important thing.

•2. RCT’s are the BEST study 2. RCT’s are the BEST study design for evaluating efficacy.design for evaluating efficacy.

•RCT’s compare mean responsesRCT’s compare mean responses–Not always what we’d like to knowNot always what we’d like to know

•RCT’s take a long time and cost $RCT’s take a long time and cost $

Take Home MessagesTake Home Messages

•What is bias?What is bias?

•What is confounding?What is confounding?

RationaleRationale

•Why do a randomized blinded trialWhy do a randomized blinded trial–minimize confounding!!!minimize confounding!!!–minimize co-interventionsminimize co-interventions–minimize biased outcome ascertainmentminimize biased outcome ascertainment

•Why not do a randomized trialWhy not do a randomized trial–major ethical issuesmajor ethical issues–narrow research questionnarrow research question–expensiveexpensive–long time from idea to paperlong time from idea to paper

•Generally reserved for mature Generally reserved for mature questionsquestions

Basic Trial DesignBasic Trial Design

PopulationPopulation

SampleSample

InterventionIntervention

RandomizationRandomization

OutcomeOutcomeOutcomeOutcome

ControlControlControlControl OutcomeOutcomeOutcomeOutcome

+ Blinding+ Blinding+ Blinding+ Blinding

PlaceboPlacebo

RandomizationRandomization

•Participants are assigned to Participants are assigned to treatment groups by chance treatment groups by chance with a known probabilitywith a known probability

•Random number table or computerRandom number table or computer

•Tamper-proof systemTamper-proof system–ordered, sealed envelopes ordered, sealed envelopes –centralized system (phone, fax, centralized system (phone, fax, web)web)

Value of RandomizationValue of Randomization

•Balances Balances baselinebaseline characteristics of the characteristics of the treatment groupstreatment groups–eliminates confounding due to eliminates confounding due to measured measured andand unmeasuredunmeasured factors factors

–provides an unbiased comparison provides an unbiased comparison between groupsbetween groups

•Does Does NOTNOT maintain balance maintain balance after randomizationafter randomization

Variations of RandomizationVariations of Randomization

•Fixed Allocation - probability Fixed Allocation - probability fixedfixed–Simple - flip a coinSimple - flip a coin–Blocked - randomize consecutive small Blocked - randomize consecutive small batchesbatches

–Stratified - separate randomization in Stratified - separate randomization in stratastrata

–Clustered - randomize groupsClustered - randomize groups

•Adaptive - probability changes Adaptive - probability changes

Cross-over DesignCross-over Design

PopulationPopulation

SampleSample

InterventionIntervention

RandomizationRandomization

PlaceboPlacebo

WashoutWashout

WashoutWashout

PlaceboPlacebo

InterventionIntervention

OutcomeOutcome OutcomeOutcome

Factorial DesignFactorial Design

PopulationPopulation

SampleSample

Int A and Int BInt A and Int B

Int A and Pbo BInt A and Pbo B

Pbo A and Int BPbo A and Int B

Pbo A and Pbo BPbo A and Pbo B

OutcomeOutcome

OutcomeOutcome

OutcomeOutcome

OutcomeOutcome

ParticipantsParticipants

•Inclusion criteria to Inclusion criteria to maximize:maximize:–rate of outcomes (old, weak)rate of outcomes (old, weak)–likely benefit from interventionlikely benefit from intervention–generalizabilitygeneralizability–ease of recruitmentease of recruitment

Exclusion CriteriaExclusion Criteria

• Intervention unsafeIntervention unsafe• Intervention unlikely to be effectiveIntervention unlikely to be effective• Unlikely to adhere to the Unlikely to adhere to the interventionintervention– Run inRun in

• Unlikely to complete follow-upUnlikely to complete follow-up

• Practical problemsPractical problems

Practice Parsimony

Preserve Generalizability

Practice Parsimony

Preserve Generalizability

Choice of InterventionChoice of Intervention

•MaximizeMaximize–effectiveness effectiveness (highest (highest tolerable dose)tolerable dose)

–safety safety (lowest effective dose)(lowest effective dose)–generalizabilitygeneralizability–trial design/conducttrial design/conduct

•recruitmentrecruitment•compliancecompliance•blindingblinding

Choice of ControlChoice of Control

•Inert placebo usually bestInert placebo usually best

•Active therapy or “standard of Active therapy or “standard of care”care”

CointerventionsCointerventions

•Unintended effective Unintended effective interventionsinterventions–participantsparticipants use other therapy use other therapy or change behavioror change behavior

–study staff, medical providers, study staff, medical providers, family or friendsfamily or friends treat treat participants differentlyparticipants differently

•Nondifferential - decreases Nondifferential - decreases powerpower

•Differential - causes biasDifferential - causes bias

Biased Outcome Biased Outcome AscertainmentAscertainmentBiased Outcome Biased Outcome AscertainmentAscertainment

• If group assignment is known If group assignment is known – participantsparticipants may report may report symptoms or outcomes symptoms or outcomes differentlydifferently

– physicians or investigatorsphysicians or investigators may elicit symptoms or may elicit symptoms or outcomes differently outcomes differently

• If group assignment is known If group assignment is known – participantsparticipants may report may report symptoms or outcomes symptoms or outcomes differentlydifferently

– physicians or investigatorsphysicians or investigators may elicit symptoms or may elicit symptoms or outcomes differently outcomes differently

Canadian Cooperative MS TrialCanadian Cooperative MS TrialCanadian Cooperative MS TrialCanadian Cooperative MS Trial

•165 patients with multiple 165 patients with multiple sclerosissclerosis–plasma exchange + cyclo + predplasma exchange + cyclo + pred–sham plasma exchange + placebo medssham plasma exchange + placebo meds

•Outcome = structured neurologic Outcome = structured neurologic exam by blinded and unblinded exam by blinded and unblinded neurologistsneurologists

•More improvement with plasma More improvement with plasma exchange by unblinded, but not exchange by unblinded, but not blinded assessmentblinded assessment

•165 patients with multiple 165 patients with multiple sclerosissclerosis–plasma exchange + cyclo + predplasma exchange + cyclo + pred–sham plasma exchange + placebo medssham plasma exchange + placebo meds

•Outcome = structured neurologic Outcome = structured neurologic exam by blinded and unblinded exam by blinded and unblinded neurologistsneurologists

•More improvement with plasma More improvement with plasma exchange by unblinded, but not exchange by unblinded, but not blinded assessmentblinded assessmentNoseworthy, Neurology, 1994

Noseworthy, Neurology, 1994

Biased Outcome AdjudicationBiased Outcome Adjudication

•Study staff who decide if a Study staff who decide if a change or outcome has occurred change or outcome has occurred maymay–classify similar events classify similar events differently in treatment groupsdifferently in treatment groups

•Problematic with “soft” Problematic with “soft” outcomesoutcomes–investigator judgementinvestigator judgement–participant reported symptoms, participant reported symptoms, scalesscales

Why Not Blind?Why Not Blind?

• ImpossibleImpossible– surgerysurgery– exerciseexercise– dietdiet– educationeducation

• Possible, butPossible, but– dangerousdangerous– painfulpainful– cumbersomecumbersome

Is It Really Blinded? Is It Really Blinded? Is It Really Blinded? Is It Really Blinded?

• Difficult even for drugsDifficult even for drugs– identical placebo difficult to identical placebo difficult to prepareprepare

– drug may smell, taste, feel drug may smell, taste, feel differentdifferent

– drug may cause side effectsdrug may cause side effects– test results may unblindtest results may unblind– participants may test drugparticipants may test drug

• Difficult even for drugsDifficult even for drugs– identical placebo difficult to identical placebo difficult to prepareprepare

– drug may smell, taste, feel drug may smell, taste, feel differentdifferent

– drug may cause side effectsdrug may cause side effects– test results may unblindtest results may unblind– participants may test drugparticipants may test drug

What if You “Can’t” What if You “Can’t” Blind?Blind?

Do the best you canDo the best you can–minimize differential minimize differential cointerventioncointervention

–blind those measuring outcome blind those measuring outcome –use “hard” outcomesuse “hard” outcomes

Measure degree of unblindingMeasure degree of unblinding

Be CourageousBe Courageous

•Laparoscopic lysis of adhesions Laparoscopic lysis of adhesions for pelvic painfor pelvic pain

•Internal mammary ligation for Internal mammary ligation for anginaangina

•Orthoscopic debridement for OAOrthoscopic debridement for OA

•Sham burr holes for fetal Sham burr holes for fetal tissue implants for Parkinson’stissue implants for Parkinson’s

Do the Best You CanDo the Best You Can

•Exercise to prevent coronary Exercise to prevent coronary eventsevents–exercise - supervised exercise to exercise - supervised exercise to 80% maximum capacity 30 min 3/wk80% maximum capacity 30 min 3/wk

–control - supervised exercise to control - supervised exercise to 40% maximum capacity 30 min 3/wk40% maximum capacity 30 min 3/wk

•Psychotherapy for schizophreniaPsychotherapy for schizophrenia–therapy - psychotherapy weeklytherapy - psychotherapy weekly–control - advice about diet, control - advice about diet, exercise, and smoking weeklyexercise, and smoking weekly

Use a “Hard” OutcomeUse a “Hard” Outcome

• DeathDeath

• MeasurementsMeasurements– test resultstest results

• MVOMVO2 2 vs.. self-reported exercise vs.. self-reported exercise abilityability

• Doppler evaluation vs.. swollen Doppler evaluation vs.. swollen leg for DVT leg for DVT

– scales and diaries vs. investigator scales and diaries vs. investigator judgmentjudgment• Geriatric Depression Scale vs. Geriatric Depression Scale vs. “improved”“improved”

• 7-day urinary diary vs. “dry”7-day urinary diary vs. “dry”

AdherenceAdherence

•Intervention cannot work if it Intervention cannot work if it isn’t usedisn’t used

•Adherence measuresAdherence measures–interventionintervention•pill count, diaries, biologic pill count, diaries, biologic measure, measuring device in measure, measuring device in dispenserdispenser

–visitsvisits–study measurementsstudy measurements

Women’s Health Initiative Women’s Health Initiative

RQRQ: Does calcium plus vitamin D reduce : Does calcium plus vitamin D reduce risk of fractures in postmenopausal risk of fractures in postmenopausal women?women?

DesignDesign: Randomized trial: Randomized trial

Subjects:Subjects: 36,282 PM women enrolled in 36,282 PM women enrolled in WHIWHI

Intervention:Intervention: 1 gm calcium + 400 IU 1 gm calcium + 400 IU vitamin Dvitamin D

OutcomeOutcome: clinical fractures: clinical fractures

Adherence at end of trial 60% and about Adherence at end of trial 60% and about 60% of placebo group was taking calcium60% of placebo group was taking calcium

Outcomes in Clinical TrialsOutcomes in Clinical Trials

•Efficacy OutcomesEfficacy Outcomes–PrimaryPrimary–SecondarySecondary–SurrogateSurrogate–CompositeComposite

•Adverse EffectsAdverse Effects–rarerare–commoncommon

Adverse Events and Side EffectsAdverse Events and Side Effects

• AnticipatedAnticipated– use specific questionsuse specific questions

• UnanticipatedUnanticipated– ask about general adverse ask about general adverse experiencesexperiences

• RareRare– sample size inadequatesample size inadequate

• CommonCommon– multiple differences between multiple differences between groupsgroups

A Randomized Controlled A Randomized Controlled Trial of a Chinese Herbal Trial of a Chinese Herbal Remedy to Increase Energy, Remedy to Increase Energy, Memory, Sexual Function, Memory, Sexual Function, and Quality of Life in and Quality of Life in

Elderly Adults in Beijing, Elderly Adults in Beijing, ChinaChina

Stephen Bent, MDStephen Bent, MD

Osher Center for Integrative Osher Center for Integrative MedicineMedicine

University of California, University of California, San FranciscoSan Francisco

Longevity TreasureLongevity Treasure

•Proprietary extractProprietary extract

•10 Chinese herbs10 Chinese herbs

•Believed to increase “Yang”Believed to increase “Yang”

•Marketed to improveMarketed to improve–Energy, Memory, Sexual Function, Energy, Memory, Sexual Function, QiQi

•Widespread use in ChinaWidespread use in China

•US sales of over $1 million US sales of over $1 million

Research QuestionResearch Question

•Does the daily use of Does the daily use of Longevity Treasure lead Longevity Treasure lead to changes in energy, to changes in energy, memory, sexual memory, sexual function, qi, or function, qi, or quality of life?quality of life?

MethodsMethods

•Design: Randomized Design: Randomized Controlled TrialControlled Trial

•ParticipantsParticipants–Chinese residents of Beijing, Chinese residents of Beijing, age age >> 60 60

–Self-reported decreased energy, Self-reported decreased energy, memory, or sexual interestmemory, or sexual interest

•Recruitment – “word of Recruitment – “word of mouth”mouth”

Exclusion CriteriaExclusion Criteria

•High YangHigh Yang

•Serious medical illnessSerious medical illness

•Currently taking Longevity Currently taking Longevity TreasureTreasure

InterventionIntervention

•Random assignment toRandom assignment to

–Longevity Treasure, 4 Longevity Treasure, 4 capsules three times a day capsules three times a day (30 days)(30 days)

–Identical placebo, 4 Identical placebo, 4 capsules, three times a day capsules, three times a day (30 days)(30 days)

OutcomesOutcomes

•Assessed at baseline and 30 Assessed at baseline and 30 daysdays

•PrimaryPrimary–Change in quality of life, SF-12 Change in quality of life, SF-12 questionnairequestionnaire

–Change in quality of life, Qi Change in quality of life, Qi scalescale

Secondary Outcome MeasuresSecondary Outcome Measures

•Energy – questionnaireEnergy – questionnaire

•Energy – physical testsEnergy – physical tests–6 minute walk6 minute walk–Step testStep test–Grip strengthGrip strength–Chair standsChair stands–Foot tapFoot tap

•Memory – word and picture recallMemory – word and picture recall

•Sexual function – 3-item Sexual function – 3-item questionnairequestionnaire

Study FlowStudy Flow

E xc lud e d (n = 8 2)H ig h Y a n g (6 5)

H T N (1 2 )O th e r (5 )

A n a lyze d (n= 1 1 5)

W ith d re w (n = 4)S id e e ffec t (2 )L e ft a re a (2 )

P la ceb o (1 1 9)

A n a lyze d (n= 1 0 8)

W ith d re w (n = 1 0)S id e e ffec t (8 )L e ft a re a (2 )

H e rb s (1 1 8)

R a nd o m ized(n = 2 3 7)

A sse sse d fo r e lig ib ility(n = 3 1 9)

Baseline CharacteristicsBaseline Characteristics

Characteristic Placebo Herbs P-value

Age 65.6 66.4 0.23

Women (%) 62.2 64.4 0.72

Decreased energy (%) 21.0 25.4 0.42

Decreased memory (%) 81.5 86.4 0.30

Decr. sexual interest (%) 97.5 94.1 0.19

Qi score 13.8 15.8 0.01

Results: Primary OutcomesResults: Primary Outcomes

•SF-12, Mental Component (Baseline SF-12, Mental Component (Baseline = 53)= 53)–Herbs: + 4.4Herbs: + 4.4–Placebo: +2.5Placebo: +2.5–Difference: +1.9 points (95% CI, 0.1 Difference: +1.9 points (95% CI, 0.1 to 3.6)to 3.6)

•SF-12, Physical Component SF-12, Physical Component (Baseline = 50)(Baseline = 50)–Herbs: +1.6Herbs: +1.6–Placebo: +1.7Placebo: +1.7–Difference: -0.1 (95% CI, -1.7 to Difference: -0.1 (95% CI, -1.7 to 1.5)1.5)

Primary Outcome: Qi scalePrimary Outcome: Qi scale

0

0.5

1

1.5

2

2.5

3

3.5

Unadjusted Adjusted

HerbPlacebo

Secondary Outcome MeasuresSecondary Outcome Measures

Measure (range) Baseline Herb Placebo P-value

Memory (0-39) 21.3 +4.5 +4.2 0.51

Physical test (-16 to 8) 0.3 -0.1 -0.1 0.8

Sexual function (0-26) 4.1 -0.5 -0.9 0.17

SF-36 Vitality 74.8 +6.1 +5.0 0.39

SF-36 Mental Health 81.0 +7.0 +5.2 0.04

Adverse EventsAdverse Events

Event Placebo (%) Herb (%) P-value

Diarrhea 3.4 % 1.7% 0.41

URI 1% 1% 1.00

Headache 0% 1% 0.31

Dry mouth 5.9% 12.7% 0.07*

Total 19% 29% 0.24

SummarySummary

•Longevity Treasure may improve Longevity Treasure may improve mental healthmental health–2 point increase on SF-12 mental 2 point increase on SF-12 mental health scorehealth score

–Similar improvement on SF-36 subscaleSimilar improvement on SF-36 subscale

•The benefit, if any, is smallThe benefit, if any, is small

•Longevity Treasure does not improveLongevity Treasure does not improve–MemoryMemory–Sexual FunctionSexual Function–Energy or QiEnergy or Qi

ConclusionsConclusions

•Longevity Treasure cannot be Longevity Treasure cannot be strongly recommended without strongly recommended without further supportive evidencefurther supportive evidence

•RCTs of Chinese herbs are RCTs of Chinese herbs are feasiblefeasible

•More work is needed to More work is needed to explore Chinese concepts of explore Chinese concepts of quality of life and qiquality of life and qi

A randomized controlled trial A randomized controlled trial of saw palmetto for the of saw palmetto for the

treatment of benign prostatic treatment of benign prostatic hyperplasiahyperplasia

Stephen Bent, MDStephen Bent, MD

Christopher Kane, Christopher Kane, MDMD

Katsuto Shinohara, Katsuto Shinohara, MDMD

John Neuhaus, PhDHarley Goldberg, DOAndrew L. Avins, MD,

MPHUniversity of California, San Francisco

Kaiser Permanente Northern California, Division of Research

Funded by the National Institute of Diabetes, Digestive and Kidney Diseases (NIDDK) and the

National Center for Complementary and Alternative Medicines (NCCAM)

Saw palmetto (Saw palmetto (serenoa serenoa repensrepens))

• Many patients seek an alternative to drug Many patients seek an alternative to drug therapytherapy

• Used daily by 1.1% of the US adult population Used daily by 1.1% of the US adult population

• Widely used in EuropeWidely used in Europe

MethodsMethods

• Randomized, double-blind, placebo-Randomized, double-blind, placebo-controlled trialcontrolled trial

• Patients recruited from Kaiser Patients recruited from Kaiser Permanente, Northern California and San Permanente, Northern California and San Francisco VAMCFrancisco VAMC– Inclusion:Inclusion:

• Age Age >> 50 50• AUASI AUASI >> 8 (range 0-35) 8 (range 0-35)• Peak flow > 4 and < 15 mls/secPeak flow > 4 and < 15 mls/sec

– Exclusion: Exclusion: • Prior prostate cancer or prostate surgeryPrior prostate cancer or prostate surgery• Using alpha-antagonist, 5-ARI, or saw palmetto Using alpha-antagonist, 5-ARI, or saw palmetto (washout permitted)(washout permitted)

• Severe concomitant illnessSevere concomitant illness

InterventionsInterventions

•Saw palmetto 160mg bid for one Saw palmetto 160mg bid for one yearyear–Indena (extract, 92% free fatty Indena (extract, 92% free fatty acids)acids)

–Cardinal Health (encapsulation)Cardinal Health (encapsulation)–Rexall Sundown (packaging)Rexall Sundown (packaging)

•Placebo: polyethylene glycol-Placebo: polyethylene glycol-400 and brown coloring agent400 and brown coloring agent

STEP Study Flow ChartSTEP Study Flow Chart

Saw palmetto 160 mg twice a day

Placebo capsule twice a day

12 mo.

6 mo.

9 mo.

3 mo.

1 mo.

0 wks.

-4 wks.

-6 wks.

Timeline

Washout

Run-in

Visit

Enrollment Visit

Randomized Treatment Period

Visits

Randomization Visit

Screening Period Visits

BBaseline Characteristicsaseline Characteristics

Characteristic Saw Palmetto (N=112)

Placebo (N=113)

Age (yr) 62.9 63.0

White 84% 79%

AUA Symptom Index 15.7 15.0

Prostate Volume (ml)

34.7 33.6

Maximal Urinary Flow Rate (ml/s)

11.4 11.6

Serum PSA (ng/dl) 1.8 1.6

Results: AUASI Results: AUASI

p = 0.99

1415

1617

18A

UA

SI

0 5 10 15Month

Saw palmetto Placebo

Randomization

Results: Maximum Flow Rate Results: Maximum Flow Rate

p = 0.37

10.5

1111

.512

12.5

Maxim

um

Uri

ne F

low

Rate

0 5 10 15Month

Saw Palmetto Placebo

Randomization

ConclusionsConclusions

•No evidence of benefit in No evidence of benefit in AUASI, Qmax, or other AUASI, Qmax, or other measured outcomemeasured outcome

•No evidence of harm; no No evidence of harm; no safety concernssafety concerns

•Discrepancy between our Discrepancy between our findings and others’ findings and others’ results needs explorationresults needs exploration

Internet-Based RCT’sInternet-Based RCT’s

•Exciting potentialExciting potential

•Reach unlimited # of Reach unlimited # of patientspatients

•Lower costLower cost

•Much shorter time to Much shorter time to completioncompletion

•Participate from home, Participate from home, convenientconvenient

Kava and Valerian for Anxiety Kava and Valerian for Anxiety and Insomnia:and Insomnia:

An Internet-based Randomized An Internet-based Randomized Blinded TrialBlinded Trial

Figure 1.

REMOTE StudyREMOTE Study

•Tolterodine ER (Detrol) Tolterodine ER (Detrol) for OABfor OAB

•11stst Internet-based RCT of Internet-based RCT of drugdrug

•https://oab.mytrus.com/home

•Great idea, but many Great idea, but many barriers currentlybarriers currently

Take Home MessagesTake Home Messages

•1. RCT’s really only do one thing1. RCT’s really only do one thing–But, it’s a very important thing.But, it’s a very important thing.

•2. RCT’s are the BEST study 2. RCT’s are the BEST study design for evaluating efficacy.design for evaluating efficacy.

•RCT’s compare mean responsesRCT’s compare mean responses–Not always what we’d like to knowNot always what we’d like to know

•RCT’s take a long time and cost $RCT’s take a long time and cost $

Take Home MessagesTake Home Messages

•Bias = a systematic Bias = a systematic deviation of an deviation of an observation from the true observation from the true clinical stateclinical state

•Confounder = a variable Confounder = a variable that is related to the that is related to the predictor and a cause of predictor and a cause of the outcomethe outcome