Design of Clinical Trials for Treatment of Invasive Fungal Infections John H. Powers, MD FACP FIDSA Senior Medical Scientist SAIC in support of Collaborative.

Design of Clinical Trials for Design of Clinical Trials for Treatment of Invasive Treatment of Invasive

Fungal InfectionsFungal Infections

John H. Powers, MD FACP FIDSAJohn H. Powers, MD FACP FIDSASenior Medical ScientistSenior Medical Scientist

SAIC in support of Collaborative Clinical Research BranchSAIC in support of Collaborative Clinical Research BranchDivision of Clinical ResearchDivision of Clinical Research

National Institute of Allergy and Infectious DiseasesNational Institute of Allergy and Infectious DiseasesNational Institutes of HealthNational Institutes of Health

2

DisclosuresDisclosures

Consultant for:Consultant for:AcureonAcureon Johnson and Johnson Johnson and Johnson

Astra-ZenecaAstra-Zeneca Merck MerckCentegenCentegen Methylgene MethylgeneCerexaCerexa Octoplus OctoplusCoNCERTCoNCERT Takeda TakedaDestinyDestiny Theravance TheravanceForestForest Wyeth Wyeth

3

IntroductionIntroduction

Why is appropriate design of trials Why is appropriate design of trials important?important?

How do clinical practice and clinical How do clinical practice and clinical research differ?research differ?

What are the principles for designing an What are the principles for designing an adequate and well-controlled, internally adequate and well-controlled, internally valid clinical trial?valid clinical trial?

How can we do better to address these How can we do better to address these issues?issues?

4

Why is Design Important?Why is Design Important? Four possible reasons for the results of a trial:Four possible reasons for the results of a trial:

1.1. Random errorRandom error – results due to chance alone – results due to chance alone

2.2. BiasBias – systematic error that results in deviation of – systematic error that results in deviation of results from “true” results (inaccurate measurement)results from “true” results (inaccurate measurement)

3.3. ConfoundingConfounding – – error where the measured result is the actual error where the measured result is the actual

measure but not causally related to treatment measure but not causally related to treatment receivedreceived

factors such as disease severity are not factors such as disease severity are not “confounders” in randomized trials, but effect “confounders” in randomized trials, but effect modifiersmodifiers

If the above reasons ruled out then………If the above reasons ruled out then………

5

Why is Design Important?Why is Design Important?

4. Valid results4. Valid results – “validity” means ability of – “validity” means ability of study to measure what is purports to study to measure what is purports to measuremeasure

Internal validityInternal validity – ability of study to measure – ability of study to measure what it purports to measurewhat it purports to measure

External validityExternal validity – ability to generalize – ability to generalize (transfer) results to population rather than just (transfer) results to population rather than just sample measuredsample measured

A trial that does not have internal validity A trial that does not have internal validity cannot have external validitycannot have external validity

6

Why is Design Important?Why is Design Important? Random error addressed by adequate sample sizeRandom error addressed by adequate sample size

P value addresses probability results may be due to P value addresses probability results may be due to chancechance

Does not address likelihood that hypothesis is likely trueDoes not address likelihood that hypothesis is likely true

Bias and confounding addressed by Bias and confounding addressed by appropriate appropriate designdesign, no statistical fix after the study is over, no statistical fix after the study is over

Increased sample size can Increased sample size can increaseincrease effects of bias and effects of bias and confounding on resultsconfounding on results

Only way to obtain valid results is Only way to obtain valid results is through appropriate design, conduct through appropriate design, conduct and analysis of trialand analysis of trial

7

Why is Design Important?Why is Design Important? Invalid clinical trial results can lead to Invalid clinical trial results can lead to

important clinical consequences:important clinical consequences:

Ineffective therapies used widely in patients Ineffective therapies used widely in patients ( cannot “figure it out later” since difficult to ( cannot “figure it out later” since difficult to determine cause and effect in individual patients)determine cause and effect in individual patients)

Unwarranted harms to patients in absence of Unwarranted harms to patients in absence of benefitsbenefits

Emergence of resistance and elimination of Emergence of resistance and elimination of benefits for other patientsbenefits for other patients

Ethical issues of exposing subjects to harm in Ethical issues of exposing subjects to harm in scientifically invalid researchscientifically invalid research Belmont Report, Belmont Report, Ethical Principles and Guidelines for Ethical Principles and Guidelines for

Research Involving Human SubjectsResearch Involving Human Subjects http://http://ohsr.od.nih.gov/guidelines/belmont.htmlohsr.od.nih.gov/guidelines/belmont.html

8

Ioannidis JP PLoS Medicine 2005;2(8):e124Ioannidis JP PLoS Medicine 2005;2(8):e124

9

Clinical Trials and Clinical Clinical Trials and Clinical PracticePractice

Clinical practice and clinical research differClinical practice and clinical research differ

Clinical practiceClinical practice based on “interventions based on “interventions designed solely to enhance the well-being designed solely to enhance the well-being of an individual patient or clients and that of an individual patient or clients and that have reasonable expectation of success”have reasonable expectation of success” Belmont Report p.3, Belmont Report p.3, Ethical Principles and Ethical Principles and

Guidelines for Research Involving Human Guidelines for Research Involving Human SubjectsSubjects

Clinical researchClinical research is “activity designed to is “activity designed to test an hypothesis” in groups of subjects test an hypothesis” in groups of subjects and “thereby to develop or contribute to and “thereby to develop or contribute to generalizable knowledge”generalizable knowledge” Belmont Report Belmont Report

http://ohsr.od.nih.gov/guidelines/belmont.htmlhttp://ohsr.od.nih.gov/guidelines/belmont.html

10

Clinical Trials and Clinical Clinical Trials and Clinical PracticePractice

Question Question notnot whether individual clinician whether individual clinician believes drug will be effective for individual believes drug will be effective for individual patient in clinical practicepatient in clinical practice

Questions is how to study drug to demonstrate Questions is how to study drug to demonstrate safety and effectiveness in group of patients in safety and effectiveness in group of patients in a clinical trial to then generalize to clinical a clinical trial to then generalize to clinical practicepractice

Medical need is reason to do a trial, not a Medical need is reason to do a trial, not a reason to accept invalid trials or lesser evidencereason to accept invalid trials or lesser evidence

Designing trials based on previously held beliefs Designing trials based on previously held beliefs in absence of evidence does not allow gathering in absence of evidence does not allow gathering of evidence to validate those beliefsof evidence to validate those beliefs

11

What are the Principles?What are the Principles?

1.1. Clear statement of objectives of the trialClear statement of objectives of the trial

2.2. Study design permits valid quantitative Study design permits valid quantitative comparison with a controlcomparison with a control

3.3. Select patients with disease (treatment) or Select patients with disease (treatment) or at risk of disease (prevention)at risk of disease (prevention)

4.4. Baseline comparability (randomization)Baseline comparability (randomization)

5.5. Minimize bias (blinding, etc.)Minimize bias (blinding, etc.)

6.6. Appropriate methods of assessment of Appropriate methods of assessment of outcomesoutcomes

7.7. Appropriate methods of analysisAppropriate methods of analysis

8. 8. Appropriate measurement of potential harmsAppropriate measurement of potential harms

12

How Can We Do Better?How Can We Do Better?1) Clear objective:1) Clear objective:

Define diseaseDefine disease and and clinical time course – clinical time course – mixing together various mixing together various infections makes interpretation of results challenginginfections makes interpretation of results challenging

Differentiate Differentiate treatmenttreatment from from preventionprevention trials – “salvage” vs primary trials – “salvage” vs primary treatmenttreatment

DifferentiateDifferentiate explanatory trials explanatory trials from from strategy/managementstrategy/management trials trials Differentiate measurement of Differentiate measurement of effectivenesseffectiveness from measurement of from measurement of

harmsharms Better Better natural history datanatural history data – what is an “invasive” infection? Does – what is an “invasive” infection? Does inin

vitro vitro resistance affect clinical outcomes and by how much?resistance affect clinical outcomes and by how much? Allows for better enrollment criteria, more homogeneous population, Allows for better enrollment criteria, more homogeneous population,

less variability, and appropriate timing of outcomesless variability, and appropriate timing of outcomes

2) Quantitative comparison with control2) Quantitative comparison with control Absence of control makes it challenging to assess causality of Absence of control makes it challenging to assess causality of

outcomesoutcomes Choice of control: no treatment, placebo, dose response, active, Choice of control: no treatment, placebo, dose response, active,

historicalhistorical Choice of study design: superiority, non-inferiorityChoice of study design: superiority, non-inferiority

13

Quantitative Comparison with a Quantitative Comparison with a ControlControl

Many ID clinical trials designed as “noninferiority” Many ID clinical trials designed as “noninferiority” (NI) trials(NI) trials

Misconceptions about goals of NI trialsMisconceptions about goals of NI trials Rule out margin by which test intervention may be Rule out margin by which test intervention may be less less

effectiveeffective than control intervention than control intervention Does not show that experimental intervention is “as good Does not show that experimental intervention is “as good

as” or “equivalent” to control unless shows statistical as” or “equivalent” to control unless shows statistical superioritysuperiority

Experimental intervention can be statistically Experimental intervention can be statistically inferior/superior and “noninferior” at same time as long as inferior/superior and “noninferior” at same time as long as not not moremore inferior than margin specified prior to trial inferior than margin specified prior to trial

Designing a noninferiority trial means one is willing Designing a noninferiority trial means one is willing to accept less effectiveness with the experimental to accept less effectiveness with the experimental intervention (for what trade off?)intervention (for what trade off?)

14

Designing a Valid Noninferiority Designing a Valid Noninferiority TrialTrial

1. Quantitative 1. Quantitative assessment that is assessment that is reliable and reproduciblereliable and reproducible (based (based on trials that are themselves adequate and well controlled) of on trials that are themselves adequate and well controlled) of benefit of control over placebo and suitably conservative benefit of control over placebo and suitably conservative evaluation examining variability (not just point estimates)evaluation examining variability (not just point estimates)

2. Maintenance of the effect of the control from trial to trial 2. Maintenance of the effect of the control from trial to trial (constancy assumption)(constancy assumption) Similar definition of disease, endpoints, timing of endpointsSimilar definition of disease, endpoints, timing of endpoints Changes in medical practice, adjunctive therapies, antimicrobial Changes in medical practice, adjunctive therapies, antimicrobial

resistanceresistance

3. Selection of margin of loss of effect of control that is less than 3. Selection of margin of loss of effect of control that is less than the benefit of control over placebo found in step 1the benefit of control over placebo found in step 1

International Conference on Harmonization Guidance E-10, International Conference on Harmonization Guidance E-10, Choice of Control Group and Related Issues in Clinical TrialsChoice of Control Group and Related Issues in Clinical Trials, , www.ich.orgwww.ich.org

15

Designing a Valid Noninferiority Designing a Valid Noninferiority TrialTrial

If these conditions not met, demonstration of similarity If these conditions not met, demonstration of similarity means experimental and control intervention may be means experimental and control intervention may be similarly effective or similarly ineffectivesimilarly effective or similarly ineffective

Experimental intervention may not be any more effective Experimental intervention may not be any more effective than placebo even if control agent previously effectivethan placebo even if control agent previously effective

Link to external “negative control” data in NI trials similar Link to external “negative control” data in NI trials similar to external (historical) trials with similar biasesto external (historical) trials with similar biases

Other forms of bias in NI trials beyond “statistical” issuesOther forms of bias in NI trials beyond “statistical” issues Not ensuring subjects have disease under studyNot ensuring subjects have disease under study Blinding less effective at preventing bias since investigators Blinding less effective at preventing bias since investigators

know all subjects receiving active interventionknow all subjects receiving active intervention Greater bias due to inappropriate conduct of trials, Greater bias due to inappropriate conduct of trials,

concomitant medications, missing data, etc.concomitant medications, missing data, etc.

16

How Can We Do Better?How Can We Do Better?3) Selection of subjects with disease (treatment) 3) Selection of subjects with disease (treatment)

or at risk of diseaseor at risk of disease Rapid diagnosticsRapid diagnostics which evaluate host response as which evaluate host response as

well as presence of organismswell as presence of organisms BiomarkersBiomarkers can be useful in diagnosis but in presence can be useful in diagnosis but in presence

of signs and symptoms of disease (positive predictive of signs and symptoms of disease (positive predictive value of test related to pre-test probability)value of test related to pre-test probability)

Better current Better current natural history datanatural history data in prevention in prevention trials to better select populations at risktrials to better select populations at risk

4) Baseline comparability using 4) Baseline comparability using Randomization controls for selection bias as well as Randomization controls for selection bias as well as

measured and unmeasured confounders; basis for measured and unmeasured confounders; basis for statisticsstatistics

Appropriate development of Appropriate development of “severity” classifications“severity” classifications (comparing baseline variables to clinical outcomes) to (comparing baseline variables to clinical outcomes) to stratify subjects at baseline and decrease variabilitystratify subjects at baseline and decrease variability

17

How Can We Do Better?How Can We Do Better?

5) Minimizing bias5) Minimizing bias

Blinding Blinding of microbiological data to persons assessing of microbiological data to persons assessing outcome in situations where impact of outcome in situations where impact of in vitroin vitro resistance resistance on clinical outcomes is unclearon clinical outcomes is unclear Could have unblinded third parties assess culture results in Could have unblinded third parties assess culture results in

serious diseasesserious diseases Will allow correlation of clinical outcomes with in vitro testing Will allow correlation of clinical outcomes with in vitro testing

to better define “resistance”to better define “resistance” Evaluate clinical outcome at time of culture result in any caseEvaluate clinical outcome at time of culture result in any case

Control for Control for concomitant medicationsconcomitant medications

Minimize Minimize loss to follow-uploss to follow-up and and missing datamissing data

18

How Can We Do Better?How Can We Do Better?6) More accurate and sensitive outcome measures6) More accurate and sensitive outcome measures

Effect of antimicrobials in severe disease based upon Effect of antimicrobials in severe disease based upon decrease in decrease in all-cause mortalityall-cause mortality

Biomarkers can make it Biomarkers can make it more difficultmore difficult to show effects in to show effects in some diseases since adds another criteria to assessment of some diseases since adds another criteria to assessment of outcomesoutcomes

Develop Develop well-defined clinical outcome criteriawell-defined clinical outcome criteria independent independent of “clinician judgment” (can cause misclassification bias and of “clinician judgment” (can cause misclassification bias and increased variability = increased sample size) based on increased variability = increased sample size) based on natural history of diseasenatural history of disease

Expert outcome assessment does not eliminate bias and Expert outcome assessment does not eliminate bias and calls into question generalizability of resultscalls into question generalizability of results

Timing of outcomes - Time to event analysesTiming of outcomes - Time to event analyses in superiority in superiority trials can inform duration of therapy, increase power to trials can inform duration of therapy, increase power to detect differences, decrease sample size, and answer detect differences, decrease sample size, and answer clinically relevant question on magnitude of effectclinically relevant question on magnitude of effect

19

Multiple/Composite EndpointsMultiple/Composite Endpoints

All cause mortalityAll cause mortality

Non-fatal clinical eventsNon-fatal clinical events

Symptoms of diseaseSymptoms of disease

Surrogate endpointsSurrogate endpoints

Lubsen J et al. Stat Med 2003;21:2159-70.Lubsen J et al. Stat Med 2003;21:2159-70.

Interested in multipleInterested in multipleaspects of how diseaseaspects of how disease

may affect patients’may affect patients’liveslives

20

Multiple/Composite EndpointsMultiple/Composite Endpoints

All cause mortalityAll cause mortality

Non-fatal clinical eventsNon-fatal clinical events

Symptoms of diseaseSymptoms of disease

Surrogate endpointsSurrogate endpoints

Lubsen J et al. Stat Med 2003;21:2159-70.Lubsen J et al. Stat Med 2003;21:2159-70.

SuccessSuccess based on based onevents from lowerevents from lower

on hierarchyon hierarchyshould not should not supersede supersede

failurefailure based on events based on eventshigher up on hierarchyhigher up on hierarchy

that occur that occur during during course of trialcourse of trial

even when surrogate even when surrogate is used as part of is used as part of primary outcomeprimary outcome

21

How Can We Do Better?How Can We Do Better?7) Appropriate analysis7) Appropriate analysis

Decrease proportions of subjects who are “indeterminate” or Decrease proportions of subjects who are “indeterminate” or “unevaluable” by eliminating inappropriate exclusions from “unevaluable” by eliminating inappropriate exclusions from “per protocol” analysis – all events post-randomization “per protocol” analysis – all events post-randomization includedincluded

Evaluation of the intent to treat, modified intent to treat Evaluation of the intent to treat, modified intent to treat analysis protects against selection bias, maintains integrity of analysis protects against selection bias, maintains integrity of randomizationrandomization

Appropriate adjustments for multiple comparisons in Appropriate adjustments for multiple comparisons in secondary endpoints and subgroup analysessecondary endpoints and subgroup analyses

Use of “gate-keeper” step wise hypothesis testing to control Use of “gate-keeper” step wise hypothesis testing to control for false positive results but requires for false positive results but requires a prioria priori specification of specification of order of hypothesis testingorder of hypothesis testing

22

8. Analysis of Harms8. Analysis of Harms SafetySafety analysis requires an adequate number of subjects to analysis requires an adequate number of subjects to

assess adverse eventsassess adverse events

““Rule of threes” – measurement of no events in a given trial Rule of threes” – measurement of no events in a given trial allows rule out rate of 3 divided by number of subjects studied allows rule out rate of 3 divided by number of subjects studied (3/300 = 1%)(3/300 = 1%)

Not evaluating “statistical significance” of harms since not Not evaluating “statistical significance” of harms since not testing a hypothesis in most clinical trials, but developing a testing a hypothesis in most clinical trials, but developing a hypothesishypothesis

Overall assessment of risks and benefits depends upon nature Overall assessment of risks and benefits depends upon nature and magnitude of bothand magnitude of both Greater risks acceptable when treatment has large effect on Greater risks acceptable when treatment has large effect on

clinically important endpoints like deathclinically important endpoints like death Serious adverse events less acceptable when benefits small Serious adverse events less acceptable when benefits small Unacceptable if benefits compared to placebo unclearUnacceptable if benefits compared to placebo unclear

23

ConclusionsConclusions

Need to accept that we can improve on Need to accept that we can improve on current level of evidence, answer questions current level of evidence, answer questions that are still unclearthat are still unclear

Many opportunities to develop more Many opportunities to develop more clinically relevant and more efficient clinical clinically relevant and more efficient clinical trialstrials

Result can be more information for clinicians Result can be more information for clinicians and patients, optimal use of antimicrobials and patients, optimal use of antimicrobials by describing who benefits, by how much by describing who benefits, by how much and with quantitative comparison to risksand with quantitative comparison to risks