An Update on FDAs Critical Path Initiative Statistical Contributions Robert T. ONeill Ph.D. Director, Office of Biostatistics Center for Drug Evaluation.

An Update on FDA’s Critical An Update on FDA’s Critical Path InitiativePath Initiative

Statistical ContributionsStatistical Contributions

Robert T. O’Neill Ph.D.Robert T. O’Neill Ph.D.

Director , Office of BiostatisticsDirector , Office of Biostatistics

Center for Drug Evaluation and Center for Drug Evaluation and ResearchResearch

Presented at the 2005 FDA/Industry Statistics Workshop: September 14-16, 2005

Marriott Wardman Park Hotel, Washington, DC

The Critical Path InitiativeThe Critical Path Initiative

Refers to the product development path Refers to the product development path from candidate selection to product launchfrom candidate selection to product launch

Covers drugs, biologics, and medical Covers drugs, biologics, and medical devices – but today’s talk is mostly devices – but today’s talk is mostly about drugs / biologicsabout drugs / biologics

Initiative was announced publicly by Dr. Initiative was announced publicly by Dr. McClellan Tuesday, March 16, 2004McClellan Tuesday, March 16, 2004

What the “Critical Path” What the “Critical Path” IsIs

A serious attempt to bring attention & focus to A serious attempt to bring attention & focus to the need for more scientific effort and publicly-the need for more scientific effort and publicly-available information on evaluative toolsavailable information on evaluative tools

Evaluative tools: The techniques & Evaluative tools: The techniques & methodologies needed to evaluate the safety, methodologies needed to evaluate the safety, efficacy & quality of pharmaceuticals as they efficacy & quality of pharmaceuticals as they move down the pathmove down the path

Despite Advances in Science, Despite Advances in Science, Success Success

Rate of Product Development Rate of Product Development hashas

NOT Improved NOT Improved

New compounds entering Phase I New compounds entering Phase I development today have 8% chance development today have 8% chance of reaching market, vs. 14% chance of reaching market, vs. 14% chance 15 years ago.15 years ago.

Phase III failure rate now reported Phase III failure rate now reported to be 50%, vs. 20% in Phase III, 10 to be 50%, vs. 20% in Phase III, 10 years ago.years ago.

Perceived Problem:Perceived Problem: The development process The development process itself is becoming a serious itself is becoming a serious

bottleneckbottleneck Current applied science and infrastructure date from last Current applied science and infrastructure date from last

centurycentury

Funding and progress in Development science has not kept Funding and progress in Development science has not kept pace with basic biomedical science.pace with basic biomedical science.

Science to evaluate safety and efficacy of potential new Science to evaluate safety and efficacy of potential new medical products, and enable manufacture, is medical products, and enable manufacture, is differentdifferent from from basic discovery science.basic discovery science.

Need to fill gap in applied science-- to increase productivity Need to fill gap in applied science-- to increase productivity and efficiency --to reduce cost of development process.and efficiency --to reduce cost of development process.

Stakeholder Input: Stakeholder Input: Overwhelming SupportOverwhelming Support

Overwhelming concurrence with:Overwhelming concurrence with:

recognition of science infrastructure recognition of science infrastructure problem problem

CP Initiative focus on research and CP Initiative focus on research and collaboration, collaboration,

We heard this fromWe heard this from: drug industry, patient : drug industry, patient groups, device companies and groups, groups, device companies and groups, biotech companies, othersbiotech companies, others

This is what we heard !This is what we heard !Demand Exceeds SupplyDemand Exceeds Supply

Docket Demand for FDA Action Docket Demand for FDA Action Exceeds FDA Capacity: Far more Exceeds FDA Capacity: Far more proposed than FDA can undertake.proposed than FDA can undertake.

Principles for setting priorities for Principles for setting priorities for FDA actions are on Science Board FDA actions are on Science Board agenda.agenda.

Overriding ConcernsOverriding Concerns

Clinical TrialsClinical Trials

Biomarkers and Biomarkers and EndpointsEndpoints

What is the problemWhat is the problem

Phase III trials are failing at a rate that Phase III trials are failing at a rate that is higher than expected - root causes ?is higher than expected - root causes ?

What is the typical planning process What is the typical planning process for drug development / phase 3 trialsfor drug development / phase 3 trials

What can we change; what new tools What can we change; what new tools can we use, and what can we do better can we use, and what can we do better in the future to improve Phase III in the future to improve Phase III success and efficiency of drug success and efficiency of drug developmentdevelopment

Possible solutions / Possible solutions / strategies strategies

Can statisticians help ?Can statisticians help ?

Are new study designs neededAre new study designs needed Impetus for Adaptive designs, two stage designs, enriched Impetus for Adaptive designs, two stage designs, enriched

target population designstarget population designs

Are we planning correctly - Rethink how Are we planning correctly - Rethink how the study planning process occursthe study planning process occurs It’s the doseIt’s the dose

It’s the scenario needing better planning - or analysis It’s the scenario needing better planning - or analysis methodsmethods

Bring consensus / closure to most pressing statistical issues at Bring consensus / closure to most pressing statistical issues at the core of decision makingthe core of decision making

Get involved in new emerging subject matter areas and impact Get involved in new emerging subject matter areas and impact them -genomics, proteonomics, nanotechnologythem -genomics, proteonomics, nanotechnology

Broaden the multi-disciplinary roles, in industry, academia and Broaden the multi-disciplinary roles, in industry, academia and regulatory bodies - internationallyregulatory bodies - internationally

Our Proposal forOur Proposal for the Critical Path the Critical Path

Conduct Research , Gain Consensus, and Conduct Research , Gain Consensus, and Develop Guidance to Remove Obstacles to Develop Guidance to Remove Obstacles to Efficient Drug Development and Enhance Efficient Drug Development and Enhance Success Rates of Clinical TrialsSuccess Rates of Clinical Trials

Improve the Processes and Approaches to Improve the Processes and Approaches to Quantitative Analysis of Clinical Safety Data Quantitative Analysis of Clinical Safety Data from Clinical Trials to Enhance Risk from Clinical Trials to Enhance Risk Assessment and Management InitiativesAssessment and Management Initiatives

Improve the Statistical Understanding and Improve the Statistical Understanding and Application of Modern Statistical Approaches Application of Modern Statistical Approaches to Product Testing and Process Control to Product Testing and Process Control

Clinical Trial Proposals for the Critical Clinical Trial Proposals for the Critical PathPath

Missing dataMissing data due to patient withdrawals and dropouts in clinical trials due to patient withdrawals and dropouts in clinical trials

Flexible / adaptive clinical trial designsFlexible / adaptive clinical trial designs to improve the information to improve the information and success rate of trialsand success rate of trials

Non-inferiority active control studiesNon-inferiority active control studies when placebos can't be used when placebos can't be used - getting to consensus on appropriate methods for margin setting, data - getting to consensus on appropriate methods for margin setting, data analysis and interpretation for various data rich and data poor scenariosanalysis and interpretation for various data rich and data poor scenarios

Development of consensus on the Development of consensus on the statistical handling of multiple statistical handling of multiple endpoints in clinical trialsendpoints in clinical trials..

Clinical trial modeling and simulationClinical trial modeling and simulation as a tool for better design as a tool for better design and interpretation of clinical trialsand interpretation of clinical trials

Application of Bayesian MethodsApplication of Bayesian Methods to Enhance the Success Rate of to Enhance the Success Rate of Clinical TrialsClinical Trials

Prioritize Efforts - Three Prioritize Efforts - Three separate yet related separate yet related

approachesapproaches

Guidance DevelopmentGuidance Development

Multiple endpointsMultiple endpoints

Non-inferiorityNon-inferiority

Topics of high interestTopics of high interest

Adaptive / Flexible designsAdaptive / Flexible designs

Modeling / simulation / planning/Phase Modeling / simulation / planning/Phase 2a2a

Other Critical Path needs: safety , product Other Critical Path needs: safety , product qualityquality

Safety and Quantitative Safety and Quantitative Risk AssessmentRisk Assessment

Clinical Trials - Pre-Clinical Trials - Pre-MarketingMarketing

Methods of applicationMethods of application

Planning, data collection, statistical Planning, data collection, statistical analysis plananalysis plan

ProcessProcess

Newly formed statistical safety team Newly formed statistical safety team for more concentrated and focused for more concentrated and focused adviceadvice

Earlier planning, modeling and Earlier planning, modeling and simulationsimulation

FDA Risk Management FDA Risk Management GuidancesGuidances

Life cycle of a drugLife cycle of a drug

Premarketing Risk Assessment Premarketing Risk Assessment (Premarketing Guidance)(Premarketing Guidance)

Development and Use of Risk Development and Use of Risk Minimization Action Plans (RiskMAP Minimization Action Plans (RiskMAP Guidance)Guidance)

Good Pharmacovigilance Practices Good Pharmacovigilance Practices and Pharmacoepidemiology and Pharmacoepidemiology Assessment (Pharmacovigilance Assessment (Pharmacovigilance Guidance)Guidance)

Enhancing Product Enhancing Product QualityQuality

Modern in process testing raises the Modern in process testing raises the possibility that alternatives to possibility that alternatives to product quality should be consideredproduct quality should be considered

There have also been advances in There have also been advances in Process Analytical Technology (PAT) Process Analytical Technology (PAT) which depends on in process which depends on in process assessment of product quality all assessment of product quality all along the drug manufacturing processalong the drug manufacturing process

The Non -Inferiority ProblemThe Non -Inferiority ProblemCurrent guidance is inadequate and Current guidance is inadequate and the issues are poorly understood - the issues are poorly understood -

must be fixedmust be fixed

Term introduced in ICH E9 Term introduced in ICH E9 ‘Statistical Principles for Clinical ‘Statistical Principles for Clinical Trials’Trials’

Some issues described in ICH E10 Some issues described in ICH E10 ‘Choice of Control Groups’‘Choice of Control Groups’

A study design that provides an A study design that provides an indirect measure of evidence of indirect measure of evidence of efficacy / safetyefficacy / safety

What are the various What are the various objectives of the non-objectives of the non-

inferiority designinferiority design To prove efficacy of test treatment by indirect To prove efficacy of test treatment by indirect

inference from the active control treatmentinference from the active control treatment

To establish a similarity of effect to a known very To establish a similarity of effect to a known very effective therapy - e.g. anti-infectiveseffective therapy - e.g. anti-infectives

To infer that the test treatment would have been To infer that the test treatment would have been superior to an ‘imputed placebo’ ; ie. had a superior to an ‘imputed placebo’ ; ie. had a placebo group been included for comparison in placebo group been included for comparison in the current trial. - a new and controversial area - the current trial. - a new and controversial area - choice of margin is the keychoice of margin is the key

To preserve a specified % effect of the ACTo preserve a specified % effect of the AC

How is the margin “ How is the margin “ “ chosen “ chosen based upon prior study databased upon prior study data

For a large treatment effect, it is For a large treatment effect, it is easier - a clinical decision of how easier - a clinical decision of how similar a response rate is needed to similar a response rate is needed to justify efficacy of a test treatment - justify efficacy of a test treatment - e.g. anti-infectives is an example.e.g. anti-infectives is an example.

For modest and variable effects, it is For modest and variable effects, it is more difficult ; and some approaches more difficult ; and some approaches suggest margin selection based suggest margin selection based upon several objectives.upon several objectives.

Complexities in choosing the Complexities in choosing the margin (how much of the control margin (how much of the control

treatment effect to give up)treatment effect to give up) Margins can be chosen depending upon which of these Margins can be chosen depending upon which of these

questions is addressed:questions is addressed:

how much of the how much of the treatment effecttreatment effect of the of the comparator can be comparator can be preserved preserved in order to indirectly in order to indirectly conclude the test treatment is effective - a clinical conclude the test treatment is effective - a clinical decision for very large effects; a statistical problem decision for very large effects; a statistical problem for small and modest effectsfor small and modest effects

how much of a how much of a treatment effecttreatment effect would one require would one require for the test treatment for the test treatment to be superior to placeboto be superior to placebo, , had a placebo been used in the current active had a placebo been used in the current active control study - a lesser standard than the abovecontrol study - a lesser standard than the above

How convincing is the How convincing is the prior evidence of a prior evidence of a treatment effect ?treatment effect ?

Do clinical trials of the comparator treatment Do clinical trials of the comparator treatment consistently and reliably demonstrate a treatment consistently and reliably demonstrate a treatment effect - when they do not, what is the reason ?effect - when they do not, what is the reason ?

Study is too small to detect the effect - under Study is too small to detect the effect - under powered for a modest effect sizepowered for a modest effect size

The treatment effect is variable, and the The treatment effect is variable, and the estimate of the magnitude will vary from study estimate of the magnitude will vary from study to study, sometimes with NO effect in a given to study, sometimes with NO effect in a given study - a BIG problem for active controlled study - a BIG problem for active controlled studies (Sensitivity to drug effect)studies (Sensitivity to drug effect)

Importance of the assumption of Importance of the assumption of constancy of the active control constancy of the active control treatment effect derived from treatment effect derived from

historical studieshistorical studies It is relevant to the design and sample size of It is relevant to the design and sample size of

the current study, to the choice of the margin, the current study, to the choice of the margin, to the amount of bias built into the to the amount of bias built into the comparisons, to the amount of effect size one comparisons, to the amount of effect size one can preserve (both of these are likely can preserve (both of these are likely confounded), and to the statistical uncertainty confounded), and to the statistical uncertainty of the conclusion.of the conclusion.

Before one can decide on how much of the Before one can decide on how much of the effect to preserve, one should estimate an effect to preserve, one should estimate an effect size for which there is evidence of a effect size for which there is evidence of a consistent demonstration that effect size exists.consistent demonstration that effect size exists.

Four approaches to the Four approaches to the problemproblem

The simple case: specify a delta - not estimatedThe simple case: specify a delta - not estimated

Indirect confidence interval comparisons (ICIC)Indirect confidence interval comparisons (ICIC) (CBER/FDA type method, etc.)(CBER/FDA type method, etc.)

- thrombolytic agents in the treatment of acute MI- thrombolytic agents in the treatment of acute MI

Virtual method (Hasselblad & Kong, Fisher, etc.) Virtual method (Hasselblad & Kong, Fisher, etc.)

- Clopidogrel, aspirin, placebo- Clopidogrel, aspirin, placebo

Bayesian approach (Gould, Simon, etc.)Bayesian approach (Gould, Simon, etc.)

- treatment of unstable angina and non-Q wave MI- treatment of unstable angina and non-Q wave MI

Current Guidance on Multiple Current Guidance on Multiple Endpoints is inadequateEndpoints is inadequate

Multiple primary endpointsMultiple primary endpointsMultiple secondary endpointsMultiple secondary endpointsComposite endpointsComposite endpointsMultiple compositesMultiple compositesHierarchiesHierarchiesPatient reported outcomesPatient reported outcomesDecision Criteria for successDecision Criteria for success

A collaborative effort: PhRMA 2004 meeting A collaborative effort: PhRMA 2004 meeting on co-primary endpoints, manuscripton co-primary endpoints, manuscript

Emerging Interest in Emerging Interest in Adaptive / Flexible Trial Adaptive / Flexible Trial

DesignsDesigns

Adaptive designsAdaptive designs

Enrichment / Enrichment / pharmacogenomicspharmacogenomics

Sample size re-estimationSample size re-estimation

Design modificationDesign modification

New study designsNew study designsWhy a need for adaptive / Why a need for adaptive /

flexible designs ? flexible designs ?

Enriching trials with patients having Enriching trials with patients having genomic profiles likely to respond or genomic profiles likely to respond or less likely to experience toxicityless likely to experience toxicity

Goal of an adaptive / flexible designGoal of an adaptive / flexible design

Mid study changes that Mid study changes that prospectively plan for prospectively plan for modifications that preserve Type modifications that preserve Type 1 errors and maximize chances 1 errors and maximize chances for successfor success

Information adaptive Information adaptive designs / flexible designsdesigns / flexible designs

ControversialControversial

Statististical Methodology is Statististical Methodology is AvailableAvailable

Why and where to use them?Why and where to use them?

Why the need for adaptation?

Design specifications often entail at least partial

knowledge of the values of many planning (primary

or nuisance) parameters that are unknown or at best

might be guessed crudelySample size planning entails “educated” guess of effect size.Selection of a composite endpoint requires “educated” guess of where the potential effects lie and what noises may be.Others….. Hung

Addressing a process issue:Addressing a process issue: Scenario Planning:Scenario Planning:A Tool to Increase the Success Rate of Phase A Tool to Increase the Success Rate of Phase III trials and to Enhance Drug Development III trials and to Enhance Drug Development PlanningPlanning

Incorporates: Incorporates: Several linked linked study phases - Several linked linked study phases - continuumcontinuumMultiple endpointsMultiple endpointsMissing dataMissing dataUse of all information in the process Use of all information in the process Safety PlanningSafety PlanningModeling and simulationModeling and simulationFlexible designs / development sequence / Flexible designs / development sequence / internationalinternational

What is Scenario What is Scenario PlanningPlanning

Modern approach to protocol planning and choice Modern approach to protocol planning and choice of clinical study designsof clinical study designs

Utilizing models for disease progression and Utilizing models for disease progression and endpoint selectionendpoint selection

Utilizing simulation strategies for what if Utilizing simulation strategies for what if scenariosscenarios

Assumes input from other studies and planning Assumes input from other studies and planning efforts - planned sequences of studies may matterefforts - planned sequences of studies may matter

An aid for prospectively planning integrated An aid for prospectively planning integrated analysesanalyses

Disease Progression Disease Progression ModelingModeling

Endpoint selection and evaluationEndpoint selection and evaluation

Trial Duration determinationTrial Duration determination

Frequency and number of subject measurementsFrequency and number of subject measurements

Tradeoffs between clinical endpoints and patient Tradeoffs between clinical endpoints and patient reported outcomesreported outcomes

Evaluate impact of missing data, informative Evaluate impact of missing data, informative treatment related censoringtreatment related censoring

Evaluate multiplicity implications Evaluate multiplicity implications

1 2 3 4 5Higher is bad

What would be observed if subjects had stayed in trial ?Impute values from subects staying in longer

TestControl

Visit

Bas

elin

e

Which path do you choose ?

Disease Progression Disease Progression Models and Clinical Models and Clinical

OutcomesOutcomes

What model captures the functional What model captures the functional relationship of the disease relationship of the disease progression and the clinical progression and the clinical outcome(s) to be used to measure outcome(s) to be used to measure treatment effecttreatment effect

Can one function capture each of the Can one function capture each of the clinical outcomes adequatelyclinical outcomes adequately

If not are several disease progression If not are several disease progression models used to express ‘response’models used to express ‘response’

Modern Protocol Modern Protocol /Development Planning/Development PlanningSensitivity / Scenario Sensitivity / Scenario

planningplanning

Different statistical tools and strategiesDifferent statistical tools and strategies

Challenge and explore assumptionsChallenge and explore assumptions

More multidisciplinary involvementMore multidisciplinary involvement

It is more than sample size planningIt is more than sample size planning

Structured planning meetings that are Structured planning meetings that are different that current – formal Q & A’s not different that current – formal Q & A’s not broad enoughbroad enough

Links between phase planning and modeling Links between phase planning and modeling efforts – currently too limited and stove efforts – currently too limited and stove pipedpiped

Concluding remarksConcluding remarks

Meeting the Challenges of the Critical Meeting the Challenges of the Critical Path will require collaboration and Path will require collaboration and resource allocationresource allocation

Multidisciplinary / collaborative planning and Multidisciplinary / collaborative planning and evaluation is needed now more than ever evaluation is needed now more than ever because issues becoming more complex - because issues becoming more complex - guidances can’t solve this - resources, exposure, guidances can’t solve this - resources, exposure, experience, training willexperience, training will

Efforts to move available appropriate statistical Efforts to move available appropriate statistical methods and concepts , possibly more complex, methods and concepts , possibly more complex, into the main stream by emphasis on into the main stream by emphasis on understanding by the audience appropriate to understanding by the audience appropriate to the applicationthe application

Guidances don’t help here - need resources that Guidances don’t help here - need resources that can understand and communicatecan understand and communicate

Efforts to maximize contributions of industry, Efforts to maximize contributions of industry, academic and regulatory statisticiansacademic and regulatory statisticians

Concluding remarkConcluding remark -Priority setting -Priority setting --

Choosing the most pressing needs Choosing the most pressing needs and the chances for success - and the chances for success - currently being updatedcurrently being updated

This is a national effort - not just This is a national effort - not just FDA’s initiative - it will take a FDA’s initiative - it will take a major coordinated effort to make major coordinated effort to make progressprogress