09/12/2008AD course for Philadelphia ASA Chapter Introduction to Adaptive Designs: Definitions and Classification Inna Perevozskaya, Merck & Co.

09/12/2008 AD course for Philadelphia ASA Chapter

Introduction to Adaptive Designs: Definitions and

Classification

Inna Perevozskaya, Merck & Co.

Acknowledgement: PhRMA Adaptive Designs Working Group

Co-Chairs: Michael KramsBrenda Gaydos

Authors: Keaven Anderson

Suman BhattacharyaAlun BeddingDon BerryFrank BretzChristy Chuang-SteinVlad DragalinPaul GalloBrenda GaydosMichael KramsQing LiuJeff MacaInna PerevozskayaJose PinheiroJudith Quinlan

Members:Carl-Fredrik BurmanDavid DeBrota Jonathan DenneGreg EnasRichard EntsuahAndy GrieveDavid HenryTony HoTelba IronyLarry LeskoGary LittmanCyrus MehtaAllan PallayMichael PooleRick SaxJerry SchindlerMichael D SmithMarc WaltonSue-Jane WangGernot WassmerPauline Williams

Recent DIJ Publications by PhRMA Working Group on Adaptive Designs(Drug Information Journal, Vol. 40, 2006)

1. P. Gallo, M. Krams

Introduction

2.2. V. Dragalin V. Dragalin

Adaptive Designs: Terminology and ClassificationAdaptive Designs: Terminology and Classification

3. J. Quinlan, M. Krams

Implementing Adaptive Designs: Logistical and Operational Considerations

4. P. Gallo.

Confidentiality and trial integrity issues for adaptive designs

5. B.Gaydos, M. Krams, I. Perevozskaya, F.Bretz; Q. Liu, P. Gallo, D. Berry; C. Chuang-

Stein, J. Pinheiro, A. Bedding.

Adaptive Dose Response Studies

6. J. Maca, S. Bhattacharya, V. Dragalin, P. Gallo, M. Krams,

Adaptive Seamless Phase II / III Designs – Background, Operational Aspects, and

Examples

7. C. Chuang-Stein, K.Anderson, P. Gallo, S. Collins.

Sample Size Re-estimation: A Review and Recommendations

Outline Adaptive design: evolution if the term

Adaptive vs. static designs

Some adaptive designs were known under different names

Formal classification effort: Structure and key elements

Classification by Objective and Phase or Stage

Adaptive designs “ahead of others” (where effort should be

focused) dose response

seamless II/III

Sample size re-estimation

Adaptive vs. Traditional Designs In traditional drug development, most designs

used (especially Phase II and III) are “static”: Key elements driving the designs are specified in

advance: Hypotheses to be tested

Population of interest

Maximum information to be collected (translated into

power, SS, and detectable treatment effect)

Randomization scheme

Early stopping rules

Adaptive vs. Traditional Designs (cont.) “Static” designs framework:

Results observed during trial are not used to guide it’s

course

This setup provides solid inferential procedures

But leaves some space for improvement in terms of

efficiency

Different ways to improve efficiency have been proposed

over time, allowing dynamic modification of trial’s design

during its course based on accumulating data

That lead to formation of a broad group of methods known

today as “adaptive designs”

Adaptive vs. Traditional Designs (cont.)Definition: (from An Executive Summary of PhRMA Working Group):

Adaptive design refers to a clinical study design that uses accumulating data to decide on how to modify aspects of the study as it continues, without undermining the validity and integrity of the trial

Essential components: changes are made by designs and not on an ad-hoc

basis

adaptation is a design feature and not a remedy for poor planning

Adaptive Designs: Evolution of the Term Many of designs we call “adaptive” today existed for quite

some time as a “class of their own”

(e.g. group-sequential designs, response-adaptive randomization, flexible designs, sample size re-estimation )

These designs

Aim at improving some feature of a rigid traditional design (such as cost efficiency or addressing an ethical dilemma)

Share a common feature of mid-course adaptation(s)

As the number of such designs grew, so did the confusion…

Strong need for a unified structured approach to terminology has emerged

Key Reference: V. Dragalin “Adaptive designs: Terminology and

Classification“. Drug Information Journal (2006), Vol 40, pp 425-435

First attempt to develop a unified approach to AD Reflects discussions within PhRMA working group on adaptive

designs Major source of AD review to follow Provides:

general definition of adaptive designs structure (key components) Classification (by objective) mapping against drug-development process

Review of “AD: Terminology and Classification” Adaptive Design Definition

Adaptive design refers to a multistage clinical study design that uses accumulating data to decide on how to modify aspects of the study without undermining the validity and integrity of the trial

Validity: Correct statistical inference Ensuring consistency across different parts Minimizing operational basis

Integrity: Providing results convincing to the scientific community Adequate pre-planning and blinding procedures

Key Elements of an Adaptive Design

1. Allocation Rule2. Sampling Rule3. Stopping Rule4. Decision Rule

Examples: Group sequential designs (stopping ) Response-adaptive allocation (allocation) Sample size reassessment (sampling) Flexible designs (all)

One or more may be applied during interim

looks

Key Elements of an Adaptive Design (cont.)1. Allocation Rules:

Determine how patients are assigned to available treatments at each stage

Can be fixed (static) or adaptive (dynamic) Fixed allocation examples:

Complete randomization Stratified randomization Restricted randomization

Adaptive allocation examples Covariate-adaptive randomization Response-adaptive randomization Bayesian response-adaptive randomization (Berry, 2001) Drop-the-loser type (Sampson, 2005)

Rosenberger and Lachin, (2001)

Key Elements of an Adaptive Design (cont.)

2. Sampling Rules How many subject will be sampled at the next stage? Examples of designs with SR:

Blinded SS re-estimation Adjustment of SS based on estimate of a nuisance parameter

Unblinded SS re-estimation Adjustment of SS based on information about trt effect

Traditional group sequential fixed sampling rule

Flexible SSR based on conditional power Probability of rejecting null at the end of study given first-

stage data Calculated for the originally specified treatment effect

Key Elements of an Adaptive Design (cont.)

3. Stopping rules Intended to protect patients from unsafe drug or to

expedite the approval of a beneficial treatment. Based on satisfying power requirements in hypothesis

testing framework “Crossing a boundary” methodology

Superiority Harm Futility

Examples: classical group-sequential (Jenisson&Turnbull, 2000)

Key Elements of an Adaptive Design (cont.)

4. Decision rules: Changing test statistics Redesigning multiple endpoints Selecting hypotheses to be tested or their hierarchy Changing patient population Choosing the number of interim analyses based on

current information For dose-response studies-selecting next dose

assignment

Classification of Adaptive DesignsRef: V. Dragalin “Adaptive designs: Terminology

and Classification“. DIJ (2006) Key elements of AD define structure and

describe algorithms of ADAllocation RuleSampling RuleStopping RuleDecision Rule

Another way to classify AD is by what their objectives are applicability to a particular stage of clinical

development

Classification of Adaptive Designs (cont.)

1. Single-arm trials 2. Comparing two treatments3. Comparing more than two treatments

Model-based dose-response assessment

4. Seamless Phase II/III

1. Adaptive Designs for Single-Arm Trials Applicability: Phase-I/POC/Phase II

a) Screening trials for 1 trt-used to screen candidate components based on short-term response

Employ small sample sizes Hypothesis testing: minimum acceptable probability

of response pre-specified Allow early stopping due to futility Ex1: Two-stage designs (Gehan, 1961 ) Ex2: Bayesian designs (Thall & Simon, 1994)

1. Adaptive Designs for Single-Arm Trials (cont.)

b) Designs for entire screening program Minimize time to identify promising compound Control Type I and Type II risk for the entire

program Ref: Wang&Leung, 1998;

Yao&Venkatraman, 1998; Hardwick & Stout, 2002;

2. Adaptive Designs for Comparing Two Treatments Applicability: predominantly Phase III, but some can be

used in Phase I-II Fully sequential design

Check boundary crossing after each patient Group-sequential Design

Check boundary crossing after a group of patients Adaptive group-sequential designs

Extend the GSD methodology: allow in SS Methodology based on P-value combination tests

Flexible designs Wide spectrum of decision rules can be applied after 1st

stage Recursive application of 2-stage combination tests Allow many mid-trial adaptations; not all prespecified (in

theory….)

3. Adaptive Designs for Comparing More Than Two Treatments Applicability: dose-response assessment studies

(mostly phase II, full range I-III) “Late stage dose-response development”

group-sequential designs (Stallard & Todd, 2003) Flexible designs (Bauer & Kieser, 1999)

“Early exploratory development” Dose-escalation studies (Phase I; Ex. CRM) Model-based dose-response assessment

D-optimal designs Bivariate response Penalized (constrained) designs Bayesian dose-finding designs

Reviewed in depth in (Gaydos et al., 2006)

4. Seamless Phase II/III designs Combine traditional Phase IIb and Phase III “learning and confirming” governed by

one protocol Can be

operationally seamless inferentially seamless

Explored in depth in Maca et al., 2006

Dose-Finding AD Example: Continual Reassessment Method (Ex.1) Bayesian dose-escalation design Designed to converge to MTD For a predefined set of doses to be studied and a binary

response, estimates dose level (MTD) that yields a particular proportion of responses

Updates MTD distribution after each patient’s response Next dose is selected as the one with predicted probability

closest to the target level of response Procedure stops after N patients enrolled

Continual Reassessment Method (cont.)

Choose initial estimate

of response distribution& choose

initial dose

Obtain nextPatient’s

Observation

Update DoseResponse Model

& estimateProb. (Resp.)@ each dose

Max NReached?

Next Pt. Dose= Dose w/

Prob. (Resp.)Closest to

Target levelno

Stop.MTD = Dose w/Prob. (Resp.)

Closest toTarget level

yes

CRM Design example (1)

Post-anesthetic care patients received a single IV dose of 0.25, 0.50, 0.75, or 1.00 μg/kg nalmefene.

Response was Reversal of Analgesia (ROA) = increase in pain score of two or more integers above baseline on 0-10 NRS after nalmefene

Patients entered sequentially, starting with the lowest dose

The maximum tolerated dose = dose, among the four studied, with a final mean posterior probability of ROA closest to 0.20 (i.e., a 20% chance of causing reversal)

Modified continual reassessment method (iterative Bayesian proc) selected the dose for each successive pt. as that having a mean posterior probability of ROA closest to the preselected target 0.20.

1-parameter logistic function for probability of ROA used to fit the data at each stage

Dougherty,et al. ANESTHESIOLOGY (2000)

CRM example (1) results

* including the 1st patient treated

(MTD), i.e., estimated mean posterior probability closest to 0.20 target^ extrapolated

Dose (ug/kg) # pts. # w/ ROA % w/ ROA

mean post.

prob. ROA

median post.

prob. ROA

0.25 4* 0 0% 0.09 0.11

0.50 (MTD)

18 3 17% 0.18 0.21

0.75 3 2 67% 0.37 0.41

1.00 0 - - 0.79^ 0.80^

CRM example (1) results

1.0

0.8

0.6

0.4

0.2

0.0Po

ste

rior

RO

A P

roba

bili

ty(w

ith 9

5%

pro

ba

bilit

y in

terv

als

)

Continual Reassessment Method (cont.)

Allocation rule: model-based Sampling rule: cohort size Stopping rule: max N or no rule Decision rule: posterior update,

select next dose

Example 2: Comparing 2 treatments Adaptive GS (Flexible) designRedesigned trial example from (Cui et al., 1999)

Actual design: group sequential design Proposed design: sample size re-estimation + combination

test statistic Phase III trial for prevention of MI in patients undergoing

coronary artery bypass graft surgery N=600 per treatment group to detect 50% reduction of

incidence (predicted 22% for placebo vs. 11% for drug) with 95% power

Interim analysis at 50% data: N=300 per treatment group Observed incidence for pbo was ~16.5%, drug~11% Given observed data, power is 40% to detect 25% reduction

Example 2 (cont.) Sponsor wanted to increase 2nd stage sample size to detect

smaller effect Type I error rate would be inflated with usual group sequential

test Trial continued with planned sample size and ended with non-

significant statistical result Instead, authors proposed to SS and use combination test Simulations were performed:

Increase total sample size to 1400 per treatment group Maintain Type I error rate; 93% power to detect 25% reduction

Example 2 (cont.) Allocation rule: fixed randomization Sampling rule: sample size of next stage

depends on results from previous stage Stopping Rule: p-value combination test Decision Rule: adapting alternative

hypothesis and test statistics

Summary: adaptive designs where attention needs to be focused1. Dose-ranging studies:

• B.Gaydos, M. Krams, I. Perevozskaya, F.Bretz; Q. Liu, P. Gallo, D. Berry; C. Chuang-Stein, J. Pinheiro, A. Bedding. Adaptive Dose Response Studies

2. Seamless Phase II/III• J. Maca, S. Bhattacharya, V. Dragalin, P. Gallo, M. Krams,

Adaptive Seamless Phase II / III Designs – Background, Operational Aspects, and Examples

3. Sample Size Re-estimation• C. Chuang-Stein, K.Anderson, P. Gallo, S. Collins.

Sample Size Re-estimation: A Review and Recommendations

Conclusions Adaptive designs provide an opportunity to redesign trials

based on accumulating data In some situations, may be more efficient than

implementing traditional designs There is no “ one-size-fits-all” recommendation for the choice

of AD In fact, it may not be the best solution at all That decision will depend on:

Trials objectives Regulatory guidelines Logistic and practical consideration

Those are collectively determined by clinicians, regulatory, statisticians and data management => complicated process

As a result, implementation may be the biggest challenge However, there are successful examples out there and that

should be encouraging!!!

Additional References1. Rosenberger WF, Lachin JM. Randomization in Clinical Trials: Theory and

Practice. New York: Wiley; 2002.2. Berry D. Adaptive trials and Bayesian statistics in drug development.

Biopharm Rep. 2001;9:1–11.3. Sampson AR, Sill MW. Drop-the-Losers design: normal case. Biometrical J.

2005;47:257–268.4. Cui L, Hung HMJ, Wang SJ. Modification of sample size in group sequential

clinical trials. Biometrics. 1999;55:853–8575. Jennison C, Turnbull BW. Group Sequential Methods With Applications to

Clinical Trials. Boca Raton, FL: Chapman and Hall; 2000.6. Gehan EA. The determination of number of patients in a follow-up trial of

a new chemotherapeutic agent. J Chronic Dis. 1961;13:346–353.7. Wang YG, Leung DHY. An optimal design for screen trials. Biometrics.

1998;54:243–250.8. Yao TJ, Venkatraman E. Optimal two-stage design for a series of pilot

trials of new agents. Biometrics.