Prior Elicitation in Bayesian Clinical Trial Design
Peter F. ThallBiostatistics Department
M.D. Anderson Cancer Center
SAMSI intensive summer research program on
Semiparametric Bayesian Inference: Applications in Pharmacokinetics and
PharmacodynamicsResearch Triangle Park, North Carolina
July 13, 2010
Disclaimer
To my knowledge, this talk has nothing to do with semiparametric Bayesian inference, pharmacokinetics, or pharmacodynamics.
I am presenting this at Peter Mueller’s behest.
Blame Him!
Outline ( As time permits )
1. Clinical trials: Everything you need to know
2. Eliciting Dirichlet parameters for a leukemia trial
3. Prior effective sample size
4. Eliciting logistic regression model parameters for Pr(Toxicity | dose)
5. Eliciting values for a 6-parameter model of
Pr(Toxicity | dose1, dose2)
6. Penalized least squares for {Pr(Efficacy),Pr(Toxicity)}
7. Eliciting a hyperprior for a sarcoma trial
8. Eliciting two priors for a brain tumor trial
9. Partially informative priors for patient-specific dose finding
Clinical Trials
Definition: A clinical trial is a scientific experiment with human subjects.
1. Its first purpose is to treat the patients in the trial. 2. Its second purpose is to collect information that may be
useful to evaluate existing treatments or develop new, better treatments to benefit future patients.
Other, related purposes of clinical trials:3. Generate data for research papers 4. Obtain $$ financial support $$ from pharmaceutical
companies or governmental agencies5. Provide an empirical basis for drug or device approval
from regulatory agencies such as the US FDA
Medical Treatments
Most medical treatments, especially drugs or drug combinations, have multiple effects.
Desirable effects are called efficacy► Shrinkage of a solid tumor by > 50%► Complete remission of leukemia► Dissolving a cerebral blood clot that caused an ischemic stroke► Engraftment of an allogeneic (matched donor) stem cell transplant
Undesirable effects are called toxicity ► Permanent damage to internal organs (liver, kidneys, heart, brain)► Immunosuppression (low white blood cell count or platelet count)► Cerebral bleeding or edema (accumulation of fluid)► Graft-versus-host disease (the engrafted donor cells attack the
patient’s organs)► Regimen-related death due to any of the above
Scientific Method
Advice from Ronald Fisher Don’t waste information
Advice From Peter Thall Don’t waste prior information when
designing a clinical trial
Standard Statistical Practice Ignore Fisher’s advice and just run your favorite
statistical software package. And be sure to record lots and lots of p-values.
A Chemotherapy Trial in Acute Leukemia
Complete Remission (CR)
Yes No
Yes 1 2
No 3 4
TOX IC ITY
Model12 , 3, 4 ) ~ Dirichlet(a1, a2, a3, a4) ≡ Dir(a)
p(a) ∝ 1a1-1 2
a2-1 3a3-1 4
a4-1, a+ = a1+a2+a3+a4 = ESSTOX = 12 ~Be(a1+a2, a3+a4)
CR = 13 ~ Be(a1+a3, a2+a4) E(TOX) = (a1+a2 )/a+ E(CR) = (a1+a3 )/a+
4 = 1 – 1 – 2 – 3
If possible, use Historical Data to establish a prior: CR and Toxicity counts from 264 AML Patients Treated
With an Anthracycline + ara-C
CR No CR
Toxicity 73
(27.7%)
63 (23.9%) 136
(51.5%)
No
Toxicity
101
(38.3%)
27
(10.2%)
128
(48.5%)
174
(65.9%)
90
(34.1%)
264
P(CR | Tox) = 73/136 = .54P(CR | No Tox) = 101/128 = .79
CR and Tox are Not Independent
S = “Standard” treatmentE = “Experimental” treatment
S ~ Dir (73,63,101,27) aS,+ = ESS = 264 (“Informative”)
Set E = S with aE,+ = 4
E ~ Dirichlet (1.11, .955, 1.53, .409) (“Non-Informative”)
Dirichlet Priors and Stopping Rules
Stop the trial if
1) Pr(S,CR + .15 < E,CR | data) < .025 (“futility”), or
2) Pr(S,TOX + .05 < E,TOX | data) > .95 (“safety”)
But what if you don’t have historical data?!!
An Easy Solution:To obtain the prior onS
1) Elicit the prior marginal outcome probability means E(TOX) = (a1+a2 )/a+ and E(CR) = (a1+a3 )/a+
2) Assume independence and solve algebraically for (1, 2, 3, 4) = (a1, a2, a3, a4)/ a+
3) Elicit the effective sample size ESS = a+ that the elicited values E(TOX) and E(CR) were based on
4) Solve for (a1, a2, a3, a4)
Sensitivity Analysis of Association in the desirable case wherePr(CR) ↑ 0.15 from .659 to .809 and Pr(TOX) = .516
i.e. there is no increase in toxicity.
p11p00
p10p01
True E Probability of Stopping the Trial Early
Sample Size
(25%,50%,75%)
.007 (.027,.489,.782,.102) >.99 4 7 14
.138 (.227,.289, .582,.102) .56 14 44 56
1.28 (.427,.089,.382,.102) .16 56 56 56
52.6 (.510,.006,.299,.185) .16 56 56 56
Oops!!
If you don’t have historical data . . .
A slightly smarter wayto obtain prior(S
1) Elicit the prior means E(TOX) = (a1+a2 )/a+ and E(CR) = (a1+a3 )/a+
2) Elicit the prior mean of a conditional probability, like Pr(CR | Tox) = 1/(1 + 2), which has mean a1/(a1 + a2), and solve for (1, 2, 3, 4) = (a1, a2, a3, a4)/ a+ . That is, do not assume independence. 3) Elicit the effective sample size ESS = a+ that the values E(TOX) and E(CR) were based on
4) Solve for (a1, a2, a3, a4)
Rocket Science!!
Example
Elicited prior values
E(TOX) = (a1+a2 )/a+ = .30
E(CR) = (a1+a3 )/a+ = .50 E{ Pr(CR | Tox) } = E{ 1/(1 + 2)} = a1/(a1 + a2) = .40
ESS = a+ = 120
(a1, a2, a3, a4) = (14.4, 21.6, 45.6, 38.4)
(1, 2, 3, 4) = (a1, a2, a3, a4)/ a+ = (.12, .18, .38, .32)
A Fundamental question in Bayesian analysis: How much information is contained in the
prior?
Priorp(θ)(((
(((
(((
(((
Determining the Effective Sample Size of a
Parametric Prior (Morita, Thall and Mueller, 2008)
The answer is straightforward for many commonly used models
E.g. for beta distributions
Be (1.5,2.5)
Be (16,19)
Be (3,8)
ESS = 16+19 = 35
ESS = 3+8 = 11
ESS = 1.5+12.5 = 5
But for many commonly used parametric Bayesian models it is not obvious how to determine the ESS of the prior.
E.g. usual normal linear regression model
22
10
210
210
inverse~ normal, bivariate~,
,,
)( ,)(
YVarXYE
Intuitive Motivation
Saying Be(a, b) has ESS = a+b implicitly refers to the wel known fact that
θ ~ Be(a, b) and Y | θ ~ binom(n, θ)
θ | Y,n 〜 Be(a +Y, b +n-Y) which has ESS = a + b + n
So, saying Be(a,b) has ESS = a + b implictly refers to an
earlier Be(c,d) prior with very small c+d = and solving
for m = a+b – (c+d) = a+b – for a very small > 0
General Approach
1) Construct an “-information” prior q0(θ), with same
means and corrs. as p(θ) but inflated variances
2) For each possible ESS m = 1, 2, ... consider a
sample Ym of size m
3) Compute posterior qm(θ|Ym) starting with prior q0(θ)
4) Compute the distance between qm(θ|Ym) and p(θ)
5) The interpolated value of 5) The interpolated value of mm minimizing the distance minimizing the distance
is the ESS.is the ESS.
A Phase I Trial to Find a Safe Dose forAdvanced Renal Cell Cancer (RCC)
Patients with renal cell cancer, progressive after treatment with Interferon
Treatment = Fixed dose of 5-FU + one of 6 doses of Gemcitabine: {100, 200, 300, 400, 500, 600} mg/m2
Toxicity = Grade 3,4 diarrhea, mucositis, or hematologic (blood) toxicity
Nmax = 36 patients, treated in cohorts of 3
Start the with1st cohort treated at 200 mg/m2
Adaptively pick a “best” dose for each cohort
Continual Reassessment Method (CRM, O’Quigley et al.
1990) with a Bayesian Logistic Regression Model
1) Specify a model for (xj, = Pr(Toxicity| , dose xj) and prior on
2) Physician specifies pTOX* = a target Pr(Toxicity)3) Treat each successive cohort of 3 pats. at the “best”
dose for which E[(xj, | data] is closest to pTOX*4) The best dose at the end of the trial is selected
exp( +xj )(xj, = 1 + exp( +xj )
using xj = log(dj) - {j=1,…klog(dj)}/k , j=1,…,6.
Prior: ~ N(, 2), ~ N(,
2)
Elicit the mean toxicity probabilities at two doses. In the RCC trial, the elicited prior values were
E{(200,)} = .25 and E{(500,)} = .75
1) Solve algebraically for = -.13 and = 2.40
2) = = 2 ~ N(-.13, 4), ~N(2.40, 4) which gives prior ESS = 2.3
Alternatively, one may specify the prior ESS and solve for = =
CRM with Bayesian Logistic Regression Model
Plot of ESS as a function of
ESS{ p(,)|}
0.1 0.2 0.3 0.4 0.5 0.7 1 2 3 4 5 10
ESS 928 232 103 58.0 37.1 18.9 9.3 2.3 1.0 0.58 0.37 0.09
For cohorts of size 1 to 3, =1 is still too small since it
gives prior ESS = 9.3
These ESS values are OK, so = 2 to 5 is OK. ?
These values give a prior with far more
information than the data in a typical phase I trial.
Prior of = Prob(tox | d = 200, )
p()
=0.1
=0.5=2.0
=10.0
ESS=928ESS=928
ESS=37.1ESS=37.1ESS=2.3ESS=2.3
ESS=0.09ESS=0.09
Why not just set = = = a very large number, so ESS = a = a very large number, so ESS = a tiny number, and have a very “non-informative” prior ?tiny number, and have a very “non-informative” prior ?
Example: A “non-informative” prior is ~ N(-.13,100) and ~ N(2.40,100), i.e. =10.0 ESS = 0.09.
But this prior has some very undesirable properties :
Prior Probabilities of Extreme
Values
Dose of Gemcitabine (mg/m2)
100 200 300 400 500 600
Pr{(x,)<.01} .45 .37 .33 .31 .31 .31
Pr{(x,)>.99} .30 .30 .32 .35 .38 .40
This says you believe, a priori, that
1) Pr{(x,< .01} = Prob(toxicity is virtually impossible) = .31 to .45
2) Pr{(x,> .99} = Prob(toxicity is virtually certain) =
.30 to .40
Making Making === = too large (a so-called “non too large (a so-called “non
informative” prior) gives a pathological prior. informative” prior) gives a pathological prior. What What is “too large” numerically is not obvious without is “too large” numerically is not obvious without
computing the corresponding ESS.computing the corresponding ESS.
Dose-Finding With Two Agents(Thall, Millikan, Mueller, Lee, 2003)
Study two agents used together in a phase I clinical trial, with
dose-finding based on (x,) = probability of toxicity for a
patient given the dose pair x = (x1, x2)
Find one or more dose pairs (x1, x2) of the two agents used
together for future clinical use and/or study in a randomized
phase II trial
Elicit prior information on (x,) with each agent used alone
Single Agent Toxicity Probabilities :
1 x11= x1= Prob(Toxicity | x1, x2=0, 1)
2 x22= x2= Prob(Toxicity | x1=0, x2, 2)
0
20
0
40
0
60
0
80
0
1,0
00
1,2
00
1,4
00 0
400
600
900
0
10
20
30
40
50
60
70
80
P(tox)
Cyclophosphamide
Gemcitabine
Hypothetical Dose-Toxicity Surface
Probability Model
x2=0 1 x11= 1 x1 / ( 1 + 1 x1
) = exp(1)/{1+exp(1)}
x1=0 2 x22= 2 x2 / (1 +2 x2
) = exp(2) / {1+ exp(2)}
where j = log(j)+j log(xj) for j=1,2
12), where 111and222have elicited informative priors and the interaction parameters23,3have non-informative priors.
Single-Agent Prior Elicitation Questions
1. What is the highest dose having negligible (<5%) toxicity?
2. What dose has the targeted toxicity * ?
3. What dose above the target has unacceptably high (60%) toxicity?
4. At what dose above the target are you nearly certain (99% sure) that toxicity is above the target (30%) ?
Resulting Equations for the Hyperparameters
Denote g() = / (1+) so (x,)} = g(x). Denote the doses given as answers to the questions by
{ x(1), x(2), x(3) = x*, x(4) }, and zj = x(j) / x*.
Assuming ~ Ga(a1 , a2 ) and ~ Ga(b1 , b2 ), solve the following equations for (a1 , a2 , b1 , b2 ) :
1. Pr{ g(z1) < .05 } = 0.99
2. E(z*)) = a1 a2 E(1) = * / (1 - * )
3. E(z3) = a1 a2 E(z3
) = 0.60 / 0.40 = 1.5
4. Pr{ g(z4) > * } = 0.99
The answers to the 4 questions for each single agent
Randy Millikan, MD
An Interpretation of this Prior
The ESS of p(θ) = p(θ1, θ2, θ3) is 1.5
Since informative priors on θ1 and θ2 and a vague prior on θ3 were elicited, it is useful to determine the prior ESS of each subvector :
ESS of marginal prior p(θ1) is 547.3 for (x1,0 | 11)}
ESS of marginal prior p(θ2) is 756.3 for (0,x2 | 22)}
ESS of marginal prior p(θ3) is 0.01 for the interaction
parameters θ3 = (33)
This illustrates 4 key features of prior ESS
1. ESS is a readily interpretable index of a prior’s informativeness.
2. It may be very useful to compute ESS’s for both the entire parameter vector and for particular subvectors
3. ESS values may be used as feedback in the elicitation process
4. Even when standard distributions are used for priors, it may NOT be obvious how to define a prior’s ESS.
For indices a=0,1 and b=0,1, and x = standardized dose,
a,b (x, ) = Pr(YE = a , YT = b | x, )
= Ea(1-E)1-a T
b(1-T)1-b + (-1)a+b E(1-E)T(1-T) (e-1)/(e+1)
with marginals
logit T(x,) = TxT
logit E(x,) = E xE,1 x2E,2 The model parameter vector is (TTEE,1 E,2 ,
Probability Model for Dose-Finding Based on Bivariate Binary Efficacy (Response) and Toxicity Indicators YE and YT
(Thall and Cook, 2004)
Establishing Priors
1) Elicit mean & sd of T(x,) & E(x,) for several values of x. 2) Use least squares to solve for initial values of the
hyperparameters in prior( | ) 3)Each component of is assumed normally distributed, r ~ N(r, r), so = (1,1,…, p,p)4) mE,j = prior mean and sE,j = prior sd of E(xj, mT,j = prior mean and sT,j = prior sd of T(xj,5) # elicited values > dim() find the vector that minimizes
the objective function
Penalty term to keep the ’s on the same numerical domain, c = .15
A trial of allogeneic stem cell transplant patients: Up to 12 cohorts of 3 each (Nmax = 36) were treated to determine
a best dose among {.25, .50, .75, 1.00 } mg/m2 of Pentostatin® as prophyaxis for graft-versus-host disease.
E = drop from baseline of at least 1 grade in GVHD at week 2 T = unresolved infection or death within 2 weeks.
Example: Elicited Prior for the illustrative application in Thall and Cook (2004)
ESS() = 8.9 (equivalent to 3 cohorts of patients!!)
ESS(E) = 13.7, ESS(T) = 5.3, ESS() = 9.0
A Slightly Smarter Way to Think About Priors
Fix the means
and use ESS contour plots to choose
Example:
A Strategy for Determining Priors in the Regression Model
To obtain desired overall ESS = 2.0 and ESSE = ESST = ESS = 2.0, one may inspect the ESS plots to choose the variances of the hyperprior. One combination that gives this is
Eliciting the Hyperprior for a Hierarchical Bayesian Model in a Phase II Trial (Thall, et at. 2003)
A single arm trial of Imatinib (Gleevec, STI571) in sarcoma, accounting for multiple disease subtypes.
i = Pr( Tumor response in subtype i )
Prior: logit(i) | ~ i.i.d Normal( ), i=1,…,k
Hyperprior: ~ N( -2.8, 1), ~ Ga( 0.99, 0.41 )
Stopping Rule: Terminate accrual within the ith subtype if
Pr( i > 0.30 | Data ) < 0.005
“Data” refers to the data from all 10 subtypes.
But where did these numbers come from?
Eliciting the Hyperprior
Denote Xi = # responders out of mi patients in subtype i.
1) I fixed the mean of at logit(.20) = -1.386, to correspond to mean prior response rate midway between the target .30 and the uninteresting value .10.2) I elicited the following 3 prior probabilities :
Pr( 1 > 0.30 ) = 0.45
Pr( 1 > 0.30 | X1 / m1 = 2/6) = 0.525
Pr( 1 > 0.30 | X2 / m2 = 2/6) = 0.47
Prior Correlation Between Two Sarcoma Subtype Response
Probabilities 1 and 2
Two Priors for a Phase II-III Pediatric Brain Tumor Trial
A two-stage trial of 4 chemotherapy combinations :S = carboplatin + cyclophosphamide + etoposide + vincristine
E1 = doxorubicin + cisplatinum + actinomycin + etoposide
E2 = high dose methotrexate
E3 = temozolomide + CPT-11
Outcome (T,Y) is 2-dimensional : T = disease-free survival timeY = binary indicator of severe but non-fatal toxicity
Both p(T | Y,Z,) and p(Y | Z,) account for patient covariates:
Age, I(Metastatic disease), I(Complete resection)
I(Histology=Choriod plexus carcinoma)
Probability Model
1) T| Z,Y, j ~ lognormal with variance T2 and
mean T,j(Z,Y,) = T,j + T(Z,Y)
T,j = effect of trt j on T, after adjusting for Z and Y
For j=0 (standard trt), T = (T,0 , T)
2) logit{Pr(Y=1 | Z, j)} = Y,j + Y Z
Y,j = effect of trt j on Y, after adjusting for Z
For j=0 (standard trt), Y = (Y,0 , Y)
Toxicity Probability as a Function of Age Elicited from Three Pediatric Oncologists
Probability Model for Toxicity
logit{Pr(Y=1 | Z, , j=0)} = Y,0 + Y,1 Age1/2 + Y,2 log(Age)
was determined by fitting 72 different fractional polynomial functions and picking the one giving the smallest BIC.
Estimated linear term with posterior mean subscripted by the posterior sd is
This determined the prior of Y
64 Elicited EFS Probabilities
How do you use these 64
probabilities to solve for 10
hyperparameters?!!
Johannes Wolff, MD
T = (T,j , T, T) has prior
Regard each prior mean EFS prob as a func of
Use nonlinear least squares to solve for
by minimizing E(T) = (0.44, -0.41, 0.56, -0.53) with common
variance 0.152
and log(T) ~ N(-0.08, 0.142)
Prior for T
YE = indicator of Efficacy
YT = indicator of Toxicity
d = assigned dose
Z = vector of baseline patient covariates
Model the marginals
E(d, Z) = Prob(E if d is given to a patient with covs Z)
T(d, Z) = Prob(T if d is given to a patient with covs Z)
Use a copula to define the joint distribution :
a,b = Pr(YE=a, YT=b) is a function of E(d, Z) and T(d, Z)
A Phase I/II Dose-Finding Method Based on E and T that Accounts for Covariates
E = link{ E(d,Z) } & T = link{ T(d,Z) }
where E(d,Z) & T(d,Z) are functions of
[ dose effects ] + [ covariate effects ]
+ [ dose-covariate interactions ]
a,b = Pr(YE=a, YT=b) = func(E, T ,for a, b = 0 or 1
Model for E(d,Z) and T(d,Z)
For the trial:
E(x, Z) = f(x,E) + EZ + x EZ
For the historical treatment j :
E( j, Z) = E,j + E,HZ + E,jZ
Linear Terms of the Model for E(,Z)
Dose effect Covariate effects Dose-Covariate Interactions
Historical trt effect Historical trt-covariate interactions
For the trial:
T(x, Z) = f(x,T) + TZ + x TZ
For the historical treatment j :
T( j, Z) = T,j + T,HZ + T,jZ
Linear Terms of the Model for T(,Z)
Dose effect Covariate effects Dose-Covariate Interactions
Historical trt effects Historical trt-covariate interactions
In planning the trial, historical data are used to estimate patient covariate main effects :
Prior(T) = Posterior(T,H | Historical data)
Prior(E) = Posterior(E,H | Historical data)
The estimated covariate effects are incorporated
into the model for E(d,Z) and T(d,Z) used to
plan and conduct the trial
Using Historical Data
For a reference patient Z*, elicit prior means of T(xj, Z*) and E(xj, Z*) at each dose xj to establish prior means of the dose effect parameters
Assume non-informative priors on dose effects and dose-covariate interactions
Use prior variances to tune prior effective
sample size (ESS) in terms of E and T
Establishing Priors
Control the prior ESS to make sure that the data
drives the decisions, rather than the prior on
the dose-outcome parameters
Application
A dose-finding trial of a new “targeted” chemo-chemo-protective agent (CPA)protective agent (CPA) given with idarubicin + cytosine arabinoside (IDA) for untreated acute myelogenous leukemia (AML)patients age < 60
Historical data from 693 AML patients
Z = (Age, Cytogenetics)
where Cytogenetics = (Poor, Intermediate, Good)
Inv-16 or t(8:21) -5 or -7
Application
Efficacy = Alive and in Complete Remission at day 40 from the start of treatment
Toxicity = Severe (Grade 3 or worse) mucositis, diarrhea, pneumonia or sepsis within 40 days from the start of treatment
Doses and Rationale
The CPACPA is hypothesized to decrease the risk of IDA-induced mucositis and diarrhea and thus allow higher doses of IDA
Fixed CPACPA dose = 2.4 mg/kg and ara-C dose = 1.5 mg/m2 daily on days 1, 2, 3, 4
IDA dose = 12 (standard), 15, 18, 21 or 24 mg/m2
daily on days 1, 2, 3 (five possible IDA doses)
Interactive
E( j, Z) = E,j + EZ + E,jZ
T( j, Z) = T,j + TZ + T,jZ
Additive
E( j, Z) = E,j + EZ
T( j, Z) = T,j + TZ
Reduced
E( j, Z) = E + EZ
T( j, Z) = T + TZ
Models for the linear terms used to fit the historical data
No treatment-covariate interactions
No differences between the
historical treatment effects
Model Selection for Historical Data
Posteriors of E(, Z) and T(, Z) based on
Historical Data from 693 Untreated AML Patients
1) Choose each patient’s most desirable dose based on his/her Z
2) No dose acceptable for that Z : No dose acceptable for that Z :
Do Not TreatDo Not Treat
3) At the end of the trial, use the fitted model to pick ( d | Z ) for future patients
Dose-Finding Algorithm
The trial’s entry criteria may change dynamically dynamically during the trial :
1) Different patients may receive different doses at the same point in the trial
2) Patients initially eligible may be ineligible (no acceptable dose) after some data have been observed
3) Patients initially ineligible may become eligible after some data have been observed
Hypothetical Trial Results :
Recommended Idarubicin Doses by Z
AGE Cyto Poor Cyto Int Cyto Good
18 – 33 1818 2424 2424
34 – 42 1818 2121 2424
43 – 58 1515 1818 2121
59 – 66 1212 1515 1818
> 66 NoneNone 1212 1515
Currently being used to conduct a 36-patient trial to select among 4 dose levels of a new cytotoxic agent for relapsed/refractory Acute Myelogenous Leukemia
Y = (CR, Toxicity) at 6 weeks
Z = (Age, [1st CR > 1 year], Number of previous trts)
Marina Konopleva, MD, PhDis the PI
Bibliography
[1] Morita S, Thall PF, Mueller P. Determining the effective sample size of a parametric prior. Biometrics. 64:595-602, 2008.
[2] Morita S, Thall PF, Mueller P. Evaluating the impact of prior assumptions in Bayesian biostatistics. Statistics in Biosciences. In press.
[3] Thall PF, Cook JD. Dose-finding based on efficacy-toxicity trade-offs. Biometrics, 60:684-693, 2004.
[4] Thall PF, Simon R, Estey EH. Bayesian sequential monitoring designs for single-arm clinical trials with multiple outcomes. Statistics in Medicine 14:357-379, 1995.
[5] Thall PF, Wathen JK, Bekele BN, Champlin RE, Baker LO, Benjamin RS. Hierarchical Bayesian approaches to phase II trials in diseases with multiple subtypes. Statistics in Medicine 22: 763-780, 2003.
[6] Thall PF, Wooten LH, Nguyen HQ, Wang X, Wolff JE. A geometric select-and-test design based on treatment failure time and toxicity: Screening chemotherapies for pediatric brain tumors. Submitted for publication.
[7] Thall PF, Nguyen H, Estey EH. Patient-specific dose-finding based on bivariate outcomes and covariates. Biometrics. 64:1126-1136, 2008.