Page 1
GeneralizedPairwiseComparisononImmuno-
Oncologyclinicaltrialdata:acasestudy
DrJulienPERON,PrDelphineMAUCORT-BOULCH,PrPascalROY,PrMarcBUYSENovember2017
DepartmentofBiosta>s>csHCL–LBBEUCBLDepartmentofMedicaloncologyHCL–LBBEUCBL
Page 3
3
TheCA184-024trial
R
502metasta>cmelamoma
Placebo+dacarbazineIpilimumab+dacarbazine
252250
Robertetal.NEJM2011
Page 4
4
OSresultsintheCA184-024trial
Pcb 252 160 89 64 44 37 26 7 0
Ipi 250 181 114 85 68 57 41 10 0
Page 5
Outline
• Theprocedureofgeneralizedpairwisecomparisons
• Apa>ent-orientedmeasureoftreatmentbenefit
• Applica>ononimmuno-oncologytrials• Simula>onstudy• Illustra>ononanipilimumabtrial
5
Page 6
Methods–Pairwisecomparisons
Let xi be the outcome of i thsubject in T (i = 1. … . n )
R
Control (C ) Treatment (T )
Let yj be the outcome of j thsubject in C (j = 1. … . m )
Yj Xi
favors T (favorable)
favors C (unfavorable)
pairwise comparison
Neutral or Uninformative6
BuyseM.statinmed2010
Page 7
Methods–DefiniQonofthresholds
CouQnuousoutcome
7Buyse.statinmed2010
Pair Rating � > � Favorable
� < (� �) Unfavorable � � � ≤ � Neutral
or missing Uninformative
ConQnuousoutcome
Page 8
Methods–Standardprocedureforpairwisescoring
innamed«netbenefit»
Anempiricaldistribu>onofcanbeobtainedbypermuta>on
8Buyse.statinmed2010
Δ =U = 1
m⋅n ijUj=1
m
∑i=1
n
∑
( )( )
otherwise 0
eunfavorabl is pair when the1
favorable is pair when the1
⎪⎪⎩
⎪⎪⎨
⎧
−
+
= Y j,X i
Y j,X i
U ij
Δ
Δ
Page 10
10
�
Favorable Unfavorable Neutral
Favorable Uninforma>ve Uninforma>ve
Uninforma>ve Unfavorable Uninforma>ve
Uninforma>ve Uninforma>ve Uninforma>ve
Buyse M. Stat in med, 2010
ThestandardproceduretoincludeQme-to-event’outcome
Page 11
11
0,5
1,0
SurvivalProbability
0,0
Time
Pa>enti:censoring
Treatmentgroup
Controlgroup
Pa>entj:event
ThestandardproceduretoincludeQme-to-eventoutcome
Gehan. Biometrika, 1965
Page 12
BasedontheKaplan-Meieres>mateofthesurvivalfunc>on
𝕡[( 𝑥↓𝑖↑0 > 𝑦↓𝑗↑0 )�( 𝑥↓𝑖↑0 > 𝑥↓𝑖↑ )]= 𝑆 ↓𝑇𝑡𝑡 (𝑦↓𝑗 )/𝑆 ↓𝑇𝑡𝑡 ( 𝑥↓𝑖 ) = 0,5/0,8
Theextendedproceduretakingintoaccount‘non-informaQve’pairs
0,5
1,0
SurvivalProbability
0,0
Time
0,8
Péron J et al, SMMR 2016
Pa>enti:censoring
Pa>entj:event
Treatmentgroup
Controlgroup
Page 13
13
𝕡[( 𝑥↓𝑖↑0 > 𝑦↓𝑗↑0 )�( 𝑥↓𝑖↑0 > 𝑥↓𝑖↑ ),( 𝑦↓𝑗↑0 > 𝑦↓𝑗↑ )]=−∑𝑡> 𝑦↓𝑗 ↑∞▒𝑆 ↓𝑇𝑡𝑡 (𝑡)/𝑆 ↓𝑇𝑡𝑡 (𝑥↓𝑖 )𝑆 ↓𝐶𝑡𝑟𝑙 (𝑦↓𝑗 ) ∙(𝑆 ↓𝐶𝑡𝑟𝑙 (𝑡↑+ )− 𝑆 ↓𝐶𝑡𝑟𝑙 (𝑡↑− ))
Efron, Berkeley Symp, 1967
0,5
1,0
0,0
Whenthees>ma>onofthesurvivalfunc>onisdiscon>nue:
SurvivalProbability
Time
Pa>enti:censoring
Treatmentgroup
Controlgroup
Pa>entj:censoring
Theextendedproceduretakingintoaccount‘non-informaQve’pairs
Page 14
14
Theextendedproceduretakingintoaccount‘non-informaQve’pairs
benefitisthen:
Page 15
• Reduc>onoftheBiasofinthepresenceofcensoredobserva>ons– Correc>onavailable
• Increasedpowerofthepermuta>ontestcomparedtostandardprocedure– Propor>onalhazardsandadministra>vecensoring<67%(BEfron,Stanford
Univ,1967)
– Latetreatmenteffect
15
Achievementsoftheextendedprocedure
(propor>onalhazards)
Péron J et al, SMMR 2016
Page 16
Outline
• Theprocedureofgeneralizedpairwisecomparisons
• Apa>ent-orientedmeasureoftreatmentbenefit
• Applica>ononimmuno-oncologytrials• Simula>onstudy• Illustra>ononanipilimumabtrial
16
Page 17
17
Probabilityforarandompa>entintheTreatmentgrouptohavea‘bederoutcome’thanarandompa>entintheControlgroup…
Δ=ℙ(𝑿>𝒀)−ℙ(𝑌>𝑋)
Thenetbenefit
Buyse M. Stat in med, 2010
Treatementgroup Controlgroup
Page 18
18
Δ=ℙ(𝑋>𝑌)−ℙ(𝒀>𝑿)
Buyse M. Stat in med, 2010
Treatementgroup Controlgroup
…minustheoppositeprobability.
Thenetbenefit
Page 19
19
Δ=ℙ(𝑋>𝑌)−ℙ(𝑌>𝑋)
ℙ(𝒀=𝑿)
Buyse M. Stat in med, 2010
Treatementgroup Controlgroup
Thenetbenefit
Page 20
20
Thenetsurvivalbenefit
ProporQonalhazards
TreatmentgroupControlgroup
Time(months)
Netsu
rvivalben
efit
Survivalprobability
Péron et al, JAMA oncology, 2016
Page 21
21
Propor>onalHazards
Delayedtreatmenteffect
TreatmentgroupControlgroup
TreatmentgroupControlgroup
Time(months)
Netsu
rvivalben
efit
Survivalprobability
Time(months)
Netsu
rvivalben
efit
Survivalprobability
Péron et al, JAMA oncology, 2016
Thenetsurvivalbenefit
Page 22
22
Oppositehazards
Péron et al, JAMA oncology, 2016
Propor>onalHazards
TreatmentgroupControlgroup
Time(months)
Netsu
rvivalben
efit
Survivalprobability
TreatmentgroupControlgroup
Time(months)
Netsu
rvivalben
efit
Survivalprobability
Thenetsurvivalbenefit
Page 23
Outline
• Theprocedureofgeneralizedpairwisecomparisons
• Apa>ent-orientedmeasureoftreatmentbenefit
• Applica>ononimmuno-oncologytrials• Simula>onstudy• Illustra>ononanipilimumabtrial
23
Page 24
SimulaQonstudy-Design
• ObjecQve:Toassessthepoweroftestsbasedongeneralizedpairwisecomparisonsfordelayedtreatmenteffect
• Simula>onofM=1000datasetswithN=200pa>ents– One>me-to-eventoutcome
Page 25
25
Scenario1:Propor>onalhazards
Scenario2:Latetreatmenteffect
SimulaQonstudy-Design
Survival
Time(months)
Survival
Time(months)
0 10 20 30 40 50
0.0
0.5
1.0
Time (months)
Haza
rd ra
tio
Page 26
• Administra>vecensoringpropor>on– Uniformdistribu>on– Between0%and20%
• Foreachsimulateddataset– Es>ma>onofthenetsurvivalbenefitofatleastτmonths[0to42
months](extendedprocedure)– Testofthenullhypothesis(Permuta>ontest,Log-Ranktest)
26
SimulaQonstudy-Design
Page 27
27
ProporQonalHazards-POWER
Page 28
28
Delayedtreatmenteffect-POWER
Page 29
Whenalong-termsurvivalbenefitisexpected
(an>cancerimmunetherapy)
Thenetsurvivalbenefitis:
– Arguablymorerelevantthantradi>onalmethodsèfocusonlongtermsurvivaldifferences
– Morepowerfulthantradi>onalmethod
29
ConclusionsofthesimulaQonstudy
Page 30
Outline
• Theprocedureofgeneralizedpairwisecomparisons
• Apa>ent-orientedmeasureoftreatmentbenefit
• Applica>ononimmuno-oncologytrials• Simula>onstudy• Illustra>ononanipilimumabtrial
30
Page 31
31
ThenetsurvivalbenefitintheCA184-024trial
R
502metasta>cmelamoma
Placebo+dacarbazineIpilimumab+dacarbazine
252250
Robertetal.NEJM2011
Page 32
32
OSresultsintheCA184-024trial
Pcb 252 160 89 64 44 37 26 7 0
Ipi 250 181 114 85 68 57 41 10 0
Page 33
33
OSresultsintheCA184-024trial
Page 34
34
OSresultsintheCA184-024trial
Δ(12)=11.5%(95%CI=3.5%-19.4%;P=0.0045)
Δ(0)=12.5%(95%CI=2.1%-23.0%;P=0.018)
LogrankP=0.0054
Page 35
35
PFSresultsintheCA184-024trial
Pcb 252 52 20 13 2 1 0 0 0
Ipi 250 70 40 25 6 2 0 0 0
Page 36
36
PFSresultsintheCA184-024trial
Page 37
37
PFSresultsintheCA184-024trial
Δ(12)=7.6%(95%CI=1.5%-13.8%;P=0.015)
Δ(0)=9.3%(95%CI=-1.0%-19.6%;P=0.076)
LogrankP=0.022
Page 38
ApackageR• AvailableonCRAN(“BuyseTest”)• Availableongithub(“hdps://github.com/bozenne/BuyseTest”)
38
SobwareimplementaQon
Page 39
Thenetbenefit– Isequivalenttostandardnon-parametrictestsinsimplecases
– IsmeaningfulandpaQent-relevant– Canfocusonlong-termsurvivaldifferences– AllowsmulQcriteriaanalysis– Mayhavebederpowerthanthelogranktest(e.g.fordelayedtreatmenteffect)
– IsOKwhenhazardsarenotproporQonals– Isavailable
39
Conclusions
Page 41
41
References
Buyse M. Reformulating the hazard ratio to enhance communication with clinical investigators. Clin Trials 5: 641-2, 2008.
Buyse M. Generalized pairwise comparisons for prioritized outcomes in the two-sample problem. Statist Med 29: 3245-57, 2010.
Péron J, Buyse M, Ozenne B, Roche L, Roy P. An extension of generalized pairwise
comparisons for prioritized outcomes in the presence of censoring. Statist Meth Med Res DOI: 10.1177/0962280216658320, 2017.
Péron J, Roy P, Ding K, Parulekar W, Roche L, Buyse M. Benejit-risk assessment of adding erlotinib to gemcitabine for the treatment of advanced pancreatic
cancer. Brit J Cancer 112: 971-976, 2015.
Péron J, Roy P, Ozenne B, Roche L, Buyse M. The net chance of a longer survival as a patient-oriented measure of benejit in randomized clinical trials. JAMA Oncology DOI: 10.1001/jamaoncol.2015. 6359, 2016.
Page 42
Methods – Definition of priority
First priority outcome
Second priority outcome
Pair rating
Favorable NA Favorable Unfavorable NA Unfavorable
Neutral/Uninf Favorable Favorable Neutral/Uninf Unfavorable Unfavorable Neutral/Uninf Neutral/Uninf Neutral/Uninf
42 Buyse. stat in med 2010
Page 43
Methods – Definition of priority
First priority outcome
Second priority outcome
Pair rating
Favorable NA Favorable Unfavorable NA Unfavorable
Neutral/Uninf Favorable Favorable Neutral/Uninf Unfavorable Unfavorable Neutral/Uninf Neutral/Uninf Neutral/Uninf
43 Buyse. stat in med 2010
Page 44
Methods – Definition of priority
First priority outcome
Second priority outcome
Pair rating
Favorable NA Favorable Unfavorable NA Unfavorable
Neutral/Uninf Favorable Favorable Neutral/Uninf Unfavorable Unfavorable Neutral/Uninf Neutral/Uninf Neutral/Uninf
44 Buyse. stat in med 2010
Page 45
Simulation study - Design
• Objective: To compare the standard and the extended procedures of generalized pairwise comparison
• Simulation of M = 1000 datasets of with N = 200 patients – One time-to-event outcome
– Threshold 𝜏 = 0 months
Page 46
HR HR HR
46
• Survival time: exponential distributions
Scenario 1 : Proportional hazards
Scenario 2 : Late treatment effect
Scenario 3 : early treatment effect
Simulation study - Design
Page 47
47
• Several treatment effect sizes
– Hazard ratio {0,5;0,7;1}
• Administrative censoring proportion – Uniform distribution – Between 0% and 70%
Simulation study - Design
Page 48
48
• For each simulated dataset – Estimation of the net chance of a better outcome (standard and extended
procedure) – Test of the null hypothesis (Permutation test, Log-Rank test)
• Endpoints – Bias – Power – Type 1 error
Simulation study - Design
Page 49
HR = 0,5
HR = 0,7
49
Scenario 1 – Proportional hazards
Péron, et al. SMMR 2016
Page 50
HR = 0,5
HR = 0,7
50
Scenario 1 – Proportional hazards
Péron, et al. SMMR 2016
Page 51
51
An explanation for this bias? 1,0
Sur
viva
l Pro
babi
lity
0,0
Time
C𝐞𝐧𝐬𝐨𝐫𝐢𝐧𝐠 𝒚↓𝒋 E𝐯𝐞𝐧𝐭 𝒙↓𝒊
Standard procedure: U𝐧𝐢𝐧𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐯𝐞 →𝑝↓𝑖𝑗 =0
U𝐧𝐢𝐧𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐯𝐞 𝐚𝐥𝐬𝐨→𝑝↓𝑖𝑗 =0
Treatment group
Control group
Page 52
52
A correction for this bias
HR=0,5
Mean bias
Censoring rate
Péron, et al. SMMR 2016
Page 53
HR = 0,5
HR = 0,7
53
Scenario 1 – Proportional hazards
Censoring rate
Péron, et al. SMMR 2016
Page 54
54
Scenario 2 et 3 – Non Proportional hazards
Censoring rate Censoring rate
Pow
er
Early treatment effect Late treatment effect
Type 1 error rate ≈ 5%
Péron, et al. SMMR 2016