Sensitivity analysis for not-at-random missing data in ... · Leurent B, et al. Missing data in trial-based cost-effectiveness analysis: an incomplete journey. Health Econ 2018 Mason

Sensitivity analysis for not-at-random missingdata in trial-based cost-effectiveness analysis

using multiple imputation

Baptiste Leurent, Manuel Gomes, Rita Faria,Steve Morris, Richard Grieve, James Carpenter

ISPOR EuropeNovember 2018

ISPOR 2018 / Sensitivity Analysis for Not-At-Random Missing Data 1/13

Introduction Pattern-mixture models Example: 10TT Conclusion

Introduction

Missing data are common in RCTs→ Loss of power→ Risk of bias

Particularly important in CEAComplex dataLong term follow-up

Typically assume data are "missing at random"But risk of being missing could depend on the data valueitself→ MNAR

ISPOR 2018 / Sensitivity Analysis for Not-At-Random Missing Data 2/13

Introduction Pattern-mixture models Example: 10TT Conclusion

Missing Not At Random

MAR: Missingness only depends of observed variablesCan get valid inference using the observed data

MNAR: missingness depends of outcome value itselfE.g. less likely to complete a health questionnaire when ill

! Cannot judge from the data! Need additional assumptions to conduct the analysis

But often plausible

Guidelines→ Should assess whether results robust toMNAR assumptionsClear gap between recommendations and practice

(NRC 2010)

ISPOR 2018 / Sensitivity Analysis for Not-At-Random Missing Data 3/13

Introduction Pattern-mixture models Example: 10TT Conclusion

Pattern-mixture models

Pattern-mixture models (PMM) are one possibleapproach for MNAR analysisDistribution = mixture of observed and missing distributionsFor example: assuming missing and observed data havesame distribution, but with mean shifted by δ

Ymiss = Yobs + δ

δ = average difference between missing and observedvalues (conditionally on observed data )Can also use a multiplicative factor c:

Ymiss = Yobs × c

ISPOR 2018 / Sensitivity Analysis for Not-At-Random Missing Data 4/13

Introduction Pattern-mixture models Example: 10TT Conclusion

Pattern-mixture models

ISPOR 2018 / Sensitivity Analysis for Not-At-Random Missing Data 5/13

Introduction Pattern-mixture models Example: 10TT Conclusion

Implementation of PMM with ultiple Imputation

Multiple Imputation (MI) commonly used in trial-based CEATypically under MAR, but can accommodate MNARIdea is simple:

1 Conduct usual MIUnder MAR

2 Modify imputed data to reflect MNAR assumptione.g. reduce imputed data by 10%

3 Analyse as usual MI datasetUsing Rubin’s rules

Sensitivity analysis can be conducted over a range ofplausible values for δ, to see how affect conclusions

ISPOR 2018 / Sensitivity Analysis for Not-At-Random Missing Data 6/13

Introduction Pattern-mixture models Example: 10TT Conclusion

Example: the 10 Top Tips trial

RCT, evaluating a brief intervention for weightloss, in UK general practicesPrimary outcome: weight loss at 3 monthsFollow-up: 2 years (3, 6, 12, 18, 24 months)CEA:

Costs: NHS resource use over 2 yearsEffectiveness: EQ-5D at each visit→ QALYsover 2 years

N=537 patientsBut only 60% at 24M, and 30%complete cost-effectiveness dataMore likely to drop out if lesssuccessful→ MNAR

ISPOR 2018 / Sensitivity Analysis for Not-At-Random Missing Data 7/13

Introduction Pattern-mixture models Example: 10TT Conclusion

MNAR sensitivity analysis of 10TT

Applying PMM approach to 10TT:Let’s denote c = MNAR multiplicative parameter for theQoL scorese.g. c = 0.9: the missing QoL are assumed 10% lower thanunder MARWhat values for c?

c = {1,0.95,0.90} (= drop out probably worst off,somewhere between MAR, and 10% worst)c could differ between arms, but more likely to be close toeach other→ 7 scenarios considered

ISPOR 2018 / Sensitivity Analysis for Not-At-Random Missing Data 8/13

Introduction Pattern-mixture models Example: 10TT Conclusion

10TT sensitivity analysis - Results

→ Under MAR, 48% probability 10TT cost-effective→ But results sensitive to departure from MAR

ISPOR 2018 / Sensitivity Analysis for Not-At-Random Missing Data 9/13

Introduction Pattern-mixture models Example: 10TT Conclusion

Discussion

Easy to implementResults can be sensitive (not always the case!)Challenges:

Choosing sensitivity parametersExpert opinionTipping-point

ReportingClarity is essential

Alternative: reference-basedimputation

ISPOR 2018 / Sensitivity Analysis for Not-At-Random Missing Data 10/13

Introduction Pattern-mixture models Example: 10TT Conclusion

Conclusion

Can never recover missing information⇒ avoiding missingdata best solution

Missing data→ make assumptions→ what if do not hold?

Multiple imputation offers a convenient way to conductthese sensitivity analyses

Some challenges: elicitation, reporting, etc.⇒ But not excuses not to conduct them

Likely will evolve over time, as become more routinelyconducted

ISPOR 2018 / Sensitivity Analysis for Not-At-Random Missing Data 11/13

Introduction Pattern-mixture models Example: 10TT Conclusion

Acknowledgements

James Carpenter1, Manuel Gomes2, Rita Faria3, SteveMorris2, and Richard Grieve1

1 London School of Hygiene and Tropical Medicine2 University College London3 University of York

The 10TT investigators

Funded by the National Institute for Health Research

ISPOR 2018 / Sensitivity Analysis for Not-At-Random Missing Data 12/13

Introduction Pattern-mixture models Example: 10TT Conclusion

References

Leurent B, et al. Sensitivity Analysis for Not-at-Random Missing Data in Trial-BasedCost-Effectiveness Analysis: A Tutorial. PharmacoEconomics 2018

Faria R, et al. A Guide to Handling Missing Data in Cost-Effectiveness AnalysisConducted Within Randomised Controlled Trials. PharmacoEconomics 2014Leurent B, et al. Missing data in trial-based cost-effectiveness analysis: an incompletejourney. Health Econ 2018Mason AJ, et al. Development of a practical approach to expert elicitation forrandomised controlled trials with missing health outcomes: Application to theIMPROVE trial. Clin. Trials. 2017Beeken RJ, et al. A brief intervention for weight control based on habit-formation theorydelivered through primary care: results from a randomised controlled trial. Int. J. Obes.2017

National Research Council, 2010. The Prevention and Treatment of Missing Data inClinical Trials, National Academies PressLittle, R. & Rubin, D., 2014. Statistical analysis with missing data, John Wiley & Sons.Carpenter, J. & Kenward, M., 2012. Multiple imputation and its application, John Wiley& Sons.

ISPOR 2018 / Sensitivity Analysis for Not-At-Random Missing Data 13/13