Collecting Real World Evidence - CADTH.ca · Dr. Kelvin Chan, MD FRCPC MSc (Clin Epi) MSc (Biostats) Clinical Lead, Provincial Drug Reimbursement Programs, CCO Co-Director, Canadian

Advancing Health Economics, Services, Policy and Ethics

Collecting Real World Evidence: HTA’s perspective

Dr. Kelvin Chan, MD FRCPC MSc (Clin Epi) MSc (Biostats)

Clinical Lead, Provincial Drug Reimbursement Programs, CCO

Co-Director, Canadian Centre for Applied Research in Cancer Control

Medical Oncologist, Sunnybrook Odette Cancer Centre

• What types of uncertainty do HTA committees encounter in their deliberations?

• How often does pCODR Expert Review Committee (pERC) request collecting evidence to reduce uncertainty?

• What types of information can be potentially obtained by collecting real world evidence (RWE)?

Contents

Disclaimer

• Member of HTAs

• pCODR Expert Review Committee

• Committee to Evaluate Drugs

• Ontario Steering Committee of Cancer Drugs

• Personal opinion

• Does not represent the views of U of Toronto, pCODR, CED, OSCCD, CCO, Ministry etc.

• Information from publicly available source

Ontario Cancer Plan IV (2015-2019)

Health Technology Assessment (HTA) Committee’s Recommendations

Recent Example

In general, HTA committees (pCODR, CED, OSCCD) consider the following inputs:

•Clinical benefit (from clinical trial data) – Efficacy (survival) data – Safety data – Quality of Life (QOL) data

•Patient values

•Cost-effectiveness and budget impact

•Adoption feasibility

Things that HTA committees consider before making a recommendation to fund (or not to fund)

• Uncertainty in clinical data – Survival data

• Missing or limited • Surrogate of overall survival • Non-comparative • Short term data

– Safety data • Missing data/Late data

– Quality of life data • Missing data (numerous examples)

Uncertainty in clinical data

Non-Comparative Data Romidepsin in Peripheral T-Cell Lymphoma

“It was noted that due to the limitations of relying on non-comparative, non-randomized evidence and the heavy reliance on extrapolation of overall survival data, there was substantial uncertainty in the

magnitude of the net clinical benefit associated with romidepsin.”

- pCODR Expert Review Committee Final Recommendation

Adapted from Piekarz et al. BLOOD 2011 Adapted from Coiffier et al. JCO 2012

Missing Quality of Life Data Vismodegib in Basal Cell Carcinoma

“… quality of life and functional outcomes are very important in this population. Patients with BCC who are inappropriate for surgery may experience severe disfigurement, leading to extreme social

isolation and decreased quality of life.”

- pCODR Expert Review Committee Final Recommendation

• Survival is main a concern for patients with locally advanced or metastatic disease

• Disease progression may lead to facial disfigurement, and thus a decreased QoL

• This study presents no QoL data, however researchers connected a response and decreased tumour size to an improvement in overall QoL

Adapted from Sekulic et al. NEJM 2012

• Cost-effectiveness analysis and budget impact analysis

• Model structure and methods – Comparator and long term clinical efficacy

• Uncertainty in the inputs of the model – Number of patients – Duration of drug treatments – Resource utilization

• Underestimation of ICER (sometimes 1-2 fold difference)

• Underestimation of BIA

Economic evidence: Estimation or “guess-timation”

Short Term Survival Data Pembrolizumab in Unresectable Metastatic Melanoma

“In the absence of longer term data, pERC was unable to accept this assumption of prolonged benefit and agreed with the EGP’s use of alternative data sources to extrapolate survival in both settings.”

- pCODR Expert Review Committee Final Recommendation

Adapted from Robert et al. NEJM 2015

• Trial patients are different from real world patients – e.g. age, co-morbidities, performance status, diffusion – Intensive monitoring on trial

• Different practice patterns in the real world – Duration of treatment (treatment until progression vs.

discontinuation before progression) – Dose intensity of treatment in the trials vs. in the real world – Management of side-effects (e.g. febrile neutropenia are

managed mostly as in-patient in Ontario)

• Sequencing of subsequent lines of available therapy – Different from what was available on the trial

• Changes in drug price over time (e.g. generics) – Drop in price in the older drugs will make the new drug less cost-

effective

Loss in “Translation”: Uncertainty about

Translating clinical trial evidence to the real world

EXPERIENCE OF pCODR EXPERT REVIEW COMMITTEE (pERC)

Requesting Real World Evidence

• pCODR 60 reviews (Up to Feb 2016) • Total of 21 pCODR reviews requested Real World

Evidence • 13 pCODR reviews explicitly requested Real World

Evidence • 10 pCODR reviews potentially requested Real World

Evidence Potential RWE Request: Unclear if pERC requested RWE, but it could be beneficial

Next Steps for Real World Evidence Collection

7

1 1

2 2

10

Inform magnitude of clinical benefit and cost-effectiveness or the true cost-effectiveness

Define the potential clinical benefit or magnitude of clinical benefit

Define the population or disease

Inform duration of treatment

Inform duration of treatment and cost-effectiveness

Inform sequencing of available therapies

23 requests for RWE for 21 studies

Breakdown of pCODR Reviews

60 Final Recommendations

11

39

10

Positive Recommendation Conditional Recommendation Negative Recommendation

(4 requested RWE)

(17 requested RWE)

Temporal Trends in RWE Requests 60 pERC Final Recommendations

0

6

12

18

24

2012 2013 2014 2015 2016

Nu

mb

er o

f R

eco

mm

end

atio

ns

Final Recommendation Issue Year

Negative Recommendations

Without RWE Request

With RWE Request

Breakdown by Tumour Site

60 pERC Final Recommendations

0

6

12

18

24

Heme Melanoma GI GU Lung Breast Other Gyne

Nu

mb

er

of

Rec

om

me

nd

atio

ns

Tumour Site

Negative Recommendations

Without RWE Request

With RWE Request

Breakdown by Study Characteristics: 21 Studies

0

2

4

6

8

10

Nu

mb

er o

f St

ud

ies

Primary Endpoint of Final Recommendations Requesting RWE

DSV (Decrease in Spleen Volume), MCyR (Major Cytogenetic Response), MaHR (Major Haematological Response)

WHAT REAL WORLD EVIDENCE CAN WE

COLLECT?

Potential Deliverables (effectiveness, safety, quality of life, cost-effectiveness)

Potential Deliverables (verify economic models)

Discussions

• HTA committees commonly make best possible recommendations based on substantial uncertainties

• pCODR ERC commonly requests for the collection for further evidence to reduce uncertainties

– How can further evidence be collected? • Routinely collected data (population-based admin data) • Evidence building program (prospective collection of data) • Real world experiments (real world pragmatic randomized trials)

– Who will use this evidence once collected? • Practitioners and patients – informed treatment decisions • Policy decision-makers and payers • Re-HTA (recommendation-makers)