FDA Guidance on PROs Breakfast Seminar 19 th March 2009 Diane Wild, MSc Oxford Outcomes Ltd., Oxford, UK
Nov 18, 2014
FDA Guidance on PROsBreakfast Seminar 19th March 2009
Diane Wild, MScOxford Outcomes Ltd., Oxford, UK
Overview
• What are PROs?
• Why and when to use them
• The FDA draft guidance
• After the guidance was released
• ePRO migration – levels of evidence
PROs – FDA Definition
• A PRO is a measurement of any aspect of a patient’s health status that comes directly from the patient (i.e., without the interpretation of the patient’s responses by a physician or anyone else) (FDA Draft Guidance, 2006).
The PRO umbrella
• Conceptual confusion between QoL, HRQoL, functional status etc.
• The term patient reported outcomes was agreed by a group of interested parties (ISPOR, FDA etc) in September 2000.
• PRO describes the source rather than the content.
Why use PROs
• Some treatment effects only known to the patient (e.g. pain, fatigue)
• Patients provide a unique perspective on effectiveness (e.g. spirometry)
• Formal assessment more reliable than informal interview
For what purpose?
• Definition of entry criteria
• Evaluation of efficacy
• Evaluation of adverse events
Timing and frequency
• As events occur
• At regular intervals
• Baseline and end of treatment
Taxonomy of PROs
• Concepts:– Functional status (physical, psychological, social).– Symptoms/signs (frequency, severity,
bothersomeness)– Health perceptions/perceived health status.– Satisfaction/preference for treatment– Adherence – QoL– HRQoL
QoL/HRQoL?
• FDA draft guidance – extremely complex.
• QoL is distinct from HRQoL as it includes perceptions of immediate environment, housing conditions etc – Have you enough money to meet your needs?– How safe do you feel in your daily life?
Hickey, A. M et al. BMJ 1996;313:29-33
Fig 1--The segments represent five areas of life nominated by the individual; the size of the segment can be adjusted to show the relative importance of each area
for the individual's quality of life
Generic & Condition Specific
Generic Measures• Designed for general use/broad base • Comparable across disease states
Disease Specific Measures• Designed for use within a particular disease • More sensitive to smaller clinically significant changes• Maybe more sensitive to specific impact of clinical
variables• Relatively narrow focus may prevent sensitivity to
treatment side-effects.
0
10
20
30
40
50
60
70
80
90
100
PF RP BP GH VT SF RE MH
Age & Gender Equivalent Norms(Ware et al, 1993, 94)
Chronic Obstructive PulmonaryDisease (Jones & Bosh, 1997)
HRQL of Patients with COPD vs. General Population Norms (SF-36)
High scores = good HRQL
FDA Draft Guidance
Draft guidance – rationale for development
• FDAs thinking on how sponsors can develop and use study results measured by PROs to support labelling claims.
• To increase efficiency of discussions, streamline FDAs review and provide optimal information about the patients view of treatment benefit.
Draft guidance contents (1)
• Selection of PROs
• Development of PROs
• Modification of PROs
• Validation of PROs
Assess Measurement PropertiesAssess score reliability, validity and ability to detect change.
Evaluate administrative & respondent burden. Add, delete or revise items. Identify meaningful differences in scores. Finalise instrument formats,
scoring, procedures & training materials.From FDA Draft PRO Guidance document 2006
Identify Concepts & Develop Conceptual Framework
Identify concepts & domains that are important to patients. Determine intended population and research application.
Hypothesize expected relationships among concepts.
Create InstrumentGenerate items.
Choose administration method, recall period &
response scales.Draft instructions.
Format instrument. Draft procedures for
scoring & administration. Pilot test draft instrument.
Refine instrument & procedures.
Modify InstrumentChange concepts measured, populations studied, research application, response options, recall periodor method of administration.
PRO
Modifications – The FDA’s View
1. Revising the PRO instrument’s measurement concept,
2. Applying an instrument to a new population or condition,
3. Changing the PRO’s item content or formatting,
4. Changing the mode of administration, OR
5. Changing the target culture or language for application of the instrument
Draft Guidance, Lines 570-671
Draft guidance contents (2)
• Specific populations (children, cognitively impaired)
• Study design (blinding and randomisation, frequency of assessments etc)
• ePROs (record keeping, maintenance and access issues)
• Data analysis (missing data, interpretation etc)
After the draft guidance
• Consultation period – comments sent to FDA• Series of papers in Value in Health expanding
on the issues in the guidance • ISPOR Task forces
– Content validity– Translation and linguistic validation– ePRO
• Final guidance will be released……
PRO Review on Critical PathPreclinical Phase 2A Phase 2B Phase 3Phase I
Pre-IND EOP2BEOP2AEOP1
Initial PRO Submission
(Updated) PRO submission
Submit Final PRO
Evidence Dossier
PRO Selection/ Development
PRO Validation/ Modification
Approved Product LabelsPRO Claims in FDA and EMEA Product Labels Approved between 1 January 2000 and 2 June 2008
FDA Approvals by Year EMEA Approvals by Year
Year HRQLSigns & Sympto
ms
AnyPRO
All Approva
lsHRQL
Signs & Sympto
ms
Any PRO
All Approva
ls
2000 0 25 25 96 3 1 4 31
2001 1 18 18 71 6 11 13 38
2002 1 14 15 79 0 10 10 38
2003 1 17 18 79 2 6 9 18
2004 0 20 23 114 6 8 11 36
2005 1 13 15 82 3 7 7 23
2006 4 15 18 98 0 5 5 42
2007 1 11 14 75 7 13 13 69
2008* 0 4 4 34 2 4 4 26
Total 9 137 150 728 29 65 76 331
8.5 yrMean
1.1 16.1 17.6 85.6 3.4 7.6 8.9 37.8
Label Claims Based on Patient-Reported Outcomes in EMEA and FDA Approvals Since 2000Jane Scott etal. Poster presented at ISPOR Athens 2008
ePRO migration:level of evidence required to establish equivalence
Migration and Re-validation
• FDA guidance states “the extent of additional validation recommended depends on the type of modification made.” (pg 20, line 582-583)
• Ambiguity
ISPOR ePRO consensus group: Modification categories
• Minor – no change in content or meaning (non substantive changes in instructions, minor changes in format)
• Moderate – May change content or meaning (changes in item wording or changes in presentation that might alter interpretability, visual to aural)
• Substantial – Clear change in content or meaning (substantial changes in response options or item wording)
ISPOR ePRO consensus group: Levels of evidence
• Minor – Small scale cognitive debriefing and usability testing
• Medium – Formal assessment of equivalence and usability testing
• Substantial – large scale usability testing and full psychometric testing
Levels of Evidence
• Cognitive Debriefing– Explores the way individuals understand, mentally
process and respond to items on a questionnaire (Willis 2005)
– Similar to target sample– 5-10 patients– Verbal probing technique– Needs to be documented
Levels of Evidence
• Usability Testing– Related to the software and device– Navigation of electronic platform– Scale should be based on the complexity of
the physical and cognitive tasks required by the ePRO application
Levels of Evidence
• Equivalence Testing– Administer paper and ePRO versions– Parallel vs cross-over– Similar to target sample– Basic classical test theory techniques (internal
consistency, means, standard deviations etc)
Levels of Evidence
• Psychometric Evaluation – Factor structure– Internal consistency– Test-retest– Construct validity– Responsiveness– MID
Comparisons of ePRO and paper PRO
• Gwaltney et al 2008 – meta-analysis of 46 studies and 275 PROs
• Very high correlations (.90)• Little evidence to suggest that size of
computer screen, respondent age, or amount of computer experience had any impact
• Respondents prefer ePRO over paper
Translation issues
• Translation can be significantly more wordy than source text – mixed modes of administration not recommended
• Possibilities of combining linguistic validation steps with migration cognitive debriefing
Summary
• PROs useful umbrella term for a variety of measures
• FDA draft guidance has helped to clarify and streamline review process
• Level of evidence for equivalence of ePROs is dependent on changes required for migration
• Growing evidence to suggest that equivalence is generally high and ePROs preferred by patients