Institute Advanced Studies Workshop: BestPractice for Patient Reported Outcomes (PROs) in Randomised Clinical Trials 23 rd July 2013 Report prepared by Dr Melanie Calvert & Mr Derek Kyte, PRO Research Group, University of Birmingham, UK For further information please contact: Dr Melanie Calvert University of Birmingham, UK mail: [email protected]E 1 ttp://www.birmingham.ac.uk/patient‐reported‐outcomes h
76
Embed
Report prepared by Dr Melanie Calvert & Mr Derek Kyte, PRO ... · Report prepared by Dr Melanie Calvert & Mr Derek Kyte, PRO Research Group, ... clinical time points – appropriate
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Institute Advanced Studies Workshop: BestPractice for Patient Reported Outcomes (PROs) in Randomised Clinical Trials
23rd July 2013
Report prepared by Dr Melanie Calvert & Mr Derek Kyte, PRO Research Group, University of Birmingham, UK
For further information please contact: Dr Melanie Calvert University of Birmingham, UK mail: [email protected]
1
ttp://www.birmingham.ac.uk/patient‐reported‐outcomes h
2
Executive Summary Patient reported outcomes (PROs) are increasingly assessed in clinical trials, but PRO trial design and data collection are often suboptimal and the results poorly reported. The University of Birmingham PRO Research Group led an Institute of Advanced Studies Workshop on 2nd‐3rd July 2013. The workshop aimed to disseminate the latest research from the group and its international collaborators, to promote and consider best practices for PRO trial design, dentify research priorities and develop collaborations to obtain funding to isupport these activities. The l i fo lowing research priorit es were identified during the workshop:
1. Determining optimal methods of patient involvement in outcome selection.
2. Establishing the optimal methods for incorporating PROs in Core Outcome Sets.
3. Development of a checklist for protocol writers, detailing recommended n PRO‐specific items for inclusion i trial protocols, with supporting online
tools. ent of ‘PRO 4. The development of consensus guidance on the managem
Alerts’ within trials.
5. Development of PRO data collection/management guidelines. tating clinician and 6. Development of methods and tools aimed at facili
omains. patient understanding of PROs.
7. Research into standardisation of PRO tools/d8. Development of a UK‐based PRO item bank.
The UoB PRO Research Group in collaboration with partner organisations are committed to addressing these priorities and to providing training and education o facilitate best practice for PROs in clinical trials. Details of the PRO Research roup and our international collaborators can be found on our website: tG http://www.birmingham.ac.uk/patient-reported-outcomes
3
Executive Summary .......................................................................................................2 Background ....................................................................................................................4 Acknowledgements........................................................................................................4 Workshop Aims: ............................................................................................................5 W ...........................................................................................................5 orkshop formatSe ..............................................................................6 ssion One: PRO SelectionKey questions:
..............................................................................6 Summary of Gr
..............................................................................7 Se ..............................................................................7 ssion Two: PRO Trial DesignKey Questions:
..................................oup discussions:Summary of Gr
..............................................................................9 Se ..............................................................................9 ssion Three: PRO Trial ConductKey questions:
...........................................11 Key questions:Summary of Group discussions:
Research priorities .......................................................................................................12 Next Steps ....................................................................................................................12 Appendix 1: Attendees:................................................................................................13 Appendix 2: Workshop Programme ............................................................................15 Ap tationspendix 3: Copy of Presen
.............................................................................17 Ap group discussionspendix 4: Notes made during
............................66
.....................................................65 Session 1: PRO Selection .......
............................68 Session 2: PRO Trial Design.......................................................
..............ractice
Session 3: PRO Trial Conduct ......................................Session 4: Use of PRO trial results to inform clinical p ............................69 Session 5: Research priorities: a clinical perspective .........................................71
Patient reported outcomes (PROs) are increasingly assessed in clinical trials and provide 'the patient voice' in evidence on treatment effectiveness. PROs commonly evaluate health‐related quality of life, symptoms such as pain or fatigue, health utility, adherence and patient satisfaction and may be used to influence clinical care and decision‐making, predict long‐term outcomes and inform health policy. Despite this, processes relating to PRO trial design and data collection are commonly suboptimal and the results are often poorly reported. These practices devalue important patient‐centred data, diluting the impact of PRO results in the clinical setting. Researchers at the University of Birmingham are leading a programme of work, which aims to enhance the design, implementation, analysis and reporting of PROs in trials, to improve the way that results are used to inform clinical care. This two‐day workshop aimed to bring together leading experts in PRO trials methodology with key personnel within the Birmingham Clinical Trials Units, UHB clinical leads, researchers involved in PRO data collection and patient representatives, who will collectively work on a strategy to improve standards of PRO trial measurement and reporting:
Locally, through collaboration, academic detailing and other knowledge transfer initiatives.
Internationally, through high‐impact publications and via collaboration with the International Society of Quality of Life Research (ISOQOL) on an international knowledge transfer process.
Acknowledgements
The Workshop was funded by the Institute for Advanced Studies (IAS), University of Birmingham and was organised by Dr Melanie Calvert and Mr Derek Kyte (PRO Research Group), with Sue Gilligan and Sarah Myring (IAS). Dr Melanie Calvert is member of the MRC Midland Hub for Trials Methodology Research, University of Birmingham, UK (Medical Research Council Grant ID G0800808). Derek Kyte is supported by a National Institute for Health Research School for Primary Care Research PhD studentship. The views expressed in the report reflect individuals opinions voiced at the workshop and may not reflect funder views. We thank the workshop participants for their valuable contributions and our speakers for stimulating lively debate (a full list of attendees and the meeting programme with the list of speakers is provided in Appendix 1 and 2 respectively). We thank Helen Duffy and Karen Biddle for assistance in preparing this report.
5
Workshop Aims:
To disseminate key findings of recent advances in PRO methodology for trials with members of clinical trials units and other key stakeholders across the UK, including clinicians and patient representatives.
Generate and prioritise key research/logistical questions surrounding the future identification, dissemination and implementation of best‐practice for PROs in randomised clinical trials and develop a resulting publication/funding/knowledge transfer strategy.
Form working groups tasked with addressing aspects of this strategy, tive. both at a local level and from an international perspec
To consider developing a theme proposal for the IAS.
ts: Benefi
Patient engagement, local & international knowledge translation, publications, collaborative grants.
t: Impac
Output from the workshop will help to improve the quality of PRO information collected from trials, thus ensuring that the clinical care of patients is informed by robust results.
Workshop format The list of attending delegates is provided in Appendix 1 and the workshop programme is provided in Appendix 2. Each session began with presentations from key stakeholders introducing current challenges and opportunities of using PROs in clinical trials. Following the presentations (Appendix 3), delegates formed break‐out groups to discuss research needs and propose research priorities (notes of these discussions are provided in Appendix 4). During day 2 of the workshop, delegates were asked to vote on research priorities (Appendix 5).
6
Session One: PRO Selection
Key questions: g e y iHow do we select PRO domains (e. . fatigu /anxiet ) and quest onnaires
that are valid, acceptable to patients and will inform clinical care? What is the difference between a patient centred and patient reported
outcome? How do we ensure that patient centred outcomes are included in clinical
trials? What other issues should we consider? How can we address these issues?
Summary of Group discussions: Methods to select PROs/ domains. Group discussions focused on identification of appropriate PRO domains associated with the research hypothesis. Delegates highlighted the need for standard methodology to aid selection of PROs with patient input. The need for ongoing validation of existing PROs in different populations was recognised. Inclusion of PROs in Core Outcome sets (COS) The groups recognised a need to develop optimal methods for incorporation of ROs into COS. The group questioned whether PROs in a COS could be used for ultiple purposes ‐ in both trials and routine practice (as an audit tool).
Pm Optimal methods of patient involvement in outcome selection & trial design. Patient involvement in trial design and selection of outcomes should be routine. Researchers need to work closely with patients to ensure that they fully understand their role. Group discussions focused on methods to recruit representative patients including those from ‘hard to reach groups’. Increased ollaboration between patients and researchers at all stages of the study may elp maximise patient benefit. ch Carer Outcomes he group felt that increased assessment of carer outcomes may be warranted in ome trials. Currently the impact of caring may not be assessed. Ts Feedback to patients on trial progress. The group recognized an increased need to provide feedback to trial participants. There is a need to develop and refine optimal methods for providing useful feedback, which could include newsletters, emails, posters, websites, phone apps. Training tools for PRO selection. The group identified a clear need for training on the optimal methods of PRO Selection. Development of methods and training for researchers (and patients) was seen as a priority area for future research.
7
Session Two: PRO Trial Design
Key Questions: What PRO‐related components should be included in the trial protocol or
other trial documentation such as SOPs? rial What are the key areas that need further work to inform optimal PRO t
design? ow do we ensure that trials adopt best‐practice in PRO trial protocol
onfiguration? Ideas to address this? Tools? Education? Hc
Summary of Group discussions:
Considerations for a PRO Protocol Checklist Groups considered a number of items which they felt should be considered for inclusion in a protocol checklist including: a clear rationale for PRO assessment with patient input, identification of the PRO as a primary or secondary endpoint, specification of valid questionnaires (with supporting references), domains linked to the study hypotheses, timing of assessments (measured at appropriate clinical time points – appropriate to the patient journey), a pre‐specified analysis plan (i.e. specific domains, global score) and plans for dealing with missing data (e.g. imputation), and assessing clinical significance. The groups discussed at length, practical issues regarding administration of PROs including ensuring patient privacy and dealing with language barriers. Delegates felt that: (1) further practical guidance on PRO administration should be provided in trial documentation, (2) SOPs should include guidance on how to deal with concerning PRO scores – for example, advocating that the research team contact the patient’s GP in response to such scores, and (3) trial documentation should also specify how feedback should be provided to patients on the progress of the trial. Delegates recognised the need to first decide which domains to measure, based on the study hypotheses, before selecting the most appropriate tool/measure. Consideration should be given to timing of assessment (i.e. before or after clinical consultation/treatment) – this should be specified. Groups generally felt that administration in advance of a consultation may have methodological benefits (patient views not influenced by clinical encounter) and those questionnaires may be more complete. The protocol should specify whether the patients are allowed to receive help (carer/nurse/doctor), e.g. for patients with visual or co‐ordination problems. The groups considered sources of potential bias when patients may ‘want to please/think there’s a right answer’. This ‘noise’ is likely to be balanced across trial arms, but should be considered at the design stage. Justifications for PRO methods, with appropriate references, should be provided in the protocol. Many practical issues associated with PRO trials, such as selecting questionnaires and time‐points for assessment could be considered during pilot work with patient input. Missing data can be a major problem and is a potential source of bias. The groups considered ways to minimise missing data. They felt that trials should pre‐specify in the protocol: approved methods to avoid or minimise missing
8
data, how trial personnel should report reasons for any missing data ncountered and how they should deal with missing data. e PRO Protocol Checklist Tools It was recognized that the development of a checklist for protocol writers, detailing recommended PRO‐specific items for inclusion trial protocols (with supporting online tools) would improve PRO trial design and implementation, and could incorporate both ‘core’ essential items and desirable items. The groups felt that this checklist (similar to CONSORT), if required for publication of protocols, would be helpful. Endorsement of use by major funding bodies would help facilitate uptake. The need for education/training for researchers and peer reviewers was recognized. A ‘stick approach’ was felt to be necessary – e.g. requirements by funders.
9
Session Three: PRO Trial Conduct Key questions:
Is experience of PRO alerts widespread? How best should they be ce we can draw upon?
be included in protocol managed? Is there existing guidan
What information for data collection staff should
guidance/trial training and why? How do we encourage uptake of new guidance?
Do patients understand why we assess PROs in trials? What PRO‐specific tion information should be included in the patient informa
documentation? Are/should PROs be discussed in consent interview?
Summary of Group discussions: PRO Alert Consensus & Guidance Experience of PRO Alerts varied amongst delegates. The groups were in general agreement that predetermined thresholds should be identified and a plan for dealing with PRO Alerts documented in the protocol or SOPs. There is no consensus on optimal methods for dealing with alerts and this was recognized as a research priority. To be effective, alert management needs to occur in a timely way (electronic data capture was discussed). The group discussed data capture of arising actions. Delegates discussed potential methodological challenges of the clinical team viewing PRO data (e.g. risk of unblinding). Data managers need guidance on how to manage unrequested additional data provided on uestionnaires or as letters. It was felt that PRO Alert management should be isk‐related. qr PRO Data Collection/Management Guidelines and Training A range of training needs were identified by delegates including: the value of PRO data, communicating with patients regarding PROs, how to minimize and report missing data, guidance on how to deal with PRO Alerts/concerning data. elegates felt that this could be incorporated into GCP/site level training. Online raining may be useful. Dt PROs in the Patient Information Sheet Delegates felt that in general patient information sheets are too long, it was felt to be a good idea to use ‘light’ versions of PIS. Concerns were expressed around the expectation gap between the benefit that a patient perceives that they will get from being involved in a trial, the requirements on the researcher to ensure that patient info sheets state clearly that there is not expected to be a direct benefit and reality (which may lie somewhere in between). Groups felt that further work is needed to consistently improve the presentation of PISs (length, accessibility, other methods DVD, multimedia, apps). Groups queried whether patients understand why we assess PROs in trials? It is unclear what PRO‐specific information should be included in the patient information. Do patients want/need this information (either in the information sheet or in the consent interview)? Further research is needed in this area and more generally on optimal methods of appropriate patient information provision.
10
Session Four: Use of PRO trial results to inform clinical practice
Key questions: What are the key challenges for reporting PROs?
How can we facilitate improved reporting? Tools, education, other KT activities?
What are the barriers to the use of PRO trial results to inform clinical care and how can we address these?
Summary of Group discussions: Research into standardisation of PROMs/reporting. Group discussions focused on the need for endorsement and uptake of the CONSORT PRO Extension. Ensuring that journals and funders adopted the extension, and that online tools were developed to support trialists, was felt to be important. Delegates agreed that PROs should be reported in the main publication. Discussions highlighted the need for high quality trial design to facilitate the reporting of meaningful results. Clinicians were keen to see standardised use of measures to facilitate reporting and interpretation. Some concerns were expressed that PRO data takes too long to analyse and thus is not ncorporated into the main publication. This was not felt to be a legitimate oncern since programming can be done using dummy data in advance. ic Methods to facilitate clinician/patient understanding of PROs. Clinicians may need training on how to interpret and communicate PROs to patients. Staff not involved in research may find the data particularly difficult to interpret. Local training could be facilitated by the Birmingham Region Research and Training Collaborative and could provide training to: (a) researchers on how to bring PRO correctly into the trial design/protocol, and (b) to site staff, to ensure appropriate procedures are followed when using PROs (e.g. timing/location of questionnaires, using translators, using persons to read out questionnaires). It was generally recognised by the group that dissemination of PRO findings to patients is not done well – if at all. Web‐based tools/Podcasts or patients may be useful. Again, these discussions highlighted the need for ncreased patient input from study design to dissemination. fi Methods to facilitate shared decision making in relation to PROs. The group felt that greater understanding of PRO data could help facilitate shared decision making. It was recognized that the presentation of PRO results can be particularly challenging. Tools/software to support clinicians should be considered. Patient lay summaries, to accompany trial publications, may be eneficial. Patients should be consulted and could help identify the optimal ontent and layout. bc
11
S
ession Five: Research priorities – a clinical perspective
Key questions: Selection of instruments (e.g. for rare diseases/ systemic vs localized
effects/ generic‐e.g. EQ‐5D vs. disease specific)
The ability of a clinical community to have a preferred PRO tool in trials ina certain area and having a central repository for use across several trials.
a significant and s this.
Use of PROs for those who do not read or speak English ‐ important minority of our population ‐ and how to addres
Questionnaire fatigue (links to selection of instruments)
Summary of Group discussions: Selection of instrument Instrument selection should be hypothesis driven and valid for the population of interest. Group discussion focused on limiting the number of questionnaires used, to ensure consistent, interpretable results. Use of both disease specific and generic questionnaires was considered, although some concerns were expressed regarding patient burden. The balance between patient burden and useful PRO data should be considered. An item‐bank approach may be helpful and may llow comparison across different clinical settings. Data repositories for PRO ata warrant further consideration. ad Assessing PROs in Diverse Populations Group discussions focused on the need to have valid questionnaires appropriate for different cultures, which should be considered at the design stage and costed appropriately. Perceptions of symptoms, such as pain or depression, may differ across cultures. Any translation needs to be validated and interviewers need raining. The use of audio system/speech‐enabled systems for illiterate or blind articipants should be considered. tp Integration of PROs in the Institute for Translational medicine PROs were identified as a strategic theme which will be included in the ITM trategy.
s
Ensuring Best Practices for PRO Assessment in Trials The group welcomed the workshop as an opportunity to learn about the latest research from the PRO Research Group, other UoB academics and international collaborators. The workshop has provided a valuable platform to share best practice, build on existing collaborations and develop new collaboration, for example, PROs will be included as a theme within the Institute for Translational Medicine at Birmingham. The PRO Research Group will work to foster partnerships with key stakeholders including the HRA, research funders such as the NIHR and CTUs.
12
Research priorities he following research priorities were identified by anonymous voting on day 2 f t w p : To
he orkshop (Ap endix 5)
1. Determining optimal methods of patient involvement in outcome selection.
2. Establishing the optimal methods for incorporating PROs in Core Outcome Sets.
3. Development of a checklist for protocol writers, detailing recommended n PRO‐specific items for inclusion i trial protocols, with supporting online
tools. ent of ‘PRO 4. The development of consensus guidance on the managem
Alerts’ within trials.
5. Development of PRO data collection/management guidelines. tating clinician and 6. Development of methods and tools aimed at facili
omains. patient understanding of PROs.
7. Research into standardisation of PRO tools/d8. Development of a UK‐based PRO item bank.
Next Steps The UoB PRO Research Group in collaboration with partner organisations are committed to addressing these priorities and to providing training and education to facilitate best practice for PROs in clinical trials. The latest details of the PRO esearch Group activities, outputs, grants, publications, our international ollaborators and project partners can be found on our website: Rc http://www.birmingham.ac.uk/patient-reported-outcomes
Appendix 1: Attendees:
Attendee List: Institute of Advanced Studies Workshop - 2nd/3rd July 2012
Best-Practice for Patient Reported Outcomes (PROs) in Randomised Clinical Trials
Title First Name
Surname
Role & Organisation Email
Dr Hareth Al-Janabi Birmingham Fellow, School of Health & Population Sciences
09:20 Meeting commencement Chair: Melanie Calvert Welcome from Pro-vice-chancellor Professor Malcolm Press, director of the IAS
09:30 Aims of the workshop and introductions Melanie Calvert
10:00 Session 1: PRO Selection • What patients/carers want from PROs [LR;MvH] • Evidence-based PRO selection (Core Outcome Sets) [JB] • Where next? Key points for discussion [JB] • Small group discussion [JB; MC] • Feedback/whole group discussion
Jane Blazeby Lesley Roberts Maria von Hildebrand
11:45 COFFEE BREAK
12:00 Session 2: PRO Trial Design • A trialists’ perspective on the quality of PRO protocol components [MK] • PRO Trial Protocol Design [MC/MB]
Melanie Calvert Madeleine King Michael Brundage Jane Blazeby
12:30-13:30 LUNCH
13:30 Session 2: PRO Trial Design cont… • Where next? Key points for discussion [MB] • Small group discussion [MC; MB; JB] • Feedback/whole group discussion
Melanie Calvert Michael Brundage Jane Blazeby
14:45 COFFEE BREAK
15:00 Session 3: PRO Trial Conduct • Challenges in assessing PROs in trials [DK] • Ethical Considerations [HD] • Where next? Key points for discussion [MC] • Small group discussion [DK; HD; MC] • Feedback/whole group discussion
Derek Kyte & Heather Draper Melanie Calvert
16:45 Round-up of the day Melanie Calvert
17:00 Drinks reception
18:00 MEETING CLOSE
Day 2 - Wednesday 3rd July 2013
Start Item Lead discussant
08:30 Meeting commencement Chair: Melanie Calvert
08:30 Priorities from Day 1/ Aims for day 2 Melanie Calvert
09:30 Session 4: Use of PRO trial results to inform clinical practice • CONSORT PRO extension [MB] • Using PRO trial results in practice [MB] • Where next? Key points for discussion [JB]
Michael Brundage Jane Blazeby Melanie Calvert
10:30 COFFEE BREAK
10.50 Session 4: Use of PRO trial results to inform clinical practice, cont… • Small group discussion [MB; JB; MC] • Feedback/whole group discussion
Michael Brundage Jane Blazeby Melanie Calvert
12:00-13:00 LUNCH
13:00 Session 5: Research priorities: a clinical perspective The Institute of Translational Medicine [PC] Priority areas for PRO research- clinicians perspective [PC; PK; CD; AD; HP] •Where next? Key points for discussion [PC] • Small group discussion [PC; PK; AD] • Feedback/whole group discussion
Paul Cockwell Paulus Kirchhof Clara Day Alastair Denniston Hardev Pall
14:30 COFFEE BREAK
14:50 Session 6: Moving PRO research forwards • Priorities from day 2 • Summary of priorities arising from the workshop • Where next? whole group discussion
• Maria has worked in patient and public involvement as parent, carer, campaigner senior manager in DH, and now working freelance in health advocacy.
• PPI Work has covered acute, chronic, primary, secondary, tertiary, health, social care, and research.
• Most recent publication: Co‐authored report with Simon Denegri on Patient and Public Survey Response to UK Clinical Trials Gateway http://www.nihr.ac.uk/files/Publications/UKCTG%20Report_Jan%202013.pdf
•In 2009 Iain Chalmers and Paul Glaziou estimated that 85% of research is wasteful or inefficient with deficiencies in four main areas:
Is the research question relevant?
Are the design methods appropriate?
Is the full report accessible?
Is it unbiased and clinically meaningful?Source: The Lancet Volume 381, Issue 9864, Page 347 2nd Feb 2013
The public and patients assume all of this to be true, because they trust in the experts.
Research is being politically exploited as an industry for economic growth due to availability of patient data in the NHS, making the above areas even more important to avoid exploiting goodwill of patients and public especially those who are vulnerable.
• At which point in the patient journey, (not clinical pathway)and stage of illness, is a patient presented with opportunity to take part in research that is relevant to them?
• ‘Only 28% of patients surveyed had accessed a clinical trial’ Source UKCTG Patient and Public Survey Report S Denegri, M von Hildebrand Jan 2013.
• Having been granted participation in a trial at what point do patients become aware of PRO questionnaire, its purpose, number of times it needs to be filled out, and its relevance to them as an individual?
• What difference do they think it will make to their quality of life and does this match with the communication they receive when consenting to take part?
• If parents/carers are to be included in study, will they be made aware of the importance of their participation in the study?
• Clarity of purpose about RCT and relevance to individual taking part.
• Knowledge that the PRO will be a useful and reliable tool to individual and future patients.
• Understanding that PRO will measure their quality of life, at the beginning, during and end of trial.
• Ability to fill in PRO and complete it at every required stage. and importance of this requirement.
• Ease of functionality, length, language, jargon, no duplication!
• Pledge that having taken part in the RCT they will be able to read published findings detailing the outcome of research in regards to clinical practice and patient benefit.
Jane M Blazeby Professor of Surgery, Honorary Consultant Upper Gastrointestinal Surgeon, Director MRC Trials Methodology Hub, Director Surgical Trials Centre, University of Bristol & University Hospitals Bristol NHS Foundation Trust
Evidence based PRO selection for core outcome sets
Overview
• Problems with patient reported outcomes and core outcome sets as a solution
• How to select PROs in core outcome sets• Future work needed
Patient reported outcomes
• Many PROMs, with scales & items assessing indistinct domains
• Impossible to synthesise, compare like with like
• Enormous risk for outcome reporting bias
HRQL Questionnaire Number of studies (n=26)
EORTC QLQ-C30/CR38 17
SF-36 4
FACT-C 1
EQ-5D 1
HADS 1
STAI 1
mCOP-QoL-Ostomy 1
FIQL 1
Spitzer’s QLI 1
CRC-QoL 1
Different PRO measures Outcome reporting bias
20
Quality of life > 20 scales/items
• EORTC QLQ-C30 and QLQ-OES18• Dermatology QoL Index
Solutions
• Core PRO sets, • Could also be used as core disclosure
sets for clinicians to discuss with patients
Advantages of core outcome sets
• Increases consistency across trials
• Maximise potential for trial to contribute to systematic reviews of these key outcomes
• Much more likely to measure appropriate outcomes
• Major reduction in selective reporting 11
Core Outcome Measures in Effectiveness Trials
http://www.comet-initiative.org/home/
@COMETinitiative
21
“Core outcome set: agreed set of outcomes
reported as a minimum in all clinical trials of a
specific clinical condition.”
What does this mean for PROs?
Core outcome set Identify core domains and then…
Creating order out of chaos
• Identify and deconstruct all PROMs (valid and not valid)
• Re-categorise into domains (experts)• Use Delphi methods to prioritise domains
Overview
• Problems with patient reported outcomes and core outcome sets as a solution
• How to select PROs in core outcome sets• Future work needed
Once selected what to measure, think about how
• ‘What’ to measure
• ‘How’ to measure (validity, reliability, feasibility)
- PROMIS, http://proqolid.org/, COSMIN, TREAT-NMD
www.comet-initiative.org
Twitter: @COMETinitiative
22
Session 2: Objectives, hypotheses, questionnaires and endpoints
QOL Protocol Checklist Workshop
Session 2: PRO Trial Design
PRO Research Group
Session 2 – PRO Trial DesignA trialists’ perspective on the
quality of PRO protocol componentsProf Madeleine King
& Ms Rebecca Merciecer‐Bebber
2 day Workshop: Best‐Practice for PROs in RCTs2‐3 July 2013 – Birmingham, UK
QOL Office, University of Sydney•Established in 2005•Funded by Cancer Australia•Provide QOL/PRO advice and services to the network of 13 Australasian Cancer Clinical Trials Group (CTGs)
Overview
• What we have learned
– things that should be included in protocols
– PROtocol Checklist
• Results of our pilot study
– Things that are not currently included in protocols
Importance of the protocol
• Procedures for good conduct of the trial– success or failure of a trial may depend on how well the protocol was designed and written
– all relevant requirements so the study can be implemented uniformly by all sites and staff
– detailed and clearly worded
• These general points apply just as much to PRO assessment as to any other aspect of a trial
SPIRIT 2013 Chan et al.
Rationale for PROs• Why measure PROs in the trial?
– What’s known / not known, why it matters
• Which PROs variables are considered relevant and why?– Acute v late effects
– Benefits v harms
– Primary v secondary PRO endpoints
– Important to anticipate those issues/domains that are most likely to be affected by the intervention, differentially between trial arms
• Proximal to disease/treatment – symptoms/side‐effects
• Distal – may include core HRQOL domains (physical, emotional, role, social functioning) and/or global assessment of HRQOL
23
Session 2: Objectives, hypotheses, questionnaires and endpoints
QOL Protocol Checklist Workshop
Choosing the most appropriate PROM(s)
• Decide on which PROs matter first, then choose the best questionnaires to measure them– ‘don’t put the cart before the horse’– ‘the right tool for the job’– FDA: ‘fit for purpose’
• Protocol should include:– Justification of choice ‐ which PROM(s) and why– Citations for validity, reliability, responsiveness,
interpretability– Which specific symptoms, side‐effects, HR‐QoL domains
covered– Number of items (questions)– Approximate time to complete (rule of thumb: 10 sec/item)
• copies of questionnaires in appendix
When, where and how PRO will be assessed
• timing of PRO assessments– acute and late effects– acceptable time windows
• mode of PRO data collection– in person; in clinic or at home; by telephone, mail or
online
• Also include this info in informed consent procedure
Statistical considerations• Scoring
– how questionnaire items are scored into summary scales
• direction of scale, e.g. higher score = more symptoms
• Minimal important difference (MID)
– Clinical importance vs statistical significance
• Responder definition (optional)
– Palliative context
– Improvement of >=MCID for specified period
Statistical considerations (cont.)
• PRO assessment schedule will result in numerous PRO variables– Early/late symptoms/side‐effects/function
• Which are the primary/secondary PRO endpoints/timepoints– For primary endpoints, corresponding hypotheses– Size and direction of expected effects
• Sample size/power– if a PRO is the primary endpoint, then required sample size
– if PROs are secondary, then the power of the trial sample size in relation to the PRO hypothesis/es
Missing data
• Prevention is better than cure– Emphasise the importance of compliance, methods for enhancing it
– Also cover in site manuals for trial co‐ordinators, research nurses, etc
• Some missing data cannot be avoided– How missing data are to be dealt with in analysis– How relevant ancillary data will be collected (e.g. ECOG‐PS, Karnovsky, Spitzer clinician rated QOL) and incorporated into analyses (e.g. used in mulitple imputation)
• Key issues addressed in protocol, detailed statistical analysis plan developed later
QOL Office
PROtocol Checklist
available @ QOL Office Website
http://www.pocog.org.au
+ PROtocol Development Workshops
24
Session 2: Objectives, hypotheses, questionnaires and endpoints
QOL Protocol Checklist Workshop
Is such a checklist needed? Pilot Study
Aim: Assess “PRO‐completeness” of Aust CTG protocols for Phase III RCT
Methods:
•13 protocols (convenience sample)
•37 checklist items, one point per item included
Results:
•Mean (SD) “completeness” score = 11.8 (4.2)
•Min = 3, max = 28 (75%)
Is such a checklist needed? Yes!Selected results from the 13 protocols assessed
Only 3 Described methods for handling missing data.
Only 2 Described the constructs used to evaluate the intervention (most just referred vaguely to “HRQOL” or “QOL”Specified acceptable time windows for each assessment.
State the Inclusion and exclusion criteria for PRO endpoint(s) and analyses.
Only 1 Stated a PRO-specific hypothesis.
Specified the timeframe of interest and rationale for this.
Included a standardised form for recording reasons for missed PRO assessments..Stated the sample size and power requirements.
None Named the PRO sub-study investigator/coordinator.
Specified how PRO was to be assessed – pencil & paper, online, etc.
Stated and justified minimal important difference (interpretation).
Next steps• Development of a definitive PRO protocol checklist
– ISOQOL Task Force: Best practices for PROs in Randomized Clinical Trials
– Extend our pilot to a large sample of existing protocols, link to quality of PRO reporting (CONSORT‐PRO guidelines)
– Evaluate the effectiveness of the PROtocol Checklist + training workshops for increasing the completeness of PRO components of Aust CTG protocols (long‐term, prospective project)
Thank you from Sydney
Session 2: PRO Trial Design
PRO Research Group
Research update: Evaluation of patient reported outcomes in clinical trials: systematic review of trial protocols
University of Birmingham UK
Investigators: Calvert M (PI), Kyte D, Draper H, Ives J, Gheorghe A, Duffy H, Brundage M, King M, Mercieca-Bebber R, Blazeby J.
25
Session 2: Objectives, hypotheses, questionnaires and endpoints
QOL Protocol Checklist Workshop
What’s the problem?Hypothesis: Information on PRO hypotheses, data collection and analysis in trial protocols is suboptimal.
Suboptimal trial design:• Unethical
• Inefficient use of funding
• Lack of guidance for trial staff‒ Potential ethical tension‒ Non-standardised (inappropriate?) practices
(Session 3)
• Poor reporting? (Session 4)
• Poor quality evidence to inform patient care.
Evaluation of patient reported outcomes in clinical trials
Eur J Cancer. 1997 Jan;33(1):20-8.Quality of life assessment in clinical trials--guidelines and a checklist for protocol writers: the U.K. Medical Research Council experience. MRC Cancer Trials Office.Fayers PM, Hopwood P, Harvey A, Girling DJ, Machin D, Stephens R.
Madeleine King, PoCoG
Review current guidance on PRO assessment in trials.
Establish the current quality of trial protocols in relation to the PRO information they provide.
NETSCC HTA Protocol 02-10-02The primary outcome measure will be complete ulcer healing at 12 weeks. Other secondary outcome measures that will be included in the analysis are healing at six months and one year, recurrence at six months and one year, EQ-5D and SF-36 health related quality of life questionnaires, and the McGill pain questionnaire.
NETSCC HTA Protocol 01-74-03Questionnaires on pain, satisfaction and HRQOLIn the absence of a widely used disease specific measure, we will analyse the responses to four separate measures, all of which are completed in private and returned by post: (Rand) SF36, EuroQol EQ5D, CWIS and a 10cm visual analogue scale for pain. [A satisfaction questionnaire will be used at the end of the study]. These are all assessment tools that have been used without problem in this patient population in previous studies.
www.hta.ac.uk/project/
Next Steps:
International Consensus on Guidance for PRO protocol writers
Taskforce led by Calvert, King, Brundage, Blazeby, Kyte
PRO Research Group
26
Session 2: Objectives, hypotheses, questionnaires and endpoints
QOL Protocol Checklist Workshop
Session 2: PRO Trial Design
Topics for group discussion:
1.What PRO-related components should be included in the trial protocol or other trial documentation such as SOPs?
2.What are the key areas that need further work to inform optimal PRO trial design?
3.How do we ensure that trials adopt best-practice in PRO trial protocol configuration? Ideas to address this? Tools? Education?
27
Session 3: PRO Trial Conduct !!
PRO Research Group
Challenges in Administering PROs in trialsDerek Kyte - PRO Research Group, University of Birmingham UK
Doctoral Research supervised by: Calvert M, Draper H, Ives J Collaborators: C Liles, Gheorghe A, Keeley T
School for Primary Care Research
Funded by the National Institute for Health Research - School for Primary Care Research
Acknowledgements
BACKGROUND• Key researcher concerns:
Inconsistencies
Uncertainty
Lack of Guidance
Additional Info
‘Concerning’ Data
METHODS
- One primary care NHS trust
- Two secondary care NHS trusts
- Two clinical trials units
• 26 semi-structured interviews
RESULTS
“If they go into our control group… they’re not gonna have any personal contact, then [the questionnaire] will go back out in the post with just a
letter, sort of, saying, ‘Oh, you accidentally missed out this one,’ and... hopefully they’ll return it back… if they’re put into an exercise, erm,
intervention, then they do get seen by [a research facilitator]. I’ll ask her to take the questionnaire with her… she’ll help them to fill it in.” [participant 25]
Inconsistency: inconsistent standards in administration of QOL measurement, both between, and within, trials, which appeared to risk the introduction of bias.
Theme 1
RESULTS
PRO Alerts
Theme 2Dealing with ‘concerning’ data: staff reported uncertainty when encountering participant self-reports of problems that potentially required intervention.
“worrying levels of psychological distress or physical symptoms that may require an immediate response.”
28
RESULTS
“A couple of times where measurable data… that the doctor had done, showed yeah, maybe a bit of a slight wobble but everything is okay. Whereas... according to their quality of life questionnaire, they will sort of say... it's all awful and it's all really getting worse… that was noted and the patient was called back in early.” [participant 17]
“One consultant called me back in because the patient had left [the HRQL questionnaire] with him… and he’d looked at it and was concerned at what he was
seeing, and saying, ‘That’s not the patient that presented to me. We need to call them back in and we need to... talk to them again’.” [participant 15]
Theme 2Dealing with ‘concerning’ data: staff reported uncertainty when encountering participant self-reports of problems that potentially required intervention.
RESULTS
Co-intervention Bias
Theme 2Dealing with ‘concerning’ data: staff reported uncertainty when encountering participant self-reports of problems that potentially required intervention.
bias caused by ‘any intervention other than the experimental manoeuvre that alters the frequency of a
trial’s outcome of interest’ (Sackett, 2011)
RESULTS
“You might feel a bit torn between… just being somebody who’s making sure all the data’s complete… you’ve got your complete datasets… and you’re also paying
attention that somebody that needs help... is gonna get some support.” [participant 25]
“I have been in that position and I have been told that the study comes first… at the end of the day… that’s not how it works… I am [a] registered nurse… I have
to act upon that as well.” [participant 8]
Theme 3 Burden: data collection could be associated with emotional and/or ethical burden for some trial staff.
RESULTS
“I don’t think there's an overall clarity about… using quality of life measures... [if the patient writes] additional information [on their questionnaire], how does that get recorded, how does that get fed back to the team? What happens… if you are concerned about somebody? What's the process? What level should we get
involved?... these sorts of issues aren't really covered anywhere.” [participant 20]
Theme 4 Lack of training: frequent lack of QOL-specific guidelines and training for trial staff.
RESULTS
“We could have done with the training on the relevance of quality of life much, much earlier on... I think that would have helped explain the role of quality of life to
patients as well, erm, because the feedback you get is… ‘I can’t understand why they’re asking me this,’… so, I think it’s very important that you educate the
interviewer so that they can explain.” [participant 4]
Theme 4 Lack of training: frequent lack of QOL-specific guidelines and training for trial staff.
• Researchers have ethical obligations to research participants• Consent, confidentiality, balance of harms & benefits
etc.• Generally assumed that participant should not be
foreseeably disadvantaged as result of participation• E.g. trials against SAT rather than placebo
• If harms foreseeable, steps taken to minimise• E.g. monitoring
• Treating clinician responsible for monitoring & treating
• Participant expectations - assume have reported
• Researchers with dual roles have to put patients first
Solution 1:Do nothing
Solution 2:Clear PIS
with disclaimer/
helpline
Solution 3:PRO alerts
and proactive response
Data lost to study
Data
collected
PRO research merely records, not intervention
• If so, then needs similar monitoring to somatic interventions
• With alerts in place if distress results
Completing a PRO questionnaire is an intervention?
30
Session 3: PRO Trial Conduct !!
Topics for group discussion:
1. Is experience of PRO alerts widespread? How best should they be managed? Is there existing guidance we can draw upon?
2. What information for data collection staff should be included in protocol guidance/trial training and why?
3. How do we encourage uptake of new guidance?
4. Do patients understand why we assess PROs in trials? What PRO-specific information should be included in the patient information documentation? Are/should PROs be discussed in consent interview?
31
Cancer Care and Epidemiology
Patterns of Reporting PRO Results from Randomized
Clinical Trials: Implications, Impact, and
Challenges
MD Brundage and ColleaguesMarch 2013
Objectives
• Current patterns of reporting PROs in RCTs of biomedical interventions
• Illustrate knowledge translation issues from a clinical perspective
• To review consensus on a CONSORT extension guidance for reporting PROs in clinical trial with implications for further research
Objectives
• Current patterns of reporting PROs in RCTs of biomedical interventions
• Illustrate knowledge translation issues from a clinical perspective
Objectives
• Current patterns of reporting PROs in RCTs of biomedical interventions
• Illustrate knowledge translation issues from a clinical perspective
• To review consensus on a CONSORT extension guidance for reporting PROs in clinical trial with implications for further research
HQL Measurement and Epidemiology
Clinical Trials
GuidelineDevelopment
Population StudiesProgram EvaluationPolicy Formulation
Patientand
Practitioner
The Adoption of Health-Related Quality of Life Innovations
32
HQL Measurement and Epidemiology
Clinical Trials
GuidelineDevelopment
Population StudiesProgram EvaluationPolicy Formulation
Patientand
Practitioner
The Adoption of Health-Related Quality of Life Innovations
HQL Measurement and Epidemiology
Clinical Trials
GuidelineDevelopment
Population StudiesProgram EvaluationPolicy Formulation
Patientand
Practitioner
The Adoption of Health-Related Quality of Life Innovations
Back to our Research Question
I’m interested in comparing two treatments:“R” and “M”
I want to evaluate how each treatment affects patients’ quality of life.
NCIC CTG PR.3/MRC PR07/SWOG JPR3:Study Scheme
Initial PSA Level: < 20 vs 20‐50 vs > 50 μg/L Hormonal Therapy: orchiectomy vs LHRH analogue+ anti androgen Method of lymph node staging: clinical vs radiological vs surgical Gleason Score: < 8 vs 8‐10 Prior hormonal therapy: yes vs no Centre
Continuous Androgen Deprivation Therapy
+ Radiotherapy
Continuous Androgen Deprivation Therapy
T3/T4 N0/NXor
T2 and PSA > 40 μg/Lor
T2 and PSA > 20 μg/L and GS: 8‐10
NCIC CTG PR.3/MRC PR07/SWOG JPR3:Study Scheme
Initial PSA Level: < 20 vs 20‐50 vs > 50 μg/L Hormonal Therapy: orchiectomy vs LHRH analogue+ anti androgen Method of lymph node staging: clinical vs radiological vs surgical Gleason Score: < 8 vs 8‐10 Prior hormonal therapy: yes vs no Centre
Continuous Androgen Deprivation Therapy
+ Radiotherapy
Continuous Androgen Deprivation Therapy
Locally Advanced Prostate Cancer
Planned Treatment
Androgen Deprivation Therapy
• Bilateral Orchiectomy
or
• LHRH agonist
– Antiandrogen for 2 weeks, optional to continue
Radiotherapy
• 45 Gy/25 F/5 weeks to pelvis
• 20‐24 Gy/10‐12 F/2‐2.5 weeks to prostate
• If treating physician felt patient inappropriate for whole pelvis then RT given to prostate only
33
Locally Advanced Prostate Cancer1990s
Canadian and UK surveys of clinicians revealed substantial uncertainty about the role of radiotherapy“These men all have metastatic disease; adding radiotherapy to hormones is unnecessary and unkind”
Baseline Characteristics
Characteristic ADT Alone ADT+RT
Median Age 69.7 years 69.7 years
T Category
< T2c
T3/T4
11%
89%
10%
88%
Gleason Score
< 7
8‐10
81%
18%
81%
18%
PSA ng/ml
<20
20‐50
>50
37%
38%
25%
36%
38%
26%
15
Final Analysis ‐ overall survival
HR = 0.70 (95% C.I. 0.57 to 0.85, P = 0.0003)
10 yr OS 55%
10 yr OS 49%
+RT
Per
cent
age
0.0
0.2
0.4
0.6
0.8
1.0
Time (Years) # At Risk(ADT)
# At Risk(ADT + RT)
0602603
2571558
4498505
6353381
8185208
107785
122832
ADT ADT + RT
16
Final Analysis: Cumulative Incidence Probability for Disease‐Specific Survival
DSS HR=0.46 (95% CI 0.34-0.61)
Estim
ated
Cum
ulat
ive
Inci
denc
e
0
20
40
60
80
100
Time (Years)0 2 4 6 8 10 12
Death related to disease ADT ADT + RT
0
10
20
30
40
50
60
70
80
90
100
0 6 12 18 24 30 36
17
Quality of Life:Bowel Domain (EORTC QLQ)
Me
an
Sym
pto
m S
core
s
Bowel and Rectum
ADT onlyADT + Radiation
Months
Few
er S
ym
pto
ms
18
Quality of Life:Bowel Domain (EORTC QLQ)
Me
an
Sym
pto
m S
core
s
0
10
20
30
40
0 6 12 18 24 30 36
0
10
20
30
0 6 12 18 24 30 36
Bowel and Rectum
ADT onlyADT + Radiation
Months
Few
er S
ym
pto
ms
Diarrhea
ADT onlyADT + Radiation
34
Proportion of Patients Worsening
• Patients deteriorating by 10 points or more at any point up to 3 years
0
10
20
30
40
50
60
70
80
90
100
ADT only ADT + XRT
Not worsened
Worsened
Rectal Symptoms
P < 0.01
0
10
20
30
40
50
0 6 12 18 24 30 36
20
Quality of Life:Urinary Domain (FACT‐P)
Me
an
Sym
pto
m S
core
s
ADT onlyADT + Radiation
Months
Few
er S
ym
pto
ms
Proportion of Patients changing
• Patients changing by 10 points or more at any point up to 3 years
0
10
20
30
40
50
60
70
80
90
100
ADT only ADT + XRT
Improved
Neither
Worsened
Urinary Symptoms
P > 0.05 0
10
20
30
40
50
60
70
80
90
100
Baseline 1 2 3 4 5
Time from Baseline in Years
PH
YS
Sc
ore
22
Quality of Life:Physical Domain (EORTC)
ADT onlyADT + Radiation
35
Objectives
1. Has the quality of RCT PRO reporting improved?
2. In what ways do RCTs report PRO findings?
Screening Articles for Review and Abstraction
ONCOLOGY papers’02 - ‘08
N=998
Non-ONCOLOGY papers’02 - ‘08N=4866
Total794
Random sample prior to screening
413381
Random sample prior to screening
Eligibility:
- Completed Phase III RCT of intervention with HRQL endpoint - Full publication was available in English- Primary report or expanded secondary analysis - Not an aggregate / meta-analysis / review
• Oncologists’ attitudes, knowledge, understanding and skills relating to clinical trial HRQL data
•A recurring theme was that HRQL data “help patients, and empower patients to make decisions based on their values” and can “level the playing field between physician and patient”.
• Oncologists’ attitudes, knowledge, understanding and skills relating to clinical trial HRQL data
• Many participants were equivocal about validity of HRQL measurement
“Many trial reports do not give enough detail. I do not know enough about the scales and it is hard to gauge sketchy, patchy information”.
0%
20%
40%
60%
80%
100%
Strongly Agree Agree Neither Agreenor Disagree
Disagree StronglyDisagree
I have a good understanding of the concept of quality of life
0%
20%
40%
60%
80%
100%
Strongly Agree Agree Neither Agreenor Disagree
Disagree StronglyDisagree
The measurement of quality of life in clinical trials isgenerally valid
0%
20%
40%
60%
80%
100%
Strongly Agree Agree Neither Agreenor Disagree
Disagree StronglyDisagree
I feel comfortable interpreting quality of life data as it isreported in the clinical trial literature
0%
20%
40%
60%
80%
100%
Strongly Agree Agree Neither Agreenor Disagree
Disagree StronglyDisagree
I feel a need to improve or increase my use of clinical trialquality of life data in my clinical practice
39
First Order Barriers: (Intrinsic to the Individual)
• Lack of knowledge of the HRQL measurement tools
•Lack of confidence in ability to interpret the published data
•Skepticism regarding current measurement techniques
Second Order Barriers: (Extrinsic to the Individual)
Mostly related to trial results:
• Questionable quality of the data
• Lack of standardization in presentation of findings
• Validity of clinically meaningful change in HRQL
Second Order Barriers: (Extrinsic to the Individual)
Mostly related to trial context:
• Lack of data that is relevant to specific populations
• Non-generalizability of the data
• Lack of time to access secondary publications
HQL Measurement and Epidemiology
Clinical Trials
GuidelineDevelopment
Population StudiesProgram EvaluationPolicy Formulation
Patientand
Practitioner
The Adoption of Health-Related Quality of Life Innovations
Enablers
*Clinician understanding of the potential value of HRQL data
*Clinician willingness to use the data if available and if relevant to their patient populations
*Clinician willingness to learn more
40
Some Background onCONSORT
(Consolidated Standards for Reporting Trials)http://www.consort-statement.org
The CONSORT group:
• Proven ability to reach consensus on reporting guidelines• Includes editors of key journals • Includes clinical trialists and other stakeholders• Promotes successful implementation of guidelines / “buy-in”
A core group of reporting guidelines has been developed and should be used to report any RCT.
More on CONSORT Guidelines……
A core group of reporting guidelines has been developed and should be used to report any RCT.
More on CONSORT Guidelines……
A core group of reporting guidelines has been developed and should be used to report any RCT.
Other more specific guidelines are contained in what are termed ‘CONSORT Extensions’.
More on CONSORT Guidelines……
Why Use this Approach?
1. To reach the right number of right people quickly.
A CONSORT extension would provide guidance for editors, reviewers, and authors on meeting the appropriate standards for HRQL reporting
2. It works.
Existing CONSORT guidelines have been widely adopted. The website has had over 100,000 hits in the 6 months following the latest CONSORT 2010 statement in March.
3. It is an inherent component of our KT strategy.
Our project plans to stream a number of its KT activities through the CONSORT / EQUATOR Network.
Recommended Process for Developing a Health Research Reporting Guideline
• Identify the need for a guideline √• Review the literature √• Identify previous relevant guidance √• Seek relevant evidence on the quality of
reporting in published research articles å Identify key information related to the
potential sources of bias in such studies √
Pre-meeting activities
• Obtain funding for the guideline initiative √(MRC-funding obtained, supplementary CIHR-funding requested to allow 30 international participants at the face-to-face meeting)
• Identify participants √( Core team of ISOQOL members form the planning committee. Broad stakeholder group to be invited to face-to-face meeting)
Pre-meeting activities currently in process:
• Conduct a Delphi exercise• Generate a list of items for consideration at the face-
to-face meeting • Prepare for the face-to-face meeting
Acknowledgements
We thank our funders and collaborators:
Funders: Medical Research Council, Canadian Institutes for Health Research
Collaborators:• International Society for Quality of Life Research
• CONSORT Executive & EQUATOR Network
• MRC Midland and ConDuCT Hubs for Trials Methodology Research
• University of Birmingham, UK, Queens University, Canada, University of Bristol, UK, University of Ottawa, Canada
• Survey participants including: ISPOR, MRC HTMR, NIHR Research design service, SCT, European Clinical Trials Units, Journal Editors, Policy Makers
• Meeting participants
Modified Delphi Approach
What does the literature say? Survey
Items include:
• Study population
• Instrument characteristics
• Study hypotheses
• Data collection procedures
• Statistical analyses
• Study results
• Quality and monitoring
• Discussion & interpretation
• Other
42
Survey of Key StakeholdersSurvey Response
Stakeholder n
International Society for Quality of Life Research 161
International Society for Pharmacoeconomics and Outcomes Research
82
Society for Clinical Trials 63
European Clinical Trials Units 58(UK (n=43); Italy (n=2); Switzerland(n=12); Denmark (n=1)
Research DesignService
28
Medical Research Council (MRC) Hubs for TrialsMethodology Research
18
Cochrane Patient‐Reported OutcomesMethods Group
8
Policy makers 4
Development of Proposed Reporting standards
• Survey results were debated by the ISOQOL Reporting Guidelines Task Force
• Feedback was obtained at the ISOQOL Annual Conference, Denver, October 2011 and used to inform proposed standards for this meeting.
Development of Proposed Reporting standards
An example: 'HRQL should be identified as an outcome in the abstract"
0
20
40
60
80
100
"Essential"
"Desirable"
"Optional"
"Rarely Neces."
When primary outcome
0
20
40
60
80
100
"Essential"
"Desirable"
"Optional"
"Rarely Neces."
When secondary outcome
Development of Proposed Reporting standards
An example: "HRQL should be identified as an outcome in the abstract"
3. The PRO hypothesis should be stated and should specify the relevant PRO domain(s) if applicable.
1 2 3 4
56%
0%4%
41%1. Include ‐ all trials with PRO
2. Include ‐ when PRO is primary outcome
3. Unsure
4. Exclude
2b. Specific objectives or hypotheses
1 2 3
96%
0%4%
1. Agree
2. Disagree
3. Further discussion
Propose: The PRO hypothesis should be stated and should specify the relevant PRO domain(s) if applicable.
Example
Potential survival benefit needs to be weighed against the burden of treatment. For this reason, HRQOL…was included as a secondary end point in the EORTC 18991 study …
The protocol hypothesized that there would be a difference in global HRQOL scale between both arms, showing worse HRQOL in the PEG‐IFN‐α‐2b arm. The remaining HRQOL variables were then examined on an exploratory basis.’
CONSORT Reporting Standard Category:
TITLE & ABSTRACT
Proposed Standard:
1. The PRO should be identified as an outcome in the abstract
1. The PRO should be identified as an outcome in the abstract
23. The proportion of patients achieving pre‐defined responder definitions should be provided where relevant.
1 2 3 4
44%
8%
16%
32%1. Include ‐ all trials with PRO
2. Include ‐ when PRO is primary outcome
3. Unsure
4. Exclude
CONSORT Reporting Standard Category:
TITLE & ABSTRACT
2010 Guidance
1a. Identification as a randomised trial in the title.
1b. Structured summary of trial design, methods, results & conclusions.
49
7a. How sample size was determined
1 2 3
96%
0%4%
1. Agree
2. Disagree
3. Further discussion
Propose: No extension
22. Interpretation consistent with results, balancing benefits and harms and considering other relevant evidence
1 2 3
69%
8%
23%
1. Agree
2. Disagree
3. Further discussion
Propose: The clinical significance of the PRO findings should be discussed.
HQL Measurement and Epidemiology
Clinical Trials
GuidelineDevelopment
Population StudiesProgram EvaluationPolicy Formulation
Patientand
Practitioner
The Adoption of Health-Related Quality of Life Innovations
CONSORT
HQL Measurement and Epidemiology
Clinical Trials
GuidelineDevelopment
Population StudiesProgram EvaluationPolicy Formulation
Patientand
Practitioner
The Adoption of Health-Related Quality of Life Innovations
CONSORT
COSMINSPIRIT
0%
20%
40%
60%
80%
100%
Strongly Agree Agree Neither Agreenor Disagree
Disagree StronglyDisagree
I feel comfortable interpreting quality of life data as it isreported in the clinical trial literature
Purpose & Objective
Attitudes
Perceived knowledge
Experience
Perceived barriers
50
We asked oncologists if they……
• find patient‐reported outcomes useful
• have positive (or negative) attitudes towards them
• use them in their clinical practice
• face barriers that prevent from using them
Methods
• Study participants
• Inclusion criteria
1) Be a practicing oncologist/ hematologist
2) Be a member of the NCIC Clinical Trials Group
3) Be involved in disease site/ scientific committee
Results
• Demographics
• Males (62%) Females (38%)
• Between 1‐20 years of medical practice (64%)
Results
• Clinical role and research exposure
• 85% spend more than half their time in clinics
• 74% work in both curative and palliative settings
• 96% have previously participated in an RCT‐ 87% have had exposure to patient‐reported outcomes in clinical trials
Results
• Perspectives on patient‐reported outcomes
• 66% have a good or very good understanding of the concept of patient‐reported outcomes
• 68% find patient‐reported outcomes useful or very useful
Results
Respondents (%)
51
Results
• Barriers to using patient‐reported outcomes in their clinical practice
#1 Concerns about the generalizability of results#2 Time
#3 Lack of understanding of results
Upcoming results
Australasian Cancer Clinical Trials Network
(Australia and New Zealand)
National Institute for Health Research Cancer Research
Network(United Kingdom)
NCIC Clinical Trials Group(Canada)
Proportion of Patients changing
• Patients changing by 10 points or more at any point up to 3 years
0
10
20
30
40
50
60
70
80
90
100
ADT only ADT + XRT
Improved
Neither
Worsened
Urinary Symptoms
P > 0.05
HQL Measurement and Epidemiology
Clinical Trials
GuidelineDevelopment
Population StudiesProgram EvaluationPolicy Formulation
Patientand
Practitioner
The Adoption of Health-Related Quality of Life Innovations
Overall Quality of Life
0
20
40
60
80
100
0 12 24
Time (Months)
Glo
bal Q
OL
Treatment "A"
Treatment "B"
Bet
ter
QO
L Overall QOL: Response at Six Months
0
20
40
60
80
100
Treatment "A" Treatment "B"
Pro
port
ion
of P
atie
nts Improved
Deteriorated
0
1020
30
4050
60
7080
90
Prefers LineGraph
Prefers BarGraph
Finds BothUseful
Adjuvant Setting
Per
cen
t o
f P
arti
cip
ants
0102030405060708090
100
Prefers LineGraph
Prefers BarGraph
Finds BothUseful
Adjuvant Setting
Palliative Setting
Per
cen
t o
f P
arti
cip
ants
52
53
Future Directions:
•Promote the CONSORT Pro extension Facilitate its use
•Explore best ways to communicate data to patient and clinician consumers
•Improve clinical trial process
• Protocol completeness
• Clinical trial execution
•Integrate with new technologies – e.g. PROMIS adaptive methods
54
Birmingham Institute of Translational Medicine
• World class clinical facility
• Large catchment area
• Clinical trials infrastructure
• Clinical academic expertise which allows delivery of world class translational medicine programme
The largest catchment area in Europe
Birmingham Biomedical Campus and the Institute of Translational Medicine
Broad work areas
• Bioclinical stratification of patient cohorts
• Bioclinical stratification of patients enrolled in clinical trials
• Early phase trials
• Phase II and Phase II clinical trials
• Informatics
• Medical device development and evaluation
• in vitro diagnostic development and evaluation
• industry‐academic partnership
• Patient reported outcomes
Specialty groups
• Audiology/ENT• Cardiology Critical Care• GI Medicine• Liver Disease• Opthalmology• Paediatrics• Rheumatology• Renal Medicine and Diabetes Medicine• Respiratory Medicine
55
Theme groups (non‐oncology)
• Auto‐immune and inflammatory diseases• Chronic Diseases• Acute disease and trauma • Transplantation• Rare Diseases • Paediatric and Transition medicine• In vitro diagnostics• Advanced Imaging• Horizon scanning• Periodontal Health• Patient Reported Outcomes and QOL
56
Neurology Trials and PROHardev Pall
Clinical and Experimental Medicine
Parkinson disease, mortality and DBS• Desire Ngoga, Rosalind Mitchell, Jamilla Kausar, James Hodson, Anwen
Harries, Hardev Pall. Deep brain stimulation improves survival in severe Parkinson’s disease. J Neurol Neurosurg Psychiat In Press 2013
• Cohort study over 10 yr follow up
• Controls – patients who were eligible for surgery but declined
• Primary endpoint – mortality
• Significant difference in mortality and rates of admission to residential care.
• Not randomised
• No patient reported outcomes
• Other studies have reported improved QoL
New Research Question?• Is survival improved by deep brain stimulation of the
subthalamic nucleus in Parkinson’s disease?
• Is this good quality improved survival?
Possible trial design?• Randomise patients to immediate DBS surgery or surgery
delayed till all medical avenues exhausted
• 10 year follow up after DBS to:
• Determine if survival is longer in those operated on at an earlier stage
• Assess ongoing QoL in the two groups
What PRO’s?• EQ5D
• PDQ39 – mobility, ADL, Emotional well-being, sense of stigma, social support, cognition, communication, bodily discomfort and a summary index
• DBS has motor effects – cognition may be helped least or possibly impaired. Does that mean we selectively pick this?
• Carer outcomes
• Driving
• Holidays
• Interaction with families
• Employment
57
Haemodialysis
• Three times a week for four hours
• Significant time travelling and waiting
• Doesn’t give normal renal function
• 15% die per year....
Improving survival in dialysis
When 100 patients were asked if they would consider more frequent haemodialysis:
94% would if increased energy levels
57% would if improved sleep quality
19% would if improved survival by up to 3 years
Ramkumar et al. Hemodial Int 2005 9 281‐295
Current measures
• KDQOL: 15 pages. – Short version 36 questions over 5 pages
• Depression: BDI / HADS
• POS‐S Renal
• EQ5D
• Dialysis recovery time
• PREM
58
Uveitis: inflammation in the eyeUveitis: inflammation in the eye
‘loss or abnormality in body structure or physiological function’ (WHO)
Activity limitation
‘difficulties an individual may have to executing activities’ (WHO)
Participation restriction
‘problems an individual may experience in involvement in a life situation’ (WHO)
Satisfaction
Not defined within the WHO ICF
Body Function/Struct
ure
Body Function/Struct
ure
ActivityActivity
ParticipationParticipation
SatisfactionSatisfaction
Che Hamzah J et al. Choosing appropriate patient‐reported outcomes instrument for glaucoma research: asystematic review of vision instruments. Qual Life Res. 2011;20:1141‐58.
PRO work shop Birmingham Session 5 Research priorities: a clinical perspectiveProfessor Paulus KirchhofChair in Cardiovascular MedicineUniversity of Birmingham Centre for Cardiovascular Sciences andSWBH NHS Trust, Birmingham, UKDepartment of Cardiovascular Medicine, University of Münster, [email protected]
WESTFÄLISCHEWILHELMS-UNIVERSITÄTMÜNSTER
• Why bother?
• The “EHRA score” as a simple PRO tool in atrial fibrillation
• PRO in cardiovascular guidelines?
• PRO in clinical trials in AF: two examples
• Why bother?
• The “EHRA score” as a simple PRO tool in atrial fibrillation
• PRO in cardiovascular guidelines?
• PRO in clinical trials in AF: two examples
Personalised cardiovascular medicine
• The medical profession (and, by extension, health care workers) aim to prolong life and to reduce suffering.
• Cardiovascular diseases are still the most common cause of death and chronic illness in the world.
• Evidence-based medicine was essential to reduce the burden of cardiovascular diseases in Europe and world wide in the last 2 decades.
• Prevention (which may reduce PROs) can help to further reduce this burden to some extent.
• Personalised approaches to therapy supported by evidence are needed to improve the situation further.
• Why bother?
• The “EHRA score” as a simple PRO tool in atrial fibrillation
• PRO in cardiovascular guidelines?
• PRO in clinical trials in AF: two examples
The “EHRA score” for AF Symptoms
Proposal for a new symptom classification scheme
Having noticed that there is no simple, easily applicable, yet AF-specific score to measure symptoms in AF patients, the panel agreed on an AF symptoms score
This classification relates not to the type of AF (to be determined by the physician), but exclusively to the patient-reported symptoms
The panel of experts suggests the following score to describe AF-related symptoms referred to as the EHRA score
1st AFNET/EHRA consensus document. Kirchhof P et al. Eur Heart J 28, 2803-2817 / Europace 9, 1006-23 (2007)
61
AF-related symptoms: EHRA score
1st AFNET/EHRA consensus document. Kirchhof P et al. Eur Heart J 28, 2803-2817 / Europace 9, 1006-23 (2007)
The „EHRA score“ assesses symptoms during AF
In addition, symptom frequency is classified (rare – occasional – frequent)
• Why bother?
• The “EHRA score” as a simple PRO tool in atrial fibrillation
• PRO in cardiovascular guidelines?
• PRO in clinical trials in AF: two examples
Principles of Antiarrhythmic Drug Therapy to Maintain Sinus Rhythm
1. Treatment is motivated by attempts to reduce AF-related symptoms
2. Efficacy of antiarrhythmic drugs to maintain sinus rhythm is modest
3. Clinically successful antiarrhythmic drug therapy may reduce rather than eliminate recurrence of AF
4. If one antiarrhythmic drug ‘fails’, a clinically acceptable response may be achieved with another agent
5. Drug-induced proarrhythmia or extra-cardiac side-effects are frequent
6. Safety rather than efficacy considerations should primarily guide the choice of antiarrhythmic agent
ESC guidelines for the management of AFCamm AJ, et al. Eur Heart J 31: 2369-2429. (2010)
Rhythm Control therapy in AF
2012 focussed update of the ESC guidelines for the management of AFCamm AJ, et al. Eur Heart J, published on line 25 August 2012
• Why bother?
• The “EHRA score” as a simple PRO tool in atrial fibrillation
• PRO in cardiovascular guidelines?
• PRO in clinical trials in AF: two examples
Flec-SL AFNET-3 Design
Standard long-term therapy6 months flecainide
Control groupNo antiarrhythmic drug therapy
Pharmacological reversal of electrical remodeling 4 weeks flecainide
cardioversion, randomization
6 months daily transtelephonic ECG monitoring
inclusion:- persistent AF- Indication for cardioversionpretreatment: - 48 h treatment with flecainide
1. superiority of flecainide vs. no therapy (4 weeks FU, initial patients)
2. noninferiority of short-term vs. long-term therapy (all patients)
62
Flec-SL AFNET 3 CONSORT chart
Kirchhof P, et al. Lancet 380: 238-246. (2012)
Flec-SL AFNET 3: Primary outcome
Kirchhof P, et al. Lancet 380: 238-246. (2012)
635 patients, mean age 64 years, flecainide 4 weeks vs long-term therapyPrimary outcome: time to persistent atrial fibrillation or death, monitored by telemetric ECG
Short-term and long-term therapy were more effective than no therapyShort-term therapy conveyed 80% of the effect of long-term therapy
Flec-SL AFNET 3 Main secondary outcomes
• number of recurrent AF episodes prior to persistent AF not different between groups: 9 (2-13) – 14 (6-23) – 13 (6-23)
• no difference in major adverse outcomes(control 1 (1.2%), short-term 9 (3.3%), long-term 10 (3.6%))
death 0 – 0 – 0
resuscitation / VT: 0 – 2 – 1
syncope: 0 – 2 – 4
stroke/TIA: 0 – 3 – 3
major bleed: 1 – 2 – 2
• improved quality of life at the end of FU:Karnowsky (base 8.3 (1.2)): 8.8 (8.5-9.1)* – 8.9 (8.7-9.0)* – 9 (8.9-9.1)*
Early comprehensive rhythm control therapy can prevent AF-related major complications (stroke,
death, heart failure) compared to usual care
Early treatment of Atrial fibrillation for Stroke prevention Trial
www.easttrial.org
NCT 01288352, ISRCTN04708680
EAST – AFNET 4 trial flow chart
Patients at risk for cardiovascular events(≈ CHADSVASc score ≥ 2*)
and havingrecent onset atrial fibrillation (≤ 1 year duration or first documented by ECG)
Early Rhythm Controlanticoagulation, rate control and either antiarrhythmic drug therapy orpulmonary vein isolation (PVI)
in case of recurrent AF:Re-PVI, adaptation of antiarrhythmic drug therapy
ECG monitoring of therapy
Usual Careanticoagulation, rate control, supplemented by rhythm control only in symptomatic patients on optimal rate control therapy
outpatient FU at 12, 24, 36 moths (both study groups)therapy of underlying heart disease (both study groups)blind assessment of primary outcomes (both study groups)
Pre-Study Screening Study Procedures
Ran
do
mis
atio
n
*Detailed inclusion criteria: One of the following: age > 75 years or prior stroke / TIAORTwo of the following: age > 65 years; female sex; arterial hypertension;diabetes mellitus; previous myocardial infarction, CABG or PCI; stable heart failure (NYHA II or LVEF<50%); left ventricular hypertrophy (>15 mm wall thickness); chronic kidney disease (MDRD stage III - IV); peripheral artery disease.
Kirchhof P, et al. Am H J; in press (2013)
77 / 2810 patients enrolled
ROs:
HRA score
Q-5D, SF-12
Ontreal Cognitive Assessment
minute walk test
EAST – AFNET 4 trial: May 2013
www.easttrial.org
63
Summary: PRO research priorities
• PROs have a huge potential in an era of cost-contained medicine that demands patient-centered and personalised practice of medicine.
• PROs are “soft outcomes” in the context of trials
• difficult to measure and monitor
• likely to remain secondary outcomes in most trials
valid, acceptable to • How do we select PRO domains (e.g. fatigue/anxiety) and questionnaires that are
patients and will inform clinical care?
ome? ials? • What is the difference between a patient centred and patient reported outcHow do we ensure that patient centred outcomes are included in clinical tr
w can we address these issues? •• What other issues should we consider? Ho
roup Discussion 1 ‐ summary of key points G Tabl 1 e
1. who is answering the questionnaires ‐ is it the patients ? 2. best practice issues – lots of good work going on but lack of dissemination.
lidated etc and
3. clinical audit issues re dh funding dependent on specific audit measures that not va
ted? ignores research that’s been done.
nd are people interes enefit) ‐ next stages?
4. What are we doing re measuring outcomes for carers? A5. me sets – (in terms of patient b6.
How you use CORE outco
bridging the gap between research and patient benefit.
include patient’s earlier. balancing act of collaboration between all key stakeholders.
group presentation to patients on the trial you’re planning to run and then getting patients to email you afterwards about what they understood about the trial.
Table 2 • Patient information sheets – too long, good idea to use ‘light’ versions of PIS; we would recommend that NIHR talks to their ethics committees to make it acceptable to have short versions; alternative to have an ‘Executive Summary’ at the start
• Concern expressed around the expectation gap between the benefit that a patient perceives that they will get from being involved in a trial, the requirements on the researcher to ensure that patient info sheets state clearly that there is not expected to be a direct benefit and reality (which may lie somewhere in between)
• Within renal, there is an agreed data set that have to be returned for audit purposes and which can be used as a core outcome set; recently DoH have introduced an EQ5D,time for recovery from dialysis, POSS‐S‐Renal (symptom score – which has not been validated in these cohorts)
Table 3
w your plans and designs • Concise patient information sheets
atients to peer revie • Involving patients with every step – getting p
• Recruitment process and appointment of patient representative
mic background etc • Remit that patients have for giving opinions
ono es? • Recognising your patient group – socioec
iv e • How local are your patient representat
rs in your centr gy for PROMS • Advertising using poste
olo • Standardised method
• Feedback to patients….
ers…email? iceboard in opd • Newslett
• Poster / notWebsite
Phone apps •• Table4 k with ethics committees to • Improve presentation of PIS (length, accessibility, other methods dvd?) – wor
encourage consistency
• What are the preferred/optimal methods for incorporating PROs into COS’s • How does one label what a domain is? • Methodology needed to ensure a fully representative patient group (hard to reach, BMEG’s, those with depression etc)
• Wording of questionnaires to aid interpretation (audio options for those who face literacy challenges)
66
• Should questionnaires be tweaked in some way for use with• Note: online or web‐based translation services (real‐time?)
proxies?
Session 2: PRO Trial Design
Key questions:
ld be included in the trial protocol or other trial documentation such • What PRO‐related components shou
as SOPs?
• Language barriers – considerations • What are the key areas that need further work to inform optimal PRO trial design? How do we ensure that trials adopt best‐practice in PRO trial protocol configuration? Ideas to address •this? Tools? Education?
Group Discussion 2 ‐ summary of key points Table1 PROTOCOL
- - Depends what the trial is
- Which domains are you measuring
- Which questionnaires used Which direction is the measure of change – high scores / low scores
- measured at appropriate clinical time points and these be standardized to Timing of assessments –
- journey of treatment…lots of discussion around this point…
- m the start) Mode of administration Rationale for measurement (how was
-
this chosen, ie were patients involved fro- obal scores How the PRO is being analyzed ie specific domains of interest or gl
How the results will be disseminated - vering the intervention? Is the PRO data used by the clinicians deli
- SOPS
Guidance re pt concerns / contacting GP - Communicatio
LANGUASimplicity
n plan re updating pts.on progress of trial GE BARRIERS / translation
- languages, eg American questionnairess some of lots of questionnaire’s not validated in different
- the language different than in UK eg feeling blue Cost implications for using translated measures
- tures Cultural concepts differ eg pain different in different cul
- KEY AREAS NEEDING FURTHER WORK
Patient / carers involvement (as co‐designers in study) - yet applied Issue re funding, ie how do you pay for patient involvement in the design stages as not
- for funding… Development grants available to put together funding bids available from RfPB PGAR Access to pts if you’re not a clinician so can utilize clinical colleagues or clinical research nurses
- Accessing the right people and information , Sharing best practice, building collaborations etc – linical research networks
-
need to access your cACTICE
- funding conditional BEST PR
Table2 S eci inp your trial design: fy
• Domains first based on what you want to measure, and then choose the
tool/measure that you want to use
• Time windows; inc pre or post intervention (eg dialysis), pre or post doctor
• consultation – generally are done on arrival
Specify whether the patients allowed to have help (carer/nurse/doctor) to fill them in eg for vision, co‐ordination problems. NB issues around bias/pts want to please/think there’s a right answer. This ‘noise’ may be balanced between treatment arms.
w to deal with missing data, sample size • Statistics – planned analysis (primary, secondary), ho
Issues aro practice need to be considered in terms of und best• Design • Delivery
• Recommend to use the BRTC network (Birmingham region Research and Training Collaborative) to provide training to a) researchers on how to bring PRO correctly into the trial design/protocol and b)to
67
site staff to ensure appropriate procedures are followed when e.g. using patient questionnaires (e.g.
timing/location of questionnaires, using translators, using persons to read out questionnaires)
• Challenges of having training around questionnaires being both disease specific assessments and generic. an English, but
pulation. • Issues around language barriers –most questionnaires not validated in languages other thif one omits non‐English speakers than the trial results may not be applicable for your po
audio system/speech‐enabled IT for those who are illiterate/blind. • Consider usage of an Ensuring best practice • Responsibility of Sponsor/CTU/CRF • Have an equivalent standard to CONSORT that is required for publication warding Bodies (try to get a standard shared between • Checklists which are published by the Grant A
MRC/NIHR) as a minimum standard Educating researchers/grant panel members
Published best practice recommendations (preferably open access) •• Table3 • Clarify if relatives can assist patients to answer questionnaires – some family members may answer instead of the patient!
lowing the patient to answer • How can we read questions to patients whilst maintaining privacy and al
s work with language barriers honestly
• SOP – can questions be read. How do
es thi • Identify scoring manual for questionnaire
• Specify time points of questionnaire
• “core” requirements as opposed to “minimum” requirements
s • Who will look at questionnaire results…will they be acted on clinically or only for study analysis • Pilot – ask previous trial patients when they think the time points should be to answer questionnaire • Identify which domains really matter to patients. Select questionnaires appropriately and document reliability / validity
the first few pages completed more fully / with more thought than later • Length of questionnaires…are
pages in the questionnaire.
• Does the above affect quality • Are questionnaires answered before or after clinical assessment time / afterwards patient may be keen to get off • Egpatients are waiting to be seen so questionnaires pass
home.
• Also clinical assessment may change patient perspective
• Commercial vs university sponsored trials • Layout of questionnaire…being clear with plenty of spaces to ensure more complete data entry the • Check complete data entry at the time and let patient know if they have missed fields to give them
er etc. opportunity to complete
• Comment why questionnaire not fully completed in crf….not well enough / language barriAim for good quality data…not just large quantity/recruitment
Conflict of interest for Research Nurses collecting data if they cannot act on their findings •• Table4 • Funding application: justification, details of tool • In protocol: What domain(s) measures, what tool, details of measurement plan (time points, pre‐
randomisation?, in relation to clinic, postal questionnaire – justify what you are going to do and do it uniformly, and be explicit about what you are doing), and data analysis plan, with references
• How to prevent missing data what is the best way, do we know (if so how best to collect, how to remind r missing data, + how to deal with missing data + analyzing missing
e trial results) etc) + Method for recording reason fo
data + plan for reporting. Publication plan (PRO data alongsid
es • Pre‐specify key secondary outcom
• Specify the minimally important difference, justify, reference • ? Cite validation refs in protocol? • Justification for the tool (should we introduce PPI’s/potential trial participants at this stage to select the right outcome for them?)
• Could we look continual validation of a PROM in an interactive way from trial to trial (so the tool becomes lidate more accurate, useful?) – possibly should be a requirement laid down by funders – you must re‐va
the PROM (during or following the trial) to inform the next trial • We should encourage publication of the PRO data alongside the main trial results, funders should encourage this from the outset
• How do we balance out having fewer q’s (not burdening pts, not collected extra info) against more comprehensive measure which might capture unexpected data
68
• Is the answer to selection adaptive trial design, start wider then narrow down options as the trial progresses
Funding panels should pay more attention to PRO elements, **should be a PRO expert on the panel** and •more time to discuss PRO‐specific issues
Need publication/funding (stick) system to force people to conform to a SPIRIT/ CONSORT‐like extension.
Session 3: PRO Trial Conduct
Key questions:
• Is experience of PRO alerts widespread? How best should they be managed? Is there existing guidance we
can draw upon?
• What information for data collection staff should be included in protocol guidance/trial training and why?• How do we encourage uptake of new guidance? Do patients understand why we assess PROs in trials? What PRO‐specific information should be included
Are/should PROs be discussed in consent interview? •in the patient information documentation?
roup Discussion 3 ‐ summary of key points G Ta1. ble1 No,
ation given is shared with the team or GP?
Management – any informDoes it change how pt’s respond if they know info is being shared with team
co se… V. little existing guidance
2. mmu training / what to do when concerning issues ari1. Make it part of GCP training which is a requirement for all researchers
nicating with pts
- induction packs - clinical trials network part of site initiation to provide training
4. Where is pt told that research will learn from their responses but not necessarily benefit – PIS… -
Ta e2 1. Experience of PRO alerts is not widespread; it is felt they are a remote and a rare event. In certain
there
bl
clinical trials there are certain parameters that could be predicted to deteriorate and for thoseshould be a contingency plan put in place to deal with them as a certain threshold is reached.
2. There are many studies where PRO data collection is independent of clinical data collection – completely anonymous. If data collected in person, the collectors should be aware of threshold levels nd act or this should be red flagged. Question whether there is anything that is discussed in the visit hat the patient would like to be raised with their clinician. at
Table3 Even if reported what action is taken?
nParameters written i to protocol as with other measures? High and low measures and also for a step increase/ decrease? f information is not relevant then don’t collect. If it is important enough to collect then it ought to be Iresponded to. • What information for data collection staff should be included in protocol guidance/trial training and Guidance on limits for data entry time period as for RCTs.
n into protocol then action to be taken on collection of data taken whether
why?
If built in parameters writtelectronic data or not. eIs the PRO data monitored?
• How do we encourage uptake of new guidance?
re agreement about what this guidance sh • ould be? Is the
• Qualitative work initially with research bodies. • Stick • Do pat at PRO‐specific information should be included in the p Os be discussed in consent interview?
ients understand why we assess PROs in trials? Whtation? Are/should PR
atient information documen
Why would they? Clinicians find this very difficult.
Is this all overcomplicated?
Could this be free text and qualitative rather than quantitative manageable data Tools and tool choice has become very complex
Interpretation of QoL data very complex and poorly understood by researchers/ clinicians.
69
Information to be included: who will read, what will be acted upon, why it is being collected, how results will be reported.
Table4 Phone?
?
Unknown how muchWhat’s responsibility – to patient carer?
patient. ty to carer?
Suicide riskMask flipping off – carer duInform GP?
. ncerned.
Guidance regarding carersPatient – act if reason to be coImplications for blinding.
written /letters ronic data collection.
Guidance for Different from elect
. pain.
Letter to GPDehydration/Autonomy Risk related. SAE life threatening. Pilot guidance Paper based/electronically. Patients and thresholds Methods for PIS – multimedia/patient groups interactive with clinician.
Session 4: Use of PRO trial results to inform clinical practice
Key esqu tions:
1. What are the key challenges for reporting PROs? 2. How can we facilitate improved reporting? Tools, education, other KT activities? 3. What are the barriers to the use of PRO trial results to inform clinical care and how can we address
these? Group Discussion 4 ‐ summary of key points Table 1 er than just add on to meet funding • Ensure PROs used appropriately in clinical trials to begin with rath
requirements
• If reporting on clear hypotheses it becomes much easier to report • Bear in mind differences between clinical and statistical significance • What do the results actually mean in terms of scales, scores, differences and how this applies to thepatient?
eople without condition so what does it all • Trials don’t compare disease qol to normal age qol for p
actually mean?
• PROMIS working on getting normalized qols for people
• Clinician understanding of data – to aid collaborative / shared decision making for patient treatment • Aren’t always RCTs in disease areas – often more cohort studies so less simple choices SF36, useful for clinicians to have pop’n norms • Many tools do have validated noms to compare against eg
reported to compare against but not necessarily for patients
• Pts need the proportionate figures i.e. 1 in 10 will benefit ness – • Events eg harmful side effect easier to present proportionately, however domains such as tiredharder to present (quantify) in this way?
• Basic training for clinicians should include higher level understanding / analysis of results and communicating these to pts.
it • Clinicians who aren’t involved in research don’t necessarily access this information on a regular basis sobecomes much harder for them to interpret findings…
urther work • There are user guides for clinicians to understand research, but not written by clinicians…f
required on this
• Publish online guide to accompany the paper for clinicians for them to interpret the data… • Dissemination to patients is not done or not done well and then communicate this to patients • We need different tools / approaches to combine data from trials
eg web based decision making / podcasts etc…
• Patient lay summaries should be included with trial publications • Patients are looking for far more simplistic information than graphs and data required for reporting and clinicians
70
• We need to get the measurements and scales right as difficult to compare scales against each other when different measures are used so need for more standardized measures and scales – will improve
interpretation.T ble 2 • Cha lena
l ges of • Qualitative data – but recognize that there is a strong quantitative limit. • Time to analyse – but not necessarily so (eg Example of a trial analysis already done with a test
of
data set and then just insert the real trial data at lock, and have the analysis done within a couple hours).
• Higher amounts of missing data – how to impute and report; this may reflect problems in design or capture techniques; too high a burden in terms of multiple time points/unnecessary
uestionnaires; issues of data queries taking time). (po
• Fac itadomains/q
il te• Ch cklist
e
based on • interactive checklist for authors
your entries regarding trial design • encourage reviewers to use the checklist
• Encouraging submission of original data • Commitment to publish (eg NIHR commit to this already, but could this become general
requirement of all registered trials eg either by registries or by funding bodies); can patient voice be used to promote this (ie ‘don’t waste my time collecting data that you don’t disseminate’)
edominantly captures the key • Improve the selection/design of the PROMs we use to ensure only/prtcomes (although may need space for exploratory variables). ou
• Bar ier
• Method of delivery needs consideration eg paper based vs electronic r s lly • Most clinicians would like a single measure that is standard for their whole speciality – not usuapossible
• Jargon – need to make sure that the results can be communicated in a way that is meaningful to patients; may be helped to have patients involved more in the design stage
a T ble 3
ean etc 1• Understanding exactly what the pro is telling you. What exactly does 30% m
• Quality of life meaning varies from person to person • Being statistically significant may not be clinically significant or vice‐versa Looking at sub‐sections for accurate reporting as well as the tool as a whole.
may relate to patient group – can make interpretation difficult • • Not all of a generic tool
2• Education and training • Encourage uptake of this Simple online training sessions for completing questionnaires.
plete questionnaires accurately in pre‐registration training • • Address “value” of knowing how to com
3• Hard to put cost against quality of life
• There can be differences in interpretation of results in relation to QOL • ‘soft data’ difficult for clinicians to interpret themselves • Surgeons are currently measured on their surgery figures not patient QOL etc. which could present a
o take on high risk patients even though it may have a conflict of interest. There is potential resistance tchance to increase QOL. It is difficult to capture quality of life as a target. •
4• The key is having accurately reported data in the first place • Quality of communication in consultation • Ensuring patients are aware that some reduced quality of life is part of the disease process initially and
may improve with time. Maximum interventions may already be in place, so other potential interventionsmay only be considered if symptoms persist.
• Follow up consultations with letter to the patient as well as the GP. This is an informative way of helping the patient to remember what was said and the opportunity to then follow it up with the GP as required. QOL could be added to this. – this has time constraints especially if ideally the patient letter should be worded differently to GP letter
Table 4 Generic route – disease specific issues. People argue disease specific gives more info to guide individual care – but can this be implemented.
71
n/countriesPolicy decisio .
Patient input up front?
Pilot studies. Focus group work at the end.Respositories
thodology PROs. measurement properties and patient views.
Submit meLook carefully of existing measures –
turally appropriateness. Software/ ore on culM
Constructs. Ask patients
ht 1. Challenge of reporting PROs
The key is to get the hypothesis right, and then this allows reporting etc and interpretation rigtility data which if not done properly (and Also problems with interpreting EQ5D scores and u
2. reported properly) this can lead to misleading decisions
3. Need better peer reviewing and editing of journals Look at the educational literature to understand risk communication Need easily accessible data for clinicians to use for everyday conditions
5. Work out how to measure and what sorting out what is core – keep them together in one paper 4.
Session 5: Research priorities: a clinical perspective
Key considerations:
1. Selection of instruments (e.g. for rare diseases/ systemic vs localized effects/ generic‐e.g. EQ‐5Dvs. disease specific)
2. The ability as a community (within one specialty such as renal) to have a preferred PRO tool in trials in a certain area (eg haemodialysis) and having a central repository for use across several trials.
3. lish which is a significant and important Use of PRO in those who do not read or indeed speak Engminority of our population and how to address this.
4. Questionnaire fatigue (links to selection of instruments) 5. Use of PROs in routine clinical care vs. trials. – Suggest we limit our discussions on this. Perhaps topic
for a future workshop?
roup Discussion 5 ‐ summary of key points G Table 1 s • Think need both generic measures and symptom specific, so generic validated measures eg eq5d for croscultural comparisons. Specific should be hypothesis driven
is there any evidence • how many are too many questions eg 4 qr’s for a trial = 80+ questions (qr fatigue),
on this...no of questions, qrs, layout etc...
• should we be able to choose independent domains from validated questionnaires? r single items? so agreed PROs used • PRO bank repositary in specialty areas eg chronic disease, renal, or fo
across trials, deposited in bank and ethical approval to access data from the bank?
• Difference between language and culture in terms of questionnaires. • Clinically difficult to exclude a large number of your cohort because you know the measures etc aren’t applicable...try to address this, eg focus group to discuss depression symptoms with South Asians, lots of work to do...
Table 2
• Selection of instruments
• Use of generic vs disease‐specific • Defining what you want to measure first and then look for tools that capture the domain ifferent • Depends on the context – look at clinical trial stratification eg could have different PROs for ddiseases/subtypes of disease or a single PRO with pre‐specified analysis of selected domains
al societies/royal colleges/NICE/FDA ‐ may be able to provide t will depend on the domains being considered
• The community – internationrecommendations although i
• Translations to be validated
72
Appendix 5: Voting Slides
Best-Practice for Patient Reported Outcomes in Clinical TrialsInstitute of Advanced Studies Workshop
PRO Research Group
Session 1: PRO Selection
Session 1: PRO SelectionPlease vote for your top priority area for research from the following
options:
5%
14%
5%
36%
27%
14%1. Methods to select PRO domains.
2. Development of COS.3. Optimal methods of
patient involvement in outcome selection.
4. Guidance on measuring carer/proxy outcomes.
5. Feedback to patients on trial progress.
6. Development of training tools for PRO selection.
Session 1: PRO SelectionPlease vote for your second priority area for research from the following
options:
23%
9%
0%
27%
23%
18%1. Methods to select PRO domains.
2. Development of COS.3. Optimal methods of
patient involvement in outcome selection.
4. Guidance on measuring carer/proxy outcomes.
5. Feedback to patients on trial progress.
6. Development of training tools for PRO selection.
Session 1: PRO SelectionPlease vote for your third priority area for research from the following
options:
30%
17%
4%
17%
9%
22%1. Methods to select PRO domains.
2. Development of COS.3. Optimal methods of
patient involvement in outcome selection.
4. Guidance on measuring carer/proxy outcomes.
5. Feedback to patients on trial progress.
6. Development of training tools for PRO selection.
Session 2: PRO Trial Design
73
Research priority: PRO Protocol checklist?
1 2
0%
100%1. Yes2. No
Research priority: PRO Protocol checklist tools?
1 2
5%
95%1. Yes2. No
Session 3: PRO Trial Conduct
Research priority: PRO Alert Consensus & Guidance
1 2
13%
88%1. Yes2. No
Research priority: PRO Data Collection/Management Guidelines and Training
1 2
8%
92%
1. Yes2. No
Research priority: PIS – content and mode of delivery
1 2
4%
96%1. Yes2. No
74
Sessions 2 & 3
Please vote for your priority area for research from the following options:
0%
17%
9%
13%
61%1. PRO Protocol Checklist.2. PRO Protocol Checklist Tools.3. PRO Alert Consensus and guidance.4. PRO Data Collection/Management Guidelines
and Training.5. PIS Content and mode of delivery.
Sessions 2 & 3Please vote for your second priority area for research from the following
options:
0%
29%
33%
17%
21%1. PRO Protocol Checklist.2. PRO Protocol Checklist Tools.3. PRO Alert Consensus and guidance.4. PRO Data Collection/Management Guidelines
and Training.5. PIS Content and mode of delivery.
Sessions 2 & 3Please vote for your third priority area for research from the following
options:
39%
35%
9%
13%
4%1. PRO Protocol Checklist.2. PRO Protocol Checklist Tools.3. PRO Alert Consensus and guidance.4. PRO Data Collection/Management Guidelines
and Training.5. PIS Content and mode of delivery.
Session 4: Use of PRO trial results to inform clinical practice
Session 4: Use of PRO trial results to inform clinical practice. Please vote for one priority area for research from the following options:
0%
19%
44%
38%1. Research into
standardisation of tools.
2. Methods to facilitate clinician/patient understanding of PROs.
3. Methods to facilitate shared decision making in relation to PROs.
4. Optimal presentation of lay summaries of PRO results.
Session 5: Research Priorities: a clinical perspective
75
Session 5: Research Priorities: a clinical perspectivePlease vote for one priority area for research from the following options:
12%
53%
29%
6%1. What generic?2. Research into questionnaire
fatigue – layout media, length.3. PRO item bank approach –
validated set of items to be selected by focus group.