Introduction to Introduction to Systematic Review Systematic Review Part 2 Part 2 David Samson, MS David Samson, MS Director, Comparative Effectiveness Director, Comparative Effectiveness Research Research Technology Evaluation Center Technology Evaluation Center Blue Cross and Blue Shield Association Blue Cross and Blue Shield Association February 18, 2010 February 18, 2010 Copyright 200 Blue Cross Blue Shield Associat
33
Embed
Introduction to Systematic Review Part 2 Introduction to Systematic Review Part 2 David Samson, MS Director, Comparative Effectiveness Research Technology.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Introduction to Systematic ReviewIntroduction to Systematic ReviewPart 2Part 2
David Samson, MSDavid Samson, MS
Director, Comparative Effectiveness ResearchDirector, Comparative Effectiveness Research
Technology Evaluation CenterTechnology Evaluation Center
Blue Cross and Blue Shield AssociationBlue Cross and Blue Shield Association
February 18, 2010February 18, 2010 Copyright 200 Blue Cross Blue Shield Association
2
Resources for Learning Systematic ReviewResources for Learning Systematic Review
• Cochrane Handbook for Systematic Reviews of Interventions, 2008.
• AHRQ Guide for Conducting Comparative Effectiveness Reviews (CERs), 2007
– Journal of Clinical Epidemiology articles• Overview: Slutsky et al. 2008, Helfand & Balshem 2009
• Key Question: for population of interest, what are the comparative effects of intervention and comparator in terms of specific outcomes (in given setting, over specified period of time)?
Copyright 2010 Blue Cross Blue Shield Association
7
Project Management/Task Order RequirementsProject Management/Task Order Requirements
Project phases:
• I: Start/Work Plan
• II: Search/Screen/Select Evidence
• III: Data Abstraction/Quality Assessment/EvidenceTables
– reported on an outcome of interest specifically among patients with head and neck cancer;
– involved an intervention of interest, excluding noncomparative studies describing use of 2DRT (defined below) only;
– reported results separately in individual patient groups according to radiation therapy modality received, except for proton beam therapy, where the results of photon and proton therapy may be combined;
– reported tumor control data compiled separately according to tumor site, or included a multivariable analysis that controlled for anatomic location and evaluated the impact of type of radiotherapy on tumor control outcomes.
Copyright 2010 Blue Cross Blue Shield Association
18
RT HNC Study Selection CriteriaRT HNC Study Selection Criteria
Types of Studies: Studies were included for Key Question 1 and Key Question 2 if they were:
• Single-arm studies with 25 or more evaluable patients that adhere to all aforementioned criteria and provide descriptive information on tumor characteristics particularly location and histology. Single-arm (noncomparative) studies of 2DRT were excluded because this radiotherapy technique is currently little practiced. Studies had to use the same type of radiotherapy for boost as for the planning treatment volume; 2DRT or electrons could be used in the lower neck.
• Dose planning studies that did not report any outcome of interest were not included.
Copyright 2010 Blue Cross Blue Shield Association
19
RT HNC Study Selection CriteriaRT HNC Study Selection Criteria
Types of Participants:
• Head and neck cancers included:– larynx;
– pharynx (hypopharynx, oropharyx, and nasopharynx);
– lip and oral cavity;
– paranasal sinus and nasal cavity;
– salivary gland; and
– occult primary of the head and neck
Copyright 2010 Blue Cross Blue Shield Association
20
RT HNC Study Selection CriteriaRT HNC Study Selection Criteria
Types of Participants:
• Other cancers excluded:– brain tumors;
– skull base tumors;
– uveal/choroidal melanoma, other ocular and eyelid tumors;
– otologic tumors;
– cutaneous tumors of the head and neck (including melanoma);
– thyroid cancer;
– parathyroid cancer;
– esophageal cancer; and
– tracheal tumors.
Copyright 2010 Blue Cross Blue Shield Association
21
RT HNC Study Selection CriteriaRT HNC Study Selection Criteria
Treatment Setting: – Primary (definitive): radiotherapy only (no surgery, with or
without chemotherapy)
– Preoperative radiotherapy: radiotherapy before surgery.(with or without chemotherapy)
– Postoperative (adjuvant): radiotherapy after surgery (with or without chemotherapy)
– Reirradiation: radiotherapy after earlier radiotherapy (other treatments irrelevant)
Copyright 2010 Blue Cross Blue Shield Association
22
RT HNC Study Selection CriteriaRT HNC Study Selection Criteria
Treatment Setting: – Primary (definitive): radiotherapy only (no surgery, with or without
chemotherapy)– Preoperative radiotherapy: radiotherapy before surgery.(with or
without chemotherapy)– Postoperative (adjuvant): radiotherapy after surgery (with or without
chemotherapy)– Reirradiation: radiotherapy after earlier radiotherapy (other treatments
irrelevant)– Concurrent chemoradiotherapy: radiotherapy and chemotherapy at the
same time (with or without surgery)– Post-radiotherapy (adjuvant) chemoradiotherapy: chemotherapy given
after radiotherapy (with or without surgery)– Pre-radiotherapy (neoadjuvant) chemoradiotherapy: chemotherapy
given before radiotherapy (with or without surgery)– Split chemoradiotherapy: chemotherapy given both before and after
radiotherapy (with or without surgery)
Copyright 2010 Blue Cross Blue Shield Association
23
RT HNC Study Selection CriteriaRT HNC Study Selection Criteria
Types of Interventions: – intensity-modulated radiotherapy (IMRT), defined as any
treatment plan where intensity-modulated radiation beams and computerized inverse treatment planning is used;
– three-dimensional conformal radiotherapy (3DCRT), defined as any treatment plan where CT-based treatment planning is used to delineate radiation beams and target volumes in three dimensions;
– proton beam therapy (PBT), defined as any treatment plan where proton beam radiation is used; and
– conventional two-dimensional radiotherapy (2DRT), defined as treatment planning where only 2D projection radiographs are used to delineate radiation beams and target volumes.
Copyright 2010 Blue Cross Blue Shield Association
24
RT HNC Study Selection CriteriaRT HNC Study Selection Criteria
Types of Outcomes: Primary (health) outcomes included:– radiation-induced toxicities;– adverse events, both acute and chronic normal tissue toxicity, such as
• xerostomia,• dysphagia;• mucositis, • skin toxicity,• osteoradionecrosis or bone toxicity, and
– effect on quality of life;– clinical effectiveness, including
• local and locoregional control,• time to any recurrence (disease-free survival), and• patient (disease-specific and overall) survival.
Secondary (intermediate) outcomes included:– salivary flow and– probability of completing treatment according to protocol.
Copyright 2010 Blue Cross Blue Shield Association
25
SR Module 5: Data AbstractionSR Module 5: Data Abstraction
• What kind of data to collect?– Guided by KQs/PICOTS
• How much data to collect?– Goldilocks standard
• How to collect data accurately and efficiently?
• What are some challenges in data abstraction?
Copyright 2010 Blue Cross Blue Shield Association
26
Data Abstraction: Head and Neck Cancer ExampleData Abstraction: Head and Neck Cancer Example
• Critical features of the study identity/design: – authors/institutions/dates– patient inclusion/exclusion criteria– number of participants and flow of participants through steps of study– treatment allocation methods (including concealment)– use of blinding
• Patient characteristics, including: – age– sex– race/ethnicity– disease and stage– tumor histology– tumor size– disease duration– other prognostic characteristics (history of tobacco use, etc.)
Copyright 2010 Blue Cross Blue Shield Association
27
Data Abstraction: Head and Neck Cancer Data Abstraction: Head and Neck Cancer
• Treatment characteristics, including:– localization and staging methods– computerized treatment planning– radiation delivery source– regimen, schedule, dose, duration of treatment, fractionation, boosts– beam characteristics– immobilization and repositioning procedures– co-intervention details
• Outcome assessment details:– identified primary outcome– secondary outcomes– response criteria– use of independent outcome assessor (important for subjective outcomes)– follow-up frequency and duration
Copyright 2010 Blue Cross Blue Shield Association
28
Data Abstraction: Head and Neck Cancer Data Abstraction: Head and Neck Cancer
• Data analysis details:– statistical analyses (statistical test/estimation results)
– test used
– summary measures
– sample variability measures
• 6 individual data abstractors:– Original entry by 1 abstractor, fact-checked by 2nd individual
• Form: MS Word tables
• Original data abstraction tables provide material for summary evidence tables
Copyright 2010 Blue Cross Blue Shield Association
29
Data Abstraction ChallengesData Abstraction Challenges
• Non-uniform outcomes (e.g., different pain measurements in different studies)
• Nonnumeric format: graphs requiring judgment to quantify outcomes
– Drawing tools
– Engauge Digitizer
• Incomplete data (frequent problem: no standard error or confidence interval)
• Discrepant data (different parts of the same report gave different numbers)
• Confusing data (cannot figure out what the authors reported)
• Missing data (only the conclusion is reported)
• Multiple (overlapping) publications of the same study with or without discrepant data
Copyright 2010 Blue Cross Blue Shield Association
30
SR Module 7: Rating Quality of Individual StudiesSR Module 7: Rating Quality of Individual Studies
• Quality can be defined as “the extent to which all aspects of a study’s design and conduct can be shown to protect against systematic bias, nonsystematic bias, and inferential error” (Lohr, 2004)
• Considered synonymous with internal validity
• Relevant for individual studies, distinct from assessment of risk of bias for a body of evidence
• Studies should be rated by predefined criteria
• AHRQ Guide recommends global ratings of good, fair, poor
Copyright 2010 Blue Cross Blue Shield Association
31
Quality Rating: RCTs and NRCSsQuality Rating: RCTs and NRCSs
• US Preventive Services Task Force (USPSTF) system (Harris et al. 2001)– Initial assembly of comparable groups: adequate randomization, including
concealment and whether potential confounders (e.g., other concomitant care) were distributed equally among groups
– Maintenance of comparable groups (includes attrition, crossovers, adherence, contamination)
– Important differential loss to follow-up or overall high loss to follow-up
– Measurements: equal, reliable, and valid (includes masking of outcome assessment)
– Clear definition of interventions
– All important outcomes considered
– Analysis: adjustment for potential confounders, intention-to-treat analysis
Copyright 2010 Blue Cross Blue Shield Association
32
USPSTF Rating SystemUSPSTF Rating System
• Good: Meets all criteria; comparable groups are assembled initially and maintained throughout the study (follow-up at least 80 percent); reliable and valid measurement instruments are used and applied equally to the groups; interventions are spelled out clearly; all important outcomes are considered; and appropriate attention is given to confounders in analysis. In addition, for randomized, controlled trials, intention to treat analysis is used.
• Fair: Studies graded “fair” if any or all of the following problems occur, without the fatal flaws noted in the “poor” category below: In general, comparable groups are assembled initially but some question remains whether some (although not major) differences occurred with follow-up; measurement instruments are acceptable (although not the best) and generally applied equally; some but not all important outcomes are considered; and some but not all potential confounders are accounted for. Intention-to-treat analysis is done for randomized, controlled trials.
• Poor: Studies graded “poor” if any of the following fatal flaws exists: Groups assembled initially are not close to being comparable or maintained throughout the study; unreliable or invalid measurement instruments are used or not applied at all equally among groups (including not masking outcome assessment); and key confounders are given little or no attention. For randomized, controlled trials, intention-to-treat analysis is lacking.