Literature review basics

Click to add title

Click to add subtitle

C.O.R.G.I.

What is a systematic review?

‘review of a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant research and to collect and analyse data from the studies that are included in the review. Statistical methods (meta-analysis) may or may not be used to analyse and summarise the results of the included studies’

Clark M and Oxman AD 2003. Cochrane Reviewers’ Handbook 4.2.0 Oxford: The Cochrane Library

Click to add title

Click to add subtitle

C.O.R.G.I.

What is a systematic review?

‘review of a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant research and to collect and analyse data from the studies that are included in the review. Statistical methods (meta-analysis) may or may not be used to analyse and summarise the results of the included studies’

Clark M and Oxman AD 2003. Cochrane Reviewers’ Handbook 4.2.0 Oxford: The Cochrane Library

Click to add title

Click to add subtitle

T

C.O.R.G.I.

Why systematic reviews?Topic=(obesity) AND Topic=(child)Timespan=All Years

Search on ISI WoK 30/11/2010(2010 dropped)

N.B. not all journals are back indexed

Click to add title

Click to add subtitle

C.O.R.G.I.

What types of review are there?

• A narrative review

• A quasi-systematic review (the norm)

• A systematic review•with a narrative (and usually tables of un-pooled study level data) •with study level pooled data (meta-analysis)•with individual level pooled data (IPD meta-analysis)

Click to add title

Click to add subtitle

C.O.R.G.I.

Interpreting types of meta-analysis

• A narrative review•Heavily open to reviewer bias•Often written by experts•Less sensitive (or sometimes overly sensitive) to changes in the body of literature, e.g. Herb Green

Click to add title

Click to add subtitle

C.O.R.G.I.

Interpreting types of meta-analysis

• A quasi-systematic review• Non-replicable• Most reviews are like this

Click to add title

Click to add subtitle

C.O.R.G.I.

Interpreting types of meta-analysis

• A systematic review•with a narrative (and usually tables of un-pooled data) •with study level pooled data (meta-analysis)

• quantitative estimation of the magnitude of the effect• studies are like subjects

•with individual level pooled data (IPD meta-analysis)• the Cochrane gold standard• both study and individual characteristics modelled

Click to add title

Click to add subtitle

C.O.R.G.I.

Searching for relevant studies• Relatively easy for an intervention

• Your question is the most important part• Population, Intervention, Comparison and Outcome (PICO to EBM buffs)

• Cochrane reviews are the gold standard – but they tend to stick to far easier topic of interventions

• Must have a defined question• Population• Variable of interest (i.e. diet, PA, TV etc)• Outcome

Click to add title

Click to add subtitle

C.O.R.G.I.

Where to look?• MeSH indexed referenced libraries (e.g. Medline, Embase)• Other databases (e.g. PubMed, Google Scholar)• Grey literature (e.g. NHS NL, NOO)• Hand searching

• Citations• Key journals

Click to add title

Click to add subtitle

C.O.R.G.I.

Using MeSH headings

Click to add title

Click to add subtitle

C.O.R.G.I.

Click to add title

Click to add subtitle

C.O.R.G.I.

Minus the duplicates=

Click to add title

Click to add subtitle

C.O.R.G.I.

If obesity is the major theme=

Click to add title

Click to add subtitle

C.O.R.G.I.

If obesity is the major theme=

Click to add title

Click to add subtitle

C.O.R.G.I.

Using MeSH headings

• As with any research project, a protocol for a systematic review should set out the aims/objectives and methods.

• Frame the research question (PICO)…– Who? (Participants)– Why? (Interventions, Comparisons)– What? (Outcomes)

…and define the study objectives

Protocol development

Protocol development - Methods• Eligibility criteria

– Types of: study, participants, interventions, outcome (primary & other)

• Search strategy– Which databases? Which years? Publication status?– Languages– Hand searching of key journals/review articles? Contact authors?

• Data extraction– Raw data or summary estimates?– Quality appraisal

• Analysis plan (data synthesis, sensitivity analyses) and tables

Screening and extraction• Screening usually in multiple stages, e.g. 1st stage

based on titles; 2nd stage based on abstracts; 3rd stage based on full-text articles

• In/Out and data extraction forms (may be combined) should be designed during protocol development– In/Out form intended for early evaluation/screening of

papers– Data extraction – think about all the data items required

for quality appraisal and data synthesis (e.g. sampling strategy, follow-up period, study size, participants, outcome measures, funding sources etc.)

Example In/Out form…

Quality appraisal for RCTs http://www.cochrane-handbook.org/

Reviewers’ judgement; ‘Yes’ indicates low risk of bias, ‘No’ indicates high risk of bias

• Other tools for assessing study quality include scales and checklists:– Olivo et al ‘Scales to assess the quality of randomized controlled

trials: a systematic review’ Phys Ther 2008; 88(2):156-75– Newcastle Ottawa Scale for nonrandomised studies

http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp

Note: Use of scales/summary scores is discouraged in Cochrane reviews…

• Consider other sources of bias, e.g. publication bias, search strategy (language bias, database)

Presenting resultsPRISMA Statement (trials)

– Preferred Reporting Items for Systematic Reviews and Meta-analyses (formerly QUOROM Statement)

– Minimum set of items for reporting in systematic reviews and meta-analyses (checklist and flow diagram)

– http://www.prisma-statement.org/

STROBE Checklist (observational studies)– Strengthening the reporting of observational studies in

epidemiology– Checklist of items that should be included in reports of

observational studies– http://www.strobe-statement.org/index.php?id=available-checkli

sts

• Introduction: rationale and objectives

• Methods: eligibility criteria, data sources, search strategy, study selection, data extraction, quality appraisal, summary measures and synthesis of results

• Results: study selection, study characteristics, study quality, results of individual studies, any synthesised results (forest plots) and other analyses

• Discussion: summary, limitations, conclusions

Flow diagram of study selection process

Tips• Keep the PRISMA/STROBE checklist to hand

• Keep a record of all the titles, abstracts and papers that you screen and reasons for rejecting them (you might record these in a database; EndNote can be useful here)