Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions 7 AUTHORS: Andrew Booth, Jane Noyes, Kate Flemming, Ansgar Gerhardus, Philip Wahlster, Gert Jan van der Wilt, Kati Mozygemba, Pietro Refolo, Dario Sacchini, Marcia Tummers, Eva Rehfuess This project is co-funded by the European Union under the Seventh Framework Programme (Grant Agreement No. 306141)
40
Embed
Guidance on choosing qualitative evidence synthesis ...
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions
7
AUTHORS: Andrew Booth, Jane Noyes, Kate Flemming,
Ansgar Gerhardus, Philip Wahlster, Gert Jan van der Wilt,
Kati Mozygemba, Pietro Refolo, Dario Sacchini, Marcia
Tummers, Eva Rehfuess
This project is co-funded by the European Union under the Seventh Framework Programme (Grant Agreement No. 306141)
PLEASE CITE THIS PUBLICATION AS:
BOOTH, A., NOYES J, FLEMMING K, GERHARDUS, A., WAHLSTER, P., VAN DER WILT, G.J., MOZYGEMBA, K.,
REFOLO, P., SACCHINI, D., TUMMERS, M., REHFUESS, E. (2016) Guidance on choosing qualitative evidence
synthesis methods for use in health technology assessments of complex interventions [Online]. Avai-
Table 2: Search terms used in the Guidance Development.
| 18
The seven sub-domains of the RETREAT framework
were mapped against 19 specific methodologies of
qualitative synthesis previously identified (See Tab-
le 4). The identified documents were used to map
whether each review method was appropriate for
each consideration. This was supplemented by ex-
periential insights from experienced reviewers from
the Cochrane Qualitative and Implementation Me-
thods Group. Where the authors/reviewers conside-
red the issue to be positively addressed by the me-
thodology items were rated as Green (Appropriate).
Where the issue is not specifically mentioned but is
compatible with existing knowledge on the metho-
dology items were rated Yellow (Potentially Appro-
Table 3: Considerations when choosing a synthesis method from identified literature.
Rev
iew
Ques
tion
Epis
tem
olo
gy
Tim
e/ T
imef
ram
e
Res
ourc
es
Exper
tise
Audie
nce
& P
urp
ose
Type
of D
ata
Paterson et al (2001)
Sandelowski & Barroso (2003)
McDermott et al (2004)
Dixon-Woods et al (2004; 2005)
Mays et al (2005)
Lucas et al (2007)
Pope et al (2007)
CRD (2008)
Garside (2008)
Barnett-Page & Thomas (2009)
Ring et al (2010)
Manning (2011) [In Hannes & Lockwood, 2011]
Noyes & Lewin (2011)
Paterson (2011) [In Hannes & Lockwood, 2011]
Urquhart (2011)
Booth (2012)
Gough et al (2012)
Saini (2012); Saini & Shlonsky (2012)
Shaw (2012)
Snilstveit et al (2012)
Tong et al (2012)
Greenhalgh & Wong (2014)
Toye et al (2014)
Whitaker et al (2014)
19 |
priate). Finally where authors/reviewers comment
on the unsuitability of a methodology for a par-
ticular consideration items were flagged with Red
(Not Appropriate) (Table 5). Other more qualitative
considerations are indicated by additional, non-in-
dicative colour shading.
4 APPLICATION OF THE GUIDANCE
To apply this guidance a reader compares the cha-
racteristics of their planned review with the requi-
rements and functions of the respective types of syn-
thesis. So, for example, they start by defining whether
their intention is to conduct a stand alone qualitative
synthesis (Table 7) or a review that integrates quan-
titative and qualitative data (Table 8). Having tracked
their decision to the appropriate Table the reader then
compares the options contained in each criteria (row)
with the available methodological options (columns).
Optimally the reader will reach a point at which the
range of available options has been reduced to a limi-
ted number, preferably a single choice.
4.1 REVIEW QUESTION
The review question is consistently identified as an
important factor when determining the methodology
of synthesis. The review question carries several con-
siderations. Unlike effectiveness questions, where the
starting point for a synthesis is a fixed PICO (Popu-
lation-Intervention-Comparison-Outcome) question
framework (Richardson et al, 1995), methods of qua-
litative synthesis may utilise either a fixed question or
a more negotiable, emerging question. Where there is
an accompanying effectiveness question review teams
often use a corresponding PICO or SPICE (Setting-Per-
spective- Interest, Phenomenon of – Comparison -
Evaluation) pre-defined question (Booth, 2006). In
essence the question structure within this type of
qualitative synthesis is used as an “anchor” (Eakin &
Mykhalovskiy, 2003) to ensure that the effectiveness
and qualitative aspects remain co-terminous. Other
methods, usually with an interpretive intent (e.g.
grounded formal theory or meta-ethnography), tre-
at the review question in a way that is more analo-
gous to primary qualitative research. The question is
seen as negotiable and thus to be explored as a result
of the initial review process. In short the question
itself becomes clearer as the review team examines
their data in a manner analogous to grounded theory
approaches in primary research. This may pose parti-
cular challenges with regard to determination of the
review protocol which may consequently need to be
delayed or produced in a more iterative manner. The
review question can be conceived as a "compass" (Ea-
kin & Mykhalovskiy, 2003) offering a general direction
of travel without predetermining its limits.
Review Question
EpistemologyTime/
TimeframeResources Expertise
Audience & Purpose
Type of Data
Fixed Generation of Theory
Degree of Iteration
Personnel In Qualitative Research
Academics Thin/Thick
Emerging Exploration of Theory
Degree of Integration
Funding In Systematic Reviewing
Policymakers Rich/Poor
Testing of Theory
Points of Integration
Effort In Topic Area Practitioners Individual Article
Idealist In Theory Developers of Interventions
Body of Literature
Realist In Literature searching
Theory
Aggregative Disciplinary, Methodological and Perspective Mix
Likely number of relevant studies
Interpretive/ Configurative
Unit of Analysis
Table 4: Subdomains representing more detailed considerations for choice of qualitative synthesis method.
| 20
Conside-rations for Review Me-thodology B
est
Fit
Fram
ework
Syn
thes
is
Conce
pt
Anal
ysis
Crit
ical
Inte
rpre
tive
Syn
thes
is
Ecolo
gica
l Tr
iangu
lati
on
Fram
ework
Syn
thes
is
Gro
unded
Form
al T
heo
ry
Met
a-Ag
greg
atio
n
Met
a-Et
hnogr
aphy
Met
a-In
terp
reta
tion
Met
a-N
arra
tive
Met
a-St
udy
Met
a-Su
mm
ary
Nar
rati
ve S
um
mar
y
Nar
rati
ve S
ynth
esis
Qual
itat
ive
Inte
rpre
tive
M
eta-
Synth
esis
Rap
id R
ealist
Syn
thes
is
Rea
list
Syn
thes
is
Text
ual
nar
rati
ve s
ynth
esis
Them
atic
syn
thes
is
Review Question
Question Type
Mult
iple
- Qual
2
Mult
iple
- Qual
Mult
iple
- Qual
3
Qual
Mult
iple
- Qual
Qual
Qual
Qual
Qual
Mult
iple
- Qual
4
Mult
iple
Mult
iple
Mult
iple
Mult
iple
Qual
Mult
iple
Mult
iple
5
Mult
iple
Mult
iple
- Qual
Fixed/ Emerging
Dixon-Woods et al (2004; 2005) FI
XED
FIXE
D
FIXE
D
FIXE
D
FIXE
D
EMER
G-
ING
FIXE
D
EMER
G-
ING
EMER
G-
ING
FIXE
D
EMER
G-
ING
FIXE
D
Uncl
ear
FIXE
D
FIXE
D
FIXE
D
FIXE
D
FIXE
D
FIXE
D
Epistemology
Epistemology Dependence
LOW
MOD
HIG
H
HIG
H
LOW
HIG
H
MOD
HIG
H
HIG
H
HIG
H
MOD
MOD
LOW
LOW
MOD
HIG
H
HIG
H
LOW
LOW
Idealist/ Realist
Rea
list
?
Uncl
ear
Idea
list
Rea
list
/ Id
ealist
Rea
list
Idea
list
Rea
list
?
Idea
list
Idea
list
?
Idea
list
Idea
list
Rea
list
?
Uncl
ear
Rea
list
?
Idea
list
?
Rea
list
Rea
list
Rea
list
Rea
list
Time/Timeframe
Time Required
LOW
MOD
MOD
MOD
LOW
HIG
H
MOD
HIG
H
HIG
H
HIG
H
HIG
H
MOD
MOD
MOD
MOD
LOW
HIG
H
LOW
LOW
Resources
Comprehensi-ve/ Purposive Sampling
COM
PRE-
HEN
SIVE
PURPO
-SI
VE
PURPO
-SI
VE
COM
PRE-
HEN
SIVE
COM
PRE-
HEN
SIVE
PURPO
-SI
VE
COM
PRE-
HEN
SIVE
PURPO
-SI
VE
PURPO
-SI
VE
PURPO
-SI
VE
COM
PRE-
HEN
SIVE
COM
PRE-
HEN
SIVE
UNCL
EAR
COM
PRE-
HEN
SIVE
PURPO
-SI
VE
PURPO
-SI
VE
PURPO
-SI
VE
COM
PRE-
HEN
SIVE
BOTH
Interlibrary loans & pho-tocopies H
IGH
MOD
LOW
HIG
H
HIG
H
MOD
HIG
H
MOD
HIG
H
HIG
H
HIG
H
HIG
H
MOD
MOD
MOD
LOW
MOD
HIG
H
MOD
Expertise
Qualitative Researcher
LOW
LOW
HIG
H
MOD
LOW
HIG
H
MOD
HIG
H
HIG
H
HIG
H
MOD
LOW
LOW
MOD
HIG
H
HIG
H
LOW
LOW
Info Specialist
HIG
H
LOW
HIG
H
MOD
MOD
LOW
MOD
LOW
HIG
H
MOD
MOD
MOD
MOD
MOD
MOD
HIG
H
MOD
MOD
Table 5: Choosing the Review Method – What are the Critical Requirements?
2 Best Fit Framework synthesis has not been used to integrate quantitative and qualitative data but in principle it meets the requirements of a framework-based mixed methods approach.
3 It is unlikely that a critical interpretive synthesis would only review qualitative literature as it seeks to purposively and theoretically sample independent of study quality and study design.
4 It is unlikely that a meta-narrative approach would only review qualitative literature as the objective is to identify different research paradig-ms which might be split across quantitative and qualitative methodologies
5 Although realist synthesis is conceived as an integrative approach published examples exist of realist synthesis of qualitative research only.
21 |
Conside-rations for Review Me-thodology B
est
Fit
Fram
ework
Syn
thes
is
Conce
pt
Anal
ysis
Crit
ical
Inte
rpre
tive
Syn
thes
is
Ecolo
gica
l Tr
iangu
lati
on
Fram
ework
Syn
thes
is
Gro
unded
Form
al T
heo
ry
Met
a-Ag
greg
atio
n
Met
a-Et
hnogr
aphy
Met
a-In
terp
reta
tion
Met
a-N
arra
tive
Met
a-St
udy
Met
a-Su
mm
ary
Nar
rati
ve S
um
mar
y
Nar
rati
ve S
ynth
esis
Qual
itat
ive
Inte
rpre
tive
M
eta-
Synth
esis
Rap
id R
ealist
Syn
thes
is
Rea
list
Syn
thes
is
Text
ual
nar
rati
ve s
ynth
esis
Them
atic
syn
thes
is
Audience & Purpose
Academics
MAY
BE
YES
YES
MAY
BE
MAY
BE
YES
MAY
BE
YES
YES
YES
YES
YES
YES
YES
YES
NO
YES
MAY
BE
MAY
BE
Designers of Interventions
YES
NO
MAY
BE
YES
YES
NO
MAY
BE
NO
NO
NO
NO
NO
NO
MAY
BE
MAY
BE
YES
YES
MAY
BE
MAY
BE
Policymakers
YES
NO
MAY
BE
YES
YES
NO
YES
NO
NO
NO
NO
NO
NO
YES
YES
YES
YES
YES
YES
Practitioners
YES
NO
MAY
BE
YES
YES
NO
YES
NO
NO
NO
NO
NO
NO
YES
YES
YES
YES
YES
YES
Commis-sioners of Research M
AYBE
NO
YES
MAY
BE
MAY
BE
MAY
BE
MAY
BE
MAY
BE
MAY
BE
YES
YES
YES
MAY
BE
YES
MAY
BE
MAY
BE
MAY
BE
YES
MAY
BE
Type of Data
Conceptually Rich Data N
O
YES
YES
NO
NO
YES
NO
YES
YES
YES
MAY
BE
NO
NO
NO
MAY
BE
YES
YES
NO
NO
Contextually Thick Data N
O
MAY
BE
MAY
BE
NO
NO
NO
NO
MAY
BE
YES
NO
MAY
BE
NO
NO
NO
MAY
BE
YES
YES
NO
NO
Large number of studies
YES
MAY
BE
YES
MAY
BE
YES
NO
YES
NO
NO
YES
YES
YES
YES
YES
MAY
BE
NO
MAY
BE
YES
YES
Other Considerations
Reporting Standards
ENTR
EQ?
NON
E
ENTR
EQ
ENTR
EQ
ENTR
EQ
ENTR
EQ
NON
E
ENTR
EQ,
eMER
GE
NON
E
RAM
ESES
ENTR
EQ
NON
E
NON
E
NON
E
NON
E
NON
E
RAM
ESES
ENTR
EQ
ENTR
EQ
Method is appropriate Method may be appropriate Method is not appropriate
| 22
Recently researchers have revisited whether a qualita-
tive review question, carried out to support an HTA ef-
fectiveness question, should actually be co-terminous
with the effectiveness question (Lorenc et al, 2012).
They note that qualitative research relating to a new
technology may necessarily be limited. As a conse-
quence the review team may need to broaden the
scope of the qualitative systematic review to include
exploration of the phenomenon of the untreated/pre-
treated condition and the lived experience of patients
with the target condition.
Other question formulations proposed for systematic
reviews of qualitative research include SPIDER (Cooke
et al, 2012), PICOC (Petticrew & Roberts, 2006) and
CHIP (Shaw, 2010; 2012) (Table 6).
4.2 EPISTEMOLOGY
Commentators tend to agree that the reviewer should
be mindful of the need to not violate the philoso-
phical foundations or the integrity of the qualitative
primary studies (Sandelowski & Barroso, 2007). For
some types of synthesis considerations of epistemolo-
gy are particularly important i.e. that the method of
synthesis should be compatible with the epistemolo-
gy of the included studies. For example meta-ethno-
graphy and grounded formal theory make frequent
recourse to epistemological considerations at each
stage of the review process. In contrast other me-
thods may be regarded as more epistemology-neu-
tral – for example best fit framework synthesis,
narrative synthesis and thematic synthesis. Within
the discipline of education Major and Savin-Baden
(2010) take the extreme position in their qualitati-
ve research synthesis methodology that only studies
with the same epistemological underpinnings should
be handled within the same synthesis. In contrast
health services research and technology assessment
pursues a more pragmatic orientation with it being
common practice to integrate qualitative studies of
different types within a single synthesis. Even me-
ta-ethnography, which implies the systematic analy-
sis of ethnographies, typically exhibits inclusion of a
wide range of study types.
Barnett-Paige and Thomas (2009) seek to characte-
rise types of qualitative synthesis on an idealist –
realist continuum. They note that the developers of
Table 8: Integrating Quantitative and Qualitative Data within a Mixed-Method Synthesis – Which Review Processes Are Required?
⊗ = Not Required = Uncertain ⊕ = Essential
6 Characterised as Systematic Meta-narrative review7 Characterised as Rapid Meta-narrative review
31 |
5 CONCLUSIONSChoice of synthesis method can be seen to be a com-
plex multifactorial decision which requires conside-
ration of multiple variables. Such is this complexity
that it has not been possible to embody methodologi-
cal advice in a single algorithm. While such an algo-
rithm has been attempted by some commentators this
approach has tended to give primacy to one or more
guiding variables (e.g. the role of theory). It is not yet
clear which considerations should be prioritised and
so, as an alternative approach we present a matrix of
considerations that can be examined for each indivi-
dual planned review.
This guidance seeks to capture the complexity present
in the source texts. Nevertheless it has been possible
to identify a relatively risk averse strategy when faced
with numerous unknown variables. The most acces-
sible method of synthesis is thematic synthesis – this
can be selected in the absence of other positive indi-
cations. It carries the added utility of being convertib-
le to meta-ethnography should the source data prove
sufficiently rich.
Experimentation and empirical testing of methods
of synthesis remains in its infancy and we anticipate
that, while the overall guiding principles will continue
to stand the test of time, the detail of considerations
will become progressively more granular and specific.
We welcome the opportunity for continued debate wi-
thin the methodological community on the determi-
nants of choice of synthesis for a qualitative evidence
synthesis.
Table 9: Mechanisms for Integration and their Point of Integration.
Mechanism Method Point of Integration Example
Textual Summary Narrative synthesis Synthesis stage Iwelunmor, J., Plange-Rhule, J., Airhihenbuwa, C. O., Ezepue, C., & Ogedegbe, O. (2015). A nar-rative synthesis of the health systems factors influencing optimal hypertension control in Sub-Saharan Africa. PloS one,10(7), e0130193.
Tables Tabulation Synthesis stage Leamy, M., Bird, V., Le Boutillier, C., Williams, J., & Slade, M. (2011). Conceptual frame-work for personal recovery in mental health: systematic review and narrative synthesis. The British Journal of Psychiatry, 199 (6), 445-452.
Matrices Tabulation Synthesis stage Nowak, P. (2011). Synthesis of qualitative linguistic research — A pilot review integrating and generalizing findings on doctor–patient interaction. Patient education and counseling, 82(3), 429-441.
Themes Thematic Synthesis stage Robinson, L., Hutchings, D., Corner, L., Finch, T., Hughes, J., Brittain, K., & Bond, J. (2007). Balancing rights and risks: Conflicting perspec-tives in the management of wandering in de-mentia. Health, Risk & Society, 9(4), 389-406.
Conceptual Model or Framework
Framework A priori and/or at synthesis stage
Hulland K, Martin N, Dreibelbis R, DeBruicker Valliant J, Winch P (2015) What factors affect sustained adoption of safe water, hygiene and sanitation technologies? A systematic review of literature. London: EPPI-Centre, Social Science Research Unit, UCL Institute of Education, Uni-versity College London.
Logic Model Framework A priori and/or at synthesis stage
Turley, R., Saith, R., Bhan, N., Doyle, J., Jones, K., & Waters, E. (2013). Slum upgrading re-view: methodological challenges that arise in systematic reviews of complex interventions. Journal of public health, 35(1), 171-175.
Context-Mechanism- Outcome (CMO) Configurations
Framework Data Extraction de Goeij, M. C., Suhrcke, M., Toffolutti, V., van de Mheen, D., Schoenmakers, T. M., & Kunst, A. E. (2015). How economic crises affect alcohol consumption and alcohol-related health prob-lems: A realist systematic review. Social Science & Medicine, 131, 131-146.
Graphic Narrative synthesis Synthesis stage Belanger, E., Rodrıguez, C., & Groleau, D. Shared decision-making in palliative care: A systematic mixed studies review using narrative synthesis.Palliative Medicine, 25(3), 242-261.
SHEMILT I. (2013) Introducing a series of methodological articles on considering complexity in systematic
reviews of interventions. Journal of Clinical Epidemiology, 66:1205-8.
ANDERSON LM, OLIVER SR, MICHIE S, REHFUESS E, NOYES J, SHEMILT I. (2013) Investigating complexity in systematic reviews of interventions by using a spectrum of methods. Journal of Clinical Epidemiology. 66:1223-9.
BARNETT-PAGE E, THOMAS J. (2009) Methods for the synthesis of qualitative research: a critical review. BMC Me-
dical Research Methodolology, 11: 59.
BONDAS, T., HALL, E. O. (2007). A decade of metasynthesis research in health sciences: A meta-method study. International Journal of Qualitative Studies on Health and Well-being, 2: 101-113.
BOOTH, A. (2006). Clear and present questions: formulating questions for evidence based practice. Library Hi Tech, 24: 355-368.
BOOTH, A (2012) Qualitative evidence synthesis for HTA. Workshop at 9th Annual Meeting HTAi Bilbao 2012 (June 23rd-24th). HTA in Integrated Care for a Patient Centered System Available from: http://www.htai2012.org/home.htm [Accessed 12.12.2015].
BOOTH, A, HARRIS, J NOYES, J HANNES, K, HARDEN, A, BURFORD, B. (2012) Incorporating Qualitative Perspectives within Your Review: Choosing the Right Approach. Workshop at: 20th Cochrane Colloquium. Auckland: 30 September -3 October.
BOOTH, A. (2011-2015) ESQUIRE Synthesis Workshops. University of Sheffield. Available from: http://esquireshef-field.pbworks.com/w/page/10399148/ [Accessed 12.12.2015].
BOOTH, A., CARROLL, C., ILOTT, I., LOW, L. L., COOPER, K. (2013a). Desperately Seeking Dissonance Identifying the
Disconfirming Case in Qualitative Evidence Synthesis. Qualitative Health Research, 23: 126-141.
BOOTH, A., CARROLL, C. (2015). Systematic searching for theory to inform systematic reviews: is it feasible? Is it desirable? Health Information & Libraries Journal, 32: 220-235.
BOOTH, A., HARRIS, J., CROOT, E., SPRINGETT, J., CAMPBELL, F., WILKINS, E. (2013b). Towards a methodology for cluster searching to provide conceptual and contextual “richness” for systematic reviews of complex in-terventions: case study (CLUSTER). BMC Medical Research Methodology, 13: 118.
BURFORD B, LEWIN S, WELCH V, REHFUESS E, WATERS E. (2013) Assessing the applicability of findings in systematic reviews of complex interventions can enhance the utility of reviews for decision making. Journal of Cli-nical Epidemiology, 66:1251-61.
BRIGGS, M., FLEMMING, K. (2007). Living with leg ulceration: a synthesis of qualitative research. Journal of Ad-vanced Nursing, 59: 319-328.
BRYMAN, A. (2008). Of methods and methodology. Qualitative Research in Organizations and Management: An International Journal, 3: 159-168.
CAMPBELL, R., POUND, P., MORGAN, M., DAKER-WHITE, G., BRITTEN, N., PILL, R., ... DONOVAN, J. (2011). Evaluating meta-ethnography: systematic analysis and synthesis of qualitative research. Health Technology Assess-ment, 15: 1366-5278.
CANDY, B., KING, M., JONES, L., OLIVER, S. (2011). Using qualitative synthesis to explore heterogeneity of complex interventions. BMC Medical Research Methodology, 11: 124.
CARROLL, C., BOOTH, A., COOPER, K. (2011). A worked example of" best fit" framework synthesis: A systematic review of views concerning the taking of some potential chemopreventive agents. BMC Medical Research Methodology, 11: 1-9.
CENTRE FOR REVIEWS AND DISSEMINATION (CRD). (2009). Systematic reviews: CRD's guidance for undertaking re-views in health care. Centre for Reviews and Dissemination.
35 |
CHILCOTT, J.B., WARD, S., SQUIRES, H. (2016) Guidance to assess economic aspects. In: LYSDAHL, K.B., MOZYGEMBA, K., BURNS, J., CHILCOTT, J.B., BRÖNNEKE, J.B., HOFMANN, B. (eds.). Guidance for assessing effectiveness, economic aspects, ethical aspects, socio-cultural aspects and legal aspects in complex technologies [On-line]. Available from: http://www.integrate-hta.eu/downloads/ [Accessed 12.12.2015].
COOKE, A., SMITH, D., BOOTH, A. (2012). Beyond PICO The SPIDER Tool for Qualitative Evidence Synthesis. Qualitative Health Research, 22: 1435-1443.
DIXON-WOODS M, AGARWAL S, YOUNG B, JONES D, SUTTON A. (2004) Integrative Approaches to Qualitative and Quan-
titative Evidence. London: NHS Health Development Agency.
DIXON-WOODS M, AGARWAL S, JONES D, YOUNG B, SUTTON AJ. (2005) Synthesising qualitative and quantitative evi-dence: a review of possible methods. Journal of Health Services Research and Policy, 10: 45-53.
EAKIN, J. M., MYKHALOVSKIY, E. (2003). Reframing the evaluation of qualitative health research: reflections on a review of appraisal guidelines in the health sciences. Journal of Evaluation in Clinical Practice, 9: 187-194.
FLEMMING K (2015). Choice of Review Methods. Personal Communication. November 2015.
GARSIDE, R. (2008). A comparison of methods for the systematic review of qualitative research: two examples using meta-ethnography and meta-study. Doctoral dissertation, Universities of Exeter and Plymouth).
GERHARDUS, A., GOYDER, E., HOFMANN, B., MOZYGEMBA, K., OORTWIJN, W, REHFUESS, E., SACCHINI, D., VAN DER
WILT, G.J. (2016). Integrated health technology assessment for evaluating complex technologies (IN-TEGRATE-HTA): An introduction to the guidances [Online]. Available from: http://www.integrate-hta.eu/downloads/ [Accessed 12.12.2015].
GOUGH, D., THOMAS, J., OLIVER, S. (2012). Clarifying differences between review designs and methods. Systematic Reviews, 1: 28.
GREENHALGH, T., WONG, G. (2014). Training materials for meta-narrative reviews. Version, 1. http://www.rame-sesproject.org/media/Meta_narrative_reviews_training_materials.pdf [Accessed 12.12.2015].
HANNES, K., LOCKWOOD, C. (2011a). Synthesizing qualitative research: Choosing the right approach. London: Wiley Blackwell.
HANNES K., LOCKWOOD C (2011b). Pragmatism as the philosophical foundation for the Joanna Briggs meta-agg-regative approach to qualitative evidence synthesis. Journal of Advanced Nursing, 67:1632-42.
HANNES, K., MACAITIS, K. (2012). A move to more systematic and transparent approaches in qualitative evidence synthesis: update on a review of published papers. Qualitative Research, 12: 402-442.
HARDEN A. (2010) Mixed-Methods Systematic Reviews: Integrating Quantitative and Qualitative Findings [mo-nograph on the internet]. London, UK: FOCUS: Technical Brief no. 25. Available from: www.ncddr.org/kt/products/focus/focus25/ [Accessed 12.12.2015].
LORENC, T., PEARSON, M., JAMAL, F., COOPER, C., GARSIDE, R. (2012). The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Research Synthesis Methods, 3: 1-10.
LUCAS, P. J., BAIRD, J., ARAI, L., LAW, C., ROBERTS, H. M. (2007). Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews. BMC Medical Research Metho-dology, 7: 4.
LYSDAHL, K.B., MOZYGEMBA, K., BURNS, J., CHILCOTT, J.B., BRÖNNEKE, J.B., HOFMANN, B. (eds.) (2016a) Guidance for assessing effectiveness, economic aspects, ethical aspects, socio-cultural aspects and legal aspects in complex technologies [Online]. Available from: http://www.integrate-hta.eu/downloads/ [Accessed 12.12.2015].
MCDERMOTT, E., GRAHAM, H., HAMILTON, V. (2004). Experiences of being a teenage mother in the UK: A report of
a systematic review of qualitative studies. Lancaster University, Lancaster.
MCGOWAN, J., SAMPSON, M. (2005). Systematic reviews need systematic searchers. Journal of the Medical Library Association, 93: 74.
| 36
MAJOR, C. H., SAVIN-BADEN, M. (2010). An introduction to qualitative research synthesis. London: Routledge.
MANNING, N (2011) Chapter 8 – Conclusion. In Hannes, K., Lockwood, C (eds) Synthesizing Qualitative Research: Choosing the right approach. London: Wiley Blackwell, 161-172.
MAYS, N., POPE, C. POPAY, J. (2005). Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. Journal of Health Services Research and Policy, 10: 6-20.
MICHIE, S., FIXSEN, D., GRIMSHAW, J. M., ECCLES, M. P. (2009). Specifying and reporting complex behaviour chan-ge interventions: the need for a scientific method. Implementation Science, 4: 1-6.
MOZYGEMBA, K., HOFMANN, B., LYSDAHL, K.B., PFADENHAUER, L., VAN DER WILT, G.J., GERHARDUS, A. (2016) Gui-dance to assess socio-cultural aspects. In: LYSDAHL, K.B., MOZYGEMBA, K., BURNS, J., CHILCOTT, J.B., BRÖN-NEKE, J.B., HOFMANN, B. (eds.). Guidance for assessing effectiveness, economic aspects, ethical aspects, socio-cultural aspects and legal aspects in complex technologies [Online]. Available from: http://www.integrate-hta.eu/downloads/ [Accessed 12.12.2015].
NOBLIT, G. W., HARE, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies (Vol. 11). London: Sage.
NOYES J LEWIN S. (2011) Chapter 6: Supplemental Guidance on Selecting a Method of Qualitative Evidence Syn-thesis, and Integrating Qualitative Evidence with Cochrane Intervention Reviews. In: Noyes J, Booth A, Hannes K, Harden A, Harris J, Lewin S, Lockwood C (editors), Supplementary Guidance for Inclusion of Qualitative Research in Cochrane Systematic Reviews of Interventions. Version 1 (updated August 2011). Cochrane Collaboration Qualitative Methods Group. Available from URL http://cqrmg.cochrane.org/supple-mental-handbook-guidance [Accessed 12.12.2015].
NOYES J, GOUGH D, LEWIN S, MAYHEW A, MICHIE S, PANTOJA T, PETTICREW M, POTTIE K, REHFUESS E, SHEMILT I, SHEP-
PERD S, SOWDEN A, TUGWELL P, WELCH V. (2013) A research and development agenda for systematic reviews that ask complex questions about complex interventions. Journal of Clinical Epidemiology, 66:1262-70.
NOYES J, HANNES K, BOOTH A, HARRIS J, HARDEN A, POPAY J, PEARSON A, CARGO M, PANTOJA T on behalf of the
Cochrane Qualitative and Implementation Methods Group. (2015) Chapter 20: Qualitative research and Cochrane reviews. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Inter-ventions Version 5.3.0 (updated October 2015). The Cochrane Collaboration. Available from http://qim.cochrane.org/supplemental-handbook-guidance [Accessed 12.12.2015].
PATERSON, B., THORNE, S., CANAM, C., JILLINGS, C. (2001). Meta-Study of Qualitative Health Research. London: Sage
PATERSON, B. L. (2011). “It Looks Great but How do I know if it Fits?”: An Introduction to Meta‐Synthesis Research. In: Hannes K, Lockwood C, (eds). Synthesizing Qualitative Research: Choosing the Right Approach, 1-20.
PETTICREW, M., ROBERTS, H. (2006). Systematic Reviews in the Social Sciences: A Practical Guide. Oxford: Black-
well.
PETTICREW M, ANDERSON L, ELDER R, GRIMSHAW J, HOPKINS D, HAHN R, KRAUSE L, KRISTJANSSON E, MERCER S, SIPE
T, TUGWELL P, UEFFING E, WATERS E, WELCH V. (2013) Complex interventions and their implications for systematic reviews: a pragmatic approach. Journal of Clinical Epidemiology, 66:1209-14.
PETTICREW M, REHFUESS E, NOYES J, HIGGINS JP, MAYHEW A, PANTOJA T, SHEMILT I, SOWDEN A. (2013b) Synthesizing evidence on complex interventions: how meta-analytical, qualitative, and mixed-method approaches can contribute. Journal of Clinical Epidemiology, 66:1230-43.
PFADENHAUER, L., ROHWER, A., BURNS, J., BOOTH, A., LYSDAHL, K.B., HOFMANN, B., GERHARDUS, A., MOZYGEMBA,
K., TUMMERS, M., WAHLSTER, P., REHFUESS, E. (2016) Guidance for the Assessment of Context and Imple-mentation in Health Technology Assessments (HTA) and Systematic Reviews of Complex Interventions: The Context and Implementation of Complex Interventions (CICI) Framework [Online]. Available from: http://www.integrate-hta.eu/downloads/ [Accessed 12.12.2015].
POPAY, J., ROBERTS, H., SOWDEN, A., PETTICREW, M., ARAI, L., RODGERS, M., DUFFY, S. (2006). Guidance on the conduct of narrative synthesis in systematic reviews. A product from the ESRC methods programme. Ver-sion, 1.
37 |
POPE C, MAYS N, POPAY J. (2007) Synthesizing qualitative and quantitative health evidence: a guide to methods. Maidenhead: Open University Press.
POUND, P., CAMPBELL, R. (2015a). Exploring the feasibility of theory synthesis: A worked example in the field of health related risk-taking. Social Science & Medicine, 124: 57-65.
POUND, P., CAMPBELL, R. (2015b). Locating and applying sociological theories of risk-taking to develop public health interventions for adolescents. Health Sociology Review, 24: 64-80.
RICHARDSON, W. S., WILSON, M. C., NISHIKAWA, J., HAYWARD, R. S. (1995). The well-built clinical question: a key to evidence-based decisions. ACP Journal Club, 123: A12-3.
RING N, RITCHIE K, MANDAVA L, JEPSON R. (2010). A guide to synthesising qualitative research for researchers undertaking health technology assessments and systematic reviews. NHS Quality Improvement Scotland (NHS QIS).
ROHWER A, PFADENHAUER L, BURNS J, BRERETON L, GERHARDUS A, BOOTH A, OORTWIJN W REHFUESS E. (2016). Logic models help make sense of complexity in systematic reviews and health technology assessments. Journal of Clinical Epidemiology [Submitted Manuscript].
RUSSELL, C. K., BUNTING, S. M., GREGORY, D. M. (1997). Protective care-receiving: the active role of care-reci-pients. Journal of Advanced Nursing, 25: 532-540.
SAINI, M., 2012. Qualitative synthesis to help explore complex interventions: an evolving approach within sys-tematic reviews, presented at the Campbell Colloquium, Copenhagen, 29–31 May 2012. Available from: http://www.sfi.dk/Default.aspx?ID=11043 [Accessed 12.12.2015].
SAINI, M., SHLONSKY, A., 2012. Systematic synthesis of qualitative research. Oxford: Oxford University Press.
SANDELOWSKI, M., BARROSO, J. (2003). Towards a metasynthesis of qualitative findings on motherhood in HIV positive women. Research in Nursing & Health, 26: 153-170.
SANDELOWSKI, M., BARROSO, J. (2007). Handbook for synthesizing qualitative research. New York: Springer.
SAUL, J. E., WILLIS, C. D., BITZ, J., BEST, A. (2013). A time-responsive tool for informing policy making: rapid realist review. Implementation Science, 8: 103.
SHAW, R. (2010). Conducting literature reviews. In M. A. Forrester (Ed.), Doing Qualitative Research in Psychology: A Practical Guide. London: Sage.
SHAW, R. L. (2012). Identifying and synthesizing qualitative literature. In: Harper, D (ed. Qualitative research me-thods in mental health and psychotherapy: A guide for students and practitioners. London: Wiley-Black-well.
SNILSTVEIT, B., OLIVER, S., VOJTKOVA, M. (2012). Narrative approaches to systematic review and synthesis of evidence for international development policy and practice. Journal of Development Effectiveness, 4: 409-429.
SQUIRES JE, VALENTINE JC, GRIMSHAW JM. (2013) Systematic reviews of complex interventions: framing the review question. Journal of Clinical Epidemiology, 66:1215-22.
SWEDISH AGENCY FOR HEALTH TECHNOLOGY ASSESSMENT AND ASSESSMENT OF SOCIAL SERVICES (SBU). (2014) Evaluation and synthesis of studies using qualitative methods of analysis. Preliminary version. Stockholm: Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU); 2014. Available from: http://www.sbu.se/upload/ebm/metodbok/sbuhandbook_qualitativemethodsofanalysis.pdf [Accessed 12.12.2015].
THOMAS, J., HARDEN, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8:45.
THOMAS, J., HARDEN, A., OAKLEY, A., OLIVER, S., SUTCLIFFE, K., REES, R., KAVANAGH, J. (2004). Integrating qualita-tive research with trials in systematic reviews. BMJ: British Medical Journal, 328:1010.
| 38
TONG, A., FLEMMING, K., MCINNES, E., OLIVER, S., CRAIG, J. (2012). Enhancing transparency in reporting the syn-thesis of qualitative research: ENTREQ. BMC Medical Research Methodology, 12: 181.
TOYE F, SEERS K, ALLCOCK N, BRIGGS M, CARR E, ANDREWS J, BARKER A. (2013). A meta-ethnography of patients’ experiences of chronic non-malignant musculoskeletal pain. Health Services Delivery Research, 1:1-189.
TOYE F, SEERS K, ALLCOCK N, BRIGGS M, CARR E, BARKER K. (2014) Meta-ethnography 25 years on: challenges and insights for synthesising a large number of qualitative studies. BMC Medical Research Methodology, 14:80.
URQUHART, C. (2011). Meta-Synthesis of Research on Information Seeking Behaviour. Information Research: an International Electronic Journal, 16: n1.
VAN HOORN, R., TUMMERS, M., KIEVIT, W., VAN DER WILT, G.J. (eds.) (2016a). Guidance for the assessment of treatment moderation and patients’ preferences [Online]. Available from: http://www.integrate-hta.eu/downloads/ [Accessed 12.12.2015].
WALSH, D. DOWNE, S. (2005). Metasynthesis method for qualitative research: a literature review. Journal of Ad-vanced Nursing, 50: 204-211.
WHITAKER, R., HENDRY, M., BOOTH, A., CARTER, B., CHARLES, J., CRAINE, N., WILLIAMS, N. (2014). Intervention Now To Eliminate Repeat Unintended Pregnancy in Teenagers (INTERUPT): a systematic review of intervention effectiveness and cost-effectiveness, qualitative and realist synthesis of implementation factors and user
engagement. BMJ Open, 4: e004733.
WILSON, K., AMIR, Z. (2008). Cancer and disability benefits: a synthesis of qualitative findings on advice and support. Psycho-Oncology, 17: 421-429.
WONG S, WILCZYNSKI N, HAYNES R. (2004) Developing optimal search strategies for detecting clinically relevant qualitative studies in Medline. Studies in Health Technology and Informatics,107:311-6.
Available from: http://qim.cochrane.org/supplemental-handbook-guidance [Accessed 12.12.2015].
7 ACKNOWLEDGEMENTSWe would like to thank all those who contributed to the production of this guidance as members of the INTEG-
RATE-HTA project. In addition to the named authors the following shared in critical reading; Wija Oortwijn and
Louise Brereton. We also thank the European Union for funding this project.
Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions
1 Integrated health technology assessment for evaluating complex technologies (INTEGRATE-HTA): An introduction to the guidances
3 Guidance for assessing effectiveness, economic aspects, ethical aspects, socio-cultural aspects and legal aspects in complex technologies
4 Guidance for the assessment of treatment moderation and patients’ preferences
6 Guidance on the use of logic models in health technology assessments of complex interventions
5 Guidance for the Assessment of Context and Implementation in Health Technology Assessments (HTA) and Systematic Reviews of Complex Interventions: The Context and Implementation of Complex Interventions (CICI) Framework
2 Guidance on the integrated assessment of complex health technologies – The INTEGRATE-HTA Model
This project is co-funded by the European Union under the Seventh Framework Programme (Grant Agreement No. 306141)
8 Integrated assessment of home based palliative care with and without reinforced caregiver support: A demonstration of INTEGRATE-HTA methodological guidances – Executive Summary