Question-led mixed methods research synthesis

Question-led mixed methods research synthesis

Centre launch 21 June 2005

David Gough and Sandy Oliver

Institute of Education, University of London

Methods for Research Synthesis Programme

• David Gough• Sandy Oliver• Angela Harden• James Thomas• Rebecca Rees• Adam Fletcher• Lisa Underwood• Ann Oakley

• Building on more than ten years’ experience of synthesising policy relevant research in health, education and social welfare

Professional Experience &

Expertise

Political Judgement

Resources

ValuesHabits & Tradition

Lobbyists & Pressure Groups

Pragmatics & Contingencies

ResearchEvidence

Adapted from Philip Davies, 2005

DIFFERENT TYPES OF EVIDENCE

Research production and evidence Evidence influenced by the context of its production• All the activities of a research commissioning or

granting agency (including training)• All the activities of a research production facility

(e.g. institute, department, university)• The activities of a potential research user

organization and its staff• The entire ‘research regime’ in a country

Adapted from Jonathan Lomas, 2005

Research methodologyIn question-led synthesis, methodology is• A system of principles, practices, and

procedures applied to a specific branch of knowledge.

• A set of procedures or methods used to conduct research.

• For asking the questions as well as findings the answers

Questions and methodsChoosing questions• Gathering questions

– Policy makers– Practitioners– Service users– Members of society– Academics

• Setting priorities– Consensus development

methods

• Framing questions– (un)answerable research

questions

Finding answers• Choosing appropriate

research designs• Choosing methods for

finding, appraising and synthesising the findings of research reports

How questions vary• Policy makers• Practitioners• Service users• Members of

society• Academics

• Collective questions

• Population wide? Costs?• Practical, delivery issues?• Personally relevant?• Ethical?

• ConceptualPhilosophical? Answerable?

• ???

My question is, are we having an impact?

User question led synthesis• What do we want to know (and who wants

to know and why)?• What do we know already (and how do we

know it)?• What more do we need to know (and how

can we know it)?• Can not be value free• Involves intellectual work

Older people, accidents and injuries

Population• People over 55• Older patients• Older people at less risk

Interventions • to prevent injury• to reduce falls & frailty• to prevent falls & injury• to prevent accidents• Exercise/ balance training• Medical interventions

Outcomes• Knowledge• Falls or fall related injury• Strength, balance, gait, sway

Design• Controlled trials• RCTs• Quality assessed RCTs

Questions do make a difference

6 reviews of older people and accident prevention

Total studies included Total studies included 137 137Common to at least two reviews 33Common to at least two reviews 33Common to all six reviews 2Common to all six reviews 2Treated consistently in all reviews 1Treated consistently in all reviews 1

Oliver et al. 1999Oliver et al. 1999

INTERPRETATION AND APPLICATION

WHAT DO YOU WANT TO KNOW?

WHAT HAS BEEN DONE?

COMMUNICATION OF KNOWLEDGE

PERSPECTIVES AND PARTICIPATION

RESEARCH STUDIES AND METHODS

WHAT IS KNOWN?(WITHIN THOSE PERSPECTIVES AND THAT FORM OF KNOWLEDGE)

TYPES OF RESEARCH EVIDENCEWhat more needs

to be known?

RESEARCH QUESTIONS

TYPES OF KNOWLEDGE

PERSPECTIVES AND PARTICIPATION

Interpretation and application

Communication

What do we know? How do we know it?What don’t we know?

How could we know it?

What has been done?

What do we want to

know?

TYPES OF KNOWLEDGE

RESEARCH QUESTIONS

RESEARCH STUDIES AND

METHODS

RESEARCH EVIDENCE

(iv) Policy community

(i) Research

(iii) Practitioner

(v) Organisational

(ii) User of service/ public

How/processes?

Nature/extent/frequency?

Effects?

Perspectives/ concepts?

Multi method

Action research

Case study

Survey

Experimental

How/processes?

Nature/extent/frequency?

Effects?

Perspectives/ concepts?

Need for synthesis• Brings together what we know – whatever we

are studying• Contextualizes information from new studies• Involves explicit systematic methods and thus

transparency• These may be lacking in non systematic reviews

and expert opinion however excellent

Systematic review: key features

• Synthesises the results of primary research

• Uses explicit and transparent method

• A piece of research, following standard set of stages

• Accountable, replicable, updateable

• Need for user involvement

Question led synthesis

• Questions looking for answers

• Make implicit assumptions explicit

• All types of question so all types of research design

• Statistical, narrative empirical and conceptual synthesis

• Mixed methods synthesis

Dimensions of difference in synthesis models

• Review/research questions (impact, process, need, explanatory concepts)

• Research designs considered relevant • Types of data - numerical or textual• Quality assessment of different designs• Breadth of question and research design• Quantitative/narrative empirical/conceptual

synthesis• Variation in contribution of each study to the

systematic synthesis

Review questione.g. What is known about the barriers to, and facilitators of, fruit

and vegetable intake amongst children aged 4 to 10 years?

Trials1. Application of inclusion criteria

2. Quality assessment3. Data extraction

4. Statistical meta-analysis

‘Qualitative’ studies1. Application of inclusion criteria

2. Quality assessment3. Data extraction

4. Qualitative synthesis

Trials and ‘views’Mixed methods synthesis

Mixed methods synthesis

Findings for synthesis 1: ‘Quantitative’ (Trials)

Findings of synthesis 2: Descriptive themes

Influences on

foods eaten

Chosen foods

Provided foods

Foods in the home

Foods in the school

Parental influence and food rules Breaking rules

Limited choiceSocial occasionContradictions

Food preferencesPerceptions of health benefitsKnowledge-behaviour gapRoles and responsibilitiesNon-influencing factors

Concepts of healthy eating

Healthy eating conceptsGood and bad foodsHealth consequences

Developing a framework• Typology of research questions in the social

science literature• Methods of synthesis for answering such

questions• Matrix of synthesis methods and methodological

development and challenges• Creating solutions with others• An emerging framework

Contacts and informationNRCM MRS webhttp://www.ncrm.ac.uk/nodes/mrs/people.php

SSRU web: http://www.ioe.ac.uk/ssru/SSRU's EPPI web: http://eppi.ioe.ac.uk

Email: [email protected] [email protected]