Critical Appraisal and Interpretation of Qualitative … · School of Health Sciences and Social Work Screening Questions What is qualitative research? Why would we use qualitative

School of Health Sciences and Social Work

Critical Appraisal and Interpretation of Qualitative Evidence


Focus of this presentation

• Critically appraising a qualitative paper using the Critical Appraisal Skills Programme (CASP) worksheet for qualitative research


Screening Questions

What is qualitative

research?

Why would we use

qualitative

research?

What types of questions

does qualitative

research ask?

How does qualitative

research differ from

quantitative? What can qualitative

research contribute to

healthcare? Critical Appraisal Skills Programme (CASP)


What is qualitative research?

So, what is qualitative research?

Wh

at

is q

ualita

tive r

esearch

?

“Not everything that can be counted counts and not everything that counts can be counted”

PEOP

LE

Experiences

Meaning

Feelings

Words

Naturalistic

Multiple realities

OBJECTIVE

EASY SMALL SAMPLE

X Y

TOUCHY-FEELY


Life History Study

Naturalistic enquiry

Interpretative

Grounded theory

Different types of qualitative research

Different types of qualitative

research

Discourse Analysis

Ethnomethodology

Action research

Interactionism

Write and Draw

Rapid Appraisal

Content Analysis

Phenomenology

Conversational

Analysis

Participative

research

Cognitive

Anthropology

Feminist Research

Ethnoscience

Narrative

Humanistic

Research

Qualitative

Evaluation

Case Study Heuristic Research

Symbolic

Interactionism

Hermeneutics

Evaluative Research

Ethnography


Reading qualitative research

Reading qualitative research: Important questions

What is the research

question and will a

qualitative approach

answer it?

What does the research

promise to offer

healthcare?

Is the research

relevant?


Sampling

Who do we collect data from in

qualitative research?

How many people are enough?

How do we find people?

Critical Appraisal Skills Programme (CASP)


Sampling: How do we find people to take part in qualitative research?

Convenience Sampling

Purposive Sampling

Snowball Sampling

Theoretical Sampling


Sampling: How do we know we have enough people?

Have we stopped finding any novel data? (Reached data saturation)

Are there enough people to cover the range of different experiences? E.g. People of different ages, with different disease severities (provide a concrete example)




Have they used

an appropriate

sampling

strategy?

Have they described the

sample well enough for you to

decide whether this applies to

your patient(s)?

How did they decide

how many people to

sample? Was this

appropriate?

Do we know

who collected

the data?

Do we know who didn’t

take part? And if not,

why not?


Data collection

What data collection tools are used in qualitative

research?

What are their advantages/disadvantages?

How do we decide which tool to use?


Data collection

FOCUS GROUPS

Affordable and

useful to explore

group interaction,

and discuss/

debate range of

experiences. Can

be difficult to

arrange.

DIARIES

Affordable, and good

at recording how

people feel at the time.

Difficult to ensure that

the data is

appropriate.

DOCUMENTS

Affordable, but

data not

recorded with

research in

mind.

INTERVIEWS

Useful to

explore in-depth

experiences,

but expensive to

conduct

OBSERVATION

Good to record actual

rather than reported

behaviour. Misses

different perspectives.

Potential ethical

issues.

How is data collected

in qualitative

research?




Was the data collected in a

way that addressed the

research issue?

Do we have enough

detail of how and

where the data was

collected?

Are the limitations of

the chosen method

presented?

Was the method

of data

collection

justified?


Reflexivity

How might the researcher affect

qualitative research?


The importance of reflexivity

In qualitative the researcher has an influence on the data in a number of ways:

• Influence on the participant • Body language • Type of questioning • Interview schedule • Manner with participant • Existing relationship with the participant

• During data analysis: • Pre-existing ideas about what they think they will/want to

find

It is important that the researcher considers their influence on the participant and the analysis, and that they present an account of their position so readers are able to judge this.




Did the researcher’s own

experience inform the

research question?

Has the researcher

talked about who they

are and the influence

they may have on the

data?

Did the researcher

do anything to limit

their impact on the

data?


Ethical issues

What ethical issues might arise

in qualitative research?


What are the ethical issues involved in qualitative research?

• As with all research:

– If conducted within the NHS must have LREC approval.

– All participants should give informed consent

• Data protection and confidentiality issues (presentation of anonymous quotes)

• Dealing with sensitive issues:

– Effect on the participant and the researcher

• Potential to reveal issues such as abuse or neglect

• Safety of the researcher whilst collecting data outside of the institution




How was informed consent,

confidentiality and data

protection managed?

Had the researcher

considered the ethical

implications of the

research?

Has approval been

sought from an

ethics committee?


Data analysis

How is data analysed in qualitative research?

How is rigour ensured in qualitative research?


What kind of data is produced by qualitative

methods?

AUDIO

Recordings of interviews

or focus groups

These will usually be

transcribed before

analysis

VISUAL

Videos of observations

(perhaps including audio)

Drawings

Photographs

WRITTEN

Documents

Diaries

Notes from

observations


Analysing qualitative data

The aim of data analysis is to go from a

large amount of data that is in no

particular order to a nice ordered

summary of what the data is telling us.

The most important thing is to make

sure that the summary really does

reflect what participants have said –

and that is the really tricky bit!


So how is data analysed?

DISPLAY THE DATA

If necessary, transcribe the recordings from interviews or focus groups

The data is read and any field notes added

Read the whole transcription and add analytical notes

Interviewer: So, you were given a prescription by your doctor for antibiotics, can you tell me what happened next? Mary: Well, I left the doctors and I meant to go straight to the chemist to put in my prescription. But I really was feeling very ill and it is a long way to the chemists. I only live about 2 minutes walk from the doctors so I went home instead. After I got home I started to feel even worse and I didn’t have anybody to ask to go and pick up my prescription for me. By the time I felt well enough to get to the chemist’s I thought it was probably not worth it anymore.

She looked

like she felt a

bit guilty

about this.

Followed this

up later on.

Intention?

Does she

mean this or

does she

think she has

to say it?



DATA REDUCTION

Identify significant statements/units of meaning

Interviewer: So, you were given a prescription by your doctor for antibiotics, can you tell me what happened next? Mary: Well, I left the doctors and //I meant to go straight to the chemist to put in my prescription.// But I really was feeling very ill and it is a long way to the chemists. I only live about 2 minutes walk from the doctors so I went home instead. //After I got home I started to feel even worse// and I didn’t have anybody to ask to go and pick up my prescription for me. //By the time I felt well enough to get to the chemist’s I thought it was probably not worth it anymore.//

She looked

like she felt a

bit guilty

about this.

Followed this

up later on.

Intention?

Does she mean

this or does

she think she

has to say it?



DATA INTERPRETATION

Group into categories

Develop into themes

Check representativeness of these from your transcripts

Find examples of these

Display

Qualitative research and analysis represents an inductive approach to research where we start with real-life data and work towards abstract ideas or general principles.


What do qualitative findings look like?

Qualitative data analysis often produces themes:

–Themes are used to describe the data in a concise and easy to understand manner

–Themes should be used to express the experiences of all participants


SYMPTOM EXPERIENCE

FEAR

BLAME AND STIGMA

Patients had not

realised that their

symptoms were

indicative of

anything serious

Patients delayed

going to the doctor

because they were

scared of finding out

that they had cancer

or were scared of

the treatments.

KNOWLEDGE

Patients did not

have much

knowledge about

lung cancer and its

symptoms

Patients were worried

that they would be

blamed or

stigmatised if they

were diagnosed with

lung cancer.

CULTURE

Some patients,

especially males,

felt the need to be

stoic about their

illness

Tod, Craven & Allmark (2008), looked at the reasons for diagnostic delay in patients with lung cancer and found the following themes.


Dependability and confirmability

How do we check the dependability and confirmability of the reported

findings?

• Can we follow the decision trail?

• Is there an audit trail?

• Have I been given enough information about the methods used?

• Is there enough detailed description for us to interpret the meaning and context of what is being researched?

Can we trust the analysis?

How careful was the

researcher to prevent bias

affecting their analysis?




Is there an in-

depth description

of the analysis

process?

Is it clear how the

themes were derived

from the data?

Do you know how the

presented quotes

were chosen?

Is there enough data to

support the findings?

Does the researcher

consider their influence

on the analysis and

presentation of

findings?


Findings

How should qualitative findings be presented?


Credibility

Are the findings based on the data? Or on what

the researcher expected to

see?

• Did the researcher have prolonged engagement

with the participants or phenomenon?

• Did the researcher attempt to triangulate their

findings?

• Did the researcher discuss their data and

interpretations with a peer(s)?

• Did the researcher check for negative cases?

• Did the researchers send the findings to

participants to ask their opinion?

• Can you see any actual data (quotes)?

• Do you know why they reported those particular

quotes?

• Do the author’s arguments make sense?

How do we check the credibility of the

reported findings?

Do we believe the findings?




Does the researcher

present data that

contradicts their

arguments?

Are the

findings

explicit?

Has the researcher

discussed

credibility?

Are the findings

related back to the

original question?


Value of the research

What makes for valuable qualitative research?


Can you tell if the findings are

transferable to your practice? • Is this relevant to our practice?

• Is there a description of the sample so you can decide it is relevant?

• Are there explanations as to what is going on in a particular social location/operation of a set of social processes?

• Does it provide comparisons that are relevant in “supporting your argument”


Critical appraisal

• Decide how trustworthy a

piece of research is (validity

and reliability)

• Determine what it is telling us

(results)

• Weigh up how useful the

research will be(relevance)

Critical appraisal helps the reader of research ………...