Qualitative Analysis Educational Skills Workshop 2011-2012 Arianne Teherani, PhD Bridget O’Brien, PhD.

Qualitative Analysis

Educational Skills Workshop

2011-2012

Arianne Teherani, PhD

Bridget O’Brien, PhD

Creative Commons License

Attribution-NonCommercial-Share Alike 3.0 Unported

You are free:• to Share — to copy, distribute and transmit the work • to Remix — to adapt the work Under the following conditions:• Attribution. You must give the original authors credit (but not in any

way that suggests that they endorse you or your use of the work).• Noncommercial. You may not use this work for commercial purposes. • Share Alike. If you alter, transform, or build upon this work, you may

distribute the resulting work only under a license identical to this one.

See http://creativecommons.org/licenses/by-nc-sa/3.0/ for full license.

Overview

Evaluating qualitative research

Overview of qualitative research

Qualitative data collection methods

Analyzing qualitative data

Combining qualitative and quantitative

Exercise 1: Judging Qualitative Research (Analyses)Do you have a clear sense of the context of the

study and the role of the researcher(s)?

How well do you feel the methodology used in this study captured the research question? Explain some of the strengths and/or limitations.

Are there sufficient examples / paraphrasing to support the findings / argument?

Has the author convinced you that his/her interpretation is plausible? How?

Has the author discussed alternate interpretations?

Qualitative Research

Research about lives, behavior, organizational functioning, interactional relationships

Exploratory

Open-ended

Data = Words

Analytic procedures are interpretivist and concurrent with and post data collection

Research

Multiple Purposes of Qualitative Methods

Curriculum Development

Instrument Development

Evaluation

Qualitative Research Questions

• What are the qualities of an ideal mentor?

• How are the professional identities of students shaped during clerkship X?

• How do clerkship students characterize the struggles they face when transitioning to the clerkships? How do clinical teachers characterize students’ struggles?

• How does participation in a teamwork & communication skills curriculum improve small group process in Year 1 of medical school?

Qualitative Paradigm

Sets of beliefs that guide action

Qualitative research takes place within a vast spectrum of paradigms:

–Positivism

–Post-positivism

–Constructivism

–Critical Theory

–Post-modernismSee: Bunniss & Kelly. Research paradigms in medical

education. Medical Education, 2010, 44: 358-366.

Qualitative Approaches

Explore social processes through interpretation of data

Three Most Common in Medical Education:

• Ethnography

• Grounded theory

• Case study

• Phenomenology

• Hermeneutics

• Narrative research

• Action research

Sampling

• Purposeful

– Seeking the best sources of information about the phenomenon of interest

• Theoretical

– Seeking the best sources of information to confirm / disconfirm your developing explanatory model

“Participants are not recruited on a representative basis, but rather because of their expert knowledge of the phenomenon under inquiry” (Green & Thorogood, 2005)

Qualitative Data Collection

Observation

Interview

Focus groups

Open-ended prompts

Artifacts (documents, photos)

Observation

Spend time with a group of people as they carry out their daily activities, to understand their way of life and how they make sense of the world in which they live.

– To understand experiences of group members, their activities, interactions, discussions

– To understand how contextual factors influence the activities and decisions of group members

Observation

Types of observation

– Field Notes

– Sketches

– Relation diagrams

Continuum of roles that range from complete participation to complete observation

Type of role adopted depends on research question

Interviews

Types of Interviews

– Structured—verbal questionnaire, scripted questions

– Semi-structured—outline of topics guides questions

– Informal--conversational, questions arise in context

– Retrospective—focuses on past events

Focus Group

• Small groups with moderator

• Concentrated data on topic of interest in short time span

• Group interview with insight into interactions on topic (compared to interviews)

• Important that topic of interest would be easy for participants to discuss in group

Data Collection Decisions

Methods

Types of Data

Focus groups

Interviews

Open ended prompts

Observations Collect Artifacts

How Interactive? Researcher’s Role?

Audio Recording

Notes

Transcripts

Written response

Typed response

Notes

Video

Documents

Photos / Visual

Video

Example: How does participation in a teamwork & communication skills curriculum improve small group process in Year 1 of medical school?

• Observation of small groups pre & post

• Interviews with small group leaders (retrospective comparison)

• In-depth interviews with students

• Focus group with students

• Written evaluations of group process by students and faculty

Exercise 2: Analysis

A researcher conducted 5 focus groups at different medical schools. Each group had 3-7 third-year medical students and the students were at least 2 months into the clerkship year. The focus groups were recorded and transcribed. This researcher approaches you for help coding the focus group data and developing a report of the findings. You will work in groups. Read the passages individually and in your group, start the initial analysis discussion.

Please document the steps you take to analyze the data and use quote/paraphrases to support your argument.

Qualitative Data Analysis

Two purposes

To identify and explain themes

To pull loosely related themes together in order to understand and explain relationships

Coding: Step 1

The process of breaking down, examining, comparing, conceptualizing, and categorizing data.

1. Conceptualize and label

2. Categorize

3. Name/define your category

4. Validate against examples

Example Coding Sheet

Coding: Step 2

Putting the data back together in new ways by making connections between categories and sub-categories.

– Link sub-categories to categories

– Define set of conditions under which categories work

– Verify statements against data

– Define differences/similarities between categories

– Propose and track relationships against data

Qualitative Analysis Continuum

Content Analysis

(quasi-qual)

Qualitative Content Analysis

Thematic Analysis

Grounded Theory

Data analysis

Count the frequency of

words or phrases

Generate a list of categories – a coding scheme

Code, organize categories into

constructs / themes

Collect, code, analyze data

simultaneously

Constant Comparative

Reporting Frequencies Frequencies Discuss themes (sometimes

counts)

Describe a model to explain a

phenomena

Rigor Inter-rater reliability

Multiple coders; Check for

consistency & reconcile

differences

Multiple coders

Review codes & themes, revise, &

reconcile differences

Seek out confirming / disconfirming info

Multiple coders & reviewers

Analysis Considerations

• Techniques to ensure rigor (credibility or trustworthiness)

• Interrater agreement

•Triangulation of data - cross checking

•Triangulation of sources – member checking

•Transparency

•Reproducibility (agreement over time, events or settings)

Exercise 3: Writing up Findings

You have finished analyzing the focus group data you collected on students’ struggles in the third year. Now you are ready to write up your findings. Please practice writing up one of your findings, making sure to consider the following:

– How will you organize the results section?

– How will you provide evidence to support your findings?

– Are there any techniques you might use to enhance trustworthiness or credibility of your results?

– What limitations would be important to mention?

Tools for Coding & Data Management

Demo of Nvivo

Other options: Atlas.ti, HyperRESEARCH, SPSS text analysis

Teaching Material References

Dr. Carol Kamin, UIC, Department of Medical Education

Lofland & Lofland. Analyzing social settings. Rubin, H.J. & Rubin, I.S. (1995). Qualitative

interviewing: The art of hearing data. Thousand Oaks, CA: Sage Publications

Strauss, A. & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park: Sage Publications

Cote, L., & Turgeon, J. (2005). Appraising qualitative research articles in medicine and medical education. Medical Teacher, 27(1), 71-75.

Also, bibliography will be emailed