What Can Qualitative Software Do for My Research? November 5, 2013 APHA Conference.

What Can Qualitative Software Do for My Research?

November 5, 2013

APHA Conference

Abt Associates | pg 2

Outline of Session

Overall Goal: Understand how qualitative data analysis software can improve the rigor of your public health research

Short introduction to qualitative analysis and computer-assisted qualitative data analysis software (CAQDAS)

Introduction to coding and coding exercise

NVivo demonstration and exploration


A Brief History of CAQDAS

1981: Lyn and Tom Richards develop NUD*IST, the precursor to NVivo

1994: Miles and Huberman discuss the use of software in qualitative analysis in their widely cited text

2007: National Science Foundation publishes guidelines for the use of software in qualitative data analysis

2013: NVivo, AtlasTi, EZ-Text, ANSWR, MaxQDA, HyperResearch and Dedoose are among the most commonly used tools today

Today at APHA 2013: Over 30 presentations mention using NVivo in their abstracts (see handout)


How can Software Help Improve the Rigor of Qualitative Analysis?

GATHER

Existing Research

Current Information

Data Sources

ORGANIZE

Data Sources

Coding Structure

Demographics

Literature Review

ANALYZECoding

Queries

Memos

Visuals

VISUALIZE RESULTSReports

Journal Articles

Dissertations

Study PresentationsX Y

Z

• Transparency• Saturation

Methodology


Considerations in Choosing to Use Software

Sample size and multiplicity of data sources

Emphasis on replicability, rigor and transparency

Likelihood that there will be future opportunities to perform secondary analyses on the same dataset

Desire to publish in peer-reviewed journals

Interest in merging close-ended attributes into the qualitative dataset

Building capacity of analysis team including training time and costs

Budgetary parameters and software investment


Promoting Reliability and Validity in Analysis Document the process of analysis including what is the

statement of a respondent and what is interpretation by a

researcher

Involve multiple analysts to check biases

Document in detail the process by which analytical themes or

codes are developed

Train coders or analysts on coding structure and create well-

defined themes. Refine again and again.

In analysis, check inter-rater reliability

Develop conventions for transcribing data so that transcripts

are comparable across data source

Develop saturation guidelines


Future Directions in Qualitative Analysis

Mixed methods tools, such as the capacity to work with datasets containing both fixed response and open-ended material

Web-based data, including social media and online discussion boards

Capacity for larger samples, especially large quantities of text (qualitative studies are no longer small)

More tools for comparing coding by researcher, theme, and participant group


A Future Look at Mixed Methods with NVivo

Descriptive Statistics Inferential Statistics Meta-Analysis

Coding of TextAudio, Video andImage Data Open-Ended Survey and Interview Responses Transcribing tools for Audio and Video

Import/Export from Excel, text and database files Open-Ended and Fixed Response Questions Within and Between Group Analysis of Coding Text Analysis Kappa Coefficient Cluster Analysis of Word and Coding Similarity

Qualitative QuantitativeNVivo


Coding Exercise


Brief Coding Exercise: Purpose

We will code a short transcript manually

– Given the short timeframe we will not do this thoroughly

We will talk about what we found and how we might begin to refine our codes

We will discuss how we would use qualitative software to accomplish the same steps


Coding

Any researcher who wishes to become proficient at doing qualitative analysis must learn to code well and easily. The excellence of the research rests in large part on the excellence of the coding.

(Anselm L. Strauss, Qualitative Analysis for Social Scientists, 1987, p. 27)


What is Coding?

Codes are short words or phrases that symbolize the essence of a piece of text, visual image, or other qualitative data.

Codes reduce a large quantity of data into more manageable “themes.”

Interpret qualitative data into meaningful themes (meaningful depends on the lens of the analyst)


EXAMPLE: Focus Group on Treating Chronic Fatigue Patients

Physician Participant

“There are also trends over times. When I was in training, everybody who we now consider chronic fatigue or even chronic fibromyalgia was largely looked into a group that they called the hypochondriacal patients. Now you hardly ever hear the diagnosis hypochondriasis anymore.”

Code Trends in Diagnoses OR Physician training OR Chronic fatigue syndrome

familiarity


Coding process

Initial codes will be defined, redefined, collapsed as more and more data are coded

Analysts will develop definitions, inclusion and exclusion criteria for each code


Analysis of Codes/Themes

Codes/themes are analyzed for patterns, e.g.

– frequency,

– similarity and differences across respondent types,

– meaning,

– sequence,

– associations with other codes,

– causation, etc.


Brief Coding Exercise - Instructions

Read the interview with Thomas – 10 minutes

Remember the purpose of the research. (Description in packet.)

Look for themes. Put parentheses around the text and write the word/phrase that summarizes the text next to it.

Note questions or ideas that occur to you as you read the interview.

Report out and discussion – 10 minutes


Brief Coding Exercise – Themes

What themes did you come up with?


Brief Coding Exercise – Discussion

What codes are similar to each other?

How will we capture the changing definitions over time?

How will we merge codes?

How will we assess inter-rater reliability?

What questions or comments occurred to you as you read the interview?


Brief Coding Exercise – Summary

What works manually with a small amount of qualitative research becomes more complicated as the number of qualitative sources increase

Defining, redefining, merging, and separating codes is easier to do and easier to keep track of with qualitative software

Documenting the analysis process systematically is a benefit of the software; replicability is possible


NVivo Demonstration

and Exploration


Let’s explore NVivo in practice

How does NVivo store data? Sources may be text, audio, video, pictures, categorical or social media data

How do I code in NVivo? Select data and assign to one or more Nodes

How can I see my coding in NVivo? Turn on Coding Stripes, and open Nodes

Isn’t there more the software can do?

Text Search and Word Frequency Queries can help you code.

Matrix Coding Queries can reveal patterns within and across themes or groups of participants.


Systematic, Rigorous, Quick

Increase accessibility of data and transparency of analysis

o Node content, memos, annotations, coding stripes, event log

Identify and test ideas about emerging patterns and themes

o Text Search Query, Matrix Coding Query

Utilize open-ended text and non-text data

o Transcripts, audio, video, pictures, survey and other categorical data, social media

Generate output for reporting

o Query results, node exports, visualizations

Support team-based analysis

o View team members’ coding, calculate Kappa coefficient


Utilizing Non-text Data

Classification (First Cell)

Person Gender Age Range Years in Down East

Primary Residence

Barbara Female 40-49 40 Yes

Charles Male 60-69 72 Yes

Dorthy Female 20-39 40 Yes

Helen Male 70-79 N/A Yes

Attributes (First Row)

Nodes (First Column)


Questions and Comments?


Contact Information

Lisa LeRoy, MBA, PhD

[email protected]

617-349-2723

Ilana Ron, MSc

[email protected]

301-347-5339

Cynthia Jacobs, EdD

[email protected]

617-491-1850

mailto:[email protected]



What Can Qualitative Software Do for My Research? November 5, 2013 APHA Conference.

Documents

qualitative software

rigor of qualitative

abt associates pg

process of analysis

analysis document

transcribing data

software investment

use of software