Working with Qualitative Data Christine Maidl Pribbenow Wisconsin Center for Education Research [email protected]
Dec 27, 2015
Working with Qualitative Data
Christine Maidl PribbenowWisconsin Center for Education Research
Session Outline
• General discussion about educational research, assumptions and misconceptions
• Contrast educational research with research in the sciences
• Define common qualitative analysis terms• Provide example using ATLAS.ti–
qualitative analysis software program • Code some text
Qualitative Data: Oxymoron or inherent tensions?
• Hard vs. soft (mushy)
• Rigor
• Validity and reliability
• Objective vs. subjective
• Numbers vs. text
• What is The Truth?
What are some of the assumptions that you have about educational
research?
How are they helping or hindering the development of your study?
Research in the sciences vs. research in education
• “Soft” knowledge• Findings based in specific
contexts• Difficult to replicate• Cannot make causal claims
due to willful human action• Short-term effort of
intellectual accumulation– “village huts”
• Oriented toward practical application in specific contexts
• “Hard” knowledge• Produce findings that are
replicable • Validated and accepted as
definitive (i.e., what we know)
• Knowledge builds upon itself– “skyscrapers of knowledge”
• Oriented toward the construction and refinement of theory
Strongly Agree
Agree Unsure Disagree Strongly Disagree
Educational research is rigorous. 4 (31 %) 8 (62 %) 1 (8 %) 0 (0 %) 0 (0 %)
I have read at least ten articles
published in educational research
journals before attending this Institute.
11 (85 %) 2 (15 %) 0 (0 %) 0 (0 %) 0 (0 %)
Educational research is more difficult
than my scientific research.
1 (8 %) 2 (15 %) 6 (46 %) 3 (23 %) 1 (8 %)
I regularly collect qualitative data
in my classes for assessment purposes.
1 (8 %) 5 (38 %) 2 (15 %) 4 (31 %) 1 (8 %)
I need a control or comparison group
to conduct educational research.
2 (15 %) 3 (23 %) 1 (8 %) 5 (38 %) 2 (15 %)
Assessment data gleaned from students
(i.e., "self report") are valuable.
2 (15 %) 9 (69 %) 2 (15 %) 0 (0 %) 0 (0 %)
I have analyzed qualitative data in the past.
1 (8 %) 3 (23 %) 1 (8 %) 4 (31 %) 4 (31 %)
Strongly Agree
Agree Unsure Disagree Strongly Disagree
Qualitative data can meet "reliability" standards.
2 (15 %) 5 (38 %) 6 (46 %) 0 (0 %) 0 (0 %)
Qualitative data can meet "validity" standards.
2 (15 %) 5 (38 %) 6 (46 %) 0 (0 %) 0 (0 %)
If I collect learning assessment data from my students and the analyzed results are "not significant" it proves that students did not learn what I intended.
0 (0 %) 0 (0 %) 1 (8 %) 5 (38 %) 7 (54 %)
If I conduct classroom research and the results are "not significant",
the study was a waste of my time.
0 (0 %) 0 (0 %) 1 (8 %) 3 (23 %) 9 (69 %)
I need human subjects approval to
conduct and publish research
about my students.
7 (54 %) 1 (8 %) 5 (38 %) 0 (0 %) 0 (0 %)
I want to conduct research
in my classroom so that I can teach better.
11 (85 %) 2 (15 %) 0 (0 %) 0 (0 %) 0 (0 %)
I want to conduct research in my classroom so that my students learn more or better.
13 (100 %) 0 (0 %) 0 (0 %) 0 (0 %) 0 (0 %)
What are some sources of qualitative data?
• Lab notebooks
• Open-ended exam questions
• Papers
• Journal entries
• On-line discussions
• Notes from observations
Qualitative Data Analysis
Qualitative analysis is the
“interplay between researchers and data.”
Researcher and analysis are
“inextricably linked.”
Qualitative Data Analysis
• Inductive process– Grounded Theory
• Unsure of what you’re looking for, what you’ll find• No assumptions• No literature review at the beginning• Constant comparative method
• Deductive process– Theory driven
• Know the categories or themes using rubric, taxonomy• Looking for confirming and disconfirming evidence• Question and analysis informed by the literature
Example Research Questions
Why do faculty leave UW-Madison?
Do UW-Madison faculty leave
due to climate issues?
Definitions: Coding and Themes
• Coding process: – Conceptualizing, reducing, elaborating and
relating text– words, phrases, sentences, paragraphs.
• Building themes:– Codes are categorized thematically to describe
or explain phenomenon.
Let’s Code #1
Read through the reflection paper written by the student from an Ecology class and highlight words, parts of sentences, and/or whole sentences with some “code” attached and identified to those sections.
Let’s Code #2
Read through this reflection paper and code based on this question:
What were the student’s assumptions or misconceptions before taking this course?
Let’s Code #3
Read through this reflection paper and code based on this question:
What did the student learn in the course?
Can we say that the students learned something in the course
using reflection papers?
Why or why not?
Ensuring “validity” and “reliability” in your research
• Use mixed methods, multiple sources.• Triangulate your data whenever possible.• Ask others to review your design methodology,
observations, data, analysis, and interpretations (e.g., inter-rater reliability).
• Rely on your study participants to “member check” your findings.
• Note limitations of your study whenever possible.
C oncept m aps o f contentfound in journa l a rticles (Both)
P re-post examof concepts (Q uantita tive)
R eflection Paper(Q ualita tive)
3 Sourcesof D ata
Does the redesign of an ecology course to include concept maps derived from
current journal articles help students to gain a more current and realistic view of
ecological issues?