TOPIC 7ACTION RESEARCHData Collection Methods
SYNOPSISTopic 7 has been designed to explore the data collection
methods that can typically be used during an action research
project. Participants will be expected to evaluate and consider how
to employ data collection methods effectively in their action
research investigation.
LEARNING OUTCOMES By the end of this topic, you will be able to:
Identify the types of data collection instruments that can be used
in action research Understand strengths and limitations of data
collection instruments in action research Become informed about how
to design and create effective data collection instruments
fRAMEWORK OF TOPICS
Preview (30 min) Work with a partner fill in table one Discuss
table one with another group Discuss table one as a whole class You
may compare answers with the Data Collection Methods example
Table 1: Data collection methodsMethodAdvantageDisadvantageWhat
I want to know/learn about this method
Survey
Interview
Document Analysis
Observation
Data Collection MethodsMethodAdvantagesDisadvantages
Interviews
-Flexibility as the researcher can observe the participant and
total situation in which they are responding.
-More participants are willing to talk and react verbally than
to write responses to questions.
-Face-to-face interviews establish rapport and motivation for
participants.
-Useful method to gather extensive data on a small number of
complex topics.
-The researcher can control the sequence of questions so the
respondent cannot predict the next question.
-More expensive and time consuming than surveys.
-Only a limited number of participants may be interviewed due to
time and financial considerations.
-Finding skilled and trained interviewers with appropriate
interpersonal skills may be difficult.
-An interviewer effect may result from interaction between the
interviewer and participant.
-Participants may feel that they must answer on-the- spot.
-Flexibility afforded by unstructured interviews may generate
difficulties when attempts are made to categorise and evaluate
responses.
Document Analysis
-More accurate and reliable data than a survey.
-Useful when the researcher is specifically interested in
precise estimates of different kinds of behaviour.-Participants may
become less proficient over time in their record keeping.
-Cognitive dissonance influences a participants recollection of
events and can lead to framing of responses.
Observations
-Can record both simple and complex behaviour.
-Can begin to theorize and draw abstractions to validate
observations.
-Researcher may use a tally sheet to categorize reoccurring
behaviour according to criteria sheet.
-Cost effective and easy to implement.-Highly subjective.
-Not all observations are equal or expert.
-Difficult to evaluate the veracity of the observation.
-Variables are ever present and that means the observed
behaviour may be impossible to replicate.
-Observations can easily become dominated by researcher
bias.
-The researcher decides what phenomena to observe and what to
ignore.
Survey -Cost effective and easy to implement
-Useful to collect data on specific populations.
-Tends to give respondent the privacy to answer truthfully.
-Standardized questions make evaluation of answers more precise
by forcing uniform definitions on participants.
-Researcher may compare responses to search for emergent trends
across participants.
-Researcher may use the survey to collect data remotely by mail,
internet or email.
-Respondents can remain anonymous. -The researcher questions may
contain bias and can potentially lead the participant to desired
answers.
-Participants may not tell the truth over controversial question
or question that cast the participant in an unfavourable light.
-Not all surveys are equal or expert.
-Very little control over the context the survey is taken. The
lack of context can make it difficult to ascertain the reasoning
behind participant responses.
-Open-ended questions generate large amounts of data that are
time consuming to analyse.
-Participants may answer superficially to complete the task.
OBSERVATION Observation is the systematic description of events,
behaviours, and artefacts in the social setting chosen for study.
Observations enable the researcher to describe existing situations
using the five senses, providing a "written photograph" of the
situation under study. Fieldwork involves active looking, improving
memory, informal interviewing, writing detailed field notes, and
perhaps most importantly, patience. Observation methods are useful
to researchers in a variety of ways. They provide researchers with
ways to check for nonverbal expression of feelings, determine who
interacts with whom, grasp how participants communicate with each
other, and check for how much time is spent on various activities.
Observation is the technique of obtaining data through direct
contact with a persons or group of persons.[footnoteRef:1] Since,
the main focus of qualitative research is naturalism, the
researcher has to observe person or persons in their natural state
as undisturbed as possible. The role of the researcher may be
viewed as a continuum (se Figure 2.1). On one extreme, the
researcher is a passive observer and on the other extreme the
researcher is a participant observer. In between these two
extremes, the researcher may be an active observer (Potter, 1996).
[1: www.peoplelearn.homestead.com/Chapter2.Methods.QR.doc]
Passive Observer Active Observer Participant Observer Figure 2.1
Continuum of Observation Techniques
Passive observer: The best way to be not involved and keep
distance from your subjects is to be a passive observer. As a
passive observer, you simply gather documents and observe the
individual or individuals without doing anything to disturb the
situation. The researcher is unobtrusive and watches the group from
the outside; i.e. the ethic or outsiders perspective. To do so, the
researcher must gain access and be accepted by the individual or
individuals being observed. For example, in collecting e-mails or
essays written by subjects or learning journals of students, the
researcher examines them without being involved. Similarly, when a
researcher interested in studying children interacting in school
canteens or the playground, merely observes them without being
involved. A certain amount of distance is maintained between the
researcher and the person or persons being observed.
Participant Observation: As the name participant suggests, the
researcher participates in the activities of the persons being
observed rather than being an observer. The researcher has two role
as observer and as participant. The researcher participates as much
as possible in the daily life of the subjects while also carefully
observing everything he or she can about it. Through this, the
researcher is seeking to gain what is called an emic perspective or
the natives point of view or the insiders perspective. The
researcher records detailed field notes, conduct interview based on
open-ended questions and gather whatever site documents might be
available in the setting as data. Participation can take many
forms. For example, the researcher could show a film or video to
stimulate discussion or question subjects and observe how they
would react to the stimulus. The researcher takes an active
position with the purpose of stimulating subjects to think about
things they might never have thought about before. However, as
pointed out by Hammersley and Atkinson (1983), there is the danger
of the researcher going native which means being too involved or
having too close a rapport with the person or persons being
observed to the extent that you loose objectivity.Participant
observation Observer participates actively, for an extended period
of time May require observer to live or work in that area Assumes
that observer will become accepted member of the group or
community
Almost any setting in which people have complex interactions
with each other, with objects, or with their physical environment
can be usefully examined through participant observation. Since
doing participant observation means being embedded in the action
and context of a social setting (going native), we consider three
key elements of a participant observation study:
1. Getting into the location of whatever aspect of the human
experience you wish to study. This means going to where the action
ispeoples communities, homes, workplaces, recreational sites,
places of commercial interaction, sacred sites, and the like.
Participant observation is almost always conducted in situ.
2. Building rapport with the participants. The point of
participant observation is that you wish to observe and learn about
the things people do in the normal course of their lives. That
means they have to accept you, to some extent, as someone they can
be themselves in front of. While you dont necessarily have to be
viewed as a complete insider, a successful participant observer has
to inspire enough trust and acceptance to enable her research
participants to act much as they would if the researcher were not
present.
Active Observer: Between being a passive observer and an active
participant, the researcher could take a middle position of being
an active observer. Here, participation is allowed but limited. The
researcher may intrude into the lives of subjects such as entering
their homes or their communities but remains passive once inside
the environment so as not to influence the natural occurring
behaviours and conversations. For example, a researcher interested
in TV viewing habits may enter a household, eat with the family,
play with the children and take part in family activities. Family
members are told not to change their routines in order to
accommodate the observer. However, the researcher tries as far as
possible to be passive, saying as little as possible so as not to
influence the behaviours and conversations of subjects.
Observer An observer is under the bed. A participant observer is
in it. John Whiting Observer is an eavesdropper Someone who
attempts to observe people without interacting with them With or
without their knowledge that they are being observed[footnoteRef:2]
[2: http://www.sagepub.com/upm-data/48454_ch_3.pdf SAGE chapter 3
December 2012.]
Direct observation is primarily a quantitative technique in
which the observer is explicitly counting the frequency and/or
intensity of specific behaviors or events or mapping the social
composition and action of a particular scene. While most direct
observation data collection is conducted by actual observers, many
direct observation studies do not technically require a human data
collector. The data captured in direct observation are, by
definition, those that can be observed and do not inherently
require any interaction (talk or discussion) between the observer
and those being studied. Direct observation is about observable
behavior and is typically associated with research objectives that
require some sort of ordinal data or purely factual description:
how often, how many, how intensely, who was there, and the like. As
such, direct observation is normally a fairly structured form of
data collection.
ParticipantsParticipants are a group that has been identified
and isolated from the general population for research study. Since
the focus in action research is usually on the teachers classroom,
participants usually involve the teachers students in a classroom
where the improvement and action research intervention will make
the biggest difference to how teachers teach and how students
learn.
DOCUMENT ANALYSIS[footnoteRef:3] [3:
www.peoplelearn.homestead.com/Chapter2.Methods.QR.doc downloaded
from the world wide web December 2012.]
If the focus of your study is the examination of documents, than
you should have access to such material which may include letters,
memos, notes, diaries, photographs, audiotapes, videotapes, films,
articles, books, manuscripts, e-mails, online discussions and so
forth. In general documents are any preserved recording of a
persons thoughts, actions or creations (Potter, 1996). The
examination of documents is especially important to historians who
investigate patterns and trends from the past. Documents may be
examined to investigate patterns and trends of the past as is
commonly done by historians. If no humans remain alive to provide
primary evidence, then documents are the in only source of data
(Potter, 1996). Documents are also examined by researchers who are
investigating subjects who are available. The examination of
documents may also provide confirmatory evidence of the information
obtained from interviews and observations.
Document for analysis student achievement data diagnostic
assessment samples of student work attendance records anecdotal
records files behavior records literature review web research
journal lesson plans portfolio activity reports pre-test and
post-test self-evaluation standardized test scores
______________________ ______________________
______________________ ______________________
______________________ ______________________
CHECKLIST
The checklist technique is commonly used in qualitative research
to denote the frequency of a particular behavior that is related to
the research question. Checklists are a popular data collection
technique because it can be applied in a wide variety of research
situation. In other words, we encourage participants to investigate
and consult with a supervisor about how to create their own
checklist as a research collection instrument to investigate action
research.
Table 1: Checklist exampleIf my action research issue is gender
question and answering in the English classroom, I can create a
checklist to denote where and who answers the majority of questions
in the classroom: boys or girls. This information may also shed
light on where the teacher (me) is directing his or her questions
towards most often (boys or girls). Second, the checklist as a data
collection instrument can also identify the type of question asked
OPEN ended (requiring a detailed response) or CLOSED (yes or no
answer) to boys or girls in the classroom.
Lets practice[footnoteRef:4] [4: Instructor can hand this out at
the start of lecture or tutorial and ask the students to fill it
out as we go through class to monitor where questions are asked.
Then at the end of the lecture, discuss findings of students
checklist and identify where questions are asked in class and what
this means.]
Read over Table 2 carefully Next, practice using this checklist
in our class (right now) Use the checklist below to observe and see
where questions are asked in our TSL 3113 class. Make sure you keep
notes on what you have observed as that will be used for discussion
at the end of class
TABLE 1: Teacher questioning checklistDate: Start TIME:
Classroom: TSL 3113 Class duration: Number of participants: Number
of girls: Number of boys: Observers Name:
Objective: This checklist is designed to collect information
on:1. Where the teacher directed his/her questions in class 2.
Responses according to gender 3. Locate the type of question (Open
or Closed) that was asked according to gender.
In class, the peer observer will indicate in writing using the
criteria below to denote the appropriate student responses as you
observe the English lesson. For example F/O means girl open ended
question and B/C = boy closed question.
Criteria
B/G; Boy/Girl Open Question (O) Closed question (C)
CLASSROOM Questioning Checklist
My anecdotal notes (what you see):
Observer reflection (should have insight and reflection)
Running Records[footnoteRef:5] [5:
http://www.decd.sa.gov.au/northernadelaide/files/links/DECD_Running_Records_1_v8.pdf]
The most powerful teaching in classrooms takes place when
teachers use the information gained from observations and
assessments of childrens literacy development to plan their
teaching. However teachers recognise that observations can
sometimes be subjective and influenced by what they already know or
believe about the students and their literacy development.
Therefore it is important to add data from more formal assessments
to their observations.
Tests of alphabet knowledge, phonics, phonemic awareness, and
sight words form part of reading assessment but they dont provide
the whole picture of how a student approaches the reading process.
Gathering information from a Running Record, which gives a reliable
and valid assessment of text reading, and adding this information
to other assessments enables a teacher to gain a richer and more
comprehensive assessment of a students reading ability.
How to take a Running Record1. Select a text at the students
appropriate reading level and one that has been previously
introduced and read by the student.2. Invite the student to read
the text.3. As the student reads, use the conventions to note the
students reading behaviours.4. Score and analyse the Running
Record.5. Use this information to plan for future teaching.
INTERVIEWS
Structured interviews Interviewing is a technique of gathering
data from humans by asking them questions and getting them to react
verbally. There are many different ways of conducting interviews
(see Figure 2.2). Structured interviews use an interview schedule
that is similar to the survey questionnaire. You could phrase the
question in such a way that so that you have a limited range of
responses. For example, Do you think the image of teachers in
society has gone down? Strongly agree, agree, somewhat agree, agree
and strongly agree. Structured interviews are widely used in
surveying opinions, beliefs and perceptions of people. Individual
interviews are expensive and you should consider whether the same
amount of data can be more efficiently collected using written
questionnaires.
Semi-structured interviews Semi-structured interviews and
unstructured interviews are widely used in qualitative research.
Semi-structured interviews consist of a list of open-ended
questions based on the topic areas the researcher intends to study.
The open-ended nature of the questions provides opportunities for
both the interviewer and interviewee to discuss certain topics in
more detail. If the interviewee has difficulty answering a question
or hesitates, the interviewer will probe. Three types of probes
commonly used by the interviewer are:a) Detail-oriented probe When
did it happen to you? Who was with you?b) Elaboration probe Tell me
more about the incident. Can you give an example.c) Clarification
probe Im not sure I understand what you mean by hanging out. Can
you explain? You said that your principal is extremely
autocratic.
What do you mean by autocratic?
Unstructured interviewsUnstructured interviews aim to obtain in
depth interviews of persons interviewed. Only a limited number of
topics are discussed, sometimes as few as one or two topics.
Although only a few topics are discussed, they are covered in great
detail. The interview may begin with a question such as Id like to
hear your views of school discipline. Subsequent questions would
follow from the interviewees responses. Unstructured interviews are
used to find out about a specific topic but has no structure or
preconceived plan or expectation as to how the interview will
proceed.
Face-to-face or personal interviews are labour intensive but can
be the best way of collecting high quality data, especially when
the subject matter is very sensitive, if the questions are very
complex or if the interview is likely to be lengthy (Mathers, Fox
and Hunn, 2002).
ReferencesKawulich, Barbara. SAGE publication: Volume 6, No. 2,
Art. 43 May 2005 Participant Observation as a Data Collection
Method
http://www.qualitativeresearch.net/index.php/fqs/article/view/466/996
Qualitative Data Collection Techniques Chapter 2:
www.peoplelearn.homestead.com/Chapter2.Methods.QR.doc downloaded
from the world wide web December 2012.
Southern Australia Department of Education, Adelaide. Running
records downloaded from the world wide web, December 2012.
http://www.decd.sa.gov.au/northernadelaide/files/links/DECD_Running_Records_1_v8.pdf.