Observations for Research

Dao Thi Bich Nguyen

DefinitionDefinition

• In the method of data collection called observation, the researcher observes participants in natural and/or structured environments.

• (Johnson and Christensen, 2004)

ObservationsObservations

• Observations have always been considered a major data collection tool in qualitative research.

• In second language acquisition research, observations are most often used to collect data on how learners use language in a variety of settings, to study language learning and teaching processes in the classroom, and to study teachers’ and students’ behaviours.

• The main use of observations is for examining a phenomenon or a behaviour while it is going on.

(Seliger and Shohamy (1989, p. 162 )

Who and what is to be observed?Who and what is to be observed?

• The focus of the observation can be ourselves as teachers: the techniques we use, our physical presence, use of gesture, management procedures.

• The focus can be on our students: the way they work, the way they interact, the way they respond to our teaching, their on-task and off-task behaviour

• The focus can be on the context in which we teach: the classroom layout, the teaching aids available and how they are used, the use of posters and visual aids in the classroom.

• (Wallace, 1998)

Who does the observing?Who does the observing?

• Observations can be made by insiders who are part of the group observed, by participant observers, or by outsiders (Seliger and Shohamy, 1989)

• Observations can be done by the teacher concerned, other colleagues, academic researchers, or students (Wallace, 1998).

Who does the observing?Who does the observing?

• The qualitative observer may take on four different roles that make up a continuum:

– Complete participant (i.e., becoming a full member of the group and not informing the participants that you are studying them).

– Participant-as-Observer (i.e., spending extensive time "inside" and informing the participants that you are studying them).

– Observer-as-Participant (i.e., spending a limited amount of time "inside" and informing them that you are studying them).

– Complete Observer (i.e., observing from the "outside" and not informing that participants that you are studying them).

• (Johnson and Christensen, 2004)

Strengths of observation dataStrengths of observation data

• Allows one to directly see what people do without having to rely on what they say they do.

• Provides firsthand experience, especially if the observer participates in activities.

• Can provide relatively objective measurement of behavior (especially for standardized observations).

• Observer can determine what does not occur. • Observer may see things that escape the awareness of people in the

setting. • Excellent way to discover what is occurring in a setting. • Helps in understanding importance of contextual factors. • Can be used with participants with weak verbal skills. • May provide information on things people would otherwise be unwilling to

talk about. • Observer may move beyond selective perceptions of people in the setting. • Good for description. • Provides moderate degree of realism (when done outside of the laboratory). • (Johnson and Christensen, 2004)

Weaknesses of observation dataWeaknesses of observation data

• Reasons for observed behavior may be unclear. • Reactive effects may occur when respondents know they are being • observed (e.g., people being observed may behave in a typical way). • Investigator effects (e.g., personal biases and selective perception of

observers) • Observer may “go native” (i.e., over-identifying with the group being

studied). • Sampling of observed people and settings may be limited. • Cannot observe large populations. • Some settings and content of interest cannot be observed. • Collection of unimportant material may be moderately high. • More expensive to conduct than questionnaires and tests. • Data analysis can be time consuming. • (Johnson and Christensen, 2004)

TypesTypes

• “Structured” observations– Researcher has determined in advance what to look for in the

observed context (Seliger and Shohamy, 1989).

– Observers are guided by some kind of observation schedule (Wallace 1998).

• “Unstructured” or “open” observations– The data recorded are broad and more general.

DataData

• Data obtained from more structured observations will be in the form of checks, tallies, frequencies, and rating (Seliger and Shohamy, 1989).

• Data obtained from open observations will be in the form of impressions, field notes, tapes or transcripts (Seliger and Shohamy, 1989).

Methods of recordingMethods of recording

• Impressions: often have no record, and this may create a problem if it is necessary to return to the data.

• Field notes: provide records, but depend on the opportunity and ability of the observer to record accurately what is observed. In addition, the presence of the observer may alter the subjects’ behaviour.

• Audio tapes: are limited to capturing the vocal aspect of verbal interaction.

• Video tapes: provide more elaborate data, although they are dependent on the capability of the camera and what is focuses on. Specific items of data may not be very accessible

• Transcription:

Methods of recordingMethods of recording

• A regular time base/ time-interval observing (every 3 seconds, 30 seconds, 60 seconds)

• Event observing (observing after an event has taken place; e.g. observing after teacher asks a question)

• Tally: • Checklist: observer check whether a specific behavior is

present or absent.• Numerical scales and rating scales: helps the observer to

rate and quantify the observed behaviors or phenomena.

Example: interaction analysis- a regular time Example: interaction analysis- a regular time basebase• Observed behavior: the

frequency/ distribution of participation in a group-work activity

Student A B C D

1

2

3

4

5

6

7

8

9

30 – second sweep

Example: interaction analysis -a regular time Example: interaction analysis -a regular time basebase• Recoding nature of

participation

• 1: initiation

• 2: response – simple agreement

• 3: response – agreement and development

• 4: response – simple disagreement

• 5: response – disagreement and development

• 6: structuring move (eg. why don’t we discuss…)

1,1,6

2 3

5,4

Student A

Student B

Student C

Student DStudent A: a dominant role in the discussion

Example: ChecklistExample: Checklist

• Observed behaviour: Student’s activities in the language classroom

• Task: Check whether or not the student performed the following:

• Asked for translation of unknown words: Yes __No ____• Used L1 in conversation with teacher: Yes ___ No ____• Used L2 in conversation with teacher: Yes ___ No ____• Used L2 in conversation with peers: Yes ___ No ____• Referred to textbook/dictionary for unknown words: Yes

___ No ____• Asked for grammaticall explanations: Yes ___ No ____

Example: Numerical scaleExample: Numerical scale

• Observed behaviour: Students’ use of L2 in asking questions

• Task: How often does each student ask a question in L2?

• Always_____; Usually________; Sometimes:_______; Never_______

Example: Rating scaleExample: Rating scale

• Observed behaviour: students’ involvement in a specific classroom task

• Task: Please mark how involved students are:• Very involved: 1 2 3 4 5 Not very involved

Example: Tallies/ Question tally sheetExample: Tallies/ Question tally sheetObserved behaviour: the level of question that teacher usesObserved behaviour: the level of question that teacher uses

Level of question Tallies Total

Literal (Simple retrieval of information from the text)

IIII IIII

IIII

14

Inferential (reading between the lines)

IIII 4

Reorganisational (combining information from different areas of the text)

II 2

Evaluative (expressing a judgment)

I 1

Example: TalliesExample: Tallies

• Observed behaviour: are some students “favoured” with the teacher’s attention more than others?

Name Tally Total

Lan I 1

Mai II 2

Hoa IIII 4

Cong I 1

Example: TalliesExample: Tallies

• Observed behaviour: are some students “favoured” with the teacher’s attention more than others?

T

IIII

IIII

I

IIIII

I

II

I

System-based approachSystem-based approach

• The observers are guided by some kind of observation schedule which allows them to collect data in a systematic way

ready made no need to be trialed or validated• Eg:• The FIAC system: Flanders Interaction Analysis

Categories• The TALOS system: The target Language Observation

Scheme• The COLT: The communicative Orientation of Language

Teaching

Example: Classroom Observation Tally Sheet from Nunan (1989)Example: Classroom Observation Tally Sheet from Nunan (1989)

Tallies Total

Teacher asks a display questions III 3

Teacher asks a referential questions IITeacher explains a grammatical point ITeacher explains meaning of a vocabulary item

Teacher explains functional point

Teacher explains points relating to the content

Teacher gives instructions/direction

Teacher praises

Teacher criticizes

Learner answers questions IIIILearner talks to another learner

Learner asks questions

Period of silence or confusion

Example: FIAC SystemExample: FIAC System

• Teacher talk– Accepting feelings or attitudes expressed by the students– Praising or encouraging a student– Accepting or using students’ ideas– Asking questions– Lecturing (explaining, informing)– Giving directions or commands– Criticizing students, justifying authority

• Student talk– Pupil-talk response (answering a question)– Pupil-talk initiation (volunteering information, asking a question of

the teacher)– Silence/confusion

Example: The TALOS Observation schemeExample: The TALOS Observation scheme

• Teaching act:– Drill

– Narrate

– Explain

– Discuss

– Compare

– Answer

– Questions

– Correct

– Reinforce

– Routine

– discipline

Example: COLT Part AExample: COLT Part A

Example COLT Part BExample COLT Part B

Example: Open observationsExample: Open observations

• Observed behaviour: Students’ involvement in the language class

• Task: Describe the level of involvement of three students in the language class activities

• ___________________________________________• ___________________________________________• ___________________________________________

Example: Open observationsExample: Open observations

• Observed behaviour: Teacher’s and students’ use of L1 in an L2 class

• Task: Describe the type and amount of language used by the teacher and by the students during a groupwork activity

• _______________________________________• _______________________________________• ______________________________________

Discussion 1Discussion 1

• Group work

• Using the table below, you are invited to think of a topic that interests you within of the three main areas of focus (yourself as a teacher, your students, or your class context). Decide who would be responsible for the observation and which method of recording you would like to use.

Focus Topic Obsever/s Method

Self as teacher

students

Context

Discussion 2Discussion 2

• Group work• Make a list of aspects of your own teaching behaviour or

your students’ behaviour that it might be useful for you to observe.

• Pick one activity that you want to observe and decide how you are going to implement the observation. What do you hope to discover?

• If you can, make a trial attempt at implementing your plan. How interesting/ useful were the data you discovered? Is there any way in which the observation could have been improved?

• [email protected]