Observation Studies

Chapter 8Observation StudiesMcGraw-Hill/IrwinCopyright 2011 by The McGraw-Hill Companies, Inc. All Rights Reserved.

Learning ObjectivesUnderstand . . .When observation studies are most useful. Distinctions between monitoring. nonbehavioral and behavioral activitiesStrengths of the observation approach in research design.Weaknesses of the observation approach in research design.

Learning ObjectivesUnderstand . . .Three perspectives from which the observer-participant relationship may be viewed. Various designs of observation studies.

How Our Brain WorksOnce a pattern becomes predictable, thebrain starts to ignore it. We get bored;attention is a scare resource, so why wasteit on something thats perfectly predictable.

Jonah Lehrer neuroscientist and author,How We Decide

PulsePoint: Research Revelation3The number of minutes the average cubicle dweller works before being interrupted by phone, e-mail, instant message, or social networking activities.

Observation and the Research Process

Selecting the Data Collection Method

Selecting an Observation Data Collection Approach

Research DesignTask Details

Observation Location

Content of Observation

FactualInferentialIntroduction/identification of salesperson and customer.Credibility of salesperson. Qualified status of customer.Time and day of week.Convenience for the customer. Welcoming attitude of the customerProduct presented.Customer interest in product.Selling points presented per product. Customer acceptance of selling points of product.Number of customer objections raised per product.Customer concerns about features and benefits.Salespersons rebuttal of objection.Effectiveness of salespersons rebuttal attempts.Salespersons attempt to restore controls.Effectiveness of salespersons control attempt. Consequences for customer who prefers interaction.Length of interview.Customers/salespersons degree of enthusiasm for the interview.Environmental factors interfering with the interview.Level of distraction for the customer.Customer purchase decision.General evaluation of sale presentation skill.

Data Collection

Using ObservationSystematic planningProperly controlledConsistently dependableAccurate account of events

Observation ClassificationNonbehavioralPhysical condition analysisProcess or Activity analysisRecord analysis

BehavioralNonverbalLinguisticExtralinguisticSpatial

Selecting an Observation Data Collection Approach Nonbehavioral

Nonbehavioral ObservationRecord AnalysisPhysical Condition AnalysisPhysical Process Analysis

Selecting an Observation Data Collection ApproachBehavioral

Wal-Mart Implements RFID Labels

RFID Changes MonitoringWe can certainly understand and appreciateconsumer concern about privacy. Thats why wewant our customers to know that RFID tags willnot contain nor collect any additional data aboutour customers. In fact in the foreseeable future,there wont even be any RFID readers on ourstores main sales floors.

Linda Dillman EVP & Chief Information OfficerWal-Mart

Behavioral ObservationWe noticed people scraping the toppings off our pizza crusts. We thought at first there was something wrong, but they said, We love it, we just dont eat the crust anymore.Tom Santor, Donatos Pizza

Systematic ObservationSystematic

Flowchart for ObservationChecklist Design

Mechanical/ Digital Behavioral ObservationDevices

SizeUSABody Measurement System

Portable People Meters

Observer-Participant RelationshipDirect vs. indirect Known vs. unknown

Involved vs. uninvolved

Extralinguistic ObservationVocalTemporalInteractionVerbal Stylistic

Desired Characteristics for ObserversConcentrationDetail-orientedUnobtrusiveExperience level

Errors Introduced by ObserversObserver DriftHalo Effect

Evaluation of Behavioral ObservationStrengthsSecuring information that is otherwise unavailableAvoiding participant filtering/ forgettingSecuring environmental contextOptimizing naturalnessReducing obtrusivenessWeaknessesEnduring long periods Incurring higher expensesHaving lower reliability of inferencesQuantifying dataKeeping large recordsBeing limited on knowledge of cognitive processes

Key TermsConcealmentEvent samplingHalo effect

ObservationDirectExtralinguisticIndirectLinguisticNonverbalParticipantSimpleSpatialSystematic

Key TermsObservation checklistObserver driftPhysical condition analysisPhysical traceProcess (activity) analysisReactivity responseRecord analysisSpatial RelationshipsTime samplingUnobtrusive measures

*This chapter provides coverage on conducting observation studies. ***See the text Instructors Manual (downloadable from the text website) for ideas for using this research-generated statistic.*Exhibit 8-1 depicts the use of observation in the research process. *Exhibit 8-2*Exhibit 8-3 illustrates the selection of an observation data collection approach. This is discussed further on the following slide.*The data collection plan specifies the details of the task. Who? The plan must specify what qualifies a person to participate as a subject of the observation. It must also assign responsibilities on the research side. What? The characteristics of the observation must be set as sampling elements and units of analysis. This is achieved when event-time dimension and act terms are defined. In event sampling, the researcher selects certain elements, behavioral acts, or conditions to record that answer the investigative questions. In time sampling, the researcher must choose among a time-point sample, continuous real-time measurement, or a time-interval sample. For a time-point sample, recording occurs at fixed points for a specified length. Time-interval sampling records every behavior in real time but counts the behavior only once during the interval. What constitutes an act depends on the study. For instance, acts could include a single expressed thought, a physical movement, a facial expression, or a motor skill.When? When specifies whether the time of the study is important, and if so, what time period will be used.How? Will the data be observed directly? How will various situations be handled? How will data be recorded?Where does the act take place?*The Lexus ad agency Team One Advertising used observation during the NY auto show. Refer to instructors manual Snapshot Lexus: The Winning Reaction.*Exhibit 8-6*Besides collecting data visually, observation involves listening, reading, smelling, and touching. Refer back to Exhibit 8-2, which describes the conditions under which observation is appropriate.*Simple observation is unstructured and not standardized. Systematic observation is structured and uses standardized procedures to observe participants or objects.Observation may be the primary methodology used to answer a research question when it is systematically planned and executed, uses proper controls, and provides a consistently dependable (reliable) and accurate and authoritative (valid) account of what happened. *Observation includes the full range of monitoring behavioral and nonbehavioral activities and conditions. These can be classified as behavioral and nonbehavioral observations. Nonverbal observation is the most prevalent and refers to recording physical actions or movements of participants. These behaviors can be measured with the human eye and with several mechanical or digital devices. These devices are discussed on the following slide.Linguistic observation is the observation of human verbal behavior during conversation, presentation, or interaction.Extralinguistic observation is the recording of vocal, temporal, interaction, and verbal stylistic behaviors of human participants. It is discussed further on slide 10-12.Spatial observation is the recording of how humans physically relate to one another.Nonbehavioral observation is the observation of the effects or traces of prior actions or of nonhuman activity. Physical condition analysis is the recording of observations of current conditions resulting from prior decisions.Process (activity) analysis is observation by a time study of stages in a process, evaluated on both effectiveness and efficiency.Record analysis is the extraction of data from current or historical records. Data mining is a type of record analysis, which is discussed further in the slide show.

**Record analysisthink data miningis a common nonbehavioral observation. What you are looking at is an accounting software report relating to late payments of customers, but such observation could as easily be looking at a defect report from operations, a record of sick days in human resources, a promotions sales-lift report, or many others that firmslarge and smallgenerate on a regular basis. Physical condition analysis might relate to a safety audit or an analysis of inventory conditions or an analysis of food preparation areas in a restaurant. Building inspectors do physical condition analyses when they allow a builder to progress to the next phase of construction following an inspection of plumbing, wiring, etc.Process analysis started with time-motion studies in manufacturers, but today relates to any organization that tracks time related to activity or process steps and stages. Some examples include: banks (ATMs resulted from such a process analysis), merchandise retailers (e-commerce sites are constantly evaluating abandon carts), restaurants, medical practices, and architects use such nonbehavioral evaluations. *Exhibit 8-3, cropped to behavioral observation elements*Radio Frequency Identification tags tracks various information about a product. For instance, Wal-Mart is using the technology to track inventory, location of customers, and when a product is actually used. This is an example of nonbehavioral observation.*Donatos Pizza discovered something about consumer preferences from simple observation of human behavior.*Systematic studies employ standardized procedures, trained observers, schedules for recording, and other devices for the observer that reflect the scientific procedures of other primary data methods.*Exhibit 8-5*Researchers can use their eyes to record behavioral observations but some things cannot be adequately observed with the human eye. Several mechanical devices are used to record behavioral observations. The most commonly used devices are the video camera and audio recorder. The galvanometer is a device that measures excitement, arousal, fear, or heightened physiological response to stimuli. It does this by measuring electrical activity in the participants skin. They are most used in advertising research.The eye camera and pupilometer are device that attach to a persons forehead. When a participant is shown various stimuli, the researcher can measure the movement of the eye and the dilation of the pupil in the eye. For example, during an episode of a television program, one can match the eye movement precisely with what one is watching on the screen.The tachistoscope is a timed shutter device that exposes a participant to some stimulus for a controlled period of time. Historically, it has been used to substantiate the effects of subliminal advertising.*SizeUSA developed a three-dimensional scaling system for the human body. It is being used to take body measurements that are and will be used in product design for clothing, cars, airline seats, and more.*Nielsen Media Research collects some of its television viewer data with electronic devices labeled people meters. The people meter measures the tuning state of the TV set, what channel is being tuned, and who is watching. Nielsen has been testing a Portable People Meter, which is shown in the slide. Participants carry the meter wherever they go. The meter records signals from media companies. The data transferred back to Nielsen when the participant returns home and docks the device.*Communication with a participant presents a clear opportunity for interviewer bias. The problem is less pronounced with observation but is still real. The relationship between observer and participant may be viewed from three perspectives:Whether the observation is direct or indirectWhether the observers presence is known or unknown to the participant, and What role the observer plays in the events or effects of events recorded.

Direct observation occurs when the observer is physically present and personally monitors what takes place. Indirect observation occurs when the recording is done by mechanical, photographic, or electronic means. When the observer is known, there is a risk of atypical activity by the participant. Concealment shields the observer from the participant to avoid error caused by the observers presence. A modified approach involves partial concealment. The presence of the observer is not concealed, but the objectives are.The third issue is whether the observer should participate in the situation while observing. Mystery shopping is an example of observer-participation.

In this situationdirect, unknown, uninvolved

*Linguistic observation is the observation of human verbal behavior during conversation, presentation, or interaction. Extralinguistic observation, the focus of this slide, is the recording of vocal, temporal, interaction, and verbal stylistic behaviors of human participants. Vocal behaviors include pitch, loudness, and timbre.Temporal behaviors include the rate of speaking, duration of utterance, and rhythm.Interaction includes the tendencies to interrupt, dominate, or inhibit.Verbal stylistic behaviors include vocabulary and pronunciation peculiarities, dialect, and characteristic expressions.Ronald Reagan was lauded for his command of extralinguistic behavior.*There are a few general guidelines for the qualification and selection of observers. Observers should have the ability to function in a setting full of distractions (concentration). They should have the ability to remember details of an experience (detail-oriented). They should have the ability to blend with the setting and not be distinctive (unobtrusive). Finally, observers should have the ability to extract the most from an observation study (experience level).For studies using direct observation with a simple checklist, prior experience is less important.

Instructors have the ads from the Akron Childrens Hospital case on the Online Learning Center. The footage in the commercials was the footage captured in the observation.*Inexperience can be an advantage if there is a risk that experienced observers may have preset convictions about the topic or if prior observations will influence what is perceived in a current observation. This is called the halo effect.Observers can also introduce error when fatigued, which can result in observer drift. Observer drift is error caused by decay in consistency and accuracy on recorded observations over time, affecting categorization.*This slide lists the strengths and limitations of observation as a data collection method. Observation is the only method available for gathering certain types of information. Another advantage is that it can capture the whole event as it occurs in its natural environment. Observation participants seem to accept an observational intrusion better than they respond to the intrusion of survey takers. Further, some observation studies are concealedA key limitation of observation is that it records what, where, who, and how, but cannot record why people behave as they do. Observation is slow and expensive. Observation may be factual or inferential. It is most reliable when it is based on fact. Exhibit 9-4, shown on the next slide, shows how we can separate the factual and inferential component of a salespersons presentation. **