Designing for Attention With Sound: Challenges and Extensions to Ecological Interface Design Marcus O. Watson and Penelope M. Sanderson HUMAN FACTORS,

Designing for Attention Designing for Attention With Sound: Challenges With Sound: Challenges

and Extensions to and Extensions to Ecological Interface DesignEcological Interface Design

Marcus O. Watson and Penelope M. SMarcus O. Watson and Penelope M. Sandersonanderson

HUMAN FACTORS, HUMAN FACTORS, Vol. 49, No. 2,Vol. 49, No. 2,April 2007, pp. 331–346April 2007, pp. 331–346

BackgroundBackground

Ecological Interface Design (EID) is a Ecological Interface Design (EID) is a framework that is based on the skills, framework that is based on the skills, rules, knowledge taxonomy of cognitive rules, knowledge taxonomy of cognitive control. control.

EID examples focus almost exclusively on EID examples focus almost exclusively on visual displays. In the anesthesia work visual displays. In the anesthesia work environment, however, auditory displays environment, however, auditory displays may provide better individual and team may provide better individual and team awareness of patient state.awareness of patient state.

Process for the design of auditory Process for the design of auditory displays using EIDdisplays using EID

MethodMethod

Using a Using a work domain analysiswork domain analysis of of physiological monitoring in anesthesia, physiological monitoring in anesthesia, we identify information to display.we identify information to display.

Using the Using the skills, rules, and knowledge skills, rules, and knowledge distinctiondistinction we identify cognitive we identify cognitive control needed.control needed.

Using Using semantic mappingsemantic mapping we map we map physiological variables and constraints physiological variables and constraints to auditory dimensions.to auditory dimensions.

Ecological Interface Design and theEcological Interface Design and theAuditory ModalityAuditory Modality

Three principles underlie EIDThree principles underlie EID

1. interface content should support 1. interface content should support knowledge-based behaviorknowledge-based behavior

2. interface content should support 2. interface content should support rule-based behaviorrule-based behavior

3. the interface should support skill-3. the interface should support skill-based behaviorbased behavior

When played through speakers, auditory When played through speakers, auditory displays can be heard regardless of a listdisplays can be heard regardless of a listener’s posture.ener’s posture.

Auditory displays will be heard by anyonAuditory displays will be heard by anyone nearby, regardless of intent.e nearby, regardless of intent.

Auditory displays occur in time and cannAuditory displays occur in time and cannot be reviewed at will by reorienting oneot be reviewed at will by reorienting one’s sense organs.’s sense organs.

PROBLEM IDENTIFICATION-Field PROBLEM IDENTIFICATION-Field StudiesStudies

Field studies suggest that anesthesiologiField studies suggest that anesthesiologists may benefit from receiving patient infsts may benefit from receiving patient information continuously but unobtrusively.ormation continuously but unobtrusively.

Auditory interfaces that provide critical iAuditory interfaces that provide critical information continuously by making displnformation continuously by making displays ubiquitous and obligatory without exays ubiquitous and obligatory without extra workload could be useful supplementtra workload could be useful supplements to visual displays.s to visual displays.

Incident Report DataIncident Report Data

Incident report data suggest that an integrateIncident report data suggest that an integrated and continuous display of patient vital signs d and continuous display of patient vital signs may speed detection of events.may speed detection of events.

A display of respiratory vital signs in a continuA display of respiratory vital signs in a continuous format compatible with pulse oximetry coous format compatible with pulse oximetry could therefore help the anesthesiologist achievuld therefore help the anesthesiologist achieve an overview of the most important aspects oe an overview of the most important aspects of a patient’s physiological state.f a patient’s physiological state.

NEEDS ANALYSIS-Work Domain NEEDS ANALYSIS-Work Domain AnalysisAnalysis

WDA produces a representation of the pWDA produces a representation of the purposes, first principles, functions, procurposes, first principles, functions, processes, and configuration of any work doesses, and configuration of any work domain main

A human agent is responsible for handliA human agent is responsible for handling disturbances and maintaining desireng disturbances and maintaining desired operation.d operation.

Sensed and Derived ValuesSensed and Derived Values

Researchers using EID usually seek to diResearchers using EID usually seek to display information about functional purpsplay information about functional purpose and abstract functions so that humaose and abstract functions so that humans can monitor the stability of such higns can monitor the stability of such high-level properties.h-level properties.

Temporal RelationsTemporal Relations

Understanding temporal relations is impUnderstanding temporal relations is important when mapping the dynamic proportant when mapping the dynamic properties of a process into the temporal proerties of a process into the temporal properties of auditory displays.perties of auditory displays.

Recipients of InformationRecipients of Information

Patient monitoring is at times a vigilancPatient monitoring is at times a vigilance task, and vigilance is sustained better e task, and vigilance is sustained better – and with a lower level of subjective str– and with a lower level of subjective stress and workload – with auditory rather ess and workload – with auditory rather than visual displays.than visual displays.

Identifying Required Cognitive ContIdentifying Required Cognitive Controlrol

we noticed a mismatch between the kinwe noticed a mismatch between the kind of cognitive control (SKR) available to d of cognitive control (SKR) available to the anesthesiologists, given the monitorthe anesthesiologists, given the monitoring equipment and the kind of cognitive ing equipment and the kind of cognitive control they sometimes preferred.control they sometimes preferred.

We hypothesized that a change in displaWe hypothesized that a change in display modality might change the way cognitiy modality might change the way cognitive control is exercised.ve control is exercised.

The AIMS study data suggested that variThe AIMS study data suggested that variable-tone pulse oximetry–a ubiquitous, able-tone pulse oximetry–a ubiquitous, obligatory display– helped anesthesioloobligatory display– helped anesthesiologists detect events early and with a low lgists detect events early and with a low level of cognitive control.evel of cognitive control.

Auditory alarms often recapture attentioAuditory alarms often recapture attention, and responding to them is faster than n, and responding to them is faster than to visual alarmsto visual alarms

DESIGN SYNTHESIS-Establishing DESIGN SYNTHESIS-Establishing a Modalitya Modality

Johannsen (2004) has suggested that deJohannsen (2004) has suggested that designers should develop auditory displaysigners should develop auditory displays principally to minimize human error at s principally to minimize human error at the skill-based level.the skill-based level.

Auditory displays can support people’s Auditory displays can support people’s peripheral awareness of system state wperipheral awareness of system state while they perform other tasks.hile they perform other tasks.

Semantic MappingSemantic Mapping

Semantic mapping is the process by whiSemantic mapping is the process by which the functional structure and constraich the functional structure and constraints in a work domain are mapped to pernts in a work domain are mapped to perceptual forms in a way that supports cogceptual forms in a way that supports cognitive control at the desired levels.nitive control at the desired levels.

We describe how we arrived at a workable soniWe describe how we arrived at a workable sonification of respiratory vital signs (Step 7, Figurfication of respiratory vital signs (Step 7, Figure 1):e 1):

1. 1. Kinds of auditory displays.Kinds of auditory displays. 2. Candidate sound dimension mappings.2. Candidate sound dimension mappings. 3. 3. Number of auditory streamsNumber of auditory streams 4. Promoting skill-based behavior4. Promoting skill-based behavior 5. 5. Choosing the carrier tone.Choosing the carrier tone. 6. 6. Piggybacking information on the carrier tone.Piggybacking information on the carrier tone.

Attentional MappingAttentional Mapping

Table 3 lists attentional mapping issues Table 3 lists attentional mapping issues and trade-offs across different possible rand trade-offs across different possible respiratory sonifications.espiratory sonifications.

Individual attention.Individual attention.

A sonification imposes obligatory and transitoA sonification imposes obligatory and transitory stimuli on an individual (whether delivered ry stimuli on an individual (whether delivered via earpiece or speakers), so auditory displays via earpiece or speakers), so auditory displays must be designed to control attention properlmust be designed to control attention properly over time.y over time.

Attentional mapping “gives designers requireAttentional mapping “gives designers requirements for how an auditory display should contments for how an auditory display should control attention alongside other interface elemenrol attention alongside other interface elements, based in a knowledge of ts, based in a knowledge of auditory attentionauditory attention””

Team attentionTeam attention..

An auditory display must be An auditory display must be tolerated by everyone who has to tolerated by everyone who has to hear it.hear it.

An auditory display must be An auditory display must be meaningful for staff who must hear meaningful for staff who must hear it, helping staff detect abnormal it, helping staff detect abnormal patient states while still performing patient states while still performing their normal tasks.their normal tasks.

If not all staff need to hear a display, theIf not all staff need to hear a display, then it can be delivered just to those who nn it can be delivered just to those who need to hear it (Patterson et al.,1999) via eed to hear it (Patterson et al.,1999) via earpieces and wireless.earpieces and wireless.

Auditory displays should avoid the speeAuditory displays should avoid the speech range so that normal verbal communich range so that normal verbal communications are preserved.cations are preserved.

Evaluation ResultsEvaluation Results

Novices and anesthesiologists perform cNovices and anesthesiologists perform concurrent tasks better when supported oncurrent tasks better when supported by respiratory sonification than by visuaby respiratory sonification than by visual display.l display.

Participants respond faster to questions Participants respond faster to questions about sonified patient status and check about sonified patient status and check visual monitors less often when sonificatvisual monitors less often when sonification is available.ion is available.

ConclusionConclusion

EID principles do not address when inforEID principles do not address when information should be displayed and to whomation should be displayed and to whom.m.

An attentional mapping stage helps to sAn attentional mapping stage helps to specify answers to these questions so thapecify answers to these questions so that a workable auditory display for anestht a workable auditory display for anesthesia monitoring is achieved.esia monitoring is achieved.