Engineering Psychology and Human Performance. Human Information Processing Knowledge of physical performance (i.e. speed and accuracy) is important in.

Engineering Psychology and Engineering Psychology and Human PerformanceHuman Performance

Human Information ProcessingHuman Information Processing

• Knowledge of physical performance Knowledge of physical performance (i.e. speed and accuracy) is important (i.e. speed and accuracy) is important in understanding a task (or the effects in understanding a task (or the effects upon the task from an ergonomics upon the task from an ergonomics intervention program).intervention program).

• But such knowledge is not sufficient to But such knowledge is not sufficient to understand understand WHYWHY performance changed performance changed from a human operator perspective.from a human operator perspective.

A MODEL for Human A MODEL for Human Information ProcessingInformation Processing

1.1. Sensory ProcessingSensory Processing

2.2. PerceptionPerception

3.3. Memory and CognitionMemory and Cognition

4.4. Response Selection and ExecutionResponse Selection and Execution

5.5. FeedbackFeedback

6.6. AttentionAttention

Sensory ProcessingSensory Processing• Information and events in the environment Information and events in the environment

must gain access to the brain.must gain access to the brain.

• Visual, Auditory, Tactile, SmellVisual, Auditory, Tactile, Smell

• All sensory systems have an associated All sensory systems have an associated short-term sensory storeshort-term sensory store– Temporary mechanism for prolonging the Temporary mechanism for prolonging the

representation of the raw stimulus (length representation of the raw stimulus (length depends on the type of stimulus)depends on the type of stimulus)

– Can be confounded if the operator is distractedCan be confounded if the operator is distracted

PerceptionPerception• Raw sensory data relayed to the brain Raw sensory data relayed to the brain

must be interpreted (given meaning) must be interpreted (given meaning) through a stage of perception.through a stage of perception.– Decode the meaning of the signalDecode the meaning of the signal

• Generally proceeds rapidly and Generally proceeds rapidly and automaticallyautomatically

• Driven by Driven by sensory inputsensory input (rapid; bottom-up (rapid; bottom-up processing) or processing) or long-term memorylong-term memory (top- (top-down processing)down processing)

Memory and CognitionMemory and Cognition• Cognitive operations generally require more Cognitive operations generally require more

“processing” time.“processing” time.

• Rehearsal, reasoning or image transformation is Rehearsal, reasoning or image transformation is carried out by carried out by working memoryworking memory..

• ““Plans”; “Diagnose” ~ these are vulnerable to Plans”; “Diagnose” ~ these are vulnerable to disruption when attentional resources are disruption when attentional resources are diverted to other mental activities.diverted to other mental activities.

• Material that is rehearsed in working memory can Material that is rehearsed in working memory can get access to get access to long-term memorylong-term memory – – learninglearning takes takes place.place.

Response Selection and ExecutionResponse Selection and Execution

• The understanding of a situation, achieved The understanding of a situation, achieved through perception and augmented by through perception and augmented by cognitive transformations will trigger the cognitive transformations will trigger the selection of a selection of a responseresponse..

• The selection of a response must precede The selection of a response must precede the the executionexecution (requiring motor effort – (requiring motor effort – which has it’s own qualities for success).which has it’s own qualities for success).

FeedbackFeedback

• Actions are directly sensed by the Actions are directly sensed by the operator OR influence the system operator OR influence the system within which the operator functions.within which the operator functions.– Flow of information can be Flow of information can be initiatedinitiated at any at any

point and is point and is continuouscontinuous..

AttentionAttention

1.1. Selective AttentionSelective Attention

2.2. Focused AttentionFocused Attention

3.3. Divided AttentionDivided Attention

A model of human information A model of human information processing stagesprocessing stages

Signal DetectionSignal Detection

Overview:Overview:

• The process of detection may involve The process of detection may involve many states of categorization:many states of categorization:

– Operator to decide whether a “signal” is Operator to decide whether a “signal” is present.present.

– Operator to attend to several signalsOperator to attend to several signals

Signal Detection Paradigm ~ Signal Detection Paradigm ~ The Four Outcomes of Signal Detection The Four Outcomes of Signal Detection

TheoryTheory

Setting the Response CriterionSetting the Response Criterion

Hypothetical distributions underlying signal detection theoryHypothetical distributions underlying signal detection theory

Consider the circumstances!Consider the circumstances!

• Radiologist examining X-Rays of referred cancer Radiologist examining X-Rays of referred cancer patient.patient.– More likely to say “yes” but increases the chances of a More likely to say “yes” but increases the chances of a

“false alarm” - CONSERVATIVE“false alarm” - CONSERVATIVE

Consider the circumstances!Consider the circumstances!

Plant operator warned about unnecessary shutdowns Plant operator warned about unnecessary shutdowns and the financial implicationsand the financial implications– More likely to say “no” but increases the chances of a More likely to say “no” but increases the chances of a

“missed event” - RISKY“missed event” - RISKY

• For what ever reasons:For what ever reasons:– When signal and noise have similar When signal and noise have similar

characteristicscharacteristics– Operator deficiencies (training, fatigue, stress)Operator deficiencies (training, fatigue, stress)

• Chances (probability) of false alarms or Chances (probability) of false alarms or misses increases!misses increases!

MISSES FALSE ALARMS

The Probability of Signal Detection TheoryThe Probability of Signal Detection Theory

• Signal detection theory is able to Signal detection theory is able to prescribe where the optimum beta (prescribe where the optimum beta (ββ) ) should fall:should fall:– The likelihood of observing a signalThe likelihood of observing a signal– The COSTS/BENEFITS (payoffs) of the four The COSTS/BENEFITS (payoffs) of the four

possible outcomespossible outcomes

• In almost all circumstances, signal and In almost all circumstances, signal and noise will overlap.noise will overlap.

Attention in Perception and Display SpaceAttention in Perception and Display Space

• The limitations of human attention represent The limitations of human attention represent one of the most formidable bottlenecks in one of the most formidable bottlenecks in human information processing.human information processing.

– I failed to notice the words of a speaker because I I failed to notice the words of a speaker because I was looking out the windowwas looking out the window

– We had so many tasks to perform that some were We had so many tasks to perform that some were omittedomitted

SELECTIVE ATTENTIONSELECTIVE ATTENTION

• Begins with the EYE Begins with the EYE and VISUAL and VISUAL SAMPLING.SAMPLING.

– Seek for Seek for information/search information/search for a targetfor a target

– We learn a lot by We learn a lot by assessing visual assessing visual scanning behaviourscanning behaviour

SELECTIVE ATTENTIONSELECTIVE ATTENTION

• The FOVEA has about The FOVEA has about a 2 degrees of visual a 2 degrees of visual angle.angle.

• To keep objects in the To keep objects in the foveal vision ~foveal vision ~– Pursuit Movement ~ Pursuit Movement ~

eye follows a target eye follows a target moving across the moving across the visual field – visual field – “smooth “smooth motion”motion”

– Saccadic Movement – Saccadic Movement – jump from one jump from one stationary point to stationary point to another – another – “jerky “jerky movements”movements”

SELECTIVE ATTENTIONSELECTIVE ATTENTION

• Supervisory Control SamplingSupervisory Control Sampling– In the aircraft cockpit or the process In the aircraft cockpit or the process

control console, many information sources control console, many information sources have to be sampled periodically.have to be sampled periodically.

– Often divide the environmental stimulus Often divide the environmental stimulus into into CHANNELSCHANNELS along which critical along which critical EVENTSEVENTS may periodically occur. may periodically occur.

SELECTIVE ATTENTIONSELECTIVE ATTENTION

• Environmental Sampling is guided by Environmental Sampling is guided by the expected cost that results when an the expected cost that results when an event is missed.event is missed.

• The probability of missing an event is The probability of missing an event is directly related to the event frequency directly related to the event frequency and uncertainty.and uncertainty.

SELECTIVE ATTENTIONSELECTIVE ATTENTIONVisual ScanningVisual Scanning

1.1. Mental Model GuidesMental Model Guides

2.2. Adjustment to Event RateAdjustment to Event Rate

3.3. Sampling Affected by ArrangementSampling Affected by Arrangement

4.4. Memory Imperfect; Sampling ImperfectMemory Imperfect; Sampling Imperfect

5.5. Preview HelpsPreview Helps

6.6. Processing Strategies – Cognitive TunnelingProcessing Strategies – Cognitive Tunneling

VISUAL SCANNINGVISUAL SCANNING

Mental Model GuidesMental Model Guides• The mental model consists of a set of expectancies The mental model consists of a set of expectancies

of how frequently and when events will occur.of how frequently and when events will occur.

• Correlation of events amongst channels.Correlation of events amongst channels.

• As expertise develops, mental models become As expertise develops, mental models become

refined as do the sampling strategies.refined as do the sampling strategies.

VISUAL SCANNINGVISUAL SCANNING

Adjustment to Event RateAdjustment to Event Rate• People learn to sample channels with higher event People learn to sample channels with higher event

rates more frequently (and lower event rates less rates more frequently (and lower event rates less

frequently)frequently)

VISUAL SCANNINGVISUAL SCANNING

Sampling Affected by ArrangementSampling Affected by Arrangement• Operators like to scan horizontally rather than Operators like to scan horizontally rather than

diagonally.diagonally.

• It is proposed that operators use simplifying rules It is proposed that operators use simplifying rules

and HEURISTICS based on channel arrangement to and HEURISTICS based on channel arrangement to

decrease attentional demands – this leads to decrease attentional demands – this leads to

systematic biases in performance.systematic biases in performance.

• Arrangements matterArrangements matter

* * HeuristicsHeuristics – creating a model as a working – creating a model as a working

hypothesis to a goal or solutionhypothesis to a goal or solution

VISUAL SCANNINGVISUAL SCANNING

Memory Imperfect; Sampling ImperfectMemory Imperfect; Sampling Imperfect• People tend to sample information sources more People tend to sample information sources more

often than they would need to if their short-term often than they would need to if their short-term

memory was better. Two consequences:memory was better. Two consequences:

• ““Oversampling” occurs of channels with low event ratesOversampling” occurs of channels with low event rates

• People forget to sample channels with low event ratesPeople forget to sample channels with low event rates

• Suggests the use of “sampling reminders”Suggests the use of “sampling reminders”

VISUAL SCANNINGVISUAL SCANNING

Preview HelpsPreview Helps• When people are given a preview of scheduled events that are When people are given a preview of scheduled events that are

likely to occur in the future, sampling a channel switching likely to occur in the future, sampling a channel switching

become somewhat more optimal.become somewhat more optimal.

• Now an “external model” is featured – somewhat else guiding Now an “external model” is featured – somewhat else guiding

attention.attention.

• If the number of channels increases too much, people fail to If the number of channels increases too much, people fail to

take advantage of the preview – because of the heavy take advantage of the preview – because of the heavy

memory load.memory load.

VISUAL SCANNINGVISUAL SCANNING

Processing Strategies – Cognitive TunnelingProcessing Strategies – Cognitive Tunneling• Scanning behaviours may reflect the operator’s Scanning behaviours may reflect the operator’s

mental model of the environment. This includes mental model of the environment. This includes

biasesbiases in the operator’s strategies. in the operator’s strategies.

• Fixation on feedback on an event can be a Fixation on feedback on an event can be a

substantial waste of visual attention.substantial waste of visual attention.

PARALLEL PROCESSING AND PARALLEL PROCESSING AND DIVIDED ATTENTIONDIVIDED ATTENTION

• Divided attention and parallel Divided attention and parallel processing is often a good thing (air processing is often a good thing (air traffic controllers).traffic controllers).

• However, it is sometimes very difficult However, it is sometimes very difficult to narrow the focus of attention when to narrow the focus of attention when needed and shut out unwanted inputs.needed and shut out unwanted inputs.

• This failure occurs when divided This failure occurs when divided attention becomes attention becomes mandatorymandatory rather rather than than optionaloptional..

Preattentive Processing and Perceptual Preattentive Processing and Perceptual OrganizationOrganization

• Preattentive phase is carried out Preattentive phase is carried out automatically and organizes the visual automatically and organizes the visual world into objects and groups of world into objects and groups of objectsobjects

Preattentive Processing and Perceptual Preattentive Processing and Perceptual OrganizationOrganization

• Grouping together of Grouping together of similar items on a similar items on a display.display.

• Knowledge of where Knowledge of where one is on the display one is on the display (proximity, similarity, (proximity, similarity, common fate, good common fate, good continuation, closure) continuation, closure) will allow an accurate will allow an accurate guess of where others guess of where others are located.are located.

Preattentive Processing and Perceptual Preattentive Processing and Perceptual OrganizationOrganization

• Because all items of an organized Because all items of an organized display must be processed together to display must be processed together to reveal organization, such parallel reveal organization, such parallel processing is sometimes called processing is sometimes called GLOBALGLOBAL or or HOLISTICHOLISTIC processing, in processing, in contrast to contrast to LOCALLOCAL processing. processing.

Global vs LocalGlobal vs Local• Global Display Global Display

Organization (top) Organization (top) allows for faster allows for faster identification of identification of Local abnormality.Local abnormality.

• Top display likely Top display likely has reduced:has reduced:– Attentional demandsAttentional demands– Risk for missed Risk for missed

eventevent

The Proximity Compatibility PrincipleThe Proximity Compatibility Principle

Display ProximityDisplay Proximity – how close together – how close together two display components are in spatial two display components are in spatial termsterms

Processing ProximityProcessing Proximity – the extent to – the extent to which two information sources are which two information sources are used within the same task (compare or used within the same task (compare or integrate).integrate).

The Proximity Compatibility PrincipleThe Proximity Compatibility Principle

• Close proximity will increase the possibility of Close proximity will increase the possibility of parallel processing by moving both dimensions into parallel processing by moving both dimensions into foveal vision. It is even more likely when objects are foveal vision. It is even more likely when objects are integrated a dimensions of a single object.integrated a dimensions of a single object.

• Emergent features (colour coding,symmetry) can Emergent features (colour coding,symmetry) can help in this respect.help in this respect.

Visualization of Complex Dynamic InformationVisualization of Complex Dynamic Information•VisualizationVisualization

•i.e. “i.e. “the use of computer-supported, interactive, visual the use of computer-supported, interactive, visual representations of abstract data to amplify cognitionrepresentations of abstract data to amplify cognition““(Card et al., 1999) (Card et al., 1999)

•supports cognition bysupports cognition by

– Relief of resources for memory and information Relief of resources for memory and information processingprocessing

– Facilitation of information retrievalFacilitation of information retrieval

– Recognition of hidden patterns, tendencies etc. Recognition of hidden patterns, tendencies etc.

– Reasoning at a perceptual level („direct Reasoning at a perceptual level („direct perception“)perception“)

– Improving visual attention during monitoring / Improving visual attention during monitoring / supervisory taskssupervisory tasks

Goal: Reduction of effort for information access and processing

Memory

Decision / response selection

Long-termmemory

Workingmemory

Perception

Proximity Compatibility Principle Proximity Compatibility Principle ((Wickens & Carswell, 1995)Wickens & Carswell, 1995)::Object Integration and Emergent FeaturesObject Integration and Emergent Features

•““Normal States” are visualized by graphical attributes Normal States” are visualized by graphical attributes likelike

– Symmetry,Symmetry,

– Regular Alignment or Regular Alignment or

– ParallelismParallelism

•

•Deviations from Normal States generateDeviations from Normal States generate– Asymmetry,Asymmetry,– Irregular Alignment,Irregular Alignment,– Skewness.Skewness.

(Images: Wickens, 1992)

• Idea:Idea: Utilisation of human skills for pattern-recognitionUtilisation of human skills for pattern-recognitionby means of graphical transformation of complex databy means of graphical transformation of complex data

Comparison of Mental Transformations:Comparison of Mental Transformations:Conventional vs. “Direct Perception“ Polar DisplaysConventional vs. “Direct Perception“ Polar Displays

Reference area= situation knowledge

Emergentfeatures

Actual value Rule ID-Crits (@ memory)

> ISR ISR = 30 NM true

Result

SU

Polar DisplayAir track

uncritical

Air track

critical

n * Integration: actual value vs. setpoint Integrationof attributes

DST 50

ALT 164

ESM Down Beat

…

EM EMciv/FF EMciv/FF = {…} false

ALT ALTciv ALTciv = [5000;FL 250] false

Conventional Display

PCP: Display Proximity PCP: Display Proximity ~~ Task Proximity & Integration of Task Proximity & Integration of KnowledgeKnowledge

Physical proximity ofaltitude and vertical speed

Proximity of frequently used displays

Basic-T

conventional HUD

Position and flight path have to be extracted by means of several flight control displays.

Integrated Display

Direct perception of position and flight path

Position and flight path have to be extracted by means of several flight control displays.Mental transformation to outside view is required.

Air Picture of a Combined Air Operations Air Picture of a Combined Air Operations Centre (CAOC)Centre (CAOC)

Usage by toolbox using well-known methaphers:

Scrolling

Cartographic zooming

Track search

Formation editing

Identification / Classification Display settings

predefined filters, styles, etc.

Visualization:

Tracks & related events and actions

A-priori information:e.g. airways, corridors, (restricted) areas etc.

Interactive ENC-chartS-57 / VMAP L1

Event- and Threat-triggeredPolar Displays

Graphical Representation of InformationGraphical Representation of Informationand Integration of A-priori Knowledge for AAW Situation Analysisand Integration of A-priori Knowledge for AAW Situation Analysis

Tactical SituationDisplay

Track Detail Display

User SupportDisplay

Polar Displays Toolbox

Secondary Display for Detailed InformationSecondary Display for Detailed Information

Relief of working memory:

Parameter ranges / tendencies

Predefined tolerances for non-critical values

State classification (IFF/ESM)

History plots: Values / states along time / distance

Information on demand

Utilize pattern recognition:

Heterogeneous values cause emergent line characteristics

Parameter constellations (Alt/Spd)

Graphical representation of user support system “explains”results

Ecological Interface Design (EID)Ecological Interface Design (EID)

Vicente & Rasmussen (1992)Vicente & Rasmussen (1992) •Approach: Approach: System errors result from violations ofSystem errors result from violations of

system operation constraints…system operation constraints…• Unexpected errors can be managed,Unexpected errors can be managed,

– if the operator possesses a valid mental model of the system ANDif the operator possesses a valid mental model of the system AND

– if the system constraints are visualized referring to environmental factors influencing if the system constraints are visualized referring to environmental factors influencing system behavior.system behavior.

•Constraints of system operationsConstraints of system operations– Physical attributesPhysical attributes temperature, radiation, illumination etc.temperature, radiation, illumination etc.

– System functionalitiesSystem functionalities different operational phasesdifferent operational phases

– Structural aspects of the work systemStructural aspects of the work system flexible vs. determinated work systemflexible vs. determinated work system

– Organizational structureOrganizational structure hierarchies, safety culturehierarchies, safety culture

– Working climateWorking climate restrictions through rules, restrictions through rules, competitors competitors

•Fundamental conceptsFundamental concepts– Abstraction hierarchiesAbstraction hierarchies

– Visualization of constraints: Invariants, system structure and interrelationsVisualization of constraints: Invariants, system structure and interrelations

– Taxonomy of human information processing (Rasmussen, 1983) Taxonomy of human information processing (Rasmussen, 1983)

EID: Abstraction HierarchiesEID: Abstraction Hierarchies

•Abstraction hierarchies offer different views which differ regarding level of Abstraction hierarchies offer different views which differ regarding level of detail and resulting information elements.detail and resulting information elements.

•Hierarchy levels:Hierarchy levels:– Functional purposeFunctional purpose::

Purpose which should be accomplished by the system Purpose which should be accomplished by the system – Abstract functionAbstract function::

Defined process / system structure Defined process / system structure regarding mass, energy, information or information flow. regarding mass, energy, information or information flow.

– Generalized functionGeneralized function: : Basis functions of system components. Basis functions of system components.

– Physical functionPhysical function:: Functional characteristics and connections of system components.Functional characteristics and connections of system components.

– Physical formPhysical form:: Structure and position of system components. Structure and position of system components.

•Each level represent another class of constraints for system operation.Each level represent another class of constraints for system operation.

Influencing Factors in Adaptive SystemsInfluencing Factors in Adaptive Systems

• SituationSituation– dynamic:dynamic: Objects / prioritized tasks, system conditionObjects / prioritized tasks, system condition– static:static: Knowledge (Scenario, Situation, Competences)Knowledge (Scenario, Situation, Competences)

• needs & possibilities of actionsneeds & possibilities of actions(“What should be done?”)(“What should be done?”) adviceadvice

•

• Operator activityOperator activity– Input sequences (Input sequences ( supervisory control) supervisory control)– Focus of attentionFocus of attention

•

• actual operator activityactual operator activity(“Which task is in focus?”)(“Which task is in focus?”) directed assistancedirected assistance

• Operator Functional State (OFS)Operator Functional State (OFS)– Estimation of resource capacityEstimation of resource capacity

and operator capability (normative / individual)and operator capability (normative / individual)

• need for user support need for user support dynamic regulation dynamic regulation (“Overload State?”) (“Overload State?”) of input loadof input load

Eye-based Assessment of Operator Functional State in Eye-based Assessment of Operator Functional State in Adaptive SystemsAdaptive Systems

•Remote / Head-based Eye TrackingRemote / Head-based Eye Tracking– Point of gaze, scan pathsPoint of gaze, scan paths– Oculomotoric parametersOculomotoric parameters

Point of interestPoint of interest Mental workload, vigilanceMental workload, vigilance

•Assessment techniquesAssessment techniques– Corneal-retinal potential (EOG)Corneal-retinal potential (EOG)– Cornea reflex infrared ocularographyCornea reflex infrared ocularography– Marker-based trackingMarker-based tracking

eye blinkfrequency

fixationfrequency

Indicators forvisual input load

ATTENTION IN THE AUDITORY MODALITYATTENTION IN THE AUDITORY MODALITY

• Auditory Sense can take input from any Auditory Sense can take input from any direction and thus there is no analog to direction and thus there is no analog to visual scanning as an index of selective visual scanning as an index of selective attention.attention.

• Most auditory input is transientMost auditory input is transient– Thus preattentive characteristics (organize Thus preattentive characteristics (organize

mentally) is more difficult after the event mentally) is more difficult after the event has passed (sound stops)has passed (sound stops)

Auditory Divided AttentionAuditory Divided Attention

• A general theory states that unattended A general theory states that unattended channel of auditory input remains in channel of auditory input remains in preattentive short-term auditory store preattentive short-term auditory store for about 3-6 seconds.for about 3-6 seconds.

• The contents of this store can be The contents of this store can be examined if a conscious switch of examined if a conscious switch of attention is made.attention is made.

Auditory Divided AttentionAuditory Divided Attention

• Even unattended auditory channels Even unattended auditory channels may make content with long-term may make content with long-term memory.memory.– Car horn honkingCar horn honking– Your name being spokenYour name being spoken– These are not meaningless blobs of noise!These are not meaningless blobs of noise!

• It is possible to think of an “auditory object’ It is possible to think of an “auditory object’ as a sound with several dimensions – thus as a sound with several dimensions – thus parallel processing can occur.parallel processing can occur.

• Auditory warning alerts have been designed Auditory warning alerts have been designed to capitalize on our parallel processing to capitalize on our parallel processing ability using redundant dimensions:ability using redundant dimensions:– PitchPitch– TimberTimber– Interruption RateInterruption Rate

– All in various combinationsAll in various combinations