Cognitive Task Analysis of Teams (Team CTA)

Team CTATeam CTA

Team CTATeam CTA

Cognitive Task Analysis

of Teams-Yogesh Tadwalkar

March 22, 2004

Team CTATeam CTA

Traditional Task Analysis

Task Analysis aims to optimize work performance by matching tasks with human capabilities and limitations

Human work is comprised of both physical and cognitive activities

Physical tasks are observable (Overt) However, cognitive processes that lead to

physical actions are not (Covert)

Team CTATeam CTA

Cognitive Task Analysis (CTA) Extension of traditional task analysis

techniques to uncover information about:– - Work domain, mental

demands on operator

– - Goal Structures – - Knowledge and Strategies– - Thought Processes and

Decision Making

Thus, providing a holistic, complete view of human work

Team CTATeam CTA

Goals of CTA

Performance improvement through: – Better Task and Interface Design– Ideal Personnel Selection– Comprehensive Training and

Instructions – Systematic Performance Assessment– Improved Planning

Team CTATeam CTA

Cognitive Task Analysis

of Teams

Team CTATeam CTA

Why Conduct Team CTA? Because 2 + 2 ≠ 4

– Individuals are often required to perform in teams

– Optimum individual performances may not lead to final mission accomplishment if team co-ordination is unsatisfactory

Thus, individual CTA would not be sufficient for designing optimum task performance

Team CTATeam CTA

Team CTA

As defined by Klein:– A reflection and representation of the

team mind / Knowledge – Description of the cognitive skills

required for effective team performance– Elaboration of the team decision

requirements– Determination of types of expertise

found in effective teams

Team CTATeam CTA

Objectives of Team CTA

Team performance enhancement through: – Restructuring and Resizing the team – Designing better:

• Information technology• Information management strategies• Human-computer interfaces• Decision support systems and Communication

– Developing methods for team training and selection

Team CTATeam CTA

Team Environment

Team CTATeam CTA

Team Environment Tasks are decomposed and

assigned to different team members

Team members may or may not have direct contact with each other (co-located vs. dispersed teams)

Tasks can be performed serially or

in parallel or overlapping fashion

Team members possess, acquire and process information, make decisions, solve problems and make plans

Team CTATeam CTA

Team Environment

Teams are intelligent entities (Thordsen and Klein, 1989)– Team members possess, acquire and

process information – Communicate with each other – Make decisions – Solve problems and – Make plans

Team CTATeam CTA

Types of Teams

Planning Teams– Military Command– Emergency Response at nuclear plants

Action Teams– Air Traffic Control– SWAT

Team CTATeam CTA

Cognitive Processes of TeamsWhat to measure through Team CTA methods

Team CTATeam CTA

Cognitive Processes of Teams Methods for team CTA should focus at capturing

these 5 processes (Klein):

1. Shared mental models- Team Goals, team member roles

2. Control of attention- Information seeking, filtering, communication

3. Shared situation awareness- Assessment of dynamism, prediction of future

4. Application of strategies - To make decisions, solve problems and plan

5. Metacognition- Self-monitoring, sensing trouble, making alternative plans

Team CTATeam CTA

A Model for Team Cognition

Knowledge

Mental Models

Attitudes

Expectations

Skills

Member A

Member CMember B

Team Performance

Pre-task Knowledge• Shared Mental Models• Strategies

Dynamic Knowledge• Control of Attention• Shared Situation Awareness• Metacognition

(Stout, Cannon-Bowers, and Salas, 1996)

Team CTATeam CTA

1. Shared Mental Models The degree to which team members have

the same understanding of key processes (Cannon-Bowers, Sales, & Converse, 1992) - Knowledge about overall team

goals and mission objectives e.g. Football game

- Roles and responsibility of each team member

- Team-mate characteristics and preferences

- Task Procedures, sequences and timing - Use and relationship of equipment

Team CTATeam CTA

Shared Mental Models and Team Performance Shared mental models affect team

performance significantly (Oransu, 1990): – Anticipating team member behaviours and

information requirements– Performing tasks from a common frame of

reference– Attending to, interpreting, communicating

about, and responding to the world more similarly than individuals with discrepant knowledge (Rentsch & Hall, 1994)

Team CTATeam CTA

Methods to Elicit and RepresentShared Mental Models

Team CTATeam CTAVariation of Goal-Directed Analysis (Woods & Hollnagel, 1987)

To solicit knowledge of Team Goals and Task Objectives

– Document Analysis and Observation– Separate interviews with each team

member, followed by group interview – Representation: Agreement Metric

Task Goal

Sub-Goal 1

Sub-Goal 2

Sub-Goal 3

Task 1 Agreement Metric

Team Goals

For Each Member For Whole Team

Task 2

Team CTATeam CTAVariation of Task-Action Mapping (Coury et al., 1991)

To solicit knowledge of Task Procedures, Sequence and Timing

– Document Analysis and Observation– Ask each team member to describe key

team concepts, associated tasks and specific actions

– Representation: Concept Maps

Description

Task 1

Task 2

Task 3

Action 1• What (Steps)• Who• WhenConcept 1

e.g. EvacuationAction 1• What (Steps)• Who• When

Team CTATeam CTA

Bootstrapping (Hoffman, Shadbolt, Burton & Klein 1995)

To solicit knowledge of Roles and Responsibilities

Bootstrapping

Documentation

Individual Interviews

ObservationGroup

Interviews

May not adequately capture dynamic, contextual data like misinterpretations, omissions due to environmental factors

Team CTATeam CTAAdapted Critical Incident Method(Flanagan, 1954; Critical Decision Method by Klein, 1989)

To solicit knowledge of Roles and Responsibilities, Strategies, Attitudes and Personal Characteristics– Document Analysis and Recall

of a critical past event e.g. fire, accident, high-stake mission.

– Ask each team member to describe various decisions taken, action triggers, who did what vs. who was supposed to do what

– Describe how each teammate reacted to the situation (perception, prediction, response time, stress, flexibility in co-ordination, etc.)

Team CTATeam CTA

2. Control of Attention The way a team engages in information

management - Information seeking - Filtering irrelevant information - Allocating attention to

important functions - Properly distributing messages

Refers to the team’s working memory– The way it uses limited resources for processing

simultaneous messages (Klein)

Team CTATeam CTA

Methods to Elicit and RepresentControl of Attention

Team CTATeam CTA

Scenario based Questionnaires (Blickensderfer, 1997)

To solicit knowledge of Information Management Strategies, and Communication Skills

– Document Analysis, observations and interviews to create a questionnaire covering a team work scenario, given to each team member

– Answers to outline relevant information protocol for the scenario, missing information, cues for identifying missing data, strategies to acquire and distribute information, and possible communication bottlenecks

Team CTATeam CTA

3. Shared Situation Awareness Extent to which team members have the

same interpretation of ongoing events (Cannon-Bowers, Sales, & Converse, 1992)

Is giving full data to each member a remedy for discrepant situation Awareness?

–How different team members perceive dynamic situations and events–How they form divergent impressions e.g. police in a riot situation–How they update each other

Team CTATeam CTA

Methods to Elicit and RepresentShared Situation Awareness

Team CTATeam CTA

Adapted Situation Awareness Global Assessment Technique (SAGAT)(Endsley, 1995)

To solicit knowledge of compatible situation awareness– Bootstrapping to create a mission simulation

– Asking team members to perform mission

– Freezing mission after specific intervals

– Asking each team member to analyze the situation and predict outcome and strategize

– Compare results for all members and assess compatibility and reasons for discrepancy

Team CTATeam CTA

Situation Awareness Linked Indicators Adapted to Novel Tasks (SALIANT) (Muniz et al. , 1998)

To solicit knowledge of compatible situation awareness 1. Delineate behaviours theoretically linked to team situations

(24 behaviours, 5 clusters).• Being able to predict next event• Managing resources (technical, system , internal, team)• Managing time• Feeling of being in control; able to implement elegant solutions• Taking the right decision at the best moment • Detecting mismatches, etc.

2. Development of task scenarios3. Identification of specific, observable responses for each

scenario4. Development of a script (to be administered to a team

member) 5. Observation form: Presence or Absence of response behavior

Team CTATeam CTA

4. Application of Strategies Every team has a prepared

list of steps, and routines for decision making, problem solving and planning

However, a team also develops shortcuts and workarounds not codified in procedures, as it gains experience

A skillful team knows its:– Key decision makers

(to be supported)– Key sources of expertise

(to be consulted) Time is critical in strategy

execution

Team CTATeam CTA

Methods to Elicit and RepresentTeam Strategies

Team CTATeam CTA

Retrospective Protocol Analysis (Means, 1993)

To solicit knowledge of agreement on strategies

– Based on non-interruption– Use video-tapes of similar mission or

actual team performance recorded previously

– Asking each team member to analyze the situation and justify strategies taken

– Analyze team agreement

Team CTATeam CTA

5. Metacognition A team needs to

monitor itself during ongoing task performance

To determine whether and when it is running into difficulty

Predict consequences of the difficulty

Determine when and how it needs to shift its strategies e.g. Police at riot situation

Team CTATeam CTA

Methods to Elicit and RepresentMetacognition

Team CTATeam CTA

Mission Simulation (Klein)

To solicit knowledge of self-monitoring or metacognition

– Bootstrapping to create a mission simulation– Asking team members to perform mission – Observing how team members recognize and

collect more information about critical incidents

– Conducting individual interviews with each team member to analyze the video-recording

– Analyzing gaps in team agreement

Team CTATeam CTA

Representing Team CTA Results

Team CTATeam CTA

Methods for Representing Team CTA Results Dealing with the ‘Envisioned World Problem’ (Woods, in

press)

– Creating prototypes and simulations to gauge how team CTA would impact the future

– Iterative testing

World Team CTA Prototypes of Changed

World

Team CTATeam CTA

Limitations / Challenges in Applying Team CTA Methods

Team CTATeam CTA

Challenges in Applying Team CTA Methods Knowledge elicitation

– Knowledge is internalized and even experts find it hard to verbalize it

– Retrospective account of work may differ from reality

Capturing dynamism – In many cases, team strategies differ

vastly from one situation to another – Generalizing based on a few situations

may be inappropriate

Team CTATeam CTA

Summary

Team CTA is critical to obtain a holistic (both overt and covert tasks) view of human work

Teams are intelligent entities with 5 key cognitive processes (Klein)

1. Shared mental models2. Control of attention3. Shared situation awareness4. Application of strategies to make decisions,

solve problems and plan5. Metacognition

Team CTATeam CTA

Summary

The 5 key processes can be spread over 2 types of team knowledge

– Pre-task knowledge– Dynamic Knowledge

Team CTA Methods should aim at eliciting team knowledge of these 5 processes

Team CTATeam CTA

Summary

Team CTA can yield significant performance improvement of teams by:

– Restructuring and Resizing the team – Designing better:

• Information technology• Information management strategies• Human-computer interfaces• Decision support systems and Communication

– Developing methods for team training and selection

Team CTATeam CTA

References Gary Klein

– Cognitive Task Analysis of Teams (1995) Cannon-Bowers, Sales, Baker

– Analyzing knowledge requirements in team tasks– Shared Mental Models in Expert Team Decision Making (1993)

Hoffman, R.R.– Use of Critical Incidents to Elicit Knowledge (1995)

Oransu, J.– Shared Mental Models and Crew Performance (1990)

Coury, B.G., Motte,S., & Seiford, L.M. – Capturing and representing decision processes in the design of

an information system Muniz, E.J., Stout, R.J., Bowers & Salas

– A methodology for measuring team situation awareness: Situation Awareness Linked Indicators Adapted to Novel Tasks (SALIANT) (1998)

Woods, D.D. & Hollnagel, E. – Mapping cognitive demands in complex problem solving worlds

(1987)

Team CTATeam CTA

Questions ?

Team CTATeam CTA

Cognitive Task Analysisof Teams

Thank You !