Cognitive Task Analysis and its Application to Restoring System Security by Robin Podmore, IncSys Frank Greitzer, PNNL.

Cognitive Task Analysisand its Application

to Restoring System Security

byRobin Podmore, IncSysFrank Greitzer, PNNL

Cognitive Task Analysis Background• Historically Task Analysis provides a list of tasks and subtasks as well as

input, actions, outputs, conditions for performance and proficiency requirements.

• Because we live in an age of Technological Complexity many jobs involve complex cognitive processing, rather than simply a collection of overt or observable behaviors.

• Analysts try to develop a model or theory of how experts in a field perform their tasks.

• This model can then be used to develop system interfaces, decision aids and training programs

• Cognitive Task Analysis is the extension of traditional task analysis techniques to yield information about the knowledge, thought processes, and goal structures that underlie observable task performance.

Respond to Forced Transmission OutageTask List

• Evaluate system conditions • Interpret relay targets, recorder readings, breaker operations, and

field observations to identify the location and cause of the event • Obtain details of equipment problem including estimated time of

return• Call out system personnel when appropriate • Direct steps needed to sectionalize a transmission line in order to

locate fault • Implement a plan for restoring the system to a safe operating

condition• Coordinate response with other affected systems as applicable • Implement a plan to restore system security including preparing for

the next contingency • Complete outage log, and update applicable displays in

SCADA/EMS • Make all required notifications.

CTA Objectives• Analysts speak with experts, journeymen, novices or any combination and

analyze their behavior to determine:

– Concepts and principles, their relationship to each other and their relationship to the tasks

– Goals and goal structures, including methods for achieving goals and their triggers

– Cognitive skills, rules, strategies and plans

– Perceptual learning, pattern recognition, implicit and explicit knowledge

– Mental models – how experts represent and run models of the system

– Problem models – how experts represent a problem and work within the problem space

– How novices move through all of the above in various stages in order to become experts

– Difficulties in acquiring domain knowledge and skills

– Instructional procedures useful for moving a person from novice to expert

Cognitive Task Analysis Framework

ODM and Cognitive

Task Analysis

Mental Model and Story Analysis

State and Situation Awareness Level Analysis

Object and Action Analysis

Conduct and Debrief Scenario – Script from Operator Interview

Scenario – Objectives, Base Case, Events, Roles

Simulator Environment – Power System Simulator, Control Center Model, Operator Consoles, Projection Map Board

Operator Decision Model

Real World Short termmemory

Long termmemory

Situation

Cues

That affect the

That let youcreate

That activate

Generates

Which youAssess by

Using your

Which youValidate by

Using your

TimeAvailable?N

MentalSimulation

Mental Simulations

ActionScript

Story(Patterns)

MentalModels

MentalModels

Y

Graphical Contingency Analysis (GCA)

• Contingency analysis, as a key function in the Energy Management System (EMS),

• Assesses the ability of the power grid to sustain various combinations of power grid component failures based on state estimates.

• Presently, most of the industrial software tools installed in control centers use tabular data to represent system status, which is not easy to interpret when the system is stressed

NORMAL

RESTORATIVE EMERGENCY

INSECURE

Preventative Action

Remedial Action

Island Blackout

Outage, Load Increase

Restoration Action

Power System States

Contingency Analysis Output - Example

Graphical Contingency Analysis Output – Crawford Baker 1 & 2 Outage

Contingency Analysis Output

Graphical Contingency Analysis Output Farlie – Grange 1 & 2 Outage

Scenario• Prebrief

– System operating at 2:00 PM, Summer peak conditions

– Crawford – Oakdale 1 & 2 out for maintenance

– Lightning Storm activity in Northeast and Southwest regions

• Simulation – 20 minutes– Restore system to a secure state based on Contingency

Analysis

– Operator plays all the roles

• Debrief – 20 minutes– Walk through recording of screen activity

– Interview on operator thought process

Participants

• Experimental Group– Three participants– Using Graphical Contingency Analysis

• Control Group– Three participants– Using Tabular Contingency Analysis

• Common Requirements– NERC Certified System Operators– Some prior experience with PowerSimulator and

PALCO– Participants blind on Control and Experiment

Groups

PNNL – Electricity Infrastructure Operations Center

Human Factors Test Laboratory

Scenario Debrief and Interview

Scenario Debrief and Interview Close Up

Example of Actions Implemented

• Decrease Farlie output from 900 MW to 600 MW• Increase Doyle output from 357 MW to 550 MW• Increase PALCO import from External from 0

MW to 300 MW.

IncorrectCritical

Assessment

IncorrectCritical

Assessment

IncorrectCritical

Assessment

Situation Awareness

Analysis

Security Restoration Requirements

• Identification of every critical contingency • Identification of all critical system constraints• Identification of impact of re-dispatch on other

critical contingencies and constraints

Operator Decision Model - Enhanced

Real World Short termmemory

Long termmemory

Situation

Cues

That affect the

That let youcreate

That activate

Generates

Which youAssess by

Using your

Which youValidate by

Using your

TimeAvailable?N

Mental andComputer

Simulations

Mental andComputer

Simulations

ActionScript

Story(Patterns)

Mental andComputer

Models

Mental andComputer

Models

Y

Conclusions

• Generic PowerSimulator with PALCO system is an effective tool for Cognitive Task Analysis

• The environment is realistic and creates tension and stress for certified system operators

• Replay of dynamic screen capture provides basis to capture operator’s thought process.

• Level of wide area and local situation awareness can be identified at each step of the scenario.

• Mental models and mental simulations can be identified at each step of the scenario.

Cognitive Task Analysis Benefits

• Improved Curriculum– Need to add Contingency Analysis to PowerSimulator

• Improved Visualization Tools– Graphical vs. Tabular Contingency Analysis

• Improved – more specific, precise and constructive feedback to system operators

• Improved capture of expert knowledge to accelerate the spiral of knowledge

• Improved ability to learn from mistakes of others• Improved power system applications• Improved insight into the Operator Decision Model