-
Designing for Wide-Area Situation Awareness in Future Power
GridOperations
by
Fiona F. Tran
A thesis submitted in conformity with the requirementsfor the
degree of Master of Applied Science
Graduate Department of Mechanical and Industrial
EngineeringUniversity of Toronto
c© Copyright 2016 by Fiona F. Tran
-
Abstract
Designing for Wide-Area Situation Awareness in Future Power Grid
Operations
Fiona F. Tran
Master of Applied Science
Graduate Department of Mechanical and Industrial Engineering
University of Toronto
2016
Power grid operation uncertainty and complexity continue to
increase with the rise of electricity market
deregulation, renewable generation, and interconnectedness
between multiple jurisdictions. Human op-
erators need appropriate wide-area visualizations to help them
monitor system status to ensure reliable
operation of the interconnected power grid. We observed
transmission operations at a control centre,
conducted critical incident interviews, and led focus group
sessions with operators. The results informed
a Work Domain Analysis of power grid operations, which in turn
informed an Ecological Interface De-
sign concept for wide-area monitoring. I validated design
concepts through tabletop discussions and a
usability evaluation with operators, earning a mean System
Usability Scale score of 77 out of 90. The
design concepts aim to support an operator’s complete and
accurate understanding of the power grid
state, which operators increasingly require due to the critical
nature of power grid infrastructure and
growing sources of system uncertainty.
ii
-
Acknowledgements
This research project was made possible with the support of many
people. I would like to thank my
supervisor Prof. Greg A. Jamieson for initiating and supporting
this project throughout; my colleague
Dr. Antony Hilliard for his mentorship, guidance, and
collaboration; and to several people from the
Independent Electricity System Operator (IESO): Len Johnson,
David Short, Steven Ferenac, David
Devereaux, Nicola Presutti, Kim Warren, and all the operators
and engineers who volunteered to par-
ticipate in interviews, observations, and evaluation
discussions. I am grateful to have received generous
support from the IESO, Mitacs, Province of Ontario, and
University of Toronto during my studies. Last
but certainly not least, I would like to thank my friends and
family for supporting me in my career and
my life outside of work.
iii
-
Contents
List of Tables viii
List of Figures ix
Abbreviations x
1 Introduction 1
1.1 Power Grid Operations . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 1
1.1.1 Wide-Area Monitoring . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 2
1.1.2 Human Factors in Power Grid Operations . . . . . . . . . .
. . . . . . . . . . . . . 2
1.2 Ecological Interface Design . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 2
1.3 Objectives . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 3
2 Scoping Study of Power Grid Visualizations 4
2.1 Power System Overviews . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 5
2.1.1 Colour Contours . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 6
2.1.2 3D Displays . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 6
2.1.3 Modelling Very Large Interconnected Grids . . . . . . . .
. . . . . . . . . . . . . . 6
2.1.4 Correlated Multi-Screen Displays . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 7
2.1.5 Hypermedia . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 7
2.1.6 Wide-Area Measurement Systems . . . . . . . . . . . . . .
. . . . . . . . . . . . . . 8
2.1.7 Decision Support Systems . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 8
2.1.8 Electricity Market Monitoring Systems . . . . . . . . . .
. . . . . . . . . . . . . . . 9
2.2 Operator Effectiveness . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 9
2.2.1 Mitigating Situation Awareness Errors . . . . . . . . . .
. . . . . . . . . . . . . . . 10
2.2.2 Distributed Situation Awareness . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 11
iv
-
2.2.3 Cognitive Workload . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 12
2.2.4 Effect of Expertise . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 12
2.2.5 Operator Training . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 13
2.3 Research Opportunities . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 13
2.4 Limitations of the Scoping Study . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 15
2.5 Scoping Study Conclusions . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 15
3 Knowledge Elicitation 17
3.1 Critical Incident Interviews . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 17
3.1.1 Methodology . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 18
3.1.2 Results and Discussion . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 18
3.2 Focus Group Sessions . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 20
3.2.1 Methodology . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 20
3.2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 21
3.2.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 22
4 Work Domain Analysis 23
4.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 23
4.2 Summary of the WDA . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 23
4.2.1 Abstraction Hierarchy . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 23
4.2.2 Means-Ends Links Between Abstraction Levels . . . . . . .
. . . . . . . . . . . . . 25
4.2.3 Part-Whole Decomposition . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . 25
4.2.4 Topographic/Causal Links . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 25
4.3 Application of the WDA to Wide-Area Monitoring . . . . . . .
. . . . . . . . . . . . . . . 26
4.3.1 Design Scope in the Abstraction Hierarchy . . . . . . . .
. . . . . . . . . . . . . . 26
4.3.2 Measures and Constraints at Different Abstraction Levels .
. . . . . . . . . . . . . 27
4.3.3 Using the Information Requirements . . . . . . . . . . . .
. . . . . . . . . . . . . . 28
5 Design and Evaluation of Wide-Area Monitoring Concepts 30
5.1 Preliminary Design Ideas and Feedback . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 30
5.1.1 Critical Parameters . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 31
5.1.2 External Geographical Scope . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 32
5.1.3 Modelling the Ontario Power Grid . . . . . . . . . . . . .
. . . . . . . . . . . . . . 33
5.1.4 Generation . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 35
v
-
5.1.5 Alarm Notifications . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 35
5.1.6 Contours . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 36
5.2 Design Prototyping . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 36
5.2.1 Context . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 36
5.2.2 First Interactive Prototype . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 36
5.3 Iterated Design Prototype . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 37
5.3.1 Wide-Area View . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 37
5.3.2 System-Wide View . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 38
5.3.3 Detailed Views . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 38
5.3.4 Alarms . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 39
5.4 Usability Evaluation . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 40
5.4.1 Methodology . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 41
5.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 42
5.5 Future Work . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 45
6 Discussion 47
6.1 Implications for Wide-Area Monitoring and Visualization . .
. . . . . . . . . . . . . . . . 47
6.2 Study Limitations . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 48
6.3 Future Research Areas . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 49
6.4 Industry Partner Feedback . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 50
7 Conclusion 52
APPENDICES 52
A Knowledge Elicitation 53
A.1 Control Room Ergonomics Questionnaire . . . . . . . . . . .
. . . . . . . . . . . . . . . . 53
A.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 58
A.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 58
B Pre-Usability Evaluation Wide-Area Monitoring Design Concept
60
C Usability Evaluation 65
C.1 Participant Feedback . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 65
C.1.1 Positive Feedback . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 65
C.1.2 Implemented Suggestions for Improvement . . . . . . . . .
. . . . . . . . . . . . . 66
vi
-
C.1.3 Suggestions for Future Work . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 67
C.1.4 Suggestions not Implemented . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 67
C.2 Usability Questionnaire . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 68
References 71
vii
-
List of Tables
3.1 Critical incidents identified from operator interviews. . .
. . . . . . . . . . . . . . . . . . . 19
5.1 Critical parameters of system operation. . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 31
5.2 Number of times each scenario question was answered
correctly during the usability eval-
uation (total: 9 participants). . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 43
viii
-
List of Figures
4.1 Abstraction hierarchy of power grid operations. Elements
relevant to wide-area monitoring
are highlighted in green boxes. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 29
5.1 Preliminary concept for displaying external jurisdictions
that affect the Ontario power grid. 32
5.2 Preliminary design for a one-line diagram showing links
between Ontario’s IROLs. . . . . 33
5.3a After tabletop discussion feedback: Stick-and-circle layout
for modelling Ontario regions
at the system overview level, showing generation and load
numbers. . . . . . . . . . . . . 34
5.3b A toggle in Figure 5.3a would also show the operating
reserve and spare generation numbers. 34
5.4 Preliminary design for alarm notification panel. . . . . . .
. . . . . . . . . . . . . . . . . . 35
5.5 Iterated design prototype: wide-area view that includes
external jurisdictions. . . . . . . . 37
5.6 Iterated design prototype: system view of Ontario power
grid. . . . . . . . . . . . . . . . . 39
5.7 Iterated design prototype: overview of the northwest section
of the Ontario power grid. . 40
5.8 Iterated design prototype: northwest overview with alarm
notification pane showing. . . . 41
5.9 System Usability Scale (SUS) responses on the usability
questionnaire. Error bars in all
graphs indicate standard error. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 44
5.10 Comparison of ease of use between the design prototype and
existing tools, as rated by
participants on the usability questionnaire. . . . . . . . . . .
. . . . . . . . . . . . . . . . 45
A.1 Responses to the control room ergonomics questionnaire at
the focus group sessions. . . . 59
B.1 Previous iteration design prototype: wide-area view that
includes external jurisdictions. . 61
B.2 Previous iteration design prototype: system view of the
Ontario grid (power flow view). . 62
B.3 Previous iteration design prototype: Hovering over OMTE
would show the contribution
of OMTE to the EWTE and FS power flows (i.e. what would happen
if the Manitoba
flow was suddenly zero). . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 62
B.4 Previous iteration design prototype: system view of the
Ontario grid (generation view). . 63
B.5 Previous iteration design prototype: overview of the
northwest section of the Ontario
power grid. . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 64
ix
-
Abbreviations
CWA Cognitive Work Analysis.
EID Ecological Interface Design.
EMS Energy Management Systems.
IESO Independent Electricity System Operator.
IROL Interconnection Reliability Operating Limit.
NERC North American Electric Reliability Corporation.
RC Reliability Coordinator.
SA Situation Awareness.
SCADA Supervisory Control and Data Acquisition.
SOL System Operating Limit.
SUS System Usability Scale.
WDA Work Domain Analysis.
x
-
Chapter 1
Introduction
Electrical power grid operation continues to increase in
complexity and uncertainty with the rise of
renewable generation [1], electricity market deregulation [2],
and cybersecurity risks [3]. These factors
are exacerbated by the tightly interconnected nature of power
grids between multiple jurisdictions [4].
The Reliability Coordinator (RC) for each jurisdiction is
responsible for the reliable operation of the
power grid within their jurisdiction. Human control room
operators monitor and control the power grid
with the help of automated Energy Management Systems (EMS). With
the growing complexity of grid
operations, EMS’s in turn need to adapt to prevent information
overload, encourage faster problem-
solving, and reduce error. Operators thus require new
visualization tools to summarize and display the
growing body of available and relevant information.
1.1 Power Grid Operations
An EMS forms the human-machine interface that gathers, computes,
and displays information about
the the grid state. In addition, it dispatches electricity
generation according to supply and demand, and
sets market prices based on near- and long-term forecast
demand.
Power grid operations by RCs are tightly regulated by standards
such as those set by the North
American Electric Reliability Corporation (NERC) [5] and the
North American Energy Standards Board
[6].
1
-
Chapter 1. Introduction 2
1.1.1 Wide-Area Monitoring
The NERC IRO-003-2 [7] standard mandates that RCs have wide-area
views of the Bulk Electric System
- generally defined as all power transmission elements and
real/reactive power sources that are operated
or connected at 100 kV or higher, and does not include local
power distribution elements [8]. The
wide-area view must include visibility within the RC Area and
adjacent RC areas, so that the RC may
determine any potential System Operating Limit (SOL) or
Interconnection Reliability Operating Limit
(IROL) violations within their respective RC Area.
Electric blackouts pose a severe risk to the welfare and
security of households, health and emergency
services, industry, and more. Large-scale blackouts, though
infrequent, may originate anywhere within
or connected to the RC Area and can propagate within
milliseconds. For instance, the major North
American blackout in 2003 started with a short-circuit and
caused cascading failures that affected 50
million people [9]. Among the U.S.-Canada Power System Outage
Task Force’s recommendations was
to improve Reliability Coordinators’ visualization capabilities
over wide geographical areas. Wide-area
monitoring and control systems thus play a crucial role by
helping determine grid security, and building
operators’ awareness of the system.
1.1.2 Human Factors in Power Grid Operations
While EMS’s are highly automated - human operators take on a
supervisory role and monitor the EMS
for abnormalities - human performance is critical for detecting
and resolving problems not handled by
the automation. Inadequate operator Situation Awareness (SA) has
been a widely cited factor in several
recent large electrical disturbances [10], making it critical
that EMS’s are designed to satisfy both the
technical constraints of power operation and the needs of the
humans using the system. One way to
accomplish this is through the Ecological Interface Design (EID)
[11, 12] approach.
The scoping study in Chapter 2 provides a more detailed overview
of the existing body of literature
on human factors in power grid visualizations.
1.2 Ecological Interface Design
The basis of the EID theoretical framework is designing to
minimize the opportunity for human error,
and support recovery from errors. EID has been used to develop
human-machine interfaces to control
complex systems, in domains including aviation, medicine,
manufacturing, and nuclear and hydro power
-
Chapter 1. Introduction 3
generation [13].
An important part of the EID process is Cognitive Work Analysis
(CWA), a framework for modelling
complex systems. CWA consists of 5 phases: Work Domain Analysis
(WDA), Control Task Analysis
(ConTA), Strategies Analysis (StrA), Social Organization and
Cooperation Analysis, and Worker Com-
petencies Analysis [14]. The CWA conducted in this study was
limited to the WDA, as obtaining
information requirements from the WDA is considered the most
formalized and systematic approach to
creating ecological displays [15].
To our knowledge, there has been no CWA, nor subset thereof,
published in the literature for the
power grid operations domain. Furthermore, EID has been sparsely
applied to creating wide-area visu-
alizations - the only examples found in the literature are
Rantanen et al.’s application of general display
guidelines [16] and the PEGASE project that modelled the
European power grid [17]. While these ex-
amples cite EID guidelines, they do not take information
requirements from a WDA, despite this step
being an important part of the EID process.
1.3 Objectives
The Cognitive Engineering Laboratory at the University of
Toronto partnered with the Independent
Electricity System Operator (IESO), the RC for the province of
Ontario, to work on designing and
evaluating displays for future power grid operations. My work
focused on visualizations for wide-area
monitoring.
The objectives of this research were to:
1. Develop an understanding of current human factors research
and issues in power grid operation;
2. Analyze operator work in the power grid control room;
3. Design and evaluate novel display concepts for wide-area
monitoring using EID.
The research-industry collaboration allowed the IESO to obtain a
third-party perspective of human
performance in their operations control room, built on standard
human factors principles. In addition,
the display design concepts were proposed for future tools to be
deployed in the control room.
-
Chapter 2
Scoping Study of Power Grid
Visualizations
To gain insight on the current state of the art in power system
visualizations for transmission operation,
we conducted a scoping study of the literature.
The scoping study framework was formalized by Arksey and
O’Malley in 2005 [18]. It is an extension
of the traditional literature review, incorporating broader
research questions with the goal of identifying
gaps in existing research literature. The methodology has been
widely used in health care, but research
synthesis in other areas such as software engineering remains
relatively untouched [19].
The scoping study consists of the stages: identifying the
research question; identifying relevant
studies; study selection; charting the data; and collating,
summarizing, and reporting the results.
Stage 1: Research questions: This scoping study pertains to the
current solutions for visualizing
growing power system complexity and future opportunities for
developing representation aids in this
area.
Stage 2: Relevant studies: Studies and reviews were found from
electronic databases (Inspec and
IEEE Xplore), and Google search engines (Scholar and Web). The
search terms were different combi-
nations of “power systems”, “electricity markets”, “dispatch”,
“visualization”, and “display”. Previous
studies in power systems visualization have largely introduced
new techniques and tools. Some have
additionally conducted usability studies; however, there has
been no comprehensive review of existing
tools and their usefulness to operators.
4
-
Chapter 2. Scoping Study of Power Grid Visualizations 5
Stage 3: Study selection: Although our topic was
multidisciplinary in nature, the search results were
dominated by power systems and electricity market modelling and
optimization algorithms rather than
user interfaces (UI). This may be partly attributed to the
ambiguity of the “visualization” and “display”
search terms - for example, an author may visualize their
mathematical model by plotting it on a graph.
To select papers for review, a preliminary reading of the
abstract, followed by reading the paper, helped
identify which were relevant to the study of control room
displays.
Stage 4: Charting the data: Studies were categorized according
to their topic of interest in the context
of the electrical transmission control centre. Theoretical
papers on future directions of power systems
and electricity market visualization were grouped last.
Stage 5: Results reporting : The results of the scoping study,
answering each of the research questions,
are described in the following sections.
2.1 Power System Overviews
Several new visualization techniques have been presented in the
literature over the years, although
empirical investigations have only begun in the past decade or
so to determine the effectiveness of those
techniques when employed in the control room.
Some visualization techniques for power systems are: tabular,
integrated, and colour-contoured one-
line. Tabular representations outline each measurement name and
corresponding value. One-line dia-
grams are a form of block diagram showing power flows, and are a
simplified graphical representation
of three-phase power systems. Integrated one-line diagrams have
the addition of showing voltage values
near the buses on the one-line.
When Overbye et al. compared the effectiveness of these three
techniques in the scenario of diagnosing
and solving low-voltage violations [20], they found that the
tabular format had the best acknowledgement
response times for low-voltage violations (1-2 seconds faster
than one-line), but users solved violations
8-9 seconds faster using the integrated one-line diagram. They
attributed this to high display proximity
[21] for the integrated information on the dynamic one-line
diagram. Colour contours (discussed below)
fared worst for acknowledgement response time, although contours
may be useful for larger real-life
systems than the hypothetical one used in the experiment.
-
Chapter 2. Scoping Study of Power Grid Visualizations 6
2.1.1 Colour Contours
Colour contours have been investigated for their effectiveness
in power system visualizations [22], since
human factors literature suggests colour codes can be
interpreted and compared faster than numeric
coding.
Display acknowledgement times do not differ among different
configurations of one-line diagrams
(number only, contour only, or number plus contour) for
low-complexity violations. For higher complexity
violations (i.e. greater number of violations), the contour-only
group was able to more quickly (taking
less than half the time) and accurately identify the low-voltage
bus than the number-only group. Solution
times are significantly slower for contour-only displays than
number-only ones, however. Overbye et al.
[22] suggest that for contour displays, once a violation has
been acknowledged, the contour may be
dimmed out or removed to aid in problem correction.
2.1.2 3D Displays
Three-dimensional (3D) displays of power system information have
been compared with conventional
two-dimensional (2D) displays, either one-line diagrams or in
tabular format [23]. The 3D display results
in faster solution times than for 2D graphical display, perhaps
because of the size and salience of the
generator representations in the 3D format. Precise judgements
are not required to solve line flow
violations, so the numerical tabular format would have no
advantages.
Overbye et al. [23] cite geographical data views (GDV) as a
viable technique for power system
visualization, by integrating information from the power system
model and geographic information.
GDVs therefore show a greater range of system information than
conventional wide-area visualizations.
2.1.3 Modelling Very Large Interconnected Grids
Power grids are highly interconnected, spanning large areas with
multiple independent system operators.
PEGASE is a European project to develop interfaces for power
grid control centres that operate very
large interconnected systems [17]. The PEGASE display provides a
map of the European power grid, with
countries coloured based on their operating state. The colour
scheme uses a traffic light analogy. Possible
operating modes are: normal (green), preventive (yellow),
corrective (red), or restorative (magenta). In
addition, black indicates a blackout and grey signifies no
information available.
-
Chapter 2. Scoping Study of Power Grid Visualizations 7
2.1.4 Correlated Multi-Screen Displays
Conventional multi-screen displays simply expand the effective
display area, and do not use their full
potential to adjust display contents quickly and conveniently.
Zhu et al. developed what they called a
correlated multi-screen display that allows operators to
indirectly modify multiple screens by controlling
just one of them [24]. The correlativity between screens (how
they would influence what is displayed on
each other) was organized as follows:
• Object correlation: 2 screens showing the same object but
different information on it (e.g. 2 maps
of the same area but different overlay content)
• Information correlation: 2 screens showing different objects
but the same information type (e.g. 2
maps of different areas but same type of overlay)
• Inheritance correlation: 1 screen showing wide area operating
state (main screen) and 1 screen
showing local detail (sub-screen which operator can control)
• Entirety correlation: 2 screens with correlated objects and
information, and whatever the operator
does on either screen, the other will change with it
The authors designed and developed an experimental platform
using the correlated multi-screen
display philosophy. They hypothesize that their platform would
allow operators to more conveniently
control objects on the screens, and perform tasks more quickly.
However, their system had not yet
undergone usability testing.
2.1.5 Hypermedia
Hypermedia is the use of interactive multimedia nodes, linked
together as a model of information rep-
resentation and management [25]. This linkage is relevant to
power grids, whereby buses and lines are
all interconnected to each other to form a complex system.
Moreno-Muñoz et al. propose hypermedia
UI design to optimize power grid data management [26], to
replace current tools that are mainly data
tables and vector-based graphical displays.
They created a hierarchy of displays, starting with the
substation view at the top level, since current
power systems are usually presented on a substation-centric map.
Clicking on each node would then lead
to a localized view showing connections to the equipment,
analogue measurements, and any additional
database information. The lowest hierarchy level would show
tables of raw variable values. Graphical
displays of power quality information would be available from
any screen so that operators can explore
-
Chapter 2. Scoping Study of Power Grid Visualizations 8
subsystems while understanding topology, composition, and
current status.
2.1.6 Wide-Area Measurement Systems
Supervisory Control and Data Acquisition (SCADA) has been used
in the past few decades for remotely
controlling equipment, and taking measurements every few
seconds. As transmission systems continue
to replace their aging assets, phasor measurement units (PMUs)
are sought as a replacement to SCADA
as they have much faster scan rates - up to 60 measurements per
second - and can directly measure bus
angles across systems [27]. This opens up the possibility of
real-time, direct visualization.
A large network that has deployed PMUs for measurement is known
as a wide-area measurement
system (WAMS). WAMS has been applied to a grid dispatching
system in China, and corresponding
visualization functions deployed on the control centre’s dynamic
displays of alarm and non-alarm infor-
mation [28]. The visual displays show basic limit violations,
power system disturbances, low-frequency
oscillations, and performance evaluations of adjoining operating
units.
2.1.7 Decision Support Systems
Moving on from experimenting with different visualization
techniques according to technological capabil-
ities, current research has examined how automation can assist
operators in decision-making and reduce
the likelihood of error.
Decision support systems advise the operator on what to do,
rather than doing the task itself. Done
incorrectly, decision support can decrease human performance, be
it taking longer to make decisions, or
forming independent assessments and succumbing to cognitive
anchoring [29]. When cues are targeted
correctly and the system provides advice after the operator has
made a decision (i.e. critiquing the
operator’s action rather than teaching the operator what to do
next), error rates may drop significantly.
One area of interest for decision support has been alarm
visualization. Tripathi et al. [30] developed
an alarm visualization concept that includes a filter to
prioritize important alarms. More critical alarms
are displayed in a larger font than non-critical ones. A root
cause diagram indicates other affected
devices in the network. Their rationale, supported by subject
matter expert feedback, was that the
salience of critical alarms improves alarm identification for
operators, and the root cause diagram helps
them analyze the alarm’s impact to resolve faults.
-
Chapter 2. Scoping Study of Power Grid Visualizations 9
2.1.8 Electricity Market Monitoring Systems
Aside from general power systems monitoring and control,
transmission dispatchers are also concerned
with market changes and how they may affect transmission status
and constraints. Market dynamics
are of particular concern to operators with the arrival of
deregulated electricity markets [2].
Independent system operators use market monitoring systems (MMS)
to ensure efficient market
performance, through detecting market inefficiencies, potential
abuses, and market power problems (i.e.
for a seller to sell at above the competitive rate). Effective
market monitoring software must encompass:
generation and transmission outages; supply, demand, and
transmission adequacy; potential or actual
market power abuses, and behavioural monitoring of market
participants [31].
One way to visualize market power is through colour contouring
[32] to show loading on each trans-
mission line. Line flows above a certain percentage are
highlighted, indicating a small generation market
available to a load pocket. Virtual reality environments can
qualitatively portray multiple layered sys-
tems on a 3D interface. For example, line flows and power
transfer distribution factors (PTDF) values
can be displayed for both the actual system and a proposed
transaction for comparison.
The deregulation of electricity markets has increased the
complexity of market operations, as they
now require collaboration between multiple operating
jurisdictions, existing market participants, and
new entrants [33]. Growth in renewable energy generation,
storage, and controllable loads has added to
the compelling need for effective market monitoring and dispatch
tools that merge with power system
operation.
2.2 Operator Effectiveness
While work in power systems and market visualization has largely
centred around developing new tech-
niques, there are gaps in the literature on how they can impact
the effectiveness of human operators.
Routine events are automated in transmission system control, but
human operators take a critical role
in managing emergency system operations [34]. The goal is to
provide operators with appropriate levels
of real-time information for them to make decisions. Of
particular interest are situation awareness and
cognitive workload.
-
Chapter 2. Scoping Study of Power Grid Visualizations 10
2.2.1 Mitigating Situation Awareness Errors
Situation awareness (SA) has been defined by Endsley [35] as
“the perception of the elements in the
environment within a volume of time and space, the comprehension
of their meaning, and the projection
of their status in the near future.” SA theory suggests that SA
is achieved in UI design by creating a
mental model for human operators that matches reality.
Operator SA is a key to preserving grid reliability [34].
Panteli et al [36] suggest best practices
for supporting human operators for increasingly complex modern
power grids, and outline methods for
dealing with problems associated with SA for power systems.
Advanced monitoring and decision-support
tools can support adequate SA, such that operators may gather
and interpret necessary information for
responding to incidents.
Panteli et al. [36] identified the main sources of SA errors in
transmission and distribution control
centres as:
• Software applications: application errors may cause operators
to be misinformed about defects,
and thus not react in time to mitigate them.
• Real-time measurements: missing or conflicting information
impedes on decision-making.
• Environmental factors: complexity in a graphical user
interface (GUI) may make perception and
interpretation difficult.
• Automation: highly automated systems may detach operators from
the real system, and have low
awareness of the state. This is known as the out-of-the-loop
performance problem [37].
• Communication with others: insufficient communication between
operators, in one or more control
rooms.
• Individual factors: operators’ training, experience, and
alertness.
Errors in operators’ SA in transmission and distribution control
centres have a profound impact on
their ability to maintain system reliability. Reports on several
large-scale blackouts in the past decade
have cited SA as a culprit [38].
To deal with these sources of SA errors, Panteli et al. propose
the following [36]:
• Detecting and eliminating data inconsistencies during the
process of state estimation. Rule-based
techniques can be implemented to detect and correct topology
errors in the event of signalling or
communication malfunction.
-
Chapter 2. Scoping Study of Power Grid Visualizations 11
• Implementing user-centred design principles to solve problems
associated with GUI complexity and
limitations. Examples include mapping system functions to the
goals of the operator, grouping
data and alarms around critical decisions, and providing system
transparency and observability
(i.e. what the system is doing, why, and what next).
• Designing the GUI and information systems to tell the user
when human intervention is required,
if the automation malfunctions or is not equipped to handle the
scenario. To enhance operator
awareness, systems should automate only when necessary, keep the
operator in control, use decision
support between human and automation, and provide automation
transparency.
• Enhancing wide-area visualization and communication between
transmission system operators in
neighbouring networks.
• Operator training that includes dealing with events outside of
their network that might affect their
responsibility area, and frequent training on new technologies,
e.g. smart monitoring and security
tools.
• Ensuring functionality of Energy Management System (EMS)
applications, including regular test-
ing and hardware maintenance.
2.2.2 Distributed Situation Awareness
Distributed situation awareness (DSA) treats team SA as a
characteristic at the overall systems level
rather than the individuals comprised within the system [39].
Although individuals may attain their
own SA for a system and share this with other members of the
team, DSA approaches assume cognitive
properties of the entire system that are not present at the
individual level.
The DSA concept has been applied to the case study of an
electrical distribution system [40]. Control
rooms house teams of operators coordinating their monitoring
activities, so SA across the team is of
particular concern. The case study found that efficient
communications links allowed DSA to propagate
throughout the network of agents, corroborating with previous
research on team SA.
Shared displays can also facilitate shared SA between team
members to supplement verbal commu-
nication [29], be they visual (e.g. computer screens) or
auditory (e.g. alarm) displays. This is especially
pertinent in a control room where operators rely on visual and
auditory cues to understand system
status, and do not have the benefit of other physical cues.
Abstracted shared displays according to each
operator’s tasks and shared goals can improve team performance,
while completely duplicating the other
-
Chapter 2. Scoping Study of Power Grid Visualizations 12
operator’s display can be detrimental to performance [41].
2.2.3 Cognitive Workload
Data overload is a prevalent issue across current control rooms.
The increasing amount of data available,
such as from PMUs, and the lack of data integration between
systems cause high attentional and memory
load on the operator. This leads to a loss of SA, and therefore
higher error rates [42].
For example, Schneiders et al. [43] conducted a cognitive task
analysis for one such control room,
and found that it required operators to monitor 20 different
information readings. Considering that the
capacity for short-term working memory has been known to be 5-9
“chunks” of information [44], the task
required more memory than was available for the human operator.
A more appropriate design would
limit the number of key indicators on the display for
monitoring; Schneiders et al. used a process of
data reduction to identify the key indicators with highest
priority according to operators. In the event
of deviations from normal operation, the display would then show
more detailed information. Multiple
displays were implemented in the control room for different
operators according to their tasks, to assist
in shared SA.
With routine events being managed by automation, human operators
take on system monitoring tasks
to supervise operations, and intervene only when problems arise.
These long periods of low temporal
demand leave them vulnerable to vigilance decrement, where they
may suffer from a decline in detection
performance after long periods of time. The decline occurs more
rapidly for cognitively demanding
environments [45], which power system control would fall under.
In addition, vigilance tasks require
hard mental work and induce stress [46]; stress in particular
can have negative effects on information
processing including a reduction in working memory and
over-arousal in emergency situations, leading
to a speed/accuracy trade-off in performance [47]. As such,
efforts should be made to ensure operators
are appropriately trained to deal with any possible scenarios
and that their tasks are not designed so as
to be repetitive.
2.2.4 Effect of Expertise
Power systems operators have varying levels of acquired
diagnostic reasoning skills. Domain experts are
able to apply cues to reduce the cognitive demands of a task,
while novices may rely on their knowledge
of the domain in order to perform a task. Cue utilization may be
measured through 4 approaches:
-
Chapter 2. Scoping Study of Power Grid Visualizations 13
• Feature identification: identifying key features from a
complex scene
• Feature discrimination: consistent perception of relative
utility of features
• Paired association: response times and discrimination for item
ratings (of the item utility)
• Transition tasks: the sequence in which the operator acquires
task-related information
Performance in these 4 cue-based cognitive tasks distinguishes
controllers into novice, competent, and
expertise groups [48]. Experts are consistently faster, are more
accurate, have greater discrimination,
and are less prone to simply view information in the manner it
was presented than their peers. Experts
are also more likely to choose the diagnostic response deemed
optimal by the subject matter expert.
2.2.5 Operator Training
Power system operators go through an Operator Training Simulator
(OTS) program to familiarize them-
selves with the human-machine interfaces in the control room,
and also practise how to deal with emer-
gency operations. The simulator goes through possible
contingency scenarios so that trainees learn
how to manage power system operations and assess dynamic
security [49]. The growing complexity of
power system operations has meant a greater need to ensure
proper operator training. Bronzini et al.
[49] developed an OTS approach that measures the knowledge and
skills of the operator for emergency
management, in relation to the required knowledge and skills for
the job.
2.3 Research Opportunities
Power transmission grids in the present and future face
environmental, customer needs, and infrastruc-
ture challenges. Energy production has turned its attention to
reducing CO2 emissions and increasing the
use of renewable power generation. Electricity markets need to
remain competitive and customers need
to be satisfied with the quality of power supply. Electricity
transmission infrastructure often suffers from
underinvestment and consists of aging assets, in spite of
growing load demands. Future power grids are
expected to be smart, taking advantage of modern technological
advances in sensing, communications,
control, computing, and information technology [33].
Power systems visualizations in the control centre typically
display the voltage magnitude on a one-
line diagram. Under abnormal conditions with the risk of voltage
collapse, voltage magnitudes are no
longer sufficient [33]. Instead, other indicators of voltage
stability - such as tracing changes in local
frequency to remote locations - need to be employed to help
operators identify fault locations.
-
Chapter 2. Scoping Study of Power Grid Visualizations 14
A lack of specific geographical location information for
displays and communication has restrained
situation awareness for control room operators [40]. Future
displays will need to present geographical
information to better understand where problems occur and to
coordinate with other agents. Overbye
et al.’s geographical data view approach [23] is expected to
garner more interest in the design of such
displays.
The shift from SCADA to WAMS for remote measurement and control
will mean more opportunities
to use real-time visualization and communication for human
operators. Wang et al. [28] presented a
visualization concept for a WAMS implementation in China;
however, no usability studies have been
conducted yet in this area.
Likewise, some of the other technologies described in previous
sections have undergone preliminary
user studies and simulations, but their effectiveness in
practice has yet to be empirically determined.
Owing to industry deregulation, power transmission control
centres are moving from centralized to
coordinated decentralized decision-making [50]. Whereas
information and communication technologies
(ICT) have rapidly evolved in recent decades, control centres
rely largely on legacy technologies. System
operators have no analytical tools to support emergency control,
due to legacy data acquisition systems
and limited computational power in the control centre. Future
control centres are projected to exploit
modern ICT in order to provide real-time information and
analytical tools to operators. This includes
taking advantage of distributed computing to offer data and
application software that is decentralized
and distributed.
This shift will require further studies on visualization tools
that transform these data into graphics,
and how operators can use these effectively. In particular, the
tools need to present rapidly growing
amounts of data to enhance the operator’s SA, without
overloading the operators’ cognitive resources.
Mobile application interfaces have been explored in the power
utility domain for field workers to
access relevant and timely information, specifically in the case
of an electrical distribution system [51].
Power transmission controllers have similar goals to field
workers, with only the distinction being that
they operate at a higher-level scope; nevertheless, the benefits
of multimedia communication for crews
and decentralization of information allow for the development of
shared SA across operators working
at different levels of the power transmission and distribution
system. Given the modern proliferation
of mobile devices in both consumer and industrial applications,
mobile application interfaces will be
another research avenue to explore to improve collaboration and
shared SA in the case of power grid
dispatch.
-
Chapter 2. Scoping Study of Power Grid Visualizations 15
The Ecological Interface Design (EID) approach forms the basis
of PEGASE, but has otherwise not
been widely used in power system control compared to other
domains such as nuclear power plants [17].
Since EID can improve operator SA especially in the monitoring
of unanticipated events [52], there are
more opportunities to apply the framework to power transmission
control and evaluate how it impacts
operator SA.
The wealth of recent literature in power systems operator SA is
a testament to significant interest in
designing tools and displays for operator effectiveness [53].
The IEEE Power & Energy Magazine featured
situational awareness for energy management analytics and
visualization [54]. Hydro-Québec [55] and
ISO New England [34] have undergone UI redesigns of their energy
management systems, and other
electrical utilities are expected to follow suit according to
the power supply and demand characteristics
of their jurisdictions.
2.4 Limitations of the Scoping Study
This scoping study presents an overview of current work and
future opportunities in visualization for
electrical transmission dispatchers in the control room.
This review may not have included all published papers in the
literature on power systems visual-
ization techniques and their usefulness to operators, despite
efforts to include all relevant search terms
and to use wide-reaching electronic databases. The review only
included English-language publications,
and scans of search results only covered the first few result
pages. Hand-searching key journals is a
method proposed by Arksey and O’Malley [18] to find articles
that may have been missed in database
and reference list searches.
2.5 Scoping Study Conclusions
Electricity market deregulation, the integration of renewable
energy sources, and interconnectedness
of the power grid have presented situation awareness and
cognitive workload challenges to power grid
operators. To help meet their goals of reliable power supply,
new algorithms and representation aids will
need to be implemented in the control centre.
The scoping study has found a great deal of work on improving
tools and representation aids for
power systems monitoring; however, many papers did not support
their claims of improving situation
-
Chapter 2. Scoping Study of Power Grid Visualizations 16
awareness with empirical studies. There is potential for more
research in integrating modern information
technologies into aging control centres, such as in providing
wide-area monitoring data, geographical
location information, and rapid communication between utilities.
Future work in user-centred approaches
to control room display design aim to improve human performance
and help meet the goals of power
system reliability, quality, and efficient electricity
markets.
-
Chapter 3
Knowledge Elicitation
We elicited the expertise of current control room operators
through critical incident interviews, focus
group sessions, and control room observations. In addition, we
observed their work in the control room
over a period of 60 non-consecutive days. The knowledge
elicitation phase informed our Work Domain
Analysis and subsequent stages of the design process.
The knowledge elicitation and Work Domain Analysis (Chapter 3
and Section 4.2) were done in
conjunction with Dr. Antony Hilliard. I developed the plan and
led the critical incident interviews. The
focus group session planning and facilitation, as well as
control room observations, were a joint effort
with Dr. Hilliard. The results reporting and analysis described
in Chapter 3 are mine.
3.1 Critical Incident Interviews
We conducted in-person interviews with operators, with the aim
of understanding operator work in
the control room, and identifying possible information gaps in
existing displays and tools. Flanagan’s
Critical Incident Technique [56] is a well-established method
for eliciting recounts of critical incidents for
this purpose. Participants were asked to recollect specific
incidents that they had experienced first-hand
in the control room, where their involvement had led to either
positive or negative outcomes.
17
-
Chapter 3. Knowledge Elicitation 18
3.1.1 Methodology
Participants
We interviewed 8 current control room operators at the IESO, who
had volunteered to participate. At
the time of the interview, 5 participants were off-shift and 3
were on-shift. Participants had a mean of
9 years of operational experience (min = 3, max = 17, SD =
5).
The IESO operations control room is staffed by 7 operators: a
market assistant, 2 systems assistants,
a system-for-markets assistant, a systems supervisor, a market
supervisor, and a shift superintendent.
We interviewed operators whose most recent control room roles
covered all of the above, except for a
market assistant.
Procedure
Participants read and signed an informed consent form prior to
the interview, and were also provided
with the list of interview questions in advance. Questions were
adapted from the Critical Incident
Technique [56]. Participants were asked to describe their
typical day on the job, and what kinds of
unusual situations they would face. They were then asked to
recall incidents where they either made a
positive or negative impact. We told them we were particularly
interested in incidents where information
was not available or hard to extract from displays, or where
team communication played a role. For each
incident, we asked guiding questions to ensure that we got a
detailed picture of the circumstances of the
incident, what the operator did to respond, and what outcomes
arose. We did not specify a restriction
to how far incidents had occurred in the past. Each interview
took approximately 1 hour.
3.1.2 Results and Discussion
All operators interviewed had examples of incidents where tool
limitations hindered their performance.
Twenty-seven separate critical incidents were recorded from
these interviews, and categorized by prop-
erties shown in Table 3.1. Some incidents fell into more than
one category. The incidents occurred
between 2002 and 2015.
The top issues mentioned in the critical incident interviews
were: data unobserved, unclear conse-
quences, model inconsistency, and wrong limit calculations. This
observation suggests a need for tools
that: (a) attract operator attention to where important activity
is occurring, and (b) integrate infor-
mation (such as limit calculations) between the different
automated systems used in the control room.
-
Chapter 3. Knowledge Elicitation 19
Table 3.1: Critical incidents identified from operator
interviews.
Type of Issue CountData unobserved : Displays did not make it
clear that a line was in service or thata region could be islanded,
misleading the operators
4
Unclear consequences: Operators performed an action, the
consequences of whichwere unexpected
3
Telemetry unavailability : Telemetry data were not transmitting,
so operators couldnot figure out system status
3
Wrong limit calculation: Voltage limits were calculated
incorrectly, leading to un-stable system or inefficient market
operation
3
Model inconsistency : Information discrepancies between
different automation sys-tems (e.g. EMS and MIS) led to those
systems optimizing for different modelparameters
2
Communication: Lack of adequate communication or cooperation
between RCsmeant IESO was unaware of a line coming back into
service or having to resort todrastic measures to protect
assets
2
Error recovery : Manual input errors were not recoverable until
after a delay, causingmarket constraints in the meantime
2
Unpredictable conditions: Variability on the power grid (e.g.
harsh weather) addedoperator workload and made communication within
the control room crucial
2
Data overload : Operator was overloaded with tasks and
interruptions, and losttrack of some of the lower-priority
tasks
1
Hardware failure: Tools were unavailable for operators during
computer hardwarefailure
1
Model inaccuracy : Model inaccurately calculated largest
contingency on the grid 1
-
Chapter 3. Knowledge Elicitation 20
Operators remarked that some displays were “cluttered” and “hard
to read,” slowing their awareness of
contingencies and data inaccuracies. Future operator tools will
need to prioritize information visibility,
particularly of unusual activity, and system data
integration.
Our observation was also in line with the IESO’s business
objectives of working toward better wide-
area monitoring and limit rules tools for operators. As such, my
ensuing design work focused on wide-area
monitoring and power grid visualization, while Dr. Antony
Hilliard’s work centred on concepts for a
limit rules engine.
3.2 Focus Group Sessions
We led focus group sessions with operators to understand their
work in the control room in a team
setting. The focus groups had 3 parts: a critical incident group
discussion, a facilitated discussion on
challenges and brainstorming, and an ergonomics
questionnaire.
Compared to the traditional one-on-one critical incident
interview, the focus group sessions provided
additional insight into team collaboration during incident
response, and multiple perspectives on the
incident from the different operator roles. Because we were able
to reach a larger audience in the focus
groups, we were able to have a wide range of responses to
brainstorming questions and questionnaires.
3.2.1 Methodology
Participants
We led 6 focus group sessions, with a total of 42 operators from
the IESO. The focus groups were
incorporated into the schedule of one of the mandatory IESO
operator training days, and participants
had the opportunity to opt out of the session. Participants came
from all roles in the control room, and
ranged widely in experience (mean = 10.5 years, min =
-
Chapter 3. Knowledge Elicitation 21
was, ”Describe an incident that your crew experienced in the
IESO control room where you had to take
action on the electricity market or system.” We were
particularly interested in events that involved team
collaboration and had different perspectives on the incident.
While participants recounted the incident,
we recorded the timeline of events and challenges encountered
during the incident, on chart paper.
A facilitated discussion on control room challenges and their
impact on day-to-day operations fol-
lowed. We asked for any other challenges that often turn up in
day-to-day operation. Once we finished
collecting challenges, participants voted on the top three in
their personal daily operation experience,
by frequency and severity. We also asked participants to
brainstorm what the concepts of “wide-area
monitoring” and “limit rules engine,” our two topics of design
interest, meant to them as an operator.
We ended with a questionnaire on sight, sound, and tools in the
control room (Appendix A.1). Each
focus group session lasted 1 hour in total.
3.2.2 Results
Challenges in Day-to-Day Operation
Issues related to wide-area monitoring featured prominently in
participants’ voted top challenges in
day-to-day operation. Four out of the six focus groups had top
challenges associated with having to
use multiple SCADA screens for a task, and difficulty navigating
between screens in order to diagnose
problems.
The top-voted challenges for each of the 6 focus groups (not
counting wide-area monitoring if it was
top-voted) were:
• Knowing system status and whether telemetry has failed.
• Knowing the situation and whether the system state is
secure.
• Contingency planning and recognizing the situation.
• Performing studies of contingencies is slow and effortful.
• Needing to open lots of screens simultaneously to piece
information together.
• Having an overly large workload during contingency
situations.
Wide-Area Monitoring Definition
The focus group sessions with operations teams formulated the
wide-area monitoring problem as:
-
Chapter 3. Knowledge Elicitation 22
• Detecting “problem areas”, both outside of Ontario and within
the province, quickly and indepen-
dently from other RCs
• Supporting situation awareness for operators
• Zoom navigation at different levels of geographical scope
• Data consistency across screens (e.g. an outage should be
visible on any screen that contains it)
• Viewing interconnections and where a Transmission Load Relief
(TLR) request would affect the
wider grid
• Fulfilling the NERC standard on wide-area monitoring
Control Room Ergonomics
Operators rated their experience of ergonomics issues and tool
development processes in the IESO control
room. The results are detailed in Appendix A.2.
3.2.3 Discussion
The brainstorming session on ”What does wide-area monitoring
mean to you?” was an exercise in
ensuring that operators’ vision of wide-area monitoring would
match the design goals set by the research
team and IESO management. During the design process, this
definition helped narrow the scope of
wide-area monitoring within the power grid operations
domain.
The responses on the control room ergonomics questionnaire were
valuable for providing specific
recommendations to the industry partner on human performance
improvements.
-
Chapter 4
Work Domain Analysis
We conducted a WDA of power grid and electricity market
operations. The WDA describes the con-
straints that govern the purposes and physical properties of the
power grid; the system decomposition;
and the means-ends links between system purposes and
components.
The analysis helped describe the information needs of power grid
operations in the context of wide-
area monitoring, and thus influenced the design and evaluation
of the wide-area monitoring design
concepts.
4.1 Methodology
To gather an understanding of work domain concepts (such as
power system theory), we consulted IESO
operator training materials and other technical documentation
[57]. We then refined the WDA based
on our literature review, control room observations, interviews,
focus groups, and questionnaire. The
WDA [58] was validated with an operator, and 2 managers who were
former operators.
4.2 Summary of the WDA
4.2.1 Abstraction Hierarchy
The electrical power grid may be described at the following five
different levels of abstraction, in order
of most to least abstract:
23
-
Chapter 4. Work Domain Analysis 24
1. Functional Purposes
2. Abstract Functions, Values, and Priority Measures
3. Purpose-related Functions
4. Physical Functions
5. Physical Objects
We started with the functional purposes, which came from our
discussions with operators. We then
described the power grid at the physical object level, and
worked our way up the rest of the levels of the
abstraction hierarchy.
Functional Purposes
The three purposes of power grid operations can be summarized
as: (1) reliability of the power system
and enforcing interconnection reliability standards; (2) quality
of service, minimizing outages, voltage
variations, and equipment failures; and (3) efficiency of
electricity markets. When making critical deci-
sions, operators have to assess the benefits and risks between
this set of overarching goals.
Physical Objects
At the most concrete level, the power grid can be described by
the physical appearance and geographical
location of the equipment that makes up a power system. Examples
include the transmission lines,
stations, and weather patterns.
Physical Functions
Grid elements have functionality most often described by circuit
schematic diagrams, which are irre-
spective of the elements’ physical appearance. Examples of
physical functions include the transmission
network (which conducts electricity between equipment, and has a
characteristic impedance), transform-
ers (which convert voltage between sections of the transmission
network), generators, and loads.
Purpose-related Functions
This level of abstraction more broadly describes what the
physical functions do, and can help illustrate
potential problem-solving paths that are available to operators.
Such purpose-related functions include
power transmission (real and reactive), generation, consumption,
and imports/exports.
-
Chapter 4. Work Domain Analysis 25
Abstract Functions, Values, and Priority Measures
This level describes the grid on the basis of general scientific
(and economic) understanding. This infor-
mation often comes from simulations, statistical analyses, and
technical models developed by planning
engineers for control room use. Priorities are defined by
regulatory standards such as those set by NERC.
Examples include maintaining dynamic stability, matching power
supply and demand, and minimizing
net inadvertent interchange.
4.2.2 Means-Ends Links Between Abstraction Levels
The abstraction hierarchy diagram in Figure 4.1 illustrates
means-ends links between elements of the
power grid. These links represent the potential trade-off
effects of equipment on system purposes, and
the tools available to operators in order to meet their
overarching goals.
4.2.3 Part-Whole Decomposition
Individual pieces of equipment on the power grid are clearly
defined as in component lists, and the
scope of the power grid can expand beyond our system boundary of
a single RC Area. Because electric
power grid components are tightly connected and interdependent,
intermediate levels of part-whole
decomposition are more complex to describe and depend on the
situation.
Functional purposes and abstract functions of the power grid are
delineated by the different regulatory
requirements they are associated with, and the concepts that
they represent. Purpose-related functions
describe power flow, either between individual lines, grouped in
the form of SOLs, or furthermore grouped
into the Area Control Error1 calculation for a whole
jurisdiction. The physical function of equipment
may be grouped according to function within an area, and
connectivity with other components. At the
physical object level, grid components can be grouped by
transmission yard.
4.2.4 Topographic/Causal Links
The electric grid is physically made up of equipment connected
together with wires. These links have
the physical function of electrical conduction, and circuit
breakers can change the grid topology - for
example, represented in node/breaker models in the EMS.
Purpose-related functions are linked through
1Area Control Error (ACE) [59] is a calculated value (in MW)
that represents power balance compared to schedulewithin an area,
plus a small “bias” obligation to maintain frequency.
-
Chapter 4. Work Domain Analysis 26
power flows, which can be derived in computational breaker
models. Abstract functions are linked by
cause-and-effect relationships between elements - for instance,
energy bids and asks affect power balance,
which may require changing generator output, and would in turn
affect system robustness and stability.
Functional purposes are somewhat interrelated in that
unreliability can lead to inefficiency and low
quality, but the reverse may not necessarily be true.
4.3 Application of the WDA to Wide-Area Monitoring
Wide-area monitoring encompasses a large geographical scope, by
definition. In order to avoid infor-
mation overload in capturing this large scope, the data should
be displayed in the form of high-level
summary views. The WDA was useful for capturing the information
requirements at the highest levels
of abstraction, and the potential paths for problem-solving at a
wide-area view of the grid.
4.3.1 Design Scope in the Abstraction Hierarchy
As discussed in the wide-area monitoring definition developed
during the focus groups (Subsection 3.2.2),
operators were primarily concerned about the impact of grid
events within and outside of Ontario, and
at the interconnections. Power flows, outages, and the external
model were recurring themes in the
discussion.
My design concept thus focused on a subset of the abstraction
hierarchy concerning: 1) matching
power supply and demand, and 2) minimizing net inadvertent
interchange. This does not preclude the
development of wide-area displays that capture other abstract
functions, but rather that these two most
closely reflect wide-area monitoring as per the NERC standard
for the scope of this project.
These two abstract functions help achieve the purpose of quality
of service, and in turn, reliability
that partly depends on quality. The means to achieve these two
abstract functions (as seen in Figure
4.1) are: real power transmission, real power consumption,
operating reserve, real power generation, and
import/export flows.
Since the prototype is intended as a high-level overview
display, physical function and physical object
descriptions were outside the design scope.
-
Chapter 4. Work Domain Analysis 27
4.3.2 Measures and Constraints at Different Abstraction
Levels
To develop wide-area monitoring information requirements from
the abstraction hierarchy [15], the model
has been converted into the following list of variables for each
level of abstraction.
Functional Purposes
• Area Control Error (ACE) (MW)
• Frequency (Hz)
Abstract Functions, Values, and Priority Measures
• Power supply (MW)
• Power demand (MW)
• Net interconnect power transmission flows (MW)
• Scheduled import/export flows (MW)
Purpose-related Functions
• Power transmission flows within Ontario (MW)
• Power consumed (MW)
• Operating reserve capacity (MW)
• Spare generation capacity (MW)
• Power generated (MW)
• Tie-line power transmission flows (MW)
• Tie-line import/export schedules (MW)
The ACE for a region is ideally at or close to 0, which would
indicate power balance according to
schedule and an acceptable transmission frequency. Any sustained
absolute value in the hundreds or
more may indicate a major failure in the system, such as a large
generator outage.
Power line frequency in North America is set at 60 Hz.
Deviations from this standard are kept to a
minimum, since power system components are designed for the
operating frequency. Frequency control
is also an important aspect of power grid operations because it
is one measure of the load and generation
balance. Therefore, operators have to monitor system frequency
despite it already being a component
of the ACE calculation.
-
Chapter 4. Work Domain Analysis 28
Power supply is constrained by the power generation
infrastructure built and connected to the grid.
Supply and demand are constrained by the transmission and
distribution systems that deliver power to
consumers and have voltage and current limits.
Import/export flows between RCs are scheduled 24 hours in
advance, although they may issue re-
quests to change import/export values in a shorter time frame.
Scheduled and actual imports/exports
are constrained by the transmission infrastructure connecting
two jurisdictions, and the power flows
within each jurisdiction.
Power flows within each jurisdiction are restricted by System
Operating Limits (SOL) developed in
engineering simulations.
4.3.3 Using the Information Requirements
I sketched basic graphics from these information requirements
and presented these preliminary design
ideas in tabletop discussions with operators. The feedback from
these tabletop discussions, and the
subsequent design and evaluation process, are described in the
next chapter.
-
Chapter 4. Work Domain Analysis 29
Fig
ure
4.1:
Ab
stra
ctio
nh
iera
rchy
ofp
ower
gri
dop
erati
ons.
Ele
men
tsre
leva
nt
tow
ide-
are
am
on
itori
ng
are
hig
hli
ghte
din
gre
enb
oxes
.
-
Chapter 5
Design and Evaluation of Wide-Area
Monitoring Concepts
Using the information requirements gathered from the WDA, I
sketched preliminary design concepts
on paper and in a MS PowerPoint slide deck. In tabletop
discussions, the slides were presented to 4
operators for their feedback. Operators were asked a series of
questions to clarify their information needs
during operation.
The ideas and feedback were used to develop an interactive
prototype, also in MS PowerPoint, from
the results of the tabletop discussions. The prototype was
validated in a usability evaluation with 9
operators. Their suggestions were incorporated into the next
iteration of the design prototype.
This chapter details the process from preliminary design ideas
to evaluating the interactive prototype
with operators.
5.1 Preliminary Design Ideas and Feedback
I sketched preliminary design ideas on paper and in MS
PowerPoint. During tabletop discussions with
4 off-shift operators, I asked a series of questions about
information requirements for control room
operations and solicited feedback on the design ideas. Each
operator was consulted individually, and
each session took approximately 30 minutes.
30
-
Chapter 5. Design and Evaluation of Wide-Area Monitoring
Concepts 31
Table 5.1: Critical parameters of system operation.
Parameter Included in prototype?Frequency YesMajor interface
flows YesACE values YesLimits (internal and external) Somewhat
(external model missing)Topology (critical infrastructure) Somewhat
(required at lower levels)Generation• Total generation Yes• Total
load Yes• Total wind, solar generation Yes• Major generating
stations YesLargest contingency - operating reserve relationship
YesFlow gates YesBoundaries (circuits that make up each interface)
No (for simplicity)Any interfaces phase-shifted NoVoltages (perhaps
compared to historical) No (mixed opinions on necessity)Automatic
Generation Control (AGC) No (mixed opinions on necessity)
5.1.1 Critical Parameters
One operator suggestion from our critical incident interviews
was to have a set of critical parameters
of system operation to gauge overall system state, similar to
what is employed in nuclear power plant
operations [60]. The notion of critical parameters is in line
with higher levels of abstraction, and might
include values that operators would want to monitor at any time.
The Macomber Map developed by the
Electric Reliability Council of Texas (ERCOT), for example, has
a dashboard showing ACE, frequency,
generation, load, wind, number of islands, and Physical
Responsive Capability1 (PRC) [61].
Table 5.1 lists what participants considered critical parameters
of system operation. Most of these
parameters were implemented in the prototype, though some were
not due to:
• Information unavailability: We did not have access to power
grid models external to Ontario.
• Appropriate level of detail for an overview display: The
wide-area overview would risk being too
cluttered with detail if circuit topology was included.
• Mixed opinions between participants: Some participants
disputed the necessity of voltages or AGC
on the overview display.
To follow up, I asked participants, “What would go on a
dashboard?” and proposed a set of parameters
that would go on a dashboard seen at the top of every screen:
ACE, frequency, total generation, total
1This term appears to be exclusive to ERCOT, and is synonymous
with operating reserve.
-
Chapter 5. Design and Evaluation of Wide-Area Monitoring
Concepts 32
load, number of islands, largest contingency, and operating
reserve. Participants suggested the removal of
number of islands, as it did not provide context, and suggested
the addition of wind and solar generation
numbers. Having these parameters appear at the top of every
screen would allow operators to gauge the
system state as it reflects the overall functional purposes.
5.1.2 External Geographical Scope
Five jurisdictions have interties with Ontario that would be
required in a wide-area overview: Quebec,
New York, Manitoba, Minnesota, and Michigan. The overview would
include the ACE (as a summary
of supply/demand), net interconnect power transmission flows,
and scheduled import/export flows for
each jurisdiction.
Participants also suggested that they would like summaries for
the reliability coordinators Midcon-
tinent Independent System Operator (MISO), which includes
Manitoba, Minnesota, and Michigan; and
the PJM Interconnection, which includes Pennsylvania, New
Jersey, and Maryland. Although the PJM
Interconnection not directly connected to Ontario, it is large
enough to affect the province’s power grid.
Figure 5.1 shows the preliminary concept for displaying these
jurisdictions and RCs. This would
later be developed into the wide-area view of the design
prototype.
Figure 5.1: Preliminary concept for displaying external
jurisdictions that affect the Ontario power grid.
-
Chapter 5. Design and Evaluation of Wide-Area Monitoring
Concepts 33
5.1.3 Modelling the Ontario Power Grid
All power flow limits fall into the category of System Operating
Limits (SOL). A subset of these SOLs
is considered crucial for maintaining stability in the overall
Eastern Interconnect, and these limits are
known as Interconnection Reliability Operating Limits
(IROL).
To summarize the Ontario grid at the system view, I proposed a
one-line diagram that showed the
links between Ontario’s IROLs (Figure 5.2). The diagram would
act as a summary of power transmission
flows within the province. The first participant pointed out an
existing zonal demand overview display
already in their EMS, where the province’s grid was separated
into “zones,” each with generation and
consumption data.
Figure 5.2: Preliminary design for a one-line diagram showing
links between Ontario’s IROLs.
The system level of the prototype borrowed the pre-existing
layout of these Ontario zones and their
relationships with IROLs. The stick-and-circle layout (Figure
5.3a) was more suitable for modelling
the Ontario regions compared to the one-line diagram, because
different status numbers for each region
could now be displayed in one graphic. These status numbers
include: power consumed, operating reserve
capacity, spare generation capacity, and power generated.
Knowing the available operating reserve or
spare generation capacity, and generation numbers, will help
operators decide what control actions are
available to match power supply and demand in the province.
IROL power flows, by definition, affect flows at the interties
between jurisdictions, and operators
monitor IROLs to minimize net inadvertent interchange. As seen
in Figure 5.3a, tie-lines are shown on
the Ontario system level, along with their power transmission
flows and import/export schedules. For
-
Chapter 5. Design and Evaluation of Wide-Area Monitoring
Concepts 34
Figure 5.3a: After tabletop discussion feedback:
Stick-and-circle layout for modelling Ontario regions atthe system
overview level, showing generation and load numbers.
Figure 5.3b: A toggle in Figure 5.3a would also show the
operating reserve and spare generation numbers.
example, a power flow reaching its IROL may cause a tie-line
flow to be above its scheduled value; the
display would alert the operator to this inadvertent interchange
and the area of the cause.
-
Chapter 5. Design and Evaluation of Wide-Area Monitoring
Concepts 35
5.1.4 Generation
One way to help operators match power supply and demand is to
allow them to monitor the supply
coming from generators. I proposed a list of generators, with
various options for sorting: type of
generation, MW generating, MW capacity, geographical region.
Operators commented that a list format
was not helpful to them, considering the importance of being
able to draw connections between the
generators and the rest of the grid (as at the physical function
level).
Instead, a graphical form that captured the generators and their
relationship with critical parameters,
such as power flow limits, would be more relevant to their
operational goals. The Ontario zonal layout
was helpful for creating a generation summary page in the
prototype that captured generation and load
amounts for each zone (Figure 5.3b).
5.1.5 Alarm Notifications
I proposed an alarm notification panel that would provide
context as to what happened, the potential
cause, and links to potential operator actions. An example is
shown in Figure 5.4.
Figure 5.4: Preliminary design for alarm notification panel.
The concept was positively received, and participants did not
have any other suggestions for notes
to add in the alarm notifications.
-
Chapter 5. Design and Evaluation of Wide-Area Monitoring
Concepts 36
5.1.6 Contours
Colour contours were explored as a way to help operators quickly
diagnose abnormal states by changes
in colour or contour shape. There are several possibilities for
what the contours may capture: voltage
[22], frequency [62], and power flows [63] have been previously
proposed.
Between these three proposed parameters for a colour contour,
participants felt that voltage would
be the most useful. Frequency deviations rarely occur, and are
caused by circuit separation that would
have already triggered other alarms. Power flows were a
possibility, but because their limits widely vary
by area (compared to voltages), comparisons between power flows
may not necessarily be useful.
Contours were not implemented in the prototype due to technical
limitations of the prototyping
software. However, there may be a need for voltage contour maps
to supplement the prototype - later
in the design process, one operator did suggest the addition of
a voltage monitoring overview.
5.2 Design Prototyping
5.2.1 Context
The wide-area monitoring display concept focuses on visualizing
the power grid at the multi-jurisdiction
(Ontario and its neighbours) and system-wide (Ontario) levels.
In practice, it would be integrated into
an EMS as a central monitoring tool to help operators detect
abnormal events across the power grid.
It would be suitable as a control room wallboard display or as a
landing screen for the EMS desktop
interface. The EMS would have other displays, not described
here, with finer levels of detail to aid
problem-solving.
5.2.2 First Interactive Prototype
The first interactive prototype was developed in MS PowerPoint.
This first iteration (see Appendix
B) was used in the usability evaluation. The hierarchy of detail
was: wide-area view – system view –
detailed overview. Further detailed views were outside the
design scope of wide-area monitoring.
A detailed description of the iterated design, after
incorporating usability evaluation feedback, is
described in the next section.
-
Chapter 5. Design and Evaluation of Wide-Area Monitoring
Concepts 37
5.3 Iterated Design Prototype
5.3.1 Wide-Area View
The wide-area view in Figure 5.5 shows the Ontario grid in
relation to its neighbours’. It namely features
imports/export schedules and ACE values.
Figure 5.5: Iterated design prototype: wide-area view that
includes external jurisdictions.
Imports/exports are shown as a trend chart comparison between
actual and scheduled flows. This
allows operators to determine whether net imports and exports
are following pre-determined schedules
for the day, and whether any net inadvertent interchange
requires a control action. If there is a significant
discrepancy between the two, a bracket (yellow for warnings, red
for alerts) labelled with the difference
appears. As these trend charts are for comparing intertie power
flows with schedules and for monitoring
trends, the MW vertical axis range is dynamic to ensure trends
are visible. The trend charts do not
have axis labels in order to reduce screen clutter.
Arrows between the jurisdictions show the direction of power
flow. The thickness of the line corre-
-
Chapter 5. Design and Evaluation of Wide-Area Monitoring
Concepts 38
sponds (though a non-linear relationship, so that small flows
are not hidden) to the MW of power flow
along all interties between two jurisdictions.
At the top of the screen (and for all others in the prototype)
is a dashboard showing ACE, frequency,
total generation, total load (+ dispatchable load), largest
contingency vs. operating reserve, and current
wind and solar generation. The alarm button opens up an alarm
notification pane, described in Subsec-
tion 5.3.4. A search box allows the operator to go directly to
the location or equipment they are looking
for, by name. Constraints on the ACE-frequency relationship and
allowed period for ACE deviation are
assumed to be already visualized in other displays like the
Balancing Authority ACE Limit radar [64].
5.3.2 System-Wide View
The system-wide display was initially split between a power flow
view (focused on monitoring power
flows and their limits) and a generation view (for examining
generation across different regions of the
ICG). However, participants suggested combining the two, and
showing limits according to the most
constraining (or situationally relevant) ones.
For example, FS (short for “Flow South”) is shown in the example
scenario in Figure 5.6 between
the Northeast and Essa, because power flow along this
transmission line grouping is flowing south, as is
typical during the daytime. During night times when power flow
is going north, the FN (“Flow North”)
flow limit is situationally relevant, so it appears in place of
FS.
When power flows are approaching the limit, a bar chart appears
as a visual warning; the bar is
yellow when approaching, and red when exceeding the limit. The
breakdown of generatio