Tablet tools and micro tasks
Aug 18, 2015
resilient procedure use
resilient procedure use study 2014
video: https://vimeo.com/98273509
products
EOP flowchart Process overview display
Micro-task tablet tool Eye tracking simulator plug-in
nuc
IFE ProcSee
what if…
operation training validation research
operation training validation research
EOP tool ✔ ✔ ✔ ✔
Overview display ✔ ✔ ✔
Micro-tasktablet app ✔ ✔ ✔
Eye trackingplugin ✔ ✔ ✔
micro taskss w i p e t o s t a r t
video: https://vimeo.com/131387407
problem
How to test performance benefitsof new interface solutions
How to increase confidence in HRA results and safety cases
solution
benefits
Generate quantitative results about performance benefits (time and reliability) of digital/innovative CR interfaces Highly efficient, objective method Compare to existing reference data Qualitative insights via eye tracking Data directly supports HRA Highly customisable and precise Generate exactly the data needed for the HRA
big picture
Micro task tablet app
Standardized question setsDatabase
training
hra interfacedesign
gamification
evaluation of training
validationa mobile companion for
HRA / HF specialists
benchmarkingwithin / across organisations
example
2014 micro task study
innovative vs conventional displays
innovative vs conventional displays
pumpsconventional innovative
mass balanceconventional innovative
Do the innovative displays lead to faster, more reliable identifications and decisions?
mass balanceconventional innovative
Is performance (time and reliability) with innovative displays at least as good as with conventional?
method
Large number of questions Related to systems, components, procedures, etc.
Varying levels of difficulty Including higher-level decision making
Different display conditions E.g. innovative vs. conventional displays
We measure response time / accuracy Compare data between conditions If needed, review eye tracking recordings to understand anomalies
detection/decision tasks under time pressure
example
4 operators working in the simulator individually(no communication)
We control which displays are available (eg innovative, conventional)
Instructions:”Please answer the questions correctly, but also as quickly as possible.
It is very important that you work as fast as you can.”
Run 1s w i p e t o s t a r t
How many condensate pumps are running on turbine 31?
1 2 3
s w i p e t o c o n ti n u e
“How many condensate pumps are running on turbine 31?”
Average identification timeConventional 8 secInnovative
6 sec
Is the subcooling margin sufficient?
Yes
s w i p e t o c o n ti n u e
No
Which steam generatorsare faulty?
SG-1
s w i p e t o c o n ti n u e
SG-2 SG-3
Should safety injection be stopped?
Yes
s w i p e t o c o n ti n u e
No
What is the narrow-range level in steam generator 1?
s w i p e t o c o n ti n u e
%
1 2 3 4 5 6 7 8 9 0
37
up to 200 questions per hourwith 4 operators in the simulator working individually, that means we can run up to 800 questions per hour
results
data sourcewhere does this data come from?
6840 data points3420 response time measures
3420 accuracy measures
20 operators
5 hours simulator timein total for the whole data collection
innovative displays are superior
HS I m e a n p e rfo rm a n ce tim e s
Cu rre n t e ffe ct: F(2 , 8 ,0 0 3 )=2 1 ,8 8 9 , p = ,0 0 0 5 7
V e rti ca l b a rs d e n o te 0 ,9 5 co n fi d e n ce i n te rva l s
L S D (In n o va ti ve ) O WD (Co n ve n ti o n a l ) L S D a n d O WD
HS I d e sig n
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
Pe
rfo
rma
nce
tim
e (
sec)
Operators were faster with innovative displays than with conventional displays
innovative
conventional
Cu rre n t e ffe ct: F(2 , 8 ,0 0 3 )=2 1 ,8 8 9 , p = ,0 0 0 5 7
V e rti ca l b a rs d e n o te 0 ,9 5 co n fi d e n ce i n te rva l s
L S D (In n o va ti ve ) O WD (Co n ve n ti o n a l ) L S D a n d O WD
HS I d e sig n
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
Pe
rfo
rma
nce
tim
e (
sec)
Highly statistically significant difference
Slightly more accurate with innovative displays than conventional displays
Percentage of correct answers
innovative conventional innovative
Performance variability
conclusion
As reliable as conventional displays
Faster to read
innovative displays in the 2014 study are
what next?
Compare analog CR to HAMMLAB
Obtain reference data about analog CR
Interested parties / collaborators
AREVA Germany
Idaho National Labs
Virginia Tech University
GSE Systems
U.S. NRC
features
MT app trigger simulator events
Simulator events trigger MT app
Integration with WL methods
Database prototype
