A System for Single Human Supervision of Multiple Robots ...imi.ntu.edu.sg/NewsEvents/Events/PastSeminars... · 0.1 0.2 0.3 0.7 0.6 0.5tated ots 0.4 2-robots 3-robots Group Size Mean

/52

A System for Single Human Supervision of Multiple Robots in Urban Search and Rescue

presented by

Wong Choon Yue

Project-3

BeingThere Centre, Institute for Media Innovation

21st June 2016

/52

Relevance to BTC research

• Single human supervision of multiple

social robots

2

Shopping and Retail Education

/52

Desire for Single-Human Multiple-Robot

System (SHMRS)

• Multiple robots:

– Efficiency, Reliability

– Some tasks require multiple robots

• Need for human supervision:

– Errors bound to occur1, 2, 3, 4

– Experience, greater awareness, flexibility, problem

solving

– Supervision possibly by only a single human5,

maximize robot to human ratio for safety, manpower

savings

1 Weir, D. (2004). Sequences of failure in complex socio-technical systems: some implications of decision and control. Kybernetes , 522 – 537

2 Cummings, M. L., & Guerlain, S. (2007). Developing operator capacity estimates for supervisory control of autonomous vehicles. Human Factors 49 (1), 1 – 15 3 Fong, T. W., & Thorpe, C. (2001). Vehicle teleoperation interfaces. Autonomous Robots, 9 – 18 4 Sheridan, T. B. (1992). Telerobotics, Automation, and Human Supervisory Control. Cambridge, MA, USA: MIT Press 5 Murphy, R., Blitch, J., & Casper, J. (2002). AAAI/RoboCup-2001 Urban search and rescue events: Reality and competitions. AI Magazine Volume 23, Number 1,

37 - 42

3

/52

Single-Human Multiple-Robot System

(SHMRS)

Human

Supervisor and

Control

Station/Interface

Robot Group

3 major components:

• Robot group

• Human supervisor

• Control station/interface

4

/52

Cognitive Concerns: Workload • Robot supervision is challenging

• Increased cognitive workload with each added robot

• Failure to perform critical tasks • Failure to spot target / victim

• Errors • Collisions

Reserve cognitive

resources capacity

Maximum

available

resources

System Performance

Resources Supplied

Cognitive Resources Demanded

(Wickens & Hollands, 2012)

Wickens, C. D., & Hollands, G. J. (2000). Engineering Psychology and Human Performance 3rd Edition. New Jersy: Prentice Hall

5

/52

Cognitive Concerns: Situation Awareness

(SA)

SITUATION AWARENESS

Perception of elements in

current situation

Level 1

Comprehension of

current situation

Level 2

Projection of

future status

Level 3

6

• Awareness and understanding of what is

happening presently with each robot.

• Awareness of what will soon happen.

/52 7

• SA is needed for – Making informed decisions

– Error prevention

– Timely intervention

Cognitive Concerns: Situation Awareness

(SA)

/52

Solution: Applying Automation

• To address workload

– Supervisor mentally relieved of

processing tasks handled by

automation

– Fewer physical actions

• To address lack of SA

Overl

oad

Maximum

workload capacity

Workload

Sit

uati

on

Aw

are

ne

ss

B

A

C

D

8

Intelligent autonomous behaviours

/52

Applying Automation in SHMRS

Exce

ssiv

ely

hig

h w

ork

loa

d

Ou

t-o

f-th

e-lo

op

syn

dro

me

Workload

Situation

Awareness

Level of Autonomy

• Individual robot level

– Obstacle avoidance

– Standard responses

• Robot-group level (coordination)

– Inter-robot coordination

9

/52

• Communication resources

– Bandwidth

– Influences volume and rate of communication1

• Computation resources

– Processing power and storage

• Affects maximum level of autonomy or

sophistication in robot behaviours1, 2

• When demand exceeds supply,

– Robots may perform slowly3 or grind to a halt

– Can affect entire robot group

Concerns with Multiple-Robot Coordination

1Barrett, A., Rabideau, G., Estlin, T., & Chien, S. (2007). Coordinated continual planning methods for cooperating rovers. IEEE Aerospace and Elecrtonic Systes

Magazine 22 (2), 27 - 33. 2Parker, L. E. (2008). Multiple mobile robot systems. In B. Siciliano, & O. Khatib, Springer Handbook of Robotics, 921 - 941. Springer. 3Burgard, W., Moors, M., Stachniss, C., & Schneider, F. E. (2005). Coordinated Multi-Robot Exploration. IEEE Transactions on Robotics. Vol. 21, No. 3 , 376 - 386.

10

/52

Problem Statement

Autonomous coordination should be designed

to alleviate workload and prevent situation

awareness degradation

BUT

Coordination must consume as little

communication and computation resources

as possible

11

/52

Approach to Problem

Apex

Robot

Subordinate

Robot 1

Subordinate

Robot 2

Centralized Robot Group (CRG)

Robot 1 Robot 2 Robot 3

Distributed Robot Group (DRG)

• Hierarchical framework

• Communication only

between Apex and

Subordinate

• Robots are peers

• Each robot

communicates with all

other robots

/52

Benefits and Drawbacks

(of CRGs)

• Prone to Single-Point-Failure (SPF) 1,2,3

• Subordinates can be smaller, simpler and less expensive4,

using microprocessors

• Allows coordination even with such robots4,5

• Communicating with fewer group members can reduce

communication costs6

Apex

Robot

Subordinate

Robot 1

Subordinate

Robot 2

1Horling, B., & Lesser, V. (2005). A survey of multi-agent organizational paradigms. The Knowledge Engineering Review Vol.19:4, 281-386

2Parker, L. E. (2008). Multiple mobile robot systems. In B. Siciliano, & O. Khatib, Springer Handbook of Robotics, 921 - 941. Springer

3Mezei, I., Malbasa, V., & Stojmenovic, I. (2010, Dec). Robot to robot: Communication aspects of coordination in robot wireless networks. IEEE Robotics and

Automation Magazine , pp. 63 - 69

4Khoshnevis, B., & Bekey, G. (1998). Centralized sensing and control of multilpe mobile robots. Computers and Industrial Engineering 35 (3-1), 503 - 506

5Seib, V., Gossow, D., Vetter, S., & Paulus, D. (2011). Hierarchical multi-robot coordination. Lecture Notes in Computer Science 6556, (pp. 314 – 323)

6Sujit, P. B., Sinha, A., & Ghose, D. (2007). Team, game, and negotiation based intelligent autonomous UAV task allocation for wide area applications. In J. S. Chahl,

L. C. Jain, A. Mitzutani, & M. Sato-llic, Studies in Computational Intelligence 70, 39 - 75

13

/52

Model of Proposed Solution

AP

H

S2 S1

Defining attributes

1. SHMRS is equipped with a

robot group that is:

a) Strongly centralized

b) Explicitly communicating

c) With a co-located apex

robot

2. Communication channel

incorporated between

supervisor and each robot 14

Single-Human Multiple-Robot System with a

Centralized Robot Group

/52

Model of Control Solution

Defining attributes

1. SHMRS is equipped with a

robot group that is:

a) Distributed

b) Explicitly communicating

c) Able to deploy all members

within mission environment

2. Communication channel

incorporated between

supervisor and each robot

H

R3 R1 R2

15

/52

Implemented SHMRS

General Information •Dimensions: 0.77 × 0.64 × 0.55m

•Weight: 50kg

•Payload: Approx. 25kg

Actuation •Motor: Two 24V brushed DC motor

•Steering: 4 wheel differential

•Maximum Speed: 1.7m/s

Computing •Pentium III 800MHz CPU

•iRobotrFlex system (for motor and

hardware control) 16

• Capable of deploying groups of two and three robots

• Capable of deploying robot group using centralized and

distributed organization structures

/52

Autonomous Coordination

17

/52

EXPERIMENTATION

18

/52

Design of Experiment

• Two factors in the

experiment

– Organization structure

• Centralized

• Distributed

– Group size

• 2 robots

• 3 robots

• 8 unpaid participants

– Each performed 4 USAR

missions

19

/52

Scenario in Experiment

• Background • Indoor USAR scenario

•Objectives of mission • Locate all simulated victims

as quickly as possible

20

/52

Mission Area

11.80m

Co

ntr

ol

sta

tio

n

En

tra

nc

e t

o

mis

sio

n a

rea

13.16m

21

/52

Simulated Victims

22

/52

Simulated Hazards

• Simulated hole

• Simulated debris pile

23

/52

Performance Measures

Measures for system

performance

Measures for supervisor

cognition

Measures for robot

communication

• Number of victims

located

• Number of robots

incapacitated

• Participant workload

• Participant situation

awareness

• Amount of

communication between

robots for coordination

•Workload • Determined with NASA-TLX (Task Load Index)

1

•Situation Awareness • Determined with SAGAT (Situation Awareness

Global Assessment Technique)2

1

Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (task load index): Results of empirical and theoretical research. In P. A. Hancock, & N. Meshkati, Human Mental Workload, 138 - 183. North-Holland: Amsterdam.

2

Endsley, M. R. (1988). Design and evaluation for situation awareness enhancement. Proceedings of the Human Factors Society 32nd Annual Meeting, 97-101.

24

/52

RESULTS

25

/52

50

52

54

64

62

60

58

56

Mean N

AS

A-T

LX

score

2-robots 3-robots

Group Size

Legend

Centralized robot group (Proposed solution) Distributed robot group (Control solution)

Mean NASA-TLX score VS Group size

Workload

26

/52

System Performance

Maximum

workload

capacity

Workload

Sit

ua

tio

n

Aw

are

ne

ss

5.0

4.5

4.0

3.5

Mean n

um

ber

of

vic

tim

s f

ound

2-robots 3-robots

Group Size

Mean number of victims found

VS Group size

27

0.0

0.1

0.2

0.3

0.7

0.6

0.5

0.4

Mean n

um

ber

of ro

bots

incapacitate

d

2-robots 3-robots

Group Size

Mean number of robots

incapacitated VS Group size

Legend

Centralized robot group (Proposed solution) Distributed robot group (Control solution)

/52

Findings

• Supervising CRG consistently resulted

in greater workload

• Participants were highly focused on

ensuring safety of Apex robot

• As a result, performance towards

finding victims declined

28

/52

Conclusions

• Robot-group organization does affect

human supervisor

• Where possible, DRG supervision may

be preferable

29

/52

THANK YOU

30