Inconsistencies in Models of Adaptive Service Robots

Inconsistencies in Models ofAdaptive Service Robots

Ivan Ruchkin *PhD Student in Software Engineering

Institute for Software ResearchCarnegie Mellon University

* with images from Jonathan Aldrich, Javier Cámara, and Bradley Schmerl

Software Research Seminar (SSSG)April 10, 2017

2

Service robot missions

● Autonomy in the face of uncertainty● Timeliness, power, safety

3

Centralized online adaptation

4

Centralized online adaptation

Does it work?

5

How to check if it works

Run the system

6

How to check if it works

Run the system

Run the simulation

7

How to check if it works

Run the system

Run the simulation

Analyze the models

8

How to check if it works

Run the system

Run the simulation

Analyze the models

Analyze the code

9

How to check if it works

Run the system

Run the simulation

Analyze the models

Analyze the code

10

Plan for today

● Inconsistency 1: power● Inconsistency 2: turns● What is common? ● Tentative approach: detection

11

Inconsistency 1: power in simulation vs hardware

12

Inconsistency 1: power in simulation vs hardware

13

Inconsistency 1: power in simulation vs hardware

Model source: hardware experiments

14

Inconsistency 1: power in simulation vs hardware

Model source: hardware experiments

Simulation: fixed increments

Simulation

15

Inconsistency 1: power in simulation vs hardware

Model source: hardware experiments

Simulation: fixed increments

Simulation

16

Inconsistency 1: models

17

Inconsistency 1: models

18

Inconsistency 1: summary

● Hardware power predictions do not match the simulation– Initially optimistic, later pessimistic

● Obscured by mixing power dynamics– Mode switching

– CPU and sensors

● Definition: – What deviations are acceptable, over what time?

● Assurance: – Simulation, corner case tests

19

Inconsistency 2: turning in instruction graphs vs planning

20

Inconsistency 2: turning in instruction graphs vs planning

A

B

C

D

21

Inconsistency 2: turning in instruction graphs vs planning

A

B

C

D

Instructiongraph A B C D

22

Inconsistency 2: turning in instruction graphs vs planning

A

B

C

D

A Planning + cost(C)

B D

Instructiongraph A B C D

23

Inconsistency 2: turning in instruction graphs vs planning

A

B

C

D

A Planning + cost(C)

B D

Instructiongraph A B C D

State estimation = ?

24

Inconsistency 2: models

25

Inconsistency 2: models

26

● Planning and execution represent turning differently ● State estimation is uncertain during a turn

– If optimistic, can run out of power

– If pessimistic, can waste time on charging

● Definition: – Bound of utility loss from state estimation uncertainty

● Assurance: – Quantitative model checking

Inconsistency 2: summary

27

What is common?

A

B D

A B C D

Power in simulation/hardware Turns in planning/execution

28

Models: big picture

29

Models: big picture

30

Models: big picture

31

What is common?

A

B D

A B C D

Power in simulation/hardware Turns in planning/execution

● Several models with differing assumptions● Definition: bounded mismatch● Checking requires multiple semantics

32

Detecting Inconsistencies

33

Detecting Inconsistencies

1. Relate models ● Manual

34

Relating models

35

Relating models

36

Detecting Inconsistencies

1. Relate models ● Manual

2. Specify consistency properties● Manual

37

Specifying consistency

“Predicted and simulated power always stay within a fixed error bound”

□ ∀p, t · p = PowPred(t) → □ ( time = t → | PowSim – p | < ε)

38

Detecting Inconsistencies

1. Relate models ● Manual

2. Specify consistency properties● Manual

3. Find potential inconsistencies● Automated

39

Finding potential inconsistencies

40

Finding potential inconsistencies

41

Detecting Inconsistencies

1. Relate models ● Manual

2. Specify consistency properties● Manual

3. Find potential inconsistencies● Automated

4. Check if the consistency properties hold ● Automated

42

Checking properties

43

Summary

Four-step approach: Relate – Specify – Find – Check