Ames Research Center Incremental Contingency Planning Richard Dearden, Nicolas Meuleau, Sailesh Ramakrishnan, David E. Smith, Rich Washington window [10,14:30]

AmesResearchCenter

Incremental Contingency Planning Richard Dearden, Nicolas Meuleau,

Sailesh Ramakrishnan, David E. Smith, Rich Washington

window

[10 ,14:30]

power power

Workspace pandata

Drive (-1)Dig(5)Visual servo (.2, -.15) NIR

Lo res Rock finder Hi res Carbonate

[10 ,14:30]

X X XX?

AmesResearchCenter

Limited onboard processingCPU, memory, time

Safetysequence checking

Anticipationsetup steps

Why Contingency Planning ??

AmesResearchCenter The Planning Problem

Visual servo (.2, -.15)

Warmup NIR

Dig(5) Drive(-1) NIR ………Compress

Drive(2)

Maximize (Expected) Scientific Return

Given:start timeposeenergy availableactions with uncertain:

durationsresource usage

Possible science objectivesimagessamples

AmesResearchCenter Technical Challenges

Continuous time (& resources)

Continuous outcomes

Time (& resource) constraints

Concurrency

Goal selection & optimizationg1, g2, g3, g4 …

Visual servo (.2, -.15)

Warmup NIR

Lo res Rock finder NIR

∆p =∆t =

NIR

E > 2 Aht [10:00, 14:00]

Time Power Storage

AmesResearchCenter Just in Case (JIC) Scheduling

1. Seed schedule2. Identify most likely failure3. Generate a contingency branch4. Integrate the branch

Advantages: TractabilitySimple plansAnytime

.4 .2.1

AmesResearchCenter Just in Case (JIC) Planning

1. Seed plan2. Identify most likely failure3. Generate a contingency branch4. Integrate the branch

.4 .2.1

AmesResearchCenter Limits of JIC Scheduling Heuristics

Dig(60)Visual servo (.2, -.15)

Lo res Rock finder LIB

= 120s = 60s

= 300s = 5s

= 1000s = 500s

t [9:00, 16:00] = 5s = 1s

= 120s = 20s V = 50

HiRes

V = 10

t [10:00, 13:50] = 600s = 60s

t [9:00, 14:30] = 5s = 1s

V = 5

Warmup LIB

= 1200s = 20s

Most probable failure points may not be the best branch-points:

It is often too late to attempt other goals when the plan is about to fail.

: most probable failures$ : most interesting branch point

ExpectedUtility

PowerStart time

10

1520

5

13:20

14:4014:20

14:0013:40

True for all initial states in the grey box.

Drive (-2) NIR

V = 100

t [10:00, 14:00] = 600s = 60s $

AmesResearchCenter Just in Case (JIC) Planning

1. Seed plan2. Identify best branch point3. Generate a contingency branch5. Evaluate & Integrate the branch

? ??

Construct plangraph

Back-propagate value tables

Compute gain

Select branch condition & goals

?

r

Vb

Vm

AmesResearchCenter Construct Plangraph

g1

g2

g3

g4

AmesResearchCenter Value Tables

g1

g2

g3

g4

V1

V2

V3

V4

r

r

r

r

AmesResearchCenter Example

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

AmesResearchCenter Simple Propagation

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

AmesResearchCenter Simple Propagation

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

5

25

AmesResearchCenter Simple Propagation

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

5

25

12

AmesResearchCenter Conjunctions

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

5

25

12

12

r

12

t

AmesResearchCenter Propagation

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

5

25

12

12

r

12

t

51

r

AmesResearchCenter Propagation

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

5

25

12

12

r

12

t

51

r

15

t

115

t

AmesResearchCenter Combining Tables

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

5

25

12

12

r

12

t

51

r

15

t

115

t

15

t

AmesResearchCenter Discharging Assumptions

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

5

25

12

r

12

t

51

r

15

t

115

t

15

t

15

15

AmesResearchCenter Propagation

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

5

25

15

15

16

AmesResearchCenter Combining Tables

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

15

15

61

18

5

2515

AmesResearchCenter Combining Tables

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

15

15

61

5

2515

5

258

18

AmesResearchCenter Ordering

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

15

15

61

5

2515

5

258

DCE

CDE

AB

18

AmesResearchCenter Achieving Multiple Goals

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

15

15

61

5

2515

5

258

18

30

g+g’g

g’ +

AmesResearchCenter Achieving Multiple Goals

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

15

15

61

18

5

2515 30

g+g’g

g’

5g+g’

g

g’

258 30

AmesResearchCenter Goal Annotation

A

E

B

e

5

(1, 5)

(3, 3)

(10, 15) (10, 15)

(2, 2)

C

Ds t

r

q

p

g

g’

e1

5

15

15

15

61

18

gg

g’

g’

g’g’

g’

5

2515 30

g+g’g

g’

5g+g’

g

g’

258 30

AmesResearchCenter Just in Case (JIC) Planning

1. Seed plan2. Identify best branch point3. Generate a contingency branch5. Evaluate & Integrate the branch

? ??

Construct plangraph

Back-propagate value tables

Compute gain

Select branch condition & goals

?

r

Vb

Vm

AmesResearchCenter Estimating Branch Value

V1

V2

V3

V4

V

r

V

r

V

r

MaxV

r

AmesResearchCenter Plan Statistics

r

V1

V2

V3

V4

P

r

plan value functionresource probability

Vm

Vb

r

AmesResearchCenter Expected Branch Gain

V1

V2

V3

V4

P

r

Gain = ∫ P(r) max{0,Vb(r) - Vm(r)} dr∞

0

Vb

r

rVb

Vm

AmesResearchCenter Selecting the Branch Condition

V1

V2

V3

V4

P

r

branch condition

rVb

Vm

Vb

r

branch condition

AmesResearchCenter Selecting Branch Goals

r

V1

V2

V3

V4

P

r

branch goals

g1

g3

g3

g1

Vb

r

rVb

Vm

AmesResearchCenter Evaluating the Branch

1. Seed plan2. Identify best branch point3. Generate a contingency branch4. Evaluate & integrate the branch

? ?? ?

r

Vb

Vm Compute value function

Compute actual gain

AmesResearchCenter Actual Branch Gain

rVb

P

r

Gain = ∫ P(r) max{0,Vb(r) - Vm(r)} dr∞

0

Vm

r

Vb Branch value function

actual branch condition

AmesResearchCenter Remarks: Single Plangraph

1. Seed plan2. Identify best branch point3. Generate a contingency branch5. Evaluate & Integrate the branch

? ??

Construct plangraph

Back-propagate value tables

Compute gain

Select branch condition & goals

?

r

Vb

Vm

AmesResearchCenter Plan Graph

g1

g2

g3

g4

V1

V2

V3

V4

r

r

r

r

AmesResearchCenter Branch Initial Conditions

g1

g2

g3

g4

V1

V2

V3

V4

r

r

r

r

v

rv

r

v

rv

r

v

r

{p}

{q,r}

{p,r}

AmesResearchCenter Single Plangraph

1. Seed plan2. Identify best branch point3. Generate a contingency branch4. Evaluate & integrate the branch

? ?? ?

r

Vb

Vm

Discharge conditions

Compute gain

Select branch condition & goals

Construct single plangraph

Back-propagate value tables

AmesResearchCenter Issues

Sensor costs

Setup steps

Utility updates

Floating contingencies

Drive (-1)Dig(5)Visual servo (.2, -.15) Hi res

Warm NIR

Lo res Rock finder NIR[11 ,14:00]

Visual servo Hi res Visual servo NIR

V = 100

V’ = 30

Lo res Rock finder NIR

Drive (-1)Dig(5)Visual servo (.2, -.15) Hi res?

AmesResearchCenter

The End.