Lecture 2: Reactive Systems Gal A. Kaminka [email protected] Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.

Lecture 2: Reactive Systems

Gal A. Kaminka

[email protected]

Introduction to Robots and Multi-Robot Systems

Agents in Physical and Virtual Environments

2 © Gal Kaminka

Previously, on Robots …

Robots are agents in physical or virtual environments Persistent, situated, responsive

Environments have different characteristics Static/dynamic, accessible?, deterministic?, ….

Since we are lazy, we want robots to do things for us Robots must consider task when deciding what to do

Action-selection problem:

What to do now in service of the task?

3 © Gal Kaminka

Physical Environments Dynamic Non-deterministic Inaccessible Continuous

4 © Gal Kaminka

A Crash Course in AI Planning

Planning: An approach to action-selection problem Very long history, since the very beginning of AI 1971, seminal paper: STRIPS (Fikes and Nilsson)

Still cited and taught today, despite much progress STRIPS originally developed for SRI robot, Shakey

This is ironic Later on, STRIPS planning was rejected for robot control

5 © Gal Kaminka

AI Planning Definition: State

{ On(A,table), On(B,table), On(C,table), In-hand(nil) }

A B C

{ On(A,B), On(B,table), On(C,table), In-hand(D) }

A

B CD

6 © Gal Kaminka

AI Planning Definition: Operators Operators change state of the world

Preconditions, add-list, delete-list Pick-A(?x)

Preconditions: On(A, ?x), In-hand(nil) Add: In-hand(A) Delete: In-hand(nil), On(A, ?x)

Put-A(?y) Preconditions: In-hand(A), not On(?x, ?y) Add: On(A,?y), In-hand(nil) Delete: In-hand(A)

7 © Gal Kaminka

STRIPS Planning Given:

Initial state of the world Operators Goal state

Produce: Plan: Ordered list of instantiated operators Will change the world from initial state to goal state

8 © Gal Kaminka

Planning Example

A B C

Initial State

A

B

C

Goal State

A

B

C

After Pick-B(table)

A

B

C

After Put-B(A)

A

B

C

After Pick-C(table)

After Put-C(B)

9 © Gal Kaminka

Planning on robots

Sense initial state using sensors Create a full plan given goal state (given task) Feed plan, step-by-step to motors

No need to sense again

What’s wrong with this?(Hint: Think about Schoppers’ paper)

10 © Gal Kaminka

Deliberative Control

Deliberative: Has internal state (typically a model of the world) Uses this internal state to make decisions Decisions made between alternatives

Sense Model Think Act

11 © Gal Kaminka

When plans goes wrong Dynamic environment

State changes even if no operator applied Non-deterministic

State changes not according to operator specs Inaccessible

Cannot sense entire state of the world Continuous

Predicate-based description of world is discrete

12 © Gal Kaminka

Reactive control

Reactive: No internal state Direct connection from sensors to actions S-R (stimulus response) systems No choices, no alternatives

Sense Hard Wiring Act

13 © Gal Kaminka

Universal Planning Have a plan ready for any possible contingency

Scouts: Be prepared! From any initial state, know how to get to goal state Input: Operators, goal state

Do not need to give initial state Output: Decision tree

What operator to take, depending on environment state Not a single ordered list of operators

14 © Gal Kaminka

Universal planning algorithm

A

B

C

After Pick-B(table)

A

B

C

Goal State

A

B

C

After Pick-C(table)

A

B

C

After Put-B(A)

A B C

Initial State

15 © Gal Kaminka

Robot Control AlgorithmUsing Universal Planning

Robot given task (goal, operators) Uses universal planner to create universal plan Robot senses environment Goal state reached?

No: Execute operator according to decision tree Yes (keep persistency)

16 © Gal Kaminka

Advantages of Universal Planning

Guaranteed to use optimal (shortest) plan to goal A very good thing! Optimal solution to action selection problem

Robust to failures Robust in dynamic and non-deterministic domains

17 © Gal Kaminka

Problems with Universal Planning

Assumes accessibility Assumes perfect sensors Assumes discrete actions (operators)

18 © Gal Kaminka

Universal plan as mapping sensors to actions

Universal plan can be viewed as a function Sensor readings to actions

u: SA

Essentially a table: For each state, give action Schoppers uses a decision-tree representation

19 © Gal Kaminka

Problems: Planning Time What is the planning time? Planning time grows with the number of states

Since we have to enumerate operator for every state What is the number of states in an environment?

Worst case:

All possible combinations of sensor readings

(state predicates)

20 © Gal Kaminka

Problems: Universal Plan Size Plan size grows with the number of possible states

“Curse of dimensionality”

X1

X2

Pick

Pick

Put

Put

X1

X2

Pick PickPut Put

X2

X1

Pick PickPut Put

21 © Gal Kaminka

Problems: Stupid executioner Schoppers:

Baby goes around knocking blocks around? Ginsberg:

What if baby repeatedly knocks down the same block? Universal plans may get into cycles This is because no deliberation is done Universal planner relies on simple executioner

Sense, consult table, act Same as regular planner – except for sensing

22 © Gal Kaminka

שאלות?

23 © Gal Kaminka

Brooks’ Subsumption Architecture

Multiple levels of control: Behaviors

Avoid Object

Wander

Explore

Map

Monitor Change

Identify objects

Plan changes

24 © Gal Kaminka

Why does this work? It breaks the ideal universal plan into behaviors

avoids the curse of dimensionality Behaviors (levels) interact to generate behavior

Note that programmer is responsible for task-oriented design

Goes both below and above universal plans Hand programmed: approximate plan

Not automatically generated

25 © Gal Kaminka

Subsuming Layers How to make sure overall output is coherent?

e.g., avoid object is in conflict with explore Subsumption hierarchy: Higher levels modify lower

Avoid Object

Wander

Explore

Map

26 © Gal Kaminka

Coherence using subsumption

Key principle

Higher layers control input/outputs of lower layers

In practice: Can be difficult to design

a lot depends on programmer Single module per layer can be restrictive Increasingly challenging to control at higher levels

27 © Gal Kaminka

Irony

Ginsberg’s article pretty much killed universal planning Though occasional papers still published

Reactive control very popular in practice But due to theory problem, no more automated planners! So we get lots of reactive plans, but no planning

28 © Gal Kaminka

Irony again

Ginsberg was right: Approximating universal plan is possible Tends to be useful only in fairly low level locomotion control

Approximation is what Brooks had done Which is why he often gets the credit for the revolution

29 © Gal Kaminka

Starting next week

Behavior-Based Control:

Expanding on Subsumption