Behrouz Haji Soleimani Dr. Moradi. Outline What is uncertainty? Some examples Solutions to uncertainty Ignoring uncertainty Markov Decision Process (MDP)

Behrouz Haji SoleimaniDr. Moradi

OutlineWhat is uncertainty?Some examplesSolutions to uncertainty

Ignoring uncertaintyMarkov Decision Process (MDP)Stochastic Motion Roadmap

A detailed exampleConclusion

What is uncertainty?Uncertainty in sensing

the current state of the robot and workspace is not known with certainty

Predictability

the future state of the robot and workspace cannot be deterministically predicted even when the current state and future actions are known

Uncertainty in sensing

It is not the world that is imperfect, it is our knowledge of it

PredictabilityUncertainty in workspace

Uncertainty in goal locationDynamic environments with moving obstacles

Uncertainty in robot’s motion

Uncertainty exampleA robot with imperfect sensing must reach a

goal location among moving obstacles (dynamic world)

Uncertainty exampleRobot created at Stanford’s ARL Lab to study

issues in robot control and planning in no-gravity space environment

air thrusters gas tank

air bearing

Uncertainty in motion

Uncertainty in motion

Markov Decision Process (MDP)MDP is a general approach to considering

uncertaintyDetermines model of the environmentDescretizes state spaceRequires explicitly defining transition

probabilities between statesWe can use dynamic programming to solve

the MDP

Stochastic Motion RoadmapCombines a roadmap representation of

configuration space with the theory of MDP’sMaximizes the probability of successUses sampling to

learn the configuration space (represented as states)

learn the stochastic motion model (represented as state transition probabilities)

Discretizes state spaceDiscretizes actions

Stochastic Motion RoadmapLearning Phase

Selecting random sample of discrete statesSample the robot’s motion model to build a

Stochastic Motion Roadmap (SMR)Calculating transition probabilities for each

actionQuery Phase

Specify initial and goal statesRoadmap is used to find a feasible pathPossibly optimizing some criteria such as

minimum length

Building the roadmap

Building the roadmap

Maximizing probability of successbuild an n × n transition probability matrix

P(u) for each u UFor each tuple (s, t, p) , we set

equals the probability of transitioning from state s to state t given that action u is performed

uE puPst )(

Maximizing probability of success

Maximizing probability of successIt is an MDP and has the form of the Bellman

equation

Where and

It can be optimally solved using infinite horizon dynamic programming

A detailed example

),,,( iiiii byxs

A detailed example

A detailed example

A detailed example

ConclusionUncertainty has a great effect on successfully

reaching the goalMDP can consider uncertainty in the modelSMR combines PRM and MDP to handle

uncertaintySMR maximizes the probability of successSMR makes balance between path safety and

minimum lengthContinuous actions in SMR is still an open

question

References [1] R. Alterovitz, T. Simeon, and K. Goldberg, “The Stochastic

Motion Roadmap: A Sampling Framework for Planning with Markov Motion Uncertainty” 2007

[2] R. Alterovitz, M. Branicky, and K. Goldberg, “Constant-curvature motion planning under uncertainty with applications in image-guided medical needle steering,” in Workshop on the Algorithmic Foundations of Robotics, July 2006.

[3] R. Alterovitz, A. Lim, K. Goldberg, G. S. Chirikjian, and A. M. Okamura, “Steering flexible needles under Markov motion uncertainty,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), Aug. 2005, pp. 120–125.

[4] B. Bouilly, T. Simeon, and R. Alami, “A numerical technique for planning motion strategies of a mobile robot in presence of uncertainty,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Nagoya, Japan, May 1995.

Questions ?

Thank you