Page 1
Confidence-aware motion prediction for real-time collision avoidance
Andrea Bajcsy
Long-term Human Motion Prediction Workshop
ICRA 2019
Work with Sylvia Herbert, Jaime Fisac, David Fridovich-Keil, Steven Wang, Sampada Deglurkar, Claire Tomlin and Anca Dragan
Page 2
When robots observe behavior that is not well explained by their predictive models, how do
they produce safe but efficient motions?
Page 3
Connections to reachability analysis
Confidence-aware prediction & planning
Scaling up to multi-robot, multi-human scenarios
Page 4
Confidence-aware prediction & planning
Scaling up to multi-robot, multi-human scenarios
Connections to reachability analysis
Page 5
Low probabilityHigh probability
π π’π» π₯π»; π, π½) β ππ½π(π₯π»,π’π»;π)
Noisily-Rational Human Motion Prediction
αΆπ₯π» = ππ»(π₯π» , π’π»)
[Luce, 1959]
[Baker et al., 2007][Ziebart et al., AAAI 2008]
[Ramachandran et al., IJCAI 2007]
[Finn et al., ICML 2016]
[Pfeiffer et al., IROS 2016]
[Herman et al., ICRA 2015]
[Schultz et al., ICRA 2017]
Page 6
Robust Robot Planning with Human Predictions
π₯π
π
αΆπ₯π
= ππ
(π₯π
, π’π
)
Page 7
Robust Robot Planning with Human Predictions
π₯π
π
αΆπ₯π
= ππ
(π₯π
, π’π
)
αΆπ π
= αππ
(π π
, ππ
)
vs.
[Herbert, 2017]
[Mitchell, 2005]
[Lygeros, 2005]
Page 8
Hamilton-Jacobi Reachability Analysis
Robust Robot Planning with Human Predictions
π₯π
π
αΆπ₯π
= ππ
(π₯π
, π’π
)
αΆπ π
= αππ
(π π
, ππ
)
vs.
[Herbert, 2017]
π π, π = supπ[π’](β)
infπ’(β)
{ supπ‘β[0,π]
πππ π‘(πππ’,π(π‘))}
αΆπ = ππ
π₯π
, π’π
β π( αππ
π π
, ππ
)
[Mitchell, 2005]
[Lygeros, 2005]
[Herbert*, Chen*, Han, Bansal, Fisac, Tomlin. "FaSTrack: a Modular Framework for Fast and Guaranteed Safe Motion Planning." CDC, 2017.]
Page 9
Robust Robot Planning with Human Predictions
π₯π
π
Error boundπ π, π = sup
π[π’](β)infπ’(β)
{ supπ‘β[0,π]
πππ π‘(πππ’,π(π‘))}
αΆπ = ππ
π₯π
, π’π
β π( αππ
π π
, ππ
)
[Mitchell, 2005]
[Lygeros, 2005]
Hamilton-Jacobi Reachability Analysis
[Herbert*, Chen*, Han, Bansal, Fisac, Tomlin. "FaSTrack: a Modular Framework for Fast and Guaranteed Safe Motion Planning." CDC, 2017.]
Page 10
Robust Robot Planning with Human Predictions
π₯π
π
[Herbert, 2017]
[Fisac, 2018]
π π, π = supπ[π’](β)
infπ’(β)
{ supπ‘β[0,π]
πππ π‘(πππ’,π(π‘))}
αΆπ = ππ
π₯π
, π’π
β π( αππ
π π
, ππ
)
[Mitchell, 2005]
[Lygeros, 2005]
Hamilton-Jacobi Reachability Analysis
Page 11
π Crash > ππππππ ππππ‘βπππ β
Robust Robot Planning with Human Predictions
π₯π
π
[Herbert, 2017]
[Mitchell, 2005]
[Fisac, 2018]
[Lygeros, 2005]
Page 12
Robust Robot Planning with Human Predictions
[Mitchell, 2008]
π₯π
π
[Herbert, 2017]
[Fisac, 2018]
[Lygeros, 2005]
π Crash > ππππππ ππππ‘βπππ β
Page 13
What if the predictive model is wrong?
Modeled human goal
UnmodeledObstacle
Robot goal
Page 14
High Confidence Low Confidence
UnmodeledObstacle
UnmodeledGoal
human goal
human goal 1
human goal 2
robot goal
robot goal 1
robot goal 2
Page 15
π π’π» π₯π»; π, π½) β ππ½π(π₯π»,π’π»;π)
Bayesian Model Confidence
αΆπ₯π» = ππ»(π₯π» , π’π»)
Page 16
π π’π» π₯π»; π, π½) β ππ½π(π₯π»,π’π»;π)
Bayesian Model Confidence
αΆπ₯π» = ππ»(π₯π» , π’π»)
π½
Page 17
π π’π» π₯π»; π, π½) β ππ½π(π₯π»,π’π»;π)
Bayesian Model Confidence
αΆπ₯π» = ππ»(π₯π» , π’π»)
Page 18
π π’π» π₯π»; π, π½) β ππ½π(π₯π»,π’π»;π)
Bayesian Model Confidence
αΆπ₯π» = ππ»(π₯π» , π’π»)
Page 19
Bayesian Model Confidence
αΆπ₯π» = ππ»(π₯π» , π’π»)
ππ‘ π½ β π π’π»π‘ π₯π»
π‘ ; π, π½)ππ‘β1(π½)
Page 20
Bayesian Model Confidence
ππ‘ π½ β π π’π»π‘ π₯π»
π‘ ; π, π½)ππ‘β1(π½)
Page 21
Fixed confidence Bayesian confidenceπ½
Page 22
Less confident!
More confident!
Page 23
Confidence-aware prediction Scaling up to multi-robot, multi-human scenarios
Connections to reachability analysis
Robust motion planning
Page 24
Confidence-aware prediction Scaling up to multi-robot, multi-human scenarios
Connections to reachability analysis
Robust motion planning
Page 25
Forward Reachable Set
αΆπ₯π» = ππ»(π₯π» , π’π»)
πΉπ
π π₯π» , π‘ β {π₯β²: βπ’π»(β), π₯β² = π(π₯π» , π‘, π’π»(β))}
π(π₯π»(0), π‘, π’π»(β))
π₯π»(0)
Page 26
Forward Reachable Set
αΆβπ₯ = π£π»cos(π’π»)αΆβπ¦ = π£π»sin(π’π»)
Ξπ‘ β π£π»
Page 27
Forward Reachable Set
αΆβπ₯ = π£π»cos(π’π»)αΆβπ¦ = π£π»sin(π’π»)
Ξπ‘ β π£π»
Page 28
Forward Reachable Set
αΆβπ₯ = π£π»cos(π’π»)αΆβπ¦ = π£π»sin(π’π»)
π πππ β π£π»
Page 29
Forward Reachable Set
αΆβπ₯ = π£π»cos(π’π»)αΆβπ¦ = π£π»sin(π’π»)
π πππ β π£π»
πππππ β« 0
Page 30
Forward Reachable Set
αΆβπ₯ = π£π»cos(π’π»)αΆβπ¦ = π£π»sin(π’π»)
π πππ β π£π»
πππππ > 0
Page 31
Forward Reachable Set
αΆβπ₯ = π£π»cos(π’π»)αΆβπ¦ = π£π»sin(π’π»)
π πππ β π£π»
πππππ β 0
Page 32
Forward Reachable Set
αΆβπ₯ = π£π»cos(π’π»)αΆβπ¦ = π£π»sin(π’π»)
2 π β π£π» 3 π β π£π»π πππ β π£π»
Page 33
Forward Reachable Set
αΆβπ₯ = π£π»cos(π’π»)αΆβπ¦ = π£π»sin(π’π»)
2 π β π£π» 3 π β π£π»π πππ β π£π»
modeled goal
Page 34
Scaling up to multi-robot, multi-human scenarios
Connections between predictions and FRS
Confidence-aware prediction
Robust motion planning
Confidently determining subsets of the FRS to avoid
Page 35
Scaling up to multi-robot, multi-human scenarios
Connections between predictions and FRS
Confidence-aware prediction
Robust motion planning
Determining subsets of the FRS to avoid
Page 36
1000 x 1000 x 1000 = 1B 1000 x 1000 x 1000 x 1000 = 1T
1000
1000 x 1000 = 1M
Page 37
1000
1000
[Bajcsy, 2019]
Page 38
Hardware Demonstration
Page 39
Hardware Demonstration
Page 40
Confidence-aware predictions offer promising directions for scaling
Connections between predictions and FRS
Confidence-aware prediction
Robust motion planning
Determining subsets of the FRS to avoid
1000
1000
1000 x 1000 = 1M
1000 x 1000 x 1000 = 1B
1000 x 1000 x 1000 x 1000 = 1T
1000
Page 41
Confidence-aware predictions offer promising directions for scaling
Connections between predictions and FRS
Confidence-aware prediction
Robust motion planning
Determining subsets of the FRS to avoid
1000
1000
1000 x 1000 = 1M
1000 x 1000 x 1000 = 1B
1000 x 1000 x 1000 x 1000 = 1T
1000
Page 42
Papers
Herbert*, Chen*, Han, Bansal, Fisac, Tomlin. "FaSTrack: a Modular Framework for Fast and Guaranteed Safe Motion Planning." CDC, 2017.
Fisac*, Bajcsy*, Herbert, Fridovich-Keil, Wang, Tomlin, and Dragan. "Probabilistically Safe Robot Planning with Confidence-Based Human Predictions." RSS, 2018.
Fridovich-Keil*, Bajcsy*, Fisac, Herbert, Wang, Dragan, and Tomlin. βConfidence-Aware Motion Prediction for Real-Time Collision Avoidance.β IJRR, 2019
Bajcsy*, Herbert*, Fridovich-Keil, Fisac, Deglurkar, Dragan, and Tomlin, βA Scalable Framework for Real-Time Multi-Robot, Multi-Human Collision Avoidance.β ICRA, 2019.
Multi-robot, multi-human planning: https://github.com/HJReachability/faSTPeople
Code
Fast and safe robot tracking: https://github.com/HJReachability/fastrack
Pedestrian prediction: https://github.com/shwang/pedestrian_prediction
ROS wrapper for pedestrian prediction: https://github.com/abajcsy/crazyflie_human