Estimating 3D Motion and Forces of Person-Object Interactions from Monocular Video Zongmian Li 1,2 , Jiri Sedlar 3 , Justin Carpentier 1,2 Ivan Laptev 1,2 , Nicolas Mansard 4 , Josef Sivic 1,2,3 1 Inria 2 DI ENS, PSL 3 CIIRC, CTU in Prague 4 LAAS-CNRS CVPR 2019, Long Beach, California
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Estimating 3D Motion and Forces of Person-Object Interactions from Monocular Video
Zongmian Li 1,2, Jiri Sedlar 3, Justin Carpentier 1,2 Ivan Laptev 1,2, Nicolas Mansard 4, Josef Sivic 1,2,3
1 Inria 2 DI ENS, PSL 3 CIIRC, CTU in Prague 4 LAAS-CNRS
CVPR 2019, Long Beach, California
Motivation: learn how to interact with objects
“Atlas” by Boston Dynamics
GoalsInput: • A monocular RGB video
Output:• Person & object 3D poses • 3D contact forces
2D pose estimation
[Cao et al., CVPR 2017]
Recognition stageMethod: a two-stage approach
2D pose estimation
Method: a two-stage approach
[He et al., CVPR 2016]
2D human pose
Contact recognizer
Recognition stageMethod: a two-stage approach
Contact recognizer
Method: a two-stage approach
[He et al., ICCV 2017]
2D human pose Contact states
Object 2D endpoint
estimation
Recognition stageMethod: a two-stage approach
Object 2D endpoint
estimation
Method: a two-stage approach
Person-object trajectory
optimization
[Tassa et al., IROS 2012][Carpentier et al., TRO, 2018]
2D object endpoints
2D human pose Contact states
Estimation stageRecognition stageMethod: a two-stage approach
2D object endpoints
2D human pose Contact states
Recognition stage Estimation stage
Estimated 3D motion and forces
[Tassa et al., IROS 2012][Carpentier et al., TRO, 2018]
Person-object trajectory
optimization
Method: a two-stage approach
Estimation stage
Object 3D posesPerson 3D poses
xx
minimizex, u, c ∑
e∈{person, object}∫
T
0le (x, u, c) dt
Person-objectperson-groundcontact forces
uu
Estimation stage
c
Contact positions
c
Estimation stage
Subject to 1. Contact motion model:
Estimation stage
Subject to: 2. Contact force constraints:
- Prevent the feet from sliding
Estimation stage
Subject to: 3. Lagrangian dynamics equation
[Carpentier, et al. Pinocchio. https://stack-of-tasks.github.io/pinocchio. 2015-2019]
M( ) + b( , ) = τ( , )··xx ·xx u c
Estimation stage
Experiments
A new dataset: Handtool videos
Barbell Hammer Scythe Spade
Zongmian Li 1,2, Jiri Sedlar 3, Justin Carpentier 1,2
Ivan Laptev 1,2, Nicolas Mansard 4, Josef Sivic 1,2,3
At the poster (#202):
1 Inria 2 DI ENS, PSL 3 CIIRC, CTU in Prague 4 LAAS-CNRS
CVPR 2019, Long Beach, California
•Additional details•Quantitative results•Discussion
Related Documents
