1 Hierarchical Part-Based Human Body Pose Estimation * Ramanan Navaratnam * Arasanathan Thayananthan Prof. Phil Torr * Prof. Roberto Cipolla * University.

1

Hierarchical Part-Based Hierarchical Part-Based Human Body Pose EstimationHuman Body Pose Estimation

* Ramanan Navaratnam

* Arasanathan Thayananthan

† Prof. Phil Torr

* Prof. Roberto Cipolla

* University Of Cambridge † Oxford Brookes University

2

IntroductionIntroduction

Input

3

IntroductionIntroduction

Input Output

4

OverviewOverview

1. Motivation

2. Hierarchical parts

3. Template search

4. Pose estimation in a single frame

5. Temporal smoothing

6. Summary & Future work

5

OverviewOverview

1. Problem motivation ???

2. Hierarchical parts

3. Template search

4. Pose estimation in a single frame

5. Temporal smoothing

6. Summary & Future work

6

OverviewOverview

1. Problem motivation ???

2. Hierarchical parts

3. Template search

4. Pose estimation in a single frame

5. Temporal smoothing

6. Summary & Future work

7

OverviewOverview

1. Problem motivation ???

2. Hierarchical parts

3. Template search

4. Pose estimation in a single frame

5. Temporal smoothing

6. Summary & Future work

8

MotivationMotivation ‘Real-time Object Detection for Smart Vehicles’

– D. M. Gavrila & V. Philomin (ICCV 1999)

‘Filtering using a tree-based estimator’ – Stenger et.al. (ICCV 2003)

9

MotivationMotivation

Exponential increase of templates with dimensions

‘Real-time Object Detection for Smart Vehicles’ – D. M. Gavrila & V. Philomin (ICCV 1999)

‘Filtering using a tree-based estimator’ – Stenger et.al. (ICCV 2003)

10

MotivationMotivation ‘Pictorial Structures for Object Recognition’

– P. Felzenszwalb & D. Huttenlocher (IJCV 2005)

‘Human upper body pose estimation in static images’ – M.W. Lee & I. Cohen (ECCV 2004)

11

MotivationMotivation

Part based approach Assembling parts together is complex

‘Pictorial Structures for Object Recognition’ – P. Felzenszwalb & D. Huttenlocher (IJCV 2005)

‘Human upper body pose estimation in static images’ – M.W. Lee & I. Cohen (ECCV 2004)

12

MotivationMotivation ‘Automatic Annotation of Everyday Movements’

– D. Ramanan & D. A. Forsyth (NIPS 2003)

‘3-D model-based tracking of humans in action:a multi-view approach’ – D. M. Gavrila & L. S. Davis (CVPR 1996)

13

MotivationMotivation ‘Automatic Annotation of Everyday Movements’

– D. Ramanan & D. A. Forsyth (NIPS 2003)

‘3-D model-based tracking of humans in action:a multi-view approach’ – D. M. Gavrila & L. S. Davis (CVPR 1996)

‘State space decomposition’

14

Hierarchical PartsHierarchical Parts

15

Hierarchical PartsHierarchical Parts

16

Hierarchical PartsHierarchical Parts

17

Hierarchical PartsHierarchical Parts

18

Hierarchical PartsHierarchical PartsConditional prior

p(x i

/xpar ent(i))

Spatial dimensions (translation)Joint Angles

19

Hierarchical PartsHierarchical Parts

Head and torsoUpper armLower Arm

False Positive

Tru

e Po

siti

ve

20

Hierarchical PartsHierarchical PartsDetection Threshold = 0.81

Detections

Head and torso

6156

Part

21

Hierarchical PartsHierarchical PartsDetection Threshold = 0.81

Detections

Head and torso

6156

13 199 44 993

Part

Lower arm

22

Template SearchTemplate Search

23

Template SearchTemplate Search

24

Template SearchTemplate Search

25

Template SearchTemplate Search

Features Chamfer distance Appearance

26

Template SearchTemplate Search

Features Chamfer distance Appearance

27

Template SearchTemplate Search

Features Chamfer distance Appearance

28

Template SearchTemplate Search

Features Chamfer distance Appearance

29

Template SearchTemplate Search

Features Chamfer distance Appearance

30

Template SearchTemplate Search

Features Chamfer distance Appearance

31

Template SearchTemplate Search

Features Chamfer distance Appearance

32

Template SearchTemplate Search

Features Chamfer distance Appearance

33

Template SearchTemplate Search

Features Chamfer distance Appearance

34

Template SearchTemplate Search

Learning Appearance Match ‘T’ pose based on edge likelihood only in initial

frames

Update 3D histograms in RGB space that approximates P(RGB/part) and P(RGB)

35

Pose Estimation in a Single FramePose Estimation in a Single Frame

36

Pose Estimation in a Single FramePose Estimation in a Single Frame

37

Pose Estimation in a Single FramePose Estimation in a Single Frame

38

Temporal SmoothingTemporal Smoothing

HMM

39

Temporal SmoothingTemporal Smoothing

HMMT = t

40

Temporal SmoothingTemporal Smoothing

HMM

Viterbi back tracking

41

Temporal SmoothingTemporal Smoothing

Viterbi back tracking

42

Temporal SmoothingTemporal Smoothing

43

Summary & Future workSummary & Future work

Summary

Realtime process (unoptimized code at 1Hz, 2.4 Ghz IG RAM)

3D pose

Automatic initialisation and recovery from failure

44

Summary & Future workSummary & Future work

Summary

Realtime process (unoptimized code at 1Hz, 2.4 Ghz IG RAM)

3D pose

Automatic initialisation and recovery from failure

Future workExtend robustness to illumination changes

Non-fronto-parallel poses

Poses when arms are inside the body silhouette

Simple gesture recognition by assigning semantics to regions of articulation space