Page 1
1
Cut-And-Stitch: Efficient Parallel Learning of Linear Dynamical Systems on SMPs
Lei LiComputer Science Department
School of Computer Science Carnegie Mellon University
[email protected]
School of Computer Science
Efficient Parallel Learning of Linear Dynamical Systems on SMPs
Page 2
2
Motion stitching via effort minimizationwith James McCann, Nancy Pollard and
Christos Faloutsos [Eurographics 2008]
Parallel learning of linear dynamical systemswith Wenjie Fu, Fan Guo, Todd Mowry and
Christos Faloutsos[KDD 2008]
Page 3
3
Background
• Motion Capture
• Markers on human body, optical cameras to capture the marker positions, and translated into body local coordinates.
• Application:– Movie/game/medical industry
Page 4
4
Outline• Background• Motivation: effortless motion stitching• Parallel learning with Cut-And-Stitch• Experiments and Results• Conclusion
Page 5
5
Motivation
• Given two human motion sequences, how to stitch them together in a natural way( = looks natural in human’s eyes)?
e.g. walking to running• Given a human motion sequence, how to find
the best natural stitchable motion in motion capture database?
Page 6
6
Intuition
• Intuition:– Laziness is a virtue. Natural motion use minimum
energy• Laziness-score (L-score) = energy used during
stitching• Objective:
– Minimize laziness-score
Page 7
7
Example
Taking off
landing
Page 8
8
Example, Natural stitching
Taking off
landing
Page 9
9
But, how about this way?
Taking off
landing
Page 10
10
Observations
• Naturalness depends on smoothness• Naturalness also depends on motion speed
Page 11
11
Proposed Method
• Estimate stitching path using Linear Dynamical Systems
Page 12
12
Proposed Method (cont’)
• Estimate the velocity and acceleration during the stitching, compute energy (defined as L-score)
Page 13
13
Proposed Method (cont’)
• Minimize L-score with respect to any stitching hops. (defined as elastic L-score)
Page 14
14
Example stitching
• Link to video
Page 15
15
Outline• Background• Motivation: effortless motion stitching• Parallel learning with Cut-And-Stitch• Experiments and Results• Conclusion
Page 16
16
Parallel Learning for LDS
• Challenge: – Learning Linear Dynamical System is slow for long
sequences• Traditional Method:
– Maximum Likelihood Estimation via Expectation-Maximization(EM) algorithm
• Objective:– Parallelize the learning algorithm
• Assumption:– shared memory architecture
Page 17
17
Linear Dynamical Systemaka. Kalman Filter
• Parameters: =(u0, V0, A, Γ, C, Σ)
• Observation: y1…yn
• Hidden variables: z1… zn
17
Z1 Z2 Z3 Z4Z5
Y1 Y2 Y3Y4 Y5
N(A z∙ 1, Γ)
N(u0, V0)
N(C z∙ 3, Σ)
N(A z∙ 2, Γ)
N(C z∙ 1, Σ) N(C z∙ 2, Σ) N(C z∙ 4, Σ)
N(A z∙ 3, Γ)
N(C z∙ 5, Σ)
N(A z∙ 4, Γ)
Page 18
18
Example
given positions, estimate dynamics (i.e. params)
z
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
z
2
Time
Position of left elbow
Page 19
19
Traditional:How to learn LDS?
Page 20
Sequential Learning (EM)z
1
z
2
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
Compute P(z1 | y1)
Time*
Measured
Estimated
Position of left elbow
Page 21
21
Sequential Learning (EM)z
1
z
2
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
From P(z1 | y1) Compute P(z2| y1 , y2)
Time*
Intuition: z2 may be close to z1
*
Measured
Estimated
Position of left elbow
Page 22
22
Sequential Learning (EM)z
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
From P(z2| y1 , y2) Compute P(z3| y1 , y2 , y3)
z
2
Time**
*
Measured
Estimated
Position of left elbow
Page 23
23
Sequential Learning (EM)z
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
From P(z3| y1 , y2 , y3) Compute P(z4| y1 , y2 , y3 , y4)
z
2
23
Time**
*
*
Measured
Estimated
Position of left elbow
Page 24
24
Sequential Learning (EM)z
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
z
2
From P(z4| y1 , y2 , y3 , y4) Compute P(z5| y1 , y2 , y3 , y4 , y5)
Time**
**
*
Measured
Estimated
Position of left elbow
Page 25
25
Sequential Learning (EM)z
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
z
2
From P(z5| y1 , y2 , y3 , y4 , y5) Compute P(z6| y1 , y2 , y3 , y4 , y5 , y6)
Time**
**
**
Measured
Estimated
Position of left elbow
Page 26
26
*
Sequential Learning (EM)z
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
z
2
From P(z6| y1 , y2 , y3 , y4 , y5 , y6) Compute P(z5| y1 , y2 , y3 , y4 , y5 , y6)
26
Time**
*
*Measured Estimated
*
*Intuition: take the future backward
Position of left elbow
Page 27
27
Sequential Learning (EM)z
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
z
2
From P(z6| y1 , y2 , y3 , y4 , y5 , y6) Compute P(z4| y1 , y2 , y3 , y4 , y5 , y6)
*
2727
Time**
*
*Measured
*
*
*
Estimated
* **
**
Position of left elbow
Page 28
Sequential Learning (EM)z
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
z
2
From P(z4| y1 , y2 , y3 , y4 , y5 , y6) Compute P(z3| y1 , y2 , y3 , y4 , y5 , y6)
28
*
2828
Time**
*
*Measured
*
*
**
Estimated
* **
**
Position of left elbow
Page 29
Sequential Learning (EM)z
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
z
2
From P(z3| y1 , y2 , y3 , y4 , y5 , y6) Compute P(z2| y1 , y2 , y3 , y4 , y5 , y6)
29
*
2929
Time**
*
*Measured
*
*
**
*
Estimated
* **
**
Position of left elbow
Page 30
30
Sequential Learning (EM)z
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
z
2
From P(z2| y1 , y2 , y3 , y4 , y5 , y6) Compute P(z1| y1 , y2 , y3 , y4 , y5 , y6)
303030
*
*
3030
Time**
*
*Measured
*
*
**
*
Estimated
* **
**
Position of left elbow
Page 31
Sequential Learning (EM)z
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
z
2
From all posterior z1 , z2 , z3 , z4 , z5 , z6
P(z1| y1 , y2 , y3 , y4 , y5 , y6), P(z2| y1 , y2 , y3 , y4 , y5 , y6)…Compute sufficient statistics E[zi] E[zizi’] E[zi-1zi’]
Page 32
32
Sequential Learning (EM)z
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
z
6
y6
z
2
*
*Time*
*
*
*Measured
*
*
**
*
with sufficient statistics, compute argmax ←likelihood(θ) θ
reconstructed signal
Position of left elbow
Page 33
34
How to parallelize it?
Speed Bottleneck: sequential computation of posterior
z
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
y6
z
2
z
6
Page 34
35
“Leap of faith”
start computation without feedback from previous node (cut),
and reconcile later (stitch)
Page 35
36
Proposed Method: Cut-And-Stitchz
1
y1
y2
z
3
y3
z
4
y4
z
5
y5
y6
z
2
z
6
υ2,Φ2,η2,Ψ2υ1,Φ1,η1,Ψ1
z
1
y1
y2
z'
2
z
2
z
3
y3
z
4
y4
z'
4
z
5
y5
y6
z
6
υ3,Φ3,η3,Ψ3
start computation without feedback from previous node (cut)
reconcile later (stitch)
Page 36
Cut-And-Stitchυ2,Φ2,η2,Ψ2υ1,Φ1,η1,Ψ1
z
1
y1
y2
z'
2
z
2
z
3
y3
z
4
y4
z'
4
z
5
y5
y6
z
6
υ3,Φ3,η3,Ψ3
Cut step: Estimate posteriors (E)
Time
Measured
Estimated
Intuition: compute all three at once*
*
*
P(z1| y1), P(z3| y3), P(z5| y5)Position of left elbow
Page 37
Cut-And-Stitchυ2,Φ2,η2,Ψ2υ1,Φ1,η1,Ψ1
z
1
y1
y2
z'
2
z
2
z
3
y3
z
4
y4
z'
4
z
5
y5
y6
z
6
υ3,Φ3,η3,Ψ3
Cut step: Estimate posteriors (E)
Time*
**
*
**
Measured
Estimated
Position of left elbow
Page 38
Cut-And-Stitchυ2,Φ2,η2,Ψ2υ1,Φ1,η1,Ψ1
z
1
y1
y2
z'
2
z
2
z
3
y3
z
4
y4
z'
4
z
5
y5
y6
z
6
υ3,Φ3,η3,Ψ3
Cut step: Estimate posteriors (E)
Time
Measured
*
**
*
***
*
* Intuition: backward adjust all at once
Position of left elbow
Page 39
Cut-And-Stitch
Stitch step: Collect sufficient Statistics (C) Maximize parameters (M)
υ2,Φ2,η2,Ψ2υ1,Φ1,η1,Ψ1
z
1
y1
y2
z'
2
z
2
z
3
y3
z
4
y4
z'
4
z
5
y5
y6
z
6
υ3,Φ3,η3,Ψ3
41
Time
Measured
*
**
*
***
*
*
Position of left elbow
Page 40
Cut-And-Stitchυ2,Φ2,η2,Ψ2υ1,Φ1,η1,Ψ1
z
1
y1
y2
z'
2
z
2
z
3
y3
z
4
y4
z'
4
z
5
y5
y6
z
6
υ3,Φ3,η3,Ψ3
Stitch together: Re-estimate block parameters (R)
Time
Measured
*
**
*
***
*
*
*
*
*
Intuition: exchange messages cross block Iterate…
reconstructed signal
Position of left elbow
Page 41
43
Outline• Background• Motivation: effortless motion stitching• Parallel learning with Cut-And-Stitch• Experiments and Results• Conclusion
Page 42
44
Experiments
Q1: How much speed up can we get?
Q2: How good is the reconstruction accuracy?
Page 43
45
Experiments
• Dataset:– 58 human motion sequences, 200 – 500 frames– Each frame with 93 bone positions in body local coordinates– http://mocap.cs.cmu.edu
• Setup:– Supercomputer: SGI Altix system, distributed shared
memory architecture – Multi-core desktop: 4 Intel Xeon cores, shared memory
• Task:– Learn the dynamics, hidden variables and reconstruct
motion
Page 44
46
Q1: How much speed up?Supercomputer Result
spee
dup
# of processors
ideal
average of 58
Page 45
47
Q1: How much speed up?Multi-core Result
spee
dup
# of cores
ideal
average of 58
Page 46
48
Q2: How good?
walking (#22) jumping (#1) running (#45)0.000%
0.500%
1.000%
1.500%
2.000%
2.500%
Sequential algCut-And-Stitch
Normalized Reconstruction Error
Result: ~ IDENTICAL accuracy
Page 47
49
Conclusion & Contributions
• A distance function for motion stitching– Based on first principle: minimize effort
• General approximate parallel learning algorithm for LDS– Near linear speed up– Accuracy (NRE): ~ identical to sequential learning– Easily extended to HMM and other chain Markovian
models• Software (C++ w. openMP) and datasets:
www.cs.cmu.edu/~leili/paralearn
Page 48
50
Promising Extensions
• Extension– HMM – other Markov models (similar graphical model)
• Open Problem:– Can prove the error bound?
Page 49
51
Thank you
• Questions