Retargetting Motion to New Characters Michael Gleicher
Retargetting Motion to New Characters
Michael Gleicher
Homework
Keep working on next homework Due on Tuesday Build a mocap data previewer
mocap.cs.cmu.edu (see Tools and Resources) Load data file Create favorite data structures Use DOFs to animate a simple character
Homework
Grading Demo your IK solution Sunday or Monday Room 002a this time! Sign up sheet up front
Readings
Interactive Control of Avatars Animted with Human Motion Data (Lee et al.)
Motion Graphs (Kovar et al.)
Spacetime Constraints
Homeworks look good Let’s discuss for a moment
Sequential Quadratic Programming aka Newton’s method and Newton-Lagrange Wilson (1963) and Beale’s SOLVER (1967)
A member of the descent family of constrained optimization techniques
Makes critical assumption that approximating constraints with linear functions is meaningful
Cost function must be quadratic
Spacetime Constraints
This is just a search method Penalty Methods permit constraints to be
violated when searching for optimality Non-linear Least Squares used by Gleicher
Conjugate gradient Simulated annealing loses benefit of
derivatives What about GA?
Spacetime Constraints
The building blocks define SQP The implementation of SQP can vary
Matrix inversion Special routines for handling sparse or
positive definite matrices
Motivation: Why Do We Care?
Have one animated motion, want another For characters with identical structure, but
different proportions Manual tweaking is slow and difficult Want to preserve high-level properties that
are hard to define
Simple Try 1
Just reuse parameters of original motion in new motion with different character
hmm … something’s not quite right …
Simple Try 2
Need to meet constraints, for instance foot touches floor
No problem, just use inverse kinematics to establish constraints each frame
… better, but not very smooth …
What’s the problem?
No temoporal notion of constraints Solving locally for each frame
generates unwanted artifacts, can’t plan ahead
Problem: introduces high frequency motions
Try again with Filtering
Apply a low-pass filter to remove unwanted high frequencies
But, may violate constraints Want to preserve high frequencies in original
motion, without introducing new ones
Original Smoothed IKIK Retargetted
Enough Failed Attempts
Need a way to solve the constrained optimization problem that takes the whole motion into account
Where have we seen this before? Spacetime constraints Through-the-lens camera control Neuroanimator Virutal Creatures
Spacetime Constraints
Mathematically encode all constraints and objectives
Not so easy to capture desirable aspects of original motion mathematically
More constraints and more complicated objective functions take longer to solve Don’t use physics constraints on every
step
Tools for Traveling through Space and Time
Constraints Identify features that must be present
Constraints
qti = parameters of motion (joint angles and root position) at time ti
Define constraint as: f(qti) = c c is a constant
Constraint examples Parameter’s value is in a certain range (joint limits) Point on character is in specific location or within a range Two points are a certain distance apart Vector between points has a certain orientation
Tools for Traveling through Space and Time
Objective Functions Stay close to the original (especially high
frequency motion) Use constraints to keep objective function
from being too simple
minimize g(x) subject to f(x) = c
Objective Functions
Goal: Minimize noticeable change Hard to define, choose something simple
instead Minimize differences between new motion
and original motions Define functions of motion parameters
Original motion: New (retargeted) Motion: Difference between the two:
Objective Function:22
0 )())()(()( tttdtmtmmg
)()()( 0 tdtmtm )(tm)(0 tm
)(td
Tools for Traveling through Space and Time
Initial Solution for Solver Some basic scaling of original data to get
a good guess
Starting Point
Good initial motion can speed numerical solver
Start just by scaling motion to match scaled character
If needed, translate motion to get as close as possible to satisfying constraints
Representation
Want to minimize introduction of high frequencies
Choose representation of to achieve this
Cubic B-splines Control point spacing determines frequency
limit Try having control points every 2, 4 or 8
frames (uniform spacing)
)(td
Retargeting Procedure
1. Take original motion and identify constraints2. Scale and translate to find initial estimate for
solution, m1(t)3. Choose representation for d(t) – what is the
control point spacing4. Solve non-linear constraint problem: find d(t)
that will satisfy constraints when added to motion estimate from step 2 (use spacetime)
5. If constraints are not well satisfied, solve again using m1(t) + d(t) as initial motion in step 2 and choose a denser spacing in step 3
Solving the Constrained Optimization Problem
Want to solve for control point values (call the vector of these x)
Define constraints and objective functions in terms of these control points minimize g(x) subject to f(x) = c g(x) = ½ xMx (a weighted sum of squares,
diagonal matrix M gives weights) In addition to SQP could use a non-linear
least squares solver to minimize constraint residuals (distance from meeting constraints)
The examples
120 Hz mocap data downsampled to 30 Hz (one example from text was rotoscoped)
Maker positions converted to articulated character parameters
Euler angles for joints (3 DOF) Except elbow, knee, ankles
Frequently, data for hand and feet is unavailable
Getting 60% shorter
Gray character results in foot skate Left shows strides getting longer (black) Right shows adjusting foot plants (black)
2-D Walking
82 frames / 15 fps 14 DOFs (2 pos + 12 ang) 146 constraints on heel/toe 328 inequality constraints
to keep feet above ground 1968 joint limit constraints
3-D Walking
34 DOF, 112 Frames 4193 scalar constraints (~354 active at
any one time… computation in seconds
Ladder
Their custom SQP, LMULT permits some constraints to be violated by ½ in.
Least squares optimization results in errors less than ¼ in.
Dancing
Woman mightlift off ground
Male permittedto move upperbody
1200 constraints 14 seconds to
solve w/ 1/8 in.errors
Extensions
Characters with different structure First use standard retargeting to a
character whose dimensions roughly match those of the new character
Find correspondences (manually) between features of original and new character, make these constraints
Use retargeting with spacetime constraints again, to find a new motion that satisfies these constraints
Conclusions
Basic retargeting works for characters with same structure, different limb lengths
Control point spacing is important for preserving desired frequencies Enforcing uniformity across time is a limitation
Enforcing certain features of original motion may not produce natural retargeted motion
Morphing simulated characters
Hodgins and Pollard Adapting Simulated Behaviors For New
Characters Siggraph 97
Morphing simulated characters
We know how character has changed in size and mass
We have no control of joint positions explicitly
We adapt character through control algorithms Timing parameters Muscle stiffnesses and damping
Morphing simulated characters
We know a lot about scaling
Animation of Dynamic Legged Locomotion – SIGGRAPH 91, Raibert and Hodgins Scale kangaroo by L
It will have to jump L times as high Because gravity is unchanged
more time will pass before landing (for L >1) Have cadence 1/sqrt(L) of original But because limbs are longer (L>1)
travel sqrt (L) as fast
We know a lot about scaling
Motion Graphs
Introduction to the paper by Kovar, Gleicher, and Pighin
Motion can be “sampled” just as videoclips were inVideoTextures
Sampling mocap
Transitioningfrom frame i toframe j
White are goodtransitions
Green arelocal min
One change vis. Video Textures
Multiple timesteps are grouped into a clip
The clips are connected to one another in a graph
Transitioning from one clip to another is not as easy as cross-fade or morph from image domain
General outline
Build motion graph Remove dead ends (find strongly
connected components) Search for paths that produce desired
motion (branch and bound search) Blend clips to form a smooth motion