Motion Synthesis for Articulated Human Bodies. Contents.

Motion Synthesis for Articulated Human Bodies

Contents

• Human motion synthesis: Zoo

• Articulated body mechanicsArticulated body mechanics

• Optimization based motion synthesisOptimization based motion synthesis

• Feedback based balance controlFeedback based balance control

Human motion components

• Bone (skeleton): ~206 (motion invariant)• Joint (articulation): limitation• Tendon and ligament: elasticity / plasticity• Muscle: ~639, works in group, limitation

– uni-articulate and bi-articulate (passive inefficiency)

– parallel or perpendicular shape

• Neuron-system: reflection speedNeuron Muscle Tedon Bone

Human motion model (I)

• Pure articulated model– Simplified skeleton (hand / foot / tibia-fibula)– Directly control force and torque on joints– Convenient for high-level control

Human motion model (II)

• Deformable model– Skin mesh motored by skeleton or captured

skin animation– Muscle shape representation

Human motion model (III)

• Tendons and muscles model– Musculoskeletal models and simulation– Muscle-tendon model (strand)

Hand can have similar DoF with coarse whole body!

Human motion model (IV)

• Visible Human data set (realistic musculature and flesh) [Teran05]

Methods for motion synthesis (I)

• Data-driven based synthesis– Captured motion skeleton data– Captured deformable data (face / skin)– Motion dynamics and naturalness is trivial– Limited response to environment

• Blending, Editing, Style transfer; no guarantee for physics correctness

– Captured data is sparse• [pose, time] space is huge

– Difficult to obtain data• High dynamic motion• Outdoor environment

Methods for motion synthesis (II)

• Pure generative-based synthesis– Physics constraint can be guaranteed – Easy to interact with dynamic environment– Hard to be natural

• Naturalness is a subset of physics correctness

– Dynamics simulator complexity• E.g. stable / dynamic frictions

– Computational complexity and stableness• High gain means high stiffness

– Controller design • Mutual controller is difficult to design and hard to generalize

Methods for motion synthesis (III)

• Pure motion planning– Can obtain solution in constrained

environment– High-level path planning– Solution has no continuous or physics

guarantee – Still time consuming

• High dimension of human skeleton

Combined methods

• Mocap data + physics: better controller– Controller for hand [Nancy05]

– Controller for whole body [Yin08]

• Mocap data + motion planning: high level motion: – Manipulating [Katsu04]

– Tangling [Edmond08]

• Dynamics + motion planning (for articulated model) [Russell07]

Other related

• Robotics [Kris06]

– Primitive based motion planning use transition motion to balance natural motion and environment constraints.

• Biomechanics– Provides many principles

for motion control: balance, Locomotion, neuroscience. [Alexandrov05] [Christine07]

Contents

• Human motion synthesis: ZooHuman motion synthesis: Zoo

• Articulated body mechanics

• Optimization based motion synthesisOptimization based motion synthesis

• Feedback based balance controlFeedback based balance control

Articulated body dynamics

• Ball-and-socket: 3 DoF• Saddle: 2 DoF• Hinge: 1 Dof

• Each joint i is 1 DoF• Ball-and-socket and saddle can

be represented by some 1 DoF

A special case of multi-body dynamics

Yin’s work gives

an example

Joint makes impulse

and penalty difficult

Parameters for articulated body

• State of root (position, rotation, linear / angular velocity)

• Configuration of joints (q, , )

• Inertia matrix (for articulated body, inertia matrix is an equivalent inertia matrix: give a test force, get an acceleration inertia)

• Lecture in COMP 790-058

Inverse dynamics

• Known: q, ,

• Unknown: f, τ• Simple: recursive [Featherstone]

– Compute v and a; compute net force on a link, similar to f = ma (downwards)

– Compute force and torque on a joint (upwards)

– Root’s p, v, a – End effector’s force and torque

Forward dynamics

• Known: q, , f, τ

• Unknown:

• More difficult– 3 loops

[Evangelos04]

Limitation of basic algorithms

• Many things are simplified in algorithms– Joint limitation– Unilateral constraint (ground constraint)– Friction (kinetic, static, rolling and spinning)– Non-interpenetrate constraint– Collision response

• Modeling contact and constraint is a well-studied problem for rigid body simulation. – See David Baraff’s papers and note.

Linear complementarity problem (LCP) [Andreas]

• One of the standard methods to handle contact and constraints.

Solution to LCP: pivot method

• Basic idea: if q is positive or zero, solution is trivial: (w = q, z = 0)

• Pivot q and w to make it true

• where

Solution to LCP: iterative method

• LCP can be represented by QP, so can be solved by iterative methods, like Gauss-Seidel, Newton

Pros and Cons

• Pivot method– Convergence is guaranteed after limited steps– suffer from numerical problem, especially for

large-scale and/or ill-conditioned problems

• Iterative method– easier to implement and– numerically robust– convergence is proven only for a limited class

of M matrix

LCP model for constraints

• Unilateral constraint (force exists only when contacting)

• Joint Limit

• LCP guarantees zero virtual work for contact forces

• w = Mz + q?

Extended LCP model for frictions

• Coulomb cone

• Dynamic friction Static friction

• Solved by extending iterative LCP solver

Other possible solutions

• Recent there are many new algorithms other than LCP to deal with unilateral constraints and friction– Nonconvex Rigid Bodies with Stacking [Eran03]

• Extension to LCP– Staggered Projections for Frictional Contact in

Multibody Systems [Danny08]

– Velocity based shock propagation[Kenny09]

– Implicit Contact Handling for Deformable Objects [Miguel09]

Impact constraints

• Non-interpenetrate constraint

• Collision reaction

• Impulse or force modeling

• See David Baraff’s papers and note.

Simulation loops

LCP solver

forward dynamics

Ode solver

Collision detector

Contact response

Contents

• Human motion synthesis: ZooHuman motion synthesis: Zoo

• Articulated body mechanicsArticulated body mechanics

• Optimization based motion synthesis

• Feedback based balance controlFeedback based balance control

Optimization based motion synthesis [Sumit09]

System overview

Lagrange mechanics

• Lagrange mechanics for rigid body

• Lagrange mechanics for articulated body

Lagrange function

Generalized force

Gravity, contact force, external force

Complete force modeling [Liu05]

Actuated and non actuated joint

• Root joint: non (passive) actuated– It’s configuration (position and rotation) is

actuated by joint constraints.

• Other joints: (active) actuated– It’s configuration is actuated by muscle

energy

Muscle force

Passive controller

• Muscle controller– Limited torque / force

– Limited torque / force change rate

• Contact controller– Static friction– Dynamic friction– Non-penetration

Optimization with passive controller

• Why (minimize q variation)?– Avoid trivial solution, like sliding or break-off

• Zero virtual work guarantee– Roll back

User controller specification

• Balance controller

• Climb controller

• Swing controller

• Multiple controller composition

• controller example

Controller protocol

• Finite State Machine

• Each state has its own constraints (in objective form)

• State transition happens when feasible solution can not find or contact breaks.

Balance controller

balance

takeStep relaxFoot

p: target position

qv: slow

spline: perpendicular to ground

cp: CoM balance

cf: support feet position

c: friction

Climb controller

allSupport

moveHand

relaxHand

moveFoot

relaxFoot

com: change CoM

Swing controller

passiveSwingtrySwing

Visual sensor

• Reachable objects search

Contents

• Human motion synthesis: ZooHuman motion synthesis: Zoo

• Articulated body mechanicsArticulated body mechanics

• Optimization based motion synthesisOptimization based motion synthesis

• Feedback based balance control

Human motion control model

Feedback based balance control

• Balance control under small perturbation [Yin03]

• Balance control under large perturbation [Yin08]

• Similar strategy: feed-back + feed-forward, large perturbation step strategy

System description

• The dynamics system does not use generalized coordinate

• Instead, use full-coordinate constrained form for dynamics

• Traditional style (based on Barraf 1996 paper)

• Control is a hybrid of generalized-coordinate form and full-coordinate form

Full-coordinate constrained matrix form (I)

• Constraint i between body a and b

• Constraint system (each item in J is 3*3 matrix)

Here f is dim-12 vector, every 3

sub-vector is force for body (a, b, c, or d), J is 9*12

Full-coordinate constrained matrix form (II)

• For torque, similar

• row number of H is number of freedom, its column number is 3 * object number.

• Each term is joint i’s torque on all objects

Balance control under small perturbation

• Preprocess: use inverse dynamics to compute force for original motion

• In each step of dynamic simulation– Forward dynamics: current state– Feedback control

– Feedforward control

– Net control

Simulation result

• Perturbation by a ball

Balance control under large perturbation

• Two differences:– Motion is modeled by finite state machine– Under large perturbation, a pre-computed

feed-forward input is not suitable (using adaptive control instead)

Motion FSM

• Difference with FSM in [Sumit09]

– For control torque heuristics– More robust (precision, static)

Feed-forward: Feed-back error learning

• Cyclic motion (function of phase)

• Learn inverse model dynamically (system identification)

Feed-back control (I)

• PD control

• Contact control (support leg)

Low gain

Follow mocap data: high gain

Feed-back control (II)

• COM feedback

+v

+v

+

d<0

v=0+

d>0

v=0

COM velocity

COM position

v>0 or d > 0 forward step quickly

Feed-back control (III)

• CoM feedback

Basic controller(default target)

Continuous feedback

General form for multiple joints (matrix form)

Control summary

• 2D to 3D:– sagittal and coronal planes

Feedback and feedforward torques

Simulation result

• demo• Overview• Downhill• Drunk• Limp• Spin• Boxes• Different friction

Reference• [Edmond08] Planning tangling motions for humanoids• [Nancy05] Physically Based Grasping Control from Example• [Yin08] SIMBICON: Simple Biped Locomotion Control• [Katsu04] Synthesizing Animations of Human Manipulation Tasks• [Russell07] Efficient Motion Planning of Highly Articulated Chains using Physics-

based sampling• [Kris06] Using motion primitives in probabilistic sample-based planning for humanoid

robots• [Teran05] Creating and Simulating Skeletal Muscle from the Visible Human Data Set• [Alexandrov05] Feedback equilibrium control during human standing• [Christine07] Bipedal locomotion: toward unified concepts in robotics and

neuroscience• [Evangelos04] Practical Physics for Articulated Characters• [Shin’schiro07] Constraint-based dynamics simulator for humanoid robots with shock

absorbing mechanics• [Featherstone] Robot Dynamics Algorithms• [Andreas] Practical Optimization• [Eran03] Nonconvex Rigid Bodies with Stacking• [Danny08] Staggered Projections for Frictional Contact in Multibody Systems

• [Sumit09] Optimization-based interactive motion synthesis• [Liu05] Towards a generative model of natural motion• [Baraff89] Analytical methods for dynamics simulation of non-penetrating rigid bodies• [Yin03] Motion Perturbation Based on Simple Neuromotor Control Models• [Kenny09] Velocity-based shock progagation for multibody dynamics animation• [Miguel09] Implicit Contact Handling for Deformable Objects