Animation CS 551 / 651

AnimationCS 551 / 651

Dynamics Modeling and CullingDynamics Modeling and Culling

Chenney, Ichnowski, and ForsythChenney, Ichnowski, and Forsyth

Dynamics Modeling and CullingDynamics Modeling and Culling

Chenney, Ichnowski, and ForsythChenney, Ichnowski, and Forsyth

The world is full of moving things

Cars, people, clouds, leaves on a treeCars, people, clouds, leaves on a treeLasseter believes everything must be moving to look “right”Lasseter believes everything must be moving to look “right”

DynamicsDynamics

• The equations that define how they moveThe equations that define how they move

SimulationSimulation

• The process of computing the dynamicsThe process of computing the dynamics

Cars, people, clouds, leaves on a treeCars, people, clouds, leaves on a treeLasseter believes everything must be moving to look “right”Lasseter believes everything must be moving to look “right”

DynamicsDynamics

• The equations that define how they moveThe equations that define how they move

SimulationSimulation

• The process of computing the dynamicsThe process of computing the dynamics

Simulation makes the world go ‘round

Simulation is expensiveSimulation is expensive

• Small timesteps for dynamics computationsSmall timesteps for dynamics computations

• Lots of moving limbsLots of moving limbs

• Flexible objects like hair/clothFlexible objects like hair/cloth

• Collision checks (nCollision checks (n22))

Simulation is expensiveSimulation is expensive

• Small timesteps for dynamics computationsSmall timesteps for dynamics computations

• Lots of moving limbsLots of moving limbs

• Flexible objects like hair/clothFlexible objects like hair/cloth

• Collision checks (nCollision checks (n22))

Reduce costs of simulation

Perception permits simplificationPerception permits simplification• What simulation fidelity is needed?What simulation fidelity is needed?

– Out-of-viewOut-of-view

No need to render correct movementsNo need to render correct movements

What happens when object returns to view?What happens when object returns to view?

– Distant or in peripheryDistant or in periphery

Some part of simulation must be accurateSome part of simulation must be accurate

Other parts can be approximatedOther parts can be approximated

Perception permits simplificationPerception permits simplification• What simulation fidelity is needed?What simulation fidelity is needed?

– Out-of-viewOut-of-view

No need to render correct movementsNo need to render correct movements

What happens when object returns to view?What happens when object returns to view?

– Distant or in peripheryDistant or in periphery

Some part of simulation must be accurateSome part of simulation must be accurate

Other parts can be approximatedOther parts can be approximated

Building simpilfications

How is simplification constructed?How is simplification constructed?

• Cull DOFsCull DOFs

• Reduce temporal resolutionReduce temporal resolution

• Permit more collisionsPermit more collisions

Current technology: simplify by handCurrent technology: simplify by hand

How is simplification constructed?How is simplification constructed?

• Cull DOFsCull DOFs

• Reduce temporal resolutionReduce temporal resolution

• Permit more collisionsPermit more collisions

Current technology: simplify by handCurrent technology: simplify by hand

Preserving accuracy

Graceful degradation Graceful degradation

• Suspension of disbeliefSuspension of disbelief

– If simplified thing looks unrealistic, belief in “virtual” If simplified thing looks unrealistic, belief in “virtual” world may be jeopardizedworld may be jeopardized

• Accuracy of outcomeAccuracy of outcome

– If simplified thing behaves differently, outcome of If simplified thing behaves differently, outcome of game or training application may be wronggame or training application may be wrong

Graceful degradation Graceful degradation

• Suspension of disbeliefSuspension of disbelief

– If simplified thing looks unrealistic, belief in “virtual” If simplified thing looks unrealistic, belief in “virtual” world may be jeopardizedworld may be jeopardized

• Accuracy of outcomeAccuracy of outcome

– If simplified thing behaves differently, outcome of If simplified thing behaves differently, outcome of game or training application may be wronggame or training application may be wrong

Related work

Geometric level of detail (LOD)Geometric level of detail (LOD)• Cost of rendering geometry must be justifiedCost of rendering geometry must be justified

– Perceptually based perception metricsPerceptually based perception metrics

– Geometric simplification algorithmsGeometric simplification algorithms

– Visibility cullingVisibility culling

• Do these translateDo these translateto simulation?to simulation?

Geometric level of detail (LOD)Geometric level of detail (LOD)• Cost of rendering geometry must be justifiedCost of rendering geometry must be justified

– Perceptually based perception metricsPerceptually based perception metrics

– Geometric simplification algorithmsGeometric simplification algorithms

– Visibility cullingVisibility culling

• Do these translateDo these translateto simulation?to simulation?

Funkhouser and Sequin, 1993

In a perfect world

For each frameFor each frame

• Compute effect on realism vs. all simplificationsCompute effect on realism vs. all simplifications

• Set “reality” dial on each object to suit its importanceSet “reality” dial on each object to suit its importance

For each frameFor each frame

• Compute effect on realism vs. all simplificationsCompute effect on realism vs. all simplifications

• Set “reality” dial on each object to suit its importanceSet “reality” dial on each object to suit its importance

Simplifying periodic systems

What does periodicity buy us?What does periodicity buy us?

• Object’s description is a function of where it is relative Object’s description is a function of where it is relative to one “cycle”to one “cycle”

– Find “t”, where it is in the cycleFind “t”, where it is in the cycle

– Build f(t), a function mapping t Build f(t), a function mapping t system state system state

Predicting where the blue-line bus is vs. Predicting where the blue-line bus is vs. predicting where Osama ispredicting where Osama is

What does periodicity buy us?What does periodicity buy us?

• Object’s description is a function of where it is relative Object’s description is a function of where it is relative to one “cycle”to one “cycle”

– Find “t”, where it is in the cycleFind “t”, where it is in the cycle

– Build f(t), a function mapping t Build f(t), a function mapping t system state system state

Predicting where the blue-line bus is vs. Predicting where the blue-line bus is vs. predicting where Osama ispredicting where Osama is

Roller coaster

Where is the car and what is its orientation?Where is the car and what is its orientation?Where is the car and what is its orientation?Where is the car and what is its orientation?

Roller coaster

Build mapping, f(t)Build mapping, f(t)

• Observe position/orientation ofObserve position/orientation ofcar during one cyclecar during one cycle

– How long is a cycle?How long is a cycle?

• Train neural network to correctly predict mappingTrain neural network to correctly predict mapping

– f(t) = x, y, z, roll, pitch, yawf(t) = x, y, z, roll, pitch, yaw

– Neural net is just a function approximator, so it can Neural net is just a function approximator, so it can do this!do this!

Build mapping, f(t)Build mapping, f(t)

• Observe position/orientation ofObserve position/orientation ofcar during one cyclecar during one cycle

– How long is a cycle?How long is a cycle?

• Train neural network to correctly predict mappingTrain neural network to correctly predict mapping

– f(t) = x, y, z, roll, pitch, yawf(t) = x, y, z, roll, pitch, yaw

– Neural net is just a function approximator, so it can Neural net is just a function approximator, so it can do this!do this!

Roller coaster

Using the simplified modelUsing the simplified model

• Replace true dynamics withReplace true dynamics withneural networkneural network

– Just keep track of t and incrementJust keep track of t and increment

A lot like motion captureA lot like motion capture

Using the simplified modelUsing the simplified model

• Replace true dynamics withReplace true dynamics withneural networkneural network

– Just keep track of t and incrementJust keep track of t and increment

A lot like motion captureA lot like motion capture

Roller coaster

Are there shortcomings Are there shortcomings with using motion capture?with using motion capture?

• Not responsive to changesNot responsive to changesin environmentin environment

• Not alterableNot alterable

Does it matter?Does it matter?

• Use this simplification when responsiveness and Use this simplification when responsiveness and flexibility are not requiredflexibility are not required

Are there shortcomings Are there shortcomings with using motion capture?with using motion capture?

• Not responsive to changesNot responsive to changesin environmentin environment

• Not alterableNot alterable

Does it matter?Does it matter?

• Use this simplification when responsiveness and Use this simplification when responsiveness and flexibility are not requiredflexibility are not required

Simplifying non-periodic systems

What does non-periodicity buy us?What does non-periodicity buy us?• People aren’t good at predicting future statesPeople aren’t good at predicting future states

– There is room for error/noise/approximationThere is room for error/noise/approximation

• People get worse at predicting as time elapsesPeople get worse at predicting as time elapses

– Short lapses are predicted using extrapolationShort lapses are predicted using extrapolation

– Longer lapses are predicted using generalizationLonger lapses are predicted using generalization

– Really long lapses lack preconceptionsReally long lapses lack preconceptions

Examples of these?Examples of these?

What does non-periodicity buy us?What does non-periodicity buy us?• People aren’t good at predicting future statesPeople aren’t good at predicting future states

– There is room for error/noise/approximationThere is room for error/noise/approximation

• People get worse at predicting as time elapsesPeople get worse at predicting as time elapses

– Short lapses are predicted using extrapolationShort lapses are predicted using extrapolation

– Longer lapses are predicted using generalizationLonger lapses are predicted using generalization

– Really long lapses lack preconceptionsReally long lapses lack preconceptions

Examples of these?Examples of these?

Tilt-a-whirl

Where are all the cars?Where are all the cars?

• A chaotic system where physics mattersA chaotic system where physics matters

Where are all the cars?Where are all the cars?

• A chaotic system where physics mattersA chaotic system where physics matters

Tilt-a-whirl

Short time lapsesShort time lapses

• Use previous state as a basis forUse previous state as a basis forprediction of future statesprediction of future states

– Extrapolation of accelerations Extrapolation of accelerations and velocitiesand velocities

Short time lapsesShort time lapses

• Use previous state as a basis forUse previous state as a basis forprediction of future statesprediction of future states

– Extrapolation of accelerations Extrapolation of accelerations and velocitiesand velocities

Tilt-a-whirl

Medium time lapsesMedium time lapses

• Use previous state as a basis forUse previous state as a basis forprediction of future statesprediction of future states

• Extrapolation only works forExtrapolation only works forsmall small tt

• Use neural network to model change in state afterUse neural network to model change in state aftert seconds have passedt seconds have passed

– f (statef (statett) = state) = statet+t+tt

Medium time lapsesMedium time lapses

• Use previous state as a basis forUse previous state as a basis forprediction of future statesprediction of future states

• Extrapolation only works forExtrapolation only works forsmall small tt

• Use neural network to model change in state afterUse neural network to model change in state aftert seconds have passedt seconds have passed

– f (statef (statett) = state) = statet+t+tt

Tilt-a-whirl

Medium time lapsesMedium time lapses

• Training a neural networkTraining a neural network

– Sample system at time tSample system at time t

– Sample system at time t + Sample system at time t + tt

– Network has one input for each DOFNetwork has one input for each DOF

– Network has one output for each DOFNetwork has one output for each DOF

– Train network to predict state after t + Train network to predict state after t + tt

Medium time lapsesMedium time lapses

• Training a neural networkTraining a neural network

– Sample system at time tSample system at time t

– Sample system at time t + Sample system at time t + tt

– Network has one input for each DOFNetwork has one input for each DOF

– Network has one output for each DOFNetwork has one output for each DOF

– Train network to predict state after t + Train network to predict state after t + tt

Tilt-a-whirl

Medium time lapsesMedium time lapses

• A particular neural network onlyA particular neural network onlypredicts state after predicts state after t secondst seconds

• What if object pops back into What if object pops back into view after ½ view after ½ t seconds?t seconds?

– Build a second neural network for ½ Build a second neural network for ½ tt

– Build a third neural network for ¼ Build a third neural network for ¼ tt

– ……

Medium time lapsesMedium time lapses

• A particular neural network onlyA particular neural network onlypredicts state after predicts state after t secondst seconds

• What if object pops back into What if object pops back into view after ½ view after ½ t seconds?t seconds?

– Build a second neural network for ½ Build a second neural network for ½ tt

– Build a third neural network for ¼ Build a third neural network for ¼ tt

– ……

Tilt-a-whirl

Medium time lapsesMedium time lapses• Any point in time is approximatedAny point in time is approximated

by series of neural networksby series of neural networks

• Ex: Approximate 3.75 secondsEx: Approximate 3.75 seconds

– Let NNs exist for Let NNs exist for t = .25, .5, and 1.0t = .25, .5, and 1.0

– statestate11 = NN = NN1.01.0 (state (state00))

– statestate22 = NN = NN1.01.0 (state (state11))

– statestate33 = NN = NN1.01.0 (state (state22))

– statestate3.53.5 = NN = NN0.50.5 (state (state33))

– statestate3.753.75 = NN = NN0.250.25 (state (state3.53.5))

Medium time lapsesMedium time lapses• Any point in time is approximatedAny point in time is approximated

by series of neural networksby series of neural networks

• Ex: Approximate 3.75 secondsEx: Approximate 3.75 seconds

– Let NNs exist for Let NNs exist for t = .25, .5, and 1.0t = .25, .5, and 1.0

– statestate11 = NN = NN1.01.0 (state (state00))

– statestate22 = NN = NN1.01.0 (state (state11))

– statestate33 = NN = NN1.01.0 (state (state22))

– statestate3.53.5 = NN = NN0.50.5 (state (state33))

– statestate3.753.75 = NN = NN0.250.25 (state (state3.53.5))

Medium time lapses

True Dynamics

Neural NetApproximation

Position after t Velocity after t

Medium time lapses

Difference image masked by Difference image masked by stationary distribution imagestationary distribution imageDifference image masked by Difference image masked by stationary distribution imagestationary distribution image

Results

Neural Network PredictionNeural Network PredictionNeural Network PredictionNeural Network Prediction

Tilt-a-whirl

Long time lapsesLong time lapses

• Previous state is not a startingPrevious state is not a startingpoint for prediction… stochasticpoint for prediction… stochastic

• What does the traffic on I-29 look What does the traffic on I-29 look like at 5:00 this afternoon?like at 5:00 this afternoon?

– I have a basic model, but no bias to previous statesI have a basic model, but no bias to previous states

Obviously if an accident happened at 4:00, my Obviously if an accident happened at 4:00, my prediction would be wrongprediction would be wrong

Long time lapsesLong time lapses

• Previous state is not a startingPrevious state is not a startingpoint for prediction… stochasticpoint for prediction… stochastic

• What does the traffic on I-29 look What does the traffic on I-29 look like at 5:00 this afternoon?like at 5:00 this afternoon?

– I have a basic model, but no bias to previous statesI have a basic model, but no bias to previous states

Obviously if an accident happened at 4:00, my Obviously if an accident happened at 4:00, my prediction would be wrongprediction would be wrong

Tilt-a-whirl

Long time lapsesLong time lapses

• How do I build a basic model?How do I build a basic model?

– Based on observationsBased on observations

– I am more likely to expect system states that I am more likely to expect system states that occurred frequently in my observationsoccurred frequently in my observations

– Some system states will be implausible because of Some system states will be implausible because of limits on feasibility that I determinelimits on feasibility that I determine

Long time lapsesLong time lapses

• How do I build a basic model?How do I build a basic model?

– Based on observationsBased on observations

– I am more likely to expect system states that I am more likely to expect system states that occurred frequently in my observationsoccurred frequently in my observations

– Some system states will be implausible because of Some system states will be implausible because of limits on feasibility that I determinelimits on feasibility that I determine

Tilt-a-whirl

Long time lapsesLong time lapses• How do I build a basic model?How do I build a basic model?

– State of world is defined by DOFsState of world is defined by DOFs

– DOFs define n-dimensional spaceDOFs define n-dimensional space

– Reduce the space to a finite volumeReduce the space to a finite volume

² Limits on feasibilityLimits on feasibility

² What are min/max for each DOF?What are min/max for each DOF?

Example: state space of two-joint armExample: state space of two-joint arm

Long time lapsesLong time lapses• How do I build a basic model?How do I build a basic model?

– State of world is defined by DOFsState of world is defined by DOFs

– DOFs define n-dimensional spaceDOFs define n-dimensional space

– Reduce the space to a finite volumeReduce the space to a finite volume

² Limits on feasibilityLimits on feasibility

² What are min/max for each DOF?What are min/max for each DOF?

Example: state space of two-joint armExample: state space of two-joint arm

Tilt-a-whirl

Long time lapsesLong time lapses

• Discretize state-space volume intoDiscretize state-space volume intocellscells

• Run the simulation for a whileRun the simulation for a while

– At each timestep, record which cell system is inAt each timestep, record which cell system is in

– Accumulate counters in each cellAccumulate counters in each cell

• Each cell is a assigned a value corresponding to the Each cell is a assigned a value corresponding to the probability system is in that stateprobability system is in that state

Long time lapsesLong time lapses

• Discretize state-space volume intoDiscretize state-space volume intocellscells

• Run the simulation for a whileRun the simulation for a while

– At each timestep, record which cell system is inAt each timestep, record which cell system is in

– Accumulate counters in each cellAccumulate counters in each cell

• Each cell is a assigned a value corresponding to the Each cell is a assigned a value corresponding to the probability system is in that stateprobability system is in that state

Tilt-a-whirl

Which model do we use?

Extrapolation vs. NN vs. StochasticExtrapolation vs. NN vs. Stochastic

• NN is accurate for designated NN is accurate for designated tt

– Start using NNs at smallest Start using NNs at smallest tt

• At some point, knowing exact state at time t doesn’t At some point, knowing exact state at time t doesn’t helphelp

– As time passes, state of system begins to match As time passes, state of system begins to match the basic prediction of stationary distributionthe basic prediction of stationary distribution

Extrapolation vs. NN vs. StochasticExtrapolation vs. NN vs. Stochastic

• NN is accurate for designated NN is accurate for designated tt

– Start using NNs at smallest Start using NNs at smallest tt

• At some point, knowing exact state at time t doesn’t At some point, knowing exact state at time t doesn’t helphelp

– As time passes, state of system begins to match As time passes, state of system begins to match the basic prediction of stationary distributionthe basic prediction of stationary distribution

Tilt-a-whirl

Medium time lapses

Difference image masked by Difference image masked by stationary distribution imagestationary distribution imageDifference image masked by Difference image masked by stationary distribution imagestationary distribution image

Results