Motion Capture Animation & Procedural Animation Dr. Midori Kitagawa Arts and Technology Program University of Texas at Dallas Gif animations by Dax Norman.

Motion Capture Animation & Procedural Animation

Dr. Midori KitagawaArts and Technology ProgramUniversity of Texas at Dallas

Gif animations by Dax Norman

Three types of 3D computer animation• Keyframe animation• Motion capture animation• Procedural animation

Keyframe Animation

Keyframe animation

• Keyframe is a drawing of a key moment in an animated sequence, where the motion is at its extreme.

• Inbetweens fill the gaps between keyframes.• Every motion is created by animators.

Keyframe animation• To produce 3D keyframe animation, the animator builds the

behavior of a 3D model by keying parameter values in key frames where the values are at their extremes.

Keyframe animation• 3D keyframe animation method is based on traditional hand-

drawn 2D keyframe animation method. • Animation is manually produced from scratch. • Examples: Pixar animations

Motion capture animation

Motion capture animation• To produce a motion capture animation, the motion of a

capture subject is recorded as 3D data and applied to a 3D model.

• The animation is produced from the capture subject’s movement.

Motion capture animation• Although no 3D data is involved in rotoscoping, motion

capture technology is sometimes referred as “devil’s rotoscoping.”

• Examples: Avatar (2009), How to Train Your Dragon 2 (2014)

Procedural animation

Procedural animation• To produce a procedural animation, the animator defines a

procedure or a set of operations to be performed.• Each operation can generate or alter data that passes through

it and can be conditionally or non-conditionally executed.

Procedural animation• With procedural animation methods (e.g., particle systems,

rigid dynamics and flexible dynamics), the user specifies a set of rules, initial conditions and parameter values and runs simulations.

• Physics-based animation is a subset of procedural animation.

Procedural animation• Procedural methods can be used in any part of production

pipeline to produce geometries, textures, lights, rigs, animation, composites, sounds and other elements.

• Examples: visual effects (e.g., smokes, steam, fire, clouds, dust, water, and plants) in live action films and 2D/3D animations.

ATEC 4371 Topics in Animation: Procedural Methods for Computer AnimationFall 2015 Semester•Prerequisite: ATEC 2326 Computer Animation Process•Textbook: The Art of 3D Computer Animation and Effects, 4th Edition by Isaac Kerlow•Software: Houdini (Side Effects Inc. ) which is node-based.

More about motion capture animatoin…

What is Mocap?• Motion capture (mocap) is sampling and recording

motion of humans, animals, and inanimate objects as 3D data.

• The data can be used to study motion or to give an illusion of life to 3D computer models.

Applications• Entertainment• Medicine / Sports • Arts / Education• Science / Engineering

Entertainment: Live Action Films

Entertainment: 3D computer animations

Entertainment: Video Games

Medicine / Sports

• Medicine (e.g., gait analysis, rehabilitation)• Sports (e.g. injury prevention, performance

analyses, performance enhancement)

Arts / Education• Dance and theatrical performances • Archiving (e.g., Marcel Marceau)

OSU/ACCAD

Science / Engineering

• Engineering (e.g., biped robot developments)• Computer Science (e.g., human motion database,

indexing, recognitions)• Design (e.g., ergonomic product design)• Military (e.g., field exercises, virtual instructors, and role-

playing games)

Mocap animation• Motion capture animation is different from keyframe

animation in terms of how motion is created.• Same principles of animations apply to mocap animation &

keyframe animation!• Combination of motion capture animation, keyframe

animation and procedural animatoin is often used.

Advantages of mocap animation

• Faster to create (only if an established production pipeline exists.)

• Secondary motions and all the subtle motions are captured. -> more realism

• Physical interactions between performers and props can be captured.

-> more realism

Disadvantages of mocap animation• Cost.• Manipulating mocap data is often difficult -> Re-capturing or

key framing a shot with bad data is often easier.• Mapping mocap data of a performer to a 3D character with a

different proportion may cause issues.

Popular types of mocap equipment

• Magnetic systems• Mechanical systems• Optical systems

Magnetic systems

• Utilize sensors placed on the body to measure the magnetic field generated by a transmitter source.

Magnetic systems

• Require no special lighting condition. • Sensors are never occluded. • X Affected by electromagnetic force.

Mechanical systems

• Exoskeleton with angle sensors.

Mechanical systems• Measure joint angles (no marker ID problems). • Sensors are never occluded. • X Breakable!• X Configuration of sensors is fixed.• X Constrains on joints.

Optical systems• The cameras are equipped with infrared LED's and filters. • The cameras see reflector markers.

Optical systems• Higher sampling rate. • Larger capture space. • X Markers are sometimes occluded -> marker ID problems.• X Provide only positional data -> joint angles need to be

computed.

Mocap system at UTD• Vicon optical system • 8 high-speed MX 13 (up to 10,000 fps) and 8 high-resolution

MX 40 (4 megapixels) cameras.• Capture up to 5 performers at once.

Mocap animation courses

Graduate level: ATEC 6352 & UG level: ATEC 4345•Group project based•Group of 3 to 5 members•Semester long projectPre-requisite: •ATEC 3317 Modeling and texturing

OR•ATEC 3327 Lighting and compositionRecommended:•ATEC 3328 Rigging I•ATEC 4337 Computer Animation

Mocap animation pipeline Preproduction

Capture -> data cleaning -> retargetingModeling -> rigging ----------------------------- mocap animationKeyframe animationEffect animationSound designTexturing/lighting -> Rendering -> Composition

Post-production

* Tasks that everyone should be involved.* Tasks that can be designated to a member or members.

2D Image -> 2D Camera Data -> 3D Markers Positions -> Trajectories -> Recon Skeleton -> Solver Skeleton2D Image -> 2D Camera Data -> 3D Markers Positions -> Trajectories -> Recon Skeleton -> Solver Skeleton

• Capture Range of Motion (ROM)• Reconstruct trajectories of ROM• Label markers

VSKVSKSubject Calibration

Character Setup

• Correlate :

Character & Character (FBX)

• Edit motion

• Bake motion data to the skeleton

FBXFBXCharacter Setup

• Create a skeleton

• Bind skin to the skeleton

• Rig the character

• Marge the rigged character (MB) and the skeleton with motion data (FBX)

• Edit motion (IK/FK blend, Trax)

• Render

MBMB

FBXFBX

Skeleton with subject’s size & proportion

Skeleton with motion data & 3D character’s size & proportion

Triangulation

BLADE

MAYA

MOTIONBUILDER

Maya Scene File

System Calibration and Capturing and Processing Data

Rendering

The processes that you go through for each character

The processes that you repeat for each shot

VST (subject template)

Reconstruction

Mocap Pipeline Flow Chart

Circle fitting FBXFBXSkeleton with subject’s size & proportion and motion data

Skeleton with 3D character’s size & proportion (and skin)

UG Mocap & Procedural Animation course scheduleSummer 2015: ATEC 4345 Motion Capture Animation•Prerequisite: ATEC 3317 Modeling and texturing

OR•Prerequisite: ATEC 3327 Lighting and composition•Mondays 1- 5 pm

Fall 2015: ATEC 4371 Topics: Procedural Methods•Prerequisite: ATEC 2326 Computer Animation Process•Mondays 1 – 3:45 pm

Spring 2016: ATEC 4345 Motion Capture Animation

Dr. Midori [email protected] 1.909