Page 1
Rishabh BattulwarResearch and Development at Digital Domain
Data-Driven Future in Visual Effects
Page 4
VFx pipelines
Fluids
FireSmoke Rigid Body
Hair
Page 6
What’s important?
Facial Animation
Natural Facial Shapes Realistic Appearance
Page 7
Facial Motion
Highly Non-Linear
Organic
Person Specific
Page 8
Facial Animation Goal
Page 9
Traditional Model
Ekman and Friesen (1978)
Facial Action Coding System (FACS)
50-60 Primary Shapes
Empirical Interpretations
GENERIC Facial Shape Basis*
Codify Facial Expressions
Inspired by Facial Musculature
Page 10
FACS Model
FACS shapesAnimation Rig Setup
50-60 Primary Shapes
More than 1000Combination Shapes
Page 11
FACS Model
50-60 Primary Shapes
More than 1000Combination Shapes
Primary Shape Primary Shape Hand-CORRECTED Combo Shape
An example of combination shape in FACS model
[A] [B] [A+B]*
Page 12
FACS Model
FACS shapesAnimation Rig Setup
50-60 Primary Shapes
More than 1000Combination Shapes
Page 13
FACS Model
FACS based process
Page 14
Geometric Approach
Low-Resolution Capture
Page 15
Geometric Approach
MASQUERADE
Geometric Mesh Deformer Output
Page 16
Geometric Approach
Correspondence Mapping
Performance Transfer using Differential Geometry
Page 17
Geometric Approach
MASQUERADE
Geometric Mesh Deformer Output
Quick Result !
Page 18
Geometric Approach
MASQUERADE
Geometric Mesh Deformer Output
NO ! Medium-scale &
Hi-resolution detail !
NOT !True to the Person
Quick Result !
Page 19
NO ! Medium-scale &
Hi-resolution detail !
NOT !True to the Person
Page 20
NO ! Medium-scale &
Hi-resolution detail !
NOT !True to the Person
Page 21
NO ! Medium-scale &
Hi-resolution detail !
NOT !True to the Person
Page 22
Data Preparation
Raw Data Capture Rigs
Page 23
Data Preparation
High-Resolution Captures
Capturing Facial Range of Motion
Offline Processing
Page 24
Data Preparation
More High-Resolution Captures
Capturing Mesoscopic Detail
Texture Maps
Page 25
MASQUERADE
Data-driven Shape ModelTaking sparse facial capture to high-resolution data
Moser et. al. ‘17 - Masquerade
Page 26
Medium-scale &Hi-frequency detail ! True to the Person !
Page 27
Medium-scale &Hi-frequency detail ! True to the Person !
Page 28
Medium-scale &Hi-frequency detail ! True to the Person !
Page 29
Issues in Geometric Approach
Specific features (lip shapes)Collapsing Geometry
Page 30
Synthetic Data-driven Corrections on CG Creatures
Sculpted Corrections for CG Creatures
No FACS-based BlendShape-Rig
Page 31
Data-driven Corrections on CG Creatures
Pre-Correction Post-CorrectionSource Hendler et. al. ‘17 - Direct Drive
Page 32
More examples - Wrinkle Map Regression
Page 33
Comparison
Takes Several Months
Shape-space modeled iteratively
Takes 1-2 Weeks
Shape-space built from physical data
Page 34
Final Result
Markeredface input Large-Scale
Geomteric Deformation using low-resolution capture
Training Data!!
Final High-Resolution Output Mesh
Overview of Data-Driven Facial Animation
Page 36
Acknowledgements
Thanks to~
David Corral, David Mclean, Rickey Cloudsdale, Lucio Moser, Geoff Wedig, Mark Williams, Ron Griswold, Nafees Bin Zafar, Doug Roble,
Jeremy Buttell, Derek Crosby, Ron Miller, Darren Hendlerand the entire Digital Domain Team!
~