Modeling and Animating Eye Blinks

Modeling and Animating Eye Blinks

Laura Trutoiu, Liz Carter, Iain Matthews, Jessica Hodgins

What is an eye blink?

QuickTime™ and aH.264 decompressor

are needed to see this picture.

Figure adapted from “Digital Character Animation”, Maestri,G . New Riders Publishing. 1996

Eye blinks: Common wisdom"... five frames is usually adequate for most situations, although four frames can make a character look more alert."

The Animator's Survival Kit by Richard Williams, Faber and Faber, NYC, 2001.

Inte

r-eyelid

dis

tance

Frame (24fps)

Inte

r-eyelid

dis

tance

Inte

r-eyelid

dis

tance

Figure adapted from “Digital Character Animation”, Maestri,G . New Riders Publishing. 1996

Eye blinks: Common wisdom"... five frames is usually adequate for most situations, although four frames can make a character look more alert."

The Animator's Survival Kit by Richard Williams, Faber and Faber, NYC, 2001.

Frame (24fps)

Inte

r-eyelid

dis

tance

Eye Blinks: What can we see?‣Temporal asymmetry:

‣Fast closing, slow opening

‣Closing amplitude:‣Eyes may not fully close

‣Variety:‣Varying duration & closing amplitude

‣Lower eyelid motionQuickTime™ and a

H.264 decompressorare needed to see this picture.

Frame (300 fps)

Inte

r-eyelid

d

ista

nce

(p

ixels

)

50 100 150 200 250 300

Eye blinks: Can you tell the difference?

QuickTime™ and a decompressor


QuickTime™ and a decompressor


Video 2:Video 1:

Eye blinks show temporal asymmetry and you can tell the difference.

forward backward

Eyes alive. LEE, S., BADLER, J., AND BADLER, N. ACM Trans. on Graphics 21, 3, 637–644. 2002.Automated eye motion using texture synthesis. DENG Z., LEWIS J. P., NEUMANN U. IEEE Computer Graphics and Applications 25, 2, 24– 30, 2005Eyelid kinematics for virtual characters. STEPTOE, W., OYEKOYA, O., AND STEED, A. Comput. Animat. Virtual Worlds 21 (May), 161–171. 2010

Related work: Eyes in graphicsEyes alive [Lee 2002]

eye gaze/saccades only

Eye motion with texture synthesis [Deng 2005]

no explicit eye blink modelrequires training

Eyelid kinematics [Steptoe 2010]parametric models for eyelid saccades and eye blinks using current position

10 participants

How do we create more natural eye-blink animations?

Track high speed video of eye blinks

Build a generative model that closely resembles the tracked data

Validate the model through perceptual experimentsWhy simple approximations are not good enough

Tracking eye blinksActor performance

Emotional vignettes and in-between pauses

High speed videoCasio Exilim F1 - 300 fps

Video trackingActive Appearance Models (AAMs) [Matthews 2004]

Inte

r-eyelid

dis

tance

(p

ixels

)

50 100 150 200 250 300

1000500 2500 3000

Inte

r-eyelid

dis

tance

(pix

els

)

Active appearance models revisited. Matthews, I. and Baker, S. International Journal of Computer Vision 60, 2, 135–164. 2004

Tracking eye blinks

Eye blink time series Normalized time series

Generated Data Re-timed data

Principal Component Analysis

Video = unobtrusive way to measure eye blink dynamicsHigh speed video => temporal resolution

Building a generative model

Perceptual experimentsQuickTime™ and a

Photo - JPEG decompressorare needed to see this picture.Goal:

Can naive users differentiate between different eye blink dynamics?

Stimuli:Animations at 30 fps

Role of appearance:photorealistic and cartoon character

Naturalness rating (1-7)Very natural clip (7) = “what you would expect to see in the real world”

Perceptual experiments


Exp 1: Model vs. real blinks

QuickTime™ and aPhoto - JPEG decompressor








1 2

3 4

Model, fully closed

Real, fully closed

Model, naturally closed

Real, naturally closed

Model vs. real + Closing amplitude


Natu

raln

ess

R

ati

ng

Real Model

n = 32, 2 (character) x 2 (profiles) x 2 (blur) x 2 (amplitude) repeated-measures ANOVA





Real

Model

300 fps

p < 0.001*


n = 32, 2 (character) x 2 (profiles) x 2 (blur) x 2 (amplitude) repeated-measures ANOVA





Modelnaturally closed

Modelfully closed

300 fps

Natu

raln

ess

R

ati

ng

Naturally closed

Fully closed


Photorealistic > Cartoon

Fully closed > Naturally closed




Exp 2: Model vs. Approximations

Model

Linear Symmetric

Linear Asymmetric

Ease In/Ease OutSymmetric

Ease In/Ease OutAsymmetric

Hypothesis: Asymmetric > Symmetric, Non-linear > Linear

Exp 2: Model vs. approximations

2 3

4 5

Model

Symmetric ease-in/ ease-out

Asymmetric linear

Symmetric linear









Asymmetric ease-in/ ease-out

1



Exp 2: Model vs. approximations

n = 43, 5 (profiles) x 4 (length) x 2 (character) repeated-measures ANOVA








Model > Approximations

Not symmetric: More time to open than to close Animation textbook - same number of frames for opening

and closing

Not linear: Fast closing (linear velocity) followed by slowly converging

opening Animation - linear or slowly accelerating, slowly decelerating

Eyes may not fully close: At least 50% of the time Animation - eyes always fully close In our experiments, fully closed animations preferred

?

Conclusions: Eye blinks are ...

Future work

Sequences of eye blinksinter-blink distancevariability may be useful

Synchronization with other facial motion

Blink sequence patterns



Exp 3: Lower eyelid motion









1 2

3 4

Vertical only

Vertical + Horizontal

Horizontal only

No Lower eyelid motion

Exp 3: Lower eyelid motion

n = 32, 4 (lower eyelid motion) x 1 repeated-measures ANOVA

Horizontal+ Vertical

No HorizontalNo Vertical

Horizontal Vertical

Modeling and Animating Eye Blinks

Documents

eye gazesaccades

natural eyeblink animations

automated eye motion

different eye blink

associated eye movements

blink physiologyrabbits

eyelid saccades

human blinks