Morphable Model - Yonseiweb.yonsei.ac.kr/hgjung/Lectures/AUE859/10. Morphable... · 2014. 12. 29. · - Automated 3D alignment of the faces with 3D-3D Absolute Orientation - Semiautomatic

E-mail: [email protected]://web.yonsei.ac.kr/hgjung

MorphableMorphable ModelModel


Graphics and Vision Research Group, University of Basel, http://gravis.cs.unibas.ch/

Volker Blanz

and Thomas Vetter, “Face Recognition Based on Fitting a 3D Morphable

Model,” IEEE PAMI, Vol. 25, No. 9, Sep. 2003, pp. 1063-1074.

SourcesSources

http://gravis.cs.unibas.ch/


MorphableMorphable Model based Face Recognition SystemModel based Face Recognition System

= Deformable 3D Model= Deformable 3D Model

+ Computer Graphics Simulation of Projection and Illumination+ Computer Graphics Simulation of Projection and Illumination

Intrinsic shape and texture are fully independent of extrinsic parameter

MorphableMorphable ModelModel

- High dimensional vector of shape

- High dimensional vector of texture

- Probability density function of natural faces within face space

MorphableMorphable

ModelModel


Paradigm 1. (The approach taken in this paper)Recognition can be based on model coefficients, which represent intrinsic shape and texture of faces, and are independent of the imaging conditions

Paradigms for Model Based RecognitionParadigms for Model Based Recognition


Paradigm 2.

Viewpoint dependentRecognition System

SyntheticViews

Original (PIE images)

Reconstructions renderedInto the originals

Same reconstructions renderedwith standard illumination

Paradigms for Model Based RecognitionParadigms for Model Based Recognition


- Vector space representation of faces

- Convex combination of shape and texture vectors Si and Ti

A A MorphableMorphable

Model of 3D FacesModel of 3D Faces


Database of Three-Dimensional Laser Scans- 100 males, 100 females, aged between 18 and 45- Recorded with a Cyberware 3030PS laser scanner(512 angular steps, 512 vertical steps = 0.615mm)

Preprocessing of Raw Scans- Filling holes and removing spikes with an interactive tool- Automated 3D alignment of the faces with 3D-3D Absolute Orientation- Semiautomatic trimming along the edge of a bathing cap- Vertical, planar cut behind the ears and a horizontal cut at the neck

Constructing Constructing MorphableMorphable

Model from 3D ScansModel from 3D Scans


- Correspondence between each face and a reference face is given bya dense vector field




Correspondence Based on Optic Flow“Objects in image sequences I(x,y,t) retain their brightnesses as they moveacross the image at a velocity (vx ,vy )T”

- Minimizing the following expression

- v is assumed to be constant on each neighborhood R(x0 ,y0 ) 5x5- In each point (x0 ,y0 ), v(x0 ,y0 ) can be found by solving 2x2 linear system




Definition of Face Vectors

- Our reference face is a triangular mesh with 75,972 vertices

vertices of this mesh be located at

- The definition of shape and texture vectors is based on a reference face I0

- To encode a novel scan I

1) Compute the flow field from I0 to I

2) Convert to Cartesian coordinates

3) Coordinates and color values for the shape and

texture vectors S and T are then sampled at




Principal Component Analysis

- Compute the eigenvectors s1 ,s2 , … by SVD- The eigenvalues of C are the variances of the data along each eigenvector- Eigenvectors form an orthogonal basis

- PCA provides an estimate of the probability density within face space :




Segments- To generate a larger variety of different faces linear combinations of shape and texture are formed separately for different regions of the face : eyes, nose, mouth and the surrounding area

- For continuous transitions between the segments, we apply a modification of the image blending technique




- Estimate shape coefficient ai and texture coefficient bi

relevant parameters of the 3D scene(pose, focal length of the camera, light intensity, color and direction)

- In an analysis-by-synthesis loop, the algorithm finds model parametersand scene parameters such that the model, rendered by computer graphicsalgorithms, produces an image as similar as possible to the input image

ModelModel--Based Image AnalysisBased Image Analysis


- For initialization, the systemcurrently requires image coordinatesof about seven facial feature points,such as the corners of the eyes orthe tip of the nose

- With an interactive tool, the userdefines these points

ModelModel--Based Image AnalysisBased Image Analysis


Image Positions of Vertices- Object-centered coordinates

Illumination and Color- The illumination model of Phong approximately describes the diffuse and

specular reflection of a surface

Image SynthesisImage Synthesis


Optimizing1)2)3) 22 rendering parameters

--------

Fitting the Model to an ImageFitting the Model to an Image


Cost Function- Minimize the sum of square differences

- Minimize feature point error

Maximum A Posteriori estimator



- Ad hoc choices of and are used to control the relative weights- At the beginning, prior probability end EF are weighted high

The final iterations put more weight on EI and no longer rely on EF



Optimization ProcedureThe core of the fitting procedure is a minimization of the cost functionwith a stochastic version of Newton’s method- Select a set K of 40 random triangles in each iteration

Evaluate EI and its gradient only at their centers- Set the probability of selecting a particular triangle proportional to its area- Area calculation once every 1,000 iterations

The optimum is

- Only consider ai



- These second derivatives are computed by numerical differentiation fromthe analytically calculated first derivatives, based on 300 random vertices,at the beginning of the optimization and once every 1,000 iterations

- In each iteration, we perform small steps



Reconstructions of 3D shape and texture from FERET images

Results of Model FittingResults of Model Fitting


Reconstructions of 3D shape and texture from CMU-PIE images

- Illumination in the third image is not fully recovered, so part of thereflections are attributed to texture.

- The error of pose estimates is within a few degrees



Similarity Measure

1) Mahalanobis distance :2) Cosine of the between angle :3) CW distance :



- Overall performance is best for the side-view gallery

Recognition PerformanceRecognition Performance




1% false alarm rate- CMU-PIE : hit rate 77.5%- FERET : hit rate 87.9%


Morphable Model - Yonseiweb.yonsei.ac.kr/hgjung/Lectures/AUE859/10. Morphable... · 2014. 12. 29. · - Automated 3D alignment of the faces with 3D-3D Absolute Orientation - Semiautomatic

Documents