03.11.2008, Tim Landgraf Active Appearance Models AG KI, Journal Club 03 Nov 2008.

03.11.2008, Tim Landgraf

Active Appearance Models

AG KI, Journal Club 03 Nov 2008

03.11.2008, Tim Landgraf

03.11.2008, Tim Landgraf

03.11.2008, Tim Landgraf

The Idea

• Objects are modelled in shape and grey-level appearance (training necessary)

• New model instances are synthesized and matched onto the new image

• Model parameters are altered according to the quality of the fit

03.11.2008, Tim Landgraf

The Idea

• Generate new model x

x = μ + P*b

from mean model μ and some (b) linear combination of principal components P

• Fit Ix to image region Ii, by altering b according to (Ix – Ii ) = ΔI

offline

online

03.11.2008, Tim Landgraf

creating the model: step by step

1. annotate landmark points2. align the shapes3. PCA (find modes of shape

variation)4. make data shape-free 5. normalize grey values6. PCA (find modes of grey value variation)7. PCA (on the combined model)

Example: 122 landmarks for the face image

03.11.2008, Tim Landgraf

What the … PCA?

• Principal Component Analysis

(aka: Karhunen-Loeve Transform)

BASICS

03.11.2008, Tim Landgraf

PCA, cont.

• used for decorrelation, dimension reduction, generalization

• Data is assumed to be:– Linear– Gaussian (unimodal)

• Principal components: eigenvectors of the Covariance matrix

BASICS

03.11.2008, Tim Landgraf

AAMexplorer

03.11.2008, Tim Landgraf

Fitting the model onto the image

• x = μ + P*b

• simplest approach: Δb = A*ΔI

• „learn“ A:– perturbate known model b‘ = b + Δb and store

the change of image ΔI – Find A by multi-variate linear regression– (note:) A connects grey-value appearance

with all model params

/* reminder */

03.11.2008, Tim Landgraf

Optimization vs. Learning

Initial position

optimum

optimum

Small perturbations

03.11.2008, Tim Landgraf

Extras

• Iterative Approach:– b1 = b0 + kΔb with k \in {0.25, …, 2.0}

– evaluate error and accept new estimate b1, if better fit, otherwise change k

• Multi-resolution: use pyramids to extend the prediction to greater ranges

03.11.2008, Tim Landgraf

AAMs: Properties

• good results if initial guess within 20 pixels and 10% scale

• depends on training image background appearance, too