OP05c-PCA

Ho Chi Minh City University of Technology

Faculty of Electrical and Electronics Engineering

Department of Telecommunications

Lectured by Ha Hoang Kha, Ph.D.

Ho Chi Minh City University of Technology

Email: [email protected]

Principal Component Analysis

PCA

Face detection and recognition

Detection Recognition “Sally”

PCA 2 H. H. Kha

Applications of Face Recognition

Digital photography

PCA 3 H. H. Kha

Applications of Face Recognition

Digital photography

Surveillance

PCA 4 H. H. Kha

Consumer application: iPhoto 2009

http://www.apple.com/ilife/iphoto/

PCA 5 H. H. Kha

Starting idea of “eigenfaces”

1. Treat pixels as a vector

2. Recognize face by nearest neighbor

x

nyy ...1

xy T

kk

k argmin

PCA 6 H. H. Kha

The space of all face images

• When viewed as vectors of

pixel values, face images

are extremely high-

dimensional

• 100x100 image = 10,000

dimensions

• Slow and lots of storage

• But very few 10,000-

dimensional vectors are

valid face images

• We want to effectively

model the subspace of face

images

PCA 7 H. H. Kha

Efficient Image Storage

PCA 8 H. H. Kha

Efficient Image Storage

PCA 9 H. H. Kha

Efficient Image Storage

PCA 10 H. H. Kha

Geometrical Interpretation

PCA 11 H. H. Kha

Geometrical Interpretation

PCA 12 H. H. Kha

Dimensionality Reduction

The set of faces is a “subspace” of the set

of images

• Suppose it is K dimensional

• We can find the best subspace using PCA

• This is like fitting a “hyper-plane” to the set of faces

- spanned by vectors u1, u2, ..., uK

Any face:

PCA 13 H. H. Kha

x≈ +w1u1+w2u2+…+wkuk

Principal Component Analysis

A N x N pixel image of a face, represented as a vector occupies a single point in N2-dimensional image space.

Images of faces being similar in overall configuration, will not be randomly distributedin this huge image space.

Therefore, they can be described by a low dimensional subspace.

Main idea of PCA for faces: • To find vectors that best account for variation of face images in

entire image space.

• These vectors are called eigenvectors.

• Construct a face space and project the images into this face space (eigenfaces).

Image Representation

Training set of m images of size

N*N are represented by vectors of

size N2

x1,x2,x3,…,xM

Example

33154

213

321

191

5

4

2

1

3

3

2

1

Principal Component Analysis (PCA)

• Given: N data points x1, … ,xN in Rd

• We want to find a new set of features that are linear

combinations of original ones:

u(xi) = uT(xi – µ)

(µ: mean of data points)

• Choose unit vector u in Rd that captures the most data

variance

PCA 16 H. H. Kha

Principal Component Analysis

• Direction that maximizes the variance of the projected data:

Projection of data point

Covariance matrix of data

The direction that maximizes the variance is the eigenvector

associated with the largest eigenvalue of Σ

N

N

1/N

Maximize

subject to ||u||=1

PCA 17 H. H. Kha

Eigenfaces (PCA on face images)

1. Compute covariance matrix of face images

2. Compute the principal components (“eigenfaces”)

• K eigenvectors with largest eigenvalues

3. Represent all face images in the dataset as linear

combinations of eigenfaces

• Perform nearest neighbor on these coefficients

PCA 18 H. H. Kha

Eigenfaces example

Training

images

x1,…,xN

PCA 19 H. H. Kha

Eigenfaces example

Top eigenvectors: u1,…uk

Mean: μ

PCA 20 H. H. Kha

Representation and reconstruction

• Face x in “face space” coordinates:

=

PCA 21 H. H. Kha

Representation and reconstruction

• Face x in “face space” coordinates:

• Reconstruction:

= +

µ + w1u1+w2u2+w3u3+w4u4+ …

=

^ x =

PCA 22 H. H. Kha

Recognition with eigenfaces

Process labeled training images Find mean µ and covariance matrix Σ

Find k principal components (eigenvectors of Σ) u1,…uk

Project each training image xi onto subspace spanned by principal components: pi=(wi1,…,wik) = (u1

T(xi – µ), … , ukT(xi – µ))

Given novel image x Project onto subspace:

p=(w1,…,wk) = (u1T(x – µ), … , uk

T(x – µ))

Optional: check reconstruction error x – x to determine whether image is really a face

Classify as closest training face in k-dimensional subspace

^

PCA 23 H. H. Kha

Recognition

The distance of p to each face class is defined by

Єk2 = ||p-pk||

2; k = 1,…,N

A distance threshold Өc, is half the largest distance between any two face images:

Өc = ½ maxj,k {||pj-pk||}; j,k = 1,…,N

PCA 24 H. H. Kha

Recognition

Find the distance Є between the original image x and its reconstructed image from the eigenface space, xf,

Є2 = || x – x ||2

Recognition process:

• IF Є≥Өc

then input image is not a face image;

• IF Є<Өc AND Єk≥Өc for all k then input image contains an unknown face;

• IF Є<Өc AND Єk*=mink{ Єk} < Өc then input image contains the face of individual k*

PCA 25 H. H. Kha

^

PCA

General dimensionality reduction technique

Preserves most of variance with a much more compact

representation

• Lower storage requirements (eigenvectors + a few

numbers per face)

• Faster matching

What are the problems for face recognition?

PCA 26 H. H. Kha

Limitations

Global appearance method: not robust to

misalignment, background variation

PCA 27 H. H. Kha