Eigenfaces In Scala

Eigenfaces inScalaA quick walkthrough and sample implementation of the Eigenfaces algorithm in Scala.

Justin Long

What is Eigenfaces?

Need to do quick and dirty object or facial recognition? Eigenfaces may be for you.

The essence of Eigenfaces

Eigenfaces is the name given to a set of eigenvectors when they are used for facial recognition.

A typical use for calculating Eigenfaces works as such:

1. Obtain a training set of faces and convert to a pixel matrix

2. Compute the mean image (which is an average of pixel intensity across each image).

3. Compute a differential matrix by subtracting the mean from each training image, pixel by pixel

4. Compute covariance matrix of the differential matrix

The essence of Eigenfaces

Your “average face” may look a little uncanny…

4. Calculate eigenvectors from covariance matrix

5. Compute Eigenfaces by multiplying eigenvectors and covariance matrices together, and normalizing them

Putting it together

def computeEigenFaces(pixelMatrix: Array[Array[Double]], meanColumn: Array[Double]): DoubleMatrix2D = {

val diffMatrix = MatrixHelpers.computeDifferenceMatrixPixels(pixelMatrix, meanColumn)

val covarianceMatrix = MatrixHelpers.computeCovarianceMatrix(pixelMatrix, diffMatrix)

val eigenVectors = MatrixHelpers.computeEigenVectors(covarianceMatrix)

computeEigenFaces(eigenVectors, diffMatrix)

}

Multiplying eigenvectors/differential

(0 to (rank-1)).foreach { i =>

var sumSquare = 0.0

(0 to (pixelCount-1)).foreach { j =>

(0 to (imageCount-1)).foreach { k =>

eigenFaces(j)(i) += diffMatrix(j)(k) * eigenVectors.get(i,k)

}

sumSquare += eigenFaces(j)(i) * eigenFaces(j)(i)

}

var norm = Math.sqrt(sumSquare)

(0 to (pixelCount-1)).foreach { j =>

eigenFaces(j)(i) /= norm

}

}

Preprocessing is key

You need to preprocess your images!

- Grayscale: important for calculating proper pixel intensity values

- Normalization: very helpful if your images were taken in different lighting conditions

- Cropping: Very important to focus only on facial features

Without preprocessing, you’re gonna have a bad time.

Potential for Eigenfaces?

Facial recognition is most obvious use.

Eigenfaces and eigenvectors are also useful for other applications of computer vision:

- Subjects that can be read 2-dimensionally, from same angle

- Optical Character Recognition (OCR)

- Image segmentation

- http://www.cs.huji.ac.il/~yweiss/iccv99.pdf

What about this author?

Well… I had a more creative use for Eigenfaces…

Tinderbox used Eigenfaces

What happened when I got fed up with all the swiping involved in Tinder? I automated it with Eigenfaces (a slightly modified strategy).

- Added a simple step of averaging all faces together for yes/no faces

- The “average” face was used for Eigenfaces selection (k-nearest neighbor between 2 models)

A quick look at Tinderbox

http://crockpotveggies.com/2015/02/09/automating-tinder-with-eigenfaces.html

Questions?