RT Face Recognition

Real-Time Facial RecognitionEECE 279: Real-Time Systems DesignVanderbilt UniversityAmes Brown & Jason CherryMATCH!

Topics Of DiscussionWhy real-time face recognition?What is difficult about real-time face recognition?In general how is face recognition done?EigenfacesOther face recognition algorithmsOppositionFuture of face recognition

Why Real-Time Face Recognition?SecurityFight terrorismFind fugitivesPersonal information accessATMSporting eventsHome access (no keys or passwords)Any other application that would want personal identificationImproved human-machine interactionPersonalized advertisingBeauty search

9/11Purdue already has fingerprint access ATMsMinority Report clip (CD1: 44:00-45:00, CD2: 11:40-16:00)Golden ratio

Real-Time Face Recognition System RequirementsWant the system to be inexpensive enough to use at many locations Match within secondsBefore the person walks away from the advertisementBefore the fugitive has a chance to run awayAbility to handle a large databaseAbility to do recognition in varying environments

What Is Difficult About Real-Time Face RecognitionLighting variationOrientation variation (face angle)Size variationLarge databaseProcessor intensiveTime requirements

General Image TypesStill image (digital photograph)AmIUDynamic imageVideo camera

Dynamic image requires motion detection and head trackingStill image can vary A LOT from picture to picture, need face detection

FERET databaseContains images of 1,196 individuals, with up to 5 different images captured for each individualOften used to test face recognition algorithmsInformation on obtaining the database can be found here: http://www.itl.nist.gov/iad/humanid/feret/

General Face Recognition StepsFace DetectionFace NormalizationFace Identification

Face DetectionIn GeneralLocate face in a given imageSeparate it from the scene

Different ApproachesMotion detecting and head trackingFace Space distance

Face Detection: Face Space

This is a picture of an image that had the face space alg applied to it. This is an application of Eigenfaces. The image is scanned to see how close the regions are to face space. The image on the right indicates how close it is to face space (darker closer to face space)

Face Detection: Motion Detecting and Head Tracking

Face Detection: Motion Detecting and Head Tracking

Face NormalizationAdjustment ExpressionRotationLightingScaleHead tiltEye location

Explain what normalizations purpose is

Face Normalization: FERET face2normImage is rotated to align the eyes (eye coordinates must be known).The image is scaled to make the distance between the eyes constant. The image is also cropped to a smaller size that is nearly just the face.A mask is applied that zeros out pixels not in an oval that contains the typical face. The oval is generated analytically.Histogram equalization is used to smooth the distribution of gray values for the non-masked pixels.The image is normalized so the non-masked pixels have mean zero and standard deviation one.

Mask oval creation could be smarter

Face IdentificationApplication of a face recognition algorithm

PCA AlgorithmsPrinciple Component AnalysisLook at the principal components of the distribution of faces, or the eigenvectors of the covariance matrix of the set of face imagesEigenfaces

Eigenfaces AlgorithmEigenfaces InitializationAcquire an initial set of face images (the training set)

Draw the face space on the boardGenerally used if need real-time because most computation is done ahead of time

Eigenfaces Algorithm2. Calculate the eigenfaces from the training set, keeping only the M images that correspond to the highest eigenvalues. These M images define the face space. As new faces are experienced, the eigenfaces can be updated or recalculated

Eigenfaces Algorithm3. Calculate the corresponding distribution in M-dimensional weight space for each known individual, by projecting their face images onto the face space.

Eigenfaces AlgorithmEigenfaces RecognitionCalculate a set of weights based on the input image and the M eigenfaces by projecting the input image onto each of the eigenfaces.Determine if the image is a face at all by checking to see if the image is sufficiently close to face space.If it is a face, classify the weight pattern as either a known person or as unknown.(Optional) Update the eigenfaces and/or weight patterns.

Eigenfaces ProblemsRecognition performance decreases quickly as the head size, or scale, is misjudged. The head size in the input image must be close to that of the eigenfaces for the system to work wellIn the case where every face image is classified as known, a sample system achieved approximately 96% correct classification averaged over lighting variation, 85% correct averaged over orientation variation, and 64% correct averaged over size variation

Parameter Based Facial RecognitionFacial image is analyzed and reduced to small set of parameters describing prominent facial featuresMajor features analyzed are: eyes, nose, mouth and cheekbone curvatureThese features are then matched to a databaseAdvantage: recognition task is not very expensiveDisadvantage: the image processing required is very expensive and parameter selection must be unambiguous to match an individuals face

Template Based Facial RecognitionSalient regions of the facial image are extractedThese regions are then compared on a pixel-by-pixel basis with an image in the databaseAdvantage is that the image preprocessing is simplerDisadvantage is the database search and comparison is very expensive

Real-Time System using Template RecognitionImplemented on a IBM PC w/ a video camera, image digitizer, and custom VLSI image correlator chip (340 Mop/second).Needed single frontal facial image under semi-controlled lighting conditions Took the system 2 to 3 seconds to identify a user from 173 images of 34 persons 88% recognition rate

How the System Worked

Artificial Neural Networks in Real-Time Face RecognitionUse many of the same algorithms described before but with back propagation ANNsDisadvantages: Complex and difficult to trainDifficult to implementSensitive to lighting variation

There are many face recognition algorithmsLDA (Linear Discriminant Analysis)Bayesian ClassifierGabor Wavelet AlgorithmElastic graphs

Get more information and source code athttp://www.cs.colostate.edu/evalfacerec/index.html

Not Everyone Loves Face RecognitionCritics say it produces too many false positivesInvasion of privacyTo easy to misuse for wrong purposes Technology is not accurate enough given the current technology and algorithms

Show Minority report clip (CD2: 23:00-24:00)

Future Of Face RecognitionSome consider the problem impossibleNo standard way of approaching the problemAdvancements in hardware and softwareSlow integration into society in limited environmentsVery large potential market

Questions?

ReferencesM. Turk and A. Pentland. Eigenfaces for Recognition. Journal of Cognitive Neuroscience, 3(1), 1991C. Nastar and M. Mitschke. Real-Time Face Recognition Using Feature Combination. In Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan, April 1998J. Gilbert and W. Yang. A Real-Time Face Recognition System using Custom VLSI Hardware. Harvard Undergraduate Honors Thesis in Computer Science, 1993.

Topics Of DiscussionWhy real-time face recognition?What is difficult about real-time face recognition?In general how is face recognition done?EigenfacesOther face recognition algorithmsOppositionFuture of face recognition

Real-Time Facial RecognitionEECE 279: Real-Time Systems DesignVanderbilt UniversityAmes Brown & Jason CherryMATCH!

9/11Purdue already has fingerprint access ATMsMinority Report clip (CD1: 44:00-45:00, CD2: 11:40-16:00)Golden ratioDynamic image requires motion detection and head trackingStill image can vary A LOT from picture to picture, need face detectionThis is a picture of an image that had the face space alg applied to it. This is an application of Eigenfaces. The image is scanned to see how close the regions are to face space. The image on the right indicates how close it is to face space (darker closer to face space)Explain what normalizations purpose isMask oval creation could be smarterDraw the face space on the boardGenerally used if need real-time because most computation is done ahead of timeShow Minority report clip (CD2: 23:00-24:00)