Face Categorization using SIFT features

Mei-Chen (Mei) Yeh

ECE 281B

06/13/2006

Bush vs Schwarzenegger

Serena Williams vs Venus Williams

Main goal Face categorization problem

Object class recognition techniques have seen progress in recent years (ex: bag-of-words models)

Different people are considered different object classes Integration of SIFT features

Does SIFT help face recognition? To what degree? Target on a few people Applications

Face annotation in family albums Name-based photo search

Method Features: SIFT Bag-of-words representation for faces

Each SIFT feature is considered a “codeword” Build a dictionary based on training samples Each face is represented as a histogram over

codewords Learning: Naïve Bayes Classifiers

Vector quantization

…

…..

fre

que

ncy

codewords

SIFT feature vectors from training samples

128-d feature space

codeword 1codeword 2

codeword 3

Datasets The BioID Face Database (simple)

1521 images with 23 people Variety of illumination, background and face size

# of images per person

121 90 150 40

2 51 94 51

78 61 40 50

88 25 6 71

98 106 46 59

60 99 35 100

22 categories, 1613 images70% for training, 30% for testing

Measurement Confusion Matrix

Classifiers

CategoriesAverage Categorization Rate

Average: 89.14%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 bg

Datasets Faces in the Wild (challenging)

851 images, 10 people + 1 non-faces Extracted from news videos

http://www.cs.berkeley.edu/~millert/faces/faceDict/NIPSdict/

# of images per person

Agassi 74 Ryder 46

Arroyo 41 Schwarzenegger 72

Bush 82 Serena Williams 70

Gates 34 Venus Williams 39

Powell 83 non-faces 137

Putin 48

11 categories, 851 images70% for training, 30% for testing

81.82% 0.00% 0.00% 0.00% 0.00% 4.55% 4.55% 0.00% 4.55% 4.55% 0.00% 0.00% 83.33% 0.00% 0.00% 0.00% 0.00% 8.33% 8.33% 0.00% 0.00% 0.00% 4.00% 4.00% 68.00% 0.00% 4.00% 16.00% 4.00% 0.00% 0.00% 0.00% 0.00% 20.00% 0.00% 0.00% 80.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 8.00% 0.00% 0.00% 8.00% 68.00% 12.00% 0.00% 4.00% 0.00% 0.00% 0.00% 0.00% 0.00% 7.14% 0.00% 0.00% 85.71% 0.00% 7.14% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 100.00% 0.00% 0.00% 0.00% 0.00% 0.00% 9.09% 9.09% 0.00% 0.00% 4.55% 0.00% 77.27% 0.00% 0.00% 0.00% 0.00% 4.76% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 85.71% 9.52% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 8.33% 8.33% 83.33% 0.00% 0.00% 2.44% 2.44% 2.44% 2.44% 0.00% 0.00% 0.00% 2.44% 0.00% 87.80%

Average: 81.91%

Conclusions SIFT features + bag-of-words representation

might work for face recognition Simple dataset: good Challenging dataset: may be improved

Consider the spatial relations between features may be the next step to improve the performance