Top Banner
Ear Recognition: A Complete System Ayman Abaza a,b and MaryAnn F. Harrison a a West Virginia High Tech Foundation, 1000 Technology Drive, Fairmont, USA; b Cairo University, Cairo, Egypt ABSTRACT Ear Recognition has recently received significant attention in the literature. Even though current ear recognition systems have reached a certain level of maturity, their success is still limited. This paper presents an efficient complete ear-based biometric system that can process five frames/sec; Hence it can be used for surveillance applications. The ear detection is achieved using Haar features arranged in a cascaded Adaboost classifier. The feature extraction is based on dividing the ear image into several blocks from which Local Binary Pattern feature distributions are extracted. These feature distributions are then fused at the feature level to represent the original ear texture in the classification stage. The contribution of this paper is three fold: (i) Applying a new technique for ear feature extraction, and studying various optimization parameters for that technique; (ii) Presenting a practical ear recognition system and a detailed analysis about error propagation in that system; (iii) Studying the occlusion effect of several ear parts. Detailed experiments show that the proposed ear recognition system achieved better performance (94.34%) compared to other shape-based systems as Scale- invariant feature transform (67.92%). The proposed approach can also handle efficiently hair occlusion. Experimental results show that the proposed system can achieve about (78%) rank-1 identification, even in presence of 60% occlusion. Keywords: Ear recognition, haar cascade AdaBoost Classifier, Local Binary Pattern, and Hair occlusion 1. INTRODUCTION Ear trait has recently received significant attention as a powerful biometric because it avoids some of the problems inherent in facial recognition. Additionally, ear-based recognition systems can efficiently extend the compatibilities of face recog- nition systems. For face recognition at a high roll angle, toward the side view, the face recognition performance is very low; while the ear recognition, at that angle, is generally yielding high performance. Ear structure is a promising characteristic because the human ear has many variation among individuals, 1 such as the various curves and geometric measurements of the ear. The first documented use of the ear for identification was put forth by the French Criminologist Alphonse Bertillon 2 in 1890. Iannarelli 3 used one of the first ear recognition systems in the literature in 1949. Burge and Burger 4 present one of the most widely cited methods for ear biometrics. To the best of our knowledge, they never present a recognition experiment. Even though the concept of using the ear as a biometric has been around since the 1890’s there are currently no commercially available systems to automatically verify the identity of individuals using ear information. Generally, ear biometric systems consist of three main stages (as shown in fig. 1): Ear Detection: Segmenting the ear region from the image. Feature Extraction: Representing the ear structure by a feature vector. Classification: Matching the probe and the gallery feature vectors to verify the subject claimed identity or to search a database in order to identify the admitted person. In this paper, we propose the use of the Block-based Local Binary Pattern (LBP) to generate features for ear recognition. The LBP operator is one of the best performing texture descriptors and it has been used in various applications, for example Further author information: (Send correspondence to Ayman Abaza) A.A.: E-mail: [email protected], Telephone: +1 304 333 6452. A.A. is also affiliated with Cairo University. M.H.: E-mail: [email protected], Telephone: +1 304 333 6432 1
11

Ear Recognition: A Complete System · Ansari and Gupta9 used a similar approach for localization of the ear from an arbitrary 2D side face image based on the ears outer helix curves.

Oct 03, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Ear Recognition: A Complete System · Ansari and Gupta9 used a similar approach for localization of the ear from an arbitrary 2D side face image based on the ears outer helix curves.

Ear Recognition: A Complete System

Ayman Abazaa,b and MaryAnn F. Harrisona

aWest Virginia High Tech Foundation, 1000 Technology Drive, Fairmont, USA;bCairo University, Cairo, Egypt

ABSTRACTEar Recognition has recently received significant attention in the literature. Even though current ear recognition systemshave reached a certain level of maturity, their success is still limited. This paper presents an efficient complete ear-basedbiometric system that can process five frames/sec; Hence it can be used for surveillance applications. The ear detection isachieved using Haar features arranged in a cascaded Adaboost classifier. The feature extraction is based on dividing the earimage into several blocks from which Local Binary Pattern feature distributions are extracted. These feature distributionsare then fused at the feature level to represent the original ear texture in the classification stage.

The contribution of this paper is three fold: (i) Applying a new technique for ear feature extraction, and studying variousoptimization parameters for that technique; (ii) Presenting a practical ear recognition system and a detailed analysis abouterror propagation in that system; (iii) Studying the occlusion effect of several ear parts. Detailed experiments show that theproposed ear recognition system achieved better performance (94.34%) compared to other shape-based systems as Scale-invariant feature transform (67.92%). The proposed approach can also handle efficiently hair occlusion. Experimentalresults show that the proposed system can achieve about (78%) rank-1 identification, even in presence of 60% occlusion.

Keywords: Ear recognition, haar cascade AdaBoost Classifier, Local Binary Pattern, and Hair occlusion

1. INTRODUCTIONEar trait has recently received significant attention as a powerful biometric because it avoids some of the problems inherentin facial recognition. Additionally, ear-based recognition systems can efficiently extend the compatibilities of face recog-nition systems. For face recognition at a high roll angle, toward the side view, the face recognition performance is verylow; while the ear recognition, at that angle, is generally yielding high performance.

Ear structure is a promising characteristic because the human ear has many variation among individuals,1 such as thevarious curves and geometric measurements of the ear. The first documented use of the ear for identification was put forthby the French Criminologist Alphonse Bertillon2 in 1890. Iannarelli3 used one of the first ear recognition systems in theliterature in 1949. Burge and Burger4 present one of the most widely cited methods for ear biometrics. To the best ofour knowledge, they never present a recognition experiment. Even though the concept of using the ear as a biometric hasbeen around since the 1890’s there are currently no commercially available systems to automatically verify the identity ofindividuals using ear information.

Generally, ear biometric systems consist of three main stages (as shown in fig. 1):

• Ear Detection: Segmenting the ear region from the image.

• Feature Extraction: Representing the ear structure by a feature vector.

• Classification: Matching the probe and the gallery feature vectors to verify the subject claimed identity or to searcha database in order to identify the admitted person.

In this paper, we propose the use of the Block-based Local Binary Pattern (LBP) to generate features for ear recognition.The LBP operator is one of the best performing texture descriptors and it has been used in various applications, for example

Further author information: (Send correspondence to Ayman Abaza)A.A.: E-mail: [email protected], Telephone: +1 304 333 6452. A.A. is also affiliated with Cairo University.M.H.: E-mail: [email protected], Telephone: +1 304 333 6432

1

Page 2: Ear Recognition: A Complete System · Ansari and Gupta9 used a similar approach for localization of the ear from an arbitrary 2D side face image based on the ears outer helix curves.

Figure 1. Ear recognition system

face5 and periocular recognition.6 It has proven to be highly discriminative and invariant to monotonic gray level changes.5

The idea of using LBP for ear description is motivated by the fact that ears can be seen as a composition of micro-patternswhich are well described by LBP operator. We present detailed experiments using USTB database I7 and UND database,collection E.8 These experiments evaluate:

• Performance optimization by tuning various LBP parameters such as the uniform pattern, number of neighbor pixels,radius of neighbor pixels, division of the input image into blocks, and feature selection techniques to assign weightsfor the blocks.

• The LBP performance compared to bench mark techniques such as the principal component analysis (PCA), andscale-invariant feature transform (SIFT).

• The error accumulates from automation of various components of the ear system.

• The system performance in case of occlusion.

This article is organized as follows: Section 2 highlights some related work on 2D ear recognition, and reviews theprevious assessment of performance in case of hair occlusion. Section 3 and Section 4 give brief overviews of the Haar-based ear detection and the LBP technique respectively. Experimental results are presented in Section 5.3. Finally, Section6 presents conclusions and sketches our future plans.

2. RELATED WORKEar biometrics refer to the automatic measurement of distinctive ear features to identify or confirm the identity of theowner. In this section, we present a review of previous work on 2D ear detection and feature extraction, followed by earocclusion.

2.1 Ear DetectionBurge and Burger4 presented one of the most cited ear biometrics method in the literature. They located the ear usingdeformable contours on a Gaussian pyramid representation of the side profile image gradient. Ear edges (contours) arethen computed. Ansari and Gupta9 used a similar approach for localization of the ear from an arbitrary 2D side face imagebased on the ears outer helix curves.

Abdel-Mottaleb and Zhou,10 and Yuizono et al.11 attempted ear detection using a model-based (template matching)technique. Later, Prakash et al.12 modified the template-based technique by adding skin color. The technique first separatesskin regions from non-skin regions and then searches for the ear within skin regions using the template matching approach.Finally, the ear region is validated using a moment-based shape descriptor.

Hajsaid et al.13 addressed the problem of fully automated ear segmentation, using morphological operators. Addition-ally, they used low computational cost, appearance based features and a learning based Bayesian classifier to determinewhether the output segment is a proper or improper ear segment.

2

Page 3: Ear Recognition: A Complete System · Ansari and Gupta9 used a similar approach for localization of the ear from an arbitrary 2D side face image based on the ears outer helix curves.

2.2 Feature ExtractionChang et al.14 used principal component analysis (PCA), and introduced the concept of Eigen-Ear. They reported aperformance of 72.7% for ear in an identification experiment, compared to 90.9% for the multi-modal system. Thistechnique has been widely used in the literature as a base reference, and that is why we used for comparison.

Dewi and Yahagi15 used ear scale-invariant feature transform (SIFT).16 They classified the owner of an ear by calcu-lating the number of key points matches and the average distance of the closest square distance. Kisku et al.17 used SIFTas feature descriptor for structural representation of ear images. They formed an ear skin color model using a Gaussianmixture model (GMM) and clustered the ear color pattern using vector quantization. After segmentation of ear images inspecified color slice regions, they extracted SIFT key-points. They fused the extracted SIFT key-points from all color sliceregions.

Feng and Mu18 combined wavelet transform and local binary patterns (LBP). They used non-uniform LBP8,1 opera-tor, and evaluated the performance of various similarity measures and two matchers (K Nearest Neighbor, and two-classSupport Vector Machine). They used 70 subjects from the USTB database III,7 and 10 images per subject at various poses.They reported 96.86% cross validation recognition rate using chi-square distance. Wang et al.19 also used wavelet trans-forms and uniform local binary patterns for ear recognition. They decomposed ear images by a Haar wavelet transform,and then applied Uniform LBP simultaneously with block-based and multi-resolution methods to describe the texture fea-tures. They used 79 subjects from the USTB database III,7 and 10 images per subject at various poses. They reported thebest recognition rate by combining uniform LBP of decomposed ear images with multi-resolution and block-based. Therecognition rate was as high as 100% for 5o angle, and deteriorated to 42.41% for 45o angle.

Feng and Mu18 and Wang et al.19 mentioned that LBP of the whole image did not yield good performance and that iswhy they transformed the image to the wavelet domain first. Based on experimental studies, shown in the experimentalsection, it was proven that dividing the image into blocks can yield the same performance as the best wavelet transforms.

2.3 Hair OcclusionYuan et al.20 proposed an Improved Non-Negative Matrix Factorization with sparseness constraints (INMFSC) by imposingan additional constraint on the objective function of NMFSC. They showed by experiments that their enhanced techniqueyielded better performance even with partially occluded images. Later Yuan et al.21 separated the normalized ear image into28 sub-windows. Then, they used Neighborhood Preserving Embedding for feature extraction on each sub-window, andselected the most discriminative sub-windows according to the recognition rate. Finally, they applied weighted majorityvoting for fusion at the decision level. They tested recognition with partially occluded images from 33% top, 33% middle,33% bottom, 50% left, and 50% right of the ear respectively.

Kocaman et al.22 applied principal component analysis (PCA), fisher linear discriminant analysis (FLDA), discrim-inative common vector analysis (DCVA), and locality preserving projections (LPP). The error and hit rates of the fouralgorithms were calculated by random sub-sampling and k-fold cross validation for various occlusion scenarios. Arbab-Zavar et al.23 used Scale Invariant Feature Transform (SIFT) to detect the features within the ear images. They presenteda comparison with PCA to show the advantage derived by the use of the model in successful occlusion handling. LaterBustard and Nixon24 evaluated the SIFT technique using various occlusion ratios and presented rank-1 of 92%, and 74%for 20%, and 30% occlusion from ear top part respectively.

3. EAR DETECTION USING CASCADED ADABOOSTIn order to detect an ear in a given image, the image is scanned using rectangular features arranged in a cascaded AdaBoostsystem.25 These rectangular features encode ad-hoc domain knowledge, and work faster than pixel-based.26

Each rectangle feature (f) represents the main component of the weak classifier (h(x, f, p, φ)), where (φ) is a threshold,and (p) is polarity indicating the direction of the inequality:

h(x, f, p, φ) =

{1 if pf(x) < pφ

0 otherwise(1)

3

Page 4: Ear Recognition: A Complete System · Ansari and Gupta9 used a similar approach for localization of the ear from an arbitrary 2D side face image based on the ears outer helix curves.

System block diagramFigure 2. Schematic diagram of the ear detection system: An input image (1) is scaled multiple times, and the set of all possible 24x16pixel sub-images are extracted from each scale, including overlapping sub-images (2). Each of the sub-images is put through a cascadedAdaboost algorithm (3), after which a decision about the presence and location of any ears is made (4).

The learner is called a weak classifier as it has very low performance, which is still better than guessing. The AdaBoostlearning algorithm is used to enhance the classification performance. Each ensemble classifier consists of T weak classifiersas follows:

H(x) =

{Continue

∑Tt=1 αtht(x) > θ

Reject otherwise(2)

where θ is the threshold of the strong classifier. These strong classifiers are arranged in cascade in order to form thedetection system, as shown in fig. 2.

The input image is divided into overlapped sub-images, and each region is evaluated using the cascaded classifier.Then, the image is scaled down by a scaling factor (s) and the above mentioned process is repeated. Finally, all thedetected regions, at various levels of the pyramid of scale, are scaled back to the original image resolutions and theoverlapped detected regions are combined. Detecting the face region, using a commercial software from Pittsburgh PatternRecognition ∗ (PittPatt), helps cut the ear detection time as follows: (i) Reducing the search area; and (ii) Cutting thepyramid of scales by defining the minimum and maximum scale relative to the head size.

4. LOCAL BINARY PATTERNS EAR DESCRIPTIONOjala et al.27 quantify the intensity patterns in local pixel neighborhood patches such as spots, line ends, edges, corners,and other distinct texture patterns, and have been used in face recognition.27 They have shown the LBP operator to behighly discriminative and computationally efficient. Using LBP operators for ear recognition is based on the descriptionof ears as a composition of micro-patterns. The basic LBP operator assigns a a decimal value to each pixel in the imageby thresholding (P=8) neighbor pixels at distance (R=1), as shown in fig. 3. The histogram (H) of these decimal valuesrepresents the feature vector. Ojala et al.5 extended the LBP operator by using neighborhood of different sizes. Generally,P neighborhood pixels, at distance R, can be used; and bilinear interpolation is used for points out of grid to calculate anapproximation of a pixel’s intensity based on the values at surrounding pixels. Figure 4 shows three examples for circularneighborhoods, that will be used through out the paper.

4.1 Uniform Local Binary PatternsOjala et al.28 called a local binary pattern “uniform”, if it contains at most two bitwise transitions from 0 to 1 or vice versawhen the bit pattern is considered circular. For example, the patterns “00000000” and “11111111” have 0 transitions, while∗http://www.pittpatt.com/

4

Page 5: Ear Recognition: A Complete System · Ansari and Gupta9 used a similar approach for localization of the ear from an arbitrary 2D side face image based on the ears outer helix curves.

Figure 3. Basic LBP operator: (1) For a given input image pixel and its 8 neighbors, (2) Each neighbor pixel greater than or equal to thecenter pixel is assigned 1 otherwise it is assigned 0, (3) These binary values are arranged to form a binary number (01110010), which istransferred to a decimal equivalent (114).

(a) (b) (c)Figure 4. Examples for circular neighborhoods: (a) P=8 and R=1, (b) P=8 and R=2, and (c) P=16 and R=2.

“00000111” and 10000001 have 2 transitions. LBP histogram, for uniform patterns, has a separate bin for every uniformpattern and only one bin for all non-uniform patterns. Experimentally, using 180 ear images from USTB database, we find:

• 90.41% of the patterns in the 8,1 neighborhood (4-a) are uniform,

• 88.84% of the patterns in the 8,2 neighborhood (4-b) are uniform,

• 81.53% of the patterns in the 16,2 neighborhood (4-c) are uniform.

This feature selection method reduced the number of features, for (p=16), from 216 using regular histogram to 243 usinguniform pattern histogram; Hence, we decided to use uniform pattern for the rest of the paper.

4.2 Block Based DivisionFor the block based division, the image is divided into N blocks. These blocks can be of arbitrary size and can overlap.The LBP operator is applied to each block separately, and their corresponding histograms H = [h1h2...hN ] are calculated.Integration of these blocks can be:

• At the feature level: By concatenating the histograms extracted from various blocks, then the overall histogram H isused for matching.

• At the score level: By fusing the various blocks scores (s1, s2, ..sN ) generated by matching the histogram extractedfrom each block alone.

Sub-blocks are expected to be more discriminative than using the whole image. However, this approach needs thedetected images to be aligned, and depends on the fusing method. We presented experiments to evaluate the effect of thisdivision on ear recognition performance.

5

Page 6: Ear Recognition: A Complete System · Ansari and Gupta9 used a similar approach for localization of the ear from an arbitrary 2D side face image based on the ears outer helix curves.

Figure 5. Sample of PittPatt aligned ear images.

5. EXPERIMENTAL RESULTSThis section presents several experiments on evaluating various components of the proposed ear recognition system, aswell as a case study of performance in case of occlusion, as follows:

• description of the databases used in the experiments.

• various experiments to tune the various parameters of the LBP.

• studies on detection and registration errors assessment.

• comparison of LBP performance against other feature extraction techniques presented in the literature.

• a case study about ear recognition performance in case of occlusion.

5.1 Ear DatabasesThree databases were used for various experiments:

1. The University of Notre Dame (UND) databases † are available to the public (free of charge). Collection E contains464 visible-light face side profile (ear) images from 114 human subjects. We refer to this data set as UND, and itcontains 106 subjects to maintain 2 images per subjects. We used this database for testing, and we call test set 1.

2. The University of Science and Technology Beijing (USTB) databases ‡ are available for academic research.7 IMAGEDATABASE I contains 180 images of 60 volunteers. The ear images in the USTB database I7 are vertically alignedthe roll Ψ rotation. We refer to this data set as USTB, and it contains 60 subjects, 3 images per subject. We used thisset to tune the parameters of LBP and to train Eigen-Ear technique, and we call it train set.

3. 200 subjects with occluded ear images from FERET database29, 30 were used in the ear occlusion assessment study,and we call it test set 2.

†http://www3.nd.edu/ cvrl/CVRL/Data Sets.html‡http://www1.ustb.edu.cn/resb/en/index.htm

6

Page 7: Ear Recognition: A Complete System · Ansari and Gupta9 used a similar approach for localization of the ear from an arbitrary 2D side face image based on the ears outer helix curves.

Table 1. Comparison of identification (Rank-1) rate of the LBP technique to tune LBP parametersOperator (Rank1%)

Whole Image LBPU8,1 41.67

Whole Image LBPU8,2 58.33

Whole Image LBPU16,2 61.67

Whole Image LBPU16,2 61.67

(2 × 2) LBPU16,2 80.00

(3 × 3) LBPU16,2 81.67

(5 × 5) LBPU16,2 83.33

(7 × 7) LBPU16,2 83.33

5.2 Tuning Local Binary PatternsTo tune various parameters of Local Binary Patterns (LBP) method, we use the train set from USTB database. For all thesubjects in the USTB data sets, we used: (i) One ear image per subject as a gallery; (ii) One ear image per subject as aprobe. To measure the similarity between the probe histogram Hp and gallery histogram Hg generated by LBP operator,we used the chi-square distance:

SChi(Hp, Hg) = Σj,iωj ∗

(Hpi,j −Hg

i,j)2

(Hpi,j +Hg

i,j)(3)

where i and j refer to the ith bin in histogram corresponding to the jth block, and ωj is the weight for block j.

We set up several identification experiments to tune LBP operators; and hence to optimize the performance (details intable 1). The first experiments select the number of neighbors points (P) and the radius of these points from the center (R).This experiment shows that the LBPU

16,2 operator achieves the best performance. In a pilot study, we evaluated fusion ofthese multi-resolution operators at the score level, and found that it did not improve the performance. Hence we decide touse the LBPU

16,2 operator for the remaining experiments.

We setup a second identification experiment to compare the LBP performance using the whole image against dividingthe image into blocks (2× 2, 3× 3, 5× 5, and 7× 7). For this experiment, we use the LBPU

16,2 operator. This experimentindicates that using block size of (5×5) yields the best performance. Hence we decide to divide the ear images into (5×5)blocks for the remaining experiments.

5.3 Ear Detection and Registration Errors AssessmentThe face, and hence the ear, can rotate along 3 axis to define rotation angles Φ the roll, Θ the pitch, and Ψ the azimuth(yaw). For a yaw angle beyond ±45, the face view is more toward profile and hence the ear structure starts to appear andcan be used for recognition.

To test the effect of automatic registration on the proposed ear recognition system, the head was automatically alignedusing commercial system developed by Pittsburgh Pattern Recognition (PittPatt). This software detects not only the faces,but also the yaw and roll of the head. We use this roll angular information to automatically align the images and registerthe ear. We applied this automatic face detection to test set 1, UND data set (as shown in fig. 5). The experiment showscorrectly detected face in 105 out of 106 subjects. We refer to the resulted data set as UNDP .

Ear detection was implemented using cascaded Adaboost based of Haar features.25 However this method yields seg-mentation errors that affect the overall performance. To test the effect of automating this step, we used the UNDP . Theexperiment shows correctly segmented ears in 103 out of 105 subjects. We refer to the resulted data set as UNDA.

We conducted an experiment to study the accumulated error effect of automating the alignment, using PittPatt, and thedetection for the proposed ear recognition system using LBP technique. The identification rate (Rank-1) for the mentionedexperiment: (i) 94.34% using UND set, (ii) 93.33% using UNDP , and 83.50% using UNDA. The general drop inperformance returns to segmentation accuracy, in other words the system detects an ear but does not accurately segmentthe ear region.

7

Page 8: Ear Recognition: A Complete System · Ansari and Gupta9 used a similar approach for localization of the ear from an arbitrary 2D side face image based on the ears outer helix curves.

Figure 6. Cumulative Match Characteristic (CMC) curve of the uniform LBP technique compared to the Eigen-Ear (PCA) and SIFTtechniques

Table 2. Comparison of identification rate (Rank1) of the LBP technique compared to the PCA and SIFT techniquesOperator UND (Rank1%) UNDA (Rank1%)LBP (5 × 5, LBPU

16,2) 94.34 83.50SIFT 67.92 60.19PCA 60.38 63.11

5.4 LBP versus other feature extraction techniquesWe setup a comparison identification experiment against a shape-based technique SIFT16 and an intensity-based techniqueEigen-Ear. Other intensity-based techniques as Linear discriminative analysis were not applicable to use as they requiremore than one image per subject for training. For this experiment, we use the LBPU

16,2 operator for an ear images dividedinto (5×5) blocks. The performance measure used to evaluate the various schemes is the Cumulative Match Characteristic(CMC) curve, which is commonly used to depict the performance of an identification system. The horizontal axis of theCMC represents rank n, and the vertical axis represents the cumulative rank probability. In the CMC curve, the y-value isthe probability of obtaining the correct identity in the top n positions cumulatively.

Figure 6 shows the CMC curves of the block-based LBP technique compared to the Eigen-Ear and SIFT techniques.Table 2, shows Rank-1 of the performance comparison experiment. This low performance of the SIFT technique is due tofailure in enrollment; in other words ear images with no or insufficient extracted SIFT points: (i) 28 out of the 106 subjectsusing UND set; and (ii) 33 out of the 103 subjects for the UNDA data set. The general drop in performance returns tosegmentation accuracy, in other words the system detect an ear but did not accurate segment the ear region.

We setup another experiment to compare the identification performance of the proposed LBP technique against tech-niques which combined wavelet transform and local binary patterns (LBP)18.19 For this experiment, we use the LBPU

16,2

operator for an ear images divided into (2 × 2) blocks. Table 3 shows the performance of applying the following waveletfilters: Daubechies (db1, db2, db10, db45), Coiflets (coif1, coif5), Symlets (sym2, sym8, sym24), Discrete Meyer (dmey),biorthogonal (bior1.1, bior1.3), reverse biorthogonal (rbio1.1, rbio3.1).18 Table 3 shows that dividing into (2 × 2) blocksachieve superior performance compared to applying wavelet filters. Hence it can be concluded that wavelet transform isnot necessary and regular portioning of the image will yield better performance at lower computational cost.

5.5 Ear Recognition incase of occlusionHair occlusion is more likely to be top down. Based on visual assessment to 200 occluded ear images from FERET database(test set 2),29, 30 we found 81% of the hair occlusion is top down, 17% from the hair side, and 2% from bottom up (Usuallythe occlusion for the bottom part return to huge ear rings) or random parts of the ear, as shown in fig. 7 (a-d).

8

Page 9: Ear Recognition: A Complete System · Ansari and Gupta9 used a similar approach for localization of the ear from an arbitrary 2D side face image based on the ears outer helix curves.

Table 3. Identification rate (Rank1) using LBPU16,2) after wavelet transform compared to block-based LBPU

16,2)

Operator UND (Rank1%) UNDA (Rank1%)(2x2 ) 90.84 82.52(db1) 65.71 48.54(db2) 70.48 53.40(db10) 60.00 46.60(db45) 31.43 36.89(coif1) 63.81 48.54(coif5) 50.48 42.72(sym2) 70.48 53.40(sym8) 61.90 50.49(sym24) 47.62 37.86(dmey) 38.10 34.95(bior1.1) 65.71 48.54(bior1.3) 74.29 49.51(rbio1.1) 65.71 48.54(rbio3.1) 79.05 57.28

(a) (b) (c) (d)

(e) (f) (g) (h)Figure 7. Example of real occluded ear images are given (a-d). Test images using mask covering part of the test images: (e) left; (f)right; (c) top; and (d) bottom.

We setup an experiment with various occlusion masks as shown in fig. 7 (e-h), simulating the real hair occlusion shownin fig. 7 (a-d). For this experiment, we use the LBPU

16,2 operator for an ear images divided into (5 × 5) blocks. Table VIshows the performance of the proposed method with various occlusion percentages up to 80%.

Table 4. Identification (Rank1) rate for various occlusion percentages and location using UND databaseOcclusion Rank1% Occlusion Rank1%Top (20%) 92.45 Top (40%) 88.68Bottom (20%) 94.34 Bottom (40%) 97.17Left (20%) 97.17 Left (40%) 94.34Right (20%) 93.40 Right (40%) 89.62Top (60%) 80.19 Top (80%) 60.38Bottom (60%) 92.45 Bottom (80%) 84.91Left (60%) 94.34 Left (80%) 90.57Right (60%) 78.30 Right (80%) 51.89

9

Page 10: Ear Recognition: A Complete System · Ansari and Gupta9 used a similar approach for localization of the ear from an arbitrary 2D side face image based on the ears outer helix curves.

6. CONCLUSION AND FUTURE WORKIn this paper, we proposed a fully automated real-time ear recognition system. The proposed system can process up tofive images (640 x 480) per sec. This system used Haar based cascaded AdaBoost for ear detection, and depends on thehead rotation angle to align these detect ears. Then LBP histograms, corresponding to various non-overlapped blocks, areconcatenated into one histogram to represent the detected ear texture.

We presented detailed experiments to tune the LBP operators, and studied the error accumulates by various components.We compared the performance of the proposed technique to the bench mark and related ear recognition techniques. Theexperimental results show that the proposed ear recognition method achieved superior performance. Then we presenteda set of experiments to determine the effect of hair occlusion, and found that the system can still achieve reasonableperformance even in presence of 60% occlusion.

Future work to enhance the performance of the proposed methods includes: (i) Advanced methods for dividing earimages into significant regions based on ear anatomy; and (ii) Integrating another preprocessing steps to enhance the earsegmentation step.

REFERENCES[1] Jain, A. and Ross, A., [Handbook of Biometrics ], ch. 1 Introduction to Biometrics, 1–22, Springer (2007).[2] Bertillon, A., [Signaletic Instructions Including: The Theory and Practice of Anthropometrical Identification ], R.W.

McClaughry translation, The Werner Company (1896).[3] Iannarelli, A., [Ear Identification, Forensic Identification Series ], Paramount Publishing Company, Fremont, Califor-

nia (1989).[4] Burge, M. and Burger, W., “Ear biometrics in computer vision,” in [the 15th International Conference on Pattern

Recognition (ICPR) ], 826–830 (2000).[5] Ojala, T., Pietikainen, M., and Maenpaa, T., “Multiresolution gray-scale and rotation invariant texture classification

with local binary patterns,” IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 24(7), 971–987(2002).

[6] Miller, P. E., Lyle, J. R., Pundlik, S. J., and Woodard, D. L., “Performance evaluation of local appearance basedperiocular recognition,” in [the Biometrics: Theory, Applications, and Systems (BTAS)], (2010).

[7] USTB, “University of science and technology beijing USTB database.” available at:http : //www1.ustb.edu.cn/resb/en/index.htm.

[8] UND, “University of notre dame UND databases.” available at:http : //www3.nd.edu/ cvrl/CV RL/Data Sets.html.

[9] Ansari, S. and Gupta, P., “Localization of ear using outer helix curve of the ear,” in [the International Conference onComputing: Theory and Applications (ICCTA) ], 688–692 (2007).

[10] AbdelMottaleb, M. and Zhou, J., “Human ear recognition from face profile images,” in [the 2nd International Con-ference on Biometrics (ICB) ], 786–792 (2006).

[11] Yuizono, T., Wang, Y., Satoh, K., and Nakayama, S., “Study on individual recognition for ear images by using geneticlocal search,” in [the Congress on Evolutionary Computation (CEC)], 237–242 (2002).

[12] Prakash, S., Jayaraman, U., and Gupta, P., “A skin-color and template based technique for automatic ear detection,”in [the 17th International Conference on Advances in Pattern Recognition (ICAPR)], (2009).

[13] HajSaid, E., Abaza, A., and Ammar, H., “Ear segmentation in color facial images using mathematical morphology,”in [the 6th Biometric Consortium Conference (BCC)], (2008).

[14] Chang, K., Bowyer, K., Sarkar, S., and Victor, B., “Comparison and combination of ear and face images inappearance-based biometrics,” IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 25, 1160–1165 (2003).

[15] Dewi, K. and Yahagi, T., “Ear photo recognition using scale invariant keypoints,” in [the 2nd International Associationof Science and Technology for Development (IASTED): Conference on Computational Intelligence], 253–258 (2006).

[16] Lowe, D. G., “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision(IJCV) 60(2), 91–110 (2004).

10

Page 11: Ear Recognition: A Complete System · Ansari and Gupta9 used a similar approach for localization of the ear from an arbitrary 2D side face image based on the ears outer helix curves.

[17] Kisku, D. R., Mehrotra, H., Gupta, P., and Sing, J. K., “SIFT-based ear recognition by fusion of detected key-points from color similarity slice regions,” in [the International Conference on Advances in Computational Tools forEngineering Applications (ACTEA) ], 380–385 (2009).

[18] Feng, J. and Mu, Z., “Texture analysis for ear recognition using local feature descriptor and transform filter,” in[SPIE: Pattern Recognition and Computer Vision], (2009).

[19] Wang, Y., chun Mu, Z., and Zeng, H., “Block-based and multi-resolution methods for ear recognition using wavelettransform and uniform local binary patterns,” in [the 19th International Conference on Pattern Recognition (ICPR)],(2008).

[20] Yuan, L., chun Mu, Z., Zhang, Y., and Liu, K., “Ear recognition using improved non-negative matrix factorization,”in [the 18th International Conference on Pattern Recognition (ICPR)], 501–504 (2006).

[21] Yuan, L., hua Wang, Z., and chun Mu, Z., “Ear recognition under partial occlusion based on neighborhood preservingembedding,” in [the SPIE: Biometric Technology for Human Identification VII], 7667 (2010).

[22] Kocaman, B., Kirci, M., Gunes, E. O., Cakir, Y., and Ozbudak, O., “On ear biometrics,” in [the IEEE Region 8Conference (EUROCON) ], (2009).

[23] ArbabZavar, B., Nixon, M., and Hurley, D., “On model-based analysis of ear biometrics,” in [the Biometrics: Theory,Applications, and Systems (BTAS)], (2007).

[24] Bustard, J. and Nixon, M., “Robust 2D ear registration and recognition based on SIFT point matching,” in [theBiometrics: Theory, Applications, and Systems (BTAS)], (2008).

[25] Abaza, A., Hebert, C., and Harrison, M. A. F., “Fast learning ear detection for real-time surveillance,” in [the Biomet-rics: Theory, Applications, and Systems BTAS], (2010).

[26] Viola, P. and Jones, M., “Robust real-time face detection,” International Journal of Computer Vision (IJCV) 57(2),137–154 (2004).

[27] Ojala, T., Pietikinen, M., and Harwood, D., “A comparative study of texture measures with classification based onfeatured distributions,” Pattern Recognition 29(1), 51–59 (1996).

[28] Ojala, T., Hadid, A., and Pietikainen, M., “Face description with local binary patterns: Application to face recogni-tion,” IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 28(12), 2037–2041 (2006).

[29] Phillips, P. J., Wechsler, H., Huang, J., and Rauss, P. J., “The feret database and evaluation procedure for facerecognition algorithms,” Image and Vision Computing 16(5), 295–306 (1998).

[30] Phillips, P., Moon, H., Rizvi, S. A., and Rauss, P. J., “The feret evaluation methodology for face recognition algo-rithms,” IEEE Transactions on Pattern Analysis and Machine Intelligence 22(10), 1090–1104 (2000).

11