Statistical Approach to a Color-based Face Detection Algorithmcarmenng/368/ee368group15.pdf · techniques to formulate a face detection algorithm that is able to give a high success

Statistical Approach to a Color-basedFace Detection Algorithm

EE 368 – Digital Image Processing

Group 15

Carmen NgThomas Pun

May 27, 2002

2

Table of ContentTable of Content .................................................................................................................................. 2Table of Figures................................................................................................................................... 3Introduction: ........................................................................................................................................ 4Assumptions:....................................................................................................................................... 4Face Detection Algorithm:................................................................................................................... 5

Stage I - Pre-Processing ............................................................................................................... 5Stage II - Skin Color Region Labeling.......................................................................................... 5Stage III – Face Selection Based on Statistics Within Image ........................................................ 7Stage IV - Edge Detection on Eyes .............................................................................................. 8

Result of Training Images:................................................................................................................. 10Advantages and Disadvantages of our Face Detection Algorithm: ..................................................... 10Conclusion: ....................................................................................................................................... 11References:........................................................................................................................................ 12

3

Table of FiguresFigure 1. Distritution of Chrominance (Cr) Plane of All Faces in Training Images............................... 5Figure 2. Original Image with Side Extension (Training_1.jpg) ........................................................... 6Figure 3. Rough Skin Region............................................................................................................... 6Figure 4. Binary Mask After a Set of Morphological Operators ........................................................... 7Figure 5. Face Regions Similar to the Most Popular Area (Note unwanted regions are marked by red

rectangle) ..................................................................................................................................... 8Figure 6 Some background .................................................................................................................. 9Figure 7. After Low Pass Filter and Edge Detection of Background..................................................... 9Figure 8. A Typical Face ..................................................................................................................... 9Figure 9. After Low Pass Filter and Edge Detection of a Typical Face................................................. 9Figure 10. Final Result for Training_1.jpg ......................................................................................... 10

4

Introduction:

The goal of our project is to detect multiple faces in an image given a set of training images similar tothe test image. Existing face detection algorithms include template matching, the use of neuralnetworks, featured-based approach in which features of the face are extracted and verified, modelbased approach which involves the shape and/or color of the face candidates. We combined thesetechniques to formulate a face detection algorithm that is able to give a high success rate in a relativelyshort time.

Our method focuses on the use of a skin color model along with its statistical information. Aimingtowards a test image containing ten or more faces, a statistical approach is fairly reliable. Besidessome pre-processing that has to be done to the image, our face detection algorithm can be divided intothree stages. Face detection is done through color-based identification. First stage involves skin colorregion labeling to identify possible face candidates. The other two stages are rejection-basedalgorithms taking into account the edge and statistical information we obtained from all the faces in theimage.

Assumptions:

The following assumptions are made to the input image in order to simplify the algorithm and improveefficiency:

- The input is a color image- There are multiple faces with frontal view and upright orientation- The size of faces within the image should approximately be the same- Little deviation in brightness for all the faces within the image- Faces have to be greater than a certain size in the image so that facial features can be detected

5

Face Detection Algorithm:

Our face detection algorithm consists of four stages: pre-processing, skin color region labeling, faceselection based on statistics of the faces within the image, and edge detection on the eyes. We takeadvantages of the fact that the test image is going to be a color image with multiple faces with a similarfrontal area and at upright orientation

Stage I - Pre-Processing

The left and right boundaries of the input image are replicated by 20 pixels. Therefore, if only half theface is on the image, the second half of the face will be mirrored and thus be easier to detect. Thisboundary replication improves accuracy on the half faces we have in the training images.

Stage II - Skin Color Region Labeling

Since we need to identify skin color of all different races, only the chrominance components isconsidered as it is observed that the major difference skin tones lies in its intensity [2]. The image,which input format is RGB, is converted to the YCbCr representation. Different methods as suggestedin [1], [2], and [3] were considered. However, we found those methods to be computation intensiveand give only slightly better results. Instead, we pick a threshold from the Cr histogram for the faceswithin all our training images. From Figure 1, we use the most popular bin (142 < Cr < 160). We thenapply this threshold to the Cr plane of the image to obtain a rough mask of all the skin color objects.Figure 2 shows the original image and Figure 3 shows the rough mask generated by the above method.

Figure 1. Distritution of Chrominance (Cr) Plane of All Faces in Training Images

6

Figure 2. Original Image with Side Extension (Training_1.jpg)

Figure 3. Rough Skin Region

A set of morphological operators are applied to this mask to remove noise and close small holes. First,we use an open filter (erosion followed by dilation) which removes small regions with 1’s. Thoseregions correspond to noise and thin lines in the image. We then apply two dilation filters to connectsmall regions together to better represent a face region. Erosion filters are used at the end to further

7

eliminate small islands and disconnect any possible faces connected because of the dilations. Figure 4shows the final binary mask that is going to pass to the next stage.

Figure 4. Binary Mask After a Set of Morphological Operators

Stage III – Face Selection Based on Statistics Within Image

We further limit the face selection by searching for holes within each region. Because the color ofhuman eyes and mouth is so much different than the skin color (independent of races), holes within theregion will be a good indication of such features. We use Euler number to calculate the number ofholes within a given region. Possible faces are then regions with one or more holes.

Given this set of preliminary face regions, we calculate the average size of the faces. Face regions withvery small area are rejected since they probably do not contain a face or even if they do, we will not beable to detect those small faces because of the lack of details within the region. Another factor is theratio between the width and height of the region. Since the region is the bounding box of a face and wecan safely assume all the faces are at up right position, we can reject those regions that are very narrowor wide. Furthermore, the ratio between the actual area and the bounding box is required to be close toone as most faces are elliptical and fill most of the area within the bounding box. After rejecting theseunlikely regions, we compute the histogram of the area. The most popular area is then estimated as themean of the top 60% of the areas. We save this most popular area as popular_area to be used later.

With this popular_area, we can further improve the list of possible faces by including regions thathave similar areas. These regions are not selected at the first place because there is no holes within theregion. This can be due to closed or blocked eyes and mouth. Regions with much smaller areas thanthe popular_area are also rejected since we assume all faces within an image have similar sizes.We also compute popular_width and popular_height similarly to popular_area. Figure

8

5 shows the corresponding face regions similar to the popular_area. Note the extra hand and partof the background were also selected.

Figure 5. Face Regions Similar to the Most Popular Area (Note unwanted regions are marked by red rectangle)

With the help of these popular_width and popular_height, the hand in Figure 5 is rejectedbecause it is much smaller than other possible faces. We can also determine if a region may containmore than one face. Due to the morphological operations and our color based selection method,multiple faces may be connected. We can now check for regions that have width and height verydifferent from the popular ones. If we find such region, we will divide this big region into smallerregions and pass them on to the next stage.

Stage IV - Edge Detection on Eyes

We now focus on rejecting regions that do not seem to have facial features (e.g., eyes) on them. Weestimate the location of the eyes as the upper half of the region. Depending on the size of the resultingregion, we then pass it through a low pass filter with the one of the following impulse responses:

9

†

h1 =14

1 11 1È

Î Í

˘

˚ ˙ when the region is small;

˙˙˙˙

˚

˘

ÍÍÍÍ

Î

È

=

1111

1111

1111

1111

161

2h when the region is big

The low pass filter helps removing noise as noise generally contains only high frequency components.Different strength for the low pass filter is used because h2 will blur small eye regions too much andmake it hard to detect any edges later. We apply a Canny edge filter and determine if this eye regioncontains enough edges. If not, we will take this region off as a possible face. Examples of abackground and a face are shown below.

Figure 6 Some background

Figure 7. After Low Pass Filter and Edge Detection ofBackground

Figure 8. A Typical Face

Figure 9. After Low Pass Filter and Edge Detection ofa Typical Face

As shown in Figure 7 and Figure 9, we can distinguish a typical face from the background. Theresulting output is shown in Figure 10.

10

Figure 10. Final Result for Training_1.jpg

Result of Training Images:

Our algorithm detects 150 faces out of the 169 possible faces within the seven training images.Subtracting the number of faces found by the falsely identified face, we achieve an accuracy of88.16%= (150-1)/169. We consider our algorithm fairly efficient. The average time the face detectiontook on a Pentium-4 1Ghz was 20.7 seconds. The profile result from MatLab indicates that more than60 % of the time is spent on RGB to YCbCr color conversion and type conversion (uint8 to double).We approximate the actual processing time is about 8 seconds for 1 mega pixels image on a P-4 1GHz.

Advantages and Disadvantages of our Face Detection Algorithm:

The main advantage of our algorithm is its computation efficiency as shown in the above section.Other advantages are that we work in YCbCr space which is the native color space for many videocompression standards such as the MPEG family, and that no training is required. Due to the nature ofthe statistical model, our detection will be more accurate as the number of images within the imageincreases.

11

Disadvantages of our algorithm come from the fact that we heavily rely on the color information. Greyscale images will not work with our algorithm. Also, we assume there will be a large number of faceswithin the image. Otherwise, our statistic approach may not work well. At last, a better facial featuredetection algorithm is needed to further improve the accuracy.

Conclusion:

We have presented a face detection algorithm for detecting multiple faces in a color image. Severalassumptions are made and faces can only be detected if those criteria are met. We first pre-processedthe image to take care of incomplete faces in the image boundary. Then, we performed a face detectionalgorithm based in the YCbCr space of the image. We looked particularly at the Cr representation forface selection. In order to obtain a set of reliable data, we performed rejection-based edge detectionand statistical analysis to our dataset. We believe that this algorithm provides an efficient and reliableway to detect multiple faces in a color image. Our model works best in the case with a large number offaces in one image as the statistical information contributes to reliability. To further improve thismodel, we would like to work on a better facial feature detection algorithm.

12

References:

[1] R.-L. Hsu, M. Abdel-Mottaleb, and A. K. Jain, "Face detection in color images", Proc. InternationalConference on Image Processing (ICIP) , pp. 1046-1049, Greece, October 7-10, 2001

[2] Jayashree Karlekar and U.B. Desai, ``Finding faces in color images using wavelet transform'' Proc.of Int. Conf. on Image Ana. and Proc. (ICIAP'99), Venice, Italy, Sept. 1999.

[3] Christophe Garcia and Georgios Tziritas, “Face Detection Using Quantised Skin Color RegionsMerging and Wavelet Packet Analysis”, IEEE Transactions on Multimedia, Vol. 1 No. 3, pp. 264-277,September, 1999