Top Banner
Term Project – Digital Image Processing Skin Detection using Support Vectors Group Members: Ankit Kumar Shrivastava : Y3051 Ankit Misra : Y3053 Gaurav Teltia : Y3120 Kuldeep Singh : Y3155 Nitin : Y3206 Saurabh Khanna : Y3321
28

Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Apr 28, 2018

Download

Documents

buituyen
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Term Project – Digital Image Processing

Skin Detection using Support Vectors

Group Members:

Ankit Kumar Shrivastava : Y3051

Ankit Misra : Y3053

Gaurav Teltia : Y3120

Kuldeep Singh : Y3155

Nitin : Y3206

Saurabh Khanna : Y3321

Page 2: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Need for Skin Detection

There have been a lot of changes in the ways humans interact with

computers.Modern Human-Computer applications

Speech synthesizers Hand gesture recognition Biometric identification

In all the above application, skin color is often used as a cue for detecting, localizing and tracking targets containing skin, like faces and hands in an image

Page 3: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Human Skin

The color of human skin is created by a combination of blood (red) and melanin (yellow, brown).

Skin colors lie between these two extreme hues and are somewhat saturated.

The human skin is a fraction of the actual color cube, about 0.25 % of the total colors

Except for extremely hairy subjects, which are rare, skin has only low-amplitude texture.

Page 4: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Human Skin

Robust skin segmentation involves the formulation of an efficient mathematical model to represent the skin color distribution. Segmenting skin from real-world images is a difficult task even though human skin is known to possess a unique color range, which is but a fraction of all the possible color combinations.

Page 5: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Human Skin Color Model In HSV space

Page 6: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Neural Network Based Color Model for Skin Detection

The training of the skin color model based on neural networks is composed of three stages :• Collection of skin color• Interpolation• Skin color classification.

This scheme proposes to model skin color, by training a three-layered network with skin and non-skin examples, using the back propagation algorithm. The trainednetwork is used to estimate the probable skin regions in a three-dimensional color cube and interpolate to provide a reasonable estimate of the skin color distribution

Since the skin color samples we have collected do not represent the skin color population, we need to interpolate for the skin color that we don’t have using the skin color we have.

Page 7: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Neural Network Based Color Model for Skin Detection

Multilayer perceptron trained using the back-propagation learning algorithm is used for skin color interpolation. Figures below illustrates the back propagation minimize the mean square error between the desired output and the actual output. The back propagation algorithm uses supervised learning where the training of the network is done with known input and output data. Once the network is trained, its weights can be used to compute outputs for new input values.

Multilayer perceptron Back-Propogation

Page 8: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Neural Network Based Color Model for Skin Detection

Applications:

Virtual Studios

To detect and track human faces in real-time video sequences

In face and gesture recognition systems to identify and index individuals for surveillance purposes.

Page 9: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Neural Network Based Color Model for Skin Detection

Advantages:

The proposed method extracts skin very well irrespective of color of the skin

It is able to detect faces in varying lighting environments too.

The method does not need to scan the entire image every time. The skin detector reduces the search area considerably, saving a lot of computation time and enhancing the speed of the system.

Page 10: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Neural Network Based Color Model for Skin Detection

Limitations:

Training a neural network for the skin color detection is challenging because of the difficulty in characterizing non-skin color.

Typically a huge database is required to train the network which is both expensive and computationally cumbersome.

Minimizing overfitting requires a great deal of computational effort.

Page 11: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

RGB Model Based Skin Detection

Distribution of skin-color of different people is clustered in a small area of chromatic space and can be represented by a Gaussian model

Skin color model should be adaptable for any skin color under any lighting conditions.

RGB representation of color images is not suitable for characterizing skin-coloras it also represents luminance.

Luminance may vary across a person's face due to the ambient lighting and is not a reliable measure in separating skin from non-skin region.

Luminance can be removed from the color representation in the chromatic color space.

Page 12: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Skin Detection Model

We collected two sets of 15 color images each with human faces from the World Wide Web.

First set are images of people with white coloured skin (Caucasian and a part of Asian race), second set of people with brown and black coloured skin (African and a part of Asian race).

Then manually selected little rectangle samples of skin from every image of each set were filtered using a low-pass filter to reduce the effect of noise.

Normalized values of red and green color for each pixel of filtered samples are calculated

A kind of normal statistical model that is estimated with parameters – meanvector and covariance matrix:

Page 13: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Estimation of Skin Color Model

Single Gaussian Model:Chromatic( normalized) colors r,g,b are defined as:

r = R/(R+G+B)g = G/(R+G+B)

Mean: m = E {x} where x = (r g)t

Covariance: C = E {(x – m)(x – m)t}

Skin likelihood of an input chrominance vector x is given by p(x) = exp [ -0.5 (x – m )t*inverse)*( x-m)]

So we could create three sets: one set of people with white skins Set W), second set of people with black-brown (Set B) skins and third set as an union of first and second set (Set WB).

Page 14: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Estimation of Skin Color Model

Mixed Gaussian Model:We estimated the weights of each Gaussian model experimentally and we will calculate skin likelihood of an input chrominance vector x by formula :

p(x) = 0.3NW(m,C) + 0.4NB(m,C) + 0.3NWB(m,C)

where x = ( r g ), NB(m,C) is Gaussian model of Set W, NB(m,C) is Gaussian model of Set B, NWB(m,C) is Gaussian model of Set WB.

Page 15: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Processing an Image

The first step in the processing of an input picture is creating a skin-likehood image in which each pixel corresponds to the probability of occurrence of skin-color (in the same pixel in the original input image).

The values of probability can be easily transformed into greyscale values.

Second step is creating a skin-segmented image by using a threshold value of probability. We can use fixed threshold value for every image or adaptive thresholding.

A fixed threshold value is generally more efficient.

Page 16: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Advantages

Skin regions can be made more integral by use of low pass filtering.

This method can be effectively used in images of faces that have some parts more affected with ambient light.

Quality of estimated color model can easily improved by using larger set of analysed samples.

EM algorithm can be used to estimate more appropriate values of mixture Gaussian model.

Page 17: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Disadvantages

To improve performance we use more number of samples but computation burden increases considerably.

Algorithm works better for images of white skin than black ones.

Page 18: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Problem Statement

What we need is a function which can tell whether a given color pixel is a skin pixel or not.

0 : Input pixel is not a skin pixel1 : Input pixel is a skin pixel

Input Color Pixel(RGB/HSV) F(R,G,B)

0

OR

1

Page 19: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Why to use Support Vector Classifier

One of the easiest way to determine if a particular RGB color corresponds to skin color or not, we built a lookup table with all possible combinations of RGB.16 million long database Large storage spaces and Computational times required.Who will build such a huge database??

R G B 0=not skin pixel1= skin pixel

0 0 0 0

0 0 1 0: : : 0 or 1: : : 0 or 1: : : :: : : :255 255 254 0

255 255 255 0

Page 20: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Why to use Support Vector Classifier

A Smart way is evaluate a non-linear function that maps all the RGB space to two sets.( One for skin pixels and the other for non-skin pixels).

Such a non-linear function will not have a closed form expression and hence annotated by several inequalities.

The number of inequalities become large if we want to represent the function accurately.

Page 21: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Support Vector Machines

We want to map the RGB space into a higher dimensional space such that the skin pixels and non-skin pixels are linearly separable.

Page 22: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Support Vector Machines

Map to Higher Dimensional space

Input Color Pixel(RGB/HSV)

Training Set Linear Classifier 1[ A skin pixel]

0 [ Not a skin pixel]

Final Algorithm

Page 23: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

DEMO

Page 24: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Thank You

Page 25: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

Appendix 1) Matlab Codes

Skin.m function skin2(fname,race) %reading the image img=imread(fname); %converting RGB image to HSV image hsv_img = rgb2hsv(img); %uploading the database data=importdata('data2.txt'); [m,n]=size(data); % x=data d=decision x=zeros(m,4); d=zeros(m,1); for i=1:m [H S V]=rgb2hsv(1/255*[data(i,1) data(i,2) data(i,3)]); x(i,1)=H; x(i,2)=S; x(i,3)=V; x(i,4)=data(i,4); d(i,1)=data(i,5); end %uploading K matrix K=importdata('K.txt'); %a=lagrangian multiplier column vector a=importdata('a.txt'); %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%detection of skin%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %displaying original image colormap(gray(256)); image(img); [q,p,r]=size(img); %final_img=zeros(q,p); for X=1:5:p for Y=1:5:q H=hsv_img(Y,X,1); S=hsv_img(Y,X,2); V=hsv_img(Y,X,3); out=0; if(race(1)==1) for i=1:m out=out+a(i)*d(i)*exp((-1/(2*0.00006))*((x(i,1)-H)^2+(x(i,2)-S)^2+(x(i,3)-V)^2+(x(i,4)-1)^2)); end

Page 26: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

if(out>0) % final_img(Y,X)=255; rectangle('Position',[X,Y,2,2]); end end if(race(2)==1) out=0; for i=1:m out=out+a(i)*d(i)*exp((-1/(2*0.00006))*((x(i,1)-H)^2+(x(i,2)-S)^2+(x(i,3)-V)^2+(x(i,4)-2)^2)); end if(out>0) % final_img(Y,X)=255; rectangle('Position',[X,Y,2,2]); end end if(race(3)==1) out=0; for i=1:m out=out+a(i)*d(i)*exp((-1/(2*0.00006))*((x(i,1)-H)^2+(x(i,2)-S)^2+(x(i,3)-V)^2+(x(i,4)-3)^2)); end if(out>0) % final_img(Y,X)=255; rectangle('Position',[X,Y,2,2]); end end end end

update_data.m function update_data %uploading the database data=importdata('data2.txt'); [m,n]=size(data); % x=data d=decision x=zeros(m,4); d=zeros(m,1); %coverting RGB data to HSV data for i=1:m [H S V]=rgb2hsv(1/255*[data(i,1) data(i,2) data(i,3)]); x(i,1)=H; x(i,2)=S; x(i,3)=V; x(i,4)=data(i,4); d(i,1)=data(i,5); end

Page 27: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

%mapping from input space to feature space %formation of K matrix K=zeros(m,m); for i=1:m for j=1:m K(i,j)=exp((-1/(2*0.00006))*((x(i,1)-x(j,1))^2+(x(i,2)-x(j,2))^2+(x(i,3)-x(j,3))^2+(x(i,4)-x(j,4))^2)); end end %updating the K matrix dlmwrite('K.txt', K, 'delimiter', '\t', 'precision', 4); %a=lagrangian multiplier column vector a=zeros(m,1); %updating the lagrangian multipliers Kd=zeros(m,m); for i=1:m for j=1:m Kd(i,j)=K(i,j)*d(i)*d(j); end end a=inv(Kd)*ones(m,1); dlmwrite('a.txt', a, 'delimiter', '\t', 'precision', 4);

Page 28: Term Project – Digital Image Processing Skin Detection ...home.iitk.ac.in/~ankitm/DIP_Project.pdf · Term Project – Digital Image Processing Skin Detection using Support Vectors

2) Results:

Before Before

After After