RECOGNISING FACES MACHINE LEARNING & COMPUTER VISION: 101 SAKSHAM GAUTAM
RECOGNISING FACESMACHINE LEARNING & COMPUTER VISION: 101
SAKSHAM GAUTAM
08.06.2016 Saksham Gautam
LET’S START WITH A QUICK SHOW OF HANDS
HOW MANY OF YOU…
▸ have heard about machine learning?
▸ have used machine learning in your projects?
▸ have implemented any ML algorithm from scratch?
▸ have done Andrew Ng’s (or other) courses on ML?
▸ understand that deep learning uses neural network?
▸ still remember what the kernel trick is?
08.06.2016 Saksham Gautam
SHOW OF HANDS ON YOUR FAMILIARITY WITH COMPUTER VISION
HOW ABOUT…
▸ know how an image can be represented as a matrix?
▸ have used openCV or MATLAB?
▸ understand how convolution can be used to detect edges?
▸ know the role of scale space in computer vision?
▸ remember how eigenvectors can be used for face recognition?
08.06.2016 Saksham Gautam
WHAT DO WE WANT TO ACHIEVE?
FACE DETECTION & RECOGNITION
http://docs.opencv.org/master/d7/d8b/tutorial_py_face_detection.html#gsc.tab=0
FACE
MONA LISA
NOT A FACE!
08.06.2016 Saksham Gautam
TO RECOGNISE AND LABEL OBJECTS IN AN IMAGE
OBJECT RECOGNITION
08.06.2016 Saksham Gautam
BASIC STEPS FOR FACE RECOGNITION
BUT HOW EXACTLY?
1. Capture image
2. Filter out noise
3. Find face in the image
4. Create a similarity metric and a model (Training)
5. Match any given face to one from the database
6. Return the closest match with the probability
08.06.2016 Saksham Gautam
FACE RECOGNITION CAN BE BROKEN DOWN INTO SIMPLE STEPS
BUILDING BLOCKS
RAW IMAGE PROCESSED IMAGE FEATURES
MACHINE LEARNING ALGORITHM
Training
Validation
MODEL
RAW IMAGE
DECISION
How can I capture image?
Remove any noise?
What’s the information in the image?
Can we match patterns?
08.06.2016 Saksham Gautam
HUMAN SEES BEAUTY BUT COMPUTER SEES NUMBERS
WHAT IS THIS?
=
08.06.2016 Saksham Gautam
PIXEL VALUES CAN BE THOUGHT OF AS ‘ELEVATION’ IN IMAGE PLANE
MONALISA IN 3D?
08.06.2016 Saksham Gautam
FILTER - I, MEAN FILTER CAN REMOVE RANDOM NOISE
MEAN FILTER
08.06.2016 Saksham Gautam
FILTERS - II, MEDIAN FILTER CAN REMOVE SALT & PEPPER NOISE
MEDIAN FILTER
08.06.2016 Saksham Gautam
FILTERS-III, ADVANCED FILTERS CAN REMOVE DIFFERENT TYPES OF NOISE
ADVANCED FILTERS
▸ Bilateral filters
▸ Adaptive bilateral filters
▸ …
08.06.2016 Saksham Gautam
38 53 38 70 84
5 56 69 82 98
34 87 85 5 40
25 99 43 69 76
11 61 86 94 59
-1 0 1
-1 0 1
-1 0 1
CONVOLUTION CAN BE APPLIED VERY QUICKLY ON AN IMAGE
CONVOLUTION
*
38 0 -38
5 0 -69
34 0 -85
-115
-115 56 16
-69 112 12
-93 3 6
08.06.2016 Saksham Gautam
CONVOLUTION CAN BE USED FOR COMPUTING IMAGE GRADIENT
IMAGE GRADIENT
0 0 0
100 100 100
0 50 100
100 50 0
-1 0 1*0
0
100
-100
=
08.06.2016 Saksham Gautam
EDGES AND CORNERS ARE FEATURES IN AN IMAGE
SOBEL FILTER FOR DETECTING EDGES
HARRIS CORNER DETECTOR
-1 0 1
-2 0 2
-1 0 1
Gx =-1 -2 -1
0 0 0
1 2 1
Gy =
$python sobel-filter.py
$python harris-corner.py
08.06.2016 Saksham Gautam
MORE ROBUST FEATURES CAN BE USED FOR OBJECT RECOGNITION
SIFT, SURF, HOG
▸ More advanced features can be used for scale invariance
▸ Some are robust even under varying lighting conditions
▸ These serve as the starting point for the ML part
08.06.2016 Saksham Gautam
CASCADES OF FILTERS ON AN IMAGE CAN BE USED FOR DETECTING FACES
DETECTING FACES
http://siret.ms.mff.cuni.cz/facereco/method/
$python viola-jones.py
08.06.2016 Saksham Gautam
FEATURES FRO THE FACE CAN BE FED TO AN ML ALGORITHM
BUILDING BLOCKS
RAW IMAGE PROCESSED IMAGE FEATURES
MACHINE LEARNING ALGORITHM
Training
Validation
MODEL
RAW IMAGE
DECISION
PERFORMANCE (P) OF A METHOD FOR A TASK (T) INCREASES WITH EXPERIENCE (E)
Tom Mitchell
BTW, WHO LEARNS? THE MACHINE, REALLY?
08.06.2016 Saksham Gautam
PROBABILITY AND STATISTICS CAN HELP ANSWER MANY QUESTIONS
T-SHIRT SIZE FOR THE SUMMITMAYBE I SHOULD HAVE PICKED ‘M’ INSTEAD OF ’S'
08.06.2016 Saksham Gautam
EXACT SOLUTION IS NOT ALWAYS POSSIBLE
T-SHIRT SIZE ~ MY BODY’S MEASUREMENTS
08.06.2016 Saksham Gautam
MAXIMUM LIKELIHOOD ESTIMATE HELPS ON THE FACE OF UNCERTAINTY
CLASSIFICATION PROBLEM?
Length #2
Length #1
S
MLength #1
Length #2
S
08.06.2016 Saksham Gautam
MEMBERSHIP TO A CLUSTER CAN BE USED FOR CLASSIFICATION
CLUSTERING
Length #2
Length #1
S
M
08.06.2016 Saksham Gautam
MARGINS CAN BE USED FOR SEPARATING CLASSES
LARGE MARGIN CLASSIFIER
Length #2
Length #1
08.06.2016 Saksham Gautam
LOG(DISTANCE FROM THE HYPERPLANE) = PROBABILITY
LOGISTIC REGRESSION
Length #2
Length #1
08.06.2016 Saksham Gautam
LARGE MARGIN CLASSIFIER CAN BE USED FOR DETECTING FACES
FEEDING VALUES FROM VIOLA JONES FILTER
X1
X2FACE
NOT A FACE
08.06.2016 Saksham Gautam
FEATURES SHOULD MAXIMISE VARIANCE (SCATTER)
EIGENVECTORS
V1
V2
X1
X2
08.06.2016 Saksham Gautam
MATRIX DECOMPOSITION ON FACES CAN BE USED TO FIND EIGENFACES
FACIAL FEATURES AS EIGENVECTORS
V1
V2
Projection
08.06.2016 Saksham Gautam
ACCURACY ITSELF IS NOT ENOUGH
CROSS VALIDATION AND F SCORE
DUMMY CANCER DETECTOR
Data
No
ACCURACY = 98%
08.06.2016 Saksham Gautam
IMAGE CAN BE DIRECTLY FED INTO NEURAL NETWORK
NEURAL NETWORK
08.06.2016 Saksham Gautam
GO WITH THE HYPE, BUT WITH CARE
DEEP LEARNING ~ MASSIVE NEURAL NETWORK
▸ Learning algorithm is the same, i.e. back propagation
▸ Has the same problem with overfitting
▸ Can be used for feature extraction and selection
▸ Mathematical foundations for neural network still not “perfect”
▸ Pointer: https://www.tensorflow.org from Google
08.06.2016 Saksham Gautam
MACHINE LEARNING PIPELINE
SUMMARY
RAW IMAGE PROCESSED IMAGE FEATURES
MACHINE LEARNING ALGORITHM
Training
Validation
MODEL
RAW IMAGE
DECISION
REFERENCES
• OpenCV Documentation. http://docs.opencv.org/3.1.0/#gsc.tab=0
• Andrew Ng. Machine Learning Courser on Coursera. http://www.coursera.org/learn/machine-learning
• Christopher Bishop. Machines that Learn. https://www.youtube.com/watch?v=icaA7gVxqSs
• Video Lecture on Face Detection and Tracking. https://www.youtube.com/watch?v=WfdYYNamHZ8
• Adam Harvey explains Viola-Jones Face Detection. http://www.makematics.com/research/viola-jones/
• Christopher, M. Bishop. "Pattern recognition and machine learning." Company New York 16.4 (2006): 049901.
• Bradski, Gary, and Adrian Kaehler. Learning OpenCV: Computer vision with the OpenCV library. " O'Reilly Media, Inc .", 2008
• Solem, Jan Erik. Programming Computer Vision with Python: Tools and algorithms for analyzing images. " O'Reilly Me dia, Inc.", 2012.
• Hartley, Richard, and Andrew Zisserman. Multiple view geometry in computer vision. Cambridge university press, 2003.
BACKUP SLIDES
08.06.2016 Saksham Gautam
IMAGE GRADIENT IS THE BASIS OF FEATURE EXTRACTION
GRADIENT IMAGE
-1 0 1gx =-1
0
1
gy =* A * A
08.06.2016 Saksham Gautam
EVERY SIGNAL CAN BE DECOMPOSED TO SINES AND COSINES
FOURIER TRANSFORM
▸ Frequency can be thought of as information in the image
▸ Fourier Transform can be used to decompose a signal into these components
▸ Signal can be multiplied with filter in frequency domain
▸ Multiplication in frequency domain is convolution in time domain