Machine Learning Algorithms Inmar Givoni
Oct 31, 2014
Machine Learning
Algorithms
Inmar Givoni
BUT…
• There appear to be problems that are really difficult for computers
• Many correspond to tasks that are really easy for humans
The face of a child can say it all… especially the mouth part of the face.
How do you write software tools to solve these problems?
sorting a list
• Why? Makes other tasks easier
– (like finding Commander Riker)
• How?
3 13 7 1 52 14
Algorithms 101
3 13 7 1 52 14
Sorting a list of numbers 1. Find for the smallest number in the entire array
(from the first to the last position) 2. If it’s not already in the 1st position, it is in some other position, k.
3 13 7 1 52 14
Sorting a list of numbers 1. Find for the smallest number in the entire array
(from the first to the last position) 2. If it’s not already in the 1st position, it is in some other position, k 3. Swap the number in the 1st position, with that smallest number at position k.
1 13 7 3 52 14
Sorting a list of numbers 1. Find for the smallest number in the entire array
(from the first to the last position) 2. If it’s not already in the 1st position, it is in some other position, k 3. Swap the number in the 1st position, with that smallest number at position k.
1 13 7 3 52 14
Sorting a list of numbers 1. Find for the smallest number in the entire array
(from the first to the last position) 2. If it’s not already in the 1st position, it is in some other position, k. 3. Swap the number in the 1st position, with that smallest number at position k 4. Find the smallest number between the 2nd position, to the last position 5. If it’s not already in the 2nd position, it is in some other position k
1 13 7 3 52 14
Sorting a list of numbers 1. Find for the smallest number in the entire array
(from the first to the last position) 2. If it’s not already in the 1st position, it is in some other position, k. 3. Swap the number in the 1st position, with that smallest number at position k 4. Find the smallest number between the 2nd position, to the last position 5. If it’s not already in the 2nd position, it is in some other position k
1 13 7 3 52 14
Sorting a list of numbers 1. Find for the smallest number in the entire array
(from the first to the last position) 2. If it’s not already in the 1st position, it is in some other position, k. 3. Swap the number in the 1st position, with that smallest number at position k 4. Find the smallest number between the 2nd position, to the last position 5. If it’s not already in the 2nd position, it is in some other position k 6. Swap the number in the 2nd position with that smallest number at position k
1 3 7 13 52 14
Sorting a list of numbers 1. Find for the smallest number in the entire array
(from the first to the last position) 2. If it’s not already in the 1st position, it is in some other position, k. 3. Swap the number in the 1st position, with that smallest number at position k 4. Find the smallest number between the 2nd position, to the last position 5. If it’s not already in the 2nd position, it is in some other position k 6. Swap the number in the 2nd position with that smallest number at position k
1 3 7 13 52 14
Sorting a list of numbers 1. Find for the smallest number in the entire array
(from the first to the last position) 2. If it’s not already in the 1st position, it is in some other position, k. 3. Swap the number in the 1st position, with that smallest number at position k 4. Find the smallest number between the 2nd position, to the last position 5. If it’s not already in the 2nd position, it is in some other position k 6. Swap the number in the 2nd position with that smallest number at position k …
1 3 7 13 14 52
Sorting a list of numbers 1. Find for the smallest number in the entire array
(from the first to the last position) 2. If it’s not already in the 1st position, it is in some other position, k. 3. Swap the number in the 1st position, with that smallest number at position k 4. Find the smallest number between the 2nd position, to the last position 5. If it’s not already in the 2nd position, it is in some other position k 6. Swap the number in the 2nd position with that smallest number at position k … ?. When you are at the last position, you are done – everything is now in place!
But for those hard problems…, ‘common sense’ doesn’t quite work!
• Recipe for finding a face in an image?
But for those hard problems…, ‘common sense’ doesn’t quite work!
• Recipe for finding a face in an image?
But for those hard problems…, ‘common sense’ doesn’t quite work!
• Recipe for finding a face in an image?
But for those hard problems…, ‘common sense’ doesn’t quite work!
• Recipe for finding a face in an image?
But for those hard problems…, ‘common sense’ doesn’t quite work!
• Recipe for finding a face in an image?
But for those hard problems…, ‘common sense’ doesn’t quite work!
• Recipe for finding a face in an image?
But for those hard problems…, ‘common sense’ doesn’t quite work!
• Recipe for finding a face in an image?
But for those hard problems…, ‘common sense’ doesn’t quite work!
• Recipe for finding a face in an image?
But for those hard problems…, ‘common sense’ doesn’t quite work!
• Recipe for finding a face in an image?
But for those hard problems…, ‘common sense’ doesn’t quite work!
• Recipe for finding a face in an image?
• Too many variations to list down…
• We probably don’t actually understand the rules ourselves!
Machine Learning approach
1. Gather data: images with their labels (face, no face)
2. Process them somehow into a machine friendly representation
3. Let the machine figure out the decision rule for itself!
A,B,C – Knobs (Parameters)
x,y,z - Numbers from the image
Sounds a bit like magic…? let’s look under the hood
• What does it mean to ‘figure out the rule’?
Face No Face
𝐴𝑥2+𝐵𝑦 + sin 𝐶𝑧 > 0 → 𝑓𝑎𝑐𝑒 𝐴𝑥2 + 𝐵𝑦 + sin 𝐶𝑧 ≤ 0 → 𝑛𝑜 𝑓𝑎𝑐𝑒
Where do the equations come from?
Non-Faces
Faces 𝑎𝑥 + 𝑏𝑦 + 𝑐 > 0
𝑎𝑥 + 𝑏𝑦 + 𝑐 = 0
𝑎𝑥 + 𝑏𝑦 + 𝑐 < 0
How do we set the parameters?
Faces
Non-Faces
Recap: figureing out the rule • Human decides on a template
• Human decides on what criteria to use for finding the parameters
• Human figures out the technique required to optimize this function
• Human (or human’s graduate students) write code that performs that optimization
• ‘Training’ - Computer does the grunt work - calculating parameter values
The dirty details of data representation
• What does it mean ‘machine friendly representation’?
• Data comes in many shapes & forms
• Algorithms crunch numbers
• We need to translate things into numbers
– Things: images, documents, speech, preferences…
– Numbers: <x1,x2,x3,…,xN> (vectors)
From data to vectors
• The good news - it’s possible
• Example 1: Images
184, 137, 119
[0,34,25, 34,52,66, 66,86,24, 4,62,75, …., 184,137,119, …]
From data to vectors
• The good news - it’s possible
• Example 2: Text
A cat and a dog went for a walk, and they
stopped by a tree. The cat climbed up the
tree. The dog continued to sit under the
tree. The cat said to the dog…
From data to vectors
• The good news - it’s possible
• Example 2: Text
0
20
40
60
80
100
120
word counts
a/the cat dog and
go/went/going me/my/I tree Commander
From data to vectors
• The bad new -
– Bad representation → bad performance
– vectors are almost always in very high dimensional space → the curse of dimensionality
Supervised Learning
• Classification – small set of labels (for every input, attach one of K labels)
• Regression – labels are continuous (curve fitting)
Unsupervised learning
• Find groups in data (clustering)
Unsupervised learning
• Find groups in data (clustering)
• Collaborative filtering
Machine Learning Everywhere…
• We’ve come a long way!
– Computer vision
– Speech processing and NLP
– Recommendation systems
– Computational Biology
– Even right here in PowerPoint!!!
Machine Learning Everywhere…
• But there’s a long way to go
• And we are pretty far off from
– “It’s an excerpt from star-trek, Riker’s annoyed, and it’s a pretty entertaining scene”
In conclusion…
• Exciting field!
• Combines mathematics, computer science, optimization theory, statistics, …
• But really – it’s mostly about
– Processing the data into a useful representation
– Coming up with a good model
– Optimizing
Thanks! Questions?