Machine Learning Overview Tamara Berg CS 560 Artificial Intelligence Many slides throughout the course adapted from Svetlana Lazebnik, Dan Klein, Stuart Russell, Andrew Moore, Percy Liang, Luke Zettlemoyer, Rob Pless, Killian Weinberger, Deva Ramanan 1
68
Embed
Machine Learning Overview Tamara Berg CS 560 Artificial Intelligence Many slides throughout the course adapted from Svetlana Lazebnik, Dan Klein, Stuart.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Machine Learning Overview
Tamara Berg
CS 560 Artificial Intelligence
Many slides throughout the course adapted from Svetlana Lazebnik, Dan Klein, Stuart Russell, Andrew Moore, Percy Liang, Luke Zettlemoyer, Rob Pless, Killian Weinberger, Deva Ramanan
• Definition– Getting a computer to do well on a task
without explicitly programming it– Improving performance on a task based on
experience
Big Data!
What is machine learning?
• Computer programs that can learn from data
• Two key components– Representation: how should we represent the data?– Generalization: the system should generalize from its
past experience (observed data items) to perform well on unseen data items.
Types of ML algorithms
• Unsupervised– Algorithms operate on unlabeled examples
• Supervised– Algorithms operate on labeled examples
• Semi/Partially-supervised– Algorithms combine both labeled and unlabeled
examples
Clustering
– The assignment of objects into groups (aka clusters) so that objects in the same cluster are more similar to each other than objects in different clusters.
– Clustering is a common technique for statistical data analysis, used in many fields, including machine learning, data mining, pattern recognition, image analysis and bioinformatics.
Euclidean distance, angle between data vectors, etc
K-means clustering
• Want to minimize sum of squared Euclidean distances between points xi and their nearest cluster centers mk
k
ki
ki mxMXDcluster
clusterinpoint
2)(),(
K-means
0 0.5 1 1.5 2 2.5 30
0.5
1
1.5
2
2.5
3
3.5
4
xx
x
x’s – indicate initialization for 3 cluster centers
Iterate until convergence:
1) Compute assignment of data points to cluster centers
2) Update cluster centers with mean of assigned points
Flat vs Hierarchical Clustering
• Flat algorithms– Usually start with a random partitioning of docs into
groups– Refine iteratively– Main algorithm: k-means
• Hierarchical algorithms– Create a hierarchy– Bottom-up: agglomerative– Top-down: divisive
Hierarchical clustering strategies
• Agglomerative clustering• Start with each data point in a separate cluster• At each iteration, merge two of the “closest” clusters
• Divisive clustering• Start with all data points grouped into a single cluster• At each iteration, split the “largest” cluster
PProduces a hierarchy of clusterings
P
P
P
P
Divisive Clustering
• Top-down (instead of bottom-up as in Agglomerative Clustering)
• Start with all data points in one big cluster
• Then recursively split clusters
• Eventually each data point forms a cluster on its own.
Flat or hierarchical clustering?
• For high efficiency, use flat clustering (e.g. k means)
• For deterministic results: hierarchical clustering
• When a hierarchical structure is desired: hierarchical algorithm
• Hierarchical clustering can also be applied if K cannot be predetermined (can start without knowing K)
Clustering in Action – example from computer vision
Recall: Bag of Words Representation
· Represent document as a “bag of words”
Bag of features for images
· Represent images as a “bag of words”
Bags of features for image classification
1. Extract features
1. Extract features
2. Learn “visual vocabulary”
Bags of features for image classification
1. Extract features
2. Learn “visual vocabulary”
3. Represent images by frequencies of “visual words”
Bags of features for image classification
…
1. Feature extraction
2. Learning the visual vocabulary
…
2. Learning the visual vocabulary
Clustering
…
2. Learning the visual vocabulary
Clustering
…Visual vocabulary
Example visual vocabulary
Fei-Fei et al. 2005
3. Image representation
…..
fre
que
ncy
Visual words
Types of ML algorithms
• Unsupervised– Algorithms operate on unlabeled examples
• Supervised– Algorithms operate on labeled examples
• Semi/Partially-supervised– Algorithms combine both labeled and unlabeled