Page 1
ICG
Professor Horst Cerjak, 19.12.20051
Horst Bischof SSL CVPR Tutorial
CVPR 2010 Tutorial Semi-Supervised Learning in
Vision
Inst. for Computer Graphics and VisionGraz University of Technology
A, Saffari, Ch. Leistner, H. Bischof
http://www.icg.tugraz.at/Members/Saffari/ssl-cvpr2010
Page 2
ICG
Professor Horst Cerjak, 19.12.20052
Horst Bischof SSL CVPR Tutorial
Typical Vision Tasks/Trends
Internet: Image Search/Classification Categorization
Huge Amounts of data, Huge labeling effort
Page 3
ICG
Professor Horst Cerjak, 19.12.20053
Horst Bischof SSL CVPR Tutorial
Typical Vision Tasks/Trends
Internet: Various Video and Image Databases
Huge Amounts of data, Partially/weakly labeled data
Page 4
ICG
Professor Horst Cerjak, 19.12.20054
Horst Bischof SSL CVPR Tutorial
Object Categorization
Typical Vision Tasks/Trends
Huge Amounts of data, Huge labeling effort
Page 5
ICG
Professor Horst Cerjak, 19.12.20055
Horst Bischof SSL CVPR Tutorial
VisipediaPerona 2009
Page 6
ICG
Professor Horst Cerjak, 19.12.20056
Horst Bischof SSL CVPR Tutorial
Surveillance: On-line data/Detection-Tracking
Typical Vision Tasks/Trends
Huge Amounts of data, On-line processing, Scene adaptation
Page 7
ICG
Professor Horst Cerjak, 19.12.20057
Horst Bischof SSL CVPR Tutorial
Many other tasks:• Tracking: On-line adaptation to object• Interactive segmentation: Changing model on
the fly• Interactive labeling: Suggestions as you label• …..
Typical Vision Tasks/Trends
Page 8
ICG
Professor Horst Cerjak, 19.12.20058
Horst Bischof SSL CVPR Tutorial
Requirements
Huge datasets
Unreliable (partial) labels
Changing environme
nts
Page 9
ICG
Professor Horst Cerjak, 19.12.20059
Horst Bischof SSL CVPR Tutorial
Requirements on Learner
On-line learning
RobustnessSemi-
supervised Learning
Page 10
ICG
Professor Horst Cerjak, 19.12.200510
Horst Bischof SSL CVPR Tutorial
Theory/Methods Covered
1. Semi-Supervised Learning2. Self-Training3. Generative Models4. Margin Assumption5. Cluster and Manifold Assumption6. Multi-View Learning7. Large-Scale, Multi-Class SSL and Online Learning8. Transfer Learning, Domain Adaptation, and Weakly
Related Data9. Multiple-Instance Learning
Page 11
ICG
Professor Horst Cerjak, 19.12.200511
Horst Bischof SSL CVPR Tutorial
Applications Covered
1. Object Detection2. Categorization3. Tracking4. Activity Recognition5. Segmentation
Using:BoostingRandom Forests
Page 12
ICG
Professor Horst Cerjak, 19.12.200512
Horst Bischof SSL CVPR Tutorial
Learning Tasks
• Unsupervised learning– Density estimation, Clustering, Dimensionality reduction.
• Semi-Supervised Clustering– Clustering with pair-wise constraints: must-link, cannot-link.
• Semi-Supervised Classification and Regression
• Supervised Learning– Classification, Regression.
Page 13
ICG
Professor Horst Cerjak, 19.12.200513
Horst Bischof SSL CVPR Tutorial
How do we get labels?Labels
Page 14
ICG
Professor Horst Cerjak, 19.12.200514
Horst Bischof SSL CVPR Tutorial
Semi-Supervised LearningSSL is Supervised Learning...
Goal: Estimate P(y|x) from Labeled DataDl={ (xi,yi) }
But: Additional Source tells about P(x)(e.g., Unlabeled Data Du={xj})
The Interesting Case:
)()()|()|(
xPypyxpxyp =
Page 15
ICG
Professor Horst Cerjak, 19.12.200515
Horst Bischof SSL CVPR Tutorial
Supervised learning
+ -
+ -
Maximum margin
Page 16
ICG
Professor Horst Cerjak, 19.12.200516
Horst Bischof SSL CVPR Tutorial
Can Unlabeled Data Help?
-?
-
???
?
?
?
?
?
?
????
?
??
??
?
? +?
+
???
?
?
?
????
?
?
?
?
?
?
?
?
? ?
low densityaround
decisionboundary
Page 17
ICG
Professor Horst Cerjak, 19.12.200517
Horst Bischof SSL CVPR Tutorial
SSL is biologically plausible
• Co-training by infants [Bahrick et.al. 2002]• Human change model once they see unlabeled data
[Zahki et.al. 2007,Zhu et.al. 2007,Vandist et.al. 2007]• Humans do On-line SSL [Zhu ICML 2010]
Page 18
ICG
Professor Horst Cerjak, 19.12.200518
Horst Bischof SSL CVPR Tutorial
Why?
1. Unlabelled data is cheap/free
2. Labeled data is hard to get
• human annotation is boring• labels may require experts• labels may require special devices• your graduate student is on vacation
Page 19
ICG
Professor Horst Cerjak, 19.12.200521
Horst Bischof SSL CVPR Tutorial
Online Learning in Perspective
Memory
Com
puta
tion
0 ∞0
∞
Online
Batch/Off-line
Incremental
Page 20
ICG
Professor Horst Cerjak, 19.12.200522
Horst Bischof SSL CVPR Tutorial
Why on-line learning?
Too much training data to fit in memory– Internet!!!
Sample generation process– Tracking, Co-Training
Changing processes– Changing Environment
Page 21
ICG
Professor Horst Cerjak, 19.12.200523
Horst Bischof SSL CVPR Tutorial
Why on-line learning?
Specializing– Forget irrelevant information– Specialize to current scene
Interactive Applications– Data labeling– Classifier Training– Specializing (Human in the loop)– Interactive Training (Segmentation)
Page 22
ICG
Professor Horst Cerjak, 19.12.200524
Horst Bischof SSL CVPR Tutorial
Robustness in Learning
Noisy input data
Label noise• Semi-supervised learning• Weakly-labeled data• Co-training• Self-learning• On-line learning
Page 23
ICG
Professor Horst Cerjak, 19.12.200525
Horst Bischof SSL CVPR Tutorial
Amir SaffariTheory
Christian LeistnerAlgorithms/Applications