This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
CVPR 2013 Diversity Tutorial
Diverse M-Best Solutions inMarkov Random Fields
Dhruv Batra Virginia Tech
Joint work with: Students: Payman Yadollahpour (TTIC), Abner Guzman-Rivera (UIUC) Colleagues: Chris Dyer (CMU), Greg Shakhnarovich (TTIC),
Pushmeet Kohli (MSRC), Kevin Gimpel (TTIC)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 2
CVPR 2013 Diversity Tutorial
Ambiguity Ambiguity Ambiguity
(C) Dhruv Batra 3
?
?
One instance / Two instances?
CVPR 2013 Diversity Tutorial
Problems with MAP
(C) Dhruv Batra 4
Model-Class is Wrong!-- Approximation Error
Not Enough Training Data!-- Estimation ErrorMAP is NP-Hard
-- Optimization ErrorInherent Ambiguity
-- Bayes ErrorMake Multiple Predictions!
Single Prediction = Uncertainty Mismanagement
CVPR 2013 Diversity Tutorial
Multiple Predictions
(C) Dhruv Batra 5
Porway & Zhu, 2011TU & Zhu, 2002Rich History
Sampling
xxx xx x xxx x xx x
CVPR 2013 Diversity Tutorial
Multiple Predictions
(C) Dhruv Batra 6
Flerova et al., 2011Fromer et al., 2009Yanover et al., 2003
M-Best MAP Ideally:
M-Best Modes ✓Porway & Zhu,
2011TU & Zhu, 2002Rich History
Sampling
CVPR 2013 Diversity Tutorial
Multiple Predictions
(C) Dhruv Batra 7
Flerova et al., 2011Fromer et al., 2009Yanover et al., 2003
M-Best MAP Ideally:
M-Best Modes ✓Porway & Zhu,
2011TU & Zhu, 2002Rich History
SamplingOur work: Diverse M-Best in MRFs [ECCV ‘12]
- Don’t hope for diversity. Explicitly encode it.
- Not guaranteed to be modes.
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 8
Example Result
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 9
Example Result
Discriminative Re-ranking of Diverse Segmentation
[Yadollahpour et al., CVPR13, Wednesday Poster]
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 10
kx1
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 11
kx1
1
0
0
0
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 12
kx1
0
1
0
0
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 13
kx1
0
0
1
0
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 14
kx1
0
0
0
1
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 15
kx1
0
0
0
1
k2x1
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 16
kx1
0
0
0
1
k2x1
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 17
Graphcuts, BP, Expansion, etc
CVPR 2013 Diversity Tutorial
Diverse 2nd-Best
(C) Dhruv Batra 18
MAPDiversity
CVPR 2013 Diversity Tutorial
Diverse M-Best
(C) Dhruv Batra 19
CVPR 2013 Diversity Tutorial
Diverse 2nd-Best
(C) Dhruv Batra 20
Q1: How do we solve DivMBest?
Q2: What kind of diversity functions are allowed?
Q3: How much diversity?
CVPR 2013 Diversity Tutorial
Diverse 2nd-Best
(C) Dhruv Batra 21
Dualize
Diversity-Augmented Score
Primal
CVPR 2013 Diversity Tutorial
Diverse 2nd-Best• Lagrangian Relaxation
(C) Dhruv Batra 22
Diversity-Augmented Score
Dual
Concave (Non-smooth)
Upper-Bound on Div2Best Score
Subgradient Descent
Div2Best score
CVPR 2013 Diversity Tutorial
Diverse 2nd-Best• Lagrangian Relaxation
(C) Dhruv Batra 23
Dualize
Diversity-Augmented EnergyMany ways to solve:
1. Subgradient Ascent. Optimal. Slow.
2. Binary Search. Optimal for M=2. Faster.
3. Grid-search on lambda. Sub-optimal. Fastest.
CVPR 2013 Diversity Tutorial
Theorem Statement• Theorem [Batra et al ’12]: Lagrangian Dual
corresponds to solving the Relaxed Primal:• Based on result from [Geoffrion ‘74]
(C) Dhruv Batra 24
Dual
Relaxed Primal
CVPR 2013 Diversity Tutorial
Effect of Lagrangian Relaxation
(C) Dhruv Batra 25
CVPR 2013 Diversity Tutorial
Effect of Lagrangian Relaxation
(C) Dhruv Batra 26
CVPR 2013 Diversity Tutorial
Effect of Lagrangian Relaxation• [Mezuman et al. UAI13]
Pose Tracking• Chain CRF with M states at each time
(C) Dhruv Batra 39
M BestSolutions
Image Credit: [Yang & Ramanan, ICCV ‘11]
CVPR 2013 Diversity Tutorial
Pose Tracking
(C) Dhruv Batra 40
DivMBest + ViterbiMAP
CVPR 2013 Diversity Tutorial
Pose Tracking
(C) Dhruv Batra 41
1 51 101 151 201 251 30145%
50%
55%
60%
65%
70%
75%
80%
85%
DivMBest (Re-ranked)
[Park & Ramanan, ICCV ‘11] (Re-ranked)
Confidence-based Perturbation (Re-ranked)
13% Gain
Same FeaturesSame Model
#Solutions / Frame
PC
P A
ccur
acy
Better
CVPR 2013 Diversity Tutorial
Machine TranslationInput:Die Regierung will die Folter von “Hexen” unterbinden und gab eine Broschüre heraus
MAP Translation:The government wants the torture of ‘witch’ and gave out a booklet
(C) Dhruv Batra 42
CVPR 2013 Diversity Tutorial
Machine TranslationInput:Die Regierung will die Folter von “Hexen” unterbinden und gab eine Broschüre heraus
5-Best Translations:The government wants the torture of ‘witch’ and gave out a booklet The government wants the torture of “witch” and gave out a booklet The government wants the torture of ‘witch’ and gave out a brochure The government wants the torture of ‘witch’ and gave out a leaflet The government wants the torture of “witch” and gave out a brochure
(C) Dhruv Batra 43
CVPR 2013 Diversity Tutorial
Machine TranslationInput:Die Regierung will die Folter von “Hexen” unterbinden und gab eine Broschüre heraus
Diverse 5-Best Translations:The government wants the torture of ‘witch’ and gave out a bookletThe government wants to stop torture of “witch” and issued a leaflet issuedThe government wants to “stop the torture of” witches and gave out a brochureThe government intends to the torture of “witchcraft” and were issued a leafletThe government is the torture of “witches” stamp out and gave a brochure
(C) Dhruv Batra 44
CVPR 2013 Diversity Tutorial
Machine TranslationInput:Die Regierung will die Folter von “Hexen” unterbinden und gab eine Broschüre heraus
Diverse 5-Best Translations:The government wants the torture of ‘witch’ and gave out a bookletThe government wants to stop torture of “witch” and issued a leaflet issuedThe government wants to “stop the torture of” witches and gave out a brochureThe government intends to the torture of “witchcraft” and were issued a leafletThe government is the torture of “witches” stamp out and gave a brochure
Correct Translation:The government wants to limit the torture of “witches,” a brochure was released