Learning sparse representations Learning sparse representations to restore, classify, and sense images and videos Guillermo Sapiro University of Minnesota Supported by NGA, ONR, DARPA, ARO, NSF, NIH
Learning sparse representationsLearning sparse representationsto restore, classify, and sense images and videos
Guillermo Sapirop
University of Minnesota
Supported by NGA, ONR, DARPA, ARO, NSF, NIH
Compressive LecturingCompressive Lecturing
Learning Sparsity 2
Ramirez
Martin Duarte
Lecumberry
Rodriguez
Learning Sparsity 3
For the past ~2 years …• Introduce and Extend the K-SVD
– Denoising– Demosaicing– Inpainting– Mairal, Elad, Sapiro, IEEE-TIP, January 2008
• Learn multiscale dictionaries– Mairal, Elad, Sapiro, SIAM-MMS, April 2008
• Incoherent dictionaries– Ramirez, Lecumberry, Sapiro, January 2009, pre-print
• Learning to classify– Mairal, Bach, Ponce, Sapiro, Zisserman, CVPR 2008, NIPS 2008Mairal, Bach, Ponce, Sapiro, Zisserman, CVPR 2008, NIPS 2008– Rodriguez and Sapiro, pre-print, 2008.
• Learning to sense sparse signals– Duarte and Sapiro pre-print May 2008 IEEE-TIP to appear
Learning Sparsity 4
Duarte and Sapiro, pre print, May 2008, IEEE TIP to appear
The Sparseland Model for Images
M Every column in D (dictionary) is M Ka prototype signal (Atom).
NN The vector α
contains very few (say L) non-zeros.
=x
N
DA fixed Dictionary A sparse
& random vector
xD α
Learning Sparsity 5
What Should the Dictionary D Be?
α=≤α−α=α ˆx̂L.t.sy21
minargˆ 00
22
DDα 2
D should be chosen such that it sparsifies the representations (for a given task!)
Learn D :
Multiscale LearningOne approach to choose D is from a known set of transforms (Steerable
Color Image Examples
Task / sensing adapted
Internal structure
wavelet, Curvelet, Contourlets, Bandlets, …)
Learning Sparsity 6
Internal structure
Dictionaries for Reconstruction
Learning Sparsity 7
Color multiscale dictionaries
Learning Sparsity 8
Example: Non-uniform noise
Learning Sparsity 9
Example: Inpainting
Learning Sparsity 10
Learning multiple reconstructive and discriminative dictionaries
With J. Mairal, F. Bach, J. Ponce, and A. Zisserman, CVPR ’08, NIPS ‘08
Learning Sparsity 11
, , , , ,
Semi-supervised detection learningp g
MIT -- Learning Sparsity 12
Supervised Dictionary Learning
With J. Mairal, F. Bach, J. Ponce, and A. Zisserman, NIPS ‘08
Learning Sparsity 13
, , , ,
Learning Incoherent Dictionaries
• Optimization depends on the incoherence• Improved generalization properties• Improved classificationImproved classification
Learning Sparsity 14
Results
Learning Sparsity 15
Results
Learning Sparsity 16
Learning for Compressed Sensing
+ “RIP (Identity Gramm Matrix)”
Learning Sparsity 17
Design the dictionary and sensing togethertogether
Learning Sparsity 18
Just Believe the PicturesJust Believe the Pictures
Learning Sparsity 19
Just Believe the PicturesJust Believe the Pictures
Learning Sparsity 20
Just Believe the PicturesJust Believe the Pictures
Learning Sparsity 21
Conclusions• State-of-the-art denoising results for still
(shared with Dabov et al ) and video(shared with Dabov et al.) and video• Vectorial and multiscale learned
dictionariesdictionaries• Dictionary learning with internal structure• Dictionary learning for classification• Dictionary learning for classification• Dictionary learning for sensing
• Dictionary learning for the task• Optimization is dictionary dependent
Learning Sparsity 22
p y p
Please do not use the wrong dictionaries thanksdictionaries… thanks
• 12 M pixel image• 7 million patches7 million patches• LARS+online
learning:learning: ~8 minutes
Learning Sparsity 23
Learning Sparsity 24