Unsupervised classification and spectral unmixing for sub-pixel labelling A.Villa ,,† , J.Chanussot , J.A. Benediktsson , C.Jutten GIPSA-lab, Signal & Image Dept., Grenoble Institute of Technology, France. Faculty of Electrical and Computer Engineering, University of Iceland, Iceland. † Aresys, Politecnico di Milano, Italy. IEEE IGARSS 2011 Vancouver, Canada - 2011
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Unsupervised classification and spectral unmixing for sub-pixel labelling
A.Villa?,�,†, J.Chanussot?, J.A. Benediktsson�, C.Jutten?
?GIPSA-lab, Signal & Image Dept., Grenoble Institute of Technology, France.�Faculty of Electrical and Computer Engineering, University of Iceland, Iceland.
† Aresys, Politecnico di Milano, Italy.
IEEE IGARSS 2011Vancouver, Canada - 2011
A new approach to classification Experiments Conclusions
Hyperspectral Images
Widely used in remote sensing:
λ
VIS0.4 μm
NIR2.4 μm
- Trees- Grass
√Wide spectral range and largenumber of wavelengths
√Very high spectral resolution
× Tradeoff between spectral andspatial resolution
Jocelyn Chanussot Gipsa-Lab 2 / 21
A new approach to classification Experiments Conclusions
Challenges
Low spatial resolution → appearance of mixed pixels
Pure pixel: 100% grass
Mixed pixel: 70% metal sheet30% grass
• Common in hyperspectral images
• Traditional classifiers inadequate,partially addressed by mixed pixeltechniques
• Critical for land cover mapping
Joint use (full + mixed techniques) desirable, but little investigated[Wang and Jia, 2010].
Jocelyn Chanussot Gipsa-Lab 3 / 21
A new approach to classification Experiments Conclusions
Challenges
Low spatial resolution → appearance of mixed pixels
Pure pixel: 100% grass
Mixed pixel: 70% metal sheet30% grass
• Common in hyperspectral images
• Traditional classifiers inadequate,partially addressed by mixed pixeltechniques
• Critical for land cover mapping
Incorporation of spectral unmixing in the classification process:
• Does it provide accuracy improvement?• Is it possible to improve the classification map spatial resolution?
Jocelyn Chanussot Gipsa-Lab 3 / 21
A new approach to classification Experiments Conclusions
1 A new approach to classification
2 Experiments
3 Conclusions
Jocelyn Chanussot Gipsa-Lab 4 / 21
A new approach to classification Experiments Conclusions
Context
Traditional techniques neglect sub-pixel and spatial information
Additional information provided by unmixing not fully exploited
Pure pixel: 100% grass
Mixed pixel: 70% metal sheet30% grass
1
1
1 1
10.6
0.6
0.9 0.8
0.8
0.9
0.9
0.6
Original image Classification Unmixing Finer resolution?
How to jointly use full and mixed pixel techniques?
Jocelyn Chanussot Gipsa-Lab 5 / 21
A new approach to classification Experiments Conclusions
Proposed Approach
Low resolutionhyperpspectral data
Spatial regularization
Abundances maps
"Upsampled" classification map
Classes identification
Final map
Unmixing
Classification
Jocelyn Chanussot Gipsa-Lab 6 / 21
A new approach to classification Experiments Conclusions
Proposed Approach
1. Abundances fractions are computed from a HSI
Low resolutionhyperpspectral data
Spatial regularization
Abundances maps
"Upsampled" classification map
Classes identification
Final map
Step 1
Step 2
Step 1:
Pure pixel: 100% grass
Mixed pixel: 70% metal sheet30% grass
Step 2:
1
1
1 1
10.6
0.6
0.9 0.8
0.8
0.9
0.9
0.6
Jocelyn Chanussot Gipsa-Lab 7 / 21
A new approach to classification Experiments Conclusions
The proposed approach
M = Mixed pixel
M
M M
M
M
M
M
M
Proposed methodThe abundances computation is in twosteps, to take the spatial information intoaccount:
1. Pixels with an abundance over acertain threshold are considered ’pure’
2. Abundances of ’mixed’ pixels arecomputed by selecting as endmemberspixels spatially close
Jocelyn Chanussot Gipsa-Lab 8 / 21
A new approach to classification Experiments Conclusions
The proposed approach
M = Mixed pixel
Proposed methodThe abundances computation is in twosteps, to take the spatial information intoaccount:
1. Pixels with an abundance over acertain threshold are considered ’pure’
2. Abundances of ’mixed’ pixels arecomputed by selecting as endmemberspixels spatially close
Jocelyn Chanussot Gipsa-Lab 8 / 21
A new approach to classification Experiments Conclusions
Proposed Approach
2. Creation of a finer classification map
Low resolutionhyperpspectral data
Spatial regularization
Abundances maps
"Upsampled" classification map
Classes identification
Final map
Step 3
Step 2
Step 2:
1
1
1 1
10.6
0.6
0.9 0.8
0.8
0.9
0.9
0.6
Step 3:
Jocelyn Chanussot Gipsa-Lab 9 / 21
A new approach to classification Experiments Conclusions
Proposed Approach
3. Final spatial regularization
Low resolutionhyperpspectral data
Spatial regularization
Abundances maps
"Upsampled" classification map
Classes identification
Final map
Step 3
Step 4
Step 3:
Step 4:
Jocelyn Chanussot Gipsa-Lab 10 / 21
A new approach to classification Experiments Conclusions
Spatial regularization
Criterion: minimization of the total perimeter of the connected areas (e.g.,belonging to the same class)
M
M M
M
M
M
M
M0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
M
M M
M
M
M
M
M0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
Criterion not satisfied Criterion satisfied
Jocelyn Chanussot Gipsa-Lab 11 / 21
A new approach to classification Experiments Conclusions
Spectral unmixing based approach [Villaet al., 2010]
1. VCA for class retrieval
2. FCLS for abundance determination
3. Simulated Annealing for spatialregularization
Novelties introduced:
1. Retrieve classes with unsupervisedclustering(→ more robust to outliers)
2. Include spatial information(→ use more accurateendmembers)
Jocelyn Chanussot Gipsa-Lab 12 / 21
A new approach to classification Experiments Conclusions
VCA vs. K-MEANS
0 1000 2000 3000 4000 5000 60000
2000
40000
1000
2000
3000
4000
5000
6000
0 1000 2000 3000 4000 5000 60000
2000
40000
1000
2000
3000
4000
5000
6000
VCA K-MEANS
Jocelyn Chanussot Gipsa-Lab 13 / 21
A new approach to classification Experiments Conclusions
1 A new approach to classification
2 Experiments
3 Conclusions
Jocelyn Chanussot Gipsa-Lab 14 / 21
A new approach to classification Experiments Conclusions
How to verify the results?
Proposedapproach
Final map(sub-pixel precision)
Decrease originalresolution
Jocelyn Chanussot Gipsa-Lab 15 / 21
A new approach to classification Experiments Conclusions
Experiments on real data
ROSIS University data set
• Classification of a metal sheet roof(120×90 pixels)
• 1.3 m spatial resolution, 103spectral bands.
• Spatial resolution of the originaldata degraded of a factor 3
AISA data set
• 400×500 pixels area, six classes ofinterest
• 6 m spatial resolution, 252 spectralbands
• Spatial resolution of the original datadegraded of a factor 5
Jocelyn Chanussot Gipsa-Lab 16 / 21
A new approach to classification Experiments Conclusions
Real data sets
ROSIS data set:
Original Image K-means (93.75%) VCA+SU (96.95%) KM+SU (95.89%)
Jocelyn Chanussot Gipsa-Lab 17 / 21
A new approach to classification Experiments Conclusions
Real data set
AISA data set:
K-means (51.61%) VCA+SU (59.69%) KM+SU (75.72%)
Jocelyn Chanussot Gipsa-Lab 18 / 21
A new approach to classification Experiments Conclusions
1 A new approach to classification
2 Experiments
3 Conclusions
Jocelyn Chanussot Gipsa-Lab 19 / 21
A new approach to classification Experiments Conclusions
Conclusions and Perspectives
New method to improve spatial resolution of thematic maps:
• Unsupervised clustering to define classes• Integration of spatial information to locally model abundances• Simulated Annealing proposed for spatial regularization
Clustering less sensitive to extreme pixels, VCA better in highly mixed scenarios
Next step: Incorporate spectral variability of the classes
Jocelyn Chanussot Gipsa-Lab 20 / 21
A new approach to classification Experiments Conclusions
Unsupervised classification and spectral unmixing for sub-pixel labelling
A.Villa?,�,†, J.Chanussot?, J.A. Benediktsson�, C.Jutten?
?GIPSA-lab, Signal & Image Dept., Grenoble Institute of Technology, France.�Faculty of Electrical and Computer Engineering, University of Iceland, Iceland.
† Aresys, Politecnico di Milano, Italy.
IEEE IGARSS 2011Vancouver, Canada - 2011
Jocelyn Chanussot Gipsa-Lab 21 / 21
A new approach to classification Experiments Conclusions
Challenges
Hyperspectral images issues:
1 Curse of dimensionality2 Exploitation of contextual information3 Presence of mixed pixels
Pure pixel: 100% grass
Mixed pixel: 70% metal sheet30% grass
• Common in hyperspectral images
• Traditional classifiers inadequate
• Usually not considered forclassification!
Jocelyn Chanussot Gipsa-Lab 22 / 21
A new approach to classification Experiments Conclusions
Context
Traditional approaches to image analysis are full pixel and mixed pixel techniques
• Full pixel techniques are traditional classification algorithms• Mixed pixel techniques are spectral unmixing, soft classification, . . .
Joint use is desirable, but little investigated [Wang and Jia, 2010].
Incorporation of spectral unmixing in the classification process:
• Does it provide accuracy improvement?• Is it possible to improve the classification map spatial resolution?
Jocelyn Chanussot Gipsa-Lab 23 / 21
A new approach to classification Experiments Conclusions
Linear Spectral Unmixing
Abundances estimation through spectral unmixing:• Goal: find extreme pixels (endmembers) that can be used to "unmix" other pixels.• Each "mixed" pixel is a combination of endmember fractional abundances.
Jocelyn Chanussot Gipsa-Lab 24 / 21
A new approach to classification Experiments Conclusions
Linear Spectral Unmixing
Abundances estimation through spectral unmixing:• Goal: find extreme pixels (endmembers) that can be used to "unmix" other pixels.• Each "mixed" pixel is a combination of endmember fractional abundances.
Jocelyn Chanussot Gipsa-Lab 24 / 21
A new approach to classification Experiments Conclusions
Context
Traditional techniques neglect information
Additional information provided by unmixing not fully exploited
Pure pixel: 100% grass
Mixed pixel: 70% metal sheet30% grass
1
1
1 1
10.6
0.6
0.9 0.8
0.8
0.9
0.9
0.6
Original image Classification Unmixing Finer resolution?
How to jointly use full and mixed pixel techniques?
Jocelyn Chanussot Gipsa-Lab 25 / 21
A new approach to classification Experiments Conclusions
The proposed approach?
M = Mixed pixel
M
M M
M
M
M
M
M
Proposed methodWe propose a technique in four steps:
1. Preliminary classification withprobabilistic classifier (SVM)
2. Choose suitable endmembercandidates and perform unmixing
3. Split every pixel into n sub-pixels, andassign them to a class
4. Perform spatial regularization in orderto correctly locate sub-pixels
?A. Villa, J. Chanussot, J.A. Benediktsson, C. Jutten, "Spectral Unmixing for the Classification of Hyperspectral Images
at a Finer Spatial Resolution", IEEE J. Sel. Topics Sign. Proc., vol. 5, n. 3, 2011
Jocelyn Chanussot Gipsa-Lab 26 / 21
A new approach to classification Experiments Conclusions
The proposed approach?
M = Mixed pixel
Proposed methodWe propose a technique in four steps:
1. Preliminary classification withprobabilistic classifier (SVM)
2. Choose suitable endmembercandidates and perform unmixing
3. Split every pixel into n sub-pixels, andassign them to a class
4. Perform spatial regularization in orderto correctly locate sub-pixels
?A. Villa, J. Chanussot, J.A. Benediktsson, C. Jutten, "Spectral Unmixing for the Classification of Hyperspectral Images
at a Finer Spatial Resolution", IEEE J. Sel. Topics Sign. Proc., vol. 5, n. 3, 2011
Jocelyn Chanussot Gipsa-Lab 26 / 21
A new approach to classification Experiments Conclusions
The proposed approach?
M
M M
M
M
M
M
M0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
Proposed methodWe propose a technique in four steps:
1. Preliminary classification withprobabilistic classifier (SVM)
2. Choose suitable endmembercandidates and perform unmixing
3. Split every pixel into n sub-pixels, andassign them to a class
4. Perform spatial regularization in orderto correctly locate sub-pixels
?A. Villa, J. Chanussot, J.A. Benediktsson, C. Jutten, "Spectral Unmixing for the Classification of Hyperspectral Images
at a Finer Spatial Resolution", IEEE J. Sel. Topics Sign. Proc., vol. 5, n. 3, 2011
Jocelyn Chanussot Gipsa-Lab 26 / 21
A new approach to classification Experiments Conclusions
The proposed approach?
M
M M
M
M
M
M
M0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
Proposed methodWe propose a technique in four steps:
1. Preliminary classification withprobabilistic classifier (SVM)
2. Choose suitable endmembercandidates and perform unmixing
3. Split every pixel into n sub-pixels, andassign them to a class
4. Perform spatial regularization in orderto correctly locate sub-pixels
?A. Villa, J. Chanussot, J.A. Benediktsson, C. Jutten, "Spectral Unmixing for the Classification of Hyperspectral Images
at a Finer Spatial Resolution", IEEE J. Sel. Topics Sign. Proc., vol. 5, n. 3, 2011
Jocelyn Chanussot Gipsa-Lab 26 / 21
A new approach to classification Experiments Conclusions
Simulated Annealing
Minimize a given Cost Function introducingrandom perturbations:
• decreases of the CF are always accepted• increases of the CF accepted with a probabilityinversely proportional to the degradation
• probability of ’bad solutions’ decreases as thesearch continues
Simulated Annealing optimization avoids local minima leading to global optimalsolution
Jocelyn Chanussot Gipsa-Lab 27 / 21
A new approach to classification Experiments Conclusions
Simulated Annealing
Cost function to be minimized: total perimeter of the connected areas (e.g.,belonging to the same class)
M
M M
M
M
M
M
M0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,60,4
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
M
M M
M
M
M
M
M0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,60,4
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
Cost function not optimized Cost function optimized
Jocelyn Chanussot Gipsa-Lab 28 / 21
A new approach to classification Experiments Conclusions
Simulated Annealing
M
M M
M
M
M
M
M0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,60,4
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
M
M M
M
M
M
M
M0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,60,4
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
Initial condition Iteration 1
M
M M
M
M
M
M
M0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,60,4
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
M
M M
M
M
M
M
M0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,60,4
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
Iteration n Final result
Jocelyn Chanussot Gipsa-Lab 28 / 21
A new approach to classification Experiments Conclusions
Simulated Annealing
Cost function to be minimized: total perimeter of the connected areas (e.g.,belonging to the same class)
M
M M
M
M
M
M
M0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
M
M M
M
M
M
M
M0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,60,4
0,60,4
0,80,2
0,5 0,3 0,2
0,90,1
0,90,1
0,70,3
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
0,60,4
0,80,2
0,90,1
0,90,1
0,70,3
0,90,1
0,90,1
0,90,1
0,90,1
Cost function not optimized Minimum cost function
Jocelyn Chanussot Gipsa-Lab 28 / 21
A new approach to classification Experiments Conclusions
Experiment on real data
AVIRIS Indian Pine data set• (145×145 pixels, 220 bands), 16 classes of interest• Spatial resolution of the original data degraded of a factor 2• 10% of the labelled samples used as training set
AVIRIS Hekla data set• (180×180 pixels, 157 bands), 9 classes of interest• Spatial resolution of the original data degraded of a factor 2• 15% of the labelled samples used as training set
Comparison with SVM 1vs1, RBF kernel
20 40 60 80 100 120 140
20
40
60
80
100
120
140
20 40 60 80 100 120 140 160 180
20
40
60
80
100
120
140
160
180
Indian Pine GT Hekla GT
Jocelyn Chanussot Gipsa-Lab 29 / 21
A new approach to classification Experiments Conclusions
Evaluation of the results
Jocelyn Chanussot Gipsa-Lab 30 / 21
A new approach to classification Experiments Conclusions
Evaluation of the results
Jocelyn Chanussot Gipsa-Lab 30 / 21
A new approach to classification Experiments Conclusions
Evaluation of the results
Jocelyn Chanussot Gipsa-Lab 30 / 21
A new approach to classification Experiments Conclusions
AVIRIS Indian Pine
20 40 60 80 100 120 140
20
40
60
80
100
120
140
10 20 30 40 50 60 70
10
20
30
40
50
60
70
Ground truth SVM map (OA = 72.31%)
20 40 60 80 100 120 140
20
40
60
80
100
120
140
20 40 60 80 100 120 140
20
40
60
80
100
120
140
Proposed, before SA (OA = 89.82%) Proposed, final map (OA = 91.10%)
Jocelyn Chanussot Gipsa-Lab 31 / 21
A new approach to classification Experiments Conclusions
AVIRIS Hekla
10 20 30 40 50 60 70 80 90
10
20
30
40
50
60
70
80
90
10 20 30 40 50 60 70 80 90
10
20
30
40
50
60
70
80
90
Low res. GT SVM map (OA = 69.19%)
20 40 60 80 100 120 140 160 180
20
40
60
80
100
120
140
160
18020 40 60 80 100 120 140 160 180
20
40
60
80
100
120
140
160
180
Proposed, before SA (OA = 78.90%) Proposed, final map (OA = 81.71%)
Jocelyn Chanussot Gipsa-Lab 32 / 21
A new approach to classification Experiments Conclusions
A robust method
0.6 0.65 0.7 0.75 0.8
70
75
80
85
90
Treshold Pure Pixels
Ove
rall
Acc
urac
y (%
)
AVIRIS Indian Pine (Complete)
Traditional SVM
Proposed Method
5 10 15 20
70
75
80
85
90
Number of ’candidates endmember’
Ove
rall
Acc
urac
y (%
)
AVIRIS Indian Pine (full data set)
Proposed method
Traditional SVM
0.6 0.65 0.7 0.75 0.868
70
72
74
76
78
80
82
Treshold Pure Pixels
Ove
rall
Acc
urac
y (%
)
AVIRIS Hekla
Proposed Method
Traditional SVM
5 10 15 2068
70
72
74
76
78
80
82
Number of ’candidates endmember’
Ove
rall
Acc
urac
y (%
)
AVIRIS Hekla
Jocelyn Chanussot Gipsa-Lab 33 / 21
A new approach to classification Experiments Conclusions
Conclusions and Perspectives
New method to improve spatial resolution of thematic maps:
• Spectral Unmixing considered to handle mixed pixels and abundances determination• Simulated Annealing proposed for spatial regularization
Better definition of spatial structures with respect to full pixel classifiers when theimage contains mixed pixels
Large quantitative improvement
Jocelyn Chanussot Gipsa-Lab 34 / 21