Motivation
Necessity of combining physical models and statistical techniques in unmixing
Methods of integration of scientific models & statistical algorithms not always obvious – Enable machine learning techniques to produce scientifically
meaningful results in unsupervised settings
Meaningful information not always readily accessible or easily readable – Representation = Simplification
Outline
Background – Mineral spectral signatures in VNIR
Meaningful features for mineral identification Data challenges for planetary data Data processing pipeline Validation: expert assessment Comparison with state-of-the-art Conclusions and future work
Mineral spectral signatures Each mineral has
a distinct spectral shape (signature)
Discriminative information mostly in absorption band positions and shapes
Difference can be subtle
Can create parameters for discrimination
Mineral spectral signatures Each mineral has
a distinct spectral shape (signature)
Discriminative information mostly in absorption band positions and shapes
Difference can be subtle
Can create parameters for discrimination
Spectral features for minerals
Use splines Select knots so that
Spectral features for minerals
Use splines Select knots so that
– Reconstruction insensitive to artifacts
Spectral features for minerals
Use splines Select knots so that
– Reconstruction insensitive to artifacts
– Reconstruction with higher sensitivity in diagnostic areas
Spectral features for minerals
Use splines Select knots so that
– Reconstruction insensitive to artifacts
– Reconstruction with higher sensitivity in diagnostic areas
– Reconstruction sharper (green) for vibrational bands and smoother (red) for electronic transition bands
B-spline coefficients as feature vector
hyperspectral data challenges
Cloud has curved boundaries and is non convex
Some dimensions uninformative
No apparent clusters, high density
Noise creates outliers Most unique spectra =
extreme points or “corners” or “image endmembers”
Objectives for spectral unmixing
Separation of spectral pixels in families (image segmentation)
Sensitivity to subtle changes in spectral absorption positions and shapes (mineral sub-families)
Sensitivity to small (spatial) outcrops Robustness with respect to noise Useful visualization
Pipeline
Dimensionality reduction
Clustering
Unmixing
Pruning
43
1
2
1 2
4
3
1 2
4 3
5
1 2
34
1 2 21
5 1 1
Preprocessing
Pipeline
Dimensionality reduction
Clustering
Unmixing
Pruning
43
1
2
1 2
4
3
1 2
4 3
5
1 2
34
1 2 21
5 1 1
Preprocessing
Operation modes
Two capabilities: – Select areas based on parameter maps (user version) – Divide the image in sections (pipeline version)
Operate on each area independently
50 pixels
100 pixels
Pipeline
Dimensionality reduction
Clustering
Unmixing
Pruning
43
1
2
1 2
4
3
1 2
4 3
5
1 2
34
1 2 21
5 1 1
Preprocessing
Dimensionality reduction: issues
Intrinsic dimensionality of data is low: benefit from dimensionality reduction.
From movie: – Need nonlinear transform. – Need to preserve local geometry – Need to highlight natural clusters
Reduce dimensionality to 2 – 3 for visualization
Dimensionality reduction High-D Feature Space Low-D Space
x1x2
x3y3
y2y1
ynxn
x1, . . . , xn (known) as vertices of a graph
y1, . . . , yn (unknown) as vertices of a graph
Edge weights proportional to spectral dissimilarities and spatial adjacency
Edge weights fixed
Dimensionality reduction High-D Feature Space Low-D Space
x1x2
x3y3
y2y1
ynxn
p12 q12
Small distance = small distance
Dimensionality reduction High-D Feature Space Low-D Space
x1x2
x3y3
y2y1
ynxn
Med/big distance = bigger distance
q13p13
Dimensionality reduction High-D Feature Space Low-D Space
x1x2
x3y3
y2y1
ynxn
P = {pij} Q = {qij}
Correl. Neighbor Embedding (Parente 2011)
Minimize relative entropy
Solve by gradient descent
Variation on t-Stochastic Neighbor Embedding (Van der Maaten et al. 2008)
High-D Space Low-D Space
dij = 1! cij
cij = !ij,spatial · cij,spect
pij =exp(!d2
ij/2!2i )!
k !=l exp(!d2kl/2!
2k )
D = argminyi,yj
!
i,j
pij(xi, xj) logpij(xi, xj)
qij(yi, yj)
!D(P ||Q)
!yi= 4
!
j
"ij(yi ! yj)(pij ! qij)
Pipeline
Dimensionality reduction
Clustering
Unmixing
Pruning
43
1
2
1 2
4
3
1 2
4 3
5
1 2
34
1 2 21
5 1 1
Preprocessing
Graph partitioning as clustering
Cluster points in the transformed space to take advantage of separated sections
The geometry is nonlinear: need clustering on curved structure
Consider the set of vertices of the graph and the edge weights
Clustering is equivalent to partitioning graph into disjoint subsets. – can be done by spectral clustering
because CNE creates several connected components
yi
qij(yi, yj)
Image segmentation
Original image
Segmentation map Clustering = mineral
family mapping = image segmentation
Pipeline
Dimensionality reduction
Clustering
Unmixing
Pruning
43
1
2
1 2
4
3
1 2
4 3
5
1 2
34
1 2 21
5 1 1
Preprocessing
Local endmember detection
The clusters in the original space are roughly convex Locally to a cluster can assume linear mixing approximate the data cloud with a conic or convex
combination of a small number of “endmembers” Have a way to extrapolate endmembers if the data
does not support clear detections
1 2
4
3
Robust Nonneg. Matrix Factorization
k is the number of local endmembers is a robust estimator D imposes smoothness and corrects MNF “problems” Solve with alternating projected gradient Zymnis 2009, Parente 2009, Parente 2011
!
minimize !(Y !WH) + "||DW ||2Fsubject to H " 0, 1T H = 1T
W ! Rm!k, H ! Rk!n
W,
Pipeline
Dimensionality reduction
Clustering
Unmixing
Pruning
43
1
2
1 2
4
3
1 2
4 3
5
1 2
34
1 2 21
5 1 1
Preprocessing
Spectral pruning
Spectrum 1 is any local endmember candidate
Spectrum 2 is either a local endmember or an estimate of the baricenter of the cloud
If the score is higher than a threshold Spectrum 1 is pruned
Cross-correlate feature vectors
Score
Features for spectrum 1
Features for spectrum 2
Validation
Martian image analysis lacks ground truth Simulation of the complete hyperspectral image
formation process (Parente et al. 2010) – Soil mixing, atmosphere, instrument response, noise
Comparison with manual expert assessment – the expert can extract the complete spectral variability (3E12) – the expert can only extract partial spectral variability (94F6) – The expert cannot extract spectral variability
Self-consistency: comparison with state-of the art (partial)
Validation: 3E12
Different mineral families evident from RGB
Low noise Good spectral
variability
Validation: 3E12
Automatically retrieved spectra over the whole scene
Manually selected spectra over the whole scene
Validation: 94F6
R=band 233, G=band 78, B=band 13 R=D2300, G=OLINDEX, B=BD2210
94F6 manual retrieval
Regions of Interest (ROIʼs) Spectra from ROIʼs
94F6
Several more spectral families detected by the algorithm
Letʼs zoom in!
94F6 automated
94F6 automated
2.205 µm
94F6 automated
2.205 µm 2.2913 µm
94F6 automated
2.205 µm 2.2913 µm 2.3046 µm
94F6 automated
2.205 µm 2.2913 µm 2.3046 µm 2.3244 µm
94F6 automated
2.205 µm 2.2913 µm 2.3046 µm 2.3244 µm 2.4038 µm
94F6 automated
2.205 µm 2.2913 µm 2.3046 µm 2.3244 µm 2.4038 µm 2.5229 µm
94F6 automated
2.205 µm 2.2913 µm 2.3046 µm 2.3244 µm 2.4038 µm 2.5229 µm 2.5295 µm
94F6 automated
2.205 µm 2.2913 µm 2.3046 µm 2.3244 µm 2.4038 µm 2.5229 µm 2.5295 µm
Carbonate !!
Some difficult data:199C7
199C7 automated
199C7 automated
2.04 µm 2.29 µm, 2.30 µm,
2.31 µm 2.52 µm, 2.53 µm
Comparison with state of the art
Current unmixing algorithms: – require convexity – developed for earth
environmental conditions are known ground truth is available
– donʼt consider impulsive noise – some require linear assumptions
Nonlinear unmixing not yet mature Not able to discriminate subtle spectral differences
Comparison with other algorithms
The proposed algorithm is insensitive to noise and picks up more surface components
The SMACC algorithm is extremely sensitive to noise
B141 Mawrth Vallis
ENVI SMACC endmembers
Proposed approach
ABCB: Nili Fossae
Endmembers from VCA Proposed
approach
More algorithms
(a) Proposed Algorithm (b) N-FINDR (c) PPI
(d) SMACC (e) SISAL
Conclusions
Presented a novel method for unmixing The algorithm effectively captures the image spectral
variability, down to subtle differences, is robust to noise and outperforms current state-of-the-art algorithms
Can be applied to any hyperspectral dataset Produces segmentation and endmember maps We proposed this technique to the CRISM and M3 teams
as the “official” data summarization tool for their processing pipelines.
Future work
Include a physical unmixing layer: use radiative transfer theory
Provide mechanism to tag “virtual” endmembers Complete validation process with expert feedback
References
L. van deer Maaten and G. Hinton, (2008). Visualizing data using t-SNE, Journal of Machine Learning, 9, pp. 2579-2605.
A. Ng, M. Jordan and Y. Weiss, (2001). On spectral clustering: Analysis and an algorithm, NIPS.
M. Parente , J.T. Clark, A. Brown and J.L. Bishop (2010). End-to-end simulation of the image generation process for CRISM spectrometer data, IEEE Transactions on Geoscience and Remote Sensing.
M. Parente, (2011). Summarization of hyperspectral images: application to Mars, IEEE Transactions on Geoscience and Remote Sensing, (in review).
M. Parente, J. L. Bishop and J. F. Bell III, (2009), Spectral unmixing and anomaly detection for mineral identification in Pancam images of Gusev soils, Icarus, Vol 203, N. 2, p. 421-436.
Questions?
Publications based on project Parente M. and A. Plaza (2010), Survey of geometric and statistical unmixing algorithms for
hyperspectral images, IEEE 2nd WHISPERS (Workshop on hyperspectral image and signal processing: evolution of remote sensing) Conf. June 14-16, Reykjavyk, Iceland (invited keynote presentation for special session on “Geometric vs. statistical unmixing algorithms”).
M. Parente Spectral unmixing using nonnegative basis learning: comparison of geometrical and statistical endmember extraction algorithms. (invited paper) Space Exploration Technologies, edited by Wolfgang Fink Proc. of SPIE Vol. 6960, 69600P, (2008). doi: 10.1117/12.777895
M. Parente Exploratory data analysis of planetary datasets – new development, (invited talk) Jet Propulsion Laboratory, Pasadena CA, December 4 2008.
Parente M., Clark J.T., Brown A.J., and Bishop J.L.. (2009). Simulation of the image generation process for CRISM spectrometer data. IEEE WHISPERS (Workshop on hyperspectral image and signal processing: evolution of remote sensing) Conf. Aug 26-28 Grenoble, France. (Best paper award)
Bishop J. L., Noe Dobrea E. Z., McKeown N. K., Parente M., Ehlmann B. L., Michalski J. R., Milliken R. E., Poulet F., Swayze G. A., Mustard J. F., Murchie S. L., and Bibring J.-., P. (2008) Phyllosilicate diversity and past aqueous activity revealed at Mawrth Vallis, Mars. Science 321, DOI: 10.1126/science.1159699, pp. 830-833.
Parente, M. and J.L. Bishop, (2010). Extracting endmember spectra from CRISM images: comparison of new Direx image transform technique with MNF, Lunar Planet Science Conf, XLI abstr. #2633.
Backup slides
MRO-CRISM: VNIR Spectra Can Characterize Small Deposits on Mars
Examples of surface features at different CRISM spatial resolutions
• Global Mode: 70 channels • Targeted Mode: 544 channels
OMEGA (300-1000 m/pixel, 13 nm/ch.)
discovers large deposits
CRISM multispectral survey (100-200 m/pix, 70 ch.) discovers small
deposits
CRISM targeted hyperspectral (15-38 m/pixel, 6.55 nm/ch)
characterizes deposits
CRISM Noise sources
Both artifacts create spikes in the spectral domain
1. Vertical striping due to miscalibration of pixel sensors (red arrows).
2. Pixels with elevated bias or abnormal dark ("bad" pixels) create stripe segments (cyan)
60/40
Noise removal with CIRRUS
CIRRUS (CRISM Iterative Recognition and Removal of Unwanted Spiking) (Parente 2008)
CIRRUS currently in use in CRISM processing pipeline
Original Cleaned
Original
Cleaned
Comparison with PCA
Proposed approach (3D) PCA (first 3 PCs)
Natural clusters well separated
Between-clusters, different spectra
Within-cluster, similar spectra
Natural clusters not evident
similar points can differ in norm
1st PC illumination gradient
Comparison with other techniques
Proposed approach (3D) PCA (first 3 PCs)
Natural clusters well separated
Between-clusters, different spectra
Within-cluster, similar spectra
Natural clusters not evident
Some endmembers evident
Clustering particularly hard
LLE (3D)
Natural clusters not evident
similar points can differ in norm
1st PC illumination gradient
Graph partitioning as clustering
Graph partitioning as clustering
Graph partitioning as clustering
Graph partitioning as clustering
Clustering for case study
Clustering performance comparison
Original image
Proposed approach
K-means in original space
Hierarchical in original space
Hierarchical in 3-D space
K-means with correlation in original space
K-Eigenvector Clustering 1. Construct matrix of normalized weights Aʼ 2. Decomposition: Find the eigenvectors of Aʼ
corresponding to the k largest eigenvalues. These form the the columns of the new matrix X.
3. Form the matrix Y – Renormalize each of Xʼs rows to have unit length – Y | – Treat each row of Y as a point in
3. Cluster into k clusters via k-means 4. Final Cluster Assignment
– Assign point to cluster j iff row i of Y was assigned to cluster j
k can be found by maximum spread between eigenvalues
(Ng et al. 2001)
Validation This software is undergoing extensive validation
aimed at confirming that the proposed method can be used pervasively and reliably in the summarization of the whole CRISM database.
The validation process starts with requesting from the community image IDʼs with manually selected endmembers.
An automated pipeline is in place that sends back via email the spectra retrieved by the algorithm to each author of manual analysis.
Upon receiving feedback on dissimilarities and quality of the detections the pipeline will calculate validation statistics and will send them to the team for review.
After validation the production stage will begin.
ID Solicitation
Processing
Feedback
Validation statistics