1 Information Extraction Principles for Hyperspectral Data David Landgrebe Professor of Electrical & Computer Engineering Purdue University [email protected]• A Historical Perspective • Data and Analysis Factors • Hyperspectral Data Characteristics • Examples • Summary of Key Factors Outline
37
Embed
1 Information Extraction Principles for Hyperspectral Data David Landgrebe Professor of Electrical & Computer Engineering Purdue University [email protected].
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Information Extraction Principles for Hyperspectral Data
David LandgrebeProfessor of Electrical & Computer Engineering
• Matched Filter - Constrained Energy Minimization
gi(X) XTCb
1i iTCb
1 i• Other types - “Nonparametric”
Parzen Window Estimators Fuzzy Set - based Neural Network implementations K Nearest Neighbor - K-NN etc.
18
Covariance Coefficients to be Estimated
• Assume a 5 class problem in 6 dimensions
• Normal maximum likelihood - estimate coefficients a and b• Ignore correlation between bands - estimate coefficients b
• Ignore correlation between bands - estimate coefficients d
Class 1 Class 2 Class 3 Class 4 Class 5b b b b ba b a b a b a b a ba a b a a b a a b a a b a a ba a a b a a a b a a a b a a a b a a a ba a a a b a a a a b a a a a b a a a a b a a a a ba a a a a b a a a a a b a a a a a b a a a a a b a a a a a b
• Assume common covariance - estimate coefficients c and d
Common Covar.dc dc c dc c c dc c c c dc c c c c d
19
EXAMPLE SOURCES OFCLASSIFICATION ERROR
Decision boundary defined by the
diagonal covariance classifier
class 2
class 1
Decision boundary defined by Gaussian ML classifier
Borsuk’s Conjecture: If you break a stick in two, both pieces are shorter than the original.
Keller’s Conjecture: It is possible to use cubes (hypercubes) of equal size to fill an n-dimensional space, leaving no overlaps nor underlaps.
Science, Vol. 259, 1 Jan 1993, pp 26-27
Counter-examples to both have been found for higher dimensional spaces.
22
The Geometry of High Dimensional Space
The Volume of a Hypercube concentrates in the corners
0.6
1 2 3 4 5 6 70
0.2
0.4
0.8
1
dimension d
The Volume of a Hypersphereconcentrates in the outer shell
1 2 3 4 5 6 7 8 9 10 110
0.2
0.4
0.6
0.8
1
dimension d
Vd (r ) Vd (r )
Vd (r)rd (r )d
rd1 1
r
d
d 1
V hypersphere
Vhypercube
d
2
d2d 1 d2
d 0
23
Some Implications
High dimensional space is mostly empty. Data in high dimensional space is mostly in a lower dimensional structure.
Normally distributed data will have a tendency to concentrate in the tails; Uniformly distributed data will concentrate in the corners.
24
Volume of a hypersphere =2rd
dd / 2
(d / 2)
How can that be?
dVdr
2d / 2
(d / 2)r (d 1)
Differential Volume at r =
0 1 2 3 4 50
20
40
60
80
Distance from Class Mean, r
1
2
3 4 5
Surface of Hypersphere
Volumn of shell
25
How can that be? (continued)
rd 1e r
2
2
2d2 1d2
The Probability Mass at r =
0 1 2 3 4 50
0.2
0.4
0.6
0.8
Distance from Class Mean, r
1
2 3 4 5 10 15 20
Probability Density of Distance r
Probability mass in shell
26
MORE ON GEOMETRY
• The diagonals in high dimensional spaces become nearly orthogonal to all coordinate axes
cos d 1d
Implication: The projectionof any cluster onto anydiagonal, e.g., by averagingfeatures could destroy information
27
STILL MORE GEOMETRY
• The number of labeled samples needed for supervised classification increases rapidly with dimensionality
In a specific instance, it has been shown that the samples required for a linear classifier increases linearly, as the square for a quadratic classifier. It has been estimated that the number increases exponentially for a non-parametric classifier.
• For most high dimensional data sets, lower dimensional linear projections tend to be normal or a combination of normals.
28
A HYPERSPECTRAL DATA ANALYSIS SCHEME
200 Dimensional Data
Class ConditionalFeature Extraction
FeatureSelection
Classifier/Analyzer
Class-SpecificInformation
29
Finding Optimal Feature Subspaces
• Feature Selection (FS)
• Discriminant Analysis Feature Extraction (DAFE)
• Decision Boundary Feature Extraction (DBFE)
• Projection Pursuit (PP)
.Available in MultiSpec via WWW at: http://dynamo.ecn.purdue.edu/~biehl/MultiSpec/Additional documentation via WWW at: http://dynamo.ecn.purdue.edu/~landgreb/publications.html
30
Hyperspectral Image of DC Mall
HYDICE Airborne System1208 Scan Lines, 307 Pixels/Scan Line210 Spectral Bands in 0.4-2.4 µm Region155 Megabytes of Data(Not yet Geometrically Corrected)
31
Define Desired Classes
Training areas designated by polygons outlined in white
32
Thematic Map of DC Mall
Legend Operation CPU Time (sec.) Analyst TimeDisplay Image 18Define Classes < 20 min.Feature Extraction 12Reformat 67Initial Classification 34Inspect and Mod. Training ≈ 5 min.Final Classification 33
Total 164 sec = 2.7 min. ≈ 25 min.
Roofs
Streets
Grass
Trees
Paths
Water
Shadows
(No preprocessing involved)
33
Hyperspectral Potential - Simply Stated
• Assume 10 bit data in a 100 dimensional space.• That is (1024)100 ≈ 10300 discrete locations
Even for a data set of 106 pixels, the probability
of any two pixels lying in the same discrete location
is vanishingly small.
34
Summary - Limiting Factors
PreprocessingData
AnalysisInformationUtilization
Human Participationwith Ancillary Data
Sensor On-BoardProcessing
Ephemeris,Calibration, etc. • Scene - The most complex
and dynamic part
• Sensor - Also not under analyst’s control
• Processing System - Analyst’s choices
35
Limiting Factors
Scene - Varies from hour to hour and sq. km to sq. km
Sensor - Spatial Resolution, Spectral bands, S/N
Processing System -
• Classes to be labeled
• Number of samples to define the classes
• Complexity of the Classifier
• Features to be used
- Exhaustive,
- Separable,- Informational Value,
36
Source of Ancillary Input
Possibilities
• Ground Observations
• “Imaging Spectroscopy”
- From the Ground
- Of the Ground
• Previously Gather Spectra
• “End Members”
Image Space
Spectral Space
Feature Space
.
37
Use of Ancillary Input
A Key Point:
• Ancillary input is used to label training samples.
• Training samples are then used to compute class quantitative descriptions
Result:
• This reduces or eliminates the need for many types of preprocessing by normalizing out the difference between class descriptions and the data