Progress Meeting M3A Presentation of TD3
Post on 30-Dec-2015
41 Views
Preview:
DESCRIPTION
Transcript
IIM-TS M3A – ESRIN 12 December 2007
Selection Procedure
Prototype selection is based on a 3-step procedure
Qualitative pre-screeningof algorithms
Quantitative evaluationof algorithms
Final ranking andprototype selection
TD2
Selected prototype
IIM-TS M3A – ESRIN 12 December 2007
Selection Criteria
8 classes of parameters are considered:
1. Scientific Background and Technical Soundness
The selected algorithms should be based on a solid theoretical background that guarantees the accuracy of its results also at an operational level. The guidelines for rating are as follows:
• The methodology is solid;
• The methodology is technical convincing;
• The methodology is at the state-of-the-art;
• The methodology is published in high quality journals;
• The methodology is included in several other scientific publications or project technical reports.
IIM-TS M3A – ESRIN 12 December 2007
Selection Criteria
2. Robustness and Generality
• The method is suitable to be used with different kind of images;
• The method shows high performances on different images and different test areas;
• There are software implementations or examples for the implementation available;
• The algorithm can be used in combination with other methodologies.
IIM-TS M3A – ESRIN 12 December 2007
Selection Criteria
3. Novelty
In order to get a high score, an algorithm should have been published or reported for the first time relatively recently in the literature. The guidelines for rating the novelty are:
• The publications are after 2003 and introduce a novel, convincing and adequately tested solution to an existing problem;
• The publications in remote sensing are after 1998;
• The method is not implemented in commercial SW packages.
IIM-TS M3A – ESRIN 12 December 2007
Selection Criteria
4. Operational Requirements
Operational requirements are evaluated in terms of computational complexity, time effort, cost etc. The guidelines for rating of operational perspectives are as follows:
• The requested modifications to KIM architecture are few;
• The algorithm works fast (e.g., near real time);
• The processing time scaling is likely to be linear with image size;
• The hardware and disk-storage requirements are low.
IIM-TS M3A – ESRIN 12 December 2007
Selection Criteria
5. Accuracy
Both absolute and relative accuracy in all operative conditions will be evaluated. The guidelines for rating the accuracy are:
• The algorithm matches the end-user requirements and can be optimized according to them;
• The accuracy does not depend on the availability/amount of prior information.
IIM-TS M3A – ESRIN 12 December 2007
Selection Criteria
6. Range of Applications
The number and kinds of applications that an algorithm can address is evaluated:
• The algorithm is suitable for a high number of application areas;
• The algorithm has a high number of estimated final users for the application areas;
• The algorithm has a high impact on the considered application areas.
IIM-TS M3A – ESRIN 12 December 2007
Selection Criteria
7. Level of Automation
From an operational point of view, it is preferable that an algorithm is able to run in a completely automatic way. The main guidelines for rating of the perspectives for automation are:
• The number of parameters to be defined by the operator is low;
• The physical meaning of parameters is clear;
• The method is automatic;
• Ground truth or prior information is not requested.
IIM-TS M3A – ESRIN 12 December 2007
Selection Criteria
8. Specific end-users requirements
From an operational point of view, capability of an algorithm to satisfy and meet different possible end-users requirements is an important parameter of evaluation. The main guidelines for driving this ranking are:
• The algorithm is flexible in meeting different possible accuracy requirements;
• The algorithm can be reasonably included in an operational procedure.
IIM-TS M3A – ESRIN 12 December 2007
Step 1: Qualitative pre-screening of algorithms
• A pre-screening of the algorithms identified and described in TD2 is carried out in order to identify the most relevant methodologies with respect to the IIM-TS project objectives.
• The preliminary qualitative evaluation is driven from the same selection criteria used also in the next quantitative steps. In this step a high level analysis of these criteria is conducted in order to identify techniques that clearly cannot reach a satisfactory ranking on several categories of parameters.
• These techniques are discarded and not further considered in the next steps.
Selection Procedure
IIM-TS M3A – ESRIN 12 December 2007
Pre-screening of algorithmsBinary Change Detection
Binary ChangeDetection Technique
Detection Algorithm
Reference
Change vector Analysis (CVA)Image Differencing (ID)
Vegetation Index Differencing (VID)
Principal Component Analysis (PCA)
Empirical thresholding
Singh (1989) Townshend et al. (1995)
Fung et al. (1990) Muchoney (1994)Fung (1987)
Thresholding based on the Bayes decision
theory
Bruzzone et al. (2000, 2002)Kittler et al. (1986)
Fuzzy thresholdingPal et al. (2000, 2001) Di Zenzo
(1998)
Context-based approaches
Bruzzone et al. (2000) Ghosh et al. (2007)
MultiscaleBovolo et al. (2005) Inglada et al.
(2007)
Reduction registration noise
Bruzzone et al. (1997, 2003)
Multispectral data
IIM-TS M3A – ESRIN 12 December 2007
Pre-screening of algorithmsBinary Change Detection
Binary ChangeDetection Technique
Detection Algorithm ReferenceKind of Data
Image Rationing (IR)
Empirical thresholdingSingh (1989) Rignot et al. (1992)
Cihlar et al. (1993)
SAR
Thresholding based onthe Bayes decision theory
Bazi et al. (2004, 2005, 2006)
Fuzzy thresholdingPal et al. (2000, 2001)
Di Zenzo (1998)
Context-based approaches Bazi et al. (2005)
MultiscaleBovolo et al. (2005) Inglada et al.
(2007)
Multivariate Alteration Detection
Nielsen et al. (1997, 1998)
Correlation coefficientContrast Ratio
Ellipticity
Empirical thresholdingDierking et al (2000, 2002)
Kersten et al. (2005)Polarimetric
SAR dataTest Statistics Conradsen et al. (2003)
Context based Molinier et al. (2007)
SAR and Polarimetric SAR data
IIM-TS M3A – ESRIN 12 December 2007
Pre-screening of algorithmsBinary Change Detection
Binary ChangeDetection Technique
DetectionAlgorithm
ReferenceKind of Data
Kullback Leibler distance (KLD)Normalized information distance
(NID)Mutual Information (I)
Variational Information (VI)Mixed Information (MI)
Single scaleInglada et al. (2003) Meila
(2003)Gueguen et al. Multispectral
and SARMultiscale Inglada et al. (2007)
Multimodal Datcu et al.
Feature and area based techniques
Thresholdingand refinement
Dell’Acqua et al. (2004, 2006)Della Ventura et al. (1990)
Multispectral and SAR
Multivariate Alteration Detection (MAD)Nielsen et al. (1997, 1998)
Liao et al. (2005)
Multispectral, SAR
multisensor
Multisensor techniques
Consensus theory Bruzzone et al. (2000)Multispectral
and SARMultivariate Alteration Detection
Nielsen et al. (1997, 1998)Liao et al. (2005)
Multisensor data
IIM-TS M3A – ESRIN 12 December 2007
Pre-screening of algorithmsMulticlass Change Detection
Multiclass Change Detection Technique
ReferenceKind of Data
Supervised Post-classification ComparisonSingh(1989) Howarth et al. (1981)Hall et al. (1991) Xu et al. (1990)
MultispectralSAR
Multisensor
Supervised Direct-Multidate ClassificationSingh (1989) Schowengerdt (1983)Hall et al. (1991) Burns et al. (1981)
Supervised Compound ClassificationBruzzone et al. (2001)
Serpico et al.
Partially Supervised approachesBruzzone et al. (1997) Cossu et al.
(2005)Fernández Prieto et al. (2001)
Unsupervised approaches
Bovolo et al. (2007) Byrne et al. (1980)
Richards et al. (1993) Häme et al. (1998) Henry et al. (2006) Multispectral
SAR
Multisensor techniquesBruzzone et al. (1999) Baraldi et al.
(2006)Macrì Pellizzeri et al. (2003)
IIM-TS M3A – ESRIN 12 December 2007
Pre-screening of algorithmsShape Change Detection
Shape Change Detection Technique
ReferenceKind of Data
Shape Measures ComparisonsLi et al. (2003) Gamba et al.
(2007)Guindon (1997) Multispectral,
SARand
Multisensor
Differential Snakes Agouris et al. (2001)
Polygon Detection and UpdatingBarr et al. (1997) Yang et al.
(2001)Masek et al. (2000)
IIM-TS M3A – ESRIN 12 December 2007
Pre-screening of algorithmsTrend Analysis of Temporal Series of Images
Trend Analysis Technique Reference Kind of data
Principal component analysisEastman et al. (1993) Hall-Beyer
(2001)Rigina et al. (2003)
Single series of Multispectral
or SAR images
Kalman filtering Joyce et al. (2001)
Regression techniques
Hansen et al. (2001) DeFries et al. (1997)
Engle et al. (1987) Johansen et al. (1995)
NDVI
Hayes et al. (2001) Wilson et al. (2002)
Rigina et al. (2003) Nemani et al. (2003)
Fuller (1998)
Neural networksBrivio et al. (2001) Bruzzone et al.
(2004)Townshend et al. (2001)
Long-term Compositing Techniques
Holben (2001)
Satellite linear-based index Rauste et al. (2007)
Fourier and Wavelet AnalysisAzzali et al. (2000) Anyambe et al.
(1996Li et al. (2000) Andres et al. (1994)
Pixel-based techniques
IIM-TS M3A – ESRIN 12 December 2007
Pre-screening of algorithmsTrend Analysis of Temporal Series of Images
Trend Analysis Technique Reference Kind of data
Integration of information from neighboring pixels and time series
transition probabilitiesBoucher et al. (2006) Single series of
Multispectral or SAR imagesThree-dimensional (or spatio-
temporal) clusteringYamamoto et al. (2001)
Heas et al. (2005)
Maximum-likelihood detectors Lombardo et al. (2002)Single series of
SAR images
Global vegetation model McCloy et al. (2004)Pairs of
Multispectral time series
Statistical analysis of areas of interest (by GIS or object analysis)
Hazel (2001) Gamba et al. (2007)
Turker et al. (2003)
Single series of Multispectral or
SAR images
Context-based techniques
IIM-TS M3A – ESRIN 12 December 2007
Pre-screening of algorithmsPre-processing Multispectral Data
Kind ofPre-
processing
Properties
ReferenceAlgorithmPrinciple
Co-registration Semi-manualGoshtasby (1988) Ton et al.(1989)
Brown (1992) Li et al. (1992)CP or structures
matching
Atmosphericcorrections
AbsoluteHäme (1991) Olsson (1995) Conese et al. (1993)
Richter(1997) Finlayson et al. (2003)Meaningful physical units identification
Relative(empirical)
Coppin et al. (1994) Hall et al. (1991) Eckhardt et al. (1990) Peddle et al. (2003) Tokola et al.
(1999)Pixel-based
Cloudsdetection
Automatic orsemi-
automatic
McIntire et al. (2002) Di Vittorio et al. (2002) Spatial coherence
Di Vittorio et al. (2002)Adaptive
thresholding
Murtagh et al. (2003) Bayesian methods
McIntire et al. (2002) Tian et al. (1999)Arriaza et al. (2003)
Neural networks
Pan-sharpening Automatic
Garzelli et al. (2006) Injection
Chavez et al. (1991) High Pass filtering
Garguet-Duport et al. (1996) Yocky et al. (1996) Zhou et al. (1998) Aiazzi et al. (1999, 2000)
Alparone et al. (1998)Multiresolution
Tu et al. (2001, 2004) HIS
Aiazzi et al. (2006) MTF
IIM-TS M3A – ESRIN 12 December 2007
Pre-screening of algorithmsPre-processing SAR Data
Kind ofPre-processing
Properties ReferenceAlgorithmPrinciple
Image focusing Automatic
Cafforio et al. (1991)
Curlander et al. (1991) Range doppler
Raney et al. (1994) Chirp scaling
Co-registration AutomaticForoosh et al. (2002) Phase Correlation
Stone et al. (2001) Fourier transform
Ortho-rectification and georeferencing
Sensor dependent
(parametric)
Hemmleb et al. (1997)Novak (1992) Meier et al. (1993) Range-Doppler
Camera ModelGCP matchingSensor
independent(non-parametric)
Rosenholm et al. (1998)Tao et al. (2001) Zhou et al. (2005)
Atmospheric corrections
AutomaticRauste et al. (2007) DEM
Datcu et al. (in press) Bayesian theory
Semi-automaticDe Grandi et al. (2003) Rauste et al.
(1999)Correlation
Manual De Grandi et al. (2004) GCP
IIM-TS M3A – ESRIN 12 December 2007
Pre-screening of algorithmsPre-processing SAR Data
Kind ofPre-processing
Properties ReferenceAlgorithmPrinciple
Radiometric calibration and normalization
Automatic
Holecz et al. (1994) Radar Equation
Ulander (1996)Normalization on pixel
surface
Small et al. (2004) Normalization on pixel area
MosaickingAfek et al. (1998) Du et al. (2001)
Guindon (1995, 1996, 1997)Linear regression
Filteringand segmentation
Automatic
Richards et al. (1999)Low-pass or edge-
preservingimage-smoothing filters
Frost et al. (1982) Lee (1980)Kuan et al. (1985) Solbø et al. (2004)
Lopes et al. (1990, 1993)Trouvé et al. (2003) Aspert et al.
(2007)
Adaptive despeckling procedures
Multitemporal filtering
Bruniquel et al. (1997) Ciuc et al. (2001)
Coltuc et al. (2000) Perona et al. (1990)
IIM-TS M3A – ESRIN 12 December 2007
Pre-screening of algorithmsPre-processing Multisensor Data
Kind of Pre-processing
Properties
ReferenceAlgorithmPrinciple
Mosaicking Automatic
Afek et al. (1998) Du et al. (2001)
Guindon (1995, 1996, 1997)Linear regression
Du et al. (2001)Last, mean, Feathering
Gradient
Co-registration Automatic
Brown (1992)Zitová et al. (2003)
Similarity measures and geometric transformation
Thépaut (1998) Orbit information
Wu et al. (1990)Djamdji et al. (1993)
Multiresolution
Ventura et al. (1990)Dai et al. (1999) Ali et al.
(2002)Feature based
Inglada et al. (2004) Multisensor
IIM-TS M3A – ESRIN 12 December 2007
Selection Procedure
Step 2: Quantitative evaluation of algorithms
• Algorithms that pass the pre-screening in step 1 are analyzed in greater detail with a quantitative evaluation.
• This analysis is based on different parameters (scientific and technical analysis, possible impacts on the application and end-users, etc).
• For each algorithm (or cluster of algorithms) a method sheet is filled in, which reports details of the algorithm and individual scores for each parameter considered.
IIM-TS M3A – ESRIN 12 December 2007
Method Sheets Organization
Method name Reference to TD2
Method category Applications
Method description
Main scientific papersCitations
Citations/year
Data type
Optical Radar Multisensor
LR MR VHR
Method suitable toSingle image Multitemporal
imagesData fusion
Existing software
Pre-processing reqs Comment X/S
Geometric correction
Radiometric correction
Other pre-processing
Remarks
Algorithm characteristics
IIM-TS M3A – ESRIN 12 December 2007
Method Sheets Organization
Technical considerations
Evaluation Criteria Y/N
Scientific background and technical soundness
The methodology is solid
The methodology is technical convincing
The methodology is at the state-of-the-art
The methodology is published in high quality journals
The methodology is included in several other scientific publications or project technical reports
Robustness and generality
The method is suitable to be used in different research and application environments
The method is suitable to be used with different kind of images
The method shows high performances on different images and different test areas
There are software implementations or examples for the implementation available
The algorithm can be used in combination with other methodologies
Novelty
The publications are after 2003 and introduce a novel, convincing and adequately tested solution to an existing problem
The publication in remote sensing are after 1998
The method is not implemented in commercial SW packages
Evaluation
IIM-TS M3A – ESRIN 12 December 2007
Method Sheets Organization
Technical considerations
Evaluation Criteria Y/N
Operational requirements
The requested modifications to KIM architecture are few
The algorithm works fast (e.g., near real time)
The processing time scaling is likely to be linear with image size
The hardware and disk-storage requirements are low
Accuracy
The algorithm matches the end-user requirements and can be optimized according to them
The accuracy does not depend on the availability/amount of prior information
Range of applications
The algorithm is suitable for a high number of application areas
The algorithm has a high number of estimated final users for the application areas
The algorithm has a high impact on the considered application areas
Level of automation
The number of parameters to be defined by the operator is low
The physical meaning of parameters is clear
The method is automatic
Ground truth or prior information is not requested
Specific end-users requirements
The algorithm is flexible in meeting different possible accuracy requirements
The algorithm can be reasonably included in an operational procedure
Evaluation
IIM-TS M3A – ESRIN 12 December 2007
Selection Procedure
Step 3: Final ranking and prototype selection
• According to an analysis of methods sheets a final score is given to each algorithm and method.
• This value is used for ranking algorithms according to their relevance with respect to IIM-TS objectives;
• The algorithms to be prototyped are identified on the basis of the score and of a final discussion of the ranking.
IIM-TS M3A – ESRIN 12 December 2007
Total score computation
• 1 point is given to each considered class of parameters for each positive answer in the corresponding category of the method sheet. Then the category score is normalized.
• Few points are assigned to each method according to the number of citations per year of the algorithms in scientific papers (or in technical reports) following this table:
Citations/year Points
0 0
1-4 1
5-8 2
8-12 3
> 12 4
Total Score Computation
IIM-TS M3A – ESRIN 12 December 2007
Total Score Computation
The score achieved for each single class is properly weighted in order to take into account its relevance with respect to the goals of the project. The following equation is used:
Total value = w1 * “Scientific Background and Technical Soundness”+ w2 * “Robustness and Generality”+ w3 * “Novelty”+ w4 * “Operational Requirements”+ w5 * “Accuracy”+ w6 * “Range of Applications”+ w7 * “Level of Automation”+ w8 * “Specific End-users Requirements”+ w9 * Citation scoreThe final score indicates the relevance of the method with respect to the prototyping
procedure within IIM-TS project.
IIM-TS M3A – ESRIN 12 December 2007
Total Score Computation
Selection criteriaWeight variable
Weight value
Scientific Background and Technical Soundness
w1 5
Robustness and Generality w2 5
Novelty w3 3
Operational Requirements w4 4
Accuracy w5 2
Range of Applications w6 3
Level of Automation w7 4
Specific End-users Requirements w8 2
citation score w9 4
wn (n = 1,…9) is the weight assigned to the n-th category of criteria, and represents the relative relevance of the considered criterion with respect to the others:
IIM-TS M3A – ESRIN 12 December 2007
Table of Ranking
Method Category(main)
Method Name
ScientificBackground
and Technical
Soundness
Robustness
andGeneralit
y
NoveltyOperation
alReq.
Accuracy
RangeOf
Appl.
LevelOf
Autom.
SpecificEnd-usersReq.
Citation
score Total
score
5 5 3 4 2 3 4 2 4
Binary CDUnsupervised Bayesian
framework to CD5 5 3 4 2 3 4 2 4 32
Binary CD & pre-processing
Kullback-Leibler divergencefirst order
4 5 3 4 2 3 3.5 1 1 28.5
Binary CD MAD+MAF or MNF 5 5 1 4 2 3 4 1 2 27Binary CD Pre-
processingImage split 4 5 3 4 2 3 4 2 0 27
SAR pre-processing, CD & Trend Analysis
SAR preprocessing and multisensor rule-based
classifier4 5 2 4 2 3 4 1 1 26
Trend analysisFourier and Wavelet
Analysis5 3 1 4 2 3 4 1 2 25
Multi-class CDAutochange change
detectionand identification
5 5 2 4 2 3 2 1 1 25
Trend analysisHot spot monitoring via
GIS fusion4 5 3 4 2 3 2 1 0 24
Trend analysis Spatio-temporal clustering 4 4.5 3 2 2 3 4 1 1 24.5
Multi-class CDDirect Multidate
Classification4 5 1 3.5 1 3 2 1 4 24.5
Shape CD Shape change index 3 4 2.5 4 2 3 3.5 1 0 23
Multi-class CDPost-classification
comparison4 4 0 4 1 3 2 1 4 23
Trend analysisClassification of
long temporal series4 5 3 3 1 3 2 1 1 23
IIM-TS M3A – ESRIN 12 December 2007
Table of Ranking
Method Category
(main)Method Name
ScientificBackground
and Technical
Soundness
Robustness
andGeneralit
y
NoveltyOperation
alReq.
Accuracy
RangeOf
Appl.
LevelOf
Autom.
SpecificEnd-usersReq.
Citation
score Total
score
5 5 3 4 2 3 4 2 4
Binary CD & pre-processing
Multi-modal change map generation
3 4 2 4 2 3 4 1 0 23
Binary CD & Pre-processing
Mixed Information Measure (comparison & co-
registration)3 4 2 4 2 3 4 1 0 23
Binary CD & pre-processing
The similarity metric based on Kolmogorov complexity
(comparison & co-registration)
3 3.5 2 4 2 3 4 1 0 22.5
Multi-class CD Compound Classification 5 5 1 3 1 2 2 1 2 22Binary CD & pre-
processingSAR polarimetric change
indices3 4 2 4 2 3 2 1 1 22
Binary CD & pre-processing
Kullback-Leibler divergence second order (comparison &
co-registration)3 4 2 4 1 3 4 1 0 22
Trend analysisPhenological changemonitoring (autumn
coloration)3 3 3 4 2 2 3 1 0 21
IIM-TS M3A – ESRIN 12 December 2007
Design of the Architecture
• The selection of the prototype algorithms among those with the highest scores in the ranking should be finalized taking into account the possible synergy between different techniques.
• The final selection should be also based on an adequate balancing among techniques belonging to the different classes.
IIM-TS M3A – ESRIN 12 December 2007
Design of the Architecture Raw SAR images
FocusingGeometric corrections
Radiometric correctionsRadiometric normalization
Mutitemporal filteringMosaiking Segmentation
Time varying segmentationPre
-pro
ce
ss
ing Registration
Ortho-rectificationMosaiking
Radiometric correctionsCloud detection
Topographic correctionsPan-sharpeningImage filtering
Feature extraction
Raw optical images
Pre
-pro
ce
ss
ing
DifferenceRatio
Information theoretically similatityMeasures (KL divergence, etc.)
Polarimetric change indeces(correlation, etc.)
Thresholding based on the Bayes decision theory
Context-based approachesMultiscale approachesMultimodal approaches
Multivatiarte Alteration Detection
Binary Change Detection
Post-classification ComparisonDirect-Multidate Classification
Compound ClassificationUnsupervised approaches
Multisensor techniques
Multiclass Change Detection
Shape MeasuresComparisons
Shape Change Detection
Neural networksSatellite linear-based index
Fourier and Wavelet AnalysisSpatio-temporal clustering
Statistical analysis of areas of interest(by GIS or object analysis)
Trend Analysis
IIM-TS M3A – ESRIN 12 December 2007
Design of the Architecture
• Pre-processing chain for multispectral images (geometric corrections and radiometric corrections)
• Pre-processing chain for SAR data (geometric corrections and radiometric corrections)
• Binary change detection:• Set of measures for image comparison (difference, magnitude of
the difference vector, ratio, log-ratio, KL, similarity measures)• Image splitting• Bayesian framework for the analysis of the results of the
comparison (minim error and cost decision rules, Gaussian model, Generalize Gaussian model (?), MRF context-sensitive decision, manual or automatic initialization?)
IIM-TS M3A – ESRIN 12 December 2007
Design of the Architecture
• Multiclass change detection:• Unsupervised method based on autochange algorithm• Supervised methods based on MDC and PCC (need for a
distribution-free classification module)• Rule based multisensor classifier
• Trend analysis of time series:• Spatio-temporal clustering (data mining)• Tools for FT and WT• Hot spot monitoring via GIS fusion
• Shape change detection measure
top related