Inversion via Bayesian Adaptive Multi-Modality Processing (AMMP) Leslie Collins Electrical and Computer Engineering Duke University Work supported by DARPA/ARO MURI
Inversion via Bayesian Adaptive Multi-Modality Processing
(AMMP)
Inversion via Bayesian Adaptive Multi-Modality Processing
(AMMP)
Leslie Collins Electrical and Computer Engineering
Duke UniversityWork supported by DARPA/ARO MURI
Leslie Collins Electrical and Computer Engineering
Duke UniversityWork supported by DARPA/ARO MURI
OutlineOutline
• Problem Background and Setup• Adaptive Feature Selection• AMMP
– HSTAMIDS results– NIITEK/GEM-3 results– Georgia Tech results
• Confidence-based Fusion
• Problem Background and Setup• Adaptive Feature Selection• AMMP
– HSTAMIDS results– NIITEK/GEM-3 results– Georgia Tech results
• Confidence-based Fusion
Adaptive Feature Selection
Subsurface Sensing (Landmines, UXO, Subsurface Structures)
• Simulate seismic, EMI, and MAG data• Large number of features available• Pre-screener needed based on RDOF feature set• Issue: best features function of sensor
implementation, test site, etc.• In some scenarios, need to adaptively select
features• Ultimately: AMMP applied to results of pre-
screener
NIITEK Radar Results
• Features selected based on training from test lanes
• Variable performance on off-road scenarios
Manual Feature Selection
RDOF Feature Selection• Guide initial selection of features• Adaptively prune features based on input from
multiple sensors• Carin et al. proposed RDOF for induction sensing
– Induction, mag, seismic still produce many features– JCFO verified by Carin on acoustic data – not yet
applied to EMI/MAG/Seismic data
• In year 2, evaluated JCFO/adaptive feature selection on: – GPR and EMI for landmines– EMI and MAG for UXO– Next year: Simulated data for subsurface structures
JCFO• Classification of a feature vector, x,
performed using kernel-based technique
• For a binary classifier with labels, l, of +1 and –1, training data T and weights w
• A Laplacian sparseness prior is placed on the weights of the training samples xn
01
( ) ( , )N
n nn
c w K w=
= +∑x x x
( )
1( 1/ , , )1
( 1/ , , ) 1 ( 1/ , , )
cp le
p l p l
−= =+
= − = − =
xx T w
x T w x T w
JCFO, Cont.
feature1 feature2 feature3 feature4…… feature17
Object1:
Object2:
feature1 feature2 feature3 feature4…… feature17
… …….Object128:Object128: feature1 feature2 feature3 feature4…… feature17
Theta[1] Theta[2] Theta[3] Theta[4]…… Theta[17]
Beta[1]
Beta[2]
Beta[128]
Associate each training sample with parameter Beta, and each feature with parameter Theta
example: training data of 128 samples, 17 features for each sample
Training Objective : Estimating parameters of Theta and BetaFeature Selection: Sparsity in Theta implies that it finds only a few features highly relevant for classification. (making some Theta[i]=0)
Kernel function selection: Sparsity in Beta implies that it finds a small subset of data highly representative of the different classes (making some Beta[i]=0)
…
Find the maximum a posterior (MAP) estimate of Theta and Beta using EM algorithm
Manual Physics-Based versus Random Feature Selection
(EMI+MAG)
Physics-Based versus JCFO for EMI and MAG (Area 2)
JCFO picks 4 features and used 5 of 128 training vectors!
Physics-Based versus JCFO for EMI and MAG (Area 1)
AMMP
Traditional Versus Collaborative/AMMP Approach
Sensor Algorithm Threshold
Sensor Algorithm Threshold
Data or Feature Fusion
Algorithm Fusion
DecisionFusion
Sensor Algorithm
Traditional:
Collaborative/AMMP:
Sensor Algorithm
Adaptive Control
AMMP Bayesian ProcessingAMMP Bayesian Processing
• Two modes of adaptation– Statistical parameters tracked and updated (e.g.
covariance matrix)– Priors on uncertain parameters modified based
on context (e.g. size, depth of radar response indicates an anti-tank mine, EMI library modified accordingly)
• Two modes of adaptation– Statistical parameters tracked and updated (e.g.
covariance matrix)– Priors on uncertain parameters modified based
on context (e.g. size, depth of radar response indicates an anti-tank mine, EMI library modified accordingly)
1 1
0 0
( / , ) ( / )( )
( / , ) ( / )
f H f H d
f H f H dΛ = ∫
∫r Θ Θ Θ
rr Ω Ω Ω
AMMP for Landmine Detection AMMP for Landmine Detection
• Prior work suggests adaptively pruning EMI/MD library using signature magnitude improved processor performance: LM vs HM
• Multi-modality processing – suggests adaptively pruning EMI/MD library using
GPR magnitude: AP vs AT– suggests adaptively pruning GPR library using
EMI/MD discrimination algorithms: mine type– New work: incorporate uncertainty in pruning
algorithm into processing algorithm• Sensor fusion at data level or decision level
• Prior work suggests adaptively pruning EMI/MD library using signature magnitude improved processor performance: LM vs HM
• Multi-modality processing – suggests adaptively pruning EMI/MD library using
GPR magnitude: AP vs AT– suggests adaptively pruning GPR library using
EMI/MD discrimination algorithms: mine type– New work: incorporate uncertainty in pruning
algorithm into processing algorithm• Sensor fusion at data level or decision level
MD Signature LibraryMD Signature LibraryResponse Library
LM HMSig 1 Sig 1Sig 2 Sig 2Sig 3 Sig 3
Sig M-1Sig M
Sig N-1Sig N
APAP
AT
AT
*Sources of uncertainty
1θ
2θ
( )21
1 , 1 ,1 1
1 11 2
( / ) ( )[ ( / , ) ( )]
AP, AT
t i
i i
N
x i x j ji j
f H p f t H p tθ
θ θθ
θ θ= =
=
= =
∑ ∑r r( )
1
itN θ
Application to Field Data: HSTAMIDS
(Handheld, co-located MD and GPR)
Application to Field Data: HSTAMIDS
(Handheld, co-located MD and GPR)
A B C D E F G H I J K L1 PMD-6
1.75"CL-2 2.5"
TAB-1 1.125"
CL-5RP 3.875"
VAL-69 2.75"
CL-5RW 3.25"
CL-5RW 2.0"
TM-46 3.5"
CL-5RP 2.5" Blank CL-3
3.375"VS2.2 1.0"
1
2 CL-1 .9375"
CL-3 1.875"
CL-2 0.5"
CL-5IP 1.5"
CL-5RS 2.125"
CL-5IP 2.875"
CL-5IS 0.5"
CL-5RS 3.75"
CL-5IP 1.0" CL-4 3.0" M-14
1.6875"CL-1
0.875" 2
3VS2.2 2.875"
CL-2 1.625"
CL-3 0.25" Blank CL-4
1.875" Blank CL-3 3.5" CL-3 0.75" Blank CL-2 4.0" CL-3 2.0" CL-3
2.25" 3
4 CL-2 0.75"
CL-4 0.75"
M409 2.375"
CL-3 2.25"
TS-50 1.6875"
CL-5IW 1.5"
M409 2.5" Blank CL-1
2.25"CL-2
1.875"CL-4 1.25" Blank 4
5 CL-1 1.5" CL-1 3.125"
CL-4 4.0"
CL-4 2.25"
CL-1 1.75"
M-14 2.0625"
CL-1 0.625"
CL-2 3.75" CL-2 2.5" Blank CL-1 3.0" CL-4 3.0" 5
6 M-19 1.25" CL-4 2.5" Blank CL-1
2.75"TMA-4 2.125"
CL-3 2.25"
CL-4 3.725 Blank CL-1 4.0" CL-4 3.0" TM-62P3
3.0625"CL-2
2.9375" 6
7 CL-2 2.25"
M-14 0.5"
CL-5IS 2.5" Blank CL-4 0.5" CL-3 2.0" M409
1.3125"CL-3 2.25"
CL-2 0.75"
TS-50 2.25"
CL-3 3.25" Blank 7
8 Blank CL-3 3.5"
TS-50 1.25"
CL-2 2.875"
TM-62P3 2.125"
CL-1 2.625" CL-3 0.5" CL-3
3.25"M-19
3.8125" CL-4 0.5" CL-1 1.375"
M409 0.5" 8
9 CL-4 0.625"
M-19 2.5625"
CL-1 0.5" CL-4 1.5" CL-2 0.5" CL-1
3.625" CL-1 1.5" VS.50 2.125"
CL-4 1.25" CL-3 2.5" Blank CL-2
0.875" 9
10 TM-62P3 1.1875"
CL-4 3.25" Blank CL-5IW
1.25"CL-5RP 1.375"
CL-5IS 2.25"
TMA-4 1.25"
CL-4 2.25"
M-14 1.625"
CL-4 2.75" CL-3 0.5" CL-1 3.5" 10
11 CL-2 1.625" CL-1 2.5" CL-1
1.875"CL-5RW
2.5"VAL-69 1.75"
CL-5RS 1.875" Blank CL-2 1.75" CL-2 4.0" CL-1
1.375"M409 1.5"
TMA-4 1.375" 11
12 CL-5RP 0.875"
CL-5IP 3.625"
VS2.2 3.25"
CL-5IW 0.75"
CL-5RW 1.625"
CL-5RW 3.5"
CL-5IS 1.6875"
CL-5RW 2.5"
CL-5IS 1.25" Blank CL-2
3.25"CL-3 3.25" 12
13 TM-46 1.0625"
CL-5RS 0.75" CL-3 1.0" CL-2
1.875" CL-1 0.5" CL-2 1.875"
CL-5RW 1.0"
TM-46 2.25"
CL-5RP 3.0" Blank CL-3
1.25"CL-4
2.625" 13
14 CL-5IP 0.5"
CL-5RS 2.125"
CL-5IP 2.875"
TAB-1 2.0625"
CL-3 0.875"
CL-5IW 2.25"
CL-5RS 1.5"
CL-5IW 2.125"
TS-50 1.375"
CL-2 1.25"
M-14 1.25"
14
15 Blank VAL-69 0.75"
CL-5RP 1.25" CL-1 3.75" TS-50
0.25"CL-3
2.375"CL-4 1.25"
CL-4 3.125"
CL-4 0.875"
VS-50 1.875" CL-1 2.5" 15
16 Blank CL5-IS 3.0"
CL-3 1.75"
CL-4 1.25" Blank CL-1
2.625"TAB-1 1.125"
CL-3 3.375"
CL-1 0.75"
TAB-1 2.25"
16
17 Blank Blank CL-2 2.5" CL-2 0.5" PMD-6 2.3125" CL-4 3.0" CL-3 1.0" Blank CL-3
1.875" 17
18 Blank CL-4 0.875" Blank Blank Blank VS-50
0.5"CL-4
1.375" Blank 18
19 CL-2 3.25"
CL-2 2.375"
CL-1 1.625"
CL-2 0.75" Blank CL-3
2.25" 19
20 Blank CL-4 3.5" TMA-4 3.6.25" CL-1 4.0" Blank 20
21 Version 02822.01 (AMD) CL-1 1.0" CL-2 3.75"
CL-4 0.875"
CL-2 3.125" 21
22 Blank CL-3 2.0" TMA-4 3.375"
CL-1 2.75" 22
A B C D E F G H I J K LC
A B C D E F G H I J K L1 PMD-6
1.75"CL-2 2.5"
TAB-1 1.125"
CL-5RP 3.875"
VAL-69 2.75"
CL-5RW 3.25"
CL-5RW 2.0"
TM-46 3.5"
CL-5RP 2.5" Blank CL-3
3.375"VS2.2 1.0"
1
2 CL-1 .9375"
CL-3 1.875"
CL-2 0.5"
CL-5IP 1.5"
CL-5RS 2.125"
CL-5IP 2.875"
CL-5IS 0.5"
CL-5RS 3.75"
CL-5IP 1.0" CL-4 3.0" M-14
1.6875"CL-1
0.875" 2
3VS2.2 2.875"
CL-2 1.625"
CL-3 0.25" Blank CL-4
1.875" Blank CL-3 3.5" CL-3 0.75" Blank CL-2 4.0" CL-3 2.0" CL-3
2.25" 3
4 CL-2 0.75"
CL-4 0.75"
M409 2.375"
CL-3 2.25"
TS-50 1.6875"
CL-5IW 1.5"
M409 2.5" Blank CL-1
2.25"CL-2
1.875"CL-4 1.25" Blank 4
5 CL-1 1.5" CL-1 3.125"
CL-4 4.0"
CL-4 2.25"
CL-1 1.75"
M-14 2.0625"
CL-1 0.625"
CL-2 3.75" CL-2 2.5" Blank CL-1 3.0" CL-4 3.0" 5
6 M-19 1.25" CL-4 2.5" Blank CL-1
2.75"TMA-4 2.125"
CL-3 2.25"
CL-4 3.725 Blank CL-1 4.0" CL-4 3.0" TM-62P3
3.0625"CL-2
2.9375" 6
7 CL-2 2.25"
M-14 0.5"
CL-5IS 2.5" Blank CL-4 0.5" CL-3 2.0" M409
1.3125"CL-3 2.25"
CL-2 0.75"
TS-50 2.25"
CL-3 3.25" Blank 7
8 Blank CL-3 3.5"
TS-50 1.25"
CL-2 2.875"
TM-62P3 2.125"
CL-1 2.625" CL-3 0.5" CL-3
3.25"M-19
3.8125" CL-4 0.5" CL-1 1.375"
M409 0.5" 8
9 CL-4 0.625"
M-19 2.5625"
CL-1 0.5" CL-4 1.5" CL-2 0.5" CL-1
3.625" CL-1 1.5" VS.50 2.125"
CL-4 1.25" CL-3 2.5" Blank CL-2
0.875" 9
10 TM-62P3 1.1875"
CL-4 3.25" Blank CL-5IW
1.25"CL-5RP 1.375"
CL-5IS 2.25"
TMA-4 1.25"
CL-4 2.25"
M-14 1.625"
CL-4 2.75" CL-3 0.5" CL-1 3.5" 10
11 CL-2 1.625" CL-1 2.5" CL-1
1.875"CL-5RW
2.5"VAL-69 1.75"
CL-5RS 1.875" Blank CL-2 1.75" CL-2 4.0" CL-1
1.375"M409 1.5"
TMA-4 1.375" 11
12 CL-5RP 0.875"
CL-5IP 3.625"
VS2.2 3.25"
CL-5IW 0.75"
CL-5RW 1.625"
CL-5RW 3.5"
CL-5IS 1.6875"
CL-5RW 2.5"
CL-5IS 1.25" Blank CL-2
3.25"CL-3 3.25" 12
13 TM-46 1.0625"
CL-5RS 0.75" CL-3 1.0" CL-2
1.875" CL-1 0.5" CL-2 1.875"
CL-5RW 1.0"
TM-46 2.25"
CL-5RP 3.0" Blank CL-3
1.25"CL-4
2.625" 13
14 CL-5IP 0.5"
CL-5RS 2.125"
CL-5IP 2.875"
TAB-1 2.0625"
CL-3 0.875"
CL-5IW 2.25"
CL-5RS 1.5"
CL-5IW 2.125"
TS-50 1.375"
CL-2 1.25"
M-14 1.25"
14
15 Blank VAL-69 0.75"
CL-5RP 1.25" CL-1 3.75" TS-50
0.25"CL-3
2.375"CL-4 1.25"
CL-4 3.125"
CL-4 0.875"
VS-50 1.875" CL-1 2.5" 15
16 Blank CL5-IS 3.0"
CL-3 1.75"
CL-4 1.25" Blank CL-1
2.625"TAB-1 1.125"
CL-3 3.375"
CL-1 0.75"
TAB-1 2.25"
16
17 Blank Blank CL-2 2.5" CL-2 0.5" PMD-6 2.3125" CL-4 3.0" CL-3 1.0" Blank CL-3
1.875" 17
18 Blank CL-4 0.875" Blank Blank Blank VS-50
0.5"CL-4
1.375" Blank 18
19 CL-2 3.25"
CL-2 2.375"
CL-1 1.625"
CL-2 0.75" Blank CL-3
2.25" 19
20 Blank CL-4 3.5" TMA-4 3.6.25" CL-1 4.0" Blank 20
21 Version 02822.01 (AMD) CL-1 1.0" CL-2 3.75"
CL-4 0.875"
CL-2 3.125" 21
22 Blank CL-3 2.0" TMA-4 3.375"
CL-1 2.75" 22
A B C D E F G H I J K LC
JUXOCO Cal Grid
• 4 types metallic clutter:- CL1: < 3g- CL2: 3 to 10g- CL3: 11 to 40g - CL4: > 40g
• Non-metallic clutter: - Wood - Plastic - Stone
• Blank Ground
• 12 Mine types (2-5 of each AT, AP, LM, HM)
HM/LM MD Energy ComparisonHM/LM MD Energy Comparison
Note difference in downtrack/crosstrack extent, as well aslevel of response
Note difference in downtrack/crosstrack extent, as well aslevel of response
Example of One Feature from MDExample of One Feature from MD
Mix of AT/AP, LM/HM
Energy Detector• 100% Detection • 73% False Alarm
Feature-Based Detector• 100% Detection • 76% False Alarm
Feature-Based Detector w/Region Processing
• 100% Detection • 24% False Alarm
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Pfa
Pd
HSTAMIDS MD Algorithm Performance
Feature/RegionEnergyFeature
Performance of HSTAMIDS MDPerformance of HSTAMIDS MD
Can reduction of AP/AT, HM/LM uncertainty improve performance?
AT/AP GPR Energy ComparisonAT/AP GPR Energy Comparison
Note difference in downtrack/crosstrack extentNote difference in downtrack/crosstrack extent
AT/AP GPR Energy ComparisonAT/AP GPR Energy Comparison
as well aslevel of response
as well aslevel of response
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Pfa
Pd
HSTAMIDS MD Algorithm Performance
Feature/RegionEnergyFeatureAMMP
Performance with Bayesian AMMPPerformance with Bayesian AMMPEnergy Detector
• 100% Detection • 73% False Alarm
Feature-Based Detector• 100% Detection • 76% False Alarm
Feature-Based Detector w/Region Processing
• 100% Detection • 24% False Alarm
Feature-Based Detector w/ Bayesian AMMP
• 100% Detection • 17% False Alarm
Application to Field Data: NIITEK GPR and Geophex’s
GEM-3 Metal Detector
Application to Field Data: NIITEK GPR and Geophex’s
GEM-3 Metal Detector
NIITEK GPR Data
GEM-3 Data
10000 20000Frequency
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
PP
M
A3
10000 20000Frequency
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
PP
M
A5
10000 20000Frequency
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
PP
M
A1Clutter
10000 20000Frequency
-2-1.75
-1.5-1.25
-1-0.75
-0.5-0.25
00.25
0.50.75
11.25
1.51.75
2
PP
M
B25
10000 20000Frequency
-1.5
-1.25
-1
-0.75
-0.5
-0.25
0
0.25
0.5
0.75
1
PP
M
B22
10000 20000Frequency
-1.5
-1.25
-1
-0.75
-0.5
-0.25
0
0.25
0.5
0.75
1
PP
M
B20 VAL69PMA-3
Clutter Clutter
10000 20000Frequency
-5-4-3-2-10123456789
B2VS50
AMMP Processing Results
Application to Lab Data: Georgia Tech’s EMI, GPR and Seismic
Sensors
Application to Lab Data: Georgia Tech’s EMI, GPR and Seismic
Sensors
EMI/GPR/Seismic Processing
• Data collected by Georgia Tech• Progress
– First, consider single sensor processing– Compare AMMP to various fusion algorithms– Conclude: need a more difficult test set for
AMMP to show its worth!
Burial Scenario #1
1.8m by 1.8m Scan Region
Rocks (3 and4 cm deep)
Dry Sand(5cm deep)
MINESVS-2.2
(7cm deep)
TS-50(1.5cm deep)
w/ Nail
M-14(0.5cm deep)
VS-50(1cm deep)
PFM-1(1.5 cm deep)
VS-1.6(6.5cm deep)
SeismicSources
Cans (3 and2.5 cm deep)
AssortedMetal Clutter (2 to 4 cm deep)
Shells(4cm deep)
ThreadedRod(3.5cm deep)
Penny(5.5cm deep)
Nails(4cm deep)
Ball Bearing(3.5cm deep)
Shells(5.5cm deep)
Burial Scenario #2
1.8m by 1.8m Scan Region
SeismicSourcesMINES
VS-50(1.3cm deep)
VS-2.2(5.4cm deep)
M-14(1cm deep)
TS-50(1.3cm deep)
PFM-1(0.6cm deep)
VS-50(0.5cm deep)
VS-1.6(5.1cm deep)
Rocks(2, 2.2, 2.5,and 1.3cm deep)
Can(2.2cm deep)
AssortedMetalClutter(<3cm deep)
CD(29cm deep)
EMI Responses
EMI Processing Results
GPR Responses
GPR Responses
GPR Processing Results
Initial Seismic Data/Results
Fusion
Confidence Fusion
Traditional Sensor Fusion
• Voting, Bayesian Decision, Bayesian Decision Statistic, Bayesian Decision Confidence (e.g. Hoballah and Varshney, Krzystofowicz and Long, Dasarathy, Blum, Kassam, and Poor, Chen and Varshney, Hall and Llinas)
• Confidence useful because normalized• Usually assume sensors/decisions are independent• Have not found Bayesian confidence fusion for
the correlated case – so have developed this metric
Confidence Fusion
• Confidence = probability of correct decision
• Compare results when confidence obtained rigorously and when confidence is assigned using a sigmoid function
*
1 2 1 2*
**
*
* * *, , 1 2
Pr( / ) Pr( ) ( / ) Pr( )Pr( / )Pr( ) ( )
( / ) Pr( )Pr( / , )( )N N
d d d dd d
d dd d d d d d N
H H f H Hc Hf
fc Hf
λ λ
λ λλλ λ
λ λ λ
=
=
= = ≈
= ≈λ λ
λ H HλK K K
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
PD
PFA
Sensor fusion performance
Majority Vote FusionBayesian Decision FusionSigmoid Confidence FusionDecision Statistic Confidence Fusion
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
PD
PFA
Individual sensor performance
Sensor 1Sensor 1 Performance PointSensor 2Sensor 2 Performance PointSensor 3Sensor 3 Performance Point
Accomplishments
• Adaptive Feature Selection (JCFO) applied to UXO problem
• AMMP successfully applied to two sets of field data (HSTAMIDS, NIITEK)
• Multi-sensor data from Georgia Tech processed – traditional fusion to date
Future Work
• JCFO application to landmines• JCFO application to subsurface structures
– Simulation– Data?
• AMMP application to EMI+GPR+Seismic• Adaptively modify sensor parameters• AMMP application to subsurface structures