Achieving Better than Human Design in Detecting Events of Interest in Bandwidth Constrained Sensor Networks Kalyan Veeramachaneni, Lisa Osadciw Development.

Achieving Better than Human Design in Achieving Better than Human Design in Detecting Events of InterestDetecting Events of Interest in in Bandwidth ConstrainedBandwidth Constrained Sensor Networks Sensor Networks

Kalyan Veeramachaneni, Lisa OsadciwKalyan Veeramachaneni, Lisa Osadciw

Development and Research in Evolutionary Algorithms for Multi sensor Smart Networks Development and Research in Evolutionary Algorithms for Multi sensor Smart Networks (DREAMSNet) (DREAMSNet)

Department of Electrical Engineering and Computer Science Department of Electrical Engineering and Computer Science

Syracuse University, New YorkSyracuse University, New York

GECCO 2008, Human Competitive Results Awards, July 14, Atlanta, U.S.A

22

What are we detecting?What are we detecting?

Modern day society relies on detection or Modern day society relies on detection or determining the meaning of the presence or determining the meaning of the presence or absence of a signal absence of a signal

Digital CommunicationsDigital Communications Pipeline/Bridges crack detection Pipeline/Bridges crack detection Genuine User detection using biometrics Genuine User detection using biometrics Presence of aircraft, ships, or motor Presence of aircraft, ships, or motor

vehicles vehicles Locating emergency personnelLocating emergency personnel Weather PhenomenaWeather Phenomena Building SecurityBuilding Security

Sensors are located in remote areas making Sensors are located in remote areas making decisions using a variety of criteriadecisions using a variety of criteria Maximum A-Posteriori CriterionMaximum A-Posteriori Criterion Maximum Likelihood CriterionMaximum Likelihood Criterion Minimum Error CriterionMinimum Error Criterion

33

Bandwidth Constrained Detection Networks Bandwidth Constrained Detection Networks

1

2

Sensor1

Sensor2

X1

X2

Fusion Rule

u1

u2

fu

Second Classifier Only

OR

AND

First SensorOnly

Likelihood density model for a sensor

Noise only

Event

44

Bandwidth Constrained Detection NetworksBandwidth Constrained Detection Networks

Two types of Errors need to be reducedTwo types of Errors need to be reduced If the entire observation value is transmitted to a central processing node, an efficient machine If the entire observation value is transmitted to a central processing node, an efficient machine

learning technique can be designed to achieve better accuracylearning technique can be designed to achieve better accuracy Shown below are 20000 samples of observations, 10000 belong to events, 10000 to noise. Shown below are 20000 samples of observations, 10000 belong to events, 10000 to noise.

9 to 32 bits required per sample if all bits are transmitted 9 to 32 bits required per sample if all bits are transmitted Reduces to 1 bit decision if decision is transmitted insteadReduces to 1 bit decision if decision is transmitted instead

Misses: Fail to detect an event

False Alarms: detecting an event that did not occur

Threshold on Sensor 1

Threshold on Sensor 2

Event is declared only in this quadrant, i.e. AND rule

Noise * Event

55

(E)Competitive Result: Correlated Sensors (E)Competitive Result: Correlated Sensors Designs for 0.1 Correlation Designs for 0.1 Correlation

LRT (Human) Based Design: 2 thresholds on each sensor 2 Sensor only fusion rule

Region where an event is declared

PSO Based Design:Simple 1 Threshold for each sensor AND fusion ruleVery few errors

Region where an eventis declared

66

Humies Categories CoveredHumies Categories Covered

(G) The result solves a problem of indisputable difficulty in its field.(G) The result solves a problem of indisputable difficulty in its field.

(B) The result is equal to or better than a result that was accepted as a new (B) The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-reviewed scientific result at the time when it was published in a peer-reviewed scientific journal.scientific journal.

(D) The result is publishable in its own right as a new scientific result - (D) The result is publishable in its own right as a new scientific result - independent of the fact that the result was mechanically created.independent of the fact that the result was mechanically created.

(E) The result is equal to or better than the most recent human-created (E) The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions.of increasingly better human-created solutions.

77

HUMIES Category GHUMIES Category G

The result solves a problem of indisputable difficulty in its The result solves a problem of indisputable difficulty in its field.field. Amount of Research and Publications on Topic Indicates ComplexityAmount of Research and Publications on Topic Indicates Complexity

Quick Check Research PublicationsQuick Check Research Publications 120 Journal Articles with Approximately 45 Discussing Similar Design Issues120 Journal Articles with Approximately 45 Discussing Similar Design Issues 48 Textbooks At Least Currently On Sale In This Area 48 Textbooks At Least Currently On Sale In This Area 5 Dissertations deal with same problem and provide human developed designs5 Dissertations deal with same problem and provide human developed designs

Paper Published that Addresses the Difficulty Paper Published that Addresses the Difficulty John N Tsitsiklis, Michael Athans, “On Complexity of Decentralized Decision John N Tsitsiklis, Michael Athans, “On Complexity of Decentralized Decision

making and detection problems” 23rd IEEE Conference on Decision and Control, making and detection problems” 23rd IEEE Conference on Decision and Control, 19841984

Optimizing Distributed Detection for 2 SensorsOptimizing Distributed Detection for 2 Sensors Independent sensors: Intractable Independent sensors: Intractable Correlated sensors: NP Complete Correlated sensors: NP Complete --

88

HUMIES Category B HUMIES Category B

The result is equal to or better than a result that was accepted as a The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-new scientific result at the time when it was published in a peer-reviewed scientific journalreviewed scientific journal

Much Research Published in Area Since the 50s/60s Beginning in RadarMuch Research Published in Area Since the 50s/60s Beginning in Radar Type I/Type II ErrorsType I/Type II Errors

Fail to detect the eventFail to detect the event Detect an Event that did not occurDetect an Event that did not occur

Decouple the two problems: optimize thresholds and design best fusion rule separatelyDecouple the two problems: optimize thresholds and design best fusion rule separately

When only labeled training datasets are available performance is sensitive to threshold search When only labeled training datasets are available performance is sensitive to threshold search precisionprecision Probability of Error for Different Threshold Search

Mechanisms and Using LRT for Fusion Rule

0.0132

0.0134

0.0136

0.0138

0.014

0.0142

0.0144

0 0.2 0.4 0.6 0.8 1

Correlation Factor -->

Bayesia

n E

rro

r --

>

Threshold Setting 1

Threshold Setting 2

Threshold Setting 3

Threshold Setting 4

When likelihood models are available Optimize Threshold

Exceed Ratio of Conditional Distributions

99

1

2

Sensor1

Sensor2

X1

X2

Fusion Rule

u1

u2

Human Design Solution: Likelihood Ratio Test (LRT) DesignHuman Design Solution: Likelihood Ratio Test (LRT) Design

fu

Optimize thresholds individually by keeping other thresholds and fusion rule constant

Use LRT for independent or correlated deriving fusion rule

Human Design Solution: Person-by-Person Optimal (PBPO) for Independent SensorsHuman Design Solution: Person-by-Person Optimal (PBPO) for Independent Sensors Human Competitive Result: Particle Swarm Optimization (PSO) Based DesignHuman Competitive Result: Particle Swarm Optimization (PSO) Based Design

Joint optimization of thresholds and Fusion RuleNo closed form solution exists

1010

HUMIES Category B (cont)HUMIES Category B (cont)

Particle Swarm Optimized Detector Simplifies Sensor Network AdaptationParticle Swarm Optimized Detector Simplifies Sensor Network Adaptation Able to combine performance parameters to simultaneously handle a variety of Able to combine performance parameters to simultaneously handle a variety of

situationssituations consider resources (energy and communication bandwidth) consider resources (energy and communication bandwidth) Reduce type I or type II errors across different degrees of correlationReduce type I or type II errors across different degrees of correlation

Simpler ReceiverSimpler Receiver Single threshold design compared to LRT based designs that can lead to multiple Single threshold design compared to LRT based designs that can lead to multiple

thresholds as the Likelihood ratio becomes non linearthresholds as the Likelihood ratio becomes non linear Adapt to design for either probability density models or a labeled training Adapt to design for either probability density models or a labeled training

dataset provideddataset provided Automatically Handles the Heterogeneity in Practical Sensor NetworksAutomatically Handles the Heterogeneity in Practical Sensor Networks

Percentage Improvement Over LRT Based Design

0

10

20

30

40

50

60

70

80

0 0.2 0.4 0.6 0.8 1

Correlation Factor --->

Per

cen

tag

e --

->

1111

HUMIES Category DHUMIES Category D The result is publishable in its own right as a new scientific result - independent of The result is publishable in its own right as a new scientific result - independent of

the fact that the result was mechanically createdthe fact that the result was mechanically created. . 5 Papers Published Including a Best Paper5 Papers Published Including a Best Paper

Correlated Sensors Correlated Sensors Kalyan VeeramachaneniKalyan Veeramachaneni and and Lisa OsadciwLisa Osadciw, “Design of Distributed Detection Systems with , “Design of Distributed Detection Systems with

Heterogeneous Correlated Sensors," Heterogeneous Correlated Sensors," 44th Annual Allerton Conference on Communications and Control44th Annual Allerton Conference on Communications and Control, , Allerton Park, Illinois, September, 2007.Allerton Park, Illinois, September, 2007.

Independent Sensors Independent Sensors Kalyan Veeramachaneni, Lisa OsadciwKalyan Veeramachaneni, Lisa Osadciw, Pramod Varshney“Adaptive Multimodal Biometric , Pramod Varshney“Adaptive Multimodal Biometric

Management Algorithm,” Management Algorithm,” IEEE Transactions on Systems Man and Cybernatics : Part C: Applications IEEE Transactions on Systems Man and Cybernatics : Part C: Applications and Reviewsand Reviews, Vol. 35, No. 3 August 2005., Vol. 35, No. 3 August 2005.

Applications Applications BiometricsBiometrics: : Kalyan VeeramachaneniKalyan Veeramachaneni, Nisha Srinivas, , Nisha Srinivas, Lisa OsadciwLisa Osadciw, and Arun Ross, “Designing , and Arun Ross, “Designing

Optimal Fusion Strategies for Correlated Biometric Classifiers”, Optimal Fusion Strategies for Correlated Biometric Classifiers”, IEEE IEEE CVPR Conference, AnchorageCVPR Conference, Anchorage, , Alaska, June, 2008. Alaska, June, 2008. (Best Paper Award)(Best Paper Award)

Pipeline Crack DetectionPipeline Crack Detection: : Kalyan VeeramachaneniKalyan Veeramachaneni, Weizhong Yan, Kai Goebel, and , Weizhong Yan, Kai Goebel, and Lisa OsadciwLisa Osadciw, , “Improving Classifier Fusion Using Particle Swarm Optimization”, “Improving Classifier Fusion Using Particle Swarm Optimization”, IEEE Multi-Criteria Decision Making IEEE Multi-Criteria Decision Making (MCDM) Symposium(MCDM) Symposium, Honolulu, Hawaii, April, 2007., Honolulu, Hawaii, April, 2007.

Adaptive Sensor Management Adaptive Sensor Management Lisa Osadciw and Kalyan VeeramachaneniLisa Osadciw and Kalyan Veeramachaneni , “Sensor Management through Efficient Fitness Function , “Sensor Management through Efficient Fitness Function

Design," Design," Proceedings of 41st Annual Asilomar Conference on Signals, Systems, and ComputersProceedings of 41st Annual Asilomar Conference on Signals, Systems, and Computers, , Asilomar, CA, November, 2007.Asilomar, CA, November, 2007.

1212

HUMIES Category EHUMIES Category E

The result is equal to or better than the most recent The result is equal to or better than the most recent human-created solution to a long-standing problem for human-created solution to a long-standing problem for which there has been a succession of increasingly better which there has been a succession of increasingly better human-created solutions.human-created solutions.

Long-standing problem since the 1950s in Radar ResearchLong-standing problem since the 1950s in Radar Research Succession of better solutions as discussed in Category BSuccession of better solutions as discussed in Category B

First Single Detectors Derived for the Following CriterionFirst Single Detectors Derived for the Following Criterion Maximum A-Posteriori Criterion – maximize the posteriori probability of Maximum A-Posteriori Criterion – maximize the posteriori probability of

belonging to one event to the other possible eventbelonging to one event to the other possible event Maximum Likelihood Criterion – maximizes probability of belonging Maximum Likelihood Criterion – maximizes probability of belonging

(likelihood) to event(likelihood) to event Minimum Error Criterion – minimize the number of errors in decisionsMinimum Error Criterion – minimize the number of errors in decisions

1313

HUMIES Category E (cont):HUMIES Category E (cont): Matched Filter Designed in 50s and 60s from RadarMatched Filter Designed in 50s and 60s from Radar

Maximum Signal to Noise Criterion – maximize signal over the noise background to Maximum Signal to Noise Criterion – maximize signal over the noise background to assist detection by matched filter (North, Van Vleck, Middleton)assist detection by matched filter (North, Van Vleck, Middleton)

Inverse Probability Criterion – (Wald, Neyman, Pearson)Inverse Probability Criterion – (Wald, Neyman, Pearson) Likelihood Ratio - based on Shannon’s information theory (Woodward & Davis)Likelihood Ratio - based on Shannon’s information theory (Woodward & Davis)

Distributed Detection (Tenney & Sandell-1979 through today)Distributed Detection (Tenney & Sandell-1979 through today) Chair Z., P. K. Varshney, "Optimal Data Fusion in Multiple Sensor Detection Systems," IEEE Chair Z., P. K. Varshney, "Optimal Data Fusion in Multiple Sensor Detection Systems," IEEE

Trans. on Aerospace and Elect. Systems, Vol. AES-22, No. 1, pp. 98-101, Jan. Trans. on Aerospace and Elect. Systems, Vol. AES-22, No. 1, pp. 98-101, Jan. 19861986. . Tang, Z. -B., K. R. Pattipati, and D. L. Kleinman, "An Algorithm for Determining the Decision Tang, Z. -B., K. R. Pattipati, and D. L. Kleinman, "An Algorithm for Determining the Decision

Thresholds in a Distributed Detection Problem," IEEE Trans. on Systems, Man, and Thresholds in a Distributed Detection Problem," IEEE Trans. on Systems, Man, and Cybernetics, Vol. SMC-21, pp. 231-237, Jan./Feb. 1991.Cybernetics, Vol. SMC-21, pp. 231-237, Jan./Feb. 1991.

Kam., M., Q. Zhu., and W. S. Gray, "Optimal Data Fusion of Correlated Local Decisions in Kam., M., Q. Zhu., and W. S. Gray, "Optimal Data Fusion of Correlated Local Decisions in Multiple Sensor Detection Systems," IEEE Transactions on Aerospace and Elect. Syst., Vol. 28, Multiple Sensor Detection Systems," IEEE Transactions on Aerospace and Elect. Syst., Vol. 28, pp. 916-920, July 1992. pp. 916-920, July 1992.

Peter Willet, Peter F. Swaszek, Rick S. Blum, "The Good, Bad, and Ugly : Distributed Detection Peter Willet, Peter F. Swaszek, Rick S. Blum, "The Good, Bad, and Ugly : Distributed Detection of Known Signal in Dependent Gaussian Noise," IEEE Transactions on Signal Processing, Vol. of Known Signal in Dependent Gaussian Noise," IEEE Transactions on Signal Processing, Vol. 48, No. 12, December 2000. 48, No. 12, December 2000.

Kalyan Veeramachaneni, Lisa Ann Osadciw, Pramod K Varshney, "An Adaptive Multimodal Biometric Fusion Kalyan Veeramachaneni, Lisa Ann Osadciw, Pramod K Varshney, "An Adaptive Multimodal Biometric Fusion Algorithm Using Particle Swarm", Proceedings of SPIE, Aerosense, April 21-25, 2003, Orlando. Algorithm Using Particle Swarm", Proceedings of SPIE, Aerosense, April 21-25, 2003, Orlando.

Saeed A. Aldosari, Jose M. F. Moura, “Fusion in Sensor Networks with Communication Constraints”, International Saeed A. Aldosari, Jose M. F. Moura, “Fusion in Sensor Networks with Communication Constraints”, International Symposium on Information Processing in Sensor Networks, April 26-27, 2004, Berkeley, CA. Symposium on Information Processing in Sensor Networks, April 26-27, 2004, Berkeley, CA.

1414

HUMIES Category E (cont):HUMIES Category E (cont):

Swarm Solution:Swarm Solution: Type I/Type II ErrorsType I/Type II Errors

Errors are Balanced in Real Time Based on Current System NeedsErrors are Balanced in Real Time Based on Current System Needs Simultaneously reduce, “Failure to detect the event”, “Detect an Event that did not occurSimultaneously reduce, “Failure to detect the event”, “Detect an Event that did not occur””

Reduce Communication BandwidthReduce Communication Bandwidth Decisions at Sensor to Reduce Message Size Saving BandwidthDecisions at Sensor to Reduce Message Size Saving Bandwidth Fusion Architecture Can Be Modified in Real-Time Based on Bandwidth and Energy Fusion Architecture Can Be Modified in Real-Time Based on Bandwidth and Energy

NeedsNeeds

Minimize EnergyMinimize Energy Save in communications with Smaller Messages and Fewer Through FusionSave in communications with Smaller Messages and Fewer Through Fusion Reduce computations with Simpler designs and Fusion RulesReduce computations with Simpler designs and Fusion Rules

Ease of Adaptation to Other ApplicationsEase of Adaptation to Other Applications Communication Management for Any Wireless Sensor Network and ArchitectureCommunication Management for Any Wireless Sensor Network and Architecture Various Sensor Networks for Aircraft Routing at Airports, First Response Networks, Various Sensor Networks for Aircraft Routing at Airports, First Response Networks,

Large, Remote Sensor Networks, Health Monitoring Sensor Networks, and etc.Large, Remote Sensor Networks, Health Monitoring Sensor Networks, and etc.

1515

(E) Competitive Result: Independent Sensors(E) Competitive Result: Independent Sensors

Number of Number of Sensors Sensors PBPOPBPO PSOPSO

% % Improvements in Improvements in accuracy accuracy

33 0.05648340.0564834 0.0555010.055501 1.73841.7384

55 0.00234260.0023426 0.00145180.0014518 38.025038.0250

77 7.022446e-0067.022446e-006 1.97107e-0061.97107e-006 71.931771.9317

99 3.539055e-0093.539055e-009 5.2906e-0115.2906e-011 98.505098.5050

Human Design Accuracy

PSO Resulting Accuracy

PBPO-Person-By-Person Optimal

PSO – Particle Swarm Optimization

1616

(E)Competitive Result: Correlated Sensors (E)Competitive Result: Correlated Sensors

Probability of Error

0.00E+00

1.00E-03

2.00E-03

3.00E-03

4.00E-03

5.00E-03

6.00E-03

7.00E-03

8.00E-03

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Correlation Factor

Pro

ba

bili

ty o

f E

rro

r

LRT Based

PSO Based

Human Design

54% 13%

2.5%

1717

Humies Categories In SummaryHumies Categories In Summary (G) (G) The result solves a problem of indisputable difficulty in its field.The result solves a problem of indisputable difficulty in its field.

This is an NP complete problem which also becomes too complex as the sensors become This is an NP complete problem which also becomes too complex as the sensors become dependent.dependent.

(B) (B) The result is equal to or better than a result that was accepted as a The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-new scientific result at the time when it was published in a peer-reviewed scientific journalreviewed scientific journal Problem Studied Since 1950s with Suboptimal SolutionsProblem Studied Since 1950s with Suboptimal Solutions PSO Allows the Coupled Threshold – Fusion Rule Problem to Remain CoupledPSO Allows the Coupled Threshold – Fusion Rule Problem to Remain Coupled PSO is able to solve the problem for different problem types PSO is able to solve the problem for different problem types

(D) (D) The result is publishable in its own right as a new scientific result - The result is publishable in its own right as a new scientific result - independent of the fact that the result was mechanically created.independent of the fact that the result was mechanically created. We have published 5 papers including 1 that recently received “Best Paper” in an We have published 5 papers including 1 that recently received “Best Paper” in an

application domainapplication domain (E) (E) The result is equal to or better than the most recent human-created The result is equal to or better than the most recent human-created

solution to a long-standing problem for which there has been a solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions.succession of increasingly better human-created solutions. As the network grows, the PSO performance also grows w.r.t. the human-created As the network grows, the PSO performance also grows w.r.t. the human-created

solution.solution.

1818

Why this is the Why this is the bestbest??

Significance of the Impact on a Wide Range of ApplicationsSignificance of the Impact on a Wide Range of Applications- Military - Health MonitoringMilitary - Health Monitoring- Homeland Security - Environmental MonitoringHomeland Security - Environmental Monitoring- Smart and Safe Buildings - Vehicle Health MonitoringSmart and Safe Buildings - Vehicle Health Monitoring

Ease of Adapting Solution to the Complexities of Practical Ease of Adapting Solution to the Complexities of Practical ProblemsProblems- Imperfect Propagation Channels - Multi-User Interference- Imperfect Propagation Channels - Multi-User Interference- Changing Architectures - Resource ConstraintsChanging Architectures - Resource Constraints

Solves Distributed Detection Problems Too Complex in PastSolves Distributed Detection Problems Too Complex in Past- Multiple ImperfectionsMultiple Imperfections- Complex Sensor ArchitecturesComplex Sensor Architectures- Complex Interference EnvironmentsComplex Interference Environments

1919

Thank you!Thank you!

2020

1

2

Sensor1

Sensor2

X1

X2

Fusion Rule

u1

u2

Human Design Solution:Human Design Solution:Likelihood Ratio Test (LRT) DesignLikelihood Ratio Test (LRT) Design

1

11 arg min R

0

0

1 1

1

1log 1 log log

(2 )1j j

j j

NFAM M

j jFAj FA FA

H

P P P Cu u

P P CPH

LRT based fusion rule for independent sensors

2

22 arg min R

fu

1 1 1 1 1 1 1

0 0 0 0 0 0 0

0

0

)1

1

1 ........

log log1 ...... (1

ij i j ijk i j kFAi j i j k

FAij i j ijk i j ki j i j k

Hz z z z z

P C

z z z z z P C

H

LRT based Fusion Rule for

correlated sensors

2121

(E)Competitive Result: Correlated Sensors (E)Competitive Result: Correlated Sensors Designs for 0.9 Correlation Designs for 0.9 Correlation

LRT (Human) Based Design: 2 thresholds on each sensor 2 Sensor only fusion rule

Region where an event is declared

PSO Based Design:Simple 1 Threshold for each sensor AND fusion ruleHigher number of errors, but still better

Region where an eventis declared