Top Banner
Active Sensing
66

Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Mar 07, 2018

Download

Documents

lethuan
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Active Sensing

Page 2: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

What is Active Sensing?

For some, it’s a hobby!

Page 3: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

“Object” to be Inferred

Page 4: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

“Measurement”

Page 5: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

“Measurement”

Page 6: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

“Feedback” Guides New “Measurements”

Page 7: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

“Feedback” Guides New “Measurements”

Page 8: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

“Feedback” Guides New “Measurements”

Page 9: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

“Feedback” Guides New “Measurements”

Page 10: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

“Feedback” Guides New “Measurements”

Page 11: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

“Feedback” Guides New “Measurements”

Page 12: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Background and Motivation

Page 13: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Sensors, Sensors Everywhere

Page 14: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Integral to Science, Engineering, Discovery

Page 15: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Inevitable Data Deluge!

The Economist, February 2010

Page 16: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Challenges for Sensing/Processing Systems

Page 17: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Active Sensing

-- An Engineering Example --

Page 18: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Wide-field Infrared Survey Explorer (WISE)

Need to shield IR (heat) from its own instruments

Sensitive instruments housed in solid hydrogen

Expected lifetime: 10 months!

WISE Mission: (http://www.nasa.gov/mission_pages/WISE/mission/index.html)

“…the infrared surveyor will spend six months

mapping the whole sky. It will then begin a second

scan to uncover even more objects and to look for

any changes in the sky that might have occurred

since the first survey. This second partial sky survey

will end about three months later when the

spacecraft's frozen-hydrogen cryogen runs out…”

Fornax Galaxy Cluster, Feb. 17 2010

Page 19: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

original signal

(~0.8% non-zero components)

Noisy, non-adaptive sampling

Recovery from adaptive samples

(1/20 “discoveries” are errors)

Recovery from non-adaptive samples

(1/20 “discoveries” are errors)

Noisy adaptive sampling

Astronomical Imaging “On a Budget”

Page 20: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Active Sensing for Sparse Recovery

-- Preliminaries and Formalization --

Page 21: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

DFT

time

Sparsity

DWT

frequency

Page 22: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

A Sparse Signal Model

signal support set

number of nonzero

signal components

Page 23: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Non-adaptive Sampling

Page 24: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Alternative: Sequential Experimental Design

Page 25: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Alternative: Sequential Experimental Design

Page 26: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Alternative: Sequential Experimental Design

Page 27: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Support Recovery

Page 28: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

A Simple Active Sensing Approach

Page 29: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Recovery From Non-Adaptive Measurements

Page 30: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Fundamentally a Multiple Hypothesis Test

Page 31: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Non-Adaptive Support Recovery

Page 32: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Adaptive Approach: Sequential Thresholding

Page 33: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

False Positives

Page 34: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

False Negatives

Page 35: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Probability of Error Bound

(Malloy & Nowak, 2011)

Page 36: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

greater sensitivity for same precision budget or lower

experimental requirements for equivalent sensitivity

Improvements Through Sequential Design

Page 37: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Active Sensing for Sparse Recovery

-- A Relaxed Error Criteria --

Page 38: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Measuring Error: False Discoveries

Here, FDP = 3/5

Page 39: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Measuring Error: Non-Discoveries

Here, NDP = 5/7

Page 40: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Limits of Non-adaptive Sampling

Page 41: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Signal Strength

Sparsity

Estimation Possible

(FDP + NDP 0)

Estimation Impossible

(FDP + NDP 0)

Sharp Delineation in “Parameter Space”

Page 42: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

What to Do in Low SNR Settings?

Page 43: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Idealized Example

Page 44: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Distilled Sensing (DS)

Page 45: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

DS

(k=4)

Estimation Possible(FDP + NDP 0)

Estimation Impossible(FDP + NDP 0)

Non-adaptive limit

Estimation Possible(FDP + NDP 0)

Achievable using DS

Equal Allocation of Sensing Resources

Page 46: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Estimation Possible(FDP + NDP 0)

Estimation Impossible(FDP + NDP 0)

DS

(k=6)

Estimation Possible(FDP + NDP 0)

Non-adaptive limit Achievable using DS

Equal Allocation of Sensing Resources

Page 47: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

original signal

(~0.8% non-zero components)

Non-adaptive observations

Adaptive recovery

(FDP = 0.05)

Non-adaptive recovery

(FDP = 0.05)

DS observations (k=6)

Recall: Noisy Astronomical Imaging

Page 48: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

(FDP + NDP 0)

(FDP + NDP 0)

(FDP + NDP 0)

Non-adaptive limit DS, k=8

Unequal Allocation of Sensing Resources…

Page 49: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Simulation

Page 50: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

The Curse of Dimensionality…

Page 51: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

…and the Virtue of Adaptivity!

Page 52: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Active Sensing for Sparse Recovery

-- Adaptive Compressive Sampling --

Page 53: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Improvements w.r.t. Other Resources?

Page 54: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Compressive DS

Page 55: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Curse of Dimensionality: Compressed Sensing

Page 56: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

…and the Virtue of Adaptivity!

Page 57: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

Generalization Beyond Gaussian Models

-- Sequential Thresholding --

Page 58: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using
Page 59: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using
Page 60: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using
Page 61: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using
Page 62: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using
Page 63: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using
Page 64: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using
Page 65: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using
Page 66: Active Sensing - University of Wisconsin–Madisonnowak.ece.wisc.edu/ASL_part2.pdf · “Feedback” Guides New “Measurements ... (FDP + NDP 0) Non-adaptive limit Achievable using

End of Presentation…