PASSIVE MILLIMETER WAVE IMAGING WITH SUPER-RESOLUTION: Application to Aviation safety
in extremely poor visibility
Dr. Isaiah M. Blankson, Research &Technology
DirectorateNASA Glenn Research Center,
Cleveland, OH
Presented at Institute of Mathematics and its Applications, University of Minnesota: May 5, 2001
PASSIVE MILLIMETER-WAVE IMAGING (PMMWI) PROJECT: OBJECTIVES &GOALS
• Explore the potential application of Radiometric sensors to alleviate atmospheric hazards to aviation.
• Develop/design an all-weather Radiometer operating at 94 GHz which employs Super-Resolution Algorithms for a Real -Time rapid image inversion processing, and is capable of producing very high resolution images ( recover scene-spatial frequencies ~or >10 X {Rayleigh Limit}).
• Construct a functioning system capable of Ground and Airborne Applications
Aeronautics & Space Transportation TechnologyStrategic Roadmap
Source: Aeronautics & Space Transportation Technology / Strategic Roadmap, NASA GRC
Space Applications
Remote Sensing of Planetary Surfaces
• Structurally Embeddable• Low Power Applications• Payload Reduction• Compact
Pillar One:
Global Civil
Aviation
Safety 2000 2010Human-Related Factors
Increase AirportCapacity
Improve Navigational Aids
Reduce AccidentRates 10X
Millimeter Wave Radiometryat 94 GHz with
Super-Resolution
Electromagnetic Spectrum
1102104106108101010121014101610181020
Millimeter & Sub-Millimeter Wave Region
0.03
Å
3 Å
300
Å
0.3
m
3 m
300
m
3 cm
3 m
300
m
30 k
m
300
km
3 km
30 m
30 c
m
0.3
cm
30
m
30 Å
0.3
Å0.4 m - 0.7 m
Visible
Infrared
Gamma Ray X Ray Radar
Radio Bands Audio AC
UV Microwave
Wavelength
Black Represents Atmospheric Attenuation
= 1
= 0
Frequency (Hz)
Heating Heating
Dissociation
PhotoIonization
PhotoDissociation
ElectronShifts
Electromagnetic Field Fluctuations
Rain & FogAttenuation
Spherics
Interaction Mechanismsor Phenomena Detected
MolecularVibration
MolecularVibration
Cosmic NoiseRadioAstronomy
Source: Manual of Remote Sensing, Vol. 1, First Edition, 1975
Black Body RadiationS
pec
tral
Exi
tan
ce
(W c
m-2
m-1 )
Wavelength (m)
1 2 3 4 5 6
30
50
10
40
20
0
1000 °K
1200 °K
1400 °K
1600 °K
1800 °K
2000 °K
22
2 22 kT
c
kTfBbb
Rayleigh-Jeans
Approximation Holds
Microwave
Infra-RedNear-Infrared
MillimeterSub-millimeter
1015 1013 1011 109 107 105
Frequency (Hz)
Rel
ati
ve
Rad
ian
ce
Why Passive Millimeter-Wave Imaging?
• All natural objects whose temperatures are above absolute zero emit passive millimeter-wave radiation.
• Millimeter-waves are much more effective (lower attenuation) than infrared in poor weather conditions such as fog, clouds, snow, dust-storms and rain. Also, images produced by passive millimeter-waves have natural appearances.
• The amount of radiation emitted in the millimeter-wave range is 108 times smaller than the amount emitted in the infrared range.
• However, current millimeter-wave receivers have at least 105 times better noise performance than infrared detectors and the temperature contrast recovers the remaining 103.
• This makes millimeter-wave imaging comparable in performance with current infrared systems.
• Electromagnetic radiation windows occur at 35 GHz, 94 GHz, 140 GHz, and 220 GHz.
• Choice of frequency depends on specific application
APPLICATIONS
Advances in Inverse Problem Solutions for :
— Geological Explorations
— Remote Sensing of Vegetation & Soil Conditions
— Non-Invasive Brain Volumetric Mapping
— Airport Safety
— All-Weather Vision
— Fused Sensor Imaging - Component
— Medical Diagnostics
— Plasma Diagnostics
— In-Situ Non-Destructive Testing
( Composites : Voids, Delaminations )
— Defense Applications
— Environmental
REMOTE SENSING
(Terrestrial & Extra-terrestrial)
DIAGNOSTICS
GENERAL
SIDE - BENEFITS
RADIOMETER CONCEPT
ELECTRONICS
BEAM Controller
COLLECTORANTENNA
……..
……….
[[[[**33
SUPER-RESOLUTIONSoftware
COMPUTER
Aviation Safety Application
Sky Radiation
Ground and VegetationEmissions
Metal Reflections of Cold Sky Radiation
Passive Radiometric Sensing - Concept
Side LobeAtmosphericContributions
Atmosphere
Antenna
Beam Width
RadiometerReceiver
VO
Side LobeBackgroundContribution
UpwardAtmospheric
Emission
Antenna PowerPattern
BUPScattered Radiation
Atmospheric Loss
Target Observation Cell
BB Self Emission
DownwardAtmosphericEmission
BDN
BSC
LATM
LATM
BB
LATM
BSC
Conceptual Diagram of 2-D Phased Array Radiometer
1 complete scan 1 video frame
ImageProcessor Receiver
Radiating Element
Low Noise Amplifier
PhaseShifter
Beam SteeringComputer
Direct Measurement Result
GOAL: Best true “Scene “ R e c o v e r y
INVERSE Problem Solution
EMR-Properties of Propagation media
Mathematical Processing of Measured Data
TRUE Scene
“True” Scene..Recovery
Why Super-Resolution?
• Images acquired from practical sensing operations usually suffer from poor resolution due to the finite size limitations of the antenna, or the lens, and the consequent imposition of diffraction limits.
• The fundamental operation underlying the sensing operation is the “low-pass” filtering effect due to the finite size of the antenna lens.
• The portions of the scene that are lost by the imaging system are the fine details (high frequency spatial spectral components) that accurately describe the object in the scene.
• For super-resolution, spatial spectral extrapolation is needed.
• Some studies have indicated that the cost of an imager increases as (1/Resolution) raised to the power 2.5.
• Hence, a possible two-fold improvement in resolution by super-resolution processing, roughly translates into a cost reduction of an imager by more than 5 times.
Regularization/Reduction MethodNoise
PSFA
Imaging System
Hypothetical operator
RI0
Original image
IDegraded Image
Î0
The Best Fit Image
I = AI0 + Î0 = R AI0 + RRI
The mathematical model of the method is shown above where: I0 = is the original image,I = is the degraded image = AI0 + A = the measured PSF of the imaging system, is the measured mean square noise of the system, R = is a hypothetical operator to be found in order to obtain the best fit image.RAI0 = the noise-free output.
The main idea of this method of reduction is to find an operator R such that the following conditions are satisfied:
=maximum required mean square noise of the system =maximum allowed error for the mean square difference between RA and E (the identity matrix) , A* is the transposed matrix of A. is a parameter to be found such that the set of conditions (2.3), (2.4) and (2.5) is satisfied. These conditions form a constrained optimization problem.
(2.5) and
(2.4)
(2.3)
22
*1*
2
min
RERAR
AEAAR
R
Mathematics of Inversion
Tikhonov - Pytiev Regularization
Tikhonov-Regularization
CONSTRAINT
LINEAR INVERSION :f = (A* A + H )-1 A* g
f = (A* A + I )-1 A* g
Errors: e1, e2, e3,…em
equal weights
related by
k
kee 22
f = ( A* R-1 R-1 A + I ) A* R-1 R-1 g
R operator is chosen so as to make
the elements of the transformed error
vector “e “ mutually independent and
possessing the same weight
R A f = R g + R e
A f = g + e
Minimize (f * ·H ·f )) subject to constraint
| Re | 2 =constant = 2
R operator is subject to 3 conditions :
1..angle target size within limits ( Fourier image size
within the recording system’s MTF)
2..signal to noise ratio no more than 3:1(preferably 5:1)
3..antenna scanning limited to avoid Gain degradation
and Grating Lobes
TIKHONOV - PYTIEV Regularization
Assembling the Building Blocks...
Antenna Design
f = ( A* R-1 R-1 A + I ) A* R-1 R-1 g
Tikhonov - Pytiev Regularization
0 200 400 600 800
0
-5
-10
-15
Spatial Frequency M
TF
Mag
nitu
de (
dB)
fRayleigh = 45 fspatial = 450
Post - Facto Determination of “ “
Constraints
Knowledge of Measurement Errors { Including Error correlations }
Precise Modeling of Media { Characterization of EM-Interactions } Optimization
Multi-Disciplinary Approach
PHYSICS
EM-Media Properties,
Field Measurements
MATHEMATICS
Forward / Inverse Problem
Regularization
MATERIALS
Radome, Substrates,
Bond-Films
COMPUTERS &
INFORMATION
SCIENCES
Neural Network
Adaptive Algorithms,
Parallel Processing,
Simulations, Optimization
ANTENNA
Design / FabricationHIGH FREQUENCY I.C
Modules, Receiver
STRUCTURAL
MECHANICS
Thermal Cycling,
Rigidity,
Stress, Durability..
RADIOMETER
Design /Fabrication
LAB. Measurements
Radiometer Noise
N.F. Measurements
Antenna Characterization
Future Radiometer Trends: Multi-Layered Integrated System
Source: University of Michigan http://www.eecs.umich.edu/RADLAB/katehi.dir/cism/index.html
Integration
• Each Layer:• Amplification• Phase Shifting• Combining
Has An Integrated Function• High Monolithic
• Permanent Layer Bonding
• Device Integration• Monolithically• Flip-Chip• Lift-Off
• Monolithic Packaging
• Integrated Processors
Combining Network
Heat Sink
Scanning ArrayAntenna
LNAsPhase Shifters
Multi-layer Microstrip PatchArray Assembly
Substrate(Silicon Wafer,Teflon)
Microstrip Patch
Aperture CouplingSlots
Stripline Feed
Combining Network
MMICs - LNAs Phase Shifters
RESOURCES: SELECTED BIBLIOGRAPHY
• Charles W. Groetsch, “Inverse Problems in the Mathematical Sciences”. VIEWEG (Bertelsman Publishing Group International) 1993
• S. Twomey, “Introduction to Mathematics of Inversion in Remote Sensing and Indirect Measurements”. Dover Publications Inc., 1977
• “Ill-Posed Problems in the Natural Sciences”. Advances in Science and Technology in the USSR, Mathematics and Mechanics Series. (Edited by A. N. Tikhonov and A.V. Goncharsky) MIR Publishers, Moscow. 1987
• “Inverse Problems in scattering and Imaging”. NATO Advanced Research Workshop, 1991. ( Edited by: E. R. Pike and M. Bertero) Adam Hilger Publishers (IOP), Bristol, England.
• M. Bertero and P. Boccacci, “Introduction to Inverse Problems in Imaging”. Institute of Physics Publishing Ltd., 1998
• “Mathematics of Profile Inversion”. Workshop Proceedings (Edited by: L. Colin) NASA Ames Research Center, Moffett Field, California. July 12 - 16, 1971. (NASA TMX-62-150)