Compressed Sensing Algorithms for Electromagnetic Imaging Applications A Thesis Presented by Richard Obermeier to The Department of Electrical and Computer Engineering in partial fulfillment of the requirements for the degree of Master of Science in Electrical and Computer Engineering Northeastern University Boston, Massachusetts December 2016
87
Embed
Compressed sensing algorithms for electromagnetic imaging ... · Abstract of the Thesis Compressed Sensing Algorithms for Electromagnetic Imaging Applications by Richard Obermeier
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Compressed Sensing Algorithms for Electromagnetic Imaging
Applications
A Thesis Presented
by
Richard Obermeier
to
The Department of Electrical and Computer Engineering
2.1 Comparison of dielectric constant of various breast tissues as a function of frequency. 62.2 Comparison of conductivities of various breast tissues as a function of frequency. . 72.3 Conceptual diagram of the DBT measurement process. . . . . . . . . . . . . . . . 72.4 Conceptual diagram of the NRI measurement process. . . . . . . . . . . . . . . . . 82.5 Overview of the Hybrid DBT / NRI system processing. . . . . . . . . . . . . . . . 92.6 Overview of the DBT segmentation process. . . . . . . . . . . . . . . . . . . . . . 102.7 Comparison of the dielectric constant composite model to the measurements pre-
sented in [10]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.8 Comparison of the conductivity composite model to the measurements presented in
3.1 Depiction of the basis pursuit problem Eq. 3.18. . . . . . . . . . . . . . . . . . . 213.2 Depiction of the PCCS basis pursuit problem of Eq. 3.19. . . . . . . . . . . . . . 213.3 Reconstruction performance of CS and PCCS programs in electromagnetic imaging
example as a function of sparsity level. . . . . . . . . . . . . . . . . . . . . . . . . 453.4 Real and imaginary parts of true contrast variable χε obtained when the DBT image
is segmented perfectly. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463.5 Real and imaginary parts of reconstructed contrast variable χε obtained when the
DBT image is segmented perfectly and there is no measurement noise. . . . . . . . 473.6 Real and imaginary parts of true contrast variable χε obtained when the fat percentage
is segmented from the DBT image with 10% error. . . . . . . . . . . . . . . . . . 473.7 Real and imaginary parts of reconstructed contrast variable χε obtained when the
fat percentage is segmented from the DBT image with 10% error and there is nomeasurement noise. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.8 Real and imaginary parts of reconstructed contrast variable χε obtained when thefat percentage is segmented from the DBT image with 10% error and and themeasurement SNR = 49dB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.9 Real and imaginary parts of reconstructed contrast variable χε obtained when thefat percentage is segmented from the DBT image with 10% error and and themeasurement SNR = 43dB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.2 Permittivity distribution of the optimized reflection mode antenna. . . . . . . . . . 574.3 log2 of the singular values of the sensing matrices obtained using the optimized
4.4 Numerical comparison of the reconstruction accuracies of Eq. 4.24 using the opti-mized reflection mode design (blue) and baseline reflection mode design (red). . . 59
4.6 Permittivity distribution of the optimized transmission mode antenna. . . . . . . . 604.7 log2 of the singular values of the sensing matrices obtained using the optimized
FISTA Fast Iterative Shrinkage Thresholding Algorithm
FDFD Finite Differences in the Frequency Domain
GMRES Generalized Minimum Residual
HWC High Water Content
LWC Low Water Content
MAP Maximum a-posteriori
MIMO Multiple Input Multiple Output
MRI Magnetic Resonance Imaging
NRI Nearfield Radar Imaging
PEC Perfect Electric Conductor
PCCS Physicality Constrained Compressed Sensing
RIP Restricted Isometry Property
v
Acknowledgments
Let me start off by saying that working on this thesis was no easy task for me. It wasextremely difficult at times to balance my research work load with my commitments to my employer,Raytheon BBN Technologies, as the two were completely orthogonal from each other. Luckily, therewere many people by my side that supported me on my endeavor. First and foremost, I would liketo thank my employer, Raytheon BBN Technologies. Without Raytheon BBN’s financial supportand flexible work hours, it would have been even more challenging for me to complete this thesis. Iwould like to thank my coworkers Steve Weeks, Paul Dryer, and Rob McGurrin for supporting me onthis journey and for always encouraging me to better myself. I would like to thank my adviser, Prof.Jose Angel Martinez Lorenzo, for his support and encouragement over the past few years, and forbeing understanding of my occasional “breaks” from research whenever I became too overwhelmedby the combined work / research work load. I would like to thank my good friend, Fernando Quivira,for directly and indirectly pushing me to achieve this goal. Last, but certainly not least, I would liketo thank my parents, who have encouraged me to pursue excellence my entire life, and who raisedme to be the man that I am today.
vi
Abstract of the Thesis
Compressed Sensing Algorithms for Electromagnetic Imaging
Applications
by
Richard Obermeier
Master of Science in Electrical and Computer Engineering
Northeastern University, December 2016
Prof. Jose Angel Martinez-Lorenzo, Advisor
Compressed Sensing (CS) theory is a novel signal processing paradigm, which statesthat sparse signals of interest can be accurately recovered from a small set of linear measurementsusing efficient `1-norm minimization techniques. CS theory has been successfully applied tomany sensing applications, such has optical imaging, X-ray CT, and Magnetic Resonance Imaging(MRI). However, there are two critical deficiencies in how CS theory is applied to these practicalsensing applications. First, the most common reconstruction algorithms ignore the constraintsplaced on the recovered variable by the laws of physics. Second, the measurement system must beconstructed deterministically, and so it is not possible to utilize random matrix theory to assess theCS reconstruction capabilities of the sensing matrix.
In this thesis, we propose solutions to these two deficiencies in the context of electromag-netic imaging applications, in which the unknown variables are related to the dielectric constant andconductivity of the scatterers. First, we introduce a set of novel Physicality Constrained CompressedSensing (PCCS) optimization programs, which augment the standard CS optimization programsto force the resulting variables to obey the laws of physics. The PCCS problems are investigatedfrom both theoretical and practical stand-points, as well as in the context of a hybrid Digital BreastTomosynthesis (DBT) / Nearfield Radar Imaging (NRI) system for breast cancer detection. Ouranalysis shows how the PCCS problems provide enhanced recovery capabilities over the standard CSproblems. We also describe three efficient algorithms for solving the PCCS optimization programs.
Second, we present a novel numerical optimization method for designing so-called “com-pressive antennas” with enhanced CS recovery capabilities. In this method, the constitutive pa-rameters of scatterers placed along a traditional antenna are designed in order to maximize thecapacity of the sensing matrix. Through a theoretical analysis and a series of numerical examples, we
vii
demonstrate the ability of the optimization method to design antenna configurations with enhancedCS recovery capabilities. Finally, we briefly discuss an extension of the design method to MultipleInput Multiple Output (MIMO) communication systems.
viii
Chapter 1
Introduction
Sensing systems attempt to extract as much information as possible about an object under
test by recording a set of independent measurements. The number of measurements and the degree of
their independence, as well as the physical limitations of the sensing modality, determine how much
information can be extracted by the sensing system. In general, the reconstruction accuracy of an
imaging system can be improved by adding more measurements. However, great care must be taken
when adding these measurements. In addition to exacerbating a number of practical issues such as
cost and processing power requirements, naively adding measurements often leads to diminishing
returns in the reconstruction accuracy.
Electromagnetic imaging systems, as the name suggests, attempt to reconstruct an image
of the object under test using electromagnetic field measurements. In general, these systems use
multiple transmitting antennas, which are distributed throughout the imaging domain, in order to
excite the object under test with broadband electromagnetic waveforms. These signals interact with
the objects in the imaging region in order to produce the scattered fields that are measured by a set
of receiving antennas. Using a model for the electromagnetic field propagation, these systems can
create an image of the objects within the imaging region. The physical meaning of the image and the
suite of reconstruction algorithms available for use depend upon the specific model that is employed.
For example, radar imaging systems often utilize linear models, which only consider the phase of the
electric field vector as it radiates in the background medium, typically freespace. This allows fast and
computationally efficient inversion methods to be used in order to generate images in quasi-real-time.
Unfortunately, the simplified linear model comes with the drawback that the reconstructed image
only recovers the scatterers’ so-called scalar reflectivity, which cannot easily be traced back to the
constitutive parameters, dielectric constant and conductivity, that govern electromagnetic radiation.
1
CHAPTER 1. INTRODUCTION
More accurate methods, such as the Contrast Source (CSI) algorithm [1, 2, 3], use the full non-
linear model for electromagnetic radiation in order to reconstruct the constitutive parameters of the
scatterers. Unfortunately, these methods tend to be slow and computationally expensive. The Born
Approximation (BA) provides middle ground between the phase-based models of radar imaging
and the accurate, but expensive non-linear methods. In the BA, the non-linear model defined by
Maxwell’s equations is linearized about some starting point, such that the resulting unknown quantity
is intimately related to the constitutive parameters of the scatterers. From here, one can simply use
any number of linear inversion techniques in order to estimate the constitutive parameters of the
scatterers.
Compressed Sensing (CS) theory is a novel signal processing paradigm, which states that
sparse signals of interest can be accurately recovered from a small set of linear measurements, even
when the number of measurements is less than the number of unknowns. In order for CS theory
to be exploited by a sensing system, several conditions must be met. First, as the definition of CS
theory implies, the unknown object of interest must have a sparse representation in some known
domain. Second, the measurement, or sensing matrix must be sufficiently “well-behaved” such that it
obeys a Restricted Isometry Property (RIP). Third, the imaging system must utilize a reconstruction
algorithm that exploits the sparsity priors. Efficient techniques based upon minimizing the `1-norm
are by far the most common techniques used in the field of CS. If the sensing system satisfies these
conditions, then CS theory can be applied in order to recover super-resolution images of the object
under test when compared to alternative methods.
CS theory has been successfully applied in many electromagnetic applications [4, 5, 6].
However, there are two critical deficiencies in how CS theory is applied to these applications. First,
standard CS theory does not always properly exploit all of the prior knowledge that is available in
electromagnetic imaging applications. In particular, when the BA is applied to form a linearized
scattering model, CS theory does not enforce the physical limitations placed upon the dielectric
constant and conductivity by the laws of physics. Intuitively, one expects the reconstruction accuracy
to improve if these so-called physicality constraints are enforced. Unfortunately, the most common
solvers used in industry and academia are specialized for the standard `1-norm minimization programs
of CS theory, such that the physicality constraints cannot easily be enforced. Second, there is no
straight-forward way to design measurement configurations such that the resulting sensing matrix
obeys the RIP. For reasons that will become clear to the reader in Chapter 3, it is NP hard to
determine whether or not given sensing matrix obeys the RIP. To overcome this, researchers have
resorted to using random matrix theory in order construct sensing matrices that obey the RIP with
2
CHAPTER 1. INTRODUCTION
high probability. Unfortunately, this approach cannot be employed in electromagnetic imaging
applications.
1.1 Contributions
This thesis has two main contributions to the application of CS theory in electromagnetic
imaging applications. First, we introduce the concept of Physicality Constrained Compressed Sensing
(PCCS). In PCCS, the standard `1-norm optimization programs of CS are augmented to force the
resulting variables to obey the laws of physics. This thesis considers PCCS from a purely theoretical
perspective, using the same tool sets that are often employed in standard CS theory, as well as from a
practical perspective. With regards to the latter, PCCS is investigated in the context of a hybrid Digital
Breast Tomosynthesis (DBT) / Nearfield Radar Imaging (NRI) system for breast cancer detection. In
general, a standalone NRI system cannot enforce sparsity to detect cancerous lesions in the breast.
However, by fusing NRI with DBT, the hybrid system is able to generate an appropriate reference
distribution for the BA so that, in theory, the breast cancer detection problem can be posed as a
sparse recovery problem. This application is investigated using a 2D full-wave model based on Finite
Differences in the Frequency Domain (FDFD) [7] in order to accurately model electromagnetic wave
propagation within the breast. PCCS is also investigated in the context of general electromagnetic
imaging applications. This analysis shows how PCCS enhances the image reconstruction capabilities
of standard CS theory. This thesis also describes in great detail three efficient algorithms for solving
the PCCS optimization programs. Each of the algorithms excels in different applications, depending
upon the size of the problem and the computational resources available.
In the second contribution, we describe a novel numerical optimization method for design-
ing so-called “compressive antennas” with enhanced CS recovery capabilities. Through a theoretical
analysis, we demonstrate how enhancing the capacity of the sensing matrix improves the lower bound
on CS reconstruction performance, as measured by the RIP. In the design method, the constitutive
parameters of scatterers placed along a traditional antenna are optimized in order to maximize the
capacity of the antenna. The design method is briefly described in its most general form, before it is
discussed in detail in simplified forms that are specialized to scatterers that are pure dielectrics and
to scatterers that consist of Electric-LC (ELC) metamaterial elements. We also briefly discuss an
extension of the design method to Multiple Input Multiple Output (MIMO) communication systems.
Using several numerical examples, which again utilize the 2D FDFD in order to accurately model
electromagnetic wave propagation, we demonstrate the ability of the optimization method to design
3
CHAPTER 1. INTRODUCTION
antenna configurations with enhanced capacity and enhanced CS recovery capabilities.
1.2 Outline
The remainder of this thesis is organized as follows. In Chapter 2, we introduce the concept
of the hybrid DBT / NRI system for breast cancer detection. Within this section, we describe the
linearized model for the electromagnetic sensing problem using the BA. This linearized model serves
as the basis for the CS imaging algorithms, which are described in Chapter 3. We begin this section
with a brief introduction to standard CS techniques, before transitioning to the specialized PCCS
techniques. In Chapter 4, we describe the novel compressive antenna design method, which is based
upon the maximization of the channel capacity, and assess its performance with a set of numerical
examples. Finally, in Chapter 5 we conclude the thesis by describing some interesting extensions and
improvements to the work presented herein that will be topics of future research.
4
Chapter 2
Hybrid DBT / NRI System for Breast
Cancer Detection
2.1 Motivation
A recent report by the Center for Disease Control and Prevention [8] states that breast
cancer is the most common type of cancer among women, with a rate of 118.7 cases per 100, 000
women, and that it is the second deadliest type of cancer among women, with a mortality rate of 21.9
deaths per 100, 000 women. It is well known that the detection of breast cancer in its early stages
can greatly improve a woman’s chance for survival, as the lesions tend to be smaller and are less
likely to have spread from the breast than more developed cancer. Although small cancers near the
surface of the breast can be detected by means of a clinical breast exam (CBE), cancers deep within
the breast can only be detected through non-invasive imaging.
Conventional Mammography (CM) is a widely used X-ray-based technology, which creates
a two-dimensional image of the breast. Because CM only creates a two-dimensional image of the
three-dimensional breast, overlapping tissue from different cross-sections of the breast can degrade
the quality of the images. Digital Breast Tomosynthesis (DBT) improves CM by generating a three-
dimensional image of the breast [9], thereby mitigating the effects of tissue overlap. Unfortunately,
CM and DBT both suffer from the small radiological contrast between healthy tissue and cancerous
tissue, which is on the order of 1%. As a result, these technologies tend to produce a large number of
false positives when used for early detection.
Nearfield Radar Imaging (NRI) is a less common technology for breast cancer detection.
5
CHAPTER 2. HYBRID DBT / NRI SYSTEM FOR BREAST CANCER DETECTION
Unlike CM and DBT, NRI excites the breast using non-ionizing microwave radiation. NRI is an
appealing technology for breast cancer detection because the contrast between healthy breast tissue
and cancerous tissue is on the order of 10% at microwave frequencies [10]. This result can be seen in
Figures 2.1 and 2.2, which display the dielectric constant and conductivity of various breast tissues
as a function of frequency. Unfortunately, the improved contrast between healthy breast tissue and
cancerous tissue comes at a cost: at microwave frequencies, the mutual coupling between the different
tissue types cannot be ignored, such that it is difficult to accurately model wave propagation within
the heterogeneous distribution of tissues within the breast. Without an accurate wave propagation
model, NRI systems fail to detect cancerous lesions within the breast.
Figure 2.1: Comparison of dielectric constant of various breast tissues as a function of frequency.
2.2 System Overview
Recent papers [11, 12, 13] have introduced the concept of a Hybrid DBT / NRI system for
breast cancer detection. The basic idea behind the hybrid system is that, by combining the strengths
of both DBT and NRI at microwave frequencies, the detection rate of cancerous lesions can be
improved. In this section, we provide an overview of how the hybrid system could be used in a
clinical setting. The hybrid system operates in a similar manner to a conventional mammogram
system. To start, the breast is placed under clinical compression in order to ensure that there is
6
CHAPTER 2. HYBRID DBT / NRI SYSTEM FOR BREAST CANCER DETECTION
minimal movement throughout the sensing process. Once the breast has been compressed, it is
excited by an X-ray source that is mechanically scanned over multiple view angles, and the radiation
that passes through the breast is measured by a set of detectors on the opposite side of the breast.
This process is depicted in Figure 2.3.
Figure 2.2: Comparison of conductivities of various breast tissues as a function of frequency.
Figure 2.3: Conceptual diagram of the DBT measurement process.
7
CHAPTER 2. HYBRID DBT / NRI SYSTEM FOR BREAST CANCER DETECTION
At this point, the measurement process is complete in a conventional DBT system. In the
hybrid DBT / NRI system, however, the NRI measurements are recorded immediately after the DBT
measurements have been completed. This ensures that the measurements between the two systems
are co-registered. Any differences in the relative position of the breast between the two measurement
periods only inhibits the ability to successfully fuse the two systems; the sequential measurement
process minimizes the probability of this occurrence. In the NRI measurement process, one or more
transmitting and receiving antennas are mechanically scanned over the the breast, as depicted in
Figure 2.4. In this figure, a single transmitting antenna and an array of receiving antennas on the
opposite side of the breast are used, although other configurations, i.e. multiple monostatic, can also
be used. In order to minimize the reflections from the surface of the breast, the transmitting and
receiving antennas are placed in a plastic container filled with a bolus matching liquid. This liquid
has minimal effect at X-ray frequencies, and so the hybrid DBT / NRI system can utilize a modified
compression paddle configuration that can be used for both the DBT and NRI measurements.
Figure 2.4: Conceptual diagram of the NRI measurement process.
The remainder of this chapter describes the data processing methodology of the hybrid
DBT / NRI system. An overview of this process is presented in Figure 2.5. The data processing
can be separated into three primary components: 1) DBT Segmentation, 2) NRI Modeling, and 3)
Image Reconstruction. The basic premise is to use the DBT measurements and the resulting DBT
reconstruction in order to establish suitable priors for the NRI imaging process, so that the enhanced
8
CHAPTER 2. HYBRID DBT / NRI SYSTEM FOR BREAST CANCER DETECTION
contrast at microwave frequencies can be maximally exploited. These processing components are
described in detail in the next three sections.
Figure 2.5: Overview of the Hybrid DBT / NRI system processing.
2.3 DBT Segmentation
The DBT segmentation process can be divided into three primary components, as depicted
in Figure 2.5. The first component, the DBT measurement process, was discussed in the previous
section. In the second component, the DBT measurements are used to create a high-resolution three-
dimensional image of the X-ray attenuation coefficients of the compressed breast. DBT imaging
techniques have been established in the literature and are outside the scope of this thesis; see [9]
for details. This image is in turn used to segment the breast into three types of tissue, skin, muscle
(pectoralis major), and breast tissue, and it is assumed that the latter only contains healthy tissue.
Each voxel of breast tissue is further characterized by its percentage of fatty tissue and fibroglandular
tissue based upon the intensity of the DBT image, as is shown in Figure 2.6. This is possible because
the X-ray attenuation coefficient is proportional to the fat content of the tissues; high fat tissues
absorb less X-rays than tissues with low fat content and high water content.
The third and final component of the DBT segmentation process establishes the priors for
the NRI imaging process. The priors are described in terms of the frequency-dependent dielectric
constant εb(r, ω) and conductivity σb(r, ω) of the breast tissues. These constitutive parameters
9
CHAPTER 2. HYBRID DBT / NRI SYSTEM FOR BREAST CANCER DETECTION
Figure 2.6: Overview of the DBT segmentation process.
are extracted directly from the fat content segmentation using the composite model developed in
[14]. This composite model was created based upon the work of Lazebnik et. al in [10]. In their
work, Lazebnik et. al experimentally measured the dielectric constant and conductivity of breast
tissue samples of various fat and fibroglandular percentages, and fit the frequency-dependence of
the parameters to a Cole-Cole model. From this data, the composite model in [14] was developed in
order to establish the dielectric constant and conductivity of breast tissue compositions that were not
directly measured in the study. The results of this composite model are displayed in Figures 2.7 and
2.8 for the dielectric constant and conductivity respectively at a 5GHz frequency. The black curves
display interpolated sample points measured in [10], and the green curves display the results of the
composite model. Overall, the composite model fits the the measurements well.
2.4 NRI Modeling
The NRI modeling process consists of two main components, which can be performed
simultaneously. Given the dielectric constants and conductivities segmented from the DBT image,
the goal is to model the NRI measurement process of the assumed healthy breast. This process
can be described using electromagnetic theory: the electric fields Eb(r, ω) produced when the NRI
source distribution I(r, ω) excites the complex permittivity εb(r, ω) = εb(r, ω) + σb(r,ω)ωε0of the
breast tissues must satisfy the vector Helmholtz equation:
Researchers have applied the standard CS programs of Eq. 3.7, 3.8, and 3.9 in many
electromagnetic applications [4, 5, 6]. In electromagnetic-based tomographic imaging, one seeks
to reconstruct the constitutive parameters, dielectric constant and conductivity, of the object under
test. These parameters are bound by the fundamental constraints placed on them by the laws of
physics, namely εr ≥ 1 and σ ≥ 0. The standard CS programs do not consider these fundamental
limitations, so it is very much possible that the optimal solution recovered by these algorithms is
physically unrealizable. To overcome these pitfalls, we propose the following Physicality Constrained
Compressed Sensing (PCCS) optimization programs for electromagnetic-based tomographic imaging
18
CHAPTER 3. COMPRESSED SENSING IN ELECTROMAGNETIC IMAGING APPLICATIONS
applications:
(P1) minimizex
‖x‖`1 (3.14)
subject to ‖Ax− y‖`2 ≤ η
Re(diag(εb)x+ εb) � 1
Im(diag(εb)x+ εb) � 0
(P2) minimizex
λ‖x‖`1 +1
2
∥∥Ax− y∥∥2`2
(3.15)
subject to Re(diag(εb)x+ εb) � 1
Im(diag(εb)x+ εb) � 0
(P3) minimizex
1
2‖Ax− y‖2`2 (3.16)
subject to ‖x‖`1 ≤ τ
Re(diag(εb)x+ εb) � 1
Im(diag(εb)x+ εb) � 0
where εb is the vector containing the background complex permittivity at each point in the imaging
region. The PCCS programs are convex, despite the odd-looking box constraints on the real
and imaginary components of the complex vector. To make this explicit, the problems can be
written in an equivalent form in terms of three variables: the contrast x ∈ CN , the real part of the
permittivity εR ∈ RN , and the complex part of the permittivity εI ∈ RN . For example, the equivalent
representation of Eq. 3.14 is:
(P1) minimizex,εR,εI
‖x‖`1 (3.17)
subject to ‖Ax− y‖`2 ≤ η
εR � 1
εI � 0
x = diag(εb)−1(εR + εI − εb)
19
CHAPTER 3. COMPRESSED SENSING IN ELECTROMAGNETIC IMAGING APPLICATIONS
3.3.1 Theoretical Considerations for the PCCS Problems
The motivation for using the PCCS programs over the traditional CS versions is straight-
forward. Not only do they produce solutions that are physically realizable, but they should also
produce more accurate solutions than the traditional programs because they consider more prior
knowledge. In this section, we address several theoretical considerations for using the PCCS pro-
grams in electromagnetic imaging problems. Consider the following related physicality constrained
problem. Suppose that we wish to find the sparse solution to the linear equation y = Ax, where
y ∈ RM , x ∈ RN , given the constraint that the elements of x are strictly non-negative, i.e. x � 0.
Traditional CS recovers x using basis pursuit:
minimize ‖x‖`1 (3.18)
subject to Ax = y
Clearly, Eq. 3.18 does not enforce the physicality constraint on the variable x. Therefore, it is
possible that basis pursuit will compute a solution that violates the physicality constraint. One
example of this is displayed in Figure 3.1. In this figure, the blue line represents the values of x that
satisfy the equality constraint, the green shaded region represents the values of x that satisfy the
physicality constraint x � 0, and the red diamond represents the `1-ball of norm ‖x‖`1 = 1. Figures
of this form are often presented in order to describe why the `1-norm produces sparse solutions
as a heuristic. The optimal solution to Eq. 3.18 is the intersection of the `1-ball and the equality
constraint, x = [−1, 0]T . Clearly, this solution vector is infeasible, as it violates the physicality
constraint x � 0.
In contrast, PCCS recovers x using the modified basis pursuit problem:
minimize ‖x‖`1 (3.19)
subject to Ax = y
x � 0
Intuitively, one expects this solution to produce sparse solutions, given the `1-norm heuristic applied
to basis pursuit. However, one also expects that the additional physicality constraint should improve
the accuracy of the solution compared to basis pursuit. This can be seen in Figure 3.2, which is
similar to Figure 3.1 except that the `1-ball intersects the equality constraint at the solution to Eq.
3.19, x = [0, 1.5]T , which is the true sparse solution to this problem.
20
CHAPTER 3. COMPRESSED SENSING IN ELECTROMAGNETIC IMAGING APPLICATIONS
Figure 3.1: Depiction of the basis pursuit problem Eq. 3.18.
Figure 3.2: Depiction of the PCCS basis pursuit problem of Eq. 3.19.
Additional insight for the PCCS programs can be obtained from a statistical perspective.
Consider a scenario where noisy measurements y = Ax + n of the vector x ∈ RN are obtained,
where the elements of n ∈ RM are i.i.d. Gaussian with zero mean and variance σ2. If the elements
of x are i.i.d. Laplacian random variables with zero mean and scale parameter λ, then the maximum
21
CHAPTER 3. COMPRESSED SENSING IN ELECTROMAGNETIC IMAGING APPLICATIONS
a-posteriori (MAP) estimation technique computes the value of x that maximizes:
xMAP = argmaxx
p(x|y) = argmaxx
p(y|x)p(x)
p(y)(3.20)
= argmaxx
p(y)−1(2πσ2
)−M/2e− 1
2σ2‖y−Ax‖2`2
λN
2e−λ‖x‖`1
Maximizing instead over log p(x|y) leads to the following optimization program for xMAP:
xMAP = argminx
1
2σ2‖y −Ax‖2`2 + λ‖x‖`1 (3.21)
which has the same form as the basis pursuit denoising problem of Eq. 3.8. Given the equivalence
of Eq. 3.7 - 3.9 for appropriate values of η, λ, and τ , we can say that the traditional CS programs
compute the MAP estimate of the unknown vector x when its elements are distributed as i.i.d.
Laplacian random variables.
Consider now the same scenario, except that the elements of x are i.i.d. expontential
random variables with scale parameter λ. In this case, the MAP estimation technique selects the
value of x that maximizes:
xMAP = argmaxx
p(x|y) = argmaxx
p(y|x)p(x)
p(y)(3.22)
= argmaxx�0
p(y)−1(2πσ2
)−M/2e− 1
2σ2‖y−Ax‖2`2λNe−λ1
T x
Once again, maximizing over log p(x|y) leads to an alternative expression for xMAP:
xMAP = argminx�0
1
2σ2‖y −Ax‖2`2 + λ1Tx (3.23)
= argminx�0
1
2σ2‖y −Ax‖2`2 + λ‖x‖`1
which has a form similar to Eq. 3.15. Indeed, Eq. 3.15 reduces to Eq. 3.23 for electromagnetic
imaging problems in a free-space background when the scattering elements have negligible conduc-
tivity, so that Im(ε) = 0. This result obviously does not hold for all values of εb, but it does provide
a statistical motivation for enforcing the physicality constraints. Considering the `1-norm regularizer
as a term from the prior probability distribution on x, we find that the traditional CS programs assign
non-zero probability to values that are not physically realizable. Using the PCCS programs to enforce
physicality of the solution resolves this issue.
Our final example provides firm theoretical justification for the reconstruction capabilities
of the PCCS problems in the context of a related CS problem. Consider a scenario where noiseless
22
CHAPTER 3. COMPRESSED SENSING IN ELECTROMAGNETIC IMAGING APPLICATIONS
measurements y = Ax of the sparse vector x ∈ RN are obtained. Ideally, one would solve the
following `0-norm minimization problem to recover x:
(P0) minimizex
‖x‖`0 (3.24)
subject to Ax = y
Suppose that we have obtained a candidate solution z to Eq. 3.24 with ‖z‖`0 = S. It is easy to show
that z is the unique minimizer of Eq. 3.24 if and only if δ2S < 1, that is, there are no 2S-sparse
vectors in the nullspace of A. To prove this, suppose that there exists another solution w satisfying
‖w‖`0 = S and Aw = y. It follows from this that the vector z − w lies within the nullspace of
A, since Aw = Az → A(z − w) = 0. Since ‖z − w‖`0 ≤ 2S, non-trivial solutions z 6= w are
guaranteed to exist if δ2S ≥ 1. Therefore, if δ2S < 1, then z is guaranteed to be the unique minimizer
of Eq. 3.24.
Consider now the problem in which the elements of x are constrained to be strictly positive.
In this case, the sparsest vector can be recovered by solving the following `0-norm minimization
problem:
(P0) minimizex
‖x‖`0 (3.25)
subject to Ax = y
x � 0 (3.26)
How can we guarantee that a candidate solution z to Eq. 3.3.1 with ‖z‖`0 = S is the unique solution?
The requirement that δ2S < 1 derived for Eq. 3.24 is sufficient, but not necessary for guaranteeing
unique solutions to Eq. 3.3.1. To see this, once again suppose that there exists another S-sparse
solution w to this problem, which leads to the necessary condition that A(z −w) = 0 as before. The
difference this time, however, is that z−w is restricted in its sign pattern. The clearest way to see this
is to analyze the case S = Smin, where Smin is the smallest integer such that the condition δ2S < 1
is violated. This implies that z and w are supported on disjoint sets, i.e. wizi = 0 ∀i = 1, . . . , N .
Since z and w must be both physically realizable, i.e. z � 0 and w � 0, in order for them to be
solutions of Eq. 3.3.1, z − w must have exactly S positive values and S negative values. Therefore,
if the nullspace of A does not have any vectors with exactly S positive elements and S negative
elements, then z must be the unique solution to Eq. 3.3.1.
This result can easily be generalized to higher sparsity levels, where S > Smin. For these
sparsity levels, we cannot assume that w and z are necessarily disjoint. Indeed, if S = Smin + L for
23
CHAPTER 3. COMPRESSED SENSING IN ELECTROMAGNETIC IMAGING APPLICATIONS
L > 0, there exists nontrivial vectors w − z in the nullspace of A satisfying ‖w − z‖`0 = 2Smin ≤2S = 2Smin + 2L. We now consider the signs of the K ≤ L values in z −w where z and w overlap.
If zk > wk for all k in the overlapping set, then w − z has S negative values and S − L positive
values. However, if zk ≤ wk for any k, then w − z has fewer than S negative values. Therefore, the
sufficient condition for Eq. produce unique S-sparse solutions is that all vectors in the nullspace of
A with at least 2S elements have at least S + 1 negative elements.
At this point, the following question has not been answered: how do we solve the PCCS
programs defined in Eq. 3.14, 3.15, and 3.16? Indeed, the physicality constraints prevent traditional
CS solvers from being used to solve the PCCS programs. For small-scale problems, the PCCS
programs can of course be solved using a general purpose solver such as CVX [22, 23], which is an
interpreter for an interior point method solver chosen by the user. For large scale problems, however,
it has been shown that specialized algorithms outperform general purpose solvers in traditional CS
applications. To this end, we describe three efficient algorithms for solving the PCCS programs in
the following sections. Each of these algorithms have their own set of strengths and weaknesses that
make them more or less appropriate to use depending upon the specific conditions of the problem
being solved.
3.4 Solving the PCCS Programs using Nesterov’s Method
In this section, we describe a first-order algorithm for solving 3.15. This algorithm utilizes
subject to ω0,i ∈ {0, ω1, . . . , ωNf }, i = 1 . . . Nr
Note that the value ω0 = 0 was added to the constraint set in order to allow the metamaterials to
act like dielectrics at all frequencies. Clearly, this problem is NP-hard, as one must loop through
all (Nf + 1)Nr possible combinations of materials. Instead of looping through all combinations,
let us consider a sub-optimal greedy approach. In this method, a single resonant frequency ω0,i is
optimized over the set {0, ω1, . . . , ωNf } while all other ω0,j for j 6= i are held fixed. Once the value
for ω0,i is determined, the optimization procedure moves on to ω0,i+1. This process continues until
the set of ω0,i converge, or until some number of iterations through the list of Nr materials has been
met.
In order to test the greedy ELC optimization procedure, let us return to the reflection
mode optimization problem discussed in Section 4.5. Consider the reflection mode configuration of
Figure 4.1, where the scatterers added to the PEC reflector are ELC metamaterials with parameters
ε∞ = 1, α = 0.1, and β = 0.001. The objective is to select the resonant frequency ω0 ∈{0, 3.1, 3.2, 3.3, 3.4, 3.5} [GHz] such that the capacity of the antenna is maximized. Starting from
the baseline configuration, i.e. ω0 = 0 for all scatterers, the greedy optimization method was executed
for a single cycle through the 40 scatterer elements. Figure 4.14 displays the log2 of the singular
values of the baseline and optimized sensing matrices. The optimization procedure clearly improves
the capacity of the sensing matrix, as the condition number decreased from 36000 in the original
design to nearly 58 in the optimized design. The optimized design also leads to an improvement in
CS reconstruction capability, as can be seen in Figure 4.15.
68
CHAPTER 4. MODEL-BASED DESIGN METHOD FOR COMPRESSIVE ANTENNAS
Figure 4.14: log2 of the singular values of the sensing matrices obtained using the optimized reflection
mode antenna (blue) and original reflection mode antenna (red) in a multi-static configuration.
Figure 4.15: Numerical comparison of the reconstruction accuracies of Eq. 4.24 using the optimized