TERM PAPER ON The Compressive Sensing Based on …home.iitk.ac.in/~amritami/index_files/ee604_term_paper.pdf · INTRODUCTION: COMPRESSIVE SENSING Conventional approaches to sampling

TERM PAPER ON

The Compressive Sensing Based on

Biorthogonal Wavelet Basis

Submitted By:

Amrita Mishra 11104163

Manoj C 11104059

Under the Guidance of

Dr. Sumana Gupta

Professor

Department of Electrical Engineering

Indian Institute of Technology Kanpur

ABSTRACT: Compressive Sensing is one of the latest tools for simultaneous sensing and

compression of data. It enables a significant reduction in the sampling and computation costs

for signals having sparse representation in some basis. In this term paper, we use wavelet

transformations such as Haar, db4, db6 and db8 wavelets for implementation of CS. We also

test several wavelet bases from Biorthogonal family. The Error Ratio between the original

coefficient and the reconstructed coefficient, the PSNR of the original image and

reconstructed image, and the Elapsed Time were used as the measurement indexes. Lena

along with CVX and l1-regularised least squares optimizer (in MATLAB) is used in

experimental for obtaining the results. The section Discussion and Results talks about the

observations and inferences in detail.

INTRODUCTION:

COMPRESSIVE SENSING

Conventional approaches to sampling signals or images follow Shannon‘s celebrated

theorem: the sampling rate must be at least twice the maximum frequency present in the

signal (the so-called Nyquist rate). In fact, this principle underlies nearly all signal acquisition

protocols used in consumer audio and visual electronics, medical imaging devices, radio

receivers, and so on. (For some signals, such as images that are not naturally bandlimited, the

sampling rate is dictated not by the Shannon theorem but by the desired temporal or spatial

resolution. However, it is common in such systems to use an antialiasing low-pass filter to

bandlimit the signal before sampling, and so the Shannon theorem plays an implicit role.) In

the field of data conversion, for example, standard analog-to-digital converter (ADC)

technology implements the usual quantized Shannon representation: the signal is uniformly

sampled at or above the Nyquist rate. The theory of compressive sampling, also known as

compressed sensing or CS, a novel sensing/sampling paradigm goes against the common

wisdom in data acquisition. CS theory asserts that one can recover certain signals and images

from far fewer samples or measurements than traditional methods use. To make this possible,

CS relies on two principles:

sparsity, which pertains to the signals of interest, and incoherence, which pertains to the

sensing modality.

Sparsity expresses the idea that the ―information rate‖ of a continuous time signal

may be much smaller than suggested by its bandwidth, or that a discrete-time signal

depends on a number of degrees of freedom which is comparably much smaller than

its (finite) length. More precisely, CS exploits the fact that many natural signals are

sparse or compressible in the sense that they have concise representations when

expressed in the proper basis Ψ.

Incoherence extends the duality between time and frequency and expresses the idea

that objects having a sparse representation in Ψ must be spread out in the domain in

which they are acquired, just as a Dirac or a spike in the time domain is spread out in

the frequency domain. Put differently, incoherence says that unlike the signal of

interest, the sampling/sensing waveforms have an extremely dense representation in

Ψ.

The crucial observation is that one can design efficient sensing or sampling protocols

that capture the useful information content embedded in a sparse signal and condense it into a

small amount of data. These protocols are non-adaptive and simply require correlating the

signal with a small number of fixed waveforms that are incoherent with the sparsifying basis.

What is most remarkable about these sampling protocols is that they allow a sensor to very

efficiently capture the information in a sparse signal without trying to comprehend that

signal. Further, there is a way to use numerical optimization to reconstruct the full-length

signal from the small amount of collected data. In other words, CS is a very simple and

efficient signal acquisition protocol which samples—in a signal independent fashion—at a

low rate and later uses computational power for reconstruction from what appears to be an

incomplete set of measurements.

Mathematical Framework of CS technique

1. Acquire n << N measurements, using a ‗special‘ sampling matrix Φ , by computing

for a signal x

2. Since the dimension of vector of the acquired samples y is ‗substantially‘ smaller than

the dimension of the signal, we obviously obtain some initial compression, which can

be further augmented by applying lossy or lossless compression to the vector y.

3. Similarly to standard transform-based compression techniques, the paradigm of CS is

based on the assumption that the signal x has a sparse representation in some basis

such as wavelets. This means that we assume that there exists a known fixed

transform T, such that from the N (or more) transform coefficients c = Tx , only k < n

coefficients are significant. Working under this ‗sparsity‘ assumption an

approximation to x can be reconstructed from y by ‗sparsity‘ minimization, such as 1 l

minimization

A key assumption in the theory of CS is that the sampling process determined by the matrix

Φand the sparsity transform T are ‗incoherent‘. Roughly speaking, this means that if a signal

has a sparse representation in one, then it must have a dense representation in the other and

visa versa, but a signal cannot have a sparse representation in both.

Figure 1: (a) Sparse wavelet representation of an image. Black- significant coefficient, white

– insignificant coefficient (b) JPEG2000 compressed image based on the sparse

representation of (a)

BIORTHOGONAL WAVELET FAMILIES

The discrete wavelet transform can be represented in matrix form as equation, where Ψ is a

matrix with columns corresponding to othonormal scaling and wavelet basis vectors. The

functions that qualify as orthonormal wavelets, such as Daubechies wavelets, lack desirable

symmetry properties. The Biorthogonal wavelets use two different wavelet bases, ψ(x) and

. One is used for decomposition (analysis) and the other one for reconstruction

(synthesis) i.e

We then have,

for decomposition

and

for reconstruction.

The two scaling functions given in the frequency domain are

and the wavelets are

and

where

The biorthogonal wavelets for the forward two-dimentional transform are given as

For the inverse transform

WAVELET TREE STRUCTURE COMPRESSIVE SENSING

The wavelet coefficients may be represented in terms of a tree structure. To a real

image, most of the significant wavelet coefficients are located in the vicinity of edges.

Wavelets can be regarded as multi-scale local edge detectors. The absolute value of a wavelet

coefficient corresponds to the local strength of the edge. The 3 level wavelet tree structure is

depicted in Figure 2 and the wavelet tree structure of Lena is shown in Figure 3. The

coefficients at the highest of the left in Figure 1 correspond to ―root nodes‖. And the

coefficients at the bottom or right in the figure correspond to ―leaf nodes‖. The top-left block

corresponds to the scaling coefficients which capture the coarse-scale representation of the

image. Each wavelet coefficient has four ―children‖ coefficients at the next level, and it has

the statistical relationship between the ―parent‖ and ―children‖ coefficients. This could be

exploited in the CS inversion model.

Figure 2: The 3 level wavelet tree structure

Figure 3: The 3 level wavelet tree structure of Lena.

(Two wavelet trees are shown in the figure)

A wavelet coefficient has a small value, then its children coefficients are likely to also

negligible. The statistics of the wavelet coefficients may be represented by the hidden

Markov tree. The structure of the wavelet tree is exploited explicitly. Each wavelet

coefficient is assumed to be drawn from one of two zero-mean Gaussian distributions in

hidden Markov tree. These distributions define the observation statistics for two hidden

states. One of the states is a ―low‖ state, defined by a small Gaussian variance. And the

―high‖ state is defined by a large variance. If a wavelet coefficient is relatively small, it is

more likely to reside in the ―low‖ state. A large wavelet coefficient has a high probability of

coming from the ―high‖state.

We do not get the wavelet coefficients directly in compressive sensing. We only get

the projections of these coefficients. The form of the hidden Markov tree will be used in the

compressive sensing inversion. If a given coefficient is negligible, we can scale its children

coefficients as ―zero‖. The subtrees of these wavelet coefficients may all be set to zero with

a little effect on the reconstruction accuracy.

Experiment:

We performed compressive sampling on the standard Lena image. Due to

computational limitations, we resized the Lena image to 64×64 pixels and performed

compressive sampling on the resized image. The original Lena image used is shown in Figure

4. The experiments were carried out in MATLAB 7.11.0 running on PC with Intel® Core™ i7

2.93 GHz CPU with 4 GB RAM.

Figure 4: Original Lena image

CS was performed using various wavelets – Haar, Daubechies-4 (db4), Daubechies-6

(db6), Daubechies-8 (db8), biorthogonal 2.8 and biorthogonal 3.5 wavelets. We also used

CVX and l1-regularized lease squares solver (l1_ls) for reconstructing the image from the

compressed samples. The performance indicators Peak Signal to Noise Ratio (PSNR), time

taken and the visual appearance of the reconstructed image were used for comparing the

performance of various wavelets and the solvers. We reconstructed the 64×64 image (4096

coefficients) using 750, 1000, 1250, 1500, 1750, 200, 2500, 3000, 3500, 4000 measurements.

Results:

The experimental results obtained by compressing the Lena image using CVX are

shown in Table 1. The performance indicators time taken in seconds, PSNR in dB and the

reconstructed image are tabulated. As expected, the PSNR and the image quality increase as

the number of measurements increases. Among the several wavelets considered, we can

observe that the biorthogonal wavelets give a good reconstruction. The time taken for

reconstruction of biorthogonal 3.5 wavelets is the least compared to other wavelets.

Table 1: Experimental results obtained using CVX.

No. of

measurements haar db4 db6 db8 bior2.8 bior3.5

750

Time 97 99 193 189 118 99

PSNR 12.88 13.17 13.11 12.88 13.03 12.88

Image

1000

Time 162 178 305 303 200 182

PSNR 13.00 13.18 12.79 12.91 12.65 12.74

Image

1250

Time 270 279 389 352 300 273

PSNR 13.26 13.18 13.06 13.47 13.12 12.60

Image

1500

Time 383 397 52 529 467 386

PSNR 12.98 13.38 13.42 13.29 13.47 12.81

Image

1750

Time 513 551 714 648 605 529

PSNR 13.46 13.70 13.5 13.52 13.20 13.05

Image

2000

Time 723 722 847 871 733 712

PSNR 14.16 14.22 14.11 14.17 13.34 12.65

Image

2500

Time 1096 1167 1268 1397 1157 1185

PSNR 15.44 16.66 16.10 16.29 13.96 13.28

Image

3000

Time 1583 1695 1972 1981 1806 1844

PSNR 22.85 21.49 20.68 21.39 19.90 16.31

Image

3500

Time 2342 2236 2387 2423 2327 2219

PSNR 30.26 28.09 28.08 28.25 28.92 28.25

Image

4000

Time 2641 2597 2769 2728 2879 2687

PSNR 43.74 43.12 41.99 41.68 44.20 43.50

Image

We also performed the same experiment using l1-regularised least squares solver

(l1_ls) to reconstruct the image from compressed samples. The results corresponding to l1_ls

are tabulated in Table 2.

Table 2: Experimental results obtained using l1-regularised least squares solver.

No. of

measurements haar db4 db6 db8 bior2.8 bior3.5

750

Time 142 178 231 178 168 375

PSNR 14.30 14.27 14.58 14.24 13.69 14.61

Image

1000

Time 277 241 351 233 476 635

PSNR 15.23 15.00 15.36 15.29 14.36 15.23

Image

1250

Time 300 326 425 395 483 721

PSNR 15.86 15.78 15.75 15.51 15.69 15.73

Image

1500

Time 388 368 357 462 582 1505

PSNR 16.73 16.98 15.43 17.03 16.96 16.20

Image

1750

Time 469 564 431 447 596.85 1237

PSNR 18.31 18.04 17.70 17.81 17.03 17.24

Image

2000

Time 590 480 641 529 836 981

PSNR 19.78 18.29 18.04 17.46 18.61 16.00

Image

2500

Time 813 631 870 592 1155 24110

PSNR 21.38 20.09 21.33 20.00 20.60 20.18

Image

3000

Time 1103 1097 713 1128 1065 2784

PSNR 26.72 23.13 24.78 24.63 24.29 22.92

Image

3500

Time 1070 1211 1214 1278 2293 3192

PSNR 32.67 29.84 29.24 27.73 29.68 28.04

Image

4000

Time 962 1024 1023 1035 1293 3523

PSNR 44.87 42.05 41.77 40.57 41.58 40.68

Image

We can observe that l1_s gives better performance in terms of both PSNR and time

taken. We also conclude that the compression performance depends on the solver used to

reconstruct the image from the compressed samples. This is the reason why the performance

observed is different from that given in the paper.

Contribution:

The reconstructed image is not of very good quality when number of measurements is

less. So, we tried denoising the image with by applying Total Variation denoising. Total

variation denoising was applied on images reconstructed using l1_ls only and the results are

shown in Table 3.

Table 3: Experimental results obtained using l1-regularised least squares solver.

No. of

measurements haar db4 db6 db8 bior2.8 bior3.5

750

Time 142 178 231 178 168 375

PSNR 15.49 15.33 15.56 15.21 14.55 15.06

Image

1000

Time 277 241 351 233 476 635

PSNR 15.72 15.74 16.62 15.82 15.97 15.90

Image

1250

Time 300 326 425 395 483 721

PSNR 17.68 17.04 17.05 17.19 16.72 16.69

Image

1500

Time 388 368 357 462 582 1505

PSNR 17.57 17.62 17.42 17.64 18.00 17.29

Image

1750

Time 469 564 431 447 596.85 1237

PSNR 19.22 19.16 18.89 18.61 18.01 17.89

Image

2000

Time 590 480 641 529 836 981

PSNR 20.26 19.79 19.62 19.99 19.79 18.79

Image

2500

Time 813 631 870 592 1155 24110

PSNR 21.27 19.82 20.47 20.43 19.79 20.61

Image

3000

Time 1103 1097 713 1128 1065 2784

PSNR 22.44 22.80 22.67 22.72 22.90 22.41

Image

3500

Time 1070 1211 1214 1278 2293 3192

PSNR 23.95 23.58 23.36 23.19 23.69 23.57

Image

4000

Time 962 1024 1023 1035 1293 3523

PSNR 24.13 24.14 24.12 24.11 24.11 24.11

Image

From the results, we can infer that the total variation filtering improves the image

quality and PSNR when number of measurements taken is less. The improvement in PSNR is

around 1-2 dB. When the number of measurements increases, the image reconstructed is

almost similar to the original image. So, the total variation filtering has resulted in blurring of

the image, resulting in a decreased PSNR. Since compression and compressive sampling aim

at reducing the number of measurements taken, number of measurements taken in practice is

very less. So, the PSNR can be increased by using denoising after reconstructing the image.

Conclusion:

In this paper, we test the quality of Haar, db4, db6, db8,bior2.8 and bior3.5 wavelet

basis for the implementation of CS and it is observed that Biorthogonal wavelets given a

better reconstruction. We used both CVX and l1-regularised least squares optimizer (in

MATLAB) to perform the optimisation step in CS and observe that l1-regularised least

squares optimizer gave better performance with respect to both PSNR and time taken. We

observed that the reconstructed image is not of very good quality when number of

measurements is less. So, we tried denoising the image with by applying Total Variation

denoising (results provided only for l1-regularised least squares optimized images) and

observed the PSNR can be increased by using denoising after reconstructing the image. We

also concluded that the type of wavelet basis used in CS depends upon the image to be

compressed. For an image with more details like Lena, the best performance was achieved for

Biorthogonal Wavelets. It can be seen that for a image with fewer details other wavelets like

Daubechies also do a fair job. Even the number of pixels required for proper reconstruction

would be less for an image with fewer details.