Top Banner
An Introduction to Signal Expansions and the Discrete Wavelet Transform James E. Fowler Department of Electrical & Computer Engineering Geosystems Research Institute Mississippi State University 1 / 38
38

Siganl Extension & DWT

Jan 16, 2016

Download

Documents

its good book
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Siganl Extension & DWT

An Introduction to Signal Expansions and the

Discrete Wavelet Transform

James E. Fowler

Department of Electrical & Computer EngineeringGeosystems Research Institute

Mississippi State University

1 / 38

Page 2: Siganl Extension & DWT

Hilbert Spaces

Vectors

C: set of complex numbers

R: set of real numbers

Z: set of integers

A vector x is a N-tuple

x = {x1, x2, . . . , xN} (1)

contained in CN (or RN)

2 / 38

Page 3: Siganl Extension & DWT

Hilbert Spaces

Vector Spaces

A vector space is any subset E ⊆ CN , coupled with addition and

multiplication operations, which satisfies:

commutation: x + y = y + x

association: (x + y) + z = x + (y + z); (αβ) x = α (βx)

distribution: α (x + y) = αx + αy; (α+ β) x = αx + βx

additive identity: there exists 0 ∈ E such that x + 0 = x

additive inverse: there exists −x ∈ E such that x + (−x) = 0

multiplicative identity: 1 · x = x

3 / 38

Page 4: Siganl Extension & DWT

Hilbert Spaces

Subspaces

A subspace is any set M ⊆ E closed under addition and

multiplication:

x, y ∈ M → x + y ∈ M (2)

x ∈ M, α ∈ C → αx ∈ M (3)

For S ⊂ E, the span of S, span (S), is a subset of E consisting of all

linear combinations of vectors in S:

span (S) =

{

i

αixi | xi ∈ S, αi ∈ C}

(4)

4 / 38

Page 5: Siganl Extension & DWT

Hilbert Spaces

Bases

The vectors S = {x1, x2, . . . } are linearly independent if

i

αixi = 0 (5)

is true only when

αi = 0, ∀i (6)

The vectors S = {x1, x2, . . . } ⊂ E form a basis of vector space E if◮ span (S) = E◮ the vectors of S are linearly independent

5 / 38

Page 6: Siganl Extension & DWT

Hilbert Spaces

Basis Expansion

A basis allows the representation, or expansion, of any vector in a

vector space using the basis vectors.

If S = {x1, x2, . . . } is a basis of E, and y ∈ E, then there exist

constants αi ∈ C such that

y =∑

i

αixi. (7)

The set of expansion coefficients αi is unique given y.

6 / 38

Page 7: Siganl Extension & DWT

Hilbert Spaces

Inner Product

An inner product on vector space E is a function mappingE × E → C such that

◮ 〈x + y, z〉 = 〈x, z〉+ 〈y, z〉◮ 〈x, αy〉 = α〈x, y〉◮ 〈x, y〉∗ = 〈y, x〉◮ 〈x, x〉 ≥ 0 with equality only if x = 0

A vector space equipped with an inner product is an inner-product

space

A complete inner-product space is a Hilbert space (completeness:

all Cauchy sequences converge to a vector in the vector space)

The norm of x is ‖x‖ =√

〈x, x〉Vector x and y are orthogonal iff 〈x, y〉 = 0

7 / 38

Page 8: Siganl Extension & DWT

Hilbert Spaces

Finite-Dimensional Spaces

N is finite: x = {x1, x2, . . . , xN} ∈ CN

The conventional inner product:

〈x, y〉 =N∑

i=1

x∗i yi (8)

The norm is then

‖x‖ =

N∑

i=1

|xi|2 (9)

Holds for RN also (R3 is 3D Euclidean space)

8 / 38

Page 9: Siganl Extension & DWT

Hilbert Spaces

Square-Summable Sequences

Let x[n] be a discrete-time signal (sequence):

n ∈ Z (10)

x[n] ∈ C, ∀n (11)

Hilbert space ℓ2(Z) is the set of all sequences x[n] ∈ C∞ such that

‖x‖ <∞ (12)

where

‖x‖ =√

〈x, x〉 (13)

〈x, y〉 =∞∑

n=−∞x∗[n] · y[n] (14)

9 / 38

Page 10: Siganl Extension & DWT

Hilbert Spaces

Square-Integrable Functions

Let f (t) be a continuous-time signal (function):

t ∈ R (15)

f (t) ∈ C, ∀t (16)

Hilbert space L2(R) is the set of all functions f (t) such that

‖f‖ <∞ (17)

where

‖f‖ =√

〈f, f〉 (18)

〈f, g〉 =∫ ∞

−∞f ∗(t)g(t) dt (19)

10 / 38

Page 11: Siganl Extension & DWT

Hilbert Spaces

Orthonormal Bases

A vector x is normalized when

‖x‖ = 1 (20)

An orthonormal set S = {xi} satisfies:

〈xi, xj〉 ={

1, i = j,

0, i 6= j(21)

An orthonormal set of vectors that is also a basis is an

orthonormal basis

11 / 38

Page 12: Siganl Extension & DWT

Hilbert Spaces

Orthonormal Basis Expansion

If S = {xi} is an orthonormal basis of Hilbert space E, then, for all

y ∈ E,

y =∑

i

αixi (22)

where the expansion coefficients are

αi = 〈xi, y〉 (23)

If S is merely an arbitrary basis, the expansion coefficients αi may

be difficult to calculate for a given vector y. The inner product

provides the coefficients for an orthonormal basis.

12 / 38

Page 13: Siganl Extension & DWT

Time and Frequency Expansions

Finite-Length Sequences

Consider the Hilbert space CN of length-N sequences—we willlook at two common bases for CN :

◮ time basis◮ frequency basis

An important signal: the Kronecker delta

δ[n] =

{

1, n = 0,

0, n 6= 0(24)

13 / 38

Page 14: Siganl Extension & DWT

Time and Frequency Expansions

The Time Basis

For 0 ≤ m ≤ N − 1, define vectors ψm

ψm[n] = δ[n− m] =

{

1, n = m,

0, n 6= m(25)

with discrete Fourier transform (DFT) Ψm

Ψm[k] = F[

δ[n− m]]

=

N−1∑

n=0

δ[n− m]e−jω0nk = e−jω0mk (26)

where ω0 = 2πN

14 / 38

Page 15: Siganl Extension & DWT

Time and Frequency Expansions

The Time Basis

The set S = {ψm,m = 0, . . . ,N − 1} forms an orthonormal basis of

length-N sequences:

〈ψm1,ψm2

〉 =N−1∑

n=0

δ[n− m1]δ[n− m2] =

{

1, m1 = m2,

0, m1 6= m2

(27)

For any length-N sequence x,

〈ψm, x〉 =N−1∑

n=0

δ[n− m]x[n] = x[m] (28)

x[n] =N−1∑

m=0

x[m]δ[n− m] =N−1∑

m=0

〈ψm, x〉ψm[n] (29)

15 / 38

Page 16: Siganl Extension & DWT

Time and Frequency Expansions

Orthonormal Time Basis for a Length-N Sequence

ψ0 = δ[n]

0 n1 2

× 〈ψ0, x〉︸ ︷︷ ︸

x[0]

=

〈ψ0, x〉ψ0

0 n1 2

Σ =

x[n]

0 n1 2

expanded as:

ψ1 = δ[n− 1]

0 n1 2

× 〈ψ1, x〉︸ ︷︷ ︸

x[1]

=

〈ψ1, x〉ψ1

0 n1 2

x[n]

0 n1 2

ψ2 = δ[n− 2]

0 n1 2

× 〈ψ2, x〉︸ ︷︷ ︸

x[2]

=

〈ψ2, x〉ψ2

0 n1 2

N−1∑ 16 / 38

Page 17: Siganl Extension & DWT

Time and Frequency Expansions

The Time Basis

In the time domain:

x[n] =N−1∑

m=0

〈ψm, x〉ψm[n] (30)

where ψm[n] = δ[n− m]← support: one value of n

In the frequency domain (take DFT of both sides):

X[k] =N−1∑

m=0

〈ψm, x〉Ψm[k] (31)

where Ψm[k] = F[

ψm[n]]

= e−jω0mk ← support: all values of k

17 / 38

Page 18: Siganl Extension & DWT

Time and Frequency Expansions

The Frequency Basis

Let φl[n] =1√N

ejω0nl where ω0 = 2πN

The set S′ = {φl, l = 0, . . . ,N − 1} forms an orthonormal basis of

length-N sequences

The DFT of length-N sequence x[n] is

X[l] = F[

x[n]]

=N−1∑

n=0

x[n]e−jω0nl =√

N

N−1∑

n=0

φ∗l [n]x[n] =√

N〈φl, x〉

(32)

The inverse DFT is

x[n] = F−1[

X[l]]

=1

N

N−1∑

l=0

X[l]ejω0nl =N−1∑

l=0

〈φl, x〉φl[n] (33)

18 / 38

Page 19: Siganl Extension & DWT

Time and Frequency Expansions

The Frequency Basis

In the time domain:

x[n] =N−1∑

l=0

〈φl, x〉φl[n] (34)

where φl[n] =1√N

ejω0nl ← support: all values of n

In the frequency domain (take DFT of both sides):

X[k] =N−1∑

l=0

〈φl, x〉Φl[k] (35)

where Φl[k] = F[

φl[n]]

=√

Nδ[k − l]← support: one value of k

19 / 38

Page 20: Siganl Extension & DWT

Time and Frequency Expansions

Representation of a SequenceExample for sequence of length N = 4

Time-basis Representation Frequency-basis Representation

ψm[n] = δ[n−m], Ψm[k] = e−jω0mk φl[n] =1

Nejω0nl, Φl[k] =

Nδ[k − l]

(corresponds to the “natural” time-domain representation) (corresponds to the frequency-domain representation given by DFT)

n

time

k

freq

uen

cy

0 1 2 3

0

1

2

3

ψ0 ψ2 ψ3ψ1

n

time

k

freq

uen

cy

φ0

φ3

φ1

φ2

0 1 2 3

0

1

2

3

20 / 38

Page 21: Siganl Extension & DWT

Time and Frequency Resolution

Resolution

Resolution: the ability to distinguish between two closely located

signal features

Time resolution: ∆n = time separation between bases

rt =1

∆n(36)

Frequency resolution: ∆k = frequency separation between bases

rf =1

∆k(37)

21 / 38

Page 22: Siganl Extension & DWT

Time and Frequency Resolution

Time-basis Representation Frequency-basis Representation

ψm[n] = δ[n−m], Ψm[k] = e−jω0mk φl[n] =1

Nejω0nl, Φl[k] =

Nδ[k − l]

n

time

k

freq

uen

cy

0 1 2 3

0

1

2

3

ψ0 ψ2 ψ3ψ1

∆n = 1, ∆k = N

rt = 1, rf = 1/N

n

time

k

freq

uen

cy0 1 2 3

0

1

2

3

φ0

φ3

φ1

φ2

∆n = N, ∆k = 1

rt = 1/N, rf = 1

22 / 38

Page 23: Siganl Extension & DWT

Time and Frequency Resolution

Continuous-Time Time/Frequency Bases

Time basis: Dirac delta, δ(t)

∆t = 0 rt =1∆t

=∞⇒

∆ω =∞ rf =1

∆ω = 0

(38)

Frequency basis: continuous-time Fourier transform

∆t =∞ rt =1∆t

= 0

⇒∆ω = 0 rf =

1∆ω =∞

(39)

23 / 38

Page 24: Siganl Extension & DWT

General Time-Frequency Tilings

Continuous-Time Time/Frequency Bases

Time and frequency (Fourier) bases are “all or nothing” (resolution

is 0 or∞)

We would like bases with 0 < rt, rf <∞Define “spread” (∆t or ∆ω) as second moment:

(∆t)2 =∫

t2|ψ(t)|2 dt∫|ψ(t)|2 dt

(∆ω)2 =∫ω2|Ψ(ω)|2 dω∫|Ψ(ω)|2 dω

time domain frequency domain(40)

24 / 38

Page 25: Siganl Extension & DWT

General Time-Frequency Tilings

The Uncertainty Principle

For any ψ(t) with Fourier transform Ψ(ω),

∆t∆ω ≥ 12

or rtrf ≤ 2 (41)

We can have arbitrarily large resolution (time or frequency) only by

giving up resolution of the other quantity

Example:◮ To get ∆t = 0, ∆ω must be∞ (which is time basis)◮ For ∆ω <∞, we must use ∆t > 0

25 / 38

Page 26: Siganl Extension & DWT

The Short-Time Fourier Transform

The Continuous-Time Fourier Transform

F(ω) = F [f (t)] is the set of coefficients of an expansion of

continuous-time function f (t) ∈ L2(R) using basis functions

ψω(t) = ejωt (42)

which provides a basis function for each ω ∈ RResolution:

∆ω = 0 (43)

∆t =∞ (44)

The short-time Fourier transform (STFT) attempts to improve the

time resolution by windowing each ψω(t) to reduce its time extent

26 / 38

Page 27: Siganl Extension & DWT

The Short-Time Fourier Transform

The STFT

The STFT:

f (t) =1

R

RS(τ, ω)ψτ,ω(t) dτ dω (45)

Basis functions—Fourier basis coupled with window function g(t):

ψτ,ω(t) = g(t − τ)ejωt (46)

Coefficients:

S(τ, ω) =⟨

ψτ,ω, f⟩

(47)

=

Rψ∗τ,ω(t)f (t) dt (48)

=

∫ ∞

−∞f (t)g(t − τ)e−jωt dt (49)

27 / 38

Page 28: Siganl Extension & DWT

The Short-Time Fourier Transform

The STFT Basis Functions

Basis function ψτ,ω(t) is indexed by two indices:◮ Time: τ◮ Frequency: ω

τ and ω determine location in time-frequency plane of the basis

function’s corresponding tile:

∆ω

∆ω

∆t

∆t

ω2

ω1

τ1 τ2t

ω

|ψτ2,ω2(t)|2|ψτ1,ω1

(t)|2

|Ψτ1,ω1(t)|2

|Ψτ2,ω2(t)|2

28 / 38

Page 29: Siganl Extension & DWT

The Short-Time Fourier Transform

The STFT Window

The window function, g(t), is chosen to give the desired resolution

tradeoff, subject to ∆t∆ω ≥ 12

Wide window (∆t large)→ good frequency resolution (∆ω small)

Narrow window (∆t small)→ poor frequency resolution (∆ω large)

f

t

f

t

Narrow window Wide window

29 / 38

Page 30: Siganl Extension & DWT

The Short-Time Fourier Transform

The Balian-Low Theorem

If the STFT basis functions ψτ,ω(t) form an orthogonal basis, then

either ∆t or ∆ω must be∞To have good resolution in time and frequency simultaneously in

STFT, one must use a non-orthogonal basis.

30 / 38

Page 31: Siganl Extension & DWT

The Short-Time Fourier Transform

Advantages of Orthonormal Bases

If f ∈ E is a vector in Hilbert space E, and S = {ψi} is a basis of E, then

f =∑

i

αiψi (50)

If S is an orthonormal basis:

Easy calculation of expansion coefficients:

αi = 〈ψi, f〉 (51)

Parseval’s theorem:

‖f‖2 =∑

i

|αi|2 (52)

That is, total energy of f is partitioned among the αi coefficients.

31 / 38

Page 32: Siganl Extension & DWT

Wavelet-Series Expansion

Wavelet-Series Expansion

Commonly called the “discrete wavelet transform” (DWT)

Expands a continuous-time function into coefficients which are

discrete in time and frequency:

f (t) =∑

k

j

aj,kψj,k(t) (53)

time: k ∈ Zfrequency: j ∈ ZSimilar to STFT in that decomposition is in terms of both time and

frequency—but, orthonormal wavelet bases exist with good time

and frequency resolution simultaneously

32 / 38

Page 33: Siganl Extension & DWT

Wavelet-Series Expansion

The Mother Wavelet

Wavelet-series expansion systems are generated from one

function, ψ(t), called the mother wavelet

The basis functions are created by scaling and translating the

mother wavelet:

ψj,k(t) = 2j/2ψ(2jt − k) for j, k ∈ Z (54)

The 2j/2 factor maintains normalization:

∥ψj,k

2= ‖ψ‖2 = 1 (55)

33 / 38

Page 34: Siganl Extension & DWT

Wavelet-Series Expansion

Scaling and Translation

ψ(t)

j = 0 j = 1 j = 2

34 / 38

Page 35: Siganl Extension & DWT

Wavelet-Series Expansion

Scaling and Translation

k controls displacement of ψj,k(t) in time

j is the scale—increasing j yields:◮ “compaction” of ψj,k(t) in time◮ ψj,k(t) functions spaced closer together in time◮ “taller” ψj,k(t) functions

When j is large:◮ ψj,k(t) functions have small ∆t; thus, ∆ω is large◮ good time resolution

When j is small:◮ ψj,k(t) functions have large ∆t; thus, ∆ω is small◮ good frequency resolution

35 / 38

Page 36: Siganl Extension & DWT

Wavelet Time-Frequency Tiling

time

frequency

36 / 38

Page 37: Siganl Extension & DWT

Wavelet Time-Frequency Tiling

Why is this a good tiling?

Natural signals are usually lowpass—wavelet tiling concentrates

frequency resolution where majority of signal energy resides

Discontinuities (edges) involve high-frequency energy that

accounts for a small portion of total energy—location in time is

more important than frequency composition—wavelet tiling has

good time resolution for high-frequency content

37 / 38

Page 38: Siganl Extension & DWT

Summary

Wavelets provide an expansion in the form:

f (t) =∑

k

j

ψj,k, f⟩

ψj,k(t) (56)

The expansion can be orthonormal—coefficients can be

calculated with inner products

The expansion is multiresolution—time and frequency resolutions

vary across time-frequency plane

The time-frequency tiling matches characteristics of natural

(lowpass) signals with salient discontinuities

38 / 38