1 / 43 3.14pt Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 1 / 43
44

# Wavelets - USTC

Mar 27, 2022

## Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript

1 / 43

3.14pt

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 1 / 43

1 / 43

Wavelets

Ming Li, Quan Xiao (USTC)

2019.06.30

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 1 / 43

2 / 43

Outline of this presentation

3.14pt

1 Introduction

2 Dilation Equation

3 Derivaton of the Wavelets from the Scaling Function

4 Sufficient Conditions for the Wavelets to be Orthogonal

5 Expressing a Function in Terms of Wavelets

6 Designing a Wavelet System

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 2 / 43

3 / 43

Outline

3.14pt

1 Introduction

2 Dilation Equation

3 Derivaton of the Wavelets from the Scaling Function

4 Sufficient Conditions for the Wavelets to be Orthogonal

5 Expressing a Function in Terms of Wavelets

6 Designing a Wavelet System

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 3 / 43

4 / 43

Fourier Transform

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 4 / 43

5 / 43

Fourier Transform

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 5 / 43

6 / 43

Difference between Fourier Transform and Wavelets

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 6 / 43

7 / 43

Dilation

A dilation equation is an equation where a function is defined interms of a linear combination of scaled, shifted versions of itself. Forinstance,

f(x) =

d−1∑k=0

ckf(2x− k)

Lemma 11.1

If a dilation equation f(x) =∑d−1

k=0 ckf(2x− k) has a solution, then∑d−1k=0 ck = 2 or

∫∞−∞ f(x)dx = 0.

Proof:∫ ∞−∞

f(x)dx =

∫ ∞−∞

d−1∑k=0

ckf(2x−k)dxFunbini=======

1

2

d−1∑k=0

ck

∫ ∞−∞

f(x)dx

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 7 / 43

8 / 43

The Haar Wavelet

Solve dilation equation f(x) = f(2x) + f(2x− 1), solution is

φ(x) =

{1, 0 ≤ x < 1

0, otherwise

The function φ is called a scale function.

Scaling and shifting of the basic scale function φ gives a twodimensional set of scale functions

φjk(x) = φ(2jx− k

)For each j, the set of functions φjk, k = 0, 1, 2 . . ., form a basis for avector space Vj and are orthogonal.

But for different values of j, φjk, j = 0, 1, 2 . . . are not orthogonal.

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 8 / 43

9 / 43

Haar wavelet scale functions

Since φjk, φj+1,2k and φj+1,2k+1 are linearly dependent, for eachvalue of j delete φj+1,k for odd values of k to get a linearlyindependent set of basis vectors.

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 9 / 43

10 / 43

Haar wavelet scale functions

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 10 / 43

11 / 43

To get an orthogonal set of basis vectors, define

ψjk(x) =

1 2k

2j≤ x < 2k+1

2j

−1 2k+12j≤ x < 2k+2

2j

0 otherwise

and replace φj,2k with ψj+1,2k.

For instance,

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 11 / 43

12 / 43

The Haar Wavelet

To approximate a function that has only finite support, select a scalevector φ(x).

Next approximate the function by the set of scale functionsφ(2jx− k

), k = 0, 1, . . ., for some fixed value of j.

Once the value of j has been selected, the function is sampled at 2j

points, one in each interval of width 2−j .

Let the sample values be s0, s1, . . . The approximation to the function

is∑2j−1

k=0 skφ(2jx− k

).

Our goal is to represent the approximation to the function using thebasis vectors rather than the nonorthogonal set of scale functionsφjk(x).

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 12 / 43

13 / 43

The Haar Wavelet Example

To represent the function corresponding to vector(3 1 4 8 3 5 7 9

), one needs to find the ci such that

The first column represents the scale function φ and subsequentcolumns the ψ’s.

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 13 / 43

14 / 43

The Haar Wavelet Example

Use tree methods to find the coefficients. Each vertex in the treecontains the average of the quantities of its two children.

The result is

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 14 / 43

15 / 43

Wavelet Systems

A wavelet system is built from a basic scaling function φ(x).

A basic scale function φ(x) comes from a dilation equation.

Scaling and shifting of the basic scale function gives a twodimensional set of scale functions

φjk(x) = φ(2jx− k

)For a fixed value of j, the φjk span a space Vj .

If φ(x) satisfies a dilation equation

φ(x) =

d−1∑k=0

ckφ(2x− k)

then φjk is a linear combination of the φj+1,k’s and this implies thatV0 ⊆ V1 ⊆ V2 ⊆ V3 · · · .

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 15 / 43

16 / 43

Outline

3.14pt

1 Introduction

2 Dilation Equation

3 Derivaton of the Wavelets from the Scaling Function

4 Sufficient Conditions for the Wavelets to be Orthogonal

5 Expressing a Function in Terms of Wavelets

6 Designing a Wavelet System

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 16 / 43

17 / 43

Solving the Dilation Equation

Consider solving a dilation equation

φ(x) =

d−1∑k=0

ckφ(2x− k)

to obtain the scale function for a wavelet system.

The easiest way is to assume a solution and then calculate the scalefunction by successive approximation.

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 17 / 43

18 / 43

Example

Example: solving

φ(x) =1 +√3

4φ(2x)+

3 +√3

4φ(2x−1)+3−

√3

4φ(2x−2)+1−

√3

4φ(2x−3)

Begin with the coefficients of the dilation equation.

c1 =1 +√3

4c2 =

3 +√3

4c3 =

3−√3

4c4 =

1−√3

4

Execute the following loop until the values for φ(x) converge.1 Calculate φ(2x) by averaging successive values of φ(x) together. Fill

out the remaining half of the vector representing φ(2x) with zeros.2 Calculate φ(2x− 1), φ(2x− 2), and φ(2x− 3) by shifting the contents

of φ(2x) the appropriate distance, discarding the zeros that move offthe right end and adding zeros at the left end.

3 Calculate the new approximation for φ(x) using the above values forφ(2x− 1), φ(2x− 2), and φ(2x− 3) in the dilation equation for φ(2x).

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 18 / 43

19 / 43

Another approach

Example: soving

φ(x) =1

2f(2x) + f(2x− 1) +

1

2f(2x− 2)

Consider continuous solutions with support in 0 ≤ x < 2:

φ(0) = 12φ(0) + φ(−1) + φ(−2) = 1

2φ(0) + 0 + 0 φ(0) = 0φ(2) = 1

2φ(4) + φ(3) + φ(2) = 12φ(2) + 0 + 0 φ(2) = 0

φ(1) = 12φ(2) + φ(1) + φ(0) = 0 + φ(1) + 0 φ(1) arbitrary

Set φ(1) = 1. Then

φ(12

)= 1

2φ(1) + φ(0) + 12φ(−1) =

12

φ(32

)= 1

2φ(3) + φ(2) + 12φ(1) =

12

φ(14

)= 1

2φ(12

)+ φ

(−1

2

)+ 1

2φ(−3

2

)= 1

4

One can continue this process and compute φ(i2j

)for larger values of

j until φ is approximated to a desired accuracy.

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 19 / 43

20 / 43

Conditions on the Dilation Equation

Lemma 11.2

Let

φ(x) =

d−1∑k=0

ckφ(2x− k)

If φ(x) and φ(x− k) are orthogonal for k 6= 0 and φ(x) has beennormalized so that

∫∞−∞ φ(x)φ(x− k)dx = δ(k), then∑d−1

i=0 cici−2k = 2δ(k).

Lemma 11.3

If 0 ≤ x < d is the support of φ(x), and the set of integer shifts,{φ(x− k)|k ≥ 0}, are linearly independent, then ck = 0 unless0 ≤ k ≤ d− 1.

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 20 / 43

21 / 43

Outline

3.14pt

1 Introduction

2 Dilation Equation

3 Derivaton of the Wavelets from the Scaling Function

4 Sufficient Conditions for the Wavelets to be Orthogonal

5 Expressing a Function in Terms of Wavelets

6 Designing a Wavelet System

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 21 / 43

22 / 43

Derivation of the Wavelets from the Scaling Function

Lemma 1

(Orthogonality of ψ(x) and ψ(x− k))Let ψ(x) =

∑d−1k=0 bkφ(2x− k). If ψ(x) and ψ(x− k) are orthogonal for

k 6= 0 and ψ(x) has been normalized so that∫∞−∞ ψ(x)ψ(x− k)dx = δ(k),

thend−1∑i=0

(−1)kbibi−2k = 2δ(k)

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 22 / 43

23 / 43

Derivation of the Wavelets from the Scaling Function

Lemma 2

(Orthogonality of φ(x) and ψ(x− k))Let φ(x) =

∑d−1k=0 ckφ(2x− k) and ψ(x) =

∑d−1k=0 bkφ(2x− k). If∫∞

−∞ φ(x)φ(x− k)dx = δ(k) and∫∞−∞ φ(x)ψ(x− k)dx = 0 for all k, then

for all kd−1∑i=0

cibi−2k = 2δ(k)

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 23 / 43

24 / 43

Derivation of the Wavelets from the Scaling Function

Proof.∫ ∞x=−∞

φ(x)ψ(x− k)dx =

∫ ∞x=−∞

d−1∑i=0

ciφ(2x− i)d−1∑j=1

bjφ(2x− 2k − j)dx

=

d−1∑i=0

d−1∑j=0

cibj

∫ ∞x=−∞

φ(2x− i)φ(2x− 2k − j)dx

=1

2

d−1∑i=0

d−1∑j=0

cibj

∫ ∞y=−∞

φ(y)φ(y − 2k − j + i)dy

=1

2

d−1∑i=0

d−1∑j=0

cibjδ(2k + j − i)

=1

2

d−1∑i=0

cibi−2k = 0

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 24 / 43

25 / 43

Derivation of the Wavelets from the Scaling Function

Lemma 3

Let φ(x) =∑d−1

k=0 ckφ(2x− k) and ψ(x) =∑d−1

k=0 bkφ(2x− k). If∫∞−∞ φ(x)φ(x− k)dx = δ(k) and

∫∞−∞ φ(x)ψ(x− k)dx = 0 for all k, then

for all kbk = (−1)kcd−1−k

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 25 / 43

26 / 43

Derivation of the Wavelets from the Scaling Function I

Proof.

By last lemma,∑d−1

j=0 cjbj−2k = 0 for all k, which can be written as

d2−1∑

j=0

c2jb2j−2k +

d2−1∑

j=0

c2j+1b2j+1−2k = 0

Similarly, By Lemmas 11.2 and 11.4, we can get

d2−1∑

j=0

c2jc2j−2k +

d2−1∑

j=0

c2j+1c2j+1−2k = 2δ(k)

andd2−1∑

j=0

b2jb2j−2k +

d2−1∑

j=0

(−1)jb2j+1b2j+1−2k = 2δ(k)

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 26 / 43

27 / 43

Derivation of the Wavelets from the Scaling Function II

Proof.c0b0 + c2b2 + c4b4 + · · ·+ c1b1 + c3b3 + c5b5 + · · · = 0 k = 0

c2b0 + c4b2 + · · · + c3b1 + c5b3 + · · · = 0 k = 1

c4b0 + · · · + c5b1 + · · · = 0 k = 2

c0c0 + c2c2 + c4c4 + · · ·+ c1c1 + c3c3 + c5c5 + · · · = 0 k = 0

c2c0 + c4c2 + · · · + c3c1 + c5c3 + · · · = 0 k = 1

c4c0 + · · · + c5c1 + · · · = 0 k = 2

b0b0 + b2b2 + b4b4 + · · ·+ b1b1 + b3b3 + b5b5 + · · · = 0 k = 0

b2b0 + b4b2 + · · · + b3b1 + b5b3 + · · · = 0 k = 1

b4b0 + · · · + b5b1 + · · · = 0 k = 2Let Ce = (c0, c2, . . . , cd−2) , Co = (c1, c3, . . . , cd−1) , Be = (b0, b2, . . . , bd−2). Equation 12.1,12.2 and 11.3 can be expressed as convolutions of these sequences. We can get the matrixformat as (

Ce Co

Be Bo

)∗(

CRe BR

e

CRo BR

o

)=

(2δ 00 2δ

)Taking the Fourier of z−transform yields(

F (Ce) F (Co)F (Be) F (Bo)

)(F(CR

e

)F(BR

e

)F(CR

o

)F(BR

o

) ) =

(2 00 2

)

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 27 / 43

28 / 43

Derivation of the Wavelets from the Scaling Function I

Proof.

Taking the determinant yields(F (Ce)F (Bo)− F (Be)F (Co)) (F (Ce)F (Bo)− F (Co)F (Be)) = 4.Thus, F (Ce)F (Bo)− F (Co)F (Be) = 2 and the inverse transform yieldsCe ∗Bo − Co ∗Be = 2δ(k). Convolution by CRe yields

CRe ∗ Ce ∗Bo + CRo ∗Bo ∗ Co = 2CRe ∗ δ(k)(CRe ∗ Ce + CRo ∗ Co

)∗Bo = 2CRe ∗ δ(k)

2δ(k) ∗Bo = 2CRe ∗ δ(k)Ce = BR

o

Thus, ci = bd−1−i for even i. Similarly, we can get ci = −bd−1−i for allodd i and hence ci = (−1)ibd−1−i for all i.

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 28 / 43

29 / 43

Outline

3.14pt

1 Introduction

2 Dilation Equation

3 Derivaton of the Wavelets from the Scaling Function

4 Sufficient Conditions for the Wavelets to be Orthogonal

5 Expressing a Function in Terms of Wavelets

6 Designing a Wavelet System

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 29 / 43

30 / 43

Sufficient Conditions for the Wavelets to be Orthogonal

Wavelets system:

Wavelets, psij(2jx− k), at all scales and shifts to be orthogonal to

the scale function phi(x)

All wavelets to be orthogonal. That is∫ ∞−∞

ψj(2jx− k

)ψl

(2lx−m

)dx = δ(j − l)δ(k −m)

φ(x) and ψjk, j ≤ l, and all k, to span Vl, the space spanned byφ(2jx− k) for all k.

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 30 / 43

31 / 43

Sufficient Conditions for the Wavelets to be Orthogonal

Lemma 4

If bk = (−1)kcd−1−k, then∫∞−∞ φ(x)ψ

(2jx− l

)dx = 0 for all j and l.

Proof.

We first show that φ(x) and ψ(x− k) are orthogonal for all values of k.Then we modify the proof to show that φ(x) and ψ(2jx− k) areorthogonal for all j and k.

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 31 / 43

32 / 43

Sufficient Conditions for the Wavelets to be Orthogonal

Proof.∫ ∞−∞

φ(x)ψ(x− k) =∫ ∞i=0

d−1∑i=0

ciφ(2x− i)d−1∑j=0

bjφ(2x− 2k − j)dx

=

d−1∑i=0

d−1∑j=0

ci(−1)jcd−1−j∫ ∞−∞

φ(2x− i)φ(2x− 2k − j)dx

=

d−1∑i=0

d−1∑j=0

(−1)jcicd−1−jδ(i− 2k − j)

=

d−1∑j=0

(−1)jc2k+jcd−1−j

= c2kcd−1 − c2k+1cd−2 + · · ·+ cd−2c2k−1 − cd−1c2k= 0

The last step requires that d be even which we have assumed for all scalefunctions.

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 32 / 43

33 / 43

Sufficient Conditions for the Wavelets to be Orthogonal

Proof.

For the case where the wavelet is ψ(2j − l), first express φ(x) as a linearcombination of φ(2j−1x− n). Now for each these terms∫ ∞−∞

φ(2jx−m

)ψ(2jx− k

)dx =

1

2j−1

∫ ∞−∞

φ(y −m)ψ(2y − k)dy = 0

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 33 / 43

34 / 43

Sufficient Conditions for the Wavelets to be Orthogonal

Lemma 5

If bk = (−1)kcd−1−k, then∫ ∞−∞

1

2jψj(2jx− k

) 1

2kψl

(2lx−m

)dx = δ(j − l)δ(k −m)

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 34 / 43

35 / 43

Sufficient Conditions for the Wavelets to be Orthogonal

Proof.This first level wavelets are orthogonal.

∫ ∞−∞

ψ(x)ψ(x− k)dx =

∫ ∞−∞

d−1∑i=0

biφ(2x− i)d−1∑j=0

bjφ(2x− 2k − j)dx

=

d−1∑i=0

bi

d−1∑j=0

bj

∫ ∞−∞

φ(2x− i)φ(2x− 2k − j)dx

=

d−1∑i=0

d−1∑j=0

bibjδ(i− 2k − j)

=

d−1∑i=0

bibi−2k

=

d−1∑i=0

(−1)icd−1−i(−1)i−2kcd−1−i+2k

=

d−1∑i=0

(−1)2i−2kcd−1−icd−1−i+2k

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 35 / 43

36 / 43

Sufficient Conditions for the Wavelets to be Orthogonal

Proof.Substituting j for d− l − i yields

d−1∑j=0

cjcj+2k = 2δ(k)

Example of orthogonality when wavelets are of different scale.

∫ ∞−∞

ψ(2x)ψ(x− k)dx =

∫ ∞−∞

d−1∑i=0

biφ(4x− i)d−1∑j=0

bjφ(2x− 2k − j)dx

=

d−1∑i=0

d−1∑i=0

bibj

∫ ∞−∞

φ(4x− i)φ(2x− 2k − j)dx

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 36 / 43

37 / 43

Sufficient Conditions for the Wavelets to be Orthogonal

Proof.

Since φ(2x− 2k − j) =∑d−1

l=0 clφ(4x− 4k − 2j − l)

∫ ∞−∞

ψ(2x)ψ(x− k)dx =

d−1∑i=0

d−1∑j=0

d−1∑l=0

bibjcl

∫ ∞−∞

ψ(4x− i)φ(4x− 4k − 2j − l)dx

=

d−1∑i=0

d−1∑j=0

d−1∑l=0

bibjclδ(i− 4k − 2j − l)

=

d−1∑i=0

d−1∑j=0

bibjci−4k−2j

Since∑d−1

j=0 cjbj−2k = 0,∑d−1

i=0 bici−4k−2j = δ(j − 2k) Thus

∫ ∞−∞

ψ(2x)ψ(x− k)dx =

d−1∑j=0

bjδ(j − 2k) = 0

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 37 / 43

38 / 43

Sufficient Conditions for the Wavelets to be Orthogonal

Proof. ∫ ∞−∞

φ(x)ψ(2x− k)dx =

∫ ∞−∞

d−1∑j=0

cjφ(2x− j)ψ(2x− k)dx

=

d−1∑j=0

cj

∫ ∞−∞

φ(2x− j)ψ(2x− k)dx

=1

2

d−1∑j=0

cj

∫ ∞−∞

φ(y − j)ψ(y − k)dy

= 0

If ψ was of scale 2j , φ would be expanded as a linear combination of φ ofscale 2j all of which would be orthogonal to ψ.

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 38 / 43

39 / 43

Outline

3.14pt

1 Introduction

2 Dilation Equation

3 Derivaton of the Wavelets from the Scaling Function

4 Sufficient Conditions for the Wavelets to be Orthogonal

5 Expressing a Function in Terms of Wavelets

6 Designing a Wavelet System

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 39 / 43

40 / 43

Expressing a Function in Terms of Wavelets

Aim: Express a function f(x) in terms of an orthogonal basis of the waveletsystem using given wavelet system with scale function φ and mother wavelet ψ.

Let f(x) =∑∞

k=0 ajkφj(x− k) where the ajk are the coefficients in the expansionof f(x) using level j scale functions. Since the φj(x− k) are orthogonal

ajk =

∫ ∞x=−∞

f(x)φj(x− k)dx

Expanding φj in terms of φj+1 yields

ajk =

∫ ∞x=−∞

f(x)

d−1∑m=0

cmφj+1(2x− 2k −m)dx

=

d−1∑m=0

cm

∫ ∞x=−∞

f(x)φj+1(2x− 2k −m)dx

=

d−1∑m=0

cmaj+1,2k+m

=

d−1∑n=2k

cn−2kaj+1,n

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 40 / 43

41 / 43

Outline

3.14pt

1 Introduction

2 Dilation Equation

3 Derivaton of the Wavelets from the Scaling Function

4 Sufficient Conditions for the Wavelets to be Orthogonal

5 Expressing a Function in Terms of Wavelets

6 Designing a Wavelet System

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 41 / 43

42 / 43

Designing a Wavelet System

If one uses d terms in the dilation equation, one defree of freedomcan be used to satisfy

d−1∑i=0

ci = 2

which insures the existence of a solution with a nonzero mean.Another d

2 degrees of freedom are used to satisfy

d−1∑i=0

cici−2k = δ(k)

which insures the orthogonal properties. The remaining d2 − 1 degrees

of freedom can be used to obtain some desirable properites such assmoothness.

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 42 / 43

43 / 43

The Haar Wavelet

Use scal function to generate the two dimensional family of functionsφjk(x) = φ

(2jx− k

).

For a given value of j, the shifted versions, {φjk|k ≥ 0}, span a spaceVj .Since φ(x) is the solution of a dilation equation, for any fixed j, φjkis a linear combination of the

{φj+1,k′ |k′ ≥ 0

}. So Vj ⊆ Vj+1.

For each j, the set of functions φjk, k = 0, 1, 2 . . ., form a basis for avector space Vj and are orthogonal. But for different values of j arenot orthogonal.Since φjk, φj+1,2k and φj+1,2k+1 are linearly dependent, for eachvalue of j delete φj+1,k for odd values of k to get a linearlyindependent set of basis vectors.To get an orthogonal set of basis vectors, define

ψjk(x) =

1 2k

2j≤ x < 2k+1

2j

−1 2k+12j≤ x < 2k+2

2j

0 otherwise

Ming Li, Quan Xiao (USTC) Section 11 2019.06.30 43 / 43