7/29/2019 Wavelets 3
1/29
Applications of Wavelets inNumerical Mathematics
Kees Verhoeven
1. Brief summary
2. Data compression
3. Denoising
4. Preconditioning
5. Adaptive grids
6. Integral equations
1
7/29/2019 Wavelets 3
2/29
1. Brief Summary
(t): scaling function.For the following 2-scale relation holds
(t) =
k=
pk(2t k), t IR.
(t): mother wavelet.For the following 2-scale relation holds
(t) =
k=
qk(2t k), t IR.
The decomposition for reads
(2tk) =
m=
h2mk(tm)+g2mk(tm), t IR.
2
7/29/2019 Wavelets 3
3/29
2. Data Compression
We consider a function f
f : [0, 1] IR.
We want to approximate this function by a function v de-fined by
v =
k
ckk,
where {k|k = 1, . . . , N } is a basis for the linear functionspace V.
The quality of the approximation can be expressed in termsof a norm
f v.
An alternative is to expand f periodically. We thereforelook at the Fourier series of f
f(x) =
m=
cme2imx
and approximate this by
v(x) =M
m=M
cme2imx.
3
7/29/2019 Wavelets 3
4/29
So
V =
e2mix |m = M , . . . , M
,
with dimension N = 2M + 1. The basis functions form anorthonormal system. Therefore
ck = (k, f) =
10
f(x)e2imxdx.
Error estimates
Given f, g : [0, 1] IR with the Fourier expansions
f =
m=
cme2imx, g =
m=
dme2imx.
Then10
f(x)g(x)dx =
m=
cmdm.
So 10
|f(x)|2dx =
m=
|cm|2.
The error then reads
2M := f v2 =
|m|>M
|cm|2.
In many cases properties of f lead to an error estimate ofthe type
M CM, C, > 0.
4
7/29/2019 Wavelets 3
5/29
Example
Given
f(x) = x(x 1
2)(x 1), x [0, 1].
We can derive
cm =3
4i3m3.
The error M can therefore be estimated via
2M =9
86
m=M+1
m6 9
86
M
y6dy =9
406M5.
M L2-error
9406 M
5
10 0.426 104 0.484 104
20 0.803 105 0.855 105
40 0.147 105 0.151 105
5
7/29/2019 Wavelets 3
6/29
How about localized functions?
It seems sensible to approximate a localized function withbasis functions which also have compact support.
Stepfunctions:
k =
h1
2 , x [(k 1)h,kh),
0, else.
Note that:
h = N1 and k = 1
more efficient for function evaluations
6
7/29/2019 Wavelets 3
7/29
Comparison
We consider the following function.
The error of the approximation using the Fourier series atM = 64, is approximately 0.001. The approximation using
first order splines is 0.01 withN
= 2M
+ 1 = 129.We would like to use localized basis functions only wherethe function to be approximated behaves like a localizedfunction.
7
7/29/2019 Wavelets 3
8/29
Therefore, we would like basis functions which all have the
same shape but different scales. Then, if we have
v =N
k=1
ckk
and |cj| , we also have
f
k
ckk
f
k=j
ckk
+ cjj.
This leads to data compression.
8
7/29/2019 Wavelets 3
9/29
Example
We denote by VJ the space of piecewise constant basis func-tions on [0,1] with width h = 2J and dimension N = 2J.
The space V0 has one basis function: the constant function1 = 1.For V1 the usual basis functions are depicted here:
The coefficients ck behave like
ck = h1
2 a+h/2
ah/2
f(x)dx h1
2f(a).
Can we chose a more appropriate basis?
9
7/29/2019 Wavelets 3
10/29
First approach: construct basis for VJ by expanding the
basis of VJ1.
Figure 1: An alternative basis for V1.
Figure 2: Together with the functions above an alternative basis for V2.
But: the new basis is no longer orthogonal.For the test function f(x) = x, the drop in the coefficientsseems to be like h3/2 (23/2 3).
generation |ck|
0 0.3541 0.1252 0.0443 0.016
10
7/29/2019 Wavelets 3
11/29
A better alternative basis for VJ
Construct an orthogonal basis {i} for VJ. This leads to
Figure 3: A better alternative basis for V1.
Figure 4: Together with the two functions above a better alternative basisfor V2.
Note that these are the Haar wavelets!
(x) = 0,0(x) = 2(x), 0,1(x) = 3(x), 1,1(x) = 4(x).
11
7/29/2019 Wavelets 3
12/29
Now:
ck = h1
2
aah/2 f(x)dx
a+h/2a f(x)dx
= h
1
2
aah/2(f(a) + (x a)f
(a) + . . .)dx
a+h/2
a (f(a) + (x a)f(a) + . . .)dx
1
4f(a)h3/2.
12
7/29/2019 Wavelets 3
13/29
Comparison of this basis and homogeneous one:
Figure 5: Approximation of f (top left) with 32, 10, 9 basis functions,respectively.
13
7/29/2019 Wavelets 3
14/29
Figure 6: Approximation of f with 22, 10 homogeneous basis functions,respectively.
If wavelets used as basis functions have several momentsequal to zero, then reduction will be better.
Because, if
v =lZZ
cl,Il,I +J1k=I
lZZ
dl,kl,k,
then
|dk,J| Chd+3/2,
where d is the number of moments equal to zero.
14
7/29/2019 Wavelets 3
15/29
Example
M L2-error for spline d = 1 L2-error for spline d = 3
256 1.149 103 2.829 104
128 1.149 103 2.829 104
64 1.356 103 2.829 104
32 2.623 103 1.143 103
15
7/29/2019 Wavelets 3
16/29
3. Denoising
Suppose we have a signal with some noise. We can make awavelet decomposition up to a certain depth.If the coefficients of the wavelets remain relatively large(say |dj, k| > ), then we have some localized noise.So cancel these contributions by forcing dj,k = 0 at L levelsdeep.
Then, we can reconstruct the filtered signal.
16
7/29/2019 Wavelets 3
17/29
The role of L can be seen as follows:
Figure 7: Filtering of f with L = 1, 3, 5 levels ( = 0.1).
17
7/29/2019 Wavelets 3
18/29
The role of is shown here:
Figure 8: Filtering of f (top) with L = 1, 2 levels ( = 0.01).
18
7/29/2019 Wavelets 3
19/29
But: if the original signal is not periodic, we encounter
problems at the boundaries.
Figure 9: Filtering of non-periodic f with 3 levels ( = 0.01).
19
7/29/2019 Wavelets 3
20/29
4. Preconditioning
Consider
D(u) = f
on a domain , with the differential operator D elliptic. IfD is linear, this would lead to a linear system
Dc = r,
with
Dj,k = (j, Dk), rj = (j, f).
The matrix D is called the stiffness matrix. If we use aniterative method to solve this system, the speed of con-vergence strongly depends on the condition number of thematrix D, (D) = DD1, with
D = supc=1
cTDc, D11 = infc=1
cTDc.
For symmetric matrices, we can express the condition num-ber in terms of the eigenvalues:
(D) =max
min.
20
7/29/2019 Wavelets 3
21/29
Example
We consider
D(u) = d2u
dx2+ u = f, x (0, 1),
with periodical boundary conditions.We assume that the numerical solution v can be writtenas a linear combination of certain scaling functions whichspan VJ.
Galerkins method and integration by parts gives us
Dc = r,
with
Di,j =
di,J
dx,
dj,J
dx
+ (i,J, j,J),
and, as before
rj = (j, f).
For linear B-splines:di,J
dx,
dj,J
dx
=
2J/2
d
dx(2Jxi)2J/2
d
dx(2Jxj)dx.
The derivatives are piecewise constant, and therefore wederive
Di,j =
2N2 + 23
, i = j,
N2 + 16 , i = (j 1) mod N,0 else.
This is a circulant matrix, that is Di,j = d(i j).
21
7/29/2019 Wavelets 3
22/29
We define the symbol of a circulant matrix as
D(z) =
j
Di,jzij.
For D now follows
max = maxzN=1
|D(z)|, min = minzN=1
|D(z)|.
For the differential equation we have
D(z) =
j
di,Jdx ,
dj,J
dx
+ (i,J, j,J)
zij.
We write
D(z) = D1(z) + D2(z),
with
D1(z) = j
di,J
dx,
dj,J
dx
zij, D2(z) =
j
(i,J, j,J)zij.
The second term can be recognized as D2(z) = R(z). Inthe same manner we can derive D1(z) = 2
2JR(z).Using this and the 2-scale relations, we can derive
D(z) = N2(2 z z1) +1
6(4 + z+ z1).
Calculating the biggest and smallest eigenvalue, we see that
1 = 4N2 + 13
.
If N , 1 .
22
7/29/2019 Wavelets 3
23/29
We now use wavelets
v =k,j
dk,jk,j.
The stiffness matrix D then looks like
Dl,m,j,k =
dm,l
dx,
dk,j
dx
+ (m,l, k,j).
After some algebra using Riesz functions and 2-scale rela-tions, we can show that
2(D) C, for all J.
Comparison
2J 1 216 1024.3 45.432 4096.3 49.7
64 16384.3 52.9128 65536.3 55.4256 262144.3 57.3
Again: this strongly depends on the periodicity of the bound-ary conditions.
23
7/29/2019 Wavelets 3
24/29
5. Adaptive grids
We consider a hyperbolic PDE
u(x, t)
t= F(u,
u
x, . . .),
together with initial condition and periodic boundary con-ditions.
The approximation of the initial condition u(x, 0) is done
by
v(x, 0) =N1i=0
ci,I(0)i,I(x) +J1j=I
iIj
di,j(0)i,j(x).
Here N = 2J is the amount of intervals on the coarsestgrid I, the set Ij is a subset of all possible wavelets on thegrids j = I , . . . , J 1. These sets Ij are found by making awavelet decomposition and leave out all wavelet coefficients
below a certain threshold .
But: this would mean that we have to approximate thefunction on the finest grid first!
24
7/29/2019 Wavelets 3
25/29
We ignore all contributions below a wavelet for which
|dk,l| . (filled circles mean |dk,l| > , open circles mean:|dk,l| )
Adaptivity means that wavelets left out in previous timesteps can occur again.
25
7/29/2019 Wavelets 3
26/29
Example: Burgers equation
Figure 10: Approximation on t = 0, 112
, 212
, 312
, 412
, 512
, for Burgers equation.
The number of basis functions with coefficient above threshold is 32, 56,122, 114, 114 and 114, respectively.
26
7/29/2019 Wavelets 3
27/29
Example: wave equation
Figure 11: Approximation on t = 0, 0.3 and 0.5, respectively, for the wave
equation. The number of basis functions with coefficient above threshold isapproximately 60.
27
7/29/2019 Wavelets 3
28/29
6. Integral Equations
We consider
u(x) =
K(x; t)u(t)dt + f(x).
We take
v(x) =
j
cjj(x).
Galerkins method would leave us with
Ac = r,
with
Aj,k = (j, k)
j(x)K(x; t)k(t)dxdt, rj = (j, f).
Often this A is well conditioned, but full.
Using wavelets reduces the number of nonzero elements.
We represent
v(x) =lZZ
cl,Il,I +J1k=I
lZZ
dl,kl,k.
Look at the second term of A
Kl,m,j,k =
m,l(x)K(x; t)k,j(t)dxdt.
We make the following assumption on K(x; t) d
xdK(x; t)
+
d
tdK(x; t)
Cd 1|x t|d+1 ,for a certain d > 0.
28
7/29/2019 Wavelets 3
29/29
Making use of the Taylor series of K(x; t) and taking a
wavelet with d zero moments, we can derive
|Kl,m,j,k| C1
|x0 t0|d+1.
Using this we can bring down the number of nonzero ele-ments (or better: the number of elements with value abovea certain threshold ).
Example
N = 106 = 109 = 1012
24 74% 92% 92%48 19% 85% 96%96 5.1% 54% 78%
192 1.1% 16% 50%384 0.34% 3.5% 25%
Table 1: The number of elements above threshold , with Daubechieswavelets with K = 2.
N = 106 = 109 = 1012
24 66% 92% 92%48 12% 93% 96%96 3.1% 47% 90%
192 0.85% 12% 56%384 0.32% 2.4% 21%
Table 2: The number of elements above threshold , with Daubechieswavelets with K = 5.
29