Wavelets: theory and applications An introduction Enrique Nava, University of Málaga (Spain)Brasov, July 2006 Grupo de Investigación: Tratamiento Digital.

Wavelets: theory and applications

An introduction

Enrique Nava, University of Málaga (Spain) Brasov, July 2006

Grupo de Investigación:Tratamiento Digital de Imágenes Radiológicas

GTDIR

What are wavelets?

Wavelet theory is very recent (1980’s) There is a lot of books about wavelets Most of books and tutorials use strong

mathematical background I will try to present an ‘engineering’ version

Overview

Spectral analysis Continuous Wavelet

Transform Discrete Wavelet

Transform Applications

A wavelet tour of signal processing, S. Mallat, Academic Press 1998

Spectral analysis:frequency

Frequency (f) is the inverse of a period (T). A signal is periodic if T>0 and

( ) ( )x t x t nT

We need to know only information for 1 period Any signal (finite length) can be periodized. A signal is regular if the signal values and

derivatives are equal at the left and right side of the interval (period)

Signals: examples

0 10 20 30 40 50 60 70 80 90 100-1

-0.5

0

0.5

1

t

x(t)

x(t)=cos(2 0.05 t)

0 200 400 600 800 1000 1200-10

-5

0

5

10Doppler signal

Signals: examples

0 1000 2000 3000 4000 5000 6000 7000 8000 90000

0.05

0.1

0.15

0.2Koch fractal curve

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5-3

-2

-1

0

1

2

3

4x(t)=cos(2 10t)+cos(2 25t)+cos(2 50t)+cos(2 100t)

Why frequency is needed?

To be able to understand signals and extract information from real world

Electrical or telecommunication engineers tends ‘to think in the frequency domain’

Fourier series

1

0 sincosk

kk kxbkxaaxf

dxxfa

2

00 21

dxkxxfak cos1 2

0

dxkxxfbk sin1 2

0

:function periodical 2any For xf

1822

Fourier series difficulties

Any periodic signal can be view as a sum of harmonically-related sinusoids

Representation of signals with different periods is not efficient (speech, images)

Fourier series drawbacks

There are points where Fourier series does not converge

Signals with different or not synchronized periods are not efficiently represented

Fourier Transform

The signal has a frequency point of view (spectrum)

Global representation Lots of math properties Linear operators

2( ) ( ) j f tX f x t e dt

2( ) ( ) j f tx t X f e df

Discrete Fourier Transform

Practical implementation Global representation Lots of math properties Linear operators Easy discrete

implementation (1965) (FFT)

knN

N

n

WnxkX

1

0

11

knN

N

k

WkXN

nx

1

0

11

1

Nj

N ew2

Fourier transform

0 50 100 150 200 250 300 350 400 450 5000

10

20

30

40

50

60

70

frequency (Hz)

Spectrum

0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05-10

-5

0

5

10Periodic Signal with Zero-Mean Random Noise

t

x(t)=sin(2 50t)+sin(2 120t)+n(t)

Random signals

Stationary signals:Statistics don’t change with timeFrequency contents don’t change with time Information doesn’t change with time

Non-stationary signals:Statistics change with timeFrequencies change with time Information quantity increases

Non-stationary signals

0 0.2 0.4 0.6 0.8 1-3

-2

-1

0

1

2

3

0 5 10 15 20 250

100

200

300

400

500

600

Time

Ma

gn

itu

de

Ma

gn

itu

de

Frequency (Hz)

2 Hz + 10 Hz + 20Hz

Stationary

0 0.5 1-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 250

50

100

150

200

250

Time

Ma

gn

itu

de

Ma

gn

itu

de

Frequency (Hz)

Non-Stationary

0.0-0.4: 2 Hz + 0.4-0.7: 10 Hz + 0.7-1.0: 20Hz

Chirp signal

0 0.5 1-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 250

50

100

150

Time

Ma

gn

itu

de

Ma

gn

itu

de

Frequency (Hz)0 0.5 1

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 250

50

100

150

Time

Ma

gn

itu

de

Ma

gn

itu

de

Frequency (Hz)

Different in Time Domain Frequency: 2 Hz to 20 Hz Frequency: 20 Hz to 2 Hz

Same in Frequency Domain

Fourier transform drawbacks

Global behaviour: we don’t know what frequencies happens at a particular time

Time and frequency are not seen together

We need time and frequency at the same time: time-frequency representation

Biological or medical signals (ECG, EEG, EMG) are always non-stationary

Short-time Fourier Transform (STFT) Dennis Gabor (1946): “windowing the signal”

Signals are assumed to be stationally local A 2D transform

Short-time Fourier Transform (STFT) dtetttxft ftj

t

2*X ,STFT

function window the:t

A function of time and frequency

Short-time Fourier Transform (STFT)

Short-time Fourier Transform (STFT)

Short-time Fourier Transform (STFT)

Narrow Window Wide Window

STFT drawbacks

Fixed window with time/frequency Resolution:

Narrow window gives good time resolution but poor frequency resolution

Wide windows gives good frequency resolution but poor time resolution

Heisenberg Uncertainty Principle

In signal processing:You cannot know at the same

time the time and frequency of a signal

Signal processing approach is to search for what spectral components exist at a given time interval

Heisenberg Uncertainty Principle

Heisenberg Box

Wavelet transform

An improved version of the STFT, but similar

Decompose a signal in a set of signals Capable of multiresolution analysis:

Different resolution at different frequencies

Continuous Wavelet Transform

Definition:

dtst

txs

ss xx

*1

, ,CWT

Translation

(The location of the window)

Scale

Mother Wavelet

Continuous Wavelet Transform

Wavelet = small wave (“ondelette”) Windowed (finite length) signal

Mother wavelet Prototype to build other wavelets with

dilatation/compression and shifting operators Scale

S>1: dilated signal S<1: compressed signal

Translation Shifting of the signal

CWT practical computation

1. Select s=1 and =0.

2. Compute the integral and normalize by 1/

3. Shift the wavelet by =t and repeat until wavelet reaches the end of signal

4. Increase s and repeat steps 1 to 3

dtst

txs

ss xx

*1

, ,CWT

s

Energy normalization

Time-frequency resolution

Time

Frequency

Better time resolution;Poor frequency resolution

Better frequency resolution;Poor time resolution

• Each box represents a equal portion • Resolution in STFT is selected once for entire analysis

From http://www.cerm.unifi.it/EUcourse2001/Gunther_lecturenotes.pdf, p.10

Comparison of transformations

Mathematical view

CWT is the inner product of the signal and the basis function

dttTX

dtst

txs

ss

s

xx

,

*

1 , ,CWT

st

sts

1,

ts ,

Wavelet basis functions

21

1

241-

0

2

20

21

1- :devivativeDOG

1!2!2

DOG :order Paul

:)frequency(Morlet

edd

mm

immi

m

ee

m

mm

mmm

j

2nd derivative of a Gaussianis the Marr or Mexican hat wavelet

Wavelet basis functionsTime domain

Frequency domain

Wavelet basis properties

Property morl mexh meyr haar dbN symN coifN biorNr.Nd rbioNr.Nd gaus dmey cgau cmor fbsp shan

Crude

Infinitely regular

Arbitrary regularity

Compactly supported orthogonal

Compactly supported biothogonal

Symmetry

Asymmetry

Near symmetry

Arbitrary number of vanishing moments

Vanishing moments for

Existence of

Orthogonal analysis

Biorthogonal analysis

Exact reconstruction

FIR filters

Continuous transform

Discrete transform

Fast algorithm

Explicit expression For splines For splines

Complex valued

Complex continuous transform

FIR-based approximation

Discrete Wavelet Transform

Continuous Wavelet Transform

Discrete Wavelet Transform

s

t

sts

1

)(, dtttxs sx )()(),( ,

][][],[1

0

nmmxanN

mj

jx

jjj a

n

an 1

][

Discrete CWT

Sampling of time-scale (frequency) 2D space Scale s is discretized in a logarithmic way

Scheme most used is dyadic: s=1,2,4,8,16,32 Time is also discretized in a logarithmic way

Sampling rate N is decreased so sN=k Implemented like a filter bank

Discrete Wavelet Transform

Approximation Details

Discrete Wavelet Transform

Discrete Wavelet TransformMulti-level wavelet decomposition tree Reassembling original signal

Discrete Wavelet Transform

Easy and fast to implement Gives enough information for analysis and

synthesis Decompose the signal into coarse

approximation and details It’s not a true discrete transform

SS

A1

A2 D2

A3 D3

D1

Examples

Wavelet: db4

Level: 6

Signal:0.0-0.4: 20 Hz0.4-0.7: 10 Hz0.7-1.0: 2 Hz

fH

fL

Examples

Wavelet: db4

Level: 6

Signal:0.0-0.4: 2 Hz0.4-0.7: 10 Hz0.7-1.0: 20Hz

fH

fL

Signal synthesis

A signal can be decomposed into different scale components (analysis)

The components (wavelet coefficients) can be combined to obtain the original signal (synthesis)

If wavelet analysis is performed with filtering and downsampling, synthesis consists of filtering and upsampling

Synthesis technique

Upsampling (insert zeros between samples)

Sub-band algorithm Each step divides by 2 time resolution and

doubles frequency resolution (by filtering)

Wavelet packets

Generalization of wavelet decomposition Very useful for signal analysis

Wavelet analysis: n+1 (at level n) different ways to reconstuct S

Wavelet packets

Wavelet packets: a lot of new possibilities to reconstruct S:

i.e. S=A1+AD2+ADD3+DDD3

We have a complete tree

Wavelet packets

A new problem arise: how to select the best decomposition of a signal x(t)?

Posible solution:Compute information at each node of the tree

(entropy-based criterium)

Wavelet family types

Five diferent types: Orthogonal wavelets with FIR filters

Haar, Daubechies, Symlets, Coiflets Biorthogonal wavelets with FIR filters

Biorsplines Orthogonal wavelets without FIR filters and with

scaling function Meyer

Wavelets without FIR filters and scaling function Morlet, Mexican Hat

Complex wavelets without FIR filters and scaling function

Shannon

Wavelet families: Daubechies

Compact support, orthonormal (DWT)

Other families

Matlab wavemenu command

Wavelet application

Physics (acoustics, astronomy, geophysics) Telecommunication Engineering (signal

processing, subband coding, speech recognition, image processing, image analysis)

Mecanical engineering (turbulence) Medical (digital radiology, computer aided

diagnosis, human vision perception) Applied and Pure Mathematics (fractals)

De-noising signals

Frequency is higher at the beginning

Details reduce with scale

De-noising images

Detecting discontinuities

Detecting discontinuities

Detecting self-similarity

Compressing images

2-D Wavelet Transform

Wavelet Packets

2-D Wavelets

Applications of wavelets

Pattern recognition Biotech: to distinguish the normal from the pathological

membranes Biometrics: facial/corneal/fingerprint recognition

Feature extraction Metallurgy: characterization of rough surfaces

Trend detection: Finance: exploring variation of stock prices

Perfect reconstruction Communications: wireless channel signals

Video compression – JPEG 2000

Practical use of wavelet

Wavelet software Matlab Wavelet Toolbox

Free software UviWave

http://www.tsc.uvigo.es/~wavelets/uvi_wave.html Wavelab http://playfair.stanford.edu/~wavelab/ Rice Tools http://jazz.rice.edu/RWT/

Useful Links to continue

Matlab wavelet tool using guide http://www.wavelet.org http://www.multires.caltech.edu/teaching/ http://www-dsp.rice.edu/software/RWT/ www.multires.caltech.edu/teaching/courses/

waveletcourse/sig95.course.pdf http://www.amara.com/current/wavelet.html