Wavelets - University of California, Los Angelesicslwebs.ee.ucla.edu/dejan/ee219awiki/images/6/6d/Lec-15-Wavelets...Wavelets EE219A – Spring 2008 Lecture 15 Slide 4 Development of

Dejan [email protected]

Wavelets

EE219A – Spring 2008Special Topics in Circuits and Signal Processing

Lecture 15

Slide 2EE219A – Spring 2008 Lecture 15

Training

Full-day event starting at 9:30am– Break 2-4pm (class)

Training Agenda– Basic PRO– Advanced PRO techniques– Basic Identify– Synplify DSP

Hands-on experience– Labs for each of the four sections– System example: IEE802.11a system with DSP blocks– All-hands-on DSP with a little competition for prizes

Wavelets


Development of Practical IdeasFrom theory to practice– Step 1: mathematicians develop theories (that only a few

people are able to understand at first)– Step 2: physicists and engineers (those who also get to

understand it) adapt the theory to make it more accessible– Step 3: the theory is applied in practice by researchers in

many different fieldsExample– Step 1: in 1807, Joseph Fourier discovered that all

periodic functions can be expressed as a weighted sum of trigonometric functions. Strong opposition, took him 15 years to publish his ideas!

– Step 2: over the next 150 years the ideas were expanded to non-periodic functions and discrete-time sequences

– Step 3: in 1965, the FFT was derived and finds use in EE and many other disciplines that require function analysis


Another Example: Wavelets

Some good reasons for wavelets– Many interesting applications where no other transform

has been applied (because it didn’t work)– Now that we have finally adopted Fourier’s theory, we

need something else to grind about…

So, what did we learn from Fourier? (so we understand the limitations, or save some time grinding…)– A complex function can be approximated with a weighted

sum of basis functions– Fourier used sinusoids with varying frequencies as the

basis functions– This representation provides the frequency content of the

original function


What’s Wrong with the Fourier Transform?Uses sinusoids as basis functions

These are not compactly supported in time domain– Stretch out in time to infinity, so they cannot be used to

approximate non-stationary signalsFourier transform assumes all spectral components are present at all times!– Time domain representation does not provide information

about spectral content of the signalLimitation: non-stationary signals whose spectral content changes in time cannot be supported by FT


Support for Non-stationary SignalsA work-around: modify Fourier transform to allow analysis of non-stationary signals by slicing in time (short-time Fourier transform)– Segment in time by applying windowing functions and

analyzing each segment separately– Many approaches (between late 1940s and early 1970s)

differing in the choice of windowing functionsSTFT, thus, provides full time-frequency representation?Not really, there is still a problem: all approaches used the same window for the analysis of the entire signal– Not applicable to signals that have

● High frequency components with short time spans● Low frequency components with long time spans

– STFT can do high frequency analysis by shortening time window or low frequency analysis by widening the window, but it cannot do both


Back to Applications

Wavelets (to be defined) are a nice example of how practical problems drive theory development– High frequency short time / low frequency long time type

of signals actually occur in geophysics

In 1970s, J. Morlet, a geophysicist solved the problem– He proposed using a different window functions for

analyzing different frequency bands– These windows were generated in a special way:

by dilation or compression of a prototype Gaussian!● Couldn’t argue with Gauss in the 1970s (well established by then),

so you’d expect Morlet to have much easier time than Fourier● Also, Morlet was a practitioner, so skip step 1 and some of step 2● None of the above actually happened, because mathematicians

didn’t like the idea!


Wavelets

The window functions (dilation or compression of a Gaussian) have compact support both in time and frequency– Fourier transform of a Gaussian is also Gaussian

Due to the “small and oscillatory” nature of the window functions, Morlet named his basis functions as wavelets of constant shape

In early 1980s, inverse function was derived– With help from A. Grossman, a theoretical physicist of

quantum mechanics– A re-discovery of Alberto Calderon’s 1964 work on

harmonic analysis


Now, Back to Mathematicians

In 1984, Yves Meyer, a French mathematician, noticed the similarity between Morlet’s and Calderon’s work– Also noticed lot so redundancy in Morlet’s choice of basis

functions, then known as wavelets– He started developing wavelets with better localization

propertiesIn 1985, Meyer constructed orthogonal wavelet basis functions– It turned out that another harmonic analyst, Stromberg,

had discovered the same wavelets about 5 years before– Neither Meyer nor Stromberg, however, have actually

discovered orthogonal wavelet basis functions– The discovery honor goes to German mathematician

Alfred Haar who did it back in 1909!


Wavelets (Cont.)

First and simplest orthogonal wavelets (wavelets = wavelet basis functions)– Little practical use due to limited frequency localization

Another important wavelet person: Ingrid Daubechies– Developed wavelet frames for discretization of time snd

scale parameters– More freedom in the choice of wavelets at the expense of

some redundancy– Ingdid Daubechies and Stehpane Mallat are credited for

developing the transition from continuous to discrete signal analysis

In 1986, Mallat developed the idea of multi-resolution analysis for discrete wavelet transform (DWT)– Ph.D. dissertation (U-Penn, 1988)


Mallat and More Recent Work

Decomposing a discrete signal into its dyadic frequency bands by a series of low-pass and high-pass frequency filters to compute its DWT from the approximations at these various scales– Not surprisingly, the idea was familiar to EEs, for about 20

years, as quadrature mirror filters (QMF) and sub-band filtering developed in the mid-1970s

– Mallat’s work was extension of time localization to the idea of QMF and sub-band coding

– Together with Daubechies’ orthonormal wavelets (1988), Mallat’s work laid foundation of modern wavelets

In the past 20 years– Search for other wavelets with different properties

● Biorthogonal wavelets (Cohen, Feauveau, Daubechies, 1992)● Wavelet packets (Coifman, Meyer, Wickerhauser)


Example

)100*2cos()50*2cos()25*2cos()10*2cos()(1 tttttx Π+Π+Π+Π=

1000ms t800ms )10*cos(2 800ms t600ms )25*cos(2 600ms t300ms )50*cos(2

300mst0 )100*2cos()(2

≤<Π≤<Π≤<Π

≤<Π=

tttttx

Fourier transform, X1(f) Fourier transform, X2(f)

Not all spectralcomponents exist at all times


Time-Frequency Representation

Short Time Fourier Transform (STFT)

– w(t) is the windowing functionMultiply signal by a window and then take a F.T. of the result (segment into stationary short-enough pieces)– S(τ,f) is STFT of x(t) at frequency f and translation τ

Translate the window to get spectral content of signal at different frequencies– w(t-τ) does time-segmentation of x(t)

Haven’t we solved the problem?

translation parameter


Heisenberg's Uncertainty PrincipleThe problem with previous approach is uniform resolution for all frequencies (same window for x(t))There exists a tradeoff between the resolution in time and frequency domain– If the signal has high frequency components for a short

time span, a narrow window is needed for time resolution, but this results in wider frequency bands (poor freq. res.)

– If the signal has low frequency components of longer time span, a wider window need to be used for good frequency resolution (but, with poor time resolution)

This is yet another demonstration of Heisenberg's uncertainty principle– F.T. is an extreme case where all time domain information

is lost to get precise frequency information– Thus, STFT offers fixed resolution which needs to be

chosen keeping the above tradeoff in mind


Continuous Wavelet TransformVarying time and frequency resolutions by using windows of different lengths– First decompose in freq domain– Analyze the signal in time

– a > 0, b: scale and translation parameters

Morlet’s Choice: Gaussian wavelets

Wave: Oscillatory; let: compact support– “small wave”

Redundant representation of signal

Search for orthonormal wavelet basis– Meyer Wavelet / Haar Wavelet Morlet Wavelet

Mother wavelet


Fourier basis functions Wavelet basis functions

Fourier vs. Wavelet Transform


Discrete Wavelet SeriesWe need discrete domain transforms

Discretize the translation and scale parameters (a, b)– Daubechies (a=2j, b=2jk)

Input signal is still continuous

Number of wavelets in the family is still infinite


Towards Practical Implementations…How to limit the number of scales for analysis?

Each wavelet is a like a constant – Q filter

Scaling function


Mallat’s MRARelation between scaling and wavelet functions at different frequency resolutions

Describes wavelet transform in terms of digital filtering and sampling operations– Iterative use of low-pass and high-pass filters,

subsequent down-sampling by 2x

Leads to an easier implementation where wavelets are abstracted away!


Mallat’s DWT

The popular form of DWT

HP LP

HP

LP


Applications

Spike Sorting

FBI finger prints

Image compression

Generality of the WT lets us take a pick for the wavelet used


Computer and Human VisionDavid Marr tried to find an answer to “Why the first attempts to construct a robot capable of understanding its surroundings were unsuccessful?”

He speculated that intensity changes occur at different scales in an image and sudden intensity changes produce a peak or two in the first derivative of the image

Vision filter has two characteristics– It should be a differential operator– It should be capable of being turned to act at any desired

scale

Marr Wavelet


1924-1995, the US FBI has collected about 30 million sets of fingerprints.In 1993, the FBI’s Criminal Justice Information Services Division developed standards for fingerprint digitization and compression in cooperation with the National Institute of Standards and Technology, Los Alamos Nat. Lab, commercial vendors and criminal justice communitiesThe data compression standard WSQ (Wavelet/Scalar Quantization) is implemented

FBI Fingerprint Compression


David Donoho’s technique: Wavelet Shrinkage and Thresholding

Denoising Noisy Data


Wavelets - University of California, Los Angelesicslwebs.ee.ucla.edu/dejan/ee219awiki/images/6/6d/Lec-15-Wavelets...Wavelets EE219A – Spring 2008 Lecture 15 Slide 4 Development of

Documents