1 Computer Science 631 Lecture 4: Wavelets Ramin Zabih Computer Science Department CORNELL UNIVERSITY
1
Computer Science 631Lecture 4: Wavelets
Ramin ZabihComputer Science DepartmentCORNELL UNIVERSITY
2
Outline
Announcement: HW#1 exists! One more try at Powerpoint formulas Wavelet transformation via the Haar basis 2D wavelets
3
Ray-based coordinates
Compute the position of the pixel X w.r.t. an oriented ray PQ• Coordinates are A (along PQ) and B
(perpendicular to PQ)
P
Q X
A
BA
Q P
Q PX P=
--
× -b gb g
BQ P
Q PX P=
--
× -^b gb g
4
Changing units
The problem is that A and B are in units of pixels• Need them in percentages of the length of PQ
uA
Q P
Q P
Q PX P=
-=
-
-× -b gb g2
vB
Q P
Q P
Q PX P=
-=
--
× -^b gb g2 P
Q X
5
General formula
Note that X’ (as well as X) is a point, not a pixel
X’
P
Q X
P’
Q’
a
b
a
b
X P a Q P b Q P' ' ( ' ' ) ' '= + × - + × - ^b g
6
Fast bilinear interpolation
The value at the interior point (x,y) is
To compute this fast:
p10
p00 p01
p11
(x,y)
p x y p x y p x y p xy pxy = - - × + - × + - × + ×1 1 1 100 10 01 11b gb g ( ) ( )
p p x p p
p p x p p
p p y p p
x
x
xy x x x
0 00 10 00
1 01 11 01
0 1 0
= + -= + -= + -
( )
( )
( )
7
Frequency decomposition of an image
An image can be described in several ways• So far, in terms of pixels (= spatial domain)• The frequency decomposition is very useful• Low-frequency components change slowly• High-frequency components change rapidly
If the image has no (little) high-frequency components, then aliasing is not a problem
8
Why use the frequency domain?
By describing an image in terms of the frequency domain, many things become clear• The image formation process itself removes really
high-frequency components– What happens when we take a picture of a checkerboard
where a pixel contains 100 squares?
– Many image operations are naturally viewed in terms of their effects on various frequency components
• Local averaging removes high-frequency components
– Image compression is best viewed in this way
9
Choice of basis
The canonical way to describe the frequency of an image is in terms of its Fourier transform• This involves a number of issues that we don’t have time
to cover in depth Instead, we will use a Wavelet representation called
the Haar basis• Same basic idea, but easier and more intuitive• For example, our basis vectors will be mostly 0
– By contrast, the Fourier basis is sine waves
10
Image representations
Consider a 1-D image (signal) with four elements: I = [9 7 3 5]• Spatial representation:
I = 9*[1 0 0 0]+7*[0 1 0 0]+3*[0 0 1 0]+5*[0 0 0 1]• The basis elements are 1-D (in this case) vectors,
each with a single 1– What are they called for an image?
11
Wavelets and DC components
Our basis vectors will have a scale, which intuitively means how many non-zero elements• To begin with, we will subtract the average value of
I, which is 6 in our exampleI - 6 = [3 1 -3 -1] I = 6*[1 1 1 1] + [3 1 -3 -1]
• [1 1 1 1] is our first (dull) new basis– No zeros, so coarsest possible scale
• The average value of an image is referred to as its DC component, others are AC components
12
Haar basis AC components
For wavelets, the basis functions are shifted and scaled versions of a single function• Like Fourier basis, with sine/cosine• The “generating function” for the Haar basis is
[1 -1] – everywhere else it’s 0
• This is a pretty simple way to describe a signal!
13
Computing detail coefficients
The average of two numbers is halfway between them• Therefore, a pair [a b] can be represented as:
avg*[1 1] + diff*[1 -1], where
avg = (a + b)/2 and diff = a - avg = avg - b• For the Haar basis, we will use this trick
– For a pair of numbers, compute the average and difference
14
Multi-resolution averaging
Given a signal of size 2n, we can average the pixels together at various scales• Compute a pairwise average, to get size 2n-1
• Compute a pairwise average on that, too• Eventually you get a single number
Example (n = 2)
[9 7 3 5][8 4][6]
421
scale average
15
Generating detail coefficients
Each time we compute the average, we can write down the difference too[9 7] -> [8 1], [3 5] -> [4 -1]
We can reconstruct [8 4] from [6] and [2]
[9 7 3 5][8 4][6]
421
scale average
[1 -1][2]
detail
16
Example
Wavelet basis
-2
-1
0
1
2
Goal: represent the input (left) in terms of a DC term, plusscaled and shifted versions of the basis function (right)
AveragesScale: 4
0
2
4
6
8
10
17
One step of average and difference
DifferencesScale:4 - Scale:2
-2
-1
0
1
2
AveragesScale:2
0
2
4
6
8
10
Add these to get our original input [9 7 3 5]
18
Last step of average and difference
AveragesScale:1
0
1
2
3
4
5
6
7
DifferencesScale:2 - Scale:1
-3
-2
-1
0
1
2
Add these to get the previous average [8 8 4 4]
19
Wavelet representation
There are several ways to do the representation• For the Haar basis, we use a single DC component,
and everything else is details
6*[1 1 1 1]+2*[1 1 -1 -1]+1*[1 -1 0 0] +-1*[0 0 1 -1]• Note that you can build the averages at one level from
the averages and differences on the next level down– We can reconstruct [8 4] from [6] and [2]
[8 4] = 6*[1 1] + 2*[1 -1]
20
Given an input signal, we can create the transformed signal by repeated averaging and differencing
[9 7 3 5]
[8 4 1 -1]
[6 2 1 -1]
Computing the transform
Algorithm:Turn each adjacent pair into an (avg,diff) pairStore all the avg’s in the left half of outputStore all the diff’s in the right halfRecurse on the left half of the outputStop when there is a single number left
Inverting is a simple exercise...
21
Scaling the Haar basis vectors
The basis vectors for the Haar basis are shifted and scaled versions of [1 -1]
We’d like the basis to be orthonormal• For any two elements u,v we want the inner
product utv to be 1 if u = v, otherwise 0• What is [1 -1]t[1 -1] ?• Solution: multiple basis vectors by the
appropriate constant
22
What about 2-D images
First issue is that images are not necessarily of size 2n by 2n
• What is the wavelet transform of [1 3 9]? This issue arises in many places
• For example, if you want to smooth an image. • Obvious solution is to replace each pixel by a
local average– But what do you do at the borders?
23
Three obvious solutions
Pad the image with something• Typically, pad with 0• For a local operation, discard the values that are
within radius of the border– Just need to be sure the process doesn’t crash
“Make up” some other stuff around the image• Question is what to make up• Obvious answer is to use the image itself!
24
Standard tiling
25
Reverse tiling
26
Reverse tiling is in general better
Why? The difference between the Fourier
transform and the transform that JPEG compression uses is (to simplify) standard versus reverse tiling• Discrete Fourier Transform = standard• Discrete Cosine Transform = reverse
27
2-D wavelet transform
The standard decomposition first computes the 1D wavelet transform of each row• The result is a row of the same length• Order by decreasing importance (resolution)
– DC coefficient first
• We treat the result as an image, then compute a 1D wavelet transform of each column
• Result is an image with a single DC coefficient– Rest are details (usually, details of details…)
28
Standard decomposition example
29
What are the 2-D basis functions?
Under the standard basis, examples are:
Gray = 0
30
Non-standard 2D Haar basis