L07 Interpolation Aliasing

2/2/17

1

Interpolation, Aliasing & Scale

CS 510 Lecture #7

February 1st, 2017

Announcements

• We are running behind– Ask at the end of class if you should read

SIFT for Friday or Monday• Expect PA2 to be handed out on Monday

2/2/17 2

Where are we?• We have

1. Discussed human vision to motivate• Attention• Recognition (classification)• Expertise• Reasoning

2. Reviewed geometric transformations3. Introduced Fourier Analysis

• Goal: work toward computational attention

2/2/17 3

More immediately…

How do geometric transformations alter the information in an image?

Let’s start with rotation and problems of interpolation.

2/2/17 4

Image Transformation• I(x,y) = I’(G�[x,y]T)

• Simple for continuous, infinite images

• Problematic for discrete, finite images

Source & Destination Images• We apply a transformation to a source image

to produce a destination image• The role of source & destination are not

symmetric

We need to know where destination pixels came from

We do not need to know where every source pixel went

2/2/17

2

Basic Transformation Algorithm

For every (x,y) in dest

{

real (u,v) = G-1 (x,y)

pixel p = Interpolate(Src, u, v)

dest(x,y) = p

}

Applying Transformations

• I assume you can invert a 3x3 matrix• So the trick is interpolation. 3 forms:

– Nearest Neighbor (fast, bad)– Bilinear (less fast, good)– Bicubic (slowest, best)

• OpenCV supports all three forms

Nearest Neighbor Interpolation

• (u’, v’) = G-1 (x, y, 1)• u = round(u’)• v = round(v’)• Interpolate(Src, x, y) = Src[u,v]

In terms of Fourier Analysis, this is awful in the frequency domain

Bilinear Interpolation(0,0) (1,0)

(0,1) (1,1)

X(u,v)

(u,0)

(u,1)

Linearly Interpolate along row

Linearly Interpolate along row

(u,v)

Bilinear Interpolation (II)• Bilinear interpolation is actually a product of two

linear interpolation• Equivalent to a least-squares approximation if you

put a plane through the four neighboring points• Typical expression:

• Linear algebraic expression

Bicubic Interpolation

• Product of two cubic interpolations– 1 in x, 1 in y

• Based on a 4x4 grid of neighboring pixels• In each dimension, create a cubic curve

that exactly interpolates all four points– Similar to Bezier curves in graphics– Except curve passes through all 4 points

2/2/17

3

Bicubic Interpolation (II)(-1,-1) (2,-1)

(-1,2) (2,2)

X(u,v)

Bicubic Interpolation (III)• The equation of a cubic function is:

• This can be rewritten as:

• We know the values of f at x = -1,0,1,2

Bicubic Interpolation (IV)• Therefore:

• And:

Bicubic Interpolation (V)

• To interpolate a value:1. Interpolate along the four rows2. Interpolate the results vertically

• Each interpolation is a matrix/vector multiply

– 16 mults, 12 adds per interpolation– 80 mults, 60 adds overall

Interpolation Implementation• You don’t need to implement geometric

transformations of interpolations• OpenCV supports geometric transformations

– warpAffine applies an affine transformation– warpPerspective applies a perspective transformation– Both give you the option of interpolation technique

• Nearest Neighbor• Bilinear• Bicubic

• The point of this lecture is to know what is happening when you use them

Back to image information…

• Interpolation introduces high-frequency noise– NN more than BiLinear more than BiCubic

• What about changes in scale?– In particular, reducing the image size?

2/2/17 18

2/2/17

4

The Nyquist Rate• What if the frequency is above N/2?

– You have fewer than one sample per half-cycle

– High frequencies look like lower frequencies

2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 19

Graphicfrom“Computer Graphics:Principles and Practice” by Foley, van Dam, Feiner & Hughes.

Low-Pass Filtering Approach #1


• Drop high frequency Fourier coefficients.To low-pass filter an image:1) convert to frequency domain2) discard all values for u > thresh3) Convert back to spatial domain

Brainflux FourierApplethttp://www.brainflux.org/java/classes/FFT2DApplet.html

Ø Butthereisabetterway.

Convolution


“Slide” a mask over an image. At each window position, multiply the mask values by the image value under them.Sum the resultsfor every pixel.

Think of this as a sliding dot product

Wearriveatthefundamentalideaofconvolution.

Convolution (II)


• Formally, convolution is often expressed as follows:

Ø Ofcourse,wearedealingwithfinite,discretefunctions:

Convolution Examples• Let F1 = [1,2,3,4,5]• Let F2 = [1,2,1,2,1]• Let G1 = [-1,2,-1]• Let G2 = [1/3,1/3,1/3]

• Then F1*G1=[0,0,0,0,6] • Then F2*G1=[0,2,-2,2,0]• Then F1*G2=[1,2,3,4,3]• Then F2*G2=[1,4/3,5/3,4/3,1]


Convolution (III)

• Why introduce convolution now?

• Because multiplying two Fourier transforms in the frequency domain is the same as convolving their inverse Fourier transforms in the spatial domain! (trust me)


2/2/17

5

Convolution Theorem

2/2/17 25

“Inmathematics,theconvolutiontheoremstatesthatundersuitableconditionstheFouriertransformofaconvolutionisthepointwise productofFouriertransforms.”

Low-Pass Filter• Low-Pass filter - multiply by a pulse in frequency

space, or• Convolve the image with the inverse Fourier

transform of a pulse...


Sinc filter

Truncated sinc


The Gibbs Phenomenon(ringing)• The truncated sinc is no longer a pulse in

frequency space– passes small amounts of some high

frequencies– passes acceptable frequencies in uneven

amounts– may create negative values in unusual

circumstances


Alternative Filters


pulse/sinc

triangle/sinc2

gaussian/gaussian


Image Reductions• Anytime the target image has a lower

resolution than the source image, prevent frequency aliasing by low-pass filtering.– In practice, convolve with a Gaussian to

avoid Gibbs ringing– Determine Nyquist rate for target image– Select s– Convolve source image with g(s)– Apply geometric transformation to result


Image Reductions (II)• Example: reduce 1Kx1K to 800x800 pixels

– Select one (source) pixel as unit length– The Nyquist rate for source is 0.5 cycles/s_pixel– Nyquist rate for target is 0.4 cycles/s_pixel

• Problem: Gaussian is not a strict cut-off– Select “pass” value (2s sounds good)– Select mask width to cover “most” of the area

under the Gaussian curve • recommend 5s (source: Trucco & Verri)• Covers 98.75% of the area under the Gaussian


2/2/17

6

Image Reduction (III)– So 2s is 0.4 cycles/pixel

• The Fourier transform of g(x, s) is g(w, 1/ s)• The inverse of 0.4 cycles/pixel is 2.5 pixels/cycle

– 2s = 2.5 pixels/cycle– s = 1.25 pixels/cycle

• (T&V): To include 5s of the curve, s = w/5, – w is the width of the mask– W = 6.25

– Create a 7x7 Gaussian mask with sigma 1.25• w should be odd, so don’t use 6x6

– Why make w odd? To avoid a geometric transformation…

– Smooth the image using this mask, then subsample.


Example of Smoothing in the Spatial Domain

• Convolve a source image, 128x128 • With a Gaussian kernel – sigma = 4.0

2/2/17 32

Another Example:Increase the Gaussian Sigma

• Now with sigma = 8.0• This is a lot of smoothing

2/2/17 33

On a real image – Sigma = 4.0

2/2/17 34

OpenCV Code for this example

2/2/17 35

Photoshop Gaussian Blur


L07 Interpolation Aliasing

Documents