Image Processing

Image Processing

Xuejin [email protected]

Ref: http://fourier.eng.hmc.edu/e161/lectures/gradient/node10.html

Image processing operations

Original image Increased contrast Change in hue

Posterized (quantized colors)

Blurred Rotated

Point operators

• Only depends on the pixel value– Plus, potentially, some globally collected

information or parameters

• Brightness scaling • Image composition

– Matting – Blending

• Histogram equalization

Linear Filter

• Smoothing– Box, Bilinear, Gaussian

Linear Filter

• Smoothing– Box, Bilinear, Gaussian

• Edge – Sobel

Linear Filter

• Smoothing– Box, Bilinear, Gaussian

• Edge – Sobel

• Corner

Separable Filter

• Convolution of K-size kernel requires K2 operations

• Can be sped up to 2K operations by – First performing a 1D horizontal convolution – Followed by a 1D vertical convolution

TK vh

Separable Filter

• Fourier transformation

Band-pass filters

• More sophisticated kernels – 1 Smoothing with a Gaussian filter

– 2 Taking first or second derivatives • Sobel • Laplacian • Coner

2 2

222

1( , ; )

2

x y

G x y e

Band-pass filters

• Undirected second derivatives: Laplacian operator

• Laplacian of Gaussian (LoG) filter

– Five point Laplacian

2 22

2 2

f ff

x y

2 22

4 2

2( , ; ) ( , ; )

x yG x y G x y

Steerable Filters

• Directional/Oriented filter– Sobel– Directional derivative

A whole family of filters can be evaluated with very little cost by first convolving the image with (Gx, Gy)

Steerable Filters

• Second-order filter

For directional Gaussian derivatives, it is possible to steer any order of derivative with a relatively small number of basis functions.

Steerable Filters

• Second-order filter

Original image orientation map Original image with oriented structures enhanced.

Steerable Filters

• Fourth-order steerable filter

test image -bars (lines) -step edges -different orientations

average orientedenergy

dominant orientation

oriented energy as a function of angle

(Freeman and Adelson 1991)

Summed Area Table (Integral Image)

• Repeatedly convolved with different box filters– different sizes at – different locations

• Precompute the summed area table (Crow1984)

Summed Area Table (Integral Image)

• Compute the sum of any rectangle area easily

Recursive filtering

Laplacian of Gaussian (LoG)

LoG

• Discrete convolution kernel– Can be any size– Sum_elements = Zero

2 2

22 2 2 2 2

22 2 4

2( , ) ( , ) ( , )

x yx y

LoG G x y G x y G x y ex y

2 22

4 2

2( , ) ( , )

x yG x y G x y

Laplacian of Gaussian (LoG)

Difference of Gaussian (DoG)

• Gaussian

• DoG

2 2

21

1

1

2

2

1( , )

2

x y

G x y e

11( , ) ( , ) ( , )g x y G x y f x y 22 ( , ) ( , ) ( , )g x y G x y f x y

1 2

1 2

1 2( , ) ( , ) ( , ) ( , )

( ) ( , )

g x y g x y G f x y G f x y

G G f x y

2 2 2 2

2 21 2

1 2

1 2

2 2

2 2

1 1 1

2

x y x y

DoG G G e e

Difference of Gaussian (DoG)

Difference of Gaussian (DoG)

LoG and DoG

LoG and DoG2 2

24 2

2( , ) ( , )

x yG x y G x y

2 2

2

22

1( , )

2

x y

G x y e

2 2 2 2 2 2

2 2 22 2 2 2

22 2 23 5 4 2 2

1 2 1

2 2

x y x y x yG x y x y

e e e G

LoG and DoG2 2

24 2

2( , ) ( , )

x yG x y G x y

2 2

2

22

1( , )

2

x y

G x y e

2 2 2 2 2 2

2 2 22 2 2 2

22 2 23 5 4 2 2

1 2 1

2 2

x y x y x yG x y x y

e e e G

2GG

LoG and DoG

2 2

( , , ) ( , , )

( 1)

( , , ) ( , , ) ( 1) ( 1)

G G x y k G x y

k

GG x y k G x y k k G

2GG

DoG LoG

Laplacian for Edge

• Zero-crossing detection

LoG DoG

Pyramids

• Change resolution – Upsampling (Interpolation)– Downsmapling (Decimation)

Interpolation

• Interpolation kernel h() with sampling rate r

• Bilinear

,

( , ) ( , ) ( , )k l

g i j f k l h i rk j rl

Interpolation

• Interpolation kernel h() with sampling rate r

• Bicubic interpolation

,

( , ) ( , ) ( , )k l

g i j f k l h i rk j rl

Bicubic Interpolation

• a specifies the derivative at x=1 • Usually a=-1, best matches the frequency

characteristics of a sinc function– A small amount of sharpening – Ringing (does not linearly interpolate straight lines

• Quadratic reproducing spline a=-0.5

Bicubic Interpolation

Bilinear Cubic a=-1

Cubic a=-0.5

windowed sinc

Windowed sinc function

• Best quality interpolator (Usually)– Both preserves details in the lower resolution

image and avoids aliasing

Windowed sinc function

• Best quality interpolator (Usually)– Both preserves details in the lower resolution

image and avoids aliasing

• Ringing effect– Instead, repeatedly interpolate images by a small

fractional amount

Decimation (Downsampling)

• Same kernel h(k,l) for both interpolation and decimation

• Avoid aliasing– Convolve the image with a low-pass filter

,

1( , ) ( , ) ( , )

k l

k lg i j f k l h i jr rr

Decimation (Downsampling)

• Linear • Binomial

– Separating the high and low frequencies, – but leaves a fair amount of high-frequency detail,

which can lead to aliasing after downsampling

Decimation (Downsampling)

• Linear• Binomial • Cubic

– a=-1, – a=-0.5

• Windowed sinc • QMF-9• Jpeg2000

Sample rate = 2

Decimation (Downsampling)

• Cubic a=-1 – Sharpest but ringing

• QMF-9 and Jpeg2000– Wavelet analysis filters– Useful for compression – More aliasing

Multi-resolution Representations

• Image pyramid– Accelerate coarse-to-fine

search algorithms– Look for objects or

patterns at different scales

– Perform multi-resolution blending operations

Multi-resolution Representations

• Laplacian pyramid [Burt and Adelson’s (1983a)]

– Best known and most widely used in computer vision

Laplacian Pyramid

• First: blur and subsample the original image with sample rate r=2

• Five-tap kernel

Octave pyramid

Laplacian pyramid

• First: blur and subsample the original image by sample rate = 2

Gaussian pyramid:Repeated convolutions of the binomial kernel

converge to a Gaussian

Laplacian Pyramid

Laplacian Pyramid

• Actual computation of high-pass filter• Results in perfect reconstruction when Q=I

Laplacian image

Gaussian image

Application: Image Blending

regular splice

pyramid blend

Low frequency part

Medium frequency part

High frequency part

Image Blending