M. Wu: ENEE631 Digital Image Processing (Spring'09) Edge Detection and Basics on 2-D Random Signal Spring ’09 Instructor: Min Wu Electrical and Computer.

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Edge Detection and Edge Detection and

Basics on 2-D Random SignalBasics on 2-D Random Signal

Spring ’09 Instructor: Min Wu

Electrical and Computer Engineering Department,

University of Maryland, College Park

bb.eng.umd.edu (select ENEE631 S’09) [email protected]

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

09

)

ENEE631 Spring’09ENEE631 Spring’09Lecture 6 (2/11/2009)Lecture 6 (2/11/2009)

M. Wu: ENEE631 Digital Image Processing (Spring'09)Lec6 – Edge Detection & 2-D

Random Signal [2]

OverviewOverview

Last Time:– Fourier Analysis for 2-D signals– Impulse Response and Frequency Response for 2-D LSI System– Image enhancement via Spatial Filtering

Denoising by averaging filter and median filter

Today– Spatial filtering (cont’d): image sharpening and edge detection– Characterize 2-D random signal (random field)

Assignment 1 Due next Monday– See course website for submission instruction and updated image

zip files

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Random Signal [4]

Review:Review:2-D Fourier Transforms2-D Fourier Transforms

Separable implementations for 2-D FT, DSFT, DFT due to separable 2-D complex exponentials

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

© 2

00

2)

F(x, y) F(x, y)

F(x, y)

2-D FT on continuous-indexed signal

1

0

1

0

1

0

1

0

),(),(

),(1

),(

M

u

N

v

muM

nvN

M

m

N

n

muM

nvN

WWvuGnmg

WWnmgMN

vuG

DFT on sampled periodic signalDSFT on sampled signal


Random Signal [5]

Review:Review:Freq. Response & Eigen Function for LSI SystemFreq. Response & Eigen Function for LSI System Relations between signal domain and Fourier domain

– I/O relation for LSI system: Convolve input with impulse response

y[n] = x[n] h[n] Y() = X() H()

– FT of complex exponentials: exp[j 0 n] ( 0)

Eigen function of 1-D LSI system– Output response of complex exponential input x[n]=exp[j 0n]:

Y() = H() ( 0) = H(0) ( 0)

=> y[n] = H(0) exp[j 0n] i.e. output has same signal shape as the input

with a possible change only in amplitude and phase specified by H(0)

– H() is the LSI system’s frequency response Extend to 2-D LSI system

– y[m, n] = x[m, n] h[m, n] Y(u, v) = X(u, v) H(u, v)– Eigen function is 2-D complex exponentials; Freq. response is H(u, v)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Random Signal [6]

Spatial Operations with Spatial MaskSpatial Operations with Spatial Mask

Spatial mask is 2-D finite impulse response (FIR) filter

– Usually has small support 2x2, 3x3, 5x5, 7x7

– Convolve this filter with image g(m,n) = f(m-x, n-y) h(x,y)

= f(x,y) h(m-x, n-y) … mirror w.r.t. origin, then shift & sum up

– In frequency domain: multiplying DFT(image) with DFT(filter)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)Note: spatial mask is often

specified as the already mirrored version of the

equivalent FIR filter.

Image examples are from Gonzalez-Woods 2/e

online slides.


Random Signal [7]

Directional Smoothing Directional Smoothing

Simple spatial averaging mask blurs edges

– Improve by avoiding filtering across edges

– Restrict smoothing to along edge direction

Directional smoothing

– Compute spatial average along several directions– Take the result from the direction giving the smallest changes

before and after filtering

Other solutions

– Use more explicit edge detection and adapt filtering accordingly

W

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)“isotropic” filter has response

independent of directions(aka. circularly symmetric

or rotation invariant)

1/9 1/9

1/9 1/9

-1 0 1

-1

0

1

1/9

1/9

1/9 1/9 1/9

0 1/8

1/8 1/2

-1 0 1

-1

0

1

0

1/8

0 1/8 0


Random Signal [8]

Median FilteringMedian Filtering

Salt-and-Pepper noise– Isolated extreme-valued (white/black) pixels

spread randomly over the image– Spatial averaging filter may lead to blurred

output when averaging with extreme values

Median filtering

– Output the median over a small window Nonlinear operation:

Median{ x(m) + y(m) } Median{x(m)} + Median{y(m)}

– Odd window size is commonly used 3x3, 5x5, 7x7; 5-pixel “+” shaped window

– Even-sized window ~ take the average of two middle values as output

Generalize: apply order statistic operations

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Random Signal [9]

Image SharpeningImage Sharpening

Use LPF to generate HPF

– Subtract a low pass filtered result from the original signal– HPF identifies the locations of a signal’s transitions

Enhance edges

I0 ILP

IHP = I0 – ILP

I1 = I0 + a*IHP

0 20 40 60 80 1000

10

20

30

40

50

0 20 40 60 80 1000

10

20

30

40

50

0 20 40 60 80 100-10

-5

0

5

10

0 20 40 60 80 1000

10

20

30

40

50

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Random Signal [10]

Example of Image SharpeningExample of Image Sharpening

– v(m,n) = u(m,n) + a * g(m,n)– Often use Laplacian operator to obtain g(m,n)– Laplacian operator is a discrete form of 2nd-order derivatives

0 -¼

-¼ 1

-1 0 1

-1

0

1

0

-¼

0 -¼ 0

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Random Signal [11]

Example of Image SharpeningExample of Image Sharpening

Original moon image is from Matlab Image Toolbox.UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)


Random Signal [12]

Other Variations of Image SharpeningOther Variations of Image Sharpening High boost filter (Gonzalez-Woods 2/e pp132 & pp188)

I0 ILP IHP = I0 – ILP I1 = (b-1) I0 + IHP

– Equiv. to high pass filtering for b=1– Amplify or suppress original image pixel values when b2

Combine sharpening with histogram equalization

Image example is from Gonzalez-Woods 2/e online slides.

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

© 2

00

2)


Random Signal [13]

Impulse and Frequency Responses of LPF / HPFImpulse and Frequency Responses of LPF / HPF

e.g. Gaussian LPF filter

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

Image example is from Gonzalez-Woods 2/e online slides.


Random Signal [14]

Edges and Gradient VectorEdges and Gradient Vector Edge: pixel locations of abrupt luminance change

For binary image– Take black pixels with immediate white neighbors as edge pixel

Detectable by XOR operations

For continuous-tone image– How to represent edge?

by intensity + direction => Edge map ~ edge intensity + directions

– Detection Method-1: prepare edge examples (templates) of different intensities and directions, then find the best match

– Spatial luminance gradient vector of an edge pixel: edge gradient gives the direction with highest rate of luminance

changes is a vector of partial derivatives along two orthogonal directions

– Detection Method-2: measure transitions along 2 orthogonal directions

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Random Signal [15]

Edge DetectionEdge Detection

Measure gradient vector– Along two orthogonal directions ~ usually horizontal and vertical

gx = L / x gy= L / y

– Magnitude of gradient vector g(m,n) 2 = gx(m,n) 2 + gy(m,n) 2

g(m,n) = |gx(m,n) | + |gy(m,n)| (preferred in hardware implement.)

– Direction of gradient vector tan –1 [ gy(m,n) / gx(m,n) ]

Characterizing edges in an image– (binary) Edge map: specify “edge point” locations with g(m,n) > thresh.– Edge intensity map: specify gradient magnitude at each pixel– Edge direction map: specify directions

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Random Signal [16]

Common Gradient Operators for Edge DetectionCommon Gradient Operators for Edge Detection

– Move the operators across the image and take the inner products Magnitude of gradient vector g(m,n) 2 = gx(m,n) 2 + gy(m,n) 2

Direction of gradient vector tan –1 [ gy(m,n) / gx(m,n) ]

– Gradient operator is HPF in nature ~ could amplify noise Prewitt and Sobel operators compute horizontal and vertical

differences of local sum to reduce the effect of noise

0 1

-1 0

-1 0

-1 0

1

1

-1 0 1

-1 0

-2 0

1

2

-1 0 1

1 0

0 -1

-1 -1

0 0

-1

0

1 1 1

-1 -2

0 0

-1

0

1 2 1

Roberts

H1(m,n)

H2(m,n)

Prewitt Sobel

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Random Signal [17]

Derivative Operators: A Closer LookDerivative Operators: A Closer Look

Spatial averaging filter perpendicular to direction of discrete derivative

discrete derivative filter in horizontal direction

Combination of the two masks gives single “averaged gradient mask” in horizontal direction.


Random Signal [18]

Examples of Examples of

Edge DetectorsEdge Detectors– Quantize edge

intensity to 0/1: set a threshold white pixel

denotes strong edge

Roberts Prewitt Sobel

UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)


Random Signal [20]

Robust Edge DetectorRobust Edge Detector Apply LPF to suppress noise, then apply edge detector or

derivative operations

E.g. Laplacian of Gaussian: in shape of Mexican hat

Figures from Gonzalez-Woods 2/e online slides.


Random Signal [22]

Summary: Spatial LPF, HPF, & BPFSummary: Spatial LPF, HPF, & BPF

HPF and BPF can be constructed from LPF

Low-pass filter

– Useful in noise smoothing and downsampling/upsampling

High-pass filter

– hHP(m,n) = (m,n) – hLP(m,n)

– Useful in edge extraction and image sharpening

Band-pass filter

– hBP(m,n) = hL2(m,n) – hL1(m,n)

– Useful in edge enhancement– Also good for high-pass tasks in the presence of noise

avoid amplifying high-frequency noise

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Random Signal [23]

2-D Random Signals (aka Random Field)2-D Random Signals (aka Random Field)

Side-by-Side Comparison with 1-D Random Process

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

(1) Sequences of random variables & joint distributions(2) First two moment functions and their properties (3) Wide-sense stationarity(4) Unique to 2-D case: separable and isotropic covariance function(5) Power spectral density and properties


Random Signal [24]

Statistical Representation of ImagesStatistical Representation of Images

Each pixel is considered as a random variable (r.v.)

Relations between pixels– Simplest case: i.i.d.– More realistically, the color value at a pixel may be statistically

related to the colors of its neighbors

A “sample” image– A specific image we have obtained to study can be considered as a

sample from an ensemble of images – The ensemble represents all possible value combinations of

random variable array

Similar ensemble concept for 2-D random noise signals– Each 2-D location can take a real value

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Random Signal [25]

Characterize the Ensemble of 2-D SignalsCharacterize the Ensemble of 2-D Signals

Specify by a joint probability distribution function– Difficult to measure and specify the joint distribution for images

of practical size=> too many r.v. : e.g. 512 x 512 = 262,144

Specify by the first few moments– Mean (1st moment) and Covariance (2nd moment)

may still be non-trivial to measure for the entire image size

By various stochastic models– Use a few parameters to describe the relations among all pixels

E.g. 2-D extensions from 1-D Autoregressive (AR) model

Important for a variety of image processing tasks– image compression, enhancement, restoration, understanding, …

=> Today: some basics on 2-D random signals

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Random Signal [26]

Discrete Random FieldDiscrete Random Field We call a 2-D sequence discrete random field if each of its

elements is a random variable

– when the random field represents an ensemble of images, we often call it a random image

Mean and Covariance of a complex random field

E[u(m,n)] = (m,n)Cov[u(m,n), u(m’,n’)] = E[ (u(m,n) – (m,n)) (u(m’,n’) – (m’,n’))* ] = ru( m, n; m’, n’)

For zero-mean random field, autocorrelation function = cov. function

Wide-sense stationary (or wide-sense homogeneity)

(m,n) = = constant

ru( m, n; m’, n’) = ru( m – m’, n – n’; 0, 0) = r( m – m’, n – n’ ) also called shift invariant or spatial invariant in some literature

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Random Signal [27]

Special Random Fields of InterestsSpecial Random Fields of Interests

White noise field

– A stationary random field– Any two elements at different locations x(m1,n1) and x(m2,n2) are

mutually uncorrelated

rx( m, n ) = x2

( m, n )

Gaussian random field

– Every segment defined on an arbitrary finite grid is Gaussian i.e. every finite segment of u(m,n) when mapped into a vector

have a joint Gaussian p.d.f. ofUM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Random Signal [29]

Properties of Covariance for Random FieldProperties of Covariance for Random Field

[Recall similar properties of covariance function for 1-D random process]

Symmetry

ru( m, n; m’, n’) = ru*( m’, n’; m, n)

• For stationary random field: r( m, n ) = r*( -m, -n )• For stationary real random field: r( m, n ) = r( -m, -n )

• Note in general ru( m, n; m’, n’) ru( m’, n; m, n’) ru( m’, n; m, n’)

Non-negativity

mnm’n’ x(m, n) ru( m, n; m’, n’) x*(m’, n’) 0

~ Recall for 1-D case, correlation matrix is non-negative definite: xH R x 0 for all x .

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

Recall: ru ( m, n; m’, n’) = E[ (u(m,n) – (m,n)) (u(m’,n’) – (m’,n’))* ]


Random Signal [31]

Separable Covariance FunctionsSeparable Covariance Functions

Separability

– If the covariance function of a random field can be expressed as a product of covariance functions of 1-D sequences

r( m, n; m’, n’) = r1( m, m’) r2( n, n’) ~ Nonstationary case

r( m, n ) = r1( m ) r2( n ) ~ Stationary case

Example:

– A separable stationary covariance function often used in image proc for its simplicity

r(m, n) = 2 1

|m| 2

|n| , |1|<1 and |2|<1

2 represents the variance of the random field; 1

and 2 are the one-step correlations in the m and n directions

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Random Signal [33]

Isotropic Covariance FunctionsIsotropic Covariance Functions

Isotropic / circularly symmetric

– i.e. the covariance function only changes with respect to the radius (the distance to the origin), and isn’t affected by the angle

Example

– A nonseparable exponential function often used as a more realistic model of the covariance function for images

– When a1= a2 = a2 , this becomes isotropic: r(m, n) = 2 d

As a function of the Euclidean distance of d = ( m 2 + n 2 ) 1/2

= exp(-|a|), i.e. correlation is decaying as distance increases

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Random Signal [34]

Estimating the Mean and Covariance FunctionEstimating the Mean and Covariance Function

Approximate the ensemble average with sample average

Example: for an M x N real-valued image x(m, n)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

Note: similar to the 1-D case, the cov estimates here are biased, in order to achieve smaller variance in estimation and to avoid the possible negative definiteness by unbiased estimate


Random Signal [36]

Spectral Density FunctionSpectral Density Function The Spectral density function (SDF) is defined as the

Fourier transform of the covariance function rx

– Also known as the power spectral density (p.s.d.)( in some text, p.s.d. is defined as the FT of autocorrelation

function )

Example: SDF of stationary white noise field with r(m,n)= 2

(m,n)

m n

x nmjnmrS )](exp[),(),( 2121

221

221 )](exp[),(),(

m n

nmjnmS

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Random Signal [38]

Properties of Power SpectrumProperties of Power Spectrum

[Recall similar properties in 1-D random process]

SDF is real: S(1, 2) = S*(1, 2) – Follows the conjugate symmetry of the covariance function

r(m, n) = r *(-m, -n)

SDF is nonnegative: S(1, 2) 0 for 1,2

– Follows the non-negativity property of covariance function– Intuition: “power” cannot be negative

SDF of the output from a LSI system w/ freq response H(1, 2)

Sy(1, 2) = | H(1, 2) |2 Sx(1, 2)

DSFT of cross correlation: Syx(1, 2) = H(1, 2) Sx(1, 2)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Random Signal [39]

Summary of Today’s LectureSummary of Today’s Lecture

Spatial filter: LPF, HPF, BPF

– Image sharpening and edge detection

Basics on 2-D random signals

Next time

– Continue on 2-D random field; – Image restoration

Readings

– Gonzalez’s book 3.6-3.7; 10.2; 5.2; Wood’s book 7.1 (on random field)

– For further readings: Woods’ book 6.6; 3.1, 3.2, 3.5.0;Jain’s book 7.4; 9.4; 2.9-2.11

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Random Signal [40]

For Next LectureFor Next Lecture

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

(1) Sequences of random variables & joint distributions(2) First two moment functions and their properties (3) Wide-sense stationarity(4) Unique to 2-D case: separable and isotropic covariance function(5) Power spectral density and properties


Random Signal [41]

Z-Transform Expression of Power SpectrumZ-Transform Expression of Power Spectrum

The Z transform of ru

– Known as the covariance generating function (CGF) or the ZT expression of the power spectrum

22

11 ,2121

2121

|),(),(

),(),(

jj ezez

m n

nmx

zzSS

zznmrzzS

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Random Signal [43]

2-D Z-Transform2-D Z-Transform

The 2-D Z-transform is defined by

– The space represented by the complex variable pair (z1, z2) is 4-D

Unit surface

If Region of Convergence(ROC) include unit surface

Transfer function of 2-D discrete LSI system

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Random Signal [44]

StabilityStability

Recall for 1-D LTI system

– Stability condition in bounded-input bounded-output sense (BIBO) is that the impulse response h[n] is absolutely summable i.e. ROC of H(z) includes

the unit circle

– H(z) for a causal and stable system should have all poles inside the unit circle

2-D Stable LSI system

– Requires the 2-D impulse response is absolutely summable

– i.e. ROC of H(z1, z2) must include the unit surface |z1|=1, |z2|=1

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Edge Detection and Basics on 2-D Random Signal Spring ’09 Instructor: Min Wu Electrical and Computer.

Documents

d fourier

d transforms

d random signal spring

d lsi system image enhancement

d lsi system ym

d signals impulse response

signal domain

d ft u convolution