Top Banner
Chapter 4 Image Segmentation
68

Chapter 4 Image Segmentation. Outline Introduction to segmentation Point,line and edge detection Thresholding techniques Watershed segmentation.

Jan 13, 2016

Download

Documents

Beryl Hardy
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Chapter 4 Image Segmentation

Page 2: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Outline

Introduction to segmentationPoint,line and edge detectionThresholding techniquesWatershed segmentationRegion based segmentationApplications

2

Page 3: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

3

Preview

• Segmentation is to subdivide an image into its component regions or objects.

• Segmentation should stop when the objects of interest in an application have been isolated.

Page 4: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

4

Principal approaches

• Segmentation algorithms generally are based on one of 2 basis properties of intensity values– discontinuity : to partition an image based on

sharp changes in intensity (such as edges)– similarity : to partition an image into regions

that are similar according to a set of predefined criteria.

Page 5: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

5

Detection of Discontinuities

• detect the three basic types of gray-level discontinuities– points , lines , edges

• the common way is to run a mask through the image

Page 6: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

6

Point Detection

• a point has been detected at the location on which the mark is centered if

|R| T

• where – T is a nonnegative threshold – R is the sum of products of the coefficients with

the gray levels contained in the region encompassed by the mark.

Page 7: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

7

Point Detection

• Note that the mark is the same as the mask of Laplacian Operation

• The only differences that are considered of interest are those large enough (as determined by T) to be considered isolated points.

|R| T

Page 8: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Example

8

Page 9: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

9

Line Detection

• Horizontal mask will result with max response when a line passed through the middle row of the mask with a constant background.

• the similar idea is used with other masks.• note: the preferred direction of each mask is weighted with

a larger coefficient (i.e.,2) than other possible directions.

Page 10: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

10

Line Detection

• Apply every masks on the image• let R1, R2, R3, R4 denotes the response of

the horizontal, +45 degree, vertical and -45 degree masks, respectively.

• if, at a certain point in the image

|Ri| > |Rj|,• for all ji, that point is said to be more likely

associated with a line in the direction of mask i.

Page 11: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

11

Line Detection

• Alternatively, if we are interested in detecting all lines in an image in the direction defined by a given mask, we simply run the mask through the image and threshold the absolute value of the result.

• The points that are left are the strongest responses, which, for lines one pixel thick, correspond closest to the direction defined by the mask.

Page 12: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

12

Example

Page 13: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

13

Edge Detection

• we discussed approaches for implementing– first-order derivative (Gradient operator)– second-order derivative (Laplacian operator)

Page 14: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

14

Ideal and Ramp Edges

because of optics, sampling, image acquisition imperfection

Page 15: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

15

Thick edge• The slope of the ramp is inversely proportional to the

degree of blurring in the edge.• We no longer have a thin (one pixel thick) path.• Instead, an edge point now is any point contained in the

ramp, and an edge would then be a set of such points that are connected.

• The thickness is determined by the length of the ramp.• The length is determined by the slope, which is in turn

determined by the degree of blurring.• Blurred edges tend to be thick and sharp edges tend Blurred edges tend to be thick and sharp edges tend

to be thinto be thin

Page 16: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

16

First and Second derivatives

the signs of the derivatives would be reversed for an edge that transitions from light to dark

Page 17: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

17

Second derivatives

• produces 2 values for every edge in an image (an undesirable feature)

• an imaginary straight line joining the extreme positive and negative values of the second derivative would cross zero near the midpoint of the edge. (zero-crossing zero-crossing propertyproperty)

Page 18: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

18

Noise in Images

• First column: images and gray-level profiles of a ramp edge corrupted by random Gaussian noise of mean 0 and = 0.0, 0.1, 1.0 and 10.0, respectively.

• Second column: first-derivative images and gray-level profiles.

• Third column : second-derivative images and gray-level profiles.

Page 19: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

19

Noise in images(contd)

• fairly little noise can have such a significant impact on the two key derivatives used for edge detection in images

• image smoothing should be serious consideration prior to the use of derivatives in applications where noise is likely to be present.

Page 20: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

20

Edge point

• to determine a point as an edge point– the transition in grey level associated with the

point has to be significantly stronger than the background at that point.

– use threshold to determine whether a value is “significant” or not.

– the point’s two-dimensional first-order derivative must be greater than a specified threshold.

Page 21: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

21

Gradient Operator

• first derivatives are implemented using the magnitude of the gradientmagnitude of the gradient.

y

fx

f

G

G

y

xf

21

22

2122 ][)f(

y

f

x

f

GGmagf yx

the magnitude becomes nonlinearyx GGf

commonly approx.

Page 22: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

22

Gradient Masks

Page 23: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

23

Diagonal edges with Prewitt and Sobel masks

Page 24: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

24

Example

Page 25: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

25

Example

Page 26: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

26

Example

Page 27: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

27

Laplacian

2

2

2

22 ),(),(

y

yxf

x

yxff

(linear operator)

Laplacian operator

)],(4)1,()1,(

),1(),1([2

yxfyxfyxf

yxfyxff

Page 28: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

28

Laplacian of Gaussian

• Laplacian combined with smoothing to find edges via zero-crossing.

2

2

2)( r

erh

where r2 = x2+y2, and is the standard deviation

2

2

24

222 )(

r

er

rh

Page 29: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

29

Mexican hat

the coefficient must be sum to zero

positive central term surrounded by an adjacent negative region (a function of distance)zero outer region

Page 30: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

30

Linear Operation

• second derivation is a linear operation

• thus, 2f is the same as convolving the image with Gaussian smoothing function first and then computing the Laplacian of the result

Page 31: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

31

Example

a). Original imageb). Sobel Gradientc). Spatial Gaussian smoothing functiond). Laplacian maske). LoGf). Threshold LoGg). Zero crossing

Page 32: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

32

Zero crossing & LoG

• Approximate the zero crossing from LoG image

• to threshold the LoG image by setting all its positive values to white and all negative values to black.

• the zero crossing occur between positive and negative values of the thresholded LoG.

Page 33: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

33

Thresholdingimage with dark background and

a light object

image with dark background and two light objects

Page 34: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

34

Multilevel thresholding

• a point (x,y) belongs to – to an object class if T1 < f(x,y) T2

– to another object class if f(x,y) > T2

– to background if f(x,y) T1

• T depends on – only f(x,y) : only on gray-level values Global threshold– both f(x,y) and p(x,y) : on gray-level values and its

neighbors Local threshold

Page 35: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

35

The Role of Illumination

f(x,y) = i(x,y) r(x,y)

a). computer generated reflectance functionb). histogram of reflectance functionc). computer generated illumination function (poor)d). product of a). and c).e). histogram of product image

easily use global thresholdingobject and background are separated

difficult to segment

Page 36: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

36

Basic Global Thresholding

generate binary image

use T midway between the max and min gray levels

Page 37: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

37

Basic Global Thresholding• based on visual inspection of histogram1. Select an initial estimate for T.2. Segment the image using T. This will produce two

groups of pixels: G1 consisting of all pixels with gray level values > T and G2 consisting of pixels with gray level values T

3. Compute the average gray level values 1 and 2 for the pixels in regions G1 and G2

4. Compute a new threshold value5. T = 0.5 (1 + 2)6. Repeat steps 2 through 4 until the difference in T in

successive iterations is smaller than a predefined parameter To.

Page 38: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

38

Example: Heuristic methodnote: the clear valley of the histogram and the effective of the segmentation between object and background

T0 = 03 iterations with result T = 125

Page 39: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

39

Basic Adaptive Thresholding

• subdivide original image into small areas.

• utilize a different threshold to segment each subimages.

• since the threshold used for each pixel depends on the location of the pixel in terms of the subimages, this type of thresholding is adaptive.

Page 40: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

40

Example : Adaptive Thresholding

Page 41: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

41

Further subdivisiona). Properly and improperly segmented subimages from previous example b)-c). corresponding histogramsd). further subdivision of the improperly segmented subimage.e). histogram of small subimage at topf). result of adaptively segmenting d).

Page 42: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Segmentation by Morphological Watersheds

Page 43: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Introduction• Based on visualizing an image in 3D

020

4060

80100

0

50

1000

5

10

15

20

25

imshow(I,[ ]) mesh(I)

Page 44: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

• Instead of working on an image itself, this technique is often applied on its gradient image.– In this case, each object is

distinguished from the background by its up-lifted edges

020

4060

80100

0

20

40

60

80

1000

2

4

6

8

Page 45: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Basic Definitions

• I: 2D gray level image• DI: Domain of I• Path P of length l between p and q in I

– A (l +1)-tuple of pixels (p0=p,p1,…,pl=q) such that pi,pi+1 are adjacent (4 adjacent, 8 adjacent, or m adjacent, see Section 2.5)

• l(P): The length of a given path P• Minimum

– A minimum M of I is a connected plateau of pixels from which it is impossible to reach a point of lower altitude without having to climb

Page 46: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Basic Definitions• Instead of working on an image itself, this technique is often applied

on its gradient image.• Three types of points

– Points belonging to a regional minimum– Catchment basin / watershed of a regional minimum

• Points at which a drop of water will certainly fall to a single minimum– Divide lines / Watershed lines

• Points at which a drop of water will be equally likely to fall to more than one minimum

• Crest lines on the topographic surface• This technique is to identify all the third type of points for

segmentation

Page 47: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Basic Steps1. Piercing holes in

each regional minimum of I

2. The 3D topography is flooded from below gradually

3. When the rising water in distinct catchment basins is about to merge, a dam is built to prevent the merging

Page 48: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

The dam boundaries correspond to the watershed lines to be extracted by a watershed segmentation algorithm

- Eventually only constructed dams can be seen from above

Page 49: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Dam Construction

• Based on binary morphological dilation• At each step of the algorithm, the binary

image in obtained in the following manner1. Initially, the set of pixels with minimum gray

level are 1, others 0.2. In each subsequent step, we flood the 3D

topography from below and the pixels covered by the rising water are 1s and others 0s. (See previous slides)

Page 50: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Notations• M1, M2:

– Sets of coordinates of points in the two regional minima

• Cn-1(M1), Cn-1(M2)– Sets of coordinates of points

in the catchment basins associated with M1 M2 at stage n-1 of flooding (catchment basins up to the flooding level)

• C[n-1]– Union of Cn-1(M1), Cn-1(M2)

Page 51: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Dam Construction• At flooding step n-1, there are

two connected components. At flooding step n, there is only one connected component

– This indicates that the water between the two catchment basins has merged at flooding step n

– Use “q” to denote the single connected component

• Steps– Repeatedly dilate Cn-1(M1), Cn-1(M2)

by the 3×3 structuring element shown, subject to the following condition

• Constrained to q (center of the structuring element can not go beyond q during dilation

Page 52: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Dam Construction• The dam is constructed by

the points on which the dilation would cause the sets being dilated to merge.

– Resulting one-pixel thick connected path

• Setting the gray level at each point in the resultant path to a value greater than the maximum gray value of the image. Usually max+1

Page 53: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Watershed Transform• Denote M1, M2, …, MR as the sets of the coordinates of the points in

the regional minima of an (gradient) image g(x,y)

• Denote C(Mi) as the coordinates of the points in the catchment basin associated with regional minimum Mi.

• Denote the minimum and maximum gray levels of g(x,y) as min and max

• Denote T[n] as the set of coordinates (s,t) for which g(s,t) < n

• Flood the topography in integer flood increments from n=min+1 to n=max+1

• At each flooding, the topography is viewed as a binary image

Page 54: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Watershed Transform• Denote Cn(Mi) as the set of

coordinates of points in the catchment basin associated with minimum Mi at flooding stage n.– Cn(Mi)= C(Mi) T[n]

– Cn(Mi)=T[n]

• Denote C[n] as the union of the flooded catchment basin portions at stage n:

• Initialization– Let C[min+1]=T[min+1]

• At each step n, assume C[n-1] has been constructed. The goal is to obtain C[n] from C[n-1]

C(n)R

ii

R

iin MCCMCnC

11

)(]1[max and )(][

Page 55: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Watershed Transform• Denote Q[n] as the set of connected components in T[n]. • For each qQ[n], there are three possibilities

1. q C[n-1] is empty (q1)

• A new minimum is encountered• q is incorporated into C[n-1] to form C[n]

2. q C[n-1] contains one connected component of C[n-1] (q2)

• q is incorporated into C[n-1] to form C[n]

3. q C[n-1] contains more than one connected components of C[n-1] (q3)

• A ridge separating two or more catchment basins has been encountered

• A dam has to be built within q to prevent overflow between the catchment basins

4. Repeat the procedure until n=max+1

Page 56: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Examplea: Original imageb: Gradient image of image ac: Watershed lines obtained

from image b (oversegmentation)

Each connected region contains one local minimum in the corresponding gradient image

d: Watershed lines obtained from smoothed image b

Page 57: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

The Use of Markers• Internal markers are used to limit the number of regions by

specifying the objects of interest

– Like seeds in region growing method

– Can be assigned manually or automatically– Regions without markers are allowed to be merged (no

dam is to be built)

• External markers those pixels we are confident to belong to the background

– Watershed lines are typical external markers and they belong the same (background) region

Page 58: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Watershed Based Image Segmentation

1. Use internal markers to obtain watershed lines of the gradient of the image to be segmented.

2. Use the obtained watershed lines as external markers3. Each region defined by the external markers contains a single

internal marker and part of the background4. The problem is reduced to partitioning each region into two

parts: object (containing internal markers) and a single background (containing external markers)

– Global thresholding, region growing, region splitting and merging, or watershed transform

Page 59: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

59

Region-Based Segmentation - Region Growing

• start with a set of “seed” points

• growing by appending to each seed those neighbors that have similar properties such as specific ranges of gray level

Page 60: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

60

Region Growing

select all seed points with gray level 255

criteria:1. the absolute gray-

level difference between any pixel and the seed has to be less than 65

2. the pixel has to be 8-connected to at least one pixel in that region (if more, the regions are merged)

Page 61: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

• Region growing: Groups pixels or sub-region into larger regions.– step1:

• Start with a set of “seed” points and from these grow regions by appending to each seed those neighboring pixels that have properties similar to the seed.

– step2:• Region splitting and merging

61

Region-based segmentation

Page 62: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

• Advantage:– With good connectivity

• Disadvantage:– Initial seed-points:

• different sets of initial seed-point cause different segmented result

– Time-consuming problem

62

Region-based segmentation

Page 63: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

• Unseeded region growing:– no explicit seed selection is necessary, the

seeds can be generated by the segmentation procedure automatically.

– It is similar to SRG except the choice of seed point

63

Region-based segmentationUnseeded region growing

Page 64: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

• Advantage:– easy to use– can readily incorporate high level knowledge of

the image composition through region threshold

• Disadvantage:– slow speed

64

Region-based segmentationUSRG

Page 65: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Region-based segmentationfast scanning

• Fast scanning Algorithm: – The fast scanning

algorithm somewhat resembles unseeded region growing

– the number of clusters of both two algorithm would not be decided before image passing through them.

65

Page 66: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

66

Region-based segmentationfast scanning

Page 67: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

Region-based segmentationfast scanning

• Last step:

– merge small region to big region

67

Page 68: Chapter 4 Image Segmentation. Outline  Introduction to segmentation  Point,line and edge detection  Thresholding techniques  Watershed segmentation.

• Advantage:– The speed is very fast– The result of segmentation will be intact with good

connectivity

• Disadvantage:– The matching of physical object is not good

• It can be improved by morphology and geometric mathematic

68

Region-based segmentationfast scanning