Region Segmentation. Find sets of pixels, such that All pixels in region i satisfy some constraint of similarity.

Region Segmentation

Region Segmentation Find sets of pixels, such that

All pixels in region i satisfy some constraint of similarity.

nRRR ,,, 21 IR

ii

ji RRji ,

K-Means Choose a fixed number of

clusters

Choose cluster centers and point-cluster allocations to minimize error

can’t do this by search, because there are too many possible allocations.

Algorithm fix cluster centers;

allocate points to closest cluster

fix allocation; compute best cluster centers

x could be any set of features for which we can compute a distance (careful about scaling)

x j i

2

jelements of i'th cluster

iclusters

K-Means

Image Segmentation by K-Means Select a value of K Select a feature vector for every pixel (color, texture,

position, or combination of these etc.) Define a similarity measure between feature vectors

(Usually Euclidean Distance). Apply K-Means Algorithm. Apply Connected Components Algorithm. Merge any components of size less than some threshold to

an adjacent component that is most similar to it.

K-means clustering using intensity alone and color alone

Image Clusters on intensity Clusters on color

K-means using color alone, 11 segments

Image Clusters on color

K-means usingcolor alone,11 segments.

K-means using colour andposition, 20 segments

How to find K Use prior knowledge about image. Apply the algorithm for different values of

K and test for goodness of clusters. Analyze Image Histograms.

How to find K

How to find K

How to Find KRealistic Histograms

How to Find KSmooth Histogram. (Convolve by averagingor Gaussian Filter)

How to Find KFind Peaks and Valleys and perform peakiness test.

Agglomerative clustering Assume that each cluster is single pixel (i.e.

every pixel is a cluster itself). Merge Clusters i.e. attach closest to cluster it

is closest to (if possible) Repeat step 2 until no more clusters can be

merged.

Divisive clustering Assume that whole image is a single cluster. Split Clusters along best boundary (if exists) Repeat step 2 until no more clusters can be

split.

Inter-Cluster distance single-link clustering: Minimum distance

between an element of the first cluster and one of the second..

complete-link clustering: Maximum distance between an element of the first cluster and one of the second.

group-average clustering: Average of distances between elements in the clusters.

Segmentation by Split and Merge Start with an initial segmentation (for example by K-

Means). Define a criteria P for goodness of region such that

P( R )=True, if R satisfies the criteria P( R )=False, otherwise

For each region R, split R in four regions (quadrants), if P( R ) = False

Merge any two adjacent regions R and Q if

Repeat until no more clusters can be split or merged.

TrueP RQ

Segmentation by Split and Merge

Graph Theoretical Techniques for Image Segmentation

Graph A graph G(V,E) is a triple consisting of a

vertex set V(G) an edge set E(G) and a relation that associates with each edge two vertices called its end points.

Path A path is a sequence of edges e1, e2, e3, …

en. Such that each (for each i>2 & i<n) edge ei is adjacent to e(i+1) and e(i-1). e1 is only adjacent to e2 and en is only adjacent to e(n-1)

Connected & Disconnected Graph

A graph G is connected if there is a path from every vertex to every other vertex in G.

A graph G that is not connected is called disconnected graph.

Graphs Representations

a

e

d

c

b

01101

10000

10000

00001

10010

Adjacency Matrix: W

Weighted Graphs and Their Representations

a

e

d

c

b

0172

106

76043

2401

310

Weight Matrix: W

6

Minimum CutA cut of a graph G is the set of edges S such that removal of S from G disconnects G.

Minimum cut is the cut of minimum weight, where weight of cut <A,B> is given as

ByAxyxwBAw

,,,

Minimum Cut and Clustering

Image Segmentation & Minimum Cut

ImagePixels

Pixel Neighborhood

w

SimilarityMeasure

MinimumCut

Minimum Cut There can be more than one minimum cut in a

given graph

All minimum cuts of a graph can be found in polynomial time1.

1H. Nagamochi, K. Nishimura and T. Ibaraki, “Computing all small cuts in an undirected network. SIAM J. Discrete Math. 10 (1997) 469-481.

Drawbacks of Minimum Cut Weight of cut is directly proportional to the

number of edges in the cut.

Ideal Cut

Cuts with lesser weightthan the ideal cut

Normalized Cuts1

Normalized cut is defined as

Ncut(A,B) is the measure of dissimilarity of sets A and B.

Minimizing Ncut(A,B) maximizes a measure of similarity within the sets A and B

VyBzVyAx

cut yzw

BAw

yxw

BAwBAN

,,,

,

,

,,

1J. Shi and J. Malik, “Normalized Cuts & Image Segmentation,” IEEE Trans. of PAMI, Aug 2000.

Finding Minimum Normalized-Cut Finding the Minimum Normalized-Cut is

NP-Hard. Polynomial Approximations are generally

used for segmentation

Finding Minimum Normalized-Cut

ImagePixels

Pixel Neighborhood

w

SimilarityMeasure

1

n

32

n-1

Finding Minimum Normalized-Cut

wherematrix, symmetric NNW

otherwise0

if,22

iNjeejiWXjiFji XXFF

Proximity Spatial

similarity feature Image

ji

ji

XX

FF

wherematrix, diagonal NND j

jiWiiD ,,

It can be shown that

such that

If y is allowed to take real values then the minimization can be done by solving the generalized eigenvalue system

Finding Minimum Normalized-Cut

Dyy

yWDyT

T

y

minmin cutN

0 and ,10 ,,1 D1yTbbiy

DyyWD

Algorithm Compute matrices W & D Solve for eigen vectors with the

smallest eigen values Use the eigen vector with second smallest eigen

value to bipartition the graph Recursively partition the segmented parts if

necessary.

DyyWD

Figure from “Image and video segmentation: the normalised cut framework”, by Shi and Malik, 1998

F igure from “Normalized cuts and image segmentation,” Shi and Malik, 2000

Drawbacks of Minimum Normalized Cut Huge Storage Requirement and time

complexity Bias towards partitioning into equal

segments Have problems with textured backgrounds

Suggested Reading Chapter 14, David A. Forsyth and Jean Ponce, “Computer

Vision: A Modern Approach”. Jianbo Shi, Jitendra Malik, “Normalized Cuts and Image

Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997

Chapter 3, Mubarak Shah, “Fundamentals of Computer Vision”