Computer Vision October 2003 L1.1 © 2003 by Davi Geiger Image Segmentation Based on the work of Shi and Malik, Carnegie Mellon and Berkley and based on the presentation of Jianbo Shi
© 2003 by Davi GeigerComputer Vision October 2003 L1.1 Image Segmentation Based on the work of Shi and Malik, Carnegie Mellon and Berkley and based on.
Dec 19, 2015
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Computer Vision October 2003 L1.1© 2003 by Davi Geiger
Image Segmentation
Based on the work of Shi and Malik, Carnegie Mellon and Berkleyand based on the presentation of Jianbo Shi
Computer Vision October 2003 L1.2© 2003 by Davi Geiger
• Edge detection by gradient operators
• Linking by dynamic programming, voting, relaxation, …- Natural for encoding curvilinear grouping- Hard decisions often made prematurely
Edge-based image segmentation
Computer Vision October 2003 L1.3© 2003 by Davi Geiger
Grouping with Bayesian Statistics
Bayes data structure = data generation model + segmentation model
X1 X2
Image asobservation f
Grouping asstate X
f1 f2
)](log)|(log[min);(min XpXfpfXE XX Segmentation is to find a partitioning of an image, with generative models explaining each partition.
Generative models constrain the observation data, f, and the prior model constrains the discrete states, X.
The solution sought is the most probable state, or the state of the lowest energy.
Texture models
Computer Vision October 2003 L1.4© 2003 by Davi Geiger
Image segmentation by pairwise similarities
• Image = { pixels }• Segmentation = partition of image into
segments• Similarity between pixels i and j
Sij = Sji ≥ 0
• Objective: “similar pixels, with large value of Sij, should be in the same segment, dissimilar pixels should be in different segments”
Sij
Computer Vision October 2003 L1.5© 2003 by Davi Geiger
Relational Graphs
G=(V, E, S) V: each node denotes a pixel E: each edge denotes a pixel-pixel relationship S: each edge weight measures pairwise similarity
Segmentation = node partitioning break V into disjoint sets V1 , V2
Computer Vision October 2003 L1.6© 2003 by Davi Geiger
Solving MRF by Graph Partitioning
p
pppp pNq
qpqp fXUXXWfXE ),(),();(min)(
,
Some simple MRF models can be translated into graph partitioning
pair relationships data measures
L1 L2
Computer Vision October 2003 L1.7© 2003 by Davi Geiger
Weighted graph partitioning
Pixels i I = vertices of graph G
Edges ij = pixel pairs with Sij > 0
Similarity matrix S = [ Sij ]
di = j Є G Sij degree of I
deg A = i Є A di degree of A G
Assoc(A,B) = i Є A j Є B Sij
Sij
ij
i
A
AB
Computer Vision October 2003 L1.8© 2003 by Davi Geiger
Cuts in a Graph
• (edge) cut = set of edges whose removal makes a graph disconnected
• weight of a cut: cut( A, B ) = i Є A, j Є B Sij =Assoc(A,B)
• the normalized cut
• Normalized Cut criteria: minimum cut(A,Ā)
NCut( A,B ) = cut(A, B)( + )
1 deg A
1
deg B
),(
1
jiij xxd
S
Computer Vision October 2003 L1.9© 2003 by Davi Geiger
Grouping with Spectral Graph Partitioning
SGP: data structure = a weighted graph, weights describing data affinity
Segmentation is to find a node partitioning of a relational graph, with minimum total cut-off affinity.
Discriminative models are used to evaluate the weights between nodes.
The solution sought is the cuts of the minimum energy.
)deg(
),(
)deg(
),(),(min
B
BAcut
A
BAcutBANcut
Ai Bj
jiSBAcut ),(),(
Ai Gj
jiSA ),()deg(
NP-Hard!
Computer Vision October 2003 L1.10© 2003 by Davi Geiger
Normalized Cut and Normalized Association
• Minimizing similarity between the groups, and maximizing similarity within the groups are achieved simultaneously.
)deg(
),(
)deg(
),(),(
B
BBAssoc
A
AAAssocBANassoc
)deg(
),(
)deg(
),(),(
B
BAcut
A
BAcutBANcut
),(),()deg(
),(2),(
AAAssocBAAssocAas
BANassocBANcut
Computer Vision October 2003 L1.11© 2003 by Davi Geiger
Some definitions
.1)(,}1,1{ in vector a be Let
);,(),( matrix, diag. thebe DLet
;),( matrix, similarity thebe Let ,
Aiixx
jiSiiD
SjiSS
N
j
ji
• Rewriting Normalized Cut in matrix form:
...
),(
),( ;
11)1(
)1)(()1(
11
)1)(()1(
)Bdeg(
B)A,(
)Adeg(
B)A,(B)A,(
0
i
x
T
T
T
T
iiD
iiDk
Dk
xSDx
Dk
xSDx
cutcutNcut
i
Computer Vision October 2003 L1.12© 2003 by Davi Geiger
Generalized Eigenvalue problem
• after simplification, we get
.01},,1{ with ,)(
),(
DybyDyy
ySDyBANcut T
iT
T
y2i i
A
y2i
i
A
DxxSD )(
Computer Vision October 2003 L1.13© 2003 by Davi Geiger
Computer Vision October 2003 L1.14© 2003 by Davi Geiger
Brightness Image Segmentation
Computer Vision October 2003 L1.15© 2003 by Davi Geiger
Brightness Image Segmentation
Computer Vision October 2003 L1.16© 2003 by Davi Geiger
Computer Vision October 2003 L1.17© 2003 by Davi Geiger
Results on color segmentation
Computer Vision October 2003 L1.18© 2003 by Davi Geiger
Motion Segmentation with Normalized Cuts
• Networks of spatial-temporal connections:
Motion “proto-volume” in space-time
Computer Vision October 2003 L1.19© 2003 by Davi Geiger
Related Documents
