Edge Preserving Spatially Varying Mixtures for Image Segmentation Giorgos Sfikas, Christophoros Nikou, Nikolaos Galatsanos (CVPR 2008) Presented by Lihan He ECE, Duke University Feb 23, 2009 by
Feb 02, 2016
Edge Preserving Spatially Varying Mixturesfor Image Segmentation
Giorgos Sfikas, Christophoros Nikou, Nikolaos Galatsanos
(CVPR 2008)
Presented by Lihan He
ECE, Duke University
Feb 23, 2009
by
Introduction
Edge preserving spatially varying GMM
Inference using MAP-EM
Experimental results
Conclusion
Outline
2/15
Introduction
3/15
Image segmentation
GMM: no prior knowledge is exploited
Adjacent pixels most likely belong to the same cluster;
Edge of objectives.
SVGMM (spatially variant GMM):
Clustering pixels or super pixels such that the same group has common characteristics (same objective, similar texture)
Spatial smoothness is imposed in the neighborhood of each pixel based on the Markov random field;
Without considering the edge of textures
Introduction
4/15
In this paper
Hierarchical Bayesian model;
Spatially varying GMM: mixing weights are different for different pixels;
Difference of mixing weights for two neighbored pixels follows a student-t distribution;
Heavy tailed student-t preserves edges of textures;
MAP-EM is used for model inference.
St-SVGMM
Nnxn ,...,1, Feature vector for each pixel:
SVGMM:
Each pixel has its own mixing weights
Each pixel xn: },,...,,{ 21nK
nn
},,...,,{ 21nK
nn zzz
weights
indicator variables
Likelihood:
Prior:
5/15
K
j
nj
nj zz
11 },1,0{
St-SVGMM
Prior for mixing weight π:
nx
d=2
d=1
d: neighborhood adjacency type
d=1: horizontald=2: vertical
γd(n): the set of neighbors of pixel n, with respect to the dth adjacency type
K×D different student-t distributions are introduced, with hyperparameters
Joint prior for π:
6/15
St-SVGMM
The student-t distribution can be modeled by introducing the latent variable
)(,:1,:1,:1}{ nkDdKjNnnkj d
uU
nkju plays an important role:
,nkju ,k
jnj neighboring pixels n, k belong to the same cluster
,0nkju ,k
jnj n, k are at the edge of two clusters
n – edge location k (d) – adjacency type (horizontal or vertical)j – cluster index (edges of which cluster)
7/15
St-SVGMM
),;(~,,| jjnnn xNzx
Model summary
)(~ nn lmultinomiaz
8/15
Inference
MAP-EM algorithm for model inference.
Complete log-likelihood
Model parameters:
E-step (update Z, U)
9/15
Inference
M-step ( update )
=
,,,,
10/15
Results
U-variable maps
nkju
j=1: sky
j=2: roof & shadows
j=3: building
d=1: horizontal d=2: vertical
n – edge location k (d) – adjacency typej – cluster index
K=3 clusters
Brighter regions represent lower values – edges.
11/15
Results
Comparison on 300 images of the Berkeley image database
Statistics on the Rand Index (RI) (measuring the consistency between the ground truth and the segmentation map); higher is better.
Statistics on the boundary displacement error (BDE) (measuring error of boundary displacement with respect to the ground truth); lower is better.
12/15
Results
Segmentation examples
K=5 K=15K=10original image
13/15
Results
K=5 K=15K=10original image 14/15
Conclusion
15/15
Proposed a GMM-based clustering algorithm for image segmentation;
Used smoothness prior to consider the adjacent pixels belonging to the same cluster;
Also captured the image edge structure (no smoothness enforced across segment boundaries);
All required parameters are estimated from the data (no requirement of empirical parameter selection).
Next: automatically estimating the number of components K.