Interactive Image Segmentation using Graph Cuts Mayuresh Kulkarni and Fred Nicolls Digital Image Processing Group University of Cape Town PRASA 2009.

Interactive Image Segmentation using Graph Cuts

Mayuresh Kulkarni and Fred NicollsDigital Image Processing Group

University of Cape Town

PRASA 2009

Outline

• Image Segmentation Problem• Our Approach• Graph cuts and Gaussian Mixture Models• Results and Discussion• Future Research

What is foreground?

Image Segmentation

Our Approach

Graph Cuts SegmentationCost Function : E(A) = λ R(A) + B(A)

Region information Boundary information Pixel connectivity

8 – pixel neighbourhoodDifference between adjacent pixels

Image propertieseg. colour, texture

Graph Cuts

Source (foreground)

Sink (background)

Cost Function : E(A) = λ R(A) + B(A)

Pixel connectivity (boundaries)Inter-pixel weights (boundaries)

Source and Sink weights (regions)

Gaussian Mixture Models

Background GMM Foreground GMM

Gaussian Mixture ModelsForeground

GMM

Background GMM

Log Likelihood Ratio = log(K *pf/pb)

pf

pb

GMM components

• Greyscale images– Intensity values– Intensity values and

MR8 filters

• Colour images– RGB values– G, (G-R), (G-B) values– Luv values– MR8 filters

Boundary information

• Inter-pixel weights– Edge detection– Difference between

adjacent pixels– Gradient

• Pixel connectivity

Results

Κ = 0.01 Κ = 0.1 Κ = 1

Results

Original Image

RGB, Luv and MR8 (Fscore = 0.916)

Luv and MR8 (Fscore = 0.921)

Luv (Fscore = 0.934)

Results

Original Image RGB, Luv and MR8 (Fscore = 0.906)

RGB (Fscore = 0.951)Luv (Fscore = 0.945)

Analysis of Results

• Accurate segmentation achieved• Components in the GMM depend on image• Segmentation can be controlled using K and λ

Future Research

• Different grid (non-pixel grid)• Ratio cuts• Exploring other statistical models• ObjCut – segmenting particular objects

References• Y. Boykov and M. P. Jolly. Interactive graph cuts for optimal boundary and region

segmentation of objects in N-D images. In ICCV, volume 1, pages 105–112, July 2001.• Yuri Boykov and Vladimir Kolmogorov. An experimental comparison of min-cut/max-flow

algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell., 26(9):1124–1137, 2004.

• Pushmeet Kohli, Jonathan Rihan, Matthieu Bray, and Philip H. S. Torr. Simultaneous segmentation and pose estimation of humans using dynamic graph cuts. International Journal of Computer Vision, 79(3):285–298, 2008.

• H. Permuter, J. Francos, and I. Jermyn. Gaussian mixture models of texture and colour for image database. In ICASSP, pages 25–88, 2003.

• D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. 8th Int’l Conf. Computer Vision, volume 2, pages 416–423, July 2001.

• Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. “GrabCut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph., 23(3):309–314, August 2004.