Geodesic Star Convexity for interactive image segmentation Varun Gulshan † , Carsten Rother ‡ , Antonio Criminisi ‡ , Andrew Blake ‡ and Andrew Zisserman † 1. Star- convexity † Visual Geometry Group, University of Oxford, UK ‡ Microsoft Research, Cambridge, UK References p c q Let y denote a binary segmentation and S * ({c}) the set of star convex shapes wrt to center c. Star-convex constraint expressed as an energy: Factorization into pairwise terms: 2. Star-convexity -- Extensions Semantics I – visibility to atleast one center (equivalently: union of star shape sets) Semantics II – visibility to nearest star center 3. Visibility Experiment Method 1 star 2 stars ESC 4.50±0.5 8 2.85±0.3 9 GSC 4.16±0.5 4 2.22±0.3 0 4. Star-convexity in an interactive system Boykov Jolly Energy: 5. Evaluation Simulat ed user [2] Dataset: 151 Images taken from GrabCut, PASCAL VOC and the alpha matting dataset. •Energy is submodular as (1,0) labeling has infinite energy. •Only needs to be imposed for neighbouring pixels – hence efficient. c •The union constraint is not submodular. •Semantics don’t extend easily to brush strokes as star centers • Tractable to implement (submodular) •Semantics extend nicely to brush strokes as star centers User interaction Color likelihood Output - BJ Output - GSC Evaluation criteria: Measure avg. number of strokes to reach desired accuracy. Metho d Avg. Effort BJ 12.35 PP 10.66 ESC 10.57 GSC 10.23 GSCse q 9.63 Metho d Avg. Effort SP 15.14 BJ 12.35 RW 12.31 GSCse q 9.63 [1] O. Veksler. Star shape prior for graph-cut based image segmentation. ECCV, 2008 [2] H. Nickisch, P. Kohli and C. Rother. Learning an interactive segmentation system. arXiv Technical Report, Dec. 2009. [3] L. Grady. Random walks for image segmentation. IEEE PAMI, 2006 [4] X. Bai and G. Sapiro. Geodesic matting: A framework for fast interactive image and video segmentation and matting. IJCV 2009 [5] S. Vicente, V. Kolmogorov and C. Rother. Graph cut based image segmentation with connectivity priors. CVPR 2008. Euclide an Geodesi c 2.1 Multiple Stars 2.2 Geodesic Stars GSC New Brush Stroke .... Comparison: Various shape constraints Comparison: Different algorithms Dij-GC [5] BJ 4.1 Sequential system Occlusion rates Veksler, ECCV 08 [1] Visualization of geodesics computed using image gradients Original image FG stroke BG stroke User interaction Original image FG stroke BG stroke SP = Shortest Paths [4] RW = Random Walker [3] PP = Post- Processing for connected components Code and dataset available at: http://www.robots.ox.ac.uk/~vgg/research/iseg/