Click here to load reader
Jan 16, 2016
Seam CarvingProject 1a due at midnight tonight.Project 1b goes out today.Well cover seam carving today.
Image resizingresamplingcroppingcontent-aware resizing
Seam carving: ideaCropping removes pixels from the image boundary.We want to remove only unimportant pixels.
How can we make sure the result is rectangular?
Seam carving: ideahttp://swieskowski.net/carve/
Something in between: remove seamsWhy not remove least important pixel from each row?
Pixel importanceMany possible measures of importanceOne simple one is gradient magnitudeor
Computing the optimal seamHow many vertical seams in an m-by-n image?
We can avoid trying all of them.Suppose this is the optimal seam:
What can we say about this one?
Computing the optimal seamM[x,y] = cost of minimum-energy vertical seam through rows 1 to yM[x,y]M[x+1,y-1]M[x,y-1]M[x-1,y-1]M[x,y] = min(M[x-1,y-1], M[x,y-1], M[x+1,y-1]) + E[x,y]
Computing the optimal seamFind pixel in bottom row with minimum seam cost.
Trace back optimal seam through image.
Remove seam pixels.M =
Seam carving algorithm
Compute energy at each pixelWhile image larger than m-by-n:Remove horizontal or vertical seam with minimum energy.
How do we choose between horizontal and vertical?
How might we enlarge an image?
Image SegmentationFrom Sandlot Science
From images to objectsWhat Defines an Object?Subjective problem, but has been well-studiedGestalt Laws seek to formalize thisproximity, similarity, continuation, closure, common fatesee notes by Steve Joordens, U. Toronto
Image histogramsHow many orange pixels are in this image?This type of question answered by looking at the histogramA histogram counts the number of occurrences of each colorGiven an image
The histogram isi.e., for each color value c (x-axis), plot # of pixels with that color (y-axis)
What do histograms look like?How Many Modes Are There?Easy to see, hard to compute
Histogram-based segmentationGoalBreak the image into K regions (segments)Solve this by reducing the number of colors to K and mapping each pixel to the closest color
Histogram-based segmentationGoalBreak the image into K regions (segments)Solve this by reducing the number of colors to K and mapping each pixel to the closest color Heres what it looks like if we use two colors
ClusteringHow to choose the representative colors?This is a clustering problem!ObjectiveEach point should be as close as possible to a cluster centerMinimize sum squared distance of each point to closest center
Break it down into subproblemsSuppose I tell you the cluster centers ciQ: how to determine which points to associate with each ci?A: for each point p, choose closest ciSuppose I tell you the points in each clusterQ: how to determine the cluster centers?A: choose ci to be the mean of all points in the cluster
K-means clusteringK-means clustering algorithmRandomly initialize the cluster centers, c1, ..., cKGiven cluster centers, determine points in each clusterFor each point p, find the closest ci. Put p into cluster iGiven points in each cluster, solve for ciSet ci to be the mean of points in cluster iIf ci have changed, repeat Step 2
Java demo: http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html
PropertiesWill always converge to some solutionCan be a local minimumdoes not always find the global minimum of objective function:
Probabilistic clusteringBasic questionswhats the probability that a point x is in cluster m?whats the shape of each cluster?K-means doesnt answer these questions
Basic ideainstead of treating the data as a bunch of points, assume that they are all generated by sampling a continuous functionThis function is called a generative model defined by a vector of parameters
Mixture of GaussiansOne generative model is a mixture of Gaussians (MOG)K Gaussian blobs with means b covariance matrices Vb, dimension dblob b defined by:
blob b is selected with probability the likelihood of observing x is a weighted mixture of Gaussians
Expectation maximization (EM)Goalfind blob parameters that maximize the likelihood function:
Approach:E step: given current guess of blobs, compute ownership of each pointM step: given ownership probabilities, update blobs to maximize likelihood functionrepeat until convergence
Grabcut [Rother et al., SIGGRAPH 2004]
What if we look at relationships between pixels?
qImages as graphsFully-connected graphnode for every pixellink between every pair of pixels, p,qcost cpq for each linkcpq measures similaritysimilarity is inversely proportional to difference in color and positionpCpqc
Segmentation by Graph CutsBreak Graph into SegmentsDelete links that cross between segmentsEasiest to break links that have low cost (low similarity)similar pixels should be in the same segmentsdissimilar pixels should be in different segments
Cuts in a graphLink Cutset of links whose removal makes a graph disconnectedcost of a cut:ABFind minimum cutgives you a segmentationfast algorithms exist for doing this
But min cut is not always the best cut...
Cuts in a graphABNormalized Cuta cut penalizes large segmentsfix by normalizing for size of segments
volume(A) = sum of costs of all edges that touch A
Interpretation as a Dynamical SystemTreat the links as springs and shake the systemelasticity proportional to costvibration modes correspond to segmentscan compute these by solving an eigenvector problemfor more details, see J. Shi and J. Malik, Normalized Cuts and Image Segmentation, CVPR, 1997
Interpretation as a Dynamical System
Color Image Segmentation
Intelligent Scissors (demo)
Around n * 3^m seams (slightly fewer due to boundaries).The shorter seam is the optimal seam ending in that pixel**