Top Banner
Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006
32

Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Dec 13, 2015

Download

Documents

Junior Lester
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Lecture 19: Solving the Correspondence Problem with Graph Cuts

CAP 5415Fall 2006

Page 2: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Announcements

PS4 Only need to search over a limited number of

disparities

Schedule Changes Project Proposals due next Monday Class canceled on Wednesday Will have class on November 22

Page 3: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Stereo Once we have rectified images, the hard problem is

find the corresponding points Then we can triangulate Known as the correspondence problem When people say “stereo algorithm”, they usually

mean “correspondence algorithm”

Page 4: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

A very simple algorithm

To find the point that matches this point

Page 5: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

A very simple algorithm

Look at a square window around that point

and compare it to windows in the other image

Could measure sum of squared differences (SSD) or correlation

Page 6: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

What's wrong with this algorithm?

This set of correspondences don't bother it

Areas of the image without texture will be a problem for this algorithm.

Page 7: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

To do better we need a better model of images

We can make reasonable assumptions about the surfaces in the world

Usually assume that the surfaces are smooth Can pose the problem of finding the

corresponding points as an energy (or cost) minimization

The data term measures how well the local windows match up for different disparities

Page 8: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

The smoothness cost

The smoothness is usually implemented by penalizing differences in the disparity

Different penalty functions lead to different assumptions about the surfaces log(1+x^2) x^2 First deriv, Second derivative

Page 9: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Today

Assuming that there a discrete set of possible disparities

How do we optimize this?

Page 10: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Today

I'll be talking about the Graph Cuts algorithm Very popular Used heavily in vision and graphics Paper: “Fast Approximate Energy Minimization Notation for the rest of the lecture

Page 11: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Simple Way to Optimize - ICM

Choose a pixel Fix the disparities at the rest of the pixels Set the pixel to the optimal disparity Iterate

Page 12: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

ICM

Advantages Energy always guaranteed to decrease Easy to code

Disadvantages Convergence Can you think of the big on?

Page 13: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Making Bigger Moves

The problem with ICM is that you can only modify one pixel at a time

Get stuck in local minima easily We need a way of moving multiple pixels at

once Boykov, Veksler and Zabih introduced two

types of moves: alpha-beta swap alpha-expansion

Page 14: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Types of Moves

ICM – One pixel moves

(From BKZ-PAMI 01)

Page 15: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

alpha-beta swap

Fix all nodes that aren't labeled alpha or beta With remaining nodes, find optimal swap

Some alpha nodes change to beta Vice Versa Some stay the same (From BKZ-PAMI 01)

Page 16: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

alpha-expansion

Any node can change to alpha

(From BKZ-PAMI 01)

Page 17: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Basic Algorithm

ICM: Compute one-pixel moves until convergence

Graph Cuts: Compute swaps (or expansions) until convergence

Since the optimal swap is being computed each time, energy always decreases

Page 18: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Important Question

How do we find the swaps? Use min-cut from graphs Slightly different from the min-cut problem that

we discussed in the context of normalized cuts

Terminal NodeTerminal Node

Page 19: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Minimum Cut A cut is a set of edges that we will remove so

that the terminal nodes are separated If a cost is assigned to each edge, the cost of

the cut is the weight assigned to each edge Minimum Cut can be found in polynomial time

(Ford and Fulkerson)

Terminal NodeTerminal Node

Page 20: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Important Rule No proper subset of the cut can also be a cut This is not a minimum cut

Terminal NodeTerminal Node

Page 21: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Using the minimum-cut to find a swap

Consider a 1-D image

Terminal Node

Terminal Node

(From BKZ-PAMI 01)

Page 22: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Can guarantee that every node will have at least one t-link

(From BKZ-PAMI 01)

Page 23: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

How to see that

(From BKZ-PAMI 01)

Page 24: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Actually Computing the Min-Cut Could use code to solve general min-cut

problems Boykov and Kolmogorov have released an

algorithm optimized for vision problems

Page 25: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Expansion Move

The difference between the optimal solution and the solution from the expansion move is bounded

Page 26: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

What functions can be minimized using graph cuts

V is called a metric if it obeys all 3 Can use Expansion Moves

V is called a semimetric if it obeys the last two Can only use Swap-Moves

Subject of recent research.

Page 27: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Graph-Cuts has been shown to perform very well

Page 28: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

From “A Comparative Study of Energy Minimization Methods for

Markov Random Fields”

Page 29: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

From “A Comparative Study of Energy Minimization Methods for

Markov Random Fields”

Page 30: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Used for much more than just stereo

Page 31: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.
Page 32: Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.