CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 1 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection.

CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II

1

Image Understanding 2Outline:

Guzman Scene AnalysisLocal and Global ConsistencyEdge DetectionLaplacians and Zero CrossingsSegmentation into RegionsImage Connected ComponentsGestalt GroupingMorphology


2




3

Blocks-World Scene Analysis(Adolfo Guzman, MIT AI Lab, 1967)• Input: a line drawing representing a 3D

scene consisting of polyhedra• Output: lists of faces grouped into

objects• Method: (a) classify vertices, (b) link

faces according to rules, (c) extract connected components of doubly linked regions.


4


5

Guzman Vertex Categories

ell, arrow, fork, tee, kay


6

Guzman Linking Rules

ell, arrow, fork, tee, kay


7

Guzman Face Grouping Criterion:Two adjacent faces must be doubly linked to be considered part of the same object.

E

F

DG

CB

A

Obj1: (A B C)

Obj2: (D E F G)


8




9

Impossible Figures

Locally consistent, globally inconsistent


10




11

Edge Detection

Why? Find boundaries of objects in order to compute their features: shape, area, moments, corner locations, etc.

Methods:Compute measures of change in the neighborhood of each pixel.Trace boundariesTransform each neighborhood into a vector space whose basis includes "edge" vectors.


12

Horizontal differences: a – b

Both horiz. & vertical diffs.

Roberts' cross operator (a-d)2 +(b-c)2

Local Edge Detectors

a b

a b

c d


13

Express each 3x3 neighborhood as a vector.

N = <a, b, c, d, e, f, g, h>

Frei-Chen Edge Detection

e

h

c d

f g

aa b


14

Frei-Chen Edge Det. (cont)Transform it linearly into the 9D vector space spanned by the following basis vectors (each shown in 3x3 format)


15




16

Instead of using 1st derivatives, let's consider 2nd derivatives.

The Laplacian effectively finds subtle changes in intensity.

2 f(x,y) = 2 f(x,y)/x2 + 2f(x,y)/y2

discrete approximation:

Edge Finding with the Laplacian

-1

0

-1 4

0 -1

00 -1


17

Remedy: First, smooth the signal with a Gaussian filter.

g(x,y) = 1/(22) exp(-(x2+y2)/22 )

Then take the Laplacian.

Or, directly convolve directly with a Laplacian of Gaussian (LOG).

Laplacian: Very Sensitive to Noise


18

The edges tend to be where the second derivative crosses 0.

Zero Crossings


19

Zero Crossings: Example

(a) orig. (b) LoG with = 1, (c) zero crossings. R Fischer et al. http://homepages.inf.ed.ac.uk/rbf/HIPR2/zeros.htm


20




21

If two neighboring pixels share a side, they are 4-adjacent.

d's 4-adjacent neighbors are: b, c, e, g

If two neighboring pixels share a corner, they are 8-adjacent

d's 8-adjacent neighbors are:

a,b,c,e,f,g,h

4- and 8-adjacency

e

h

c d

f g

aa b


22

A set of pixels R is said to be 4-connected, provided for any pi, pj in R, there is a chain [(pi, pi' ), (pi' , pi'' ), ..., (pk, pj)] involving only pixels in R, and each pair in the chain is 4-adjacent. The chain may be of length 0.

R is 8-connected, iff there exists such a chain where each pair is 8-adjacent.

4- and 8-connectedness


23

4- and 8-connected components

4-connected components of black: 48-connected components of black: 24-connected components of white: 28-connected components of white: 2


24

"Dual approach" to edge detection.

Let P be a digital image: P = {p0, p1, ..., pn-1}

A segmentation S = {R0, R1, ..., Rm-1} is a partition of P, such that

For each i, Ri is 4-connected, and Ri is "uniform",

and for all i,j if Ri is adjacent to Rj,

then (Ri U Rj) is not uniform.

Segmentation into Regions


25




26

Can compute a segmentation if U(R) means all pixels in R have the same pixel value (e.g., intensity, color, etc.)

Algorithm: Initialize a UNION-FIND ADT instance, with each pixel in its own subset.Scan the image. At each pixel p, check its neighbor r to the right. p and r have the same pixel value, determine whether FIND(p) = FIND(r), and if not, perform UNION(FIND(p), FIND(r)). Then check the pixel q below doing the same tests and possible UNION.

When the scan is complete, each subset represents one 4-connected component of the image.

Image Connected Components


27




28

Gestalt Grouping

Identification of regions using a uniformity predicate based on wider-neighborhood features (e.g., texture).


29

Gestalt Grouping


30

Gestalt Grouping

Texture element = “texel”

Texel directionality

Texel granularity

Alignments of endpoints

Spacing of texels

Groups cue for surfaces, objects.


31




32

"Mathematical Morphology"

Developed at the Ecoles des Mines, France by J. Serra, from mathematical ideas of Minkowski.

Idea is to take two sets of points in space and obtain a new set.

Set A: the main set.Set B: the "structuring element"Set C: the result.


33

Math. Morph. Operation 1: "Erosion"

Consider each point b of B to be a vector and use it to translate a copy of A getting Ab

Take the intersection of all the Ab

The result is called the erosion of A by B.


34

Erosion: example

A: 0 1 1 0 0

1 1 1 0 0

0 1 1 1 0

0 0 1 0 0

B: 1 1

C: 0 0 1 0 0

0 1 1 0 0

0 0 1 1 0

0 0 0 0 0


35

Dilation: Take Union instead of Intersection.

A: 0 1 1 0 0

1 1 1 0 0

0 1 1 1 0

0 0 1 0 0

B: 1 1

C: 0 1 1 1 0

1 1 1 1 0

0 1 1 1 1

0 0 1 1 0


36

Opening: Erode by B, then Dilate by –B (B rotated by )

A: 0 1 1 0 0

1 1 1 0 0

0 1 1 1 0

0 0 1 0 0

B: 1 1

C: 0 1 1 0 0

1 1 1 0 0

0 1 1 1 0

0 0 0 0 0


37

Closing: Dilate by B, then Erode by –B (B rotated by )

A: 0 1 1 0 0

1 1 1 0 0

0 1 1 1 0

0 0 1 0 0

B: 1 1

C: 0 1 1 0 0

1 1 1 0 0

0 1 1 1 0

0 0 1 0 0

(In this example, erosion undoes the dilation.)


38

Closing – another example

A: 0 1 1 0 0

1 0 1 0 0

0 1 0 1 0

0 0 1 0 0

B: 1 1

C: 0 1 1 0 0

1 1 1 0 0

0 1 1 1 0

0 0 1 0 0

(In this example, closing fills in holes.)


39

Math. Morphol. UsesRepair regions corrupted by noise;

Eliminate small regions

Identify holes, bays, protrusions, isthmi, peninsulas, and specific shapes such as crosses.

Printed circuit board inspection (find cracks, spurious wires.)

Popular primitives for writing industrial machine vision apps.

CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 1 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection.

Documents

image understanding

regions image

c extract

morphology slide

kay slide

b c obj2

inconsistent slide

guzman scene analysis