Spatial Random Partition for Common Visual Pattern Discovery

CVPR 2006 New York City

Spatial Random Partition for Common Visual Pattern Discovery

Junsong Yuan and Ying Wu EECS Dept. Northwestern Univ.

{j-yuan,yingwu}@northwestern.edu

ICCV 2007

ICCV 2007 Rio de Janeiro, Brazil 2

The Problem

Can you find common posters in the two images?

ICCV 2007 Rio de Janeiro, Brazil 3

Challenges

No prior knowledge of the common patterns– What are they ? appearances– Where are they ? locations– How large are they ? scales– How many of them ? number of

instances

Computationally demanding– Exponentially large solution space– Large image dataset

Robust similarity matching

ICCV 2007 Rio de Janeiro, Brazil 4

Related Work Pattern Discovery by Matching Visual Words

– J. Sivic and A. Zisserman, CVPR04– J. Yuan, Y. Wu and M.Yang, CVPR07– S. Nowozin, K. Tsuda, T. Uno, T. Kudo and G. Bakir,

CVPR07– T. Quack, V. Ferrari, B. Leibe and L. V. Gool, ICCV07– …

Pattern Discovery by Direct Matching – O. Boiman and M. Irani, ICCV05, NIPS06– K. Grauman and T. Darrell, CVPR06, NIPS07– K.-K. Tan and C.-W. Ngo, ICCV05 – N. Ahuja and S. Todorovic, CVPR06, ICCV07– …

ICCV 2007 Rio de Janeiro, Brazil 5

Spatial Random Partition

DI

DR

SR

Votingmaps

CommonPatterns

RandomPartition

Matching

Voting

Localization

ICCV 2007 Rio de Janeiro, Brazil 6

Visual Primitives

Visual Primitives: Scale Invariant Feature Transformation (SIFT, D.Lowe,

IJCV04 )

Locality Sensitive Hashing (LSH) for matching visual primitives– For each visual primitive, search for its

matches from other images, based on Euclidean distance

ICCV 2007 Rio de Janeiro, Brazil 7

Matching Subimages A many-to-many assignment problem.

Fast approximation by set intersection:

where

is the # of visual primitives in the subimage

ICCV 2007 Rio de Janeiro, Brazil 8

Another View: Max Flow

VR MVQ

VR VQ..

..

.

... .. .

..

....

. ... . ..

Visual primitives

subimage

Problem: Matching two sets of m and n points (feature vectors)

Fast Approximate Solution: set intersection (linear complexity)

ICCV 2007 Rio de Janeiro, Brazil 9

An Example Final estimation of similarity score:

= 3

T K

A EB DC F

S

P

P Z

H

Z X

I R

QG

Y

J

O

SX

Z

W

L

M

Y

VR

VQ

MVQ

N

Z

X

Y=

LSH for Fast Query

N

ICCV 2007 Rio de Janeiro, Brazil 10

Voting for Common Patterns

. . .

+

W1W2 WK

VotingMap

ICCV 2007 Rio de Janeiro, Brazil 11

Asymptotic Property Theorem: Given two pixel i and j, where

i locates in a common pattern while j locates in the background, let and the total votes i and j receives regarding to K random partitions. Both and are discrete random variables and we have

Proof: using the weak law of large numbers, see the Appendix for details

ICCV 2007 Rio de Janeiro, Brazil 12

Localization of Common Patterns

ICCV 2007 Rio de Janeiro, Brazil 13

Various Number of Partitions

ICCV 2007 Rio de Janeiro, Brazil 14

ICCV 2007 Rio de Janeiro, Brazil 15

Image Irregularity Detection

Differences from common pattern discovery– disocver unpopular subimages instead of popular

ones– Adjust voting weight proportional to the subimage

size: the larger the unpopular subimage, the more possible it contains an irregular pattern

ICCV 2007 Rio de Janeiro, Brazil 16

Evaluation Collect 8 image datasets, each contains 4-8 images. An image

dataset contains 1-3 common patterns each has 2-4 instances (* indicates the dataset containing multiple common patterns )

Comparisons of computational complexity, around 12 sec. for 2 images

J. Sivic & A. Zisserman

04O. Boiman & M. Irani

05

ICCV 2007 Rio de Janeiro, Brazil 17

Conclusion A novel spatial random partition

method for common pattern discovery and irregularity detection in images– No construction of visual vocabularies– Trade-off of performance and efficiency

by the total number of random partitions– Efficient by using LSH and approximate

matching between subimages– Theoretically justified by the asymptotic

property of the algorithm