1 Michael Bronstein Heat diffusion descriptors deformable Michael Bronstein Weizmann Institute of Science, 4 November 2010 Institute of Computational Science.

1Michael Bronstein Heat diffusion descriptors

deformable

Michael Bronstein

Weizmann Institute of Science, 4 November 2010

Institute of Computational ScienceUniversita della Svizzera Italiana

Lugano, Switzerland

shapesfor

Heat diffusion descriptors


Dan Raviv

Technion

Ron Kimmel

Technion

Maks Ovsjanikov

Stanford

Leo Guibas

Stanford

Iasonas Kokkinos

ECP Paris

Alex Bronstein

TAU


The next challenge

Text Visual data Geometric data


Shape retrieval today


Bags of words

Notre Dame de Paris is a Gothic cathedral in the fourth quarter of Paris, France. It was the first Gothic architecture cathedral, and its construction spanned the Gothic period.

con

stru

ctio

nar

chit

ectu

reIt

aly

Fra

nce

cath

edra

lch

urc

hb

asili

caP

aris

Ro

me

Go

thic

Ro

man

St. Peter’s basilica is the largest church in world, located in Rome, Italy. As a work of architecture, it is regarded as the best building of its age in Italy.

Notre Dame de Paris is a Gothic cathedral in the fourth quarter of Paris, France. It was the first Gothic architecture cathedral, and its construction spanned the Gothic period.

St. Peter’s basilica is the largest church in world, located in Rome, Italy. As a work of architecture, it is regarded as the best building of its age in Italy.


Outline

Feature descriptor

Geometric words

Bag of geometric words

Geometric expressions

Spatially-sensitive bag of words

“ ”

“ ”

Volumetric descriptors

Scale invariance


Curvature

Rigid Bending Topology

Integralvolume1

Scale

Spin image2

Shape context3

Representation

HKS4

SI-HKS5

vHKS6

1 Gelfand et al. 2005; 2 Johnson, Hebert 1999; 3 Belongie et al. 2002; 4 Sun et al. 2009

Shape descriptors

Any

Volume/Mesh

Any

Any

Any

Any

Volume/Mesh

5 B, Kokkinos 2010; 6 Raviv, BBK 2010


Diffusion geometry

Amount of heat transferred from point x to point y in time t

Heat equation

where

- positive semidefinite Laplace-Beltrami

operator

- heat distribution

Fundamental solution (heat kernel, ) – heat equation solution

for initial conditions

Spectral expression


Sun, Ovsjanikov, Guibas, 2009

Heat kernel interpretation

Geometric interpretation: “multiscale Gaussian curvature”

Probabilistic interpretation: the probability of a random walk to remain

at point x after time t.


Sun, Ovsjanikov, Guibas, 2009

Heat kernel signature

Multiscale descriptor

Time (scale)

■ Intrinsic, hence deformation-invariant

■ Provably informative

■ Efficiently computable on different shape representations

■Multiscale


Ovsjanikov, BB, Guibas, 2009BB. Ovsjanikov, Guibas 2010

Shape

Geometric vocabulary



Index in geometric vocabulary1 64

Ovsjanikov, BB, Guibas, 2009BB. Ovsjanikov, Guibas 2010

Bags of geometric words


B et al. 2010

SHREC 2010: Robust shape retrieval benchmark

Transformation

Query set

Database (>1K shapes)


B et al. 2010

Query Toldo et al. 2009 Shape


Bags of words using HKS descriptor,vocabulary of size 48

Shape

Bags of words using spin imagedescriptor

Performance results

Toldo et al. 2009

Toldo et al. 2009B et al. 2010

Performance criterion: mean average precision (mAP) in %


Scale invariance

Original shape Scaled by

Not scale invariant!


Scale-invariant HKS

B, Kokkinos CVPR 2010

Log scale-space

Scaling = shift and multiplicative constant

log + d/d

Undo scaling

Fourier transformmagnitude

Undo shift

0 100 200 300-15

-10

-5

0

t0 100 200 300

-0.04

-0.03

-0.02

-0.01

0

t0 2 4 6 8 10 12 14 16 18 20

0

1

2

3

4

=2k/T


Scale invariant HKS

B, Kokkinos 2010

HKS SI-HKS


B, Kokkinos 2010

Query HKS SI-HKS


B, Kokkinos 2010

HKS, vocabulary of size 48 SI-HKS, vocabulary of size 48

HKS vs SI-HKS



Expressions

In math science, matrix decomposition is a factorization of a matrix into some canonical form. Each type of decomposition is used in a particular problem.

In biological science, decomposition is the process of organisms to break down into simpler form of matter. Usually, decomposition occurs after death.

Matrix is a science fiction movie released in 1999. Matrix refers to a simulated reality created by machines in order to subdue the human population.

mat

rix

dec

om

po

siti

on

mat

rix

fact

ori

zati

on

scie

nce

fic

tio

nca

no

nic

al f

orm


In biological science, decomposition is the process of organisms to break down into simpler form of matter. Usually, decomposition occurs after death.


mat

rix

dec

om

po

siti

on is a

the of in to by

scie

nce

form



Ovsjanikov, BB & Guibas 2009


Expressions


mat

rix

dec

om

po

siti

on is a

the of in to by

scie

nce

form

In particular matrix used type a some science, decomposition form a factorization of is canonical. matrix math decomposition is in a Each problem. into of

mat

rix

dec

om

po

siti

on

mat

rix

fact

ori

zati

on

scie

nce

fic

tio

nca

no

nic

al f

orm





“Yellow Yellow”Yellow

No total order between points (only “far” and “near”)

Geometric expression = a pair of spatially close geometric words



Spatially-sensitive bags of words


B et al. 2010

HKS, vocabulary of size 48 Spatially-sensitive HKS, vocabulary of size 8x8

HKS vs SI-HKS



Is our shape model good?

Raviv, BBK 2010

Boundary ∂X

Interior

X


Is our shape model good?

Camel illustration from Sumner et al.Raviv, BBK 2010

Volume isometryBoundary isometry

Preserves geodesic distances on

the boundary surface

Preserves geodesic distances

inside the volume


where

Diffusion equation

Raviv, BBK 2010

Volumetric diffusionBoundary diffusion

- Laplace-Beltrami operator - Euclidean Laplacian

- normal to boundary surface


where

Heat kernels

Raviv, BBK 2010

Volumetric heat kernelBoundary heat kernel

Geometric interpretation

“Multiscale Gaussian curvature”


Heat kernel signatures

Raviv, BBK 2010

vHKSHKS

Boundary+volume isometry Boundary+volume isometry

Boundary isometry Boundary isometry


Raviv, BBK 2010

vHKSHKS


HKS, vocabulary of size 48 vHKS, vocabulary of size 48

HKS vs vHKS

Raviv, BBK 2010



Summary

Feature descriptor

Geometric words



Spatially-sensitive bag of words

“ ”

“ ”

Volumetric descriptors

Scale invariance


Thank you

1 Michael Bronstein Heat diffusion descriptors deformable Michael Bronstein Weizmann Institute of Science, 4 November 2010 Institute of Computational Science.

Documents

shape descriptors

heat equation solution

time t heat equation

gothic cathedral

gothic architecture

gothic period

quarter of paris

work of architecture