Shape Representation Soma Biswas Department of Electrical Engineering, Indian Institute of Science, Bangalore.
Shape Representation
Soma Biswas
Department of Electrical Engineering,
Indian Institute of Science, Bangalore.
Shape-Based Recognition
2
Analysis of anatomical structures Figure from Grimson & Golland
Pose
Recognition, detection Fig from Opelt et al.
Applications
3
Geometric Transformations
4
Related Problems
Shape representation and decomposition
Finding a set of correspondences between shapes
Transforming one shape into another
Measuring the similarity between shapes
Shape localization and model alignment
Finding a shape similar to a model in a cluttered image
5
Comparing Images Using the Hausdorff Distance
6 D. Huttenlocher, G. Klanderman, and W. Rucklidge, 1993
7
8
Shape Contexts
9
Shape Matching and Object Recognition Using Shape Contexts, S. Belongie, J. Malik, and J.
Puzicha, 2002
Approach for measuring similarity between shapes and apply it for object recognition
Solve for correspondences between points on the two shapes
● Using shape contexts – describe coarse distribution of the rest of the shape
w.r.t. a given point on the shape
Use the correspondences to estimate an aligning transform
● Using regularized thin-plate splines
Compute the distance between the two shapes
10
- Not required to be landmarks/curvature
extrema, etc
- More samples -> better approximation of
underlying shape
11
SC: extremely rich descriptors Finding correspondence between 2 shapes = for each
sample pt on one shape, find sample pt on the other
shape with most similar SC
Maximizing similarities and enforcing uniqueness ->
bipartite graph matching problem / optimal assignment
Can add local appearance similarity at
the 2 points (gray scale images)
Choice is application dependent
For robust handling of outliers, add
dummy nodes to each pt set
When there is no real match, a pt will be
matched to the dummy
Invariance and Robustness
Matching approach should be
1) invariant under scaling and translation
2) robust under small geometrical distortions,
occlusion & presence of outliers
Invariant to translation -> since all measurements are
taken w.r.t. pts on the object
Scale invariance: Normalize all radial distances by
mean distance between the pt pairs in the shape
From expts: insensitive to small perturbations of parts
of the shape, small non-linear transformations,
occlusions and outliers
Can provide complete rotation invariance: use relative
frame – tangent vector at each point as the x-axis
(not suitable for say 6 and 9)
12
-a,b- sampled points
- correspondence found using
bipartite matching
Thin-Plate Spline Model
13
Minimizing Bend Energy
14
Matching Process
15
Object Recognition
Prototype based recognition: categories represented by ideal examples, rather than
logical rules
Eg. Prototype for bird category: sparrow
Soft category membership – as one moves further away from the ideal example, the
association with that prototype falls off
3 distances:
Shape Context Distance between shapes P and Q: sum of SC matching costs over
best matching points
Local image appearance difference: texture and color
Amount of transformation necessary to align the shapes: Bending energy in TPS
16
Results – Digit Recognition
17
Result: Trademark Retrieval
18
-300 trademarks
- 300 sample points
-Computational Needs:
-For 100 sample points - ~200ms
Inner Distance
19
Using the Inner-Distance for Classification of Articulated Shapes, H. Ling and
D. Jacobs, 2005
Model of Articulated Objects
20
Inner Distance
21
Computing the Inner Distance
22
Example
23
Experiments
24
Hierarchical Matching of Deformable Shapes
25 P. Felzenszwalb and J. Schwartz, 2007
Use:
- Compare pair of objects
- Detect objects in cluttered images
The Shape Tree
A be an open curve (a1, . . . , an).
ai be a midpoint on A.
L(ai|a1, an) -> location of ai relative to a1 and an
First & last sample points define a canonical scale
and orientation, so L invariant to similarity transf.
26
o Left child of a node: describes the subcurve from the start to the midpoint
o Right child describes the subcurve from the midpoint to the end.
o Bottom nodes capture local geometric properties such as the angle formed at a point,
o Root nodes capture more global information encoded by the relative locations of points
that are far from each other.
o Representation invariant to similarity transformations : Since contains only the
locations of points relative to two other points.
o Given the tree representation for A, along with the location of its start and end points a1 and an, the
curve can be recursively reconstructed – translated, rotated & scaled version of A
27
Bookstein coordinate of B
w.r.t. A and C
Deformation Model
28
Elastic Matching
A and B be 2 open curves
Build shape tree for A -> look for mapping from points in A to points in B such that
the shape tree of A is deformed as little as possible
Total deformation = sum over deformations applied to each node in the A shape-tree
Hierarchical nature of the shape-tree ensures that both local and global geometric
properties are preserved by a good matching.
Allow larger deformations near the bottom of a shape-tree as these do not change the
global appearance of an object.
29
Experiments
30
31
Results
32
What is a CAPTCHA?
33 Recognizing Objects in Adversarial Clutter: Breaking a Visual CAPTCHA,
G. Mori and J. Malik , 2003