Slide 1 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Shape Recovery from Medical Image Data Using Extended Superquadrics Talib Bhabhrawala.

Slide 1 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo

Shape Recovery from Medical Image Data Using Extended

Superquadrics

Talib Bhabhrawala

Advisor : Dr. Venkat KroviDepartment of Mechanical and Aerospace EngineeringState University of New York at Buffalo

Master of Science Thesis DefenseDecember 14th, 2004


Introduction Background Methodology Results Conclusion

• Introduction • Background • Methodology Development • Case Studies• Interfaces• Conclusion & Future Work

Overview


Explos ion inavailability and

form of 3Ddata.

Ubiquitous availability of computation and communication infrastructure

Introduction Superquadrics Methodology Results Conclusion

Introduction

Create, manipulate & distribute such data.


Model Based Reconstruction– Building geometric shape models from raw input data – Data reduction, Analysis, Manipulation, Storage

Computer Vision & Animation Life sciencesEngineering


Introduction

Point Cloud Data is adequate for Visualization

Application Areas

– Models & Methods are defined by the final application– Visualization – Surface Geometry– Dynamic & Finite Element Analysis – Volumetric Information


Desired Characteristics

Low Order Models– Computational ease.– Fitting, Visualizing & Analysis

Parametric Models - Intuitive and Easy to use

- Meaningful and repeatable

- Great Success in Engineering

Models whose nature is approximation

- Tractability for infinite dimensional data


How can we leverage the same framework to additionally parametrically explore multi-resolution hierarchical indexing, storage, searching, reconstruction and retrieval?


Research Issues

Which kind of a parametric approximation framework would be most suitable for rapid, easy, accurate and computationally inexpensive shape modeling and conversion to volumetric solid model from a dense sampling of the surface?


- flexible family of parametric objects - using low order parameterization, variety of shapes maybe obtained - simple mathematical representation- good explicit and implicit form

Superquadrics (SQ)


powerful & compact shape representation


cosm( ) = , / 2 / 2

sin

cosh( ) = ,

sin

A 3D surface can be obtained by the spherical product of two 2D curves.

When a half circle in a plane orthogonal to the (x, y) plane.

is crossed with the full circle in (x, y) plane

/2 /2cos cos

( , ) ( ) ( ) cos sin ,

sin

x

r m h y

z


Spherical Product


A superellipse is a closed curve defined by


Superellipses

1x y

a b


Superellipsoids are obtained from superellipses

a1, a2, a3 - scaling factors ε1 , ε 2 - relative roundness & squareness.

1 2

1 1

2 2

3

( , ) ( ) ( )

cos cos2 2cos sin ,

sin

r s s

a a

a a

a


Superquadrics


10.3 3

20.3 3


Superquadrics


Valuable single implicit function.

• The object is continuous everywhere.

• Point membership classification can be done

• Inside–outside function.

1

22/2 / 2 /

1 2 3

1x y z

a a a


Implicit Representation


Advantages

– can model a diverse set of objects– compact representations– controllability and intuitive meaning– can be recovered from 3D information robustly


Limitations– Basic representation can only model symmetrical shapes

SQ Discussion


• Superquadric’s applications

– Computer environments (Montiel, 1997; Pentland, 2000)– Graphics & vision (Chella, 2000; Jacklic, 2000)

• Local and Global Deformations

– Nonlinear deformable models (Solina & Bajcsy, 1991)– Simulating equations of motion (Terzopoulos, 1993)

• Increasing the DOF

– Segmentation (e.g. Löffelman and Gröller, 1994)– blending multiple models (DeCarlo & Metaxas, 1998)– free form deformations (Bardinet et al., 1994)


Literature


To represent more complex shapes there is a trade off between

- degrees of freedom & expressive power

Zhou and Kambhamettu (2001) first examined

- exponents need not be fixed- possibly be spatially varying functions- extended superquadrics


Literature


Analogous to a SQ it is defined by

• & are the latitude & longitude angles • Exponents are now functions of these angles.

( ) ( )1 1

( ) ( )

2 2

( )

3

cos cos

2 2cos sin ,

sin

f f

f f

f

a a

a a

a


Extended Superquadrics (ESQ)


Measure the difference between a modeled shape and the given data set

2 2

2 2 2

1 2 1

atan 2

atan 2

1

f f f

f

f

z

x y

y

x

x y x

a a a

where


Inside-Outside Function


The shape of the exponent functions have to be controllable

We introduce a spline as the exponent function.

This interpolated curve acts like a look up table for the algorithm

-2 -1.5 -1 -0.5 0 0.5 1 1.5 20.5

0.6

0.7

0.8

0.9

1

1.1

1.2

( )f


Exponent Functions



Intuitive Example


Problem Statement

• Recovering a superquadric model from a set of 3D points– Superquadric model

– Vector of superquadric parameters – Input Points

– Minimize

• Least square distance between SQ surface & data points

( )f P

P

3:g

, ,( ) 1,..,Ti i i iX x y z i N

1

( ( )) ( ( )) ( ( ))N

Ti i i

i

g X f P X f P X f P


3

s

s 1, 2, , 1 2

s

( , ε ,ε )

x

y F a a a

z

SQ in a local coordinate system

1

s

s

s

x x

y T y

z z

SQ in the general position Transform the points to the object coordinated system


Five parameters which

define the size & shape

Initial Model Definition


31 2 , , 1 1 p 1 1 1 p( , , , , , , ( )..... ( ), ( )..... ( ))s

s x y z

s

x

y F a a a p p p f f f f

z

,

Applying the Inverse Transform & using Euler angles

31, 2, , 1 2 , ,( , , , , )s

s x y z

s

x

y F a a a p p p

z

, Additional six parameters which

define the position and orientation

Case of Extended Superquadrics

Exponent is a spline interpolating ‘p’ control points increases the total number of parameters by 2(p-1).

Number of parameters are now 9+2(p-1).


Initial Model Definition


Farthest range point along each coordinate axis which gives an estimate of a1, a2 & a3

Object recognition and pose estimation

xx xy xz

yx yy yz

zx zy zz

I I I

M I I I

I I I

Obtain the rotation matrix and eigen vectors.

Orient axes along minimal & maximal moment of inertia

2 2

1

2 2

1

2

1

1

1

1

1( ) ( )

1( ) ( )

1( ) ( )

1( )( )

1( )( )

1( )( )

N

xx i ii

N

yy i ii

N

zz i ii

N

xy i ii

N

xz i ii

N

xz i ii

I y y z zN

I x x z zN

I y y x xN

I y y x xN

I z z x xN

I z z x xN


Moment Based Estimation


The error-of-fit function is define using the inside–outside function

2N data

i i ii=1EOF = 1- f x ,y , z

EOF varies quickly where the exponents are largeand slowly where exponents are small

2N data

i i ii=1EOF2 = 1- f x ,y ,

fz


Added to remove the bias

Error of Fit Function


2N data

1 2 3 i i ii=1EOF3 = a a a 1- f x ,y ,

fz

Ambiguity in Description– A set of exponent functions in conjunction with scaling

parameters can generate the same shape as another set

To solve the ambiguity, the minimum volume constraint is added


Error of Fit Function

Metric to be minimized for the “fitting”


Minimize

where

Variables

tan 2 0

tan 2 0

1, 2, 3 0

f a

f a

a a a

2N data

1 2 3 i i ii=1F = a a a 1- f x ,y ,

fz


Optimization Problem

3

11, 2, 1 1 p 1 1 1 p[ , ( )..... ( ), ( )..... ( )]P a a a f f f f


Guided random T echniques

EvolutionaryA lgorithms

GeneticA lgorithms

•The conventional method used is the Levenberg-Marquardt algorithm• Fast and accurate • Problems of local minima• Heavily dependent on initial estimates

Search T echniques

Calculus -B ased Guided random Enumerative

I ndirec tM ethods

Direc t M ethods

S imulatedAnnealing

EvolutionaryA lgorithms

DynamicP rogramming

Calculus - B ased


Choice of Optimization Method


• Inspired by Biological Evolution and its principles

• The evolution of life on earth can be

regarded as one long optimization process though it’s up to debate if this process has reached a optimum yet…


Genetic Algorithms


Salient Features

• Requires little insight into the problem

• Ideal if a problem is non convex or has a very large multimodal solution space

• Heuristics Based, does not require derivatives

• Provides with “Good” Solutions

• Ideal exploratory tool to examine new approaches

Genetic Algorithms


Genetic Algorithms

Components of a GA

Population size: 20 – 100

Crossovers: 50-60%

Mutations: < 5%

Generations: 20 – 2000

- Encoding technique (double vector, binary)- Object function (environment)- Genetic operators (selection, mutation, crossover)

“Typical” tuning parameters

initialize population;evaluate population;while TerminationCriteriaNotSatisfied{

select parents for reproduction;perform crossover & mutation;evaluate population;

}



T r an s f o r m toO b jec t C o o r d in a te

S y s tem

G o o d F it?

O b ta in P o in tC lo u d D ata

S tar tM o m en t Bas ed

E s tim atio n o f s izean d O r ien ta tio n

S et N u m b er o fC o n tr o l P o in ts to

2

S et G AP ar am eter s

M in im ize E O F 3T r an s f o r m to

O b jec t C o o r d in a teS y s tem

D is p lay th eF itted ES Q

Ad d a C o n tr o lP o in t

Ye s

N o

Shape Recovery Algorithm



2D Case Study



3D Case Study


• Difficult to fit a complex model using a single extended superquadric

• Segment an object into primitives

2N data

i ii=1EOF2 = 1- f x ,y

f

Maximum Error2

M data

i=1

( )EOF

M

-25 -20 -15 -10 -5 0 5 10 15 20 25

-10

-5

0

5

10

15

20

25

Partitioning Line

dminimize

-20 -15 -10 -5 0 5 10 15 20 25-20

-15

-10

-5

0

5

10

15

20

25

Two Superquadrics to approximate the data

EOF1= 0.575

EOF2= 0.326


Iterative Segmentation & Recovery



Volume Segmentation Using ESQ

• 2D contours are obtained and are stacked • Topological Accuracy is high• Loses Compact representation• Laborious process & model has inconsistencies • Requires a post-processing step



PC Based Interface


Java EnabledBrowser On Remote

Computer

Web Server

Matlab

Internet TCP/IP

Computational /VisualizationCapabilities


Web Based Interface


• Flexible enough for an asymmetric object that deform smoothly on spheres

• Variable coefficients of the continuous exponents offer a compact parameter space and broad coverage

• The descriptive parameterization is directly incorporated into the model formulation


Conclusion


• A more intuitive and robust segmentation scheme

• Techniques for creating “tailored” models from such simple general purpose models

• More intelligent precursor steps to improve convergence speed of the algorithm

• Systematic way to extract and store characteristic signatures of shape


Future Work


Thank You!

Acknowledgments:Dr. V. Krovi, Dr. C. Bloebaum &

Dr. A. Patra

Slide 1 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Shape Recovery from Medical Image Data Using Extended Superquadrics Talib Bhabhrawala.

Documents