Medical Image Analysis Image Registration Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Medical Image AnalysisMedical Image AnalysisImage Registration

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Image RegistrationImage Registration

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Atlas◦Study the variability of anatomical

and functional structures among the subjects

Structural computerized atlas (SCA): CT or conventional MRI. ◦A model for image segmentation and

extraction of a structural volume of interest (VOI)

Image RegistrationImage Registration

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Functional computerized atlas (FCA): SPECT, PET, or fMRI. ◦Understanding the metabolism of

functional activity in a specific structural VOI

Image registration methods and algorithms◦Transformation of a source image

space to the target image space

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Analysis

Analysis

Anatomical Reference

(SCA)

(SCA)

Functional Reference

(FCA)

Functional Reference(FCA)

ReferenceSignatures

ReferenceSignatures

MR Image(New Subject)

MR Image(New Subject)

PET Image(New Subject)

PET Image(New Subject)

MR-PETRegistration

MR-PETRegistration

Figure 9.1. A schematic diagram of multi-modality MR-PET image analysis using computerized atlases.

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

AB

f

g

Figure 9.2. Image registration through transformation.

Image RegistrationImage Registration

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

External markers and stereotactic frames based landmark registration◦External markers◦Coordinate transformation (rotation,

translation and scaling) and interpolation computed from visible markers

◦Optimizing the mean squared error◦Stereotactic frames are usually

uncomfortable for the patient

Image RegistrationImage RegistrationRigid-body transformation based

global registration◦Principal axes transformation◦PET-PET, MR-MR, MR-PET

Image feature-based registration◦Boundary and surface matching based

registration Edges, boundary and surface information A geometric transformation is obtained by

minimizing a predefined error function between the surfaces

Image RegistrationImage Registration

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

◦Image landmarks and features based registration Utilize pre-defined anatomical landmarks

or features Bayesian model based probabilistic

methods Neuroanatomical atlases for elastic

matching of brain images Landmark-based elastic matching

algorithm Maximum likelihood estimation

Rigid-Body TransformationRigid-Body TransformationRigid transformation

◦ : rotation matrix◦ : translation vector

Translation along -axis by

Rt

tRxx'

x p

zz

yy

pxx

'

'

'

Rigid-Body TransformationRigid-Body TransformationTranslation along -axis by

Translation along -axis by

y q

zz

qyy

xx

'

'

'

z r

rzz

yy

xx

'

'

'

Rigid-Body TransformationRigid-Body TransformationRotation about -axis by

Rotation about -axis by

x

cossin'

sincos'

'

zyz

zyy

xx

y

cossin'

'

sincos'

zxz

yy

zxx

Rigid-Body TransformationRigid-Body TransformationRotation about -axis byz

zz

yxy

yxx

'

cossin'

sincos'

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Translation of z

Translation of y Translation of x

Rotation by

Rotation by

Rotation by

Figure 9.3. The translation and rotation operations of a 3-D rigid transformation.

Rigid-Body TransformationRigid-Body Transformation

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

The rotation matrix for the rotational order:

R

cossin0

sincos0

001

cos0sin

010

sin0cos

100

0cossin

0sincos

RRRR

Affine TransformationAffine TransformationAffine matrix including the

translation, rotation and scaling transformation

Axx'

11000

000

000

000

1000

0cossin0

0sincos0

0001

1000

0cos0sin

0010

0sin0cos

1000

0100

00cossin

00sincos

1000

100

010

001

1

'

'

'

z

y

x

c

b

a

r

q

p

z

y

x

Principal Axes RegistrationPrincipal Axes RegistrationPrincipal axes registration (PAR)

◦ Global matching of two binary volumes

: a binary segmented volume

: centroid of

),,( zyxB

Tggg zyx ),,( ),,( zyxB

zyx

zyxg zyxB

zyxxB

x

,,

,,

),,(

),,(

Principal Axes RegistrationPrincipal Axes Registration

zyx

zyxg zyxB

zyxyB

y

,,

,,

),,(

),,(

zyx

zyxg zyxB

zyxzB

z

,,

,,

),,(

),,(

Principal Axes RegistrationPrincipal Axes RegistrationThe principal axes of

are the eigenvectors of the inertia matrix:

),,( zyxB

zzzyzx

yzyyyx

xzxyxx

III

III

III

I

zyx

ggxx zyxBzzyyI,,

22 ),,()()(

zyx

ggyy zyxBzzxxI,,

22 ),,()()(

Principal Axes RegistrationPrincipal Axes Registration

zyx

ggzz zyxByyxxI,,

22 ),,()()(

zyx

ggyxxy zyxByyxxII,,

),,())((

zyx

ggzxxz zyxBzzxxII,,

),,())((

zyx

ggzyyz zyxBzzyyII,,

),,())((

Principal Axes RegistrationPrincipal Axes RegistrationResolve 6 degrees of freedom

◦ Three rotations and three translations

Equate the normalized eigenvector matrix to the rotation matrix RRRE

333231

232221

131211

eee

eee

eee

E

Principal Axes RegistrationPrincipal Axes Registration

cossin0

sincos0

001

cos0sin

010

sin0cos

100

0cossin

0sincos

RRR

)arcsin( 31e

)cos/arcsin( 21 e

)cos/arcsin( 32 e

Principal Axes RegistrationPrincipal Axes RegistrationPAR for two volumes and

◦ 1. Translate the centroid of to the origin

◦ 2. Rotate the principal axes of to coincide with the , and axes

◦ 3. Rotate the , and axes to coincide with the principal axes of

◦ 4. Translate the origin to the centroid of

◦ is scaled to match the volume using the scaling factor

1V 2V

1V

1Vx y z

x y z

2V

2V

1V2V

3

2

1

V

VFs

Principal Axes RegistrationPrincipal Axes RegistrationProbabilistic models

◦ Counting the occurrence of a particular binary subvolume that is extracted from the registered volumes corresponding to various images

n

ii zyxS

nzyxM

1

),,(1

),,(

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Figure 9.4. A 3-D model of brain ventricles obtained from registering 22 MR brain images using the PAR method.

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Figure 9.5. Rotated views of the 3-D brain ventricle model shown in Figure 9.3.

Iterative Principal Axes Iterative Principal Axes RegistrationRegistrationIterative principal axes

registration (IPAR)◦ Developed by Dhawan et al.◦ Register MR and PET brain images◦ Used with partial volumes

Iteration 1

Figure 9.6. Three successive iterations of the IPAR algorithms for registration of vol 1 and vol 2: The results of the first iteration (a), the second iteration (b) and the final iteration (c). Vol 1 represents the MR data while the PET image with limited filed of view (FOV) is represented by vol 2.

(a)

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Iteration 2

(b)

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Iteration 3

(c)

Figures 9.7 a, b and c: Sequential slices of MR (middle rows) and PET (bottom rows) and the registered MR-PET brain images (top row) of the corresponding slices using the IPAR method.

(a)

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

(b)

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

(c)

Image Landmarks and Features Image Landmarks and Features Based RegistrationBased RegistrationImage landmarks and features Image landmarks and features

based registrationbased registration◦ Rigid and non-rigid transformationsRigid and non-rigid transformations◦ Image landmarks (points) and Image landmarks (points) and

featuresfeatures

Similarity Transformation for Similarity Transformation for Point-Based RegistrationPoint-Based RegistrationA non-rigid transformation

: ratation : scaling : translation : the total number of landmarks

)(xT

trxx s'

yxx )()( TE

sr

tN

N

iiii sw

1

22 || ytrx

Similarity Transformation for Similarity Transformation for Point-Based RegistrationPoint-Based RegistrationAlgorithm

◦ 1.◦ 2. Find

1sr

N

ii

N

iii

w

w

1

2

1

2xx

N

ii

N

iii

w

w

1

2

1

2yy

xxx ii

yyy ii

Similarity Transformation for Similarity Transformation for Point-Based RegistrationPoint-Based Registration

Singular value decomposition

◦ 3. Compute the scaling factor

◦ 4. Compute

N

i

tiiiwZ

1

2 yx

tVUZ

),,( 321 diag 0321 tdiag UVUVr ))det(,1,1(

N

iiii

N

iiii

w

ws

1

2

1

2

xxr

yxr

xryt s

Weighted Features Based Weighted Features Based RegistrationRegistration , = 1, 2, 3,…, : a set of

corresponding data shapes in and spaces

}{ iX i sN

x y

s iXN

i

N

jijijij TwTd

1 1

22 )()( yx

Elastic Deformation Based Elastic Deformation Based RegistrationRegistrationElastic deformation

◦Mimic a manual registration◦Map the elastic volume to the

reference volume◦The elastic volume is deformed by

applying external forces such that it matches the reference model

◦Constraints Smoothness incompressibility

Elastic Deformation Based Elastic Deformation Based RegistrationRegistrationMotion of a deformable body in

Lagrangian form ◦ : the force acting on a particle◦ : the position◦ : time◦ : the mass◦ : the damping constant◦ : the internal energy of

deformation

rtttf

)(

),(2

2 rrrr

),( tf rr

t

)(r

Elastic Deformation Based Elastic Deformation Based RegistrationRegistrationFind the displacement vector

that maximizes the similarity measure◦ : metric tensor◦ : curvature tensor

ijkG

ijkB

kjiijkijkijkijk dadadaBBGGS )()',(221221 xx

)',( xxS

u

MR ReferenceBrain Image

Data

Global RegistrationIPAR

Algorithm

MR NewBrain Image

Data

AnatomicalReference

Model

LandmarksLocalization and

VOICharacterization

Expert ViewerEditing andValidation

Low-ResolutionDeformation and

Matching

Spatial Relaxationand Constraint

Adapation

High-ResolutionDeformation and

Matching

Multi-Resolution DeformationBased

Local Registration andMatching

Figure 9.8. Block diagram for the MR image registration procedure.

Figure 9.9. Results of the elastic deformation based registration of 3-D MR brain images: The left column shows three images of the reference volume, the middle column shows the respective images of the brain volume to be registered and the right column shows the respective images of the registered brain volume.