Non-rigid Slice to Volume Medical Image Registrationmpawankumar.info/talks/slice-to-volume.pdf · 2015-04-21 · Test was performed over 20 manual segmentations of the left endocardium

Non-rigid Slice to Volume Medical Image Registration

Using Markov Random Fields

Enzo Ferrante, Nikos Paragios

Argentina / France

Motivation

3D ModalitiesCT, MRI

2D ModalitiesUltrasound (sliced image)X-Ray (projective image)

Medical Images

Nowadays: surgery planning with pre-operative images

High definition pre-operative 3D images

+ Annotated data + Doctor mind

Examples: CT, MRI, PET/CTObjects of interest (as organ

or tumor) segmentations.Surgeon experience

Future: surgery planning with pre and intra-operative images

High definition pre-operative 3D images + Annotated data

+ Intra-operative 2D images = Fused images and

plane location

*

* Image taken from Ewertsen, C. et al (2013). Real-time image fusion involving diagnostic ultrasound. AJR. American journal of roentgenology, 200(3), W249–55. doi:10.2214/AJR.12.8904

Future: surgery planning with pre and intra-operative images

Non-Rigid Slice to Volume Image registration

Slice to volume image registration Formal definition

Given a 2D source image I and a 3D target volume J, we seek the slice from the volume J

that best matches the image I.

It is “non-rigid” because image I can be deformed.

GPS based navigation systems (Electromagnetic sensors)

Image based navigation systems

● Absence of compensation for respiration and patient motion.

● Restrictions in the material of tools that can be used in the surgery.

● It's possible to deform intra-operative images in order to compensate respiration and patient motion.

● No restrictions about the tools that can be used in the surgery.

Slice to Volume image registration Different approaches

Data term

Regularization term

Deformation field

Plane (bi-dimensional slice)

Slice to Volume image registration as an optimization problem

G = < V, E >

V = Set of vertices

E = Set of pairwise cliques

L = Label space

Data term Regularization term

Slice to Volume image registration using Markov Random Fields

d N

Slice to Volume image registration 5 dimensional label space

Slice to Volume registration 5 dimensional label space

d N

Slice to Volume image registration Data term: Unary potentials

Monomodal registration

Multimodal registration

Slice to Volume image registration Data term: Examples

Grid regularity Plane structure

Slice to Volume image registration Regularization term: Pairwise potentials

Slice to Volume image registration Grid regularization: Distance preserving

Slice to Volume image registration Plane Structure regularization

[2] N. Komodakis, G. Tziritas, and N. Paragios. Fast, approximately optimal solutions for single and dynamic mrfs. In Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE Conference on, pages 1–8. IEEE, 2007.

● FastPD as optimization algorithm [2]

● Pyramidal approach: from coarse to fine spacing between the control points, refining the label space in every iteration

● Gaussian pyramids to improve the performance

Slice to Volume image registration Implementation details

Slice to Volume image registration Workflow

Slice to Volume image registration Workflow: initialization

Te: Estimated Rigid

Transformation using Horn's method

[3] B. Horn. Closed-form solution of absolute orientation using unit quaternions. JOSA A, 4(4):629–642, 1987.

[3]

Slice to Volume image registration Workflow: Optimization process

Grid projection over regression plane

Final 2D FFD that approximates the Deformation Field

Slice to Volume image registration Workflow: Solution recontruction

Experiments and Results

Monomodal DatasetCardiac Sequence

Monomodal DatasetCardiac Sequence

Parameters estimation over 10 temporal series of 20 slices from a beating heart MRI series (total of 200 registration cases) registered with

a MRI volume.

The average error is less than 0.013rad for rotation and less than 1mm for translation parameters.

Monomodal DatasetRigid parameters estimation

● Test was performed over 20 manual segmentations of the left endocardium

● Each slice was registered with the initial volume starting from a random position around the ground truth

● The estimated Deformation Field was applied to the initial segmentation

● The average DICE coefficient between the deformed segmentations and the ground truth was 0.93.

Monomodal DatasetDeformation field estimation

Monomodal Datasetsome qualitative results

Time

Beforeregistration

Deformationfield

Afterregistration

Six slices from the MRI heart sequence before and after registration

Monomodal Datasetsome qualitative results

Monomodal Datasetsome qualitative results

Multimodal DatasetBrain Sequence

Multimodal DatasetBrain Sequence

● DICE increases after registration process an average of 0.05

● CMD decreases an average of 0.4mm.

● Note that average DICE coefficients are always greater than 0.7.

Conclusions

● Registration is the key to bring high-resolution pre-operative images into the operating room and improve the accuracy of Image Guided Surgeries.

● Image guidance is a fundamental component that has started to be progressively incorporated in surgeries.

● We have proposed an image based slice to volume registration algorithm that gives promising results in our experimental dataset.

● Our method is metric free so it can be used to register different types of images

Future worksDecoupled model

Future worksLearning energy parameters

● Learning parameters w using SSVM

● Loss function (that measure the cost of predicting a labeling given GT) must decompose over the random variables.

● Main limitation: small dataset and poor ground truth

Related publications

Method and device for elastic registration between a two-dimensional digital image and a slice of a three-dimensional volume with overlapping contentN Paragios, E Ferrante, R Marini SilvaUS Patent 20,140,192,046

Non-rigid 2D-3D Medical Image Registration using Markov Random FieldsE Ferrante, N Paragios. Medical Image Computing and Computer-Assisted Intervention MICCAI 2013, Pages 163-170

¡Muchas Gracias!