MSc project Janneke Ansems 21-08-2007 Intensity and Feature Based 3D Rigid Registration of Pre- and Intra-Operative MR Brain Scans Committee: Prof. dr.

MSc project Janneke Ansems

21-08-2007

Intensity and Feature Based 3D Rigid Registration

of Pre- and Intra-Operative MR Brain Scans

Committee:Prof. dr. ir. B.M. ter Haar RomenyProf. dr. ir. F.N. van de VosseDr. ir. B. PlatelDr. ir. G.J. Strijkers

Two 3-D point sets are given: fpi g and fp0i g; i = 1, 2, : : :, N. The equation

that needs to be solved is then:

p0i = Rpi + T + Ni (1)

where R is a 3£3 rotation matrix, T a translation vector (3£1 column matrix)and Ni a noisevector. Therotation R and translation T haveto befound suchthat the following equation is minimized:

§ 2 =NX

i=1

k p0i ¡ (Rpi + T) k2: (2)

First the point sets are translated such that both point sets have the samecentroid. Then the rotation matrix R is calculated using the singular valuedecomposition (SVD). Finally the translation vector T is computed. Theregis-tration is completed by transforming the entire dataset using the just found Rand T.

RMS =1N

kp0i ¡ (Rpi + T)k: (3)

2

Outline Introduction Registration Materials and Methods Results Discussion and Conclusion Recommendations

3

IntroductionMedical Background

Brain tumors Cancer is 2nd major cause of death in the Netherlands at

present time Each year 1000 people in the Netherlands are diagnosed with a

brain tumor

Treatment Radiotherapy Resection surgery

Figure 1: benign (left) and malignant tumor

4

IntroductionImage-Guided Surgery

Image-Guided Surgery The use of images to guide a surgeon during the procedure

Medtronic Stealth Station Surgeon is able to verify the location

of a tumor directly with the imagesusing an image guided probe

But: pre-operative images do not alwaysresemble the real-time situation duringsurgery!

Figure 2: Medtronic Stealth Station

5

IntroductionIntra-Operative Imaging

Brain shift Intra-operative imaging during surgery gives a more

accurate view on the real-time situation

Figure 3: Axial slices during a craniotomy showing brain shift.

6

IntroductionIntra-Operative MRI

The Polestar N20 open intra-operative MR scanner (Medtronic Inc.) Field strength: 0.15 Tesla Resolution: 128x128x64 Field of view: 20x20x19 cm

Chosen for its: Relative low cost Open access to the patient Mobility Local shielding Compatibility with Medtronic

Stealth Station Compromise: image qualityFigure 4: The Polestar N20 system in the operating

room.

7

IntroductionImage quality Polestar

Due to low field the Polestar scanner is susceptible to noise and artifacts Intensity Gradient Distortions

Figure 5: Images of Phantoms scanned by the polestar N20 showing distortion (left) and intensity gradient.

8

IntroductionAim

Register 3D pre-operative high resolution MR data and the intra-operative MR data from the Polestar N20 with maximum accuracy.

In this way high resolution accurate information is available for navigational purposes during neurosurgery, focus on datasets with skull intact.

9

Outline Introduction

Registration Materials and Methods Results Discussion and Conclusion Recommendations

10

Registration Definition:

Given a reference image R and a template image T, find a suitable transformation y such that the transformed image T[y] looks similar to the reference image R

ReferenceTemplate

11

RegistrationTransformation Model

Definition: A mapping of locations of one image to new locations in another

image

12

RegistrationSimilarity Measure Definition:

Equation that measures how much two images are alike

Intensity-based Methods Sum of Squared Differences:

Gradient-based Methods Normalized Gradient Field:

x

2SSD ))( - )(( T] [R,S xRxT

22

2NGF

),(

))(())(( T] [R,S

T

TxTn

xTnxRn

Reference Template

13

RegistrationSimilarity Measure

14

RegistrationOptimizing Scheme

Definition: An optimizing scheme calculates the transformation

parameters to achieve maximum similarity

Steepest Descent Gauss-Newton Levenberg-Marquardt

Figure 6: A comparison of steepest descent (green) and Gauss-Newton's method (red) for minimizing a function

15

OptimizationGauss-Newton

The objective function f (w), w are transformation parameters:

f (w) =12

k T(y(w)) ¡ R k22 : (1)

Updatew by s, thesecond order Taylor approximation is:

f (w+ s) = f (w) + sr f (w) +12sT r 2f (w)s: (2)

The derivativeof s is taken and set to zero:

s =¡ r f (w)r 2f (w)

; (3)

in which r f (w) and r 2f (w) are:

r f (w) = Tyyw(T(y(w)) ¡ R) and r 2f (w) = (Tyyw)T (Tyyw): (4)

By updating w by s the new transformation parameters w are calculated.

16

RegistrationMultilevel approach

Definition: Register from coarse to fine to optimize for speed and

robustness

17

RegistrationSummary Intensity Based

18

RegistrationFeatures Definition:

Features are a finite number of pixels or groups of pixels that are unique and exist in both images.

Given features r1, … , rn in reference image and t1, … , tn in template image, find a transformation y such that:ntyr ii ,...,1ifor )(

19

Automatic feature detection Scale Invariant Feature Transform (SIFT) by Lowe 2D version gave promising results

RegistrationFeatures

Figure 6: Matches found by SIFT algorithm in Polestar data (left) and high resolution data

20

Manual feature selection Transformation using Arun’s algorithm

Demo Brainmark

RegistrationFeatures

test

p0i = Rpi + T + Ni (1)

where R is a 3£3 rotation matrix, T a translation vector (3£1 column matrix)and Ni a noisevector. The rotation R and translation T haveto be found suchthat the following equation is minimized:

§ 2 =NX

i=1

k p0i ¡ (Rpi + T) k2: (2)

test

Two 3D point sets are given, the equation that needs to be solved is then:

ri = Rti + T (1)

Root mean square error (RMS):

RMS =1N

kri ¡ (Rti + T)k: (2)

21

22

Outline Introduction Registration

Materials and Methods Results Discussion and Conclusion Recommendations

23

Materials and MethodsDatasets

Four datasets: Pre-operative 1.5 Tesla and intra-operative 0.15 Tesla MR data Two datasets of healthy volunteers Two partial datasets of patients

Figure 7: Mid-sagittal slices of the high and low resolution MR data of a healthy volunteer (left) and patient.

24

Materials and MethodsPreprocessing

Initial Alignment Important for optimization scheme Gravity point of nonzero voxels in sagittal direction

Gradient removal Skull removal

25

Materials and MethodsPreprocessing

Global Intensity Gradient Removal Subtract peaks from Gaussian blurred image

26

Materials and MethodsPreprocessing

Gradient Removal Subtract peaks from Gaussian blurred image

27

Materials and MethodsPreprocessing

‘Skull’ stripping Dilation and erosion of a binary mask

28

Materials and MethodsPreprocessing

‘Skull’ stripping Dilation and erosion of a binary mask

29

Materials and MethodsRegistration programs

Intensity-based registration programs: Rigid transformation model Sum of Squared Differences and Normalized Gradient Field Gauss-Newton optimization scheme Multilevel approach

Feature-based registration program: Manual selection of 10-15 features Arun’s algorithm for transformation

30

Materials and MethodsExperiments

Intensity-based registrations Four datasets were registered using the SSD and NGF

programs Multilevel approach: 1, 2 and 3 levels (resolution steps) were

used

Feature-based registration Four datasets were registered The results will be used as initial parameter guess for the

optimizing scheme of the NGF program to register the patient datasets

31

Materials and MethodsVisualization

To inspect registration results, a Graphical User Interface (GUI) was built

Figure 8: A screenshot of Regview to inspect registration results visually. The green lines indicate the cross-section with the other two views.

32

Materials and MethodsVisualization

Three different settings to inspect registration results

Figure 9: Checkerboard (left), fusion (middle) and transition visualization.

33

Outline Introduction Registration Materials and Methods

Results Discussion and Conclusion Recommendations

34

ResultsTable 1.1: Results from the SSD program.

Dataset Nr of Nr of F inal SSD Computing Quality Commentsresolution iterations value time [s] A ssessment

steps V isualInspection

VolunteerNeuro001 3 9+11+3=23 1539509 65 Good2 24+3=27 1539294 66 Good1 35 1540095 196 Average

VolunteerNeuro002 3 16+13+10=39 1427983 82 Bad No skull2 44+9=53 1378021 91 Average removal1 35 1378671 378 Bad

PatientNeuro003 3 54+47+6=107 759877 98 Good Manual2 125+7=132 983955 370 Bad initial1 154 1059637 730 Bad alignment

PatientNeuro004 3 - - - Unable2 - - - Unable1 - - - Unable

35

Results Influence of resolution steps, VolunteerNeuro002

One resolution step

Two resolution steps

36

ResultsTable 1.2: Results from the NGF program.

Dataset Nr of Nr F inal NGF Computing Quality A ssessmentresolution steps iterations value time [s] V isual Inspection

VolunteerNeuro001 3 18+11+10=39 20,13 117 Bad2 13+5=18 8,97 117 Excellent1 31 8,97 205 Excellent

VolunteerNeuro002 3 15+12+7=34 8,59 168 Good2 14+7=21 8,59 138 Excellent1 22 17,64 172 Bad

PatientNeuro003 3 40+20+10=70 40,96 195 Bad2 14+30=44 98,84 233 Bad1 7 121,0 88 Bad

PatientNeuro004 3 - - - Unable2 - - - Unable1 - - - Unable

37

ResultsRegistration of VolunteerNeuro001

using NGF, two resolution steps:

38

ResultsTable 1.3: Results from the feature-based program.Dataset Nr of selected R M S error [cm] Quality A ssessment Comments

features V isual InspectionVolunteerNeuro001 5 0.157 Poor

10 0.166 Average15 0.128 Average

VolunteerNeuro002 5 0.203 Poor10 0.130 Average15 0.093 Average

PatientNeuro003 4 0.119 Poor5 0.120 Poor6 0.121 Poor7 0.162 Poor8 0.139 Average8 - Good After NGF

PatientNeuro004 4 0.201 Poor5 0.161 Poor6 0.146 Poor7 0.136 Poor8 0.130 Poor9 0.124 Average10 0.150 Average10 - Excellent After NGF

39

Results

After manual selection of 10 features

Using feature based initialization for NGF registration program

40

Results The intensity-based registration programs managed to

register the datasets of the healthy volunteers

However both intensity-based programs were not able to register the partial datasets of the patients without manual initial parameter guess

Best results were obtained by using a feature-based registration as initial parameter guess for the intensity-based programs.

41

Outline Introduction Registration Materials and Methods Results

Discussion and Conclusion Recommendations

42

Conclusion and Discussion All datasets are registered, results were inspected using

visual inspection

Preprocessing important for intensity-based programs

Accuracy of the voxelsize is feasible in the center of the field of view

However this accuracy is not attainable at the edge of the field of view due to distortions and artifacts resulting from the low field of the Polestar N20

43

Outline Introduction Registration Materials and Methods Results Discussion and Conclusion

Recommendations

44

Recommendations Image quality Polestar

Currently a phantom is developed to measure and correct the distortion

Next step: registration after skull opening but before tumor resection Non rigid transformation model Computation time Validation

45

Thank you for your attention!Questions?