THE STUDY OF MS USING MRI, IMAGE PROCESSING, AND VISUALIZATION By Jeremy Michael Nett B.S.E.E., University of Louisville, 2000 A Thesis Submitted to the Faculty of the University of Louisville Speed Scientific School As Partial Fulfillment of the Requirements For the Professional Degree MASTER OF ENGINEERING Department of Electrical and Computer Engineering December, 2001
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
8/12/2019 Jeremy Nett
http://slidepdf.com/reader/full/jeremy-nett 1/124
THE STUDY OF MS USING MRI, IMAGE PROCESSING, AND VISUALIZATION
By
Jeremy Michael NettB.S.E.E., University of Louisville, 2000
A ThesisSubmitted to the Faculty of the
University of LouisvilleSpeed Scientific School
As Partial Fulfillment of the RequirementsFor the Professional Degree
MASTER OF ENGINEERING
Department of Electrical and Computer Engineering
December, 2001
8/12/2019 Jeremy Nett
http://slidepdf.com/reader/full/jeremy-nett 2/124
ii
THE STUDY OF MS USING MRI, IMAGE PROCESSING, AND VISUALIZATION
Submitted by:
Jeremy Michael Nett
A Thesis Approved on
By the Following Reading and Examination Committee:
Aly A. Farag, Thesis Director
Tom Cleaver
Kyung Kang
Robert Falk
Christina Kaufman
Stephen Hushek
8/12/2019 Jeremy Nett
http://slidepdf.com/reader/full/jeremy-nett 3/124
iii
ACKNOWLEDGMENTS
The author would like to thank all those that made his education at the University
of Louisville possible, including the many contributors to the scholarship funds of which
he was profoundly honored to be a recipient.
The author would also like to thank Dr. Aly Farag and the CVIP Lab for an
interesting thesis topic, and support during the completion of this work. Additionally, the
author would like to thank Dr. Robert Falk of Jewish Hospital for spending much of his
valuable time guiding this work, commenting on its quality, and for many suggestions on
how to make the tools developed more applicable in the setting of medical research and
clinical use. The author would also like to thank the additional members of his thesis
committee for their review and critique of his work, including Dr. Stephen Hushek of
Norton Healthcare, Dr. Christina Kaufman of the Institute for Cellular Therapeutics of
the School of Medicine at the University of Louisville, Dr. Thomas Cleaver of the
Electrical and Computer Engineering Department, and Dr. Kyung Kang of the Chemical
Engineering Department.
Last but not least, the author would like to sincerely thank his parents, Michael
and Kathy Nett, for putting up with him, and his grandmother, Agnita Nett, for a place to
stay. Without their support, this work would have not been possible.
8/12/2019 Jeremy Nett
http://slidepdf.com/reader/full/jeremy-nett 4/124
iv
ABSTRACT
Multiple sclerosis (MS), a well known disease of the nervous system in which the
myelin sheaths of axons are damaged, may be imaged using magnetic resonance imaging
(MRI) modalities. In this research, image analysis, volume registration, and scientific
visualization techniques are applied for quantitative and qualitative analysis of the
response of the disease to a proposed treatment regimen. Image analysis techniques are
applied, with the objective of automated and reliable quantitative evaluation of MS
lesions in the brain, through segmentation and classification of the brain and MS lesions.
To facilitate a time-series analysis of lesions, a volume registration technique is applied
to geometrically align MRI scans taken at different times over the course of treatment.
Additionally, scientific visualization techniques are utilized to facilitate three-
dimensional analysis of the disease pathology, and evaluation of changes in the structure
of lesions over a period of treatment. The result of these efforts is a preliminary system
for the study of MS using MRI.
8/12/2019 Jeremy Nett
http://slidepdf.com/reader/full/jeremy-nett 5/124
v
TABLE OF CONTENTS
PageAPPROVAL PAGE…………………………………………………………….. ii
ACKNOWLEDGMENTS……………………………………………………… iii
ABSTRACT……………………………………………………………………. iv
NOMENCLATURE……………………………………………………………. viii
LIST OF TABLES……………………………………………………………… xi
LIST OF FIGURES…………………………………………………………….. xii
I. INTRODUCTION……………………………………………………... 1
A. General Introduction……………………………………………. 1
B. Introduction to Multiple Sclerosis……………………………… 2
C. Introduction to Magnetic Resonance Imaging of MS………….. 2
D. Introduction to Computer-Assisted Evaluation of MS………… 5
E. Previous Work in MS Studies Using MRI and ImageProcessing Approaches…………………………………….. 7
F. Introduction to Lab Facilities and Data Acquisition……………. 9
G. Outline of the Components of the Researched Solution……….. 11
H. Summary……………………………………………………….. 12
II. VOLUME REGISTRATION…………………………………………. 13
A. Introduction to Volume Registration…………………………... 13
B. Imaging Model…………………………………………………. 19
C. Transformations………………………………………………... 26
8/12/2019 Jeremy Nett
http://slidepdf.com/reader/full/jeremy-nett 6/124
vi
D. Volume Interpolation …………………………………………... 30
E. Overview of Registration Approaches…………………………. 31
F. Registration Metric/Criteria……………………………………. 32
G. Computation of the Mutual Information Metric……………….. 37
H. Search Algorithm………………………………………………. 43
I. Implementation………………………………………………….. 44
J. Results: MS Studies…………………………………………….. 44
K. Summary……………………………………………………….. 50
III. BRAIN SEGMENTATION FROM THE HEAD……………………. 51
A. Introduction and Necessity…………………………………….. 51
B. Brain Segmentation Utilizing Registration…………………….. 52
C. Results: Manual a priori Segmentation………………………... 55
D. Results: Semi-Automatic a priori Segmentation………………. 59
E. Summary……………………………………………………….. 62
IV. TISSUE SEGMENTATION…………………………………………. 63
A. Introduction and Necessity……………………………………... 63
B. Feature Selection……………………………………………….. 64
C. Thresholding……………………………………………………. 67
D. Segmentation by Image Enhancement…………………………. 69
E. Segmentation by Unsupervised Clustering…………………….. 73
F. Bayesian Classification………………………………………… 77
G. Bayesian Classification: Gaussian Conditionals, Equal a priori Probabilities………………………………………………... 80
8/12/2019 Jeremy Nett
http://slidepdf.com/reader/full/jeremy-nett 7/124
vii
H. Bayesian Classification: Nonparametric Conditionals, a priori Probabilities Modeled by Markov Random Fields………… 82
I. Quantification…………………………………………………… 89
J. Accuracy………………………………………………………… 90
K. Summary……………………………………………………….. 91
V. VISUALIZATION……………………………………………………. 92
A. Introduction…………………………………………………….. 92
B. Visualization for the Study of Volume Registration…………… 92
C. Visualization for the Study of Volume Segmentation…………. 96
D. Web-Based Presentation of Results…………………………….. 98
E. Summary……………………………………………………….. 99
VI. CONCLUSIONS AND RECOMMENDATIONS…………………… 101
REFERENCES……….…………………………………………………………. 104
VITA……………………………………………………………………………. 109
8/12/2019 Jeremy Nett
http://slidepdf.com/reader/full/jeremy-nett 8/124
viii
NOMENCLATURE
l = a volume coordinate vector in the image coordinate system
w = a volume coordinate vector in the world coordinate system
c =coordinate vector locating the center of a volumeorthe number of states of nature
c(i) = a diffusion function
C = transformation matrix for alignment of the center of a volume tothe origin in the world coordinate system
V = transformation matrix for scaling a volume to account forspatial sampling rates
Γ = transformation matrix for accounting of a gantry angle
A i,w = transformation matrix for conversion of image to worldcoordinates
R = reference volume for registration
F =floating volume for registrationora random field
T = transformation matrix for translation
R x, R y, R z = axis-angle rotation matrices
R = transformation matrix for rotation
TFR = transformation matrix from image coordinates in the floatingvolume to the reference volume
lower_left_N1,lower_left_N2,lower_left_N3
= coordinates for alignment of a bounding cube for trilinearvolume interpolation
8/12/2019 Jeremy Nett
http://slidepdf.com/reader/full/jeremy-nett 9/124
ix
V(sample i) = interpolated sample from a volume for a sample coordinate ofsample i
delta x, delta y, delta z = translational differences between a coordinate to interpolate at,and a point that forms an origin for trilinear interpolation
weight i = a weight and index i for trilinear interpolation
sample i = a sample at index i in a bounding cube for trilinear interpolation
p(x), q(x) = a probability distribution function
D(p||q) = relative entropy
I(X, Y) = mutual information of the random variables X and Y
H(X) = entropy of the random variable X
Tα = a registration transformation with a parameter set α
h(·) = a histogram
µ =coordinates of a cluster center locationora parameter of a Gaussian distribution
x = a feature vector
d = the dimension of the feature vector x
ω = a state of nature, or class
P(ω i|x) = the a posteriori probability of class ω i given a featuremeasurement x
p(x| ω i) = the class conditional probability of observing a featuremeasurement x, given a class ω i
1. Sample slices from FLAIR MRI studies of patients with MS.Hyperintense regions in the brain are indicative of plaques caused by MS.. 4
2. The front-end to the patient database of Jewish Hospital…………………. 9
3. The SGI Onyx2 visual supercomputer used as a computing platform in thisresearch…………………………………………………………………….. 10
4. Illustration of the registration problem in two dimensions. The squares inthe left and right figures represent the respective scanning area. Comparedwith one another, the anatomy is located at a different position betweenthe two volumes, and has been rotated…………………………..……….... 14
5. Scout scans for the same patient, at different time points. Note that thelines that indicate the slice planes do not correspond in the two differentstudies……………………………………………………………….……… 15
6. Sample slice comparison between two scans of the same patient, taken atdifferent points in time. Each column contains sample slices from onestudy. The rows contain the same slice number from each study. Asapparent, the anatomy is not geometrically aligned between the twoscanning volumes…………………………………………………….…….. 16
7. Examples of sagittal scans (a), coronal scans(b), and axial scans (c)….…... 19
8. Formation a volume from an ordered set of images……………………….. 20
9. Illustration of eight samples in an isotropic volume. Each sample islocated at lattice points, with integer coordinates, and equal distances
between lattice locations which have a difference of 1.0 between acoordinate………………………………………………………………….. 22
10. Illustration of the parallel computation of the joint histogram necessary forcomputation of the mutual information metric…………………………….. 41
11. Sample registration results from each of the seven MS data sets. The firstcolumn(a, d, g, j, m, p, and s) is a sample slice from the floating volumeused. The second column (b, e, h, k, n, q, and t) is the corresponding slicefrom the re-sampled reference volume. The third column (c, f, i, l, o, r,and u) is the checkerboard composite image of the two correspondingslices from the floating and re-sampled reference volumes. The floatingvolume and reference volumes used in each trial were from the same
patient……………………………………………………………………… 49
12. Samples of the (non-expert) manual segmentation of a patient’s brain fromthe remaining tissue of the head. The slices obtained from the MRI study((a), (b), and (c)) are manually segmented to obtain slices containing the
brain ((d), (e) and (f), respectively). The manually segmented images arethen made into binary masks ((g), (h), and (i), respectively)……………… 56
13. Registration of two MRI studies of the same patient, taken at differenttimes. The first study ((a), (b), and (c)) was manually segmented toextract the brain from the head. The second study ((d), (e), and (f)) wasregistered to the first (the re-sampled slices are shown)…………………… 57
14. Segmentation of the second study by a binary mask calculated from theregistration parameters, and the binary mask used to segment the a prioristudy. The raw input slices are shown ((a), (b), and (c)). The binary maskwas obtained by use of the registration parameters and the a priori mask((d), (e) and (f), respectively). By application of the mask, the segmentedvolume is obtained ((g), (h), and (i), respectively)……………………….. 58
15. Sample brain segmentation results of the a priori scan using a semi-automatic technique improved upon in the CVIP Lab. The input slices((a), (b), and (c)) are semi-automatically segmented to form a binary maskof the brain ((d), (e), and (f), respectively). The mask is then applied toform the segmented volume ((g), (h), and (i), respectively)………………. 60
16. Brain segmentation of the second study. The second study ((a), (b), and(c)) was registered to the a priori scan. The brain mask for the secondstudy ((d), (e), and (f), respectively) is then obtained using the a priori
brain mask, and the registration parameters. The binary brain mask is thenapplied to generate the segmented volume ((g), (h), and (i), respectively)... 61
17. A sample slice from an MRI study of an MS patient, with FLAIR (a andd), T1 (b and e), and T2 (c and f) weightings. The appearance of lesions isnot consistent between different weightings. MS lesions are easilyidentified in the FLAIR images as hyperintense regions in the brain……… 66
18. Segmentation by thresholding. The input slice (a) is classified into threeclasses, based on selection of class on the interval in which an intensityfalls. Corresponding labeled image slice is shown in (b), with the darkgray indicating CSF, the light gray indicating normal brain material, andthe red indicating MS lesions……………………………………………… 68
19. The histogram of the input slice shown in figure 18 (a). Intervalscontaining a tissue class can be visually identified by a human. Peaking ofthe histogram at low intensities represents CSF, and at high intensities, MSlesions. Peaking of the histogram in the intermediate intensities indicatesnormal brain tissue…………………………………………………………. 68
20. Non-linear anisotropic diffusion filtering of MRI slices for segmentation.Input slices (a and c) are FLAIR weighted imagery. The correspondingoutput slices (b and d, respectively) show much enhanced images, but withthe loss of some MS lesions………………………………………………... 72
21. Illustration of types of errors encountered with segmentation via imageenhancement………………………………………………………………... 73
22. Results of k-means clustering, using 5 clusters. The input slice is shownin (a). The resulting clustered output image is shown in (b). By manualre-labeling of clusters such that clusters of similar tissue belong to thesame cluster, the output slice in (c) is obtained…………………………... 76
23. Sample result of tissue classification in the brain using a simple Bayesianclassifier, with equal a priori probabilities, and Gaussian class conditional
probabilities………………………………………………………………… 82
24. Neighborhood systems for an image modeled as a Markov random field…. 86
25. Sample classification results obtained using a Bayesian classifier, withclass conditional probabilities modeled nonparametrically, and class a
priori probabilities obtained from a Markov random field model of theimage. Input slices are shown in (a) and (c), and the classified images areshown in (b) and (d). In (b) and (d), normal brain material is indicated bythe color yellow, CSF by gray, and lesions as red…………………………. 89
26. Generation of the checkerboard image, by formation of a composite imageof square pixel regions from the re-sampled reference and floatingvolumes……………………………………………………………………. 93
27. Sample slices from different registration studies, showing the a slice fromthe floating volume ((a), (d), and (g)), corresponding slices from the re-sampled, registered reference volume ((b), (e),and (h), respectively), andthe checkerboard slices ((c), (f), and (i), respectively)…………………….. 94
28. Volume rendering and arbitrary volume re-slicing for comparison of twovolumes that are registered via maximization of mutual information……... 95
FIGURE 4 : Illustration of the registration problem in two dimensions. The squares inthe left and right figures represent the respective scanning area. Compared with one
another, the anatomy is located at a different position between the two volumes, and hasbeen rotated.
Figure 5 below provides further evidence of this problem. Shown are two
different images, known as scout images, for the same patient, for MRI studies done at
different points in time. The scout images show the positions of the imaging planes in a
study, and are recorded by the technologist controlling the MRI machine at the time of
data acquisition. As can be observed from the two scout images shown, the positions of
these lines do not correspond with one another, indicating that a slice-by-slice
comparison of the studies will not allow for comparison between the same physical
second column are the slices from a later study, having the same slice number as the
slices shown in the first column. As can be observed, the anatomy imaged is not
geometrically aligned.
(a) (b)
(c) (d)
(e) (f)FIGURE 6 : Sample slice comparison between two scans of the same patient, taken at
different points in time. Each column contains sample slices from one study. The rowscontain the same slice number from each study. As apparent, the anatomy is not
geometrically aligned between the two scanning volumes.
FIGURE 9 : Illustration of eight samples in an isotropic volume. Each sample is locatedat lattice points, with integer coordinates, and equal distances between lattice locations
which have a difference of 1.0 between a coordinate.
The world coordinate system places each volume into 3D space, accounting
properly for origins, voxel sizes, and a gantry angle. Conversion from image coordinates
to world coordinates must consider these factors.
The convention used for setting an origin for a volume is to calculate the center
location of the volume. Thus, if a volume has a dimension d i for the dimension i, then the
center coordinate is given by c i = (d i – 1) / 2. Therefore, for volume dimensions d x, dy,
and d z, the center is computed as given in eq. 2.
For two discrete random variables X and Y, with marginal probability mass
functions p X(x) and p Y(y), and a joint distribution p(x, y), the mutual information
function I(x,y) is the relative entropy between the joint distribution p(x, y), and the
distribution p X(x)•p Y(y), the joint distribution when X and Y are independent random
variables. Thus, the mutual information of X and Y is given in eq. 17 below [17].
∑⋅
⋅=⋅=Y X Y X
Y X y p x p y x p y x p y p x p y x p DY X I
,2 )()(
),(log),())()(||),((),( (17)
If X and Y are independent random variables, then p(x, y) is given by the product
of the marginals p X(x) and p Y(y), and therefore, the quantity that is the argument of the
logarithm function in eq. 17 is one. As the logarithm of one is zero, the mutualinformation I(X, Y), when X and Y are statistically independent, is zero.
There is a close relationship between entropy, a measure of information content,
and the mutual information quantity. The entropy H(X) of a discrete random variable X
is defined in eq. 18. The joint entropy H(X,Y) of the discrete random variables X and Y
is defined below in eq. 19. The conditional entropy H(Y|X) of the discrete random
variables X and Y is defined below in eq. 20 [21].
TABLE II PARAMETERS OF THE MS STUDIES USED FOR ASSESSING THE REGISTRATION
SOFTWARE DEVELOPED .
Reference Studystudy patient
date img.plane weighting dimensions voxel sizes (mm)
1 A 8/22/2001 axial FLAIR 256 x 256 x 21 0.937503 x 0.937500 x 5.02 B 7/27/2001 axial FLAIR 256 x 256 x 20 0.9375 x 0.9375 x 5.03 C 11/7/2000 axial FLAIR 256 x 256 x 20 0.9375 x 0.9375 x 5.04 A 8/22/2001 coronal T1 256 x 256 x 25 0.78125 x 0.78125 x 5.05 C 11/7/2000 sagittal T1 256 x 256 x 12 0.9375 x 0.9375 x 5.06 D 10/13/2001 axial FLAIR 256 x 256 x 21 0.937494 x 0.9375 x 5.07 E 3/27/1999 axial FLAIR 256 x 256 x 21 0.93749 x 0.9375 x 5.0
Floating Studydate img.
plane weighting dimensions voxel sizes (mm)
1 A 9/12/2000 axial FLAIR 256 x 256 x 20 0.9375 x 0.9375 x 5.02 B 1/29/2001 axial FLAIR 256 x 256 x 21 0.9375 x 0.9375 x 5.03 C 4/8/2000 axial FLAIR 256 x 256 x 20 0.9375 x 0.9375 x 5.04 A 9/12/2000 coronal T1 256 x 256 x 23 0.859375 x 0.859375 x 5.05 C 4/8/2000 sagittal T1 256 x 256 x 12 0.9375 x 0.9375 x 5.06 D 6/9/2000 axial FLAIR 256 x 256 x 20 0.9375 x 0.9375 x 5.07 E 11/3/2001 axial FLAIR 256 x 256 x 20 0.937496 x 0.9375 x 5.0
Table III below shows the registration parameters obtained, as well as the
execution time of the registration application. The registration parameters shown are the
six parameters comprising the parameter vector, including the three translation quantities
(tx, ty, and t z), and the three rotation angles ( φx, φy, and φz). The execution time shown
includes the entire execution time of the program, including initialization, the search for
the optimal registration parameters, and generation of a re-sampled output volume.
(s) (t) (u)FIGURE 11 : Sample registration results from each of the seven MS data sets. The first
column(a, d, g, j, m, p, and s) is a sample slice from the floating volume used. The second column (b, e, h, k, n, q, and t) is the corresponding slice from the re-sampled
reference volume. The third column (c, f, i, l, o, r, and u) is the checkerboard compositeimage of the two corresponding slices from the floating and re-sampled reference
volumes. The floating volume and reference volumes used in each trial were from the same patient.
binary mask of the a priori scan, and the registration parameters discovered in the
registration process.
(a) (b) (c)
(d) (e) (f)
(g) (h) (i)FIGURE 12 : Samples of the (non-expert) manual segmentation of a patient’s brain
from the remaining tissue of the head. The slices obtained from the MRI study ((a), (b),and (c)) are manually segmented to obtain slices containing the brain ((d), (e) and (f),respectively). The manually segmented images are then made into binary masks ((g),
(d) (e) (f)FIGURE 13 : Registration of two MRI studies of the same patient, taken at differenttimes. The first study ((a), (b), and (c)) was manually segmented to extract the brain from the head. The second study ((d), (e), and (f)) was registered to the first (the re-
(g) (h) (i)FIGURE 14 : Segmentation of the second study by a binary mask calculated from theregistration parameters, and the binary mask used to segment the a priori study. Theraw input slices are shown ((a), (b), and (c)). The binary mask was obtained by use of
the registration parameters and the a priori mask ((d), (e) and (f), respectively). Byapplication of the mask, the segmented volume is obtained ((g), (h), and (i), respectively).
The accuracy of the resulting segmentation depends on the accuracy of the
registration, and the accuracy of the a priori segmentation. The registration results were
found to be acceptable to an expert (as evaluated in section II). The a priori
segmentation is assumed to be accurate on the basis of a manual segmentation. Despite
(g) (h) (i)FIGURE 15 : Sample brain segmentation results of the a priori scan using a semi-
automatic technique improved upon in the CVIP Lab. The input slices ((a), (b), and (c))are semi-automatically segmented to form a binary mask of the brain ((d), (e), and (f),respectively). The mask is then applied to form the segmented volume ((g), (h), and (i),
(g) (h) (i)FIGURE 16 : Brain segmentation of the second study. The second study ((a), (b), and(c)) was registered to the a priori scan. The brain mask for the second study ((d), (e),
and (f), respectively) is then obtained using the a priori brain mask, and the registration parameters. The binary brain mask is then applied to generate the segmented volume
((g), (h), and (i), respectively).
As can be observed from the results of the semi-automatic method, there are
errors that prevent this approach from further use. For example, an MS lesion in the slice
TR = 3577 ms, TE = 114 ms FIGURE 17 : A sample slice from an MRI study of an MS patient, with FLAIR (a and d),T1 (b and e), and T2 (c and f) weightings. The appearance of lesions is not consistentbetween different weightings. MS lesions are easily identified in the FLAIR images as
hyperintense regions in the brain.
It should be noted that differences in lesion appearance across different study
weightings may be attributed to different stages of the disease on a per-lesion basis. This
information, however, is not incorporated in the classification scheme developed in thisresearch.
Gross observation of the FLAIR imagery, following segmentation of the brain
from the head, shows that there are three dominant intensity levels, corresponding with
expected number of tissue classes present, and the selection of the thresholds is
influenced by this observation.
(a) (b)FIGURE 18 : Segmentation by thresholding. The input slice (a) is classified into three
classes, based on selection of class on the interval in which an intensity falls.Corresponding labeled image slice is shown in (b), with the dark gray indicating CSF,
the light gray indicating normal brain material, and the red indicating MS lesions.
FIGURE 19 : The histogram of the input slice shown in figure 18 (a). Intervalscontaining a tissue class can be visually identified by a human. Peaking of the histogram
at low intensities represents CSF, and at high intensities, MS lesions. Peaking of thehistogram in the intermediate intensities indicates normal brain tissue .
Although this technique is very intuitive, it is demanding on an expert to select
intensity ranges for each class, and for each slice. This process is time consuming,
(c) (d)FIGURE 20 : Non-linear anisotropic diffusion filtering of MRI slices for segmentation.
Input slices (a and c) are FLAIR weighted imagery. The corresponding output slices (b
and d, respectively) show much enhanced images, but with the loss of some MS lesions.
Unfortunately, while promising, the results introduce several difficulties. First of
all, lesions that do not have a strong edge on all sides are smoothed into brain material,
which leads to large classification errors when subsequent labeling is applied. Secondly,
large numbers of iterations of the filter are required before regions of similar tissue have
nearly the same intensity. Even at this point, some type of thresholding is necessary, andincurs similar difficulties as discussed above. Lastly, the resulting images do not entirely
contain the idealized three similar intensities. This is best illustrated by the input slice in
figure 20 (a), and the output slice in figure 20 (b), where intermediate intensity levels are
The above approaches, while generating tantalizing results, all fail with regard to
accuracy, ease of application, and logical relation to the classification problem. A
statistical classification approach addressees these concerns, providing an optimal
solution to the classification problem when tissue models are known or well
approximated. Additionally, statistical classification is well founded in probability
theory, and has enjoyed success in a number of applications, including classification of
remote sensing data, scene classification, and classification of medical imagery.
Statistical classification is fundamentally based upon the application of Bayes rule
for quantifying a decision procedure. Bayes rule, given in eq. 28 below, specifies how to
compute a posterior probability of a class, given feature measurements [33].
)(
)()|()|(
x p
P x p x P j j
j
ωωω
⋅=
(28)
In eq. 28, ω j denotes a class, or state of nature. The vector x denotes a feature
vector, obtained from measurements of quantities of interest. For image analysis, thefeature vector x is composed for each pixel, and consists of the features of the pixel. For
the approach here, the feature x therefore shall consist of one component, the intensity of
the pixel from the FLAIR image. The quantity p(x| ω j) is known as the class conditional
probability density function for class ω j. This distribution quantifies the probability of
measurement of a feature, given a state of nature. The quantity P( ω j) is known as the a
priori probability, and quantifies the probability of observing a state of nature, regardless
of any feature measurement. The quantity p(x) is known as the evidence, and serves only
as a scale factor, such that the quantity in eq. 28 is indeed a true probability, with values
between zero and one. The quantity P( ω j|x) is known as the posterior probability, and
quantifies the probability of observing class ω j, given that the feature x was measured.
The quantity p(x) in eq. 28 can normally be ignored for implementing decision
theory, as it is only a constant for establishing a probability between zero and one, and
constant for all classes. Therefore, the quantity given below in eq. 29, known as the
maximum a posteriori (MAP) estimate of eq. 28, is used.
)()|()|( j j j P x p x P ωωω ⋅= (29)
Bayes decision rule, based upon the application of eq. 28 or 29, gives a
framework for how to make a sound decision. For c possible classes, Bayes decision rule
mandates the selection of class ω j for a feature x , if the a posteriori quantities in eq. 28 or29 are maximum for class ω j, compared to ω i, i = 1…c, i ≠ j. This rule is stated below in
( ) ( )( )1,1,2,1,11,, )()( +−+− +⋅++⋅+⋅== l k l k l k l k l k l k f f f f f f U f U ββα (39)
The quantity f i,j in eq. 39 refers to the sample at indices (i,j) in the labeled image.
The parameters α, β1, and β2 are parameters that allow for adjusting relative weights or
contributions of neighborhood interactions. In the experiments here, α is taken to be 1.0,
and β1 and β2 are both taken to be 0.75. These weightings allow the current classificationat index (i, j) to take an importance somewhat greater than the neighborhood
classifications, and gives equal weighting to the neighbors of the feature measured at (i,
j). Also, it should be noted that the temperature parameter T is taken to be 1.0.
The implemented algorithm accepts a slice of data, and training points. An initial
classification solution is obtained by assumption of equal a priori class probabilities.
Subsequent iterations implement modeling of a priori class probabilities using the MRF
model discussed above. Each iteration classifies the entire image, using the previous
classification as a basis to model neighborhood interactions. The iterative process
terminates when no changes are made in the classification. Class conditional
probabilities are modeled throughout from the training points initially supplied.
A sample of the results obtained is given below in figure 25. The classifier
successfully detects the lesions, and in general, is observed to perform well. Additional
comments on the behavior of the classifier are given below. In terms of computational
performance, each segmentation executes within seconds on the SGI Onyx2
supercomputer, as a non-threaded application.
(a) (b)
(c) (d)FIGURE 25 : Sample classification results obtained using a Bayesian classifier, with
class conditional probabilities modeled nonparametrically, and class a priori probabilities obtained from a Markov random field model of the image. Input slices are shown in (a) and (c), and the classified images are shown in (b) and (d). In (b) and (d),normal brain material is indicated by the color yellow, CSF by gray, and lesions as red.
I. Quantification
With the classification results obtained, quantification of disease burden as judged
from MRI may be obtained by computing the total volume of lesions, and the total
volume of normal brain material. This can be simply accomplished by having the
volume. With the re-sampled reference volume, the reference and floating volumes may
then be compared on a slice-by-slice basis, as corresponding anatomy is registered in 3D
space.
As an aid to judging the quality of registration, a simple data fusion technique was
implemented for visualization. This technique takes regions from the re-sampled
reference volume and the floating volume to form a patchwork, composite image that
contains components of the imaged anatomy in both the reference and floating volumes.
This composite image, referred to as a checkerboard image, facilitates judgment of the
accuracy of the volume registration by allowing contours (such as the skull, or features
within the brain) to be followed between volumes, directly within one image. The
regions used here are 32 x 32 square pixel regions. Figure 26 below illustrates the
generation of the checkerboard images from the floating and re-sampled reference
volumes. Figure 27 below shows sample results obtained, from several registration
studies.
FIGURE 26: Generation of the checkerboard image, by formation of a composite imageof square pixel regions from the re-sampled reference and floating volumes.
(g) (h) (i)FIGURE 27 : Sample slices from different registration studies, showing the a slice from
the floating volume ((a), (d), and (g)), corresponding slices from the re-sampled,registered reference volume ((b), (e),and (h), respectively), and the checkerboard slices
((c), (f), and (i), respectively).
An additional approach to the comparison of registration results was made with
the utilization of volume rendering, and arbitrary volume re-slicing. Figure 28, below,
shows an example of this application. The graphics components of this application were
[1] J. H. Noseworthy, C. Lucchinetti, M. Rodriguez, and B. G. Weinshenker,“Multiple sclerosis,” The New England Journal of Medicine, vol. 343, no. 13, pp.938-952, Sept. 28, 2000.
[2] National Institutes of Neurological Disorders and Stroke, “Multiple Sclerosis:Hope Through Research,” NIH Publication No. 96-75, July 1, 2001.
[3] Z. P. Liang and P. C. Lauterbur, Principles of Magnetic Resonance Imaging: ASignal Processing Perspective. New York: IEEE Press, 2000.
[4] D. H. Miller, P. S. Albert, F. Barkhof, G. Francis, J. A. Frank, S. Hodgkinson, F.D. Liblin, D. W. Paty, S. C. Reingold, and J. Simon, “Guidelines for the Use ofMagnetic Resonance Techniques in Monitoring the Treatment of MultipleSclerosis,” Annals of Neurology, vol. 39, no. 1, Jan. 1996.
[5] C. R. G. Guttmann, R. Kikinis, M. C. Anderson, M. Jakab, S. K. Warfield, R. J.Killiany, H. L. Weiner, F. A. Jolesz, “Quantitative Follow-up of Patients withMultiple Sclerosis: Reproducibility,” JMRI, vol. 9, no. 4., pp. 509-518, June1999.
[6] R. Kikinis, C. R. G. Guttmann, D. Metcalf, W. M. Wells, G. J. Ettinger, H. L.Weiner, and F. A. Jolesz, “Quantitative Follow-up of Patients with MultipleSclerosis: Technical Aspects,” JMRI, vol. 9, no. 4, pp. 519-530, June 1999.
[7] W. M. Wells, W. E. L. Grimson, R. Kikinis, and F. A. Jolesz, “AdaptiveSegmentation of MRI Data,” IEEE Transactions on Medical Imaging, vol. 15, no.4, pp. 429-442, August 1996.
[8] M. Kamber, R. Shinghal, D. L. Collins, G. S. Francis, and A. C. Evans, “Model-Based 3-D Segmentation of Multiple Sclerosis Lesions in Magnetic ResonanceBrain Images,” IEEE Transactions on Medical Imaging, vol. 14., no. 3, pp. 442-453, Sept. 1995.
[9] B. Johnston, M. S. Atkins, B. Mackiewich, and M. Anderson, “Segmentation ofMultiple Sclerosis Lesions in Intensity Corrected Multispectral MRI,” IEEETransactions on Medical Imaging, vol. 15, no. 2, pp. 154-169, April 1996.
[10] J. Besag, “On the Statistical Analysis of Dirty Pictures,” Journal of the RoyalStatistical Society B, vol. 48, no. 3, pp. 259-30, 1986.
[11] J. K. Udupa, L. Wei, S. Samarasekera, Y. Miki, M. A. van Buchem, and R. I.Grossman, “Multiple Sclerosis Lesion Quantifiction Using Fuzzy-ConnectednessPrinciples,” IEEE Transactions on Medical Imaging, vol. 16, no. 5, pp. 598-609,Oct. 1997.
[12] J. K. Udupa and S. Samarasekera, “Fuzzy Connectedness and Object Definition:Theory, Algorithms, and Applications in Image Segmentation,” Graphical Modelsand Image Processing, vol. 58, no. 3, pp. 246-261, May 19996.
[13] K. V. Leemput, F. Maes, D. Vandermeulen, A. Colchester, and P. Suetens,“Automated Segmentation of Multiple Sclerosis Lesions by Model OutlierDetection,” IEEE Transactions on Medical Imaging, vol. 20, no. 8, pp. 677-688,August 2001.
[14] F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens,“Multimodality Image Registration by Maximization of Mutual Information,”IEEE Transactions on Medical Imaging, vol. 16, no. 2, pp. 187-198, April 1997.
[15] W. M. Wells, P. Viola, H. Atsumi, S. Nakajima, and R. Kikinis, “Multi-modalVolume Registration by Maximization of Mutual Information,” Medical ImageAnalysis, vol. 1, no. 1, pp. 35-51, 1996.
[16] C. R. Meyer, J. L. Boes, B. Kim, P. H. Bland, K. R. Zasadny, P. V. Kison, K.Koral, K. A. Frey, and R. L. Wahl, “Demonstration of Accuracy and ClinicalVersatility of Mutual Information for Automatic Multimodality Image FusionUsing Affine and Thin-Plate Spline Warped Geometric Deformations,” MedicalImage Analysis, vol. 1., no. 3, pp. 195-206, 1996/7.
[17] J. Nolte, The Human Brain: An Introduction to its Functional Anatomy. St.Louis: Mosby Year Book, 3 rd edition, 1993.
[18] J. B. A. Maintz and M. A. Viergever, “A Survey of Medical Image Registration,”Medical Image Analysis, vol. 2., no. 1, pp. 1-36, 1998.
[19] M. A. Audette, F. P. Ferrie, and T. M. Peters, “An Algorithmic Overview ofSurface Registration Techniques for Medical Imaging,” Medical Image Analysis,vol. 4, pp. 201-217, 2000.
[20] G. Wolberg, Digital Image Warping. Los Alamitos: IEEE Computer SocietyPress, 1990.
[21] T. M. Cover and J. A. Thomas, Elements of Information Theory. New York:John Wiley & Sons, 1991.
[22] G. P. Penney, J. Weese, J. A. Little, P. Desmedt, D. L. G. Hill, and D. J. Hawkes,“A Comparison of Similarity Measures for Use in 2D-3D Medical ImageRegistration,” IEEE Transactions on Medical Imaging, vol. 17, no. 4, pp. 586-595, August 1998.
[23] W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, NumericalRecipes in C: The Art of Scientific Computing. Cambridge University Press,Second Edition, 1992.
[24] F. Maes, D. Vandermeulen, and P. Suetens, “Comparative Evaluation ofMultiresolution Optimization Strategies for Multimodality Image Registration byMaximization of Mutual Information,” Medical Image Analysis, vol. 3, no. 4, pp.373-386, 1999.
[25] S. L. S. Jacoby, J. S. Kowalik, and J. T. Pizzo, Iterative Methods for NonlinearOptimization Problems. Englewood Cliffs: Prentice-Hall, 1972.
[26] M. Joliet and B. M. Mazoyer, “Three-Dimensional Segmentation andInterpolation of Magnetic Resonance Brain Images,” IEEE Transactions onMedical Imaging, vol. 12, no. 2, pp. 269-277, June 1993.
[27] M. E. Brummer, R. M. Mersereau, R. L. Eisner, and R. R. J. Lewine, “AutomaticDetection of Brain Contours in MRI Data Sets,” IEEE Transactions on MedicalImaging, vol. 12, no. 2, pp. 153-166, June 1993.
[28] T. Kapur, W. E. L. Grimson, W. M. Wells, and R. Kikinis, “Segmentation ofBrain Tissue from Magnetic Resonance Images,” Medical Image Analysis, vol. 1,no. 2, pp. 109-127, 1996.
[29] M. S. Atkins and B. T. Mackiewich, “Fully Automatic Segmentation of the Brainin MRI,” IEEE Transactions on Medical Imaging, vol. 17, no. 1, pp. 98-107, Feb.1998.
[30] C. Sites, A. Farag, and T. Moriarty, “Visualization Algorithms for MedicalImaging with Telemedicine and High Performance Computing Applications,”Technical Report, CVIP Laboratory, December 2000.
[31] N. Shareef, D. L. Wang, and R. Yagel, “Segmentation of Medical Images UsingLEGION,” IEEE Transactions on Medical Imaging, vol. 18, no. 1, pp. 74-91, Jan.1999.
[32] G. Gerig, O. Kübler, R. Kikinis, and F. A. Jolesz, “Nonlinear AnisotropicFiltering of MRI Data,” IEEE Transactions on Medical Imaging, vol. 11, no. 2,
pp. 221-232, June 1992.
[33] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. New York: JohnWiley & Sons, 2 nd edition, 2001.
[34] H. E. Cline, W. E. Lorensen, R. Kikinis, and F. Jolesz, “Three-DimensionalSegmentation of MR Images of the Head Using Probability and Connectivity,”Journal of Computer Assisted Tomography, vol. 14, no. 6, pp. 1037-1045,
[35] R. V. Hogg and E. A. Tanis, Probability and Statistical Inference. Upper SaddleRiver: Prentice-Hall, 6 th edition, 2001.
[36] S. Z. Li, Markov Random Field Modeling in Image Analysis. Tokyo: Springer-Verlag, 2001
[37] K. Held, E. R. Kops, B. J. Krause, W. M. Wells, R. Kikinis, and H. W. Müller-Gärtner, “Markov Random Field Segmentation of Brain MR Images,” IEEETransactions on Medical Imaging, vol. 16, no. 6, pp. 878-886, Dec. 1997.
[38] C. W. Therrien, Decision, Estimation, and Classification. New York: John Wiley& Sons, 1989.
[39] J. P. W. Pluim, J. B. A. Maintz, and M. A. Viergever, “Image Registration byMaximization of Combined Mutual Information and Gradient Information,” IEEETransactions on Medical Imaging, vol. 19, no. 8, pp. 809-814, August 2000.
[40] C. Studholme, D. L. G. Hill, and D. J. Hawkes, “An Overlap Invariant EntropyMeasure of 3D Medical Image Alignment,” Pattern Recognition, vol. 32, pp. 71-86, 1999.
[41] B. Likar and F. Pernus, “A Hierarchical Approach to Elastic Registration Basedon Mutual Information,” Image and Vision Computing, vol. 19, pp. 33-44, 2001.
[42] American College of Radiology, National Electrical Manufacturers Association,"Digital Imaging and Communications in Medicine (DICOM): Version 3.0",Draft Standard, ACR-NEMA Committee, Working Group VI, Washington, DC,1993.
[43] W. J. Schroeder, L. S. Avila, K. M. Martin, W. A. Hoffman, and C. C. Law, TheVTK User’s Guide. Kitware, 2001.
[44] W. Schroeder, K. Martin, and B. Lorensen, The Visualization Toolkit: An Object-Oriented Approach to 3D Graphics. Upper Saddle River: Prentice-Hall, 2 nd Edition, 1998.