Page 1
Fast Segmentation of Vessels in MR Liver Images using Patient Specific Models
by
Sameer Zaheer
A thesis submitted in conformity with the requirements for the degree of Master’s of Health Science
Institute of Biomaterials and Biomedical Engineering University of Toronto
© Copyright by Sameer Zaheer 2013
Page 2
ii
Fast Segmentation of Vessels in MR Liver Images using Patient
Specific Models
Sameer Zaheer
Master’s of Health Science
Institute of Biomaterials and Biomedical Engineering
University of Toronto
2013
Abstract
Image-guided therapies have the potential to improve the accuracy of treating liver cancer. In
order to register intraoperative with preoperative liver images, joint segmentation and
registration methods require fast segmentation of matching vessel centerlines. The algorithm
presented in this thesis solves this problem by tracking the centerlines using ridge and cross-
section information, and uses knowledge of the patient’s vasculature in the preoperative image to
ensure correspondence. The algorithm was tested on three MR images of healthy volunteers and
one CT image of a patient with liver cancer. Results show that in the context of join
segmentation registration, if the registration error is less than 2.0mm, the average segmentation
error is 0.73-1.68mm, with 88-100% of the vessels having an error less than a voxel length. For
registration error less than 4.6mm, the average segmentation error is 1.17-2.11mm, with 79-98%
of the vessels having an error less than a voxel length.
Page 3
iii
Acknowledgments
I would like to thank Dr. James Drake for his supervision and support. I would like to
thank Dr. (Edward) Xishi Huang for his supervision and patiently teaching me important
concepts in mathematics, imaging and programming. I would like to thank my committee
members Dr. Allan Jepson and Dr. Walid Farhat for their constructive criticism. I would like to
thank Dr. Brian Carillo for his help in writing. I would like to thank Anwar Bari for helping me
process data. I would like to thank CIGITI members for their support.
I am grateful to my parents for always pushing me. I am grateful to my sisters and my
friends for their encouragement. I am grateful to God Almighty for giving me strength.
Page 4
iv
Table of Contents
Acknowledgments .......................................................................................................................... iii
Table of Contents ........................................................................................................................... iv
List of Tables ................................................................................................................................ vii
List of Figures .............................................................................................................................. viii
Chapter 1 ......................................................................................................................................... 1
1 Introduction ................................................................................................................................ 1
1.1 Liver anatomy ..................................................................................................................... 1
1.2 Liver carcinoma .................................................................................................................. 2
1.3 Imaging liver cancer ........................................................................................................... 3
1.3.1 Ultrasonography ...................................................................................................... 4
1.3.2 Computed Tomography .......................................................................................... 4
1.3.3 Magnetic Resonance Imaging ................................................................................. 5
1.3.4 Comparison of the imaging modalities ................................................................... 7
1.4 Role of imaging in interventions ......................................................................................... 8
1.5 Image registration ............................................................................................................... 9
1.6 Thesis outline .................................................................................................................... 12
Chapter 2 ....................................................................................................................................... 13
2 Previous Work and Research Objective ................................................................................... 13
2.1 Vessel Segmentation ......................................................................................................... 13
2.1.1 Active Contours .................................................................................................... 14
2.1.2 Region-Growing ................................................................................................... 14
2.1.3 Centerline Tracking .............................................................................................. 15
2.2 Vessel Matching ................................................................................................................ 16
2.3 Research Objective ........................................................................................................... 17
Page 5
v
Chapter 3 ....................................................................................................................................... 19
3 Algorithm for vessel segmentation .......................................................................................... 19
3.1 Vessel modeling ................................................................................................................ 19
3.1.1 Cross-Section ........................................................................................................ 20
3.1.2 Direction ............................................................................................................... 21
3.2 Centerline tracking ............................................................................................................ 23
3.2.1 Mathematical Representation of the Image .......................................................... 23
3.2.2 Vessel Center Likelihood ...................................................................................... 24
3.2.3 Generating Seed Point ........................................................................................... 25
3.2.4 Computing the Next Point .................................................................................... 26
3.2.5 Termination Condition .......................................................................................... 27
3.2.6 Radius Computation .............................................................................................. 28
3.3 Orientation Correction ...................................................................................................... 28
3.4 Bifurcation Correction ...................................................................................................... 29
Chapter 4 ....................................................................................................................................... 33
4 Experiments, Results and Discussion ...................................................................................... 33
4.1 Experiments ...................................................................................................................... 33
4.1.1 Image Acquisition ................................................................................................. 33
4.1.2 Guidance Model .................................................................................................... 33
4.2 Results and Discussion ..................................................................................................... 34
4.2.1 Subject A ............................................................................................................... 36
4.2.2 Subject B ............................................................................................................... 42
4.2.3 Subject C ............................................................................................................... 46
4.2.4 Subject D ............................................................................................................... 50
4.3 Accuracy ........................................................................................................................... 55
4.4 Speed ................................................................................................................................. 57
Page 6
vi
Chapter 5 ....................................................................................................................................... 58
5 Conclusion and Future Direction ............................................................................................. 58
5.1 Contribution of This Thesis .............................................................................................. 58
5.2 Other clinical applications ................................................................................................ 59
5.3 Future Work ...................................................................................................................... 60
References ..................................................................................................................................... 61
Page 7
vii
List of Tables
Table 1.1 The different phases of acquiring contrast enhanced images of the liver ....................... 5
Table 3.1 The description of the shape being described for various combinations of eigenvalues.
....................................................................................................................................................... 22
Table 4.1 MR images acquired using the following sequences and resolutions. ......................... 33
Table 4.2 Summary of the number of vessels segmented in each data set, the number of different
guidance models used on each image, and the total number of individual vessel segmentations 34
Table 4.3 Summary of accuracy and speed results of the algorithm. ........................................... 35
Page 8
viii
List of Figures
Figure 1.1 The vascular anatomy of the liver. ................................................................................ 2
Figure 1.2 Example of a workflow diagram of image-guided treatment administration. ............. 10
Figure 1.3 Huang et al.’s joint registration and segmentation method to iteratively obtain an
accurate registration. ..................................................................................................................... 11
Figure 3.1 Ray-casting to determine center of cross-section.. ...................................................... 24
Figure 3.2 Illustration of the centerline extraction process. .......................................................... 25
Figure 3.3 Tracking the next point in the algorithm. .................................................................... 27
Figure 3.4 Illustration of the centerline extraction process. .......................................................... 28
Figure 3.5 Points used to compute bifurcation point.. .................................................................. 30
Figure 3.6 Corrected centerlines (dark blue dots and lines) at a branching point, after the new
point of bifurcation (green square) has been computed. ............................................................... 32
Figure 4.1 An example of the vessel centerline segmentation results (green) compared with the
gold standard (red) for the image of subject A. ............................................................................ 36
Figure 4.2 The SE vs. ME for all vessel centerlines segmented for subject A (length) ............... 37
Figure 4.3 The SE vs. ME for all vessel centerlines segmented for subject A (radius)................ 38
Figure 4.4 Percentage of vessels with an average error in each category vs. maximum ME for
subject A. ...................................................................................................................................... 39
Figure 4.5 The SE vs. ME for all bifurcation points segmented for subject A.. ........................... 40
Figure 4.6 Percentage of bifurcation points with an average error in each category vs. maximum
ME for subject A. .......................................................................................................................... 41
Page 9
ix
Figure 4.7 An example of the vessel centerline segmentation results (green) compared with the
gold standard (red) for the image of subject B. ............................................................................ 42
Figure 4.8 The SE vs. ME for all vessel centerlines for segmented for subject B (length) .......... 43
Figure 4.9 The SE vs. ME for all vessel centerlines for segmented for subject B (radius). ......... 44
Figure 4.10 Percentage of vessels with an average error in each category vs. maximum ME for
subject B. ....................................................................................................................................... 45
Figure 4.11 An example of the vessel centerline segmentation results (green) compared with the
gold standard (red) for the image of subject C. ............................................................................ 46
Figure 4.12 The SE vs. ME for all vessel centerlines for segmented for subject C (length). ....... 47
Figure 4.13 The SE vs. ME for all vessel centerlines for segmented for subject C (radius).. ...... 48
Figure 4.14 Percentage of vessels with an average error in each category vs. maximum ME for
subject C. ....................................................................................................................................... 49
Figure 4.15 Subject D, a pediatric patient with a lesion in the liver. ............................................ 50
Figure 4.16 An example of the vessel centerline segmentation results (green) compared with the
gold standard (red) for the image of subject D. ............................................................................ 51
Figure 4.17 A close-up of the vessel centerline segmentation results (green) compared with the
gold standard (red) for the image of subject D. ............................................................................ 52
Figure 4.18 The SE vs. ME for all vessel centerlines for segmented for subject D (length). ....... 53
Figure 4.19 The SE vs. ME for all vessel centerlines for segmented for subject D (radius).. ...... 54
Figure 4.20 Percentage of vessels with an average error in each category vs. maximum ME for
subject D. ...................................................................................................................................... 55
Page 10
1
Chapter 1
1 Introduction
Liver cancer is one of the leading causes of cancer related deaths in the world. Medical
imaging plays not only a diagnostic role, but can also be used to more accurately guide
treatments. At heart of image-guided interventions is the registration of preoperative and
intraoperative images. Fast and accurate vessel segmentation is often a critical step in clinically
relevant registration methods. Many segmentation techniques have been previously explored, but
they have not been fast robust or automated. These problems could be overcome by making use
of a priori information about the vasculature. This thesis proposes a ridge-tracking algorithm that
makes use of a previously segmented image of the patient to segment all subsequent images.
The objective of the first chapter is to provide the necessary background on liver
anatomy, liver cancer and its diagnosis and treatment, and the importance of vessel segmentation
in guiding therapies to result in better patient outcomes.
1.1 Liver anatomy
The liver is the largest internal organ in the human body, weighing 1.2-1.5 kg [21]. A
vital organ necessary for survival, the liver plays a major role in metabolism, protein synthesis,
hormone production and detoxification. It is located inferior to the diaphragm. During normal
respiration the liver and other nearby organs such as the pancreas and kidneys undergo motion.
The liver chiefly undergoes motion in the craniocaudal direction [26]. As it is a soft tissue, it also
undergoes deformation, including compression, stretching and shear [27].
The liver is a vessel rich organ that normally receives 25% of cardiac output through a
dual blood supply. The hepatic artery supplies with oxygenated blood from the aorta, and about
25% of the total blood volume received by the liver. The other 75% of the blood volume is
venous blood from the pancreas, intestines, and spleen supplied via the portal vein. The blood
leaves the liver through the hepatic venous system, which is typically made of three major veins:
the right, the middle and the left [22].
Page 11
2
Figure 1.1 The vascular anatomy of the liver. The inferior vena cava (IVC) gives rise to the
right (RHV), middle (MHV) and left (LHV) hepatic veins. The portal vein (PV) and the
hepatic artery (HA) enter the liver alongside the common bile duct (CBD). The PV
bifurcates into right (RPV) and left (LPV), as does the HA (RHA and LHA) [54].
1.2 Liver carcinoma
There are two types of liver cancer, primary and metastatic. Together, they are one of the
leading causes of cancer death worldwide, being the second most frequent cause of cancer death
in men, and the sixth most frequent in women. In 2008, 695,900 deaths were attributed to liver
cancer, while another 748,300 patients were diagnosed with it. Primary liver cancer represented
the majority of liver cancer cases worldwide [30]. However, in the United States and Europe,
malignancies in the liver are mostly due to metastatic deposits originating in other organs [29].
Page 12
3
Primary liver malignancies include hepatocellular carcinoma (HCC), hepatoblastoma and
cholangiocarcinoma, although HCC constitutes for about 85% of all primary malignancies. HCC
most often occurs in patients with cirrhosis, Hepatitis B and C infections, and history of alcohol
abuse. Such patients are often regularly screened for HCC. When the diagnosis of the disease is
done as a result of screening, the prognosis is better than when it occurs only after a patient
presents the symptoms of the disease, such as jaundice and abdominal pain. Yuen et al. showed
that 26.8% of patients diagnosed as a result of regular screening were eligible for curative
resection as opposed to only 7.9% of those presenting symptomatic disease [30].
Besides tumors that originate in the liver, tumors can metastasize to the liver from other
organs. In fact, after the lymph nodes, the liver is the organ most frequently involved in organ
metastases from tumors in the breast, colon, lung, pancreas and stomach [28]. There are at least
four reasons for this. Firstly, the dual blood supply of the liver provides metastatic cells with two
ports of entry. Secondly, the long, slow and tortuous character of hepatic microcirculation
provides circulating cancer cells with access to all parts of the liver and may facilitate their
mechanical arrest there. Thirdly, the liver contains rich cell populations that normally serve to
restore hepatic tissue under various pathophysiological circumstances; these same cells can be
exploited for growth by cancer cells. Finally, the local production of anti-inflammatory and
immune suppressing factors, which normally prevent damage to liver parenchyma, can also
contribute to the tolerance of cancer metastasis [31]. As in primary liver cancer, resection of the
tumor(s) is widely considered to be the only potentially curative treatment, although less than
20% of patients are eligible for such treatment [40].
1.3 Imaging liver cancer
Various medical imaging modalities are used to visualize the structure, function and
pathology of the liver. Medical imaging is critical in diagnosing liver cancer, determining the
treatment options for the patient, planning the treatment, and, more recently, in guiding the
intervention [8]. It is also important as a follow-up procedure during chemotherapy [34].
Imaging of the liver presents certain challenges. First, the liver is usually moving due to
respiration. It can be temporarily immobilized via a breath-hold, and the imaging modality must
complete acquisition within that time. Second, both benign and malignant lesions are common in
Page 13
4
the liver, and the imaging modality must provide enough information to distinguish between
them.
Common modalities to image the liver are ultrasound, computed tomography and
magnetic resonance imaging. In all three imaging modalities, a contrast agent is often
administered intravenously to improve image quality. The acquisition is timed with agent’s
arrival into the relevant vasculature [23]. In the context of the liver, the agent travels first to the
hepatic artery, then the hepatic and portal vein systems and then may travel into the interstitial
fluid depending on whether it can pass out of the vasculature.
1.3.1 Ultrasonography
Ultrasonography (US) is an imaging technique that transmits high frequency sound
waves through the body, and captures their echoes to create an image. The modality is
inexpensive, safe, widely used in radiology and employs relatively small and portable
machinery. Conventional, gray-scale US is considered a “non-specific technique” in the
diagnosis of focal liver lesions (FLL). While tissue harmonic imaging, which exploits the non-
linear propagation of a sound-wave, significantly improves the spatial and contrast resolution of
conventional US, the detection rate of FLLs is only slightly improved. The use of Doppler US,
which can provide a quantitative measure of blood flow, is also limited since metastases are
overall poorly vascularized; although Doppler US can detect vascular invasion [34].
Two methods can significantly extend the role of US in liver cancer diagnosis: contrast
agents and intraoperative imaging. US contrast agents are commonly microbubbles, consisting of
gas in a protein, polymer or lipid shell, and can increase the contrast of vasculature. When the
agent is predominantly in arteries, it can be useful in imaging metastases, which, while overall
poorly vascularized, receive almost all their blood from the hepatic artery. Intraoperative, or
invasive US, can also significantly improve liver cancer diagnosis as the ultrasound probe can be
brought to the surface of the liver. This is useful because it can overcome the limited depth of
penetration of ultrasound due to attenuation and artifacts caused by tissue covering the liver.
1.3.2 Computed Tomography
Computed Tomography (CT) provides significantly better image quality than US and is
one of the most widely used imaging techniques. CT delivers electromagnetic radiation in the
Page 14
5
range of 100 eV and 100 keV (“x-rays”) to the body and measures its attenuation. The
attenuation depends on the composition and density of tissue it along its beam path. Since it is
impossible to determine the exact 3-D volume based simply on a single 2-D projection image,
CT takes multiple images from different angles of the same volume and then reconstructs the
volume based on algorithms. The radiation used by CT is ionizing, and therefore potentially
harmful, limiting CT’s usefulness as a modality, particularly in pediatric patients.
CT is currently the most common imaging modality for detecting and characterizing liver
lesions. The development of multi-detector CT has allowed for a high resolution whole liver scan
to be acquired in 5-10 seconds. CT scans can be taken with or without a contrast agent. Scans
without a contrast agent (also called “precontrast”) are useful in showing calcified and
hemorrhagic metastases [34]. Scans taken with an iodinated agent intravenously injected can
improve the contrast between healthy and diseased tissue. There are three phases of contrast-
enhanced liver imaging, depending on how long after the injection the liver is imaged: arterial-
phase, portal-venous phase, and the equilibrium phase. Table 1.1 summarizes the location of the
agent at the time of acquisition and the type of tissue enhanced in each phase [33].
Table 1.1 The different phases of acquiring contrast enhanced images of the liver
Phase Acquisition time
(after contrast
injection)
Location of
contrast agent
Tissues enhanced
Arterial 15-25 seconds Hepatic arteries Hypervascular
malignancies (lesions
predominantly supplied
by the hepatic artery)
Portal-venous 40-70 seconds Hepatic and portal
veins
Liver parenchyma,
excluding hypovascular
lesions
Equilibrium 80-120 seconds Interstitium Edematous tissues, e.g.
neoplasms and inflamed
areas
1.3.3 Magnetic Resonance Imaging
Magnetic Resonance Imaging (MRI) is an imaging technique that exploits the nuclear
magnetic resonance (NMR) of hydrogen atoms inside the body. NMR is the phenomenon where
nuclei of certain atoms can respond to magnetic fields and create magnetic fields of their own. In
this model, nuclei are like bar magnetics that are always spinning, and their axis of rotation
Page 15
6
normally points in an arbitrary direction [35]. Hydrogen nuclei, which make up the majority of
atoms inside the body, are especially magnetically susceptible. In MRI, the axes of rotation of
hydrogen nuclei are first aligned in the same direction using a strong static magnetic field. A
radiofrequency (RF) magnetic wave then deflects the axes of rotation (or simply “axes”) of the
hydrogen nuclei, causing them to tip into the transverse plane. The frequency of the RF wave
must be the same as the hydrogen nuclei’s Larmor frequency, which is the rate at which they
spin. As the axes of the nuclei then begin to “relax”, or return themselves to the original
alignment with the strong magnetic field, their magnetic field is measured. The strength of the
field is dependent upon the amount of hydrogen nuclei (“proton density”), which varies for
different tissue types, and gives the “image intensity”.
The nuclei relax in two primary ways: T2 and T1. When the nuclei axes are deflected,
they tip, generating a magnetic field transverse to the static magnetic field. T2 relaxation is the
exponential decay of this signal, which occurs because the precession is slightly different for
different nuclei and becomes incoherent over time. T1 relaxation is the phenomenon whereby the
deflected nuclei axes re-align themselves with the strong magnetic field. T2 and T1 relaxation
times vary for different tissues and can give excellent soft tissue contrast.
An MR dataset is typically acquired one slice at a time, where a slice is the plane normal
to the anteroposterior axis (defined in the thesis as the “z-axis”). In order to “select” a slice in the
body, a 1-D linear magnetic field gradient is applied during the period that the RF pulse is
applied, causing the magnetic field to vary linearly with the position on the z-axis. Since the
Larmor frequency is proportional the magnetic field, and because nuclei will only deflect if their
Larmor frequency is the same as the RF wave frequency, the position on the z-axis can be
selected by choosing the frequency of the RF wave.
While the z-axis information is determined before deflecting the axes, the x- and y-axis
information is determined during relaxation. A 1-D linear magnetic gradient is applied along the
x direction, increasing the precession where the magnetic field is stronger. Over time this results
in the phase progressively shifting along the x direction, which implies a particular spatial
frequency (depending on the strength and duration of application of the gradient). Applying a
similar gradient in the y direction gives spatial frequencies in that direction; together these
frequencies form “k-space”. Taking the 2-D inverse Fourier transform of k-space then gives the
Page 16
7
image slice. The maximum spatial frequency sampled in k-space corresponds with the x and y
resolutions (“in-slice resolution”), while the distance between slices selected corresponds to the z
resolution (“slice thickness”). The order in which the deflections and gradients are applied
(“pulse-sequence”) affects the duration of acquisition.
Recent advances in both hardware and software have allowed for fast pulse sequences
that are suitable for liver imaging, including fast dynamic, parallel, and echo-planar imaging.
More recently, parallel acquisition techniques have been introduced, which have the potential of
reducing the acquisition time 2-4 fold [34], at the expense of poorer signal to noise and
reconstruction artifacts.
There are three different types of MRI sequences of the liver that are commonly acquired:
T2-weighted, T1-wieghted, and contrast enhanced Gradient Echo (GRE). T2-wieghted sequences
predominantly provide information on the fluid content, fibrotic tissue and iron content of
tissues. T1-wieghted images are very useful in lesion characterization. Much like US and CT
images, MRI images can be contrast enhanced; gadolinium chelate is typically used as a contrast
agent [34]. As shown in Table 1, the time of acquisition determines the type of tissue that is
maximally contrasted.
1.3.4 Comparison of the imaging modalities
US, CT and MRI are all commonly used to image the liver with similar sensitivity in
detecting liver cancer [34]. US is inexpensive and can be rapidly acquired. CT is more expensive
than US, but less so than MRI, and can also be rapidly acquired. However, CT exposes the
patient to potentially harmful ionizing radiation. MRI is expensive with a relatively long
acquisition time, but has few known harmful effects. CT and MRI also both allow for multi-
plane reformatting and 3-D display; this allows surgeons to plan for surgical resections.
MRI offers better image quality than CT and both offer superior image quality than US.
US images suffer primarily from poor contrast and effective resolution. US also has a limited
depth of penetration, this is especially true if higher frequencies are used to achieve a higher
resolution. CT images, while better than US images, don’t give as good a liver parenchyma to
lesion contrast as MRI [34]. Also, while both CT and MRI rely on contrast agents for improved
Page 17
8
image quality, MRI contrast agents like gadolinium are not as often contraindicated as the
iodinated contrast agents used for CT [33].
For the above reasons, MRI is probably one of the best imaging modalities for the liver
due to its safety and image quality, with great potential for the future.
1.4 Role of imaging in interventions
Surgical resection is often considered the “only curative”… Patients can be considered
ineligible for surgical resection either due to the extent of disease or poor medical fitness or both
[46]. For those who are ineligible for resection, systematic chemotherapy is currently the only
significant treatment. However, more local, but invasive, treatments have recently been
developed. In radiofrequency ablation (RFA), alternating electric current is delivered to the
tumor via a needle, heating it to 50-100°C, which causes coagulation and tumor necrosis [47].
Other invasive methods for ablation include laser-induced thermotherapy, where laser energy is
deliver via inserted optical fibers, and microwave ablation, where microwaves (a type of
electromagnetic radiation) are delivered via a microwave antenna inserted into the tumor [48].
An example of a local but non-invasive treatment is focused ultrasound surgery (FUS), where
tumor temperature is increased by delivering mechanical energy via high intensity ultrasonic
waves. Because it is a noninvasive form of treatment, the energy is localized using some sort of
intraoperative imaging modality (usually MRI). In spite of challenges, including the fact that the
liver is often behind ribs which attenuate ultrasound, initial results are promising [49]. A far
more common non-invasive local treatment is radiation therapy.
Intraoperative imaging (i.e. images acquired during intervention), is increasing used in
the treatment of liver cancer. It is an essential part of interventions such as radiation therapy [36],
radiofrequency ablation [38] and high frequency ultrasound (HIFU) ablation, where the
interventions are either non-invasive or minimally invasive, and intraoperative imaging is
necessary to direct treatment to diseased tissue while sparing healthy tissue. Intraoperative
imaging is also increasingly used in surgical resection of the liver tumors [37], where it can
increase the accuracy of the resection, leading to clear tumor margins, improved outcomes, and
an increased number of patients eligible for resection [38].
Page 18
9
However, there are inherent limitations to intraoperative imaging. Due to time constraints
associated with a single breath-hold (reportedly 12-16 seconds [57]), and the duration of the
treatment, it is logistically difficult to complete a comprehensive liver scan intraoperatively.
Furthermore, intraoperative imaging is usually limited to non-contrast enhanced images, and
does not embed the clinician’s treatment plans. For this reason, using the information provided
by pre-operative images is necessary to guide the intervention.
In order to use the pre-operative images, they must be mathematically mapped onto the
intraoperative images in what is known as registration. From this mapping, a real-time update of
the location of intervention can be displayed in reference to the detailed pre-operative images
and plans [39]. This real-time information can be used to much better improve accuracy of the
treatment. Figure 1.1 shows a workflow diagram of how preoperative and intraoperative imaging
would be incorporated to deliver therapy. It could apply to, for example, radiation therapy or MR
guided ultrasound ablation.
1.5 Image registration
In the context of liver cancer treatment, the image registration must take into account not
just liver motion, but also liver tissue deformation, and be rapid enough to be implemented in the
workflow outlined above.
Most deformable image registration methods can be divided into two categories:
intensity-based and feature based. Intensity-based methods, including Fourier methods and
Mutual Information (MI) methods, correlate the voxel intensities in both images. Because these
methods take only voxel intensities into account, without any structural analysis, they are
susceptible to intensity changes caused by noise [11], and the presence of contrast in the
preoperative image. Since the method of acquisition of pre-operative and intra-operative images
is often different, these methods require the normalization of intensities and the resampling of
one image to the resolution of another [12].
Page 19
10
Figure 1.2 Example of a workflow diagram of image-guided treatment administration.
Preoperative images are acquired and the treatment is planned before the intervention,
which is boxed in blue. After the intervention is prepared, the treatment is administered
using image guidance, boxed in pink, as many times as necessary until the required dose
has been administered. In between the administrations the patient is freely breathing.
Feature-based methods attempt to match the corresponding features of both images, and
are faster and more robust than most other methods [13,14]. These methods’ speed advantage
over intensity-based methods stems from the fact that features are more sparse than voxels [5]. In
order for these methods to be accurate, the precise segmentation and correspondence of the
features they rely upon is critical. In the context of the liver, vessels are ideal features because
they are well distributed throughout the liver and capture its non-rigid transformations [7]. While
microvasculature may not be detectable in medical images, the larger vessels are quite
prominent. The registration would then use these more visible vessels as features to generate a
transformation.
Page 20
11
Recently, Huang et al. showed a joint registration and segmentation method where
vessels are iteratively registered and segmented, with each process exploiting the information
from the previous one [41] (see Figure 1.3). Osario et al. have presented a similar scheme [42].
These methods use corresponding vessel centerlines in both images, as opposed to segmented
vessel volumes, as features. This is because vessel centerlines, a 1-D curve in 3-D space, capture
the location of the vessel with far less information than vessel volumes, making registration
efficient. Because the registration aims to minimize the distance between features in both
images, correspondence between those features is necessary. This is not a trivial task considering
that vascular trees are dense and vessels look quite similar.
Figure 1.3 Huang et al.’s joint registration and segmentation method to iteratively obtain
an accurate registration. Vessel centerline extraction is a critical part of this method.
Page 21
12
Thus the segmentation of vessel centerlines in the liver allows for fast and accurate image
registration, which is a critical component of image-guided treatments of liver cancer.
1.6 Thesis outline
The remainder of the work is organized as follows.
Chapter 2 discusses the previous work done in rapidly segmenting vessel centerlines in
an image and establishing correspondence between two sets of centerlines. It identifies the
absence of a fast method that has been used to do both. Noticing that few of the previous
methods have used a priori information about the liver vasculature, the research goal is defined
as developing an algorithm that accurately and rapidly segments liver vessel centerlines using
patient specific models.
Chapter 3 presents the proposed algorithm to meet the research goal. It describes the
vascular model and centerline tracking scheme used to segment vessel centerlines, and the
method to correct centerline orientation.
Chapter 4 discusses the results of using the algorithm to segment centerlines in sets of
MR and CT images of healthy and diseased individuals. The strengths and weakness of the
accuracy and speed of the algorithm are discussed.
Finally, Chapter 5 summarizes the contribution of this thesis, suggesting future direction
for the work and presenting ideas on integrating this algorithm into clinical procedures.
Page 22
13
Chapter 2
2 Previous Work and Research Objective
As shown in the previous chapter, there is a demonstrated need for fast and accurate
segmentation of vessel centerlines in the liver. There also needs to be a correspondence between
two sets of segmented vessel centerlines so that the registration may determine a transform that
minimizes the distance between corresponding centerlines. In order to be clinically relevant the
speed of these methods is critical. Ultimately faster algorithms leave more time for registration,
image acquisition and treatment administration. This chapter looks at the previous work done in
the field before identifying the gap that this thesis aims to fill.
2.1 Vessel Segmentation
A preponderance work has been done in segmenting vessels in medical images. However,
much of the work is computationally intensive and/or requiring significant user interaction, and
therefore clinically irrelevant for rapid image guidance. A hallmark of most computationally
cheap segmentation techniques is that they explore the image only very sparsely. Another
important aspect of fast techniques is their parallelizability. This is because computationally
complex methods can sometimes be significantly speeded up using a GPU based
implementation. Erdt et al, for example, presented a vessel segmentation method where the
filtering step for a liver volume took 26-32s on a CPU, but only 1.6-2.0s with a GPU
implementation [45]. (They did not report the total times for the entire segmentation process).
Interestingly, Osario et al, the only other group to use segmentation in a joint
segmentation-registration framework, have reported a segmentation time of 3s per vessel, with
the total segmentation-registration time of the entire liver upwards of 22 minutes [42]. Their
segmentation involves first computing the likelihood of a vessel being present at each voxel, and
then skeletonizing likelihood images to generate centerlines, before implementing another
method to determine correspondence.
Most vessel segmentation can be categorized by their vessel extraction schemes as active
contours, region growing, or centerline-based methods [20].
Page 23
14
2.1.1 Active Contours
Active contours methods delineate the outline of the vascular region by evolving an
interface (contour in two dimensions, surface in three dimensions) to minimize its energy.
External energy is derived from the image, while the internal energy of an interface is based on
its geometry and regularity.
The deformable interface can be represented explicitly in a parametric form, also called
“snakes”. Segmentation using parametric contours is robust to image noise and gaps because it
constrains the interface to be smooth and can be simple to implement and computationally
efficient. However, it is topologically inflexible to splitting or merging of parts. Such topological
challenges can be handled using “level set” methods, which represent contours implicitly [50].
However, these methods are computationally intensive [20], making them unsuitable for online
use.
2.1.2 Region-Growing
Region-growing methods begin with at least some voxels identified inside a vessel
(“seeds”), and classify neighboring voxels as vessel or non-vessel, incrementally, until a
termination condition is met. Classic region growing approaches were voxel-wise processes that
were prone to false positives (“leaking”) and false negatives (“holes”). This can be rectified with
growth limiting criteria, or using region growing to segment both vessel and non-vessel voxels
(“competitive region growing”). The region being grown can be made spatially coherent by
imposing conditions on the interface of the region (which is similar to the active contours
approach). One way of doing that is through ordered region growing (ORG) approach, where
regions for inclusion are ranked in accordance to their potential effect on the geometry of the
growing front. Fast Marching methods, where voxels are visited based on their geodesic
distance from seeds, have been used to segment vasculature [20]. All region growing methods
give a true segmentation of the image, i.e. they classify the voxels into vessel and non-vessel.
However, as mentioned in 2.4, efficient registration requires that centerlines be computed from
the segmentation. The resulting skeletonization problem is not trivial as it can result in false
branches.
Yim et al. used ORG to segment vessels, followed by skeletonization to extract the
centerline, and achieved this in 10-20 s. In order to avoid false branches, the method required the
Page 24
15
user to input the maximum radius and pruned all branches smaller than that [51]. However, since
a liver data set would contain vessels that vary widely in radius, such an approach would require
significant user interaction increasing the actual time taken by the algorithm.
2.1.3 Centerline Tracking
A more straightforward approach to extracting vessel centerlines is by directly tracking
them, as in centerline-tracking approaches. These methods output focus on tracking the center
points of a vascular tree, not its entire volume. Just as region growing methods can be augmented
to later extract the centerline from vascular volumes, centerline tracking methods can be
followed by one of several methods to generate the vessel volume from its centerline: 2D cross-
section segmentation, 3D constrained active contours, or generalized cylinders [20].
Like region-growing, centerline-tracking requires initial seeds inside the vessel. Many
previous authors have relied on user specified seeds, making the speed of their methods
dependent upon user interaction. Some methods, not concerned with the processing time, use
seeds generated, more or less, arbitrarily [16]. Many of these seeds are later discarded if they are
determined to be false positives, or if they have been visited via tracking from a different seed
[52]. Another method is to select seeds only at branchpoints, which can be detected pre-
segmentation using a corner-detector algorithm, or during tracking as a result of cross-sectional
changes associated with passing a bifurcation [53].
Tracking methods rely on one or more models to determine direction and recenter each
tracked point in the vessel cross-section. There are three models that can be used: ridges, cross-
section, and tubular. Ridge-tracking assumes the center line of the vessel is a “ridge”. However,
at very low contrast, the ridge will be sensitive to noise. Cross-section tracking, which models
the vessel cross-section as an ellipse, is more suitable for such low contrast. But this method is
not suitable for small vessels (usually less than 3 voxels in diameter) [15]. By contrast, ridge
tracking is suitable for vessels as small as 1 mm in diameter [16]. Tubular models of blood
vessels have been developed to deal with both low contrast and small vessel diameter. These,
however, take several minutes to compute [15], and therefore unsuitable for applications that
require near real-time operation. Thus a combination of ridge and cross-section methods can
provide a robust and accurate model for the vessel, yet still be fast enough to be suitable for near
real-time applications.
Page 25
16
Aylward, in particular amongst ridge-tracking algorithms has developed a fast and
accurate ridge-tracking algorithm and optimized it using dynamic parameters [16]. The
algorithm further had the advantage of modeling the vessel as curving centerline, with varying
radii. This reduces the amount of data required to represent a vessel, therefore making feature-
feature registration more computationally efficient. However, the authors did not propose an
efficient method for defining a search space. The search space could either be defined by a user,
or arbitrarily the entire image. The former case reduces automation, and the latter case increases
computational complexity. In the latter case, Aylward’s method would also have to be followed
by a method that established the correspondence between the vessels segmented in the
intraoperative and preoperative images.
2.2 Vessel Matching
As mentioned in section 2.5, correspondence between features is necessary to obtain a
transformation. The problem is similar to the “assignment problem” in computer science: given
two sets (of possibly unequal sizes), assign each member in the smaller set exclusively to one
member in the larger set. A brute force approach could lead a combinatorial explosion. One
efficient approach is to use the Hungarian algorithm, where the cost is the linear combination of
Mahalanobis distance and orientation similarity [55]. However, the accuracy of this approach
depends on the initial alignment of the two sets, with a strong likelihood of the existence of local
minima in the cost, especially in dense vasculature. A more accurate alignment is likely to give
better results.
Thus, in the context of registration, the most robust methods solve for both the
correspondence and transformation, iteratively. The simplest of these methods is the iterative
closest point (ICP) method, whose steps can be:
1. Assign elements in one set to the elements in the second set based on the nearest
neighbor criteria,
2. Estimate a rigid transformation that would minimize the distance between
corresponding elements,
3. Apply the transformation to the moving set of elements, and start again at step 1.
Page 26
17
The above algorithm still has drawbacks: the transformation is not necessarily rigid and
ICP may assign two elements to one element. ICP also generates local minima [56]. More
advanced approaches use the Expectation Maximization (EM) algorithm, where E-step estimates
the correspondence and the M-step finds the transformation. Osario et al use the Thin-Plate
Spline Robust Point Matching (TPS-RPM) algorithm. The algorithm is similar to ICP above.
However, in step 1, each element in the smaller set is assigned in a “soft” or “fuzzy” manner to a
spread of elements in the larger set. The spread is determined by the “annealing temperature”,
which is reduced at each iteration. In step 2, a non-rigid TPS transformation is used. The original
authors of this method reported the worst case computational complexity at O(N3), where N is
the number of points in the sets. Osario et al reported the time taken by correspondence-
transformation iteration to be on the order of several minutes [42]. If vessel centerlines in a set of
two images are to be used as corresponding features, the correspondence problem must be
solvable in near real-time.
2.3 Research Objective
One common aspect of all the algorithms mentioned above is that they don’t take into
account that an image of the same patient may already have been segmented (either by the said
algorithm or another algorithm). Detailed information about the patient’s vasculature is readily
available, if it has been segmented in the preoperative image. Unlike the intraoperative image,
the time limits on segmenting the preoperative image are far more relaxed and the preoperative
image is usually studied carefully anyway by the radiologist. This thesis hypothesizes that this
information can be quite powerful, especially in solving the correspondence problem. Thus the
thesis proposes using a patient specific model to segment vessel centerlines in liver images.
The objective of the thesis is to develop an accurate, fast and automatic vessel
segmentation algorithm that exploits patient-specific knowledge of the vasculature. The
algorithm should segment the vessel centerlines with an error less than a voxel length. This is
because reducing the error beyond a voxel length does not necessarily have any clinical
advantages, as clinicians do not aim to resolve tumor boundaries or surgical instruments at the
subvoxel level. The percentage of vessels that need to be correctly segmented should be as high
as possible; however, since no previous work has used patient-specific models to segment vessel
centerlines rapidly, the number achieved by this thesis will set the benchmark for future work.
Page 27
18
The algorithm should execute in less than 2s, the lower the better. The reason for such a short
window is because breath-holds are typically 12-16 seconds[57], during which the image must
be acquired and registered and the treatment administered. Finally, the algorithm should execute
automatically, since there would not be enough time for user interaction. The first image taken
after the patient is in the image scanner may require a rough initial alignment from the user,
which is often the case in clinical protocols.
Page 28
19
Chapter 3
3 Algorithm for vessel segmentation
This thesis proposes an algorithm that exploits the vascular segmentation of one image to
segment the vessels in another image of the same patient. The proposed algorithm takes as input
the intraoperative image (called the “image” in this section) in which vessels are to be segmented
and the vessel centerlines segmented in the preoperative image (called the “guidance model” in
this section). The algorithm outputs the vessel centerlines in the intraoperative image. The
algorithm assumes that the guidance model has been registered onto the image with some degree
of accuracy, and thus both are in the same coordinate frame of reference. For each centerline in
the guidance model, the algorithm searches for the corresponding centerline in the current image.
The segmentation of the centerline is initialized by determining an appropriate seed point
using the guidance model. It is then tracked point by point, and the radius is calculated at each
point. The tracking terminates once the length of the current vessel matches that of the guidance
model. Finally, the algorithm checks the segmented centerline to ensure it matches the guidance
model. If not, the algorithm backtracks and re-segments. Upon segmentation, the algorithm
corrects the bifurcations of the segmented tree of vessel centerlines.
3.1 Vessel modeling
Selecting appropriate models for vessel cross-section and direction are important to
ensure accurate and robust tracking on MR images of the liver vasculature.
The following notations are used in this chapter, given a point P in a 3-D image:
is used to denote a point in 3-D image coordinates
is the intensity of the image at point
Page 29
20
is the Hessian matrix at point and scale σ
are the eignevalues of H, where
are the respective eignevectors of , and
3.1.1 Cross-Section
The vessel cross-section is theoretically a sharp edged circle or ellipse (anatomically this
corresponds to the fluid being surrounded by an endothelium). However, the edge often appears
slightly blurred in MR images, due to three artifacts.
Firstly, due to the partial volume effect, if a voxel straddles the boundary of a vessel, part of
the voxel will generate a vessel signal, while the other part will generate a non-vessel signal.
These will often average out and give the impression of blurring at the edge. This would also
apply to CT images.
Secondly, because of slice selection, as discussed in section 1.3.3, only nuclei in a slice have
their axes deflected. Since there is time between this axis deflection and the measuring of the
signal, some of the nuclei are inevitably replaced by non-deflected nuclei via blood flow. Due to
the no-slip condition of fluid flow, where flow at the boundary is assumed to be 0, the blood flow
is slower at the boundary than at the center of the vessel, making this effect the most pronounced
in the center of the cross-section [58]. Note that this results in the vessel cross-section being a
region of low intensity, which would be then used in the current algorithm by inverting the
intensity values.
Thirdly, the blurring could be the result of windowing in the k-space, which, for practical
reasons, can’t be sampled infinitely. The 1-D vessel cross-section and it’s k-space representation
can be written as, respectively,
and
. . After
applying windowing, the k-space representation becomes
,
Page 30
21
and the cross-section is imaged in 1-D as
, where W is the
width of the window.
Previous literature has sometimes modeled the cross-section as a 2-D Gaussian distribution.
This was not observed in the MR images this thesis examined, nor could it be explained in terms
of the artifacts above. However, two important features were observed on the cross-section:
1. The center portion of the vessel cross-section has a higher intensity than the peripheral
regions; the center portion could be a single voxel or several voxels wide, depending
on the size of the vessel cross-section.
2. While the intensity drops off from the center portion of the vessel cross-section to the
background, the decrease is the sharpest at the edge of the vessel cross-section.
These two features will be exploited later in the chapter to determine the center of the vessel
cross-section and the radius.
3.1.2 Direction
In order to model the vessel direction, first consider the neighborhood of an image point.
+
If the neighborhood is approximated by the Taylor series expansion up to the second derivative,
the following is obtained,
Page 31
22
The second coefficient of the final term in the above equation is the Hessian matrix,
which takes into account the second order intensity variation [24]. In fact, if the eigen-
decomposition of the Hessian matrix gives three distinct eigenvalues ( , the
corresponding eigenvectors (
are orthogonal. Each eigenvalue represents the sign and
magnitude of the second derivative in the direction of the corresponding eigenvector. Also recall
that the second derivative of a graph is negatively correlated with its curvature (where concave
up is defined as positive). So, for example, the eigenvector associated with the largest eigenvalue
would represent the direction in which the curvature (where concave up is positive) of the
intensity is the highest. Similarly, different possibilities of describe various structures,
described in the table below. Eigenvalues much larger than zero are not considered as those
would give rise to concave up curvatures, which would result in low intensity structures in the
image, whereas vessels in MR images are of a high intensity.
Table 3.1 The description of the shape being described for various combinations of
eigenvalues.
Description
≈ 0 ≈ 0 ≈ 0 Noise or uniformly bright/dark
<< 0 ≈ 0 ≈ 0 A flat “sheet” or “plate”
<< 0 << 0 ≈ 0 A “tube” or vessel
<< 0 << 0 << 0 An ellipsoid or “blob”
Page 32
23
From Table 3.1 it can be seen that the eigen-decomposition of the Hessian computed near
the center of a vessel would result in ≈ 0, << , and ≈ . The last condition assumes
that if the cross-section of the vessel is approximated as an ellipse, the length of the major axis
would be similar to the minor axis. Thus, and
are the vectors representing directions of
high magnitude of (negative) curvature, which corresponds to the cross-section, whereas the
direction of lowest magnitude of curvature would correspond to the vessel direction.
3.2 Centerline tracking
The centerline of the vessel is tracked as a three dimensional intensity “ridge”, adapting a
similar tracking method described by Aylward [16]. The starting point of the tracking, the seed,
is derived from the guidance model. Parameters of scale and step size are calculated dynamically
based on radius and curvature, respectively. Radius calculation is based on maximizing the sum
of gradients along the circumference of a circle [17].
The assumption that the centerline of the blood vessels is an intensity ridge is justified as
it has been shown in 3.1.1 that the center of the vessel cross-section is higher in intensity than the
edges.
3.2.1 Mathematical Representation of the Image
Mathematically the vessel centerlines constitute a three-dimensional ridge. When viewed
on a cross-section, or a 2-D plane orthogonal to the direction of the vessel, the ridge appears as a
local maximum. If a Hessian matrix is computed at a voxel on this ridge, two of its eigenvalues
will be negative, the eigenvector corresponding to the largest eigenvalue will be tangent to the
direction of the vessel, and the other two eigenvectors will form a plane orthogonal to the
direction of the vessel. The Hessian matrix can be calculated by convoluting the image with the
first and second derivatives of a Gaussian, which is mathematically equivalent to taking the
derivative of the image after convolving with a Gaussian. The elegance of using the derivatives
of the Gaussian to compute the Hessian is that convolving with a Gaussian will reduce noise and
generate maximum intensity at the center of the cross-section, given the correct scale.
Page 33
24
3.2.2 Vessel Center Likelihood
In 4.1.1, it was shown that the vessel cross-section is blurred for several possible reasons.
When the vessel diameter is less than 10 voxels, it has been observed that the point of maximum
intensity is close to the center. Otherwise, however, the central portion of the cross-section is of
uniform intensity (meaning the 1-D cross-section profile of the vessel has a plateau rather than a
peak). Because of noise, the point of maximum intensity is often not close to the center. In that
case vessel center likelihood is determined using a method adapted from Wink et al [18].
The likelihood of a point being at the
center of the cross-section is computed by casting four pair of opposite rays from it at angles 0˚
and 180˚, 45˚ and 225˚, 90˚ and 270˚, and 135˚ and 315˚. The ray computes the 1-D intensity
gradient at each point; the point of minimum gradient is assumed to be the boundary of the
vessel cross-section. To avoid false positives due to noise, the minimum gradient must be lower
than -30. If the point is at the center of the vessel cross-section then its distance to the edge of the
vessel should be equal in opposite directions.
Formally, let Q be the point under consideration, and be the distance between Q and
the boundary of the vessel along the bth
ray of the ath
pair of rays (see Figure 3.1). The likelihood
that point Q is the center of the vessel is,
Figure 3.1 Rays are casted from the point
Q (yellow) in eight directions. Each ray
halts at the point the intensity gradient is
minimum, with the length of the ray
indicating the distance between Q and the
vessel edge. Opposite rays of similar
length indicate a high likelihood of Q
being the center of the cross-section. In
this image, Q is off-center, yielding some
asymmetric pairs of rays.
Page 34
25
It can be seen that this approach is more computationally intensive than simply finding a
maximum, and thus is only used for vessels with diameter equal to or larger than 10 voxels,
where there is observed to be a difference between center point and point of maximum intensity
of the cross-section.
3.2.3 Generating Seed Point
The starting point of the current vessel is determined using the starting point of its
guidance vessel. As shown in Figure 3.2, the guidance model (red) is superimposed on the
current image, via coarse registration. The starting point of the guidance vessel (G1) is placed in
the current image, and a number of particles are uniformly generated, using a Mersenne-Twister
distribution, in a cylinder with a normal in the direction of the guidance vessel (blue crosses in
figure 3). Each particle, S, is assigned a score based on the following equation,
where G1 and G2 are points from the guidance vessel as shown in Figure 3.2. The numerator
ensures a vessel in the current image with orientation closest to the guidance vessel is selected,
while the denominator ensures that distant vessels in the current image oriented in the same
direction as the guidance vessel are not matched. The numerator is squared to give orientation
matching more weight than distance. The particle with the highest score is chosen as the seed.
Figure 3.2 Illustration of the centerline extraction
process. Points G1, G2 are on the guidance model
(red line) The blue ‘+’ points are the seed
candidates generated around G1, with direction
associated with each candidate indicated. The seed
candidate with the highest score, a function of the
direction it is associated with, is indicated in light
blue.
Page 35
26
The seed is then refined by searching for a local maximum of intensity along a straight
line starting at the seed and point in the direction of the gradient at the seed. This refined seed is
the first centerline point and should approximately be located on a ridge. After this first point is
computed, each subsequent point of the centerline is found by ridge tracking.
3.2.4 Computing the Next Point
Using the eigenvectors of the Hessian computed at a given point, PG, the next point is
first estimated, PN1, and then refined iteratively as PN2 and PN3. The next point is estimated using
the following equation
where the magnitude of t is the desired “step-size” for vessel tracking. Through testing, 0.4 times
shortest voxel length was found to work well as the step-size. If vector is pointing in the
direction of the previous point (i.e. pointing “backwards”) the value of t is negative, otherwise t
is positive. To compute PN2, a plane is defined as the following set of points, where is the
search radius.
The refined point is taken to be the point in with the highest intensity,
To compute PN3, a plane is defined as the following set of points,
Once again the refined point is taken to be the point in with the highest vessel center
likelihood,
Page 36
27
Essentially the algorithm is creating a plane of the vessel cross-section at a given point,
shifting the plane in the direction of its normal, and finding the point with the maximum intensity
at this shifted plane, as depicted in Figure 3.3. This is similar to the pseudo-arc length numerical
continuation method, where a solution curve is parameterized in terms of its arclength, the next
solution is predicted by its tangent at a known solution point, and the next solution is “corrected”
by solving the set of equations that define the curve (this is usually approximated using a variant
of Newton’s method) [43, 44].
3.2.5 Termination Condition
Just as the algorithm needs a place to start tracking, it needs to know when to stop.
Currently it is proposed that the vessel extraction be terminated when the length of the current
vessel reaches that of the guidance vessel. After extracting each centerline, the points are
averaged using an eleven point long Gaussian-weighted window:
The first and last points are anchored while this smoothing is applied to prevent vessel
centerline shrinkage. This had the effect of smoothing the points, eliminating curvatures in the
final vessel due to inaccuracies, while preserving genuine curves in the vessel structure.
Figure 3.3 The tracking algorithm first
computes point B by adding A and , which is
a scalar times the eigenvector corresponding to
the largest eigenvalue, where the Eigenmatrix is
computed at point A. C is taken as the point of
maximum vesselness in the plane normal to .
D is taken as the point of maximum vesselness
in the plane normal to .
Page 37
28
3.2.6 Radius Computation
To model the vessel as a tubular structure, both information about its centerline and its
radius at each centerline point are needed. To compute the radius at each point, an estimate is
needed. For the seed point of the current vessel, the radius is estimated to be that at its guidance
vessel seed. For all subsequent points the estimate is the computed radius at the previously
tracked point.
The algorithm computes the optimal radius, , by maximizing a medialness function
over radii values close to the estimated radius, ,
For practicality, discrete values of r, spaced at 0.1 times the voxel size is considered. This
requires linearly interpolating the intensity data. A convenient medialness function, M, has been
formulated by Pock et al. [17] to optimize the radius, r, at the refined point, ,
3.3 Orientation Correction
Often vessel tracking encounters an unknown bifurcation and may track the “wrong” one
of the two possible vessels. The algorithm evaluates every tracked segment to check if it has
tracked a “wrong” segment. If so, it backtracks and restarts the tracking from seed generation.
While tracking the current vessel centerline, every 4 mm of the current vessel centerline
is designated a segment (e.g. in Figure 3.4, segment from T1 to T2, or from T2 to T3). The
Figure 3.4 Illustration of the centerline
extraction process. The red line is the
guidance model, and points G1 to G4
delineate segments. The teal square points
are those segmented by the algorithm, and
points T1 to T4 delineate the tracked
segments.
Page 38
29
guidance vessel centerline is treated similarly (e.g. in Figure 3.4, the portion of the centerline
from G1 to G2 is a segment). Furthermore, correspondence is established between segments of
the current vessel and guidance vessel by equating the distance of both segments to the
respective starting point of the centerline.
A current vessel segment is classified as “correct” if it meets the following criterion,
where = 0.65, corresponding to a maximum deviation angle between the corresponding
segments of ~50˚. This value was found to give the best result on the data tested.
If the segment is not correct, then the algorithm will eliminate that segment and backtrack
to the last good segment. It will attempt to regenerate it by placing the seeds in the same fashion
as described in 5.2.2. The model vessel segment corresponding to the “wrong” vessel serves as
the starting model segment. However, if the newly tracked segment again doesn’t satisfy the
relationship above, the tracking of the vessel terminates. It is better to produce no information on
a vessel than the wrong information.
3.4 Bifurcation Correction
Vessel centerlines, upon segmentation, often do not give accurate bifurcations. As can be
seen Figure 3.5, the centerlines bisect earlier than one would expect the bifurcation of the
vasculature. While the definition of bifurcation can be subjective, it should correspond to the
geometry of the vasculature and be robust to small changes in the centerline segmentation. For
this reason, the definition proposed by Piccinelli et al. was used [19]. It is a weighted average of
four points on the centerlines, reducing the impact of a single point on the bifurcation location.
Because the points are on the centerline and because it also uses the radius computed at the
centerline, it is robust to small changes in one measure that are not reflected in the other
measure.
Page 39
30
Figure 3.5 Centerlines A and B are shown with their respective “tubes” shown as the
shaded regions (pink and blue). Points and
(black) are on the edges of the red and
blue circles, respectively, while and
(dark blue and dark red) are at the centers of
the two respective circles.
Assuming the centerline of a vessel is given, and its radius is known at each centerline
point, a “tube” can be constructed as shown for two vessels in Figure 3.5. and radii of a vessel at
each point. Given centerlines A and B, the following four points are defined:
1. and
are the points on centerlines A and B, respectively, that intersect tubes B
and A, respectively.
2. and
are points on centerlines A and B, respectively, whose distance from the
edge of the tube is the same as the distance from points and
, respectively. In
Page 40
31
other words, these points are the centers of maximally inscribed spheres within tubes
A and B, respectively, that touch points and
, respectively.
The bifurcation point is then the weighted average of the four points:
where is the radius associated with point j on centerline i.
After the correct bifurcation is computed, the centerlines are re-generated in the vicinity
of the bifurcation. Because the correct bifurcation point is always farther from the root than the
bifurcation of the centerlines, part of the parent centerline has to be reconstructed, while the
children centerlines need to be modified to be connected to the new branchpoint. An example of
the results is shown in Figure 3.6. The new lines are reconstructed by fitting a second order
polynomial 3-D function to the following points:
For reconstructing the parent, the points used are: the new branchpoint, the old
branchpoint, the last few points of the parent centerline
For reconstructing each of the children, the points used are: the new branchpoint,
the first few points of the child centerline
If enough points are not available for a second order fit, a first order fit is generated.
Page 41
32
Figure 3.6 Corrected centerlines (dark blue dots and lines) at a branching point, after the
new point of bifurcation (green square) has been computed. The uncorrected centerlines
(cyan dots and lines) had an incorrect point of bifurcation (red square). The blue centerline
between the red and green squares is generated by fitting a second order polynomial to the
two squares, and points on the blue line preceding the red square. Similarly the lines
between the green square and orange circles are regenerated by fitting a second order
polynomial to the shapes and some points outside the region. In the rare case when enough
points were not available, a first order polynomial was fit.
Page 42
33
Chapter 4
4 Experiments, Results and Discussion
The above described algorithm was tested on MR and CT images of the livers of several
humans, including a patient with liver cancer. The results are that the algorithm can accurately
segment vessel centerlines.
4.1 Experiments
4.1.1 Image Acquisition
Dynamic MR images of three human volunteers (subjects A, B and C) were acquired in
the axial plane using a 1.5T GE scanner (GE Medical Systems, Milwaukee, WI). All three
images were from black blood pulse sequences and the acquisition details are shown in table 4.1.
A set of two images was acquired for each subject with a breath-hold at exhalation in the same
imaging session. The variance in MR images was due to the variance in the duration the
volunteers held their breath; the longer the breath-hold the finer the resolution of the acquired
image.
In addition, one set of two CT images (resolution 0.355 x 0.355 x 2.5mm) of a pediatric
patient (subject D) with liver cancer were used to evaluate the performance of the algorithm on a
liver image with a lesion, especially in the vicinity of the lesion.
Table 4.1 Three MR images were acquired using the following sequences and resolutions.
Subject Pulse sequence TR TE Flip angle Slice thickness In-plane resolution
A LAVA gradient echo
3.79ms 1.72ms 12˚ 1.5 mm 1.3 mm x 1.3 mm
B DwiSE/SE 1.3s 34ms 90˚ 5.0 mm 1.95 mm x 1.95 mm
C DwiSE/SE 1.6s 34ms 90˚ 3.0 mm 1.95 mm x 1.95 mm
4.1.2 Guidance Model
To prepare the guidance model, the volume of the liver was manually segmented and
processed in Slicer 3.0. As a pre-processing step, the image was contrast-enhanced using the
Page 43
34
Vascular Modeling Toolkit (VMTK) Vessel Enhancement tool (steps: 10, diameter: 1.0-2.5
physical units, plate and line-line structures 0.5, blob-like structures: 0.5, contrast: 60). The
image was segmented using Easy Level Set tool (inflation: 10, curvature: 70, attraction to ridges:
60, iterations: 10). The centerlines were extracted using the Centerline Extraction tool. Because
this tool generates overlapping centerlines, in-house software separated the centerlines to what
we call “segmented format” for the guidance model. In this format, a vessel is defined from one
bifurcation point up to the next bifurcation point. The model was stored as a .vtk file with the
centerline points with corresponding radii at each point. As VMTK produces incorrect
bifurcations, and because correct bifurcations are necessary for a good guidance model,
centerline bifurcations had to be corrected as described in 3.4.
For each set of images, one image was registered to the second image using different
types of registration with a different set of parameters each time. Each registration was applied to
the guidance model associated with the set of images to bring it into the coordinate system of the
image to be segmented. The objective was to produce a wide variety of guidance models with
which to test the algorithm.
Table 4.2 Summary of the number of vessels segmented in each data set, and the number of
different guidance models used on each image, and thus the total number of individual
vessel segmentations
Subject Vessels segmented in each
segmentation
Number of segmentations Total number of vessels
segmented
A 102 10 1020
B 15 3 45
C 12 5 60
D 35 5 175
The results of the centerline extraction method described above were evaluated against a
“gold standard”. The gold standard was constructed in the same way as the guidance model was.
An example of the results of the segmentation and the gold standard are shown in Figure 4.1.
4.2 Results and Discussion
To quantitatively evaluate the extraction accuracy, the segmentation error (SE) of each
vessel centerline was calculated between segmented vessels and the gold standard, by averaging
Page 44
35
the shortest distance between the two centerlines for each point on a segmented centerline. The
model error (ME) was calculated similarly, finding the average distance between the gold
standard and the model centerlines from which the segmentation is computed. The performance
of the algorithm is heavily dependent on the model it is given: a vessel centerline generated from
a model closer to the gold standard is more accurate than one generated from a model centerline
much farther than the gold standard. Therefore SE was plotted against ME for each subject and
horizontal lines are drawn to indicate the voxel length (taken to be the largest dimension of the
voxel) for the set of images.
Every vessel segmentation is classified in terms of its average error compared to the
voxel length (e.g. 1 voxel < SE < 2 voxels). As ME increases, the proportion of vessels in each
category of error is expected change. The “maximum model error” refers to the set of all models
whose error is less than or equal to a particular model error. (For example, “90% of vessels had
error < 1 voxel at maximum model error of 2.5 mm” means that of the segmentations
corresponding to all models with an average error ≤ 2.5 mm, the SE is less than a voxel length
90% of the time). The measure of maximum model error is relevant because it imposes a
constraint on the accuracy of the registration portion of the joint segmentation-registration
method which is one of the main applications of this segmentation algorithm.
Times were measured while running on a computer running Microsoft Windows XP with
a 3.2-GHz i3 processor and 3.5 GB of memory, and included the entire time required for
segmenting all vessels in the image.
Table 4.3 Summary of accuracy and speed results of the algorithm. *Features refer to
vessels in all cases except the one where it refers to bifurcation points.
Subject Average error of all features*
when maximum model error is: Percentage of features* with average error < voxel length,
when maximum model error is:
Average time to run each
segmentation
2.0 mm 4.6 mm 2.0 mm 4.6 mm
A 0.97 mm 1.28 mm 88% 79% 35.2s A (for
bifurcation points)
0.81 mm 1.01 mm 95% 89%
B 1.67 mm 2.03 mm 100% 98% 9.8s
C 1.68 mm 2.11 mm 93% 87% 5.2s
D 0.73 mm 1.17 mm 96% 82% 44.7s
Page 45
36
A summary of the accuracy and speed are given in Table 4.3. Maximum model errors of
2.0 mm and 4.6 mm were used as they were quoted by Osario et al[42] and Huang et al[41],
respectively, as the maximum error for their registration algorithms in their joint-segmentation
registration framework. For subject A, additional analysis was performed for measuring the
accuracy of bifurcation points instead of the entire centerline. This analysis was not performed
for other data sets, as the number of bifurcation points was very small.
4.2.1 Subject A
Figure 4.1 An example of the vessel centerline segmentation results (green) compared with
the gold standard (red) for the image of subject A.
Data from subject was comprehensive in terms of vascular coverage, with 102 vessels
from the hepatic and portal system. Figure 4.1 shows that all the vessels were successfully
segmented in most cases. Figure 4.2 shows that SE is less than a voxel length for low ME.
Visually, there is no clear pattern of vessel length on SE or ME, but it appears that vessels with a
larger radius may have both smaller ME and SE (Figure 4.3). Figure 4.4 confirms the effect of
Page 46
37
increasing maximum ME on decreasing accuracy. While the only category relevant to the goal of
the thesis is SE < voxel length, the other categories are shown to illustrate the distribution of
error.
The figures mentioned above all show the average error of vessel centerlines. Figure 4.5
displays the error of the bifurcation points only. SE for bifurcation point was measured as the
distance between the segmented bifurcation point and the bifurcation point in the gold standard;
ME was the distance between the bifurcation point in the model and that in the gold standard.
The SE of the bifurcation is comparably less than that of vessel centerlines. This is likely
because the location of the bifurcation point is influenced most heavily by the parent centerline
(since the algorithm segments from the root in the direction of the leaves). Parent vessels will
Figure 4.2 The SE vs. ME for all vessel centerlines segmented for subject A. Each blue
point represents the average error of a vessel centerline with a particular length. The
total number of points on the graph is 1020. The horizontal lines are at intervals of 1.5
mm (the voxel length for this data set).
Page 47
38
have a larger radius than the children vessels, and Figure 4.3 shows that vessel with larger radii
tend to have lower ME and SE. Figure 4.6 confirms that bifurcation SE is less than the vessel
centerline SE, showing a higher percentage of vessel centerline whose SE < voxel length.
Figure 4.3
Figure 4.3 The SE vs. ME for all vessel centerlines segmented for subject A.
Each blue point represents the average error of a vessel centerline with a
particular radius. The total number of points on the graph is 1020.The
horizontal lines are at intervals of 1.5 mm (the voxel length for this data set).
Page 48
39
Figure 4.4 Percentage of vessels with an average error in each category vs. maximum ME
for subject A.
Page 49
40
Figure 4.5 The SE vs. ME for all bifurcation points segmented for subject A. Each blue
point represents the error of a bifurcation point. The total number of points on the graph is
420. The horizontal lines are at intervals of 1.5 mm (the voxel length for this data set).
Page 50
41
Figure 4.6 Percentage of bifurcation points with an average error in each category vs.
maximum ME for subject A.
Page 51
42
4.2.2 Subject B
Figure 4.7 An example of the vessel centerline segmentation results (green) compared with
the gold standard (red) for the image of subject B.
In the dataset from subject B, 15 vessels were segmented, including those from the
hepatic and portal systems. Figure 4.7 shows that most vessels were accurately segmented. From
Figure 4.8 it can be seen that most segmented centerlines have error less than 5mm, the voxel
length. This is confirmed by the distribution in Figure 4.10. Figures 4.8 and 4.9 don’t show any
vessel length or radius related pattern.
Page 52
43
Figure 4.8 The SE vs. ME for all vessel centerlines for segmented for subject
B. Each blue point represents the average error of a vessel centerline with a
particular length. The total number of points on the graph is 45. The
horizontal lines are at intervals of 5 mm (the voxel length for this data set).
Page 53
44
Figure 4.9 The SE vs. ME for all vessel centerlines for segmented for subject
B. Each blue point represents the average error of a vessel centerline with a
particular radius. The total number of points on the graph is 45. The
horizontal lines are at intervals of 5 mm (the voxel length for this data set).
Page 54
45
Figure 4.10 Percentage of vessels with an average error in each category vs. maximum ME
for subject B.
60
65
70
75
80
85
90
95
100
1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5
Perc
enta
ge o
f ve
ssel
s in
err
or
cate
gory
Maximum model error
Effect of maximum model error on segmentation error
>3 voxel lengths
2-3 voxel lengths
1-2 voxel lengths
0-1 voxel lengths
Category of error
Page 55
46
4.2.3 Subject C
Figure 4.11 An example of the vessel centerline segmentation results (green) compared with
the gold standard (red) for the image of subject C.
12 vessel centerlines from the hepatic system were segmented in the image from subject
C. A large number of vessel centerlines had SE < voxel length, especially for ME < 5mm. This is
confirmed in figure 4.14. Figures 4.12 and 4.13 do not reveal any length or radius related
patterns.
Page 56
47
Figure 4.12 The SE vs. ME for all vessel centerlines for segmented for subject C.
Each blue point represents the average error of a vessel centerline with a
particular length. The total number of points on the graph is 60. The horizontal
lines are at intervals of 3 mm (the voxel length for this data set).
Page 57
48
Figure 4.13 The SE vs. ME for all vessel centerlines for segmented for subject
C. Each blue point represents the average error of a vessel centerline with a
particular radius. The total number of points on the graph is 60. The
horizontal lines are at intervals of 3 mm (the voxel length for this data set).
Page 58
49
Figure 4.14 Percentage of vessels with an average error in each category vs. maximum ME
for subject C.
Page 59
50
4.2.4 Subject D
Figure 4.15 Subject D was a pediatric patient with a lesion in the liver.
The objective of testing the CT image of subject D, a pediatric cancer patient, was to
demonstrate that the algorithm could segment vessel centerlines even in the presence of a lesion.
While the focus of the thesis was to segment MR images, it can also be applied to CT images.
Figure 4.15 shows the lesion in the axial view. Figure 4.16 shows that the algorithm was able to
accurately segment 35 vessels. Figure 4.16 shows a close-up of the segmentation, revealing that
segmentation is accurate even when the vessel is near or partially inside the lesion volume.
Figures 4.18 and 4.19 show that most vessel centerlines have an average SE <voxel length and
this is confirmed by Figure 4.20.
LiverLi
ne of
lowest
possibl
e
Differe
ntial
Error
LesionSe
gmentati
on Error
> Model
Error
Page 60
51
Figure 4.16 An example of the vessel centerline segmentation results (green) compared with
the gold standard (red) for the image of subject D. The volume of lesion is shown in blue,
along with a slice of the image.
Page 61
52
Figure 4.17 A close-up of the vessel centerline segmentation results
(green) compared with the gold standard (red) for the image of subject
D. The centerlines indicated by yellow arrows are either partially inside
the lesion volume or in close proximity to it. The volume of lesion is
shown in blue, along with a slice of the image.
Page 62
53
Figure 4.18 The SE vs. ME for all vessel centerlines for segmented for
subject D. Each blue point represents the average error of a vessel
centerline with a particular length. The total number of points on the graph
is 175. The horizontal lines are at intervals of 2.5 mm (the voxel length for
this data set).
Page 63
54
Figure 4.19 The SE vs. ME for all vessel centerlines for segmented for subject
D. Each blue point represents the average error of a vessel centerline with a
particular radius. The total number of points on the graph is 175. The
horizontal lines are at intervals of 2.5 mm (the voxel length for this data set).
Page 64
55
Figure 4.20 Percentage of vessels with an average error in each category vs. maximum ME
for subject D.
4.3 Accuracy
In all of the segmentation results, the number of vessel centerlines with an average error
of less than a voxel length was singled out. This is because accuracies better than the image
resolution would be clinically irrelevant as it is difficult to resolve parameters such as tumor
margin, needle or surgical tool position at a subvoxel level. It is also challenging to segment
vessel centerlines at a subvoxel level without some modeling of vascular anatomy and MRI
physics that is beyond the scope of this thesis. All other errors represent room for improvement
in the accuracy of the algorithm.
There are several explanations for error greater than voxel length. The first is that in a
small number of cases the “gold standard” is actually inaccurate. Since the gold standard was
Page 65
56
generated using 3D Slicer and in the absence of easy editing tools in the software, there was no
quick and easy way of fixing this.
The second was the problem of correspondence. There were several instances of the
algorithm tracking the “wrong vessel”, with the orientation correction aspect of the algorithm
failing. This happened if the guidance model was closer to the incorrect vessel than to the vessel
that was meant to be segmented. Sometimes, if there was no vessel within the model’s search
space, the algorithm would instead tracked noise or other nonvascular structures. Had the
algorithm been able to predict vascular density and model error, it would be better able to define
a search space and thus avoid tracking the “wrong vessel”.
The third source of error was due to not finding the center of the cross-section. Neither of
the models discussed in section 3.2.2 were robust enough to always ensure that the center was
found accurately. There are a lot of models in the literature, many computationally efficient,
which should be explored in the future for use in this algorithm.
The fourth problem was that orientation correction was intended to detect tracking the
“wrong vessel” and correct the problem close to where it started spatially. However, in practice
this often happened a few millimeters downstream of when the problem occurred leading to
portions of the centerline that were completely incorrect. This problem could potentially be
solved by detecting bifurcations using the shape of the cross-section. So once the “wrong vessel”
has been detected, the algorithm would backtrack all the way to the last bifurcation found, which
would better localize where the “wrong vessel” starts.
The accuracy was seen to remain consistent with vessel length. There was, however, a
slight trend noticed where vessels with a larger radius had lower ME and SE than those with
smaller radius. This can be explained in several ways. Since vessels with larger radius represent
those closer to the root, the vascular density in their vicinity is lower, reducing the chances of
tracking something else. That the ME is lower for these cases suggests that the registration may
be yielding closer models for thicker vessels than thinner ones. This could be because thicker
vessels are more prominent features that would influence registration more than thinner ones.
The accuracy that this thesis deals with is dependent upon the registration error, with
segmentation error worsening with worsening model error. The segmentation works relatively
Page 66
57
well for registration errors less than 5mm. Thus, the segmentation errors were given in terms of a
registration error. This allows the thesis to evaluate the accuracy of the segmentation
independent of the registration. However, understandably clinicians would be interested in the
final accuracy of the entire joint segmentation registration framework. Unfortunately, since the
registration portion of the framework was not available, the final accuracy couldn’t be tested and
is left as future work.
4.4 Speed
The execution of the algorithm took 35 seconds on the most comprehensive MRI dataset.
The goal was for the algorithm to execute in less than 2 seconds. However, the algorithm does
have the potential to execute within that time frame for several reasons.
The first reason is that the algorithm developed is highly parallelizable – the extraction of
one vessel centerline is totally independent of all others and can thus be done in parallel.
Previous work has shown that algorithms can be speeded up to 15 times with a GPU
implementation [45]. Therefore it is quite possible that the entire segmentation can be completed
in 2 seconds.
Secondly, the implementation wasn’t optimized. 40% of the time was consumed by a
single inefficient class that computed the Hessian over the entire image. The area explored for
vessel segmentation, however, is a small fraction of the entire image. Therefore an
implementation that computes the Hessian only as necessary could reduce the computation
significantly.
Finally, implementing the algorithm on a more high performance computer would
increase the speed.
Ultimately, the goal of the algorithm is to be used in a segmentation-registration method
to find a transformation for the preoperative image into the intraoperative image space. Thus, the
accuracy and speed that matter the most need to be evaluated from the segmentation-registration
method as a whole.
Page 67
58
Chapter 5
5 Conclusion and Future Direction
Image-guided procedures are a promising avenue to treat liver cancer as they can
improve accuracy of treatments, and therefore patient outcomes. In the context of the liver, a
rapid registration of the preoperative and intraoperative images is required as treatments are
often administered within a single breath-hold. Feature-based registration is robust and
computationally efficient and uses corresponding features in both images to find an accurate
registration. Vessels are a well-distributed feature in the liver. The problem to be solved is the
fast and accurate segmentation of vessel centerlines in an intraoperative liver images and
corresponding them with vessel centerlines in a preoperative liver image.
Previous work in vessel segmentation has often been intended for offline use. While there
are fast vessel segmentation algorithms, they do not necessarily establish correspondence.
Solving the correspondence problem after the segmentation is computationally expensive. This
thesis approaches the problem with the aim of exploiting the information in the preoperative
image to segment the intraoperative image.
5.1 Contribution of This Thesis
The algorithm developed by this thesis project uses a previous segmentation of the vessel
centerlines in another image and superimposes it upon the image to be segmented (usually after
some sort of registration). This guidance model is used to generate centerline seeds, which are
then tracked until termination to generate vessel centerlines. If the wrong centerline is tracked,
the guidance model can be used to detect that and correct the orientation in order to track the
correct centerline. This elegantly solves the correspondence problem at the time of segmentation,
whereas previous works have treated the correspondence problem as one to be solved post-
segmentation.
The algorithm successfully segmented centerlines accurately and rapidly on four sets of
images. If the registration was constrained to 2.0mm, the average segmentation error was 0.73-
1.68mm, with 88-100% of the vessels having an error less than a voxel length. If the registration
Page 68
59
error was less than 4.6mm, the average segmentation error was 1.17-2.11mm, with 79-98% of
the vessels having an error less than a voxel length.
5.2 Other clinical applications
There are many clinical applications to the work done in this thesis, beyond those that
have been discussed. The first possible expansion of the application of the segmentation
algorithm is to other organs. While the aim of the thesis was to segment the vessels of the liver,
the proposed algorithm is not specific to liver vessels, and can, in fact, be applied to the
segmentation of vessels in other vasculature rich organs such as the brain, kidneys and lungs.
Secondly, the segmentation only clinical application of segmentation discussed above was
registration. While this is indeed an important application, there are two other applications of
vessel segmentation: computer aided diagnosis and treatment planning.
Computer Aided Diagnosis (CAD) systems aim to detect, classify and quantify certain
tissues (including lesions) in 3D medical images and thereby estimate the probability of a disease
in a given patient [59]. These systems can save the time and labor required to interpret medical
images and increase the accuracy of detecting abnormalities. The segmentation of vessels in the
medical images can be an important step in CAD [60]. For example, hepatic vessel segmentation
in CT images is an important part of the CAD of hepatocellular carcinoma [60]. Another
example is the CAD of pulmonary emboli in CT pulmonary angiography images, which depends
on the segmentation of pulmonary vessels to define its search space [61]. In breast tumors, the
delineation of vessels is an important step since vessels can display similar contrast
enhancements as the tumors [62]. Thus, the proposed vessel segmentation could potentially be
used to aid CAD algorithms.
Treatment planning is an important part of intervention, and helps ensure effectiveness
and minimizes the complications that may result; planning is necessary to consistently achieve
high quality results [63]. Vessel segmentation is often a key part of this planning because it
allows the clinician to visualize the vasculature and perform quantitative analysis. For example,
in liver surgery, vessel segmentation enables the surgeon to visualize the vasculature, which is an
important part of the preoperative planning [64]. Since vessel segmentation also quantifies the
location and size of vessels, it is used to characterize the importance of certain vessels in
supplying and draining tissue, which in turn is used to compute the volume of tissue remaining
Page 69
60
post-resection [65]. The segmentation of vessels can also be used to visualize cerebral
vasculature and plan the path of tumor biopsies or the placement of electrodes deep in the brain
for epilepsy evaluation [66]. The visualization can also assist in selecting the best treatment (e.g.
surgery, radiosurgery, embolization, etc.), evaluating surgical risk and localizing lesions [67].
Thus, the segmentation of vessels has applications in treatment planning.
5.3 Future Work
The algorithm needs further improvement in accuracy and speed before it can be tested in
the clinical context. Changes to the algorithm to improve accuracy are proposed in section 4.3.
As discussed in section 4.4, the algorithm is highly parallelizable, and major improvements in
speed could be achieved by implementing it using parallel programming.
After improvements in speed and accuracy have been achieved the algorithm needs to be
tested within a suitable joint segmentation registration framework to determine the actual error in
registering the intraoperative image to the preoperative one. In the thesis, the error is presented
as a function of registration error; however, clinicians are more concerned with the final
registration error.
After the joint segmentation registration method has been tested, it could be tested as part
of clinical trials. These trials would validate the method, and also elicit useful feedback from
clinicians. This would improve the method so that ultimately it can improve treatment accuracy,
creating better outcomes for patients living with liver cancer.
Page 70
61
References
[1] K. S. Lehmann et al. “Portal Vein Segmentation of a 3D-Planning System for Liver
Surgery—In vivo Evaluation in a Porcine Model,” Annals of Surg. Oncology, vol. 15, no. 7, pp.
1899-1907, 2008.
[2] M. Lin et al. “Algorithm-Based Method for Detection of Blood Vessels in Breast MRI for
Development of Computer-Aided Diagnosis,” Jour. Of Magn. Resonance Imaging, vol. 30, pp.
817-824, 2009.
[3] M. Descoteaux, D. L. Collins, K. Siddiqui. “A geometric flow for segmenting vasculature in
proton-density weighted MRI,” Medical Image Analysis, vol. 12, pp. 497-513, 2008.
[4] Q. Shang et. al. “Adaptive Directional Region Growing Segmentation of the Hepatic
Vasculature,” Proc. Of the Soc. Of Photo-Optical Instr. Eng., vol. 6914, pp. F9141, 2008.
[5] S. Kawajiri et al. “Automated segmentation of hepatic vessels in non-contrast X-ray CT
images,” Radio. Physics and Tech, vol. 1, no.2, pp. 214-222, 2008.
[6] C. Zhou et al. “Automatic multiscale enhancement and segmentation of pulmonary vessels in
CT pulmonary angiography images for CAD applications,” Med. Physics, vol. 34, no. 12, pp.
4567–4577, 2007.
[7] X. Zhu. “Voles: Vascularity-Oriented Level Set Algorithm For Pulmonary Vessel
Segmentation In Image Guided Intervention Therapy,” IEEE Inter. Symp. on Biomedical Imag. -
From Nano to Macro, pp. 1247-1250, 2009.
[8] I. Bankman, Handbook of Medical Imaging: Processing and Analysis Management, San
Diego, CA: Academic Press, 2000.
[9] B. J. Erickson and B. Bartholmai. “Computer-Aided Detection and Diagnosis at the Start of
the Third Millennium,” Journal of Digital Imag, vol. 15, no. 2, pp. 59-68, 2002.
Page 71
62
[10] S. P. Edwards. “Computer-Assisted Craniomaxillofacial Surgery,” Oral and Maxillofacial
Surgery Clinics of North America, vol. 22, no. 1, pp.117-134, 2010.
[11] N. Shin et al. “Computer-assisted Surgical Planning for Cerebrovascular Neurosurgery,”
Neurosurgery, vol. 41, no. 2, pp. 403-410, 1997.
[12] P. Markelj et. al. “A review of 3D/2D registration methods for image-guided interventions,”
Medical Image Analysis, vol. 16, pp. 642-661, 2012.
[13] M. Groher et al, “Planning and Intraoperative Visualization of Liver Catherizations: New
CTA Protocol and 2D-3D Registration Method,” Academic Radiology, vol. 14, no. 11, pp. 1325-
1340, 2007.
[14] B. Zitova and J. Flusser, “Image registration methods: a survey,” Image Vis. Comput. Vol.
21, no. 11, pp. 977–1000, 2003.
[15] O. Friman, M. Hindennach, C. Kühne. “Multiple hypothesis template tracking of small 3D
vessel structures,” Medical Image Analysis, vol. 14, pp. 160-171, 2010.
[16] S. R. Aylward. “Initialization, Noise, Singularities, and Scale in Height Ridge traversal for
Tubular Object Centerline Extraction,” IEEE Transactions on Medical Imaging, vol. 21, no. 2,
2002.
[17] T. Pock, C. Janko, R. Beichel and H. Bischof. “Multiscale Medialness for Robust
Segmentation of 3D Tubular Structures,” Proceedings of the Computer Vision Winter
Workshop, 2005.
[18] O. Wink and W. J. Niessen. “Fast Delineation and Visualization of Vessels in 3-D
Angiographic Images,” IEEE Trans. On Medical Imag., vol. 19, no. 4, pp.337-346, 2000.
[19] M. Piccinelli et al. "A Framework for Geometric Analysis of Vascular Structures:
Application to Cerebral Aneurysms," IEEE trans. on medical imaging, vol. 28, no. 8, pp. 1141-
1155, Aug. 2009.
[20] D. Lesage et. al. “A review of 3D vessel lumen segmentation techniques: Models, features
and extraction schemes,” Medical Image Analysis, vol. 13, pp. 819-845, 2009.
Page 72
63
[21] S. Sherlock and J. Dooley, “Anatomy and function,” in Diseases of the Liver and Biliary
System. John Wiley & Sons, 2008, pp.1-4.
[22] L. Schlossberg and G. Zuidema, “The Liver,” in The Johns Hopkins Atlas of Human
Functional Anatomy. JHU Press, 1997, pp. 137-141.
[23] L. DeLeve and G. Garcia-Tsao, Vascular Liver Disease: Mechanisms and Management,
[24] A. Frangi et al, “Multiscale Vessel Enhancement Filtering,” Lecture Notes in Comput. Sci.,
vol. 1496, pp. 130-137, 1998.
[25] Y. Sato et al, “Three-dimensional multi-scale line filter for segmentation and visualization
of curvilinear structures in medical images,” Medical Image Analysis, vol. 2, no. 2, pp. 143-168,
1998.
[26] L. Cervino and S. Jiang, “Intrafraction Variations and Management Technologies” in
Adaptive Radiation Therapy, USA: CRC Press, 2011, pp. 204.
[27] P. Villard et al, “A Method to Compute Respiration Parameters for Patient-based
Simulators,” Medicine Meets Virtual Reality, no. 173, pp. 529-33, 2012.
[28] A. Khan et al (2012). Liver Metastases Imaging [Online]. Available:
http://emedicine.medscape.com/article/369936-overview
[29] M. Oliva and S. Saini, “Liver cancer imaging: role of CT, MRI, US and PET,” Cancer
Imaging, vol. 4, Spec No A, pp. S42–S46.
[30] A. Jemal et al. “Global Cancer Statistics,” CA Cancer J Clin, vol. 61, no. 2, pp. 69-90,
March/April 2011.
[31] N. Reau and F. Poordad, Primary Liver Cancer: Surveillance, Diagnosis and Treatment,
New York: Springer, 2012, pp. 1-12.
[32] F. Vidal-Vanaclocha, “Architectural and Functional Aspects of the Liver with Implications
for Cancer Metastasis,” in Liver Metastasis: Biology and Clinical Management, Springer, 2011,
pp. 9-42.
Page 73
64
[33] Advanced liver diagnosis: Computed Tomography (CT) [Online]. Available:
http://www.liver-imaging.com/scripts/pages/en/liver_imaging/computed_tomography_ct/
[34] M. El Khodary et al, “Imaging of Hepatic Metastases,” in Liver Metastasis: Biology and
Clinical Management, New York: Springer, 2011, pp. 307-352.
[35] A. Berger, “How does it work?: Magnetic Resonance Imaging,” BMJ, vol. 324, no. 7328,
pp. 35, 2002.
[36] K. Brock, “Imaging and Image-Guided Radiation Therapy in Liver Cancer,” Semin
Radiation Onc, vol. 21, no. 4, pp. 247-255, 2011.
[37] P. Dumpuri et al, “Model-updated image-guided liver surgery: Preliminary results using
surface characterization,” Prog Biophys Mol Biol., vol. 103, no. 2-3, pp. 197-207, 2010.
[38] H. Rempp et al, “Image-Based Monitoring of Magnetic Resonance-Guided Thermoablative
Therapies for Liver Tumors,” Cardiovasc Intervent Radiol, vol. 35, no. 6, pp. 1281-94, Dec.
2012.
[39] D. Cash, “Concepts and Preliminary Data Toward the Realization of Image-guided Liver
Surgery,” Journ Gastrointest Surg, vol. 11, no. 7, pp. 844-59, 2007.
[40] D. Ribero, “Epidemiology and Natural History,” in Surgical Treatment of Colorectal Liver
Metastases, Milan: Springer, 2011.
[41] Huang et al. “Extraction of liver vessel centerlines under guidance of patient-specific
models,” in Engineering in Medicine and Biology Society, San Diego, CA, 2012.
[42] Osario et al. “Accurate CT/MR vessel-guided nonrigid registration of largely deformed
livers,” Med Phys, vol. 39, no. 5, May 2012.
[43] H. Keller, Lectures on Numerical Methods in Bifurcation Theory, Bangalore: Tata Institute
of Fundamental Research, 1986, pp.66.
[44] C. Kelley (2005). Arclength Continuation and Bifurcation[Online]. Available:
http://www4.ncsu.edu/~ctk/MA784/continue.pdf
Page 74
65
[45] Erdt et al. “Automatic Hepatic Vessel Segmentation Using Graphics Hardware,” Lec. Not.
Comp. Sci., vol. 5128, pp. 403-412, 2008.
[46] A. Muratore and L. Capussotti. “Evolution of Resectability Criteria,” in Surgical Treatment
of Colorectal Liver Metastases, Turin: Springer, 2011, pp. 28.
[47] R. Cirocchi et al. “Radiofrequency ablation in the treatment of liver metastases from
colorectal cancer,” Cochrane Database Syst. Rev., no.6, Jun 13, 2012.
[48] T. Vogl et al. “Thermal ablation therapies in patients with breast cancer liver metastases: A
review,” Eur. Radiol, vol. 23, no. 3, pp. 797-804, 2013.
[49] K. Hynynen. “MRIgHIFU: A Tool for Image-Guided Therapeutics,” Journ. Magn.
Resonan. Imag., vol. 34, pp. 482-493, 2011.
[50] S. Weeratunga and C Kamath. “An Investigation of Implicit Active Contours for Scientific
Image Segmentation,” Vis Comm Image Process Conf, San Jose, CA, 2004.
[51] P. Yim et al. “Gray-Scale Skeletonization of Small Vessels in Magnetic Resonance
Angiography,” IEEE Trans. Med. Imag., vol. 19, no. 6, 2000.
[52] J. Tyrrell et al. “Robust 3-D Modeling of Vasculature Imagery Using Superellipsoids,”
IEEE Trans. Med. Imag., vol. 26, no. 2, 2007.
[53] Y Fridman et al. “Segmenting 3D Branching Tubular Structures Using Cores,” MICCAI
2003, LNCS, vol 2879, pp. 570–577, 2003.
[54] O. Catalano et al. “Vascular and Biliary Variants in the Liver: Implications for Liver
Surgery,” Radiographics, vol. 28, no. 2, pp. 359-78, 2008.
[55] E. Serradell et al. “Simultaneous Correspondence and Non-Rigid 3D Reconstruction of the
Coronary Tree from Single X-ray Images,” Intern Conf on Comp Vision, 2011.
[56] H. Chui and A. Rangarajan. “A new point matching algorithm for non-rigid registration,”
Comp. Vis. Image Underst., vol. 89, pp. 114-141, 2003.
Page 75
66
[57] P. Giraud and A. Houle. “Respiratory Gating for Radiotherapy: Main Technical Aspects and
Clinical Benefits,” ISRN Pulmonology, vol. 2013, id. 519602, 2013.
[58] C. Ethier and C. Simmons. “Hemodynamics,” in Introductory Biomechanics: From cells to
Organisms, Cambridge: Cambridge University Press, 2007, p.136.
[59] B. J. Erickson and B. Bartholmai. “Computer-Aided Detection and Diagnosis at the Start of
the Third Millennium,” Journal of Digital Imag, vol. 15, no. 2, pp. 59-68, 2002.
[60] S. Kawajiri et al. “Automated segmentation of hepatic vessels in non-contrast X-ray CT
images,” Radio. Physics and Tech, vol. 1, no.2, pp. 214-222, 2008.
[61] C. Zhou et al. “Automatic multiscale enhancement and segmentation of pulmonary vessels
in CT pulmonary angiography images for CAD applications,” Med. Physics, vol. 34, no. 12, pp.
4567–4577, 2007.
[62] M. Lin et al. “Algorithm-Based Method for Detection of Blood Vessels in Breast MRI for
Development of Computer-Aided Diagnosis,” Jour. Of Magn. Resonance Imaging, vol. 30, pp.
817-824, 2009.
[63] S. P. Edwards. “Computer-Assisted Craniomaxillofacial Surgery,” Oral and Maxillofacial
Surgery Clinics of North America, vol. 22, no. 1, pp.117-134, 2010.
[64] Q. Shang et. al. “Adaptive Directional Region Growing Segmentation of the Hepatic
Vasculature,” Proc. Of the Soc. Of Photo-Optical Instr. Eng., vol. 6914, pp. F9141, 2008.
[65] K. S. Lehmann et al. “Portal Vein Segmentation of a 3D-Planning System for Liver
Surgery—In vivo Evaluation in a Porcine Model,” Annals of Surg. Oncology, vol. 15, no. 7, pp.
1899-1907, 2008.
[66] M. Descoteaux, D. L. Collins, K. Siddiqui. “A geometric flow for segmenting vasculature in
proton-density weighted MRI,” Medical Image Analysis, vol. 12, pp. 497-513, 2008.
[67] N. Shin et al. “Computer-assisted Surgical Planning for Cerebrovascular Neurosurgery,”
Neurosrugery, vol. 41, no. 2, pp. 403-410, 1997.