AUTOMATIC DETECTION AND CORRECTION OF ...mlai.cs.huji.ac.il/publications/Leo_Joskowicz/files...similar. Intensity-based graph-cut methods produce leaks be-cause the min-cut is unlikely

AUTOMATIC DETECTION AND CORRECTION OF SEGMENTATION LEAKS IN MEDICALIMAGES

Achia Kronman1 Leo Joskowicz1,2 Jacob Sosna3

1 School of Engineering and Computer Science, The Hebrew University of Jerusalem, Israel.2 The Edmond and Lily Safra Center for Brain Sciences - ELSC.

3 Dept. of Radiology, Hadassah Hebrew University Medical Center, Jerusalem, Israel.

ABSTRACT

We present a new method for the detection and correction ofsegmentation leaks in volumetric medical images. Segmen-tation leaks occur when the segmentation volume expandsoutside the actual target structure boundary into neighboringstructures. Our method identifies the segmentation leak ba-sis boundary by gradient magnitude likelihood classification,fits a surface to the leak basis boundary, and separates the leakfrom the target structure after finding their common boundary.Our method is automatic, does not rely on prior information,and is independent of the initial segmentation method. Ex-perimental evaluation on 139 segmentations of kidneys andabdominal aortic aneurysms from CT scans from two com-mon segmentation methods yield an improvement of 56.8%(std=15.4) and 49.4% (std=19.5) in the mean surface distanceand the volume overlap error between the initial and the cor-rected segmentations.

Index Terms— Segmentation errors, detection, correc-tion, kidney, abdominal aortic aneurysm.

1. INTRODUCTION

Image segmentation of volumetric medical images is a keytask in the generation of patient-specific models of anatomicalstructures and pathologies. The models play a central role inall aspects of patient care, from the initial diagnosis throughthe planning, delivery, and evaluation of patient treatment.

Although many segmentation methods have been devel-oped for various anatomical structures and imaging modal-ities, only few are in clinical use. Indeed, segmentation is avery challenging task due to: 1) great image variability result-ing from the wide range of imaging devices, imaging proto-cols, imaging inhomogeneity, and imaging artifacts; 2) largeintra and inter patient anatomical variability, and; 3) inten-sity values overlap between structures tissues and the prox-imity between them. Experience shows that each anatomicalstructure, pathology, and imaging modality has unique char-acteristics that cause significant segmentation errors in a non-negligible number of cases. Correcting faulty segmentationsrequires extensive user interaction, trial-and-error parameter

(a) initial segmentation (b) leak components (c) 3D leak components

Fig. 1. Example of a kidney contour segmentation: (a-b)sagittal CT slice; (c) 3D surface mesh.

tuning, and/or developing custom algorithms, all of which areimpractical in a clinical environment.

Segmentation leaks are one of the most pervasive segmen-tation errors. They occur when the segmentation volumecstd-def expands outside the target structure boundary into neigh-boring structures, beyond the expected ground-truth expertboundary delineation variability (Fig. 1). The segmentationproduces leaks when the voxels gradient intensity magnitudesof the target and neighboring structures boundaries are small,i.e., when their tissue imaging characteristics are similar, be-cause of the partial volume effect, and/or because of imagingnoise. Segmentation leaks can occur in all types of segmen-tation methods. In intensity-based thresholding and regiongrowing methods, they appear when the voxels intensity inthe leak boundaries and inside the target structure are close.In adaptive region growing methods, they appear when theintensity distribution of the leak and the target structure aresimilar. Intensity-based graph-cut methods produce leaks be-cause the min-cut is unlikely to separate voxels with simi-lar intensities, as this penalizes the global energy function.Energy-based methods, including snakes and level-set , moveaway the segmentation contour in low-gradient regions be-cause the image term in the energy equation yields a higheroverall energy. Adding a curvature-based penalty may onlyhelp to prevent bottleneck-like leak contour shapes. Shapeprior based methods, including atlas-based ones can producesegmentation leaks for target structures with low gradients in

their outer boundaries since the registration may to convergeto a local minima, far away from the actual boundary.

The correction of segmentation leaks is necessary and of-ten requires extensive manual user interaction, regardless ofthe segmentation method used. Shape prior models [1, 2, 3]have been proposed to reduce segmentation leaks. Its draw-backs are that: 1) it is structure-specific; 2) it relies on a repre-sentative set of expert-approved delineations; 3) it is inappro-priate for pathologies due to their high shape variability, and;4) the generation of shape priors is often time-consuming.

Heiman et al. [4] describe a method that explicitly detectsa segmentation leak by computing a path between two user-defined points – one inside the segmentation leak region andone inside the target structure region. This requires user in-teraction, is limited to a single bottleneck-like shape leak andcannot handle leaks to several neighboring structures.

In this paper, we present a novel method for the automaticdetection and correction of segmentation leaks. For each leak,our method first finds the segmentation leak basis boundaryby gradient magnitude likelihood classification and then fits asurface to voxels of the leak basis boundary (Fig 1b-c). Theleak is then separated from the target structure by by findingthe largest component after re-labeling the leak basis voxelsas background. The advantages of our method are that: 1) itis independent of the segmentation method used; 2) it doesnot require any prior shape, location, and/or intensity infor-mation; 3) it is automatic; 4) it has multiple uses, e.g., forsegmentation leaks correction post-processing, for automaticsegmentation with shape prior, and for interactive segmenta-tion leaks elimination. Experimental evaluation on 139 seg-mentations of kidneys and abdominal aortic aneurysms fromCT scans generated with two common segmentation methodsshow an improvement of about 50% in both the mean surfacedistance and the volume overlap error.

2. METHOD

We formally define the segmentation leaks and their structure,and then describe our leak detection and correction algorithm.

Let I = {Ii} be a volumetric medical image composed ofvoxels Ii. Let S = {Sj} be the set of anatomical structures ofinterest Sj in I , e.g., organs, vessels and tumors. Let T ∈ Sbe the target structure of interest, and let N ∈ S be a neighborstructure of T . Let ST be a segmentation of T in I .

The leak L between the target structure T and a neighborstructure N is the set of voxels in I that are classified by STas belonging to T but actually belong to N : L = ST ∩N .When L = φ, there is no leak.

The leak front LF of the initial segmentation ST is the set ofvoxels on the outer boundary of ST that are inside the neigh-bor structure N : LF = N ∩ boundary(ST ) for L 6= φ.

The leak basis LB of segmentation leak L is the set of voxelsthat belong to L and are on the boundary of the target struc-

ture T : LB = L ∩ boundary(T ).

The leak basis boundary LBB is the set of voxels on theouter contour of the segmentation leak basis LB: LBB =LB ∩ ST .

The segmentation leak L can extend into several neighboringstructures N and can have several components. In the follow-ing, we refer to each component as one leak. To distinguishbetween small segmentation errors and segmentation leaks,we define an absolute or relative lower threshold on the leaksize, i.e., its dimensions, and/or volume, usually above theintra and inter observer manual delineation variability.

Our algorithm first detects the leaks and then correctingeach one of them. We describe each step in detail next.

2.1. Leak detection

The goal is to find the segmentation leak basis, since this sur-face is the actual boundary between the target and the neigh-bor structures. This boundary was not detected in the initialsegmentation because the corresponding voxels gradient in-tensity magnitudes are relatively small.

The leak basis boundary is the common interface of theinitial segmentation contour and the segmentation leak front(Fig. 1b-c). We compute the leak front, derive from it the leakbasis boundary, and then obtain the leak basis from it.

The leak front is derived from the unknown leak gradientmagnitude distribution. We model this distribution with theknown gradient magnitude distribution inside the neighborstructure, which is that of interior of the initial segmentation.This assumes that: 1) the leak front is inside the initial seg-mentation, i.e., the initial and ground-truth segmentations arenot too far from each other, and; 2) the target and the neighborstructure interior gradient magnitude distributions are similar,which we empirically verified in CT images.

The leak detection consists of four steps. First, we com-pute the gradient magnitude likelihood histogram GL(|∇(Ii)|)in the interior of the initial segmentation ST . We obtain theinterior of ST by eroding ST with a sphere of radius rcentered on the outer boundary of ST .

Next, we find the leak front LF by classifying the vox-els in the initial segmentation contour according to GL. Theprobability PLFi of each voxel Ii to belong to the leak frontis defined as:

PLFi =

L(|∇(Ii)|)

maxIj∈SA

L(|∇(Ij)|)if Ii ∈ boundary(ST )

0 otherwise

where SA is the sampling area. Each voxel Ii is then classi-fied to belong to LF if PLFi ≥ t1. This yields the segmen-tation leak front.

Then, we estimate the leak basis boundary LBB by find-ing the leak front neighboring voxels in the initial segmenta-tion contour:LBB = {IiIi∈boundary(ST )

Ii 6∈LF

: minIj∈LF

(gdBST (Ii, Ij)) ≤ t2}

where gdBST(Ii, Ij) is the geodesic distance between Ii andIj on the boundary of ST , and t2 is a predefined voxel dis-tance constant. We compute the geodesic distance by findingthe shortest path on the contour using Dijkstra’s shortest pathalgorithm on the graph representation of the image voxels.

Finally, we approximate the leak basis LB by fitting asurface to the leak basis boundary voxels Ij ∈ LBB withthe grid-fit method [5]. The fitting minimizes the energy ofthe pulling forces on each voxel Ij subject to surface stiffnessconstraints, which control the surface smoothness and reducethe effect of noise and outliers.

2.2. Leak correction

We produce a corrected segmentation by eliminating the leaksidentified in the previous step in the original image I . Foreach segmentation leak L, we re-label the segmentation leakbasis voxels LB in the original segmentation ST as back-ground, and find the target connected component.

3. EXPERIMENTAL RESULTS

We conducted an experimental study to quantify the accuracyand robustness of our method, and to evaluate its scope.

We retrospectively selected 28 CT scans of kidneys (or-gan) and AAA (vascular pathology) which have differencesin shape, intensity homogeneity, and surrounding structures.For the kidney study, we used 16 clinical CT scans [6] of size512 × 512 × 350 − 500 voxels, 0.5 − 1.0 × 0.5 − 1.0 ×1.0 − 1.5mm3, with and without contrast agent, from a Bril-liance 64-row CT scanner (Phillips Healthcare, Cleveland,OH). Ground-truth segmentations of the left and right kidneyswere obtained from three manual segmentations of expertswith STAPLE [7]. For the AAA study, we used 12 clinicalCTA scans [8] of size 512 × 512 × 500 voxels, 0.7 − 1.2 ×0.7 − 1.2 × 0.7 − 1.2mm from the same scanner. Some ofthe scans included imaging streaking artifacts caused by im-planted stents. Ground-truth segmentations were obtained bymanual segmentation from an expert radiologist.

We chose two common segmentation algorithms: geodesicactive contour level-set and graph cut with shape prior. Forthe level-set initialization, the input was a user-defined seedinside the target structure and parameter settings for the level-set differential equation terms. The user stopped manuallythe segmentation when the taret structure was completleysegmented. For the the graph-cut initialization, the input wasa coarse, manually defined shape prior and user-defined in-tensity parameters for the foreground and background labels.

We obtained initial segmentations for the kidneys and theAAA with both segmentation algorithms with various initialconditions. We then applied our algorithm, and produced cor-rected segmentations. For the kidney, we performed 112 runson the 16 datasets, of which four cases where initially seg-mented with level-set, and three where initially segmented

(a) Initial segmentation (b) Leak basis (c) Final segmentation

Fig. 2. Illustration of the algorithm steps. The top two rowsshow kidney coronal CT slices and 3D meshes. The bottomtwo rows show AAA sagittal slices and 3D meshes: a) ini-tial segmentation with leak (orange); (b) leak basis (blue); (c)corrected (red) and ground-truth segmentations (blue).

with graph-cut, for each dataset. For the AAA, we performed48 runs on 12 datasets, of which two where initially seg-mented with level-set, and two with graph-cut. All the re-sulting segmentations had leaks. In all cases the algorithminternal constants where fixed to: t1 = 0.25, t2 = 7, andr = 15. The mean running time of the leak algorithm was106 secs (std=13.3) on a 64-bit quad-core 2.80GHz proces-sors and 6.0GB memory PC. Fig 2 shows two examples.

To quantify the accuracy of the leak basis detection, wecomputed the mean/maximum distance of the leak basis fromthe ground truth outer boundary. The average mean distanceswere 1.47mm (std=1.32) and 1.3mm (std=0.62); the averagemaximum distances were 11.03mm (std=3.98) and 12.02mm(std = 3.38) for the kidney and the AAA, respectively.

AVD (%) ASSD (mm) MSSD (mm) VOE (%)mean std mean std mean std mean std

KIDNEY initial 22.5 11.4 2.18 1.09 16.22 5.28 25.36 10.98corrected 4.98 4.65 0.87 0.44 10.56 4.18 12.31 4.65

improvement 17.52 6.75 1.31 0.65 5.66 1.10 13.05 6.33AAA initial 24.2 9.42 3.03 1.18 21.2 5.7 26.42 9.1

corrected 5.8 5.03 1.53 0.75 14.46 8.89 14.68 7.13improvement 18.4 4.46 1.49 0.42 6.74 -3.18 11.74 1.97

Table 1. Summary of evaluation metrics for the kidney and AAA. The metrics are the Absolute Volume Difference (AVD)from the ground-truth in %, Average Symmetric Surface Distance (ASSD) in mm, the Maximal Symmetric Surface Distance(MSSD) in mm, and the Volumetric Overlap Error (VOE) in %. The first (fourth) and second (fifth) are the initial and correctedsegmentation measures; the third (sixth) row is the relative improvement (the negative value indicates worsening).

To quantify the segmentation quality improvement, wecomputed the segmentation error of the initial segmentationand of the corrected segmentation. We used both volumet-ric and surface based metrics. In a few cases, the algorithmdid not improve the initial segmentation. For the kidney,for 14 out of 112 (12.5 %) segmentations, the average volu-metric overlap error and average symmetric surface distanceincreased by 4.29% (std=6.3%) and 0.45mm (std=1.1mm),respectively. For the AAA, for 7 out of 48 (14.5%) seg-mentations, the average volumetric overlap error and averagesymmetric surface distance increased by 5.68% (std=5%) and0.44mm (std=0.44mm), respectively.

Table 1 shows the results for the remaining cases inwhich there was a significant improvement. Our method sig-nificantly improved both the mean quality and its variabilityfor all evaluation metrics. For the kidney, the improvementwas 60% (std=20%) and 51.5% (std=4.65%) in the aver-age surface distance and the volume overlap error betweenthe initial and the corrected segmentations with respect tothe ground-truth; for the AAA it was 49.2% (std=24.7) and44.4% (std=7.13), respectively.

To estimate the robustness of our method to different seg-mentation methods and initializations, we computed for eachscan the mean correlation coefficient of the final segmenta-tion volumes after segmentation leaks correction. The aver-age correlation coefficient was 0.96 (std=0.026) for the kid-ney and 0.94 (std=0.02) for the AAA.

4. CONCLUSIONS

We have developed a new method for the explicit detectionand correction of segmentation leaks. Its advantages are that itis automatic, that it is independent of the initial segmentationmethod, and that it does not rely on prior information.

The limitations of our method are that it requires suffi-cient non-leaky boundaries segments in the initial segmenta-tion, that the leak front is inside the neighboring structures,not extending beyond their boundaries, and that the imageshave low noise. Within this scope, our results indicate that it

may be effective as a post-processing method for correctingsegmentation leaks, for automatic segmentation with shapepriors, and for interactive leaks elimination.

5. REFERENCES

[1] S. Chen and R.J. Radke, “Level set segmentation withboth shape and intensity priors,” in Proc. IEEE 12th Int.Conf. on Computer Vision and Pattern Recognition, 2010,pp. 763–770.

[2] Chittajallu DR. et al., “A shape-driven MRF model forthe segmentation of organs in medical images,” in Proc.12th Int. Conf. on Computer Vision and Pattern Recogni-tion, 2010, pp. 3233–3240.

[3] Rose J.L. et al., “3D region growing integrating adaptiveshape prior,” in Proc. 5th IEEE Int. Symp. on BiomedicalImaging, 2008, pp. 967–970.

[4] Heimann T. et al., “New methods for leak detection andcontour correction in seeded region growing segmenta-tion,” Int. Archives of Photogrammetry and Remote Sens-ing, vol. 35, pp. 16, 2004.

[5] J. DErrico, “Surface fitting using gridfit,” MATLAB Cen-tral File Exchange, 2006.

[6] Freiman M. et al., “Non-parametric iterative model con-straint graph min-cut for automatic kidney segmentation,”Medical Image Computing and Computer-Assisted Inter-vention, pp. 73–80, 2010.

[7] Warfield S.K. et al., “Simultaneous truth and performancelevel estimation (STAPLE): an algorithm for the valida-tion of image segmentation,” IEEE Trans. Med. Imaging.,vol. 23, no. 7, pp. 903–921, 2004.

[8] Freiman M. et al., “An iterative model-constrained graph-cut algorithm for abdominal aortic aneurysm thrombussegmentation,” in Biomedical Imaging, 2010 IEEE Inter-national Symposium on, 2010, pp. 672–675.