3d segmentation of rodent brain structures using active .../67531/metadc... · 3D SEGMENTATION OF RODENT BRAIN STRUCTURES USING ACTIVE VOLUME MODEL WITH SHAPE PRIORS Shaoting Zhang

BNL-100635-2013-CP

3d segmentation of rodent brain structures using active volume model with shape priors; conference paper

Shaoting Zhang1; J. Huang

1; M. Uzunbas

1; T. Shen

2;

F. Delis3; N. Volkow

3; P. Thanos

3; J. Fowler

2

1CBIM, Rutgers, The State University of New Jersey, Piscataway, NJ

2Computer Science and Engineering Department, Lehigh University, Bethlehem,

PA 3NIAAA Intramural Program, Bethesda, MD & Brookhaven National Laboratory,

Upton, NY

To be presented at: International Symposium on Biomedical Imaging (ISBI 2011),

Chicago, IL, March 20-April 2, 2011

January 2013

Biosciences Department

Brookhaven National Laboratory

U.S. Department of Energy

Office of Science, Office of Biological and Environmental

Research

Notice: This manuscript has been co-authored by employees of Brookhaven Science Associates, LLC under

Contract No. DE-AC02-98CH10886 with the U.S. Department of Energy. The publisher by accepting the

manuscript for publication acknowledges that the United States Government retains a non-exclusive, paid-up,

irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others

to do so, for United States Government purposes.

This preprint is intended for publication in a journal or proceedings. Since changes may be made before publication, it may not be cited or reproduced without the author’s permission.

DISCLAIMER

This report was prepared as an account of work sponsored by an agency of the

United States Government. Neither the United States Government nor any

agency thereof, nor any of their employees, nor any of their contractors,

subcontractors, or their employees, makes any warranty, express or implied, or

assumes any legal liability or responsibility for the accuracy, completeness, or any

third party’s use or the results of such use of any information, apparatus, product,

or process disclosed, or represents that its use would not infringe privately owned

rights. Reference herein to any specific commercial product, process, or service

by trade name, trademark, manufacturer, or otherwise, does not necessarily

constitute or imply its endorsement, recommendation, or favoring by the United

States Government or any agency thereof or its contractors or subcontractors.

The views and opinions of authors expressed herein do not necessarily state or

reflect those of the United States Government or any agency thereof.

3D SEGMENTATION OF RODENT BRAIN STRUCTURES USINGACTIVE VOLUME MODEL WITH SHAPE PRIORS

Shaoting Zhang1, Junzhou Huang1, Mustafa Uzunbas1, Tian Shen2, Foteini Delis3,Xiaolei Huang2, Nora Volkow3, Panayotis Thanos3, Dimitris Metaxas1

1 CBIM, Rutgers, The State University of New Jersey, Piscataway, NJ, USA2 Computer Science and Engineering Department, Lehigh University, PA, USA

3 Behavior Neuropharmacology and Neuroimaging Lab, Brookhaven National Laboratory, NY, USA

ABSTRACTObject boundary extraction is an important task in brain im-age analysis. Acquiring detailed 3D representations of thebrain structures could improve the detection rate of diseasesat earlier stages. Deformable model based segmentationmethods have been widely used with considerable success.Recently, 3D Active Volume Model (AVM) was proposed,which incorporates both gradient and region informationfor robustness. However, the segmentation performance ofthis model depends on the position, size and shape of theinitialization, especially for data with complex texture. Fur-thermore, there is no shape prior information integrated. Inthis paper, we present an approach combining AVM andActive Shape Model (ASM). Our method uses shape infor-mation from training data to constrain the deformation ofAVM. Experiments have been made on the segmentation ofcomplex structures of the rodent brain from MR images, andthe proposed method performed better than the original AVM.

Index Terms— Segmentation, deformable models, Ac-tive Volume Model, Active Shape Model, Shape prior, rodentbrain

1. INTRODUCTION

Diagnosis of neurological and phychiatric disorders is mostlybased on behavioral observations and on data from neu-roanatomical measures (MR, CT). Retrospective studies haveshown that neurological disorders are associated with specificmorphological changes of the brain, which could be usedfor the early or differential diagnosis of the disease [9]. Inthis study, we propose an approach that will provide fast andaccurate 3D segmentation of brain regions based on MR im-ages of the rodent brain. Rodents are often used as modelsof human disease not only because they frequently exhibitkey features of abnormal neurological conditions [5] but alsobecause they are a convenient starting point for novel studies.

Related works: Deformable model based segmentationmethods have been widely used with considerable success.We review some relevant work using contour or mesh based

shapes. Kass et al. proposed Snakes [7], which are energy-minimizing splines with smoothness constraints and influ-enced by image forces. Gradient Vector Flow (GVF) [12] isthen proposed to increase the attraction range of the originalSnakes. Depending solely on the image gradient information,however, these methods may be trapped by noise and spuriousedges. Region analysis strategies [15] have been incorporatedin Snake-like models to improve their robustness to noise.Huang et al. present a strategy aimed at integrating shapeand appearance in a unified space, which is named as Meta-morphs [6]. The model has not only boundary shape but alsointerior appearance, making it more robust in segmentation.Efforts have been made on the integration of interior appear-ance into 3D models. A volumetric deformable model, ActiveVolume Model (AVM) [11], is proposed. The AVM model’sshape is represented by a simplex mesh and its volumetricinterior carries the various visual appearance feature statis-tics. However, this online learning based model does not haveany shape prior information. Thus it is relatively sensitive tothe quality of initialization and image information. Anothergroup of deformable models is level set based methods [8].Region information has also been incorporated [4]. These ap-proaches have been widely used in tubular structure and 3Dcortex segmentation tasks since they are topologically freeand can be easily used in any dimension. Statistical modelingapproaches such as Active Shape Model (ASM) [3] or ActiveAppearance Model (AAM) [2] are also widely used and havebeen successfully applied in cardiac [14] and brain [1] seg-mentation. These methods may need a large amount of 3Dtraining data, whose creation and maintenance can be difficultand time consuming in practice.

In this paper, we present an approach to combine the ad-vantages of both AVM and ASM, i.e., use online learningbased AVM to deform the 3D mesh from image information,and then use ASM shape constraint to refine this intermedi-ate result, and repeat these two steps until convergence. AVMdoes not rely on any offline learning, so we do not need alarge set of 3D training data. As a tradeoff, AVM is some-times sensitive to image noise and intensity inhomogeneity.

Thus shape constraint from ASM is then applied to refine theintermediate segmentation result. We train the ASM modelin our approach using a small number of training data sincewe only need a rough statistical model to help AVM out oflocal minima, and the overall approach is mostly data driven.By combining these two approaches, the proposed method isparticularly useful when there are a limited number of train-ing samples and large variations among the samples. It isapplied to segment complex structures in rodent brains, suchas the cerebellum and the striatum. Numerous experimentshave been designed to evaluate this method.

2. METHODOLOGY

Image alignment: First all brain scans (images) are alignedto the same reference brain whose orientation corresponds tothe Paxinos stereotaxic position [10]. Alignment is performedwith rigid transformations (rotations and translations), thusafter alignment the volume and shape of brain structures donot change. Then the brain structures are manually segmentedby clinical experts. A 3D surface mesh, composed of around1, 000 vertices, is created for each reconstructed structuresin brains. The results from traditional marching cubes maycontain many artifacts, thus all meshes are post-processedusing isotropic remeshing and detail preserving smoothingtools1. To obtain the one-to-one correspondence for each ver-tex, we choose one specific mesh as the reference mesh anduse a local deformation technique [13] to warp it into all othermeshes. Thus all resulting meshes share the same connectiv-ity and topology, and a mean shape can be calculated. Fromobservation, the variance of the mass centroid of these meshesis small (the standard deviation is less than 0.3mm). Thusgiven an aligned test image, it is reasonable to predict the cen-troid using the mean value.

Deformation module: The smoothed mean shape isused as an initialization. However, it may not be close to theboundary of the testing data because of the variance. Thus de-formable models are still needed for accurate segmentation.In order to fit to the boundary of an object, the traditionalAVM is driven by both a gradient based data term and a re-gion data term which are derived from image information.The overall external energy function consists of two terms:the gradient term Eg and the region term ER. The overallenergy function is:

E = Eint + ke · Eext = Eint + ke · (Eg + kR · ER) (1)

where kR is a constant to balance the contributions of the twoexternal energy terms. ke is for the balance between internal(smoothness) and external forces.

The gradient data term can be defined using the gradientmap, edge distance map, or a combination of both. Denote a

1http://www.openflipper.org/

gradient magnitude map or the distance transform of an edgemap as Fg , the gradient data term is defined as:

Eg =∫

Λ

Fg(x)dΛ, Fg = D2edge or Fg = − |∇I|2 (2)

where Λ denotes the surface mesh, Dedge refers to the un-signed distance transform of the edge map, and ∇I representsthe image gradient.

The region term encodes constraints for the AVMs model-interior appearance statistics. Considering a module usingintensity statistics, the object region is predicted accordingto the current model-interior intensity distribution. Havingboth foreground object and background probabilities, we ob-tain a binary map that represents the predicted object regionby applying the Bayesian Decision rule. Connected compo-nent analysis is then applied on the binary map to retrieve theconnected component that overlaps the current model. Thisconnected region is considered as the current ROI. Let us de-note the signed distance transform of the current model’s sur-face shape as ΦΛ, and the signed distance transform of theROI boundary shape as ΦR, the region-based external energyterm is defined using voxels within a narrow band around themodel surface as:

ER =∫

Λ

ΦΛ(v)ΦR(v)dΛ (3)

The multiplicative term provides two-way balloon forcesthat deform the model toward the predicted ROI boundary.This allows flexible model initializations either overlappingthe object or inside the object. The mesh deformation isbased on standard Finite Element Method (FEM) and solvedefficiently in a linear system.

Shape refinement module: The above formulation maynot be able to avoid the local minimum or keep a specificshape, especially when the texture of image is complex. Thusa shape refinement procedure is added in our method to con-strain the deformation. This procedure is similar to the shapeupdating step in ASM. First all meshes are aligned using sim-ilarity transformation. Note that the alignment here is forshapes instead of images. Then their statistics is capturedby principal component analysis (PCA). Given a intermediatesegmentation result, it is first transformed to the mean shape,and then mapped into PCA space to update the pose and shapeparameters. Thus we can guarantee that the shape only de-forms into shapes consistent with the training data. This stepcan prevent over-segmentation and provide a shape constraint.We adopt the following steps to deform the proposed modeltoward matching the desired object boundary.

1. Manually segment a small number of training data.2. Use PCA to capture statistics (mean and variance) of

these 3D shapes.3. Initialize the AVM, i.e., stiffness matrix and step size

for FEM, and the gradient magnitude or edge map.

Fig. 1. Segmentation results of the striatum (the 1st to the 3rd column) and the cerebellum (the 4th to the 6th column) of therodent brain. The 1st row: results from the proposed method. The 2nd row: results from AVM.

4. Compute ΦΛ based on the current model; predict objectROI R by applying the Bayesian Decision rule to bina-rizing the estimated object probability map, and com-pute ΦR. Calculate the external force vector.

5. Deform the model using FEM and external forces.6. Refine this intermediate result by ASM shape con-

straint (transform it to the mean shape, then update thepose and shape parameters consistent with the trainingdata).

7. Repeat steps 4-6 until convergence.

Alternatively employing the AVM deformation and theASM shape refinement is more robust to noises and canhandle more complex textures than purely using AVM. Ourmodel converges fast and robust towards the boundary be-cause of the benefit of the good-quality initialization and theconstraint of the shape prior. One more benefit of our modelis that the size of the training data can be small, because AVMis driven by current image information. Training informationis only used for the shape refinement procedure, which is anauxiliary step in our model.

3. EXPERIMENTS

Experimental Settings: Adult male Sprague-Dawley ratswere transcardially perfused with 4% paraformaldehyde.Heads were stored in paraformaldehyde and scanned formagnetic resonance imaging. The brains remained in theheads during scanning in order to avoid tissue and shape dis-tortions during brain extraction. The heads were scanned on a21.1T Bruker Biospin Avance scanner (Bruker Biospin Cor-poration, Massachusetts, USA). The protocol consisted of a3D T2-weighted scan with echo-time (TE) 7.5ms, repetitiontime (TR) 150ms, 27.7 kHz bandwidth, field of view (FOV)of 3.4 × 3.2 × 3.0mm, and voxel size 0.08mm, isotropic.Out of 10 3D data, 7 were used as training to get mean masscentroid and other statistics, and the remaining 3 were used

for testing. Both AVM and the proposed method were imple-mented in C++ and Python2.6 and tested on a 2.40 GHz IntelCore2 Quad computer with 8G RAM.

Table 1 compares the segmentation performance betweenour framework (including more sophisticated initializationand shape refinement) and the original AVM system. Cere-bellum, left and right striatum of rodent brains are seg-mented. They are challenging for deformable models becauseof the relatively complex shapes and inside textures. Themean value of sensitivity (p), specificity (q), dice similaritycoefficient (DSC) and relative errors of volume magnitudecompared to ground truth are reported for evaluation. Thenumber of iterations is also listed since the running time isproportional to it. Figure 1 shows the segmentation results ofthe methods with different view points in 3D. In the originalAVM system, we manually place an ellipsoid and carefullyrotate and resize it to an ideal position (close to the boundary)as the initialization, following the same procedure as the orig-inal paper [11]. Parameter tuning is in general a challengingproblem for deformable models. We chose the parametervalues in both models empirically. In the proposed method,shape constraints contribute to the stableness. Thus the resultis less sensitive to parameter values.

Our proposed method achieves better quantitative perfor-mance and also better visual results. Its sensitivity, specificityand DSC are generally higher, while the number of iterationtimes for convergence are less. Note that all specificities arevery high. The reason is that the segmented structures are rel-atively small, thus most background areas (true negatives) aresuccessfully recognized.

Segmenting the left and the right striatum is challeng-ing because the contrast between the structure and other sur-rounding brain structures is relative low. AVM can easily de-form too much and over-segment the structure. Furthermore,with large smoothness energy (small ke), AVM may fail toreach the sharp boundary of the striatum, while with large ex-ternal energy (large ke), AVM results may contain many small

Table 1. Quantitative evaluations and performance comparisons between AVM and the proposed method. RE-V denotes therelative error of estimated volume magnitude.

AVM Our methodp q DSC #iterations RE-V p q DSC #iterations RE-V

Left striatum 0.75 0.98 0.62 28 1.138 0.82 0.99 0.89 7 0.084Right striatum 0.70 0.99 0.61 32 1.025 0.81 0.99 0.84 8 0.129Cerebellum 0.71 0.99 0.78 43 0.266 0.84 0.99 0.87 13 0.039

bumps because of the local intensity inhomogeneity and lackof smoothness constraints. The results can be observed inthe second row of Figure 1. Using our method alleviates thisproblem since the ASM shape refinement step keeps the shapeand prevents the over segmentation problem (the first row ofFigure 1).

The cerebellum is also difficult to segment because of itscomplex internal textures and two protruding features. Its in-terleaving high gradient may make the model trapped in thelocal minimum. Large external force is needed to segment theprotruding features, which adversely affects the smoothnessof the model. Thus traditional AVM underperforms this task.Even though the quantitative results of AVM are reasonablygood, we can clearly observe the artifacts from visual results.Our method uses ASM shape constraint to refine the interme-diate result. The shape already escapes many local minimaand it has two protruding features presented as priors. Theseproperties not only improve the model’s performance but alsodecrease the number of iterations to converge.

4. CONCLUSIONS AND FUTURE WORKS

In this work we proposed an algorithm to segment structuresof the rodent brain. Based on a small set of training data, ourmethod automatically places the initialization using the meanmass centroid and the smoothed mean shape, and also uses theshape information as a prior to constrain the deformation. It ismore accurate and robust, and needs less iterations than AVM.Our method is more applicable when there is only a smallnumber of training sampless. In the future, we would liketo analyze and obtain statistics for more structures from moredata. Furthermore, our current model can not handle structuremissing. Efforts will be put on the modeling of these atypicalsituations. We are also interested in using the relative positionand orientation among structures to facilitate the deformation.

5. REFERENCES

[1] K. Babalola, T. Cootes, C. Twining, V. Petrovic, andC. Taylor. 3D brain segmentation using active appear-ance models and local regressors. In MICCAI, pages401–408, 2008.

[2] T. Cootes, G. Edwards, and C. Taylar. Active appear-ance models. Proc. Of European Conf. on ComputerVision, 2:484–498, 1998.

[3] T. Cootes, C. Taylor, D. Cooper, and J. Graham. Ac-tive shape model - their training and application. CVIU,61:38–59, 1995.

[4] D. Cremers, M. Rousson, and R. Deriche. A re-view of statistical approaches to level set segmentation:Integrating color, texture, motion and shape. IJCV,72(2):195–215, 2007.

[5] F. Delis, M. Xenos, D. Grandy, G.-J. Wang, and N. V. P.Thanos. Effects of chronic alcohol intake and dopamineD2 receptor gene expression on brain anatomy: an in-vivo MRI morphometric study of the mouse brain. InAnnual meeting of the Research Society on Alcoholism,2009.

[6] X. Huang, D. Metaxas, and T. Chen. Metamorphs: De-formable shape and texture models. In CVPR, pages496–503, 2004.

[7] M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Activecontour models. IJCV, 1:321–331, 1987.

[8] R. Malladi, J. Sethian, and B. Vemuri. Shape modelingwith front propagation: A level set approach. PAMI,17:158–175, 1995.

[9] R. McCarley, C. Wiblea, M. Frumina, Y. Hirayasua,J. Levitta, I. Fischera, and M. Shenton. MRI anatomyof schizophrenia. Biological Psychiatry, 45(6):1099–1119, 2009.

[10] G. Paxinos and C. Watson. The rat brain in stereotaxiccoordinates. Elsevier, Amsterdam, 2007.

[11] T. Shen, H. Li, Z. Qian, and X. Huang. Active vol-ume models for 3D medical image segmentation. CVPR,2009.

[12] C. Xu and J. Prince. Snakes, shapes and gradient vectorflow. TIP, 7:359–369, 1998.

[13] S. Zhang, X. Wang, D. Metaxas, T. Chen, and L. Axel.LV surface reconstruction from sparse tMRI using lapla-cian surface deformation and optimization. In ISBI,2009.

[14] Y. Zheng, A. Barbu, B. Georgescu, M. Scheuering, andD. Comaniciu. Four-chamber heart modeling and au-tomatic segmentation for 3D cardiac ct volumes usingmarginal space learning and steerable features. IEEETrans. Med. Imaging, 27:1668–1681, 2008.

[15] S. Zhu and A. Yuille. Region Competition: Unifyingsnakes, region growing, and Bayes/MDL for multi-bandimage segmentation. PAMI, 18(9):884–900, 1996.