An Analysis of Segmenting and Classifying Tumor Regions in ... · exible and high ca-pacity of Deep Neural Network (DNN).The deep learning ... perbolic tangent and Recti ed Linear

An Analysis of Segmenting andClassifying Tumor Regions in MRI

Images using CNN

PL.Chithra1 and G.Dheepa2

1,2Department of Computer Science,University of Madras, Chennai-5,

Tamilnadu, India

March 21, 2018

Abstract

Brain tumor extraction from Magnetic Resonance Imag-ing (MRI) is an important task in medical image processing.It is one of the difficult and time consuming tasks becausethe structural variability among the tumor is entirely dif-ferent from normal images. Manual segmentation requiresexpert knowledge and it has very less accuracy. So, weneed intelligent machine learning technique for automaticdetection of brain tumor through magnetic resonance im-ages. These reasons motivate our exploration through ma-chine learning solution that exploits flexible and high ca-pacity of Deep Neural Network (DNN).The deep learningarchitectures are performing very accurately and they grad-ually outperform previous state-of-the-art classical machinelearning algorithms. MRI sequence consists of four intra-tumoral structures like edema, necrotic, non-enhancing andenhancing regions. The purpose of this paper is to providedetailed examination of segmenting brain tumor and theirregions using deep learning methods in MRI sequences.

Key Words:Brain tumor segmentation, deep learning,Magnetic Resonance Imaging (MRI), Convolution NeuralNetwork (CNN).

1

International Journal of Pure and Applied MathematicsVolume 118 No. 24 2018ISSN: 1314-3395 (on-line version)url: http://www.acadpubl.eu/hub/Special Issue http://www.acadpubl.eu/hub/

1 INTRODUCTION

Brain tumor extraction of magnetic resonance imaging show a highcorrelation between the intensities of nearby voxels and the differ-ent image modalities of intensity patterns acquired from the samevolume [10]. Based on high mortality rate and prevalence, the braintumors are graded into Low Grade Gliomas (LGG) and High GradeGliomas (HGG). Each grades of brain images in MRI sequence hav-ing four sequences of contrast images, namely T1-weighted (T1),T2-weighted contrast enhanced image (T1c), T2-weighted image(T2), and T2-weighted FLAIR image (FLAIR) [5].

Some tumor structures may appear dark in one contrast, whilebrighter in another, making different tumor substructures easierto segment in different contrasts.Z. Akkus et al [8] describes thatthe tumor structures vary considerably across patients in terms ofsize, extension, shape and localization of intensities. S. Valverde[4] suggested that the deep learning architecture proves better per-formance than the conventional methods of segmentation. Beforeapplying deep learning, all (testing, training, and validation) imageshave to be pre-processed and then extract the 2D patches. Finally,all patches have been trained by using deep learning techniques.

The objective of this paper is to analyse the detailed architec-ture of deep learning techniques for segmenting tumor in brain MRIimages. Manual, semi-automatic and automatic methods have beendescribed in section II. Database used for brain images and exper-imental setup are detailed in section III. Training, evaluation andresult validation are discussed in section IV. Finally, conclude theirresults in section V.

2 RELATED WORK

Manual and Semi-Automatic MethodsManual segmentation of tumor is a complicated and labour-

intensive task, which normally involves slice-by-slice procedures. Itis one of the radiologist dependent methods and it has very less ac-curacy compared to automatic methods [11].Semi-automatic meth-ods can be classified with four major categories, namely thresholdbased methods, region based methods, clustering based methodsand classification based methods, in which four major types require

2

International Journal of Pure and Applied Mathematics Special Issue

interactions from the users (initialization of parameters, processingof images and evolution of results) have been discussed [12].

Automatic MethodsAutomatic methods have been used to reduce rater variability,

human interaction and time complexity. Dong et al [9] proposeda fully automatic segmentation method and it has highly in de-mand for multimodal and multi-dimensional structural variability.Machine learning is a set of algorithmic techniques that performcomputer systems to make data-driven predictions from large data[2].One of the best and emerging machine learning technique iscalled deep learning.

There is several deep learning architectures are used for the fea-ture extraction on images. The most popular deep learning archi-tectures are feed forward Neural Network, Recurrent Neural Net-works (RNNs), Long Short-Term Memory (LSTM)/gated recurrentunit (GRU), Convolution Neural Networks (CNNs), and Deep Be-lief Networks (DBN). S. Periana et al [2] described that the con-volution neural network is most widely used architecture for themedical image processing to extract tumours in 3D MRI images.

B.H. Menze et al [5] has collected various tumor extractionchallenges in the Multimodal Brain Tumor image Segmentation(BRATS) benchmarks. He analysed that deep CNN architectureis giving more accurate results compare to other networks. Beforewe started the brain tumor segmentation methods, intensities haveto be normalised in pre-processing step. Then we convert three di-mensional MRI images into two dimensional (2D) slices of images.After normalizing, it needs to be trained images using CNN andpredict its cancer pixel class. Nyul et al [3] has discussed variousnormalization techniques for medical images.

K. Kamnitsas et al [13], S.valverde et al [4], and D. G. Loranzoet al [7] are used brain lesion segmentation from multi convolu-tion pipeline. These three authors have been used more than oneconvolutional layer for segmenting brain tumor.

Mohammad Havaei et al [1] have proposed more convolutionallayers for segmenting brain tumor in MRI images. In this pro-posed architecture has 30 times faster than the results reported inBRATS 2013 dataset. Fig.1 presents the different types of contrastimages. Each contrast images have four intra tumor structural re-gions namely necrosis, edema, non-enhancing and enhancing tumor.

3


Fig. 1 Four imaging modalities: (a) T1-weighted MRI; (b)T2-weighted MRI; (c) FLAIR; and (d) FLAIR with contrast

enhancement.

3 EXPERIMENTAL SETUP

A CNN consists of more number of convolutional and pooling layersoptionally followed by fully connected layers. Input to a convolu-tional layer is an m * n * r image where m and n is the height andwidth of the image and r is the number of channels, e.g. an RGBimage has 3 channels. Each convolutional layer will have k filters(or kernels) of size x * y * q, where x and y is smaller than thedimension of image and q can be same as the number of channelsr.

Brain images have to be pre-processed by using any one of thepre-processing methods namely, De-noising, skull-stripping and in-tensity normalization etc. Patches have been prepared and ex-tracted in normalized tumor image by augmentation. The methodof augmentation is mostly used to increase the size of the dataset.MRI brain images can be prepared by 2D or 3D patches.

Deep CNN based methods are used to extract the image featuresautomatically. These extracted features can be classified by usingConvolutional Neural Network. Finally calculate the loss functionbetween network output and some targeted output. Overall ar-chitecture brain tumor segmentation in MRI images using CNN isillustrated in fig.2.

4


Fig 7 Output Waveforms of proposed zeta Vi, Vo&Vco

Brain image datasetsThere are different kinds of datasets are available [6], namely

the Multimodal Brain Tumor image Segmentation (BRATS), OpenAccess Series of Imaging Studies (OASIS), Magnetization PreparedRapid Acquisition Gradient Echo (MPRAGE) and Internet BrainSegmentation Repository(IBSR V2.0).

Medical image consist of array of 2D images (x1, x2..n). In asingle image, x be the image patch of size u*v from image space I.Focusing on 2D patches, a patch x is represented as x (u, v, I) where(u, v) denotes the patch top left corner coordinates in multimodalimage I(s, V ) where s denotes the slice position in image volume V.The tumor can present in all 2 dimensional MRI images. Each slicehas four intra-tumor structures like necrotic, edema, non-enhancingand enhanced tumor. This proposed method will have to classifythese tumor regions by using CNN.

MRI Pre-processing MethodsMost segmentation methods implement an identical processing

pipeline. These pipelines are pre-processing, classification and postprocessing steps. Before classifying tumor region, Pre-processingmethods have been applied for de-noising, skull-stripping, intensitynormalization, etc [6]. Noises present in MRI images have removedby delineate region of interest between brain tumor and normalbrain tissues. The non-cerebral tissue region such as skull and scalpare removed in the skull stripping methods.

Intensity of same tissue can vary across the image. So, we haveto normalize the intensities before segmenting brain tumor. Mostmedical images have been used N4ITK and Nyul et al intensity

5


normalization methods [3]. The intensity of each MRI sequenceis more similar across subjects. Then some statistical measureszero mean and variance values have been used to normalize theintensities.

Weight initializationBefore training the network, weight parameters are initialized.

There are four major categories of weight initializations. They are,Zero (When all weights are set to 0), Random (When weights are setcompletely randomly), Random between -1 to +1(Random weightson the scale of -1 to +1) and Xavier-Glorot initialization [1].

Activation functionThe activation function is the non-linear transformation of in-

tensities over the input signal is given in (1).

Y = Activation(∑

(weights ∗ input) + bios) (1)

This transformed output is then sent to the next layer of neu-rons. Most popular types of Activation functions are Sigmoid, Hy-perbolic tangent and Rectified Linear Units. The formula for sig-moid activation function is mentioned in equation (2).

f(x) = 1/1 + exp(−x) (2)

Its Range is between 0 and 1 and it is an S- Shaped curve.The mathematical formula for hyperbolic tangent function (tanh)is given in equation (3).

f(x) = 1 − exp(−2x)/1 + exp(−2x) (3)

Tanh output is zero centred because its range arrives between -1to 1 i.e. -1 < output < 1.ReLu (Rectified Linear units) has becomevery popular in the past few years. It was recently proved that ithad 6 times improvement in convergence from tanh function hasmentioned in equation 4.

R(x) = max(0, x) (4)

If x < 0, R(x) = 0 and if x >= 0, R(x) = x.PoolingThe output of the first convolutional layer has been send to

the pooling layer. It combines spatially nearby features in the fea-ture map and also reduces the size and computational load of the

6


network. There are two types of pooling techniques, namely Maxpooling and average pooling. These two methods are mostly usedto down sampling the size of 2D images. Input 2D images arecombined with filter weights and bios to produce an output of thefirst convolutional layer. The output can be send to the poolinglayer. Three hyper parameters (The number of filters stride andzero padding) are used to control the size of output volume, namelythe number of filters, padding and stride.

Data augmentationData Augmentation always improves its performance though

the amount of data present in the dataset. 2D image data can beaugmented to produce an increased number of images. Some of themost widely used augmentation methods are Rotation (Randomwith angle between 0◦ and 360◦), Translation (Random with shiftbetween -10 and 10 pixels),Rescaling (Random with scale factorbetween 1/1.6 and 1.6), Flipping (Yes or no), Shearing (Randomwith angle between -20◦ and 20◦ ) and Stretching (random withstretch factor between 1/1.3 and1.3.

RegularizationRegularization in used to improve the training performance and

reduce over fitting. It is mostly used in the Fully Connected (FC)layers. This method is used to dropping out the regions from net-work, based on some fixed probability. The commonly used dropoutvalue for brain tumor images is 0.5. There are at least 4 ways toregularize the network [2] (L1 Regularization, L2 Regularization,Dropout, and Batch Normalization). Over segmentation reducedby using these regularization techniques.

Fig 8 Waveforms in the inductors

Calculating loss functionIt is used to measure the errors between network outputs and

some target outputs. There are several types of loss functions. They

7


are Cross-entropy (XE), Maximum Mean Discrepancy (MMD), Mean-absolute-error (MAE), Mean-squared-error (MSE). But cross-entropyis one of the common loss functions to minimize the distance fromnetworks distribution over class labels. Cross-entropy (XE) is usedto find the loss function between true distribution over discrete classp(t) and model distribution over predicted class q(x). Mathemati-cally loss function computes in the equation 5.

L(x, t) = −∑

(=)log(=) (5)

When using this loss function, targets are assumed to be integersand it can be computed between the intervals of (0, k).

4 TRAINING AND EVALUATION

The experiments have been discussed on real patient data [6]. Firstthe data can be split into 3 parts namely testing, training andvalidation. All brain images on the dataset have four modalities(FLAIR, T1, T1c, and T2). Each modal have the slices of images.These images can be augmented to prepare more patches. Everypatches of 2D image combined with weights and bios to producefirst convolutional layer output. Activation functions are appliedat the end of each convolutional layer for non-linearly transformingour input data.

Then down sampling of image can be done by using pooling.For a single convolution network contains more number of convolu-tional and pooling layers. Finally our patches have transformed into fully connected layer. Here, regularization techniques have ap-plied to improve the training performance and reduce over fitting.Finally classify the input images by using any one of the classifica-tion techniques, namely Conditional Random Field (CRF), MarkavRandom field (MRF), Random Forests (RF), and Support VectorMachine (SVM). Overall designing of convolutional neural networkis described in fig.2. Then the classified regions have labelled asnecrosis, edema, non-enhancing, and enhancing tumor. Finally, theresults have evaluated for testing, training and validation images.

The training of CNN performs continuously, until it reached theepoch (complete pass over of all data from the dataset). A result

8


has been evaluated by using all the values of calculated brain im-ages. Finally calculate the performance of training for the classifiedtumor regions namely necrosis, edema, non-enhancing and enhanc-ing tumor.

Three metrics are mostly used in the evaluation of segmentingbrain tumor. They are, Dice Similarity Coefficient (DSC), PositivePredictive value (PPV), and Sensitivity. Dice similarity Coefficient(DSC) is used to find the amount of overlap between false positiveand false negative values mentioned in equation 6.

DSC =2TP

FP + 2TP + FN(6)

Where TP, FN and FP are the numbers of True Positive, FalseNegative and False Positive detection. Positive Predictive value(PPV) is the measure of finding true positive and false positivevalues. These values have given in equation 7.

PPV =TP

TP + FP(7)

Sensitivity is used to calculate the number of true positive andfalse negative detections is given in equation 8

Sensitivity =TP

TP + FN(8)

The convolutional neural network can be designed either singlepipeline or multiple pipelines. Different authors observed variousperformance scores for whole tumor, core tumor and active tumor.The highest performance in BRATS 2013 dataset is 88%, 78%, 71%for whole tumor, core tumor and active tumor. Accuracy of thetraining performance is depending on increased number of epoch.The dice score performance of different CNN pipeline from variousauthors is illustrated in table1.

Table 1, Comparison of various brain tumor segmentation meth-ods (Results based on BRATS data set).

9


Fig 9 Waveforms of input(Vo),output(Vi) & diode voltage(VD)

5 Conclusion

Segmenting brain tumor is one of the challenging tasks in medicalimage processing. In this paper, we discussed overview of tradi-tional semi automatic methods and the detailed architecture of au-tomatic deep learning methods. The different scores of performanceachieved from various authors are also discussed. Convolutionalneural network has the advantage of learning image features auto-matically for both healthy brain tissues and tumor tissues directlyfrom the multi-model MRI images. The computation time was re-duced and the performance was increased by using Convolutionalneural network pipeline. Finally, this study is concluded with, thedeep CNN based architectures has given better performance whilecomparing to other traditional methods.

References

[1] M. Havaei,A.Davy, Brain tumor segmentation with deep neuralnetwork. Medical image analysis,No. 35, pp.1831, 2017.

[2] S. Pereira, A. Pinto, and V. Alves, Brain tumor segmenta-tion using convolutional neural network in MRI images., IEEETransactions on Medical Imaging, vol.35, no.5, pp. 12401251,2016.

10


[3] L.G. Nyul, J.K. Udupa, New variants of a method of MRI scalestandardization. IEEE Transactions On Medical Imaging, vol.19, No. 2, , pp. 143150, 2000.

[4] S. Valverde, M. Cabezas, Improving automated multiple scle-rosis lesion segmentation with a cascaded 3D convolutionalneural network approach.,Neuro Image, No. 155, pp.159-168,2017

[5] B.H. Menze, A. JakabMultimodal Brain tumor image seg-mentation benchmarks, IEEE transactions of medical imaging.vol.34, no.10, pp. 1993-2023, 2015.

[6] J.Kleesiek, G. Urban, Deep MRI brain extraction: A 3D con-volutional neural network for skull stripping, Neuro Image,no.129, pp. 460-469, 2016.

[7] D.G. Loranzo, S. Francis, Review of automatic segmentationmethods of multiple sclerosis white matter lesions on convolu-tional MRI., Medical Image Analysis, no.17, pp.1-18, 2013

[8] Z. Akkus, A. Galimzianova, Deep learning for brain MRI seg-mentation: state of the art and feature directions, Journal ofdigital imaging, no.30, pp. 449-459, 2017

[9] H. Dong, G. ysng, Automatic brain tumor detection and seg-mentation using U-net based fully convolutional network ,Springer no. 69, pp.1-12, vol.3, 2017

[10] P. Devork, B. Menze, Structured prediction with convolutionalneural network for multimodal brain tumor segmentation.,In-Proceedings MICCAI-BraTS ( Brain Tumor Segmentationchallenge ), pp.13-24, 2015.

[11] Jin Liu, Min Li, A survey of MRI-Based brain tumor segmen-tation methods, TSINGHUA Science and Technology, ISSN:1007-0214, Vol.19, no.6 , 2014

[12] Yao Wu, W. Yang, semi-automatic segmentation of brain tu-mors using population and individual information, Springer,no.26, pp. 786-796, 2013.

11


[13] K.Kamnitsas, C.Ledic, Efficient multi-scale 3D CNN with fullyconnected CRF for accurate brain lesion segmentation MedicalImage Analysis, no. 36, pp.61-78, 2017.

[14] A. Islam, S. Reza, Multiracial texture estimation for detectionand segmentation of brain tumors, IEEE Transactions of Bi-medical Engineering, vol. 60, no. 11, pp. 32043215, Nov. 2013.

12


An Analysis of Segmenting and Classifying Tumor Regions in ... · exible and high ca-pacity of Deep Neural Network (DNN).The deep learning ... perbolic tangent and Recti ed Linear

Documents