-
Research ArticleA Pilot Study of Diabetes Mellitus
Classification from rs-fMRIData Using Convolutional Neural
Networks
Yunfei Liu ,1 Xian Mo,1 Hao Yang,2 Yan Liu ,1 and Junran Zhang
1
1Department of Automation, College of Electrical Engineering,
Sichuan University, Chengdu, China2Information Center, West China
Hospital, Sichuan University, Chengdu, China
Correspondence should be addressed to Yan Liu;
[email protected] and Junran Zhang; [email protected]
Received 23 July 2020; Revised 21 September 2020; Accepted 28
September 2020; Published 21 October 2020
Academic Editor: Jianbing Ma
Copyright © 2020 Yunfei Liu et al. ,is is an open access article
distributed under the Creative Commons Attribution License,which
permits unrestricted use, distribution, and reproduction in any
medium, provided the original work is properly cited.
Background. As a chronic progressive disease, diabetes mellitus
(DM) has a high incidence worldwide, and it impacts on cognitiveand
learning abilities in the lifetime even in the early stage, may
degenerate memory in middle age, and perhaps increases the riskof
Alzheimer’s disease. Method. In this work, we propose a
convolutional neural network (CNN) based classification method
tohelp classify diabetes by distinguishing the brains with abnormal
functions from the normal ones on resting-state functionalmagnetic
resonance imaging (rs-fMRI). ,e proposed classification model is
based on the Inception-v4-Residual convolutionalneural network
architecture. In our workflow, the original rs-fMRI data are first
mapped to generate amplitude of low-frequencyfluctuation (ALFF)
images and then fed into the CNN model to get the classification
result to indicate the potential existence ofDM. Result. We
validate our method on a realistic clinical rs-fMRI dataset, and
the achieved average accuracy is 89.95% in
fivefoldcross-validation. Our model achieves a 0.8690 AUC with
77.50% and 77.51% sensitivity and specificity using our local
dataset,respectively. Conclusion. It has the potential to become a
novel clinical preliminary screening tool that provides help for
theclassification of different categories based on functional brain
alteration caused by diabetes, benefiting from its accuracy
androbustness, as well as efficiency and patient friendliness.
1. Background
Diabetes affects more than 451 million people (18–99
years)worldwide, and this figure will rise to 693 million by
2045;more surprisingly still, it is estimated that almost half of
allpeople living with diabetes are undiagnosed [1]. It is a groupof
metabolic diseases characterized by hyperglycemia andfrequently
accompanied by complications. For many years,the most well-known
complications caused by diabetes aredysfunction and failure of
organs, like kidney, retina, pe-ripheral nerve, and vasculature
[2]. Children diagnosed withT1DM are more likely to perform poorly
in school than theirnondiabetic classmates and are particularly
vulnerableshowing impaired results on cognitive tests, learning
abil-ities, and affectingmemory [3].,ere are now several
studiesdemonstrating a linkage between T2DM and mild
cognitiveimpairment (MCI) and Alzheimer’s disease (AD). ,e
co-existence of cerebrovascular disease and T2DM enhances the
correlation with MCI and the development of dementia
[4].Compared to nondiabetic individuals, several studies
havedemonstrated an increase of AD in T2DM patients [5].Nowadays,
clinical studies have shown that much moreattention should be paid
to the brain complications of di-abetes, like diabetic
encephalopathies and cognitive dys-function. Diabetic
encephalopathies are now acceptedcomplications of diabetes, which
manifest themselves as agradual decline of cognitive function and
result in brainstructural lesions (neural slowing, increased
cortical atro-phy, microstructural abnormalities in white matter
tracts)[6, 7]. ,ere is a growing literature indicating that
indi-viduals with diabetes have impairments in recent memory[8],
and the mechanism might be due to the fact that glucosetransport is
significantly reduced in diabetic animals [9].Most of the current
research has formed a consensus thatDM affects brain function and
brain structure, and thechanges in brain function often precede
those in brain
HindawiMathematical Problems in EngineeringVolume 2020, Article
ID 1903734, 11 pageshttps://doi.org/10.1155/2020/1903734
mailto:[email protected]:[email protected]://orcid.org/0000-0002-5168-3978https://orcid.org/0000-0003-4881-8429https://orcid.org/0000-0002-8035-8824https://creativecommons.org/licenses/by/4.0/https://doi.org/10.1155/2020/1903734
-
structure. Previous studies have shown that white mattercould
alter after a six-week training period [10]. ,at is tosay, the
impact of diabetes on the brain is initiated frombrain function and
gradually extends to the brain structure.,e reason might be that
the long-term abnormal bloodglucose level damages the cognitive
function through itsnegative effect on target organs, namely, the
central nervoussystem [11], which may help explain why animals that
haveabnormal glucose metabolism have more hippocampaldamage [12].
Glucose borne by the blood accounts for 99%of the brain energy
requirements, and metabolic substratedelivery also may influence
brain function and structure[13]. What is more serious is that the
damage is progressiveand irreversible; in most cases, it will
develop to dysfunction.It will cause patients to suffer long-term
pain and seriouslyaffect the quality of life. A few studies have
indicated thatreasonably controlling blood glucose with
antidiabetictreatments on time may help prevent the dysfunction
indiabetic patients [14]. Accordingly, the detection of
diabetesmellitus and other changes in brain function can reduce
theproduction probability of those serious complications; it
alsoplays an important role in treatment planning and makes
apositive influence on prognosis.
As diabetic encephalopathy is a degenerative disease ofthe
nervous system, we can choose to directly observe thedegree of
cognitive impairment of patients through modernimaging methods such
as magnetic resonance imaging(MRI), so as to explore the
correlation between it and di-abetes. MRI imaging technology can be
divided intostructural modalities and functional modalities.
Comparedwith structural images, functional images are more
sensitiveto early brain changes and so can better reflect
earlierchanges in brain function. fMRI uses oxygen consumptionin
brain tissue to determine whether a part of the brain isactive in
the resting state. ,e study found that the restingbrain was not
completely calm but had spontaneous BOLDsignal fluctuations which
account for 60 to 80 percent of thetotal energy consumed by the
brain. A large number ofstudies have also shown that the inherent
spontaneousneural activity signals of the brain at rest have
importantphysiological significance [15]. Besides, resting-state
func-tional MRI (rs-fMRI), which reflects the variation
charac-teristics of spontaneous resting-level activities in
restingstate, has been increasingly applied in the field of
brainscience with unparalleled advantages recently. rs-fMRI is
thepreferred modality to investigate brain function due to itshigh
temporal resolution and has already been used tomeasure spontaneous
brain activity in patients with diabetesand to reflect changes in
brain functional damage caused bydiabetes encephalopathy [11]. We
can observe the dys-function degree directly through modern imaging
modali-ties in a noninvasive way and explore its correlation
withdiabetes and its complication. By using rs-fMRI,
researchershave demonstrated that diabetes is related to different
in-dices of functional brain alterations, including
regionalhomogeneity (ReHo) and amplitude of
low-frequencyfluctuations (ALFF) [16]. ALFF analysis is an
importantmethod for depicting the various characteristics of global
rs-fMRI signals through measuring the intensity of neural
activity at the single-voxel level and evaluating the
differ-ences in the amplitude of low-frequency oscillations
ofbrains [17]. Abundant ALFF studies have demonstrated thatthe
brain function of diabetic patients has changed. Previousstudies by
Cui et al. [16] showed that patients with T2DMhad significantly
decreased ALFF values in the postcentralgyrus and occipital lobe.
Patients performed worse onseveral cognitive tests; the researchers
speculated that thisimpaired cognitive performance was correlated
with de-creased activity in the cuneus and lingual gyrus in the
oc-cipital lobe. Recent studies by Xia et al. [18] indicated
thatT1DM patients showed significantly decreased ALFF valuesin the
posterior cingulate cortex (PCC) and right inferiorfrontal gyrus
compared with the healthy controls. Fur-thermore, they found a
positive correlation between de-creased ALFF values in the PCC and
Rey–OsterriethComplex Figure Test- (CFT-) delay scores in T1DM
patients.
Although ALFF has already been widely used in brainfunction
research, the main analysis still follows the tradi-tional ways
(statistical analysis, functional connectivityanalysis, and
correlation analysis) depending on well-trainedexperts and is
always qualitative and subjective. By contrast,we try to analyze
the rs-fMRI sequence through deeplearning technology and reflect
the quantitative relationshipbetween brain dysfunction and
diabetes. In this work, wepropose a learning-based classification
model to distinguishthe abnormal ALFF signals from the normal ones.
Weemploy a convolutional neural network architecture toconstruct
our model. ,e entire pipeline of the proposedmethod consists of
three successive blocks, as shown inFigure 1(a). ,is fully
automated solution can processthousands of heterogeneous images
quickly for accurate,objective diabetes detection. Furthermore, we
seek tocharacterize the association between DM and brain
functionand structure. We study the differences in the brain
functionchanges caused by diabetes between diabetic patients
andhealthy control groups to prove the reliability of the
clas-sification. All information learned in our end-to-end
algo-rithmic pipeline is visualized through the
Gradient-weightedClass Activation Mapping (Grad-CAM), and the
subregionswithin the classified image are highlighted intuitively
tofurther observe the extent of diabetes affecting differentbrain
regions. ,e differential brain regions reflect the in-fluence of
diabetes on brain function and structure andprovide some insights
for the study of the brain compli-cations of diabetes.
2. Methods
,e pipeline of the proposed method is represented inFigure
2.
2.1. Data
2.1.1. Data Acquisition. Our retrospective study includes
47patients with type 1 diabetes mellitus (denoted as “T1DM”),73
patients with type 2 diabetes mellitus (denoted as“T2DM”), and 50
healthy controls (denoted as “HC”).Physicians make all the
diagnoses according to the criteria
2 Mathematical Problems in Engineering
-
from the American Diabetes Association [19]. All subjectsare
right-handed and undergo brain scans at the Huaxi MRResearch Center
of the West China Hospital. Exclusioncriteria for all participants
include a history of substance oralcohol abuse, a psychiatric or
neurological disorder unre-lated to diabetes, contraindications
toMRI, and a history of abrain lesion such as tumor or stroke.
Control subjects areexcluded if they have a fasting blood glucose
level≥7.0mmol/L, glucose level ≥7.8mmol/L after oral
glucosetolerance test (OGTT).
All rs-fMRI images are acquired via a Siemens (Erlangen,Germany)
3-Tesla Trio scanner. Subjects are instructed torelax, to keep
their eyes closed but to remain awake, to keeptheir heads still
during the scanning, and to avoid thinking of
anything in particular. ,e functional images arerecorded using
an echo-planar imaging (EPI) sequencewith the following parameters:
repetition time/echo time(TR/TE) � 2000/30ms, 90°flip angle, 240 ×
240mmmmfield of view (FOV), slice thickness/gap � 5/0mm, 3.75 ×3.75
× 5mm3 voxel size, and 64 × 64matrix resolution. Novery obvious
structural damage is found in any subjectbased on MRI scans, which
are examined by two expe-rienced specialists.
2.1.2. Functional Image Preprocessing. ,e raw rs-fMRIdata are
preprocessed with the Statistical ParametricMapping software (SPM8)
on the MATLAB (R2013b)
BasicConv2d:Conv-BN-PReLU-Dropout
ResidualBlock:BN-Conv-BN-ReLU-Conv-BN-ReLU-Conv-BN-Dropout
Binary output
FC-So
ftmax
AvgPo
ol-Dr
opou
t
FeaturemappingRe
sidual
Block
Sum
Concat
Block 2Inception
Block 1
Conv-B
N-PReL
U
MaxPo
ol
BasicC
onv2d
1 × 1
BasicC
onv2d
1 × 1
BasicC
onv2d
1 × 1
BasicC
onv2d
3 × 3
BasicC
onv2d
3 × 3
BasicC
onv2d
3 × 3
BasicC
onv2d
1 × 1
AvgPo
ol
Block 3Residual Visualization:
gradients via backprop
Shortcu
t
1 × 1
(a)
Gradients
Activations
Backprop till convBackprop till conv
Back
prop
till
conv
Thetask-
specificnetwork
db1-control
Layer4
FC layer
10 Image classification
Block3 (residual):rectified convfeature maps
Block2 (Inception):rectified convfeature maps
Block1: rectifiedconv feature maps
Visualization process
Inception-v4-
residual
Input
PReLU
I
Laye
r1
Laye
r2
Laye
r2
PReLU
+ +
A1 A2 A3
PReLU
wc 2(n–1)wc 11 wc 12 w
c 1(n–1) w
c 1n w
c 21 w
c 22 wc 31 w
c 32 w
c 3(n–1) wc 3nw
c 2n
(b)
Figure 1: ,e abstraction of the proposed algorithmic pipeline
and visualization process. (a) ,e proposed Inception-v4-Residual
networkwith seven parameter layers (about 0.09 million parameters).
(b) Grad-CAM overview: Given an image and a class of interest as
input, weforward-propagate the ALFF image through the CNN part of
the model and then through the task-specific computations to obtain
a rawscore for the category. ,en, the gradients are set to 0 for
all classes except the desired class, which is set to 1. Finally,
this signal is back-propagated to the parametric rectified
convolutional feature maps of interest, which we combine to compute
the coarse Grad-CAMlocalization (blue heatmap), which represents
where the model has to look to make the particular decision.
Mathematical Problems in Engineering 3
-
platform. ,e adopted preprocessing operation is shownin Figure
1. ,e first 10 volumes of the scanning sessionsare discarded due to
the instability of the initial MRIsignal and adaptation of the
participants to the situation.,e remaining volumes with size 61 ×
73 × 61 × 170 areanalyzed. For each subject, the remaining rs-fMRI
imagesfirst undergo section-timing correction and are then
arerealigned. Here, participants with head displacement>2mm in
any x, y, and z direction and >2° in any angulardimension are
excluded. After realignment, the resultingimages are spatially
normalized using 3 mm isotropicvoxels. ,e last step is to smooth
the images using aGaussian kernel of 4 mm full width at
half-maximum toimprove signal-to-noise ratio and reduce the
differencebetween individuals after standardization.
2.1.3. Feature Mapping. ALFF measures the intensity ofneural
activity at the single-voxel level and can be calculatedthrough the
Data Processing Assistant for Resting-StatefMRI (DPARSF) [20]
(http://rfmri.org/DPARSF). After theabove preprocessing, band-pass
filtering (0.01−0.08Hz) isperformed on the time series of each
voxel to remove theeffect of low-frequency drift and high-frequency
respiratoryand cardiac noise [21]. For each subject, the time
series ofeach voxel is transformed into the frequency domain
usingFast Fourier Transform in turn, and the power spectrum
isobtained. ,en, the square root of the power spectrum iscomputed
and averaged across a predefined frequency in-terval. ,e averaged
square root is ALFF, an effective in-dicator of spontaneous
neuronal or regional intrinsic activityin the brain [22]. Following
the above steps, ALFF, thefunction map is generated for each
subject.
2.1.4. Training and Testing Dataset. Each participant is
as-sociated with a diagnostic label of 1 or 0 referring to DM orHC,
confirmed by medical specialists. Our data augmen-tation protocol
is as follows. Firstly, all images are nor-malized to [0, 1] and
resized to the standard resolution of73 × 73. Secondly, images are
flipped horizontally or
vertically to capture the reflection invariance. ,e final
datatransformation is brightness adjustment with one randomscale
factor α per image, sampled from a uniform distri-bution over
[−0.2, 0.2]. We add above transformations thatextend the
translation invariance, improve our model’sability to generalize,
and correctly classify images without aloss of accuracy. To address
the imbalance of dataset cate-gories, about 20% of the subjects
from HC and DM areassigned to the testing data, and the remaining
are used fortraining.
2.1.5. Model and Visualization. A full diagram of
theclassification model can be viewed in Figure 1(a), and
theabstraction of the visualization process is represented inFigure
1(b).
2.1.6. Classification Model. ,e network used in this study
isinspired by the CNN architecture, Inception-v4-Residual,presented
in [23]. It speeds up the flow of information,extracts features
from different scales, accelerates networktraining, and avoids
gradient disappearing. Limited by theamount of data, we simplify
the original Inception-v4-Re-sidual architecture into three
integrated convolutionalblocks (block1, block2 (Inception), block3
(Residual)), oneaverage pooling (AvgPool) layer, and one fully
connected(FC) layer, as depicted in Figure 1(a). All integrated
blocksare stacked by basic blocks as follows.
Block1 � Conv − BN − PReLU − MaxP, (1)
BasicConv � Conv − BN − PReLU − Dropoutbranch1 �
BasicConvbranch2 � BasicConv − BasicConvbranch3 � BasicConv −
BasicConv − BasicConvbranch4 � AvgP − BasicConv
⎧⎪⎪⎪⎪⎪⎪⎪⎨
⎪⎪⎪⎪⎪⎪⎪⎩
,
Block2 � Concat(branch1, branch2, branch3, branch4),(2)
DM
Testingdata
Classificationmodel
HCAugmentation
Trainingdata
73 × 73Cropresizing
axslice
7361
61
ALFF
Discard the first 10 volumesSlice timing correctionMotion
correctionSpatial smoothingLow-pass filteringSpatial
normalization
(i)(ii)
(iii)(iv)(v)
(vi)
Featuremapping
Preprocessing
Inputrawdata
Figure 2: ,e pipeline of the proposed method.
4 Mathematical Problems in Engineering
http://rfmri.org/DPARSF
-
Block3 � BN − Conv − BN − ReLU − Conv − BN − ReLU− Conv − BN −
Dropout,
(3)
where (branch1, branch2, branch3, branch4) in (2) refers tothe
concatenation of the feature maps produced inbranch 1, · · · , 4.
Conv is the convolutional layer which learnsiteratively filters
that transform the input into hierarchicalfeature maps. BN is the
normalization layer used to nor-malize the activation for fast
convergence and to improveperformance [24]. A BN layer conducts an
affine transfor-mation with the following equation:
y � cx + β, (4)
where c and β are learned for every activation in featuremaps.
Parametric Rectified Linear Unit (PReLU) [25] orRectified Linear
Unit (ReLU) [26] layer applies an ele-mentwise activation function,
such as the max (αx, x) ormax (0, x) thresholding, and does not
change the size of theimage volume, as defined by
xl � (P)ReLU convl xl−1( ( , (5)
where convl represents a convolutional layer l, which returnsthe
previous convolutional layer’s output volume. ,epooling layer (MaxP
or AvgP) performs a downsamplingoperation along the spatial
dimensions to reduce hyper-parameters and prevent overfitting. ,e
key idea of dropoutlayer (dropout) [27] is to randomly drop units
(along withtheir connections) from the neural network during
training.,is strategy prevents too much coadaptation of
units,significantly reduces overfitting, and gives major
improve-ments over other regularizationmethods. FC layer
computesthe class scores, resulting in a volume of several classes.
Asthe name implies, each neuron in the FC layer will beconnected to
all the numbers in the previous volume[28, 29]. An abstraction of
this feature-learning architectureis represented in Figure 1(a),
and see Table 1 for detailedarchitectures.
2.1.7. Implementation. Our implementation for
Inception-v4-Residual follows the practice in [30, 31]. We
initialize theweight as in [25] and train the Inception-v4-Residual
net-work from scratch. We use Adam and SGD in the early andlate
stages of model training with a minibatch size of 10,respectively.
,e learning rate (lr) starts from 0.001, and themodel is trained
for up to 20 × 104 iterations. Figure 3 showshow the learning rate
changes and how the optimizer isselected during training. We use
weight decay of 0.0005 andmomentum of 0.8. Since the data are
relatively unbalanced,we also use the 2-class categorical focal
loss for discrimi-nation [32]. In practice, we use an α−balanced
variant of thefocal loss:
FL pt( � −αt 1 − pt( clog pt( , (6)
where t ∈ 0, 1{ } specifies the ground-truth class andpt ∈ [0,
1] is the model’s estimated probability for the classwith a label
t. A weighting factor α ∈ [0, 1] is applied for class
t � 0 and 1 − α for class t � 1. In practice, α is set by
inverseclass frequency, and we set c � 2 in our experiment.
2.1.8. Visualization Process. ,e efficient visualizationprocess
used in this work is Grad-CAM proposed by Sel-varaju et al.
[33]—for making any CNN based models moretransparent by producing
“visual explanations.” Grad-CAMuses the gradients of any target
concept, flowing into thecorresponding convolutional layer to
produce a coarse lo-calization map highlighting the most important
regions inthe image for predicting the concept. We apply Grad-CAM,a
class-discriminative localization technique, to find
char-acteristic brain regions that can classify diabetic
patients.,eprocedure for generating these maps is illustrated
inFigure 1(b).
As shown in Figure 1(b), to obtain the class-discrimi-native
localization map Grad-CAM Lcn,Grad−CAM ∈ R
μ×υ ofwidth μ � 73 and height ] � 73 for any class c � 2, we
firstcompute the gradient of the score for the class c, yc
withrespect to feature maps Akn (A
k1, A
k2, A
k3) of a convolutional
block, i.e., zyc/zAkn. ,ese gradients flowing back are
global-average-pooled to obtain the neuron importance weightsαcn,k.
,e final localization map is calculated through (7) andvisualized
as a heatmap in Figure 1(b). ,e highlightedregions in the ALFF
image might be used to help real-timeclinical validation of
automated detection in the future.
Lcn,Grad−CAM � PReLU k
1Z
ijzyc
zAkn,ij
√√√√backprop gradients√√√√√√√√√√√√√√
global average pooling:αcn,k
⎛⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝
⎞⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠
· Akn
⎛⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝
⎞⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠
√√√√√√√√√√√√√√√√√√√√√√√√√√linear combination
. (7)
3. Results
We use 5-fold cross-validation in this study. Average
metricsoriginated from 5 test runs on respective held-out data.
3.1. Local Cross-Validation Results. In our work, we mainlyuse
the accuracy for evaluating our network architecture. Allthe
networks are trained using the same experimental setupas in Section
2.1.7. ,e initial lr is set to 0.001, and we trainmodels for 300
epochs with a batch size of 10 with the samedata augmentation
strategies as in Section 2.1.4. Note that weuse early stopping [34]
to prevent overfitting in all thenetworks. ,e accuracy of the
single-crop is provided inTable 2 for evaluating our models. Our
algorithm scores anAUC of 0.8690 during cross-validation and also
achieves anaverage 77.50% sensitivity and a 77.51% specificity.
,isROC curve is plotted in Figure 4. In the training phase,accuracy
and also loss of training and testing aremeasured asis depicted in
Figure 5. ,e experimental results in Figure 5show that the
Inception-v4-Residual network with only0.091M trainable parameters
converges fairly fast, whichindicates that our network has a
real-time runtime per-formance and has computational efficiency. We
empirically
Mathematical Problems in Engineering 5
-
demonstrate the Inception-v4-Residual network’s effec-tiveness
on ALFF classification accuracy grouped by subjectin Figure 6.
,rough Figure 6, we can see that the recog-nition rate of the model
to diabetic patients is quite high.However, classification for HC
is slightly worse because ofthe lack of distinct image features.
Compared with false-negatives, false-positives could be expected
when using anautomated method for the detection of a patient.
3.2. Grad-CAMVisualization Results. For efficiently
triagingreferrals and focusing on one’s clinical examination, it
ishighly important to interpret the output of
detection-guidingsoftware reasonably. Toward that end, we apply the
Grad-CAM visualization method to locate the most
discriminativefeatures of our deep learning network. Figure 7
illustrate somefeature maps of the final filter (kernel) of block1,
block2, andblock3, where the regions that contributed most to the
finalclassification result are highlighted on the heatmaps.
,osehighlighted regions tie the mathematical learning of thenetwork
to the domain of clinical data. ,e distributiondifferences of these
important regions corroborate the do-main-guided learning procedure
of our model. ,e average
Table 1: Inception-v4-Residual architectures. Building blocks
are shown in brackets (see also Figure 1(a)).
Layer name Output size Conv
Block137 × 37 [3 × 3, 16, stride 2, padding 1]
19 × 19 [3 × 3,maxpool, stride 2]
Block2inception
19 × 19 [1 × 1, 16, stride 1]
19 × 19 1 × 1, 16, stride 13 × 3, 32, stride1, padding 1
19 × 19 1 × 1, 16, stride 13 × 3, 16, stride1, padding 13 × 3,
32, stride1, padding 1
⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦
19 × 19 3 × 3, stride1, padding 11 × 1, 8, stride 1
cat branch_all � Concat[branch1, branch2, branch3, branch4] �
88
sum shortcut � 1 × 1, 88, stride 1shortcut + branch_all
Block3residual 9 × 9
1 × 1, 44, stride 13 × 3, 44, stride 21 × 1, 88, stride 1
⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦ + [3 × 3, 88, stride2]
Adaptive average pool, 2d FC, softmax# Params. 0.09MFLOPs 1 ×
107
Trunk depth 7
0.0010
0.0008
0.0006
0.0004
0.0002
0.0000
Lear
ning
rate
Epoch_Ir
Adam (≤epoch); SGD(≥epoch)
StepLR (≤epoch); CosineannealingLR (≥epoch)
0 20 40 60 80 100Epoch
Epoch < 2020 ≤ epoch < 4040 ≤ epoch
Figure 3: Schematic diagram of optimizer selection and
dynamicadjustment of learning rate. Green: the initial lr decays
bygamma� 0.5 every four epochs. Blue and red: after the 20th
epoch,cyclical lr is a function of epoch. Epoch ≤40: Adam. Epoch
>40:SGD.
6 Mathematical Problems in Engineering
-
activation of HC is much higher than DM, which proves tosome
extent the reliability of the classification results.
3.3. Effectiveness of Different Network Architectures.Apart from
the Inception-v4-Residual network, we exploreseveral other CNN
models, including ResNet [35] and
Inception [36] for performance comparison. ,e totalnumber of
parameters in these networks is shown in Table 3.,e same image
preprocessing is adopted for a fair com-parison. We train network
entirely for several epochs at acertain resolution from scratch.
Table 3 shows the experimentresults. It is observed that the
network architectures have aslight impact on the performance, and
Inception-v4-Residual
Table 2: Accuracies achieved from Inception-v4-Residual network
across five runs.
Inception-v2-Residual Training time per epoch (s) Total time (m)
Top-1 accuracy (%) AUCrun1 5.65 9.51 89.51 0.8708run2 564 9.49
89.88 0.8862run3 5.71 9.61 90.12 0.8721run4 5.70 5.60 90.12
0.8762run5 5.59 12.03 90.12 0.8391# of params. �0.09MMean accuracy
89.95%Mean AUC 0.8690
1.0
0.8
0.6
0.4
0.2
0.0
True
pos
itive
0.0 0.2 0.4 0.6 0.8 1.0False positive
Random chance
Equal error rate
ROC curve
Figure 4: ,e average ROC curve derived from the
Inception-v4-Residual network’s run3. ,e black dotted line
represents the trade-offresulting from random chance. ,e red curve
represents the model’s trade-off, with the blue dot marking the
threshold point.
0.60
0.55
0.50
0.45
0.40
0.35
0.300.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0
Epoch0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0
Epoch
90
80
70
60
50
40
30
20
10
Accu
racy
(%)
Training accTesting acc
Training lossTesting loss
Loss
Figure 5: Loss and accuracy of training and testing for
Inception-v4-Residual network’s run3.
Mathematical Problems in Engineering 7
-
network achieves better performance than the other twonetworks,
which demonstrates the efficiency of our method.
3.4. Ablation Studies. ,e main design choices we make forthe
training and testing procedure are the data augmentationprotocols
and test-time rotations. To show their impact, weperform two
ablation studies.
3.4.1. Rotation. ,e experimental setting of the trainingprocess
remains the same as the above experiment, but duringtesting the
images are only presented in their original orien-tation or show
the direction after being rotated at a certainangle. To improve
accuracy, we average the classification scores.
3.4.2. Data Augmentation Protocols. During training, wehave not
adopted any data augmentation protocols.Meanwhile, we separately
test the impact of each of the dataaugmentation protocols and their
combination on the ex-perimental results. ,e testing procedure is
unchanged.
Figure 8 shows results on the preliminary test set for
ourproposed setting and the two ablation studies. ,e influenceof
data augmentation protocols is significant but smaller; wenote that
the brightness adjustment is beneficial to themodel’s
generalization ability, but other data augmentation
protocolshave reduced the model’s ability to learn features. Not
usingtest-time rotation slightly decreases the mean accuracy(0.24%)
and AUC (0.006) in fivefold cross-validation.
4. Discussion
In this work, the convolution neural network based on
theclassification model is proposed to help DM and brain re-lated
abnormalities classification based on rs-fMRI, and theresults
indicate that it is effective and accurate. In addition
tofacilitating detection, for the first time, our algorithmpipeline
visualizes the abnormal brain areas, which mightprovide critical
information to understand the effect ofdiabetes on brain
dysfunction. Our approach provides anattempt to utilize deep
learning to detect DM and brainrelated abnormalities from
resting-state fMRI data. Resultsobtained suggest that rs-fMRI holds
the potential to increasethe translation of rs-fMRI data into
clinical detection.
In classifying HC and DM, we can achieve an AUC of0.869 on the
local dataset. ,e visualization results in Fig-ure 7 provide some
reliable evidence for the results of thisexperiment. As we have
seen, the degree of activation of HCand DM is significantly
different, which confirms the pre-vious research finding that DM
patients have a decreasedspontaneous brain activity on rs-fMRI.
However, the dif-ference in activation between T1DM and T2DM is not
thatobvious. To a certain degree, this visualization result
canexplain the high level of accuracy in the experiment.
Asexpected, decreased neural activity is significantly
associatedwith DM, a result which is in agreement with other
studies[16]. It is extremely difficult to accurately classify
diabetesand its type in the early stages even for medical care
per-sonnel because many diabetic individuals do not easily fitinto
a single class, and assigning a type of diabetes to anindividual
often depends on the circumstances present at thetime of
diagnosis.,e detection of subtle differences in brainfunction
between T1DM and T2DM poses an importantlimitation on accurate
identification in future DM typingdetection systems. In future
research, we look forward tocombining manual features for targeting
specific charac-teristics of DM and the robust potential of deep
learningsystems to characterize the type of diabetes accurately
toyield more clinically useful results.
Further optimization of the sensitivity metric might benecessary
to ensure a minimum false-negative rate forproper clinical
application of our algorithm. ,e computer-aided system for DM
classification must minimize false-
1.0
0.5
0.0 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5
6 7 8 9 10 11 12 13 14 15 16 17 18T2DMT1DMHC
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 10.95
0 0 0 0
Figure 6: Subject-level ALFF classification accuracy on the test
set using the best model.
Subject of HC
Subject of T1DM
Subject of T2DM
Figure 7: Examples of visualization map generated from
deepfeatures. Note that, in all subjects, red regions correspond to
a highscore for the class. ,e figure is best viewed in color.
Table 3: Accuracies achieved from Inception-v4-Residual,
ResNet,and Inception models.
Ours Inception ResNet #of params. (M) Top-1 accuracy (%)✓ 0.091
90.12
✓ 0.152 89.88✓ 2.039 88.78
8 Mathematical Problems in Engineering
-
negative results to provide necessary glucose care for
pa-tients. During clinical use, it may be quite critical to
controlspecific variances in local dataset, such as age, to
optimizethe model for certain demographics. Patient history,
du-ration of diabetes, symptom type, hemoglobin A1C value,genetic
factors, and other clinical data may play a crucial rolein
investigating different types of common patient metadatathat may
assist healthcare professionals in making a correctdiagnosis of the
type of DM. Adding confirmed clinicalinformation into the
classification system may yield in-sightful correlations into
underlying DM risk factorsoutside of imaging information,
potentially enhancingclassification accuracy between T1DM and T2DM.
Severallimitations of this study should be noted. First,
duringsubsequent experiments, a large patient and control
subjectcohort is needed to create a more robust model, and
in-dependent testing on external datasets is required toconfirm its
predictive properties. Second, the impact ofdifferent background
ethnicity and geographic location ofdemographics on the
classification model needs to beconsidered. ,ird, our experiments
assume that diabeticpatients with chronic insulin deficiency or
hyperglycemiamay cause corresponding changes in brain function
andmicrostructure. ,ese changes can be reflected in rs-fMRIimages,
which can use deep learning methods to identifythe disease but
cannot exclude the presence of diabeteswithout causing changes in
brain function and micro-structure, for instance, when diabetes is
quite early. Inshort, much effort is still required to achieve the
clinicalimplementation of a texture-based decision-support sys-tem
in further research.
Diabetic encephalopathies are now accepted complica-tions of
diabetes. ,ey appear to differ in T1DM and T2DMas to the underlying
mechanisms and the nature of resultingcognitive deficits [3]. Both
types of diabetes are associatedwith increased risks for micro- and
macrovascular diseaseand cerebrovascular accidents with compounding
effects oncognitive deficits [13]. Studies of brain function
andstructural neuroimaging have demonstrated associatedanomalies.
As shown in Figure 7, it should be mentioned
though that the class activation mapping in T1DM appearsto be
different from that of T2DM, which coincides withprevious studies.
,ere is evidence suggesting that theprogressive deficits in brain
function and structure maydevelop already in patients with
prediabetes [13] (T1DMleads to neuronal loss and disintegration of
neuronal net-working fundamental to cognitive function; T2DM
results inneuronal loss). However, some of the underlying
pathoge-netic mechanisms are different in the encephalopathies
ofthe two types of diabetes. ,erefore, continued investiga-tions
are needed to start to formulate precise therapeuticinterventions
to curtail the increase in these major com-plications. In future
research, we will seek to characterize theassociation between DM
and brain function and structureusing CNN and Grad-CAM to further
quantitatively in-vestigate the impact of diabetes on brain
complications.Definitely, this can reduce the production
probability ofthose serious complications; it also plays an
important rolein treatment planning and makes a positive influence
onprognosis.
In this study, we distinguish diabetic patients fromhealthy
controls and seek to characterize the associationbetween DM and
brain function and structure. We proposethat, among DM patients,
the activation degree of the featuremap would be associated with
decreased spontaneous brainactivity. ,is study may also help create
the view that dif-ferences in brain activity of different diabetes
types areclosely related to the corresponding brain complications.
Onthe whole, we propose a cost-effective and
time-efficientautomatic diagnosis algorithm of diabetes which shows
thepotential of automated feature-learning systems instreamlining
current diabetes screening programs.
5. Conclusion
In this work, we explore an approach based on deep learningto
distinguish the DM data from normal control data with89.95%
accuracy. ,e results showed that rs-fMRI holdsgreat promise for the
prediction of DM. However, furthervalidation on independent
datasets is required to confirm its
0 5 10 15 20 25
90.50
90.25
90.00
89.75
89.50
Accu
racy
Accuracy
0.90
0.89
0.88
0.87
0.86
0.85
0.84
0.83
0.82
AUC
0 2 4 6 8 10 12 14 16 18 20 22 24 26Rotation numberRotation
number
AUC
90.2
90.0
89.8
89.6
89.4
Accu
racy
(%)
Fold_one Fold_two Fold_three Fold_four Fold_fiveFold name
89.5122
89.3902
89.878 89.878
90 9090
90.122 90.122 90.12290.122
90.2439 90.2439 90.24
NoneColorjitterRandomaffine
RandomrotationAll
Figure 8: Metrics on the preliminary test set measured after the
early stopping for the proposed setting and the two ablation
studies. ,eshaded area marks one standard deviation. ,e figure is
best viewed electronically and zoomed in.
Mathematical Problems in Engineering 9
-
predictive properties. ,is deep learning solution and
ouralgorithm pipeline provide a new idea for the diagnosis ofDM. It
will potentially alleviate the workload of manualanalysis and guide
high-risk patients for referral to furthercare. We believe that the
methodology presented in thiswork can be also generalized to
predict the different types ofDM for different age groups, and it
can eventually lead toimprovements in treatment personalization and
patientsurvival.
Abbreviations
ALFF: Amplitude of low-frequency fluctuationHC: Healthy
controlsDM: Diabetes mellitusT1DM: Type 1 diabetes mellitusT2DM:
Type 2 diabetes mellitusMRI: Magnetic resonance
imagingGrad-CAM:
Gradient-weighted Class Activation Mapping
AUC: Area under the receiver operating characteristiccurve
GDM: Gestational diabetes mellitusDR: Diabetic retinopathyReHo:
Regional homogeneitylr: Learning rate.
Data Availability
,e data used to support the findings of this study areavailable
from the corresponding author upon request.
Conflicts of Interest
,e authors declare that they have no conflicts of
interestregarding the publication of this paper.
Authors’ Contributions
YFL, YL, HY, and JRZ substantially contributed to methodand
design. XM and YFL participated in data acquisition.YFL was
responsible for code and network structure design.YFL, YL, and XM
drafted the article. YL and JRZ criticallyrevised the article for
important intellectual content. Allauthors approved the final
version to be published.
Acknowledgments
,e publication of this article was sponsored by the
NationalScience Foundation of China under Grant 61902264 and theKey
Research and Development Projects in Sichuan Prov-ince under Grant
2019YFS0125, Sichuan University-ZigongCity Cooperation Project
2018CDZG-19.
References
[1] N. H. Cho, J. E. Shaw, S. Karuranga et al., “IDF diabetes
atlas:global estimates of diabetes prevalence for 2017 and
projec-tions for 2045,”Diabetes Research & Clinical Practice,
vol. 138,pp. 271–281, 2018.
[2] A. D. Association, “Classification and diagnosis of
diabetes,”Diabetes Care, vol. 40, no. Supplement 1, pp. S11–S24,
2017.
[3] A. A. Sima, “Encephalopathies,” >e Emerging
DiabeticComplications, vol. 47, no. 4, pp. 279–293, 2010.
[4] R. Peila and L. J. Launer, “Type 2 diabetes, apoe gene, and
therisk for dementia and related
pathologies,”>eHonolulu-AsiaAging Study, vol. 51, no. 4, pp.
1256–1262, 2011.
[5] J. A. Luchsinger, M. Tang, Y. Stern, S. Shea, and R.
Mayeux,“Diabetes mellitus and risk of alzheimer’s disease and
de-mentia with stroke in amultiethnic cohort,”American Journalof
Epidemiology, vol. 154, no. 7, pp. 635–641, 2001.
[6] Z. S. Nasreddine, N. A. Phillips, V. Bédirian et al.,
“,emontreal cognitive assessment, MoCA: a brief screening toolfor
mild cognitive impairment,” Journal of the AmericanGeriatrics
Society, vol. 53, no. 4, pp. 695–699, 2005.
[7] R. J. McCrimmon, C. M. Ryan, and B. M. Frier, “Diabetes
andcognitive dysfunction,” >e Lancet, vol. 379, no. 9833,pp.
2291–2299, 2012.
[8] S. Kalmijn, E. J. M. Feskens, L. J. Launer, T. Stijnen,
andD. Kromhout, “Glucose intolerance, hyperinsulinaemia
andcognitive function in a general population of elderly
men,”Diabetologia, vol. 38, no. 9, pp. 1096–1102, 1995.
[9] W. M. Pardridge, D. Triguero, and C. R. Farrell,
“Down-regulation of blood-brain barrier glucose transporter in
ex-perimental diabetes,” American Diabetes Association, vol. 39,no.
9, pp. 1040–1044, 2013.
[10] J. Scholz, M. C. Klein, T. E. Behrens, and H.
Johansen-Berg,“Training induces changes in white-matter
architecture,”Nature Neuroscience, vol. 12, no. 11, pp. 1370-1371,
2009.
[11] Z.-G Li, W. Zhang, and A. A. Sima, “,e role of
impairedinsulin/IGF action in primary diabetic
encephalopathy,”Brain Research, vol. 1037, no. 1-2, pp. 12–24,
2005.
[12] A. M. Magariños and B. S. McEwen, “Experimental diabetesin
rats causes hippocampal dendritic and synaptic reorga-nization and
increased glucocorticoid reactivity to stress,”Proceedings of the
National Academy of Sciences, vol. 97,no. 20, p. 11056, 2000.
[13] A. Convit, O. T. Wolf, C. Tarshish, and M. J. De
Leon,“Reduced glucose tolerance is associated with poor
memoryperformance and hippocampal atrophy among normal el-derly,”
Proceedings of the National Academy of Sciences,vol. 100, no. 4, p.
2019, 2003.
[14] C. Messier, “Impact of impaired glucose tolerance and type
2diabetes on cognitive aging,” Neurobiology of Aging, vol. 26,no.
1, pp. 26–30, 2005.
[15] B. B. Biswal, M. Mennes, X.-N. Zuo et al., “Toward
discoveryscience of human brain function,” Proceedings of the
NationalAcademy of Sciences, vol. 107, no. 10, p. 4734, 2010.
[16] Y. Cui, Y. Jiao, Y.-C. Chen et al., “Altered spontaneous
brainactivity in type 2 diabetes: a resting-state functional
MRIstudy,” Diabetes, vol. 63, no. 2, pp. 749–760, 2014.
[17] D. S. Margulies, J. Böttger, X. Long et al., “Resting
devel-opments: a review of fMRI post-processing methodologies
forspontaneous brain activity,”Magnetic Resonance Materials
inPhysics, Biology and Medicine, vol. 23, no. 5-6, pp.
289–307,2010.
[18] W. Xia, Y. C. Chen, Y. Luo et al., “Decreased
spontaneousbrain activity and functional connectivity in type 1
diabeticpatients without microvascular complications,”
CellularPhysiology and Biochemistry, vol. 51, 2018.
[19] V. Gulshan, L. Peng, M. Coram et al., “Development
andvalidation of a deep learning algorithm for detection of
di-abetic retinopathy in retinal fundus photographs,”>e
Journal
10 Mathematical Problems in Engineering
-
of the American Medical Association, vol. 316, no. 22,pp.
2402–2410, 2016.
[20] C. Yan and Y. Zang, “DPARSF: a MATLAB toolbox for“pipeline”
data analysis of resting-state fMRI,” Frontiers inSystems
Neuroscience, vol. 4, p. 13, 2010.
[21] M. Lowe, B. Mock, and J. Sorenson, “Functional connectivity
insingle and multislice echoplanar imaging using
resting-statefluctuations,” Neuroimage, vol. 7, no. 2, pp. 119–132,
1998.
[22] Z. Yu-Feng, H. Yong, Z. Chao-Zhe et al., “Altered
baselinebrain activity in children with ADHD revealed by
resting-statefunctional MRI,” Brain and Development, vol. 29, no.
2,pp. 83–91, 2007.
[23] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi,
“In-ception-v4, inception-resnet and the impact of
residualconnections on learning,” in Proceedings of the
>irty-firstAAAI conference on artificial intelligence, San
Francisco, CL,USA, February 2017.
[24] S. Ioffe and C. Szegedy, “Batch normalization:
acceleratingdeep network training by reducing internal covariate
shift,”2015, https://arxiv.org/abs/1502.03167.
[25] K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep
intorectifiers: surpassing human-level performance on
imagenetclassification,” in Proceedings of the IEEE
InternationalConference on Computer Vision, Venice, Italy, October
2015.
[26] V. Nair and G. E. Hinton, “Rectified linear units
improverestricted Boltzmann machines,” in Proceedings of the
27thInternational Conference on Machine Learning (ICML-10),Haifa,
Israel, June 2010.
[27] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever,
andR. Salakhutdinov, “Dropout: a simple way to prevent
neuralnetworks from overfitting,” >e Journal of Machine
LearningResearch, vol. 15, no. 1, pp. 1929–1958, 2014.
[28] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner,
“Gradient-based learning applied to document recognition,”
Proceedingsof the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
[29] I. Arel, D. C. Rose, and T. P. Karnowski, “Deep
machinelearning-a new frontier in artificial intelligence
research[research frontier],” IEEE Computational Intelligence
Maga-zine, vol. 5, no. 4, pp. 13–18, 2010.
[30] K. Simonyan and A. Zisserman, “Very deep
convolutionalnetworks for large-scale image recognition,” 2014,
https://arxiv.org/abs/1409.1556.
[31] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet
clas-sification with deep convolutional neural networks,” in
Pro-ceedings of the NIPS Proceedings Advances In Neural
InformationProcessing Systems, Lake Tahoe, NV, USA, December
2012.
[32] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár,
“Focalloss for dense object detection,” in Proceedings of the
IEEEInternational Conference on Computer Vision, Venice,
Italy,October 2017.
[33] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D.
Parikh,and D. Batra, “Grad-cam: visual explanations from
deepnetworks via gradient-based localization,” in Proceedings ofthe
IEEE International Conference on Computer Vision,Venice, Italy,
October 2017.
[34] L. Prechelt, Early Stopping-but when? Neural networks:
Tricksof the Trade, Springer, 1998.
[35] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual
learningfor image recognition,” in Proceedings of the IEEE
Conferenceon Computer Vision and Pattern Recognition, Las Vegas,
NV,USA, July 2016.
[36] C. Szegedy, W. Liu, Y. Jia et al., “Going deeper with
con-volutions,” in Proceedings of the IEEE Conference on
ComputerVision and Pattern Recognition, Boston, MA, USA, June
2015.
Mathematical Problems in Engineering 11
https://arxiv.org/abs/1502.03167https://arxiv.org/abs/1409.1556https://arxiv.org/abs/1409.1556