V09N02P2-41-51

8/12/2019 V09N02P2-41-51

http://slidepdf.com/reader/full/v09n02p2-41-51 1/11

Neural Information Processing - Letters and Reviews Vol.9, No.2, November 2005

41

Levenberg-Marquardt Learning Algorithm for Integrate-and-Fire NeuronModel

Deepak Mishra, Abhishek Yadav, Sudipta Ray, and Prem K. Kalra

Department of Electrical EngineeringIndian Institute of Technology, Kanpur, India

E-mail: [email protected], [email protected]

(Submitted on July 22, 2006)

Abstract — In this paper, Levenberg-Marquardt (LM) learning algorithm for a singleIntegrate-and-Fire Neuron (IFN) is proposed and tested for various applications in which a

neural network based on multilayer perceptron is conventionally used. It is found that asingle IFN is sufficient for the applications that require a number of neurons in different

hidden layers of a conventional neural network. Several benchmark and real-life problemsof classification and function-approximation have been illustrated. It is observed that theinclusion of robust algorithm and more biological phenomenon in an artificial neuralnetwork can make it more powerful.

Keywords —Levenberg-Marquardt learning algorithm, Integrate-and-Fire neuron model,Multilayer Perceptron, Classification, Function approximation.

1. Introduction

Various researchers have proposed many neuron models for artificial neural networks. Although all ofthese models were primarily inspired from the biological neuron, there is still a gap between the philosophiesused in neuron models for neuroscience studies and neuron models used for artificial neural networks. Some ofthese models exhibit a close correspondence with their biological counterparts while others do not. Author in [1]has pointed out that while brains and neural networks share certain structural features such as massive parallelism, biological networks solve complex problems easily and creatively, and existing neural networks donot. He discussed the issues related to the similarities and dissimilarities between biological and artificial neural

systems of present days. The main focus in the development of a neuron model for artificial neural networks isnot its ability to represent biological activities with its maximum intricacy, but some mathematical properties,

e.g., its capability as a universal function approximator. However, it can be advantageous for artificial neuralnetworks if we can bridge the gap between biology and mathematics by investigating the learning capabilities of biological neuron models for use in the applications of classification, time-series prediction, function

approximation etc. In this work, we used the simplest biological neuron model i.e. integrate and fire model forthis purpose.The first artificial neuron model was proposed by McCulloch and Pitts [2] in 1943. They developed this

neuron model based on the fact that the output of the neuron is 1 if the weighted sum of its inputs is greater thana threshold value and 0 otherwise. Later on a learning rule was proposed in [3] that became initiative for

artificial neural networks. He postulated that the brain learns by changing its connectivity patterns. Authors in[4] presented the most analyzed and most applied learning rule. It was called the least mean square learning rule.

Later it was found that this rule converges in the mean square to the solution that corresponds to least meansquare output error if all the input patterns are of the same length [5]. A single neuron of all the above and manyother neuron models proposed by several scientists and researchers are capable of linear classification [6]. In [7],authors have incorporated various aggregation and activation functions to model the nonlinear input-outputrelationships. In [8], the chaotic behavior in neural networks that represent biological activities in terms of firingrates has been investigated. Author in [16] discussed biologically inspired artificial neurons and authors in [15]

introduced neuronal models with current inputs. Training the integrate-and-fire model with the Informax

LETTER

8/12/2019 V09N02P2-41-51


Learning Algorithm for Integrate-and-Fire Neuron Model D. Mishra, A. Yadav, S. Ray, and P. K. Kalra

42

principle was discussed in [13] and [14]. All the previous approaches are based on the backpropagation learning

algorithm, the only drawback of backpropagation learning algorithm is slow convergence and in order toaccelerate the convergence of the algorithm we have proposed Levenberg-Marquardt [20] Learning Algorithmfor Integrate-and-Fire Neuron Model.

This paper describes implementation of training algorithm for Integrate-Fire neuron (IFN) model based onthe Levenberg-Marquardt (LM) algorithm. In Section 2, a brief discussion on the biological neuron is presentedand formation of Integrate-and-Fire neuron model from Hodgkin-Huxley model is described. Inspired from therelationship between injected current and interspike interval for integrate-and-fire neuron (IFN) model for thelearning is proposed in Section 3. The comparison of the proposed model with classical multi layer perceptron(MLP) is described in Section 4. In Section 5, we concluded our work with a brief discussion.

2. Biological Neurons

2.1 Biological Neuron

A neuron is the fundamental building block of the biological neural networks. A typical neuron has threemajor regions: the soma, the axon and the dendrites. Dendrites form a dendritic tree which is a very fine bush of

thin fibers around the neuron’s body. Dendrites receive information from neurons through axons, i.e., long fibersthat serve as transmission lines. An axon is along cylindrical connection that carries impulses from the neuron.The end part of an axon splits into a fine arborization which terminates in a small end-bulb almost touching thedendrites of neighboring neurons. The axon-dendrite contact organ is called synapse. Details of the biologicalneuron can be found in [11].

2.2 Integrate-and-Fire Neuron Models

The integrate-and-Fire (IF) neuron is the simplest of the threshold-fire neuron models. It is based on themembrane potential equations of HH model [21] with the membrane potential currents omitted. Although theintegrate and-fire model is a very simple, it captures almost all of the important properties of the cortical neuron.Figure 1 shows the basic circuit of this model consists of a capacitor C in parallel with a resistor R driven by a

current I EXT . The driving current can be split into two components, I EXT = I R(t) + I C (t). The first component is theresistive current which passes through the linear resistor R and the second component charges the capacitor C .

Thus

dt

dvC

R

v I EXT += (1)

where, v(t) is the membrane potential. A spike occurs when v(t) reaches a threshold V TH . After the occurrence ofa spike, next spike cannot occur during a refractory period T REF.

This model divides the dynamics of the neuron into two regimes: the subthreshold and the suprathreshold .The Hodgkin-Huxley equations [21] show that in subthreshold regime, sodium and potassium active channelsare almost closed. Therefore, the corresponding terms can be neglected in the voltage equation of the Hodgkin-Huxley model. This gives a first order linear differential equation similar to Eq.(1). In case of the suprathreshold region, if the voltage hits the threshold at time t 0, a spike at time t 0 will be registered and the membrane potential

will be reset to V RESET . The system will remain there for a refractory period T REF . Figure 2 shows the response ofan integrate-and-fire model. The solution of the first-order differential equation in Eq.(2) describing thedynamics of this model in subthreshold region can be found analytically. With v(0) = V REST , the solution ofEq.(2) is given as Eq.(3).

EXT L L I vV Gdt

dvC +−= )( (2)

Lc

tG

L REST c

tG

L

EXT V eV V eG

I t v

L L

+−+−= −−

)()1()( (3)

Here G L is the leakage conductance.Let us assume that v(t) hits V TH at t=T TH. Thus

8/12/2019 V09N02P2-41-51



43

Figure 1. Circuit diagram of an integrate-and-fire Figure 2.Response of an integrate-and-fire neuron model Neuron model

( ) LC

GT

L REST C

GT

L

EXT TH V eV V e

G

I V

LTH LTH

+−+⎟⎟

⎠

⎞

⎜⎜

⎝

⎛ −= −−1 (4)

Therefore T TH can be written as

⎟⎟ ⎠

⎞⎜⎜⎝

⎛

−−

−−=

)(

)(ln

LTH L EXT

L REST L EXT

LTH

V V G I

V V G I

G

C T (5)

Interspike interval T ISI is the summation of T TH and T REF . Thus

T ISI = T TH + T REF (6)

Therefore

REF LTH L EXT

L REST L EXT

L ISI T

V V G I

V V G I

G

C T +⎟

⎟ ⎠

⎞⎜⎜⎝

⎛

−−

−−=

)(

)(ln (7)

Frequency f is reciprocal of interspike interval and hence is given by

REF LTH L EXT

L REST L EXT

L

T V V G I

V V G I

G

C f

+⎟⎟ ⎠

⎞⎜⎜⎝

⎛

−−

−−=

)(

)(ln

1 (8)

3. The Proposed Model

Inspired from the relationship between injected current and interspike interval for integrate-and-fire neuronin Eq.(7), following aggregation function is assumed instead of the weighted sum of a conventional neuron:

....3,2,1))log((1

P k for d xbanet ikiii

n

ik =+= Π

=

(9)

where n is the number of inputs and P is the number of patterns. Eq.(9) corresponds to the first part of integrate-and-fire model which is represented by an RC circuit in Figure 1. In Eq.(9), the net input to the activationfunction of the neuron is considered analogous to the interspike interval and the input x to the neuron isconsidered to be analogous to a function of the injected current I EXT . Weight b is assumed to be associated withthis input to represent the temporal summation when inputs from other synapses are also present.

)(

)(

LTH L EXT

L REST L EXT

V V G I

V V G I x

−−

−−= (10)

8/12/2019 V09N02P2-41-51



44

Figure 3. f-I relationship of an integrate-and-fire neuron model

In view of evidence in support of the presence of multiplicative-like operations in the nervous system,multiplication of net inputs to the activation function is considered. In biological neural systems, this operation

depends on the timings of various spikes. Aggregation of the exponential waveforms with different time-delayshas been approximated by considering a weight associated with the input to the aggregation function. The second part of the integrate-and-fire neuron model is represented in terms of a threshold type of nonlinear block. In this paper, we considered sigmoid function to represent the activity in this block.

k net k e

y−+

=1

1 (11)

3.1 Biological Significance of the Proposed Model

This model is inspired from the fact that the actual shape of the action potential does not contain any

neuronal information. It is the timing of spikes that matters. As the firing frequency is directly related to theinjected current, we considered the f-I characteristic of integrate-and-fire neuron as the backbone of our model.

All artificial neuron models have two functions associated with them, i.e., aggregation and activation. In case ofthe integrate-and-fire model there are two parts in its circuit representation, i.e., RC -circuit and threshold-typenonlinearity. While aggregation is inspired from the f-I relation derived from the response of the RC circuit,nonlinearity is introduced in terms of sigmoid activation function. This activation function is continuous anddifferentiable; therefore it can easily be incorporated in learning. As its output f(x) approaches zero when input xapproaches a large value, and is always greater than 0.9933 for x > 5.0, it can be considered to represent anapproximation of threshold-type nonlinearity to some extent. A substantial body of evidence supports the presence of multiplicative-like operations in the nervous system [10]. Physiological and behavioral data strongly

suggest that the optomotor response of insects to moving stimuli is mediated by a correlation-like operation [11].

Another instance of a multiplication-like operation in the nervous system is the modulation of the receptive fieldlocation of neurons in the posterior parietal cortex by the eye and head positions of the monkey [11].Multiplication operation is used for aggregation of inputs to the artificial neuron in many research papersincluding [7]. In our work, we incorporated this multiplication operation while aggregating inputs to theactivation function.

3.2 Development of the Training Algorithm

Pattern classification with neural classifier basically involves understanding the class boundaries by theclassifier. To attain this capability, the classifier has to undergo a training phase. The same is achieved with thehelp of a training algorithm.

In this paper we have focused on the application of Levenberg-Marquardt algorithm for the learning ofsingle IFN model. In paper [9], author presented a simple steepest descent method to minimize the following

error function:

8/12/2019 V09N02P2-41-51



45

∑=

−= P

k k k yt e

1

2)(

2

1 (12)

where t is the target and y is the actual output of the neuron and e is the function of parameters ai, bi and d i; i = 1,2, ...n. Therefore, the parameter update rule (weight update rule) can be expressed in terms of the following

equations:

i

old i

newi

a

eaa

∂

∂−= η (13)

i

old i

newi

b

ebb

∂

∂−= η (14)

i

old i

newi

c

ecc

∂

∂−= η , for i = 1, 2, 3, ...n. (15)

Partial derivatives of e with respect to parameters ai, bi and d i (i = 1, 2, ..., n) can be given by the followingequations:

⎟⎟

⎠

⎞⎜⎜

⎝

⎛

+

−−=

∂

∂∑= ikiii

kiik k k

P

k

k k

i d xba

xbnet y y yt

a

e

)log(

)log()1()(

1

(16)

⎟⎟ ⎠

⎞⎜⎜⎝

⎛ ⎟⎟ ⎠

⎞⎜⎜⎝

⎛

+−−=

∂

∂∑= i

i

ikiiik k k

P

k k k

i b

a

d xbanet y y yt

b

e

)log(

1)1()(

1

(17)

⎟⎟ ⎠

⎞⎜⎜⎝

⎛

+−−=

∂

∂∑= ikiii

k k k

P

k k k

i d xbanet y y yt

d

e

)log(

1)1()(

1

. (18)

In [9], it is found that the performance of IFN model is quite comparable to the performance of classicalMLP when simple gradient descent algorithm is used for the training of the neural network. We incorporated the

LM method for the training of proposed IFN model and compared with classical feedforward neural network bysolving several classification and function approximation problems.

3.3 The Levenberg-Marquardt Algorithm

Gradient-based training algorithms, like backpropagation, are most commonly used by researchers. Theyare not efficient due the fact that the gradient vanishes at the solution. Hessian-based algorithms allow thenetwork to learn more subtle features of a complicated mapping. The training process converges quickly as thesolution is approached, because the Hessian does not vanish at the solution. To benefit from the advantages ofHessian based training, we focused on the Levenberg-Marquardt Algorithm. The LM algorithm is basically aHessian-based algorithm for nonlinear least square optimization [20]. For neural network training the objective

function is the error function of the type

∑=

−= P

k k k yt e

1

2)(

2

1r

, (19)

where yk is the actual output for the k-th pattern and t k is desired output. P is the total number of training patterns.

z represents the weights and biases of the network. The steps involved in training a neural network using LMalgorithm are as follows:

1) Present all inputs to the network and compute the corresponding network outputs and errors. Compute themean square error over all inputs as in Eq.(19).

2) Compute the Jacobian matrix, J(z) where z represents the weights and biases of the network.

3) Solve the Lavenberg-Marquardt weight update equation to obtain ∆z.

4) Recompute the error using z +∆ z. If this new error is smaller than that computed in step 1, then reduce the

training parameter µ by µ−, let z = z +∆ z, and go back the step 1. If the error is not reduced, then increase µ by µ

+ and go back step 3. The µ− and µ

+ are defined by user.

5) The algorithm is assumed to have converged when the norm of the gradient is less than some predetermined value, or when the error has been reduced to some error goal.

The weight update vector ∆ z is calculated as

8/12/2019 V09N02P2-41-51



46

[ ] E z J I z J z J z T T )()()(1−

+=∆ µ , (20)

where E is a vector of size P calculated as

[ ]

T

P P yt yt yt E −−−= ...

2211 . (21)Here J

T (z)J(z) is referred as the Hessian matrix. I is the identity matrix, µ is the learning parameter. For µ = 0

the algorithm becomes Gauss-Newton method. For very large µ the LM algorithm becomes steepest decent orthe error backpropagation algorithm. The parameter is automatically adjusted at each iteration in order to secureconvergence. The LM algorithm requires computation of the Jacobian J(z) matrix at each iteration step and theinversion of J

T (z)J(z) square matrix.

4. Illustrative Examples

4.1 Classification Problems

XOR Problem: The XOR problem, as compared with other logic operations (NAND, NOR, AND and OR), is probably one of the best and most used nonlinearly separable pattern associator, and consequently provides one

of the most common examples of artificial neural systems for input remapping. We compared the performance ofintegrate-and-fire neuron (IFN) with that of multilayer perceptron (MLP). For this purpose, we considered anMLP with 3 hidden units. Figure 4 shows the mean-square-error (MSE) vs. number of epochs curves for trainingwith MLP and IFN while dealing with the XOR-problem. It is clear from this figure that the proposed modeltakes only 10 iterations while MLP takes 23 iterations for training to achieve an MSE of the order of 0.0001. Table 1 and Figure 5 exhibit the comparison between MLP and IFN in terms of the deviation of actual outputsfrom corresponding targets. It can be seen here that the performance of IFN is almost same as compared withMLP. It means that a single IFN is capable to learn XOR relationship almost 2 times faster than an MLP with 3hidden units. Table 2 shows the comparison of training and testing performance with MLP and IFN whilesolving the XOR-problem.

3-bit Parity Problem: The 3-input XOR has been a very popular benchmark classification problem among theresearchers of ANN. The problem deals with the mapping of 3-bit wide binary numbers into its parity. If the

input pattern consists the odd numbers of 1’ s then the parity is 1, otherwise it is 0. This is considered as adifficult problem because the patterns that are close in the sample space, i.e. the numbers that differ in only one bit, require their classes to be different. For comparison of the performance with IFN and MLP in case of 3-bit Parity Problem, we considered MLP with 5 hidden units. Figure 6 shows the comparison of MSE vs. number ofepochs curves with conventional multilayer perceptron and the proposed single IFN model while training theartificial neural systems for 3-bit Parity Problem. It is clear from this figure that the proposed model takes only45 iterations as compared to 70 iterations taken by MLP for training to achieve an MSE of the order of 0.00001.

Figure 4. Learning profiles for XOR-problem Figure 5. Target vs. Actual Output with MLP and IFNfor XOR-problem

8/12/2019 V09N02P2-41-51



47

Table 1. Outputs of IFN and MLP for XOR-Problem

Input Target Output with MLP Output with IFN

0.1

0.10.90.9

0.1

0.90.10.9

0.1

0.90.90.1

0.1

0.90.90.1

0.1

0.90.90.1

Table 2. Comparison of Training and Testing Performance for XOR-Problem

S.No. Parameter MLP IFN

123

456

78

Training Goal in terms of MSEIterations Needed

Training Time in seconds

Testing Time in secondsMSE for Testing Data

Correlation Coefficient

Percentage Misclassification Number of Parameters

0.0001523

0.312

0.0010.0

1.0

0%11

0.0001510

0.094

0.0010.0

1.0

0%6

Table 3. Outputs of IFN and MLP for 3-bit Problem

Input TargetOutput with

MLPOutput with

IFN

0.10.10.10.10.90.9

0.90.9

0.10.10.90.90.10.1

0.90.9

0.10.90.10.90.10.9

0.10.9

0.10.90.90.10.90.1

0.10.9

0.10.90.90.10.90.1

0.10.9

0.10.90.90.10.90.1

0.10.9

Table 3 and Figure 7 exhibit the comparison between MLP and IFN in terms of the deviation of actual outputsfrom corresponding targets. It can be observed that the performance of IFN is almost same as compared withMLP but IFN is capable to learn this relationship almost 1.5 times faster than that in case of an MLP with 5hidden neurons. Table 4 shows the comparison of training and testing performance with MLP and IFN whilesolving the 3-bit Parity problem.

Figure 6. Learning profiles for 3-bit Parity problem Figure 7. Target vs. Actual Output with MLP and IFN

for 3-bit parity problem

8/12/2019 V09N02P2-41-51



48

Table 4. Comparison of Training and Testing Performance for 3-bit Problem


1

2345678

Training Goal in terms of MSE

Iterations NeededTraining Time in secondsTesting Time in secondsMSE for Testing DataCorrelation Coefficient

Percentage Misclassification Number of Parameters

0.00003

700.3120.0002

0.01.00%22

0.00003

450.0940.0002

0.01.00%9

4.2 Function Approximation Problems

Internet Traffic Data: Short term internet traffic data was supplied by HCL Infinet Ltd. (a leading Indian ISP).This data represents weekly internet traffic (in kbps) with a 30-minute average. Four measurements y(t − 1), y(t−

2), y(t−

4) and y(t −

8) were used to predict y(t). For comparison of the performance with IFN and MLP withInternet Traffic Data, we considered MLP with 6 hidden neurons. Figure 8 shows the comparison of MSE vs.number of epochs curves while training MLP and IFN artificial neural systems for Internet Traffic Data. Thisfigure shows that the proposed model exhibits faster training on this data. Figure 9 shows the comparison between MLP and IFN in terms of the deviation of actual outputs from corresponding targets. Data till 250 sampling instants was used for training and rest of the data was used for testing. It can be observed that the

performance of IFN for training data is almost same as compared with MLP while its performance is better fortesting data, i.e., after 250 sampling instants. Table 5 shows the comparison of training and testing performance

with MLP and IFN for the Internet Traffic Data.

Table 5. Comparison of Training and Testing Performance for Internet Traffic Data


123

4567

Training Goal in terms of MSEIterations Needed

Training Time in seconds

Testing Time in secondsMSE for Testing DataCorrelation Coefficient Number of Parameters

0.005308.9

0.0320.000530.911

32

0.00530

0.79

0.0310.000660.902

12

Figure 8. Learning profiles for Internet Traffic Data Figure 9. Target and Actual Output with MLP and IFNfor Internet-Traffic Data

8/12/2019 V09N02P2-41-51



49

Figure 10. Learning profiles for EEG Data Figure 11. Target and Actual Output with MLP and IFNfor EEG Data

Table 6. Comparison of Training and Testing Performance for EEG Data


1

234567

Training Goal in terms of MSE

Iterations NeededTraining Time in secondsTesting Time in secondsMSE for Testing DataCorrelation Coefficient Number of Parameters

0.01

258.9

0.0320.008140.829

32

0.01

50.790.031

0.009130.812

12

Electroencephalogram Data: Electroencephalogram (EEG) data used in this work was taken from [19].Presence of randomness and chaos [8] in this data makes it interesting for neural network related research. In this problem also, four measurements y(t −1), y(t −2), y(t −4) and y(t −8) were used to predict y(t). We considered

MLP with 5 hidden neurons for the comparison purposes. Figure 10 shows the comparison of MSE vs. numberof epochs curves with conventional multilayer perceptron and the proposed single IFN model while training the

artificial neural systems for EEG data. It is clear from this figure that the proposed model exhibits faster trainingon this data. Figure 11 shows the comparison between MLP and IFN in terms of the deviation of actual outputsfrom corresponding targets. Data till 200 sampling instants was used for training. It can be seen here that the performance of IFN for training as well as testing data is much better than that of MLP. It means that a singleIFN is capable to learn this relationship faster than that in case of an MLP with 5 neurons and its performance onseen as well as unseen data is significantly better. Table 6 shows the comparison of training and testing performance with MLP and IFN while applying for the EEG Data.

5. Conclusions

The training and testing results with different benchmark and real-life problems show that the proposedartificial neural system with a single neuron inspired from the integrate-and-fire neuron model is capable of performing classification and function approximation tasks as efficiently as a multilayer perceptron with manyneurons and in some cases its learning is even better than that of a multilayer perceptron. It is also observed thattraining and testing times in case of IFN are significantly less as compared with MLP. The LM algorithm isemployed for the training and the results indicate that the application of LM algorithm is very efficient for thetraining. Future scope of this work includes incorporation of these neurons in a network and analyticalinvestigation of its learning capabilities (e.g., as universal functions approximator).

Acknowledgment: The authors would like to thank HCL Infinet Ltd. (India) for providing Internet-Traffic Data.

8/12/2019 V09N02P2-41-51



50

References

[1] W.J. Freeman, “Why neural networks don’t yet fly: inquiry into the neurodynamics of biological

intelligence,” IEEE International Conference on Neural Networks, 24-27 July 1988 pp.1-7, vol.2, 1988.

[2] W. McCulloch and W. Pitts, “A logical calculus of the ideas immanent in nervous activity,” Bulletin of Mathematical Biophysics, vol.5, pp. 115-133, 1943.

[3] D. Hebb, “Organization of behavior,” John Weiley and Sons, New York, 1949.

[4] B. Widrow and M. E. Hoff, “Adaptive switching circuits,” IREWESCON Connection Recors, IRS, New York,1960.

[5] B. Widrow and S. Steams, “Adaptive signal processing,” Prentice-Hall , Englewood Cliffs, NJ., 1985.

[6] M. Sinha, D.K. Chaturvedi and P.K. Kalra, “Development of flexible neural network.” Journal of IE(I),vol.83, 2002.

[7] R. N. Yadav, V. Singh and P. K. Kalra, “Classification using single neuron,” Proceedings of IEEE International Conference on Industrial Informatics, 2003, pp.124-129, 21-24 Aug. 2003, Canada.

[8] D. Mishra, A. Yadav and P. K. Kalra, “Chaotic Behavior in Neural Networks and FitzHugh-Nagumo Neuronal Model,” Proceedings of ICONIP-2004, LNCS 3316, pp.868-873, Dec. 2004, India.

[9] A. Yadav, D. Mishra, R.N. Yadav, S. Ray, and P. K. Kalra, “Learning with Single Integrate-and-Fire Neuron,” IEEE International Joint Conference on Neural Network , IJCNN-2005, Montreal (Canada), 2005.

[10] C. Koch and T. Poggio, “Multiplying with synapses and neurons,” Single Neuron Computation, AcademicPress: Boston, Massachusetts, pp.315-315, 1992.

[11] C. Koch, “Biophysics of Computation: Information Processing in Single Neurons,” Oxford University Press,1999.

[12] P. Chandra and Y. Singh, “Feedforward sigmoidal networks - equicontinuity and fault-tolerance properties,” IEEE Transactions on Neural Networks, vol.15, pp.1350-1366, Nov. 2004.

[13] J. Feng, H. Buxton and Y. C. Deng, “Training the integrate-and-fire model with the Informax principle I,” J.

Phys. A, vol. 35, pp. 23792394, 2002.

[14] J. Feng, Y. Sun, H. Buxton and G. Wei, “Training integrate-and-fire neurons with the Informax principleII,” IEEE Transactions on Neural Networks, vol.14, pp. 326-336, March 2003.

[15] J. Feng and G. Li, “Neuronal models with current inputs,” J. Phys. A, vol. 24, pp. 1649-1664, 2001.

[16] M. Scholles, B. J. Hosticka, M. Kesper, P. Richert and M. Schwarz, “Biologically-inspired artificialneurons: modeling and applications,” Proceedings of 1993 International Joint Conference on Neural Networks, IJCNN ’93-Nagoya, vol.3, 25-29 Oct. 1993, pp.2300-2303, vol.3.

[17] N. Iannella and A. Back, “A spiking neural network architecture for nonlinear function approximation,”

Proceedings of the 1999 IEEE Signal Processing Society Workshop - Neural Networks for Signal Processing IX, 23-25 Aug. 1999, pp.139-146.

[18] S. C. Liu and R. Douglas, “Temporal coding in a silicon network of integrate-and-fire neurons,” IEEETransactions on Neural Networks, pp.1305-1314, vol.15, Sept. 2004.

[19] www.cs.colostate.edu/eeg/

[20] M.T. Hagan, M.B. Mehnaj, “Training Feedforward Networks with the Marquardt Algorithm,” IEEETransaction on Neural Networks, Vol. 5, No. 6, 1994.

[21] A.L. Hodgkin and A.F. Huxley, “A quantitative description of membrane current and its application toconduction and excitation in nerve,” Journal of Physiology, 117:500–544, 1952.

8/12/2019 V09N02P2-41-51



51

Deepak Mishra was born in Seoni (Madhya Pradesh, India) on 18th July, 1978. He is

pursuing his Ph. D. in Electrical Engineering in Indian Institute of Technology Kanpur,India. His major field of study is Neural Networks and Computational Neuroscience.

Mr. Mishra is a student member of IEEE society. (Home page: http://home.iitk.ac.in/

~dkmishra)

Abhishek Yadav was born in Mainpuri (Uttar Pradesh, India) on 21st October, 1976.He is pursuing his M. Tech. in Electrical Engineering in Indian Institute of TechnologyKanpur, India. His major field of study is Computational Neuroscience. He is workingas ASSISTANT PROFESSOR in the Department of Electrical Engineering, College of

Technology, G. B. Pant University of Agriculture and Technology, Pantnagar,Uttaranchal, India. He is currently on leave to pursue his higher studies. Mr. Yadav is a

student member of IEEE society.

Sudipta Ray was born in Rishra (West Bengal, India) on 3rd September, 1980. He is pursuing his M. Tech. in Electrical Engineering in Indian Institute of TechnologyKanpur, India. His major field of study is Computational Neuroscience.

Prem K. Kalra was born in Agra (Uttar Pradesh, India) on 25th October , 1956. Hereceived his BSc (Engg.) degree from DEI Agra, India in 1978, M.Tech degree fromIndian Institute of Technology, Kanpur, India in 1982 and Ph.D. degree fromManitoba University, Canada in 1987. He worked as assistant professor in theDepartment of Electrical Engineering, Montana State University Bozeman, MT, USAfrom January 1987 to June 1988. In July-August 1988 he was the visiting assistant professor in the Department of Electrical Engineering, University of Washington

Seattle, WA, USA. Since September 1988 he is with Department of ElectricalEngineering, Indian Institute of Technology Kanpur, India where he is a Professor. Dr.

Kalra is a member of IEEE, fellow of IETE and Life member of IE(I), India. He has

published over 150 papers in reputed National and International journals andconferences. His research interests are Expert Systems applications, Fuzzy Logic, Neural Networks and PowerSystems.

V09N02P2-41-51

Documents