-
IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, VOL. 4,
NO. 3, JUNE 2010 139
Analog VLSI Biophysical Neurons and SynapsesWith Programmable
Membrane Channel Kinetics
Theodore Yu, Student Member, IEEE, and Gert Cauwenberghs, Senior
Member, IEEE
Abstract—We present and characterize an analog VLSI networkof 4
spiking neurons and 12 conductance-based synapses, imple-menting a
silicon model of biophysical membrane dynamics anddetailed channel
kinetics in 384 digitally programmable parame-ters. Each neuron in
the analog VLSI chip (NeuroDyn) implementsgeneralized
Hodgkin-Huxley neural dynamics in 3 channel vari-ables, each with
16 parameters defining channel conductance,reversal potential, and
voltage-dependence profile of the channelkinetics. Likewise, 12
synaptic channel variables implement arate-based first-order
kinetic model of neurotransmitter andreceptor dynamics, accounting
for NMDA and non-NMDA typechemical synapses. The biophysical origin
of all 384 parameters in24 channel variables supports direct
interpretation of the resultsof adapting/tuning the parameters in
terms of neurobiology. Wepresent experimental results from the chip
characterizing singleneuron dynamics, single synapse dynamics, and
multi-neuronnetwork dynamics showing phase-locking behavior as a
functionof synaptic coupling strength. Uniform temporal scaling of
thedynamics of membrane and gating variables is demonstrated
bytuning a single current parameter, yielding variable speed
outputexceeding real time. The 0.5 � CMOS chip measures 3 mm3 mm,
and consumes 1.29 mW.
Index Terms—Neuromorphic engineering, reconfigurableneural and
synaptic dynamics, silicon neurons, subthresholdmetal–oxide
semiconductor (MOS), translinear circuits.
I. INTRODUCTION
N EUROMORPHIC engineering [1] takes inspiration fromneurobiology
in the design of artificial neural systems insilicon integrated
circuits (ICs), based on the function and struc-tural organization
of biological nervous systems. By emulatingthe form and
architecture of biological systems, neuromorphicengineering seeks
to emulate their function as well. Since thefirst silicon model of
a biophysical neuron in 1990 [2], great ad-vances have been made in
the detail and scale of the modelingthe neural function in silicon.
Recently, the focus of the neu-romorphic engineering effort in
silicon modeling of the nervoussystem has shifted from the sensory
periphery to central nervous
Manuscript received October 12, 2009; revised January 06, 2010.
Currentversion published May 26, 2010. This work was supported by
the National In-stitute of Aging, National Science Foundation, and
Defense Advanced ResearchProjects Agency. This paper was
recommended by Associate Editor P. Häfliger.
T. Yu is with the Department of Electrical and Computer
Engineering, JacobsSchool of Engineering, University of California,
San Diego, CA 92039 USA.He is also with the Institute for Neural
Computation, University of California,San Diego, CA 92039 USA
(e-mail: [email protected]).
G. Cauwenberghs is with the Department of Bioengineering, Jacobs
Schoolof Engineering, University of California, San Diego, CA 92039
USA. He is alsowith the Institute for Neural Computation,
University of California, San Diego,CA 92039 USA (e-mail:
[email protected]).
Color versions of one or more of the figures in this paper are
available onlineat http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TBCAS.2010.2048566
Fig. 1. An ion membrane channel is pictured [13] (top) with an
accompanyingmathematical expression (bottom) describing the channel
kinetics in terms ofthe opening � and closing � rates.
function addressing higher levels of integration and
cognitiveprocessing in the cortex and other brain regions [3]–[6],
settingthe stage for further advances toward closed-loop
integration ofbiological and silicon neural systems [8]–[12].
Biophysical modeling and implementation of the neural func-tion
require a careful account of channel opening and closingkinetics
and their role in ion transport through membranes thatgive rise to
the rich neuronal and synaptic dynamics observedin neurobiology
[13] (Fig. 1). Hodgkin and Huxley’s seminalwork in the
investigation and formalization of neuron membranedynamics have
long been the standard of biophysical accuracy[14]. The difficulty
of realizing the complex functional formof the Hodgkin–Huxley
membrane currents and channel vari-ables in analog circuits has
motivated alternative realizationsby simplifications in the model
[15]–[18]. The prevailing ap-proach in neuromorphic engineering
design has been to abstractthe neuron membrane action potential to
discrete-time spikeevents in simplified models that capture the
essence of integrate-and-fire dynamics and synaptic coupling
between large numbersof neurons in an address-event representation
[19]–[25]. The ad-vantage of these approaches is that they support
event-basedinterchip communication, including direct input from
neuro-morphic audition [26] and vision [27] sensors, and may leadto
highly efficient and densely integrated implementations inanalog
very-large scale integrated (VLSI) silicon [28], [29].
Here, we offer an alternative neuromorphic engineeringapproach
that targets applications where biophysical detail inmodeling
neural and synaptic dynamics at the level of channelkinetics is
critical. Examples of these applications includemodeling of the
effect of neuromodulators, neurotoxins, aswell as neurodegenerative
diseases on neural and synaptic
1932-4545/$26.00 © 2010 IEEE
-
140 IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, VOL.
4, NO. 3, JUNE 2010
function through parameter changes in the channel kinetics.
Forthese and other applications in computational neuroscience,a
direct correspondence between the parameters governingthe
biophysics of neural and synaptic function and those inthe
implemented computational model is greatly beneficial[30]–[33]. The
approach we propose here is the first in analogVLSI to fully model
the general voltage dependence of ratekinetics in the opening and
closing of membrane ion chan-nels. We illustrate this approach with
the implementation anddemonstration of NeuroDyn, an analog VLSI
network of fourneurons and 12 chemical synapses with a total of 384
digitallyprogrammable parameters governing the channel
conductance,reversal potential, and opening and closing kinetics
voltageprofile of 24 individually configurable channel
variables.
Hodgkin and Huxley in their landmark paper [14] resortedto
heuristics in curve-fitting the rate kinetics of channel vari-ables
observed through ingenious measurements on the squidgiant axon. Any
model replicating the precise functional formof this heuristic fit
would, at best, produce an approximationto squid giant axons. To a
large extent, the variety in dynamicsbetween different neuron types
in different organisms as wellas the anomalies due to biomolecular
agents and neurodegen-erative processes acting on membrane channels
arise from theresulting differences in channel properties. These
channel prop-erties are compactly characterized in our model by
16
parameters for each channel variable specifyingthe voltage
dependence of channel opening and closing rates(seven regression
points each), besides values for the channelconductance and
reversal potential. Fewer parameters wouldimpair the flexibility in
modeling neural and synaptic diversityin healthy and diseased
nervous systems, although fewer pa-rameters would be appropriate in
special purpose implementa-tions of specific model instances and
their functional abstrac-tions [34]–[38] where the application
warrants efficiency ratherthan flexibility and biophysical
explanatory power in parameterselections. Likewise, extensions to
the models to incorporatefurther biophysical detail, such as
short-term synaptic adapta-tion [39]–[42] and multicompartmental
dynamics through linear[43] and nonlinear [44] dendritic coupling
would incur largernumbers of parameters where the need for the
extended modelsjustifies the increase in implementation
complexity.
While the implementation of parameterized channel kineticrate
equations in NeuroDyn provides the capacity to model alarge variety
of neuron and synapse behaviors, it requires tuningover a large
number of parameters. Since each of these pa-rameters has a direct
physical correspondence in channel ki-netics and membrane dynamics,
values for these parameters canbe obtained from physical
considerations and measurements.Fine tuning of these parameters, to
account for uncertaintiesin the modeling as well as imprecisions in
the implementedmodel, would still be desirable. Extensive parameter
fine-tuningwas found to be unnecessary to address transistor
mismatch. Asimple calibration and parameter fitting procedure
proved ade-quate to counteract mismatch and nonlinearities in
setting pa-rameters in the biophysical model to desired values. The
cor-respondence between biophysical and circuit parameters is
de-scribed in Section IV, with experimental alignment
documented
Fig. 2. NeuroDyn chip micrograph. (a, top left) and system
diagram (b,top right). Four neurons are interconnected with 12
synapses, each withprogrammable channel kinetics, conductances, and
reversal potentials (seeTable I).
in Section V and the parameter alignment procedure detailed
inAppendix B.
In the present implementation, we have aimed for
functionalflexibility and real-time control over parameters and
internaldynamics, rather than efficiency and density of
integration.The NeuroDyn system interfaces through the universal
serialbus (USB) to Matlab software on a workstation to update
the384 parameters in rea time, and to continuously control
andobserve each of the four membrane potentials and 24
channelvariables. To support this level of programmability, all
parame-ters are stored locally on-chip in digital registers
interfacing toa bank of 384 current multiplying digital-to-analog
converters(DACs). The neuromorphic modeling approach presented
hereis extendable to other implementations where parameters maybe
shared across individual neurons and/or channels for
greaterefficiency, and combined with floating-gate nonvolatile
analogstorage [45]–[51] or dynamically refreshed volatile
analogstorage [52] of the parameters for greater density of
integration.For example, central pattern generators require only a
fewneurons for implementation, yet they can characterize
complexbehavior [11].
We envisage NeuroDyn as an enabling tool for computationaland
systems neuroscience, since all internal dynamical variablesand
their parameters are grounded in the biophysics of mem-branes and
ion channels. With its analog interface to the phys-ical world, the
NeuroDyn chip (Fig. 2) may also serve as an elec-tronic training
tool for budding neuroscientists and neurobiolo-gists to practice
patch clamp recording and other experimentaltechniques on “virtual”
neurons. The NeuroDyn system containsvarious analog and digital
exposed probes in the circuit boardthat allow for a real-time
interface to the internal membrane po-tential and channel
dynamics.
Furthermore, the low-power and efficient circuit
implemen-tation, combined with extensions for hardcoded parameter
set-tings or high-density analog storage, support applications of
thedevice as an implantable computational neural interface in
vivo.These approaches, as described in [7], have great potential in
therealization of intelligent neural prostheses when combined
withembedded signal processing to process incoming neural spikedata
streams [8], [9] and activate prostheses [10], [12].
The analog VLSI design of the NeuroDyn system, and prelim-inary
experimental results were presented in [53]. First resultson
coupled neural dynamics with inhibitory synapses were re-ported in
[54]. Here, we provide details on the circuit implemen-
-
YU AND CAUWENBERGHS: ANALOG VLSI BIOPHYSICAL NEURONS AND
SYNAPSES 141
TABLE INEURODYN DAC PARAMETERS
All rates �, � are functions of voltage as 7-point sigmoidal
splines
(Section IV-A-1).
tation and complete experimental characterization of the
neuraland synaptic circuits, and present calibration and
parameter-fit-ting procedures to align neural and synaptic
characteristics frommodels or recorded data onto the digitally
programmable analoghardware. We demonstrate the operation of the
system by repli-cating opening and closing rates, gating variable
kinetics, andaction potentials of the Hodgkin–Huxley model, and
study thedynamics of a network of two neurons coupled through
recip-rocal inhibitory synapses.
II. NEURODYN ARCHITECTURE
A. System Overview
The NeuroDyn board consists of four Hodgkin–Huxley (HH)-based
neurons fully connected through 12 conductance-basedsynapses as
shown in Fig. 2(b). All parameters are individuallyaddressable and
individually programmable and are biophysi-cally based governing
the conductances, reversal potentials, andvoltage dependence of the
channel kinetics. There are a totalof 384 programmable parameters
governing the dynamics asshown in Table I. Each parameter is stored
on-chip in a 10-bDAC.
B. Chip Architecture
The NeuroDyn chip is organized into four quadrants witheach
quadrant containing one neuron, and three synaptic inputsfrom the
other neurons. Each neural and synaptic membranechannel current
follows the same general form as illustrated inFig. 3. Each channel
current is a product of a conductance termmodulated by a product of
gating variables and the differencebetween the membrane voltage and
reverse potential as in (2).The similar form for the neuron channel
currents and synapticcurrent allows for a small number of circuits
to model each com-ponent of the channel current.
III. BIOPHYSICAL MODELS
A. Membrane Dynamics
The HH membrane dynamics [14], including conductance-based
synaptic input, are described by
(1)
Fig. 3. System diagram for one of the four neurons in the
NeuroDyn chip.
where , and
(2)
All of the conductances in the model, including the
synapticconductances , are positive. Excitatory synapses are
char-acterized by reversal potentials above the rest
potential,whereas for inhibitory synapses, the reversal potential
isbelow the rest potential.
B. Channel Kinetics
The neuron channel gating variables , , and , as in theHH neuron
formulation, are modeled by a rate-based first-orderapproximation
to the kinetics governing the random openingand closing of membrane
channels
(3)
(4)
(5)
where the three channel variables , , and for each neurondenote
the fractions of corresponding channel gates in the open
state, and where the and parameters are the
correspondingvoltage-dependent opening and closing rates.
Similarly, the synaptic channel currents are modeled by
usingfirst-order kinetics in the receptor variables , the fraction
ofreceptors in the open state [55]
(6)
The opening rates are dependent on presynaptic voltage, modeling
the release of the neurotransmitter and its binding
at the postsynaptic receptor, affecting the channel opening.
In
-
142 IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, VOL.
4, NO. 3, JUNE 2010
Fig. 4. Generalized channel rate variables � and � implemented
in the currentdomain with additive seven-point sigmoidal functions.
Programmable parame-ters scaling the sigmoidal currents are stored
in 10-b MOSFET-only R-2R DACswith additional bit governing
polarity.
contrast, the closing rates are generally dependent on
post-synaptic voltage . For non-NMDA synapses, this dependenceis a
constant, given the rate of unbinding and resulting decreasein the
channel conductance following presynaptic deactivation.For NMDA
synapses, this dependence models the effect of mag-nesium blocking
of synaptic conductance triggered by postsy-naptic potential.
The point of departure from the prevailing models in
compu-tational neuroscience is that the channel gate/receptor
openingand closing rates are not specified and implemented as
ana-lytic functions, but are parameterized as regression
functions,leaving significant flexibility in accommodating
diversity inchannel properties in the implemented model.
IV. NEUROMORPHIC IMPLEMENTATION ANDCHARACTERIZATION
A. Voltage-Dependent Channel Kinetics
1) Seven-Point Sigmoidal Spline Regression: Opening ratesand
closing rates are modeled and regressed as 7-point
additive spline sigmoidal functions implemented in the
circuitillustrated in Fig. 4. Each sigmoid in the regression
splineis implemented by a simple differential pair of
metal–oxidesemiconductor (MOS) transistors operating in
subthreshold,where a bias current scales the sigmoid while a bias
voltagedetermines the sigmoid offset [1]. These bias voltages
arelinearly spaced and are set through a voltage divider
resistorstring. A programmable 10-b metal–oxide
semiconductorfield-effect transistor (MOSFET)-only R-2R
digital-to-analogconverter (DAC) supplies the bias currents. An
additional signbit controls a switch circuit that determines the
polarity of theoutput current slope, which selects either
monotonically in-creasing or monotonically decreasing voltage
dependence. Theoutput currents from each differential pair are then
additivelycombined to provide the composite function for the
opening orclosing rate. Each of the spline amplitudes and sign
selectionbits are individually programmable. By properly setting
thecurrent bias values and sign bit for each of the seven
sigmoidal
functions, the summation can accommodate a wide range
offunctions approximating typical rate functions and
(7)
where the output current denotes either one of the andrates, and
where is the thermal voltage.To enforce a consistent temporal scale
of the dynamics across
membrane and gating variables, the currents implementing
theopening and closing rates as well as the membrane
conductancesare globally scaled with a current that drives the
multiplyingDACs
(8)
(9)
(10)
and, thus, uniformly controls the time base of all dynamic
vari-ables with a global temporal scale parameter .
2) Programmable Channel Kinetics: The gate opening andclosing
variables for one neuron were programmed to imple-ment the HH model
(Section III), with the target functions forthe channel kinetics
defined according to the HH opening andclosing rate functions. The
sigmoidal spline functions weremeasured from the chip to provide
the basis functions at eachspline location. Rectified linear
least-squares optimization wasthen applied to determine the current
bias parameters basedon chip characteristics. Further parameter
fitting details areprovided in Appendix B. The 10-b programming for
each of theseven spline amplitude levels in the regression
functions resultsin the fit illustrated in Fig. 5. The closeness of
fit is limited bythe dynamic range of the 10-b DACs to
simultaneously fit thesteep slope of the gating variable and the
gradual slopesof the and parameters. Parameter fitting was
achievedby applying rectified linear regression and iterative
linearleast-squares residue correction as described in Appendix
B.
B. Gated Conductances
Gating variables , , , and are implemented as cur-rents by the
log-domain circuit shown in Fig. 6, which imple-ments the kinetics
(3)–(6) as
(11)
where represents the gating variable output current, andwhere is
a current reference that only affects the amplitudescale of the
gating variables, but not the temporal scale of theirdynamics.
The use of MOS transistors operating in the subthreshold re-gion
allows analog multiplication through the exponential rela-tionship
between the transistor input voltage and output currentin
translinear circuits [56]. The addition of the capacitor
trans-forms the circuit from a translinear multiplier into a
log-domainfilter [59] that implements the desired first-order
dynamics. Thederivation is provided in Appendix A.
-
YU AND CAUWENBERGHS: ANALOG VLSI BIOPHYSICAL NEURONS AND
SYNAPSES 143
Fig. 5. Target and measured channel opening and closing rates �
and � forgating variables �, �, and � of a single NeuroDyn neuron
approximating theHH model, obtained by fitting the on-chip
programmed parameters to the HHmodel.
The circuit is similar in implementation complexity to a
pre-vious implementation of rate kinetics [33] but avoids the
back-gate effect in the bulk CMOS process on the linearity in
thefirst-order dynamics, and provides full programmability in
thevoltage profile of the dynamics. The circuit offers 14
parame-ters, specifying the detailed voltage dependence of the
openingand closing rates offering flexibility in accurately
modeling thechannel kinetics.
1) Steady-State (in)Activation Functions: The
steady-state(in)activation functions for one NeuroDyn programmed to
repli-cate the HH model are shown in Fig. 7. These data were
gath-ered by clamping the membrane voltage and slowly sweepingthe
membrane voltage while recording the values for the , ,and gating
variables. The results closely match the expectedsteady-state
values according to the HH model [14]. Notice that
Fig. 6. Log-domain circuit implementing channel kinetics
(3)–(6).
Fig. 7. Steady-state (in)activation functions measured on one
neuron of Neu-roDyn programmed to replicate the HH model (a) for
fast setting of the neuronparameters and (b) for slow setting of
the parameters, obtained without recali-bration by increasing the
global temporal scale parameter � 2.5-fold.
there is little variation between the fast and slow time-scale
im-plementations obtained by varying the global temporal scale
pa-rameter . This desirable time independence in the steady
state(in)activation functions is clearly reflected in Fig. 7.
2) Voltage-Dependent Time Constants: The
measuredvoltage-dependent time constants of the implemented HHmodel
are shown in Fig. 8. The time constants were estimatedby averaging
the measured rise and fall times of changes inthe gating variables
under alternating small-amplitude voltagesteps around the swept
membrane voltage. The observed dy-namics are consistent with the HH
model [14] except for the
-
144 IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, VOL.
4, NO. 3, JUNE 2010
Fig. 8. Voltage-dependent time constants measured for one
NeuroDyn neuronapproximating the HH model.
larger observed time constants of the gating variable due
todelays imposed by the on-chip output buffers.
C. Translinear Multiplier
A translinear multiplier shown in Fig. 9 implements gatingof the
membrane conductances with the gating variables. Atranslinear
multiplier exploits the zero sum of voltages along aloop to
implement a multiplication of current sources [57] and[56]
(12)
where is the same current reference controlling the ampli-tude
of the gating variables (11) for dimensionless operation.Similar
translinear circuits implement the other gated conduc-tances of the
form with stages, where and
for the K+ channel; and and for the conduc-tance-based
synapses.
1) Channel Conductance Dynamics: The channel conduc-tance
dynamics of an implemented HH model are shown inFig. 10. The
membrane voltage was clamped to the specifiedvoltage levels and
then released to measure the conductance dy-namics for the Na+ and
K+ channels. The results for the Na+channel show an increase in the
magnitude and speed (as seenin the width of the curve) of the curve
proportional to the mag-nitude of the depolarizing voltage step.
The results for the K+channel also reflect an increase in magnitude
and slope propor-tional to the magnitude of the depolarizing
voltage step.
D. Membrane Dynamics
Each membrane conductance is implemented by a
differentialtransconductance amplifier, linearized through shunting
in thedifferential pairs for wide dynamic range in subthreshold
MOSoperation [58]. Unity gain connection of the amplifier yields
amembrane current
(13)
For each of the membrane conductances, one amplifier is
con-nected in parallel as shown in Fig. 11. A capacitanceon the
membrane node realizes the membrane dynamics (1).
V. EXPERIMENTAL RESULTS
A. Neuron Spiking Dynamics
We observed the dynamics of the membrane and gating vari-ables
for one neuron programmed to implement the HH model.We also
demonstrated temporal control through the variation ofthe global
temporal scale parameter set by current . As shownin Fig. 12, the
variation of scales the time axis of the wave-forms by a factor
greater than 2. The amplitude scaling in thegating parameters
reflects scaling proportional to consistentwith (8)–(10). We
implemented the HH model in one neuron andobserved the dynamics of
the membrane and gating variables asshown in Fig. 12. A small,
constant is applied to the neuronin order to provide dc input
inducing spiking dynamics.
B. Synapse Dynamics
To observe the synapse dynamics, we took the spiking HHneuron
from before and connected that as a presynaptic inputto a synapse.
The synapse parameters were configured to im-plement a inhibitory
synapse. The configuration is il-lustrated in Fig. 13(a). The
conductance curves of the synapseare shown in Fig. 13(b). The
synapse conductance curve was ob-served to rise quickly in time
with the spiking neuron and slowlydecay in accordance with the
expected behavior.
C. Neuron Network Dynamics
We chose to demonstrate synaptic dynamics using a simplenetwork
of two neurons coupled with reciprocal inhibitorysynapses as
illustrated in Fig. 14. The neuron parameters wereconfigured to
implement the channel kinetic rate equations ofthe HH model. The
synapse parameters were configured toimplement inhibitory synapses.
The network wasfirst initialized by disconnecting all of the
synaptic connectionsby setting each synaptic conductance to zero.
Then, separateexternal currents and were applied to the neuronsand
, respectively, to induce spiking behavior. The values for
and were chosen so that there was a small differencein the
spiking frequency between the two neurons. Then, thesynaptic
conductances were increased until coupling was ob-served between
the neurons, in the form of phase-locking. Theresulting waveforms
are shown in Fig. 15. Notice that especiallyin the oscilloscope
capture from the coupled neurons, that thereis observable timing
jitter in the spiking neuron waveforms.This phase noise is
primarily due to the noise intrinsic in theanalog circuit
implementation. Noise has also been observedin in vivo recordings
of neuronal activity that can be attributedto thermal, stochastic,
and other sources [60]. Thus, the noisefrom the circuit
implementation may prove advantageous toprovide a more biorealistic
implementation.
VI. CONCLUSION
We presented an analog VLSI network of biophysical neu-rons and
synapses that implements general detailed models of
-
YU AND CAUWENBERGHS: ANALOG VLSI BIOPHYSICAL NEURONS AND
SYNAPSES 145
Fig. 9. Translinear circuit implementing gated conductances of
the form � �� such as � �� and � � . Synaptic gated conductances ��
are implementedby a two-stage version of the five-stage translinear
circuit shown.
Fig. 10. Channel conductance dynamics measured from one NeuroDyn
neuronapproximating the HH model. (a) for the Na+ channel and (b)
for the K+channel. Channel conductance was measured for different
depolarizing voltagesteps away from the resting potential.
continuous-time membrane dynamics and channel kinetics ina fully
digitally programmable and reconfigurable interface.Each neuron and
synapse in the network offer individuallyprogrammable parameters
setting reversal potentials, conduc-tances, and voltage-dependent
channel opening and closingrates. Least squares parameter fitting
was shown to accuratelyreproduce biophysical neural data of channel
opening andclosing rates, gating variable dynamics, and action
potentials.We further observed coupled neural spiking dynamics in
anetwork with inhibitory synapses.
The implemented neural model extends on the HH formu-lation by
allowing for arbitrary voltage profiles for channel
Fig. 11. Transconductance-C circuit implementing membrane
dynamics forone neuron with synaptic input from the other three
neurons.
opening and closing rates. The approach can be further
extendedby using similar principles to include adaptation
mechanismsusing calcium dynamics, and to implement resistively
coupledmulticompartment neurons. The work shown here representsa
first step toward detailed silicon modeling of general neuraland
synaptic dynamics, combining digital and analog VLSI formaximum
configurability and functionality.
APPENDIX ADERIVATION OF THE CIRCUIT IMPLEMENTING KINETICS OF
CHANNEL GATING VARIABLES
Here, we derive the dynamics of the circuit in Fig. 6 by
im-plementing the kinetics in the channel gating variables by
com-bining the and rate currents. The log-domain circuit [59]uses
the dynamic translinear principle, exploiting the exponen-tial
current-voltage dependence of MOS transistors operatingin the
subthreshold region [56]. Drain currents are modeled as
in gate voltage and sourcevoltage relative to the bulk, where is
the thermal voltage,
and is the bulk back-gate effect factor [57]. The re-sulting
translinear loop relation combinedwith Kirchhoff’s current law
leads to
(14)
-
146 IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, VOL.
4, NO. 3, JUNE 2010
Fig. 12. Measured dynamics of membrane voltage � and gating
variables�, �, and � for a single HH neuron. (a) and (b) show the
effect of settingthe global temporal scale parameter � , uniformly
speeding or slowing the dy-namics across all variables.
Fig. 13. (a) Synapse with presynaptic spiking neuron diagram
(above). (b) Os-cilloscope trace of the conductance curve of a
synapse with a spiking presynapticneuron input. Notice that the
spiking neuron waveform (purple) and conduc-tance curves for the
synapse (pink) are in phase (below).
Since , thevoltage dynamics of in (14) are expressed in the
currentlog domain as
(15)
leading to (11).
Fig. 14. Coupled neurons diagram. Two spiking neurons are
connected withinhibitory synapses.
APPENDIX BCALIBRATION PROCEDURE FOR AND PARAMETER FITTING
A. Rectified Linear Regression
Let be the measured or functionof obtained with current bias
parameter settings .Then, for calibration, we measure the
individual sigmoid con-tributions
(16)
where for , and 0 otherwise. Hence, because oflinearity in
current summation, we may assume
(17)
To proceed, we perform a first linear fit ofto the target
function by using rectified linear least-squares regression in the
coefficients
(18)
The rectification is necessary because of the positivity
con-straints on the bias current parameters.
B. Iterative Linear Least-Squares Residue Correction
Next, we correct for residual errors due to nonlinearities inthe
current multiplying DACs by implementing the sigmoidweighting (17).
To do so, we linearize the system around thecurrent operating
point, by regressing the residue to locallydifferential sigmoid
contributions
where is chosen sufficiently small for the linear analysis to
bevalid, but sufficiently large for reliable measurement. We
pro-ceed with another round of rectified linear least-squares
regres-sion in the parameters subject to the same
positivityconstraints, and iterate until the changes in parameter
values
are small compared to the DAC precision.
ACKNOWLEDGMENT
The authors would like to thank S. Deiss and M. Chi forhelp with
the experimental setup, and the MOSIS EducationalProgram for
fabricating the chip. The authors also benefited
-
YU AND CAUWENBERGHS: ANALOG VLSI BIOPHYSICAL NEURONS AND
SYNAPSES 147
Fig. 15. Oscilloscope traces showing synaptic coupling in neural
dynamics.(a) Through (c) individual uncoupled spiking neurons and
(d) neurons coupledwith inhibitory synapses spiking in
synchrony.
from interactions with participants at the 2008 National
ScienceFoundation Workshop on Neuromorphic Cognition Engineeringin
Telluride, CO.
REFERENCES
[1] C. A. Mead, Analog VLSI and Neural Systems. Reading, MA:
Ad-dison-Wesley, 1989.
[2] M. Mahowald and R. Douglas, “A silicon neuron,” Nature, vol.
354,pp. 515–518, 1991.
[3] R. Serrano-Gotarredona, M. Oster, P. Lichtsteiner, A.
Linares-Bar-ranco, R. Paz-Vicente, F. Gomez-Rodriguez, K.
Camunas-Mesa,R. Berner, M. Rivas-Perez, T. Delbruck, S. C. Liu, R.
Douglas, P.Hafliger, G. Jimenez-Moreno, A. C. Ballcels, T.
Serrano-Gotarredona,A. J. Acosta-Jimenez, and B. Linares-Barranco,
“CAVIAR: A 45 kneuron, 5 M synapse, 12 G connects/s AER hardware
sensory-pro-cessing-learning-actuating system for high-speed visual
objectrecognition and tracking,” IEEE Trans. Neural Netw., vol. 20,
no. 9,pp. 1417–1438, Sep. 2009.
[4] S. C. Liu and R. Douglas, “Temporal coding in a silicon
network ofintegrate-and-fire neurons,” IEEE Trans. Neural Netw.,
vol. 15, no. 5,pp. 1305–1314, Sep. 2004.
[5] R. J. Vogelstein, U. Mallik, E. Culurciello, G.
Cauwenberghs, andR. Etienne-Cummings, “A multi-chip neuromorphic
system forspike-based visual information processing,” Neural
Comput., vol. 19,no. 9, pp. 2281–2300, 2007.
[6] D. Sridharan, B. Percival, J. Arthur, and K. Boahen, “An
in-siliconeural model of dynamic routing through neuronal
coherence,” Adv.Neural Inf. Process. Syst., vol. 20, 2008.
[7] R. Sarpeshkar, Ultra Low Power
Bioelectronics—Fundamentals,Biomedical Applications, and
Bio-Inspired Systems. Cambridge,U.K.: Cambridge Univ. Press,
2010.
[8] M. C. Hsiao, C. H. Chan, V. Srinivasan, A. Ahuja, G.
Erinjippurath, T.P. Zanos, G. Gholmieh, D. Song, J. D. Wills, J.
LaCoss, S. Courellis, A.R. Tanguay, J. J. Granacki, V. Z.
Marmarelis, and T. W. Berger, “VLSIimplementation of a nonlinear
neuronal model: A ‘Neural Prosthesis’to restore hippocampal
trisynaptic dynamics,” in Proc. IEEE Eng. Med.Bio Conf., 2006, pp.
4396–4399.
[9] R. Z. Shi and T. K. Horiuchi, “A neuromorphic VLSI model of
bat in-teraural level difference processing for azimuthal
echolocation,” IEEETrans. Circuits Syst. I, vol. 54, no. 1, pp.
74–88, Jan. 2007.
[10] T. A. Hudson, J. A. Bragg, P. Hasler, and S. P. DeWeerth,
“An analogVLSI model of muscular contraction,” IEEE Trans. Circuits
Syst. II:Analog and Digital Signal Process., vol. 50, no. 7, pp.
329–342, Jul.2003.
[11] R. J. Vogelstein, F. Tenore, L. Guevremont, R.
Etienne-Cummings, andV. K. Mushahwar, “A silicon central pattern
generator controls loco-motion in vivo,” IEEE Trans. Biomed.
Circuits Syst., vol. 2, no. 3, pp.212–222, Sep. 2008.
[12] V. Aggarwal, S. Acharya, F. Tenore, H. C. Shin, R.
Etienne-Cummings,and N. V. Thakor, “Asynchronous decoding of
dexterous finger move-ments using M1 neurons,” IEEE Trans. Neural
Syst. Rehab. Eng., vol.16, no. 1, pp. 3–14, Feb. 2008.
[13] E. Kandel, J. Schwartz, and T. Jessell, Principles of
Neural Science.New York: McGraw-Hill Medical, 2000.
[14] A. L. Hogdkin and A. F. Huxley, “A quantitative description
of mem-brane current and its application to conduction and
excitation in nerve,”J. Physiol., vol. 117, pp. 500–544, 1952.
[15] B. Linares-Barranco, E. Sanchez-Sinencio, A.
Rodriguez-Vazquez, andJ. L. Huertas, “A CMOS implementation of
FitzHugh-Nagumo neuronmodel,” IEEE J. Solid-State Circuits, vol.
26, no. 7, pp. 956–965, Jul.1991.
[16] D. Dupeyron, S. Le Masson, Y. Deval, G. Le Masson, and J.
Dom,“A BiCMOS implementation of the Hodgkin--Huxley formalism,”
inProc. 5th Int. Conf. Microelectronics for Neural Networks and
FuzzySystems, 1996, pp. 311–316.
[17] J. Georgiou, E. M. Drakakis, C. Toumazou, and P. Premanoj,
“Ananalogue micropower log-domain silicon circuit for the Hodgkin
andHuxley nerve axon,” in Proc. IEEE Int. Symp. Circuits Syst.,
1999, vol.2, pp. 286–289.
[18] M. F. Simoni, G. S. Cymbalyuk, M. E. Sorensen, R. L.
Calabrese,and S. P. DeWeerth, “A multiconductance silicon neuron
with biolog-ically matched dynamics,” IEEE Trans. Biomed. Eng.,
vol. 51, no. 2,pp. 342–354, Feb. 2004.
[19] K. A. Boahen, “Point-to-point connectivity between
neuromorphicchips using address-events,” IEEE Trans. Circuits Syst.
II, vol. 47, no.5, pp. 416–434, May 2000.
[20] G. Indiveri, E. Chicca, and R. Douglas, “A VLSI array of
low-powerspiking neurons and bistable synapses with spike-timing
dependentplasticity,” IEEE Trans. Neural Netw., vol. 17, no. 1, pp.
211–221, Jan.2006.
[21] D. Badoni, M. Giulioni, V. Dante, and P. Del Giudice, “An
aVLSIrecurrent network of spiking neurons with reconfigurable and
plasticsynapses,” in Proc. IEEE ISCAS , 2006, pp. 1227–1230.
[22] R. J. Vogelstein, U. Mallik, J. T. Vogelstein, and G.
Cauwenberghs,“Dynamically reconfigurable silicon array of spiking
neurons with con-ductance-based synapses,” IEEE Trans. Neural
Netw., vol. 18, no. 1,pp. 253–265, Jan. 2007.
-
148 IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, VOL.
4, NO. 3, JUNE 2010
[23] R. Serrano-Gotarredona, T. Serrano-Gotarredona, A.
Acosta-Jimenez,C. Serrano-Gotarredona, J. A. Perez-Carrasco, B.
Linares-Barranco,A. Linares-Barranco, G. Jimenez-Moreno, and A.
Civit-Ballcels, “Onreal-time AER 2-D convolutions hardware for
neuromorphic spike-based cortical processing,” IEEE Trans. Neural
Netw., vol. 19, no. 7,pp. 1196–1219, Jul. 2008.
[24] S. Mitra, S. Fusi, and G. Indiveri, “Real-time
classification of com-plex patterns using spike-based learning in
neuromorphic VLSI,” IEEETrans. Biomed.Circuits Syst., vol. 3, no.
1, pp. 32–42, Feb. 2009.
[25] P. Livi and G. Indiveri, “A current-mode conductance-based
siliconneuron for address-event neuromorphic systems,” in Proc.
IEEEISCAS, 2009, pp. 2898–2901.
[26] V. Chan, S. C. Liu, and A. van Schaik, “AER EAR: A matched
siliconcochlea pair with address event representation interface,”
IEEE Trans.Circuits Syst. I: Reg. Papers, vol. 54, no. 1, pp.
48–59, Jan. 2007.
[27] J. Costas-Santos, T. Serrano-Gotarredona, R.
Serrano-Gotarredona,and B. Linares-Barranco, “A spatial contrast
retina with on-chipcalibration for neuromorphic spike-based AER
vision systems,” IEEETrans. Circuits Syst. I: Reg. Papers, vol. 54,
no. 7, pp. 1444–1458, Jul.2007.
[28] H. Yang and R. Sarpeshkar, “A bio-inspired
ultra-energy-efficientanalog-to-digital converter for biomedical
applications,” IEEE Trans.Circuits Syst. I, vol. 53, no. 11, pp.
2349–2356, Nov. 2006.
[29] Y. Wong, P. Xu, and P. Abshire, “Ultra-low spike rate
silicon neuron,”in Proc. IEEE Biomedical Circuits Systems Conf.,
2007, pp. 95–98.
[30] A. van Schaik, “Building blocks for electronic spiking
neural net-works,” Neural Netw., vol. 14, no. 6–7, pp. 617–628,
Jul.-Sep. 2001.
[31] E. Farquhar and P. Hasler, “A bio-physically inspired
silicon neuron,”IEEE Trans. Circuits Syst. 1: Reg. Papers, vol. 52,
no. 3, pp. 477–488,Mar. 2005.
[32] S. Saighi, Y. Bornat, J. Tomas, and S. Renaud,
“Neuromimetic ICs andsystem for parameters extraction in biological
neuron models,” in Proc.IEEE ISCAS, 2006, pp. 4207–4211.
[33] K. Hynna and K. Boahen, “Thermodynamically equivalent
siliconmodels of voltage-dependent ion channels,” Neural Comput.,
vol. 19,pp. 327–350, 2007.
[34] E. M. Izhikevich, “Simple model of spiking neurons,” IEEE
Trans.Neural Netw., vol. 14, no. 6, pp. 1569–1572, Nov. 2003.
[35] J. Wijekoon and P. Dudek, “Spiking and bursting firing
patterns of acompact VLSI cortical neuron circuit,” in Proc. Int.
Joint Conf. NeuralNetworks, 2007, pp. 1332–1337.
[36] A. Basu, C. Petre, and P. Hasler, “Bifurcations in a
silicon neuron,” inProc. IEEE ISCAS, pp. 428–431.
[37] A. Cassidy and A. G. Andreou, “Dynamical digital silicon
neurons,” inProc. IEEE BioCAS, 2008, pp. 289–292.
[38] F. Folowosele, A. Harrison, A. Cassidy, A. Andreou, R.
Etienne-Cum-mings, S. Mihalas, E. Niebur, and T. Hamilton, “A
switched capacitorimplementation of the generalized linear
integrate-and-fire neuron,” inProc. IEEE ISCAS, 2009, pp.
2149–2152.
[39] E. Chicca, G. Indiveri, and R. Douglas, “An adaptive
silicon synapse,”in Proc. IEEE ISCAS, 2003, pp. I-81–I-84.
[40] J. Schemmel, A. Grubl, K. Meier, and E. Mueller,
“Implementingsynaptic plasticity in a VLSI spiking neural network
model,” in Proc.IEEE IJCNN, 2006, pp. 1–6, IEEE Press.
[41] C. Bartolozzi and G. Indiveri, “Synaptic dynamics in analog
VLSI,”Neural Comput., no. 19, pp. 2581–2603, 2007.
[42] Z. Yang, A. Murray, and J. Huo, “Deterministic coincidence
detec-tion and adaptation via delayed inputs,” in ICANN (Part 2),
2008, pp.453–461.
[43] A. Basu, S. Ramakrishnan, C. Petre, S. Koziol, S. Brink,
and P. Hasler,“Neural dynamics in reconfigurable silicon,” IEEE
Trans. Biomed. Cir-cuits Syst., 2009, to be published.
[44] Y. Wang and S. C. Liu, “Input evoked nonlinearities in
silicon dendriticcircuits,” in Proc. IEEE ISCAS, 2009, pp.
2894–2897.
[45] K. Kotani, T. Shibata, and T. Ohmi, “Neuron-MOS
binary-logiccircuits featuring dramatic reduction in transistor
count and intercon-nections,” in Proc. Int. Electron Devices
Meeting Tech. Digest., Dec.13–16, 1992, pp. 431–434.
[46] G. Cauwenberghs, C. F. Neugebauer, and A. Yariv, “Analysis
and veri-fication of an analog VLSI incremental outer-product
learning system,”IEEE Trans. Neural Netw., vol. 3, no. 3, pp.
488–497, May 1992.
[47] P. Hafliger and C. Rasche, “Floating gate analog memory for
parameterand variable storage in a learning silicon neuron,” in
Proc. IEEE ISCAS,1999, pp. 416–419.
[48] R. R. Harrison, J. A. Bragg, P. Hasler, B. A. Minch, and S.
P. De-weerth, “A CMOS programmable analog memory-cell array
usingfloating-gate circuits,” IEEE Trans. Circuits Syst. II: Analog
DigitalSignal Process., vol. 48, no. 1, pp. 4–11, Jan. 2001.
[49] F. Tenore, R. J. Vogelstein, R. Etienne-Cummings, G.
Cauwenberghs,and P. Hasler, “A floating-gate programmable array of
silicon neuronsfor central pattern generating networks,” in Proc.
IEEE ISCAS, 2006,pp. 3157–3160.
[50] S. C. Liu and R. Mockel, “Temporally learning floating-gate
VLSIsynapses,” in Proc. IEEE ISCAS, 2008, pp. 2154–2157.
[51] M. Pankaala, M. Laiho, and P. Hasler, “Compact
floating-gate learningarray with STDP,” in Proc. Int. Joint Conf.
Neural Networks, 2009, pp.2409–2415.
[52] G. Cauwenberghs and A. Yariv, “Fault-tolerant dynamic
multi-levelstorage in analog VLSI,” IEEE Trans. Circuits Syst. II:
Analog andDigital Signal Process., vol. 41, no. 12, pp. 827–829,
Dec. 1994.
[53] T. Yu and G. Cauwenberghs, “Analog VLSI neuromorphic
networkwith programmable membrane channel kinetics,” in Proc. IEEE
Int.Symp. Circuits and Systems, 2009, pp. 349–352.
[54] T. Yu and G. Cauwenberghs, “Biophysical synaptic dynamics
in ananalog VLSI network of Hodgkin-Huxley neurons,” in Proc. IEEE
Eng.Med. Biol. Conf., 2009, vol. 1, pp. 3335–3338.
[55] A. Destexhe, Z. F. Mainen, and T. J. Sejnowski, “Synthesis
of modelsfor excitable membranes, synaptic transmission and
neuromodulationusing a common kinetic formalism,” J. Comp.
Neurosci., vol. 1, pp.195–230, 1994.
[56] A. Andreou and K. Boahen, “Translinear circuits in
subthresholdMOS,” Analog Integr. Circuits Signal Process., vol. 9,
pp. 141–166,1996.
[57] T. Serrano-Gotarredona, B. Linares-Barranco, and A. G.
Andreou,“A general translinear principle for subthreshold MOS
transistors,”IEEE Trans. Circuits Syst. I: Fundam. Theory Appl.,
vol. 46, no. 5, pp.607–616, May 1999.
[58] P. M. Furth and A. G. Andreou, “Linearised differential
transconduc-tors in subthreshold CMOS,” Electron. Lett. Online,
vol. 31, no. 7, pp.545–546, 1995.
[59] T. Edwards and G. Cauwenberghs, “Synthesis of log-domain
filtersfrom first-order building blocks,” J. Analog Integr.
Circuits SignalProcess., vol. 22, pp. 177–186, 2000.
[60] A. Manwani and C. Koch, “Detecting and estimating signals
in noisycable structures, I: Neuronal noise sources,” Neural
Comput., vol. 11,pp. 1797–1829, 1999.
Theodore Yu (S’04) received the B.S. and M.S.degrees in
electrical engineering from the CaliforniaInstitute of Technology,
Pasadena, in 2004 and 2005,respectively, and is currently pursuing
the Ph.D.degree in electrical and computer engineering at
theUniversity of California, San Diego.
His research interests include neuromorphicanalog very-large
scale integrated models of neuraland synaptic circuits and circuit
implementations oflearning algorithms.
Gert Cauwenberghs (SM’89–M’94–SM’04) re-ceived the Ph.D. degree
in electrical engineeringfrom the California Institute of
Technology,Pasadena.
Currently, he is Professor of Bioengineeringat University of
California, San Diego, where heco-directs the Institute for Neural
Computation. Pre-viously, he was Professor of Electrical and
ComputerEngineering at Johns Hopkins University, Baltimore,MD, and
Visiting Professor of Brain and CognitiveScience at the
Massachusetts Institute of Tech-
nology, Cambridge. His research interests cover neuromorphic
engineering,computational and systems neuroscience, neuron-silicon
and brain-machineinterfaces, as well as learning and intelligent
systems. He is Associate Editorfor IEEE TRANSACTIONS ON BIOMEDICAL
CIRCUITS AND SYSTEMS, and IEEETRANSACTIONS ON NEURAL SYSTEMS AND
REHABILITATION ENGINEERING. Heis a Senior Editor for the IEEE
SENSORS JOURNAL.
Dr. Cauwenberghs received the National Science Foundation Career
Awardin 1997, the ONR Young Investigator Award in 1999, and
Presidential EarlyCareer Award for Scientists and Engineers in
2000.