8/3/2019 Vlsi in Neural
1/15
Analog VLSI circuits
for spiking neural networks
Giacomo Indiveri
Neuromorphic Cognitive Systems group
Institute of Neuroinformatics
University of Zurich and ETH Zurich
September 17, 2009
G.Indiveri (NCS @ INI) ICANN09 Tutorial 1 / 78
Custom VLSI implementations of neural networks
Early attempts
The idea of making custom analog VLSI implementations of neural networks
dates back to the late 80s - early 90s:
[Holler et al. 1989, Satyanarayana et al. 1992, Hammerstrom 1993, Vittoz 1996]
General purpose computing
Full-custom analog
implementation
Neural network accelerator
PC-boards
Competing with Intel steamroller
Communication - bandwidth
limited
Difficult to program
Current research
Technological progress
Power-dissipation/computational
power
Application-specific focus
Embedded system integration
G.Indiveri (NCS @ INI) ICANN09 Tutorial 3 / 78
Silicon neuron designs
Many VLSI models of spiking neurons have been developed in the past, and
many are still being actively investigated:
Most designs can be traced back to one of two types of silicon neurons
G.Indiveri (NCS @ INI) ICANN09 Tutorial 4 / 78
Silicon neural network characteristics
Above threshold (strong inversion)
Mixed analog/digital
Rate-based
Real-time
Conductance-based
Large-scale, event-based
networks
Below threshold (weak inversion)
Fully analog
Spiking
Accelerated-time
Integrate-and-Fire
Small-scale, hard-wired
G.Indiveri (NCS @ INI) ICANN09 Tutorial 5 / 78
http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/8/3/2019 Vlsi in Neural
2/15
Most designs can be traced back to one of two types of silicon neurons
Why subthreshold neuromorphic VLSI
Exploit the physics of silicon to reproduce the
bio-physics of neural systems.
G.Indiveri (NCS @ INI) ICANN09 Tutorial 6 / 78
MOSFETs in subthreshold
Vg
Vs
Vd
Ids n-FET subthreshold transfer function
Ids = I0enVg/UT
eVs/UTeVd/UT
where
I0 denotes the nFET current-scaling parameter
n denotes the nFET subthreshold slope factor
UT the thermal voltage
Vg the gate voltage, Vs the source voltage, and Vd the drain voltage.
The current is defined to be positive if it flows from the drain to the source
G.Indiveri (NCS @ INI) ICANN09 Tutorial 9 / 78
Diffusion and saturation
If
Vg
Vg
Vs
Vs
Vd
Vd
IfIr
Ir
VE Qs Qd
Ids = I0enVg/UT
eVs/UTeVd/UT
is equivalent to:
Ids = I0e
VgUT Vs
UT I0e
VgUT
VdUT
Ids = If Ir
If Vds > 4UT the Ir term becomes negligible,and the transistor is said to operate in the
saturation regime:
Ids = I0enVg/UTVs/UT
G.Indiveri (NCS @ INI) ICANN09 Tutorial 10 / 78
Exponential voltage dependence
Subthreshold n-FET
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
10-12
10-11
10-10
10-9
10-8
10-7
10-6
10-5
10-4
10-3
10-2
Ids(A)
Vgs (V)
above threshold
subthreshold
G.Indiveri (NCS @ INI) ICANN09 Tutorial 11 / 78
http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/8/3/2019 Vlsi in Neural
3/15
n-FETs and p-FETs
In Complementary Metal-Oxide Semiconductor (CMOS) technology, there are
two types of MOSFETs: n-FETs and p-FETs
Vg
Vd
Vs
Vg
Vs
Vd
Vb
In traditional CMOS circuits, all n-FETs have the common bulk potential (Vb)
connected to Ground (Gnd), and all p-FETs have a common bulk potential
(typically) connected to the power supply rail (Vdd).
The corresponding (complementary) equation for the p-FET is
Ids = I0ep(VddVg)/UT
e(VddVs)/UTe(VddVd)/UT
G.Indiveri (NCS @ INI) ICANN09 Tutorial 12 / 78
One, two, and three transistor circuits
Ideal current source
I out
Vin
Vd
Inverting amplifier
Vin Vout
Current-mirror
M 2M1
I outI in
Differential pair
Vbn
V1 V2M3
M 1
M 2
I 1 I 2
Vs
G.Indiveri (NCS @ INI) ICANN09 Tutorial 14 / 78
The differential-pair
I1 = I0eV1Vs
UT
I2 = I0eV2Vs
UT
Ib = I1 + I2 = I0eVbUT
eVsUT = Ib
I01
eV1UT + e
V2UT
I1 = Ibe
V1UT
eV1UT + e
V2UT
I2 = Ibe
V1UT
eV1UT + e
V2UT
Vbn
V1 V2M3
M 1
M 2
I 1 I 2
Vs
0.3 0.2 0.1 0 0.1 0.2 0.30
0.2
0.4
0.6
0.8
1x 10
8
V1V
2(V)
I1,
I2(A)
I1
I2
G.Indiveri (NCS @ INI) ICANN09 Tutorial 15 / 78
The transconductance amplifier
Vb
V1 V2
Ib
I1 I2
Vs
Vdd Vdd
VoutIout
M1 M2
M3
M4 M5
+
Vb
VoutIout
V2
V1
Iout = Ibtanh
2UT(V1V2)
In the linear region (|V1V2|< 200mV):
Iout gm(V1V2)
where
gm =Ib
2UT
is a tunable conductance.
G.Indiveri (NCS @ INI) ICANN09 Tutorial 16 / 78
http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/8/3/2019 Vlsi in Neural
4/15
What is a synapse?
In 1897 Charles Sherrington
introduced the term synapse to
describe the specialized
structure at the zone of contact
between neurons as the point
in which one neuroncommunicateswith another.
2005 winner of the Science and Engineering
Visualization Challenge.
by G. Johnson, Medical Media, Boulder, CO.
G.Indiveri (NCS @ INI) ICANN09 Tutorial 19 / 78
Synapses in the nervous system
GABA
Glutammate
Eex(Na+, ...)
Einh(K+, Cl, ...)
CmemGl
Vmem
Electrical | Chemical
Excitatory | Inhibitory
Depressing | Facilitating
AMPA | NMDA
. . .
G.Indiveri (NCS @ INI) ICANN09 Tutorial 20 / 78
Synaptic transmission
In chemical synapses the presynaptic and
postsynaptic membranes are spearated by
extracellular space.
The arrival of a presynaptic action potential
triggers the release of neurotransmitter in theextracellular space.
The neurotransmitters react with the
postsynaptic receptors and depolarize the cell.
Chemical synaptic transmission is
characterized by specific temporal dynamics.
G.Indiveri (NCS @ INI) ICANN09 Tutorial 21 / 78
EPSC and EPSP
, , ,
.-
.
.
. .0 - :
.0 0 . .
-. -
-i ti t t i i -
. .i i i ti ti i it ti t i t t ti t 0
0 i t - t t - itit 0 t t t , t i t , t
t i ti t tt i 0 0 t i t / t ti
- .-
- - -0 0 . -
. .
-
Superimposed excitatory
post-synaptic currents (EPSCs)
recorded in a neuron at different
membrane potentials (from Sacchi et
al., 1998).
Excitatory post-synaptic potential
(EPSP) in response to multiple
pre-synaptic spikes (from Nicholls et al.
1992).
G.Indiveri (NCS @ INI) ICANN09 Tutorial 22 / 78
http://www.sciencemag.org/sciext/vis2005/show/slide1.dtlhttp://www.sciencemag.org/sciext/vis2005/show/slide1.dtlhttp://ncs.ethz.ch/http://www.sciencemag.org/sciext/vis2005/show/slide1.dtlhttp://www.sciencemag.org/sciext/vis2005/show/slide1.dtlhttp://ncs.ethz.ch/http://ncs.ethz.ch/http://www.sciencemag.org/sciext/vis2005/show/slide1.dtlhttp://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://www.sciencemag.org/sciext/vis2005/show/slide1.dtlhttp://www.sciencemag.org/sciext/vis2005/show/slide1.dtl8/3/2019 Vlsi in Neural
5/15
Neural network models
In classical neural network theory
signals are (tipically) continuous values that represent the neurons mean
firing rate,
neurons implement a saturating non-linearity transfer function (S) on the
inputs weightedsum,
the synapse implements a multiplication between the neurons input
signal (Xi) and its corresponding synaptic weight (wi).
G.Indiveri (NCS @ INI) ICANN09 Tutorial 24 / 78
VLSI synapses in classical neural networks
The role of the VLSI synapse in implementations of classical neural network
models is that of a multiplier.
Multiplying synaptic circuits have been implemented using a wide range of
analog circuits, ranging from the single MOS-FETs to the Gilbert multiplier.
Iin1 Iin2
Ib
I1 I2
Figure: Schematic of half of a Gilbert multiplier. This circuit multiplies Iin1 and Iin2 by Ibif Iin1 + Iin2 = Ib.
G.Indiveri (NCS @ INI) ICANN09 Tutorial 26 / 78
VLSI synapses in pulse-based neural networks
Vdd
Vmem
Cmem
WiIWi Vi =
IWi
Cmemt
In pulse-based neural networks the weightedcontribution of a synapse can be
implemented using a single transistor.
In this case p-FETs implement excitatory synapse, and n-FETs implement
inhibitory synapses.
The synaptic weight can be set by changing the Wi bias voltage or the tduration.
G.Indiveri (NCS @ INI) ICANN09 Tutorial 27 / 78
Linear pulse integrators
A linearintegrator is a linear low-pass filter. Its impulse response should be a
decaying exponential.
With VLSI and subthreshold MOSFETS its fairly easy to implement
exponential voltage to current conversion, and linear voltage increase or
decrease over time.
Vdd
Vg
C
Vc
Id
Id = Cddt
Vc
Vg(t)
Id(t)
Vdd
Id(t) = I0e
UT(VddVg(t))
G.Indiveri (NCS @ INI) ICANN09 Tutorial 28 / 78
http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/8/3/2019 Vlsi in Neural
6/15
Linear charge-and-discharge integrator
Vw
M
Mw
V
Msyn
Isyn
Vsyn
Csyn
Mpre
Iw
I
Iw = I0eVwUT , c
CsynUT
I
I = I0e(VddV)
UT , dCsynUT
I
Ic = Cd
dt(VddVsyn)
Isyn = I0e(V
ddV
syn)
UT
Isyn(t) =
Isyne+(tti )
c (charge phase)
I+syne(tt+i )d (discharge phase)
Isyn(t) = I0e cft(c+d)
cdt, with f =
nt
, f=
IgIin
I
t, =
1
t+ ISI
G.Indiveri (NCS @ INI) ICANN09 Tutorial 32 / 78
http://ncs.ethz.ch/http://ncs.ethz.ch/http://www.ini.unizh.ch/~giacomo/papers/pdf/Bartolozzi_Indiveri07.pdfhttp://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://www.ini.unizh.ch/~giacomo/papers/pdf/Bartolozzi_Indiveri07.pdfhttp://ncs.ethz.ch/http://ncs.ethz.ch/8/3/2019 Vlsi in Neural
7/15
DPI measured response
0 0.05 0.1 0.150
50
100
150
200
250
300
Time (s)
EPS
C(
nA)
Vw
=420mV
Vw
=440mV
Vw
=460mV
0 0.5 1 1.5 2 2.5 3 3.50
50
100
150
200
250
300
350
400
450
Time (s)
EPS
C(
nA)
Vw
=300mV
Vw
=320mV
Vw
=340mV
G.Indiveri (NCS @ INI) ICANN09 Tutorial 33 / 78
DPI response to spike-trains
50 100 150 2000
20
40
60
80
100
120
Input Frequency (Hz)
OutputFrequency(Hz)
Vw
=1.30V
Vw
=1.33V
Vw
=1.35V
G.Indiveri (NCS @ INI) ICANN09 Tutorial 34 / 78
Short-term depression
Vw
Cd
Va
Vd Ir
(C. Rasche, R. Hahnloser, 2001)
G.Indiveri (NCS @ INI) ICANN09 Tutorial 35 / 78
Short-term depression
Vd
Vx
Vpre
C
Va
IrIsyn
C2
IdM1
M3
M2M4
M5
M6 M7
Vgain
Vpre
0.04 0.06 0.08 0.1 0.12 0.14 0.16
0.1
0.15
0.2
0.25
0.3
0.35
Time (s)
Vx
(V) Update
Slow recovery
Fast recoveryV
d=0.26 V
Vd=0.28 V
Vd=0.3 V
de
f
$g i
dp
S
@
Sr S t
f
$g i
St
(M. Boegerhausen, P. Suter, and S.-C. Liu 2003)
G.Indiveri (NCS @ INI) ICANN09 Tutorial 36 / 78
https://www.ini.uzh.ch/node/11958https://www.ini.uzh.ch/node/11958http://ncs.ethz.ch/http://ncs.ethz.ch/https://www.ini.uzh.ch/node/11958http://ncs.ethz.ch/https://www.ini.uzh.ch/node/11918http://ncs.ethz.ch/http://ncs.ethz.ch/https://www.ini.uzh.ch/node/11918http://ncs.ethz.ch/https://www.ini.uzh.ch/node/11958http://ncs.ethz.ch/http://ncs.ethz.ch/8/3/2019 Vlsi in Neural
8/15
STDP and beyond
Alternative spike-driven learning algorithm
Spike-driven weight change depends on the value of
the post-synaptic neurons membrane potential, and on
its recent spiking activity.
Fusi et al. 2000; Brader et al. 2007
Recipe for efficient VLSI implementation
1 bistability: use two synaptic states;
2 redundancy: implement many synapses that see
the same pre- and post-synaptic activity
3 stochasticity & inhomogeneity: induce LTP/LTD
only in a subset of stimulated synapses.
- Slow learning: only a fraction of the synapses memorize the pattern.
+ The theory is matched to the technology: use binary states, exploit
mismatch and introduce fault tolerance by design.
G.Indiveri (NCS @ INI) ICANN09 Tutorial 37 / 78
Spike-driven learning in VLSI I
J. Arthur and K. Boahen.
Learning in silicon: Timing is everything.
In Y. Weiss, B. Schlkopf, and J. Platt, editors, Advances in Neural Information Processing Systems 18. MIT Press, Cambridge, MA,
2006.
A. Bofill-i Petit and A. F. Murray.
Synchrony detection and amplification by silicon neurons with STDP synapses.
IEEE Transactions on Neural Networks, 15(5):12961304, September 2004.
E. Chicca, D. Badoni, V. Dante, M. DAndreagiovanni, G. Salina, S. Fusi, and P. Del Giudice.
A VLSI recurrent network of integrateandfire neurons connected by plastic synapses with long term memory.
IEEE Transactions on Neural Networks, 14(5):12971307, September 2003.
P. Hfliger, M. Mahowald, and L. Watts.
A spike based learning neuron in analog VLSI.
In M. C. Mozer, M. I. Jordan, and T. Petsche, editors, Advances in neuralinformation processing systems, volume 9, pages 692698.
MIT Press, 1997.
G. Indiveri, E. Chicca, and R. Douglas.
A VLSI array of low-power spiking neurons and bistable synapses with spiketiming dependent plasticity.
IEEE Transactions on Neural Networks, 17(1):211221, Jan 2006.
S. Mitra, G. Indiveri, and S. Fusi.
Learning to classify complex patterns using a VLSI network of spiking neurons.
In J.C. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20, pages 10091016,
Cambridge (MA), 2008. MIT Press.
G.Indiveri (NCS @ INI) ICANN09 Tutorial 38 / 78
Neurons . . . in a nutshellA quick tutorial
Complexity
Real Neurons
Conductance based models
Integrate and fire models
Rate based models
Sigmoidal units Linear threshold units
G.Indiveri (NCS @ INI) ICANN09 Tutorial 40 / 78
Neurons of the world
(adapted from B. Mel, 1994)
G.Indiveri (NCS @ INI) ICANN09 Tutorial 42 / 78
http://www.mitpressjournals.org/doi/abs/10.1162/neco.1994.6.6.1031http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://www.mitpressjournals.org/doi/abs/10.1162/neco.1994.6.6.1031http://ncs.ethz.ch/http://ncs.ethz.ch/http://www.mitpressjournals.org/doi/abs/10.1162/neco.1994.6.6.1031http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/8/3/2019 Vlsi in Neural
9/15
Equivalent Circuit
GABA
Glutammate
Eex(Na+, ...)
Einh(K+, Cl, ...)
CmemGl
Vmem
If excitatory input currents are relatively small, the neuron behaves exactly like
a first order low-pass filter.
G.Indiveri (NCS @ INI) ICANN09 Tutorial 43 / 78
Spike generating mechanism
ENa
CmemGl
Vmem
gNa
EK
gK
If the membrane voltage increases above a certain threshold, a
spike-generating mechanism is activated and an action potential is initiated.
G.Indiveri (NCS @ INI) ICANN09 Tutorial 44 / 78
Spike properties
Refractory Period
Pulse Width
Iin=I1
Iin=I2 > I1
G.Indiveri (NCS @ INI) ICANN09 Tutorial 45 / 78
The F-I curve
Input Current (I)
Spike
Freq
uency(F) Refractory
Period
G.Indiveri (NCS @ INI) ICANN09 Tutorial 46 / 78
http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/8/3/2019 Vlsi in Neural
10/15
Hardware implementations of spiking neurons
The first artificial neuron model was proposed in the 1943 by McCulloch and
Pitts. Hardware implementations of this model date almost back to the same
period.
Hardware implementations of spikingneurons are relatively new.
One of the most influential circuits that implements an
integrate and fire(I&F) model of a neuron was the
Axon-Hillock Circuit, proposed by Carver Mead in the late
1980s.
G.Indiveri (NCS @ INI) ICANN09 Tutorial 47 / 78
Conductance-based models of spiking neurons
In 1991 Misha Mahowald and Rodney Douglas proposed a
conductance-based silicon neuron and showed that it had properties
remarkably similar to those of real cortical neurons.
G.Indiveri (NCS @ INI) ICANN09 Tutorial 49 / 78
Conductance based Si-Neurons
-
+
-
+
-
+
-
+
-
+
Vdd Vdd
-
+
Vdd Vdd
-
+
-
+
-
+
Vdd Vdd
-
+
-
+
Vdd Vdd
-
+
Sodium
Potassium
Passive
Vmem
Vmem
Gleak
Eleak
Cmem
EK
EK
VNa
VK
Vthr
Vthr
Vthr
ENa
INa
INaoff
GNaon
GK
IK I
K
GNaoff
INaon
Passive Leak
Sodium Current
Potassium Current
G.Indiveri (NCS @ INI) ICANN09 Tutorial 50 / 78
Conductance based Si-NeuronsSilicon neurons measurements
1V
Vm
[Ca]
Vm
[Ca]
Vm
[Ca]
50 ms
I
G.Indiveri (NCS @ INI) ICANN09 Tutorial 51 / 78
http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/8/3/2019 Vlsi in Neural
11/15
The Axon-Hillock Circuit
A
Vpw
Vmem Vout
Input current
Membrane voltage
Output voltage
Positive Feedback
Reset
G.Indiveri (NCS @ INI) ICANN09 Tutorial 53 / 78
The Axon-Hillock Circuit
A
Vpw
Vmem Vout
Vout
Vmem
time
voltage
Vmem
Vout
Slope = A
G.Indiveri (NCS @ INI) ICANN09 Tutorial 54 / 78
Capacitive Divider
Given the change V2, what is V1?
Q= C1V1 + C2(V1V2) = constant
C1V1 + C2(V1V2) = 0
V1 =C2
C1 + C2V2
A V2V1 C1
C2
Q
G.Indiveri (NCS @ INI) ICANN09 Tutorial 55 / 78
Positive Feedback
A
Vpw
Vmem Vout
Vout
Vmem
time
voltage
Positive Feedback
Vmem =
Cm
Cfb
CfbCm + Cfb
Vdd
G.Indiveri (NCS @ INI) ICANN09 Tutorial 56 / 78
http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/8/3/2019 Vlsi in Neural
12/15
Axon-Hillock Circuit Dynamics
A
Vpw
Vmem Vout
Vout
Vmem
time
voltage
Cm
Cfb
Iin
tH tL
Ir
tL =Cfb+ Cm
IinVmem =
Cfb
IinVdd
Frequency Iin
tH =Cfb+ Cm
Ir IinVmem =
Cfb
Ir IinVdd
Pulse width 1/Ir for Ir Iin
G.Indiveri (NCS @ INI) ICANN09 Tutorial 57 / 78
Gain
How to make voltage gain
A
Whats bad about this?
G.Indiveri (NCS @ INI) ICANN09 Tutorial 58 / 78
Power Dissipation
The Axon-Hillock circuit is very compact and allows for
implementations of dense arrays of silicon neurons
BUTit has a major drawback: power consumption
During the time when an inverter switches, a large amount
of current flows from Vdd to Gnd.
G.Indiveri (NCS @ INI) ICANN09 Tutorial 59 / 78
Conductance-based modelsIntegrate and Fire vs Hodgkin-Huxley
Traditionally there have been two main classes of neuron models:
Integrate and fi re (I -C) Conduc tanc e-bas ed ( R-C)
G.Indiveri (NCS @ INI) ICANN09 Tutorial 61 / 78
http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/8/3/2019 Vlsi in Neural
13/15
Conductance-based modelsIntegrate and Fire vs Hodgkin-Huxley
But recently proposed models bridge the gap between the two:
Generalized Integrate and Fire models can account for a very large set of
behaviors captured by far more complicated Hodgkin-Huxley models.
d
dtumem =
iin
Cmem+ F(umem)
where F(umem) is a non-linear function of umem(t).
G.Indiveri (NCS @ INI) ICANN09 Tutorial 62 / 78
An ultra low-power generalized I&F circuit
Cahp
Cmem
Vtau_ahp
Vahp
Vspk
Vtau
Vmem
Vrf
Positive Feedback
Refractory Period
Iin
Leak
Adaptation
Vthr
M1
M3M2
VrestVthr_ahp
M8M7
M4
M5
M9
M6
M10
M11
M12
M13
M14
M15
M17
M16
M19
M21
M20
M18
M22
Imem
DPI
DPI
Iahp
Ifb
(G. Indiveri, P. Livi, ISCAS 2009)
G.Indiveri (NCS @ INI) ICANN09 Tutorial 63 / 78
DPI neuron sub-threshold equations
Cahp
Cmem
Vtau_ahp
Vahp
Vspk
Vtau
Vmem
Vrf
Positive Feedback
Refractory Period
Iin
Leak
Adaptation
Vthr
M1
M3M2
VrestVthr_ahp
M8M7
M4
M5
M9
M6
M10
M11
M12
M13
M14
M15
M17
M16
M19
M21
M20
M18
M22
Imem
DPI
DPI
Iahp
Ifb
d
dtImem+ Imem
IgIin
I+(Imem)
1
II
1+1
0 , 2+ 1
+ 1
G.Indiveri (NCS @ INI) ICANN09 Tutorial 64 / 78
SPICE simulations
2 4 6 8 105
0
5
10
15
20
Time (ms)
Imem(
A)
Vthr=0.325 V
Vthr=0.350 V
Vthr=0.375 V
0 20 40 60 80 100
2
0
2
4
6
8
10
12
14
Time (ms)
Ime
m(
A)
G.Indiveri (NCS @ INI) ICANN09 Tutorial 66 / 78
http://ncs.ethz.ch/http://www.ini.uzh.ch/~giacomo/papers/pdf/iscas09.pdfhttp://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://www.ini.uzh.ch/~giacomo/papers/pdf/iscas09.pdfhttp://ncs.ethz.ch/8/3/2019 Vlsi in Neural
14/15
Experimental resultsSingle spike
0.01 0.02 0.03 0.04 0.051.9
2
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
Voltage
(V)
Time (s)
Vthr
= 0.0V
Vthr
= 0.3V
Vthr
= 0.6V
Vthr
= 0.9V
G.Indiveri (NCS @ INI) ICANN09 Tutorial 67 / 78
Experimental resultsPopulation activity
0 0.5 1 1.5 20
5
10
15
20
25
30
Neurons
0 0.5 1 1.5 20
5
10
15
20
25
30
0 0.5 1 1.5 20
5
10
15
20
25
30
Neurons
Time (s)0 0.5 1 1.5 2
0
5
10
15
20
25
30
Time (s)
G.Indiveri (NCS @ INI) ICANN09 Tutorial 70 / 78
Spiking multi-neuron architectures
Networks of I&F neurons with adaptation,
refractory period, etc.
Synpases with realistic temporal dynamics
Winner-Take-All architectures
Spike-based plasticity mechanisms
G.Indiveri (NCS @ INI) ICANN09 Tutorial 72 / 78
Spikes and Address-Event Systems
1 2 3 2 3 12
Inputs
Encode Decode
Address Event Bus
SourceChip
Outputs
DestinationChip
Action Potential
Address-Eventrepresentation ofaction potential
2
1
32
1
3
0 0.05 0.1 0.15 0.20
0.2
0.4
0.6
0.8
1
Time (s)
Vmem(
V)
G.Indiveri (NCS @ INI) ICANN09 Tutorial 73 / 78
, ,
http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/8/3/2019 Vlsi in Neural
15/15
A spike-based learning chip
A minimum-sizechip implementing a
reconfigurable AER neural network.
Neurons and synapses have realistic
temporal dynamics. Local circuits at
each synapse implement the bi-stable
spike-based plasticity mechanism.
Indiveri, Fusi, 2007
Technology: AMS 0.35mSize: 3.9mm2.5mmNeurons: 128
AER plastic synapses: 28128AER non-plastic synapses 4128Dendritic tree multiplexer: 32128 | . .. | 1 4096
G.Indiveri (NCS @ INI) ICANN09 Tutorial 74 / 78
Distributed multi-layer networksAnalog processing, asynchronous digital communication
, ,- ,
, -
,x x x
, - x
xx, -
x x, -x
x ,,
2
,
x x
x, , xx
,,
x, x,
x
xx 1 ,
1 ,1 ,2 2 2
x , l1 -
'x,
-- ,
2
, ,x ,
x, -
2 , 2 , 2 x
- -
x -
i i i r r l c r x. k r i r c i
r - i i l rcli r l l i i ic r . r
i l i l r r c r c ri i c i ri i
r ik r r r i l r r i ri r i r r l
c r x r i - r ci ic rcli . i c rr i
r i c r k i c i irr r i .
c i i c i - r Fi . c r i i -i ri , c
ci ic i i ll r c i r c r r . r
L i .1
.
ik
i i
l i l l r i . l ri i i i r
r l i l r c i i i r l r
i c r x. c rr c l r i l i l r i
r i li ic i . ircl r r r rr r r c c i
; i i r r . r
i l l- i i l ck rr i , r i x l -
l r i r li i , r i rl i r r x -lik
r i r rr , ic r i i ri c i ic l . x ri l x l i l r c ll i
l r l ll i r r l i l i i Fi . . l cc
ll r i i r i i - i r r l c r x c ll 1
. l , r l c ic i r r x r i . r i i
i - lik i r i r li i r - i i l r i
r i - x i ri c c l i i . I ,
i r r l c r x, I , ri r I ; I , ri r I ; F , r r l c r x.
r . i r, r l c ic i .
V1
V4
V1
PFC
AIT
IT
Categ. Ident.
V4/PIT
| | | . r . r
AERINPUTY
AER
INPUTX
AEROUTPUT
G.Indiveri (NCS @ INI) ICANN09 Tutorial 75 / 78
Summary
If
Vg
Vg
Vs
Vs
Vd
Vd
IfIr
Ir
VE Qs Qd
Cahp
Cmem
Vtau_ahp
Vahp
Vspk
Vtau
Vmem
Vrf
Positive Feedback
Refractory Period
Iin
Leak
Adaptation
Vthr
M1
M3M2
VrestVthr_ahp
M8M7
M4
M5
M9
M6
M10
M11
M12
M13
M14
M15
M17
M16
M19
M21
M20
M18
M22
Imem
DPI
DPI
Iahp
Ifb
G.Indiveri (NCS @ INI) ICANN09 Tutorial 77 / 78
Thank you for your attentionAdditional information available at:
http://ncs.ethz.ch/
G.Indiveri (NCS @ INI) ICANN09 Tutorial 78 / 78
http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/http://ncs.ethz.ch/