Page 1
Perspec'vesonApplica'onsofaStochas'cSpikingNeuronModelto
NeuralNetworkModelingAntonioC.Roque
USP,RibeirãoPreto,SP,BrazilJointworkwithLudmilaBrochini1,AriadneCosta3,
ViníciusCordeiro2,RenanShimoura2,MiguelAbadi1,OsameKinouchi2andJorgeStolfi3
1USP,SãoPaulo;2USP,RibeirãoPreto;3Unicamp,Campinas
PNLD2016,Berlin
Page 2
Whystochas'cneuronmodels?
• Invivoandinvitrorecordingsofsingleneuronspiketrainsarecharacterizedbyahighdegreeofvariability
• ThefollowingexamplesaretakenfromthebookbyGerstner,Kistler,NaudandPaninski,NeuronalDynamics,CUP,2014
Page 3
Awakemouse,cortex,freelywhisking
Crochetetal.,2011
Spontaneousac'vityinvivo
Page 4
Trialtotrialvariabilityinvivo15repe''onsofthesamerandomdotmo'onpa\ern
AdaptedfromBairandKoch,1996;DatafromNewsome,1989
Page 5
Trialtotrialvariabilityinvitro
4repe''onsofthesame'me-dependents'mulus
ModifiedfromNaudandGerstner,2012
Page 6
Sourcesofnoise:extrinsicandintrinsictoneurons
Lindner,2016
Page 7
Twotypesofnoisemodelforaneuron
• Spikegenera'onisdirectlymodeledasastochas'cprocess
• Spikegenera'onismodeleddeterminis'callyandnoiseentersthedynamicsviaaddi'onalstochas'cterms
Page 8
Stochas'cmodelforsystemsofinterac'ngneurons
Page 9
Thestochas'cmodel
• Vi(t): time dependent membrane potential of neuron i at time t for i = 1, …, N; • t: discrete time given by integer multiples of constant step Δ small enough to
exclude possibility of a neuron firing more than once during each step; • Xi(t): number of times neuron i fired between t and t+1, namely 0 or 1; • If neuron fires between t and t+1, its potential drops to VR by time t+1; • wij: weight of synapse from neuron j to neuron i; • µ: decay factor (in the interval [0, 1]) due to leakage during time step Δ; • Xi(t) = 1 with probability Φ(Vi(t)); • Φ(V) is assumed to be monotonically increasing and saturating at some
saturation potential VS.
Page 10
Comment• IfΦ(V) = Θ(V−Vth),i.e.0 for V<Vth and1for
V>Vth,themodelbecomesthedeterminis'cdiscrete-'meleakyintegrate-and-firemodel(LIF).
• AnyotherchoiceofΦ(V) givesastochas'cneuron
Vs
Page 11
Inthefollowing,Iwillshowsomeanaly'calandnumericalresultsof
networkmodelsusingthisstochas'cneuronmodel
Page 12
Networkwithall-to-allcouplingMeanfieldanalysis
Analy'calandnumericalresults
Page 13
Macroscopicquan''es• Poten'aldistribu'on: frac'onofneuronswithpoten'alin therange(V,V+dV)at'met
• Networkac'vity: frac'onofneuronsthatfiredbetweentand t+1
Page 14
Shapeofthepoten'aldistribu'onP(V,t)hasacomponentthatisaDiracpulseatV=VRwithamplitude,accoun'ngfortheneuronsthatfiredbetweentandt+1
Page 15
Mean-fieldanalysis
• Fullyconnectednetwork:eachneuronreceivesinputsfromallotherN−1neurons;
• VR=0;• Uniformconstantexternalinput:Ii(t)=I;• Allweightsareequal:
Page 16
Themean-fieldpoten'aldistribu'on• Onceallneuronshavefiredatleastonce,thedensityP(V,t)becomesacombina'onofdiscreteimpulseswithamplitudesη0(t),η1(t),η2(t),…,atpoten'alsU0(t),U1(t),U2(t),...,suchthat.
• Thevaluesofηk(t)andUk(t)evolvebytheequa'ons:
Page 17
• Theamplitudeisthefrac'onofneuronswith“age”k:neuronsthatfiredbetweent – k – 1andt – k anddidnotfirebetweent – k andt
• Forthistypeofdistribu'ondenetworkac'vityρ(t)is:
Page 18
• Givenvaluesforµ,Wandthefunc'onΦ(V):– Therecurrenceequa'onscanbesolvednumerically
– Insomecasestheycanbesolvedanaly'cally
Page 19
ExamplesofΦ(V)Satura'ngmonomialfunc'onofdegreer
Ra'onalfunc'on
Γ = 1; VT = 0 NB.:Thedeterminis'cLIFmodelcorrespondstothemonomialfunc'onwith
S
Page 20
Resultsforthemonomialsatura'ngfunc'onwithμ=0
Page 21
• Inthecasewithµ = 0,neurons“forget”allpreviousinputsignals,exceptthosereceivedat t – 1.
• P(V,t)containsonlytwopeaksatpoten'als:V0(t)=0andV1(t)=I+Wρ(t−1)
• Takingintoaccountthenormaliza'oncondi'on,thefrac'onsη0(t)andη1(t)evolveas:
Page 22
• Assumingthatneuronscannotfireatrest,Φ(0) = 0:
• In a stationary regime, the recurrence equations
reduce to:
• Since Φ(V) ≤ 1, any stationary regime must have ρ ≤
1/2
Page 23
Φ(V) = (ΓV)r, I = 0; r = 1Con'nuousphasetransi'ons
Absorbing State ρ = 0
Fixed point ρ > 0
2-cycle ρ1 = ½ − a ρ2 = ½ + a
a ≤ ½ − Vs/W
WC=1/Γ
Γ=1
WB=2/Γ
Brochinietal.,2016
Page 24
Φ(V) = (ΓV)r, I = 0; r > 1Discon'nuousphasetransi'ons
r=1.2 r=2
ρ+
ρ−
Nontrivialsolu'onρ+onlyfor1≤r≤2Forr=2thissolu'onisapointatWC=2/ΓThediscon'nuitygoestozeroforr=1
W=WC(r)
Γ = 1 Γ = 1
Brochinietal.,2016
Page 25
Φ(V) = (ΓV)r, I = 0; r < 1Ceaselessac'vity
Noabsorbingρ=0solu'on
Brochinietal.,2016
Γ = 1
ρ>0foranyW>0
Page 26
Numerical solutions for µ > 0 Φ(V) = (ΓV)r, I = 0; r = 1
Brochinietal.,2016
Page 27
Discon'nuousphasetransi'onsforVT > 0
Φ(V) = (Γ(V-VT))r, I = 0, µ = 0 ; r = 1, Γ = 1
VT=0 VT=0.05 VT=0.1
Thediscon'nuityρCgoestozeroforVTà0
Brochinietal.,2016
Page 28
Neuronalavalanches(simula'onstudiesatthecri'calpointof
thecon'nuousphasetransi'on)
• Anavalanchethatstartsat'met=aandendsat'met=bhas:– Dura'ond=b−a;– Size
Page 29
Neuronalavalanchesatthecri'calpointΦ(V) = (ΓV)r, I = 0, µ = 0; r = 1, ΓC = WC = 1
Brochinietal.,2016
Avalanchesizesta's'cs
Page 30
Avalanchedura'onsta's'cs
Brochinietal.,2016
Page 31
Self-organiza'onwithdynamicneuronalgainsIdea:fixtheweightsatW=1andallowthegainsΓtovary
u=1,A=1.1,τ=1000ms Brochinietal.,2016
Page 32
Networkwithrealis'cconnec'vityExcitatoryandinhibitoryneurons
Simula'onresults
Page 33
ThePotjans-DiesmannModel
105neurons(80%excit.20%inhibit.)109synapses
Availableatwww.opensourcebrain.org
• Modelforlocalcor'calmicrocircuit
• Integratesexperimentaldataofmorethan50experimentalpapers
• Excitatoryandinhibitoryneuronsmodeledbythesamedeterminis'cLIFmodel
• Asynchronous-irregularspikingofneurons
• Higherspikerateofinhibitoryneurons
• Replicateswellthedistribu'onofspikeratesacrosslayers
Potjans&Diesmann,2014
Page 34
Fitofaveragebehaviorofstochas'cmodel(monomialΦ(V))toIzhikevichmodelneurons
Regularspikingneuron(excitatory) Fastspiking
neuron(inhibitory)
Page 35
−40 −35 −30 −25
μ=0.9ΓΓ
Page 36
win<<wex win<wex
win>wex win>>wex
Page 37
Computa'onalcost
TIzhikevich
Tstochas'c_______
No.ofsynapses
----------------------------------------------------
Timetosimulate5secofnetworkac'vity(reducednetworkwith4000neurons)
Whichmodeltouseforcor'calspikingneurons?Izhikevich,2004
Page 38
Conclusions
• Thestochas'cneuronmodelintroducedbyGalvesandLöcherbachisaninteres'ngelementforstudiesofnetworksofspikingneurons
• Enablesexactanaly'cresults:– Phasetransi'ons– Avalanches,SOC
• Simpleandefficientnumericalimplementa'on
Page 39
ResearchTeam
USP,SaoPaulo USP,RibeirãoPreto
Thanks!
Unicamp,Campinas
NeuroMat
L.Brochini M.Abadi
A.Galves
A.CostaO.Kinouchi J.StolfiR.Shimoura V.Cordeiro
E.LöcherbachUniv.Cergy-PontoiseUSP,SaoPaulo