Lecture 13: Associative Memory References: D Amit, N Brunel, Cerebral Cortex 7, 237- 252 (1997) N Brunel, Network 11, 261-280 (2000) N Brunel, Cerebral Cortex 13, 1151-1161 (2003) J Hertz, in Models of Neural Networks IV (L van Hemmen, J Cowan and E Domany, eds) Springer Verlag, 2002; sect 1.4
Lecture 13: Associative Memory. References: D Amit, N Brunel, Cerebral Cortex 7 , 237-252 (1997) N Brunel, Network 11 , 261-280 (2000) N Brunel, Cerebral Cortex 13 , 1151-1161 (2003) - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Lecture 13: Associative Memory
References:
D Amit, N Brunel, Cerebral Cortex 7, 237-252 (1997)
N Brunel, Network 11, 261-280 (2000)
N Brunel, Cerebral Cortex 13, 1151-1161 (2003)
J Hertz, in Models of Neural Networks IV (L van Hemmen, J Cowan and E Domany, eds) Springer Verlag, 2002; sect 1.4
What is associative memory?
What is associative memory?
• “Patterns”: firing activity of specific sets of neurons (Hebb: “assemblies”)
What is associative memory?
• “Patterns”: firing activity of specific sets of neurons (Hebb: “assemblies”)
• “Store” patterns in synaptic strengths
What is associative memory?
• “Patterns”: firing activity of specific sets of neurons (Hebb: “assemblies”)
• “Store” patterns in synaptic strengths
• Recall: Given input (initial activity pattern) not equal to any stored pattern, network dynamics should take it to “nearest” (most similar) stored pattern
What is associative memory?
• “Patterns”: firing activity of specific sets of neurons (Hebb: “assemblies”)
• “Store” patterns in synaptic strengths
• Recall: Given input (initial activity pattern) not equal to any stored pattern, network dynamics should take it to “nearest” (most similar) stored pattern
(categorization, error correction, …)
Implementation in balanced excitatory-inhibitory network
Model (Amit & Brunel): p non-overlapping excitatory subpopulations
Implementation in balanced excitatory-inhibitory network
Model (Amit & Brunel): p non-overlapping excitatory subpopulationseach of size n = fN (fp < 1)
Implementation in balanced excitatory-inhibitory network
Model (Amit & Brunel): p non-overlapping excitatory subpopulationseach of size n = fN (fp < 1)
stronger connections within subpopulations (“assemblies”)
Implementation in balanced excitatory-inhibitory network
Model (Amit & Brunel): p non-overlapping excitatory subpopulationseach of size n = fN (fp < 1)
stronger connections within subpopulations (“assemblies”)weakened connections between subpopulations
Implementation in balanced excitatory-inhibitory network
Model (Amit & Brunel): p non-overlapping excitatory subpopulationseach of size n = fN (fp < 1)
stronger connections within subpopulations (“assemblies”)weakened connections between subpopulations
Looking for selective states: higher rates in a single assembly
Model
Like Amit-Brunel model (Lecture 9) except for exc-exc synapses:
Model
Like Amit-Brunel model (Lecture 9) except for exc-exc synapses:
From within the same assembly:
Model
Like Amit-Brunel model (Lecture 9) except for exc-exc synapses:
From within the same assembly: 1111 JgJ
Model
Like Amit-Brunel model (Lecture 9) except for exc-exc synapses:
From within the same assembly: 1111 JgJ (strengthened, “Hebb” rule)
Model
Like Amit-Brunel model (Lecture 9) except for exc-exc synapses:
From within the same assembly:
From outside the assembly:
1111 JgJ
1111 JgJ
(strengthened, “Hebb” rule)
(weakened, “anti-Hebb”)
Model
Like Amit-Brunel model (Lecture 9) except for exc-exc synapses:
From within the same assembly:
From outside the assembly:
Otherwise: no change
1111 JgJ
1111 JgJ
(strengthened, “Hebb” rule)
(weakened, “anti-Hebb”)
Model
Like Amit-Brunel model (Lecture 9) except for exc-exc synapses:
From within the same assembly:
From outside the assembly:
Otherwise: no change
1111 JgJ
1111 JgJ
(strengthened, “Hebb” rule)
(weakened, “anti-Hebb”)
To conserve average strength: 1)1( gffg
Model
Like Amit-Brunel model (Lecture 9) except for exc-exc synapses:
From within the same assembly:
From outside the assembly:
Otherwise: no change
1111 JgJ
1111 JgJ
(strengthened, “Hebb” rule)
(weakened, “anti-Hebb”)
To conserve average strength:
=>
1)1( gffg
ffg
g
11
Mean field theoryRates: active assembly
inactive assembliesrest of excitatory neuronsinhibitory neuronsext input neurons
actr
r1r
2r0r
Mean field theory
Input current to neurons in the active assembly:
212111010 ])1()1([ rJrgpfrgpfrfgJrJI actact
Rates: active assemblyinactive assembliesrest of excitatory neuronsinhibitory neuronsext input neurons
actr
r1r
2r0r
Mean field theory
Input current to neurons in the active assembly:
212111010 ])1()1([ rJrgpfrgpfrfgJrJI actact
Rates: active assemblyinactive assembliesrest of excitatory neuronsinhibitory neuronsext input neurons
actr
r1r
2r0r
to rest of assemblies:
212111010 ])1(])2([[ rJrgpfrgpgfrfgJrJI act
Mean field theory
Input current to neurons in the active assembly:
212111010 ])1()1([ rJrgpfrgpfrfgJrJI actact
Rates: active assemblyinactive assembliesrest of excitatory neuronsinhibitory neuronsext input neurons
actr
r1r
2r0r
to rest of assemblies:
212111010 ])1(])2([[ rJrgpfrgpgfrfgJrJI act
to other excitatory neurons:
2121110101 ])1()1([ rJrpfrpffrJrJI act
Mean field theory
Input current to neurons in the active assembly:
212111010 ])1()1([ rJrgpfrgpfrfgJrJI actact
Rates: active assemblyinactive assembliesrest of excitatory neuronsinhibitory neuronsext input neurons
actr
r1r
2r0r
to rest of assemblies:
212111010 ])1(])2([[ rJrgpfrgpgfrfgJrJI act
to other excitatory neurons:
2121110101 ])1()1([ rJrpfrpffrJrJI act
to inhibitory neurons:
2221210202 ])1()1([ rJrpfrpffrJrJI act
Mean field theory (2)Noise variances (white noise approximation):
Mean field theory (2)Noise variances (white noise approximation):
2
22
121
222
1
211
0
02
102 ])1()1([K
rJrgpfrgpfrfg
KJ
KrJ
actact
Mean field theory (2)Noise variances (white noise approximation):
2
22
121
222
1
211
0
02
102 ])1()1([K
rJrgpfrgpfrfg
KJ
KrJ
actact
2
22
121
2222
1
211
0
02
102 ])1(])2([[K
rJrgpfrgpgfrfg
KJ
KrJ
act
Mean field theory (2)Noise variances (white noise approximation):
2
22
121
222
1
211
0
02
102 ])1()1([K
rJrgpfrgpfrfg
KJ
KrJ
actact
2
22
121
2222
1
211
0
02
102 ])1(])2([[K
rJrgpfrgpgfrfg
KJ
KrJ
act
2
22
121
1
211
0
02
1021 ])1()1([
KrJ
rpfrpffrKJ
KrJ
act
Mean field theory (2)Noise variances (white noise approximation):
2
22
121
222
1
211
0
02
102 ])1()1([K
rJrgpfrgpfrfg
KJ
KrJ
actact
2
22
121
2222
1
211
0
02
102 ])1(])2([[K
rJrgpfrgpgfrfg
KJ
KrJ
act
2
22
121
1
211
0
02
1021 ])1()1([
KrJ
rpfrpffrKJ
KrJ
act
2
2222
11
221
0
02202
2 ])1()1([K
rJrpfrpffr
KJ
KrJ
act
Mean field theory (2)Noise variances (white noise approximation):
2
22
121
222
1
211
0
02
102 ])1()1([K
rJrgpfrgpfrfg
KJ
KrJ
actact
2
22
121
2222
1
211
0
02
102 ])1(])2([[K
rJrgpfrgpgfrfg
KJ
KrJ
act
2
22
121
1
211
0
02
1021 ])1()1([
KrJ
rpfrpffrKJ
KrJ
act
2
2222
11
221
0
02202
2 ])1()1([K
rJrpfrpffr
KJ
KrJ
act
Rate of an I&F neuron driven by white noise:
Mean field theory (2)Noise variances (white noise approximation):