PART 5 Supervised Hebbian Learning
Dec 22, 2015
PART 5Supervised Hebbian
Learning
Outline Linear Associator The Hebb Rule Pseudoinverse Rule Application
Hebb’s Postulate
“When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.”
D. O. Hebb, 1949
A
B
Linear Associator
a Wp= ai w ijp jj 1=
R
=
Hebb Rule(1/2)
Hebb Rule(2/2)
Batch Operation
Performance Analysis(1/2)
Performance Analysis(2/2)
Example(1/2)
Example(2/2)
Pseudoinverse Rule(1/2)
Pseudoinverse Rule(2/2)
Relationship to the Hebb Rule
Relationship to the Hebb Rule
Example
p1
1–
11–
t1 1–= =
p2
1
11–
t2 1= =
W TP+1– 1
1– 11 1
1– 1– +
= =
P+
PT
P 1–PT 3 1
1 3
1–1– 1 1–1 1 1–
0.5– 0.25 0.25–0.5 0.25 0.25–
= = =
W T P+1– 1
0.5– 0.25 0.25–
0.5 0.25 0.25–1 0 0= = =
Wp1 1 0 01–1
1–
1–= = Wp2 1 0 011
1–
1= =
Autoassociative Memory
Tests50% Occluded
67% Occluded
Noisy Patterns (7 pixels)
Variations of Hebbian Learning
Wnew
Wold
tqpqT
+=
Wnew
Wold
tqpqT
+=
Wnew
Wold
tqpqT
Wold
–+ 1 – Wold
tqpqT
+= =
Wnew Wol d tq aq– pqT
+=
Wnew Wold aqpqT
+=
Basic Rule:
Learning Rate:
Smoothing:
Delta Rule:
Unsupervised:
Solved Problems
Solved Problems
Solved Problems
Solution:
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solution:
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solution:
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems
Solved Problems