CVG-UPM COMPUTER VISION Machine Learning & Neural Networks 7.- Non supervised Neural Networks: Self-organizing Maps by Pascual Campoy Grupo de Visión por Computador U.P.M. - DISAM 3 CVG-UPM COMPUTER VISION Machine Learning and Neural Networks P. Campoy P. Campoy Unsupervised learning Feature space Unsupervised learning concept ? area length Working structure y 1 . . y m . . x n x 1 Clustering
12
Embed
Machine Learning & Neural Networks 7.- Non supervised ... · P. Campoy Machine Learning and Neural Networks Self organizing Maps (SOM) Bio-inspired idea: Similar inputs map onto neighbor
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
CVG-UPMC
OM
PU
TE
R V
ISIO
N
Machine Learning and Neural NetworksP. CampoyP. Campoy
Machine Learning & Neural Networks
7.- Non supervised NeuralNetworks:
Self-organizing Mapsby
Pascual CampoyGrupo de Visión por Computador
U.P.M. - DISAM
3
CVG-UPM
CO
MP
UT
ER
VIS
ION
Machine Learning and Neural NetworksP. CampoyP. Campoy
Unsupervised learning
Feature space
Unsupervised learning concept
?
area
leng
th
Working structure
y1..ym
.
.xn
x1
Clustering
4
CVG-UPMC
OM
PU
TE
R V
ISIO
N
Machine Learning and Neural NetworksP. CampoyP. Campoy
Self organizing Maps (SOM)
Bio-inspired idea:
Similar inputs map onto neighbor outputs.
SOM objective:
Neighbor inputs map onto neighbor outputs
and vice versa
Rn → R2, R1 D.R. into a pattern space
5
CVG-UPM
CO
MP
UT
ER
VIS
ION
Machine Learning and Neural NetworksP. CampoyP. Campoy
recent paper
6
CVG-UPMC
OM
PU
TE
R V
ISIO
N
Machine Learning and Neural NetworksP. CampoyP. Campoy
SOM working principle
Objective: To obtain a bijective application Rn ⇔ R2, such as neighborhood in the input space ⇔ neighborhood in the output space Procedure: To distribute an elastic 2D lattice
into the nD input space, wherethe every cross represent aneuron that has:
- a position in the input space(defined by its weights)
- a position in the output space (defined by its coordinates in the lattice)
w
7
CVG-UPM
CO
MP
UT
ER
VIS
ION
Machine Learning and Neural NetworksP. CampoyP. Campoy
SOM: viability
is it possible in this cubic example that any two neighbor inputsample are represented by neighbor neurons?
and in this Swiss roll example?
concept ofIntrinsic Dimensionalityof the data
8
CVG-UPMC
OM
PU
TE
R V
ISIO
N
Machine Learning and Neural NetworksP. CampoyP. Campoy
SOM: running and learning
the neurons whose weight vector is the closestto this input data
Learning:
How are weights updated for every train inputin order to fulfill the SOM objectives? The weights of which neurons are updated? How are they updated?
Running:Which neuron is activated by every input data?
9
CVG-UPM
CO
MP
UT
ER
VIS
ION
Machine Learning and Neural NetworksP. CampoyP. Campoy
SOM: learning procedure
The neuron whose weights are the closest tothe present train sample x, called the winningneuron wb (also the best matching unit), andits neighbors are the ones that learn (i.e.update their weights)
α
ds(wi-wb)
α
k
where α = α(dos(wi-wb),k) is function of:
- the distance to the winning neuron in the output space dos(wi-wb),- the training instant k (e.g. epoch)
Learning rule: Δkwi = α (x-wi)
10
CVG-UPMC
OM
PU
TE
R V
ISIO
N
Machine Learning and Neural NetworksP. CampoyP. Campoy
SOM: neural implementation
Training and running imply distance calculation, thatcan be implemented by scalar product in a onedimensional incremented space
.
.
.
.
.
x1
xI
xi
.
.
.
.
.
.
11
CVG-UPM
CO
MP
UT
ER
VIS
ION
Machine Learning and Neural NetworksP. CampoyP. Campoy
SOM: discussion on objective fulfillment
feature 1
feat
ure
2
feature 1
feat
ure
2
examples R2 → R1
13
CVG-UPMC
OM
PU
TE
R V
ISIO
N
Machine Learning and Neural NetworksP. CampoyP. Campoy
SOM: examples 1
14
CVG-UPM
CO
MP
UT
ER
VIS
ION
Machine Learning and Neural NetworksP. CampoyP. Campoy
SOM: example 2
15
CVG-UPMC
OM
PU
TE
R V
ISIO
N
Machine Learning and Neural NetworksP. CampoyP. Campoy
σv=0.25
α0=0.1
σv=0.5
σv=0.75
α0=2.1
SOM results:influence of learning parameters
16
CVG-UPM
CO
MP
UT
ER
VIS
ION
Machine Learning and Neural NetworksP. CampoyP. Campoy different instances
SOM result different order
different # neurons
SOM: influence of training samples and # of neurons
17
CVG-UPMC
OM
PU
TE
R V
ISIO
N
Machine Learning and Neural NetworksP. CampoyP. Campoy