Biologically Inspired Robotics Group,EPFL Associative memory using coupled non-linear oscillators Semester project Final Presentation Vlad TRIFA.

QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.

Biologically Inspired Robotics Group,EPFLQuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.

Associative memory using coupled

non-linear oscillators

Semester project

Final Presentation

Vlad TRIFA

Project summary

Litterature review Implementation of an associative memory using coupled

oscillators and analysis of performance/drawbacks. Mixture with the BIRG model

Generalization to complex signals Better control on the capacity

Final discussion about relevant issues concerning the performances of both models

Conclusion

Associative Memory

Animal and human memory works by association.Able to retrieve a stored pattern upon presentation of a partial and noisy representation of an input signal.

Many models developed since early 80’s• Concepts taken from statisic mechanics and hebbian learning

rule turned neural networks into dynamic systems.• Useful into understanding dynamics of networks (emergence)

but…• Lack of biologically plausible mechanisms (coupling, binary,…)• Low capacity and performance (Global coupling: N2 parameters)

Oscillators

Oscillating systems are very common in nature and possess very intersting properties.

• Synchronization• Energy efficient mechanism for temporal correlation

Many brain processes rely on interaction of oscillators• CPG• Olfactory and visual cortex • Temporal correlation hypothesis and binding problem

Information can be stored as phase relationships patterns, where coupled oscillators converge.

Analyzed model

Can be found in [Borisyuk, 2001].

Oscillators described by phase, amplitude, and frequency.

The model

Dynamics

Model performance

The capacity of this model is not easy to derive, due to the random phase shifts, and to the dynamics of the nonlinear term:

We do not know what percentage of overlapping is

possible, as memorized patterns can be “overwritten“. Implies that error increases as memory is filled. Robustness due to distributed memorization. But, loss of groups influence strongly the retrieval error.

Discussion

The model is interesting as it is based on oscillating systems, thus can be easily implementable on many oscillating systems (PLL, etc…).

A very nice methodology that is embedded in the system is proposed in order to decide where to store an input signal is proposed.

Random phase shifts ensure some robustness to the system, but too big influence on the performance.

Drawbacks

The all-to-all coupling into groups is not efficient computationally and it uses too many oscillators.

Due to the explicit input signal embedded in the equations, we can only learn sine functions.

The input dimension is annoying. Complexity is increased with no performance increase.

The time is reset after each stimulus. We need to present the input in-phase with the oscillators. We cannot learn sequences.

Improvements

We want to be able to learn complex signals.

Starting from the model in [Righetti et al, 2005], we want to extend the model to form a network.

Discussion

Simpler model, more computationally efficient. We gained a much better control on the amount of

oscillators to dedicate for a frequency component. We are able to memorize complex signals in a robust

and fault tolerant manner, under some constraints.

but… Unfortunately, the capacity depends on the complexity of

the signals to store. We lost the selection of the storage sites based on

phase relationships we had with the previous model.

Future work

We need to find a mechanism (embedded in the dynamics) that can select where each component should be stored depending on the signal.

It would be very intersting to create links between different clusters activated by the same signal, similar to associative connections forming according to the correlation of neural activity between assemblies, enhancing robustness if attenuated components.

Reduce parameters, so we need only to select the amount of oscillators allocated per component.

Conclusion

This work should be considered as an attempt to provide insights on how it is possible to store information encoded as a complex signals in a reliable manner, simply by using oscillating systems with local interactions.

Our approach is interesting as it uses some concepts that are common in biological neuronal networks such as Oscillating components with local interactions No global external process to supervise the learning procedure

Thank you!

References :

[Borisyuk, 2001]

[Righetti et al., 2005]

[Singer, 1995]