Kirchhoff Institute for Physics Johannes Schemmel Ruprecht-Karls-Universität Heidelberg 1 Accelerated Neuromorphic Hardware : Hybrid Plasticity - The Next Generation Johannes Schemmel Human Brain Project Subproject Neuromorphic Computing Neuromorphic Computing with Physical
17
Embed
Kirchhoff Institute for Physics Johannes Schemmel Ruprecht-Karls-Universität Heidelberg 1 Accelerated Neuromorphic Hardware : Hybrid Plasticity - The Next.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Kirchhoff Institute for PhysicsJohannes SchemmelRuprecht-Karls-Universität Heidelberg 1
Accelerated Neuromorphic Hardware :
Hybrid Plasticity - The Next Generation
Johannes Schemmel
Human Brain Project
Subproject Neuromorphic Computing
Neuromorphic Computing with Physical Models
Kirchhoff Institute for PhysicsJohannes SchemmelRuprecht-Karls-Universität Heidelberg 2
Overview
• Overview of the NM-PM1
system
• Modeling with the NM-PM1
system
• Hybrid Plasticity
• NM-PM2 – HICANN DLS
• Prototype
• Results
NM-PM : Neuromorphic Computing with Physical Models
Kirchhoff Institute for PhysicsJohannes SchemmelRuprecht-Karls-Universität Heidelberg 3
Physical Model Example : Continuous Time Integrating Membrane Model
DV [V] gleak [S] Cm [F] (gV)/C [V/s]
Biology(*)
10-2 10-8 10-10 100
VLSI 10-1 10-6 10-13 106
Consider a simple physical model for the neuron’s cell membrane potential V:
VEgdt
dVC leakleakm Cm
R = 1/gleak
Eleak
V(t)
(*) from Brette/Gerstner, J. Neurophysiology, 2005
Kirchhoff Institute for PhysicsJohannes SchemmelRuprecht-Karls-Universität Heidelberg 10
Hybrid Plasticity
Problem : millions of parameters• network topology• neuron sizes and AdEx-parameters• synaptic strengthsCurrent status : everything is pre-computed on host-computer• requires precise calibration of hardware• takes long time
(much longer than running the experiment on the accelerated system)
Integrate flexible plasticity mechanisms : “Hybrid Plasticity”• no calibration of synapses necessary• plastic topology and delays• learning replaces calibration• combination of analog correlation measurement and digital
Plasticity Processing Unit (PPU)
Kirchhoff Institute for PhysicsJohannes SchemmelRuprecht-Karls-Universität Heidelberg 11
Second Generation Neuromorphic ASIC : HICANN-DLS
analognetwork
core
bottom ppu
top ppu
digitalcorelogic
fast ADC
verticallayer1
repeaters
horizontal layer1
repeaters
SERDESchannel 0
output amplifier
main PLL
SERDESchannel 1
SERDESchannel 2
SERDESchannel 3
synthesized RTLmixed full customanalog full custom
analog outputs
TX data
TX clk
RX clk
RX data
extclkJTAG and reset
TX dat
L1 top
L1 right
L1 left
L1 bot
synapse tl, tr, bl, br
Kirchhoff Institute for PhysicsJohannes SchemmelRuprecht-Karls-Universität Heidelberg 12
Plasticity : Hybrid Scheme Provides Flexibility
• analog correlation measurement in synapses
• A/D conversion by parallel ADC
• digital Plasticity Processing Units→ full access to synapse weights→ full access to configuration data
SIMD Plasticity Processing Unit
ADC arrayparallel conversion of STDP readout
Kirchhoff Institute for PhysicsJohannes SchemmelRuprecht-Karls-Universität Heidelberg 13
NM-PM2 Prototype
plasticity processor
synapse array
neuron circuits
FPGA based controller board
Kirchhoff Institute for PhysicsJohannes SchemmelRuprecht-Karls-Universität Heidelberg 14
Concept of Hybrid Plasticity Operation
• Synapse measures time-difference between pre- and post synaptic signals• Time-difference is exponentially weighted• Results are accumulated within each synapse for causal and anti-causal
correlations separately• Accumulated correlation measures are digitized • PPU uses digitized values together with current weights to calculate new weight
Kirchhoff Institute for PhysicsJohannes SchemmelRuprecht-Karls-Universität Heidelberg 15
𝜔 ′−=𝜔−𝑏−𝜔 exp(− ∆ 𝑡
𝑐−)
Measurement Results for Multiplicative STDP Rule
𝜔+¿ ′=𝜔+𝑏+¿ (𝜔max−𝜔 ) exp¿ ¿¿ ¿
Kirchhoff Institute for PhysicsJohannes SchemmelRuprecht-Karls-Universität Heidelberg 16
Measurements Demonstrating Possible STDP Rules
Hebbian :Anti-Hebbian :
AsymmetricSensitivity :
Bistablelearning :
• very early results using only variations of the STDP PPU code
• PPU also supports : • supervised plasticity• reinforcement
learning• including neuron
firing rates in plasticity rules
• adding additional digital synaptic state variables
• anything you can code …
The research leading to these results has received funding from the EU FP7 Programme under grant
agreement nos. 269921 (BrainScaleS) and 604102 (HBP).
This endeavor would not have been possible without the tireless commitment of all the involved students and
colleagues, which unfortunately are too many to name them all here individually.