2018 International Conference on Intelligent Autonomous ... · Transistor nbr doubles every year, but we cannot get energy to operate the whole chip - Dark Silicon. ii. ... Cell Leakage

March 1, 2018 1

Neuro-inspired Computing Systems

& ApplicationsBen Abdallah Abderazek

Adaptive Systems Laboratory

[email protected]

2018 International Conference on Intelligent Autonomous Systems (ICoIAS’2018), March 1-3, 2018, Singapore

The University of Aizu

2

Aizu City / Aizu University

March 1, 2018

March 1, 2018 3

Outline

• Technology Transformation

• Neuron Modeling

• Neuro-inspired Systems/Chips

• Concluding Remarks

March 1, 2018 4

Outline


• Neuron Modeling

• Neuro-inspired Systems/Chips


March 1, 2018 5

Massive amounts of data is generated.

Technology Transformation

Source: https://practicalanalytics.files.wordpress.com/2012/10/newstyleofit.jpg

March 1, 2018 6

Constant Increase of the number of transistors/cores


March 1, 2018 7

Emerging

transistors

Emerging

memories

3D

Integration

Special

architectures

There are many emerging technologies


March 1, 2018 8Khanh N. Dang, Akram Ben Ahmed, Yuichi Okuyama, and Abderazek Ben Abdallah, ”Scalable Design Methodology and Online Algorithm for TSV-cluster Defects Recovery

in Highly Reliable 3D-NoC Systems”, IEEE Transactions on Emerging Topics in Computing, 2017 (in press). DOI: 10.1109/TETC.2017.2762407

Technology Transformation3D-NoC with TSV-cluster Defects Recovery

http://ieeexplore.ieee.org/document/8070972/

https://doi.org/10.1109/TETC.2017.2762407

March 1, 2018 9

Khanh N. Dang, Akram Ben Ahmed, Xuan-Tu Tran, Yuichi Okuyama, Abderazek Ben Abdallah, ”A Comprehensive Reliability Assessment of Fault-

Resilient Network-on-Chip Using Analytical Model”, IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 25, Issue: 11, pp. 3099 –

3112, Nov. 2017. DOI:10.1109/TVLSI.2017.2736004

Technology TransformationRobust Scalable NoC


Single layer layout illustrating the TSV

sharing areas (red boxes). The layout

size is 865µm × 865µm.

The sharing TSV area are the red

boxes. Each sharing area has 8

clusters for 4 ports and 2 routers.

Khanh N. Dang, Akram Ben Ahmed, Xuan-Tu Tran, Yuichi Okuyama, Abderazek Ben Abdallah, ”A Comprehensive Reliability Assessment of Fault-

Resilient Network-on-Chip Using Analytical Model”, IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 25, Issue: 11, pp. 3099 –

3112, Nov. 2017. DOI:10.1109/TVLSI.2017.2736004

March 1, 2018 10



OASIS Network-on-Chip System

11

(7/14)

March 1, 2018


12March 1, 2018

3D-PHENIC System Architecture

Technology TransformationHybrid Electro-Photonic NoC

13

Achraf Ben Ahmed, Tsutomu Yoshinaga, Abderazek Ben Abdallah, “Scalable Photonic Networks-on-Chip Architecture Based on a Novel

Wavelength-Shifting Mechanism”, IEEE Transactions on Emerging Topics in Computing, 2017 (in press). DOI: 10.1109/TETC.2017.2737016

March 1, 2018

Technology TransformationHybrid Electro-Photonic NoC


https://doi.org/10.1109/TETC.2017.2737016

What is the issue with the current computing technology?

March 1, 2018 14


What is the issue with the current computing technology?

Scalability issue.

March 1, 2018 15


What does that mean ?

March 1, 2018 16



What does that mean ?i. Transistor nbr doubles every year, but we

cannot get energy to operate the whole chip - Dark Silicon.

ii. We double the number of transistors with smaller sizes, but we are producing much more heat in the same space.

iii. The speed of the chip increases, but the memory bandwidth does not keep-up.

March 1, 2018 17

March 1, 2018 18

stored-program

Computer.

John von Neumann Machine

John von Neumann Machine

March 1, 2018 19

stored-program

Computer.

‘’Computers are like humans- they do everything except think.’’ John von Neumann

Neuro-inspired Computing

March 1, 2018 20

Why is the brain computing style better?


March 1, 2018 21

Why is the brain computing style better?BECAUSE

Consumes low power - ～20W)

Fault tolerant -brain continues to operate even when the

circuit (neuron, neuroglia) is died)

Works in parallel ->106 parallelism vs. <101 for VN)

Faster than current computers - i.e. simulation of a 5 s

brain activity takes ~500 s on state-of-the- art supercomputer [US PTN 2016O125287A1]

Learn and think - needless to prove

How do we design this new brain-like machine?

March 1, 2018 22


How do we design this new brain-like machine?WE NEED

New Software

Parallel programming

abstraction

New Hardware

Massively Parallel

Scalable connectivity

Low-powered coresMarch 1, 2018 23


Type of Neuro-inspired Computing Systems

• Neuromorphic Sensors -electronic models of retinas and cochleas.

• Smart sensors – tracking chips,

motion, pressor, auditory classifications and localization sensors.

• Models of specific systems:

e.g. lamprey spinal cord for swimming, electric fish lateral line.

• Pattern generators – for

locomotion or rhythmic behavior

• Large-scale multi-core/chip systems – for investigating models

of neuronal computation and synaptic plasticity.

March 1, 2018 24

Neurogrid

(Stanford)

Brainscales/HBP (Heidelberg, Lausanne)

SpiNNaker

(Manchester)

TrueNorth

(IBM)

March 1, 2018 25

Outline


• Neuron Modeling

• ASL Neuro-inspired Systems/Chips


Connectivity in Human Cortex

March 1, 2018 26

Desmann, 4th Biosupercomputing Symposium, 2012

There are three known level of connections in the Human Cortex:

• connectivity of local microcircuit

• within-area connectivity with space constant

• long-range connections between areas

Neuron Modelling

Biology: Single Neuron Machine Learning

March 1, 2018 27

Dendrites

Axon

Suma

W1j

W1j

W1j

W1j

Transfer

function

Activation

function

Oj

Weights

Neuron

Biology: Tree Neurons Machine Learning

March 1, 2018 28

convolution fully connected

Neuron Modelling

How neurons communicate?1. An electrical signal travels down

the axon.

2. Chemical neurotransmitter molecules are released.

3. The neurotransmitter molecules bind to receptor sites.

4. The signal is picked up by the second neuron and is either passed along or halted.

5. The signal is also picked up by the first neuron, causing reuptake, the process by which the cell that released the neurotransmitter takes back some of the remaining molecules.

March 1, 2018 29

[From health.harvard.edu]

March 1, 2018 30

Spiking Neuron

• Computing with precisely timed spikes is more powerful than with “rates”. [W. Maass, 1999]

Electronic devise vs chemical device

March 1, 2018 32

• Deliver the concentration difference of K+,Na+• Action potential ~ 80 mV

Extreme low voltage operation Noise problem Multiple signal input/ integration

• Spatial and temporal multiplexing → Active sharing of the interconnect

• Chemical computing, extremely low operation voltage (<100mV) Low power

Fundamental interactions

March 1, 2018 33

34March 1, 2018

The electrical resistor is not constant but depends on the history of current that had previously flowed through the device.

Action Potential (Synapse) Storage(Dr. Leon Chua, 1971)

Voltage pulses can be applied to a memristor to change its

resistance, just as spikes can be applied to a synapse to change

its weight.

Wiring via AER (Asynchronous)

March 1, 2018 35

(Courtesy: iStock/Henrik5000)

March 1, 2018 36

Spike-timing-dependent plasticity (STDP)

• Adjusts the strength of connections between neurons in the brain. Adjusts the connection strengths based on the relative

timing of a particular neuron's output and input action potentials.

outputs

…

…time(ms)0 5 10 15 20 25 30 35 40

Reset

0

10

20

30

40

Mem

bra

ne

po

ten

tial

inputs

Integration & Fire

MEMRISTOR

March 1, 2018

NASH: Neuro-inspired ArchitectureS

in Hardware

36

NASH: Neuro-inspired ArchitectureS in Hardware

March 1, 2018 37

Outline


• Neuron Modeling

• ASL Neuro-inspired Systems/Chips


39

Synaptic Integration

i=8

Adder

Subtractor

Magnitude Comparator

Spike,ResetWrite Vj(t) and Delay

If >=If <

Vj(t-1)8b

Vj(t)8b

8b

Vj(t) 8b 8b λj

8b αj8b

xi(t) 8b si

SynapticIntegration

LeakIntegration

Threshold, Fireand Reset

LIF Neuro-core Architecture

• Xi(t) – Spike input to the synapse • Si – synaptic weight • Vj(t) – Membrane potential • αj – Neuron threshold• Λj – Leak value

LIF Neuro-core for NASH System

Item NC-1N NC-4N

Cell Internal Power 6.9680 μW 20.5040 μW

Net Switching Power 4.8271 μW 14.8272 μW

Total Dynamic Power 11.7950 μW 35.3312 μW

Cell Leakage Power 4.6943 μW 14.3147 μW

Item NC-1N NC-4N

Combinational Area 186.998 μm 562.856001 μm

Non-Comb Area 47.88002 μm 213.864000 μm

Total Cell Area 234.878002 μm 776.720001 μm

Table 1: Area Evaluation

Table 1: Power Evaluation

Placement of LIF-1N (Left) and LIF-4N (right)

Kanta Suzuki, Yuichi Okuyama, Abderazek Ben Abdallah, ”Hardware Design of a Leaky Integrate and Fire Neuron Core Towards the Design of a Low-power Neuro-

inspired Spike-based Multicore SoC”, Proc. Of IPSJ, 2018March 1, 2018 39

Application INeuro-inspired Hardware System for

Image Recognition

40

The H. Vu, Ryunosuke Murakami, Yuichi Okuyama, Abderazek Ben Abdallah, ”Efficient Optimization and Hardware Acceleration of CNNs towards the

Design of a Scalable Neuro-inspired Architecture in Hardware”, Proc. of the IEEE International Conference on Big Data and Smart Computing

(BigComp-2018), January 15-18, 2018March 1, 2018

March 1, 2018 41

Training with BP example

Application II Neuro-inspired Hardware System for

Autonomous Vehicles

Yuji Murakami, Yuichi Okuyama, Abderazek Ben Abdallah, ”SRAM Based Neural Network System for Traffic-Light Recognition in Autonomous

Vehicles”, Information Processing Society Tohoku Branch Conference, Feb. 10, 2018

March 1, 2018 42

Application IIINeuro-inspired Hardware System for Visual

Pattern Recognition in FARM Monitoring

Ryunosuke Murakami, Yuichi Okuyama, Abderazek Ben Abdallah, ”Animal Recognition and Identification with Deep Convolutional Neural Networks for Farm

Monitoring”, Information Processing Society Tohoku Branch Conference, Feb. 10, 2018

Application IVBrain-inspired Drone Control with BCI

Numerical computation with

SNNsBrain to Brain drone system

43March 1, 2018

March 1, 2018 44

1. Khanh N. Dang, Akram Ben Ahmed, Yuichi Okuyama, and Abderazek Ben Abdallah, ”Scalable Design Methodology and Online Algorithm for TSV-cluster Defects Recovery in Highly Reliable 3D-NoC Systems”, IEEE Transactions on Emerging Topics in Computing, 2017 (in press). DOI: 10.1109/TETC.2017.2762407

2. Achraf Ben Ahmed, Tsutomu Yoshinaga, Abderazek Ben Abdallah, “Scalable Photonic Networks-on-Chip Architecture Based on a Novel Wavelength-Shifting Mechanism”, IEEE Transactions on Emerging Topics in Computing, 2017 (in press). DOI: 10.1109/TETC.2017.2737016

3. Khanh N. Dang, Akram Ben Ahmed, Xuan-Tu Tran, Yuichi Okuyama, Abderazek Ben Abdallah, ”A Comprehensive Reliability Assessment of Fault-Resilient Network-on-Chip Using Analytical Model”, IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 25, Issue: 11, pp. 3099 – 3112, Nov. 2017. DOI:10.1109/TVLSI.2017.2736004

4. The H. Vu, Ryunosuke Murakami, Yuichi Okuyama, Abderazek Ben Abdallah, ”Efficient Optimization and Hardware Acceleration of CNNs towards the Design of a Scalable Neuro-inspired Architecture in Hardware”, Proc. of the IEEE International Conference on Big Data and Smart Computing (BigComp-2018), January 15-18, 2018.

5. Ryunosuke Murakami, Yuichi Okuyama, Abderazek Ben Abdallah, ”Animal Recognition and Identification with Deep Convolutional Neural Networks for Farm Monitoring”, Information Processing Society Tohoku Branch Conference, Feb. 10, 2018

6. Yuji Murakami, Yuichi Okuyama, Abderazek Ben Abdallah, ”SRAM Based Neural Network System for Traffic-Light Recognition in Autonomous Vehicles”, Information Processing Society Tohoku Branch Conference, Feb. 10, 2018.

7. Kanta Suzuki, Yuichi Okuyama, Abderazek Ben Abdallah, ”Hardware Design of a Leaky Integrate and Fire Neuron Core Towards the Design of a Low-power Neuro-inspired Spike-based Multicore SoC”, Information Processing Society Tohoku Branch Conference, Feb. 10, 2018.

Conclusion & References

Thank you!

45

Ben Abdallah Abderazek

Adaptive Systems Laboratory

[email protected]

2018 International Conference on Intelligent Autonomous ... · Transistor nbr doubles every year, but we cannot get energy to operate the whole chip - Dark Silicon. ii. ... Cell Leakage

Documents