Josephson Junction Based Neuromorphic ComputingTalk3... · 2017. 12. 20. · Josephson Junction Based Neuromorphic Computing Mike Schneider, Stephen Russek, Christine Donnelly, Burm

Josephson Junction Based Neuromorphic Computing

Mike Schneider, Stephen Russek, Christine Donnelly, Burm Baek, Matt Pufall, Ian Haygood, Pete Hopkins, Paul Dresselhaus, Sam Benz, Bill Rippard

12/14/2017 2

Artificial Intelligence

“Intelligence” exhibited by machines

Machine Learning

Field of study that gives computers the ability to learn

without being explicitly programmed

Bio – Inspired

Drawing inspiration from the human brain

Neuromorphic Hardware

Deep Neural Networks

eyeriss.mit.edu

12/14/2017 3

Software neural networks are mainstream

• Apple: A11 bionic SoC, Siri

• Google: Search results (RankBrain), translate, street view, …

• Microsoft: Computational Network Toolkit (speech recognition)

• Amazon: from product recommendation to robotic picking routines

• Facebook Ranking, Language translation, …

M. Nielsen, “Neural Networks and Deep Learning” (2015)

12/14/2017 4

Neural networks dominate modern image recognition

2010 2011 2012 2013 2014 2015 2016 20170

5

10

15

20

25

30

Cla

ssific

atio

n E

rro

r (%

)

Year

Deep Convolutional Neural Network (Trained with GPUs)

Human Accuracy ~ 5% error rate

Feature based algorithms

ImageNet Competition

12/14/2017 5

What interaction is being modeled in neural networks?

Wikimedia neuronimage

x0

axon

w0

synapse

w0 x0

x1w1

Cell body

w1 x1

𝑓

𝑖

𝑤𝑖𝑥𝑖 + 𝑏output

…

Neuromorphic Single Flux Quantum

6

Information transfer: quantized pulse trains

Long distance “lossless” pulse transmission

3D architecture

Memory/ plasticity

Neural Single Flux Quantum

axons Josephson/passive transmission lines

Magnetic JJsynapse

• Barrier = insulator, semiconductor, normal metal, ferromagnet, …

7

Josephson junction (JJ)

sincs II

1

101

ie 2202

ie

𝜑0

2𝜋𝐶 ሷ𝜃 +

𝜑0

2𝜋𝑅𝑛

ሶ𝜃 + 𝐼𝑐𝑠𝑖𝑛 𝜃 − 𝐼𝑏 = 0

Ib

Circuit Model of a Josephson Junction

8

𝑈 =𝐼𝑐𝜙02𝜋

(−cos 𝜃 −𝐼𝑏𝐼𝑐𝜃)DampingMass

Vse

h 150 1003.2

2 where

Interaction of order parameters

9

S F

Phase Progression

Exchange Field

Ph

ase

FM Thickness|C

riti

cal C

urr

ent

Den

sity

|

Jc > 0 Jc < 0(-JJ)

Jc > 0

S-F-S

– eiQ∙R – e-iQ∙R

= ( – ) cos(Q∙R)+ i( + ) sin(Q∙R)

Singlet(S = 0)

Singlet + Triplet(S = 0) (S = 1)

Superconducting Spintronics Reviews

Buzdin, Rev Mod Phys 2005Bergeret, Rev Mod Phys 2005Eschrig, Phys Today 2011Blamire, J of Phys Cond Matt 2014Linder and Robinson, Nature Phys 2015

Fulde-Ferrell-Larkin-Ovchinnikov(FFLO) state

S

Magnetic nanoclusters in a Josephson junction

10

Thin Films

Si

Mn

0 100 200 300

0

30

60

90 10 mT

5 mT

0.5 mT

0 mT

Mom

ent (n

Am

2)

Temperature (K)

ordered

disordered

Nb

Nb

Changing magnetic order with fast electrical pulses in devices

12/14/2017 11

H

H

Small applied magnetic field

250 psElectrical pulse

Only the small applied magnetic field

250 psElectrical pulse

+

H=0

The basic neuro-inspired cell

12

Magnetic Josephson junction neural network simulation

12/14/2017 13

Pre-synaptic Neurons

Synaptic Weights

Post-synaptic Neurons

• 9 input pixel (JJs)• 27 weights (MJJs)• 3 outputs (MJJ SQUIDs)

Real time (~ 3ns) recognition

14

100111001

Prezioso et al. Nature 2015

z

v

n

Real time (~ 3ns) recognition

15

100111001

0 20 40 60 80 100

0

20

40

60

80

100

120

Time (ns)

Offse

t V

olta

ge

(

V)

Prezioso et al. Nature 2015

Real time (~ 3ns) recognition

16

000111001

0 20 40 60 80 100

0

20

40

60

80

100

120

Time (ns)

Offse

t V

olta

ge

(

V)

Prezioso et al. Nature 2015

Large scale image recognition

• ARGUS

• 1.8 billion pixels at 12 fps

• resolution of 6 inches over 10 square miles

12/14/2017 17

• Currently pre-processed on board using kilowatts

• Full stream processed on a supercomputer

Scale to a large network

12/14/2017 18

• Recent ImageNet winners use ~ 1010 MA-operations / image

• ~ 1017 MA-operations /sec to process ARGUS real time

• JJ neural network would take ~ 3 watt (including cooling)

( 1 spike / MA-operation)

Benchmarks

12/14/2017 19

• Energy• Operational energy < 10-18

• Training energy < 10 -17

• Cooling overhead ~ 103

• (Human brain ~ 10-15)

• Speed• Operational (device) ~ 100 GHz• Operational (circuit) ~ 10 GHz

• Size • Demonstrated 1.5 x 3 µm• Demonstrated for MJJ 100 nm

• Scalability• Demonstrated ~ 101

• There is a lot of scaling to go…

Conclusions

12/14/2017 20

• Josephson junctions are a very promising neuromorphic computing technology

• Magnetic Josephson junctions are a cryogenic non-volatile memory

• Magnetic Josephson junctions can be used for the synaptic function

• There is a lot of scaling needed for Josephson junction based neuromorphic computing