Energy Efficient and Scalable Neuromemristive Computing Substratesornlcda.github.io/neuromorphic2016/presentations/... · 2016-09-13 · 4/18/2014 NanoComputing Research Lab nano.ce.rit.edu

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 1

EnergyEfficientandScalableNeuromemristive Computing

SubstratesDhireesha Kudithipudi†, Cory Merkel‡, James Mnatzaganian†, Nicholas

Soures†, Qutaiba Saleh†

†NanoComputing Research LabRochester Institute of Technology

‡Information DirectorateAir Force Research Laboratory

R I T

DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 22DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

Study Model Implement

§ Brain-inspired adaptive computing platforms based on nanoscale resistive memory (memristors)

§ Memristor characteristics facilitate efficient computation and learning

§ Improve the efficiency (over conventional computers) of natural processing tasks

NeuromemristiveSystems

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 33DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

WhytheBrain?

▪ Incrediblyversatile– Canlearnanything!

▪ Energyefficient– ~1016 ops/sec@afewWatts!

▪ Robust/Resilient– Functionswithnoise!– Unreliableanddamagedcomponents!

[1]ScientificAmerican[2]www.transhumanist.com[3]http://sites.psu.edu/cigerber02141993/2014/04/14/are-two-halves-better-the-one-whole/

Whynot thebrain?

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 44DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

Brainvs.ConventionalComputing

4

Brain-like computing is better for massively parallel applicationswith noisy data and relaxed precision requirements

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 55DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

Neuromemristive Systems

5 5

Concepts

High-Level Features

Low level Features

Low level features

High-Level Features

Low level Features

Low level features Ears, Fur, Stripes

Face

Tiger Cub

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 66DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

MemristorsforPlasticity

▪ CompatibilitywithCMOS▪Memristor characteristics facilitateefficient computationandlearning

6

Biological Synapse Memristor as a Synapse

2-terminal device with state-dependent Ohm’s Law

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 77DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

ReconfigurableSynapses

Inhibitory and Excitatory Synapses

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 88DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

ReconfigurableNeurons

!Edge Detection

Non-Monotonic Neuron

Energy-Delay Product

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 99DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

On-ChipTraining▪ Variationisexploitedinthetrainingprocess

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 1010DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

RandomWeightSynapses▪Exploitrandommismatch incurrentmirrors▪Controldistributionwithsizing

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 1111DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

▪

11

On-ChipTraining

Expected output

Sensors, real-time data

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 1212DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

UnsupervisedClustering

𝑢"

𝑢#+1

-1

+1-1

𝑢# 𝑢$ 𝑢%…

…

𝑥# 𝑥$ 𝑥'

Distance

Calcu

latio

n

Mem

ristor

Crossbar

Synapses

BoostUpdate

WeightUpdate

Inpu

tsWTA

Manhattan Distance

Metric

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 1313DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

HierarchicalTemporalMemoryInspiration/Motivation

Critical Aspects Applications/Results

• Inspired by the neocortex• Highly parallelizable• Suitable for hardware

design

• Spatiotemporal data• Online, unsupervised learning• Classification & prediction• Distinct learning components• Customizable architecture

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 1414DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

MathematicalFormalizationoftheSpatialPooler

14

http://arxiv.org/abs/1601.06116

https://github.com/tehtechguy/mHTM

Overlap

Learning

inhibition

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 1515DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

ReconfigurableHTMArchitecture

15

Storage processor units may leverage PCIe SSD technology

unpublished

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 1616DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

ReconfigurableHTMArchitecture

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 1717DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

GeneralizableIntelligenceEngine

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 1818DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

ReconfigurableReservoirArchitecture

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 1919DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

ReconfigurableReservoirArchitecture

User Authentication based on Gait Patterns

Rebooting’16

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 2020DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

SmartGridLoadForecasting

20

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 2121DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

SmartGridLoadForecasting

21

0 5 10 15 20 252

3

4

5x 10 4

Time [hours]

Loa

d [M

W]

Actual Load DataPredicted−−Ideal SynapsePredicted−−CBRAM fBefore TraininggPredicted−−CBRAM fAfter Trainingg

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 2222DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

Summary

▪ Reconfigurabilityisintegraltothenatureofcomputation

–Precomputation isoccurringincommunicationchannels

–Nostandardizedmetrics/benchmarkstoevaluate– Designingtechnologyagnosticvs.technologyawaresystems

▪ Lookingforward– oneshotlearning….

4/18/2014NanoComputingResearchLab

nano.ce.rit.edu 2323DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited; 27 June 2016.

Team&Collaborators