RANC as Neural Network Accelerator

DNNARA: A Deep Neural Network Accelerator

using Residue Arithmetic and

Integrated Photonics

Jiaxin Peng, Yousra Alkabani, Shuai Sun, Volker Sorger, and Tarek El-Ghazawi

49th International Conference on Parallel Processing – ICPP

August 2020

Outline

➢Introduction

➢Background

➢Integrated Photonic Residue Arithmetic Computing Engine for Convolutional Neural Network

➢Performance Evaluation

➢Conclusion

2

Introduction

3

Introduction

➢Some NN applications require real-time analysis for inference

➢Computation intensive; includes billion multiply-accumulate (MAC) operations

➢We propose DNNARA: a deep neural network accelerator using residue arithmetic based on integrated photonics

➢All the computations through the neural network are done in residue number system (RNS) to avoid extra binary to/from RNS conversions

4Block Diagram of a DNNARA System

Introduction

➢DNNARA: RNS with wavelength-division multiplexing (WDM)

• Execute multiple MVMs due to WDM feature

• Speedup MVMs due to digit-independent feature

• Residues are small-sized

• Increase the system parallelism – save area/hardware resources

5

Background➢ Convolutional Neural Network

➢ Residue Number System

6

Background – Convolutional Neural Network

➢Widely applied in classification• Image recognition

➢Including several layers/functions• Convolutional layers• Activation functions – add non-linearity

• ReLu (Rectified Linear Unit)• Sigmoid function / Hyperbolic tangent function

• Pooling layers – down ample the output• Max pooling• Average pooling

• Fully-connected layers

➢Contains up to billion multiply-accumulate (MAC) operations

7

Background - Residue Number System (RNS)

➢Each Integer X is represented by its “residue,” or remainder obtained by dividing it by a modulus Mi

• Example: Moduli are M1=2, M2=3, M3=5, M4=7• X = 20 is represented as X={0, 2, 0, 6}[2, 3, 5, 7]

• Range of numbers that can be represented: 0 to (M– 1) (here 0 to 219) (M=M1*M2*M3*M4)• Moduli should be relatively prime

➢Negative Number Notation: Similar to 2’s compliment• r = |m-|-X|m|m (where X is negative)• Example: -20 = {|2-0|2, |3-2|3, |5-0|5, |7-6|7}[2, 3, 5, 7] = {0, 1, 0, 1}[2, 3, 5, 7]

• Range of numbers that can be represented: [−(𝑀−1)/2,(𝑀−1)/2]if M is odd, or[−𝑀/2,𝑀/2−1]if M is even

➢Residue Arithmetic: Operations carried out on residues• Example: Addition of X=20={0, 2, 0, 6}[2, 3, 5, 7] and Y=5={1, 2, 0, 5 }[2, 3, 5, 7]

• X+Y = {0+1, 2+2, 0+0, 6+5 }[2, 3, 5, 7] → = {1, 1, 0, 4 }[2, 3, 5, 7]

• X*Y = {0*1, 2*2, 0*0, 6*5 }[2, 3, 5, 7] → = {0, 1, 0, 2 }[2, 3, 5, 7]

• Residue arithmetic is carried out as modulo additions and multiplication on the residues• Residue arithmetic is carried out on each residue in parallel

8

Integrated Photonic Residue Arithmetic Computing Engine for Neural Network

➢ Overview

➢ Residue Adders and Multipliers

➢ Residue Matrix-Vector Multiplication Unit

➢ Sigmoid Unit

➢ Max Pooling Unit

9

• R-MVM: Residue Matrix-Vector Multiplication

• R-Multiplier: Residue Multiplier

• R-Adder: Residue Adder

• MRR: Micro-Ring Resonator

• PD: Photo-Detector

• LUT: Look-up Table

• RNS2Bin: RNS to Binary

• Bin2RNS: Binary to RNS

• T: tile

Overview Architecture

10

Integrated Photonic Residue Adder and Multiplier

➢Basic block• An electro-optical 2×2 switch• Light either propagates through (“bar” state – (a))or

propagates cross (“cross” state – (b))

➢Residue Adder [1] – one-hot encoding• Could be considered as a mapping (injection)• Arbitrary Size Benes (AS-Benes) Network (c – even

number & d – odd number)• Switch states are precomputed and stored in a look-

up table (LUT)

➢An AS-Benes modulo-5 adder (e)• Example with |3+4|5 = 2

➢A Modulo-N Residue Multiplier Implementation (f)

➢WDM capable

11

Residue MVM (R-MVM) Computing Block

➢Schematic of designed R-MVM (b)

➢Wavelength-Division Multiplexing (WDM) Capable

➢Lasers, MRRs, PDs, LUTs, Registers, as well as photonic and electrical connections are needed

➢sel to choose either the partial sum or bias

➢Example: 5x5 input feature and a 2x2 kernel

12

Pipeline of a MAC operation

• Cycle 1:• Input feature (x) are encoded as light with

different wavelengths• Weights (w) are encoded as the selection line,

loading the states of switches in the LUT

• Cycle 2:• Setup the switch states accordingly• Inject light and detect light - multiply• MRRs & PDs act like filter to derive the

solutions for all the multiplications

13

Pipeline of a MAC operation

• Cycle 3:• Results from last cycle (w*x) are decoded as

the selection line to load the states for adders• According to sel, either the partial sum or bias

is decoded as the light

• Cycle 4:• Setup the adders• Inject light and detect light – add

• Cycle 5: Write back to the register

14

Sigmoid Function Unit - Polynomial

➢In residue domain, it is hard to calculate the sigmoid function

➢Instead, it could be considered as a polynomial because sigmoid function could be represented as Taylor series

➢Need to pre-calculate the terms that include x, and build the connection accordingly

➢Example: P(x) = ax4 + bx3 + cx2 + dx + e in modulo-5 system

15

Max pool Function Unit

➢Sign detection in RNS is implicit

➢Instead, we convert the number from RNS to MRS (mixed-radix number system) [2]

➢From the MRS, the coefficient of even number 2 (a4) separates the number to negative or non-negative

➢It is serial but could be pipelined

16

Performance Evaluation

17

Experiment Setup

➢Electrical memory component• CACTI 7.0 [3],

➢Optical Switch [4]• Lumerical FTDT

➢Optical circuit• Lumerical Interconnect

➢Lasers/MRRs/PDs• Data from other work ([5], [6],and [7],

respectively)

➢HyperTransport serial link • Data from [8]

➢System Level Design• Our own simulator

18

Configurations of Selected Benchmarks

Design Space Exploration

➢Swept Parameters• WDM size

• # of tiles in a chip

• # of MVMs in a tile

➢Computation capability• # of operations

/(time*area*power)

19

Hardware Specification

20

Speed & Power Analysis

➢Real benchmarks

➢The more chip the faster but did not scaled proportionally

➢Consumes more power

➢Due to communication

➢19 times faster compared to a GPU (Nvidia Tesla V-100) for VGG-4 with same power budget

21

Conclusion

➢Proposed DNNARA, a deep neural network accelerator that using residue number system

➢DNNARA is a hybrid electro-optical design

➢Proposed a system-level CNN accelerator chip with nano-photonic

➢Built a system-level simulator for experimental estimation

➢Could reach up to 12.6 GOPS/(second·mm2· watt)

➢Reached 19 times faster compared to a state-of-art GPU (Nvidia Tesla V-100) for VGG-4 with same power budget

22

References

➢ [1] Jiaxin Peng, Yousra Alkabani, Shuai Sun, Volker Sorger, and Tarek El-Ghazawi.2019. Integrated Photonics Architectures for Residue Number System Computations. In IEEE International Conference on Rebooting Computing (ICRC 2019).129–137.

➢ [2] Nicholas S Szabo and Richard I Tanaka. 1967.Residue arithmetic and its applications to computer technology. McGraw-Hill.

➢ [3] Naveen Muralimanohar, Rajeev Balasubramonian, and Norm Jouppi. 2007. Optimizing NUCA organizations and wiring alternatives for large caches with CACTI6.0. In Proceedings of the 40th Annual IEEE/ACM International Symposium on Microarchitecture. IEEE Computer Society, 3–14.

➢ [4] Shuai Sun, Vikram K Narayana, Ibrahim Sarpkaya, Joseph Crandall, Richard ASoref, Hamed Dalir, Tarek El-Ghazawi, and Volker J Sorger. 2017. Hybrid photonic-plasmonic nonblocking broadband 5×5 router for optical networks. IEEE Pho-tonics Journal10, 2 (2017), 1–12.

➢ [5] Rupert F Oulton, Volker J Sorger, Thomas Zentgraf, Ren-Min Ma, Christopher Gladden, Lun Dai, Guy Bartal, and Xiang Zhang. 2009. Plasmon lasers at deep subwavelength scale.Nature461, 7264 (2009), 629

➢ [6] Erman Timurdogan, Cheryl M Sorace-Agaskar, Jie Sun, Ehsan Shah Hosseini, Aleksandr Biberman, and Michael R Watts. 2014. An ultralow power a thermal silicon modulator. Nature communications5 (2014), 4008.

➢ [7] Yannick Salamin, Ping Ma, Benedikt Baeuerle, Alexandros Emboras, Yuriy Fedoryshyn, Wolfgang Heni, Bojun Cheng, Arne Josten, and Juerg Leuthold. 2018.100 GHz plasmonic photo detector. ACS photonics5, 8 (2018), 3291–3297.

➢ [8] Yunji Chen, Tao Luo, Shaoli Liu, Shijin Zhang, Liqiang He, Jia Wang, Ling Li,Tianshi Chen, Zhiwei Xu, Ninghui Sun, et al.2014. Dadiannao: A machine-learning supercomputer. In Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture. IEEE Computer Society, 609–622.

23

24

Thank you!