Aspire / RISC-V / Rocket / Accelerators Lecture 02cs250/fa14/lectures/lec02.pdf · Aspire / RISC-V / Rocket / Accelerators – Lecture 02 Jonathan Bachrach EECS UC Berkeley September

Aspire / RISC-V / Rocket / Accelerators – Lecture 02

Jonathan Bachrach

EECS UC Berkeley

September 2, 2014

Today 1

Parlab / AspireRISC-V + RocketIron Law + OptimizationsAccelerators + RoCC— break —Decoupled Interfaces in ChiselRoCC Implementation in ChiselTowards General Purpose Accelerators

ParLab Project 2

Got parallel computers but how do we write parallel software?

Principle Investigators: Krste Asanovic Ras Bodik, Jim Demmel,Armando Fox, Tony Keaveny, Kurt Keutzer, John Kubiatowicz, NelsonMorgan, David Patterson, Koushik Sen, David Wessel, Kathy Yelick

Founding Companies: Intel and MicrosoftAffiliates: National Instruments, NEC, Nokia, Nvidia, Samsung, andOracle/Sun.

ParLab Project 3

Got parallel computers but how do we write parallel software?

How to Make Parallelism Visible? 4

In a new general-purpose parallel language?An oxymoron?Won’t get adopted?Most big applications written in >1 languages

Par Lab bet on Patterns at all levels of programmingPatterns provide a good vocabulary for domain expertsAlso comprehensible to efficiency-level experts or hardware architectsLingua franca between the different levels in ParLab

Patterns and Hardware Platforms 5

Mapping Patterns to Hardware 6

Specializers: Pattern-specific and platform-specific compilers

Aspire Project 7

Algorithms and Specializers for Provably Optimal Implementationswith Resiliency and Efficiency

http://aspire.eecs.berkeley.edu

Principle Investigators: Krste Asanovic (Director), Jonathan Bachrach,Armando Fox, Jim Demmel, Kurt Keutzer, Borivoje Nikolic, DavidPatterson, Koushik Sen, and John Wawrzynek

Future App Drivers 8

Compute Energy Iron Law 9

performance = power ∗ energy efficiency( tasks

second ) = ( joulessecond ) ∗ ( tasks

joule )

when power is constrained, need better energy efficiency for moreperformancewhere performance is constrained (real-time), want better energyefficiency to lower power

Improving energy efficiency is critical goal for all future systems andworkloads

Good News: Moore’s Law Continues 10

Bad News: Dennard Scaling Over 11

Aspire Goal 12

Keep computers’ performance and energy efficiency improving past endof CMOS transistor scaling until new switch technology deployed

[Graph from “Advancing Computers withoutTechnology Progress”, Hill, Kozyrakis, et al.,DARPA ISAT 2012 ]

Modern CMOS givesbillions of transistors,reliably interconnected,clocking at GHz,for a few dollars

Future Efficiency Gains Above Transistor Level

End of Sequential Processor Era 13

Parallelism: A one-time gain 14

use more, slower cores for better energy efficiency, either

simpler coresLimited by smallest sensible core

or

run cores at lower Vdd/frequencyLimited by Vdd/Vt scaling, errors

Now what?

Dark Silicon 15Opportunity: If only 10% die usable, build 10 different specializedengines and only use one at a time.

End of General-Purpose Processors 16

Most computing happens inspecialized, heterogeneousprocessors

Can be 100-1000X moreefficient thangeneral-purposeprocessor

Challenges:Hardware design costsSoftware developmentcosts

Nvidia Tegra2

Real Scaling Challenge: Communication 17

As transistors become smaller and cheaper, communication dominatesperformance and energy

All scales:Across chipUp and down memoryhierarchyChip-to-chipBoard-to-boardRack-to-rack

Provable Optimal Comm Lower Bounds 18

1) Prove lower bounds on communication for a computation2) Develop algorithm that achieves lower bound for system3) Find that communication time/energy cost is >90% of resultingimplementation4) We know we’re within 10% of optimal!

Supporting technique: Optimizing software stack and computeengines to reduce compute costs and unavoidable communicationcosts

ESP: an Applications Processor Arch for Aspire 19

Intel Ivy Bridge (22nm)

Qualcomm SnapdragonMSM8960 (28nm)

Future server and mobileSoCs will have manyfixed-function acceleratorsand a general-purposeprogrammable multicoreWell-known how tocustomize hardwareengines for specific taskESP challenge is usingspecialized engines forgeneral-purpose code.

ESP: Ensembles of Specialized Processors 20

General-purpose hardware, flexible but inefficientFixed-function hardware, efficient but inflexibleParLab Insight: Patterns capture common operations across manyapplications, each with unique communication and computationstructureBuild an ensemble of specialized engines, each individuallyoptimized for particular pattern but collectively covering applicationneedsAspire Bet: ESP will give efficiency and flexibility

ESP Engines 21

Optimize compute and data movement per patternDense Engine: Provide sub-matrix load/store operations, supportin-register reuseStructured Grid Engine: Supports in-register operand reuseacross neighborhoodSparse Engine: Support load/store of various sparse datastructuresGraph Engine: Provide load/store of bitmap vertex representations,support many outstanding request

Richer semantics of new load/stores preserved throughout memorysystem for memory-side optimizations

RISC-V 22

BackgroundDesigned at BerkeleyFifth Berkeley RISC design

AdvantagesOpen source with modified BSD license www.riscv.orgEfficient to implementExtensible

State2.0 Spec outFast functional simulatorGCC tool chainLLVM tool chainBoots linuxlowRISC and india government support

RISC-V Programming Model 23

31 General Purpose IntegerRegistersRegister to RegisterOperationsLoad / Store withAddressing ModesControl Transfer Operations

XLEN-1 0x0 / zerox1x2x3x4x5x6x7x8

...x24x25x26x27x28x29x30x31XLEN

XLEN-1 0pc

XLEN

RV32 Instruction Encoding 24

simple symmetric formateasy and efficient to decode

integer instruction format

coprocessor instruction format

RISC-V Vec Inc Example 25

int a[64];

for (int i = 0; i < 64; i++)

a[i] += 1;

gcc -O3 -S ...

move x3,x0 // x3 count

li x6,64 // x6 lim

$LOOP: lw x5,0(x4) // x4 idx

addw x2,x3,1 // inc count

move x3,x2

addw x5,x5,1 // inc val

sw x5,0(x4) // update

add x4,x4,4 // inc idx

bne x2,x6,$LOOP

7 cycles / element inc

C compiler -O3 versus Hand Assembly 26

gcc -O3 -S ...

move x3,x0

li x6,64

$LOOP: lw x5,0(x4)

addw x2,x3,1

move x3,x2

addw x5,x5,1

sw x5,0(x4)

add x4,x4,4

bne x2,x6,$LOOP


optimized by hand

lw t0, a

lw t1, a+64*4

$LOOP: lw t2, 0(t0)

addw t0, t0, 4 // inc idx

addw t2, t2, 1 // inc val

sw t2, -4(t0)

bne t0, t1, $LOOP


Iron Law 27

timeprogram

=instructions

program∗ cycles

instruction∗ time

cycle

Instructions / program depends on source code, compiler, and ISACPI = cycles/instruction – depends on ISA and microarchitectureTime / cycle depends on microarchitecture + underlying technology

By pipelining can lower time / cycle without increasing CPIBy issuing multiple instructions can lower CPI further

Rocket 28

in-order 6 stage pipelinesingle issueCPI = 1 with no hazards

I$

+4

decode

RF

BTB

D$

pcgen fetch decode execute mem commit

Pipelining CPI 29

from Krste’s CS152 slide

Rocket Data Memory 30

64 byte cache linenon-blocking L1 cache withfour cache line misses inflight1 cycle L1 hit read but50-60 cycles for misslocality of accesses tocache lines important

lines

...

...

...

...

0

1

n-1

0 1 2 3 60 61 62 63

bytes

cache organized as n 64B lines

CPU L1Cache

LkCache DRAM

64KB1 cycle

1GB50-60 cycles

…...

memory hierarchy

Memory Fence Instruction 31

allows coordinating memory between threadsfence waits until all outstanding memory reads/writes are complete

producer1 write input data2 fence3 request execution on data

consumer1 request execution on data2 fence3 read result data

Rocket Hazards 32

branch resolutionexceed capacitymismatchesCPI = 1 with hit and CPI = 3 with branch mispredict

bypassing limitations1 cycle delay between load and its useloads have address calculation that adds a cycle (versus alu ops)can have instruction right behind to fill load to use delay slot

core can continue to execute after cache miss but ...cache is non blocking and can allow multiple requests in parallelwill stall as soon as produced register is accessed

so only works for up to 31 registers which is big limitation

Pipeline CPI Examples 33


Vec Inc: Unrolling 34

replicate loop bodyamortizes loop overhead

li t0, a

lw t1, a+64*4

$LOOP: lw t2, 0(t0)

addw t2, t2, 1 // 1 cycle stall

sw t2, 0(t0)

lw t3, 4(t0)

addw t3, t3, 1 // 1 cycle stall

sw t3, 4(t0)

addw t0, t0, 8

bne t0, t1, $LOOP

4 instructions / element in limit

Vec Inc: Unrolling + Scheduling 35

avoid ld / st hazard by moving ld upachieves approximately 3 instructions / element

li t0, a

lw t1, a+64*4

$LOOP: lw t2, 0(t0)

addw t2, t2, 1 // stall

sw t2, 0(t0)

lw t3, 4(t0)

addw t3, t3, 1 // stall

sw t3, 4(t0)

addw t0, t0, 8

bne t0, t1, $LOOP

4 instructions / element

lw t0, a

lw t1, a+64*4

$LOOP: lw t2, 0(t0)

lw t3, 4(t0) // reschedule

addw t2, t2, 1

sw t2, 0(t0)

addw t3, t3, 1

sw t3, 4(t0)

addw t0, t0, 8

bne t0, t1, $LOOP

3 instructions / element

Vec Inc Unrolling Limit 36

pipeline memory operations to fully saturate memory

lw t0, a

lw t1, a+64*8

$LOOP: lw t2, 0(t0)

lw t3, 4(t0)

lw t4, 8(t0)

...

addw t2, t2, 1

addw t3, t3, 1

addw t4, t4, 1

...

sw t2, 0(t0)

sw t3, 4(t0)

sw t4, 8(t0)

...

addw t0, t0, n

bne t0, t1, $LOOP

GCC -funroll-all-loops 37

in fact gcc can unroll andschedule perfectly for thisexample

move x3,x0

li x13,64

$L2:

lw x5,0(x4)

lw x2,4(x4)

lw x19,8(x4)

lw x18,12(x4)

lw x17,16(x4)

lw x16,20(x4)

lw x15,24(x4)

lw x14,28(x4)

addw x12,x5,1

addw x11,x2,1

addw x10,x19,1

addw x9,x18,1

addw x8,x17,1

addw x7,x16,1

addw x6,x15,1

addw x5,x14,1

addw x2,x3,8

sw x12,0(x4)

sw x11,4(x4)

sw x10,8(x4)

sw x9,12(x4)

sw x8,16(x4)

sw x7,20(x4)

sw x6,24(x4)

sw x5,28(x4)

move x3,x2

add x4,x4,32

bne x2,x13,$L2

History of Coprocessors and Accelerators 38

Reasonssplit functionality that wouldn’t fit on chipoff load computation

Examplesx87 floating point coprocessorMIPS coprocessor interfaceAXI SOC coprocessor interface

Accelerator Metrics 39

efficiencypowerlatencythroughputbottlenecks?

programmabilitysharing datacoordinationhazardslanguage / compiler friendliness

RISC-V Rocket / Accelerator Interface 40

decoupled interfaces2 src regs + 1 dst regstalls on dst reg accessmcmd is load, store, ...mtype is 1,2,4,8 bytesloads + stores taggedctrl is busy and error

Rocket Pipeline with Coprocessor 41

latency 5-6 cycles min

I$

+4

decode

RF

BTB

D$

pcgen fetch decode execute mem commit

Coprocessor

opReq

opResp

memResp

memReq

busyerror

Rocket Coprocessor Coordination 42

coordinatinginput <= 2 scalars to coprocessorinput data to coprocessoroutput data from coprocessoroutput scalar from coprocessor

techniquesmemory fencesstall on reading dst register

Programming Template for Memory Result 43

Rocket Corewrite input vec datafence

coprocessor instructionfence

use result data

Coprocessor.........executes + writes mem...

Programming Vec Inc 44

Rocket Corewrite vec data in aset x1 = a, x2 = 64fence

vecinc x1, x2fence

...

...result data in x1

Coprocessor............busy = truevec inc writing x1 databusy = false...

Vec Sum ( Scalar Result ) 45

int sum = 0;

int a[64];

for (int i = 0; i < 64; i++)

sum += a[i];

Programming Template for Scalar Result 46

Rocket Corewrite input vec datafence

coprocessor instructionuse res value......

Coprocessor............exec + store res in reg...

Programming Vec Sum 47

Rocket Corewrite vec dataset vec x1 = a, x2 = 64fence

vecsum x1, x2, x3use x3 stalls...use x3 completes

Coprocessor............vec sumx3 = sum...

GCC Programming VecInc 48

int* vec = { 33, 17, ... };

int n = 64;

// vecinc opcode = 0

asm volatile // don’t move

("fence; custom0 0, %0, %1, 0; fence",

: // destination

: "r"(vec), "r"(n) // sources

: "memory"); // clobbers

for (int i = 0; i < n; i++)

printf("elt[%d] = %d\n", i, vec[i]);

GCC Programming VecSum 49

int sum;

int* vec = { 33, 17, ... };

int n = 64;

// vecsum opcode = 1

asm volatile

("fence; custom0 %0, %1, %2, 1",

: "=r"(sum)

: "r"(vec), "r"(n)

: "memory");

printf("sum = %d\n", sum);

Decoupled Interfaces in Chisel 50

class DecoupledIO[T <: Data](data: T)

extends Bundle {

val ready = Bool(INPUT)

val valid = Bool(OUTPUT)

val bits = data.clone.asOutput

}

object Decoupled {

def apply(data: Data) =

new DecoupledIO(data)

}

val results =

Decoupled(UInt(width = 64))

Bool

Bool

T

ready

valid

bits

Ready / Valid 51

ConsumerProducerBool

Bool

T

ready

valid

bits

Bool

Bool

T

ready

valid

bits

Decoupled(T) Decoupled(T).flip

Ready / Valid + Queue 52

QueueProducerBool

Bool

T

ready

valid

bits

Bool

Bool

T

ready

valid

bits

ConsumerBool

Bool

T

ready

valid

bits

Bool

Bool

T

ready

valid

bits

Ready / Valid Transfer 53

Clock

Valid

Ready

transfer

Two Ready / Valid Transfers 54

Clock

Valid

Ready

transfer transfer

No Ready / Valid Transfer Both Low 55

Clock

Valid

Ready

No Ready / Valid Transfer Valid Low 56

Clock

Valid

Ready

No Ready / Valid Transfer Ready Low 57

Clock

Valid

Ready

No Ready / Valid Transfer Out of Phase 58

Clock

Valid

Ready

No Ready / Valid Combinational Loops 59

Bool

Bool

T

ready

valid

bits

Bool

Bool

T

ready

valid

bits

CombinationalLogic

CombinationalLogic

Producer Consumer

How do We Avoid Combinational Loops 60

producer is valid regardless of whether consumer is readyconsumer is ready regardless of whether producer is valid

but how do you know when to move on?

How to Update Valid from Ready 61

Bool

Bool

T

ready

valid

bits

Bool

Bool

T

ready

valid

bits

f

Producer Consumer

g f g

Using Decoupled Interfaces in Chisel 62

producer

val results =

Decoupled(UInt(width = 64))

val result =

Reg(UInt(width = 64))

val isResult =

Reg(Bool())

...

results.valid := isResult

results.bits := result

...

when (results.ready) {

// update state inc isResult

}

consumer

val cmds =

Decoupled(UInt(width = 32)).flip

val cmd =

Reg(UInt(width = 32))

val isReady =

Reg(Bool())

...

cmds.ready := isReady

cmd := result

...

when (cmds.valid) {

// update state inc isReady

}

Rule is neverready in terms of valid orvalid in terms of ready

Accelerator Chisel Interface 63

class RoCCInstruction extends Bundle {

val funct = Bits(width = 7)

val rs2 = Bits(width = 5)

val rs1 = Bits(width = 5)

val xd = Bool()

val xs1 = Bool()

val xs2 = Bool()

val rd = Bits(width = 5)

val opcode = Bits(width = 7)

}

Accelerator Chisel Interface 64

def class MemReq extends Bundle {val cmd = UInt(width = 2)val mtype = UInt(width = 2)val tag = UInt(width = 9)val addr = UInt(width = 64)val data = UInt(width = 64)

}

def class MemResp extends Bundle {val cmd = UInt(width = 2)val tag = UInt(width = 9)val mtype = UInt(width = 2)val data = UInt(width = 64)

}

def class OpReq extends Bundle {val code = new RoccInst()val a = UInt(width = 64)val b = UInt(width = 64)

}

def class OpResp extends Bundle {val idx = UInt(width = 5)val data = UInt(width = 64)

}

def class RoccIO extends Bundle {val busy = Bool(OUTPUT)val isIntr = Bool(OUTPUT)val memReq = Decoupled(new MemReq).flipval memResp = Decoupled(new MemResp)val opReq = Decoupled(new OpReq)val opResp = Decoupled(new OpResp).flip

}

opReq

opResp

memResp

memReq

busyerrorRocket Coprocessor

Accelerator Details 65

tags on read / writecommands,responses to read / writecommands,must keep busy asserteduntil all reads and writeshave completed, andmemory system has singleport with accelerator havingpriority

opReq

opResp

memResp

memReq

busyerrorRocket Coprocessor

Vec Inc Accelerator 66

two cycles per element assuming no cache missessaturate single memory op per cycleneed to pipeline this because memreq takes 4 cycle min latency

ld 0

ld 1

ld 2

ld 3

inc st 0

inc st 1

inc st 2

inc st 3

ld 4...t -->

Vec Inc Accelerator 67

use vec idx as tag

val i = Reg(init = UInt(0, 32))val v = Reg(init = UInt(0, 64))val n = Reg(init = UInt(0, 32))when (io.opRequests.valid) {val op = io.opRequests.deq()i := UInt(0)v := op.an := op.b

// is load coming back?} .elsewhen (io.memResponses.valid && io.memRequests.ready) {val resp = io.memResponses.deq()when (resp.cmd === M_LOAD) {io.memRequests.enq(memWrite(v + resp.tag, resp.bits + 1))

}// else issue more loads} .elseWhen (i < n && io.memRequests.ready) {io.memRequests.enq(memRead(v + i, i))i := i + i(1)

}

Vec Inc Accelerator Busy 68

set busy based on i and n

val i = Reg(init = UInt(0, 32))val v = Reg(init = UInt(0, 64))val n = Reg(init = UInt(0, 32))io.busy := i != nwhen (io.opRequests.valid) {val op = io.opRequests.deq()i := UInt(0)v := op.an := op.bio.busy := Bool(true)

// is load coming back?} .elsewhen (io.memResponses.valid && io.memRequests.ready) {val resp = io.memResponses.deq()when (resp.cmd === M_LOAD) {io.memRequests.enq(memWrite(v + resp.tag, resp.bits + 1))

}// else issue more loads} .elseWhen (i < n && io.memRequests.ready) {io.memRequests.enq(memRead(v + i, i))i := i + UInt(1)

}

Bigger vec size 69

What if vec is bigger than 512 max tag size?have mapping from tags to indices

manage free list but could be expensivebreak up vec into chunks

don’t run ahead until done with previous chunk

or just restrict vec ops to specific size

How could we do better? 70

can we achieve >= one element / cycle?8 bytes / cycle so could add 8/4/2 1/2/4 byte numbersfatter memory interface with banked memory?

What are goals of CPU / Coprocessor? 71

CPU sets up coprocessor (like scripting language)Coprocessor performs bigger computeRun at point of stalling in order pipeline with most workaccomplished in coprocessorSaturate memory if memory boundOverlap CPU and coprocessor

General Purpose Processor as Accelerator 72

prosMore applications work wellEasier to program (in C)

consLargePower inefficient

Out of Order Core Comparison 73

Good at soaking up ILP from C codeDatapath small portion of energy consumptionBigger consumer is all control logic and data trafficLots of dynamic dataflow control logic to reorder operationCan achieve similar sustained Incs / Cycle butLots of overhead in reg renaming, load / store unit etc

Energy Breakdown for CPU by Horowitz et al.

VLIW Comparison 74

Wide instruction with multiple ops / cycleStatically scheduled (so less energy)Still need to read / decode instructionsMight not use all ops / instructions every cycleNon determinism in memory system causes stalls

More General and Programmable Accelerator 75

Hard to Justify Vec Inc (or VecSum) Operation as AcceleratorAllow Range of Operations with Similar Form

examplesDense Linear Algebra OperationsFFT Accelerator

Vector Programming Model 76


Vector Code 77

Vector Registers

int a[64];

for (int i = 0; i < 64; i++)

a[i] += 1;

3 cycle / element in limit

li vlr, 64

lv v1, x1

addvi.w v2, v1, 1

sv v2, x1

1 or 2 cycle / element in limit

Vector Machine Improvements 78

domainsSparse MatrixStructured GridsConvolutionFFT

ideasshared infrastructurespecialized memory access patternsspecialized ALU

Acknowledgements 79

parlab and aspire slides by Krste Asanovic“Ready / Valid” based on Chris Fletcher’s CS150 Writeup which isbased on Greg Gibeling’s Writeupsome computer architecture slides by Krste Asanovic

Aspire / RISC-V / Rocket / Accelerators Lecture 02cs250/fa14/lectures/lec02.pdf · Aspire / RISC-V / Rocket / Accelerators – Lecture 02 Jonathan Bachrach EECS UC Berkeley September

Documents