SAM: Optimizing Multithreaded Cores for Speculative ... · SAM : OPTIMIZING MULTITHREADED CORES FOR SPECULATIVE PARALLELISM 2. Executive Summary Analyzes the interplay between hardware

SAM:OptimizingMultithreadedCoresforSpeculativeParallelismMALEENABEYDEERA, SUVINAY SUBRAMANIAN, MARKJEFFREY,JOEL EMER, DANIEL SANCHEZ

PACT 2017

ExecutiveSummaryAnalyzestheinterplaybetweenhardwaremultithreadingandspeculativeparallelism

(eg:ThreadLevelSpeculationandTransactionalMemory )

Conventionalmultithreadingcausesperformancepathologiesonspeculativeworkloads• Increaseinabortedwork• Inefficientuseofspeculationresources

Why?Allthreadsaretreatedequally

SpeculationAwareMultithreading(SAM)• Prioritizethreadsrunning tasksmorelikelytocommit

SAMmakesmultithreadingmoreuseful

SAM:OPTIMIZINGMULTITHREADEDCORESFORSPECULATIVEPARALLELISM 2

ExecutiveSummaryAnalyzestheinterplaybetweenhardwaremultithreadingandspeculativeparallelism

(eg:ThreadLevelSpeculationandTransactionalMemory )


Why?Allthreadsaretreatedequally

SpeculationAwareMultithreading(SAM)• Prioritizethreadsrunning tasksmorelikelytocommit



Outline

BackgroundonspeculativeparallelismPitfallsofspeculativeparallelismwithconventionalmultithreadingSAMonin-ordercoresSAMonout-of-ordercores


BackgroundonSpeculativeParallelism


ParallelizetaskswhenthedependencesarenotknowninadvanceHardwareexecutesalltasksinparallel,abortinguponconflictsWhichtasktoabort?Conflictresolutionpolicy




SpeculativeParallelism

Orderede.g.Thread-LevelSpeculation(TLS)

(Programorderdictatestheconflictresolutionorder)

Unorderede.g.HardwareTransactionalMemory

(Anyexecutionorderisvalid,buthigh-performanceconflictresolutionpoliciesdefineanorder)




Implicitorderamongalltasksinanyspeculativesystem

SpeculativeParallelism

Orderede.g.Thread-LevelSpeculation(TLS)

(Programorderdictatestheconflictresolutionorder)

Unorderede.g.HardwareTransactionalMemory

(Anyexecutionorderisvalid,buthigh-performanceconflictresolutionpoliciesdefineanorder)

BaselineSystem- Swarm[Jeffrey,MICRO’15]


void desTask(Timestamp ts , GateInput* input) {Gate* g = input ->gate ();bool toggledOutput = g.simulateToggle(input); if ( toggledOutput ) {

for (GateInput* i : g-> connectedInputs ()) {swarm::enqueue(desTask , ts+delay(g,i), i);

}}

}





}}

} Taskscreatechildrentasks(functionptr,timestamp,args)

Timestampedtasks


Tasksappeartoexecuteintimestamporder

Unorderedexecutionviaequaltimestamps




}}

} Taskscreatechildrentasks(functionptr,timestamp,args)

Timestampedtasks

SwarmMicroarchitecture


Equaltimestamps:globalorderviaVirtualTime(VT)

Timestamp Tiebreaker

Virtual Time



Mem / IO

Mem

/ IO

Mem / IO

Mem

/ IO

16-tile, 64-core CMP Tile Organization

Core Core Core Core

L1I/D L1I/D L1I/D L1I/D

L2

L3 SliceRouter

Task Unit

Tile



Virtual Time



Mem / IO

Mem

/ IO

Mem / IO

Mem

/ IO

16-tile, 64-core CMP Tile Organization

Core Core Core Core

L1I/D L1I/D L1I/D L1I/D

L2

L3 SliceRouter

Task Unit

Tile


Tasksexecuteout-of-order,butcommitinVTorder


Virtual Time

Commitqueue:stateoftaskswaitingtocommit

Outline



PitfallsofSpeculation-ObliviousMultithreading


Systemconfiguration:64-coreSMTsystemIn-ordercorewith2-wideissueSpeculation-oblivious round-robin order






Insights:1.Multithreadingcanbehighlybeneficial


Micro-opsissuedfromcommitted tasks

Noreadymicro-opstoissue




However,multithreadingcanalsoleadto:2.Increasedaborts




Micro-opsissuedfromabortedtasks




However,multithreadingcanalsoleadto:2.Increasedaborts3.Inefficientuseofspeculationresources





Resourcestalls




However,multithreadingcanalsoleadto:2.Increasedaborts3.Inefficientuseofspeculationresources

Unlikely-to-committaskshurtthethroughputoflikely-to-commitones





Resourcestalls

Speculation-AwareMultithreading


Prioritizethreadsaccordingtotheirconflictresolutionpriorities

ReduceSpeculationResourceStalls(taskscommitearly)

ReduceAborts(focusresourcesontaskslikelytocommit)

Outline



SAMonin-ordercores


SMTIssue

Fetch Decode leslesRegisterFiles

Pipe 0Pipe 1

Int ALUFP ALU

Int ALUMem/DCache

Thread micro-op queues

SAMonin-ordercores


SMTIssue


Pipe 0Pipe 1

Int ALUFP ALU

Int ALUMem/DCache


SAMonin-ordercores


SMTIssue


Pipe 0Pipe 1

Int ALUFP ALU

Int ALUMem/DCache


Conflict resolutionpriority updates(Virtual Times)

Task Unit

SAMonin-ordercores


SMTIssue


Pipe 0Pipe 1

Int ALUFP ALU

Int ALUMem/DCache


SAM issue priorities(higher is better)

SortMax

Ready

52:9

52:717:195:4

Virtual Times

3

24

1

IssueThreadID

Conflict resolutionpriority updates(Virtual Times)

Task Unit

ExperimentalMethodology

BaselineSystem• Swarm+Wait-N-GoTM [Jafrietal.ASPLOS’13]conflictresolutiontechniques• Cycle-accurate,event-driven,Pin-basedsimulator• Modelsystemsupto64cores• Cores:2wideissue,upto8threadspercore

Benchmarks• Ordered:Swarm[Jeffreyetal.MICRO’15,MICRO’16]– 8benchmarks• Unordered:STAMP[Minhetal. IISWC’08]– 8benchmarks


SAMmakesmultithreadingmoreeffective


1 Thread

Ordered Benchmarks Unordered Benchmarks



8 Thread Round Robin1 Thread




8 Thread SAM8 Thread Round Robin1 Thread






8threadedcoresoutperformsinglethreadedcoresby1.85X

WithSAM,thebenefitincreasesto2.33X













WhydoesSAMhelp?


SAMmatchesRRwhentherearenopathologies

Micro-opsissued Unusedissueslots(reason)Committed Aborted Resource Notready Other

WhydoesSAMhelp?



SAMreduceswastedwork


WhydoesSAMhelp?



SAMreduceswastedwork

SAMreducesresourcestalls


Outline



SAMonout-of-ordercoresUnlikein-ordercores,prioritiesaffectpipelineefficiency• Asinglethreadcanclogcoreresources• Increasedwrongpathexecution

Despitethese,prioritizingtasksisbetter

Needforaggressiveprioritizationaffectscoredesign• Shared,notpartitionedROBs


SMTIssue

Fetch Decode


Issue Buffer

PhysicalRegFile

Pipe 0 ReorderBuffer

Pipe 1

In-flight uops (for ICount)

3 9 4 2

SAM priorities

3 4 2 1

Conflict resolutionpriority updates(from task unit)

Conflict res. priorities

2 3 2 1

SAMtradeoffs without-of-ordercores


Micro-opsissued Unusedissueslots(reason)Committed Aborted Resource Notready OtherWrongpath

Baselinepolicy- ICount(IC)

sssp – 8threads





SAMismorebeneficialwithdynamicallysharedROBsReducesaborts+resourcestalls

sssp – 8threads






Butreducedpipelineefficiency

sssp – 8threads






ButreducedpipelineefficiencyIncreaseinwrong-pathissues+not-readystalls

sssp – 8threads

AdaptiveSAMpolicy



AdaptiveSAMpolicy


HardwarecounterstotrackcyclesMicro-opsissued Unusedissueslots(reason)Committed Aborted Resource Notready OtherWrongpath

AdaptiveSAMpolicy


Aborted Resource NotreadyWrongpath

HardwarecounterstotrackcyclesMicro-opsissued Unusedissueslots(reason)Committed Aborted Resource Notready OtherWrongpath

AdaptiveSAMpolicy



Hardwarecounterstotrackcycles

+


AdaptiveSAMpolicy




Cycleslosttotasklevelspeculation

+


AdaptiveSAMpolicy




Cycleslosttotasklevelspeculation

Cycleslosttopipelineinefficiencies

+ +

>

UseSAM UseICount

True False


SAMonOoO cores(allbenchmarks)


At8threads/core:• Multithreadingimprovesperformance

oversinglethreadedcoresby1.1x

Averageoverallbenchmarks





oversinglethreadedcoresby1.1x• WithSAM,improvementrisesto1.5x






oversinglethreadedcoresby1.1x• WithSAM,improvementrisesto1.5x

Adaptivepolicyslightlyincreasesperformanceat2and4threads



Conclusion



SpeculationAwareMultithreading(SAM)Prioritizethreadsrunningtasksmorelikelytocommit


Questions?



SpeculationAwareMultithreading(SAM)Prioritizethreadsrunningtasksmorelikelytocommit


SAM: Optimizing Multithreaded Cores for Speculative ... · SAM : OPTIMIZING MULTITHREADED CORES FOR SPECULATIVE PARALLELISM 2. Executive Summary Analyzes the interplay between hardware

Documents