Instruction Level Parallelism (ILP) Limitations
Post on 01-Jul-2015
476 Views
Preview:
DESCRIPTION
Transcript
Jose P. Pinilla
EECE528: Parallel and Reconfigurable Computing
Instruction-Level Parallelism Limitations
CONTENT
I. ILP Background
II. Hardware Model
III. Study of Limitations
IV. Simultaneous Multithreading
V. ILP today
CONTENT
I. ILP Background
II. Hardware Model
III. Study of Limitations
IV. Simultaneous Multithreading
V. ILP today
I. ILP
• MIPS Example
– Hazards
• Structural
• Data
• Control
• Power5
• ILP Optimizations
– Register Renaming
– Branch/Jump Prediction
– Alias Analysis
I$
Load
Instr 1
Instr 2
Instr 3
Instr 4
I$ Reg D$ Reg
AL
U I$ Reg D$ Reg
I$ Reg D$ Reg
AL
UReg D$ Reg
I$ Reg D$ Reg
Time (clock cycles)
AL
U
AL
U
AL
U
I. ILP: MIPS
I. ILP: Hazards
1. Structural
2. Data
3. Control
I. ILP: Structural HazardsConflict over the use of resources
I$
Load
Instr 1
Instr 2
Instr 3
Instr 4
I$ Reg D$ Reg
AL
U I$ Reg D$ Reg
I$ Reg D$ RegA
LU
Reg D$ Reg
I$ Reg D$ Reg
Time (clock cycles)
AL
U
AL
U
AL
U
I. ILP: Structural HazardsConflict over the use of resources
I$
Load
Instr 1
Instr 2
Instr 3
Instr 4
I$ Reg D$ Reg
AL
U I$ Reg D$ Reg
I$ Reg D$ Reg
AL
UReg D$ Reg
I$ Reg D$ Reg
Time (clock cycles)
AL
U
AL
U
AL
U
I. ILP: Structural Hazards
I$
Load
Instr 1
Instr 2
Instr 3
Instr 4
I$ Reg D$ Reg
AL
U I$ Reg D$ Reg
I$ Reg D$ Reg
AL
U
Reg D$ Reg
I$ Reg D$ Reg
Time (clock cycles)
AL
U
AL
U
AL
U
Solutions R/W:
*On same clock cycle
On different R/W ports
I. ILP: Data Hazards
I$
Instr 1
Instr 2
Instr 3
Instr 4
Instr 5
I$ Reg D$ Reg
AL
U I$ Reg D$ Reg
I$ Reg D$ RegA
LU
Reg D$ Reg
I$ Reg D$ Reg
Time (clock cycles)
AL
U
AL
U
AL
U
I. ILP: Data Hazards
add $t0, $t1, $t2
sub $t4, $t0 ,$t3
and $t5, $t0 ,$t6
or $t7, $t0 ,$t8
xor $t9, $t0 ,$t10
I. ILP: Data Hazards
I. ILP: Data HazardsForwarding
I. ILP: Data HazardsForwarding
I. ILP: Data HazardsHardware Interlock
Allows Forwarding
I. ILP: Data HazardsStalling by compiler
I. ILP: Data HazardsStalling by compiler
The compiler could schedule a better use of that cycle. Hardware can also do it.
I. ILP: Data HazardsAvoid Stalling
I. ILP: Control Hazards
I. ILP: Control Hazards
Solutions:
Add HW to be able to compute branch on stage 2 (DECODE)
Predict Branch: To simplify hardware, predict branch as NOT TAKEN most of the times. End
of the loop will always be wrong, but then is just once
Insert instruction after branch, always gets executed. Compiler. MIPS
I. ILP: Power5 Architecture
16 different stages
I. ILP: OptimizationsInstruction window: Trace of incoming instructions to analyze for execution.
Register Renaming: On false data dependences, hardware can rename the register.
Compilers should optimize this false dependences: R2R memory model.
Branch Prediction:
Static: Always not taken, always taken. Forward/Backward taken. Branch delay slot.
Dynamic: One-level (1bit, 2bit...), Two-level and Multiple Component
Jump Prediction: Static profiling. Dynamic: Last taken, 2bit tables, return stack.
Alias Analysis: Indirect memory references. Instruction Inspection.
I. ILP: Branch Prediction
Saturated counter: Increment on branch
taken, decrement on not taken. No
Over or Under flow.
Branch correlation: Inter/Intra
Two-level: Remembers the history of the
last n occurrences of the branch and
uses one saturating counter for each of
the possible 2n history patterns.
Many more...
CONTENT
I. ILP Background
II. Hardware Model
III. Study of Limitations
IV. Simultaneous Multithreading
V. ILP today
II. HARDWARE MODEL
• Profiling Framework
– Assumptions
– Window Size
II. HW MODEL: Profiling Framework
A set of assumptions and a methodology to, experimentally, extract a parallelism
profile out of a set of benchmarks.
Program is executed completely, resulting in a trace of instructions.
Trace includes data addresses referenced, and the results of branches and jumps.
(D. Wall's 1993 study) Divides the trace in cycles of 64 instructions in flight.
The only limits on ILP in such a processor are those imposed by the actual data
flows through either registers or memory.
II. HW MODEL: Assumptions
• No limits on replicated functional units or ports to registers or memory.
• Register Renaming: Perfect, Infinite, Finite, None
• Branch Prediction: Perfect, Infinite, Finite, None
• Jump Prediction: Perfect, Infinite, Finite, None
• Memory Address Alias Analysis:
• Perfect Caches
• Unit cycle
• 2k Window size
II. HW MODEL: Register Renaming
• Perfect: Infinite number of registers to avoid false register dependencies.
• Finite: Normally 256 integer registers and 256 floating point registers used in LRU (Least Recently Used) fashion.
• No renaming: Number of registers used in the code.
II. HW MODEL: Branch Prediction
• Perfect: All branches are correctly predicted.
• 2bit predictor with infinite tables: Dynamic. A 2bit counter per branch option (2). Indexed by low-order bits of branch's address. Incremented on branch taken. Does not overflow. Branch is taken if table entry is 2 or 3. Up to 512 2bit entries.
• 2bit predictor with infinite tables: Infinite number of counters.
• Tournament-based branch predictor: 2 2bit counters competing. A 2bit selector that is decremented/incremented according to the correct prediction of the table entries.
• Profile based: Static predictions.
• No prediction: Every branch is predicted wrong.
Not in order of performance
II. HW MODEL: Jump Prediction
• Direct Jumps are known.
• Indirect jumps
– Perfect: Always performed correctly.
– Finite prediction: A table with destination addresses. The address of a jump provides the index of the table. Whenever a jump is executed, we put its address in the table. Next jump should be to address in the table.
– Infinite prediction: Infinite table entries.
• No prediction: Every jump is predicted wrong.
II. HW MODEL: Alias Analysis
• If two memory references do not refer to the same address, then they may be safely interchanged.
• Indirect memory references are previous to the instruction execution.
• No need to predict the actual values, only whether those values conflict.
• Perfect: All global and stack reference predictions are perfect, heap
• Inspection: Examine base and offset
• None: All indirect memory references conflict.
II. HW MODEL: Window Size
• The set of instructions which is examined for simultaneous execution.
• The cycle width limits the number of instructions which can be scheduled.
• A window size of 2k will look at 2048 instructions.
• Cycle width: Assume we have found 111 instructions which can be parallelized. A cycle width of 64 would limit actual parallelism to 64 in flight instructions.
II. HW MODEL
ctr: counter
gsh: gshared (global history)
CONTENT
I. ILP Background
II. Hardware Model
III. Study of Limitations
IV. Simultaneous Multithreading
V. ILP today
III. STUDY OF LIMITATIONS
• Effects of...
– Register Renaming
– Branch/Jump Prediction
– Alias Analysis
– Realizable processor
• Window Size (Discrete/Continuous)
• Results
III. LIMITATIONS: Benchmarks
III. LIMITATIONS: Register Renaming
III. LIMITATIONS: Branch Prediction
III. LIMITATIONS: Branch Prediction
III. LIMITATIONS: Alias Analysis
III. LIMITATIONS: Results
III. LIMITATIONS: Realizable Processor
• Up to 64 instruction issues per clock with no issue restrictions, or roughly 10 times the total issue width of the widest processor in 2011
• A tournament predictor with 1K entries and a 16-entry return predictor. This predictor is comparable to the best predictors in 2011; the predictor is not a primary bottleneck
• Perfect disambiguation of memory references done dynamically—this is ambitious but perhaps attainable for small window sizes (and hence small issue rates and load-store buffers) or through address aliasing prediction
• Register renaming with 64 additional integer and 64 additional FP registers, which is slightly less than the most aggressive processor in 2011
• No issue restrictions, no cache misses, unit latencies
• Variable Window Size (Power5 200, Intel Core i7 ~128)
III. LIMITATIONS: Realizable Processor
III. LIMITATIONS: Conclusions
• Plateau behavior
• Window size effect on integer programs (3 top) is not as severe. Due to loop-level parallelism.
• Designers are faced with the challenge:
– Simpler processors with larger caches and higher clock rates
Vs
– ILP with slower clock and smaller caches
• Persistent limitations:
– WAW and WAR hazards through memory
– Unnecessary dependences
– Data flow limit
CONTENT
I. ILP Background
II. Hardware Model
III. Study of Limitations
IV. Simultaneous Multithreading
V. ILP today
IV. SIMULTANEOUS MULTITHREADING
• TLP Background
– TLP approaches
– Design Challenges
• Limits of Multiple-Issue Processors
– Power
– Complexity
IV. SMT: TLP Background
• Largely independent
– Separate copy of regFile, PC and page table
• Thread could represent
– A process that is part of a parallel program consisting of multiple processes
– An independent program on its own
• Thread level parallelism occurs naturally
• It can be used to employ the functional units idle when ILP is insufficient
IV. SMT: TLP Approaches
IV. SMT: Changes
• Increasing the associativity of the L1 instruction cache and the instruction address translation buffers
• Adding per-thread load and store queues
• Increasing the size of the L2 and L3 caches
• Adding separate instruction prefetch and buffering
• Increasing the number of virtual registers from 152 to 240
• Increasing the size of several issue queues
IV. SMT: Results
IV. SMT: Results
• SMT reduces energy by 7%
• “Because of the costs and diminishing returns in performance, however, rather than implement wider superscalars and more aggressive versions of SMT, many designers are opting to implement multiple CPU cores on a single die with slightly less aggressive support for multiple issue and multithreading; we return to this topic in the next chapter.” - Hennessy et al.
CONTENT
I. ILP Background
II. Hardware Model
III. Study of Limitations
IV. Simultaneous Multithreading
V. ILP today
V. ILP TODAY: x86
• Instruction fetch—The processor uses a multilevel branch target buffer to achieve a balance between speed and prediction accuracy. There is also a return address stack to speed up function return. Mispredictions cause a penalty of about 17 cycles. Using the predicted address, the instruction fetch unit fetches 16 bytes from the instruction cache.
• Micro-code and Macro-code
• Total pipeline depth is 14 stages
• 128 reorder (renaming) buffer size
V. ILP TODAY: x86
• Hyper-Threading:
– SMT
– The processor may stall due to a cache miss, branch misprediction, or data dependency.
– Branch misprediction costs 17 cycles
V. ILP TODAY: x86
V. ILP TODAY: ARM
- The average CPI for the ARM7 family is about 1.9 cycles per instruction.
- The average CPI for the ARM9 family is about 1.5 cycles per instruction.
- The average CPI for the ARM11 family is about 1.39 cycles per instruction.
SOURCESComputer Architecture: A Quantitative Approach. Hennessy, J.L., Patterson, D.A., Asanović, K..
5th Ed. 2012. Morgan Kaufmann/Elsevier.
Limits of instruction-level parallelism. D. W. Wall. IV international conference on Architectural
Support for Programming Languages and Operating Systems (ASPLOS), pages 176–188, 1991.
Computer Science 61C - Lecture 31: Instruction Level Parallelism. Mike Franklin, Dan Garcia.
UC Berkeley. Fall. 2011
ILP and TLP in Shared Memory Applications: A Limit Study. E. Fatehi, P. V. Gratz, Proceedings of
the 23rd international conference on Parallel architectures and compilation, pages 113-126, 2014.
MIPS Multicycle Model: Pipelining. Michael Langer. Introduction to Computer Systems. McGill
University. 2012.
IBM Power5 Chip: A Dual-Core Multithreaded Processor. R. Kalla, B. Sinharoy, J. M. Tendler. IBM.
IEEE CS. 2004.
top related