Slide 1 Instruction-Level Parallelism • Review of Pipelining (the laundry analogy)
Slide 1
Instruction-Level Parallelism
• Review of Pipelining (the laundry analogy)
Slide 2
Instruction-Level Parallelism
• Review of Pipelining (Appendix A)
Slide 3
Instruction-Level Parallelism
• Review of Pipelining (Appendix A) – MIPS pipelineMIPS pipeline five stages:
» IF – instruction fetch
» ID – instruction decoding and operands fetch
» EX – execution using ALU, including effective address and target address computing
» MEM – accessing memory for L & S instructions
» WB – write result back to (destination) register
Slide 4
• The “naïve” MIPS pipeline
Slide 5
• The “naïve” MIPS pipeline -- implementation
Instruction-Level Parallelism
Slide 6
Instruction-Level Parallelism• A series of datapaths shifted in time
Slide 7
Instruction-Level Parallelism• A pipeline showing the pipeline registers between stages
Slide 8
The major hurdles of pipelining: pipeline hazards• Structural HazardsStructural Hazards: resource conflicts, such as bus,
register file ports, memory ports, etc.
Slide 9
The major hurdles of pipelining: pipeline hazards• Data HazardsData Hazards: data dependency (producer-consumer
relationship, or read after write). Some can be resolved by forwardingforwarding
Slide 10
The major hurdles of pipelining: pipeline hazards• Data HazardsData Hazards: data hazards detection in MIPS pipeline
Slide 11
The major hurdles of pipelining: pipeline hazards• Data HazardsData Hazards: the logic for forwarding of data in MIPS
pipeline
Slide 12
The major hurdles of pipelining: pipeline hazards• Data HazardsData Hazards: the forwarding of data in MIPS pipeline
Slide 13
The major hurdles of pipelining: pipeline hazards• Data HazardsData Hazards: Some cannot be resolved by forwarding, forwarding,
thus requiring stalls
Slide 14
The major hurdles of pipelining: pipeline hazards• Data HazardsData Hazards: Avoid non-forwardable data hazards
through compiler scheduling:
Slide 15
The major hurdles of pipelining: pipeline hazards• Branch (Control) HazardsBranch (Control) Hazards: can cause greater
performance loss (e.g., a 3-cycle loss in the “naïve” MIPS pipeline)
Slide 16
The major hurdles of pipelining: pipeline hazards• Branch (Control) HazardsBranch (Control) Hazards: improved MIPS pipelined with
one-cycle loss
Slide 17
The major hurdles of pipelining: pipeline hazards• Reducing branch penaltiesReducing branch penalties:
1. Freeze or Flussh
2. Predict-not-taken or Predict-taken
3. Delayed Branch1) Branch instruction
2) Sequential successor
3) Branch target if taken
“Canceling/nullifying”
Branch if prediction
incorrect
Slide 18
The major hurdles of pipelining: pipeline hazards• Scheduling the branch delay slotScheduling the branch delay slot:
Slide 19
Performance of Pipelining• Example 1Example 1:
– Consider an unpipelined machine A and a pipelined machine B where CCTA = 10ns, CPI(A)ALU = CPI(A)Br = 4, CPI(A)l/s = 5, CCTB = 11ns. Assuming an instruction mix of 40% for ALU, 20% for branches, and 40% for l/s, what is the speedup of B over A under ideal conditions?
Slide 20
Performance of Pipelining• Impacts of pipeline hazardsImpacts of pipeline hazards:
Slide 21
Performance of Pipelining• Performance of branch schemesPerformance of branch schemes:
Overall costs of a variety of branch schemes with the MIPS pipeline
Slide 22
Performance of Pipelining• Example 2Example 2: For a deeper pipeline such as that in a MIPS R4000, it takes three pipeline stages
before the target-address is known and an additional stage before the condition is evaluated. This leads to the branch penalties for the three simplest branch schemes listed below:
Find the effective addition to the CPI arising from branches for this pipeline, assuming that unconditional, untaken conditional, and taken conditional branches account for 4%, 6%, and 10%, respectively.Answer:Answer:
Slide 23
What Makes Pipelining Hard to Implement?1. Exceptional conditions (e.g., interrupts, etc) often change the order of
instruction execution;
Slide 24
What Makes Pipelining Hard to Implement?• Actions needed for different types of exceptional conditions:
Slide 25
What Makes Pipelining Hard to Implement?
• Stopping and Restarting Execution: Two Challenges
Slide 26
What Makes Pipelining Hard to Implement?
• Stopping and Restarting Execution: Two Challenges (cont’d)
Slide 27
What Makes Pipelining Hard to Implement?• Precise Exception Handling in MIPS
Pipeline Stage
Problem exceptions occurring
IF Page fault on instruction fetch; misaligned memory access; memory protection violation
ID Undefined or illegal opcode
EX Arithmetic exception
MEM Page fault on data fetch; misaligned memory access; memory-protection violation
WB None
Slide 28
What Makes Pipelining Hard to Implement?• Precise Exception Handling in MIPS
Slide 29
Extending MIPS Pipeline to Handle Multicycle Operations• Handle floating point operations: single cycle (CPI=1) very long CCT
or highly complex logic circuit
• Multiple cycle long latency: with EX cycle repeated many times and/or with multiple PF function units
The MIPS pipeline with three additional unpipelined, floating point units
Slide 30
Extending MIPS Pipeline to Handle Multicycle Operations• Pipelining FP functional units:
• LatencyLatency: number of intervening cycles between the producer and the consumer of an operand -- 0 for ALU and 1 for LW
• Initiation intervalInitiation interval: number of minimum cycles between two issues of instructions using the same functional unit.
F. Unit Int. ALU Data Mem FP Add Multiply Divide
Latency 0 1 3 6 24Init. Interval 1 1 1 1 25
Slide 31
Extending MIPS Pipeline to Handle Multicycle Operations• Pipeline timing of a set of independent FP instructions:
• A typical FP code sequence showing the stalls arising from RAW hazards:
• Three instructions want to perform a write back to the FP register simultaneously
MUL.D IF ID M1 M2 M3 M4 M5 M6 M7M7 MEM WB
ADD.D IF ID A1 A2 A3 A4A4 MEM WB
L.D IF ID EX MEMMEM WB
S.D IF ID EX MEM WB
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
L.D. F4,0(R2) IF ID EX MM WB
MUL.D F0,F4,F6 IF ID Stall M1 M2 M3 M4 M5 M6 M7 MM WB
ADD.D F2,F0,F8 IF St’l ID St’l St’l St’l St’l St’l St’l A1 A2 A3 A4 MM WB
S.D. F2,0(R2) IF St’l St’l St’l St’l St’l St’l ID EX St’l St’l St’l MM
1 2 3 4 5 6 7 8 99 10 11
MUL.D F0, F4, F6 IF ID M1 M2 M3 M4 M5 M6 M7 MEM WBWB
… IF ID EX MEM WB
… IF ID EX MEM WB
ADD.D F2, F4, F6 IF ID A1 A2 A3 A4 MEM WBWB
… IF ID EX MEM WB
… IF ID EX MEM WB
L.D. F2, 0(R2) IF ID EX MEM WBWB
Slide 32
Extending MIPS Pipeline to Handle Multicycle Operations
• Difficulties in exploiting ILP:Difficulties in exploiting ILP: various hazards that impose dependency among instructions, as a result:– RAW(read after write): j tries to read a source before i writes to it
– WAW(write after write): j tries to write an operand before it is written by i
– WAR(write after read): j tries to write a destination before it is read by
• Implementing pipeline in FP: hazards and forwarding in Implementing pipeline in FP: hazards and forwarding in longer latency pipelineslonger latency pipelines– Divide not fully pipelined (structural hazard)Divide not fully pipelined (structural hazard)
– Multiple Multiple writes in a cycle and arrive at WB variably,WAW and structural hazards. Would there be WAR?
– Out-of-order completion of instructions more problems for exception handling
– Higher RAW frequency and longer stalls due to longer latency
Slide 33
Extending MIPS Pipeline to Handle Multicycle Operations• Introduce Introduce interlockinterlock::
– tracking the use of write port at ID and stalling issue if detected
– use shift register for tracking issued instructions' use of write port
– stall when entering MEM:
1. can stall any of the contending instructions,
2. no need to detect conflict early when is it harder to see,
3. give priority to the unit with the longest latency,
4. can cause bottleneck stalling
– WAW occurs if LD is issued one cycle earlier and has F2 as destination (WAW with ADDD); Solution:
1. delay issuing LD until ADDD enters MEM, or,
2. stamp out result of ADD
» Hazard detection with FP pipeline:Hazard detection with FP pipeline:1. check for structural hazards: a. functional units, b. write ports
2. check for RAW hazard: source reg. in ID = dest. reg. (issued)
3. check for WAW hazard: dest reg. in ID = dest. reg. (issued)
Slide 34
Extending MIPS Pipeline to Handle Multicycle Operations• Maintain precise exception:Maintain precise exception:
– Example of out-of-order completion:» DIVF F0, F2, F3 ; exception of SUBF at end of ADDF» ADDF F10, F10, F8 ; cause imprecise exception which» SUBF F12, F12, F14 ; cannot be solved by HW/SW
– Solutions:1. Fast imprecise (tolerable in 60's & 70s, but much less so now due to pipelined FP, virtual memory,
and IEEE standard) or slow precise2. Buffering of result until all predecessors finish:
– the bigger the difference among instruction execution lengths, the more expensive to implement (e.g., large number of comparators and MUXs and large amount of buffer space)
– history file: keeps track of register values– future file: keeps newer values of registers until all predecessors are completed
3. Quasi-precise exception: keep enough information for trap-handling routine to create a precise sequence for exception:– operations in the pipeline and their PCs– software finishes all instructions issued prior to the latest completed instruction
4. Guarded issuing: issue only if it is certain that all prior instructions will complete without causing an exception – stalling to maintain precise exception
Slide 35
The MIPS R4000 Pipeline
– R4000 pipeline leads to a 2-cycle load delay
Slide 36
The MIPS R4000 Pipeline
– R4000 pipeline leads to a 3-cycle basic branch delay since the condition evaluation is performed during the EX stage
Slide 37
Dynamic Scheduling with Scoreboard
Dynamic Scheduling:Dynamic Scheduling: hardware re-arranges the instruction execution order to reduce stalls:1. handles situations where dependences are unknown or
difficult to detect at compile time, thus simplifying the compiler design;
2. increases portability of the compiled code;
3. solves problems associated with the so-called “head-of-the-queue” (HOTQ) blocking caused by “in-order issue” of earlier pipelines. Example:
4. MIPS, which is “in-order issue”, can be made to “out-of-order” execute (implying “out-of-order” completion) by splitting ID into two phases: (1) In-order Issue: check for structural hazards. (2) Read operands: wait until no data hazards, then read operands (and then execute, possibly out-of-order!). The HOTQ problem above can be solved in this new MIPS!
Slide 38
Dynamic Scheduling with Scoreboard
Slide 39
Dynamic Scheduling with Scoreboard
Slide 40
Dynamic Scheduling with Scoreboard
Slide 41
Dynamic Scheduling with Scoreboard
Slide 42
Dynamic Scheduling with Scoreboard
Slide 43
Dynamic Scheduling with Scoreboard
Slide 44
Dynamic Scheduling with Scoreboard
Slide 45
Unpipelined Processor (MIPS)
Slide 46
Pipelined Processor (MIPS)
Slide 47
The Eight-stage Pipeline of the R4000
Slide 48
A 2-cycle Load Delay of The R4000 Integer Pipeline