18-447 Computer Architecture Lecture 9: Pipelining and Related Issues Prof. Onur Mutlu Carnegie Mellon University Spring 2012, 2/15/2012
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
18-447
Computer Architecture
Lecture 9: Pipelining and Related Issues
Prof. Onur Mutlu
Carnegie Mellon University
Spring 2012, 2/15/2012
Reminder: Homeworks
Homework 3
Due Feb 27
Out
3 questions
LC-3b microcode
Adding REP MOVS to LC-3b
Pipelining
2
Reminder: Lab Assignments
Lab Assignment 2
Due Friday, Feb 17, at the end of the lab
Individual assignment
No collaboration; please respect the honor code
Lab Assignment 3
Already out
Extra credit
Early check off: 5%
Fastest three designs: 5% + prizes
3
Reminder: Extra Credit for Lab Assignment 2
Complete your normal (single-cycle) implementation first, and get it checked off in lab.
Then, implement the MIPS core using a microcoded approach similar to what we are discussing in class.
We are not specifying any particular details of the microcode format or the microarchitecture; you should be creative.
For the extra credit, the microcoded implementation should execute the same programs that your ordinary implementation does, and you should demo it by the normal lab deadline.
4
Readings for Today
Pipelining
P&H Chapter 4.5-4.8
Pipelined LC-3b Microarchitecture Handout
Optional
Hamacher et al. book, Chapter 6, “Pipelining”
5
Review: Pipelining: Basic Idea
More systematically:
Pipeline the execution of multiple instructions
Analogy: “Assembly line processing” of instructions
Idea:
Divide the instruction processing cycle into distinct “stages” of processing
Ensure there are enough hardware resources to process one instruction in each stage
Process a different instruction in each stage
Instructions consecutive in program order are processed in consecutive stages
Benefit: Increases instruction processing throughput (1/CPI)
Downside: Start thinking about this… 6
Example: Execution of Four Independent ADDs
Multi-cycle: 4 cycles per instruction
Pipelined: 4 cycles per 4 instructions (steady state)
7
Time
F D E W
F D E W
F D E W
F D E W
F D E W
F D E W
F D E W
F D E W
Time
Review: The Laundry Analogy
“place one dirty load of clothes in the washer”
“when the washer is finished, place the wet load in the dryer”
“when the dryer is finished, take out the dry load and fold”
“when folding is finished, ask your roommate (??) to put the clothes away”
8
- steps to do a load are sequentially dependent - no dependence between different loads - different steps do not share resources
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task
order
Task
order
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
Review: Pipelining Multiple Loads of Laundry
9
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task
order
Task
order
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task
order
Task
order
- latency per load is the same - throughput increased by 4
- 4 loads of laundry in parallel - no additional resources
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
Review: Pipelining Multiple Loads of Laundry: In Practice
10
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task
order
Task
order
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task
order
Task
order
the slowest step decides throughput
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
Pipelining Multiple Loads of Laundry: In Practice
11
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task
order
Task
order
A
B
A
B
Throughput restored (2 loads per hour) using 2 dryers
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task
order
Task
order
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
An Ideal Pipeline
Goal: Increase throughput with little increase in cost (hardware cost, in case of instruction processing)
Repetition of identical operations
The same operation is repeated on a large number of different inputs
Repetition of independent operations
No dependencies between repeated operations
Uniformly partitionable suboperations
Processing can be evenly divided into uniform-latency suboperations (that do not share resources)
Good examples: automobile assembly line, doing laundry
What about instruction processing pipeline?
12
Ideal Pipelining
13
combinational logic (F,D,E,M,W) T psec
BW=~(1/T)
BW=~(2/T) T/2 ps (F,D,E) T/2 ps (M,W)
BW=~(3/T) T/3 ps (F,D)
T/3 ps (E,M)
T/3 ps (M,W)
More Realistic Pipeline: Throughput
Nonpipelined version with delay T
BW = 1/(T+S) where S = latch delay
k-stage pipelined version
BWk-stage = 1 / (T/k +S )
BWmax = 1 / (1 gate delay + S )
14
T ps
T/k ps
T/k ps
More Realistic Pipeline: Cost
Nonpipelined version with combinational cost G
Cost = G+L where L = latch cost
k-stage pipelined version
Costk-stage = G + Lk
15
G gates
G/k G/k
Pipelining Instruction Processing
16
Remember: The Instruction Processing Cycle
Fetch
Decode
Evaluate Address
Fetch Operands
Execute
Store Result
17
1. Instruction fetch (IF) 2. Instruction decode and register operand fetch (ID/RF) 3. Execute/Evaluate memory address (EX/AG) 4. Memory operand fetch (MEM) 5. Store/writeback result (WB)
Remember the Single-Cycle Uarch
18
Shift left 2
PC
Instruction memory
Read address
Instruction [31– 0]
Data memory
Read data
Write data
RegistersWrite register
Write data
Read data 1
Read data 2
Read register 1
Read register 2
Instruction [15–11]
Instruction [20–16]
Instruction [25–21]
Add
ALU result
Zero
Instruction [5– 0]
MemtoReg
ALUOp
MemWrite
RegWrite
MemRead
Branch
JumpRegDst
ALUSrc
Instruction [31–26]
4
M u x
Instruction [25–0] Jump address [31– 0]
PC+4 [31–28]
Sign extend
16 32Instruction [15–0]
1
M u x
1
0
M u x
0
1
M u x
0
1
ALU control
Control
AddALU
result
M u x
0
1 0
ALU
Shift left 2
26 28
Address
PCSrc2=Br Taken
PCSrc1=Jump
ALU operation
bcond
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
T BW=~(1/T)
Dividing Into Stages
19
200ps
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instruction
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
Address
Data
memory1
ALU result
M u x
ALU
Zero
IF: Instruction fetch ID: Instruction decode/
register file read
EX: Execute/
address calculation
MEM: Memory access WB: Write back
Is this the correct partitioning? Why not 4 or 6 stages? Why not different boundaries?
100ps 200ps 200ps 100ps
RF write
ignore for now
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
Instruction Pipeline Throughput
20
Instruction
fetchReg ALU
Data
accessReg
8 nsInstruction
fetchReg ALU
Data
accessReg
8 nsInstruction
fetch
8 ns
Time
lw $1, 100($0)
lw $2, 200($0)
lw $3, 300($0)
2 4 6 8 10 12 14 16 18
2 4 6 8 10 12 14
...
Program
execution
order
(in instructions)
Instruction
fetchReg ALU
Data
accessReg
Time
lw $1, 100($0)
lw $2, 200($0)
lw $3, 300($0)
2 nsInstruction
fetchReg ALU
Data
accessReg
2 nsInstruction
fetchReg ALU
Data
accessReg
2 ns 2 ns 2 ns 2 ns 2 ns
Program
execution
order
(in instructions)
200 400 600 800 1000 1200 1400 1600 1800
200 400 600 800 1000 1200 1400
800ps
800ps
800ps
200ps 200ps 200ps 200ps 200ps
200ps
200ps
5-stage speedup is 4, not 5 as predicated by the ideal model
Enabling Pipelined Processing: Pipeline Registers
21 T
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instruction
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
Address
Data
memory1
ALU result
M u x
ALU
Zero
IF: Instruction fetch ID: Instruction decode/
register file read
EX: Execute/
address calculation
MEM: Memory access WB: Write back
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uctio
n
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALU
Zero
ID/EX
Data
memory
Address
No resource is used by more than 1 stage! IR
D
PC
F
PC
D+4
PC
E+4
nP
CM
AE
BE
Imm
E
Ao
ut M
B
M
MD
RW
A
ou
t W
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
T/k ps
T/k ps
Pipelined Operation Example
22
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uctio
n
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALU
Zero
ID/EX
Instruction fetch
lw
Address
Data
memory
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uctio
n
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALU
Zero
ID/EX MEM/WB
Instruction decode
lw
Address
Data
memory
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uctio
n
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALU
Zero
ID/EX
Instruction fetch
lw
Address
Data
memory
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uctio
n
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALU
Zero
ID/EX MEM/WB
Instruction decode
lw
Address
Data
memory
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uctio
n
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALU
Zero
ID/EX MEM/WB
Execution
lw
Address
Data
memory
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uction
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
Data
memory
1
ALU result
M u x
ALU
Zero
ID/EX MEM/WB
Memory
lw
Address
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uctio
n
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write data
Read dataData
memory
1
ALU result
M u x
ALU
Zero
ID/EX MEM/WB
Write back
lw
Write register
Address
97108/Patterson
Figure 06.15
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uction
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
Data
memory
1
ALU result
M u x
ALU
Zero
ID/EX MEM/WB
Memory
lw
Address
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uctio
n
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write data
Read dataData
memory
1
ALU result
M u x
ALU
Zero
ID/EX MEM/WB
Write back
lw
Write register
Address
97108/Patterson
Figure 06.15
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uctio
n
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0
Address
Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
Data
memory
1
ALU result
M u x
ALU
Zero
ID/EX
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
All instruction classes must follow the same path and timing through the pipeline stages. Any performance impact?
Pipelined Operation Example
23
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uctio
n
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALU
Zero
ID/EX
Instruction decode
lw $10, 20($1)
Instruction fetch
sub $11, $2, $3
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uction
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALU
Zero
ID/EX
Instruction fetch
lw $10, 20($1)
Address
Data
memory
Address
Data
memory
Clock 1
Clock 2
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uctio
n
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALU
Zero
ID/EX
Instruction decode
lw $10, 20($1)
Instruction fetch
sub $11, $2, $3
Instruction
memory
Address
4
32
0
AddAdd
result
Shift
left 2
Instr
uction
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALU
Zero
ID/EX
Instruction fetch
lw $10, 20($1)
Address
Data
memory
Address
Data
memory
Clock 1
Clock 2
Instruction
memory
Address
4
0
AddAdd
result
Shift
left 2
Instr
uctio
n
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
3216Sign
extend
Write register
Write data
Memory
lw $10, 20($1)
Read data
1
ALU result
M u x
ALU
Zero
ID/EX
Execution
sub $11, $2, $3
Instruction
memory
Address
4
0
AddAdd
result
Shift
left 2
Instr
uction
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
Write register
Write data
Read data
1
ALU result
M u x
ALU
Zero
ID/EX
Execution
lw $10, 20($1)
Instruction decode
sub $11, $2, $3
3216Sign
extend
Address
Data
memory
Data
memory
Address
Clock 3
Clock 4
Instruction
memory
Address
4
0
AddAdd
result
Shift
left 2
Instr
uctio
n
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
3216Sign
extend
Write register
Write data
Memory
lw $10, 20($1)
Read data
1
ALU result
M u x
ALU
Zero
ID/EX
Execution
sub $11, $2, $3
Instruction
memory
Address
4
0
AddAdd
result
Shift
left 2
Instr
uction
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
Write register
Write data
Read data
1
ALU result
M u x
ALU
Zero
ID/EX
Execution
lw $10, 20($1)
Instruction decode
sub $11, $2, $3
3216Sign
extend
Address
Data
memory
Data
memory
Address
Clock 3
Clock 4
Instruction
memory
Address
4
32
0
AddAdd
result
1
ALU result
Zero
Shift
left 2
Instr
uction
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
lw $10, 20($1)
Instruction
memory
Address
4
32
0
AddAdd
result
1
ALU result
Zero
Shift
left 2
Instr
uctio
n
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
sub $11, $2, $3
Memory
sub $11, $2, $3
Address
Data memory
Address
Data
memory
Clock 6
Clock 5
Instruction
memory
Address
4
32
0
AddAdd
result
1
ALU result
Zero
Shift
left 2
Instr
uction
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
lw $10, 20($1)
Instruction
memory
Address
4
32
0
AddAdd
result
1
ALU result
Zero
Shift
left 2
Instr
uctio
n
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1
Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
sub $11, $2, $3
Memory
sub $11, $2, $3
Address
Data memory
Address
Data
memory
Clock 6
Clock 5
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
Illustrating Pipeline Operation: Operation View
24
MEM
EX
ID
IF Inst4
WB
IF
MEM
IF
MEM
EX
t0 t1 t2 t3 t4 t5
ID
EX IF ID
IF ID
Inst0 ID
IF Inst1
EX
ID
IF Inst2
MEM
EX
ID
IF Inst3
WB
WB MEM
EX
WB
Illustrating Pipeline Operation: Resource View
25
I0
I0
I1
I0
I1
I2
I0
I1
I2
I3
I0
I1
I2
I3
I4
I1
I2
I3
I4
I5
I2
I3
I4
I5
I6
I3
I4
I5
I6
I7
I4
I5
I6
I7
I8
I5
I6
I7
I8
I9
I6
I7
I8
I9
I10
t0 t1 t2 t3 t4 t5 t6 t7 t8 t9 t10
IF
ID
EX
MEM
WB
Control Points in a Pipeline
26
PC
Instruction memory
Address
Instr
uction
Instruction [20– 16]
MemtoReg
ALUOp
Branch
RegDst
ALUSrc
4
16 32
Instruction [15– 0]
0
0Registers
Write register
Write data
Read data 1
Read data 2
Read register 1
Read register 2
Sign extend
M u x
1Write
data
Read
data M u x
1
ALU
control
RegWrite
MemRead
Instruction [15– 11]
6
IF/ID ID/EX EX/MEM MEM/WB
MemWrite
Address
Data memory
PCSrc
Zero
AddAdd
result
Shift
left 2
ALU
result
ALU
Zero
Add
0
1
M u x
0
1
M u x
Identical set of control points as the single-cycle datapath!!
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
Control Signals in a Pipeline
For a given instruction
same control signals as single-cycle, but
control signals required at different cycles, depending on stage
decode once using the same logic as single-cycle and buffer control signals until consumed
or carry relevant “instruction word/field” down the pipeline and decode locally within each stage (still same logic)
Which one is better?
27
Control
EX
M
WB
M
WB
WB
IF/ID ID/EX EX/MEM MEM/WB
Instruction
Pipelined Control Signals
28
PC
Instruction memory
Instr
uctio
n
Add
Instruction [20– 16]
Me
mto
Re
g
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction [15– 0]
0
0
M u x
0
1
AddAdd
result
RegistersWrite register
Write data
Read data 1
Read data 2
Read register 1
Read register 2
Sign extend
M u x
1
ALU result
Zero
Write data
Read data
M u x
1
ALU control
Shift left 2
Re
gW
rite
MemRead
Control
ALU
Instruction [15– 11]
6
EX
M
WB
M
WB
WBIF/ID
PCSrc
ID/EX
EX/MEM
MEM/WB
M u x
0
1
Me
mW
rite
Address
Data memory
Address
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
An Ideal Pipeline
Goal: Increase throughput with little increase in cost (hardware cost, in case of instruction processing)
Repetition of identical operations
The same operation is repeated on a large number of different inputs
Repetition of independent operations
No dependencies between repeated operations
Uniformly partitionable suboperations
Processing an be evenly divided into uniform-latency suboperations (that do not share resources)
Good examples: automobile assembly line, doing laundry
What about instruction processing pipeline?
29
Instruction Pipeline: Not An Ideal Pipeline Identical operations ... NOT!
different instructions do not need all stages
- Forcing different instructions to go through the same multi-function pipe
external fragmentation (some pipe stages idle for some instructions)
Uniform suboperations ... NOT!
difficult to balance the different pipeline stages
- Not all pipeline stages do the same amount of work
internal fragmentation (some pipe stages are too-fast but take the same clock cycle time)
Independent operations ... NOT!
instructions are not independent of each other - Need to detect and resolve inter-instruction dependencies to ensure the pipeline operates correctly
Pipeline is not always moving (it stalls) 30
Issues in Pipeline Design
Balancing work in pipeline stages
How many stages and what is done in each stage
Keeping the pipeline correct, moving, and full in the presence of events that disrupt pipeline flow
Handling dependences
Data
Control
Handling resource contention
Handling long-latency (multi-cycle) operations
Handling exceptions, interrupts
Advanced: Improving pipeline throughput
Minimizing stalls
31
Causes of Pipeline Stalls
Resource contention
Dependences (between instructions)
Data
Control
Long-latency (multi-cycle) operations
32
Dependences and Their Types
Also called “dependency” or much less desirably “hazard”
Dependencies dictate ordering requirements between instructions
Two types
Data dependence
Control dependence
Resource contention is sometimes called resource dependence
However, this is not fundamental to (dictated by) program semantics, so we will treat it separately
33
Handling Resource Contention
Happens when instructions in two pipeline stages need the same resource
Solution 1: Eliminate the cause of contention
Duplicate the resource or increase its throughput
E.g., use separate instruction and data memories (caches)
E.g., use multiple ports for memory structures
Solution 2: Detect the resource contention and stall one of the contending stages
Which stage do you stall?
Example: What if you had a single read and write port for the register file?
34
Data Dependences
Types of data dependences
Flow dependence (true data dependence – read after write)
Output dependence (write after write)
Anti dependence (write after read)
Which ones cause stalls in a pipelined machine?
For all of them, we need to ensure semantics of the program are correct
Flow dependences always need to be obeyed because they constitute true dependence on a value
Anti and output dependences exist due to limited number of architectural registers
They are dependence on a name, not a value
We will later see what we can do about them
35
Data Dependence Types
36
Flow dependence r3 r1 op r2 Read-after-Write r5 r3 op r4 (RAW)
Anti dependence r3 r1 op r2 Write-after-Read r1 r4 op r5 (WAR) Output-dependence r3 r1 op r2 Write-after-Write r5 r3 op r4 (WAW) r3 r6 op r7
How to Handle Data Dependences
Anti and output dependences are easier to handle
write to the destination in one stage and in program order
Flow dependences are more interesting
Four fundamental ways of handling flow dependences
Detect and stall
Detect and forward/bypass data to dependent instruction
Eliminate the dependence at the software level
No need to detect
Do something else (fine-grained multithreading)
No need to detect
Predict the needed values and execute “speculatively”
37
Interlocking
Detection of dependence between instructions in a pipelined processor to guarantee correct execution
Software based interlocking
vs.
Hardware based interlocking
MIPS acronym?
38
Approaches to Dependence Detection (I)
Scoreboarding
Each register in register file has a Valid bit associated with it
An instruction that is writing to the register resets the Valid bit
An instruction in Decode stage checks if all its source and destination registers are Valid
Yes: No need to stall… No dependence
No: Stall the instruction
Advantage:
Simple. 1 bit per register
Disadvantage:
Need to stall for all types of dependences, not only flow dep.
39
Approaches to Dependence Detection (II)
Combinational dependence check logic
Special logic that checks if any instruction in later stages is supposed to write to any source register of the instruction that is being decoded
Yes: stall the instruction/pipeline
No: no need to stall… no flow dependence
Advantage:
No need to stall on anti and output dependences
Disadvantage:
Logic is more complex than a scoreboard
Logic becomes more complex as we make the pipeline deeper and wider (superscalar)
40
We did not cover the following slides in lecture.
These are for your preparation for the next lecture.
Control Dependence
Question: What should the fetch PC be in the next cycle?
Answer: The address of the next instruction
All instructions are control dependent on previous ones. Why?
If the fetched instruction is a non-control-flow instruction:
Next Fetch PC is the address of the next-sequential instruction
Easy to determine if we know the size of the fetched instruction
If the instruction that is fetched is a control-flow instruction:
How do we determine the next Fetch PC?
In fact, how do we know whether or not the fetched instruction is a control-flow instruction?
42
Branch Types
Type Direction at fetch time
Number of possible next fetch addresses?
When is next fetch address resolved?
Conditional Unknown 2 Execution (register dependent)
Unconditional Always taken 1 Decode (PC + offset)
Call Always taken 1 Decode (PC + offset)
Return Always taken Many Execution (register dependent)
Indirect Always taken Many Execution (register dependent)
43
Different branch types can be handled differently
How to Handle Control Dependences
Critical to keep the pipeline full with correct sequence of dynamic instructions. Potential solutions:
If the instruction is a control-flow instruction:
Stall the pipeline until we know the next fetch address
Guess the next fetch address. How?
Employ delayed branching (branch delay slot)
Do something else (fine-grained multithreading)
Eliminate control-flow instructions (predicated execution)
Fetch from both possible paths (if you know the addresses of both possible paths) (multipath execution)
44
Delayed Branching (I)
Change the semantics of a branch instruction
Branch after N instructions
Branch after N cycles
Idea: Delay the execution of a branch. N instructions (delay slots) that come after the branch are always executed regardless of branch direction.
Problem: How do you find instructions to fill the delay slots?
Branch must be independent of delay slot instructions
Unconditional branch: Easier to find instructions to fill the delay slot
Conditional branch: Condition computation should not depend on instructions in delay slots difficult to fill the delay slot
45
Delayed Branching (II)
46
A
B
C
BC X
D
E
F
F E
A
A B
B C
C BC
BC
G X:
--
A
B
C
BC X
D
E
F
G X:
F E
A
A C
C BC
BC B
B G
-- G
Normal code: Timeline: Delayed branch code: Timeline:
6 cycles 5 cycles
Fancy Delayed Branching (III)
Delayed branch with squashing
In SPARC
If the branch falls through (not taken), the delay slot instruction is not executed
Why could this help?
47
A
B
C
BC X
D
E
X:
Normal code: Delayed branch code:
A
B
C
BC X
D
E
X:
NOP
Delayed branch w/ squashing:
A
B
C
BC X
D
E
X:
A
Delayed Branching (IV) Advantages:
+ Keeps the pipeline full with useful instructions assuming
1. Number of delay slots == number of instructions to keep the pipeline full before the branch resolves
2. All delay slots can be filled with useful instructions
Disadvantages:
-- Not easy to fill the delay slots (even with a 2-stage pipeline)
1. Number of delay slots increases with pipeline depth, issue width, instruction window size.
2. Number of delay slots should be variable with variable latency operations. Why?
-- Ties ISA semantics to hardware implementation
-- SPARC, MIPS, HP-PA: 1 delay slot
-- What if pipeline implementation changes with the next design?
48
Fine-Grained Multithreading
Idea: Hardware has multiple thread contexts. Each cycle, fetch engine fetches from a different thread.
By the time the fetched branch/instruction resolves, there is no need to fetch another instruction from the same thread
Branch resolution latency overlapped with execution of other threads’ instructions
+ No logic needed for handling control and
data dependences within a thread
-- Single thread performance suffers
-- Does not overlap latency if not enough
threads to cover the whole pipeline
-- Extra logic for keeping thread contexts
49
Pipelining the LC-3b
50
Pipelining the LC-3b
Let’s remember the single-bus datapath
We’ll divide it into 5 stages
Fetch
Decode/RF Access
Address Generation/Execute
Memory
Store Result
Conservative handling of data and control dependences
Stall on branch
Stall on flow dependence
51
An Example LC-3b Pipeline
53
54
55
56
57
58
Control of the LC-3b Pipeline
Three types of control signals
Datapath Control Signals
Control signals that control the operation of the datapath
Control Store Signals
Control signals (microinstructions) stored in control store to be used in pipelined datapath (can be propagated to later stages than decode)
Stall Signals
Ensure the pipeline operates correctly in the presence of dependencies
59
60
Control Store in a Pipelined Machine
61
Pipeline stall: Pipeline does not move because an operation in a stage cannot finish
Stall Signals: Ensure the pipeline operates correctly in the presence
Why could an operation in a stage not finish?
Stall Signals
62
Related Documents
