CS 230: Computer Organization and Assembly Language

CCMMLLCCMMLL

CS 230: Computer CS 230: Computer Organization and Organization and

Assembly LanguageAssembly LanguageAviral

ShrivastavaDepartment of Computer Science and

EngineeringSchool of Computing and Informatics

Arizona State University

Slides courtesy: Prof. Yann Hang Lee, ASU, Prof. Mary Jane Irwin, PSU, Ande Carle, UCB

CCMMLLCCMMLL

AnnouncementsAnnouncements• Project 4

– MIPS Simulator– Due Nov 10, 2009

• Alternate Project

• Quiz 5– Thursday, Nov 19, 2009– Pipelining

• Finals– Tuesday, Dec 08, 2009– Please come on time (You’ll need all the time)– Open book, notes, and internet– No communication with any other human

CCMMLLCCMMLL

Benefits of PipeliningBenefits of Pipelining

• Pipeline latches: pass the status and result of the current instruction to next stage

• Comparison:

Clock

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle 10

Ifetch

lw sw

Dec/Reg Exec Mem Wr Dec/Reg Exec MemIfetchSingle- cycle inst.

Ifetch Dec/Reg Exec Mem Wr

Ifetch Dec/Reg Exec Mem Wr

Ifetch Dec/Reg Exec Mem Wr

pipelined

CCMMLLCCMMLL

Passing control w/pipe Passing control w/pipe registersregisters

• Analogy: send instruction with car on assembly line– “Install Corinthian leather interior on car 6 @ stage 3”

WB

M

EX

WB

M WB

Control

IF/ID ID/EX EX/MEM MEM/WB

Ins

tru

cti

on

RegDst

ALUOp

ALUSrc

Branch

MemRead

MemWrite

MemtoReg

RegWrite

strip off signals for

execution phase

strip off signals for write-back phase

strip off signals for memory phase

Genera-tion

CCMMLLCCMMLL

The hazards of The hazards of pipeliningpipelining

• Pipeline hazards prevent next instruction from executing during designated clock cycle

• There are 3 classes of hazards:– Structural Hazards:

• Arise from resource conflicts • HW cannot support all possible combinations of

instructions

– Data Hazards:• Occur when given instruction depends on data from an

instruction ahead of it in pipeline

– Control Hazards:• Result from branch, other instructions that change flow of

program (i.e. change PC)

CCMMLL

Structural HazardStructural Hazard

ALU

RegMem DM Reg

ALU

RegMem DM Reg

ALU

RegMem DM Reg

ALU

RegMem DM Reg

Time

ALU

RegMem DM Reg

Load

Instruction 1

Instruction 2

Instruction 3

Instruction 4

What’s the problem here?04/21/23 6CSE 420: Computer Architecture

CCMMLLCCMMLL

Structural hazardsStructural hazards

• A way to avoid structural hazards is to duplicate resources– i.e.: An ALU to perform an arithmetic operation and

an adder to increment PC

• If not all possible combinations of instructions can be executed, structural hazards occur

• Most common instances of structural hazards:– When a functional unit not fully pipelined– When some resource not duplicated enough

04/21/237 CSE 420: Computer Architecture

CCMMLL

How is it resolved?How is it resolved?

ALU

RegMem DM Reg

ALU

RegMem DM Reg

ALU

RegMem DM Reg

Time

ALU

RegMem DM Reg

Load

Instruction 1

Instruction 2

Stall

Instruction 3

Bubble Bubble Bubble Bubble Bubble

Pipeline generally stalled by inserting a “bubble” or NOP

04/21/23 8CSE 420: Computer Architecture

CCMMLL

Or alternatively…Or alternatively…

Inst. # 1 2 3 4 5 6 7 8 9 10

LOAD IF ID EX MEM WB

Inst. i+1 IF ID EX MEM WB

Inst. i+2 IF ID EX MEM WB

Inst. i+3 stall IF ID EX MEM WB

Inst. i+4 IF ID EX MEM WB

Inst. i+5 IF ID EX MEM

Inst. i+6 IF ID EX

Clock Number

• LOAD instruction “steals” an instruction fetch cycle which will cause the pipeline to stall.

• Thus, no instruction completes on clock cycle 8

04/21/23 9CSE 420: Computer Architecture

CCMMLLCCMMLL04/21/23 CSE 420: Computer Architecture10

Example: Dual-port vs. Single-Example: Dual-port vs. Single-portport

• Machine A: Dual ported memory (“Harvard Architecture”)• Machine B: Single ported memory, but its pipelined

implementation has a 1.05 times faster clock rate• Ideal CPI = 1 for both• Loads are 40% of instructions executed

SpeedUpA = Pipeline Depth/(1 + 0) x (clockunpipe/clockpipe)

= Pipeline DepthSpeedUpB = Pipeline Depth/(1 + 0.4 x 1) x (clockunpipe/(clockunpipe /

1.05) = (Pipeline Depth/1.4) x 1.05 = 0.75 x Pipeline Depth

SpeedUpA / SpeedUpB = Pipeline Depth/(0.75 x Pipeline Depth) = 1.33

• Machine A is 1.33 times faster

CCMMLL

Illustrating a data Illustrating a data hazardhazard

ALU

RegMem DM Reg

ALU

RegMem DM Reg

ALU

RegMem DM

RegMem

Time

ADD R1, R2, R3

SUB R4, R1, R5

AND R6, R1, R7

OR R8, R1, R9

XOR R10, R1, R11

ALU

RegMem

ADD instruction causes a hazard in next 3 instructions b/c register not written until after

those 3 read it.

CCMMLLCCMMLL

stall

stall

One Way to “Fix” a Data One Way to “Fix” a Data HazardHazard

Instr.

Order

add $1,

AL

UIM Reg DM Reg

sub $4,$1,$5

and $6,$7,$1

AL

UIM Reg DM Reg

AL

UIM Reg DM Reg

Fix data hazard by waiting –

stall – but impacts CPI

CCMMLLCCMMLL

stall

stall

One Way to “Fix” a Data One Way to “Fix” a Data HazardHazard

Instr.

Order

add $1,

AL

UIM Reg DM Reg

sub $4,$1,$5

and $6,$7,$1

AL

UIM Reg DM Reg

AL

UIM Reg DM Reg

Fix data hazard by waiting –

stall – but impacts CPI

CCMMLLCCMMLL

Another Way to “Fix” a Another Way to “Fix” a Data HazardData Hazard

Instr.

Order

add $1,

AL

UIM Reg DM Reg

sub $4,$1,$5

and $6,$7,$1A

LUIM Reg DM Reg

AL

UIM Reg DM Reg

Fix data hazards by

forwarding results as

soon as they are

available to where they

are needed

sw $4,4($1)

or $8,$1,$1

AL

UIM Reg DM Reg

AL

UIM Reg DM Reg

CCMMLLCCMMLL

Data Forwarding (aka Data Forwarding (aka Bypassing)Bypassing)

• Take the result from the earliest point that it exists in any of the pipeline state registers and forward it to the functional units (e.g., the ALU) that need it that cycle

• For ALU functional unit: the inputs can come from any pipeline register rather than just from ID/EX by– adding multiplexors to the inputs of the ALU

– connecting the Rd write data in EX/MEM or MEM/WB to either (or both) of the EX’s stage Rs and Rt ALU mux inputs

– adding the proper control hardware to control the new muxes

• Other functional units may need similar forwarding logic (e.g., the DM)

• With forwarding can achieve a CPI of 1 even in the presence of data dependencies

CCMMLL

When can we When can we forward?forward?

ALU

RegMem DM Reg

ALU

RegMem DM Reg

ALU

RegMem DM

RegMem

Time

ADD R1, R2, R3

SUB R4, R1, R5

AND R6, R1, R7

OR R8, R1, R9

XOR R10, R1, R11

ALU

RegMem

SUB gets info. from EX/MEM pipe register

AND gets info. from MEM/WB pipe register

OR gets info. by forwarding fromregister file

Rule of thumb: If line goes “forward” you can do forwarding. If its drawn backward, it’s physically impossible.

CCMMLLCCMMLL

Datapath with Forwarding Datapath with Forwarding HardwareHardware PCSrc

ReadAddress

InstructionMemory

Add

PC

4

Write Data

Read Addr 1

Read Addr 2

Write Addr

Register

File

Read Data 1

Read Data 2

16 32

ALU

Shiftleft 2

Add

DataMemory

Address

Write Data

ReadData

IF/ID

SignExtend

ID/EXEX/MEM

MEM/WB

Control

ALUcntrl

Branch

ForwardUnit

ID/EX.RegisterRt

ID/EX.RegisterRs

EX/MEM.RegisterRd

MEM/WB.RegisterRd

ForwardA

ForwardB

CCMMLLCCMMLL

Data Forwarding Control Data Forwarding Control ConditionsConditions

1. EX/MEM hazard: if (EX/MEM.RegWriteand (EX/MEM.RegisterRd != 0)and (EX/MEM.RegisterRd = ID/EX.RegisterRs))

ForwardA = 10if (EX/MEM.RegWriteand (EX/MEM.RegisterRd != 0)and (EX/MEM.RegisterRd = ID/EX.RegisterRt))

ForwardB = 10

Forwards the result from the previous instr. to either input of the ALU

Forwards the result from the second previous instr. to either input of the ALU

2. MEM/WB hazard:if (MEM/WB.RegWriteand (MEM/WB.RegisterRd != 0)and (MEM/WB.RegisterRd = ID/EX.RegisterRs))

ForwardA = 01if (MEM/WB.RegWriteand (MEM/WB.RegisterRd != 0)and (MEM/WB.RegisterRd = ID/EX.RegisterRt))

ForwardB = 01

CCMMLLCCMMLL

Yet Another Complication!Yet Another Complication!

Instr.

Order

add $1,$6,$2

AL

UIM Reg DM Reg

add $1,$1,$3

add $5,$1,$4

AL

UIM Reg DM Reg

AL

UIM Reg DM Reg

• Another potential data hazard can occur when there is a conflict between the result of the WB stage instruction and the MEM stage instruction – which should be forwarded?

CCMMLLCCMMLL

Corrected Data Forwarding Control Corrected Data Forwarding Control ConditionsConditions

2. MEM/WB hazard:if (MEM/WB.RegWriteand (MEM/WB.RegisterRd != 0)and (EX/MEM.RegisterRd != ID/EX.RegisterRs)and (MEM/WB.RegisterRd = ID/EX.RegisterRs))

ForwardA = 01

if (MEM/WB.RegWriteand (MEM/WB.RegisterRd != 0)and (EX/MEM.RegisterRd != ID/EX.RegisterRt)and (MEM/WB.RegisterRd = ID/EX.RegisterRt))

ForwardB = 01

CCMMLLCCMMLL

Inserting bubblesInserting bubbles

PC IF/ID

EX/MEM

ID/EX

MEM/WB

WB

M

EX

WB

M WB

Mux0

HazardDetection

Unit

Control

Insert Bubble

CCMMLLCCMMLL

Datapath with Forwarding Datapath with Forwarding HardwareHardware PCSrc

ReadAddress

InstructionMemory

Add

PC

4

Write Data

Read Addr 1

Read Addr 2

Write Addr

Register

File

Read Data 1

Read Data 2

16 32

ALU

Shiftleft 2

Add

DataMemory

Address

Write Data

ReadData

IF/ID

SignExtend

ID/EXEX/MEM

MEM/WB

Control

ALUcntrl

Branch

ForwardUnit

ID/EX.RegisterRt

ID/EX.RegisterRs

EX/MEM.RegisterRd

MEM/WB.RegisterRd

CCMMLLCCMMLL

Yoda says…Yoda says…

Death is a natural part of life. Rejoice for those around you who transform into the Force. Mourn them do not. Miss them do not