Branch Prediction Techniques Alex Ramirez (based on the work of others) UPC-Barcelona.

Branch Prediction TechniquesBranch Prediction Techniques

Alex RamirezAlex Ramirez

(based on the work of others)(based on the work of others)

UPC-BarcelonaUPC-Barcelona

Apr 18, 2023 Branch Prediction Strategies 2

Motivation (Run-time)Motivation (Run-time) Pipelined execution

A new intruction enters the pipeline every cycle...

…but still takes several cycles to execute

Control flow changes Two possible paths after a

branch is fetched Introduces pipeline

"bubbles"• Branch delay slots

Prediction offers a chance to avoid this bubbles

Problem increases with superscalar execution

FF WWMMAADD

FF WWMMAADD

FF WWMMAADD

A branch is fetched

But takes N cycles to execute

Pipeline bubble

FF WWMMAADD


Motivation (Compile-time)Motivation (Compile-time)

Many compiler optimizations depend on accurate branch prediction Trace / Superblock scheduling

Routine inline expansion

Inter-procedural register allocation

Code transformations

Code layout optimizations


Performance MetricsPerformance Metrics

Misprediction rate Mispredicted branches per executed branch

• Unfortunately the most usually found

Instructions per mispredicted branch Gives a better idea of the program behaviour

• Branches are not evenly spaced


Talk OutlineTalk Outline Static branch prediction

Simple heuristics Complex heuristics

Semi-static branch prediction Profile-based

Dynamic branch prediction Bimodal predictors Two level predictors Hybrid predictors

Multiple branch prediction Adapted branch predictors Tree-like subgraph predictors Trace predictors


Simple Static PredictorsSimple Static Predictors[J.E.Smith ISCA´81][J.E.Smith ISCA´81]

Simple heuristics Always taken Always not taken Backwards taken / Forward not taken

• Relies on the compiler to arrange the code following this assertion

Certain opcodes taken

Programmer provided hints


Simple Static Predictors (II)Simple Static Predictors (II)

20

30

40

50

60

70

80m

88ks

im gcc li

ijpe

g

perl

vort

ex

post

gres

takennot takenBTFNT


Simple Static Predictors (III)Simple Static Predictors (III)Optimized code layoutOptimized code layout

0

20

40

60

80

100m

88ks

im gcc li

ijpe

g

perl

vort

ex

post

gres

takennot takenBTFNT


Complex Static PredictorsComplex Static Predictors[Ball & Larus PLDI´93][Ball & Larus PLDI´93]

Based on: Control flow analysis of the code to determine loops Classify branches in:

• Pointer comparison: false• Avoid executing subroutine calls

– exception handlers

• Less than zero: false, Greater than zero: true– error codes less than zero

• FP equal: false• Avoid returning from a subroutine

– recursion base case

• Avoid blocks containing a store instruction• Go towards loop headers, avoid exiting loops• Favor reuse of the branch operand

Misprediction rate about 20%


Improving Static PredictorsImproving Static Predictors[Mueller & Whalley PLDI´92/95, Krall PLDI´94, Young & Smith ASPLOS´94][Mueller & Whalley PLDI´92/95, Krall PLDI´94, Young & Smith ASPLOS´94]

The compiler can lay out the code to match the static prediction

Code replicating techniques allow: Unconditional jump elimination

• Replicate code after the if-then-else convergence in both conditional paths

• Change unconditional loop edges to conditional ones Conditional branch elimination

• In some situations the branch outcome is known– Test a variable which has not been modified or has just been set

Static prediction using branch history information• Replicate if-then-else bodies to know the previous

branch outcomes


Removing Unconditional JumpsRemoving Unconditional Jumps


Removing Unconditional Jumps (II)Removing Unconditional Jumps (II)


Removing Conditional BranchesRemoving Conditional Branches

flag = 1;while (cnd1 && flag) { A; if (cnd2) { B; flag = 0; } C;}

while (cnd1) { A; if (cnd2) { B; C; break; } C;}


Static Prediction Using Branch HistoryStatic Prediction Using Branch History

If x>10

True False

If x>20

True

False

If x>10

FalseTrue

If x>20 If x>20

True

False

True

False


Semi-static PredictorsSemi-static Predictors[Mc Farling & Hennessy ISCA´86, Fisher & Freudenberger ASPLOS´92][Mc Farling & Hennessy ISCA´86, Fisher & Freudenberger ASPLOS´92]

Use profile information from previous program runs Branches tend to behave in a fixed way Branches tend to behave in the same way across

different program executions

Performance metric: How close to a perfect static predictor

• Best direction to statically predict a branch Real static predictors

• Based on a past dataset / group of datasets


Semi-static Predictors (II)Semi-static Predictors (II)

50

60

70

80

90

100m

88ks

im gcc li

ijpe

g

perl

vort

ex

post

gres

perfectBTFNT


Bimodal Branch PredictorsBimodal Branch Predictors Dynamically store information about the

branch behaviour Branches tend to behave in a fixed way Branches tend to bevave in the same way across

program execution Index a Pattern History Table using the branch

address 1 bit: branch behaves as it did last time Saturating 2 bit counter: branch behaves as it

usually does Usually used as an extension to the BTB


Bimodal Predictors & BTBBimodal Predictors & BTB

Branch Address

Tag Taken Target Pred TypeFall Through


Bimodal Branch Predictors (II)Bimodal Branch Predictors (II)

70

75

80

85

90

95

100m

88ks

im gcc li

ijpe

g

perl

vort

ex

post

gres

bimodalsemi-staticBTFNT


Two-level Branch PredictorsTwo-level Branch Predictors[Pan, So & Rahmeh ASPLOS´92, Yeh & Patt ISCA´93][Pan, So & Rahmeh ASPLOS´92, Yeh & Patt ISCA´93]

A branch outcome depends on the outcomes of previous branches

First level: Branch History Registers (BHR) Global history / Branch correlation: past executions

of all branches Self history / Private history: past executions of the

same branch Second level: Pattern History Table (PHT)

Use first level information to index a table• Possibly XOR with the branch address [McFarling ´93]

PHT: Usually saturating 2 bit counters Also private, shared or global


Gshare Two-level PredictorGshare Two-level Predictor

Branch History

Branch Address

Index

PHT


PAp Two-level PredictorPAp Two-level Predictor

Branch Address

Branch History

PHT


Two-level Branch Predictors (II)Two-level Branch Predictors (II)

85

87

89

91

93

95

97

99m

88ks

im gcc li

ijpe

g

perl

vort

ex

post

gres

Gsharebimodalsemi-static


Improving Dynamic PredictorsImproving Dynamic Predictors Combining branch

predictors [McFarling WRL'93] Use two different branch

predictors• Access both in parallel

A third table determines which prediction to use

Using value prediction to predict branch outcomes [Gonzalez & Gonzalez PACT'99]

Combine a branch predictor with a value predictor

• Predict the branch inputs

• Compute branch outcome

Variable history length [Juan Sanjeevan & Navarro ISCA'98, Stark Evers & Patt ASPLOS'98]

Record accuracy of a given history length

• Define a time frame Use the best recorded

history length


Hybrid Branch PredictorsHybrid Branch PredictorsBranch Address

Predictor A:•static

•bimodal•PA{psg}

Predictor B:•GA{psg}•GshareMeta

Predictor:•bimodal

•two-level*

* does not replicatethe history register


Hybrid Branch Predictors (II)Hybrid Branch Predictors (II)

90

92

94

96

98

100m

88ks

im gcc li

ijpe

g

perl

vort

ex

post

gres

Gshare+Bimod (12K)

Gshare (16K)

Gshare (4K)


Multiple Branch PredictorsMultiple Branch Predictors Adapt existing branch predictors

Multi-bank BTB with bimodal predictor• Access many consecutive addresses Invalid predictions after the first taken branch

Adapted two-level predictor [Yeh,Marr & Patt SC'93]

• Fast sequential accesses to the same PHT Tree-like subgraph predictors [S.Dutta & M.Franklin

MICRO'95]

Predict which of the tree-like paths to follow Next trace predictor [Jacobson, Rottenberg & Smith MICRO'97]

Designed to work in conjuntion with the trace cache


Multi-bank Branch Target BufferMulti-bank Branch Target Buffer

Target

Pred

Tag

Type

=

=

==

Branch Address

Taken!Not taken


Adapted Gshare PredictorAdapted Gshare Predictor

Branch History

Branch Address

Index

PHT

P3P2P1


Adapted (fast) Gshare PredictorAdapted (fast) Gshare Predictor

Branch History

Branch Address

Index

PHT

P3P2P1


Tree-like Subgraph Predictor (I)Tree-like Subgraph Predictor (I)Subgraph Address A0

L0

A1

L1

A2

L2

A3

L3

A4

L4

A5

L5

A6

L6

T0 T1 T2 T3

Path 0 Attributes: {{A0,A1,A2},{L0,L1,L2},T0}


Tree-like Subgraph Predictor (II)Tree-like Subgraph Predictor (II)

Tag Path AttributesSubgraph history 2-bit counters

PathSelector

To FetchUnit

Subgraph Address


Next Trace PredictorNext Trace Predictor

Hashed ID Hashed ID Hashed ID

History register

IndexGeneration Next Trace Cnt

=

HashFunction

Next Trace Cnt Tag

Correlating table

Secondary table


Bibliography


BibliographyBibliography Static branch prediction

J.E.Smith. A study of branch prediction strategies. ISCA-8, 1981.

F.Mueller and D.B.Whalley. Avoiding unconditional jumps by code replication. PLDI, 1992.

T.Ball and J.R.Larus. Branch prediction for free. PLDI, 1993.

C.Young and M.D.Smith. Improving the accuracy of static branch prediction using branch correlation. ASPLOS-6, 1994.

F.Mueller and D.B.Whalley. Avoiding conditional branches by code replication. PLDI, 1995.


BibliographyBibliography Semi-static branch prediction

S.McFarling and J.Hennessy. Reducing the cost of branches. ISCA-13, 1986.

D.E.Wall. Predicting program behavior using real or estimated profiles. PLDI, 1991.

J.A.Fisher and S.M.Freudenberger. Predicting conditional branch directions from previous runs of a program. ASPLOS-5, 1992.

A.Krall. Improving semi-static branch prediction by code replication. PLDI, 1994.

B.Calder and D.Grunwald. Reducing branch costs via branch alignment. ASPLOS-6, 1994.

B.Calder, D.Grunwald, D.Lindsay, J.Martin, M.Mozer and B.Zorn. Corpus-based static branch prediction. PLDI, 1995.


BibliographyBibliography Bimodal predictors and BTBs

J.E.Smith. A study of branch prediction strategies. ISCA-8, 1981.

J.Lee and A.Smith. Branch prediction strategies and branch target buffer design. IEEE Computer 21(7). 1984.

S.McFarling and J.Hennessy. Reducing the cost of branches. ISCA-13, 1986.

T.Yeh and Y.N.Patt. A comprehensive instruction fetch mechanism for a processor supporting speculative execution. MICRO-25, 1992.

B.Calder and D.Grunwald. Fast & accurate instruction fetch and branch prediction. ISCA-21, 1994.


BibliographyBibliography Two-level branch predictors

S.Pan, K.So and J.Rahmeh. Improving the accuracy of dynamic branch prediction using branch correlation. ASPLOS-5, 1992.

T.Yeh and Y.N.Patt. Alternative implementations of two-level adaptive branch prediction. ISCA-19, 1992.

T.Yeh and Y.N.Patt. A comparison of dynamic branch predictors that use two levels of branch history. ISCA-20, 1993.

S.McFarling. Combining branch predictors. Technical note TN-36, DEC-WRL, 1993.


BibliographyBibliography Combining branch predictors

S.McFarling. Combining branch predictors. Technical note TN-36, DEC-WRL, 1993.

P.Y.Chang, E.Hao and Y.N.Patt. Alternative Implementations of hybrid branch predictors. MICRO-28, 1995.

Dynamic history length T.Juan, S.Sanjeevan and J.Navarro. Dynamic history

length fitting: a third level of adaptivity for branch prediction. ISCA-25, 1998.

J.Stark, M.Evers and Y.N.Patt. Variable length path branch prediction. ASPLOS-8, 1998.


BibliographyBibliography Novel branch predictors

C.C.Lee, I.Chen and T.Mudge. The bi-mode branch predictor. MICRO-30 1997.

E.Sprangle, R.Chappell, M.Alsup and Y.N.Patt. The agree predictor: a mechanism for reducing negative branch history interference. ISCA-24, 1997.

J.Gonzalez and A.Gonzalez. Control-flow speculation through value prediction for superscalar processors. PACT 1999.


BibliographyBibliography Multiple branch predictors

T.Yeh, D.Marr and Y.N.Patt. Increasing instruction fetch rate via multiple branch prediction and a branch address cache. SC-7, 1993.

S.Dutta and M.Franklin. Control flow prediction with tree-like subgraphs for superscalar processors. MICRO-28, 1995.

T.Conte, K.Menezes, P.Mills and B.Patel. Optimization of instruction fetch mechanisms for high issue rates. ISCA-22, 1995.

S.Wallance and N.Bagherzadeh. Multiple branch and block prediction. 1997.


BibliographyBibliography Trace predictors

Q.Jacobson, E.Rottenberg and J.E.Smith. Path-based next trace prediction. MICRO-30, 1997.

B.Black, B.Ryclick and J.P.Shen. The block-based trace cache. ISCA-26, 1999.

Branch Prediction Techniques Alex Ramirez (based on the work of others) UPC-Barcelona.

Documents

r static prediction

mispredicted branch

loops r

fetched r

r programmer

branch predictors r

static prediction r

r control flow changes