Cache Architectures for Real-Time University of Embedded

CACHES FOR

ES

UNIVE

k

me

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob

University ofMaryland

C Summer ’99

RSITY OF MARYLAND

Cache Architecturesfor Real-TimeEmbedded Systems

Prof. Bruce Jacob

Electrical & Computer EngineeringUniversity of Maryland, College Par

OUTLINE:

• Cache Primer

• Memory Management Primer

• Caches & Embedded Systems

• Cache Architectures for Real-Ti

CACHES FOR

ES

M

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

What is a Cache?

Von Neumann Bottleneck

P

PAGE 2

CACHES FOR

ES

M

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

What is a Cache?

P M

CACHE

PAGE 3

CACHES FOR

ES

CK

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache Organizations

Fundamental Unit: CACHE BLO

Purpose: HOLD DATA

PAGE 4

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache Organizations

...

A Simple Cache

PAGE 5

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache Organizations

ADDRESS

...

Cache Addressing Mechanism

DATA

PAGE 6

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache Organizations

DATA OUT

ADDRESS

...


DATA

PAGE 7

CACHES FOR

ES

s

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache Organizations

DATA OUT

ADDRESS

...


DATA

MODULO

block number

#blk

CACHE INDEX

PAGE 8

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache Organizations

...

A Simple Cache

PAGE 9

CACHES FOR

ES

.

....

s)

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache Organizations

.....

..

A More Complex Cache(similar to having several cache

PAGE 10

CACHES FOR

ES

.

....

E)E SIZE)

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache Organizations

.....

..

A Single CACHE BLOCK (or LINParameter: BLOCK SIZE (or LIN

PAGE 11

CACHES FOR

ES

.

....

nce class)

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache Organizations

.....

..

A Single CACHE SET (equivaleParameter: ASSOCIATIVITY

PAGE 12

CACHES FOR

ES

.

....

AY)

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache Organizations

.....

..

A Single CACHE COLUMN (or W

PAGE 13

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache Organizations

ADDRESS

Associative Lookup

DATA DATA

PAGE 14

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache Organizations

DATA OUT

ADDRESS

Associative Lookup

DATA DATA

PAGE 15

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache Organizations

DATA OUT

ADDRESS

The Purpose of TAGS

TAG DATA TAG DATA

(!CMP) (+CMP)

PAGE 16

CACHES FOR

ES

ive

emory)

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache OrganizationsGiven 8 cache blocks ...

DirectMapped

2-WaySet Associative

4-WaySet Associat

8-Way Set Associative(Fully Associative, or Content-Addressable M

PAGE 17

CACHES FOR

ES

DirectMapped

..

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache OrganizationsAssociativity vs. the Memory Space

n/4-WayAssoc.

n/2-WayAssoc.

FullyAssoc.

.

n = blocks in cache

PAGE 18

CACHES FOR

ESMORY

CEHES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Memory Management

TRADITIONALLY:

The MANAGEMENT ofone’s USE of PHYSICAL ME

THIS TALK’S CONTEXT:

a DIFFERENT NAMESPAused for ADDRESSING CAC

i.e. VIRTUAL ADDRESSING

PAGE 19

CACHES FOR

ES

PAGETABLE

(SOFTWARE)

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Memory ManagementWhat is a TLB?(Translation Lookaside Buffer)

APPLICATION

PAGETABLE

NAME SPACE

PHYSICALNAME SPACE

(HARDWARE)

MANY

APPLICATION

TLB

NAME SPACE

PHYSICALNAME SPACE

(HARDWARE

TODAY

CACHE)

MOONS AGO

PAGE 20

CACHES FOR

ES

ed

ffset

CHE

ffset

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache AddressingPhysically Indexed, Physically Tagg

TLB

Virtual Page Number

ASID

Page O

Page Frame Number

Tag: Page Frame Number

Cache Data

Virtual Address

CA

Cache Index

Page Frame Number Page O

Physical Address

PAGE 21

CACHES FOR

ES

d

HE

fset

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache AddressingPhysically Indexed, Virtually Tagge

CAC

Virtual Page Number Page Of

Virtual Page Number

Cache Index

Tag: Virtual Page Number

Cache Data

Virtual Address

PAGE 22

CACHES FOR

ES

d

Offset

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache AddressingVirtually Indexed, Physically Tagge

CACHETLB

Virtual Page Number

ASID

Page

Page Frame Number

Cache Index

Tag: Page Frame Number

Cache Data

Virtual Address

PAGE 23

CACHES FOR

ES

CHE

fset

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache AddressingVirtually Indexed, Virtually Tagged

CA

Virtual Page Number Page Of

Virtual Page Number

Cache Index

Tag: Virtual Page Number

Cache Data

Virtual Address

PAGE 24

CACHES FOR

ES

set

ta)E

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

A Little More DetailVirtual Page Number Page Off

PFNVPN

TLB (CAM)Cache Block (DaP-Tag

DIRECT-MAPPED CACH

Match?

Pr

ProtectionViolation?

AND

CACHE DATA

PAGE 25

CACHES FOR

ESMORY

UCTURESY

helpent

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Memory Management

TRADITIONALLY:

The MANAGEMENT ofone’s USE of PHYSICAL ME

NEW DEFINITION:

The MANAGEMENT of ALL STRassociated with MEMOR

WHAT WE WILL SEE:

VIRTUAL ADDRESSING canSIMPLIFY memory managem

PAGE 26

CACHES FOR

ES

RAM

requires

d RAMs

SRAM0

SRAM1

DRAM

IBUF

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Cache vs. Scratch-pad

MAIN DIFFERENCE: scratch-pad EXPLICIT MANAGEMENT

UNIFORMADDRESS

SPACE

Traditional Caches

NON-UNIFORMADDRESS

SPACE

Scratch-Pa

I-CACHE

D-CACHE

PAGE 27

CACHES FOR

ES

Suck

ISM

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Why Traditional Caches

NON-DETERMIN

PAGE 28

CACHES FOR

ES

or MISS?

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC01 3 34

Time

HIT

PAGE 29

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #1

Set 3:

Set 2:

Set 1:

Set 0:


0

PAGE 30

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #1

1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0 133ABABC01 3 34

0

PAGE 31

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #1

3

1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 01 33ABABC01 3 34

0

PAGE 32

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #1

3

1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 013 3ABABC01 3 34

0

PAGE 33

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #1

ASet 3:

Set 2:

Set 1:

Set 0:

Refs: 0133 ABABC01 3 34

3

10

PAGE 34

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133A BABC01 3 34

AB

10

PAGE 35

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133AB ABC01 3 34

AB

10

PAGE 36

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABA BC01 3 34

AB

10

PAGE 37

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABAB C01 3 34

AB

1C

PAGE 38

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC 01 3 34

AB

10

PAGE 39

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC0 1 3 34

AB

1C

PAGE 40

CACHES FOR

ES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #1

Set 3:

Set 2:

Set 1:

Set 0:


A3

1C

PAGE 41

CACHES FOR

ES4

IT or MISS?

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC

45645601 3 3

Time

H

PAGE 42

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133

0

ABABC

45645601 3 3

PAGE 43

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0 133

0

ABABC

45645601 3 3

PAGE 44

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

3

1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 01 33

0

ABABC

45645601 3 3

PAGE 45

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

3

1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 013 3

0

ABABC

45645601 3 3

PAGE 46

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

3

1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133

4

ABABC

45645601 3 3

PAGE 47

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

3

5

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133

4

ABABC

45645601 3 3

PAGE 48

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

3

5

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133

4

ABABC

45645601 3 3

6

PAGE 49

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

3

5

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133

4

ABABC

45645601 3 3

6

PAGE 50

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

3

5

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133

4

ABABC

45645601 3 3

6

PAGE 51

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

3

5

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133

4

ABABC

45645601 3 3

6

PAGE 52

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

3

5

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133

0

ABABC

45645601 3 3

6

PAGE 53

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

3

1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133

0

ABABC

45645601 3 3

6

PAGE 54

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #2

3

1

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133

0

ABABC

45645601 3 3

6

PAGE 55

CACHES FOR

ES4

IT or MISS?

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC

45645601 3 3

Time

H

PAGE 56

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC

45645601 3 3

0

PAGE 57

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0 133ABABC

45645601 3 3

10

PAGE 58

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 01 33ABABC

45645601 3 3

3

10

PAGE 59

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 013 3ABABC

45645601 3 3

3

10

PAGE 60

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC

45645601 3 3

3

10

A

PAGE 61

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC

45645601 3 3

3

10

BA

PAGE 62

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC

45645601 3 3

3

10

BA

PAGE 63

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABA BC

45645601 3 3

3

10

BA

PAGE 64

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABAB C

45645601 3 3

3

10

B

C

A

PAGE 65

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC

45645601 3 3

3

10

B

C

A

PAGE 66

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC

45645601 3 3

3

10

B

C

A

PAGE 67

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC

45645601 3 3

3

10

B

C

A

PAGE 68

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC

45645601 3 3

3

10

B

C

A

PAGE 69

CACHES FOR

ES4

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC

45645601 3 3

3

10

54

6

PAGE 70

CACHES FOR

ES4

HIT?or MISS?

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Example #3

Set 3:

Set 2:

Set 1:

Set 0:

Refs: 0133ABABC

45645601 3 3

3

10

–/B

5/–4/C

6/A

PAGE 71

CACHES FOR

ES

nce)

t

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Traditional Caches

Require TAGS

Soon into program execution,contents of cache are indeterminate(thus the term “hit rate” for performa

Set associativity delays problems,but only to a point

Associativity > 2 does not implemenTRUE Least-Recently-Used

PAGE 72

CACHES FOR

ES

in)

d RAMs

SRAM0

SRAM1

DRAM

IBUF

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Scratch-pad RAMs (aga

Traditional Caches Scratch-Pa

I-CACHE

D-CACHE

Require EXPLICIT MANAGEMENT

UNIFORMADDRESS

SPACE

NON-UNIFORMADDRESS

SPACE

PAGE 73

CACHES FOR

ES

ent:

rved

CTS

sES

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Scratch-Pad RAMs

No TAGS (save die area)

As long as everything fits, GREAT!

Otherwise, addressing is impedim

CONTIGUITY must be prese

DISTANCE BETWEEN OBJEmust be preserved

DSPs go one step further:

Multiply-accumulate requireTWO DISJOINT DATA SPAC

PAGE 74

CACHES FOR

ES

GONAL

ARD

ES

es)

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Scratch-Pad RAMs

Access to memory is NON-ORTHOSeparate spaces are DISJOINT

Bottom Line: COMPILATION IS H

Trend: UNIFORM ADDRESS SPAC

(i.e. more like traditional cach

PAGE 75

CACHES FOR

ES

E

sse

CACHE

che Organization

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

WHAT WE WANT

Guaranteed slow access-time

CODE

Statistically fast access-time

Guaranteed fast access-time

DATA

AddressSpace

DATA

AddressSpace

COD

DATA

AddreSpac

CACHE

No Cache Traditional Cache Ideal Ca

CODE

PAGE 76

CACHES FOR

ES

T

D LAYOUT

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

WHY IT’S DIFFICULT

DATA NAME => DATA PLACEMENMust Group Data & InstructionsSo as to Minimize Cache Conflicts

CODE

DATA

BAD LAYOUT GOO

PAGE 77

CACHES FOR

ESs ...

Mts)

easily

atter ...

...

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Data Placement

DATA SPACE

• Relatively easy to rearrange item

• ... Unless part of a LARGER ITE(cannot rearrange array elemen

CODE SPACE

• Can move FUNCTIONS around

• PORTIONS of code is another m

THERE IS A FAMILIAR SOLUTION

PAGE 78

CACHES FOR

ES

HEES

OVERLAP:ds 1-way

O-WAY

assoc.

RLAP:ds 2-wayassoc.

EE-WAYRLAP:ds 3-wayassoc.

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Solution #1

A BIG, HIGHLY ASSOCIATIVE CAC+ ability to PIN DOWN CACHE LIN

CODE

DATA

NO Nee

TW

set

OVENeeset

THROVENeeset

PAGE 79

CACHES FOR

ES

ent items

TION

ystems)

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Solution #1

• Choose items to cache,Bring each into the cache,Pin each down

• Can CACHE/NOT-CACHE adjac

• Must know CACHE ORGANIZAat COMPILE TIME(not huge issue for embedded s

• SIMPLEST, but perhapsMOST EXPENSIVE solution

PAGE 80

CACHES FOR

ES

0

MAX

NOTCACHED

CACHED

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Solution #2 (var. on #1)

Software-Managed Caches

Top bits determinememory-access behavior(CACHED/NON-CACHED)

Other possibilities:

• Physical/virtual

• Faulting/non-faulting

• Which cache ormemory structure

Enables on-the-fly decision-makingRe: memory behavior

PAGE 81

CACHES FOR

ES

/ YES; // NO

MAYBE

ENCE

0000

cache_it

;

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Application Behavior

int *array = malloc (N * sizeof int); /int *stream = malloc (N * sizeof int)int *mix = malloc (N * sizeof int); //

for (i=0; i<N; i++)x = array[i]; // CACHED REFER

stream |= MIN_NEG_INT; // 0x8000for (i=0; i<N; i++)

x = stream[i]; // NON-CACHED

for (i=0; i<N; i++) // DEPENDS ONx = (cache_it (i)) ? mix[i]

: (mix | MIN_NEG_INT)[i]

PAGE 82

CACHES FOR

ESISIONS

IZATION

CACHE

MENT ...

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Solution #2

Advantages over Solution #1:

Allows DYNAMIC CACHE DEC

LESS TIED to CACHE ORGAN

Many of the same weaknesses:

Requires BIG CACHE

Requires SET ASSOCIATIVE

Have to deal with DATA PLACE

PAGE 83

CACHES FOR

ES

T

OVERLAP:ds 1-way

O-WAY

assoc.

RLAP:ds 2-wayassoc.

EE-WAYRLAP:ds 3-wayassoc.

TE DATA

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Issue: Data Placement

DATA NAME => DATA PLACEMEN

CODE

DATA

NO Nee

TW

set

OVENeeset

THROVENeeset

TO MINIMIZE OVERLAP, RELOCA
PAGE 84

CACHES FOR

ESEMENT

n

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Issue: Data Placement

GOALS:

• Disassociate NAME and PLAC

• Fine-grained code/data relocatioat granularity of TLB pageor (better) cache line

PAGE 85

CACHES FOR

ES

D LAYOUT

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Enter Virtual Memory

Disassociates NAME from PLACE

Allows you to go from THIS:

CODE

DATA

BAD LAYOUT GOO

PAGE 86

CACHES FOR

ES

OD LAYOUT

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Enter Virtual Memory

Disassociates NAME from PLACE

... to THIS:

CODE

DATA

GOOD LAYOUT GO

TRANSLATIONADDRESS

PAGE 87

CACHES FOR

ES

ation

TCH-PAD

Page Offset

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Real-Time TLB Organiz

Works with either CACHE or SCRA

Physical Address

Page Number

Physical Page #SRAM Page #

Translation Lookaside Buffer:

PAGE withing SRAM

SEARCHONPHYS. PAGENUMBER

SRAM (Cache or Scratch-Pad RAM)

PAGE 88

CACHES FOR

ES

32 entries)

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Solution #3

Fully-Associative Real-Time TLB+ Direct-Mapped SRAM

• TLB must fully map SRAM(8KB SRAM, 256-byte page =>

• Can place ANY 256-byte pageANYWHERE in the SRAM

• Benefit: simple SRAM design

• Drawback: fully assoc. TLB

PAGE 89

CACHES FOR

ES

3
REAL-TIME
EMBEDDEDSYSTEMS

Bruce Jacob


C Summer ’99

Variations on Solution #

WANT A LARGER CACHE ?

• Larger TLB

• Larger Page Size

WANT A SMALLER TLB ?

• Smaller Cache

• Larger Page Size

WANT LESS ASSOCIATIVITY ?

• That’s a little more involved ...

PAGE 90

CACHES FOR

ES

s

DirectMapped

y.

1

1

1

1

1

1

1

1

...

n

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Set-Associative RT-TLBAssociativity vs. the Memory Space

2-WaAssoc

4-WayAssoc.

FullyAssoc.

...

n = entries in TLB

n

2

2

4

4

2

2...

...

n/2n/4

n/2-Way

n/2

n/2

Assoc.

PAGE 91

CACHES FOR

ES

s

blems

lifted

ORTH IT

ION

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Set-Associative RT-TLB

LIMITING CASE:

Direct-Mapped TLB

Direct-Mapped SRAM

Same set of data placement prowe had with NO TLB ...

EXCEPT: contiguity restriction

Bottom Line: PROBABLY NOT W

INTERMEDIATE SOLUTIONS:

Obvious Trade-Offs Exist

NEED MORE INVESTIGAT

PAGE 92

CACHES FOR

ES NFLICT)

PHYS. MEM

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Solution #4

What if SRAM Still Too Small?

(i.e. — previous solution reduces COproblems, not CAPACITY problems

Real-Time SRAM-Management

BROUGHT IN AT START

BROUGHT INON DEMAND

SRAM

PAGE 93

CACHES FOR

ES

ED

Y

TLB

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Real-Time SRAMManagement

CLASSIFY ALL CODE & DATA:

• MUST ALWAYS REMAIN CACH

• MUST NEVER BE CACHED

• EXHIBITS PERIODIC LOCALIT(i.e. loop code & data)

FOR PERIODIC ITEMS:

• Add code at beginning to set up

• Add code at end to unmap TLBand write out any dirty values

PAGE 94

CACHES FOR

ES

aceaging

nce

xecution

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Real-Time SRAMManagement

RESULTS:

• VM-style extending of SRAM spinto DRAM space via demand-p

• PROACTIVE demand-paging,not REACTIVE demand-paging

• Deterministic memory performafor all references

• Slight overhead in code size & e

PAGE 95

CACHES FOR

ESory

S:

ormanceent)

ACES

r ...

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

Summary

UNIFORM MEMORY SPACES:

• Provide Orthogonal Look at Mem

• Cache Architectures ExhibitNon-Deterministic Performance

NON-UNIFORM MEMORY SPACE

• Non-Orthogonal Memory Map

• Caches Offer Deterministic Perf(at the Price of Explicit Managem

TREND IS TOWARD UNIFORM SP

• Easier to Program & Compile fo

PAGE 96

CACHES FOR

ES

UNIVE

RES:

aches

TION

ast ...

blj/talks/

REAL-TIMEEMBEDDED

SYSTEMS

Bruce Jacob


C Summer ’99

RSITY OF MARYLAND

Summary, cont’d

REAL-TIME CACHE ARCHITECTU

• Really Big, Highly Associative C

• Software-Managed Caches

• Virtual Addressing w/ RT-TLB

• Real-Time SRAM Management

VIRTUAL MEMORY:

• Nice Programming Paradigm

• Separates NAMING from LOCA

• Like Tang ®, Not Just for Breakf

slides at http://www.ece.umd.edu/~

Cache Architectures for Real-Time University of Embedded

Documents