Transactional Memory - csg.csail.mit.educsg.csail.mit.edu/6.823/Lectures/L22split.pdf · Data Management Policy • Manage uncommitted (new) and committed (old) versions of data for

L22-1MIT 6.823 Spring 2020

Daniel SanchezComputer Science & Artificial Intelligence Lab

M.I.T.

Based on slides from Christos Kozyrakis

Transactional Memory

May 5, 2020

MIT 6.823 Spring 2020

Reminder: Why Multicore?

May 5, 2020

Performance

Cos

t (a

rea,

ene

rgy…

)

Cost/perf curve ofpossible core designs

L22-2

MIT 6.823 Spring 2020

Reminder: Why Multicore?

May 5, 2020

Performance

Cos

t (a

rea,

ene

rgy…

)

Cost/perf curve ofpossible core designs

High-perf,expensivecore

L22-2

MIT 6.823 Spring 2020

Reminder: Why Multicore?

May 5, 2020

Performance

Cos

t (a

rea,

ene

rgy…

)

Cost/perf curve ofpossible core designs

High-perf,expensivecore

Moderate perf,efficient core

L22-2

MIT 6.823 Spring 2020

Reminder: Why Multicore?

May 5, 2020

Performance

Cos

t (a

rea,

ene

rgy…

)

Cost/perf curve ofpossible core designs

High-perf,expensivecore

Moderate perf,efficient core

2 cores

L22-2

MIT 6.823 Spring 2020

Reminder: Why Multicore?

May 5, 2020

Performance

Cos

t (a

rea,

ene

rgy…

)

Cost/perf curve ofpossible core designs

High-perf,expensivecore

Moderate perf,efficient core

2 cores

4 cores

L22-2

MIT 6.823 Spring 2020

But Parallel Programming is HARD

• Divide algorithm into tasks

• Map tasks to threads

• Add synchronization (locks, barriers, …) to avoid data races and ensure proper task ordering

May 5, 2020 L22-3

MIT 6.823 Spring 2020

But Parallel Programming is HARD

• Divide algorithm into tasks

• Map tasks to threads

• Add synchronization (locks, barriers, …) to avoid data races and ensure proper task ordering

• Pitfalls: scalability, locality, deadlock, livelock, fairness, races, composability, portability…

May 5, 2020 L22-3

MIT 6.823 Spring 2020

Example: Hash Table

• Sequential implementation:

May 5, 2020

V lookup(K key) {

int idx = hash(key);

for (;; idx++) {

if (buckets[idx].empty)

return NOT_FOUND;

if (buckets[idx].key == key)

return buckets[idx].val;

}

}

L22-4

MIT 6.823 Spring 2020

Example: Hash Table

• Sequential implementation:

• Not thread-safe– e.g., concurrent inserts and lookups cause races

– Need synchronization

May 5, 2020

V lookup(K key) {

int idx = hash(key);

for (;; idx++) {

if (buckets[idx].empty)

return NOT_FOUND;

if (buckets[idx].key == key)

return buckets[idx].val;

}

}

L22-4

MIT 6.823 Spring 2020

Thread-Safe Hash Table with Coarse-Grain Locks

• Also add lock(mutex)/unlock(mutex) pairs to all other hash table methods (insert, remove, …)

May 5, 2020

V lookup(K key) {int idx = hash(key);V result = NOT_FOUND; lock(mutex);for (;; idx++) {if (buckets[idx].empty) break;if (buckets[idx].key == key) {

result = buckets[idx].val;break;

}}unlock(mutex);return result;

}

L22-5

MIT 6.823 Spring 2020

Thread-Safe Hash Table with Coarse-Grain Locks

• Also add lock(mutex)/unlock(mutex) pairs to all other hash table methods (insert, remove, …)

• Problem?May 5, 2020

V lookup(K key) {int idx = hash(key);V result = NOT_FOUND; lock(mutex);for (;; idx++) {if (buckets[idx].empty) break;if (buckets[idx].key == key) {

result = buckets[idx].val;break;

}}unlock(mutex);return result;

}

L22-5

MIT 6.823 Spring 2020

Thread-Safe Hash Table with Coarse-Grain Locks

• Also add lock(mutex)/unlock(mutex) pairs to all other hash table methods (insert, remove, …)

• Problem?May 5, 2020

Serializes operations to independent buckets

V lookup(K key) {int idx = hash(key);V result = NOT_FOUND; lock(mutex);for (;; idx++) {if (buckets[idx].empty) break;if (buckets[idx].key == key) {

result = buckets[idx].val;break;

}}unlock(mutex);return result;

}

L22-5

MIT 6.823 Spring 2020

V lookup(K key) {int idx = hash(key);V result = NOT_FOUND; for (;; idx++) {lock(buckets[idx].mutex);if (buckets[idx].empty) {

unlock(buckets[idx].mutex);break;

}if (buckets[idx].key == key) {

result = buckets[idx].val;unlock(buckets[idx].mutex);break;

}unlock(buckets[idx].mutex);

}return result;

}

Thread-Safe Hash Table with Fine-Grain Locks

• Per-bucket locks

May 5, 2020 L22-6

MIT 6.823 Spring 2020

V lookup(K key) {int idx = hash(key);V result = NOT_FOUND; for (;; idx++) {lock(buckets[idx].mutex);if (buckets[idx].empty) {

unlock(buckets[idx].mutex);break;

}if (buckets[idx].key == key) {

result = buckets[idx].val;unlock(buckets[idx].mutex);break;

}unlock(buckets[idx].mutex);

}return result;

}

Thread-Safe Hash Table with Fine-Grain Locks

• Per-bucket locks

• Problems?

May 5, 2020 L22-6

MIT 6.823 Spring 2020

V lookup(K key) {int idx = hash(key);V result = NOT_FOUND; for (;; idx++) {lock(buckets[idx].mutex);if (buckets[idx].empty) {

unlock(buckets[idx].mutex);break;

}if (buckets[idx].key == key) {

result = buckets[idx].val;unlock(buckets[idx].mutex);break;

}unlock(buckets[idx].mutex);

}return result;

}

Thread-Safe Hash Table with Fine-Grain Locks

• Per-bucket locks

• Problems?

May 5, 2020

Locking overheads

L22-6

MIT 6.823 Spring 2020

V lookup(K key) {int idx = hash(key);V result = NOT_FOUND; for (;; idx++) {lock(buckets[idx].mutex);if (buckets[idx].empty) {

unlock(buckets[idx].mutex);break;

}if (buckets[idx].key == key) {

result = buckets[idx].val;unlock(buckets[idx].mutex);break;

}unlock(buckets[idx].mutex);

}return result;

}

Thread-Safe Hash Table with Fine-Grain Locks

• Per-bucket locks

• Problems?

May 5, 2020

Locking overheads

Still overserializes!(e.g., concurrent readsto the same bucket)

L22-6

MIT 6.823 Spring 2020

Performance: Locks

0

0.2

0.4

0.6

0.8

1

1 2 4 8 16

Processors

Ex

ec

uti

on

Tim

e

coarse locks fine locks

0

1

2

3

4

5

1 2 4 8 16

Processors

Ex

ec

uti

on

Tim

e

coarse locks fine locks

Ba

lan

ce

d T

ree

Ha

sh

-Ta

ble

May 5, 2020 L22-7

MIT 6.823 Spring 2020

Concurrency Control

• We need to implement concurrency control to avoid races on shared data!

• Options?

May 5, 2020 L22-8

MIT 6.823 Spring 2020

Concurrency Control

• We need to implement concurrency control to avoid races on shared data!

• Options?– Stall

• Mutual exclusion: Ensure at most one processin critical section; others wait

May 5, 2020 L22-8

MIT 6.823 Spring 2020

Concurrency Control

• We need to implement concurrency control to avoid races on shared data!

• Options?– Stall

• Mutual exclusion: Ensure at most one processin critical section; others wait

– Speculate

May 5, 2020 L22-8

MIT 6.823 Spring 2020

Concurrency Control

• We need to implement concurrency control to avoid races on shared data!

• Options?– Stall

• Mutual exclusion: Ensure at most one processin critical section; others wait

– Speculate

• Guess: No conflicts will occur during the critical section

May 5, 2020 L22-8

MIT 6.823 Spring 2020

Concurrency Control

• We need to implement concurrency control to avoid races on shared data!

• Options?– Stall

• Mutual exclusion: Ensure at most one processin critical section; others wait

– Speculate

• Guess: No conflicts will occur during the critical section

• Check: Detect whether conflicting data accesses occur

May 5, 2020 L22-8

MIT 6.823 Spring 2020

Concurrency Control

• We need to implement concurrency control to avoid races on shared data!

• Options?– Stall

• Mutual exclusion: Ensure at most one processin critical section; others wait

– Speculate

• Guess: No conflicts will occur during the critical section

• Check: Detect whether conflicting data accesses occur

• Recover: If conflict occurs, roll back; otherwise commit

May 5, 2020 L22-8

MIT 6.823 Spring 2020

Transactional Memory (TM)

• Memory transaction [Lomet’77, Knight’86, Herlihy & Moss’93]

– An atomic & isolated sequence of memory accesses

– Inspired by database transactions

• Atomicity (all or nothing)

– At commit, all memory writes take effect at once

– On abort, none of the writes appear to take effect

• Isolation

– No other code can observe writes before commit

• Serializability

– Transactions seem to commit in a single serial order

– The exact order is not guaranteed

May 5, 2020 L22-9

MIT 6.823 Spring 2020

Programming with TM

• Declarative synchronization

– Programmers says what but not how

– No declaration or management of locks

• System implements synchronization

– Typically through speculation

– Performance hit only on conflicts (R-W or W-W)

May 5, 2020

void deposit(account, amount) {

lock(account.mutex);

int t = bank.get(account);

t = t + amount;

bank.put(account, t);

unlock(account.mutex);

}

void deposit(account, amount) {

atomic {

int t = bank.get(account);

t = t + amount;

bank.put(account, t);

}

}

L22-10

MIT 6.823 Spring 2020

Advantages of TM

• Easy-to-use synchronization– As easy to use as coarse-grain locks

– Programmer declares, system implements

• High performance– Performs at least as well as fine-grain locks

– Automatic read-read & fine-grain concurrency

– No tradeoff between performance & correctness

• Composability– Safe & scalable composition of software modules

(nested transactions)

May 5, 2020 L22-11

MIT 6.823 Spring 2020

Performance: Locks vs Transactions

May 5, 2020

0

0.2

0.4

0.6

0.8

1

1 2 4 8 16

Processors

Ex

ec

uti

on

Tim

e

coarse locks fine locks TCC

0

0.5

1

1.5

2

2.5

3

3.5

4

1 2 4 8 16

Processors

Ex

ec

uti

on

Tim

e

coarse locks fine locks TCC

Ba

lan

ce

d T

ree

Ha

sh

Ma

p

TCC: a HW-based TM system

[Hammond et al, ISCA’04]L22-12

MIT 6.823 Spring 2020

TM Implementation Basics

• Use speculation to provide atomicity and isolation without sacrificing concurrency

• Basic implementation requirements– Data versioning

– Conflict detection & resolution

• Implementation options– Hardware transactional memory (HTM)

– Software transactional memory (STM)

– Hybrid transactional memory

• Hardware accelerated STMs and dual-mode systems

May 5, 2020 L22-13

MIT 6.823 Spring 2020

Motivation for Hardware TM

• Single-thread software TM performance:

• Software TM suffers 2-8x slowdown over sequential– Short-term issue: demotivates parallel programming– Long-term issue: not energy-efficient

• Industry adopting Hardware TM: Intel (since Haswell), IBM (POWER8+, Blue Gene, and zSeries)

May 5, 2020

0.0

0.5

1.0

1.5

2.0

kmeans

Execu

tio

n T

ime

(n

orm

alized

to

seq

uen

tial)

0

1

2

3

4

5

6

vacation

STMwrite

STMread

STMcommit

Busy

L22-14

MIT 6.823 Spring 2020

Data Management Policy

• Manage uncommitted (new) and committed (old) versions of data for concurrent transactions

May 5, 2020 L22-15

MIT 6.823 Spring 2020

Data Management Policy

• Manage uncommitted (new) and committed (old) versions of data for concurrent transactions

1. Eager versioning (undo-log based)

May 5, 2020 L22-15

MIT 6.823 Spring 2020

Data Management Policy

• Manage uncommitted (new) and committed (old) versions of data for concurrent transactions

1. Eager versioning (undo-log based)– Update memory location directly

May 5, 2020 L22-15

MIT 6.823 Spring 2020

Data Management Policy

• Manage uncommitted (new) and committed (old) versions of data for concurrent transactions

1. Eager versioning (undo-log based)– Update memory location directly

– Maintain undo info in a log

+ Fast commits

– Slow aborts

May 5, 2020 L22-15

MIT 6.823 Spring 2020

Data Management Policy

• Manage uncommitted (new) and committed (old) versions of data for concurrent transactions

1. Eager versioning (undo-log based)– Update memory location directly

– Maintain undo info in a log

+ Fast commits

– Slow aborts

2. Lazy versioning (write-buffer based)

May 5, 2020 L22-15

MIT 6.823 Spring 2020

Data Management Policy

• Manage uncommitted (new) and committed (old) versions of data for concurrent transactions

1. Eager versioning (undo-log based)– Update memory location directly

– Maintain undo info in a log

+ Fast commits

– Slow aborts

2. Lazy versioning (write-buffer based)– Buffer data until commit in a write buffer

May 5, 2020 L22-15

MIT 6.823 Spring 2020

Data Management Policy

• Manage uncommitted (new) and committed (old) versions of data for concurrent transactions

1. Eager versioning (undo-log based)– Update memory location directly

– Maintain undo info in a log

+ Fast commits

– Slow aborts

2. Lazy versioning (write-buffer based)– Buffer data until commit in a write buffer

– Update actual memory locations at commit

+ Fast aborts

– Slow commits

May 5, 2020 L22-15

MIT 6.823 Spring 2020

Eager Versioning Illustration

May 5, 2020

Begin Xaction

Thread

X: 10 Memory

Undo

Log

L22-16

MIT 6.823 Spring 2020

Eager Versioning Illustration

May 5, 2020

Begin Xaction

Thread

X: 10 Memory

Undo

Log

Write X←15

Thread

X: 15 Memory

Undo

LogX: 10

L22-16

MIT 6.823 Spring 2020

Eager Versioning Illustration

May 5, 2020

Begin Xaction

Thread

X: 10 Memory

Undo

Log

Write X←15

Thread

X: 15 Memory

Undo

LogX: 10

Commit Xaction

Thread

X: 15 Memory

Undo

LogX: 10

L22-16

MIT 6.823 Spring 2020

Eager Versioning Illustration

May 5, 2020

Begin Xaction

Thread

X: 10 Memory

Undo

Log

Write X←15

Thread

X: 15 Memory

Undo

LogX: 10

Commit Xaction

Thread

X: 15 Memory

Undo

LogX: 10

Abort Xaction

Thread

X: 10 Memory

Undo

LogX: 10

L22-16

MIT 6.823 Spring 2020

Lazy Versioning Illustration

May 5, 2020

Begin Xaction

Thread

X: 10 Memory

Write

Buffer

L22-17

MIT 6.823 Spring 2020

Lazy Versioning Illustration

May 5, 2020

Begin Xaction

Thread

X: 10 Memory

Write

Buffer

Write X←15

Thread

X: 10 Memory

Write

BufferX: 15

L22-17

MIT 6.823 Spring 2020

Lazy Versioning Illustration

May 5, 2020

Begin Xaction

Thread

X: 10 Memory

Write

Buffer

Write X←15

Thread

X: 10 Memory

Write

BufferX: 15

Commit Xaction

Thread

X: 15 Memory

Write

BufferX: 15

L22-17

MIT 6.823 Spring 2020

Lazy Versioning Illustration

May 5, 2020

Begin Xaction

Thread

X: 10 Memory

Write

Buffer

Write X←15

Thread

X: 10 Memory

Write

BufferX: 15

Abort Xaction

Thread

X: 10 Memory

Write

BufferX: 15

Commit Xaction

Thread

X: 15 Memory

Write

BufferX: 15

L22-17

MIT 6.823 Spring 2020

Conflict Detection

• Detect and handle conflicts between transaction

– Read-Write and (often) Write-Write conflicts

– Must track the transaction’s read-set and write-set

• Read-set: addresses read within the transaction

• Write-set: addresses written within transaction

May 5, 2020 L22-18

MIT 6.823 Spring 2020

Conflict Detection

• Detect and handle conflicts between transaction

– Read-Write and (often) Write-Write conflicts

– Must track the transaction’s read-set and write-set

• Read-set: addresses read within the transaction

• Write-set: addresses written within transaction

1. Pessimistic detection

– Check for conflicts during loads or stores

• SW: SW barriers using locks and/or version numbers

• HW: check through coherence actions

– Use contention manager to decide to stall or abort

• Various priority policies to handle common case fast

May 5, 2020 L22-18

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1

Success

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1

X0 X1

Success

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1

X0 X1

Success

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1

X0 X1

rd A

Success

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1

X0 X1

rd Acheck

Success

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1

X0 X1

rd Acheck

Success

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1

X0 X1

rd A

wr B

check

check

Success

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1

X0 X1

rd A

wr B

check

check

Success

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1

X0 X1

rd A

wr B

check

check

wr Ccheck

Success

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

Early Detect

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr Acheck

Early Detect

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

Early Detect

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

Early Detect

stall

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

Abort

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

rd Acheck

Abort

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

rd A

wr A

check

check

Abort

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

rd A

wr A

check

check

Abort

restart

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

rd A

wr A

check

check

commit

Abort

restart

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

rd A

wr A

check

check

commit

commit

Abort

restart

rd Acheck

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3 Case 4

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

rd A

wr A

check

check

commit

commit

Abort

restart

rd Acheck

X0 X1

No progress

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3 Case 4

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

rd A

wr A

check

check

commit

commit

Abort

restart

rd Acheck

X0 X1

rd A

check

No progress

wr ATIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3 Case 4

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

rd A

wr A

check

check

commit

commit

Abort

restart

rd Acheck

X0 X1

rd A

check

No progress

wr A

rd Awr A

check

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3 Case 4

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

rd A

wr A

check

check

commit

commit

Abort

restart

rd Acheck

X0 X1

rd A

check

No progress

wr A

rd Awr A

check

restart

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3 Case 4

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

rd A

wr A

check

check

commit

commit

Abort

restart

rd Acheck

X0 X1

rd A

check

No progress

wr A

rd Awr A

check

restart

rd A

check

wr A

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3 Case 4

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

rd A

wr A

check

check

commit

commit

Abort

restart

rd Acheck

X0 X1

rd A

check

No progress

wr A

rd Awr A

check

restart

rd A

check

wr A

restart

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3 Case 4

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

rd A

wr A

check

check

commit

commit

Abort

restart

rd Acheck

X0 X1

rd A

check

No progress

wr A

rd Awr A

check

restart

rd A

check

wr A

restart

rd Awr A

check

TIM

E

L22-19

MIT 6.823 Spring 2020

Pessimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3 Case 4

X0 X1

rd A

wr B

check

check

wr Ccheck

commit

commit

Success

X0 X1

wr A

rd A

check

check

commit

commit

Early Detect

stall

X0 X1

rd A

wr A

check

check

commit

commit

Abort

restart

rd Acheck

X0 X1

rd A

check

No progress

wr A

rd Awr A

check

restart

rd A

check

wr A

restart

rd Awr A

check

restart

TIM

E

L22-19

MIT 6.823 Spring 2020

Conflict Detection (cont)

2. Optimistic detection– Detect conflicts when a transaction attempts to commit

– SW: validate write/read-set using locks or version numbers

– HW: validate write-set using coherence actions

• Get exclusive access for cache lines in write-set

• On a conflict, give priority to committing transaction

• Other transactions may abort later on

– On conflicts between committing transactions, use contention manager to decide priority

• Note: optimistic & pessimistic schemes together– Several STM systems are optimistic on reads, pessimistic on

writes

May 5, 2020 L22-20

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

TIM

E

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1

X0 X1

Success

TIM

E

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1

X0 X1

rd A

wr B

Success

TIM

E

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1

X0 X1

rd A

wr B

wr C

commit

Success

check

TIM

E

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1

X0 X1

rd A

wr B

wr C

commit

commit

Success

check

check

TIM

E

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

Abort

check

check

TIM

E

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

wr A

rd A

Abort

check

check

TIM

E

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

wr A

rd A

commit

Abort

check

check

check

TIM

E

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

wr A

rd A

commit

Abort

restartcheck

check

check

TIM

E

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

wr A

rd A

commit

Abort

restartcheck

check

check

rd A

TIM

E

commitcheck

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

wr A

rd A

commit

Abort

restart

X0 X1

Success

check

check

check

rd A

TIM

E

commitcheck

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

wr A

rd A

commit

Abort

restart

X0 X1

rd A

wr A

Success

check

check

check

rd A

TIM

E

commitcheck

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

wr A

rd A

commit

Abort

restart

X0 X1

rd A

wr A

Success

check

check

check

rd A

commitcheck

TIM

E

commitcheck

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

wr A

rd A

commit

Abort

restart

X0 X1

rd A

wr A

commit

Success

check

check

check

rd A

check

commitcheck

TIM

E

commitcheck

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3 Case 4

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

wr A

rd A

commit

Abort

restart

X0 X1

rd A

wr A

commit

Success

X0 X1

Forward progress

check

check

check

rd A

check

commitcheck

TIM

E

commitcheck

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3 Case 4

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

wr A

rd A

commit

Abort

restart

X0 X1

rd A

wr A

commit

Success

X0 X1

rd A

Forward progress

wr A

rd Awr A

check

check

check

rd A

check

commitcheck

TIM

E

commitcheck

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3 Case 4

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

wr A

rd A

commit

Abort

restart

X0 X1

rd A

wr A

commit

Success

X0 X1

rd A

Forward progress

wr A

rd Awr A

check

check

check

rd A

check

commitcheck

commitcheck

TIM

E

commitcheck

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3 Case 4

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

wr A

rd A

commit

Abort

restart

X0 X1

rd A

wr A

commit

Success

X0 X1

rd A

Forward progress

wr A

rd Awr A

check

check

check

rd A

check

commitcheck

commitcheck

restart

TIM

E

commitcheck

L22-21

MIT 6.823 Spring 2020

Optimistic Detection Illustration

May 5, 2020

Case 1 Case 2 Case 3 Case 4

X0 X1

rd A

wr B

wr C

commit

commit

Success

X0 X1

wr A

rd A

commit

Abort

restart

X0 X1

rd A

wr A

commit

Success

X0 X1

rd A

Forward progress

wr A

rd Awr A

check

check

check

rd A

check

commitcheck

commitcheck

restart

rd Awr A

commitcheck

TIM

E

commitcheck

L22-21

MIT 6.823 Spring 2020

Conflict Detection Tradeoffs

1. Pessimistic conflict detection

+ Detect conflicts early

• Undo less work, turn some aborts to stalls

– No forward progress guarantees, more aborts in some cases

• Requires additional techniques to guarantee forward progress

(e.g., backoff, prioritize older transactions)

– Locking issues (SW), fine-grain communication (HW)

2. Optimistic conflict detection

+ Forward progress guarantees

+ Potentially less conflicts, shorter locking (SW), bulk

communication (HW)

– Detects conflicts late, still has fairness problems

May 5, 2020 L22-22

MIT 6.823 Spring 2020

HTM Implementation Overview

• Data versioning: Use caches

– Cache the write-buffer or the undo-log

– Cache metadata to track read-set and write-set

– Can do with private, shared, and multi-level caches

May 5, 2020 L22-23

MIT 6.823 Spring 2020

HTM Implementation Overview

• Data versioning: Use caches

– Cache the write-buffer or the undo-log

– Cache metadata to track read-set and write-set

– Can do with private, shared, and multi-level caches

• Conflict detection: Use the cache coherence protocol

– Coherence lookups detect conflicts between transactions

– Works with snooping & directory coherence

May 5, 2020 L22-23

MIT 6.823 Spring 2020

HTM Implementation Overview

• Data versioning: Use caches

– Cache the write-buffer or the undo-log

– Cache metadata to track read-set and write-set

– Can do with private, shared, and multi-level caches

• Conflict detection: Use the cache coherence protocol

– Coherence lookups detect conflicts between transactions

– Works with snooping & directory coherence

• Note: On aborts, must also restore register state take register checkpoint

– OOO cores support with minimal changes(recall rename table snapshots…)

May 5, 2020 L22-23

MIT 6.823 Spring 2020

HTM Design

• Cache lines track read-set & write-set– R bit: indicates data read by transaction; set on load

– W bit: indicates data written by transaction; set on store

– R/W bits can be at word or cache-line granularity

– R/W bits gang-cleared on transaction commit or abort

• Coherence requests check R/W bits to detect conflicts – Shared request to W-word is a read-write conflict

– Exclusive request to R-word is a write-read conflict

– Exclusive request to W-word is a write-write conflict

May 5, 2020

V D E Tag R W Word 1 R W Word N...

L22-24

MIT 6.823 Spring 2020

Example HTM: Lazy Optimistic

CPU

Cache

ALUs

TM State

Tag DataV

Registers

• CPU changes

– Register checkpoint

– TM state registers (status, pointers to handlers, …)

• Cache changes

– Per-line R/W bits

• Assume a bus-based system

WR

May 5, 2020 L22-25

MIT 6.823 Spring 2020

HTM Transaction Execution

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

May 5, 2020 L22-26

MIT 6.823 Spring 2020

HTM Transaction Execution

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

• Transaction begin

• Initialize CPU & cache state

• Take register checkpoint

0 0

May 5, 2020 L22-26

MIT 6.823 Spring 2020

HTM Transaction Execution

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

0 0

May 5, 2020 L22-27

MIT 6.823 Spring 2020

HTM Transaction Execution

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

0 0

A 3311 0• Load operation

• Serve cache miss if needed

• Set line’s R-bit

May 5, 2020 L22-27

MIT 6.823 Spring 2020

HTM Transaction Execution

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

0 0

A 3311 0

May 5, 2020 L22-28

MIT 6.823 Spring 2020

HTM Transaction Execution

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

0 0

A 3311 0

B 510 1

• Store operation

• Serve cache miss if needed (if other cores have line, get it shared anyway!)

• Set line’s W-bit

May 5, 2020 L22-28

MIT 6.823 Spring 2020

HTM Transaction Execution

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

1 0

A 3311 0

B 510 1

May 5, 2020 L22-29

MIT 6.823 Spring 2020

HTM Transaction Execution

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

1 0

A 3311 0

B 510 1

May 5, 2020 L22-29

MIT 6.823 Spring 2020

HTM Transaction Execution

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

1 0

A 3311 0

B 510 1 upgradeX B

• Fast 2-phase commit:

1. Validate: Request exclusive access to write-set lines (if needed)

May 5, 2020 L22-29

MIT 6.823 Spring 2020

HTM Transaction Execution

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

1 0

A 3311 0

B 510 1

0 0

0 0

0 0

• Fast 2-phase commit:

1. Validate: Request exclusive access to write-set lines (if needed)

2. Commit: Gang-reset R&W bits, turns write-set data to valid (dirty) data

May 5, 2020 L22-29

MIT 6.823 Spring 2020

HTM Conflict Detection

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

1 0

A 3311 0

B 510 1

• Fast conflict detection & abort:

May 5, 2020 L22-30

MIT 6.823 Spring 2020

HTM Conflict Detection

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

1 0

A 3311 0

B 510 1

upgradeX D

• Fast conflict detection & abort:

– Check: Lookup exclusive requests in the read-set and write-set

May 5, 2020 L22-30

MIT 6.823 Spring 2020

HTM Conflict Detection

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

1 0

A 3311 0

B 510 1 upgradeX A

• Fast conflict detection & abort:

– Check: Lookup exclusive requests in the read-set and write-set

– Abort: Invalidate write-set, gang-reset R and W bits, restore checkpoint

May 5, 2020 L22-30

MIT 6.823 Spring 2020

HTM Conflict Detection

Xbegin

Load A

Store B 5

Load C

Xcommit

CPU

Cache

ALUs

TM State

Tag DataV

C 91

WR

Registers

1 0

A 3311 0

B 510 1 upgradeX A

11 0

01 0

00 1

0 0

0 0

0 0

• Fast conflict detection & abort:

– Check: Lookup exclusive requests in the read-set and write-set

– Abort: Invalidate write-set, gang-reset R and W bits, restore checkpoint

May 5, 2020 L22-30

MIT 6.823 Spring 2020

HTM Advantages

• Fast common-case behavior

– Zero-overhead tracking of read-set & write-set

– Zero-overhead versioning

– Fast commits & aborts without data movement

– Continuous validation of read-set

May 5, 2020 L22-31

MIT 6.823 Spring 2020

HTM Advantages

• Fast common-case behavior

– Zero-overhead tracking of read-set & write-set

– Zero-overhead versioning

– Fast commits & aborts without data movement

– Continuous validation of read-set

• Strong isolation

– Conflicts detected on non-transactional loads/stores as well

May 5, 2020 L22-31

MIT 6.823 Spring 2020

HTM Advantages

• Fast common-case behavior

– Zero-overhead tracking of read-set & write-set

– Zero-overhead versioning

– Fast commits & aborts without data movement

– Continuous validation of read-set

• Strong isolation

– Conflicts detected on non-transactional loads/stores as well

• Simplifies multi-core coherence and consistency

[Hammond’04, Ceze’07]

– Recall: Sequential consistency hard to implement

– How would you enforce SC using HTM?

May 5, 2020 L22-31

MIT 6.823 Spring 2020

HTM Challenges

• Performance pathologies: How to handle frequent contention?

– Should HTM guarantee fairness/enforce priorities?

• Size limitations: What happens if read-set + write-set exceed size of cache?

• Virtualization, I/O, syscalls…

May 5, 2020 L22-32

MIT 6.823 Spring 2020

HTM Challenges

• Performance pathologies: How to handle frequent contention?

– Should HTM guarantee fairness/enforce priorities?

• Size limitations: What happens if read-set + write-set exceed size of cache?

• Virtualization, I/O, syscalls…

• Hybrid TMs may get the best of both worlds:

– Handle common case in HW, but with no guarantees

• Abort on cache overflow, interrupt, syscall instruction, …– On abort, code can revert to software TM

– Current approach in Intel’s RTM…– … but still unclear how to integrate HTM & STM well

• Currently, slow/limited adoption by programmers,who must still support non-HTM systems

May 5, 2020 L22-32