Multicoreexpo2011 05-05fridiusingtemplate-110518193827-phpapp02

1

Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH

aicas Technology

Multicore for Real-Timeand Safety-Critical Software:

Avoid the Pitfal ls

Dr. Fridtjof Siebert, CTO, aicasDr. James J. Hunt, CEO aicas

MultiCoreExpo 2011, 5th May 2011

2


Agenda

Race conditions

Synchronization

Atomic operations

Memory model

Values out of thin air

Reaching peak performance

CPU affinities

Multicore scheduling

Lock free algorithms

Compare and Swap

3


Typical Problems on Mult icore

typical code sequence (C/C++ or Java)

int counter;

void increment(){ counter++; }

4




int counter;

void increment(){ counter++; } r1 = counter;

r2 = r1 + 1; counter = r2;

5




int counter;


r2 = r1 + 1; counter = r2;

r1 = counter; r2 = r1 + 1; counter = r2;

Thread 1 Thread 2

6




int counter;


r2 = r1 + 1; counter = r2;

r1 = counter; r2 = r1 + 1; counter = r2;

Thread 1 Thread 2

An increment can get lost!

7




int counter;


8




int counter;


code lacks synchronization

but on a single core, it practically always works!

on a multicore, chances for failure explode!

9


Synchronization

solution: synchronize

int counter;

synchronized void increment(){ counter++; }

Easy, problem solved.

Right? See later.

10


Atomic Operations

What is the result of

int a, b; /* 32 bit, initially 0 */

Thread 1 Thread 2 b = a; a = 1;

?

11


Atomic Operations


int a, b; /* 32 bit, initially 0 */


?

b == 0b == 1

12


Atomic Operations


long a, b; /* 64 bit, initially 0 */


?

13


Atomic Operations


long a, b; /* 64 bit, initially 0 */


?

b == 0b == 1b == 4294967296b == 4294967295

14


Cache Structure

CPUs may have local caches for performance

CPU0 CPU1CPU0 CPU1CPU0 CPU1CPU0 CPU1 CPU0 CPU1CPU0 CPU1CPU0 CPU1CPU2 CPU3

CPU0 CPU1CPU0 CPU1L1 Cache L1 CacheL1 Cache L1 Cache

L2 Cache L2 Cache

Main Memory

15


Cache Structure

Modifications do not become visible immediately

Modifications may be re-ordered

Reads may refer to outdated (cached) data

Reads may be re-ordered

CPU0 CPU1 CPU2 CPU3

L1 Cache L1 CacheL1 Cache L1 Cache

L2 Cache L2 Cache

Main Memory

16



polling update

long counter;[..]do { doSomething(); }while (counter < MAX);

counter is incremented by parallel thread

on a Multicore, changes to counter may not become visible!

17



polling update

long counter;[..]do { doSomething(); }while (counter < MAX);

counter is incremented by parallel thread

on a Multicore, changes to counter may not become visible!

18


Solution: volati le?

polling update

volatile long counter;[..]do { doSomething(); }while (counter < MAX);

works for Java

does not work for C!

19


Solution: volati le?

polling update

volatile long counter;[..]do { doSomething(); }while (counter < MAX);

works for Java

does not work for C!

20


Understanding the Memory Model

Memory model specifies what optimizations are permitted by the compiler or underlying hardware

C/C++ programs have undefined semantics in case of race conditions

Java defines a strict memory model

21


Java's Memory Model

ordering operations aresynchronized block

accessing a volatile variable

The presence of an ordering operation determines the visible state in shared memory

22


Java's Memory Model: Defined Order

all reads are completed before entering synchronized block, or

reading a volatile variable

read fence

all writes are completed before exiting a synchronized block, or

writing a volatile variable

write fence

23


Java's Memory Model: Data Races

data races are not forbidden in Java you can use shared memory variables

your code has to tolerate optimizations

examples collecting debugging / profiling information

useful if occasional errors due to data races are tolerable

24


Memory Model Example

Shared memory communicationPtr p;boolean p_valid;

Thread 1

p = new Ptr(); p_valid = true;

25




Thread 1 Thread 2

p = new Ptr(); p_valid = true;

26




Thread 1 Thread 2

p = new Ptr(); if (p_valid)p_valid = true; p.call();

27



Shared memory communication

Thread 1 Thread 2 p = new Ptr(); if (p_valid)p_valid = true; p.call();

28





What may happen:

29





What may happen: t1 = new Ptr(); t2 = true; p_valid = t2; p = t1;

30






Writes reordered!

31






Writes reordered!

32





What may happen: t1 = new Ptr(); t3 = p;t2 = true; if (p_valid) p_valid = t2; t3.call(); p = t1;

Writes reordered! Reads reordered!

33





What may happen: t1 = new Ptr(); t3 = p;t2 = true; if (p_valid) p_valid = t2; t3.call(); p = t1;

Writes reordered! Reads reordered!

34


Shared memory communicationvolatile Ptr p;volatile boolean p_valid;


in Java

Example Use of Java's Memory Model

35




in Java


36




in Java


37


Shared memory communicationvolatile Obj *p;volatile boolean p_valid;

Thread 1 Thread 2 p = malloc(..); if (p_valid)p_valid = TRUE; p>f = ..;();

in C?

Example Use of C's Memory Model

38




in C? CPU may reorder memory accesses!


39






40






41



Thread 1 Thread 2 p = malloc(..); if (p_valid)p_valid = TRUE; p>f = ..;

How to fix it?


42



Thread 1 Thread 2 p = malloc(..); if (p_valid)asm volatile( p>f = ..; "sfence":::"memory"); p_valid = TRUE;

How to fix it? Add memory fences!


43



Thread 1 Thread 2 p = malloc(..); if (p_valid)asm volatile( { "sfence":::"memory"); asm volatile(p_valid = TRUE; "lfence":::"memory"); p>f = ..; }



44




Thread 1 Thread 2 p = malloc(..); if (p_valid)asm volatile( { "sfence":::"memory"); asm volatile(p_valid = TRUE; "lfence":::"memory"); p>f = ..; }


45


Out of Thin Air

imagine this codeint x = 0, n = 0;

Thread 1 Thread 2 for (i=0; i<n; i++) x = 42; x += f(i); print(x);

46


Out of Thin Air

imagine this codeint x = 0, n = 0;

Thread 1 Thread 2 for (i=0; i<n; i++) x = 42; x += f(i); print(x);

can only print 42 in Java

47


Out of Thin Air: Introduction of Writes

loop optimization in C/C++int x = 0, n = 0;

Thread 1 Thread 2 tmp = x; for (i=0; i<n; i++) x = 42; tmp += f(i);x = tmp; print(x);

48


Out of Thin Air: Introduction of Writes

loop optimization in C/C++int x = 0, n = 0;

Thread 1 Thread 2 tmp = x; for (i=0; i<n; i++) x = 42; tmp += f(i);x = tmp; print(x);

can print 0 in C/C++

49


Out of Thin Air

imagine this codeint x = 0, y = 0;

Thread 1 Thread 2 r1 = x; r2 = y;y = r1; x = r2;

50


Out of Thin Air


Thread 1 Thread 2 r1 = x; r2 = y;y = r1; x = r2;

Expected result x == 0; y == 0;

Only possible result in Java

51


Out of Thin Air: Optimization in C/C++


Thread 1 Thread 2 y = 42; r2 = y;r1 = x; x = r2;if (r1 != 42) y = r1;

Possible results in upcoming C++ MM x == 42; y == 42;

52


Performance on Mult icore: Example

Single core application, 3 threads

All threads synchronize frequently on same lock

53




All threads synchronize frequently on same lock

while (true) { synchronized (lock) { counter++; } doSomething(); }

54




55



On a multicore


56


Performance on a Mult icore

Frequent synchronization can kill the performance

Typical non-RTOS will use heuristics to improve average performance

spin-lock for a short time

blocking for longer periods

57


Performance on a Mult icore

These heuristics introduce priority-inversion and generally destroy predictability

A typical semaphore implementationdoes not take thread priority into account

does not limit worst-case-execution-time

58


CPU Affinit ies

OSes provide APIs to lock threads certain CPUs

This limits the decision space of the RTOSa global scheduler for n CPUs would always run the n highest priority threads

with affinities, this may not be possible, e.g., if the two highest priority threads are locked to the same CPU

CPU affinities can introduce priority inversion!

So what are CPU affinities good for?

59


CPU Affinit ies: No More interrupts

Locking interrupts to a dedicated CPUprotects all other CPUs from interrupts, and

invalidated cashes

WCETA is simplified considerably

60


CPU Affinit ies: No More interrupts

Locking interrupts to a dedicated CPUprotects all other CPUs from interrupts, and

invalidated cashes

WCETA is simplified considerably

non-RT Task

Interrupt

CPU0

CPU1 A A A A A A A A A A A RT Task

61


CPU Affinit ies: Separating Threads

Locking thread A to one CPU and threads B, C, ... to other CPUs may increase A's performance.

A will not be preempted by B, C, ..

A will not see its caches invalidated by B, C, ...

62


CPU Affinit ies: Separating Threads

Locking thread A to one CPU and threads B, C, ... to other CPUs may increase A's performance.

A will not be preempted by B, C, ..

A will not see its caches invalidated by B, C, ...

non-RT Task

RT Task

CPU0

CPU1

CPU2

A AA A A A

B

B B

B

B

A A A A A

B

B

B

C C C

63


CPU Affinit ies: Grouping Threads

Locking threads A and B to the same CPUwill increase shared memory communication between A and B

will avoid performance degradation on locking

will enable simple scheduling analysis (RMA)

64


CPU Affinit ies: Grouping Threads

Locking threads A and B to the same CPUwill increase shared memory communication between A and B

will avoid performance degradation on locking

will enable simple schedule feasibility analysis (RMA)

RT Task ACPU0

CPU1

CPU2

BAA B A RT Task B

Other Task

C

C

DD D D

E E E E

E

E

EC C

GG

G

BA

65


CPU Affinit ies: Diff icult Decisions

what if A and B both computation intensive and both access the same shared memory?

How can we use idle time?

...

66


Multicore Scheduling for Realt ime

A pure priority based scheduler is not sufficient:

Imagine three tasks A, B, C on 2 CPUS

A

B

C

Release Deadline

67



Priorities will cause deadline miss

A pri=10

B pri=10

CPU0

CPU1

C pri=9

Release Deadline

68



Priorities will cause deadline miss

A pri=10

B pri=10

CPU0

CPU1

C pri=9Release Deadline

69



CPU affinities do not help

A pri=10, {CPU0}

B pri=10, {CPU1}

CPU0

CPU1

C pri=9

Release Deadline

70



Round robin could help

ACPU0

CPU1

C B

B A C B A C

A C B

Release Deadline

71


A C B

B A C B A C

A C B


Round robin could help

Release Deadline

CPU0

CPU1

72


Performance on Multicore

Synchronization is expensive

Can synchronization be avoided?

Can lock free algorithms be used?Use compare and swap (CAS) instructions instead

73


Lock Free Algorithms

Typical code sequence

do { x = counter; result = CAS(counter,x,x+1); }while (result != x);

74


Compare-and-Swap Issues



What is the WCET? ∞?

75


On dual core:

# iterations

fre

qu

en

cy



What is the WCET? ∞?

Compare-and-Swap Issues

76


Lock Free Library Code

Using libraries helpsAtomicInteger counter = new AtomicInteger();void increment(){ (void)counter.incrementAndGet();}

Code is easier and safer

Hand made lock free algorithms are not for normal application development

77


Compare-and-Swap Solutions

One way state changes, without retry

Bounded number of retries

78


One Way State Changes Using CAS

Example codefor (i=0; i<N; i++) { new Thread() { public void run() { CAS(state,INIT,STARTING); [..] } }.start(); }

79


Bounding Retries for CAS

introduce long enough code sections in between 2 compare-and-swap loops

then, if a retry is required, one other CPU was successful

after n-1 conflicts, one can be sure that all other CPUs are outside the CAS loop

80


Bounding Retries for CAS: Example

AtomicInteger counter = new AtomicInteger(); static final int GRANULARITY = 64; [..] new Thread(){ int local_counter; public void incCounter() { local_counter++; if (local_counter >= GRANULARITY) { local_counter = 0; counter.addAndSet(GRANULARITY); } } }.start();

81


Measurements

alloc free sweep available Memory1E-1

1E+0

1E+1

1E+2

1E+3

1E+4

1E+5

1E+6

1E+7

1E+8

1E+9

1E+10

1E+11

123456789

Number of CAS tries is bounded:

On 8-CPU x86 system (Linux)

82


CAS Lists

List modifaction using CAS, single linked list

next

data

head next

data

next

data

A B C

83


CAS Lists

Add

next

data

head next

data

next

data

next

data

X

B CA B C

84


CAS Lists

Add

next

data

head next

data

next

data

next

data

X

B CA B C

85


CAS Lists

Add: CAS(head,A,X)

next

data

head next

data

next

data

next

data

X

B CA B C

86


CAS Lists

Add: CAS(head,A,X)

next

data

head next

data

next

data

next

data

A B C

X

87


CAS Lists

Add

next

data

head next

data

next

data

next

data

A B C

X

88


CAS Lists

Remove

head next

data

next

data

next

data

A B C

89


CAS Lists

Remove: CAS(head,A,B)

head next

data

next

data

next

data

A B C

90


CAS Lists

Remove: CAS(head,A,B)

head next

data

next

data

next

data

A B C

91


CAS Lists

Remove

head next

data

next

data

B C

92


CAS Lists, ABA Problem

Now, consider concurrent modifications:

next

data

head next

data

A B

next

data

C Thread 2Thread 1

93



next

data

head next

data

A B

next

data

C Thread 2Thread 1remove:CAS(head,A,B)

94



next

data

head next

data

A B

next

data

C Thread 2Thread 1remove:

CAS(head,A,B)

preempted

95



next

data

head next

data

A B

next

data

C Thread 2remove:CAS(head,A,B)

Thread 1remove:

CAS(head,A,B)

preempted

96



next

data

head next

data

A B

head next

data

B

next

data

C

next

data

C

Thread 2remove:CAS(head,A,B)

Thread 1remove:

CAS(head,A,B)

preempted

97



next

data

head next

data

A B

head next

data

B

next

data

C

next

data

C


remove:CAS(head,B,C)

Thread 1remove:

CAS(head,A,B)

preempted

98



next

data

head next

data

A B

head next

data

B

next

data

C

next

data

C

head next

data

C



Thread 1remove:

CAS(head,A,B)

preempted

99



next

data

head next

data

A B

head next

data

B

next

data

C

next

data

C

head next

data

C



add A:CAS(head,C,A)

Thread 1remove:

CAS(head,A,B)

preempted

100



next

data

head next

data

A B

head next

data

B

next

data

C

next

data

C

head next

data

C

head next

data

C

next

data

A


remove:CAS(head,A,B)

add A:CAS(head,C,A)

Thread 1remove:

CAS(head,A,B)

preempted

101



next

data

head next

data

A B

head next

data

B

next

data

C

next

data

C

head next

data

C

head next

data

C

next

data

A

head next

data

C

next

data

A

next

data

B



add A:CAS(head,C,A)

Thread 1remove:

CAS(head,A,B)

preempted

102



next

data

head next

data

A B

head next

data

B

next

data

C

next

data

C

head next

data

C

head next

data

C

next

data

A

head next

data

C

next

data

A

next

data

B



add A:CAS(head,C,A)

Thread 1remove:

CAS(head,A,B)

preempted

B was re-introduced!

103


ABA Problem: Solutions

Non solution: In C: free() after remove

Reason: newly allocated block may be sameSolution: In Java: only add new references

Reason: GC ensures old values no longer visibleSolution: Sync all threads before reuse

Example: Phase 1 moves elements from List1 to List2, Phase 2 moves elements back

Solution: Use 64-/128-bit CAS and mod. counter

104


ABA Solved via Modification Counter

next

data

head: 42 next

data

A B

next

data

C Thread 2Thread 1remove:

preempted

105



next

data

head: 42 next

data

A B

next

data

C Thread 2remove:CAS(head,A:42,

B:43)

Thread 1remove:

preempted

106



next

data

head: 42 next

data

A B

head: 43 next

data

B

next

data

C

next

data

C

Thread 2remove:CAS(head,A:42,

B:43)

Thread 1remove:

107



next

data

head: 42 next

data

A B

head: 43 next

data

B

next

data

C

next

data

C

head: 44 next

data

C


B:43)

remove:CAS(head,B:43,

C:44)

Thread 1remove:

108



next

data

head: 42 next

data

A B

head: 43 next

data

B

next

data

C

next

data

C

head: 44 next

data

C

head: 45 next

data

C

next

data

A


B:43)


C:44)

add A:CAS(head,C:44,

A:45)

Thread 1remove:

109



next

data

head: 42 next

data

A B

head: 43 next

data

B

next

data

C

next

data

C

head: 44 next

data

C

head: 45 next

data

C

next

data

A


B:43)


C:44)


A:45)

Thread 1remove:

CAS(head,A:42,B:43)

retry!

110



next

data

head: 42 next

data

A B

head: 43 next

data

B

next

data

C

next

data

C

head: 44 next

data

C

head: 45 next

data

C

next

data

A

head: 46 next

data

C


B:43)


C:44)


A:45)

Thread 1remove:

CAS(head,A:42,B:43)

retry!CAS(head,A:45,

C:46)

111


ConclusionCode that runs well on single CPU may fail on multicore

Clear semantics of concurrent code is required for functional correctness

Cost of locking may be prohibitive

CPU affinities may help, but it is difficult to make the application more efficient

Lock free code is very hard to get right

A reliable memory model and good concurrent libraries are basis to avoid pitfalls.

112


Thank you.

Please come see us at booth 2306

Multicoreexpo2011 05-05fridiusingtemplate-110518193827-phpapp02

Technology

counter r2

fridtjof siebert

aicas gmbhdr

aicas gmbh10

r2 multicore

aicas technology multicore

single core

typical problems