Parallel? Sleep well!

© Acando AS© Acando AB

Parallel? Sleep well!

Fredrik Bertilsson

© Acando AS

Fredrik [email protected]

© Acando AS 2023-04-074

Contents

●Introduction Why this talk? The free lunch

●Solution patterns Active object Lock ordering Execution pipeline

●Wrap-up

© Acando AS 2023-04-075

Why This Talk?

●Something new? No.

●Multithreading available for decades

●What I will show: Modern APIs not enough to make safe and friendly classes

Simple, language independent techniques

Sleep well

© Acando AS 2023-04-076

The Free Lunch

1970 – 2005

●Exponential CPU frequency

●Exponential transistor density

© Acando AS 2023-04-077

The Free Lunch is …

●2013 – The free lunch is

OVER

© Acando AS 2023-04-078

Now and Future

●Exponential cores If developers want it...

●Good! I’ll parallelize my whole program ●Amdahls law

The speedup of a program … is limited by the time needed for the sequential fraction …

© Acando AS 2023-04-079

Amdahl’s Law

© Acando AS 2023-04-0710

Sequential Program

●Enter threads

●Let’s speed up this sequential program:

private static long a;

for (long i = 0; i < LargeConstant; i++)

{

a++;

}

© Acando AS 2023-04-0711

Sequential Program

●15 s

© Acando AS 2023-04-0712

Remedy - Parallel

●Parallelize it

const long k = LargeConstant / 2;

var thread1 = new Thread(() =>

{ for (long i = 0; i < k; i++) { a++; } });

var thread2 = new Thread(() => { for (long i = k; i < LargeConstant; i++) { a++; } });

thread1.Start(); thread2.Start();

thread1.Join(); thread2.Join();

© Acando AS 2023-04-0713

Remedy - Parallel

●40 s●Not 15 / 2

●Not 15 * 2

© Acando AS 2023-04-0714

Memory Caching for One Core

Core

L1 Cache

L2 Cache

L3 Cache

DRAM

© Acando AS 2023-04-0715

Memory Caching for Multiple Cores

Core

L1 Cache

L2 Cache

L3 Cache

DRAM

Core

L1 Cache

L2 Cache

L3 Cache

Core

L1 Cache

L2 Cache

L3 Cache

© Acando AS 2023-04-0716

Memory Caching – Cache Misses are Expensive

© Acando AS 2023-04-0717

What Have We Learned?

●Each thread should use “private” data, not shared among threads

●Better code:

var thread1 = new Thread(() =>

{

long temp = 0;

for (long i = 0; i < k; i++) { temp++; }

a += temp;

});

Corresponding for thread 2

© Acando AS 2023-04-0718

What Have We Learned?

●Each thread should use “private” data, not shared

●Better code than our previous “remedy”:

var thread1 = new Thread(() =>

{

long temp = 0;

for (long i = 0; i < k; i++) { temp++; }

Interlocked.Add(ref a, temp);

});

© Acando AS 2023-04-0719

Speed now

●9 s●Down to 9 from 15

© Acando AS 2023-04-0720

Active Object Pattern – A Real Remedy

●Externally Safe and friendly interface

● Internally Private thread Private data Safe termination

© Acando AS 2023-04-0721

Active Object Pattern – Externally

●Safe and friendly interface

●What if we could code something like this:

PrimeCalculator calculator = new PrimeCalculator();

var future = calculator.Calculate(7);

// do other work here WHILE CALCULATOR RUNS

var result = future.Result;

© Acando AS 2023-04-0722

Active Object Pattern – Internally 1(2)

●First, the synchronous version:

public class PrimeActiveObject

{

public IEnumerable<int> Calculate(int limit)

{

…

}

© Acando AS 2023-04-0723

Active Object Pattern – Internally 2(2)

●The asynchronous version:

public Task<IEnumerable<int>> CalculateAsync(int limit)

{

var future = new Task<IEnumerable<int>>(

() => Calculate(limit));

future.Start();

return future;

}

© Acando AS 2023-04-0724

Active Object Demo

© Acando AS 2023-04-0725

Where are we now?

●We have the Active Object pattern Works for NET 4.5 Works for Java 1.5? Works for C++11

●What about the rest of us?

© Acando AS 2023-04-0726

Active Object with Thread

●A queue of commands

●Commands hold private data

●Thread executes commands in turn

●Properties Execution is parallel Execution is ordered

© Acando AS 2023-04-0727

Lock Ordering

●Order your locking so that you Avoid deadlock Avoid starvation

●E.g. by file name File 1File 2

Thread A Thread B

© Acando AS 2023-04-0728

Execution Pipeline

© Acando AS 2023-04-0729

Execution Pipeline

●If you have a sequence of operations on data Like a pipeline

●Write your program that way, to avoid cache misses

●Techniques Manually Microsoft TPL (Task Parallel Library) Dataflow

© Acando AS 2023-04-0730

Wrap-Up

●Free lunch is … Over

●Use new APIs and … Crash! They are nice, but not

enough

●Safe and friendly patterns Active Object Execution Pipeline Lock Ordering

●Now, you can: Deliver safe asynchronous objects Use the CPUs of today and

tomorrow

© Acando AS 2023-04-0731

References

●Watch this presentation on slideshare (soon)http://www.slideshare.net/FredrikBertilsson

●Source code: https://github.com/fbertilsson/parallel-sleep-well.git

●Ordered Execution With ThreadPool, Stephen Toub, MSDNhttp://msdn.microsoft.com/en-us/magazine/dd419664.aspx

© Acando AS 2023-04-0732

Thank you for listening. Sleep well!