Introduce Concurrency Threading in Servers. A Recurring Theme Concurrent programming is not for the…

Introduce Concurrency

Threading in Servers

A Recurring Theme• Concurrent programming is not for the weak

hearted!– Deadlocks, blocked threads, critical regions, …

• Tactics and Patterns– Ressource pool– Introduce Concurrency

CS@AU Henrik Bærbak Christensen 2

Thread Pool• Thread pool

– Keep a fixed size pool of N threads ready– Accept request and

• Retrieve a thread from pool to process request• Return thread to pool after processing has finished

– Result• N requests can be processed in parallel

CS@AU Henrik Bærbak Christensen 3

Why Not…• Multi-threaded

– Accept request and• Spawn a new thread• Terminate thread after processing has finished

• Why is this a lesser solution?– Performance (time/space)– Availability

CS@AU Henrik Bærbak Christensen 4

Java Support• ExecutorService

• Hint: Jenkov has nice tutorial– SocketReactor is modeled over his single-thread srv.

CS@AU Henrik Bærbak Christensen 5

In Action…

CS@AU Henrik Bærbak Christensen 6

But…

CS@AU Henrik Bærbak Christensen 7

Parallel Execution• The nasty-ness of shared resources hits us…

• Let us analyze the flow in SkyCave

– Consider that a max peak we have 15 concurrent threads commited to executing requests from 15 clients

• Can some of these 15 requests represent a request from the same player?

• Some may include ‘login’s?CS@AU Henrik Bærbak Christensen 8

In Reactor• s = serverSocket.accept()

– readAndDispatch(s)

CS@AU Henrik Bærbak Christensen 9

Any problems?

In Invoker

CS@AU Henrik Bærbak Christensen 10

In (Player) Dispatcher

CS@AU Henrik Bærbak Christensen 11

Ex: digRoom

CS@AU Henrik Bærbak Christensen 12

MongoDB

• Phew…– Seems we are safe there…

CS@AU Henrik Bærbak Christensen 13

Java Concurrency

Super fast reiteration

CS@AU Henrik Bærbak Christensen 14

Monitor• Add ‘syncronized’ to a method

– Becomes critical region• Only one thread allowed to execute at a time

– Multiple threads wait in queue for execution• If several threads call the method at the same time• The ‘queue’ controlling object is the object of the method

– i.e. there is a ‘per object’ queue

– If the method freeze• All queued threads does as well!

CS@AU Henrik Bærbak Christensen 15

ReentrantLock• You can declare a specific ‘queue/lock’ object

and synchronize on that– More tedios, but much finer control

• Locking across a set of objects– Allows timeout! Avoid ‘blocked threads’

CS@AU Henrik Bærbak Christensen 16

Idiom

• Alternative: tryLock(timeout)

CS@AU Henrik Bærbak Christensen 17

lock.lock();try { // critical region} finally { lock.unlock();}

Tutorials…• Jenkov’s tutorials appear very nice…

CS@AU Henrik Bærbak Christensen 18

Our Tool Stack

Help! Help!

CS@AU Henrik Bærbak Christensen 19

Blocked Threads!• Load Generator, my faulty first version

– N players execute M sets of operations• Output

CS@AU Henrik Bærbak Christensen 20

How to diagnose?• ‘jvisualvm’

CS@AU Henrik Bærbak Christensen 21

Thread Dump• Pick VM, Threads Pane, Click Thead Dump

CS@AU Henrik Bærbak Christensen 22

Review Dump• Locate the blocking thread, see stack trace !

CS@AU Henrik Bærbak Christensen 23

Monitoring Performance• Started ‘load.cave’

CS@AU Henrik Bærbak Christensen 24

More Load• Started SkyCave app server on small machine

CS@AU Henrik Bærbak Christensen 25

More Load• Started 1 x load.cave of 100 players each

CS@AU Henrik Bærbak Christensen 26

More Load• Started 15 x load.cave of 100 players each

CS@AU Henrik Bærbak Christensen 27

One server,Approx 13.100 req/s

The ‘Lyon Airport’ Test• Karibu System: Daemon died from exhausted memory

CS@AU Henrik Bærbak Christensen 28

Going Further

CS@AU Henrik Bærbak Christensen 29