Java Core | Modern Java Concurrency | Martijn Verburg & Ben Evans

Slide Design by http://www.kerrykenneally.com

Modern Java Concurrency

Ben Evans and Martijn Verburg

(@kittylyst @karianna)

http://www.teamsparq.net

Creative Commons - Attribution-NonCommercial-NoDerivs 3.0

2

No, we’re not in sales!

Directors of TeamSparq– “Optimisation for Technical Teams”

Ben is known for his performance and concurrency work

Martijn is “the Diabolical Developer” - @diabolicaldev

Authors of “The Well-Grounded Java Developer”

Co-Leaders of the LJC (London’s Java User Group)– Hold a seat on the JCP SE/EE Executive Committee

Regular conference speakers (JavaOne, Devoxx, OSCON, etc)

2

3

Who are these two anyway?

3

4

About this talk

We only have 1 hour to talk today

Huge amount to talk about

This subject fills 2 (or even 4) days worth of training

We will give you some highlights today

4

5

Modern Java concurrency

Not a new subject– Underwent a revolution with Java 5

More refinements since– Still more coming in 7

java.util.concurrent (j.u.c) really fast in 6– Better yet in 7

However, still under-appreciated by a surprising number

Too much Java 4-style concurrency code still in PROD– Why...?

5

6

Java concurrency - Why not upgrade?

Too much legacy code?– People scared to refactor?

People don’t know j.u.c is easier than “classic” Java concurrency?

People don’t know j.u.c is faster than classic?

People don’t know that you can mix-and-match (with a bit of care)?

Still not being taught at Universities?

Not enough people reading Doug Lea or Brian Goetz?

6

7

Modern Java concurrency

Perhaps: Previous treatments haven’t involved enough otters.

We will rectify this.

If you suffer from lutraphobia, you may want to leave now...

7

8

Otterly Amazing Tails of Modern Java Concurrency

Srsly.

Otters!

8

9

Why Otters?

Otters are a very good metaphor for concurrency

Not just because they look a bit like threads (i.e. long and thin)

Collaborative, Competitive & Sneaky

Can hare off in opposite directions

Capable of wreaking havoc if not contained

9

10

Otter Management

Safety– Does each object stay self-consistent?

– No matter what other operations are happening?

Liveness– Does the program eventually progress?

– Are any failures to progress temporary or permanent?

Performance– How well does the system take advantage of processing cores?

Reusability– How easy is it to reuse the system in other applications?

10

11

Some History

Until recently, most computers had only one processing core

Multithreading was simulated on a single core– Not true concurrency

Serial approach to algorithms often sufficed

Interleaved multithreading can mask errors– Or be more forgiving than true concurrency

Why and how (and when) did things change?

11

12

Moore’s Law

“The number of transistors on an economic-to-produce chip roughly doubles every 2 years”

Originally stated in 1965– Expected to hold for the 10 years to 1975

– Still going strong

Named for Gordon Moore (Intel founder)– About transistor counts

– Not clock speed

– Or overall performance

12

13

Transistor Counts

13

14

Moore’s Law - Problems

Not about overall performance

Memory latency exponent gap– Need to keep the processing pipeline full

– Add memory caches of faster SRAM “close” to the CPU• (L1, L2 etc)

Code restricted by L1 cache misses rather than CPU speed– After JIT compilation

14

15

Spending the transistor budget

More and more complex contortions...

ILP, CMT, Branch prediction, etc, etc

15

16

Multi-core

If we can’t increase clock speed / overall performance– Have to go multi-core

– Concurrency and performance are tied together

Real concurrency– Separate threads executing on different cores at the same moment

The JVM runtime controls scheduling– Java thread scheduling does NOT behave like OS scheduling

Concurrency becomes the performance improver

16

17

Classic Concurrency

Provides exclusion

Need locking to make mutation concurrency-safe

Locking gets complicated

Can become fragile

Why synchronized ?

17

18

Three approaches to Concurrent Type Safety

Fully-synchronized Objects– Synchronize all methods on all classes

Immutability– Useful, but may have high copy-cost

– Requires programmer discipline

Be Very, Very Careful– Difficult

– Fragile

– Often the only game in town

18

19

The JMM

Mathematical description of memory

Most impenetrable part of the Java language spec

JMM makes minimum guarantees

Real JVMs (and CPUs) may do more

19

Primary concepts

• synchronizes-with• happens-before• release-before-acquire• as-if-serial

20

Synchronizes-with

Threads have their own description of an object’s state– This must be flushed to main

memory and other threads

synchronized means that this local view has been synchronized-with the other threads

Defines touch-points where threads must perform synching

20

21

java.util.concurrent

Thanks, Doug!

j.u.c has building blocks for concurrent code

– ReentrantLock

– Condition

– ConcurrentHashMap

– CopyOnWriteArrayList

– Other Concurrent Data Structures

21

22

Locks in j.u.c

Lock is an interface

ReentrantLock is the usual implementation

22

23

Conditions in j.u.c

23

24

Conditions in j.u.c

24

25

ConcurrentHashMap (CHM)

HashMap has:– Hash function

– Even distribution of buckets

Concurrent form– Can lock independently

– Seems lock-free to users

– Has atomic operations

25

26

CopyOnWriteArrayList (COWAL)

Similar idea to CHM– Makes separate copies of

underlying structure

Iterator will never throw ConcurrentModificationException

26

27

CountDownLatch

A group consensus construct

countDown() decrements the count

await() blocks until count == 0– i.e. consensus

Constructor takes an int param– The count

Quite a few use cases– e.g. Multithreaded testing

27

28

Example - CountDownLatch

28

29

Handoff Queue (in-mem)

Efficient way to hand off work between threadpools

BlockingQueue a good pick

Has blocking ops with timeouts – e.g. for backoff / retry

Two basic implementations– ArrayList and LinkedList backed

Java 7 introduces the shiny new TransferQueue

29

30

Example - LinkedBlockingQueue

Imagine multiple producers and consumers30

31

Executors

j.u.c execution constructs– Callable, Future,

– FutureTask

In addition to the venerable– Thread and Runnable

Stop using TimerTask!

Executors class provides factory methods for making threadpools– ScheduledThreadPoolExecutor is one standard choice

Final building block for modern concurrent applications with Java

31

3232

Example - ThreadPoolManager

3333

Example - QueueReaderTask

3434

Fork/Join

Java 7 introduces F/J– similar to MapReduce

– useful for a certain class of problems

– F/J executions are not really threads

In our example, we subclass RecursiveAction

Need to provide a compute() method– And a way of merging results

F/J provides an invokeAll() to hand off more tasks

3535

Fork/Join

Typical divide and conquer style problem– invokeall() performs the threadpool/worker queue magic

36

Concurrent Java Code

Mutable state (objects) protected by locks

Concurrent data structures– CHM, COWAL

Be careful of performance– especially COWAL

Synchronous multithreading - explicit synchronization

Executor-based threadpools

Asynch multithreading communicates using queue-like handoffs36

37

Stepping Back

Concurrency is key to the future of performant code

Mutable state is hard

Need both synch & asynch state sharing

Locks can be hard to use correctly

JMM is a low-level, flexible model– Need higher-level concurrency model

– Thread is still too low-level

37

38

Imagine a world...

The runtime & environment helped out the programmer more– Runtime-managed concurrency

– Collections were thread-safe by default

– Objects were immutable by default

– State was well encapsulated and not shared by default

Thread wasn’t the default choice for unit of concurrent execution

Copy-on-write was the basis for mutation of collections / synchronous multithreading

Hand-off queues were the basis for asynchronous multithreading

38

39

What can we do with Java?

We’re stuck with a lot of heritage in Java

But the JVM and JMM are very sound

You don’t have to abandon Java– LMAX’s OSS “Disruptor” framework proves this

– Modern low-latency Java trading systems are screamingly fast

– Mechanical sympathy and clean code get you far

– The JIT compiler just gets better and better

If we wanted to dream of a new language– It should be on the JVM

– It should build on what we’ve learned in 15 years of Java

39

40

New Frontiers in Concurrency

There are several options now on the JVM– New possibilities built-in to the language syntax

– Synch and asynch models

Scala offers an Actors model– And the powerful Akka framework

Clojure is immutable by default– Has agents (like actors) & shared-nothing by default

– Also has a Software Transactional Memory (STM) model

Groovy has GPARs

40

41

Acknowledgments

All otter images Creative Commons or Fair Use

Photos owned by Flickr Users– moff, spark, sodaro, lonecellotheory, tomsowerby

– farnsworth, prince, marcus_jb1973, mliu92, Ed Zitron,

– NaturalLight & monkeywing

Dancing Otter by the amazing Nicola Slater @ folksy

Slide design by http://www.kerrykenneally.com

The Disruptor! http://code.google.com/p/disruptor/

41

42

Thanks for listening! (@kittylyst, @karianna)

42

http://www.teamsparq.net http://www.java7developer.com