Ken Wadland, PhD Refactoring to Concurrency (Java Edition) Prepared for: Boston Java Meetup Group February 2011.

Ken Wadland, PhD

Refactoring to Concurrency

(Java Edition)Prepared for: Boston Java Meetup

GroupFebruary 2011

Overview

• Step 1: Establish Performance Criteria• Step 2: Annotate Existing Class Thread

Safety• Step 3: Increase Thread Safety (as

needed)• Step 4: Use Tasks instead of loops/steps• Step 5: Use Thread Pool• Step 6: Increase Concurrency

Copyright 2010, Concurrency Magic

Common Concurrency Problems

• Data Race Condition• Deadlock• Liveliness• Memory Visibility• And, “Heisenbugs”


Example Problem

Solving Sudoku Puzzles

Single Threaded Sudoku (4x4

• There are 12 “Houses” – 4 Rows– 4 Columns– 4 Boxes

• Each Value (1 - 4) must appear exactly once in each House

• Algorithm:– Look for “Singles”– (Cells where only

one value is valid)3

41 32

24

1

1

312

2

4

4 3

Concurrent Sudoku: Thread per value

**

1 *

**

* *

3*

* 3

*4

* *X X

XX

X X

X X

XXX X

XX

X X

X

XX X

X

X1 X

3

X

X

2

34

2

X X

X

X X

X

4 X

4

X 2

X2X

1 X

1

Thread Algorithm:4 Threads. Each

considers one value

Mark which cells cannot have that value

Look for Singles for that value

Alternative Algorithm #1:Thread per cell

34

1 3

13

24

13

24

13

24

13

24

13

24

13

24

13

24

13

24

13

24

13

24

13

24

13

24

Thread Algorithm:16 threads each

consider one cell and its 4 “candidates“

Look at neighbor cells in 3 “houses”

Remove invalid candidates

Alternative Alogorithm #2:Thread per “House”

34

1 3

34

1 3

34

1 3

4

4 2

2

2

2

2

4

2

4

4

4

2

2

2

31

3

31

1

4

4

4

31

3

31

1

2

2

2

1

1

1

Alternative Algorithm #3:Thread per “Candidate”

34

1 3

13

24

13

24

13

24

13

24

13

24

13

24

13

24

13

24

13

24

13

24

13

24

13

24

Thread Algorithm:64 Threads. Each

considers one candidate

Look at neighbor cells in 3 “houses”

Remove if invalid

Package Architecture


Step 1: Establish Criteria

“If you don’t know where you’re going, how will you know when you gotten

there?”

Success Criteria Examples

• Good Examples:– Response time < 1ms for typical queries– Process >100 queries per second– Deterministic Results

• Bad Examples:– Keep all processors busy– Keep all threads busy– Avoid waiting for locks


Performance Test


Timing Java Apps (1st attempt)

• Start timer• Run algorithm A• Stop timer• Report elapsed time• Start timer• Run algorithm B• Stop timer• Report elapsed time

. . .• Start timer• Run algorithm Z• Stop timer• Report elapsed time


Timing Java Apps (2nd attempt)

• Repeat 1000 times• Start timer• Repeat 100 times (with 100 different puzzles)• Run algorithm A• Stop timer• Report min/ave/max times (divided by 100,000)• Repeat 1000 times• Start timer• Repeat 100 times (with 100 different puzzles)• Run algorithm B• Stop timer• Report min/ave/max times (divided by 100,000). . .


Timing Java Apps (3rd attempt)

Repeat 1000 times• Start timer A• Repeat 100 times: Run algorithm A• Stop timer A• Start timer B• Repeat 100 times: Run algorithm B• Stop timer B

. . .• Start timer Z• Repeat 100 times: Run algorithm Z• Stop timer ZReport min/ave/max (divided by 100,000)


Timing Java Apps (4th attempt)

• Start JVM• Repeat 1000 times• Start timer• Repeat 100 times (with 100 different puzzles)• Run algorithm A• Stop timer• Report min/ave/max times (divided by 100,000)• Stop JVM

Repeat for each algorithm



Statistics Gathering

Performance results solving 100 puzzles 1000 times.

Sample Goal: cut time in half using 4 cores

Best Time

Ave Time

Nickname

Algorithm

15.261 ms

16.456 ms

STP Pencil

30.569 ms

32.806 ms

STS Scanner

Step 2: Annotate Type Safety

Assign Type Safety Annotation to every class that might be used

concurrently

Concurrency Categories for Classes

• Immutable• Thread-Safe• Thread-Friendly• Thread-Neutral• Thread-Hostile



New Package: Annotations

public @interface Immutable { }public @interface ThreadSafe { }public @interface ThreadFriendly { }public @interface ThreadHostile { }public @interface ThreadNeutral { }

public @interface GuardedBy { String value();}


Value

public enum Value { UNKNOWN(0, '.'), ONE(1, '1'), TWO2, '2'), . . . FIFTEEN (15, 'F'), SIXTEEN (16, 'G');

final private int number;

final private char name;

final static private Map<Character,Value> charMap = new HashMap<Character, Value>();

final static private List<Value> intList = new ArrayList<Value>(16);. . .

@Immutable


Location

public class Location { private int row; private int col; private int offset; private int boxSize;

. . .

@ThreadNeutral


ValueSet

public class ValueSet { private Value value = Value.UNKNOWN;

private EnumSet<Value> includedValues;

. . .

@ThreadNeutral


Puzzle

public class Puzzle implements Cloneable { public static final List<Integer> SUPPORTED_SIZES

= Arrays.asList(4, 9, 16);

static int size = 0; static int boxSize = 0; static int gridSize = 0;

private static List<List<Location>> rows; private static List<List<Location>> cols; private static List<List<Location>> boxes; private static List<List<Location>> houses; private static List<Location> allLocations;

private List<Value> values;. . .

@ThreadHostile

Step 3: Increase Thread Safety (as needed)

Modify classes or create new classes with higher levels of

atomicity


Concurrency in Java


AbstractQueuedSynchronizer

Your Thread Safe Code

java.util.concurrent.lock

compareAndSwap, AtomicReference

java.util.concurrent.atomic

“happens before” Java JVM

Lock (ReentrantLock), Condition java.util.concurrent.lock

CountDownLatch, Semaphore java.util.concurrent

AtomicInteger, AtomicLongArrayjava.util.concurrent.atomi

c

Executer, Future, ConcurrentMap java.util.concurrent

Your Thread Neutral Code

http://download-llnw.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks/package-summary.html




Location

final public class Location { final private int row; final private int col; final private int offset; final private int boxSize;

// Getters but not Setters


Puzzle Hierarchy


Puzzle (Base Class)

public abstract class Puzzle implements Cloneable {/** List of Puzzle sizes supported (4x4, 9x9 and 16x16). */public static final List<Integer> SUPPORTED_SIZES = Arrays.asList(4, 9, 16);

/** Puzzle table used when this Puzzle was created. */protected final PuzzleTables tables;

/** Size of this Puzzle (4, 9 or 16). */protected final int size;


Basic Puzzle (Thread Neutral)

@ThreadNeutralpublic final class BasicPuzzle extends Puzzle {/** Table containing current Values for each cell in

grid. */private final List<Value> values;


BasicPuzzle get/set

@Overridepublic Value getValue (Location loc) { return values.get(loc.getOffset());}

@Overridepublic void setValue (Location loc, Value v) { values.set(loc.getOffset(), v);}


SynchronizedPuzzle

@ThreadSafepublic class SynchronizedPuzzle extends Puzzle { @GuardedBy("this") private final List<Value> values;

@Override public synchronized Value getValue (Location loc) { return values.get(loc.getOffset()); }

@Override public synchronized void setValue (Location loc, Value v) { values.set(loc.getOffset(), v);}


LockingPuzzle.LockableValue

@ThreadSafepublic final class LockingPuzzle extends Puzzle {

/** Value that can be locked */ private class LockableValue { private Value value; public LockableValue() { value = Value.UNKNOWN;} public synchronized Value get() { return value; } public synchronized void set(Value v) { value = v; } }


LockingPuzzle get/set

@GuardedBy("itself") private final List<LockableValue> values;

@Overridepublic Value getValue (Location loc) { return values.get(loc.getOffset()).get();}

@Overridepublic void setValue (Location loc, Value v) { values.get(loc.getOffset()).set(v);}


AtomicPuzzle

@ThreadSafepublic final class AtomicPuzzle extends Puzzle { AtomicReferenceArray<Value> values;

@Override public Value getValue (Location loc) { return values.get(loc.getOffset()); }

@Override public void setValue (Location loc, Value v) { values.set(loc.getOffset(), v); }

. . .


ImmutablePuzzle

@Immutablepublic final class ImmutablePuzzle extends Puzzle { private final List<Value> values;

@Overridepublic Value getValue (Location loc) { return values.get(loc.getOffset());}

@Overridepublic void setValue (Location loc, Value v) { throw new IllegalArgumentException( "setValue is not implemented for ImmutablePuzzle");}

New class: GenericScanSolver

public class ScanSolver extends Solver { /** Puzzle currently being solved. */ final BasicPuzzle puzzle; ...

public class GenericScanSolver extends Solver { /** Puzzle currently being solved. */ final Puzzle puzzle; ...


Use SynchronizedPuzzleST_SCANNER_WTH_SYNCHRONIZED_PUZZLE("STS.S") { public String getName() {

return "ST Scanner with SynchronizedPuzzle"; } public Solver newSolver(Puzzle input,

Verbosity verbosity) { Puzzle p = SynchronizedPuzzle.makeCopy(input); Solver s = GenericScanSolver.newSolver(p, verbosity); return s; }},


Use LockingPuzzleST_SCANNER_WTH_LOCKING_PUZZLE("STS.L") { public String getName() {

return "ST Scanner with LockingPuzzle"; } public Solver newSolver(Puzzle input, Verbosity verbosity) { Puzzle p = LockingPuzzle.makeCopy(input); Solver s = GenericScanSolver.newSolver(p, verbosity); return s; }},


Step 4: Use Tasks

Convert loops and/or separate steps into Tasks.

Execute Tasks from a Queue.(Still Single Threaded)


Creating Executor

private boolean runAllTests() { // Create pool of threads to execute tasks ExecutorService exec =

Executors.newSingleThreadExecutor();

try { for (PuzzleResults r : results) { if (runOneTest(exec, r) != true) { // Report error

return false; } } } finally { exec.shutdown(); } return true;}



Pass (unused) Executor

public static Solver newSolver(Executor exec, Puzzle input, Verbosity verbosity) { return new ScanSolver(input, verbosity);}

Using Tasks in seekHiddenSingles

protected void seekHiddenSingles() { final CountDownLatch latch = new CountDownLatch(unsolvedValues.count()); for (final Value v : unsolvedValues.getAllValues()) { Runnable task = new Runnable() { public void run() { boolean foundAll = seekOneHiddenSingle(v); if (foundAll) { unsolvedValues.removeValue(v); } latch.countDown(); } }; exec.execute(task); }

try { latch.await(); ... Error handling}


Step 5: Use a Thread Pool

Replace Single Threaded Executor with Multi-threaded

Executor


Creating Thread Pool

private boolean runAllTests() { // Create pool of threads to execute tasks ExecutorService exec =

Executors.newFixedThreadPool(numberOfThreads);

try { for (PuzzleResults r : results) { if (runOneTest(exec, r) != true) { // Report error

return false; } } } finally { exec.shutdown(); } return true;}


Step 6: Increase Concurrency

In this case, rather than solve one Puzzle in parallel,

solve different Puzzles in parallel


Concurrent Sudoku #5:Thread per puzzle

34

1 341

1

21

2

2 3

342

3

44 12

24

1 3 2

2

23

11

14 4

4

4

3

4 3 1

2

1

4

3 3 3

1

2

2

Task to Solve one Puzzle

class PuzzleSolver implements Callable<Solvability> {

private final SolverType solverType; private final PuzzleInfo info;

public PuzzleSolver (SolverType s, PuzzleInfo info) { … }

public Solvability call () { … }}


Submit Tasks to Executorprotected boolean solveOneSetOfPuzzles(SolverType s)

{ ExecutorService exec = Executors.newSingleThreadExecutor();

for (PuzzleInfo info : puzzles) { Callable<Solvability> task = new PuzzleSolver(s, info); exec.submit(task); } exec.shutdown();

// Error handling}


Use a Thread Pool

protected boolean solveOneSetOfPuzzles(SolverType s) {// ExecutorService exec = // Executors.newSingleThreadExecutor(); ExecutorService exec = Executors.newFixedThreadPool(numberOfThreads);


Summary

• Step 1: Establish Performance Criteria• Step 2: Annotate Existing Class Thread

Safety• Step 3: Increase Thread Safety (as

needed)• Step 4: Use Tasks instead of loops/steps• Step 5: Use Thread Pool• Step 6: Increase Concurrency


Ken Wadland, PhD Refactoring to Concurrency (Java Edition) Prepared for: Boston Java Meetup Group February 2011.

Documents

concurrency magic slide

value slide

concurrency copyright

algorithm copyright

thread algorithm

immutable slide

threadneutral slide

invalid slide