Principles of Software Construction

115-214

SchoolofComputerScience

PrinciplesofSoftwareConstruction:Concurrency,Pt.3– java.util.concurrent

JoshBloch CharlieGarrod

215-214

Administrivia

• Homework5bdueTuesday11:59p.m.– TurninyourworkbyWednesday9a.m.tobeconsideredasaBestFramework

315-214

Canyoufindthebug?Publicserviceannouncement(1/1)/* From Linux 2.3.99 drivers/block/radi5.c */ static struct buffer_head * get_free_buffer(

struct stripe_head *sh, int b_size) {struct buffer_head *bh; unsigned long flags;

save_flags(flags);cli();if ((bh = sh->buffer_pool) == NULL)

return NULL;sh->buffer_pool = bh->b_next;bh->b_size = b_size; restore_flags(flags);return bh;

}

415-214

Canyouwriteaprogramtofindthebug?Publicserviceannouncement(2/2)• TakeProgramAnalysis(17-355)andlearn(e.g):– Abstractinterpretation,atheoryforreasoningaboutprogramsevenbeforeyouknowtheirinput

– Concolic testing,combinessymbolicexecutionwithrandomizedtestingtoexercisehard-to-reachcornercases

– Andmore:interprocedural analysis,control-flowanalysis,shapeanalysis,anddynamicanalysis

• Thenbuildawesometoolstofindbugs,verifysecurityproperties,andgeneratetests!

• Newcourse,Spring2017– T/Th 10:30inGHC4102Prof.JonathanAldrich

515-214

KeyconceptsfromTuesday…• Neverusewaitoutsideofawhileloop!– Thinktwicebeforeusingitatall

• Neitheranunder- noranover-synchronizerbe– Under-synchronizationcausessafety(&liveness)failures– Over-synchronizationcausesliveness(&safety)failures

• TwothingsthatIshouldhavesaidTuesday…

615-214

1.Doaslittleaspossibleinsynchronizedregions

• Getin,getdone,andgetout– Obtainlock– Examineshareddata– Transformasnecessary– Droplock

• Ifyoumustdosomethingslow,moveitoutsidesynchronizedregion

715-214

2.Avoidingdeadlock

• Deadlockcausedbyacycleinwaits-forgraph– T1:synchronized(a){ synchronized(b){ … } }– T2:synchronized(b){ synchronized(a){ … } }

• Toavoidthesedeadlocks:–Whenthreadshavetoholdmultiplelocksatthesametime,allthreadsobtainlocksinsameorder

T1 T2b

a

815-214

java.util.concurrent isBIG(1)

I. Atomicvars - java.util.concurrent.atomic– Supportvariousatomicread-modify-writeops

II. Executorframework– Tasks,futures,threadpools,completionservice,etc.

III. Locks- java.util.concurrent.locks– Read-writelocks,conditions,etc.

IV. Synchronizers– Semaphores,cyclicbarriers,countdownlatches,etc.

915-214

java.util.concurrent isBIG(2)

V. Concurrentcollections– Sharedmaps,sets,lists

VI. DataExchangeCollections– Blockingqueues,deques,etc.

VII. Pre-packagedfunctionality-java.util.arrays– Parallelsort,parallelprefix

1015-214

I.Overviewofjava.util.atomic• Atomic{Boolean,Integer,Long}

– Boxedprimitivesthatcanbeupdatedatomically• AtomicReference<T>

– Objectreferencethatcanbeupdatedatomically– CoolpatternforstatemachineAtomicReference<StateEnum>

• Atomic{Integer,Long,Reference}Array– Arraywhoseelementsmaybeupdatedatomically

• Atomic{Integer,Long,Reference}FieldUpdater– Reflection-basedutilityenablingatomicupdatestovolatilefields

• LongAdder, DoubleAdder– Highlyconcurrentsums

• LongAccumulator,DoubleAccumulator– Generalizationofadder toarbitraryfunctions(max,min,etc.)

1115-214

AtomicInteger example(review)[EJItem66]public class SerialNumber {

private static AtomicLong nextSerialNumber = new AtomicLong();

public static long generateSerialNumber() {return nextSerialNumber.getAndIncrement();

}}

1215-214

VI.ExecutorframeworkOverview

• Flexibleinterface-basedtaskexecutionfacility• Keyabstractions– Runnable,Callable<T> - kindsoftasks

• Executor – thingthatexecutestasks• Future<T> – apromisetogiveyouaT• Executorservice– Executor that– Letsyoumanagetermination– CanproduceFuture instances

1315-214

Executors – yourone-stopshopforexecutorservices• Executors.newSingleThreadExecutor()– Asinglebackgroundthread

• newFixedThreadPool(int nThreads)– Afixednumberofbackgroundthreads

• Executors.newCachedThreadPool()– Growsinresponsetodemand

1415-214

Averysimpleexecutorserviceexample

• Backgroundexecutiononalong-livedworkerthread– Tostarttheworkerthread:

ExecutorService executor = Executors.newSingleThreadExecutor();

– Tosubmitataskforexecution:executor.execute(runnable);

– Toterminategracefully:executor.shutdown(); // Allows tasks to finish

• BetterreplacementforourrunInBackground andWorkQueue examplesfrompreviouslectures.

1515-214

Otherthingsyoucandowithanexecutorservice• Waitforatasktocomplete

Foo foo = executorSvc.submit(callable).get();

• Waitforanyorallofacollectionoftaskstocompleteinvoke{Any,All}(Collection<Callable<T>> tasks)

• RetrieveresultsastaskscompleteExecutorCompletionService

• ScheduletasksforexecutioninthefutureScheduledThreadPoolExecutor

• etc.,adinfinitum

1615-214

ForkJoinPool:executorserviceforForkJoinTask instancesclass SumSqTask extends RecursiveAction {

final long[] a; final int lo, hi; long sum;SumSqTask(long[] array, int low, int high) {

a = array; lo = low; hi = high;}protected void compute() {

if (h - l < THRESHOLD) {for (int i = l; i < h; ++i)

sum += a[i] * a[i];} else {

int mid = (lo + hi) >>> 1;SumSqTask left = new SumSqTask(a, lo, mid);left.fork(); // pushes task SumSqTask right = new SumSqTask(a, mid, hi);right.compute();right.join(); // pops/runs or helps or waits sum = left.sum + right.sum;

}}

}

1715-214

II.Overviewofj.u.c.locks (1)

• ReentrantReadWriteLock– Shared/Exclusivemodelockswithtonsofoptions• Fairnesspolicy• Lockdowngrading• Interruptionoflockacquisition• Conditionsupport• Instrumentation

• ReentrantLock– LikeJava'sintrinsiclocks– Butwithmorebellsandwhistles

1815-214

Overviewofj.u.c.locks (2)

• Condition– wait/notify/notifyAllwithmultiplewaitsetsperobject

• AbstractQueuedSynchronizer– SkeletalimplementationoflocksrelyingonFIFOwaitqueue

• AbstractOwnableSynchronizer,AbstractQueuedLongSynchronizer– Moreskeletalimplementations

1915-214

ReentrantReadWriteLock exampleDoesthislookvaguelyfamiliar?private final ReentrantReadWriteLock rwl =

new ReentrantReadWriteLock();

rwl.readLock().lock();try {

// Do stuff that requires read (shared) lock} finally {

rwl.readLock().unlock();}

rwl.writeLock().lock();try {

// Do stuff that requires write (exclusive) lock} finally {

rwl.writeLock().unlock();}

2015-214

III.Overviewofsynchronizers

• CountDownLatch– Oneormorethreadstowaitforotherstocountdown

• CyclicBarrier– asetofthreadswaitforeachothertobeready

• Semaphore– Likealockwithamaximumnumberofholders(“permits”)

• Phaser – Cyclicbarrieronsteroids• AbstractQueuedSynchronizer – rollyourown!

2115-214

CountDownLatch exampleConcurrenttimer[EJItem69]public static long time(Executor executor, int nThreads,

final Runnable action) throws InterruptedException {CountDownLatch ready = new CountDownLatch(nThreads);CountDownLatch start = new CountDownLatch(1);CountDownLatch done = new CountDownLatch(nThreads);for (int i = 0; i < nThreads; i++) {

executor.execute(() -> {ready.countDown(); // Tell timer we're readytry {

start.await(); // Wait till peers are readyaction.run();

} catch (InterruptedException e) {Thread.currentThread().interrupt();

} finally {done.countDown(); // Tell timer we're done

}});}ready.await(); // Wait for all workers to be readylong startNanos = System.nanoTime();start.countDown(); // And they're off!done.await(); // Wait for all workers to finishreturn System.nanoTime() - startNanos;

}

2215-214

IV.ConcurrentCollections

• Providehighperformanceandscalability

Unsynchronized ConcurrentHashMap ConcurrentHashMapHashSet ConcurrentHashSetTreeMap ConcurrentSkipListMapTreeSet ConcurrentSkipListSet

2315-214

Youcan’t excludeconcurrentactivityfromaconcurrentcollection• Thisworksforsynchronizedcollections…

Map<String, String> syncMap =Collections.synchronizedMap(new HashMap<>());

synchronized(syncMap) {if (!syncMap.containsKey("foo"))

syncMap.put("foo", "bar");}

• Butnot forconcurrentcollections– Theydotheirowninternalsynchronization– Neversynchronizeonaconcurrentcollection!

2415-214

Concurrentcollectionshaveprepackagedread-modify-writemethods• V putIfAbsent(K key, V value)• boolean remove,(Object key, Object value)• V replace(K key, V value)• boolean replace(K key, V oldValue, V newValue)• V compute(K key, BiFunction<...> remappingFn);• V computeIfAbsent,(K key, Function<...> mappingFn)• V computeIfPresent,(K key, BiFunction<...> remapFn)• V merge(K key, V value, BiFunction<...> remapFn)

2515-214

Concurrentcollectionexample:canonicalizing mapprivate static final ConcurrentMap<String, String> map =

new ConcurrentHashMap<String, String>();

// This implementation is OK, but could be betterpublic static String intern(String s) {

String previousValue = map.putIfAbsent(s, s);return previousValue ==

null ? s : previousValue;}

2615-214

Anoptimizedcanonicalizing map[EJItem69]• ConcurrentHashMap optimizedforread– Socallget first,putIfAbsent onlyifnecessary

// Good, fast implementation! public static String intern(String s) {

String result = map.get(s);if (result == null) {

result = map.putIfAbsent(s, s);if (result == null)

result = s;}return result;

}

2715-214

ConcurrentobserverpatternrequiresopencallsThiscodeispronetolivenessandsafetyfailures!private final List<SetObserver<E>> observers =

new ArrayList<SetObserver<E>>();public void addObserver(SetObserver<E> observer) {

synchronized(observers) { observers.add(observer); }}public boolean removeObserver(SetObserver<E> observer) {

synchronized(observers) { return observers.remove(observer); }}private void notifyElementAdded(E element) {

synchronized(observers) {for (SetObserver<E> observer : observers)

observer.notifyAdded(this, element); // Callback!}

}

2815-214

Adecentsolution:snapshotiterationprivate void notifyElementAdded(E element) {

List<SetObserver<E>> snapshot = null;

synchronized(observers) {snapshot = new ArrayList<SetObserver<E>>(observers);

}

for (SetObserver<E> observer : snapshot) {observer.notifyAdded(this, element); // Open call

}}

2915-214

Abettersolution:CopyOnWriteArrayList [EJItem67]private final List<SetObserver<E>> observers =

new CopyOnWriteArrayList<SetObserver<E>>();

public void addObserver(SetObserver<E> observer) {observers.add(observer);

}public boolean removeObserver(SetObserver<E> observer) {

return observers.remove(observer);}private void notifyElementAdded(E element) {

for (SetObserver<E> observer : observers)observer.notifyAdded(this, element);

}

3015-214

V.DataexchangecollectionssummaryHoldelementsforprocessingbyanotherthread• BlockingQueue – Supportsblockingops– ArrayBlockingQueue,LinkedBlockingQueue– PriorityBlockingQueue,DelayQueue– SynchronousQueue

• BlockingDeque – Supportsblockingops– LinkedBlockingDeque

• TransferQueue – BlockingQueue inwhichproducersmaywaitforconsumerstoreceiveelements– LinkedTransferQueue

3115-214

SummaryofBlockingQueuemethods

Throwsexception Specialvalue Blocks Timesout

Insert add(e) offer(e) put(e) offer(e, time, unit)

Remove remove() poll() take() poll(time, unit)

Examine element() peek() n/a n/a

3215-214

SummaryofBlockingDequemethods• Firstelement(head)methods

• Lastelement(tail)methods

Throws exception Specialvalue Blocks Timesout

Insert addFirst(e) offerFirst(e) putFirst(e) offerFirst(e,time, unit)

Remove removeFirst() pollFirst() takeFirst() pollFirst(time,unit)Examine getFirst() peekFirst() n/a n/a

Throws exception Specialvalue Blocks Timesout

Insert addLast(e) offerLast(e) putLast(e) offerLast(e,time, unit)

Remove removeLast() pollLast() takeLast() pollLast(time,unit)

Examine getLast() peekLast() n/a n/a

3315-214

Summary

• java.util.concurrent isbigandcomplex• Butit’swelldesignedandengineered– Easytodosimplethings– Possibletodocomplexthings

• ExecutorframeworkdoesforexecutionwhatCollectionsframeworkdidforaggregation

• Thistalkjustscratchedthesurface– Butyouknowthelayofthelandandthejavadoc isgood

• Alwaysbettertousej.u.c thantorollyourown!