Parallel computing with GPars

Parallel Computing(with GPars)

NOV 2011

Evolution

● Old Days ○ threads○ synchronize○ locks○ wait-notify

Evolution

● Recent enhancements ○ java.util.concurrent.*○ Future task○ Executor Pools○ Blocking Queues○ ConcurrentMap

Evolution

● Emerging Technologies○ Map-Reduce○ API's (Google Collections, JSR-166)○ Non-blocking I/O ○ Parallel Friendly Languages (Scala, Clojure, Groovy)

Groovy Parallel Systems

High level abstractions for writing concurrent code!

● Code-level Helpers○ Parallel Collections○ Asynchronous Processing○ Fork/Join - Map/Reduce

● Architecture-level Concepts○ Dataflows ○ Actors

● Protecting Shared Mutable State ○ Agents

Groovy Parallel Systems

High level abstractions for writing concurrent code!

● Code-level Helpers○ Parallel Collections○ Asynchronous Processing○ Fork/Join - Map/Reduce

● Architecture-level Concepts○ Dataflows ○ Actors (For next time...)

● Protecting Shared Mutable State ○ Agents (For next time...)

DEMO 1 (Parallel encryption)

Optimal Thread Pool Size

Pool size = ???

Optimal Thread Pool Size

Pool size = Ncores

Optimal Thread Pool Size

Pool size = Ncores + 1

Optimal Thread Pool Size

Pool size = Ncores + Block. Op. Ratio

Optimal Thread Pool Size

Pool size = Ncores + wait time cpu time

DEMO 2

"Dado un rango obtener la cantidad de nr. primos agrupados por él ultimo digito"

Don't mess up with mutable shared data!

Danger

DEMO 3 (Facebook Search)

Collections● eachParallel()● eachWithIndexParallel()● collectParallel()● findAllParallel()● findAnyParallel● findParallel()● everyParallel()● anyParallel()● grepParallel()● groupByParallel()● foldParallel()● minParallel()● maxParallel()● sumParallel()● splitParallel()● countParallel()● foldParallel()

DEMO 4 (Facebook Singles)

Fork-Join

Map-Reduce

DEMO 5 (API Calls)

DEMO 6 (Multiget)

Multiget

+ Info● http://gpars.org/0.12/guide/guide/single.html● http://www.slideshare.net/paulk_asert/concurrency-with-

gpars● http://gee.cs.oswego.edu/dl/papers/fj.pdf● http://static.googleusercontent.

com/external_content/untrusted_dlcp/labs.google.com/en/us/papers/mapreduce-osdi04.pdf

● Java Concurrency in Practice (ISBN-0321349601)

Pics Credits● Stone age: http://tinyurl.com/bvnlv74● Steam train: http://tinyurl.com/6lp3krq● Speed train: http://tinyurl.com/7jjc3v5● CPU: http://www.flickr.com/photos/olivander/149865626/

@pmolnar

https://github.com/pablomolnar/gpars_samples