Scott Andreas - Garbage, Garbage Everywhere: GC Strategies for Event Processing Systems on the JVM, Boundary Tech Talks 11/17/11

Garbage, Garbage EverywhereGC Strategies for Event Processing Systems on the JVM

C. Scott AndreasPizza, Beer, and Tech TalksNovember 17, 2011

What’s ESP / CEP?

• Event Stream ProcessingSelecting events on dimensions among a stream of movingdata, maintaining them for a brief period, emitting aggregations.

• Complex Event ProcessingIdentifying correlations between events, predicting trends,and programmatically reacting to emergent trends.

ESP and Network Analytics

• Packet flows are event streams with many dimensions.

• Blast them into the engine, select over the stream, emit aggregations based on queries.

• Ipfix data flows in, JSON comes out.

Back of the Envelope

• 500 Mbps / sec data comes into the JVMxxx Mbps / sec data goes out of the JVM

• This memory must be allocated, retainedfor processing, freed, and collected.

• Actual allocation rates far higher than data in / out(Memory also used for deserializing, aggregations, etc).

Opening the Grimoire

http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html

http://stas-blogspot.blogspot.com/2011/07/most-complete-list-of-xx-options-for.html

• GC Tuning to the rescue

• Oracle guide with 84 -XX: options

• Stas’s guide: 830 options (OpenJDK debug builds)

Moment of Pause

• Don’t touch the knobs unless you need to

• Server defaults are a decent place to start for local development

• Defaults shipped with Cassandra decent for bimodal GC profiles

• Basic rule of thumb: if you’re aggressively tuning garbage collection, you can trade hours of frustration for ~10% gain

Generational Garbage Collection

• Modern JVMs divide heap space up into multiple “generations.”

• Most applications have a lot of objects which live for a very short time, and a lot which live (nearly) forever.

• Generational collection enables the JVM to collect unused memory more efficiently by avoiding unnecessarily scanning heap / object graphs for references or free regions.

first attempt: “deploy the g1”

Low-Pause Collector: The G1

v% => threshold violations. lower is better.

G1 Collector

• Hundreds of tiny 1ms collections / second rather thanParNew’s ~100 - 200ms larger collections.

• Capable of meeting ambitious pause targets.

• Powered by a gang of threads working in parallel

• ...cooperating to chew through CPU like it’s free.

second attempt:you’re gonna laugh

the unsafe

Unsafe

• A OpenJDK/HotSpot class exposing direct access to the underlying VM, OS, and memory.

• This includes the ability to allocate, manage, and free memory.

• Perhaps we can outsmart the JVM and do a better job than it!

Learned While Astray

• Finalization occurs in a single thread.

• Jumping from native finalization back into Java is expensive.

• Attempting to outsmart the garbage collector by creating hundreds of thousands of tiny ByteBuffers is...a thing.

• Java’s collectors are very good at collecting garbage.Your home-grown in-app GC go-kart is probably not.

returning to earth ::attempt 3[a]

Lessons from Science

• Your rate of “freeing” must be equal to or exceed your rate of object allocation on the heap.

• High rates of allocation speed up heap fragmentation,which compounds the problem.

• Creating less garbage reduces your rate of allocation(and freeing).

• This means less work for the garbage collector.

best way to help out the gc ::

PRODUCE LESS GARBAGE

Breaking out YourKit

attempt 3[b]:responsible tuning of the old hat

Optimizing for Infant Mortality

• Java 6 AMD64 (server) defaults to allocating1/3 of heap to the new gen, 2/3 to the old gen.

• ESP/CEP workloads place tremendous pressure on the newgen. The vast majority of objects survive less than five seconds.

• Experiment: Allocate 80% of heap to the new gen, set a higher tenuring threshold, and lean hard on the ParNew collector.

default newgen ratios in java 6

CMS Collector

• Guardian of the tenured generation, favorite workhorse for years.

• Primarily parallel, easier on the CPU than the G1.

• ...But contains a significant pause phase, is less suited to meeting low pause targets.

ParNew Collector

• Designed for the small, but works great in the large.Excellent throughput, parallel collection.

• Can collect ~5GB in ~200ms on a quad-core Xeon w/HT.

• 200ms pause every several seconds favorable compared toless frequent multi-second pauses and promotion failures.

Explosions in the Barrel

“real-time” and the jvm

Real Time and the JVM

• Real TimeAbility to meet specific targets with low variance is critical to the bare minimum functionality of the product (e.g., air bags).

• “Soft” Real TimeAbility to meet targets important but not critical. Value of system’s functionality is diminished but not eliminated by delay.

Real Time and “The Pause”

• To what extent can a system which can endure pauses of unpredictable duration be considered “real-time”?

• Is it sufficient to mitigate the frequency and duration of pauses for a system to still deliver value as “soft real-time”?

• Is the alternative worth the cost?

what does your app sound like?

Garbage, Garbage EverywhereGC Strategies for Event Processing Systems on the JVM

C. Scott AndreasPizza, Beer, and Tech TalksNovember 17, 2011