Heap Off Memory WTF ?? Reducing Heap memory stress · PDF fileAbstract * Java memory fundamental * heap off memory principles * heap-off cache with Apache DirectMemory

Heap Off Memory

WTF ??

Reducing Heap memory stress

Abstract

* Java memory fundamental

* heap off memory principles

* heap-off cache with Apache DirectMemory

/me

Olivier Lamy

* Open Source Architect @Talend Apache Team* Apache Member : Maven, Archiva, Tomcat, DirectMemory, Build Infra, etc..* Jenkins* Add bugs in various OpenSource projects* twitter.com/olamy olamy.blogspot.com

Java Memory Fundamental

Java Memory Fundamental

The cool part

* Automatic memory allocation (do you remember when you used malloc)

* Garbage collector (GC) (no more free)

Java Memory Fundamental

The bad part

* stop-the-world mode

* proportionnal to memory's size

* unpredictable unresponsive application : complicated with tight SLA

Heap Off Memory

Not in the Heap process

Limited GC stop

« Unlimited storage »

Heap Off MemoryHow to use that ?

Native libraries with JNAhttps://github.com/twall/jna

ByteBuffer @since 1.4 with nioByteBuffer.allocate( size ) (not off heap)

ByteBuffer.allocateDirect( size )

You can use sun.misc.Unsafe but can cause issue and not portable

Heap Off MemoryHow to use that ?

You can manipulate only Buffer (ByteBuffer byte[] etc..) !

So you must serialize/deserialize your datas

Maximum size with -XX:MaxDirectMemorySize=

Our use case : Cache

RAM : 10-60 nsNetwork : 10000-30000 nsSSD Disk : 70000-120000 nsDisk : 3000000-10000000 ns

Cache must use memory !

On-Heap Cache

Objects stored by reference (no de/serialisation)

GC storm effect when refreshing/removing objects!

Off-Heap Cache

de/serialisation overhead (hopefully some very performant libraries exist for that)

Cache objects payload no more affecting GC

Solutions :

* Terracotta BigMemory (off-heap storage on top of ehcache)

* Infinispan (by Jboss)

* Huge collections

* Apache DirectMemory

Apache Direct Memory

Goals :

Apache Direct Memory is a multi layered cache implementation featuring off-heap memory storage to enable caching of java objects without degrading

jvm performance.

Apache Direct Memory

● Joined Apache Incubator end 2011● 12 developpers ATM● Under development :

– Memory allocation service just rewrite

– APIs are subject to be changed and bugs to be found !

Design & principles

● ByteBuffer.allocateDirect is the foundation of the cache

● ByteBuffers are allocated in big chunk and splitted for internal use

Design & principlesBuild on layers

– CachingService : serialize object (pluggable)

– MemoryManagerService: Compute ByteBuffer access

– ByteBufferAllocatorService: Eventually deals with ByteBuffer

ByteBuffers Allocation

Merging Strategy – No memory wasted

– Suffers from fragmentation

– Need synchronization at de/allocation

Fixed Size buffers allocation– Memory wasted if size not correctly configured

– No fragmentation

Use case (1)Multi layers cache

● Most used objects are cached on heap, the rest off-heap (maybe overflow to disk)

● Sounds like ehcache with BigMemory● Hard coded class to use :

net.sf.ehcache.store.offheap.OffHeapStore● So same package class name in Apache

DirectMemory (https://jira.terracotta.org/jira/browse/EHC-940)

● Demo

Use case (2)Cache Server

● « à la » memcache tru http(s)● But with a REST Api and written in Java● Client API available● Demo

> PUT /dm/cache/bordeaux HTTP/1.1> Content-Type:text/plain{"millesime":"2003","description":"so good so good"}< HTTP/1.1 200 OK< X-DirectMemory-SerializeSize: 58< Content-Length: 0

> GET /dm/cache/bordeaux HTTP/1.1> Accept:text/plain< HTTP/1.1 200 OK< Content-Type: text/plain< Content-Length: 51{"millesime":"2003","description":"so good so good"}> DELETE /dm/cache/foo HTTP/1.1

Next Steps

● JSR 107 ?● Benchmarks● Components Integration (Cassandra, Tomcat etc..)● Dynamic cache size modification● Management/Monitoring

Links● http://incubator.apache.org/directmemory/

● https://issues.apache.org/jira/browse/DIRECTMEMORY

● Demo project : https://bitbucket.org/olamy/heap-off-memory-wtf

● Slides : http://www.slideshare.net/olamy

Thanks&&

Questions

Heap Off Memory

WTF ??

Reducing Heap memory stress

Abstract

* Java memory fundamental

* heap off memory principles

* heap-off cache with Apache DirectMemory

/me

Olivier Lamy

* Open Source Architect @Talend Apache Team* Apache Member : Maven, Archiva, Tomcat, DirectMemory, Build Infra, etc..* Jenkins* Add bugs in various OpenSource projects* twitter.com/olamy olamy.blogspot.com

Java Memory Fundamental

Java Memory Fundamental

The cool part

* Automatic memory allocation (do you remember when you used malloc)

* Garbage collector (GC) (no more free)

Java Memory Fundamental

The bad part

* stop-the-world mode

* proportionnal to memory's size

* unpredictable unresponsive application : complicated with tight SLA

Heap Off Memory

Not in the Heap process

Limited GC stop

« Unlimited storage »

Heap Off MemoryHow to use that ?

Native libraries with JNAhttps://github.com/twall/jna

ByteBuffer @since 1.4 with nioByteBuffer.allocate( size ) (not off heap)

ByteBuffer.allocateDirect( size )

You can use sun.misc.Unsafe but can cause issue and not portable

Heap Off MemoryHow to use that ?

You can manipulate only Buffer (ByteBuffer byte[] etc..) !

So you must serialize/deserialize your datas

Maximum size with -XX:MaxDirectMemorySize=

Our use case : Cache

RAM : 10-60 nsNetwork : 10000-30000 ns

SSD Disk : 70000-120000 nsDisk : 3000000-10000000 ns

Cache must use memory !

On-Heap Cache

Objects stored by reference (no de/serialisation)

GC storm effect when refreshing/removing objects!

Off-Heap Cache

de/serialisation overhead (hopefully some very performant libraries exist for that)

Cache objects payload no more affecting GC

Solutions :

* Terracotta BigMemory (off-heap storage on top of ehcache)

* Infinispan (by Jboss)

* Huge collections

* Apache DirectMemory

Apache Direct Memory

Goals :

Apache Direct Memory is a multi layered cache implementation featuring off-heap memory storage to enable caching of java objects without degrading

jvm performance.

Apache Direct Memory

● Joined Apache Incubator end 2011● 12 developpers ATM● Under development :

– Memory allocation service just rewrite

– APIs are subject to be changed and bugs to be found !

Design & principles

● ByteBuffer.allocateDirect is the foundation of the cache

● ByteBuffers are allocated in big chunk and splitted for internal use

Design & principlesBuild on layers

– CachingService : serialize object (pluggable)

– MemoryManagerService: Compute ByteBuffer access

– ByteBufferAllocatorService: Eventually deals with ByteBuffer

ByteBuffers Allocation

Merging Strategy – No memory wasted

– Suffers from fragmentation

– Need synchronization at de/allocation

Fixed Size buffers allocation– Memory wasted if size not correctly configured– No fragmentation

Use case (1)Multi layers cache

● Most used objects are cached on heap, the rest off-heap (maybe overflow to disk)

● Sounds like ehcache with BigMemory● Hard coded class to use :

net.sf.ehcache.store.offheap.OffHeapStore● So same package class name in Apache

DirectMemory (https://jira.terracotta.org/jira/browse/EHC-940)

● Demo

Use case (2)Cache Server

● « à la » memcache tru http(s)● But with a REST Api and written in Java● Client API available● Demo

> PUT /dm/cache/bordeaux HTTP/1.1> Content-Type:text/plain{"millesime":"2003","description":"so good so good"}< HTTP/1.1 200 OK< X-DirectMemory-SerializeSize: 58< Content-Length: 0

> GET /dm/cache/bordeaux HTTP/1.1> Accept:text/plain< HTTP/1.1 200 OK< Content-Type: text/plain< Content-Length: 51{"millesime":"2003","description":"so good so good"}> DELETE /dm/cache/foo HTTP/1.1

Next Steps

● JSR 107 ?● Benchmarks● Components Integration (Cassandra, Tomcat etc..)● Dynamic cache size modification● Management/Monitoring

Links● http://incubator.apache.org/directmemory/

● https://issues.apache.org/jira/browse/DIRECTMEMORY

● Demo project : https://bitbucket.org/olamy/heap-off-memory-wtf

● Slides : http://www.slideshare.net/olamy

Thanks&&

Questions