Heap Off Memory
WTF ??
Reducing Heap memory stress
Abstract
* Java memory fundamental
* heap off memory principles
* heap-off cache with Apache DirectMemory
/me
Olivier Lamy
* Open Source Architect @Talend Apache Team* Apache Member : Maven, Archiva, Tomcat, DirectMemory, Build Infra, etc..* Jenkins* Add bugs in various OpenSource projects* twitter.com/olamy olamy.blogspot.com
Java Memory Fundamental
Java Memory Fundamental
The cool part
* Automatic memory allocation (do you remember when you used malloc)
* Garbage collector (GC) (no more free)
Java Memory Fundamental
The bad part
* stop-the-world mode
* proportionnal to memory's size
* unpredictable unresponsive application : complicated with tight SLA
Heap Off Memory
Not in the Heap process
Limited GC stop
« Unlimited storage »
Heap Off MemoryHow to use that ?
Native libraries with JNAhttps://github.com/twall/jna
ByteBuffer @since 1.4 with nioByteBuffer.allocate( size ) (not off heap)
ByteBuffer.allocateDirect( size )
You can use sun.misc.Unsafe but can cause issue and not portable
Heap Off MemoryHow to use that ?
You can manipulate only Buffer (ByteBuffer byte[] etc..) !
So you must serialize/deserialize your datas
Maximum size with -XX:MaxDirectMemorySize=
Our use case : Cache
RAM : 10-60 nsNetwork : 10000-30000 nsSSD Disk : 70000-120000 nsDisk : 3000000-10000000 ns
Cache must use memory !
On-Heap Cache
Objects stored by reference (no de/serialisation)
GC storm effect when refreshing/removing objects!
Off-Heap Cache
de/serialisation overhead (hopefully some very performant libraries exist for that)
Cache objects payload no more affecting GC
Solutions :
* Terracotta BigMemory (off-heap storage on top of ehcache)
* Infinispan (by Jboss)
* Huge collections
* Apache DirectMemory
Apache Direct Memory
Goals :
Apache Direct Memory is a multi layered cache implementation featuring off-heap memory storage to enable caching of java objects without degrading
jvm performance.
Apache Direct Memory
● Joined Apache Incubator end 2011● 12 developpers ATM● Under development :
– Memory allocation service just rewrite
– APIs are subject to be changed and bugs to be found !
Design & principles
● ByteBuffer.allocateDirect is the foundation of the cache
● ByteBuffers are allocated in big chunk and splitted for internal use
Design & principlesBuild on layers
– CachingService : serialize object (pluggable)
– MemoryManagerService: Compute ByteBuffer access
– ByteBufferAllocatorService: Eventually deals with ByteBuffer
ByteBuffers Allocation
Merging Strategy – No memory wasted
– Suffers from fragmentation
– Need synchronization at de/allocation
Fixed Size buffers allocation– Memory wasted if size not correctly configured
– No fragmentation
Use case (1)Multi layers cache
● Most used objects are cached on heap, the rest off-heap (maybe overflow to disk)
● Sounds like ehcache with BigMemory● Hard coded class to use :
net.sf.ehcache.store.offheap.OffHeapStore● So same package class name in Apache
DirectMemory (https://jira.terracotta.org/jira/browse/EHC-940)
● Demo
Use case (2)Cache Server
● « à la » memcache tru http(s)● But with a REST Api and written in Java● Client API available● Demo
> PUT /dm/cache/bordeaux HTTP/1.1> Content-Type:text/plain{"millesime":"2003","description":"so good so good"}< HTTP/1.1 200 OK< X-DirectMemory-SerializeSize: 58< Content-Length: 0
> GET /dm/cache/bordeaux HTTP/1.1> Accept:text/plain< HTTP/1.1 200 OK< Content-Type: text/plain< Content-Length: 51{"millesime":"2003","description":"so good so good"}> DELETE /dm/cache/foo HTTP/1.1
Next Steps
● JSR 107 ?● Benchmarks● Components Integration (Cassandra, Tomcat etc..)● Dynamic cache size modification● Management/Monitoring
Links● http://incubator.apache.org/directmemory/
● https://issues.apache.org/jira/browse/DIRECTMEMORY
● Demo project : https://bitbucket.org/olamy/heap-off-memory-wtf
● Slides : http://www.slideshare.net/olamy
Thanks&&
Questions
Heap Off Memory
WTF ??
Reducing Heap memory stress
Abstract
* Java memory fundamental
* heap off memory principles
* heap-off cache with Apache DirectMemory
/me
Olivier Lamy
* Open Source Architect @Talend Apache Team* Apache Member : Maven, Archiva, Tomcat, DirectMemory, Build Infra, etc..* Jenkins* Add bugs in various OpenSource projects* twitter.com/olamy olamy.blogspot.com
Java Memory Fundamental
Java Memory Fundamental
The cool part
* Automatic memory allocation (do you remember when you used malloc)
* Garbage collector (GC) (no more free)
Java Memory Fundamental
The bad part
* stop-the-world mode
* proportionnal to memory's size
* unpredictable unresponsive application : complicated with tight SLA
Heap Off Memory
Not in the Heap process
Limited GC stop
« Unlimited storage »
Heap Off MemoryHow to use that ?
Native libraries with JNAhttps://github.com/twall/jna
ByteBuffer @since 1.4 with nioByteBuffer.allocate( size ) (not off heap)
ByteBuffer.allocateDirect( size )
You can use sun.misc.Unsafe but can cause issue and not portable
Heap Off MemoryHow to use that ?
You can manipulate only Buffer (ByteBuffer byte[] etc..) !
So you must serialize/deserialize your datas
Maximum size with -XX:MaxDirectMemorySize=
Our use case : Cache
RAM : 10-60 nsNetwork : 10000-30000 ns
SSD Disk : 70000-120000 nsDisk : 3000000-10000000 ns
Cache must use memory !
On-Heap Cache
Objects stored by reference (no de/serialisation)
GC storm effect when refreshing/removing objects!
Off-Heap Cache
de/serialisation overhead (hopefully some very performant libraries exist for that)
Cache objects payload no more affecting GC
Solutions :
* Terracotta BigMemory (off-heap storage on top of ehcache)
* Infinispan (by Jboss)
* Huge collections
* Apache DirectMemory
Apache Direct Memory
Goals :
Apache Direct Memory is a multi layered cache implementation featuring off-heap memory storage to enable caching of java objects without degrading
jvm performance.
Apache Direct Memory
● Joined Apache Incubator end 2011● 12 developpers ATM● Under development :
– Memory allocation service just rewrite
– APIs are subject to be changed and bugs to be found !
Design & principles
● ByteBuffer.allocateDirect is the foundation of the cache
● ByteBuffers are allocated in big chunk and splitted for internal use
Design & principlesBuild on layers
– CachingService : serialize object (pluggable)
– MemoryManagerService: Compute ByteBuffer access
– ByteBufferAllocatorService: Eventually deals with ByteBuffer
ByteBuffers Allocation
Merging Strategy – No memory wasted
– Suffers from fragmentation
– Need synchronization at de/allocation
Fixed Size buffers allocation– Memory wasted if size not correctly configured– No fragmentation
Use case (1)Multi layers cache
● Most used objects are cached on heap, the rest off-heap (maybe overflow to disk)
● Sounds like ehcache with BigMemory● Hard coded class to use :
net.sf.ehcache.store.offheap.OffHeapStore● So same package class name in Apache
DirectMemory (https://jira.terracotta.org/jira/browse/EHC-940)
● Demo
Use case (2)Cache Server
● « à la » memcache tru http(s)● But with a REST Api and written in Java● Client API available● Demo
> PUT /dm/cache/bordeaux HTTP/1.1> Content-Type:text/plain{"millesime":"2003","description":"so good so good"}< HTTP/1.1 200 OK< X-DirectMemory-SerializeSize: 58< Content-Length: 0
> GET /dm/cache/bordeaux HTTP/1.1> Accept:text/plain< HTTP/1.1 200 OK< Content-Type: text/plain< Content-Length: 51{"millesime":"2003","description":"so good so good"}> DELETE /dm/cache/foo HTTP/1.1
Next Steps
● JSR 107 ?● Benchmarks● Components Integration (Cassandra, Tomcat etc..)● Dynamic cache size modification● Management/Monitoring
Links● http://incubator.apache.org/directmemory/
● https://issues.apache.org/jira/browse/DIRECTMEMORY
● Demo project : https://bitbucket.org/olamy/heap-off-memory-wtf
● Slides : http://www.slideshare.net/olamy
Thanks&&
Questions
