Top Banner

Click here to load reader

Heap Off Memory WTF ?? Reducing Heap memory stress · PDF fileAbstract * Java memory fundamental * heap off memory principles * heap-off cache with Apache DirectMemory

Feb 07, 2018

ReportDownload

Documents

doanthien

  • Heap Off Memory

    WTF??

    Reducing Heap memory stress

  • Abstract

    * Java memory fundamental

    * heap off memory principles

    * heap-off cache with Apache DirectMemory

  • /me

    Olivier Lamy

    * Open Source Architect @Talend Apache Team* Apache Member: Maven, Archiva, Tomcat, DirectMemory, Build Infra, etc..* Jenkins* Add bugs in various OpenSource projects* twitter.com/olamy olamy.blogspot.com

  • Java Memory Fundamental

  • Java Memory Fundamental

    The cool part

    * Automatic memory allocation (do you remember when you used malloc)

    * Garbage collector (GC) (no more free)

  • Java Memory Fundamental

    The bad part

    * stop-the-world mode

    * proportionnal to memory's size

    * unpredictable unresponsive application: complicated with tight SLA

  • Heap Off Memory

    Not in the Heap process

    Limited GC stop

    Unlimited storage

  • Heap Off MemoryHow to use that?

    Native libraries with JNAhttps://github.com/twall/jna

    ByteBuffer @since 1.4 with nioByteBuffer.allocate( size ) (not off heap)

    ByteBuffer.allocateDirect( size )

    You can use sun.misc.Unsafe but can cause issue and not portable

    https://github.com/twall/jnahttps://github.com/twall/jna

  • Heap Off MemoryHow to use that?

    You can manipulate only Buffer (ByteBuffer byte[] etc..)!

    So you must serialize/deserialize your datas

    Maximum size with -XX:MaxDirectMemorySize=

  • Our use case: Cache

    RAM:10-60 nsNetwork: 10000-30000 nsSSD Disk: 70000-120000 nsDisk: 3000000-10000000 ns

    Cache must use memory!

  • On-Heap Cache

    Objects stored by reference (no de/serialisation)

    GC storm effect when refreshing/removing objects!

  • Off-Heap Cache

    de/serialisation overhead (hopefully some very performant libraries exist for that)

    Cache objects payload no more affecting GC

  • Solutions:

    * Terracotta BigMemory (off-heap storage on top of ehcache)

    * Infinispan (by Jboss)

    * Huge collections

    * Apache DirectMemory

  • Apache Direct Memory

    Goals:

    Apache Direct Memory is a multi layered cache implementation featuring off-heap memory storage to enable caching of java objects without degrading

    jvm performance.

  • Apache Direct Memory

    Joined Apache Incubator end 2011 12 developpers ATM Under development :

    Memory allocation service just rewrite APIs are subject to be changed and bugs to be

    found !

  • Design & principles

    ByteBuffer.allocateDirect is the foundation of the cache

    ByteBuffers are allocated in big chunk and splitted for internal use

  • Design & principlesBuild on layers CachingService : serialize object

    (pluggable)

    MemoryManagerService: Compute ByteBuffer access

    ByteBufferAllocatorService: Eventually deals with ByteBuffer

  • ByteBuffers Allocation

    Merging Strategy No memory wasted Suffers from fragmentation Need synchronization at de/allocation

    Fixed Size buffers allocation Memory wasted if size not correctly configured No fragmentation

  • Use case (1)Multi layers cache

    Most used objects are cached on heap, the rest off-heap (maybe overflow to disk)

    Sounds like ehcache with BigMemory Hard coded class to use :

    net.sf.ehcache.store.offheap.OffHeapStore So same package class name in Apache

    DirectMemory (https://jira.terracotta.org/jira/browse/EHC-940)

    Demo

  • Use case (2)Cache Server

    la memcache tru http(s) But with a REST Api and written in Java Client API available Demo

  • > PUT /dm/cache/bordeaux HTTP/1.1> Content-Type:text/plain{"millesime":"2003","description":"so good so good"}< HTTP/1.1 200 OK< X-DirectMemory-SerializeSize: 58< Content-Length: 0> GET /dm/cache/bordeaux HTTP/1.1> Accept:text/plain< HTTP/1.1 200 OK< Content-Type: text/plain< Content-Length: 51{"millesime":"2003","description":"so good so good"}> DELETE /dm/cache/foo HTTP/1.1

  • Next Steps

    JSR 107 ? Benchmarks Components Integration (Cassandra, Tomcat etc..) Dynamic cache size modification Management/Monitoring

  • Links http://incubator.apache.org/directmemory/ https://issues.apache.org/jira/browse/DIRECTMEMORY Demo project : https://bitbucket.org/olamy/heap-off-

    memory-wtf Slides : http://www.slideshare.net/olamy

    https://issues.apache.org/jira/browse/DIRECTMEMORYhttps://issues.apache.org/jira/browse/DIRECTMEMORY

  • Thanks&&

    Questions

  • Heap Off Memory

    WTF??

    Reducing Heap memory stress

  • Abstract

    * Java memory fundamental

    * heap off memory principles

    * heap-off cache with Apache DirectMemory

  • /me

    Olivier Lamy

    * Open Source Architect @Talend Apache Team* Apache Member: Maven, Archiva, Tomcat, DirectMemory, Build Infra, etc..* Jenkins* Add bugs in various OpenSource projects* twitter.com/olamy olamy.blogspot.com

  • Java Memory Fundamental

  • Java Memory Fundamental

    The cool part

    * Automatic memory allocation (do you remember when you used malloc)

    * Garbage collector (GC) (no more free)

  • Java Memory Fundamental

    The bad part

    * stop-the-world mode

    * proportionnal to memory's size

    * unpredictable unresponsive application: complicated with tight SLA

  • Heap Off Memory

    Not in the Heap process

    Limited GC stop

    Unlimited storage

  • Heap Off MemoryHow to use that?

    Native libraries with JNAhttps://github.com/twall/jna

    ByteBuffer @since 1.4 with nioByteBuffer.allocate( size ) (not off heap)

    ByteBuffer.allocateDirect( size )

    You can use sun.misc.Unsafe but can cause issue and not portable

  • Heap Off MemoryHow to use that?

    You can manipulate only Buffer (ByteBuffer byte[] etc..)!

    So you must serialize/deserialize your datas

    Maximum size with -XX:MaxDirectMemorySize=

  • Our use case: Cache

    RAM:10-60 nsNetwork: 10000-30000 ns

    SSD Disk: 70000-120000 nsDisk: 3000000-10000000 ns

    Cache must use memory!

  • On-Heap Cache

    Objects stored by reference (no de/serialisation)

    GC storm effect when refreshing/removing objects!

  • Off-Heap Cache

    de/serialisation overhead (hopefully some very performant libraries exist for that)

    Cache objects payload no more affecting GC

  • Solutions:

    * Terracotta BigMemory (off-heap storage on top of ehcache)

    * Infinispan (by Jboss)

    * Huge collections

    * Apache DirectMemory

  • Apache Direct Memory

    Goals:

    Apache Direct Memory is a multi layered cache implementation featuring off-heap memory storage to enable caching of java objects without degrading

    jvm performance.

  • Apache Direct Memory

    Joined Apache Incubator end 2011 12 developpers ATM Under development :

    Memory allocation service just rewrite APIs are subject to be changed and bugs to be

    found !

  • Design & principles

    ByteBuffer.allocateDirect is the foundation of the cache

    ByteBuffers are allocated in big chunk and splitted for internal use

  • Design & principlesBuild on layers CachingService : serialize object

    (pluggable)

    MemoryManagerService: Compute ByteBuffer access

    ByteBufferAllocatorService: Eventually deals with ByteBuffer

  • ByteBuffers Allocation

    Merging Strategy No memory wasted Suffers from fragmentation Need synchronization at de/allocation

    Fixed Size buffers allocation Memory wasted if size not correctly configured No fragmentation

  • Use case (1)Multi layers cache

    Most used objects are cached on heap, the rest off-heap (maybe overflow to disk)

    Sounds like ehcache with BigMemory Hard coded class to use :

    net.sf.ehcache.store.offheap.OffHeapStore So same package class name in Apache

    DirectMemory (https://jira.terracotta.org/jira/browse/EHC-940)

    Demo

  • Use case (2)Cache Server

    la memcache tru http(s) But with a REST Api and written in Java Client API available Demo

  • > PUT /dm/cache/bordeaux HTTP/1.1> Content-Type:text/plain{"millesime":"2003","description":"so good so good"}< HTTP/1.1 200 OK< X-DirectMemory-SerializeSize: 58< Content-Length: 0> GET /dm/cache/bordeaux HTTP/1.1> Accept:text/plain< HTTP/1.1 200 OK< Content-Type: text/plain< Content-Length: 51{"millesime":"2003","description":"so good so good"}> DELETE /dm/cache/foo HTTP/1.1

  • Next Steps

    JSR 107 ? Benchmarks Components Integration (Cassandra, Tomcat etc..) Dynamic cache size modification Management/Monitoring

  • Links http://incubator.apache.org/directmemory/ https://issues.apache.org/jira/browse/DIRECTMEMORY Demo project : https://bitbucket.org/olamy/heap-off-

    memory-wtf Slides : http://www.slideshare.net/olamy

  • Thanks&&

    Questions

    Slide 1Slide 2Slide 3Slide 4Slide 5Slide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24Slide 1Slide 2Slide 3Slide 4Slide 5Slide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.