Top Banner
On-heap cache vs Off-heap cache Radek Grębski @RadekGrebski https://github.com/rgrebski
39

On heap cache vs off-heap cache

Jan 15, 2017

Download

Software

rgrebski
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
  • On-heap cache vs Off-heap cacheRadek Grbski

    @RadekGrebskihttps://github.com/rgrebski

  • PRESENTATION PLAN

    1. Heap vs off-heap2. Heap memory3. Off-heap memory

    3.1 Memory mapped file3.2 Unsafe and ByteBuffers3.3 Off-heap memory advantages

    4. Cache:4.1 Chronicle4.2 Hazelcast4.3 Redis

    5. Comparison

  • 1. HEAP VS OFF-HEAP

    vs

  • 2. HEAP MEMORY

    JVM memoryJDK8

    S1Eden S0 Old Generation

    Heap (-Xmx)

    Metaspace(-XX:MaxMetaspaceSize)

    Native memory

    JVM Process memory

    OS memory

    Young Generation

  • Objects on heap

    Source: http://www.ibm.com/developerworks/library/j-codetoheap/

    Integer (64-bit JVM): 7:1 (28 bytes)

    9:1 (36 bytes)

    Integer (32-bit JVM):

    3:1 overhead ratio

    16 bytes

    String (32-bit JVM):

    3.75:1

    60 bytes

    int[1]

    2. HEAP MEMORY

  • 3. OFF-HEAP MEMORY

    Off-heap (native) memoryJVM Process

    OS memory

    Other processes / unallocated

    byte[]

    byte[]

  • 3.1. MEMORY MAPPED FILE

    Off-heap memoryJVM Process

    OS memory

    JVM Process

    /tmp/myFile.dat

    byte[]

    byte[] byte[]

    byte[]

  • 3.2. UNSAFE AND BYTEBUFFERS

    How to allocate memory using standard Java classes?

    java.nio.ByteBuffer: HeapByteBuffer (on-heap, up to 2gb) DirectByteBuffer (off-heap, up to 2gb) MappedByteBuffer (off-heap, up to 2gb, persisted)

    sun.misc.Unsafe public native long allocateMemory(long allocationSizeInBytes);

  • 3.2. UNSAFE AND BYTEBUFFERS

    ByteBuffer.allocate ( 2(char) + 4(integer)

    byte[] bytesToBeReadInto = new byte["test".getBytes("UTF-8").length];

    char charA = byteBuffer.getChar(/*address*/ 0); // 'a'

    int int123 = byteBuffer.getInt(/*address*/ 2); //123

    byteBuffer.position(6); //set cursor position

    byteBuffer.get(bytesToBeReadInto); //"test" as byte[] read into "bytesToBeReadIntoRead"

  • 3.2. UNSAFE AND BYTEBUFFERS

    ByteBuffer.allocateDirect (off-heap)

    JvmUtils.verifyJvmArgumentsPresent(of("-Xmx64m", "-XX:MaxDirectMemorySize=2g"));

    long capacity2GB = (2 * ByteUtil.GB) - 1;

    assertThat(capacity2GB).isLessThanOrEqualTo(Integer.MAX_VALUE);

    //after this line process memory grows up to 2GB+

    ByteBuffer byteBuffer = ByteBuffer.allocateDirect((int) capacity2GB)

    .putChar('a') //2bytes, position =0

    .putInt(123); //4bytes, position = 2(0 + 2(char))

    char charA = byteBuffer.getChar(/*address*/ 0); // 'a'

    int int123 = byteBuffer.getInt(/*address*/ 2); //123

    //perform "GC" on created byte buffer (

    ByteBufferUtils.callCleaner(byteBuffer);

    byteBuffer.putInt(1);

    // # A fatal error has been detected by the Java Runtime Environment:

    // #

    // # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000000006fca432d, pid=9356, tid=11152

  • 3.2. UNSAFE AND BYTEBUFFERS

    Unallocating ByteBuffer memory

    public static void callCleaner(ByteBuffer byteBuffer){

    //DirectByteBuffer.cleaner().clean()

    Method cleanerMethod = byteBuffer.getClass().getMethod("cleaner");

    cleanerMethod.setAccessible(true);

    Object cleaner = cleanerMethod.invoke(byteBuffer);

    Method cleanMethod = cleaner.getClass().getMethod("clean");

    cleanMethod.setAccessible(true);

    cleanMethod.invoke(cleaner);

    }

  • 3.2. UNSAFE AND BYTEBUFFERS

    Allocating memory using sun.misc.Unsafe

    sun.misc.Unsafe::getUnsafe():

    public static Unsafe getUnsafe() {

    Class cc = sun.reflect.Reflection.getCallerClass(2);

    if (cc.getClassLoader() != null)

    throw new SecurityException("Unsafe");

    return theUnsafe;

    }

    Creating an instance of Unsafe:

    public static Unsafe createUnsafe() {

    Constructor unsafeConstructor = Unsafe.class.getDeclaredConstructor();

    unsafeConstructor.setAccessible(true);

    return unsafeConstructor.newInstance();}

    Memory allocation:

    long memorySizeInBytes = ByteUtil.GB * 4;

    long startAddress = unsafe.allocateMemory(memorySizeInBytes);

    unsafe.setMemory(startAddress, memorySizeInBytes, (byte) 0);

    unsafe.freeMemory(startAddress);

  • 3.2. UNSAFE AND BYTEBUFFERS

    Memory mapped file using Java API

    File mappedFile = new File("/tmp/mappedFile.tmp");

    mappedFile.delete();

    try (FileChannel fileChannel = new RandomAccessFile(mappedFile, "rw").getChannel()) {

    long buffer8MB = 8 * ByteUtil.MB;

    MappedByteBuffer mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, 0,

    buffer8MB);

    long startAddress = 0;

    long elementsToPut = 200_000_000;

    for (long counter = 0; counter < elementsToPut; counter++) {

    if (!mappedByteBuffer.hasRemaining()) {

    startAddress += mappedByteBuffer.position();

    mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, startAddress,

    buffer8MB);

    }

    mappedByteBuffer.putLong(counter);

    }

    }Time: 1,068 sFilesize: 1,49 GB

  • 3.2. UNSAFE AND BYTEBUFFERS

    Memory mapped file content

  • 3.2. UNSAFE AND BYTEBUFFERS

    Big endian vs Little endian

    0xCAFEBABE

    Address 00 01 02 03

    Big endian CA FE BA BE

    Address 00 01 02 03

    Little endian BE BA FE CA

  • 3.2. UNSAFE AND BYTEBUFFERS

    ByteBuffer endianess

    @Test

    public void testEndianess() throws NoSuchMethodException, IllegalAccessException,

    InvocationTargetException {

    ByteBuffer bigEndianByteBuffer = ByteBuffer.allocate(4);

    bigEndianByteBuffer.order(ByteOrder.BIG_ENDIAN);

    ByteBuffer littleEndianByteBuffer = ByteBuffer.allocate(4);

    littleEndianByteBuffer.order(ByteOrder.LITTLE_ENDIAN);

    littleEndianByteBuffer.putInt(0xCAFEBABE);

    bigEndianByteBuffer.putInt(0xCAFEBABE);

    String bigEndianHexString = Hex.encodeHexString(bigEndianByteBuffer.array());

    String littleEndianHexString = Hex.encodeHexString(littleEndianByteBuffer.array());

    assertThat(bigEndianHexString).isEqualToIgnoringCase("CAFEBABE");

    assertThat(littleEndianHexString).isEqualToIgnoringCase("BEBAFECA");

    }

  • 3.3. OFF-HEAP MEMORY ADVANTAGES

    Off-heap storagePros and cons

    No GC! Manual GC! Memory leaks Persistence IPC Latency Durability Scalability (1TB+)

  • 4. CACHE

    HashMap Collections.synchronizedMap(..) ConcurrentHashMap

  • 4.1. CHRONICLE

    http://chronicle.software

    Chronicle Map Chronicle Set Chronicle Queue

    Chronicle Logger Java Thread Affinity

  • Features

    No GC! No low-level coding Persistence Shared between JVMs Replication Loading of Chronicle Map ~10ms Concurrent across processes Synchronization is on CPU level Uses proxy Zero-copy getUsing(key, existingValueObject) close()

    4.1. CHRONICLE MAP

  • 4.1. CHRONICLE MAP

    Creating off-heap map instance

    private Map chronicleMap;

    private static final String KEY_SAMPLE = "12345678901234567890";

    private static final long MAX_ENTRIES = 10_000_000;

    {

    chronicleMap = ChronicleMapBuilder.of(String.class, OffHeapUser.class)

    .averageKeySize(KEY_SAMPLE.getBytes("UTF-8").length)

    .constantValueSizeBySample(new OffHeapUserSample())

    .createPersistedTo(new File("/tmp/mappedFile.bin"))

    .entries((long) (MAX_ENTRIES))

    .create();

    }

  • 4.1. CHRONICLE MAP

    Complex structures

    public interface OffHeapUser {

    String getUsername();

    void setUsername(@MaxSize(30) String username);

    long getAccountValidUntil();

    void setAccountValidUntil(long accountValidUntil);

    void setRoleAt(@MaxSize(2) int index, Role role);

    Role getRoleAt(@MaxSize(2) int index);

    static interface Role {

    String getRole();

    void setRole(@MaxSize(10) String role);

    }

    }

  • 4.1. CHRONICLE MAP

    Creating value instance

    //fill the data

    long accountValidUntil = System.currentTimeMillis() + YEAR_IN_MILLIS;

    String username = RandomStringUtils.randomAlphabetic(20);

    OffHeapUser offHeapUser = chronicleMap.newValueInstance();

    offHeapUser.setAccountValidUntil(accountValidUntil);

    offHeapUser.setUsername(username);

    OffHeapUser.Role role0 = offHeapUser.getRoleAt(0);

    role0.setRole("Role0");

    OffHeapUser.Role role1 = offHeapUser.getRoleAt(1);

    role0.setRole("Role1");

    //put

    chronicleMap.put("someKey", offHeapUser);

    //get

    OffHeapUser offHeapUserActual = chronicleMap.get("someKey");

  • 4.1. CHRONICLE MAP

    Throughput

    Key = u:0123456789, value = counter

    *ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap.Source: https://github.com/OpenHFT/Chronicle-Map

    0

    20

    40

    60

    80

    100

    120

    140

    160

    180

    10 000 000 50 000 000 250 000 000 1 250 000 000

    Th

    rou

    ghp

    ut

    Mu

    pd

    /s

    Map entries

    Throughput - ChronicleMap vs ConcurrentHashMap

    Cronicle Map

    ConcurrentHashMap

    OutOfMemory

  • 4.1. CHRONICLE MAP

    Memory usage

    OutOfMemory

    0,0

    20,0

    40,0

    60,0

    80,0

    100,0

    120,0

    140,0

    10 000 000 50 000 000 250 000 000 1 250 000 000

    Mem

    ory

    in G

    B

    Map entries

    Memory used - ChronicleMap vs ConcurrentHashMap

    Cronicle Map

    ConcurrentHashMap

    Key = u:0123456789, value = counter

    *ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap.Source: https://github.com/OpenHFT/Chronicle-Map

  • 4.1. CHRONICLE MAP

    GC pauses

    0,0

    5,0

    10,0

    15,0

    20,0

    25,0

    30,0

    35,0

    40,0

    45,0

    50,0

    10 000 000 50 000 000 250 000 000 1 250 000 000

    Wo

    rst

    GC

    pa

    use

    in

    se

    con

    ds

    Map entries

    Worst GC pause [s] - ChronicleMap vs ConcurrentHashMap

    Cronicle Map

    ConcurrentHashMap

    OutOfMemory

    Key = u:0123456789, value = counter

    *ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap.Source: https://github.com/OpenHFT/Chronicle-Map

  • 4.1. CHRONICLE

    Chronicle Map250M entries comparison

    Source: https://github.com/OpenHFT/Chronicle-Map

    250 Million entries ConcurrentHashMap ChronicleMap

    Throughput 120million updates/s 30 million updates/s

    Worst case latency 17s 160s (avg 0,3s)

    Throughput persisted NA 28 million updates/s

    Maximum entries(128 GB)

    400 million 2500 million(32mb heap)

  • 4.2. HAZELCAST

    On-heap cache Off-heap support (High Density Memory - commercial) List, Map, Set, Queue Topics Executor Service Dynamic clustering Transactional

  • 4.2. HAZELCAST

    Example

    @Test

    public void hazelcastClusterTest(){

    Config hazelcastConfig = new Config();

    HazelcastInstance hazelcastInstance1 = Hazelcast.newHazelcastInstance(hazelcastConfig);

    HazelcastInstance hazelcastInstance2 = Hazelcast.newHazelcastInstance(hazelcastConfig);

    Map node1Map = hazelcastInstance1.getMap("someMapName");

    Map node2Map = hazelcastInstance2.getMap("someMapName");

    node1Map.put("key", "value");

    Assertions.assertThat(node2Map.get("key")).isEqualTo("value");

    }

    Output:

    Members [2] {

    Member [192.168.1.23]:5701 this

    Member [192.168.1.23]:5702

    }

  • 4.3. REDIS

    REmote DIctionary Server Key/Value cache + store ANSI C Used by:

    StackOverflow GitHub Twitter Instagram Alibaba

    Clients for almost all programming languages

  • 4.3. REDIS

    Redis data types

    HashMaps (hset, hget) LinkedList (lpush, ltrim, lrange) Queue (lpush, rpop, brpop) Topics (publish, subscribe) Set (sadd, srem) SortedSet (zadd, zrem)

  • 4.3. REDIS

    Redis Java clients

    Jedis

    Redisson

    Aredis JDBC-Redis Jredis Lettuce RedisClient

  • 4.3. REDIS

    Redis HashMap example

    @Test

    public void testRedisMap(){Jedis jedis = new Jedis("localhost");

    // Pipeline pipeline = jedis.pipelined();

    // pipeline.multi();

    jedis.hset(/*map name*/ "user:1", /*key*/ "firstName", /*value*/"Radek");

    jedis.hset("user:1", "lastName", "Grebski");

    jedis.hset("user:1", "email", "rgrebski@gmail.com");

    // pipeline.sync()

    Map mapFromRedis = jedis.hgetAll("user:1");

    assertThat(jedis.hget("user:1", "firstName")).isEqualTo("Radek");

    assertThat(mapFromRedis)

    .hasSize(3)

    .contains(entry("firstName", "Radek"))

    .contains(entry("lastName", "Grebski"))

    .contains(entry("email", "rgrebski@gmail.com"));

    }

  • 5. COMPARISON

    Chronicle Hazelcast Redis

    Deployment model Embedded Embedded / Separate Separate

    Replication Yes Yes Master/Slave

    Topics (pub/sub) No Yes Yes

    Executor Service No Yes No

    Persistence Yes (Memory mapped file) Yes (db, file, custom) Yes (periodically)

    Java collections support Yes Yes Yes (Redisson)

    Clustering No Yes Yes (since 3.0)

    Distributed events No Yes Yes

    Features

  • 5. COMPARISON

    R/W Performance

    Chronicle Hazelcast Redis ConcurrentHashMap

    Write (2mln entries) 2 555 583 178 491 110 668 1 100 715

    Read (2mln entries) 937 207 214 638 156 177 4 092 769

    0

    500 000

    1 000 000

    1 500 000

    2 000 000

    2 500 000

    3 000 000

    3 500 000

    4 000 000

    4 500 000

    Op

    s/s

    R/W Performance

    Heap: 1gb, G1 GC, Threads: 2, Records: 2mln, Entries: counter, User(username, 2x role, long)

  • 5. PERFORMANCE COMPARISON

    GC Pauses

    Heap: 1gb, G1 GC, Threads: 2, Records: 2mln, Entries: counter, User(username, 2x role, long)

    Chronicle Hazelcast Redis ConcurrentHashMap

    GC pauses summarized 39 1452 480 726

    GC pause max 8 28 15 360

    0

    200

    400

    600

    800

    1000

    1200

    1400

    1600

    Tim

    e in

    mill

    is

  • Which cache should I use in my app ?

  • Questions?

  • Thank You !

    Radek Grbski@RadekGrebski

    https://github.com/rgrebski