Transcript
On-heap cache vs Off-heap cacheRadek Grębski
@RadekGrebskihttps://github.com/rgrebski
PRESENTATION PLAN
1. Heap vs off-heap2. Heap memory3. Off-heap memory
3.1 Memory mapped file3.2 Unsafe and ByteBuffers3.3 Off-heap memory advantages
4. Cache:4.1 Chronicle4.2 Hazelcast4.3 Redis
5. Comparison
1. HEAP VS OFF-HEAP
vs
2. HEAP MEMORY
JVM memoryJDK8
S1Eden S0 Old Generation
Heap (-Xmx)
Metaspace(-XX:MaxMetaspaceSize)
Native memory
JVM Process memory
OS memory
Young Generation
Objects on heap
Source: http://www.ibm.com/developerworks/library/j-codetoheap/
Integer (64-bit JVM): 7:1 (28 bytes)
9:1 (36 bytes)
Integer (32-bit JVM):
3:1 overhead ratio
16 bytes
String (32-bit JVM):
3.75:1
60 bytes
int[1]
2. HEAP MEMORY
3. OFF-HEAP MEMORY
Off-heap (native) memoryJVM Process
OS memory
Other processes / unallocated
byte[]
byte[]
3.1. MEMORY MAPPED FILE
Off-heap memoryJVM Process
OS memory
JVM Process
/tmp/myFile.dat
byte[]
byte[] byte[]
byte[]
3.2. UNSAFE AND BYTEBUFFERS
How to allocate memory using standard Java classes?
� java.nio.ByteBuffer:� HeapByteBuffer (on-heap, up to 2gb)� DirectByteBuffer (off-heap, up to 2gb)� MappedByteBuffer (off-heap, up to 2gb, persisted)
� sun.misc.Unsafe� public native long allocateMemory(long allocationSizeInBytes);
3.2. UNSAFE AND BYTEBUFFERS
ByteBuffer.allocate ( <2GB )
JvmUtils.verifyJvmArgumentsPresent("-Xmx2g");
ByteBuffer byteBuffer = ByteBuffer.allocate((int) ByteUtil.GB);
byteBuffer.putChar('a') //2bytes, position =0
.putInt(123) //4bytes, position = 2(0 + 2(char))
.put("test".getBytes("UTF-8")); //6 => 2(char) + 4(integer)
byte[] bytesToBeReadInto = new byte["test".getBytes("UTF-8").length];
char charA = byteBuffer.getChar(/*address*/ 0); // 'a'
int int123 = byteBuffer.getInt(/*address*/ 2); //123
byteBuffer.position(6); //set cursor position
byteBuffer.get(bytesToBeReadInto); //"test" as byte[] read into "bytesToBeReadIntoRead"
3.2. UNSAFE AND BYTEBUFFERS
ByteBuffer.allocateDirect (off-heap)
JvmUtils.verifyJvmArgumentsPresent(of("-Xmx64m", "-XX:MaxDirectMemorySize=2g"));
long capacity2GB = (2 * ByteUtil.GB) - 1;
assertThat(capacity2GB).isLessThanOrEqualTo(Integer.MAX_VALUE);
//after this line process memory grows up to 2GB+
ByteBuffer byteBuffer = ByteBuffer.allocateDirect((int) capacity2GB)
.putChar('a') //2bytes, position =0
.putInt(123); //4bytes, position = 2(0 + 2(char))
char charA = byteBuffer.getChar(/*address*/ 0); // 'a'
int int123 = byteBuffer.getInt(/*address*/ 2); //123
//perform "GC" on created byte buffer (
ByteBufferUtils.callCleaner(byteBuffer);
byteBuffer.putInt(1);
// # A fatal error has been detected by the Java Runtime Environment:
// #
// # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000000006fca432d, pid=9356, tid=11152
3.2. UNSAFE AND BYTEBUFFERS
Unallocating ByteBuffer memory
public static void callCleaner(ByteBuffer byteBuffer){
//DirectByteBuffer.cleaner().clean()
Method cleanerMethod = byteBuffer.getClass().getMethod("cleaner");
cleanerMethod.setAccessible(true);
Object cleaner = cleanerMethod.invoke(byteBuffer);
Method cleanMethod = cleaner.getClass().getMethod("clean");
cleanMethod.setAccessible(true);
cleanMethod.invoke(cleaner);
}
3.2. UNSAFE AND BYTEBUFFERS
Allocating memory using sun.misc.Unsafe
sun.misc.Unsafe::getUnsafe():
public static Unsafe getUnsafe() {
Class cc = sun.reflect.Reflection.getCallerClass(2);
if (cc.getClassLoader() != null)
throw new SecurityException("Unsafe");
return theUnsafe;
}
Creating an instance of Unsafe:
public static Unsafe createUnsafe() {
Constructor<Unsafe> unsafeConstructor = Unsafe.class.getDeclaredConstructor();
unsafeConstructor.setAccessible(true);
return unsafeConstructor.newInstance();}
Memory allocation:
long memorySizeInBytes = ByteUtil.GB * 4;
long startAddress = unsafe.allocateMemory(memorySizeInBytes);
unsafe.setMemory(startAddress, memorySizeInBytes, (byte) 0);
unsafe.freeMemory(startAddress);
3.2. UNSAFE AND BYTEBUFFERS
Memory mapped file using Java API
File mappedFile = new File("/tmp/mappedFile.tmp");
mappedFile.delete();
try (FileChannel fileChannel = new RandomAccessFile(mappedFile, "rw").getChannel()) {
long buffer8MB = 8 * ByteUtil.MB;
MappedByteBuffer mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, 0,
buffer8MB);
long startAddress = 0;
long elementsToPut = 200_000_000;
for (long counter = 0; counter < elementsToPut; counter++) {
if (!mappedByteBuffer.hasRemaining()) {
startAddress += mappedByteBuffer.position();
mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, startAddress,
buffer8MB);
}
mappedByteBuffer.putLong(counter);
}
}Time: 1,068 sFilesize: 1,49 GB
3.2. UNSAFE AND BYTEBUFFERS
Memory mapped file content
3.2. UNSAFE AND BYTEBUFFERS
Big endian vs Little endian
0xCAFEBABE
Address 00 01 02 03
Big endian CA FE BA BE
Address 00 01 02 03
Little endian BE BA FE CA
3.2. UNSAFE AND BYTEBUFFERS
ByteBuffer endianess
@Test
public void testEndianess() throws NoSuchMethodException, IllegalAccessException,
InvocationTargetException {
ByteBuffer bigEndianByteBuffer = ByteBuffer.allocate(4);
bigEndianByteBuffer.order(ByteOrder.BIG_ENDIAN);
ByteBuffer littleEndianByteBuffer = ByteBuffer.allocate(4);
littleEndianByteBuffer.order(ByteOrder.LITTLE_ENDIAN);
littleEndianByteBuffer.putInt(0xCAFEBABE);
bigEndianByteBuffer.putInt(0xCAFEBABE);
String bigEndianHexString = Hex.encodeHexString(bigEndianByteBuffer.array());
String littleEndianHexString = Hex.encodeHexString(littleEndianByteBuffer.array());
assertThat(bigEndianHexString).isEqualToIgnoringCase("CAFEBABE");
assertThat(littleEndianHexString).isEqualToIgnoringCase("BEBAFECA");
}
3.3. OFF-HEAP MEMORY ADVANTAGES
Off-heap storagePros and cons
� No GC!� Manual GC!� Memory leaks� Persistence� IPC� Latency� Durability� Scalability (1TB+)
4. CACHE
� HashMap� Collections.synchronizedMap(..)� ConcurrentHashMap
4.1. CHRONICLE
http://chronicle.software
Chronicle Map Chronicle Set Chronicle Queue
Chronicle Logger Java Thread Affinity
Features
� No GC!� No low-level coding� Persistence� Shared between JVMs� Replication� Loading of Chronicle Map ~10ms � Concurrent across processes� Synchronization is on CPU level� Uses proxy� Zero-copy – getUsing(key, existingValueObject)� close()
4.1. CHRONICLE MAP
4.1. CHRONICLE MAP
Creating off-heap map instance
private Map<String, OffHeapUser> chronicleMap;
private static final String KEY_SAMPLE = "12345678901234567890";
private static final long MAX_ENTRIES = 10_000_000;
{
chronicleMap = ChronicleMapBuilder.of(String.class, OffHeapUser.class)
.averageKeySize(KEY_SAMPLE.getBytes("UTF-8").length)
.constantValueSizeBySample(new OffHeapUserSample())
.createPersistedTo(new File("/tmp/mappedFile.bin"))
.entries((long) (MAX_ENTRIES))
.create();
}
4.1. CHRONICLE MAP
Complex structures
public interface OffHeapUser {
String getUsername();
void setUsername(@MaxSize(30) String username);
long getAccountValidUntil();
void setAccountValidUntil(long accountValidUntil);
void setRoleAt(@MaxSize(2) int index, Role role);
Role getRoleAt(@MaxSize(2) int index);
static interface Role {
String getRole();
void setRole(@MaxSize(10) String role);
}
}
4.1. CHRONICLE MAP
Creating value instance
//fill the data
long accountValidUntil = System.currentTimeMillis() + YEAR_IN_MILLIS;
String username = RandomStringUtils.randomAlphabetic(20);
OffHeapUser offHeapUser = chronicleMap.newValueInstance();
offHeapUser.setAccountValidUntil(accountValidUntil);
offHeapUser.setUsername(username);
OffHeapUser.Role role0 = offHeapUser.getRoleAt(0);
role0.setRole("Role0");
OffHeapUser.Role role1 = offHeapUser.getRoleAt(1);
role0.setRole("Role1");
//put
chronicleMap.put("someKey", offHeapUser);
//get
OffHeapUser offHeapUserActual = chronicleMap.get("someKey");
4.1. CHRONICLE MAP
Throughput
Key = „u:0123456789”, value = counter
*ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap.Source: https://github.com/OpenHFT/Chronicle-Map
0
20
40
60
80
100
120
140
160
180
10 000 000 50 000 000 250 000 000 1 250 000 000
Th
rou
ghp
ut
Mu
pd
/s
Map entries
Throughput - ChronicleMap vs ConcurrentHashMap
Cronicle Map
ConcurrentHashMap
OutOfMemory
4.1. CHRONICLE MAP
Memory usage
OutOfMemory
0,0
20,0
40,0
60,0
80,0
100,0
120,0
140,0
10 000 000 50 000 000 250 000 000 1 250 000 000
Mem
ory
in G
B
Map entries
Memory used - ChronicleMap vs ConcurrentHashMap
Cronicle Map
ConcurrentHashMap
Key = „u:0123456789”, value = counter
*ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap.Source: https://github.com/OpenHFT/Chronicle-Map
4.1. CHRONICLE MAP
GC pauses
0,0
5,0
10,0
15,0
20,0
25,0
30,0
35,0
40,0
45,0
50,0
10 000 000 50 000 000 250 000 000 1 250 000 000
Wo
rst
GC
pa
use
in
se
con
ds
Map entries
Worst GC pause [s] - ChronicleMap vs ConcurrentHashMap
Cronicle Map
ConcurrentHashMap
OutOfMemory
Key = „u:0123456789”, value = counter
*ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap.Source: https://github.com/OpenHFT/Chronicle-Map
4.1. CHRONICLE
Chronicle Map250M entries comparison
Source: https://github.com/OpenHFT/Chronicle-Map
250 Million entries ConcurrentHashMap ChronicleMap
Throughput 120million updates/s 30 million updates/s
Worst case latency 17s 160µs (avg 0,3µs)
Throughput persisted NA 28 million updates/s
Maximum entries(128 GB)
400 million 2500 million(32mb heap)
4.2. HAZELCAST
� On-heap cache� Off-heap support (High Density Memory - commercial)� List, Map, Set, Queue� Topics� Executor Service� Dynamic clustering� Transactional
4.2. HAZELCAST
Example
@Test
public void hazelcastClusterTest(){
Config hazelcastConfig = new Config();
HazelcastInstance hazelcastInstance1 = Hazelcast.newHazelcastInstance(hazelcastConfig);
HazelcastInstance hazelcastInstance2 = Hazelcast.newHazelcastInstance(hazelcastConfig);
Map<String, String> node1Map = hazelcastInstance1.getMap("someMapName");
Map<String, String> node2Map = hazelcastInstance2.getMap("someMapName");
node1Map.put("key", "value");
Assertions.assertThat(node2Map.get("key")).isEqualTo("value");
}
Output:
Members [2] {
Member [192.168.1.23]:5701 this
Member [192.168.1.23]:5702
}
4.3. REDIS
� REmote DIctionary Server� Key/Value cache + store� ANSI C� Used by:
� StackOverflow� GitHub� Twitter� Instagram� Alibaba
� Clients for almost all programming languages
4.3. REDIS
Redis data types
� HashMaps (hset, hget)� LinkedList (lpush, ltrim, lrange)� Queue (lpush, rpop, brpop)� Topics (publish, subscribe)� Set (sadd, srem)� SortedSet (zadd, zrem)
4.3. REDIS
Redis Java clients
� Jedis
� Redisson
� Aredis� JDBC-Redis� Jredis� Lettuce� RedisClient
4.3. REDIS
Redis HashMap example
@Test
public void testRedisMap(){Jedis jedis = new Jedis("localhost");
// Pipeline pipeline = jedis.pipelined();
// pipeline.multi();
jedis.hset(/*map name*/ "user:1", /*key*/ "firstName", /*value*/"Radek");
jedis.hset("user:1", "lastName", "Grebski");
jedis.hset("user:1", "email", "rgrebski@gmail.com");
// pipeline.sync()
Map<String, String> mapFromRedis = jedis.hgetAll("user:1");
assertThat(jedis.hget("user:1", "firstName")).isEqualTo("Radek");
assertThat(mapFromRedis)
.hasSize(3)
.contains(entry("firstName", "Radek"))
.contains(entry("lastName", "Grebski"))
.contains(entry("email", "rgrebski@gmail.com"));
}
5. COMPARISON
Chronicle Hazelcast Redis
Deployment model Embedded Embedded / Separate Separate
Replication Yes Yes Master/Slave
Topics (pub/sub) No Yes Yes
Executor Service No Yes No
Persistence Yes (Memory mapped file) Yes (db, file, custom) Yes (periodically)
Java collections support Yes Yes Yes (Redisson)
Clustering No Yes Yes (since 3.0)
Distributed events No Yes Yes
Features
5. COMPARISON
R/W Performance
Chronicle Hazelcast Redis ConcurrentHashMap
Write (2mln entries) 2 555 583 178 491 110 668 1 100 715
Read (2mln entries) 937 207 214 638 156 177 4 092 769
0
500 000
1 000 000
1 500 000
2 000 000
2 500 000
3 000 000
3 500 000
4 000 000
4 500 000
Op
s/s
R/W Performance
Heap: 1gb, G1 GC, Threads: 2, Records: 2mln, Entries: counter, User(username, 2x role, long)
5. PERFORMANCE COMPARISON
GC Pauses
Heap: 1gb, G1 GC, Threads: 2, Records: 2mln, Entries: counter, User(username, 2x role, long)
Chronicle Hazelcast Redis ConcurrentHashMap
GC pauses summarized 39 1452 480 726
GC pause max 8 28 15 360
0
200
400
600
800
1000
1200
1400
1600
Tim
e in
mill
is
Which cache should I use in my app ?
Questions?
Thank You !
Radek Grębski@RadekGrebski
https://github.com/rgrebski
top related