Blackboard DevCon 2013 - Advanced Caching in Blackboard Learn Using Redis Building Block

Blackboard LearnRedis Cache B2

Noriaki Tatsumi, Director, EngineeringBlackboard, Inc.

2

Redis Cache B2What is it?

Enables Blackboard Learn’s caching framework to use Redis as an additional type of memory store outside the Java heap.

3

RedisWhat is it?

• Open source @ http://redis.io• Networked advanced key-value store• Powerful but simple + easy• Written in ANSI C• Companies that use Redis

– Craigslist– Github– Stackoverflow– Instagram

4

Why Expand on Caching?

• The key to scaling large environment• Database is hard to scale horizontally

5

Caching in LearnEhcache RMI Replicated Caching + Local Disk Overflow

6

Redis Cache B2Ehcache RMI Replicated Caching + Redis Overflow

7

Performance TestTest case: Database is pressured

Redis Cache B2 App Nodes %Fail Total Txns95th% RespTime (s)

90th% RespTime (s) Stdev

Disabled 11 0.40 235,852 14.2 10.61 6.09

Enabled 11 0.35 276,602 9.05 7.2 4.03Delta 40,750 -5.15 -3.41 -2.06Delta (%) 17.28% -36.27% -32.14% -33.83%

95th% Resp Time(s) 90th% Resp Time(s)0

4

8

12

16Ehcache Redis Cache B2

Ehcache Redis Cache B20

50,000100,000150,000200,000250,000300,000

Total Txns

8

Performance TestTest case: JVM heap is pressured

Redis Cache B2 App Nodes %Fail Total Txns95th% RespTime (s)

90th% RespTime (s) Stdev

Disabled 11 0.61 292,864 5.86 4.41 18.49

Enabled 11 0.69 317,058 4.8 3.81 14.77Delta 24,194 -1.06 -0.6 -3.72Delta (%) 8.26% -18.09% -13.61% -20.12%

95th% Resp Time(s) 90th% Resp Time(s)0

2

4

6

Ehcache Redis Cache B2

Ehcache Redis Cache B20

50,000100,000150,000200,000250,000300,000350,000

Total Txns

9

Performance TestTest case: No pressure on any component

Redis Cache B2 App Nodes %Fail Total Txns95th% RespTime (s)

90th% RespTime (s) Stdev

Disabled 11 0.57 361,817 4.22 3.39 1.67

Enabled 11 0.47 363,220 4.02 3.28 1.69Delta 1,403 -0.20 -0.11 +0.02Delta (%) 0.39% -4.74% -3.24% -1.20%

95th% Resp Time(s) 90th% Resp Time(s)0

1

2

3

4

5Ehcache Redis Cache B2

Ehcache Redis Cache B20

100,000

200,000

300,000

400,000Total Txns

10

Every Second Counts

http://www.stevesouders.com/blog/2009/07/27/wikia-fast-pages-retain-users/

11

Redis Cache B2 in ActionNo risk of performance loss – Redis fails safe

12

Redis Cache B2 in ActionCache is shared and the hottest subset of data is kept local

13

Redis Cache B2 in ActionCache is available under JVM heap pressure

PanicRoom thread • Triggers when free heap memory is less than 129MB• Flushes cache to avoid running out of memory• Log message:

14

Redis Cache B2 in ActionKeeps a manageable JVM heap size

From performanceterracotta.blogspot.com

• GC pause increases with heap size

• Compressed oops feature limited to 32 GB heap size

15

Redis Cache B2 in ActionBigger cache and faster overflow storage

JVM Heap

Network Memory

Disk

SizeSpeed

16

Redis Cache B2 in ActionFast and efficient object graph serialization

CacheJavabytes

Kryobytes

class blackboard.data.course.Course 4506 1837class blackboard.data.navigation.CourseNavigationApplication 2551 621class blackboard.data.user.User 7304 1735class blackboard.platform.session.impl.BbSessionImpl 8107 2119

Course

CourseNav

AppUse

r

BbSessi

onImpl

040008000

Java bytes Kryo bytes

17

Redis Cache B2 in ActionAsynchronous overflow operations with write-behind strategy

Test case:10,000 serial iterationsper operation X 5 runs

Test Put: Mean(s) Put: Min(s) Put: Max(s) Put: StdDev(s)Disk overflow on SSD 3.116242472 3.053443009 3.315864087 0.11230668Redis overflow 0.033371059 0.030807671 0.038541155 0.003137204 -3.082871413 -3.022635338 -3.277322932 -0.109169476 -98.93% -98.99% -98.84% -97.21%

Put: Mean(s)

Put: Min(s)

Put: Max(s)

Put: StdDev(

s)

0

1

2

3

Disk overflow on SSDRedis overflow

18

Redis Cache B2 in ActionReduces on-heap cluster invalidation overhead

-• On-heap cache size

-• P(updates + removes)

-• Cluster invalidations

19

Redis Cache B2 in ActionCache survives application restarts and crashes

20

Monitoring and ManagementRich monitoring features for easy management

21

Monitoring and ManagementRich monitoring features for easy management

22

Monitoring and ManagementRich monitoring features for easy management

23

Monitoring and ManagementRich monitoring features for easy management

24

Monitoring and ManagementRich monitoring features for easy management

25

Monitoring and ManagementRich monitoring features for easy management

26

Summary of Redis B2

• Provides better user experience from performance and availability boost

• Easy to manage– Redis is simple and easy to install/operate– Redis cache fails safe– Redis Cache B2 has tools to monitor and diagnose problems

• When do you need it?– High cache evictions– Database pressure– JVM heap pressure

27

Release

• Date: August 2013• Requirements: 9.1 SP 11+

– Cumulative Patch 8 for 9.1 SP 11– Cumulative Patch 2 for 9.1 SP 12

28

Learn More!help.blackboard.com

• Performance, scalability, and availability highlights• Installation guide for Redis server and Redis B2• Hardening Redis server security

29

Thank You!

Noriaki TatsumiDirector, EngineeringBlackboard, Inc.noriaki.tatsumi@blackboard.com