Redis Adaptor Swapnil Bawaskar @sbawaskar (incubating)
Redis Adaptor
Swapnil Bawaskar@sbawaskar
(incubating)
• Redis Primer • Need • Implementation • Advantages • Roadmap
Agenda
2
Redis Primer
3
Data Structures Supported • Strings • List • Hashes • Sets • SortedSets • HyperLogLog
Redis Primer
4
Redis Commands
Redis Primer
5
redis>SETkey1"value"OKredis>LPUSHkey2"value"(integer)1redis>SADDkey3"value"(integer)1redis>TYPEkey1stringredis>TYPEkey2listredis>TYPEkey3setredis>
In Redis commands • key refers to a data structure name • Command determine the data structure type
Redis Cluster
6
Back in 2014
7
Y U No
8
Geode Implementation
9
Redis Adaptor
10
RedisClient
RedisClient
RedisClient
RedisClient
Redis Server
Redis Adaptor
11
RedisClient
RedisClient
RedisClient
RedisClient
Geode Server
Drop-in Replacement
Start Using GFSH
Redis Adaptor - Usage
12
gfsh>startserver--name=server1--redis-port=11211--redis-bind-address=localhost
Creates a region type PARTITION by default • Change Using system property gemfireredis.regiontype
gfsh>startserver--name=server1--redis-port=11211--J=-Dgemfireredis.regiontype=PARTITION_PERSISTENT
• Supports all Redis data structures • Each data structure (except Strings) is
backed by a Geode Partitioned Region • which is a horizontally scalable distributed
ConcurrentHashMap • We create just one Partitioned Region for
storing all Strings • Uses Geode’s OQL and indexes
Redis Adaptor - Implementation
13
• One Partitioned Region for Each List
• Track “head” and “tail” position in the Partitioned Region itself as special entries
Future Exploration • Embed the position in “Value” • Place an index on the position,
should help with List Scan
Redis Adaptor - List
14
user dataposition
Key Value
• One Partitioned Region for Each Sorted Set
• key is user data, value is user provided score
• Index on score
Redis Adaptor - Sorted Set
15
scoreuser data
Key Value
Advantages
16
• Scalability • Server side stored procedures • Network Partitioning • WAN replication
Advantages
17
• Scale Vertically • Redis server is single threaded • Geode server supports high concurrency • Utilize CPUs on big boxes without having to
manage more instances • Scale data structures - Horizontally
Scalability
18
• Redis lost 56% writes during Network Partition*
• Geode servers fence themselves
Network Partition
19
* Aphyr - https://aphyr.com/posts/283-jepsen-redis
• Data can be replicated to other clusters over WAN
WAN Replication
20
WAN
Roadmap
21
• Implement “All” Commands • String: SETRANGE (overwrites part of string) • List: BLPOP, BRPOPLPUSH (Block/insert into another) • Sorted Set: ZINTERSTORE (intersect multiple Sorted
Sets and store result in another) • etc.
• Performance benchmark • Optimize
• Geode API for these data structures?
Roadmap
22
Questions?
23
24
Join the Apache Geode Community! • Check out http://geode.incubator.apache.org
• Subscribe: [email protected]
• Download: http://geode.incubator.apache.org/releases/