An Experimental Analysis of Different Key-Value Stores and … · 2017-03-08 · Key-Value Stores and Relational Databases David Gembalczyk Felix Martin Schuhknecht Jens Dittrich
Post on 13-Jul-2020
0 Views
Preview:
Transcript
An Experimental Analysis of DifferentKey-Value Stores
andRelational Databases
David GembalczykFelix Martin Schuhknecht
Jens Dittrich
BTW 2017
Information Systems Group, Saarland Universityinfosys.uni-saarland.de
An Experimental Analysis of DifferentKey-Value Stores
andRelational Databases
David GembalczykFelix Martin Schuhknecht
Jens Dittrich
BTW 2017
Information Systems Group, Saarland Universityinfosys.uni-saarland.de
How different are they really?
2/20
OLTP OLAP
2/20
Range QueriesPoint Queries
OLTP OLAP
2/20
Range QueriesPoint Queries
Single Entry All Entries
OLTP OLAP
2/20
Range QueriesPoint Queries
Single Entry All Entries
Relational DBs
OLTP OLAP
2/20
Range QueriesPoint Queries
Single Entry All Entries
Relational DBs Relational DBs
OLTP OLAP
2/20
Range QueriesPoint Queries
Single Entry All Entries
Relational DBs Relational DBs
OLTP OLAP
Key-Value Stores
2/20
Range QueriesPoint Queries
Single Entry All Entries
Relational DBs Relational DBs
OLTP OLAP
Key-Value Stores Key-Value Stores
2/20
Systems
3/20
Systems
3/20
Systems
3/20
Systems
3/20
Systems
3/20
HyPer(Demo 0.5)
( )
Systems
3/20
Relational Systems
4/20
Relational Systems
Primary Key A B C
1 a1 b1 c1
2 a2 b2 c2
Relational Table
4/20
Relational Systems
Primary Key A B C
1 a1 b1 c1
2 a2 b2 c2
Relational Table
PostgreSQL, MonetDB,
Hyper
4/20
Relational Systems
Primary Key A B C
1 a1 b1 c1
2 a2 b2 c2
Relational Table
PostgreSQL, MonetDB,
Hyper
N_NATIONKEY(integer) N_NAME(char(25)) N_REGIONKEY(integer) N_COMMENT(varchar(152)))
123 “Germany” 5 “some comment”
... ... ... ...
TPC-H NATION
4/20
Redis
“any string”Key Value
“any string”
5/20
Redis
“any string”Key
[ “a”, “b”, “c”, “d”]Value
5/20
Redis
“any string”Key Value
{ ( 0.5, „a“ ), { ( 1.0, „c“ ), { ( 1.5, „b“ ), { ( 2.0, „d“ ) }
5/20
Redis
“any string”Key Value
[ “1” ➞ “a“ , [ “2” ➞ “b“, [ “3” ➞ “c“, [ “4” ➞ “d“ ]
5/20
Redis
“any string”Key Value
[ “1” ➞ “a“ , [ “2” ➞ “b“, [ “3” ➞ “c“, [ “4” ➞ “d“ ]
“NATION_123”Key
Value
[ N_NAME ➞ “Germany“ , [ N_REGIONKEY ➞ “5“, [ N_COMMENT ➞ “some comment“ ]
TPC-H NATION
5/20
Aerospike
6/20
Aerospike
“any string 1”Key
6/20
Aerospike
“any string 1”Key
Value
Bin
“binA”: “a”Bin
“binB”: 42Bin
“binC”: [“a”, “b”, “c”]
6/20
Aerospike
“any string 1”Key
Value
Bin
“binA”: “a”Bin
“binB”: 42Bin
“binC”: [“a”, “b”, “c”]
Value
Bin
“binB”: “b”Bin
“sub”: {“binD”: “d”, “binE”: “e”}“any string 2”Key
6/20
Aerospike
“any string 1”Key
Value
Bin
“binA”: “a”Bin
“binB”: 42Bin
“binC”: [“a”, “b”, “c”]
Column?
Value
Bin
“binB”: “b”Bin
“sub”: {“binD”: “d”, “binE”: “e”}“any string 2”Key
6/20
Aerospike
“any string 1”Key
Value
Bin
“binA”: “a”Bin
“binB”: 42Bin
“binC”: [“a”, “b”, “c”]
Column?
Value
Bin
“binB”: “b”Bin
“sub”: {“binD”: “d”, “binE”: “e”}“any string 2”Key
6/20
Aerospike
“any string 1”Key
Value
Bin
“binA”: “a”Bin
“binB”: 42Bin
“binC”: [“a”, “b”, “c”]
SetColumn?
Value
Bin
“binB”: “b”Bin
“sub”: {“binD”: “d”, “binE”: “e”}“any string 2”Key
6/20
Aerospike
“any string 1”Key
Value
Bin
“binA”: “a”Bin
“binB”: 42Bin
“binC”: [“a”, “b”, “c”]
SetColumn?
Table?
Value
Bin
“binB”: “b”Bin
“sub”: {“binD”: “d”, “binE”: “e”}“any string 2”Key
6/20
Aerospike
“any string 1”Key
Value
Bin
“binA”: “a”Bin
“binB”: 42Bin
“binC”: [“a”, “b”, “c”]
SetColumn?
Table?
Value
Bin
“binB”: “b”Bin
“sub”: {“binD”: “d”, “binE”: “e”}“any string 2”Key
123Key
Value
Bin
N_NAME: “Germany”
Bin
N_REGIONKEY: 5
Bin
N_COMMENT: “some comment”
TPC-H NATIONSet
6/20
PostgreSQL
7/20
PostgreSQLHSTORE
Key Value
1
[ “A” ➞ “a1”, “B” ➞ “b1”, “C” ➞ “c1”]
7/20
PostgreSQLHSTORE
Key Value
1
[ “A” ➞ “a1”, “B” ➞ “b1”, “C” ➞ “c1”]
JSONB
Key Value
1
{ “A”: “a1”, “B”: “b1”, “C”: “c1”}
2
{ “A”: “a2”, “B”: “b2”, “C”: “c2”, “sub”: { “D”: “d” “E”: “e” }}
7/20
PostgreSQLHSTORE
Key Value
1
[ “A” ➞ “a1”, “B” ➞ “b1”, “C” ➞ “c1”]
JSONB
Key Value
1
{ “A”: “a1”, “B”: “b1”, “C”: “c1”}
2
{ “A”: “a2”, “B”: “b2”, “C”: “c2”, “sub”: { “D”: “d” “E”: “e” }}
Key Value
123
[ “N_NAME” ➞ “Germany”, “N_REGIONKEY” ➞ “5”, “N_COMMENT” ➞ “some comment”]
Key Value
123
[ “N_NAME”: “Germany”, “N_REGIONKEY”: 5, “N_COMMENT”: “some comment”]
7/20
Comparing Apples and Oranges
8/20
Comparing Apples and Oranges
vs
8/20
Comparing Apples and Oranges
vsRAM-disk
enlarge caches
warm-up run
8/20
Comparing Apples and Oranges
vsRAM-disk
enlarge caches
warm-up run
vs Custom Interface(s)
8/20
Comparing Apples and Oranges
vsRAM-disk
enlarge caches
warm-up run
vs Custom Interface(s)
8/20
Comparing Apples and Oranges
vsRAM-disk
enlarge caches
warm-up run
vs Custom Interface(s)
vs1 2 3 41
2
3
4
8/20
Experimental Setup
9/20
Experimental Setup
LINEITEMLINEITEMKEY
ORDERKEYPARTKEYSUPPKEY
LINENUMBERQUANTITY
EXTENDEDPRICEDISCOUNT
TAXRETURNFLAGLINESTATUSSHIPDATE
COMMITDATERECEIPTDATESHIPINSTRUCT
SHIPMODECOMMENT
PARTSUPPPARTSUPPKEY
PARTKEYSUPPKEYAVAILQTY
SUPPLYCOSTCOMMENT
modifiedTPC-H
9/20
Experimental Setup
LINEITEMLINEITEMKEY
ORDERKEYPARTKEYSUPPKEY
LINENUMBERQUANTITY
EXTENDEDPRICEDISCOUNT
TAXRETURNFLAGLINESTATUSSHIPDATE
COMMITDATERECEIPTDATESHIPINSTRUCT
SHIPMODECOMMENT
PARTSUPPPARTSUPPKEY
PARTKEYSUPPKEYAVAILQTY
SUPPLYCOSTCOMMENT
modifiedTPC-H
Scale Factor 1 9/20
Experimental Setup
LINEITEMLINEITEMKEY
ORDERKEYPARTKEYSUPPKEY
LINENUMBERQUANTITY
EXTENDEDPRICEDISCOUNT
TAXRETURNFLAGLINESTATUSSHIPDATE
COMMITDATERECEIPTDATESHIPINSTRUCT
SHIPMODECOMMENT
PARTSUPPPARTSUPPKEY
PARTKEYSUPPKEYAVAILQTY
SUPPLYCOSTCOMMENT
modifiedTPC-H
Scale Factor 1
warm-uprun
3 measurementruns
50,000 ops/run
9/20
Experimental Setup
LINEITEMLINEITEMKEY
ORDERKEYPARTKEYSUPPKEY
LINENUMBERQUANTITY
EXTENDEDPRICEDISCOUNT
TAXRETURNFLAGLINESTATUSSHIPDATE
COMMITDATERECEIPTDATESHIPINSTRUCT
SHIPMODECOMMENT
PARTSUPPPARTSUPPKEY
PARTKEYSUPPKEYAVAILQTY
SUPPLYCOSTCOMMENT
modifiedTPC-H
Scale Factor 1
warm-uprun
3 measurementruns
50,000 ops/run
Intel Xeon X5690 Intel Xeon X5690
3.47 GHz
3.47 GHz
3.47 GHz
3.47 GHz
3.47 GHz
3.47 GHz
3.47 GHz
3.47 GHz
3.47 GHz
3.47 GHz
3.47 GHz
3.47 GHz
96GBRAM
96GBRAM
9/20
Round-Trip Time
jdbc.executeQuery( "SELECT 1;" ); // Relational DBsredis.echo( "1" ); // Redisaerospike.exists( non_existing_key ); // Aerospike
0 2 4 6 8 10 12 14 16 18 20
100 000
200 000
300 000
400 000
500 000
Clients
Perform
ance
[Q/sec]
PostgreSQL MonetDBHyPer (Demo) AerospikeRedis
10/20
Round-Trip Time
jdbc.executeQuery( "SELECT 1;" ); // Relational DBsredis.echo( "1" ); // Redisaerospike.exists( non_existing_key ); // Aerospike
PostgreSQL MonetDBHyPer (Demo) AerospikeRedis
10/
0 2 4 6 8 10 12 14 16 18 20
100 000
200 000
300 000
400 000
500 000
Clients
Perform
ance
[Q/sec]
PostgreSQL MonetDBAerospike RedisNO-DB YCSBCUSTOM
PostgreSQL MonetDBAerospike RedisNO-DB YCSBCUSTOM
20
Modifying the Store
Inserts
Batch Size 1
50,000 Inserts into LINEITEM and ORDERS
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
0 2 4 6 8 10 12 14 16 18 200
100 000
200 000
300 000
Clients
Perform
ance
[ops/sec]
12/20
Inserts
Batch Size 1
50,000 Inserts into LINEITEM and ORDERS
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
0 2 4 6 8 10 12 14 16 18 200
100 000
200 000
300 000
Clients
Perform
ance
[ops/sec]
12/20
Inserts
Batch Size 1
50,000 Inserts into LINEITEM and ORDERS
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
0 2 4 6 8 10 12 14 16 18 200
100 000
200 000
300 000
Clients
Perform
ance
[ops/sec]
12/20
Inserts
Batch Size 1
50,000 Inserts into LINEITEM and ORDERS
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
0 2 4 6 8 10 12 14 16 18 200
100 000
200 000
300 000
Clients
Perform
ance
[ops/sec]
12/20
Inserts50,000 Inserts into LINEITEM and ORDERS
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Single Client
1 10 100 1 000100
1 000
10 000
100 000
Batch Size
Perform
ance
[ops/sec]
13/20
Inserts50,000 Inserts into LINEITEM and ORDERS
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Single Client
1 10 100 1 000100
1 000
10 000
100 000
Batch Size
Perform
ance
[ops/sec]
13/20
Deletes50,000 Deletes in LINEITEM and ORDERS
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Batch Size 1
0 2 4 6 8 10 12 14 16 18 200
100 000
200 000
300 000
400 000
Clients
Perform
ance
[ops/sec]
14/20
Deletes50,000 Deletes in LINEITEM and ORDERS
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Batch Size 1
0 2 4 6 8 10 12 14 16 18 200
100 000
200 000
300 000
400 000
Clients
Perform
ance
[ops/sec]
14/20
Deletes50,000 Deletes in LINEITEM and ORDERS
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Batch Size 1
0 2 4 6 8 10 12 14 16 18 200
100 000
200 000
300 000
400 000
Clients
Perform
ance
[ops/sec]
14/20
Reading from the Store
Queries on Primary Key50,000 Point-Queries in LINEITEM and ORDERS
0 2 4 6 8 10 12 14 16 18 200
100 000
200 000
300 000
400 000
Clients
Perform
ance
[ops/sec]
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Batch Size 1
16/20
Queries on Primary Key50,000 Point-Queries in LINEITEM and ORDERS
0 2 4 6 8 10 12 14 16 18 200
100 000
200 000
300 000
400 000
Clients
Perform
ance
[ops/sec]
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Batch Size 1
16/20
Queries on Primary Key50,000 Point-Queries in LINEITEM and ORDERS
0 2 4 6 8 10 12 14 16 18 200
100 000
200 000
300 000
400 000
Clients
Perform
ance
[ops/sec]
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Batch Size 1
16/20
Queries on Primary Key50,000 Point-Queries in LINEITEM and ORDERS
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Single Client
1 10 100 1 000
10 000
100 000
Batch Size
Perform
ance
[ops/sec]
17/20
Queries on Primary Key50,000 Point-Queries in LINEITEM and ORDERS
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Single Client
1 10 100 1 000
10 000
100 000
Batch Size
Perform
ance
[ops/sec]
17/20
Queries on Primary Key50,000 Point-Queries in LINEITEM and ORDERS
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Single Client
1 10 100 1 000
10 000
100 000
Batch Size
Perform
ance
[ops/sec]
17/20
Queries on Secondary Index
1 10 100 1 0000
20 000
40 000
60 000
80 000
100 000
Range Size
Perform
ance
[ops/sec]
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
SELECT MAX([ ps_supplycost | o_totalprice | l_discount ]) FROM [PARTSUPP | ORDERS | LINEITEM]
8 Clients
18/20
Queries on Secondary Index
1 10 100 1 0000
20 000
40 000
60 000
80 000
100 000
Range Size
Perform
ance
[ops/sec]
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
SELECT MAX([ ps_supplycost | o_totalprice | l_discount ]) FROM [PARTSUPP | ORDERS | LINEITEM]
WHERE [ ps_partkey | o_custkey | l_orderkey ] = value if Range Size = 1
8 Clients
18/20
Queries on Secondary Index
1 10 100 1 0000
20 000
40 000
60 000
80 000
100 000
Range Size
Perform
ance
[ops/sec]
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
SELECT MAX([ ps_supplycost | o_totalprice | l_discount ]) FROM [PARTSUPP | ORDERS | LINEITEM]
WHERE [ ps_partkey | o_custkey | l_orderkey ] = value if Range Size = 1
WHERE start <= [ ps_partkey | o_custkey | l_orderkey ] AND [ ps_partkey | o_custkey | l_orderkey ] <= end if Range Size ≥ 1
8 Clients
18/20
Queries on Secondary Index
1 10 100 1 0000
20 000
40 000
60 000
80 000
100 000
Range Size
Perform
ance
[ops/sec]
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
SELECT MAX([ ps_supplycost | o_totalprice | l_discount ]) FROM [PARTSUPP | ORDERS | LINEITEM]
WHERE [ ps_partkey | o_custkey | l_orderkey ] = value if Range Size = 1
WHERE start <= [ ps_partkey | o_custkey | l_orderkey ] AND [ ps_partkey | o_custkey | l_orderkey ] <= end if Range Size ≥ 1
8 Clients
18/20
Complex Analytical Queries
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Q01 Q060.01
0.1
1
10
100
TPC-H Query
Duration
[sec]
19/20
Complex Analytical Queries
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Q01 Q060.01
0.1
1
10
100
TPC-H Query
Duration
[sec]
19/20
Complex Analytical Queries
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Q01 Q060.01
0.1
1
10
100
TPC-H Query
Duration
[sec]
19/20
Complex Analytical Queries
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
PG-row PG-hstore PG-jsonb MonetDB HyPer (Demo)Aerospike Redis
Q01 Q060.01
0.1
1
10
100
TPC-H Query
Duration
[sec]
19/20
Conclusion
20/20
Conclusion
UDF support
20/20
Conclusion
UDF support
secondary index support
20/20
Conclusion
UDF support
secondary index support
build-in data structures
20/20
Conclusion
UDF support
secondary index support
bulk operations
build-in data structures
20/20
Conclusion
UDF support
secondary index support
bulk operations
build-in data structures
HSTORE
20/20
Conclusion
UDF support
secondary index support
bulk operations
build-in data structures
HSTOREJSONB
20/20
top related