Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster Driving high performance
Ivan Ma 11th October 2014
Open Source Tech Talk 2014.10
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 2
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster Overview
• Auto-Sharding, Multi-Master
• ACID Compliant, OLTP + Real-Time Analytics HIGH SCALE, READS +
WRITES
• Shared nothing, no Single Point of Failure
• Self Healing + On-Line Operations 99.999% AVAILABILITY
• In-Memory Optimization + Disk-Data
• Predictable Low-Latency, Bounded Access Time REAL-TIME
• Key/Value + Complex, Relational Queries
• SQL + Memcached + JavaScript + Java + HTTP/REST & C++ SQL + NoSQL
• Open Source + Commercial Editions
• Commodity hardware + Management, Monitoring Tools LOW TCO
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster Architecture
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster Scaling
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster HA
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster 7.3: 1.2 Billion UPDATEs per Minute
• NoSQL C++ API, flexaSynch benchmark
• 30 x Intel E5-2600 Intel Servers, 2 socket, 64GB
• ACID Transactions, with Synchronous Replication
0
5
10
15
20
25
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
Millio
ns o
f U
PD
AT
Es p
er
Se
co
nd
MySQL Cluster Data Nodes
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Scaling & Linear Scaling
• Partitioning & Sharding
• Push Down Query
• Explain Query
• Multi-threaded Data Node – ndbd vs ndbmtd
• SQL and NoSQL
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1
Data Node 2
F1
Primary Fragment
Secondary Fragment
Data Node 3
Data Node 4 Fx
Fx
Table T1
P2
P3
P4
Px Partition
4 Partitions * 2 Replicas = 8 Fragments
P1
Automatic Data Partitioning Nodes & Node Groups
Copyright 2011 Oracle Corporation 11
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1
Data Node 2
F1
Primary Fragment
Secondary Fragment
F1
Data Node 3
Data Node 4 Fx
Fx
Table T1
P2
P3
P4
Px Partition
4 Partitions * 2 Replicas = 8 Fragments
P1
Automatic Data Partitioning Nodes & Node Groups
Copyright 2011 Oracle Corporation 12
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1
Data Node 2
F1
Primary Fragment
Secondary Fragment
F3 F1
Data Node 3
Data Node 4 Fx
Fx
Table T1
P2
P3
P4
Px Partition
4 Partitions * 2 Replicas = 8 Fragments
P1
Automatic Data Partitioning Nodes & Node Groups
Copyright 2011 Oracle Corporation 13
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1
Data Node 2
F1 F3
Primary Fragment
Secondary Fragment
F3 F1
Data Node 3
Data Node 4 Fx
Fx
Table T1
P2
P3
P4
Px Partition
4 Partitions * 2 Replicas = 8 Fragments
P1
Automatic Data Partitioning Nodes & Node Groups
Copyright 2011 Oracle Corporation 14
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1
Data Node 2
F1 F3
Primary Fragment
Secondary Fragment
F3 F1
Data Node 3
Data Node 4
F2 F4
F4 F2
Node Group 1
Node Group 2 Fx
Fx
Table T1
P2
P3
P4
Px Partition
4 Partitions * 2 Replicas = 8 Fragments
P1
Automatic Data Partitioning Nodes & Node Groups
Copyright 2011 Oracle Corporation 15
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Scalability: Auto-Sharding
• Partitioning happens automatically & transparent to the application
• Transparency maintained during failover, upgrades and scale-out
• No need for application-layer sharding logic
• Transactional across shards (COMMIT transaction across shards/partitions)
http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster_perfomance.php
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Adaptive Query Localization Scaling Distributed Joins
• Perform Complex Queries across Shards • JOINs pushed down to data nodes
• Executed in parallel
• Returns single result set to MySQL
• Opens Up New Use-Cases
• Real-time analytics
• Recommendations engines
• Fraud-detection
mysqld
Data Nodes
mysqld
AQL
Data Nodes
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
AQL – How to Use it
•Activated when ndb_join_pushdown is on (default)
•Run ANALYZE TABLE <tab-name> on each table once
•Use EXPLAIN to see what components are being pushed down: – Extra: Child of 'd' in pushed join@1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1 Data Node 2
Need more throughput? Scale Out
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1 Data Node 2
Need more throughput? Scale Out
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1 Data Node 2
Need more throughput? Scale Out
Oops, need to increase capacity as well!
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1 Data Node 2
Scale Out
Data Node 3 Data Node 4
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
NoSQL
Simple access patterns
Compromise on consistency for performance
Ad-hoc data format
Simple operation
SQL
Complex queries with joins
ACID transactions
Well defined schemas
Rich set of tools
• No best single solution fits all • Mix and match
Still a role for SQL (RDBMS)?
Scalability
Performance
HA
Ease of use
SQL/Joins
ACID Transactions
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps
JPA
Cluster JPA
PHP Perl Python Ruby JDBC Cluster J JS Apache Memcached
MySQL JNI Node.JS mod_ndb ndb_eng
NDB API (C++)
NoSQL Access to MySQL Cluster data
MySQL Cluster Data Nodes
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster 7.4.1 DMR Available Now!
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Better performance and operational simplicity
MySQL Cluster 7.4.1 DMR
• Performance gain over 7.3
– 47% (Read-Only)
– 38% (Read-Write)
• Faster maintenance operations
– Nodal & Rolling restarts
– Upgrades
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Asynchronous replication between MySQL Clusters
• Active-Active
– Update anywhere
– Conflict detection • Application notified through exception tables
• Can opt to have conflicts resolved automatically
– Auto-conflict-resolution • Conflicting transaction and dependent ones
are rolled-back
• No changes to application schema
Active-Active Geo-Replication
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Restart Times
• Restarting data node with locally checkpointed data
– Major improvement
• Restarting data node which must recover data from peer
– Major improvement
– Further speedups to come in 7.4.X (greater parallelization)
• Upgraded/rolling restarts
– Major improvement
• Cluster shutdown and restart
– Minor improvement
What operations benefit?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle MySQL HA & Scaling Solutions MySQL
Replication MySQL Fabric
Oracle VM Template
Oracle Clusterware
Solaris Cluster
Windows Cluster
DRBD MySQL Cluster
App Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Data Layer Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Platform Support All All Linux Linux Solaris Windows Linux All
Clustering Mode Master +
Slaves Master +
Slaves Active/Pas
sive Active/Passi
ve Active/P
assive Active/Pas
sive Active/P
assive Multi-Master
Failover Time N/A Secs Secs + Secs + Secs + Secs + Secs + < 1 Sec
Scale-out Reads ✔ ✖ ✖ ✖ ✖ ✖ ✔
Cross-shard operations N/A ✖ N/A N/A N/A N/A N/A ✔
Transparent routing ✖ For HA ✔ ✔ ✔ ✔ ✔ ✔
Shared Nothing ✔ ✔ ✖ ✖ ✖ ✖ ✔ ✔
Storage Engine InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ NDB
Single Vendor Support ✔ ✔ ✔ ✔ ✔ ✖ ✔ ✔
30
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
When to Consider MySQL Cluster
• Scalability demands
– Sharding for write performance?
• Latency demands
– Cost of each millisecond?
• Uptime requirements – Cost per minute of downtime?
– Failure versus maintenance?
• Application agility
– Developer languages and frameworks?
– SQL or NoSQL?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 32