Database Cluster as a Service in OpenStackIntegrated, Scalable, Highly Available and Secure
Ivan Zora>, CTO -‐ 14.11.2013
Thursday, 14 November 13
Who is Ivan
?Thursday, 14 November 13
SkySQL•Leading provider of open source databases, services and solutions
•Home for the founders and the original developers of the core of MySQL
•The creators of MariaDB, the drop-‐in, innovative replacement of MySQL
Thursday, 14 November 13
MySQL , MariaDB and Percona Serverin the Cloud
Thursday, 14 November 13
The Status of MySQLMySQL
•From Oracle, acquired with Sun in 2010
•Trademark, code, documentation, bugs are available at *.mysql.com
•5.6 GA now•5.7 DMR available for testing
MariaDB
•5.5 GA: branch from Oracle with add-‐ons from Percona and other key contributors
•10.0 Alpha: fork from Oracle with add-‐ons and new storage engines
Percona
•5.6 GA: branch from Oracle with add-‐ons from MariaDB and other key contributors
5Thursday, 14 November 13
More in-‐depth featuresMySQL
•5.6• Scales up to 48 CPU threads• Performance Schema• Memcached plugin for InnoDB• InnoDB API• InnoDB Full Text Indexing• Replication with Global Transaction ID• HA utilities• Improved DDL online operations• Transportable tablespaces• Subquery optimization• Batched key access and multi-‐range read
• Thread pool• PAM Authentication• Audit plugin
MariaDB•5.5
• Subquery optimization• Batched key access and multi-‐range read
• Index condition pushdown• and many, many other improvements in the optimizer...
• SphinxSE and ScaleDB engines• Handler Socket• Dynamic and virtual columns• Group Commit for binary log• Microsecond Support• GIS operations• XtraDB Engine• PAM Authentication• Thread pool• Audit plugin
Percona•5.6
• Porting of the MySQL 5.6 features• XtraDB 5.6• Per-‐table/index/user/client performance counters
• FlashCache support
6Thursday, 14 November 13
More in-‐depth featuresMySQL
•5.7• Improved DDL online operations• EXPLAIN improvements• Improved triggers• MySQL Client > syslog• Parallel thread replication• EXPLAIN in JSON format• EXPLAIN for running statements• Subquery optimization• Loss-‐less MySQL Replication• Multi-‐source replication
MariaDB
•10.0• Includes XtraDB 5.6• Performance schema• Fusion I/O atomic writes• Further optimizer improvements• Improved DDL online operations• Replication with Global Transaction ID• Parallel thread replication• Multi-‐source replication• EXPLAIN for running threads and logged queries
• Engine-‐independent table statistics• Per-‐engine table attributes• Per-‐thread memory usage
• TokuDB, Spider, CASSANDRA and CONNECT storage engine
7Thursday, 14 November 13
High AvailabilityMySQL
•MySQL Replication with HA utilities
•DRBD from Linbit•Shared storage with Oracle Enterprise Linux
MariaDB•MySQL Replication with MHA and Pacemaker
•MariaDB Galera Cluster Synchronous Replication with Codership Galera
•Support from SkySQL for:•DRBD•Shared storage
Percona•MySQL Replication with Percona Replication Manager
•Percona XtraDB ClusterSynchronous Replication with Codership Galera
•Support for:•DRBD•Shared storage
8Thursday, 14 November 13
ScalabilityMySQL
• Improved InnoDB scalability•Group commit, parallel and multi-‐source replication in 5.7
•MySQL Replication for read scalability
•MySQL Cluster for high volume/small transactions
MariaDB• In MariaDB 5.5:
• XtraDB 5.5• Group commit•MySQL Replication for read scalability
• TokuDB and ScaleDB engine
• Plus, in MariaDB 10:• XtraDB 5.6• Parallel and multi-‐source replication• Spider and Cassandra engine
• MariaDB Galera Cluster for synchronous replication
• MySQL Cluster for high volume/small transactions
Percona
•XtraDB 5.6•Group commit•MySQL Replication for read scalability
•Percona XtraDB Cluster for synchronous replication
9Thursday, 14 November 13
Ease of UseMySQL
•MySQL Replication Utilities
•Enterprise Monitoring
•Online Backup•MySQL Workbench
MariaDB
•MHA
•Enterprise Monitor
•Xtrabackup•Query Editor•MariaDB Manager
Percona
•Percona Replication Manager
•Nagios and Cacti plugins•Xtrabackup•Percona Toolkit
10Thursday, 14 November 13
MySQL in Rackspace
•Rackspace Cloud Database•MySQL Community 5.1•Up to 16GB RAM, 150GB storage
•Standard Servers•Up to 8 vCPUs, 30GB RAM, 1.2TB storage•BYODBMS
11Thursday, 14 November 13
MySQL in HPCloud
•Manage a single DB instance through the REST API
•Standard Servers•Up to 8 vCPUs, 30GB RAM, 960GB storage•BYODBMS
•Marketplace•GenieDB•MariaDB ScaleDB
12Thursday, 14 November 13
MySQL in AWS
•Amazon RDS•MySQL Community 5.1, 5.5 & 5.6•Reserved instances, provisioned IO and Flash storage•32 vCPUs, 244GB Memory, 10GB network
•Automatic backups, Storage replication, MySQL Replication•Limited tuning
•Standard Servers and Marketplace•Similar instances•MariaDB on CentOS, Debian and Ubuntu•GenieDB
13Thursday, 14 November 13
MySQL and OpenStackThursday, 14 November 13
MySQL [or MariaDB, or Percona] in OpenStack
•Used as repository -‐ alternative to SQLite•Increased portability and availability
•High Availability is the key factor:•MySQL Replication•DRBD•Galera
15Thursday, 14 November 13
MySQL Replication
16
DatabaseDatabase
Database DatabaseDatabase
Database
Node Manager
HA Manager
Control Network Compute
Thursday, 14 November 13
DRBD
17
Control Network Compute
Database Database Database Database
Thursday, 14 November 13
Galera Cluster
18
Control Network Compute
Database Database DatabaseDatabase Database Database
Thursday, 14 November 13
MySQL as a ServiceThursday, 14 November 13
Automatic Provisioning
•REST-‐based for interoperation
•GUI-‐assisted for end-‐users
•From server to cluster
20Thursday, 14 November 13
The false promise of the elasticity•Distributed MySQL is not something you can have out of the box•Or not for all your applications
21
Read-Write
Read-Only Read-Only
binlog
relaylog relaylog relaylog relaylog
DatabaseDatabase
Database Database Database
LU
LU LU LU LU
SHARD0-99
SHARD 100-199
SHARD 200-299
SHARD 300-399
1 2 3 4M
Thursday, 14 November 13
DBaaS vs PaaS
22
Database Database
vs.
Thursday, 14 November 13
Storage
•Backups on Swift•Full or incremental
•Moving tablespaces in Cinder•For high availability•For backups and archiving•For maintenance
23
in01
DatabaseDatabase
in02
Database
in03
Database
in04
Database Database
in05
Database
Thursday, 14 November 13
Networking
•Multiple NICs and floating IPs
•Public IPs
•Local proxies vs enhanced connectors
24
Database Database Database Database Database Database
Thursday, 14 November 13
Security
•LDAP & Keystone
•Integration with MySQL•LDAP / OpenLDAP•MySQL Security Plugin using OpenLDAP
25Database
Keystone
LDAP
Thursday, 14 November 13
MariaDB Cluster in OpenStackThursday, 14 November 13
What’s new with MariaDB
27
MariaDB Manager MariaDB Cluster
MariaDBServer(s)
MHA Galera
GUI
API
Monitor
MaxScale
Configuration& Provisioning
Thursday, 14 November 13
SkySQLMaxScale
28
MaxScaleCore
Data Stores
Client Applications
Backend Backend Backend
Frontend Frontend Frontend Frontend Frontend
Clie
nt
Clie
nt
Clie
nt
Clie
nt
Clie
nt
Router
Router
Router
Monitor
Monitor
Auth
Auth
Auth
Service Service Service
Listener Listener Listener Listener
Server Server Server ServerServer
Thursday, 14 November 13
IntroducingMariaDBManager
29
RESTful(APICore
Monitor/ManagementInterface
Monitor/ManagementInterface
Monitor/ManagementInterface
Monitor(DataProvider
Agent Agent Agent
MariaDB(Galera(Cluster
Thursday, 14 November 13
MariaDB Galera Cluster -‐ State
30Thursday, 14 November 13
Provision a new node
31
HTTP POST method -‐ Create a nodeURI:
• .../restfulapi/system/systemid/node• Parameters:• name• hostname• publicip, privateip, port• instanceid• dbusername, dbpassword• ...
1
Thursday, 14 November 13
Provision a new node
32
HTTP POST method -‐ Create a nodeURI:
• .../restfulapi/system/systemid/node• Parameters:• name• hostname• publicip, privateip, port• instanceid• dbusername, dbpassword• ...
1
HTTP POST method -‐ Run the commandConnect to the new nodeURI:
• .../restfulapi/command/connect• Parameters:• systemid• nodeid• rootpassword
2
Thursday, 14 November 13
Provision a new node
33
HTTP POST method -‐ Create a nodeURI:
• .../restfulapi/system/systemid/node• Parameters:• name• hostname• publicip, privateip, port• instanceid• dbusername, dbpassword• ...
1
HTTP POST method -‐ Run the commandConnect to the new nodeURI:
• .../restfulapi/command/connect• Parameters:• systemid• nodeid• rootpassword
2
HTTP POST method -‐ Run the commandProbe the state of the nodeURI:
• .../restfulapi/command/probe• Parameters:• systemid• nodeid
3
Thursday, 14 November 13
Provision a new node
34
HTTP POST method -‐ Create a nodeURI:
• .../restfulapi/system/systemid/node• Parameters:• name• hostname• publicip, privateip, port• instanceid• dbusername, dbpassword• ...
1
HTTP POST method -‐ Run the commandConnect to the new nodeURI:
• .../restfulapi/command/connect• Parameters:• systemid• nodeid• rootpassword
2
HTTP POST method -‐ Run the commandProbe the state of the nodeURI:
• .../restfulapi/command/probe• Parameters:• systemid• nodeid
3
HTTP POST method -‐ Run the commandProvision the nodeURI:
• .../restfulapi/command/provision• Parameters:• systemid• nodeid
4
Thursday, 14 November 13
Start a node
35
HTTP POST method -‐ Run the commandStart the nodeURI:
• .../restfulapi/command/start• Parameters:• systemid• nodeid
Thursday, 14 November 13
Retrieve the status of a cluster
36
• .../restfulapi/system/systemid
Thursday, 14 November 13
Monitor the # of connections of a node
37
• ../res^ulapi/system/systemid/node/nodeid/monitor/{monitorid}/data
Thursday, 14 November 13
What’s coming•MariaDB Replication and MHA
•MariaDB 10
•MaxScale integration
•Juju charms for MariaDB Cluster and MariaDB Manager
•Puppet & Chef manifests and recipes•MariaDB and Trove
•Trove for MariaDB Manager38
Thursday, 14 November 13
For More Information...
39
• MariaDB Enterprise:www.mariadb.orgwww.mariadb.com
• MariaDB Manager and MaxScale:http://downloads.skysql.com/archives
• Bugs: http://bugs.skysql.com
Thursday, 14 November 13
Thank You!
www.slideshare.net/izorattiwww.skysql.com
Pictures courtesy of:•www.pluralsight.com
Thursday, 14 November 13