Harnessing the Power of Master/Slave Clusters to Operate Data-Driven Businesses on MySQL

©Continuent 2014

Harnessing the Power of Master/Slave Clusters for Data-

Driven Business on MySQL

Robert Hodges, CEO Giuseppe Maxia, Director of QA

©Continuent 2014 2

Introducing Continuent

©Continuent 2014

Introducing Continuent

3

• The leading provider of clustering and replication for open source DBMS

• Our Product: Continuent Tungsten

• Clustering - Commercial-grade HA, performance scaling and data management for MySQL

• Replication - Flexible, high-performance data movement

©Continuent 2014

Quick Continuent Facts

• Largest Tungsten installation by data volume processes over 800 million transactions per day on 225 terabytes of relational data

• Largest installation by transaction volume handles up to 8 billion transactions daily

• Wide variety of topologies including MySQL, Oracle, Vertica, and Hadoop in production

• Cross-site topologies widely deployed

4

©Continuent 2014©Continuent 2014

Continuent Tungsten Customers

5

1

©Continuent 2014 6

Too Good to Be True

©Continuent 2014

The Dream: multiple, active DBMS servers with identical data

7

Server A

Server B

Server C

Transparent read/write

to any server

Updates propagated immediately

High Performance

High Availability

©Continuent 2014

Multi-Master Clusters in Action: Galera

8

Group Ordering and

Delivery

Server A

Server B

Server C

foo pk=1, v=5

foo pk=6, v=25

foo pk=1, v=6

[1] pk=1, v=6

[2] pk=1, v=5

[3] pk=6, v=25

[1] pk=1, v=6

[2] pk=1, v=5

[3] pk=6, v=25

[1] pk=1, v=6

[2] pk=1, v=5

[3] pk=6, v=25

©Continuent 2014

Synchronous Optimistic Locking

9

Group Ordering and

Delivery

Server A

Server B

Server C

foo pk=1, v=5

foo pk=6, v=25

foo pk=1, v=6

[1] pk=1, v=6

[2] pk=1, v=5

[3] pk=6, v=25

[1] pk=1, v=6

[2] pk=1, v=5

[3] pk=6, v=25

[1] pk=1, v=6

[2] pk=1, v=5

[3] pk=6, v=25

COMMIT

COMMITDEADLOCK

©Continuent 2014

Trouble in Paradise

• Transaction failures due to conflicts

• Operations like SELECT FOR UPDATE not supported

• Slow writes due to synchronous messaging

• Large transactions lock system/cause failures

• Unstable cross-site replication

10

©Continuent 2014

Can Master/Slave Offer Same Benefits?

11

Server A

Server B Server C

Updates propagated

immediately?

High Performance?

High Availability?

Transparent read/write

to any server?

©Continuent 2014 12

Tungsten Master/Slave Clusters

©Continuent 2014

Tungsten Database-as-a-Service

13

Tungsten clusters combine off-the-shelf open source DBMS servers into data services with: !

• 24x7 data access • Scaling of load on replicas • Simple management commands !...without app changes or data migration

Amazon US West

apache /php

GonzoPortal.com

Connector Connector

©Continuent 2014

Cluster Architecture

14

Application

Tungsten Connector

Application

Tungsten Connector

Replicator

Db2

Replicator

Db3Db1

Replicator

Slave SlaveMaster

Manager Manager Manager

Monitoring and Control

Monitoring and Control

Data Service: sjc

©Continuent 2014

Tungsten Connector Basics

• Any MySQL client can connect

• Initiates connections using JDBC

15

App Native protocol

!COM_QUERY

COM_INIT_DBCOM_DROP_DB

Connector

©Continuent 2014

Tungsten Connector Basics

• Pass-through operation after connection

• Full transparency & low overhead for clients

16

App

Protocol packet COM_QUERY

select * from t

ConnectorProtocol packet

OKResultSet Rows:1

©Continuent 2014

SmartScale Session Load Balancing

17

App

Connect/Write

Connector

Transaction History Log

Slave replay

position

Session x Session y

Slave DBMS

Master DBMS

Slave DBMS

Initial Write

©Continuent 2014

SmartScale Session Load Balancing

18

AppRead

Connector

Transaction History Log

Slave replay

position

Session x Session y

Slave DBMS

Master DBMS

Slave DBMS

Read from master

NO read from slave!

©Continuent 2014

SmartScale Session Load Balancing

19

AppRead

Connector

Transaction History Log

Slave replay

position

Session x

Slave DBMS

Master DBMS

Slave DBMS

Read from master

Read from slave

©Continuent 2014

Connectivity Example

20

Tungsten Connector Tungsten Connector

Db2 Db3Db1

SlaveMasterSlave

(SmartScale) (Strict Consistency)

©Continuent 2014 21

Demo: Transparent Connectivity to Replicas

©Continuent 2014 22

Transaction Scaling with Master/Slave Clusters

©Continuent 2014

Understanding Replication

23

Master

(Transactions + Metadata)

Slave

THL

DBMS Logs

Replicator

(Transactions + Metadata)

THLReplicator

Download transactions via network

Apply using JDBC

©Continuent 2014

Master Replication Service

24

Extract Filter Apply

StageExtract Filter Apply

Stage

Pipeline

MySQL Master

Transaction History Log

In-Memory Queue

Slave ReplicatorsBinlog

tcp/ip

©Continuent 2014

Slave Service with Parallel Apply

25

Extract Filter Apply

StageExtract Filter Apply

StageStage

Pipeline

Transaction History Log

Parallel Queue

Slave DBMS

Extract Filter ApplyExtract Filter ApplyExtract Filter Apply

Master Replicator

tcp/

ip

©Continuent 2014

Understanding Transaction Loads

26

Large TransactionsSlow

queriesReports

Db2 Db3Db1

SlaveMasterSlave

Small Transactions

©Continuent 2014 27

Demo: Scalable Transaction Processing

©Continuent 2014 28

Master/Slave Clustering Across Regions

©Continuent 2014

Primary/Disaster Recovery Clustering

29

App Server + Connector

App Server + Connector

Master Slave

San Jose NYC

©Continuent 2014

Further Cross-Site Topologies

30

Data Warehouse

All Masters

Fan-In

©Continuent 2014 31

Failover and Maintenance

©Continuent 2014

Automatic Failover

32

Application

Tungsten Connector

©Continuent 2014

Automatic Failover

33

Application

Tungsten Connector

1. Detect non-responsive master

2. Halt master connections

©Continuent 2014

Automatic Failover

34

Application

Tungsten Connector

3. Select most advanced slave and wait for pending transactions to apply

©Continuent 2014

Automatic Failover

35

Application

Tungsten Connector

4. Promote to master, reconfigure slave(s), and re-enable connections

©Continuent 2014

Automatic Failover

36

Application

Tungsten Connector

5. Recover failed master

©Continuent 2014

Rolling MySQL maintenance

• Task: change the innodb log file size

• Problem: it requires a database server restart

• Constraint: we do not want to restart

• Solution: rolling maintenance

• shun node2 (slave) - do maintenance - put node online

• shun node3 - do maintenance - put node online

• promote node2 to master

• shun node1 - do maintenance - put node online

• promote again node1 to master

37

©Continuent 2014

Rolling MySQL Maintenance

38

Final UpgradeSwitchSlave Upgrade Slave Upgrade

• Shun slave 1, run upgrade, and return to cluster. Discard and recover on failure

• Upgrade old master. Cluster schema is now fully upgraded

• Repeat for additional slave(s)

• Switch master to promote an upgraded slave

©Continuent 2014 39

Demo: Zero-Downtime Upgrade and Failover

©Continuent 2014 40

Wrap-Up

©Continuent 2014

Master/Slave Performs Very Well!

41

Server A

Server B Server C

Updates propagated

without cost to applications

Very High Performance

Very High Availability

Transparent connectivity with full SQL

semantics

©Continuent 2014

In Conclusion: Tungsten Offers...

• Transparent access to a highly available cluster of off-the-shelf MySQL replicas

• High performance transaction processing regardless of cluster size or distance between nodes

• Zero-downtime maintenance and upgrade of DBMS and applications

42

©Continuent 2014

www.continuent.com Follow us on Twitter @continuent

!

Tungsten Replicator: http://code.google.com/p/tungsten-replicator

Our Blogs: http://scale-out-blog.blogspot.com http://datacharmer.org/blog http://www.continuent.com/news/blogs http://flyingclusters.blogspot.com/

560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009 e-mail: [email protected]