Seminar : "The Future of MySQL - Roadmap to Success" session MySQL High Availability Options

<Insert Picture Here>

MySQL High Availability Options

Ryusuke KajiyamaMySQL Principal Sales Consultant, Asia Pacific and Japan

2

Designing for High Availability

Copyright Oracle Corporation 2010 2

3

Selecting the Right HA Architecture


4

MySQLCluster

MySQLCluster

MySQLServer

Application / Web / Web AP

• MySQL ReplicationAsynchronous replication

• MySQL+DRBD (for Linux)Shared Nothing Active/Passive



Shared disk

• Shared Disk BasedActive/Passive


• MySQL ClusterShared Nothing Active/Active

MySQLServer

MySQLServer

MySQLServer

MySQLServer

MySQLServer

AsynchronousReplication

SynchronousReplication

Fail Over on failure


Load Balancing


Load Balancing

MySQL High Availability Option

5

Common MySQL HA SolutionsMySQL Replication is Common Foundation for MySQL HA


Requirements Replication Cluster MySQL &DRBD

AvailabilityAutomatic Fail Over No Yes Yes

Fail Over Time Varies < 1 sec < 5 min

Resynch of Data No Yes Yes

Geographic Redundancy Yes Yes No

ScalabilityLoad Balancing Scale-Out Yes No

Read Intensive Yes Yes No

Write Intensive No Yes No

# of Nodes 100’s (reads) 255 1 Active

SQL FunctionalityPrimary Key Lookups Yes Yes Yes

Simple JOINs Yes Yes Yes

Complex JOINs Yes No Yes

Transactions Yes Yes Yes

6

MySQL Replication Overview

• Native in MySQL• Used for Scalability

and HA• Asynchronous as

standard• Semi-Synchronous

support added inMySQL 5.5

• Each slave addsminimal load onmaster


7

MySQL ReplicationDelivering Read Scalability

• Used by leading web properties for scale-out• Reads are directed to slaves, writes to master• Delivers higher performance & scale with efficient resource utilization

Clie

nts

Slaves Master

MySQL Replication


8

Master > Slave

Masters > Slave (Multi-Source)

Master < > Master (Multi-Master)

Master > Slaves

Circular (Multi-Master)

Master > Slave > Slaves

MySQL Replication Topologies

9

Building on ReplicationFailure Detection & Failover

• Linux Heartbeat implements heartbeat protocol between nodes• Failover initiated by Cluster Resource Manager (Pacemaker) if heartbeat

message is not received• Virtual IP address failed over to ensure failover is transparent to apps


10


■MySQL Replication+ IP address failover


Shared disk

■RedHat Cluster + Sharedstorage for Master + Slave

MySQLServer

MySQLServer

MySQLServer

MySQLServerAsynchronous

Replication

Fail Over on failureLoad Balancing

MySQLServer

Master(Read + Write)

Slave(Read)

Master(Read + Write) Asynchronous

Replication

Slave(Read)

Standby

MySQL HA Configuraiton

11


■MySQL Replication+ IP address failover


Shared disk

■RedHat Cluster + Sharedstorage for Master + Slave

MySQLServer

MySQLServer

MySQLServer

MySQLServer


MySQLServer


Slave(Read)


New Master(Read + Write)

New Master(Read + Write)

In case of failure of Master Server

12

MySQL 5.5 Replication Features1. Semisynchronous replication

Improved resilience by having master wait for slave toreceive events.

2. Slave fsync tuning & Automatic relay log recoveryTune fsyncs so corruption is less likely on slave crashes.Let the slave recover from corrupted relay logs.

3. Replication HeartbeatHave a more precise failure detection mechanism.Avoid spurious relay log rotation when the master is idle.

4. Per server replication filteringInstruct slave to discard events from a master with aspecific server id.


13

MySQL 5.5 Replication Features5. Precise Slave Type Conversions

Use different types on master and slave and getautomatic type promotion and demotion when using RBR

6. Individual Log FlushingSelectively flush server logs when using 'FLUSH LOGS'

7. Safe logging of mixed transactionsReplicate transactions containing both InnoDB andMyISAM changes


14

Application

Master

Connection Thread

Data Binlog

Slave

DataRelaylog

Commit

ChangingData

ChangingBinlog

Replication

Response

ChangingData

Asynchronous Replication

15

Application

Master

Connection Thread

Data Binlog

Slave

DataRelaylog

Commit

ChangingData

ChangingBinlog

Replication

ResponseChanging

Data

Response

Semi-synchronous Replication

16

“A high-performance, distributed memory object cachingsystem, generic in nature, but intended for use in speeding updynamic web applications by alleviating database load” *

* http://www.socialtext.net/memcached/index.cgi?faq

• Created by Danga Interactive to speed up LiveJournal’s 20million+ dynamic page views per day for 1 million+ users

• Significantly dropped database load, delivering faster pageloads, better resource utilization, and faster access todatabases

• Perfect for dynamic sites that generate high database load

• Used by Facebook, YouTube, Wikipedia, others!

What is Memcached?

17

• Created to speed up blogging site LiveJournal• 20 million+ dynamic page views per day• 1 million+ users

• Results…• Faster page loads• Lowered database load• Better resource utilization• Faster access to databases

• Perfect for dynamic sites that generate high database load

Why was Memcached created?

18

• Application is modified so data isread from memcached not thedatabase

• In the event the data is stale ornon-existent…

– data is read from thedatabase

– written into memcached

• Next request for the same data isretrieved from memcached

Typical Use Case: Read/Pass-Through

19

Memcached Functions for MySQL

• Overview– Uses UDF API and libmemcached– Manage memcached Cluster via SQL– Read through Cache– Write through Cache

• Installation– CREATE FUNCTION memc_servers_add

RETURNS INT SONAME"libmemcached_functions_mysql.so";

20

memcached UDF Example

– Creating Trigger which kicks UDF is one of the best practices

21

Linux Heartbeat, Block-Replication & MySQL

• Distributed Replicated Block Device (DRBD)– Runs over standard IP networks– Distributed storage– Similar to network RAID

• Synchronous• Characteristics

– Higher complexity to install and configure– No special networking components (except Heartbeat)– Excellent performance (blocks vs. rows of data)– Manages inconsistencies of data during a failure– Hides the complexity of many recovery actions– Linux heartbeat manages fail over and virtual IPs

22

DRBD Architecture

23

MySQL w/ Shared Storage & Clustering Agents

• Active/Passive likely configuration– Multiple instances not allowed concurrent access to same

data files• Automated management

– Virtual IPs– Fail over– Data synchronization– Mounting file systems

• Characteristics– High cost (storage, hardware, software)– Idle resources– Longer fail over times– High initial complexity– Many options and proven vendors

24

MySQL Cluster

• Shared-Nothing Clustering Solution• Synchronous (2-phase commit)• Fast Automatic Fail Over• High Performance• High Transactional Throughput• No Special Component Requirements• In-Memory & Disk Data Support• Heart-beat protocol

25

MySQL Cluster Data Nodes

MySQL Cluster Application Nodes

MySQLClusterMgmt

Clients

MySQLClusterMgmt

MySQL Cluster ArchitectureParallel Database with no SPOF: High Read & Write Performance & 99.999% uptime

26

Out of the Box Scalability: Data Partitioning

• Data partitioned across Data Nodes• Rows are divided into partitions, based on a hash of all or part of the primary

key• Each Data Node holds primary fragment for 1 partition

– Also stores secondary fragment of another partition

• Records larger than 8KB stored as BLOBs

27

Geographic Replication

Cluster 1

Synchronousreplication

Cluster 2

MyISAM MyISAM InnoDB

Asynchronousreplication

• Synchronous replication withina Cluster node group for HA

• Bi-Direction asynchronousreplication to remote Cluster forgeographic redundancy

• Asynchronous replication tonon-Cluster databases forspecialised activities such asreport generation

• Mix and match replication types

28

High Throughput, Low Latency Transactional Performance

• MySQL Cluster delivered:– 250k TPM, 125k operations per second– Average 3ms response time– 4.3x higher throughput than previous MySQL Cluster 6.3 release

1 4 8 12 16 20 24 28 32 36 400

2500050000

75000

100000125000

150000

175000

200000

225000

250000

275000

DBT2 Benchmark, 4-MySQL Cluster Data Nodes

MySQL C luster 7.0MySQL C luster 6.3

Number of MySQL Server Nodes

Tran

sact

ions

Per

Min

ute

http://www.mysql.com/why-mysql/benchmarks/mysql-cluster/

29

SQL Node(MySQL)

NDB API

Data Node(NDB Storage Engine)

X

Low-Level Access via NDB API

• High performance C++ API• Implements indexes, scans, transactions & events• ACID-compliant• Object-oriented error-handling• Additional performance features not available in SQL

30

MySQL Cluster Connector for Java

• New Domain Object Model PersistenceAPI (ClusterJ) :– Java API– High performance, low latency– Feature rich

• JPA interface built upon this new Javalayer:– Java Persistence API compliant

• Implemented as an OpenJPA plugin– Uses ClusterJ where possible, reverts to

JDBC for some operations– Higher performance than JDBC– More natural for most Java designers– Easier Cluster adoption for web

applicationsData Nodes

Network

32

• Application: Service Delivery Platform– Roaming platform to support 7m roaming subscribers per day FIFA World Cup 2010– Database supports AAA, routing, billing, messaging, signalling, payment processing– MySQL Cluster 7.1 delivered 1k TPS on 1TB data with carrier-grade availability

• Key business benefits– Local carriers to monetize new subscribers– Users enjoy local pricing with full functionality of their home network– Reduced deployment time by 75%

32

”MySQL Cluster 7.1 gave us the perfect combination of extreme levels of transactionthroughput, low latency & carrier-grade availability. We also reduced TCO by being able

to scale out on commodity server blades and eliminate costly shared storage”

- Phani Naik, Head of Technology at Pyro Group

Learn More: http://www.mysql.com/why-mysql/case-studies/mysql_cs-pyro_telecoms.php

3333

“Since deploying MySQL Cluster as our eCommerce database, we have had continuous uptime with linear scalability enabling us to exceed our most stringent SLAs”

— Sean Collier, CIO & COO, Shopatron Inc

Shopatron: eCommerce Platform

• Applications– Ecommerce back-end, user authentication,

order data & fulfilment, payment data &inventory tracking. Supports severalthousand queries per second

• Key business benefits– Scale quickly and at low cost to meet

demand– Self-healing architecture, reducing TCO

• Why MySQL?– Low cost scalability– High read and write throughput– Extreme availability

Learn More: http://www.mysql.com/why-mysql/case-studies/mysql_cs_shopatron.php

34

MySQLCluster

MySQLCluster

MySQLServer


• MySQL ReplicationAsynchronous replication

• MySQL+DRBD (for Linux)Shared Nothing Active/Passive



Shared disk

• Shared Disk BasedActive/Passive


• MySQL ClusterShared Nothing Active/Active

MySQLServer

MySQLServer

MySQLServer

MySQLServer

MySQLServer





Load Balancing


Load Balancing

MySQL High Availability Option

Seminar : "The Future of MySQL - Roadmap to Success" session MySQL High Availability Options

Technology

mysql server slaveread

replication features

replication heartbeathave

slave fsync

slave crashes

leading web properties

high database load

slave toreceive events