Page 1
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |Friday, November 20, 2015 OSI - Bangalore
An Overview of the New Replication Features in MySQL 5.7 and Beyond
Sujatha Sivakumar ([email protected] )Software Engineer, MySQL Server Sustaining Team
Page 2
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Friday, November 20, 2015 OSI - Bangalore
Page 3
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Friday, November 20, 2015 OSI - Bangalore
Page 4
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Background
Celebrating MySQL 5.7 Replication
The New Replication Features in MySQL 5.7
Development Sneak Peek
Roadmap
Summary
1
2
3
4
5
Friday, November 20, 2015 OSI - Bangalore
6
Page 5
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Celebrating MySQL 5.7 ReplicationGA ready!
1
Friday, November 20, 2015 OSI - Bangalore
Page 6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Thank You!
• After two and half years baking the team is very happy to share the outcome with you!
– We did it for you!
• Thank you for being part of it!
• Enjoy MySQL 5.7, in particular the replication features! ;)
Friday, November 20, 2015 OSI - Bangalore
Page 7
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Background2
Friday, November 20, 2015 OSI - Bangalore
Page 8
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Background: Replication Components
Friday, November 20, 2015
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Senderthread
Receiverthread
ApplierThreads
ReceiverMeta-data
Update
ApplierMeta-data
Update
Network
OSI - Bangalore
Page 9
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Background: Replication Components
• Binary Log
– File based logical log that records the changes on the master.
– Statement or Row based format (may be intermixed).
– Each transaction is split into groups of events.
– Control events: Rotate, Format Description, Gtid, ...
Friday, November 20, 2015
Layout of a Binary Log File
BEGIN ...E1 E2 COMMIT BEGIN ...E1 E2 COMMIT
OSI - Bangalore
Page 10
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
M S
S
S
S
M
write clients read clientsread clients
write clients
Morereads?More
slaves!
Read scale-out
Background: What is Replication Used For?
Friday, November 20, 2015 OSI - Bangalore
Page 11
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
M M?
write clients write clients
Morewrites?More
Masters?
M?
M?
MySQL Fabric helps sharding your data with MySQL...
What about write scale-out?
Friday, November 20, 2015 OSI - Bangalore
Page 12
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
C
B
A
C
B
ACrash
C
B
A
B is thenew master
Uh Oh! Whew!
Redundancy: If master crashes, promote slave to master
Background: What is Replication Used For?
Friday, November 20, 2015 OSI - Bangalore
Page 13
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
M
S
write clients
business intelligent client applicationsreporting client applicationsbig queries client applications
On-line Backup and Reporting
Background: What is Replication Used For?
Friday, November 20, 2015 OSI - Bangalore
Page 14
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
CB
BAAC
Image fromwww.ginkgomaps.com
Background: What is Replication Used For?
Friday, November 20, 2015 OSI - Bangalore
Page 15
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
The New Replication Features in MySQL 5.73
Friday, November 20, 2015 OSI - Bangalore
Page 16
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
The New Replication Features in MySQL 5.7Usability / Online
3
Friday, November 20, 2015
3.1
OSI - Bangalore
Page 17
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Online Reconfiguration of Global Transaction Identifiers
Friday, November 20, 2015
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Senderthread
Receiverthread
ApplierThreads
ReceiverMeta-data
Update
ApplierMeta-data
Update
Network
OSI - Bangalore
Page 18
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Online Reconfiguration of Global Transaction Identifiers
• The procedure is online.
– Both reads and writes are allowed while global transaction identifiers are being turned on or off in the entire replication cluster.
• No need to synchronize servers, ever.
• No need to restart servers.
• No need to change replication topology.
– Works in arbitrary topologies.
• Should anything happen in the middle of the procedure, you can always roll back.
Friday, November 20, 2015 OSI - Bangalore
Page 19
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Online Reconfiguration of Global Transaction Identifiers
• Over simplified procedure to turn on global transaction identifiers:
• Details: http://dev.mysql.com/doc/refman/5.7/en/replication-mode-change-online-enable-gtids.html
Friday, November 20, 2015
1) SET @@GLOBAL.GTID_MODE = OFF_PERMISSIVE; (on every server)2) SET @@GLOBAL.GTID_MODE = ON_PERMISSIVE; (on every server)3) wait for a bit longer than your replication lag4) SET @@GLOBAL.GTID_MODE = ON; (on every server)
OSI - Bangalore
Page 20
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Online Reconfiguration of Replication Filters
Friday, November 20, 2015
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Senderthread
Receiverthread
ApplierThreads
ReceiverMeta-data
Update
ApplierMeta-data
Update
Network
OSI - Bangalore
Page 21
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Online Reconfiguration of Replication Filters
• Change Slave's Replication Filters dynamically.
– No need to stop and restart the slave server for setting new replication filtering rules.
– All slave filters are supported.
– Values can be input in various character sets.
Friday, November 20, 2015
mysql> CHANGE REPLICATION FILTER REPLICATE_DO_DB= (db1, db2)
OSI - Bangalore
Page 22
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Online Reconfiguration of Replication Receiver/Applier
Friday, November 20, 2015
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Senderthread
Receiverthread
ApplierThreads
ReceiverMeta-data
Update
ApplierMeta-data
Update
Network
OSI - Bangalore
Page 23
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Change master from A to B without stopping the applier threads.
Online Reconfiguration of Replication Receiver/Applier
Friday, November 20, 2015
C
B
A
C
B
ACrash
C
B
A
B is thenew master
Uh Oh! Whew!
OSI - Bangalore
Page 24
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Change master from A to B without stopping the applier threads.
Online Reconfiguration of Replication Receiver/Applier
Friday, November 20, 2015
C
B
A
C
B
ACrash
C
B
AUh Oh! Whew!
DBA did not need C to stop applying its relay log to change C's master from A to B.
OSI - Bangalore
Page 25
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Online Reconfiguration of Replication Receiver/Applier
• Enables more online operations during fail-over:
• Stopping, changing master and restarting the receiver thread are all done while the applier thread is running.
• Change applier properties while the receiver thread is working:
Friday, November 20, 2015
mysql> STOP SLAVE IO_THREAD;
mysql> CHANGE MASTER TO MASTER_HOST='master2', …;
mysql> START SLAVE IO_THREAD;
mysql> STOP SLAVE SQL_THREAD;
mysql> CHANGE MASTER TO MASTER_DELAY=3600, …;
mysql> START SLAVE SQL_THREAD;
OSI - Bangalore
Page 26
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Improved Replication Monitoring
Friday, November 20, 2015
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Senderthread
Receiverthread
ApplierThreads
ReceiverMeta-data
Update
ApplierMeta-data
Update
Network
OSI - Bangalore
DBA
Page 27
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Improved Replication Monitoring
• Monitoring through SQL.
• Logically unrelated information into different places.
• Extensible, can be adapted to new features.
• More accurate and consistent identifier names.
• Master-slave, Multi-source, Group Replication support.
Performance Schema Replication Tables
Friday, November 20, 2015 OSI - Bangalore
Page 28
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Improved Replication Monitoring
Friday, November 20, 2015
Connection Configuration
Connection Status
Applier Configuration
ApplierStatus
(Slave) Receiver and Applier Status
Applier / CoordinatorStatus
WorkersStatus
OSI - Bangalore
Page 29
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Improved Replication Monitoring
Friday, November 20, 2015
mysql> select * from performance_schema.replication_applier_status_by_worker\G
*************************** 1. row ***************************
CHANNEL_NAME:
WORKER_ID: 1
THREAD_ID: 35
SERVICE_STATE: ON
LAST_SEEN_TRANSACTION: 4ba0eb86-63c3-11e4-92ba-28b2bd168d07:2368
LAST_ERROR_NUMBER: 0
LAST_ERROR_MESSAGE:
LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
*************************** 2. row ***************************
CHANNEL_NAME:
WORKER_ID: 2
THREAD_ID: 36
SERVICE_STATE: ON
LAST_SEEN_TRANSACTION: 4ba0eb86-63c3-11e4-92ba-28b2bd168d07:2367
LAST_ERROR_NUMBER: 0
LAST_ERROR_MESSAGE:
LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
2 rows in set (0,00 sec)
OSI - Bangalore
Page 30
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
The New Replication Features in MySQL 5.7Usability / Online
Performance
3
Friday, November 20, 2015
3.1
3.2
OSI - Bangalore
Page 31
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Improved Applier Throughput – Locking-based Parallelism
Friday, November 20, 2015
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Senderthread
Receiverthread
ApplierThreads
ReceiverMeta-data
Update
ApplierMeta-data
Update
Network
OSI - Bangalore
Page 32
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Improved Applier Throughput – Locking-based Parallelism
Friday, November 20, 2015
Single Threaded Apply 8 Threads Applier 24 Threads Applier 48 Threads Applier 96 Threads Applier
0%
50%
100%
150%
200%
250%
Fast and Scalable Multi-threaded Replication Applier
(statement-based replication, durable settings, SSD storage)
Sla
ve a
pply
tim
e (
Sysbench R
W o
n t
he m
aste
r w
ith 9
6 t
hre
ads =
100%
)
10X
OSI - Bangalore
Page 33
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Improved Applier Throughput – Locking-based Parallelism
• Leverage parallelization information from master execution:
– Concurrent transactions, which have not blocked each other on the master, are marked as non-contending in the binary log.
• Meanwhile, at the slave:
– Non-contending transactions are scheduled in parallel;
– Concurrent transactions commit independently, thus no waiting involved.
Friday, November 20, 2015 OSI - Bangalore
Page 34
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Improved Applier Throughput – Locking-based Parallelism
• T2 is scheduled to execute together with T1.
• T3 is scheduled to execute as soon as T1 finishes.
Friday, November 20, 2015
T1
T2
T3
Time
Commit Finishes
Last Lock Acquired
Concurrent Execution Historyon the Master
Execution
Commit
T1 and T2 execute inparallel on the slave.
T2 and T3 execute in parallel on the slave.
OSI - Bangalore
Page 35
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Improved Applier Throughput – Locking-based Parallelism
• Supports statement-based or row-based formats.
• Scheduling policy controlled through:
– logical_clock - means schedule based on the locking interval timestamps.
– database - the scheduling policy from 5.6 (concurrency control done per database).
• Work to improve slave scalability continues, does not stop here.
Friday, November 20, 2015
mysql> SET slave_parallel_type= [logical_clock|database]
OSI - Bangalore
Page 36
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Improved User Threads-Sender Threads Synchronization
Friday, November 20, 2015
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Senderthread
Receiverthread
ApplierThreads
ReceiverMeta-data
Update
ApplierMeta-data
Update
Network
OSI - Bangalore
Page 37
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Improved User Threads-Sender Threads Synchronization
• Concurrent reads by the sender thread with ongoing writes from user threads.
– Sender thread does not block user sessions more than necessary.
– Higher throughput for both sender threads and user sessions.
Friday, November 20, 2015
Sender Thread Reads Binary Log User Thread Writes to Binary Log
Sender Thread Reads Binary Log
User Thread Writes to Binary Log
pre 5.7.2
5.7.2+
OSI - Bangalore
Page 38
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Faster Sender Thread
Friday, November 20, 2015
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Senderthread
Receiverthread
ApplierThreads
ReceiverMeta-data
Update
ApplierMeta-data
Update
Network
OSI - Bangalore
Page 39
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Faster Sender Thread
Friday, November 20, 2015
• Send buffer is not allocated and freed every time an event is sent.
• When sender threads require larger send buffer - buffer grows as needed.
• When buffer is larger than needed - buffer shrinks dynamically.
• Together with the sender thread enhancements released on MySQL 5.7.2:
– increases master scalability;
– reduces resource consumption (CPU);
– Master, with dump threads attached, copes better with peak loads.
OSI - Bangalore
Page 40
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Faster Sender Thread
• Ad hoc microbenchmark using mysqlslap
• 1M queries, concurrency=200, commit=1
• N slaves attached (fake slaves using remote mysqlbinlogs)
• 48 cores HT / 512 GB RAM / HDD
Friday, November 20, 2015
0
2000
4000
6000
8000
10000
12000
14000
16000
0 1 5 10 20 30 40 50 100
Qp
S
Number of Connected Sender Threads
Master’s Throughput
MySQL 5.6.16 MySQL 5.7.4
OSI - Bangalore
Page 41
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Faster Semi-sync Replication – ACK Receiver Thread
Friday, November 20, 2015
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
ACKReceiverthread
Receiverthread
ApplierThreads
ReceiverMeta-data
Update
ApplierMeta-data
Update
Network
Senderthread
OSI - Bangalore
Page 42
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Faster Semi-sync Replication – ACK Receiver Thread
• Consecutive transactions do not block each other while waiting for ACKs
– Transaction t1 and t2 are sent immediately to the slave by the sender thread
– ACKs are received only by a special thread
– Transaction t2 does not include t1 round trip in its semi-sync overall latency
• Thread starts when semi-sync is activated
• Thread stops when semi-sync is deactivated
Friday, November 20, 2015
mysql> SET GLOBAL rpl_semi_sync_master_enabled= ON
mysql> SET GLOBAL rpl_semi_sync_master_enabled= OFF
OSI - Bangalore
Page 43
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Faster Semi-sync: Durability over the Network
Friday, November 20, 2015 OSI - Bangalore
Orange bar percentage shows the throughput gain (orange) when compared to the corresponding blue bar. Blue and stacked bars show throughput ratio computed against the best blue bar.
Page 44
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
The New Replication Features in MySQL 5.7Usability / Online
Performance
Dependability
3
Friday, November 20, 2015
3.1
3.2
3.3
OSI - Bangalore
Page 45
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Loss-less Semi-sync Replication
Friday, November 20, 2015
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Senderthread
Receiverthread
ApplierThreads
ReceiverMeta-data
Update
ApplierMeta-data
Update
Network
OSI - Bangalore
Page 46
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Loss-less Semi-sync Replication
Friday, November 20, 2015
Master
Slave
T1: INSERT INTO t1 VALUES (1000)
ACK
Time
T2: SELECT * FROM t1; empty set
execute prepare binlog
relay log
commitexecute
OSI - Bangalore
Page 47
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Loss-less Semi-sync Replication
• Master waits for slave's ACK before committing (as opposed to: master waits for slave's ACK after committing).
– Therefore, concurrent transactions do not see changes while this transaction waits for ack.
• Should a master fail, then any transaction that it may have externalized is also persisted on a slave.
• User can choose between the original semi-sync behavior and the new one.
Friday, November 20, 2015
mysql> SET rpl_semi_sync_master_wait_point= [AFTER_SYNC|AFTER_COMMIT]
OSI - Bangalore
Page 48
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Semi-sync Replication – Wait for Multiple ACKs
Friday, November 20, 2015
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Senderthread
Receiverthread
ApplierThreads
ReceiverMeta-data
Update
ApplierMeta-data
Update
Network
OSI - Bangalore
Page 49
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Semi-sync Replication – Wait for Multiple ACKs
• Master does not commit transaction until it receives N ACKs from N slaves.
• Dynamically settable:
Friday, November 20, 2015
mysql> SET GLOBAL rpl_semi_sync_master_wait_for_slave_count= N
OSI - Bangalore
Page 50
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Semi-sync Replication – Wait for Multiple ACKs
• Master does not commit transaction until it receives N ACKs from N slaves.
• Dynamically settable:
Friday, November 20, 2015
mysql> SET GLOBAL rpl_semi_sync_master_wait_for_slave_count= N
Master
Slave 2
T1: COMMIT
ACK
Slave 1ACK
T1: COMMITsucceeds
mysql> SET GLOBAL rpl_semi_sync_master_wait_for_slave_count= 2
Slave 3
Time
relay log
relay log
relay log
OSI - Bangalore
ACK
Page 51
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
The New Replication Features in MySQL 5.7Usability / Online
Performance
Dependability
Flexibility
3
Friday, November 20, 2015
3.1
3.2
3.3
3.4
OSI - Bangalore
Page 52
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Storing Global Transaction Identifiers in a Table
Friday, November 20, 2015
Insert...
B
Insert...
relay log
Insert...
A
binary log
Client
Senderthread
Receiverthread
ApplierThreads
ReceiverMeta-data
Update
ApplierMeta-data
Update
Network
OSI - Bangalore
Page 53
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Storing Global Transaction Identifiers in a Table
• Slaves can use GTIDs with binary logs disabled.
– Slaves that are never candidates to become a master can still use GTIDs for auto positioning.
Use Cases
Friday, November 20, 2015 OSI - Bangalore
Page 54
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Storing Global Transaction Identifiers in a Table
• GTIDs are saved into a mysql system table (range-compressed).
• GTIDs are inserted into the table as transactions commit.
• A compression thread runs periodically and compresses the table into ranges.
– New global variable controls the period: gtid_executed_compression_period.
Mechanics
Friday, November 20, 2015
CREATE TABLE gtid_executed(
source_uuid CHAR(36) NOT NULL,
interval_start BIGINT NOT NULL,
interval_end BIGINT NOT NULL,
PRIMARY KEY(source_uuid, interval_start)
);
mysql> SET GLOBAL gtid_executed_compression_period= N;(N – number of transactions)
OSI - Bangalore
Page 55
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Storing Global Transaction Identifiers in a Table
• Binary Log Enabled
– Store GTID in binary log (transactionally).
– Copy GTID set to table on binary log rotation.
• Binary Log Disabled– Store GTID in table (transactionally).
– New transactions are do not get an automatic GTID assigned.
• Always store GTIDs transactionally.
Mechanics
Friday, November 20, 2015 OSI - Bangalore
Page 56
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Multi-Source Replication
Friday, November 20, 2015
S
M M Slave can have more than one master.M M
The need for gathering data in a central server:• Integrated backup;• Complex queries for analytics purposes;• Data HUB for inter-cluster replication.
OSI - Bangalore
Page 57
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Multi-Source Replication• A server (slave) can replicate from multiple sources (masters).
• Multiple channels (channel: receiver thread, relay logs, applier threads) that can be stopped started individually.
• Integrated with Multi-threaded Slave (each channel has its own multi-threaded applier set of threads).
• Integrated with the new P_S tables.
– replication_applier_status_by_coordinator shows multiple entries, one per channel/source coordinator.
– replication_applier_status_by_worker shows multiple entries, multiple entries per channel
– replication_connection_status shows multiple entries, one per connection to different sources.
Friday, November 20, 2015 OSI - Bangalore
Page 58
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Multi-Source Replication
• Integrated with GTIDs.
• Integrated with crash-safe tables.
– Progress state is stored in these tables for recoverability purposes.
• Virtually no limit on the number of sources (capped to 256, but can be changed if server binary is rebuilt).
• Able to manage each source separately.
Friday, November 20, 2015 OSI - Bangalore
Page 59
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
The New Replication Features in MySQL 5.7Usability / Online
Performance
Dependability
Flexibility
Misc
3
Friday, November 20, 2015
3.1
3.2
3.3
3.4
3.5
OSI - Bangalore
Page 60
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
“Smaller”, yet interesting, enhancements!• Multi-threaded applier is able to retry failed transactions.
• Option to make the Multi-threaded applier preserve commit order.
• SSL options for mysqlbinlog tool.
• Rewrite DB rules for the mysqlbinlog tool.
• Function to wait for transactions to be applied, regardless of the replication stream they come from.
• Options to track global transaction identifiers in the return packet of the mysqlprotocol. Useful for tracking GTID session state and enabler for session consistency.
• Support for XA transactions when the binary log is enabled.
• Change in the defaults. E.g., binlog_format=ROW and sync_binlog=1.
• Options to fine tune the binary log group commit procedure.
Friday, November 20, 2015 OSI - Bangalore
Page 61
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
The New Replication Features in MySQL 5.7Usability / Online
Performance
Dependability
Flexibility
Misc
Other Components Working with Replication
3
Friday, November 20, 2015
3.1
3.2
3.3
3.4
3.5
OSI - Bangalore
3.6
Page 62
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Router
• Motivation
– The need for transparent integration with Fabric.• No need to upgrade existing connectors.
• Use Fabric with connectors that do not support it (e.g., PHP, Ruby, Perl, C).
– The need for read-write and read-only automatic routing.• Do not need to know which server is the master.
• Transparent fail-over support for a new master.
• End Result– Generic and versatile framework: MySQL Router.
Friday, November 20, 2015 OSI - Bangalore
NEW
M
Router
App
M M
MySQLFabric
GA
Page 63
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Router
• Highlights
– High Performance
– Plugin Driven Architecture
– Simple to setup, configure and deploy.
• Features
– Connection routing and simple load balancing.
– Seamless fail-over based on Fabric HA Groups.
– Fail-over without Fabric (needs 3rd party tool to handle fail-over itself).
Friday, November 20, 2015 OSI - Bangalore
M
Router
App
M M
MySQLFabric
NEWGA
Page 64
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Development Sneak Peek4
Friday, November 20, 2015 OSI - Bangalore
Page 65
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Group Replication Plugin
Friday, November 20, 2015
M M M M M
Replication Group
• Multi-master update everywhere
• Automatic group membership management and failure detection.
• No need for server fail-over.
• Automatic reconfiguration.
• No single point of failure.
• Shared-nothing state machine replication.
• InnoDB compliant. Off-the-shelf hardware friendly.
OSI - Bangalore
Page 66
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Group Replication Plugin
• Great technology for deployments where elasticity is a requirement, such as cloud based infrastructures.
– Integrated with the server core through a well defined API.
– It is integrated with GTIDs, row based replication.
– It is integrated with performance schema tables.
– It is elastic and self-healing: adding or removing servers does not need human intervention.
Friday, November 20, 2015 OSI - Bangalore
Page 67
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Strong development cycles and continuous community engagement through regular and frequent releases on labs.mysql.com.
MySQL Group Replication Plugin
Friday, November 20, 2015 OSI - Bangalore
2014-Sep-29Labs release: 0.2.0
2015-Apr-06Labs release: 0.3.0
2014-Aug-06Labs release: 0.4.0
2015-Sep-14Labs release: 0.5.0
Introduces standalone plugin,with its own release cycle!
2015-Oct-26Labs release: 0.6.0
Multi-platform, No need forCorosync.
Page 68
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Group Replication Plugin
• With MySQL Group Replication for 5.7.9 we introduce a new group communication engine.
– Corosync is not needed anymore.
– All pieces are integrated and is now easier to build, install and use.
– Plugin now builds on multiple platforms: Linux, Solaris, FreeBSD, OSX and Windows.
– Cloud friendly transport – TCP transport.
Friday, November 20, 2015 OSI - Bangalore
Page 69
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Roadmap5
Friday, November 20, 2015 OSI - Bangalore
Page 70
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
What is Next?
• MySQL Group Replication
– Rapid releases.
– Improve performance, stability and usability.
• MySQL Replication Usability– Instrument even more replication and extend replication P_S tables
– Simpler administrative commands, more online operations
• MySQL Replication Performance– Continue to improve replication stream multi-threaded (slave) applier
– Continue to improve replication stream pipeline performance and efficiency
Friday, November 20, 2015 OSI - Bangalore
Page 71
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Summary6
Friday, November 20, 2015 OSI - Bangalore
Page 72
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Summary
• Replication feature set in MySQL 5.7 shows many improvements:
– Semi-sync is more flexible and faster.
– Less contention on the binary log.
– Slave is faster and provides an inter-transaction parallelization scheme.
– More online reconfiguration: Global Transaction Identifiers, filters, configuration.
– Improved monitoring.
– Even more flexible replication with multi-source enabling new deployment scenarios.
– Enhancements all over: transaction retries, sequential commits, XA support, new replication functions, security.
• Lab releases provide a sneak peek at what is coming - a new replication plugin and exciting new infrastructure: MySQL Group Replication and MySQL Router.
Friday, November 20, 2015 OSI - Bangalore
Page 73
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Where to go from here?
• Packages
– http://dev.mysql.com
– http://labs.mysql.com
• Reference Documentation– http://dev.mysql.com/doc/#manual
• Blogs from the Engineers (news, quirks, technical information and much more)
– http://mysqlhighavailability.com
Friday, November 20, 2015 OSI - Bangalore
Page 74
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Other Interesting Presentations From Us …
• Topic: Optimizer Enhancements and JSON Support in MySQL 5.7
Time: 12:30 to 1.15 PM
Speaker: Sree Harsha
• Topic: State of the Dolphin Time: 2.15 to 3.00 PM
Speaker: Sanjay Manwani
• Topic: Security and Performance – out of the box in MySQL 5.7
Time: 3:15 to 4:00 PM
Speaker: Nitin Mehta and Mayank Prasad
Friday, November 20, 2015 OSI - Bangalore