Oracle OpenWorld - Getting started with MySQL Cluster

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

Getting started with MySQL Cluster Hands-on Lab

Benedita Paúl Vasconcelos Principal Technical Support Engineer Oracle, MySQL Support October 29, 2015


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

3


Program Agenda

Aim of this Hands-on Lab

Brief Introduction to MySQL Cluster

Installation of MySQL Cluster

Start & Monitoring of MySQL Cluster

Connecting to MySQL Cluster

Safe Shutdown of MySQL Cluster

Overview of MySQL Cluster’s Admin Commands & Operations

MySQL Cluster CGE & Conclusions

1

2

3

4

5

6

7

8


Program Agenda with Highlight









1

2

3

4

5

5

6

7

8



• Objectives

1. Learn the basics of the MySQL Cluster Architecture

2. Learn the basics of MySQL Cluster Configuration and Administration

3. Learn how to start a new Cluster for evaluation purposes and how to connect to it

• Benefits

– Familiarize with MySQL Cluster

– Know where to go next to start (or continue) Cluster evaluation

6











1

2

3

4

5

7

6

7

8



• Memory optimized tables with durability

• Predictable Low-Latency, Bounded Access Time REAL-TIME

• Auto-Sharding, Multi-Master

• ACID Compliant, OLTP + Real-Time Analytics HIGH SCALE, READS + WRITES

• Shared nothing, no Single Point of Failure

• Self Healing + On-Line Operations 99.999% AVAILABILITY

• Key/Value + Complex, Relational Queries

• SQL + Memcached + JavaScript + Java + HTTP/REST & C++ SQL + NoSQL

• Open Source + Commercial Editions

• Commodity hardware + Management, Monitoring Tools LOW TCO

MySQL Cluster Overview – WHAT

8


Brief Introduction to MySQL Cluster MySQL Cluster GA – WHAT

9



Web

High volume OLTP

eCommerce

User Profile Management

Session Management & Caching

Content Management

On-Line Gaming

MySQL Cluster Deployments – WHERE

10

Telecoms

Subscriber Databases (HLR / HSS)

Service Delivery Platforms

VAS: VoIP, IPTV & VoD

Mobile Content Delivery

Mobile Payments

LTE Access

Case studies: www.mysql.com/customers/cluster/

http://www.mysql.com/customers/cluster/


• Scalability demands

– Sharding for write performance?

• Latency demands

– Cost of each millisecond?

• Uptime requirements – Cost per minute of downtime?

– Failure versus maintenance?

• Application agility

– Developer languages and frameworks?

– SQL or NoSQL?

• MySQL Cluster is designed for: – Short transactions

– Many parallel transactions


11

MySQL Cluster Architecture – WHEN


Brief Introduction to MySQL Cluster MySQL Cluster Architecture – HOW

12

MySQL Cluster Data Nodes

Clients

Application Layer

Data Layer Management


Brief Introduction to MySQL Cluster MySQL Cluster Scaling – HOW

13


Clients

Management

Application Layer

Data Layer


Brief Introduction to MySQL Cluster MySQL Cluster HA – HOW

14

Clients

Management Data Layer

Application Layer




15

MySQL Cluster Customers – WHO

Questions?


16

WHAT

WHERE

WHEN

HOW

WHO











1

2

3

4

5

17

6

7

8



Open the HOL instructions and enter the VM:

Open the terminal:

• gedit HOL3348_Instructions.txt &

• ssh [email protected] (Password: ouser)

It’s your turn

18

Hands-on Lab

mailto:[email protected]

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 19

Generic Linux package


Unpack of the package

Creation of the needed directories

Installation of SQL nodes

Creation of the configuration

files

1 2 3 4



Execute the commands at section #1 of the HOL instructions

It’s your turn

20

Hands-on Lab


Section #1.1 of HOL instructions

Commands:

• cd $HOME/software/generic

• tar -xzfv mysql-cluster-gpl-7.4.7-linux-glibc2.5-i686.tar.gz

21

Unpack of the package



Commands:

• mkdir $HOME/cluster

• mkdir $HOME/cluster/7.4.7

• mkdir $HOME/cluster/7.4.7/conf $HOME/cluster/7.4.7/mysqld1_data

$HOME/cluster/7.4.7/mysqld2_data

$HOME/cluster/7.4.7/ndb_data

• ls -l $HOME/cluster/7.4.7/

22

Creation of the needed directories



Commands:

• cd $HOME/software/generic/mysql-cluster-gpl-7.4.7-linux-glibc2.5-i686

• scripts/mysql_install_db --no-defaults --datadir=$HOME/cluster/7.4.7/mysqld1_data/

• scripts/mysql_install_db --no-defaults --datadir=$HOME/cluster/7.4.7/mysqld2_data/

23

Installation of SQL nodes

Initialize MySQL Data Directory

Do not read any option files



Commands:

• cd $HOME/cluster/7.4.7/conf/

The files to be created are:

–my1.cnf

–my2.cnf

–config.ini

24

Creation of the configuration files

Specifies options for all MySQL Cluster executables

Contains the configuration of each node involved in the Cluster



[mysqld]

ndbcluster

datadir=/home/ouser/cluster/7.4.7/mysqld1_data

basedir=/home/ouser/software/generic/mysql-cluster-

gpl-7.4.7-linux-glibc2.5-i686

port=3307

log_error=/home/ouser/cluster/7.4.7/mysqld1_data/mys

qld.log

25

my1.cnf

Enables NDBCLUSTER storage engine



[ndb_mgmd]

hostname=localhost

PortNumber=1186

datadir=/home/ouser/cluster/7.4.7/ndb_data

NodeId=49

ArbitrationRank=1

[ndbd default]

NoOfReplicas=2

datadir=/home/ouser/cluster/7.4.7/ndb_data

[ndbd]

hostname=localhost

NodeId=1

[ndbd]

hostname=localhost

NodeId=2

[mysqld]

NodeId=50

[mysqld]

NodeId=51

26

config.ini

The [ndb_mgmd] sections are used to configure the behavior of each one of the Cluster’s Management server: • PortNumber: port number (default value) on which the management server listens for configuration requests and management commands • NodeID: each node in the cluster has an unique identity • ArbitrationRank: used to define which nodes can act as arbitrators (1 is the default value)

The [ndbd default] section is used to configure the behavior of all the Cluster's Data nodes: • NoOfReplicas: defines the number of replicas for each table stored in the cluster (2 is the default value)

The [ndbd] sections are used to configure the behavior of each one of the Cluster's Data nodes: • NodeID: each node in the cluster has an unique identity

The [mysqld] sections are used to define the behavior of the MySQL servers (SQL nodes): • NodeID: each node in the cluster has an unique identity











1

2

3

4

5

27

6

7

8


Start of MySQL Cluster

28

Ordered start of the nodes

Management node (ndb_mgmd)

Data nodes (ndbd/ndbmtd)

SQL nodes (mysqld)

1 2 3


Start of MySQL Cluster


It’s your turn

29

Hands-on Lab



Command:

• ndb_mgmd -f $HOME/cluster/7.4.7/conf/config.ini --configdir=$HOME/cluster/7.4.7/conf/ --initial

-f : specify the cluster configuration file

--configdir : specify the cluster management server's configuration cache directory

--initial : causes the management server reload its configuration data from the configuration file, bypassing the configuration cache

Start of Management node

30

Suggestion: open now a new terminal to monitor the Cluster log: tail –f ndb_49_cluster.log


Sections #2.2 & #2.3 of HOL instructions

Commands:

• ndbd -c localhost:1186 --initial --ndb-nodeid=1

• ndbd -c localhost:1186 --initial --ndb-nodeid=2

-c : set connection string for connecting to ndb_mgmd

--initial : perform initial start of the Data node, including cleaning the file system

--ndb-nodeid : set the respective Data node’s ID

31

Start of Data nodes (ndbd/ndbmtd)

Note: an --initial start is to be used only when starting the ndbd process under very special circumstances; this is because this option causes all files to be removed from the MySQL Cluster file system and all redo log files to be re-created.


Sections #2.4 & #2.5 of HOL instructions

Commands:

• mysqld --defaults-file=$HOME/cluster/7.4.7/conf/my1.cnf &

• mysqld --defaults-file=$HOME/cluster/7.4.7/conf/my2.cnf &

32

Start of SQL nodes (mysqld)


Monitoring of MySQL Cluster

•Cluster log: ndb_<mgm-nodeid>_cluster.log

•Data node’s logs: output ; error ; trace

Log files

•shell> ps aux | grep -i ndb

•shell> ps aux | grep -i mysql

Cluster processes

•SHOW

•ALL | <nodeid> STATUS

•ALL | <nodeid> REPORT MEMORY | BackupStatus | EventLog

ndb_mgm commands

•memoryusage

•nodes

•cluster_transactions

•restart_info

•disk_write_speed_aggregate

•etc

ndbinfo database

•MySQL Cluster Memory

•MySQL Cluster Log buffers and files

•MySQL Cluster cache

•MySQL Cluster node status

MySQL Enterprise Monitor (MEM)

33

Manual: ndbinfo

Manual: MEM

https://dev.mysql.com/doc/en/mysql-cluster-ndbinfo.html

https://www.mysql.com/products/enterprise/monitor.html


Monitoring of MySQL Cluster processes


It’s your turn

34

Hands-on Lab


Section #3 of HOL instructions

Are the Cluster processes running?

• ps aux | grep -i ndb

• ps aux | grep –i mysqld

Tail of the Management log:

• tail –f ndb_49_cluster.log

35

Monitoring of MySQL Cluster processses & logs











1

2

3

4

5

36

6

7

8



37

Administration + SQL & NoSQL access to Cluster data

Management client (administration)

MySQL client (SQL access)

Connectors & APIs (SQL or NoSQL access)

1 2 3




It’s your turn

38

Hands-on Lab



Command in shell:

• ndb_mgm -c localhost:1186

Command in Management Client (ndb_mgm):

• SHOW

• EXIT

Management Client (administration)

39



Command in shell:

• mysql -uroot -P3307

Commands in MySQL Client (mysql):

• USE ndbinfo

• SHOW TABLES;

• SELECT * FROM memoryusage;

• SELECT * FROM nodes;

• exit

MySQL Client (SQL access)

40











1

2

3

4

5

41

6

7

8



42

A two-step process

Use of the Cluster’s “shutdown” command

Shutdown (or not) all the SQL nodes

1 2


MySQL Cluster shutdown

• Cluster's "shutdown" command: ndb_mgm> shutdown

Node 1: Cluster shutdown initiated

Node 2: Cluster shutdown initiated

Node 2: Node shutdown completed.

Node 1: Node shutdown completed.

3 NDB Cluster node(s) have shutdown.

Disconnecting to allow management server to shutdown.

• Tail of the Management log: 2015-08-26 12:30:15 [MgmtSrvr] INFO -- Node 1: Cluster shutdown initiated

2015-08-26 12:30:15 [MgmtSrvr] INFO -- Node 2: Cluster shutdown initiated

2015-08-26 12:30:22 [MgmtSrvr] ALERT -- Node 49: Node 2 Disconnected

2015-08-26 12:30:22 [MgmtSrvr] INFO -- Node 2: Node shutdown completed.

2015-08-26 12:30:22 [MgmtSrvr] INFO -- Node 1: Node shutdown completed.

2015-08-26 12:30:22 [MgmtSrvr] ALERT -- Node 49: Node 1 Disconnected

2015-08-26 12:30:25 [MgmtSrvr] INFO -- Shutting down server...

2015-08-26 12:30:33 [MgmtSrvr] INFO -- Shutdown complete

43

Sample output











1

2

3

4

5

44

6

7

8


Overview of main MySQL Cluster’s Admin Commands & Operations

• Management Client's (ndb_mgm) Commands: HELP, SHOW, CREATE | DROP NODEGROUP

<nodeid> START | STOP | RESTART

START BACKUP, ENTER | EXIT SINGLE USER MODE

CLUSTER LOG ON | OFF | INFO | FILTER <severity_level>

<nodeid> CLUSTERLOG category=<threshold>

ALL | <nodeid> REPORT MEMORY | BackupStatus | EventLog

SHUTDOWN, QUIT | EXIT

• Cluster Operations:

– On-line Rolling Restart • On-line Scaling (add/remove nodes)

• On-line Reconfiguration

• On-line Upgrades

– On-line Backup & Restore

45



Command in shell:

• ndb_mgm -c localhost:1186


• ALL REPORT MEMORY

• ALL REPORT BackupStatus

46

MySQL Cluster’s Admin Commands




• 1 RESTART

• ALL STATUS Note: keep running ALL STATUS during the restart of node 1 so you may see the node passing through some different start phases.

47

MySQL Cluster’s Admin Commands











1

2

3

4

5

48

6

7

8


MySQL Cluster Carrier Grade Edition

49

More: www.mysql.com/buy-mysql

No Trade-Offs

On the Top

http://www.mysql.com/buy-mysql




MySQL Cluster Manager (MCM)

50

Automated Management

• Start / Stop node or whole cluster

• On-Line Scaling

• On-Line Reconfiguration

• On-Line Upgrades

• On-Line Backup & Restore

• Import Running Cluster

Self-Healing

• Node monitoring

• Auto-recovery extended to SQL + mgmt nodes

HA Operations

• Cluster-wide configuration consistency

• Persistent configurations

• HA Agents


MySQL Enterprise Monitor (MEM) Enterprise Monitor, Advisors & Query Analyzer

51

Your virtual MySQL DBA Assistant


MySQL Cluster Auto-Installer

• Fast configuration

• Auto-discovery

• Workload optimized

• Repeatable best practices

52

Specific Workload

Auto-Discover

Define Topology

Deploy


MySQL Cluster Auto-Installer


Oracle Premier Support for MySQL

• Straight from the Source • Largest Team of MySQL Experts • Backed by MySQL Developers • Forward Compatible Hot Fixes • MySQL Maintenance Releases • MySQL Support in 29 Languages • 24/7/365 • Unlimited Incidents • Knowledge Base • MySQL Consultative Support

54

Rely on The Experts - Get Unique Benefits

Only From

Oracle

"The analyst quickly answered my questions, and went beyond what I expected, providing resources to get additional information and understanding.“

mysql.com/support/quotes

http://www.mysql.com/support/quotes.html


Resources m

ysq

l.co

m TCO calculator

White papers

dev

.mys

ql.c

om

Community downloads

Documentation

Dev articles

Forums

PlanetMySQL

eDel

iver

y.co

m Oracle Software Delivery

Cloud

Download and evaluate all MySQL products

sup

po

rt.o

racl

e.co

m Knowledge Base

• Doc ID 1389140.1: Is MySQL Cluster the Right Choice Compared with InnoDB?

• Doc ID 1023119.1: How to Add New Nodes To an Existing MySQL Cluster Setup?

• Doc ID 1926680.1: Best Practices For MySQL Cluster

• Doc ID 2028705.1: How to Upgrade/Downgrade MySQL Cluster When Using MySQL Cluster Manager (MCM)

• Doc ID 1941472.1: How to Upgrade MySQL Cluster using RPM

Service request (SR)

Patches

Bugs/Defects

Etc

55

MySQL Support: www.mysql.com/support

https://support.oracle.com/epmos/faces/DocumentDisplay?id=1389140.1





http://www.mysql.com/support


Summary

• Web-Scale Performance with Carrier-Grade Availability

• Scale-Out

• Real Time Performance

• 99.999% Uptime

• SQL & NoSQL Access Methods

56

Q&A

Thank you for your attention

57


Safe Harbor Statement

The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

58


Oracle OpenWorld - Getting started with MySQL Cluster

Technology