Webinar - Manage Galera Cluster with Puppet

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

Manage Galera Cluster with Puppet

Configuration management with puppet

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

Who am I?

• Krishna Prajapati, MySQL Engineer at Olindata

http://www.olindata.com/

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

Overview• What is puppet (for those not aware)?• What is Galera Cluster?• Terminology• Block Diagram/Feature• MySQL Parameter for Galera

• Puppet Module for Galera

• Install Galera health check

• Haproxy

• wsrep status

• Questions

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

What is Puppet and why do we care?

• Configuration management software- http://www.olindata.com/blog/2014/08/puppet-master-agent-setup- http://olindata.com/blog/2015/03/setup-puppet-server-centos-70

• Scales very well (from 1 to 200k+ nodes)

• Multi-platform (windows, *nix, Mac OS, BSD)

• Commercially supported Open Source

• Infrastructure as code

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

What is Galera Cluster ?

• Galera is a synchronous multimaster database cluster

• Opensource

• Multi-platform (*nix)

• Consists of

- Percona

- MariaDB

- MySQL

• HA

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

Terminology

• Synchronous vs Asynchronous

• Failover

• Split Brain

• GTID

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

Block Diagram

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

Galera Features• True Multi-master Read and write to any node at any time.

• Synchronous Replication No slave lag, no data is lost at node

crash.

• Tightly Coupled All nodes hold the same state. No diverged

data between nodes allowed.

• Multi-threaded Slave For better performance. For any

workload.

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

• No Master-Slave Failover Operations or Use of VIP.

• Hot Standby No downtime during failover (since there is no

failover).

• Automatic Node Provisioning No need to manually back up

the database and copy it to the new node.

• Supports InnoDB.

• Transparent to Applications Required no (or minimal)

changes) to the application.

• No Read and Write Splitting Needed.

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

[mysqld]

# (This must be substituted by wsrep_format)

binlog_format=ROW

# Currently only InnoDB storage engine is supported

default-storage-engine=innodb

# to avoid issues with 'bulk mode inserts' using autoinc

innodb_autoinc_lock_mode=2

# This is a must for paralell applying

innodb_locks_unsafe_for_binlog=1

# Query Cache is not supported with wsrep

query_cache_size=0

query_cache_type=0

# Override bind-address

# In some systems bind-address defaults to 127.0.0.1, and with mysqldump SST

bind-address=0.0.0.0

MySQL Parameter for Galera

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

1. Github/olindata https://github.com/olindata/puppet-galera

2. Demo.

https://github.com/olindata/olindata-galera-puppet-demo

3. Haproxy (load-balancer)

https://github.com/olindata/puppetlabs-haproxy

Puppet Module for Galera?

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

Install Galera using puppet moduleclass profile::mysql::base {

include mysql::client

# the base profile contains logic to choose either profile::apt or profile::yum

Class['profile::base'] ~> Package['mysql_client']

Class['profile::base'] ~> Package['percona-toolkit']

Class['profile::base'] ~> Package['mysql-server']

Exec['apt_update'] -> Package['mysql_client'] -> Package['mysql-server']

# load the mysql options from hiera and pass them to mysql::server

$override_options = hiera_hash('profile::mysql::base::override_options', undef)

if $override_options {

class { 'mysql::server':

override_options => $override_options,

require => [Package['mysql_client']]

}

} else {

# we don't use default options, so error if nothing is found

fail('No hash found in hiera to override default mysql settings in class profile::mysql::base.')

}

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

# make sure percona toolkit and xtrabackup are present

package { 'percona-toolkit': ensure => latest, }

package { 'xtrabackup':

ensure => latest,

require => Class['profile::apt'],

}

}

From hiera

mysql::client::package_name: 'percona-xtradb-cluster-client-5.6'

mysql::server::package_name: 'percona-xtradb-cluster-server-5.6'

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

Install galera health checkclass profile::mysql::galera {

include profile::mysql::base

include ::galera

include galera::health_check

# load haproxy frontends

$balancermembers = hiera_hash('profile::mysql::galera::balancermembers', undef)

if $balancermembers {

create_resources('@@haproxy::balancermember', $balancermembers)

}

}

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

health checkroot@db01:~# clustercheck HTTP/1.1 200 OK

Content-Type: text/html

Content-Length: 43

<html><body>MySQL is running.</body></html>

root@db01:~#

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

Install haproxy moduleclass profile::haproxy::galera {

include ::haproxy

# Make sure the wheezy backports repo is present before attempting the package

if $::lsbdistcodename == 'wheezy' {

Apt::Source['wheezy-backports'] -> Package['haproxy']

}

# load haproxy frontends

$listens = hiera_hash('profile::haproxy::galera::listen', undef)

if $listens {

create_resources('haproxy::listen', $listens)

} else {

fail('No listen settings found in hiera in class profile::haproxy::galera.')

}

}

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

Rolesclass role::db::galera inherits role {

include profile::mysql::galera

}

class role::db::galeramaster inherits role::db::galera {

include profile::haproxy::galera

}

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

wsrep statusmysql> show global status like '%wsrep%';

+------------------------------+----------------------------------------------------------+

| Variable_name | Value |

+------------------------------+----------------------------------------------------------+

| wsrep_local_state_uuid | 8fbd0b15-8db0-11e4-8fa3-7770ec850cad |

….

| wsrep_commit_window | 0.000000 |

| wsrep_local_state | 4 |

| wsrep_local_state_comment | Synced |

| wsrep_cert_index_size | 0 |

| wsrep_causal_reads | 0 |

| wsrep_cert_interval | 0.000000 |

| wsrep_incoming_addresses | 192.168.56.32:3306,192.168.56.34:3306,192.168.56.33:3306 |

| wsrep_evs_delayed | |

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

wsrep status| wsrep_incoming_addresses | 192.168.56.32:3306,192.168.56.34:3306,192.168.56.33:3306 |

| wsrep_evs_delayed | |

| wsrep_evs_evict_list | |

| wsrep_evs_repl_latency | 0/0/0/0/0 |

| wsrep_evs_state | OPERATIONAL |

| wsrep_gcomm_uuid | 0d150685-d121-11e4-9961-0f0d19f43e8f |

| wsrep_cluster_conf_id | 6 |

| wsrep_cluster_size | 3 |

| wsrep_cluster_state_uuid | 8fbd0b15-8db0-11e4-8fa3-7770ec850cad |

| wsrep_cluster_status | Primary |

| wsrep_connected | ON |

| wsrep_local_bf_aborts | 0 |

| wsrep_local_index | 0 |

| wsrep_provider_name | Galera |

| wsrep_provider_vendor | Codership Oy <[email protected]> |

| wsrep_provider_version | 3.8(rf6147dd) |

| wsrep_ready | ON |

+------------------------------+----------------------------------------------------------+

56 rows in set (0.00 sec)

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

node 'master.olindata.vm' {

# this server is a puppetmaster

include role::master

}

node 'galera000.olindata.vm' {

# this server will be a galera node, also having an haProxy instance

include role::db::galeramaster

}

node /^galera\d{3}.olindata.vm$/ {

# all other nodes are galera nodes

include role::db::galera

}

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

• Other mysql flavours like MariaDB, Percona can be installed with the existing module

class { '::mysql::server':

package_ensure => 'present',

package_name => 'mariadb-server',

}

Galera Flavour Install

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

Upcoming traininghttp://olindata.com/training/upcoming

Puppet Fundamentals Training, Manila – April 2015

Monday, April 6, 2015 - 09:00

Philippines

Puppet

Puppet Fundamentals Training, Hyderabad – April 2015

Wednesday, April 15, 2015 - 09:00

India

Puppet

Puppet Fundamentals Training, Jakarta – April 2015

Monday, April 20, 2015 - 09:00

Indonesia

Puppet

Puppet Fundamentals Training, Vienna – April 2015

Monday, April 27, 2015 - 09:00

Austria

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

Previous Webinars

• https://www.youtube.com/results?search_query=olind

ata

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

We’re hiring!EU and Asia based trainers

[email protected]

OlinData Webinar 2015 -https://docs.google.com/a/olindata.com/presentation/d/1vlwtPNRGkaIvJN0olgwknyDvVha910yb4_bM5UfXE8

I/edit?usp=sharing

Questions?

@krishna / @olindata

http://www.olindata.com

[email protected]

http://github.com/olindata