Copyright 2017 Severalnines AB I'm Jean-Jérôme from the Severalnines Team and I'm your host for today's webinar! Feel free to ask any questions in the Questions section of this application or via the Chat box. You can also contact me directly via the chat box or via email: [email protected] during or after the webinar. Your host & some logistics
57
Embed
Webinar slides: MySQL on Docker: Understanding the Basics
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Copyright 2017 Severalnines AB
I'm Jean-Jérôme from the Severalnines Team and I'm your host for today's webinar!
Feel free to ask any questions in the Questions section of this application or via the Chat box.
You can also contact me directly via the chat box or via email: [email protected] during or after the webinar.
Your host & some logistics
Copyright 2017 Severalnines AB
Copyright 2017 Severalnines AB
About Severalnines and ClusterControl
Copyright 2017 Severalnines AB
What We Do
Copyright 2017 Severalnines AB
ClusterControl Automation & Management
Management● Multi-Cluster / Multi-DC
● Automate Repair &
Recovery
● Database Upgrades
● Backups
● Configuration Management
● Database Cloning
● One-Click Scaling
Deployment● Deploy a Cluster in Minutes
● On-Premises or in the Cloud (AWS)
Monitoring● Systems View with 1sec Resolution
● DB / OS stats & Performance Advisors
● Configurable Dashboards
● Query Analyzer
● Real-time / historical
Copyright 2017 Severalnines AB
Supported Databases
Copyright 2012 Severalnines ABCopyright 2012 Severalnines AB
# Run a MySQL container$ docker run -d \--name my-test \--env MYSQL_ROOT_PASSWORD=mypassword \mysql
# Get the container layer size$ docker ps -sCONTAINER ID IMAGE NAMES SIZEd51f459239fb mysql my-test 4B (virtual 412MB)
Image: docker.io/mysql:latest
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
d51f459239fb
0639788facc8
35369f9634e1
937bbdd4305a
my-test
Image layers (RO)
412 MB
Container layers (RW)4 B
Docker Container - Run another Container
Copyright 2012 Severalnines AB
# Run a MySQL container$ docker run -d \--name my-test2 \--env MYSQL_ROOT_PASSWORD=mypassword \mysql
# Get the container layer size$ docker ps -sCONTAINER ID IMAGE NAMES SIZEd51f459239fb mysql my-test 4B (virtual 412MB)e9de9ed50ced mysql my-test2 4B (virtual 412MB)
Image: docker.io/mysql
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
d51f459239fb
0639788facc8
35369f9634e1
937bbdd4305a
my-test
Image layers (RO)
412 MB
Container layers (RW)
4 B + 4 B
e9de9ed50ced
my-test2
Docker Container - Container Layer Changes
Copyright 2012 Severalnines AB
# Operation 2 - Create a schema and table$ docker exec -it my-test \mysql -uroot -pmypassword -e \'CREATE SCHEMA testdb; CREATE TABLE t1 (id INT PRIMARY KEY AUTO_INCREMENT, data TEXT);'
# Mount a named volume into container-v mysql1-data:/var/lib/mysql
# Mount tmpfs into container--tmpfs /tmp
● Bind mount:○ A file or directory on the host machine is
mounted into a container.○ You manage the directory’s contents.
● Named volume:○ A new directory is created within Docker’s
storage directory.○ Docker manages the directory’s contents.○ Extensible via volume plugin.
● tmpfs:○ Stored on host machine’s memory (or swap).○ Linux only.
# Bind mount a directory into container-v /data/mysql1/data:/var/lib/mysql
# Bind mount a file into container-v /data/mysql1/my.cnf:/etc/mysql/my.cnf
Docker Volume - Persistent Volumes
Copyright 2012 Severalnines AB
# Run with persistent named volume, using -v or --volume$ docker run -d \--name mysql-local \--env MYSQL_ROOT_PASSWORD=mypassword \--volume local-datadir:/var/lib/mysql \mysql:5.7
Image: docker.io/mysql:5.7
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-local
416f2a92355f
/var/lib/mysql
local-datadir
# Run with persistent named volume, using --mount$ docker run -d \--name mysql-local \--env MYSQL_ROOT_PASSWORD=mypassword \--mount source=local-datadir,target=/var/lib/mysql \mysql:5.7
Docker Volume - Changes on Volume
Copyright 2012 Severalnines AB
# Run a container with port 3308$ docker run -d \--name mysql-local \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v local-datadir:/var/lib/mysql \mysql
Image: docker.io/mysql:latest
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-local
416f2a92355f
/var/lib/mysql
local-datadir
# Run with persistent named volume, using --mount$ mysql -uroot -p -h127.0.0.1 -P3308 < dump.sql
dump.sql
3308
Docker Volume - Remote Volume
Copyright 2012 Severalnines AB
# Run a container with NFS bind mount$ docker run -d \--name mysql-nfs \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v /nfs/mysql-nfs:/var/lib/mysql \mysql
Image: docker.io/mysql
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-nfs
416f2a92355f
/var/lib/mysql
192.168.1.100/storage/docker
# Mount NFS with extra options for MySQL$ mount 192.168.1.100:/storage/docker /nfs -o noatime,nodiratime,hard,intr
3308
/nfs/mysql-nfs
Further Reading - Using NFS with MySQL:https://dev.mysql.com/doc/refman/5.7/en/disk-issues.html#disk-issues-nfs
General log general_log_file {datadir}/{hostname}.log
Docker Volume - Non-Persistent Volume
Copyright 2012 Severalnines AB
# Run a container with NFS bind mount and tmpfs volumes$ docker run -d \--name mysql-nfs-tmp \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v /nfs/mysql-nfs:/var/lib/mysql \--tmpfs /tmp:rw,size=1g,mode=177 \mysql
Image: docker.io/mysql:latest
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-nfs-tmp
416f2a92355f
/var/lib/mysql
192.168.1.100/storage/docker
3308
/nfs/mysql-nfs
/tmp
Docker Volume - Drivers
Copyright 2012 Severalnines AB
● Default is local.● You can use other different volume driver (or plugin).
○ Install the plugin.○ Use --volume-driver option.
● Volume drivers allow you to extend Docker's volume capabilities:
● Packet forwarding by iptables to the bridge network.● --publish is supported. Run multiple containers with
same image through different ports.● --link is supported. Link with other containers to
expose environment variables.● docker-proxy redirects connection to the correct
container through NAT.
docker0(virtual Ethernet bridge)
vethxxx vethyyy
Docker Network - User-Defined Bridge Network
Copyright 2012 Severalnines AB
# Run a container on user bridge network$ docker run -d \--name mysql1 \--net db_single \-p 3308:3306 \--ip 192.168.10.10 \--hostname mysql1 \-e MYSQL_ROOT_PASSWORD=mypassword \mysql:5.7
Container 1
eth0
eth0
Container 2
eth0
Docker
Host
# Create a new bridge network$ docker network create subnet=192.168.10.0/24 db_single
Similar to the default docker0 bridge, plus:● --hostname is supported for sticky hostname.● --ip is supported for sticky IP address.● --link is not supported. Use embedded DNS to
resolve container's name in the same network.
db_single(virtual Ethernet bridge)
vethxxx vethyyy
DNS
Docker Network - Plugins
Copyright 2012 Severalnines AB
● Similar to volume, Docker network can be extended through plugins:
○ Multi-host networking (must use outside of Swarm)
○ Name resolver and service discovery.○ Encryption.
● Popular network plugins:○ Contiv○ Calico○ Weave○ Flannel
Further Reading - Multi-host Networking with Calico:https://severalnines.com/blog/mysql-docker-multi-host-networking-mysql-containers-part-2-calico
# Run a service on default overlay network$ docker service create \--name mysql-service \--replicas 1 \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v nfs-datadir:/var/lib/mysql \mysql:5.7
Container 1
eth0
eth0 eth1
Docker
Host
# Initialize Docker Swarm$ docker swarm init
● Docker Swarm must be enabled.● Each task (container) has:
○ eth0 - Interface to external network via docker_gwbridge.
○ eth1 - Interface to overlay network via VXLAN.
docker_gwbridge
vethxxx vethyyy
br0
External
Container 2
eth0 eth1
Docker
docker_gwbridge
vethxxx vethyyy
br0
eth0Host
VXLANudp/4789
Docker Network - User-Defined Overlay Network
Copyright 2012 Severalnines AB
# Run a service on default overlay network$ docker service create \--name mysql-service \--replicas 1 \--net db_multi \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v nfs-datadir:/var/lib/mysql \mysql:5.7
# Bring up the production MySQL container in background$ docker-compose up -dCreating network "compose_db-prod" with driver "bridge"Creating compose_mysql-prod_1
Image: docker.io/mysql:5.7.18
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-prod
416f2a92355f
/var/lib/mysql
mysql-datadir
/tmp
mysql-tmp
3307
# Stop the container$ docker-compose stop
# Destroy everything$ docker-compose down
# Start the container$ docker-compose start
Scaling is supported via docker-compose scale, however conflicting options must be omitted:hostname, ports, ipv4_address, volume
Docker Orchestration - Docker Swarm
Copyright 2012 Severalnines AB
● Multi-host deployment, through overlay network.● Concepts:
○ Service■ Replicated - Number of replicas■ Global - One task per available node
○ Task■ Container
○ Mesh routing■ All Swarm nodes route to the running tasks■ Ingress load balancing (round-robin only)
○ Scheduler■ Resources availability■ Label and constraints
Docker Orchestration - Simple Auto Failover using Swarm
Copyright 2012 Severalnines AB
# Run a replica of MySQL service$ docker service create \--name mysql-service \--replicas 1 \--net db_multi \-p 3307:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \--mount source=ebs-datadir,destination=/var/lib/mysql,volume-driver=rexray \--mountsource=ebs-conf,destination=/etc/my.cnf.d,volume-driver=rexray \mysql:5.7
● Recommended volumes for data persistence:○ /etc/cmon.d○ /var/lib/mysql○ /root/.ssh○ /root/backups
● ClusterControl UI is exposed at 80 and 443 by default.● Service control via supervisord: sshd, mysqld, httpd,
cmon, cmon-ssh, cmon-events, cc-auto-deployment● Run it as:
○ --net=host, if you want this container to manage host or container outside of Docker network.
○ --net={net_name}, if you want CC container to manage containers in the same Docker network.
# Run a ClusterControl container$ docker run -d --name=clustercontrol \-p 5000:80 \-p 5001:443 \-h clustercontrol \-v /storage/clustercontrol/.ssh:/root/.ssh \-v /storage/clustercontrol/datadir:/var/lib/mysql \-v /storage/clustercontrol/cmon.d:/etc/cmon.d \-v /storage/clustercontrol/backups:/backups \severalnines/clustercontrol
# Run a 3 node Galera cluster, with auto deployment$ for i in {1..3}; do docker run -d \ --name galera${i} \ -p 666{$i}:3306 \ --link clustercontrol:clustercontrol \ -e CLUSTER_TYPE=galera \ -e CLUSTER_NAME=mygalera \ -e INITIAL_CLUSTER_SIZE=3 \ severalnines/centos-sshdone
ClusterControl on Docker - Auto DB Cluster Deployment
Copyright 2012 Severalnines AB
● Use "severalnines/centos-ssh" image - configures passwordless SSH automatically from CC to DB containers and register the container with CC.
● Only Galera is supported (more to come).● Background script, cc-auto-deployment:
○ Monitors new containers registration.○ Triggers deployment job if AUTO_DEPLOYMENT=1
(default).○ Uses ClusterControl CLI, called 's9s' to interact
with cmon backend.● Deployment flow:
○ You create the container.○ cc-auto-deployment picks up the registered
containers and creates a deployment job.○ ClusterControl deploys the cluster.
ClusterControl on Docker - Manual DB Cluster Deployment
● Use "severalnines/centos-ssh" image with AUTO_DEPLOYMENT=0.
● Allows user to have better control on the deployment process initiates by ClusterControl.
● All supported database cluster can be deployed using this method.
● Deployment flow:a. You create the containers.b. You create the database deployment job
through ClusterControl UI.c. ClusterControl deploys the cluster.
Copyright 2012 Severalnines AB
Further Reading - Example at Github:https://github.com/severalnines/docker/tree/master/examples/docker