MySQL overview

MySQL Overview a quick look at the general aspects of the most popular open source database

Marco Tusa

MySQL Team Leader

16 September 2011

2 © 2010/2011 Pythian

Why Companies Trust Pythian •  Recognized Leader: •  Global industry-leader in remote database administration services and consulting for Oracle,

Oracle Applications, MySQL and SQL Server

•  Work with over 150 multinational companies such as Forbes.com, Fox Interactive media, and MDS Inc. to help manage their complex IT deployments

•  Expertise:

•  One of the world’s largest concentrations of dedicated, full-time DBA expertise.

•  Global Reach & Scalability:

•  24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response

2

3 © 2010/2011 Pythian

Why Companies Trust Pythian

•  Recognized Leader: • Global industry-leader in remote database administration services and consulting for Oracle, Oracle Applications, MySQL and SQL Server

• Work with over 150 multinational companies such as Toyota, Fox Interactive media, and MDS Inc. to help manage their complex IT deployments

•  Expertise: • One of the world’s largest concentrations of dedicated, full-time DBA expertise.

•  Global Reach & Scalability: • 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response

2

4 © 2010/2011 Pythian

Who I am • Currently: Team Lead at The Pythian Group for MySQL database

support

• Manager Professional Service South EMEA at MySQL/SUN/Oracle

• In MySQL before the SUN gets on us

• Leading the team responsible for Oracle & MySQL DBs service in support to technical systems, at Food and Agriculture Organization of United Nations (FAO of UN)

• Leading developer & system administrator teams in FAO managing the Intranet/Internet infrastructure.

• Working (a lot) in developing countries like (Ethiopia, Senegal, Ghana, Egypt …)

• My Profile http://it.linkedin.com/in/marcotusa

3

5 © 2010/2011 Pythian

What is MySQL?

• MySQL is a database management system

• MySQL is a relational database management system

• MySQL software is Open Source.

• MySQL Server works in client/server or embedded systems.

• The official way to pronounce “MySQL” is “My Ess Que Ell” (not “my sequel”)

20

6 © 2010/2011 Pythian

Who is using MySQL

List of customer from MySQL official presentation

20

7 © 2010/2011 Pythian

Main insight on MySQL 1/2

• Written in C and C++.

• Works on many different platforms:

•  FreeBSD 5.x.

•  HP-UX 11.x.

•  Linux.

•  Mac OS X.

•  Solaris 2.8 on SPARC and x86

•  Windows XP, Windows Vista, Windows Server 2003, and Windows Server 2008.

20

8 © 2010/2011 Pythian

Main insight on MySQL 2/2

• Uses multi-layered server design with independent modules.

• Designed to be fully multi-threaded using kernel threads, to easily use multiple CPUs if they are available.

• Provides transactional and non-transactional storage engines. • Designed to make it relatively easy to add other storage engines. This is useful if you want to provide an SQL interface for an in-house database.

• Uses a very fast thread-based memory allocation system.

• Full Localization support

20

9 © 2010/2011 Pythian

What Standards MySQL Follows?

• Supports the full ANSI/ISO SQL standard, but with some flexibility (SQL-92=1992 Release; SQL:1999=1999 Release; SQL:2003=2003 Release; SQL:2008= 2008 Release). http://www.iso.org/iso/home.htm

• ODBC levels 0 to 3.51.

• MySQL can adapt its behavior using SQL Modes like:

•  ANSI (SET GLOBAL sql_mode='ANSI';)

•  Oracle

•  Custom …

• MySQL DATA is structured on the file system by directory and subdirectory (symbolic links widely used)

20

10

© 2010/2011 Pythian

Installing MySQL in 4 easy steps

Installing MySQL is a simple operation:

1. Download software from site like (http://dev.mysql.com/get/Downloads/MySQL-5.5/mysql-5.5.15-linux2.6-x86_64.tar.gz/from/http://mirror.csclub.uwaterloo.ca/mysql/)

2. Expand it in /usr/local/mysql (and set all grants for mysql user)

3. Copy one of the pre-configured my.cnf (from support-files directory) to /etc/my.cnf; copy from same source mysql.server file to /etc/init.d/mysql

4. Start mysql /etc/init.d/mysql start

You can have MySQL running in 30 minutes.

20

11

© 2010/2011 Pythian

MySQL Architecture 1/2 MySQL is composed of two main layers

20

12

© 2010/2011 Pythian

MySQL Architecture 2/2

• MySQL Layer (Parser; Optimizer; Query cache; threads handling )

•  Storage engines:

•  MyISAM ( no ACID)

•  Innodb (ACID)

•  Cluster (ACID)

•  Archive (only archive and append)

•  Memory (Main engine for in memory operations)

•  Blackhole (it is exactly what the name indicate)

•  .. many others

20

13

© 2010/2011 Pythian

MySQL Storage engine more details 1/2 They have different advantages / trade-offs - know when to use them.

1. MyISAM •  No transaction handling; no isolation level; no data buffered (only index); full Table

lock; no foreign key;

•  Fully file base; minimal overhead in write; very fast in reading; Btree index; Full text search; Geospatial support; 256TB Storage limit;

2. InnoDB •  No full text (5.6 will); no Geospatial support; no cluster support; 64TB Storage limit;

•  Full ACID; possible to modify the Isolation level; data is buffered; Write lock by ROW; Foreign-key support; automatic recovery from redo log in case of crash; Btree index; Clustered table; different file format (internal); data compression.

3. Memory •  No transaction; no version control; no Geospatial support; no foreign-key; full Table

lock;

•  Data cached in memory; Btree index; Hash index; Storage limit is the available RAM.

20

14

© 2010/2011 Pythian

MySQL Storage engine more details 2/2 1.  MySQL Cluster

•  No geospatial support; no foreign-key; no “Temporary” tables; partition only by [LINEAR] KEY; only the READ COMMITTED isolation level is supported; no partial roll-back (full or nothing); a Select on Blob or Text fields will create a READ LOCK on the table to guarantee consistency.

•  Fully ACID; share nothing architecture; single point of failure; write scale by node-group; thread requests parallelization; and a lot more …

20

15

© 2010/2011 Pythian

MySQL Logs 1/2 What they are and how MySQL uses them. • General Log

•  each SQL statement received from clients; connection/disconnection as well

•  Write intensive to be used ONLY for checking issues

• Binary log

•  Contains the statements “events” that modify data structure and datasets

•  Point in time recovery; Replication; Backup; can be (statement/mixed/row)

• Relay log

•  Is the local copy of the Slave; contains the same information of the Binlogs

• Slow log

• The Error Log

•  Self explanatory in the latest MySQL can write to syslog

20

16

© 2010/2011 Pythian

MySQL Logs 2/2

What they are and how MySQL uses them.

• Slow log

•  Information related to SLOW QUERY

•  Could be write intensive; long time and no index usage configurable

•  Crucial for SQL review

• The Error Log

•  If not specify --log-error errors are written to stderr

•  Warnings are written as well (log_warnings) to disable

•  From MySQL 5.1 can also write to syslog

20

17

© 2010/2011 Pythian

MySQL Replication

Replication topology, and when to use what.

• Master -> Slave

• Master <> Master

• Master -> Multi slave

• MySQL replication pro and cons

•  Single thread

•  Parallel replication

•  Asynchronous 20

18

© 2010/2011 Pythian

Scaling with MySQL Replication.

• Scaling for reads

•  Add slave(s) but … keep in mind PT = (ND + QT) * NL PT = Propagation Time ND = Network time delay QT = Query Execution Time NL = Number of Level

• Scaling for writes

•  Sharding or MySQL Cluster

• Scaling out vs. scaling up

•  Partition

•  Spread the load in smaller chunk

• Using the right approach

•  Database dimension

•  Activity (r/w) 20

19

© 2010/2011 Pythian

Partitioning 1/2 Use Partition to organize data

Max number of partition per table !1024!

Types of partitioning

• RANGE

•  Numeric range like (store Id; Year) less then “value”

• LIST

•  Declared list of value (matching one in the list)

• COLUMNS

•  RANGE: List of a single column value or more (partition definition value list) evaluation left->right

•  LIST: Declared list of value coming from one or more columns

•  No DECIMAL, FLOAT,TIMESTAMP,TEXT,BLOB

20

20

© 2010/2011 Pythian

Partitioning 2/2 Types of partitioning (cont.)

• HASH

•  Calculate an hash value on the given value (also expression) then equally distribute the data

• KEY •  Calculate an hash value on Primary/Unique

• Subpartitioning

•  Works for LIST and RANGE Partitioned tables

•  Subpartition only by HASH or KEY

Partition cannot be done on table with: Full Text index ; Spatial columns

20

21

© 2010/2011 Pythian

MySQL programming 2/2 A few point about performance • Not as good you would expect

• Optimize IF

• Recursive VS not recursive calculating Fibonacci numbers

20

22

© 2010/2011 Pythian

MySQL programming Using stored procedure (code for performance on prime numbers)

20

PROCEDURE N_PRIMES ( p_num NUMBER) IS i INT; j INT; nprimes INT; isprime INT; BEGIN i:=2; nprimes:=0; <<main_loop>> WHILE (i<p_num) LOOP isprime:=1; j:=2; <<divisor_loop>> WHILE (j<i) LOOP IF (MOD(i,j)=0) THEN isprime:=0; EXIT divisor_loop; END IF; j:=j+1; END LOOP ; IF (isprime=1) THEN nprimes:=nprimes+1; END IF; i:=i+1; END LOOP; dbms_output.put_line(nprimes||' prime numbers

less than '||p_num); END;

CREATE PROCEDURE sp_nprimes(p_num int) BEGIN DECLARE i INT; DECLARE j INT; DECLARE nprimes INT; DECLARE isprime INT;

SET i=2; SET nprimes=0;

main_loop: WHILE (i<p_num) do SET isprime=1; SET j=2; divisor_loop: WHILE (j<i) DO IF (MOD(i,j)=0) THEN SET isprime=0; LEAVE divisor_loop; END IF; SET j=j+1; END WHILE ; IF (isprime=1) THEN SET nprimes=nprimes+1; END IF; SET i=i+1; END WHILE; SELECT CONCAT(nprimes,' prime numbers less

than ',p_num); END;

23

© 2010/2011 Pythian

MySQL programming Using Store procedure (code for Fibonacci test)

20

Recursive

CREATE PROCEDURE rec_fib(n INT,OUT out_fib INT)

BEGIN DECLARE n_1 INT; DECLARE n_2 INT;

IF (n=0) THEN SET out_fib=0; ELSEIF (n=1) then SET out_fib=1; ELSE CALL rec_fib(n-1,n_1); CALL rec_fib(n-2,n_2); SET out_fib=(n_1 + n_2); END IF; END

Not Recursive CREATE PROCEDURE nonrec_fib(n INT,OUT out_fib INT) BEGIN DECLARE m INT default 0; DECLARE k INT DEFAULT 1; DECLARE i INT; DECLARE tmp INT;

SET m=0; SET k=1; SET i=1;

WHILE (i<=n) DO SET tmp=m+k; SET m=k; SET k=tmp; SET i=i+1; END WHILE; SET out_fib=m; END

24

© 2010/2011 Pythian

MySQL programming 1/2 •  Using Store procedure

• Programming is real at a basic level; no packages etc…

• Triggers and function could leads to unexpected results by Storage engine

•  Real status about store procedure

• Triggers; Functions; Store procedure

• Using SELECT INTO is fine for single-row queries; CURSORS; Unbound result set (return result set to calling application); Non SELECT actions (update, insert, delete); Internal CALL to other Store Procedure; few info about performance

•  Error handling available from 5.5 onwards with SIGNAL and RESIGNAL

20

25

© 2010/2011 Pythian

MySQL HA solution How many nines do you need?

9 0. 0 0 0 % (36 days) MySQL Replication

9 9. 0 0 0 % (8 hours) Linux Heartbeat with DRBD

9 9. 0 0 0 % (8 hours) Linux Heartbeat with Shared Storage (Active/Passive)

9 9. 0 0 0 % (8 hours) Linux Heartbeat with Shared Storage (Active/Active)

9 9. 9 0 0 % (52 minutes) Linux Heartbeat with DRBD and Replication

9 9. 9 0 0 % (52 minutes) Linux Heartbeat with Shared Storage and Replication

9 9. 9 9 9 % (5 minutes) MySQL Cluster

9 9. 9 9 9 % (5 minutes) MySQL Cluster & Replication

9 9. 9 9 9 % (5 minutes) MySQL Cluster Carrier Grade Edition

[Time is possible outage per year]

20

26

© 2010/2011 Pythian

MySQL Backup •  Logical MySQL Backup

•  mysqldump

•  export

•  Physical MySQL backup

•  LVM Snapshot

•  ZFS snapshot

•  File copy

•  Hot Backup (xtrabackup)

•  Incremental backup

–  Xtrabackup

–  binlogs

20

27

© 2010/2011 Pythian

Managing MySQL MySQL tools

• Client application

•  Non GUI

•  mysql — The MySQL Command-Line Tool

•  mysqladmin — Client for Administering a MySQL Server

•  mysqlcheck — A Table Maintenance Program

•  mysqldump — A Database Backup Program

•  mysqlimport — A Data Import Program

•  mysqlshow — Display Database, Table, and Column Information

• GUI

•  Workbench; Navicat; SQLyog … Google

20

28

© 2010/2011 Pythian

Managing MySQL Administration tools

•  Non GUI •  innochecksum — Offline InnoDB File Checksum Utility

•  myisam_ftdump — Display Full-Text Index information

•  myisamchk — MyISAM Table-Maintenance Utility

•  myisampack — Generate Compressed, Read-Only MyISAM Tables

•  mysqlaccess — Client for Checking Access Privileges

•  mysqlbinlog — Utility for Processing Binary Log Files

•  mysqldumpslow — Summarize Slow Query Log Files

•  mysqlhotcopy — A Database Backup Program (Archive MyISAM)

•  perror — Explain Error Codes

20

29

© 2010/2011 Pythian

Monitoring MySQL Status •  MySQL enterprise monitor

•  Historical monitor

•  Cacti

•  MONyog

•  Live monitoring

•  What we do at Pythian

  Avail software for monitoring of paging/alerts and daily pro-active checks

•  Scripts

•  Nagios plugin

•  Issue is: define the correct parameters to monitor

20

30

© 2010/2011 Pythian

Use the right tool for the right use What MySQL is good for • OTLP

• Small/mid size datasets (per instance) say less then 10Tb

• Applications using many simultaneous connections

What is better to avoid

• Data warehouse

• Applications requiring heavy usage of Stored Procedure

• Applications requiring Hierarchical query (connect by prior etc.)

• Datasets that could not scale by SHARD

20

31

© 2010/2011 Pythian

Q & A

Thank you! [email protected] http://www.pythian.com/ http://www.pythian.com/news/

Interested in joining our elite team of trusted database expert? Pythian is hiring.

More details at: http://www.pythian.com/about/careers/job-listings/

20