MySQL PHP native driver : Advanced Functions / PHP forum Paris 2013

{

FORUM PHP PARIS 2013

MySQL mysqlnd PHP driver Serge Frezefond / Twitter : @sfrezefond

{


MySQL User Survey: Top Languages

{


MySQL Connectors

MySQL Server

Application

libmysql

Connector/C

++

Connector/O

DB

C

Connector/Java

Connector/PH

P

Connector/Python*

Connector/.N

et

Perl

Ruby …

LISP

{


MariaDB Connectors

LGPL connectors(non contaminating) : •  MariaDB Client Library for Java Applications •  MariaDB Client Library for C •  MariaDB ODBC Driver (work in progress)

{


MySQL Protocol Transport Layer

•  Unix Domain Socket – Local communication on Linux /tmp/mysql.sock

•  Windows Named Pipes – Local communication on Windows systems

•  TCP/IP Socket – Network-based communication. TCP Port 3306

•  Embedded MySQL Server – Single process with application code and

MySQL Daemon

{


MySQL Protocol Compression • MySQL compresses each packet individually

– Each returned row is (usually) one packet • Compression costs CPU to reduce network IO

– With bad luck compression consumes more IO

• SELECT id FROM table – Will most likely show bad compression ratio,

even with many rows • SELECT complete_article_text FROM table

– Might have good compression ratio

{


MySQL Protocol Encryption • Encryption is being enabled during the initial

handshake • All subsequent communication (incl.

authentication) will be encrypted – Passwords will always be transferred in a

scrambled safe manner only • SSL handshake time (incl. certificate

validation etc.) takes relatively long

{


MySQL Protocol Character Sets •  MySQL has a rich support for character set and

encodings •  Character set can be changed using

– SQL: SET NAMES – API: mysql_set_character_set() –  In some environments (p.e. Connector/J) the

character set is handled internally and should not be changed

•  Always use these API calls to set the character set so

escape routines know about it!

{


MySQL Protocol Support for Prepared Statements

Client Server

SELECT foo FROM bar WHERE id = ?

• Query database

Resultset(s)

Handle

Handle Param 1: 42

• Create basic Execution plan

prepare()

execute()

{


Buffered and Unbuffered Result Sets

•  Buffered Results – Copies the complete

result set to the client – Minimal memory usage

on Server

Unbuffered Results - Copies rows in sequence as needed - Minimal memory usage on Client

{


PHP mysqlnd driver

• Under the PHP license. no licensing issues that have been problematic in the past.

• MySQL PHP drivers part of the PHP distribution • Actively maintained by Oracle as part of the

php.net project • Three APIs (mysql, mysqli, pdo_mysql) and one

base library (mysqlnd)

• http://php.net

{


PHP and mysqlnd

PHP

PHP Memory IO: PHP Streams Infrastructure

ext/mysql PDO_mysql mysqli

mysqlnd – MySQL native driver for PHP

MySQL Server

{


libmysql vs. mysqlnd

MySQL Server MySQL Server

mysqlnd libmysql

PHP PHP

PHP Memory

libmysql Memory

PHP Memory

PHP Memory

copy

copy

use directly

copy

{


Mysqli API Key mysqli extension benefits : l  Object-oriented interface l  Support for Prepared Statements l  Support for Multiple Statements l  Support for Transactions l  Enhanced debugging capabilities l  Embedded server support

{


Mysqlnd easy Install / few dependancies PHP 5 ChangeLog 5.4 : ext/mysql, mysqli and

pdo_mysql now use mysqlnd by default. $ yum install mysqlnd $ rpm -ql php-mysqlnd-5.3.13-1.fc16.x86_64 /etc/php.d/mysqlnd.ini mysqlnd_mysqli.ini

pdo_mysqlnd.ini /usr/lib64/php/modules/mysqlnd.so

mysqlnd_mysqli.so pdo_mysqlnd.so

...

{


Mysqlnd unique features

Special features compare to MySQL Client Library :

»  Improved persistent connections

»  The special function mysqli_fetch_all()

»  Performance statistics calls: mysqli_get_cache_stats(), mysqli_get_client_stats(), mysqli_get_connection_stats()

{


Mysqlnd supported features SSL Support

MySQL Native Driver supports SSL since PHP 5.3.3 Compressed Protocol Support

As of PHP 5.3.2 MySQL Native Driver supports the compressed client server protocol.

Note that PDO_MYSQL does NOT support compression when used together with mysqlnd.

Named Pipes Support

Named pipes support for Windows was added in PHP version 5.4.0.

{


Mysql asynchronous query only with mysqlnd

$link1->query("SELECT 'test'", MYSQLI_ASYNC);

... if (!mysqli_poll($links, $errors, $reject, 1)) { ... $result = $link->reap_async_query();

{


client per-process statistics only with mysqlnd

• Around 150 statistic values collected –  mysqli_get_client_stats () –  mysqli_get_connection_stats()

• Check PS Execution Ratio –  $stats = mysqli_get_client_stats();

echo $stats['com_execute'] / $stats['com_prepare'];

•  JSMysqlndBundle a symfony2 bundle

{


Mysqlnd Extensible with plugins

Plugin Hook

mysqlnd Query

mysqli::query() mysql_query() PDO::query()

Wire Protocol

Plugin Hook

Network

{


mysqlnd Plugins Client Proxy

– Load Balancing •  Read / Write splitting •  Failover •  Sharding suport ( Fabric framework)

– Monitoring •  Query Logging / Auditing

– Performance •  Caching •  throtling •  multiplexing

{


mysqlnd Plugins through PECL/PHP Extension Community Library

• mysqlnd_ms Replication /load balancing plugin • mysqlnd_qc Query cache plugin • mysqlnd_uh mysqlnd Userland Handler • mysqlnd_memcache

Transparent access to memcached • mysqlnd_mux Multiplexes MySQL connections

• # pecl install mysqlnd_qc-beta … and you are done!

{


mysqlnd Query Cache

PHP

mysql / mysqli / PDO_mysql

mysqlnd

Cache Backend

Query Cache

MySQL Server Local Memory, APC,

Memcache, Custom Handler

{


mysqlnd Query Cache Key Properties

• Transparent – PHP Extension hooking into mysqlnd • Works with ext/mysql, mysqli, pdo_mysql • Pluggable storage handler

– By default: local memory, APC, memcache, SQLite – PHP Userspace • Invalidation via TTL – No automatic invalidation by server

– Custom handlers may use custom invalidation logic

{


mysqlnd Query Cache Transparent ?

$mysqli = new mysqli($host, $user, $pw, $db); $sql = sprintf(“/*%s*/SELECT SLEEP(10) FROM

table”, MYSQLND_QC_ENABLE_SWITCH); ... mysqlnd_qc.cache_by_default = 1

{


mysqlnd Query Cache User-Defined Storage Handler

public function is_select($query) { if (preg_match("@from employees where@ism",

$query)) {return true;} .. $qc = new my_qc(); mysqlnd_qc_change_handler($qc);

{


myslqnd_ms

• mysqlnd replication and load balancing plugin (mysqlnd_ms = “mysqlnd master/slave splitting”)

{


mysqlnd_ms

mysqlnd_ms

mysqli::query() mysql_query() PDO::query()

Application

Master Slave(s)

{


mysqlnd_ms Configuration

{ "myapp": { "master": { "master_0": { "host": "localhost", "socket": "/tmp/mysql.sock" } }, "slave": { "slave_0": { "host": "192.168.2.27", "port": "3306" } } } }

{


mysqlnd_ms: Connecting

$mysqli = new MySQLi(“myapp”, “user”, “pwd”); $mysqli->query(“SELECT * FROM t”); $mysqli->query(“SET @sessionvar='value' ”); $mysqli->query(“SELECT @sessionvar”);

{


mysqlnd_ms Quality of Service

mysqlnd_ms is a load balancer at the driver level controlled by :

–  SQL hints –  mysqlnd_ms_set_qos() defines the quality of

service (QoS). tells load balancer how to select database cluster nodes .

Without GTIDs the rules for a MySQL Replication cluster are simple:

–  eventual consistency – any slave, session –  strong consistency – master only.

{


myslqnd_mux multiplexes MySQL connections

•  Less connections to the MySQL server.

•  Pooling saves connection time.

•  Multiplexed connection

•  Persistent connection

{


•  Fast, simple access to InnoDB • Memcached for key-value

operations • Memcached daemon plug-in to

mysqld

MySQL 5.6: NotOnlySQL: Memcached API .

InnoDB Storage Engine

MySQL Server

Memcached plugin

Application SQL

(MySQL Client)

NoSQL (Memcached

Protocol) mysqld

{


PHP NoSQL data Access memcached API

$mysqli = new mysqli("localhost", "usr", "pass", "test");

$memcache = new memcached(); $memcache->addServer("localhost", 11211); mysqlnd_memcache_set($mysqli, $memcache); …. $q1 = $mysqli->query("SELECT firstname, lastname FROM test WHERE id = 1");

{


myslqnd_uh Write your own plugin in PHP

l  Monitoring l  Auditing l  Load Balancing connections l …

l  Two classes are provided by the extension: MysqlndUhConnection and MysqlndUhPreparedStatement.

{


myslqnd_uh Installing a connection proxy

class proxy extends MysqlndUhConnection { public function connect($res, $host, $user, $pwd,

$db, $port, $socket, $mysql_flags) { printf("%s(%s)\n", __METHOD__,

var_export(func_get_args(), true)); … } mysqlnd_uh_set_connection_proxy(new proxy()); $mysqli = new mysqli("localhost", "root", "", "test");

{


Write your own mysqlnd plugin !

A mysqlnd plugin is itself a PHP extension. A mysqlnd plugin works by intercepting calls made to

mysqlnd by extensions that use mysqlnd. This is achieved by obtaining the mysqlnd function

table, backing it up, and replacing it by a custom function table, which calls the functions of the plugin as required.

{


Sharding with PHP MySQL Fabric

MySQL Fabric is a sharding framework. Some connectors can route query based on

sharding metadata stores in Fabric Server. Fabric aware connectors : •  connector python •  connector J •  connect PHP

{


connector

Application

Shard 1 Shard 2 Shard n

Fabric Server

…

MySQL Fabric aware connector

{


Sharded Highly Available setup with Fabric

… $ mysqlfabric group create shard1 $ mysqlfabric group add shard1 127.0.0.1:3303 root

secret $ mysqlfabric group promote shard1 $ mysqlfabric group add shard1 127.0.0.1:3304 root

secret …

{


Driver

Application

Shard 1 Shard 2 Shard n

Sharded table

Global Tables

…

MySQL Fabric Global and sharded tables

{


PHP access to MySQL Fabric shards

fabric.json: { "test" : { "fabric":{ "hosts": [ { "host": "localhost", "port”:8080 } ] }}}

php.ini: extension=mysqlnd_ms.so mysqlnd_ms.enable=1 mysqlnd_ms.config_file=/path/to/fabric.json

{


PHP Access to Fabric sharded database

$c = new mysqli("test", "root", "", "test"); mysqlnd_ms_fabric_select_shard($c, "test.ftest", 10); $c->query("INSERT INTO ftest VALUES (10)")); mysqlnd_ms_fabric_select_shard($c, "test.ftest", 110); $r = $c->query("SELECT * FROM ftest WHERE id =

110"); $r->fetch_row();

{


PHP Access to Fabric sharded database

$c = new mysqli("test", "root", "", "test"); mysqlnd_ms_fabric_select_global($c, "test.fabrictest"); $c->query("CREATE TABLE fabrictest (id INT NOT

NULL) »);

{


Other Sharding Framework Mature alternative to Fabric

•  Google/Youtube has Vitess (in Go open source)

•  Twitter has built Gizzard (in Scala open source) •  Tumblr has built Jetpants (in Ruby open source) •  Facebook has also it own sharding framework

which is a range based model. The HA is addressed with MHA. they have presented their architecture at various events

•  Many others : ETSY, ….

{


PHP Access to MariaDB Galera Cluster

Ø Read & write access to any node Ø Client can connect to any node Ø There can be several nodes Ø Automatic node provisioning Ø Replication is synchronous

{


Galera Replication

…

… MariaDB Galera Cluster mysqlnd multi master

Clients1

Read Write

Read Write

Read Write

{


Galera Replication

…

… MariaDB Galera Cluster R/W splitting with HAproxy

Clients

Write READ on HAproxy

{


Node 2

Node 1

MariaDB Node 0

MariaDB CONNECT Storage Engine remote and ODBC Access

free

col col2

ODBC col3 col1 col

ODBC table

MySQL table

MYSQL

{


Node 3

Node 2

Node 1 MariaDB Node 0

MariaDB Connec Storage Engine Parallel execution

TBL

free

col1

col2

MYSQL /ODBC

col3 col1 col2

ODBC table

MySQL table

col3 col1 col2

col4

{


Conclusion

l  Mysqlnd makes MySQL & PHP stronger l  Fabric sharding and HA from PHP worth

testing

Thank You Serge Frezefond @sfrezefond http://serge.frezefond.com

MySQL PHP native driver : Advanced Functions / PHP forum Paris 2013

Technology

php php memoryinfrastructureio

php forum php version

mysql php drivers

php mysqlnd driver

progressforum php paris

sfrezefondforum php

languagesforum php paris

queryforum php paris