Developing Information Schema Plugins

Presented by,

MySQL AB® & O’Reilly Media, Inc.

Developing INFORMATION_SCHEMA Plugins

Mark Leith

Support Manager, Americas

mleith@mysql.com

Overview

INFORMATION_SCHEMA overview Plugin Overview Developing a simple Hello World I_S table Building and installing Interacting with another program Interacting with another library

INFORMATION_SCHEMA

Specified within the SQL Standard (ISO/IEC 9075-11:2003) Virtual tables which give database metadata MySQL implements a subset of the Standard http://dev.mysql.com/doc/refman/5.1/en/information-schema.html http://www.xcdsql

.org/MySQL/information_schema/5.1/MySQL_5_1_INFORMATION_SCHEMA.html

Plugins

A means to load shared libraries in to a running MySQL instance

Developed in C/C++ http://dev.mysql.com/doc/refman/5.1/en/plugin-a

pi.html User Defined Functions Storage Engines Fulltext parsers Daemons INFORMATION_SCHEMA Tables!

Located in include/mysql/plugin.h

The Plugin Interface

struct st_mysql_plugin{ int type; /* the plugin type (a MYSQL_XXX_PLUGIN value) */ void *info; /* pointer to type-specific plugin descriptor */ const char *name; /* plugin name */ const char *author; /* plugin author (for SHOW PLUGINS) */ const char *descr; /* general descriptive text (for SHOW PLUGINS ) */ int license; /* the plugin license (PLUGIN_LICENSE_XXX) */ int (*init)(void *); /* the function to invoke when plugin is loaded */ int (*deinit)(void *) /* the function to invoke when plugin is unloaded */ unsigned int version; /* plugin version (for SHOW PLUGINS) */ struct st_mysql_show_var *status_vars; struct st_mysql_sys_var **system_vars; void * __reserved1; /* reserved for dependency checking */};

Plugin Types Define the plugin type (types listed below) Used within plugin functions such as plugin_foreach()

MYSQL_UDF_PLUGIN 0 /* User-defined function */MYSQL_STORAGE_ENGINE_PLUGIN 1 /* Storage Engine */MYSQL_FTPARSER_PLUGIN 2 /* Full-text parser plugin */MYSQL_DAEMON_PLUGIN 3 /* The daemon/raw plugin type */MYSQL_INFORMATION_SCHEMA_PLUGIN 4 /* The I_S plugin type */

I_S plugins loaded within sql/sql_show.cc

int schema_tables_add(THD *thd, List<LEX_STRING> *files, const char *wild){… if (plugin_foreach(thd, add_schema_table, MYSQL_INFORMATION_SCHEMA_PLUGIN, &add_data)) DBUG_RETURN(1);

Starting a HELLO_WORLD Table

Must have MySQL source tree Create a new plugin/hello_world/hello_world.cc

file plugin.h and mysql_priv.h includes required Add a prototype for

schema_table_store_record()

/* Required for schema_table_store_record() */#include "mysql_priv.h"#include <mysql/plugin.h>

bool schema_table_store_record(THD *thd, TABLE *table);

Define the Table Structure Uses an array of ST_FIELD_INFO structs

field_name Column name field_length Column length or display length field_type Column datatype value Not used within I_S plugins field_flags Set NULL / UNSIGNED attributes old_name Internal mapping for I_S tables to SHOW output open_method Open table using supplied method

Last entry in the array is an end markerST_FIELD_INFO hello_world_fields[]={ {"HELLO", 10, MYSQL_TYPE_STRING, 0, 0, "Hello", 0}, {"WORLD", 10, MYSQL_TYPE_STRING, 0, 0, "World", 0}, {0, 0, MYSQL_TYPE_NULL, 0, 0, 0, 0}};

A Closer Look

MYSQL_TYPE_STRING

MYSQL_TYPE_LONG

MYSQL_TYPE_LONGLONG

MYSQL_TYPE_DECIMAL

MYSQL_TYPE_DATETIME

VARCHAR

INT

BIGINT

DECIMAL

DATETIME

Field_type

Field_flags

MY_I_S_MAYBE_NULLMY_I_S_UNSIGNED

Open_method

SKIP_OPEN_TABLE OPEN_FRM_ONLY

OPEN_FULL_TABLE

Define Function to Fill Table

Called every time the table is accessed Fills the virtual table with data

int fill_hello_world(THD *thd, TABLE_LIST *tables, COND *cond){ DBUG_ENTER("fill_hello_world_is_plugin"); CHARSET_INFO *scs= system_charset_info; TABLE *table= tables->table; int rc= 0;

table->field[0]->store("Hello", strlen("Hello"), scs); table->field[1]->store("World", strlen("World"), scs); if (schema_table_store_record(thd, table)) rc= 1;

DBUG_RETURN(rc);}

Create the Init Function Runs when the plugin is installed/loaded ST_SCHEMA_TABLE is an internal table representation Points to the ST_FIELD_INFO struct / table definition Points to the function to fill the table Manages any other init needed for the plugin

int hello_world_plugin_init(void *p){ DBUG_ENTER("init_hello_world_is_plugin"); ST_SCHEMA_TABLE *schema= (ST_SCHEMA_TABLE*) p; schema->fields_info= hello_world_fields; schema->fill_table= fill_hello_world; DBUG_RETURN(0);}

Create the Deinit Function

Run when the plugin is unloaded Does nothing for HELLO_WORLD Use this function to do any clean up in your plugin

int hello_world_plugin_deinit(void *p){ DBUG_ENTER("deinit_info_schema_example_plugin"); DBUG_RETURN(0);}

Create the Plugin Definition Info struct points to the interface version built against

struct st_mysql_information_schema hello_world_plugin_info={ MYSQL_INFORMATION_SCHEMA_INTERFACE_VERSION };

mysql_declare_plugin(hello_world_plugin){ MYSQL_INFORMATION_SCHEMA_PLUGIN, &hello_world_plugin_info, /* Pointer to info struct */ "HELLO_WORLD", /* Plugin Name (used in INSTALL PLUGIN) */ "Mark Leith, MySQL AB", /* Plugin Author */ "HELLO_WORLD example plugin", /* Plugin Description */ PLUGIN_LICENSE_GPL, /* _GPL, _BSD or _PROPRIETARY */ hello_world_plugin_init, /* Pointer to plugin init function */ hello_world_plugin_deinit, /* Pointer to plugin deinit function */ 0x0100, /* 1.0 */ NULL, /* status variables */ NULL, /* system variables */ NULL /* config options */}mysql_declare_plugin_end;

Build the Plugin Define MYSQL_DYNAMIC_PLUGIN Make shared Include the sql and include source directories

g++ -DMYSQL_DYNAMIC_PLUGIN -shared \> -I/home/leithal/mysql/mysql-5.1/include \> -I/home/leithal/mysql/mysql-5.1/sql \> -o is_hello_world.so hello_world.cc

ls -ltotal 16-rw-r--r-- 1 leithal leithal 1712 2008-04-11 14:20 hello_world.cc-rwxr-xr-x 1 leithal leithal 8226 2008-04-11 14:27 is_hello_world.so

cp is_hello_world.so /usr/local/mysql/lib/mysql/plugin

Install and Use!

mysql> INSTALL PLUGIN HELLO_WORLD SONAME 'is_hello_world.so';Query OK, 0 rows affected (0.01 sec)

mysql> USE INFORMATION_SCHEMA;Database changedmysql> SHOW TABLES LIKE 'HELL%';+--------------------------------------+| Tables_in_information_schema (HELL%) |+--------------------------------------+| HELLO_WORLD |+--------------------------------------+1 row in set (0.00 sec)

mysql> SELECT * FROM HELLO_WORLD;+-------+-------+| HELLO | WORLD |+-------+-------+| Hello | World |+-------+-------+1 row in set (0.00 sec)

So We'll Do Something Interesting

How's the disk space doing on my db server?

root@achilles:~# df -hFilesystem Size Used Avail Use% Mounted on/dev/hda1 4.6G 3.5G 886M 81% /varrun 126M 84K 125M 1% /var/runvarlock 126M 0 126M 0% /var/lockprocbususb 10M 76K 10M 1% /proc/bus/usbudev 10M 76K 10M 1% /devdevshm 126M 0 126M 0% /dev/shmlrm 126M 18M 108M 14% /lib/modules/2.6.17-12-generic/volatile

If I can get this in a table I can track it over time I can also report on it easily How about an I_S table with events to catalog?

Create Template & Table Structure Add stdio.h,string.h and stdlib.h includes Mirror init, deinit and info functions Fill in the plugin declaration scruct appropriately Define the new table:

ST_FIELD_INFO fs_info_schema_fields[]={ {"FILESYSTEM", 120, MYSQL_TYPE_STRING, 0, 0, "Filesystem", 0}, {"SIZE", 8, MYSQL_TYPE_LONGLONG, 0, 0, "Mountpoint Size", 0}, {"USED", 8, MYSQL_TYPE_LONGLONG, 0, 0, "Used Space", 0}, {"AVAILABLE", 8, MYSQL_TYPE_LONGLONG, 0, 0, "Available Space", 0}, {"CAPACITY", 4, MYSQL_TYPE_STRING, 0, 0, "Percent Used", 0}, {"MOUNTED_ON", 120, MYSQL_TYPE_STRING, 0, 0, "Filesystem Mounted On" , 0}, {0, 0, MYSQL_TYPE_STRING, 0, 0, 0, 0}};

Running Other Programs

Open with popen() and read in the results

/* get filesystem information from df on linux like systems */ FILE *f; char buf[128]; char *c, *s, *size; const char delim[]= " "; unsigned long long uli; double d;

if (NULL != (f= popen("/bin/df -h", "r"))) { int ln= 0; while(!feof(f)) { fgets(buf, sizeof(buf), f);

switch(ln++) {

Tokenize and Push Results

Can do this how you like

/* skip the header line of df */ case 0: break; default:

c= buf; /* hack to stop processing when falling off the end of output */ if (strchr(c, ' ') == NULL) break;

/* Filesystem */ s= strtok(c, delim); table->field[0]->store(s, strlen(s), scs); /* Size */ s= strtok(NULL, delim); d= strtod(s, &size); uli= get_bytes(d, size); table->field[1]->store(uli, TRUE); ….

Store the Row and Finish up

/* Use% */ s= strtok(NULL, delim); table->field[4]->store(s, strlen(s), scs); /* Mountpoint */ s= strtok(NULL, delim); stripnl(s); table->field[5]->store(s, strlen(s), scs); /* store the row */ if (schema_table_store_record(thd, table)) rc= 1; break; } } pclose(f); } else rc= 1; DBUG_RETURN(rc);}

Build, Install, Try..

Build as before and try it out!mysql> select * from file_system_mountpoints;+------------+------------+------------+-----------+----------+-----------------------------------------+| FILESYSTEM | SIZE | USED | AVAILABLE | CAPACITY | MOUNTED_ON |+------------+------------+------------+-----------+----------+-----------------------------------------+| /dev/hda1 | 4939212390 | 3758096384 | 930086912 | 81% | / | | varrun | 132120576 | 86016 | 131072000 | 1% | /var/run | | varlock | 132120576 | 0 | 132120576 | 0% | /var/lock | | procbususb | 10485760 | 77824 | 10485760 | 1% | /proc/bus/usb | | udev | 10485760 | 77824 | 10485760 | 1% | /dev | | devshm | 132120576 | 0 | 132120576 | 0% | /dev/shm | | lrm | 132120576 | 18874368 | 113246208 | 14% | /lib/modules/2.6.17-12-generic/volatile | +------------+------------+------------+-----------+----------+-----------------------------------------+7 rows in set (0.06 sec)

mysql> desc file_system_mountpoints;+------------+--------------+------+-----+---------+-------+| Field | Type | Null | Key | Default | Extra |+------------+--------------+------+-----+---------+-------+| FILESYSTEM | varchar(120) | NO | | | | | SIZE | bigint(11) | NO | | 0 | | | USED | bigint(11) | NO | | 0 | | | AVAILABLE | bigint(11) | NO | | 0 | | | CAPACITY | varchar(4) | NO | | | | | MOUNTED_ON | varchar(120) | NO | | | | +------------+--------------+------+-----+---------+-------+6 rows in set (0.01 sec)

Now Let's Use another Library

The SIGAR library from Hyperic Lets us do cross platform OS stats monitoring Released under GPL

Again, start with the same base functions etc. as before

Include Lib Header, Use Lib in Fillsigar_t *t;sigar_open(&t);

sigar_file_system_list_t fslist;sigar_file_system_list_get(t, &fslist);

for (uint i = 0; i < fslist.number; i++) { sigar_file_system_t fs = fslist.data[i]; sigar_file_system_usage_t fsusage; rc= sigar_file_system_usage_get(t, fs.dir_name, &fsusage);

if (fs.type == 2 || fs.type == 3) { table->field[0]->store(fs.dir_name, strlen(fs.dir_name), scs); table->field[1]->store(fsusage.total, TRUE); table->field[2]->store(fsusage.used, TRUE); table->field[3]->store(fsusage.free, TRUE); table->field[4]->store(fsusage.files, TRUE); if (schema_table_store_record(thd, table)) rc= 1; } } sigar_file_system_list_destroy(t, &fslist); sigar_close(t);

Building and Including the Library

g++ -DMYSQL_DYNAMIC_PLUGIN -Wall -shared \-lsigar -Wl,-rpath -Wl,/home/leithal/os_stats_info_schema/sigar/ \-L/home/leithal/os_stats_info_schema/sigar/ \-I/home/leithal/os_stats_info_schema/sigar/include/ \-I/home/leithal/mysql/mysql-5.1/include \-I/home/leithal/mysql/mysql-5.1/sql \-o os_stats_info_schema.so os_stats_info_schema.cc

-lsigar links the library Use -rpath to pass in library location

This is for runtime Can also use LD_LIBRARY_PATH

-L (linker) and -I (include) paths -L points to the directory with the built library

Try it out!

mysql> INSTALL PLUGIN os_disk_usage SONAME 'os_stats_info_schema.so';Query OK, 0 rows affected (0.00 sec)

mysql> SHOW TABLES LIKE 'OS%';+------------------------------------+| Tables_in_information_schema (OS%) |+------------------------------------+| OS_DISK_USAGE | +------------------------------------+1 row in set (0.00 sec)

mysql> SELECT * FROM OS_DISK_USAGE;+------------+---------+---------+---------+--------+| FILESYSTEM | SIZE | USED | FREE | FILES |+------------+---------+---------+---------+--------+| / | 4799024 | 3656800 | 1142224 | 610432 | +------------+---------+---------+---------+--------+1 row in set (0.00 sec)

Resources and Questions!

http://www.markleith.co.uk/?p=18 http://rpbouman.blogspot.com/2008/02/mysql-inf

ormation-schema-plugins-best.html http://rpbouman.blogspot.com/2008/02/reporting

-mysql-internals-with.html (Great resource for monitoring server internals)

Questions?