PHP tips by a MYSQL DBA

1

PHP tips by a MYSQL DBA

-- Shantanu Oak

2

Signing query

Sign your queries...

select * from mytable;

SeLeCt * from mytable;

Or use comments in the query like...

select /* query by shantanu login.php file */ name, age from

customers

This helps while watching process-list and slow query logs

No line breaks in a query but space after , of column name

3

Output Query echo the query and make it hidden

Display style none as shown below:

 <?php echo $qrysearch ?>



Anyone can go to View – Source and find the query

This will be useful for debugging. Remove the code

before taking the code to production server.

4

MySQL command history

[root@databaseserver215 ~]# tail .mysql_history

show processlist;

show slave status;

stop slave;

SET GLOBAL SQL_SLAVE_SKIP_COUNTER=1; START

SLAVE;

show slave status;

show variables like '%innod%';

show slave status;

5

Mysqld --print-defaults [root@localhost mysql]# mysqld --print-defaults

--port=3306 --socket=/var/lib/mysql/mysql.sock --skip-external-locking

--log-bin=/var/log/mysql/binary/mysql-bin.log --binlog-ignore-db=mysql --binlog-ignore-db=test --

server-id=4 --expire_logs_days=15 --max_binlog_size=1024M --report-host=172.29.120.121 --relay-

log=/var/log/mysql/binary/cluster1-relay-bin --sync_binlog=1 --group_concat_max_len=500000 --

innodb_data_home_dir= --innodb_data_file_path=ibdata1:2800M;ibdata2:2800M:autoextend --

innodb_buffer_pool_size=4G --innodb_additional_mem_pool_size=1M --innodb_log_files_in_group=2

--innodb_log_file_size=1G --innodb_log_buffer_size=8M --innodb_flush_log_at_trx_commit=1 --

innodb_lock_wait_timeout=50 --log_slow_queries=/var/log/mysql/mysql-slow.log --long_query_time=2

--skip-name-resolve --max_connections=400 --max_user_connections=1600 --

max_connect_errors=50 --wait_timeout=1200 --connect_timeout=5 --interactive_timeout=120 --

join_buffer_size=1M --read_buffer_size=4M --read_rnd_buffer_size=16M --table_cache=512 --

max_allowed_packet=4M --key_buffer_size=6144M --sort_buffer_size=2M --

myisam_sort_buffer_size=64M --thread_cache=128 --thread_concurrency=8 --thread_cache_size=40

--thread_stack=128K --low_priority_updates=1 --query_cache_limit=16M --

query_cache_min_res_unit=1 --query_cache_size=500M --query_cache_type=1 --skip-bdb --

group_concat_max_len=3072 --ft_min_word_len=1 --ft_stopword_file=/etc/stopword.txt

6

Use Strict Modemysql> SET local sql_mode='strict_all_tables';

mysql> SELECT @@local.sql_mode;

+--------------------------------+

| @@local.sql_mode |

+--------------------------------+

|

STRICT_ALL_TABLES,NO_ZERO_DA

TE |

+--------------------------------+

1 row in set (0.00 sec)

mysql> SELECT @@global.sql_mode;

+-------------------+

| @@global.sql_mode |

+-------------------+

| |

+-------------------+


7

Use Cron

If you want to delete rows older than current date use

crontab -e command to edit the crontab file. Make sure

that the mysql command can be executed at

command prompt.

# remove records earlier than current date

1 0 * * * mysql -h 172.219.0.1 -uroot -e'delete from

testdb.test_tb where d < current_date()'

8

Using Log files

MySQL can generate different log files like...

General, slow, binlog and error

Apache error log files also contain some

valuable information.

9

Error log

grep -n "File" -v dataentry_error_log | tail

cat -n iro_error_log | tail -300 | more

10

Binlog I

Binlogs are enabled in order to send insert

/update queries to the slave.

But you can enable binlogs even if the server is

not part of the replication.

This log will automatically store all the important

statements along with it's date-time and IP

addresses.

11

Binlog II

mysqlbinlog --stop-datetime="2008-05-01 00:00:00" mysql-

bin.000005 >> prit1.txt

grep -i 'PA003111' prit1.txt | sort | uniq > sendprit.txt

mysqlbinlog –start-datetime="`date +%Y-%m-%d' '%H:00:00 -d\"1

hour ago\"`" --stop-datetime="`date +%Y-%m-%d' '%H:00:00`" mysql-

bin.* | awk '/tbl_company_master/,/;/' | replace tbl_company_master

new.tbl_company_master | more

12

Slow-query-log mysqldumpslow

You can save the first 1000 expensive queries into a separate

file using built-in mysqldumpslow utility.

mysqldumpslow /var/log/mysql/mysql-slow.log | head -1000 >

slowextract.txt

sort by count instead of time (default) and show actual values of

integers and text instead of N and S

mysqldumpslow mysql-slow-demaster.log -s c -a > extract.txt

13

Slow-query log - I

# Time: 080602 11:37:50

# User@Host: root[root] @ [172.29.0.12]

# Query_time: 103 Lock_time: 0 Rows_sent: 82

Rows_examined: 1060213

use d_Jd_ClientFB;

Select dosdate, count(*) as SmsCount, operator as Operator

from ClientFeedback_backup where SMSSent = 'Y' and

CompanyMobile is not null group by dosDate, operator;

14

Slow-query log II

The explain plan shows as below:

id: 1

select_type: SIMPLE

table: ClientFeedback_backup

type: ref

possible_keys: SentIndex,CompanyMobileIndex

key: SentIndex

key_len: 2

ref: const

rows: 434148

Extra: Using where; Using temporary; Using filesort

15

Slow-query log III

The index is being used on 'Sent' column which is of

low cardinality. It means only 'Y' and 'N' values are

stored in this column and it is as good as using no

index at all. Because Index column looks for unique

values and it did not find such values in this column.

A composite index on dosDate, operator in that

order is necessary.

16

General Log

General logs can become very big in a very

short time since they have all the selects along

with update/ delete

Worth enabling on test servers where you are

testing your PHP code

17

Low and High Priority The HIGH_PRIORITY hint can be used on SELECT or INSERT

statements to let MySQL know that this is a high priority query. This

hint will basically allow the query to skip in line.

The LOW_PRIORITY hint can be used on INSERT and UPDATE

statements. If you use the LOW_PRIORITY keyword, execution of

the query is delayed until no other clients are reading from the table.

This means that you may wait a LONG time, or forever on servers

with a heavy read volume.

insert HIGH PRIORITY into logappointment (emailid, appflag,

date1, callerid, tel, mob) values ('[email protected]', 'Y', now(),

'2191361', '22579950', '9415023611');

mailto:'[email protected]

18

sql_big_results The SQL_BIG_RESULT hint can be used with DISTINCT and GROUP BY

SELECT statements. It as you might guess, tells MySQL that the result set will

be big. According to the MySQL documentation, if invoked

MySQL directly uses disk-based temporary tables if needed, and prefers

sorting to using a temporary table with a key on the GROUP BY elements.

SeleCT SQL_BUFFER_RESULT SQL_BIG_RESULT t1.contactid, t1.parentid,

t1.companyname, t1.createdby, t1.curTime, group_concat(t1.contract_type)

as contract_type, t1.promptype, t1.freez, t1.mask, t1.contract_series

FROM tbl_company_master t1 WHERE t1.contract_type <> 'prompt'

GROUP BY t1.parentid ORDER BY t1.companyName ASC, t1.parentId ASC

19

Insert delayed Part I It will return immediately, but it will still wait until other clients have closed the table

before executing the statement.

Note: INSERT DELAYED only works on MyISAM, MEMORY, and ARCHIVE tables.

You can delay INSERT's from happening until the table is free by using the

DELAYED hint in your SQL statement. For example:

INSERT DELAYED INTO table (col) VALUES ('val');

The above SQL statement will return quickly, and it stores the insert statement in a

memory queue until the table you are inserting into is free from reads. This means

that if there are multiple inserts in the queue they can be written in one block, which is

a more optimal use of IO.

20

Insert Delayed Part II The downside to this is that it is not transactionally safe at all. You don't really know

how long its going to take for your INSERT to happen. If the server crashes, or is

forcefully shutdown you will loose your INSERTs. So don't use this on any critical

information.

One great use for the DELAYED keyword would be for storing web stats in a

database. You don't want the client waiting for the stats to insert, and its not that big

of a deal if you loose a few stats (for most people).

21

Store IP addresses CREATE TABLE Sessions (session_id INT UNSIGNED NOT NULL

AUTO_INCREMENT, display_address varchar(15), ip_address INT

UNSIGNED NOT NULL, session_data TEXT NOT NULL, PRIMARY KEY

(session_id), INDEX (ip_address) ) ENGINE = InnoDB;

insert into Sessions values (NULL, '172.29.0.217',

INET_ATON('172.29.0.217'), 'some more data');

insert into Sessions values (NULL, '172.29.0.227',

INET_ATON('172.29.0.227'), 'data from other IP');

select session_id, display_address, ip_address as ip_raw,

INET_NTOA(ip_address) as ip, session_data from Sessions WHERE

ip_address = INET_ATON('172.29.0.217') or ip_address =

INET_ATON('172.29.0.227') ;

22

Replication

A client that has the SUPER privilege can disable

binary logging of its own statements by using a SET

SQL_LOG_BIN=0 statement. As a brief example, if I

am loading a large table it could be good to disable

logging before beginning the import.

mysql> SET SQL_LOG_BIN=0;

mysql> LOAD DATA INFILE 'honking_big_file' INTO

BIG_TABLE;

23

Normalize your database

The first normal form doesn't allow you to store

values like this in a single cell.

/

146/,/834/,/3483/,/4043/,/20852/,/221554/,/221561/,/

222987/,/223154/,/223539/

'Z008677','Z004949','Z008572','Z004951'

24

Table Types There are 2 important types of tables.

MyISAM when there are many selects or many inserts or when you need Full text

index.

InnoDB when there are a lot of select, update, inserts and deletes happening

simultaneously. This is the only table type that supports transactions and foreign

keys.

Archive table type supports only inserts and selects. No Update/ delete is allowed.

Good for log tables.

Federated tables allows you to connect to remote tables as if they are local. Doesn't

work for very big tables and is buggy.

Memory tables are temporary tables living in memory and are dropped when the

server is restarted.

25

Column Types

Do not use float or double to store numbers. Use decimal or integer

Do not use BLOB to store files, images. Save them in a directory and

store the path in the table.

Avoid 'text' datatype. Use varchar. No need to use varchar(255) since

we can now have varchar(3000)

But it does not mean that you should have varchar(3000) when

varchar(100) is enough. Indexes will have issues with that figure.

Do not use set or enum if you are not comfortable with database

concepts.

26

Learn how to use “load data infile”

The method of 'Load data in file' is usually 20 times

faster than using INSERT statements.

27

Load file You can read the file contents from within mysql. This feature is useful to read the file

saved on the server and send the contents to the user through PHP.

[root@irodb2 mysql]# echo ' this is bold in xml file ' > mytest.xml

[root@irodb2 mysql]# mysql test

mysql> select load_file("/var/log/mysql/mytest.xml"), 'test constant column', bse_code

from bsecode limit 1

*************************** 1. row ***************************

load_file("/var/log/mysql/mytest.txt"): this is bold in xml file 

test constant column: test constant column

bse_code: 513375

28

Balance PHP and MySQL //

userID,posts,runningTotal|

// output running total

// 2, 23434, 28330|

// 6, 3443, 4896|

// 1, 422, 1453|

// 3, 344, 1031|

// 4, 344, 687|

// 5, 343, 343|

echo 'userID,posts,runningTotal| ';

$q = mysql_query("select * from `members`

order by `posts` DESC");

while($a = mysql_fetch_row($q)){

echo "$a[0],$a[1],$total| ";

$total = $total - $a[1];

}

29

Use Joins

Do not execute the query and take the rows

one at a time to compare it's value with another

row. Use joins.

Do not use IN

Use different joins like inner, left

30

Update 2 tables in one query You can join 2 tables in a single select. You can update those 2

tables in a single statement as well.

select * from packet_date_sent AS pds

INNER JOIN tempAdvSep2007 AS tas

ON pds.enroll_no = tas.enroll_no

INNER JOIN packet_sent AS ps

ON ps.enroll_no = tas.enroll_no

AND ps.material_id = pds.material_id

AND ps.course_id = pds.course_id

AND ps.enroll_date = pds.enroll_date

WHERE pds.date_sent = '2007-09-01'

AND pds.material_id BETWEEN 62 AND 97;

31

Update 2 tables in one query UPDATE packet_date_sent AS pds

INNER JOIN tempAdvSep2007 AS tas ON pds.enroll_no = tas.enroll_no

INNER JOIN packet_sent AS ps ON ps.enroll_no = tas.enroll_no AND

ps.material_id = pds.material_id AND ps.course_id = pds.course_id AND

ps.enroll_date = pds.enroll_date

SET pds.sent_mode = '5', pds.postofficecode = 1, pds.system_date = NOW(),

pds.branch_id = '99', ps.sent_bit = '1', ps.system_date = NOW(), ps.branch_id =

'99'

WHERE pds.date_sent = '2007-09-01' AND pds.material_id BETWEEN 62 AND 97;

32

Execute linux commands

You can use quotes to execute Linux command

from within PHP

<?php

echo `mysql -h 172.29.0.131 -uroot -H -e'set

@count:= 0; select @count:= @count + 1 as "sr

no", p,d from test.t'`;

?>

33

Full Text search

Only MyISAM table types supports this.

You can use match – against syntax when you

want to use this type of search.

34

SQL_CALC_FOUND_ROWS

mysql> select SQL_CALC_FOUND_ROWS name, country from india

limit 1\G

name: Punjab Plains

country: IN

mysql> select found_rows();

+--------------+

| found_rows() |

+--------------+

| 38202 |

+--------------+


35

Use MySQL variablesmysql> set @count:= 0;

Query OK, 0 rows affected (0.00 sec)

mysql> select @count:= @count + 1 as myid, p from t;

+------+-----+

| myid | p |

+------+-----+

| 1 | 20% |

| 2 | 20% |

| 3 | 30% |

| 4 | 30% |

| 5 | 50% |

| 6 | 50% |

+------+-----+

10 rows in set (0.00 sec)

36

Collation issues

SET @@SESSION.collation_connection =

'latin1_general_ci';

37

error_reporting(E_ALL);

declaring a variable ahead of time,

referencing a variable that isn’t available in that

segment of code, or

using a define that isn’t set.

38

Use deterministic functions

A function is considered deterministic if it always

produces the same result

Non-deterministic (changing value) functions return

different values every time.

MySQL does not have to process the function and

hence faster

Replication will see the actual value and not now() that

will have different time on Master and slave

39

Save these values in a variable and

use that value in the SQL statement

mysql> select now(), rand(), curtime();

+---------------------+------------------+-----------+

| now() | rand() | curtime() |

+---------------------+------------------+-----------+

| 2008-09-17 18:04:26 | 0.38422505500189 | 18:04:26 |

+---------------------+------------------+-----------+

mysql> select now(), rand(), curtime();

+---------------------+------------------+-----------+

| now() | rand() | curtime() |

+---------------------+------------------+-----------+

| 2008-09-17 18:04:29 | 0.98001360425727 | 18:04:29 |

+---------------------+------------------+-----------+

40

Forget select *mysql> explain select * from tbl_compcatarea a inner join bid_details b on a.contactid=b.contractid;

+----+-------------+-------+------+---------------+-----------+---------+--------------------+-------+-------+

| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |

+----+-------------+-------+------+---------------+-----------+---------+--------------------+-------+-------+

| 1 | SIMPLE | b | ALL | ContractID | NULL | NULL | NULL | 24386 | |

| 1 | SIMPLE | a | ref | contactid | contactid | 47 | d_jds.b.contractID | 1 | |

+----+-------------+-------+------+---------------+-----------+---------+--------------------+-------+-------+

mysql> explain select a.contactid, a.compname from tbl_compcatarea a inner join bid_details b on

a.contactid=b.contractid;

+----+-------------+-------+-------+---------------+------------+---------+--------------------+-------+-------------+

| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |

+----+-------------+-------+-------+---------------+------------+---------+--------------------+-------+-------------+

| 1 | SIMPLE | b | index | ContractID | ContractID | 48 | NULL | 24386 | Using

index |

| 1 | SIMPLE | a | ref | contactid | contactid | 47 | d_jds.b.contractID | 1 | |

+----+-------------+-------+-------+---------------+------------+---------+--------------------+-------+-------------+

41

Standards and conventions for

naming columns, tables and indexes The name of the key is misleading in the table

tbl_company_source. The key name is

'ContractID' and the column used for indexing is

'contactID'. I overlooked this problem because

of the similarity in the key name and column

name. In fact we need two different indexes on

ContractID and contactID.

PHP tips by a MYSQL DBA

Technology