Performance optimisations PHP meetup Rotterdam

Dim0 (Dimitri Vanoverbeke) PHP Meetup Rotterdam

November 12th 2015

Practical MySQL Performance Optimization

www.percona.com 2

Who am I?

•  Linux system engineer and Consultant for 8 years

•  Solution Engineer @ Percona

•  www.twitter.com/dim0

2

www.percona.com 3

Contents of the presentation

How do we approach performance optimisations

Discuss practical optimisation tips.

What tools exist?

www.percona.com 4

MySQL performance

MySQL Performance

does not Matter

www.percona.com 5

What matters?

My Website/App is

important

www.percona.com 6

Things to Note

Performance issues can be unrelated to MySQL

Maybe the solution is not MySQL

www.percona.com 7

Many other tools

Cassandra MongoDB Hadoop ElasticSearch Redis

www.percona.com 8

Putting it in Practice

Keep an unbiased view

Focus on the Stack

www.percona.com 9

Performance “sandbags”

Security

Manageability

Compatibility

Compliance

Ease of use by developers

www.percona.com 10

“Good Enough”

KNOW WHEN TO

STOP

www.percona.com 11

So what is important in MySQL?

Queries!!!!

•  Inserts •  Selects •  Updates •  Deletes

www.percona.com 12

What are we aiming for?

Quickly, Efficiently, Reliably…

www.percona.com 13

Quickly

Good Response Time

For all Users and Use Cases

www.percona.com 14

Efficiency

Resources

CPU, Memory, IO, Disk Space

Euro’s

Human Effort

www.percona.com 15

Reliably

High Availability!

High Availability and Performance Come together

www.percona.com 16

Consider things

•  But avoid focus on extreme outliers

Look beyond averages

•  Daily and Weekly cycle •  Growth of load and Data Size

Make sure to look at trends

over time

•  Data size changes? Think about the Future

www.percona.com 17

Schema

Look at the Schema and queries together

•  Minor Schema changes •  Data Architecture

www.percona.com 18

Schema Tips

Learn indexing •  http://bit.ly/1rAtamE

Design Schema for Data access •  Rethink your schema

Technical voodoo •  Partitioning and Sharding •  Normalization and denormalization

www.percona.com 19

The following also Matters

Infrastructure •  Storage systems, virtualization, etc.

Operating system and configuration •  Disk scheduler, FS, etc…

MySQL Version •  Are you using Percona Server, MariaDB, MySQL?

MySQL configuration •  The default configuration of Hell…

www.percona.com 20

Optimization Process

Low Hanging Fruit

•  MySQL Configuration

•  OS settings •  Indexes •  Caching

Medium Level

•  InfraStructure •  MySQL

Version

Hard Changes

•  Major schema changes

•  Application architecture changes

www.percona.com 21

QUICK TIPS

www.percona.com 22

Hardware

CPU

Memory

Disk

Network

www.percona.com 23

OS choices are simple

Linux

Server Grade

Recent

Well supporting MySQL

www.percona.com 24

OS tuning

Defaults are good for Common workload

Filesystem: EXT4 or XFS

More details: http://bit.ly/MySQLonLinux

Well supporting MySQL

www.percona.com 25

MySQL Version

New versions typically improve performance

Scalability, improved optimizer

Expect some regression

Try maybe a different flavor? Percona Server 5.6

www.percona.com 26

MySQL config optimisations

Do NOT run with defaults

Do not ever obsess with tuning

More details: http://bit.ly/1vth5Cu

www.percona.com 27

MySQL tuning in 1 slide

•  max_connections •  log_bin •  open_files_limit •  innodb_buffer_pool_size •  innodb_log_file_size •  innodb_flush_log_at_trx_commit •  innodb_flush_method=O_DIRECT

www.percona.com 28

TOOLS

www.percona.com 29

Tools to Consider

•  pt-query-digest (Percona Toolkit) •  Other Percona-toolkit tools •  Percona Monitoring plugins •  MySQL Enterprise Monitor •  Monyog •  VividCortex •  New Relic

www.percona.com 30

Percona toolkit helps

Identify Manage

Understand Monitor

Percona Toolkit

www.percona.com 31

What is Percona Toolkit?

•  Collection of command-line tools for a wide range of MySQL DBA tasks

•  Works with MySQL 5.0+, MariaDB, Percona Server, Percona XtraDB Cluster

•  Professionally developed, supported, and maintained by Percona

•  7 years maturity, thousands of tests, 20 testing environments, 1 full-time developer

•  Free, open-source www.percona.com/software/percona-toolkit

2

www.percona.com 32

Installation

•  Rpm and deb packages are available http://www.percona.com/downloads/percona-toolkit /LATEST/

•  Or you can use a tarball wget percona.com/get/percona-toolkit.tgz

•  Extract, then make, make install Or, if you only need a specific tool

wget percona.com/get/TOOL

2

www.percona.com 33

pt-query-digest

Command Line tool from Percona Toolkit

Process Logs Manualy

Powerful Filtering

Storing History

www.percona.com 34

pt-query-digest example

www.percona.com 35

Top Queries

www.percona.com 36

Query Details

www.percona.com 37

More Query Details

www.percona.com 38

Where to Analyze ?

• Development to catch bad queries early

• Production to catch good queries starting to go bad

Both !

www.percona.com 39

Query Performance can show modifying parameters

Workload Changes

Data Size

Changing Query Plans

Upgrading MySQL Version

Changing MySQL Settings

Hardware Changes

www.percona.com 40

But Why Slow ?

• MySQL Level • OS Level • Hardware/Environment Level

Need to know more than query

level information

www.percona.com 41

Hardware visibility

Eeek! I have a new machine

What are the specs?

What is the current memory usage?

How is the storage system currently set up?

What are the OS specifics that are currently applied?

www.percona.com 42

Lack of visibility

www.percona.com 43

pt-summary

[ec2-user@ip-172-31-0-249 ~]$ pt-summary # Percona Toolkit System Summary Report ###################### Date | 2015-08-31 11:20:01 UTC (local TZ: UTC +0000) Hostname | ip-172-31-0-249 Uptime | 14 days, 23 min, 1 user, load average: 0.00, 0.01, 0.05 Platform | Linux Release | Amazon Linux AMI release 2015.03 Kernel | 3.14.35-28.38.amzn1.x86_64Architecture | CPU = 64-bit, OS = 64-bit Threading | NPTL 2.17 SELinux | No SELinux detected Virtualized | Xen

www.percona.com 44

Continue pt-summary

# Processor ################################################## Processors | physical = 1, cores = 2, virtual = 2, hyperthreading = no Speeds | 2x2494.026 Models | 2xIntel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Caches | 2x25600 KB# Memory ##################################################### Total | 3.9G Free | 1.9G Used | physical = 2.0G, swap allocated = 0.0, swap used = 0.0, virtual = 2.0G Buffers | 147.4M Caches | 1.6G Dirty | 52 kB UsedRSS | 134.3M Swappiness | 60 DirtyPolicy | 20, 10 DirtyStatus | 0, 0

www.percona.com 45

Continue pt-summary (2)

www.percona.com 46

Continue pt-summary (3)

www.percona.com 47

pt-mysql-summary

MySQL visibility

www.percona.com 48

Continue pt-mysql-summary

www.percona.com 49

Continue pt-mysql-summary(2)

www.percona.com 50

Continue pt-mysql-summary(3)

www.percona.com 51

Pt-stalk

•  You are having performance problems, however it’s difficult to pinpoint what is causing it. •  Analyse Data •  Fix what is wrong

•  Problems can occur randomly •  Problems can happen while you are not monitoring •  pt-stalk is a tool which gives you the ability to monitor variables while

you are not keeping an eye on them

www.percona.com 52

Continue pt-stalk

www.percona.com 53

Innotop

Innotop

•  Identify •  Understand

•  Monitor

www.percona.com 54

Innotop

•  Sometimes it’s hard to keep track of the active queries on your system. •  Amount of queries are confusing •  Which query should I explain •  Which query is problematic? •  Why, how, where, what?

www.percona.com 55

Innotop: Enter innotop

•  It’s like top, but for queries

55

www.percona.com 56

Innotop: Innodb I/O information

56

www.percona.com 57

Innotop: Variables overview

57

www.percona.com 58

FINAL WORDS

www.percona.com 59 www.percona.com

Dim0 [email protected]

Thank You!