High Performance Rails with MySQL

High Performance Rails with MySQLJervin Real, March 2014

I am…

• Consultant, Percona

• @dotmanila

• http://dotmanila.com/blog/

• http://www.mysqlperformanceblog.com/

http://www.mysqlperformanceblog.com/

Rails Fu Mastah!

http://walksalong.files.wordpress.com/2007/05/bruce_on_rails.jpg

Beginner

http://www.devonring.ca/img/ouch.png

Customer Problems

Web Apps Performance

• Powerful servers

• CPUs, higher clock speeds

• Lots of memory

• Fast storage

• Scale in the cloud

• Agile development techniques

Why Not?

• Premature scaling is expensive

• Cost inefficient

• Agile means less effective measurement to compensate

for fast deployments

Squeeze the Software

• Exhaust application optimizations first

• You cannot optimize what you can’t measure

• Cacti, NewRelic, Scout

• NewRelic RPM Developer Mode, Rails Footnotes (2, 3,

4!), Google PerfTools for Ruby

• Dumb schemas and queries (somewhat)

Characteristics of Rails

:primary_key, :string, :text, :integer, :float, :decimal, :datetime, :timestamp, :time, :date, :binary, :boolean

• Dumb schemas

• Likes to use SHOW FIELDS


• Dumb schemas


• N+1 queries problem

• Well documented and discouraged for use


• Dumb schemas


• N+1 queries problem

• Well documented and discouraged for use

• Does SELECT FOR UPDATE

• NOOP transactions still wrapped in BEGIN/COMMIT


• FK relationships - logical - GOOD

Rails - Good


• Knows how to use PRIMARY KEYs — VERY GOOD!

Rails - Good



• Knows NOOP changes - GOOD

Rails - Good



• Knows NOOP changes - GOOD

• Database agnostic - GOOD

Rails - Good

MySQL by Default

• Assumes you have less powerful hardware

• ironically on 5.5, assumes you have very powerful

CPUs - innodb_thread_concurrency = 0

MySQL by Default




• Not optimized for faster storage

MySQL by Default





• Still have bad configuration assumptions

• Query cache

• MyISAM < 5.5.5

MySQL by Default





• Still have bad configuration assumptions

• Query cache

• MyISAM < 5.5.5

• Still haunted by mutexes

What to Optimize - MySQL

• Disable Query Cache



• query_cache_size = 0



• query_cache_size = 0

• query_cache_type = 0



• skip_name_resolve




• Use >= 5.5




• Use >= 5.5

• Use Indexes, EXPLAIN should be your friend!




• Use >= 5.5

• Use Indexes, EXPLAIN should be your friend!

• 5.6 does Subquery Optimizations


• Slow queries - search and destroy

• long_query_time = 0

• log_slow_verbosity - Percona Server

• pt-query-digest/Percona Cloud Tools


• Use InnoDB

innodb_buffer_pool_size # keep hot data in memory innodb_log_file_size # allow more IO buffer innodb_flush_method = O_DIRECT # skip OS cache innodb_[read|write]_io_threads > 4 innodb_io_capacity innodb_adaptive_flushing_method

What to Optimize - Rails

• counter_cache

• Good for InnoDB if you often SELECT COUNT(*)

• Indexes for find_by, where and family@posts = Post.where(title: params[:keyword]) !mysql> EXPLAIN SELECT `posts`.* FROM `posts` WHERE `posts`.`title` = 'LongTitles' \G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: posts type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 2 Extra: Using where 1 row in set (0.00 sec)


• dependent: :destroy

24 Query BEGIN 24 Query SELECT `comments`.* FROM `comments` WHERE `comments`.`post_id` = 1 24 Query DELETE FROM `comments` WHERE `comments`.ìd` = 2 24 Query DELETE FROM `posts` WHERE `posts`.ìd` = 1 24 Query COMMIT

24 Query BEGIN 24 Query DELETE FROM `comments` WHERE `comments`.`post_id` = 4 24 Query DELETE FROM `posts` WHERE `posts`.ìd` = 4 24 Query COMMIT

BAD

Use dependent: :delete_all


• Cache SHOW FIELDS output - patches for now

• Disable innodb_stats_on_metadata


• Cache SHOW FIELDS output - patches for now

• Disable innodb_stats_on_metadata

• Pull only the columns you need - especially excluding

BLOBS

@posts = Post.select(“title”)


• Avoid pessimistic locks when possible - SELECT … FOR

UPDATE - UPDATE directly and return affected rows




• Avoid N+1 queries - use JOIN or includes


@posts = Post.limit(2) [email protected] do |post| puts post.authors.name end

24 SELECT `posts`.* FROM `posts` LIMIT 2 24 Query SELECT àuthors`.* FROM àuthors` WHERE àuthors`.ìd` = 1 ORDER BY àuthors`.ìd` ASC LIMIT 1 24 Query SELECT àuthors`.* FROM àuthors` WHERE àuthors`.ìd` = 2 ORDER BY àuthors`.ìd` ASC LIMIT 1

With this:

You get this:


@posts = Post.includes(:authors).limit(2)

24 Query SELECT `posts`.* FROM `posts` LIMIT 2 24 Query SELECT àuthors`.* FROM àuthors` WHERE àuthors`.ìd` IN (1, 2)

With this:

You get this:


@posts = Post.joins(:authors).limit(2);

24 Query SELECT `posts`.* FROM `posts` INNER JOIN àuthors` ON àuthors`.ìd` = `posts`.àuthors_id` LIMIT 2

With this:

You get this:





• Compress MySQL requests, especially transactions!






• Learn and use SQL - find_by_sql






• Learn and use SQL - find_by_sql

• Don’t accept Model defaults

Further Thoughts

• GDB, Strace, OProfile

• Smart aggregation, use summary tables when possible

• Rails > Caching > MySQL

• Avoid:config.action_controller.session_store = :active_record_store

http://www.amyvernon.net/wp-content/uploads/2014/01/upward-graph-striving.png

Thank you!

… and

• We’re hiring!

• http://www.percona.com/about-us/careers/open-

positions

High Performance Rails with MySQL

Software

query cache query

rails cache

cache good

query cache skip

mysql use innodb innodb

use characteristics

os cache innodb

rails dependent