MariaDB Performance TuningMariaDB Training
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
Introduction
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Introducing MariaDB AbFounders from MySQL the
Company and Community Funded by Founders, Employees,
and Venture Capital Over 100 Employees Several former MySQL Employees
and Community Members in over 14 Countries
3
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Personal IntroductionsInstructor
Name and Background
Participants Name and Company MariaDB Experience How You Use MariaDB Needs Related to Course Topics
4
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Class Schedule & Personal ConcernsStarting and Ending Times Planned Breaks On- Site
Location of Rest Rooms Smoking Areas Snacks and Drinks
LVC Classes Chat with Everyone
5
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Course OutlineBasics & Core
Tuning Overview Collecting Information
Tweaking Schema Tuning Query Tuning Common Bottlenecks
Server & Storage Engines Architecture Server Configuration MyISAM InnoDB
Administration MariaDB Replication Maintenance Impact
6
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
Tuning Overview
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Tuning TermsPerformance Efficiency Scalability Benchmark Throughput Risk
Queuing SLA Response Time Capacity Resources Uptime
8
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Value in Performance TuningEfficient Use of Hardware Resources
Anticipate Needs
Best Performance for Users Machines Cost less than User Time Competitive Advantage
Maintain Spare Infrastructure Capacity Be Prepared for Application Development Needs (New
Features) Allow for Unexpected Traffic Spikes or Other Changes
9
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Tuning LevelsApplication
Primary Responsibility of Architects and Developers Secondary Responsibility of DBA for Input
Database Primary Responsibility of DBA — Tuning and Monitoring
Secondary Responsibility of Developers — Gain Insight and Understanding
10
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Tuning LevelsServer
Primary Responsibility of System Administrator Secondary Responsibility of DBA — Knows Well Databases
Network Primary Responsibility of Network Administrator Secondary Responsibility of DBA — Replication or Clusters
11
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Tuning Routine - When to Tune
Tune from Start in the SDLC Start Early to Ensure Schema is Well Constructed Test Queries on Real Data — Watch for Bottlenecks Over Tuning without Production Data or Traffic is Counter
Productive
Conduct Periodic Reviews of Production Systems Watch for Schema, Query and Significant Changes Check Carefully New Application Features
Monitor System Resources — Disk, Memory, Network, CPU
Eliminate the Obvious Potential Problems Be Proactive in Solving Problems
Emergency Response Tuning
12
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Tuning Routine - When Not to Tune
Identify Objectives in Advance Adhere to Objectives
Decide on Best Use of Resources Tune Appropriately Don't Excessively Tune
Affect Adversely Data Integrity Where is Speed Most Important Where is Integrity Most Important Adhere to these Boundaries
13
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Establish BaselineNumber of Queries Expected per Unit of Time Number of Concurrent Client Connections Expected
Average Number of Clients Peak Number of Concurrent Clients
Length of a Client Connection Expected — In Milliseconds Size of Dataset — Fit in Memory When all Requirements will Double
14
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Construct TestsDetermine System Resource Limits and Design Test
Should Run Normally Should Exceed Server Resource Limits Should Simulate Nightly Back-up Load without Users
Noticing
Examine Impact of Configuration Changes Remove or Restrict Access to CPU core or Hard Drives Start another Process via cron that Consumes Suddenly
Memory Observe What Happens When a Caching Layer goes
Offline
Use Tests to Identify Easy Solutions
15
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Examine Test ResultsTest System Identical to the Production System
Operating System (Version, Libraries, Patches)
Hardware (Number of Cores, Physical Memory, Disk Speed) Other Resources (Memory Available to MariaDB, Competing
Services) Test System Simulating Real Data and Traffic
Number of Clients (Concurrently, per Hour per Day, Peak Traffic)
Any Caching that might Skew Results
16
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
HardwareOne Service per Server is Ideal to Prevent Contention More CPU Cores is generally Good More Disk is usually Better
Large Datasets, Fast Disks are Ideal
RAM is usually Best — Traffic Dependent
More of Dataset in Memory, Fewer Slow Disk Operations
17
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
Collecting Information
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Tools & StatisticsIdentify Accurately and Carefully Trouble Spots
Guessing is Rarely Useful
Gather Performance Stats with MariaDB and OS Tools SHOW Statements PERFORMANCE_SCHEMA CPU, Disk, Network, Memory, & Swap Stats
Retain Snapshots of Multiple Stats Data from a Single Point Shows very Little
Automate the Collection of Stats into Logs Can be Useful for Emergency Tuning
19
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
SHOW PROCESSLISTSnapshot of mysqld Activity
mysqld is Multi-Threaded, One Thread per
Client Connection (i.e., query, transaction) — a
"process" is a "thread"
Accumulate SHOW PROCESSLIST Snapshots to
build History of Thread Activities
20
Time: Length of Time Thread has been in Current State State: Indicates Thread Activity
SHOW PROCESSLIST \G
****** 1. row ******** Id: 9546 User: monyog Host: localhost:40600 db: NULL
Command: Sleep Time: 192 State: Info: NULL Progress: 0.000
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Thread States
21
After Create
Analyzing
Checking Permissions
Checking Table
Cleaning Up
Closing Tables
Converting HEAP to MyISAM
Copy to tmp Table
Copying to Group Table
Copying to tmp Table
Copying to tmp table on Disk
Creating Index
Creating Sort Index
Creating Table
Creating tmp Table
Deleting from Main Table
Deleting from Reference Tables
discard_or_import_tablespace
End
Executing
Execution of init_command
Flushing Tables
Freeing Items
FULLTEXT initialization
Init
Killed
Locked
Logging Slow Query
Login
Manage Keys
NULL
Opening Tables, Opening Table
Optimizing
Preparing
Purging Old Relay Logs
Query End
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Thread States
22
Reading from Net
Removing Duplicates
Removing tmp Table
Rename
Rename Result Table
Reopen Tables
Repair by Sorting
Repair Done
Repair with Keycache
Rolling Back
Saving State
Searching Rows for Update
Setup
Sorting for Group
Sorting for Order
Sorting Index
Sorting Result
Statistics
System Lock
Table Lock
Updating
Updating Main Table
Updating Reference Tables
User Lock
User Sleep
Waiting for (tables|tables|table flush)
Waiting for All Running Commits to Finish
Waiting for Commit Lock
Waiting for Global Read Lock
Waiting for lock_type Lock
Waiting for Release of Read Lock
Waiting on Cond
Waiting to Get Read Lock
Writing to Net
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
SHOW STATUS - Global or Session
Returns List of Internal Counters GLOBAL for System-Wide Status — Since Start-Up
SESSION for Local to Client Connection
FLUSH STATUS Resets Local Counters Monitor Changes to Counters to Identify Hot Spots Collect Periodically Status Snapshots to Profile Traffic
23
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Status Counters
24
Aborted_clients
Aborted_connects
Binlog_cache_*
Bytes_received
Bytes_sent
Com_*
Compression
Connections
Created_tmp_disk_tables
Created_tmp_files
Created_tmp_tables
Delayed_errors
Delayed_insert_threads
Delayed_writes
Flush_commands
Handler_*
Innodb_*
Key_*
Last_query_cost
Max_used_connections
Not_flushed_delayed_rows
Open_*
Opened_files
Opened_table_definitions
Opened_tables
Prepared_stmt_count
Qcache_*
Queries
Questions
Rpl_status
Select_*
Slave_open_temp_tables
Slave_retried_transactions
Slave_running
Slow_launch_threads
Slow_queries
Sort_*
Ssl_*
Table_locks_immediate
Table_locks_waited
Tc_log_max_pages_used
Tc_log_page_size
Tc_log_page_waits
Threads_cached
Threads_connected
Threads_created
Threads_running
Uptime*
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
PERFORMANCE_SCHEMASimilar to INFORMATION_SCHEMA, but Performance Tuning Monitors MariaDB Server Events
Function Calls, Operating System Waits, Internal Mutexes, I/O Calls
Detailed Query Execution Stages (Parsing, Statistics, Sorting)
Some Features Storage Engine Specific
Monitoring Lightweight and Requires No Dedicated Thread Not a Simple, Quick Fix
Interpreting Output Requires knowledge of Internals Designed to be Used Iteratively with Successive Refinement
25
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
PERFORMANCE_SCHEMA Tables
26
SET PROFILING=ON; SHOW PROFILES; SHOW PROFILE FOR QUERY 1;
cond_instances
events_waits_current
events_waits_history
events_waits_history_long
events_waits_summary_by_instance
events_waits_summary_by_thread_by_event_name
events_waits_summary_global_by_event_name
file_instances
file_summary_by_event_name
file_summary_by_instance
mutex_instances
performance_timers
rwlock_instances
setup_consumers
setup_instruments
setup_timers
threads
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
External ToolsApacheBench htop httperf innotop iostat / vmstat / top maatkit MyBench MariaDB Benchmark Suite (sql-bench)
mysqladmin (extended-status) mysqlslap mytop statpack Super Smack SysBench
27
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
Schema Tuning
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Naming ConventionConsistent, Descriptive Names Reduce Mistakes in
Development and Maintenance Plural Form for Table Names (products)
Singular Form for Column Names (name_last, email)
Alphabetical Order for Link Tables (posts_users)
Long Names are Fine — Use Aliases in Queries
29
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Choosing Data TypesUse Appropriate Data Type (INT for Numbers, VARCHAR)
Use Accurate Types where required (DECIMAL vs. DOUBLE)
Use Smallest Useful Type On-Disk Size does Not Equal In-Memory Size Variable Length Fields are often Padded Use NOT NULL, where Practical
A NULL field uses slightly More Disk and Memory (Depends on Storage Engine)
Use PROCEDURE ANALYSE( )
30
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Data Types for Integers
31
Documentation on Integer Data Types: https://mariadb.com/kb/en/mariadb/data-types/
Use UNSIGNED when Appropriate
INT(n) Specifies Display Precision, Not Storage Precision
Size and Precision is Storage Engine Dependent Define Handling of Out-of-Range Values with sql_mode
Default Mode: Values are Truncated Silently Strict Mode: Errors are Generated
TINYINT
SMALLINT
MEDIUMINT
INTEGER,INT
BIGINT
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
String Data Types
32
Documentation on String Data Types: https://mariadb.com/kb/en/mariadb/string-data-types/
All String Data Types have a Character Set CHAR(n) — number of characters, not bytes, wide VARCHAR(n)— Changes to CHAR in Implicit Temporary Tables and mysqld internal buffers
TEXT —Not Supported by the MEMORY Storage Engine; Implicit Temporary Tables may Convert to MyISAM
CHAR
VARCHAR
TINYTEXT
TEXT
MEDIUMTEXT
LONGTEXT
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Character Set & Collation
33
Documentation on Character Sets and Collation: https://mariadb.com/kb/en/mariadb/data-types-character-sets-and-collations/
SELECT * FROM table1 ORDER BY col1 COLLATE latin1_german2_ci;
Character Set may be Global or for Schema, Table or Column
Multi-Byte Character Sets Increase Disk Storage and Working Memory Requirements (e.g, UTF-8 Requires 3 or 4 bytes per Character)
Collations affect String Comparison (Character Order) Collations can be Changed for Query:
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Date & Time Data Types
34
DATE — from 1000-01-01 to 9999-12-31 (YYYY-MM-DD)
TIME — from -838:59:59 to 838:59:59
DATETIME — Same Range with Time (YYYY-MM-DD HH:mm:ss) TIMESTAMP — Unix timestamp, in seconds from
1970-01-01 Many Apps Store UNIX_TIMESTAMP() values in unsigned integer
field
YEAR — Accepts YY or YYYY
DATE
TIME
DATETIME
TIMESTAMP
YEAR
Documentation on Date and Time Data Types: https://mariadb.com/kb/en/mariadb/date-and-time-data-types/Documentation on Microseconds in MariaDB: https://mariadb.com/kb/en/mariadb/microseconds-in-mariadb/
SELECT CURTIME(4); +---------------+ | CURTIME(4) |
+---------------+ | 05:33:09.1061 | +---------------+
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Listing Acceptable Values
35
Documentation on ENUM: https://mariadb.com/kb/en/mariadb/enum/ Documentation on SET: https://mariadb.com/kb/en/mariadb/set-data-type/
CREATE TABLE colors (primary_colors ENUM('red', 'yellow', 'blue'));
ENUM is an Enumerated List of String Values — uses a 2-byte integer index
CREATE TABLE colors
(primary_colors SET('red','yellow','blue'));
INSERT INTO colors VALUES('red'), ('red,blue');
Simple Example for an ENUM Column
Simple Example for a SET Column
SET is a Specified List of String Values — Can Hold Multiple Specified Values
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Keys and IndexesMake PRIMARY KEY as Small as Practical InnoDB Stores Copy of PRIMARY KEY with each Index Entry
Secondary Indexes Grow with PRIMARY KEY
Use Surrogate PRIMARY KEY, Not Natural Ones AUTO_INCREMENT is like a Sequence AUTO_INCREMENT UNSIGNED NOT NULL Careful of AUTO_INCREMENT Contention for Pre-Plugin InnoDB
Consider Partial (Prefix) Index for String Indexes With Good Index Selectivity, Won't Affect Performance
Much Without Good Selectivity is a Different Problem
36
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
File StorageStore Files in BLOB Useful
Included in Back-Up and Replication Guaranteed to be Consistent with Other Data
Store Files in BLOB has Problems Database gets Too Large MariaDB Can't do Partial Field Read — MyISAM Reads Entire File InnoDB can Skip if Field Not Referenced in Query
Can Minimize by putting Files in Separate Table (1:1 relationship)
Store Files in File System and File Name and Path in Table Database is Smaller and More Efficient Organize Separate Back-Up, possibly with External Locking
for Consistency
37
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Tuning Tables OverallMinimize Table Size on Disk and in Memory Archive Table Data if Possible and Appropriate Remove Duplicate or Unused Indexes
No need to Index a Column if it’s First Field in Another Index
Use Appropriate Data Types — Smaller is Better
Consider Sharding Large Tables across Multiple Servers
38
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Normalization of TablesProcess of Optimizing and Factoring a Schema
1:1, 1:n and n:n 1NF / 2NF / 3NF
Makes a Data Set as Smaller by Eliminating Redundant or Duplicated Data
Problem if Data is Smaller than Key
Ensures Data Set Consistency and Integrity Often Improves Concurrency by Reducing Locking
Overhead - Not Always Increases Number of Tables and Associated
Maintenance
39
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Denormalization of TablesComplete Normalization can Slow Queries More Complex JOIN Queries are Drains and Harder to Maintain
Adding Redundant Data Back to Simplify Problems with JOIN Queries
Combine 1:n Relationships Add Pre-Computed Columns Use Fake Materialized Views
Denormalization Adds Redundant Data to Schema Write Queries become More Complex or Numerous as Multiple
Locations must be Maintained
It's Easier to Normalize First, Then Denormalize when Appropriate
40
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
ViewsMERGE
Query and Original View SELECT are Merged Often the Fastest Option, as Base Table Indexes Used
TEMPTABLE View Data Materialized to Temporary Table, then Query
Executed Can be Slower, as Temporary Table Doesn't have Indexes
UNDEFINED Algorithm is Chosen on Fly Usually Defaults to TEMPTABLE
41
CREATE VIEW ALGORITHM = ...
AS SELECT ...
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
Query Tuning
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Query Tuning OverviewTry Not to Query Tune on Production Server Use Test Server with same Hardware and OS Use Copy of Production Data Set Query Frequency is as Important as Query Speed
Moderately Slow Queries are often a Bigger Problem than a Rarely Run Very Slow Query
43
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
IndexingIndexes Improve Read Performance
Without Index, MariaDB Must Read Every Row — a Full Table Scan
With Index, MariaDB can jump to Requested Rows Reduced I/O and Improving Performance Index Increase cost of Writes
Index Improve Writes since they Require Reads Find Balance
Index for Speed, but Avoid Indexing Excessively or Arbitrarily Remove Dead or Redundant Indexes
44
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Index SizeKeep Indexes as Small as Practical
Faster since More Likely to Fit in Memory Rebuilds Faster after Writes PRIMARY KEY should be Minimum Useful Size
Use Partial Prefix Indexes for String Columns
May Slow Searches Slightly, but Reduce Index Size
Use Index Cardinality (Uniqueness Measure) Only If Necessary — Re-evaluate as Data Grows
Low Cardinality Indicates many Duplicates High Cardinality is More Useful
45
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
B-Tree IndexesBalanced Binary Search Trees holding Pointers to
Rows Precision depends on Storage Engine MyISAM stores within .MYD File Pointers to Rows (Offsets) InnoDB Primary Keys store Row — Other Indexes Store Copy
of Key
Good for many Operations Equality (= <>) Range Access (> < >= <= BETWEEN)
Partial String (i.e., LIKE) — String Constants without Leading % Sorting (ORDER BY) — Indexes are Pre-Sorted
46
Version 3.1, Slide Copyright 2017. MariaDB Ab. Commercial in Confidence
Advanced MariaDB
InnoDB Adaptive Hash IndexInnoDB performs like In-Memory
Database with AHI Leads to High CPU Usage
Mutex for Adaptive Hash Index
47
SHOW ENGINE INNODB MUTEX;
+--------+------------------------------+---------------+
| Type | Name | Status |
+--------+------------------------------+---------------+
| InnoDB | &rseg->mutex | os_waits=1 |
| InnoDB | &dict_sys->mutex | os_waits=1 |
| InnoDB | &log_sys->mutex | os_waits=210 |
| InnoDB | &buf_pool->flush_state_mutex | os_waits=5095 |
| InnoDB | &buf_pool->LRU_list_mutex | os_waits=41 |
| InnoDB | combined &block->mutex | os_waits=11 |
| InnoDB | &new_index->lock | os_waits=2 |
| InnoDB | &new_index->lock | os_waits=16 |
| InnoDB | &new_index->lock | os_waits=1 |
| InnoDB | &dict_operation_lock | os_waits=221 |
| InnoDB | &log_sys->checkpoint_lock | os_waits=2920 |
| InnoDB | &btr_search_latch_arr[i] | os_waits=6 |
| InnoDB | combined &block->lock | os_waits=62 |
+--------+------------------------------+---------------+
[mysqld] innodb_adaptive_hash_index=OFF
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Composite IndexesSpans Multiple Columns Increases Search Speed Best when All Columns in Index
Specified in Query Not Used with Latter Columns of Index
48
CREATE TABLE people (id INT AUTO_INCREMENT KEY,
name_first CHAR(20), name_last CHAR(20), city CHAR(20), KEY names(name_first,name_last,city));
SELECT * FROM people WHERE name_first = 'Bob' AND name_last = 'Smith';
SELECT * FROM people WHERE name_last = 'Smith';
Index Used
Index Not Used
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Joining TablesJOIN is a Cartesian Product of Table Subsets with
Variations The Optimizer can Help
Orders Tables for Minimum Cost — Fewest Rows Examined Converts OUTER to INNER when Possible
Ensure Columns in ON and USING Clauses are Indexed
Restrict GROUP BY and ORDER BY to Columns in same Table
Consider JOIN Decomposition
49
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Query AnalysisUse the Slow Log to find Problem Queries (--
log-slow-queries)
Use mysqldumpslow Utility for Manageable Reports
Use EXPLAIN to see how MariaDB Executes a Troublesome Query and if Indexes are Used
Use EXPLAIN EXTENDED and SHOW WARNINGS to see how MariaDB Rearranges a Query before Execution
50
Documentation on EXPLAIN: https://mariadb.com/kb/en/mariadb/explain/ Documentation on EXPLAIN EXTENDED: https://mariadb.com/kb/en/mariadb/explain/#explain-extended Documentation on mysqldumpslow: https://mariadb.com/kb/en/mariadb/mysqldumpslow/
EXPLAIN SELECT * FROM employees WHERE MONTH(birth_date) = 8 \G id: 1 select_type: SIMPLE
table: employees type: ALL possible_keys: NULL key: NULL
key_len: NULL ref: NULL rows: 299587 Extra: Using where
EXPLAIN EXTENDED SELECT …; SHOW WARNINGS;
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Fields from EXPLAINField Descriptionid Query identifierselect_type Type of SELECT performed
table The table in use (each table in a JOIN will use one row)
*type The table access strategy
possible_keys Any available indexes that could resolve the query
key Index MariaDB chose from possible_keyskey_len The portion of the chosen index to be used (width in bytes)ref Columns from the index, or a constant, usedrows Number of rows the query is expected to touchExtra Extra information relating to the access strategy
51
Documentation on EXPLAIN: https://mariadb.com/kb/en/mariadb/explain/ Documentation on EXPLAIN EXTENDED: https://mariadb.com/kb/en/mariadb/explain/#explain-extended
* See Next Slide for List of Types
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Scan or Index Method Types
Field Description
const A constant value can be read once and cached
eq_ref One index access per outer query row
ref Multiple index accesses per outer query row
index_merge Multiple indexes used, merging into a single result set
range Multiple index accesses to return all rows within a range
index Full index scan (every index entry is read sequentially)
all Full table scan (every record is read sequentially)
Method Used to Access Table
52
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
EXPLAIN select_typeType of SELECT being Performed
53
Type of SELECT Description
SIMPLE Normal SELECT Statement
PRIMARY First (Outermost) SELECT in JOIN
UNION Subsequent Query in JOIN
DEPENDENT UNION Subsequent Query Dependent on an Outer Query Result
UNION RESULT Query is the Results Set from a UNION
SUBQUERY First SELECT in a Sub-Query
DEPENDENT SUBQUERY A Sub-Query that must be Re-Executed for Each Combination of Values in Outer Query
DERIVED Sub-Query in a FROM Clause
UNCACHEABLE SUBQUERY
Sub-Query that must be Re-Executed for Every Iteration of an Outer Query
UNCACHEABLE UNION Subsequent Query in a UNION belonging to an Uncacheable Subquery
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
EXPLAIN Extraconst row not found
Impossible HAVINGNo matching min/max rowNot existsSelect tables optimized awayOpen_trigger_onlyUsing filesortUsing join buffer Using
intersect(...)
Distinct Impossible WHEREno matching row in const
tableRange checked for each
record (index map: N)Skip_open_table Open_full_tableUsing indexUsing index conditionUsing sort_union(...)Using temporary
Using WHERE with pushed condition
Full scan on NULL keyImpossible WHERE noticed after
reading const tablesNo tables used Scanned N
databasesOpen_frm_only
unique row not found Using index for group-by Using UNION(...) Using where
54
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
MySQL OptimizerOptimizer calculates Best Plan
Can't Take too Long or Defeats Purpose
Many Possible Plans for Large Joins Greedy Searches used to Reduce
Possibilities Possible to Adjust Decision Parameters
Limit Number of Tables Considered (optimizer_search_depth)
Skip Plans Based on Row Count Estimates (optimizer_prune_level)
55
Documentation on optimizer_search_depth: https://mariadb.com/kb/en/mariadb/server-system-variables/#optimizer_search_depth Documentation on optimizer_prune_level: https://mariadb.com/kb/en/mariadb/server-system-variables/#optimizer_prune_level Article on Setting Optimizer Search Depth: https://mariadb.com/resources/blog/setting-optimizer-search-depth-mysql
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Optimizer HintsGiving Hints can Affect the Optimizer’s
Choices Index Choices USE INDEX IGNORE INDEX FORCE INDEX
Buffering Methods for GROUP BY SQL_BUFFER_RESULT SQL_SMALL_RESULT SQL_BIG_RESULT
56
SELECT * FROM people FORCE INDEX names WHERE name_first = 'Bob';
Documentation on Forcing Indexes: https://mariadb.com/kb/en/mariadb/index-hints-how-to-force-query-plans/ Documentation on Indexing with GROUP BY: https://mariadb.com/kb/en/mariadb/index-hints-how-to-force-query-plans/#forcing-an-index-to-be-used-for-order-by-or-group-by
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Slow Query LogContains Queries that took Longer
than long_query_time Seconds to Execute — Off by Default
Activate when First Using an Application to Monitor Slow Queries
Default Location is in Data Directory
Use mysqldumpslow to Process Log
Use also pt-query-digest as an Alternative to Process Log
57
Documentation on Slow Query Log: https://mariadb.com/kb/en/mariadb/slow-query-log-overview/ Documentation on mysqldumpslow: https://mariadb.com/kb/en/mariadb/mysqldumpslow/ Documentation on pt-query-digest: https://www.percona.com/doc/percona-toolkit/2.1/pt-query-digest.html
[mysqld] log-slow-queries = ON long-query-time = 2
log-queries-not-using-indexes log-slow-admin-statements log-slow-vervosity = query_plan,explain
Excerpt from /etc/my.cnf.d/server.cnf
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
ORDER BY NULLGROUP BY Triggers an Implicit ORDER BY
(filesort)
ORDER BY NULL Avoids It
58
EXPLAIN SELECT Continent, COUNT(*) FROM country GROUP BY Continent \G
... Extra: Using temporary; Using filesort
EXPLAIN SELECT Continent, COUNT(*) FROM country
GROUP BY Continent ORDER BY NULL\G ... Extra: Using temporary
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
ORDER BY RAND( )MariaDB Can’t Use an Index for Ordering Entire Results Set must be Generated in a Temporary
Table Combined with GROUP BY, it can kill servers
Used Often be Developers when an Application is New or Small
Doesn't Necessarily Scale Well
59
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
LIMIT ClausesLIMIT is Evaluated at End of Query Plan
ORDER BY with LIMIT needs to Sort All Rows
LIMIT(o, n) will still Parse o + n Rows LIMIT in Sub-Queries Before Sorting
Scan in Index Order and Apply LIMIT vs. Sort and LIMIT
60
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
SELECT * StatementSELECT with * Performs Badly
Increases Size of Results Set — Memory and Network Overhead on Client and Server
Increases Disk Activity for Storage Engines that can Read Partial Records (not MyISAM, but InnoDB)
Causes Development and Usage Problems Ambiguity with JOIN tables having same Column Names Problems when Schema has been Changed
61
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Pagination — Common Scenarios
Scenario 1 Processes Data Set Twice Depends on Complexity of WHERE Clause Worse if User Controlled Filtering
Allowed
Scenario 2 The SQL_CALC_FOUND_ROWS Option Causes
Entire Results Set to be Generated The ORDER BY and LIMIT Clauses become
a Drain as Data Grows
62
SELECT SQL_CALC_FOUND_ROWS FROM table1
JOIN table2 USING(id) ORDER BY col2 LIMIT 4;
SELECT FOUND_ROWS();
SELECT COUNT(*)
FROM table1 WHERE col1 = 'something' AND col2 = 'something else';
SELECT * FROM table1
JOIN table2 WHERE col1 = 'something' ORDER BY col2 LIMIT 100;
Scenario 1
Scenario 2
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Pagination — Limit Size of Data Set
Scenario 1 Archive Old Data to Another
Table Add an Index to Column (e.g.,
col_time)
Scenario 2 The First Column is Assumed to be
PRIMARY KEY Value Can Be a Constant, Re-
Generated Periodically
63
SELECT * FROM table1 WHERE id < 1000;
SELECT * FROM table1 WHERE col_time > NOW()
- INTERVAL 6 MONTH;
Scenario 1
Scenario 2
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Pagination — Reduce Data Set in ORDER BY and LIMIT
Scenario 1 First column Assumed to be PRIMARY KEY The ORDER BY and LIMIT Clauses often Use a
Temporary Table Retrieving Only PRIMARY KEY keeps
Temporary Table Small Allows 'Using Index' — Especially with InnoDB
Scenario 2 A Safe, Fixed Size Range Access
64
SELECT id FROM table1 WHERE col1 = 'something'
ORDER BY col2 LIMIT 10;
SELECT col2, col4 FROM table1
JOIN table2 USING(id) WHERE id IN(10, 14, 28);
Scenario 1
Scenario 2
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Sub-QueriesAdvantages of Sub-Queries before MariaDB 5.3 for Certain
SQL Constructs SELECT...FROM (( (SELECT filter|order|limit) JOIN SELECT
order|limit ) JOIN column)
...WHERE exists (SELECT... LIMIT1) to remove DISTINCT or GROUP BY
SELECT ( SELECT ) FROM to remove DISTINCT or GROUP BY
Disadvantages of Sub-Queries before MariaDB 5.3 for Certain SQL Constructs IN or NOT IN that can be Rewritten with JOIN or LEFT JOIN
WHERE IS NULL Replay for Each Row of Outer Table, No Materialization
VIEW is a Sub-Query with Additional Security Features
65
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
Common Bottlenecks
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Lack of CachingCaching can Occur at Many Levels
67
Application NetworkUser Session Data
Pre-Compiled, Static ContentProxy Server
Content Distribution Network memcached
MariaDB OS
Query Cache Storage Engine
File System Cache
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Using Caches — Questions to Consider
Do Users Care if Counter on Web Page is Real Time? Can Any Content be Re-Generated Periodically in a
Controlled Fashion and Served from a Cache? Can the Application Scale Cheaply, Simply by Sharing
Caches between Servers? Can the Queries be Rewritten to be Cacheable in the
Query Cache, or their Results put in memcached?
For MyISAM tables, does the Server have Sufficient Memory to Operate a File System Cache?
68
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Using CacheQuery Cache has to be Enabled to Use Use memcached if controlling Server - If not, try MEMORY tables
Web Application Use for Pagination (Page Count and id = N)
Use if Data Not Specific to User and Not Required to Update in Real Time (Static, or Mostly Static)
Use if User Specific Data lasts Longer than One Page Request
69
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Slow at StartSlow at Start could be from Cold Index & Data
Caches — Try Preloading
InnoDB Clustered Index
InnoDB without PRIMARY Indexes For Speed, Try Preloading PK Ranges
in Parallel MyISAM Key Cache
70
SELECT count(*) FROM tbl WHERE idx_col LIKE '%0%';
LOAD INDEX INTO CACHE;
SELECT count(*) FROM tbl WHERE non_idx_col=0;
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Temporary Tables on DiskWhen MEMORY Engine Can't Handle an Implicit
Temporary Table, it Converts to MyISAM on Disk Table contains TEXT or BLOB Columns - Not Supported Small Values for tmp_table_size or max_heap_table_size Record Size greater than 8K (VARCHAR Converting to CHAR)
SHOW STATUS, created_tmp_disk_tables Incrementing SHOW PROCESSLIST, Threads in "Creating tmp table on disk"
Move TEXT and BLOB Columns to another Table Make Large VARCHAR Columns as Small as Possible
Increase tmp_table_size and max_heap_table_size
71
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Persistent ConnectionsSome Client Languages and Connection Pool Solutions
use Persistent Connections (PHP mysql_pconnect( ))
Not very Effective — a New Client Connection Simpler
Client Session Buffers can Stay Allocated too Long if the Client Pool of Application is not Efficient, Increasing Memory Usage
Transactions can be Held Open if Application is Built Poorly, Reducing Concurrency
72
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Domain Name ServiceMariaDB uses DNS and Reverse DNS for IP and
Hostname with Client Connections A Slow DNS Server can Slow MariaDB Client
Connection Running a Local DNS Server Helps with Lookup Speed
and Security Use --skip-name-resolve to Tell MariaDB Not to do
DNS Lookup — Then Use Only IP Addresses in Permissions Tables
73
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
Architecture
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
MariaDB Terrain
mysql
application (e.g., php)
mysqladmin
mysqld
Archive
InnoDB
MyISAM
cron
DBA
web user
socket file
tcp/ip
socket file
ssh & tcp/ip
Memory
SQL Tier — ClientsStorage Engine
TierServer Tier
Handler Interface
75
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Client Connections
76
TCP/IP Connections available on All Platforms
--skip-networking (disables)
Socket Files available on Unix Platforms - fastest choice Named Pipe and Shared Memory available on Windows
--enable-named-pipe, --shared-memory
MariaDB Connections require little resources and easy to open.
Most use External Connection Pools (not needed usually) Set Global Client Connection Limit (max_connections=n)
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Client Libraries and the Like
Information on MariaDB Connectors: https://mariadb.com/products/connectors-plugins
Various Libraries for connecting to MariaDB — Most are Wrappers for MariaDB C API, which uses MariaDB Network Protocol
PHP API, Perl DBI, Python, Ruby MariaDB Connector for Java Native C/C++ driver MariaDB ODBC Drive Embedded MariaDB (libmysqld)
77
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
MariaDB Query Process
Thread Cache
Archive
InnoDB
Aria
Memory
Storage Engines
Query Cache
SQL Parser
Optimizer
Clients
78
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Thread Cache
Thread Cache
Query Cache
SQL Parser
Optimizer
Storage Engines
Thread is Assigned to Each Connection
Threads may be Reused from the Thread Cache or Created
79
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Connection Pool
Thread Cache
Query Cache
SQL Parser
Optimizer
Storage Engines
Main Thread Listens for Client Connections User Authentication based on Host, User, and
Password Client Buffers for Session Variables and Network
Communications
80
thread_handling = pool-of-threads thread_pool_size = 3
Excerpt from /etc/my.cnf.d/server.cnf
Documentation on Thread Pool: https://mariadb.com/kb/en/mariadb/thread-pool-in-mariadb/#using-thread-pool-scheduler
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Query Cache
Thread Cache
Query Cache
SQL Parser
Optimizer
Storage Engines
Documentation on Query Cache: https://mariadb.com/kb/en/mariadb/query-cache/
SHOW VARIABLES LIKE 'query_cache_type';
Possible to Stipulate in a Query
Stores SELECT Query Result Sets
Useful for High Read, Low Write Servers
Cache Purged when Related Data Changed
SELECT SQL_CACHE * FROM ...
SELECT SQL_NO_CACHE * FROM ...
81
No Longer Recommended
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
SQL Parser
Thread Cache
Query Cache
SQL Parser
Optimizer
Storage Engines
Converts SQL Text to Binary Format Parses SQL into Tokens (i.e., keyword, table, field, value) Applies Grammar Rules to check Validity (Lexical
Scanner, Grammar Rules Module) Construct a Parse Tree to be passed to Optimizer
82
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Optimizer
Thread Cache
Query Cache
SQL Parser
Optimizer
Storage Engines
The Optimizer Reads the Parse Tree and Determines an Execution Plan
Locate any Related Indexes Compare Efficiency of Index Access to Table Scan Determine JOIN order of Table
Eliminate Unnecessary Tables and WHERE clauses
Find Indexes that can handle GROUP BY and ORDER BY
83
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Information Schema (information_schema)
Generated as Needed No On-Disk Presence Read-Only Tables
Some Accessible with SHOW statement
CHARACTER_SETSCOLLATION_CHARACTER_SET_APPLICABILITYCOLLATIONSCOLUMN_PRIVILEGESCOLUMNSENGINESEVENTSFILESGLOBAL_STATUS
GLOBAL_VARIABLESKEY_COLUMN_USAGEPARTITIONSPLUGINSPROCESSLIST SCHEMATAPROFILINGREFERENTIAL_CONSTRAINTSROUTINESSCHEMA_PRIVILEGES
SESSION_STATUSSESSION_VARIABLESSTATISTICSTABLE_CONSTRAINTSTABLE_PRIVILEGESTABLESTRIGGERSUSER_PRIVILEGESVIEWS
84
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Methods of Accessing Information Schema
SHOW STATUS LIKE '%qcache%';
Using SHOW Statement
SELECT * FROM information_schema.global_status
WHERE VARIABLE_NAME LIKE '%qcache%';
Accessing Directly
85
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Information Schema — InnoDB Tables
SHOW TABLES LIKE 'INNODB%'; SHOW TABLES LIKE 'XTRADB%';
SELECT r.trx_id AS waiting_trx_id, r.trx_mysql_thread_id AS waiting_thread,
r.trx_query AS waiting_query, b.trx_id AS blocking_trx_id, b.trx_mysql_thread_id AS blocking_thread, b.trx_query AS blocking_query FROM information_schema.innodb_lock_waits w
INNER JOIN information_schema.innodb_trx b ON b.trx_id = w.blocking_trx_id INNER JOIN information_schema.innodb_trx AS r ON r.trx_id = w.requesting_trx_id;
Locked & Locking QueriesINNODB_LOCKS
INNODB_TRX
INNODB_BUFFER_POOL_STATS
INNODB_TABLE_STATS
INNODB_INDEX_STATS
86
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
Server Configuration
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Global Variables
GLOBAL Variables are System Wide Settings
Drawn from Configuration File at Start
Some can be Changed Dynamically by a SUPER user with SET
88
SET GLOBAL tmp_table_size = 32*1024*1024;
SHOW GLOBAL VARIABLES;
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Session VariablesSESSION Variables are Client Specific
Settings
Default Values Same as Global Variables, but may be Changed for Client
89
SET SESSION tmp_table_size = 64*1024*1024;
SHOW GLOBAL VARIABLES;
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Memory FootprintCheck if Server is Dedicated to MariaDB Check How much Memory does OS and Other Services
Need MariaDB Allocates Global and per Session Buffers Global Buffers are Once-Only Allocations at Start (MyISAM
Key Cache, InnoDB Buffer Pool, etc.)
Session Buffers are Allocated Multiple Times per Client Connection, Depending on Queries
MariaDB's Footprint needs to have Space, Depending on how many Concurrent Client Connections Expected
90
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Global Memory — Buffers Allocated once at mysqld Start
91
io_read io_write
Global Memory Buffers are Allocated once when mysqld Starts table_open_cache table_definition_cache query_cache_size thread_cache
Permissions Tables Global Memory Buffers are also Allocated for Storage Engines
MyISAM: key_buffer_size InnoDB: innodb_buffer_pool_size, innodb_additional_mem_pool_size, innodb_log_buffer_size
Storage Engines may Allocate internally Other Memory
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Session Memory
92
Buffers Allocated as required for each Client Connection Some Allocated more than once for Joins and Sorting Released when Query is Finished or Client Session Closed Persistent Connections need to be Reset Client Session Memory Variables:
thread_stack read_buffer_size join_buffer_size bulk_insert_buffer_size
binlog_cache_size net_buffer_length
read_rnd_buffer_size max_heap_table_size max_allowed_packet sort_buffer_size
tmp_table_size myisam_sort_buffer_size
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Estimating Memory Usage
93
(sum(session_buffers) * max_connections)
+ sum(global_buffers) = total memory usage
Worst Case, Maximum Memory Usage Scenario:
Chances of All Clients Allocating All Possible Session Buffers at once is usually Small
Normal Practice is to Commit Over Memory Requirements
Out-of-Memory (OOM) may occur if mysqld is near System Memory Limits
Add Memory or Use a Crash-Safe Storage Engine
Use SWAP space, but Avoid letting MySQL Swap for Normal Load
Basic Formula for Estimating Memory Usage
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
File HandlesLogs and Replication Files need to be Open Client Connections each need a File (socket = file_handle)
Storage Engines may have Specific Needs: MyISAM uses a file per table.frm, and multiple for
table.MYI and table.MYD InnoDB needs only a few for Global Table-Space and
Logs (unless using innodb_file_per_table)
Implicit Disk-Based Temporary Tables are Files SELECT INTO OUTFILE uses a file
94
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Temporary Disk SpaceMariaDB Uses Temporary Disk Space
Large Implicit Temporary Tables for Resolving Queries Some Sort Operations (ORDER BY, GROUP BY)
tmpdir defaults to system temporary location Some Unix platforms mount /tmp as tmpfs
slave_load_tmpdir for Replicating LOAD DATA INFILE Defaults to Location of tmpdir — Never use tmpfs Slave Exports from binlog to here, then LOAD DATA
INFILE
95
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Disk SpeedUnless a Data Set fits Memory, Disk Speed is Important Disk Reads are Alleviated by Caching Some Storage Engines Implement their own Caches Some Rely on OS Disk Caching Mechanism Disk Writes are Often a Bottleneck Battery Backed Writes are useful for Fast fsync( ) Consider Disk Access Patterns — Multiple Disks to Separate
Random & Sequential Disk IO Random: Most Data Access — Can be Query Dependent Sequential: Transaction and Replication Logs
96
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Group CommitInnoDB Flushes the Redo Log for a Transaction before it’s Committed Group Commit makes one Write to Logs for Multiple Transactions, instead of Individual Flushes Group Commit as of MariaDB 5.3
97
Graphs taken from Facebook Article show MySQL version 5.1.52Benchmark Articles on Group Commit with MariaDB: https://www.facebook.com/note.php?note_id=10150211546215933; https://www.facebook.com/note.php?note_id=10150261692455933 Documentation on Group Commit: https://mariadb.com/kb/en/mariadb/group-commit-for-the-binary-log/
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Server CPUSince mysqld is multi-threaded, it can
use many CPU cores A Client Connection is a single thread —
a Query uses only One Core
Query Concurrency is very Schema and Traffic Dependent
98
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
MyISAM
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Advantages & Disadvantages of MyISAM
Indexes are Cached by mysqld in key_buffer Don't Disable — System Tables use MyISAM Storage
Engine (InnoDB in Future)
100
Not Transactional
Table-Level Locking
Disadvantages
Poor Crash Recovery
(Relies on OS to Flush Data to
Disk)
Data Caching Relies on OS Disk Cache
High Read, Low Write
Traffic
Concurrent Writes
Advantages
Many Column Data Types
Documentation for InnoDB & XtraDB: https://mariadb.com/kb/en/mariadb/xtradb-and-innodb/
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Architecture
101
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Architecture
102
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Key BufferOne Key Buffer by Default (key_buffer_size)
For MyISAM Only, Set Key Buffer to 25 - 50% of Memory (max 4GB)
Don't set key_buffer_size to 0 - Used for Internal Table Operations
Consider Key Buffer Hit and Miss Rates Ideal Values are Application Specific Better to have More Reads than Writes
Run Multiple Key Buffers Allows More than 4GB Dedicating a Key Buffer to a Single Table Prevents Index
Flushing
103
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Delayed Key WritesMyISAM Supports Delayed Key Writes Data is Written Immediately to Disk Index Writes are Postponed (delay_key_writes)
ON needs DELAY_KEY_WRITE in Table Definition ALTER TABLE...DELAY_KEY_WRITE Rebuilds Whole Table
After Crash, Index Files must be Rebuilt myisam_recover_options = DEFAULT, BACKUP, FORCE,
QUICK
104
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
InnoDB & MyISAM Row Formats
105
Fixed-Length Data Types Less Data File Fragmentation Heaviest Disk Space Usage
Static
Fixed and Variable Length Data Types More Data File Fragmentation More Efficient Disk Space Usage
Dynamic
Decompressed as Needed Minimal Disk Space Usage Read-Only
Compressed
INT, CHAR, etc.
VARCHAR, TEXT, etc.
105
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Concurrent InsertsMyISAM leaves Gaps in Data File when Deletes Rows
No Automatic Defragmentation
MyISAM tries to fill Gap with Data from INSERT MyISAM has no MVCC Need to Lock the Table to Avoid tripping a SELECT
Concurrent Inserts allows INSERT to add Data Concurrently with SELECT
Not Equivalent of InnoDB's MVCC, but can Improve Concurrency
106
concurrent_inserts = 0 Concurrent Inserts Disabled
concurrent_inserts = 1 Allow Concurrent Inserts when a Table has no gaps (Default)
concurrent_inserts = 2 Allow even When a Table has Gaps
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
TrapsMyISAM always Reads Whole Row from Disk, Not Partial
Row Large, Infrequently Needed Fields may be Better
Elsewhere in one-to-one Relationship MyISAM used for Implicit Temporary Tables on Disk
Never Reduce MyISAM Resources to Zero (key_buffer_size, etc.)
Ensure tmpdir Location has Plenty of Space - Cartesian Products can be Large
Check Settings of myisam_sort_buffer_size and myisam_max_sort_file_size
MyISAM Repairs use Key Buffer when too Small
107
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
InnoDB
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Advantages & Disadvantages of InnoDB
109
Read Uncommitted Read Committed Repeatable Read (no fantom rows) Serializable
Fully Transactional,
ACID Compliant
Data & Indexes Cached by mysqld
in Buffer Pool
Row-Level Locking for High-
Concurrency
Supports Four Isolation Levels
Supports Foreign Keys and Multi-
Version Concurrency
Control
Reliable Crash Recovery
Slower than MyISAM, but
Higher Concurrency
Uses a Global Table-Space File
and two Redo Log Files
Advantages Disadvantages
Isolation Levels
Documentation for InnoDB & XtraDB: https://mariadb.com/kb/en/mariadb/xtradb-and-innodb/
109
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
InnoDB Architecture
110
Additional Memory Pool
Buffer Pool *
Data Files * **
Redo Log (File 1/2)
Redo Log (File 2/2)
Memory
Disk
* Cached Data & Indexes, Stored in Pages ** Transaction Data Writes *** Data, Index, and Undo Log Files
INSERT, UPDATE, DELETE
1
COMMIT
CHECKPOINT
Log Buffer * *
2
3
110
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Buffer PoolFixed Size Pool of Memory Pool by InnoDB
Row Data and Index Pages Adaptive Hash Index Insert Buffer Lock Structures Misc. Other Housekeeping
111
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Buffer PoolLarger is Generally Better for innodb_buffer_pool_size
For Servers with Only InnoDB Tables, 80% of Available Memory
MyISAM is Used Internally and Needs Resources along with OS
Dirty Buffer Pool Pages are Flushed in Background Allows Changes to be Merged and Written Sequentially to Disk Applications with Plenty of Writes, or Write Spikes, may need
Adjusting Use innodb_max_dirty_pages_pct to Control Frequency of
Flushing Dirty Pages
InnoDB Plugin can Disable the Adaptive Hash Index Mostly Improves Performance, but Not Always
112
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
IndexesInnoDB Indexes are Clustered
Row Data is Stored with Primary Key
Use Short Surrogate Primary Keys (INT UNSIGNED AUTO_INCREMENT)
Secondary Index Entries hold a Copy of PK — Large PK Increases All
Table Index Statistics are Vague for InnoDB Gathered by a Series of Random Index Dives It's Possible for Stats to be Inaccurate and Cause
Optimizer to Choose Bad Query Plan Run Periodically ANALYZE TABLE to Regenerate Stats
113
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
InnoDB Clustered Index
114
root
node node
leafleaf leaf
PAGE (16KB) PAGE (16KB) PAGE (16KB)
B-Tree
HEADER PRIMARY KEY ROW DATA
leaf
114
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
InnoDB Secondary Index
115
root
node node
leafleaf leaf
PAGE (16KB) PAGE (16KB) PAGE (16KB)
B-Tree
HEADER VALUES OF INDEXED COLUMNS PRIMARY KEY
leaf
115
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Transaction LogConverts Transactions from Random I/O to Sequential
Transaction Data is Written to Transaction Log for Durability and Speed - Not Directly to Data Files
Log Files Rotate as a Circular Buffer with Data Merged to Disk in Background
116
fsync() Frequency
Variable Setting Write and Flush Action
innodb_flush_log_at_trx_commit = 0 Flush on Checkpoint
innodb_flush_log_at_trx_commit = 1 Flush at Commit (Default)
innodb_flush_log_at_trx_commit = 2 Write at Commit, but No Flush
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Transaction LogLarge innodb_log_file_size (Default 5MB) Results
Usually in Less I/O and Better Performance Crash Recovery Delay Increases with Log File Size
Increasing innodb_log_buffer_size (Default 1MB) May Reduce Log File I/O for Large Transactions
117
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Isolation LevelsInnoDB Supports All Four Transaction Isolation Levels*
READ UNCOMMITTED READ COMMITTED REPEATABLE READ SERIALIZABLE
* Listed in Order of Increasing Performance Potential
Increasing Isolation Protection means More Locks Needed
Reduces Query Concurrency Increases Memory Usage
Set Isolation Level on Per-Session Basis for Performance for Transactions Requiring More Protection
Leave Bulk of Traffic to Operate at Lower Isolation Level
118
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
The InnoDB CoreVariable innodb_thread_concurrency Sets Number of
Threads that may Access simultaneously InnoDB Core
Set Based on Application and Traffic
Start about Double number of CPU Cores Increase Slowly to Watch for System Spikes or General
Thread Thrashing, Reduced Through-Put
119
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
The InnoDB Core
Variable innodb_flush_method Controls How Data is Written to Disk
Default is a Normal fsync( ) Operation
Leads to "Double Buffering" of Data (Buffer Pool and OS Disk Cache)
O_Direct Allows InnoDB to Use Direct I/O
Bypasses any OS Cache Changing Flush Method should be Tested Thoroughly Some Combinations of Device, Filesystem, OS may
Not Benefit
120
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
TablespacesInnoDB Stores Tables in Tablespace Files
Single Global Table Space is Used by Default Use innodb_file_per_table to Use Separate
Tablespaces Easier Maintenance More File Handles Required — Two Additional for Each
Table
121
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
InnoDB Plugin
New Pluggable version of Original Built-In InnoDB
Better Diagnostics Benchmarking Shows Faster & Higher
Throughput Optional Compressed File Format
Configurable IO Threads on All Platforms
MariaDB 5.5 Uses InnoDB Plugin by Default
122
CREATE TABLE… ROW_FORMAT=COMPRESSED;
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
InnoDB General TipsDo Bulk Data Loads
Do in PK Order (Sequential Ascending) Batch to Allow Background Threads Keep Pace
Avoid Large ROLLBACK Operations No Equivalent of Log and Insert Buffers Exist to Reduce
Rollback Disk I/O
RAID with Battery Back-Up Unit is Preferable Durability Benefit and some Drivers Allow Disk Sync
Operations Return after Writing Only to BBU Cache
Use when possible Newer InnoDB Plugin or XtraDB Many former Bottlenecks have been Fixed Supports Compression
123
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
MariaDB Replication
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Purpose of MariaDB Replication
125
Load Balancing (Scaling SELECT Queries) Move Slow, Heavy Queries to Slave Take Slave Off-Line to Make Back-ups
Multiple Data Centers Need Fast Reads Gain Redundancy (High Availability)
Fail Over - Promote Quickly a Slave to Master Fail Over Isn’t Automatic — Requires External Monitoring
Minimal Downtime for Upgrades or Schema Changes Apply Changes to a Slave Promote Slave to Master and Redirect Traffic Apply Changes to Master and Switch
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Replication Terrain
126
mysqld Data Storage
Master
Binary Log
mysqld Data Storage
Slave 1
Relay Log
Data Storage
Slave 2/Master 2
Relay Log
Binary Log
mysqldSlave 2A
Slave 2B
Slave 2C
INSERT UPDATE DELETE
CREATE ALTER DROP
Dump Thread
Client Threads
SQL Thread
IO Thread
IO Thread
IO Thread
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
MariaDB Replication Factors
127
One Master, Multiple Slaves No true Multi-Master Solution, but Circular Replication
Close to Real Time, but Asynchronous Semi-Synchronous Replication Mode Crash-Safe Slaves with Transactional Storage Engines
A Slave may also be a Master Set log_slave_updates in Configuration File Apply Optionally Replication Filtering Rules or Storage
Engine Changes on Intermediate Slaves
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Replication — Threads
128
Master binlog Dump Thread Pushes binlog Events to Slave Visible in SHOW PROCESSLIST as "Binlog Dump"
Slave IO Thread — Visible in SHOW SLAVE STATUS
Requests and receives binlog events from the Master Writes them to the local relay log
Slave SQL Thread — Visible in SHOW SLAVE STATUS
Reads the Relay Log and Executes Queries on Local Data Checks the Query Result Codes Match those Recorded by Master
Slave Multiple Execution Threads Multi-Threaded Slave separates events based on Database Names Updates are Applied in Parallel, Not Sequence
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Parallel Replication
129
Documentation on Parallel Replication: https://mariadb.com/kb/en/mariadb/documentation/replication/standard-replication/parallel-replication/
Replication Process on Slaves Events Received from Master by IO Thread and Queued
in Relay Log Each Relay Log Entry is Retrieved by the SQL Thread Each Transaction is Applied to the Slave
On Non-Parallel Systems, Application Performed Sequentially by SQL Thread
On Parallel Systems, Application Performed in Pool of Separate Replication Worker Threads
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Topologies
130
Master to Slave Simplest Solution and Used Most Widely Allows Off-Loading of SELECT Traffic to Slave
Master1 to MasterN ... to Master1 (circular) Servers Replicate in a Circle, with binlog Events
Traversing the ring until Originating Server Does Not Alleviate Heavy Write Load Needs Careful setup of server-id and
auto_increment_offset, auto_increment_increment settings
Master to Slave to Slaves Can build Complex Trees Useful for Replication Rules or Storage Engine Changes
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Master Configuration
131
GRANT REPLICATION SLAVE ON *.* TO 'maria_replicator'@'52.34.19.24' IDENTIFIED BY 'rover123';
Enable Binary Log — Choose Binary Log Format
Set server-id in Configuration File to Unique Value
Create Replication User Account on Master
Make a Consistent Snapshot of Data on Master
mysqldump -p -u admin_backup --master-data --flush-logs \ --all-databases > full-dump.sql
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Slave Configuration
132
Documentation on Slave Options: https://mariadb.com/kb/en/mariadb/replication-and-binary-log-server-system-variables/ Documentation on CHANGE MASTER TO: https://mariadb.com/kb/en/mariadb/change-master-to/
CHANGE MASTER TO
MASTER_HOST='35.161.145.71', MASTER_PORT=3306, MASTER_USER='maria_replicator', MASTER_PASSWORD='rover123';
Set server-id in Configuration File to Unique Value
Add read-only in Configuration File to Prevent Writes Set Optionally Replication Rules — Covered Later in Class
Restart MariaDB
Load Data from Master
Execute START SLAVE on Slave
mysql -p -u root < full-dump.sql
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Monitoring Replication
133
Documentation on SHOW MASTER STATUS: https://mariadb.com/kb/en/show-master-status/ Documentation on SHOW SLAVE STATUS: https://mariadb.com/kb/en/mariadb/show-slave-status/
SHOW MASTER STATUS;Check Regularly Status on Master — Includes binlog number and position
Check More Often Status of Replication on Slave
SHOW SLAVE STATUS \G
Slave_IO_State: Waiting for master to send event Slave_IO_Running: Yes
Slave_SQL_Running: Yes Last_Errno: 0 Last_Error: Seconds_Behind_Master: 300
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Replication Files
134
Documentation mysqlbinlog: https://mariadb.com/kb/en/mariadb/using-mysqlbinlog/
Binary Log Files (Master)
Master Records Write-Queries to File Rotated when Flushed or Periodically to New Log File — File
Name Pattern (.000001)
Relay Log File (Slave)
Record of Master binlog Events Rotated when Flushed or Periodically — File Name Pattern (.
000001)
Replication Configuration Recorded in master.info (Slave)
Name of Relay Log File Recorded in relay-log.info (Slave)
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Slave Configuration Files
135
52.89.128.176 maria_replicator rover123 3306 60
0 ...
master.info
196803 mariadb-bin.000040
196513 30217 9 ...
relay-log.info
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Replication File Maintenance & Back-Ups
136
Replication Files Updated & Purged Automatically — Don’t Edit or Move Manually
Include Replication Files when Making Binary Back-ups
Use --raw option with mysqlbinlog to Back-up Binary Log
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Binary Log FormatStatement Based (SBR) — Original Queries are Replicated
Least Data Sent over Wire and Tested for Years Non-Deterministic Statements Executed on Slave — Slave Load is
Increased vs. RBR
Row Based (RBR) - Table Rows are Replicated
Only Non-Deterministic Statements Executed on Master — Slave Load is Reduced vs. SBR
More Data sent over Wire — Not Supported by All Engines
Mixed (default) - Smart Switching between SBR and RBR Checksum (--binlog-checksum) in Binary and Relay Logs to detect Errors
Includes Errors in Memory, Disk, Network and Database Can be Implemented for each Slave
137
Documentation on Binary Log Format: https://mariadb.com/kb/en/mariadb/binary-log-formats/
[mysqld] binlog-format=MIXED
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Slave Filtering Rules — Database Level
138
Documentation on Slave Options: https://mariadb.com/kb/en/mariadb/replication-and-binary-log-server-system-variables/
Exclude Specific Databases (e.g., mysql)
CHANGE REPLICATION FILTER REPLICATE_IGNORE_DB = (), REPLICATE_DO_DB = (sales,inventory);
CHANGE REPLICATION FILTER REPLICATE_IGNORE_DB = (mysql);
Include Specific Databases
Excluding can Cause Problems with Joins
SET GLOBAL replicate_ignore_db = ''; SET GLOBAL replicate_do_db = 'sales,inventory';
SET GLOBAL replicate_ignore_db = 'mysql';
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Slave Filtering Rules — Table Level
139
Documentation on Slave Options: https://mariadb.com/kb/en/mariadb/replication-and-binary-log-server-system-variables/
Ignore Specific Tables
CHANGE REPLICATION FILTER REPLICATE_IGNORE_DB = (employees), REPLICATE_DO_TABLE = (employees.names, employees.contacts);
CHANGE REPLICATION FILTER REPLICATE_IGNORE_TABLE = (employees.salary);
CHANGE REPLICATION FILTER
REPLICATE_IGNORE_DB = (sales), REPLICATE_DO_TABLE = (sales.europe_%), REPLICATE_IGNORE_TABLE = (sales.europe_uk_%);
Include Specific Tables
Wildcards for Multiple Tables
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
MariaDB Replication - Asynchronous
140
Master Doesn’t Wait for Slaves IO Thread may be Slow to Receive binlog Packets
Network Congestion or Disconnects
SQL Thread may be Slow in Processing Relay Log Events
Load on Slave or Network Problems
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Semi-Synchronous
141
Documentation on Semi-Synchronous Replication: https://mariadb.com/kb/en/mariadb/semisynchronous-replication/
Implemented with an optional Plug-In A COMMIT on Master can Wait for a Slave to
Acknowledge it’s Received Transaction Master Waits for Slave to Write Transaction
to Relay Log, Not to Execute Transaction — Slave SQL Thread may still Lag
One Slave Response needed for Master to Continue (Semi-Synchronous, Not Synchronous)
Can Affect Significantly Performance of Master
INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so';
INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so';
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Lagging Slave
142
Documentation on Time-Delayed Replication: http://dev.mysql.com/doc/refman/5.6/en/replication-delayed.html
When Slave SQL Thread is Slow or Disabled due to Errors, Slave is said to Lag behind Master
Slave SQL Thread must Execute Serially Queries that were Executed in Parallel on the Master
Slave Multiple Execution Threads Multi-Threaded Slave separates events based on
Database Names Updates are Applied in Parallel, Not Sequence
Time-Delayed Replication - Set Provides a Buffer to Stop Replication of Mistakes
CHANGE MASTER TO MASTER_DELAY = seconds;
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Troubleshooting Problems with Replication
143
Check Slave Error Log for Errors affecting Replication Look for Disconnects from Network Problems Binary or Relay Log Event Corruption will cause Slave
SQL Thread to Stop Different Query Error Codes on Slave indicate it’s Not
Synchronized with Master
Tools like Maatkit can help with Replication Troubleshooting and Recovery
May Need to Rebuild Slave from a fresh Snapshot (Back-up of Master or Another Slave)
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
Maintenance Impact
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
OverviewMaintenance Tasks can Impact Heavily
a System Backups, Table and Index Optimization,
Schema DDL, etc. Some can Alleviate such Impact
145
Type of Operation PerformedHot Operation Warm Operation Cold Operation
No Locks Required Possible High Disk/Network I/O Runs usually Concurrently with Normal Traffic
System must be in Read-Only Mode Impact Depends on Application and Timing Related to Traffic
System must be Stopped Completely Results in Complete Down Time for Application
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Back-UpsBinary Back-Up is Usually Faster than Logical Back-Up
Faster Often for Creation and Restoration May be Transparent to Application, Depending on
Method Often a Warm Operation (Read Lock Only) — Consider Storage
Engine
Incremental Back-Ups can be very Low Impact Synchronized with Binary Log Full Back-Up Daily and Periodic binlog Snapshots
Require Only FLUSH LOGS Slower to Restore than Full Back-Up
146
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Table & Index OptimizationOPTIMIZE is Warm Operation (Read Lock)
Perform during Low Traffic or Scheduled Maintenance Time
OPTIMIZE LOCAL is Non-Replicated Version Can be Applied to Slaves one-by-one so Overall
Cluster Impact is Minimal and Predictable
ANALYZE is Warm operation (Read Lock)
Impact is Storage Engine Specific MyISAM is Equivalent to Full Table Scan InnoDB is Fast Series of Random Index Dives
147
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Schema AlterationsMost DDL Commands are Warm Operations DDL Involves Generally a Table Rebuild
Table is Read Locked DDL is Performed on Copy of Table Two Tables are Swapped Lock is Released
pt-online-schema-change Might Help
DDL Operations can be a Long Process Best Performed in Scheduled Maintenance Time Test Carefully on Staging System Changes can be Made on Slave First, then Promoted to Master
148
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
Conclusion
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
On-Line ResourcesDocumentation Knowledge Base Open-Source Community
Post Questions on Forums Ask Questions on IRC — FreeNode
150
MariaDB Documentation and Knowledge Base: http://mariadb.com/kb
Version 3.2, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB Performance Tuning
Other Training CoursesPublic In-Person Courses Private In-Person Courses Live Virtual Training Courses Self-Paced On-Line Courses
151
Training Courses: https://mariadb.learnupon.com/store Upcoming Courses: https://mariadb.learnupon.com/store/sessions
MariaDB Performance Tuning
Introduction
Tuning Overview
Collecting Information
Schema Tuning
Query Tuning
Common BottlenecksArchitecture
Server Configuration
MyISAM
InnoDB
MariaDB Replication
Maintenance Impact
Thanks for Participating