Dec 27, 2015
MySQL Performance SchemaNew enhancements in MySQL 5.7
Mayank PrasadPrincipal Member Technical StaffOracle, MySQLOctober 1, 2014
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 3
Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 4
Program Agenda
Need and Origin
The Journey : Then and now …
New in MySQL-5.7 (DMRs)
Configuration
An example
Q&A
1
2
3
4
5
6
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 5
Program Agenda
Need and Origin
The Journey : Then and now …
New in MySQL-5.7 (DMRs)
Configuration
An example
Q&A
1
2
3
4
5
6
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 6
Why Performance Schema?
SystemLow throughput?
?
DBAHot table?
NetworkHigh traffic on link?
StorageToo much disk spin?App Developer
Slow application?
MySQL Developer Code contention?
End User Stuck session?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 7
History and Origin
“The Performance Schema is a mechanism to give user an insight of what is happening behind the scenes when MySQL server is running.”
Development was started by Marc Alff in 2008. He is Chief Architect and Lead of Performance Schema Team (Christopher Powers and me) at Oracle-MySQL.
Introduced in MySQL 5.5.• New storage engine : Performance Schema• New Database : performance_schema• Statistics stored in tables (hard-coded DDLs). • Non-persistent data.• SQL user interface.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 8
MySQL 5.7 Performance Schema : Design
MySQLServer
Instrumentation points(P_S hooks)
P_S Internal Buffers
P_SStorageEngine
StorageEngine
Interface StatisticsReport
FetchData
SQL Query
Statistics Collection Statistics Reporting
Collect DataStoreData
P_STables
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 9
Program Agenda
Need and Origin
The journey : Then and now …
New in MySQL-5.7 (DMRs)
Configuration
An example
Q&A
1
2
3
4
5
6
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 10
The Journey : Then and now..• MySQL 5.5
• Only waits events– Mutex waits– Conditions waits– R/W locks waits– File I/O waits
• Setup_(timers/consumers/instruments) tables
• Few summary tables
• 17 Tables and 218 Instruments
• MySQL 5.6
• Statements and stages
• Table, Index and network I/O. Table locks
• Host Cache
• User, account, host
• Setup_(actor/objects)
• Multiple summary tables
• Events’ hierarchy
• 52 Tables and 550 Instruments
• MySQL 5.7 (DMRs)
• Instrumentation:– Transactions– Metadata locks– Prepared statements– Stored programs– Memory usage
• User variables
• Replication summary tables
• 76 Tables and 885 Instruments
• …
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 11
Program Agenda
Need and Origin
The Journey : Then and now …
New in MySQL-5.7 (DMRs)
Configuration
An example
Q&A
1
2
4
3
5
6
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 12
Transaction Instrumentation
• Exposes all aspects of Transactions within server
• Implicit vs Explicit
• Quantitative (counts, timing) and Qualitative (isolation level) statistics
• Aggregated across several dimensions (user, account, session etc.)
• Statements to non-transactional tables are ignored (MyISAM)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 13
Transaction Instrumentation (cont.)Transactions stats at runtime
* Units in PICOSECONDs
EVENTS_TRANSACTIONS_SUMMARY_BY_THREAD_BY_EVENT_NAME
THREAD_ID 23
EVENT_NAME Transaction
COUNT_STAR 3
SUM_TIMER_WAIT 18558993066000*
… …
COUNT_READ_WRITE 3
SUM_TIMER_READ_WRITE 18558993066000*
… …
EVENTS_TRANSACTIONS_CURRENT
THREAD_ID 23
EVENT_ID 11
EVENT_NAME Transaction
STATE COMMITTED
TIMER_WAIT 22073180000*
ACCESS_MODE READ WRITE
ISOLATION LEVEL REPEATABLE READ
AUTOCOMMIT NO
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 14
Event Hierarchy
Session
Transaction
Statement*
Stage(closing tables, preparing, optimizing)
Wait(sync, lock, i/o)
* Transactional tables only.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 15
Event Hierarchy (cont.)An example
START TRANSACTION;INSERT INTO test.t1 VALUES(‘3’);COMMIT;
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 16
Event Hierarchy (cont.)An example
START TRANSACTION;INSERT INTO test.t1 VALUES(‘3’);COMMIT;
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 17
Metadata Locks InstrumentationMETADATA_LOCKS
OBJECT_TYPE: TABLE
OBJECT_SCHEMA: test
OBJECT_NAME: tab1
LOCK_TYPE: SHARED_READ
LOCK_DURATION: TRANSACTION
LOCK_STATUS: GRANTED
OWNER_THREAD_ID: 23
OWNER_EVENT_ID: 41
wait/lock/metadata/sql/mdl
• What are MDLs?• Exposes metadata lock
dependencies between session.• Which session holds MDL• Which session waiting MDL• Lock request status
– PENDING, GRANTED, KILLED, TIMEDOUT
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 18
Stored Programs Instrumentation
• New objects types in setup_objects.– PROCEDURE– FUNCTION– TRIGGER– EVENT
• Can be configured individually.• Nesting is maintained.
Triggers, Events, Functions, Procedures
OBJECT_TYPES
OBJECT_SCHEMA
OBJECT_NAME ENABLED TIMED
TABLE test tab1 YES NO
PROCEDURE test proc1 NO NO
TRIGGER db1 trg1 YES YES
FUNCTION % % NO NO
EVENT % % NO NO
SETUP_OBJECTS
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 19
Stored Programs Instrumentation (contd..)Example : create trigger test.trg before insert on test.tab1 for each row
insert into test.tab2 values ('2');
create procedure test.proc () BEGIN select count(*) from test.tab1; insert into test.tab1 values('123'); END
call test.proc();
EVENTS_STATEMENTS_SUMMARY_BY_PROGRAM
OBJECT_TYPE OBJECT_SCHEMA
OBJECT_NAME
COUNT_STAR
SUM_TIMER_WAIT
COUNT_STATEMENT
SUM_STATEMENTS_WAIT
SUM_ERRORS
SUM_ROWS_AFFECTED
PROCEDURE test proc 1 21233759000 2 21126786000 0 1
TRIGGER test trg 1 2001201000 1 1976505000 0 1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 20
Stored Programs Instrumentation (contd..)Example : (Nesting)
EVENTS_STATEMENTS_HISTORY
EVENT_ID 461
SQL_TEXT insert into d1.tab1 values('123')
OBJECT_TYPE PROCEDURE
OBJECT_NAME proc
NESTING_EVENT_ID 442
EVENTS_STATEMENTS_HISTORY
EVENT_ID 469
SQL_TEXT insert into d1.tab2 values ('2')
OBJECT_TYPE TRIGGER
OBJECT_NAME trg
NESTING_EVENT_ID 461
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 21
Prepared Statements Instrumentation
• What are Prepared Statements?
• Exposes Prepared Statements in use by server.
• Covers– PREPARE <stmt>– mysql_stmt_prepare()
SQLCOM_PREPARE, COM_STMT_PREPARE
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 22
Prepared Statements Instrumentation (contd.)Example prepare stmt1 from “select * from employee where emp_id=?”;
Set @emp_id=12345;execute stmt1 USING @emp_id;
EVENTS_STATEMENTS_HISTORY_LONG
THREAD_ID: 20
EVENT_ID: 4
EVENT_NAME: statement/sql/prepare_sql
SQL_TEXT: prepare stmt1 from "select * from employee where emp_id=?’;"
…
PREPARED_STATEMENTS_INSTANCES
STATEMENT_ID: 1
STATEMENT_NAME: stmt1
SQL_TEXT: select * from employee where emp_id=?
OWNER_THREAD_ID: 20
OWNER_EVENT_ID: 4
OWNER_OBJECT_TYPE: NULL
OWNER_OBJECT_NAME: NULL
COUNT_EXECUTE: 1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 23
User Variables
• Exposes user-defined variables • Max name limit 64
USER_VARIABLES_BY_THREAD
Set @var_1=12345;--------Set @var_2=67890;
USER_VARIABLES_BY_THREAD
THREAD_ID VARIABLE_NAME VARIABLE_VALUE
23 var_1 12345
24 var_2 67890
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 24
Memory Usage Instrumentation
• Aspects covered :– Memory size– Operation Count (1M bytes once vs. 1 byte 1M times)– Low/high watermarks (spikes)
• Aggregation– Type of memory (various cache, internal buffers)– Thread/account/user/host
• Memory behavior of code/choice of design
Who is taking what, how much and how many times.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 25
Memory Usages Instrumentation (contd..)MEMORY_SUMMARY_GLOBAL_BY_EVENT_NAME
EVENT_NAME memory/sql/TABLE
COUNT_ALLOC 1381
COUNT_FREE 924
SUM_NUMBER_OF_BYTES_ALLOC 2059873
SUM_NUMBER_OF_BYTES_FREE 1407432
CURRENT_COUNT_USED 457
HIGH_COUNT_USED 461
CURRENT_NUMBER_OF_BYTES_USED 652441
HIGH_NUMBER_OF_BYTES_USED 669269
• Say :• T1 : 1MB-2MB
• T2 : 10MB-12MB
• Together : 11MB - 14MB
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 26
Replication Summary TablesTables for replication statistics
• SHOW SLAVE STATUS (Limitations)– Lots of information packed together– No logical division of information– No cherry picking (difficult for automation)– Difficult to scale (more new fields)
• Why Performance Schema Tables?– SQL Interface. Fetch what is required– Split logically into different tables– Easier to extend
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 27
Replication Summary TablesTables for replication statistics
Replication status
Connection Information Execute Information
Connection Configuration Connection StatusExecute Configuration Execute Status
Coordinator/SQL Thread
Worker Thread
replication_connection_configuration
replication_connection_status
replication_execute_status_by_coordinator
replication_execute_status_by_worker
replication_execute_status
replication_execute_configuration
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 28
Replication Summary TablesTables for replication statistics
Table Information
replication_connection_configuration (Host, Port, User etc.)
replication_connection_status (Server UUID, Receiver thread ID, Service State etc.)
replication_execute_configuration (Desired Delay)
replication_execute_status (Remaining Delay)
replication_execute_status_by_coordinator (Thread Id, Service State, Last Error info.)
replication_execute_status_by_worker (WID, WTID, Service State, Last error Info.)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 29
Program Agenda
Need and Origin
The Journey : Then and now …
New in MySQL-5.7 (DMRs)
Configuration
An example
Q&A
1
2
3
5
4
6
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 30
Configuration• Build
• Startup– Configuration file
– Command line
[mysqld]performance_schema_consumers_event_waits_history = ON.
performance_schema_events_waits_history_size = 1000.
performance_schema_instruments=‘statement/sql/% = COUNTED’
--performance_schema_consumers_event_waits_history = ON.
--performance_schema_events_waits_history_size = 1000
cmake . -DWITH_PERFSCHEMA_STORAGE_ENGINE=1 -DDISABLE_PSI_MUTEX=1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 31
Configuration (contd..)• Runtime
• Auto tune (value=-1)– Use heuristics to compute
• Max number of objects (mutex, rwlocks, files, sockets, threads etc.)• Memory for statistical data (history, history_long, digest etc.)• NOT ALL VARIABLES ARE AUTO TUNED.
– Aligned with server tuning• Use hints from server variables
update setup_consumers set ENABLED=‘NO’ where NAME=‘global_instrumentation’;.
update setup_instruments set ENABLED=‘YES’ where NAME=‘statement/sql/%’;
max_connections
table_open_cache
table_definition_cache
open_files_limit
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 32
Program Agenda
Need and Origin
The Journey : Then and now …
New in MySQL-5.7 (DMRs)
Configuration
An example
Q&A
1
2
3
4
6
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 33
An Example
• Lets say thread, with thread id T0, is trying to alter a table tab1 and is hung.
• Now, what T0 is waiting for
– LOCK_STATUS=PENDING and OBJECT_NAME=‘tab1’
• Lets see who has taken MDL on table tab1.
– Thread T1 (column OWNER_THREAD_ID) is holding MDL.
• Is T1 waiting for some MDL?
SELECT * FROM metadata_locks WHERE THREAD_ID = T0;
SELECT * FROM metadata_locks WHERE OBJECT_NAME=‘tab1’, LOCK_STATUS=GRANTED;
SELECT * FROM metadata_locks WHERE OWNER_THREAD_ID = T1, LOCK_STATUS=PENDING;
Diagnose a stuck session
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 34
An Example (contd…)
• If T1 is not waiting for a MDL, then ?
• What T1 is waiting for
– Say T1 is waiting for mutex_A (column OBJECT_INSTANCE_BEGIN)
• Lets see who has taken this mutex_A
– Ok, so thread T2 is holding mutex_A (column LOCKED_BY_THREAD_ID)
• Find out what thread t2 is waiting for
SELECT * FROM events_waits_current WHERE THREAD_ID = T1;
SELECT * FROM mutex_instances WHERE OBJECT_INSTANCE_BEGIN = mutex_A;
SELECT * FROM events_waits_current WHERE THREAD_ID = T2;
Diagnose a stuck session
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 35
Program Agenda
Need and Origin
The Journey : Then and now …
New in MySQL-5.7 (DMRs)
Configuration
An example
Q&A
1
2
4
5
6
3