Practical SQL query monitoring and optimization

Practical SQL Query Monitoring

and Optimization Identify Data Layer Bottlenecks

About me

� Project Manager @

� 10 years professional experience

� Microsoft Certified Specialist

� [email protected]

� http://www.linkedin.com/in/ivelin

� Business Interests

� ASP.NET, AJAX, jQuery

� SOA, Integration

� GIS, Mapping

� SQL optimization

2 |

A Well Known Story

Monitor and Measure Application

Analyze Tier Interaction

Getting Performance Hints

Identify Problematic Queries

Optimization Query Performance

Measure the Effect

Maintain

3 |

Warning: A Bottleneck

� Visual Studio Performance Wizard� CPU and Memory Sampling

� Concurrency

� Tier Interaction

� SQL Server Profiler� Create trace

� Analyze trace results

� Replay trace

� Log analyzer� Most frequent and timely queries

� Identify long-running locks (and deadlocks)

� Identify I/O activity

� SQL Solutions Trace Analyzer / Deadlock Detector

Row vs. Chunk at a Time

� Row at a time (RAT) processing� SELECT * FROM Activities WHERE ParentId = 1

� SELECT * FROM Activities WHERE ParentId = 2

� SELECT * FROM Activities WHERE ParentId = 16

� Chunk at a time (CAT) processing� SELECT * FROM Activities WHERE ParentId IN (1,2,16)

� Likely to happen when:

� Components with own DataSource extraction

� Complex code by different developers

DEMO 1

� Row vs. Chunk at a Time

� Analyzing Trace Logs

Getting Help

� Analyze Execution Plan

� Database Tuning Advisor

� Suggests indexes and statistics

� Opt.1: Improvement > 20%

� Opt.2: Improvement < 20%

� Dynamic Management Views

� Missing Indexes (sys.dm_db_missing_index_groups)

� Slowest Queries (sys.dm_exec_query_stats)

DEMO 2

� Tuning Advisor

� Dynamic Management Views

Analyze Query Plan

� Tools

� SQL Server Management Studio

� SQL Sentry: free @ http://www.sqlsentry.com

� Query Plan� Colour-scaled cost information

� Cost filter by CPU, I/O or both

� Better plan layout

� Top Operations

� Tuning Tips

Query Plan Basics

� Table Scan (Heap)� Clustered index missing

� Scans data pages (all data)

� Index Scan� Scans index pages (less data)

� Relies on order of pages

� Index Seek� Highly selective queries (10-15% of rows)

� Indexed Views� Require: Schema binding

� Require: Deterministic functions

DEMO 3

� Query Plan Explorer

� OR vs. UNION

Indexes

� Index types

� Clustered / Non-clustered

� Filtered index(SQL 2008)

� Covering index

� Objective

� Optimal performance

� Reduce slow scans

� Difference in large data sets

Are Indexes Magical ?

� Execute non-indexable queries� SELECT * FROM Employees WHERE CAST(PNumber AS INT) > 1

� SELECT * FROM Customers WHERE ContactName LIKE ‘%a%’

� SELECT * FROM Employees WHERE FName=‘a’ OR LName=‘b’

� Reject indexes because of their overhead

� Choosing Indexes� Measure query impact

� Measure index size

� Maintenance� Fragmentation = poor disk I/O

� Reorganize vs. Rebuild

� Automate: Nightly Job

� Anti-patterns� Create indexes blindly

SQL Server Statistics

� Objective� WHERE clause order

� Statistics on data distribution

� Query Optimizer creates efficient plan

� Up-to-date check� Automatically created for index keys

� Estimated vs. Actual Rows

�Maintenance� DB Default: AUTOCREATE and AUTOUPDATE

� EXEC sp_updatestats

� Automate: Nightly Job

User Defined Functions

� Procedures vs. Functions

� Performance vs. Reusability

� Complex conditions

� Used through the whole application

� i.e. IsCaseClosed

� Chunk at a Time

� Table valued parameters (SQL 2008 and on)

� Pass result from one function to next

Optimize Locks

�Data Integrity

� Data commit waits

�NOLOCK

� Prevent Deadlocks

� Returns uncommitted data

� Incorrect results

� Less/More rows could be returned

� Update one entity, retrieve another

� Index page splits under heavy load(GUID)

DEMO 4

� The NOLOCK Problem

Common Table Expressions (CTE)

� Alias of a query (SQL 2005)

� Advantages

� Readability and reusability

� Implement recursive query

� Materializes when used

� Uses indexes (Table v@r not)

� Disadvantages� Can be used only once

� Must be used immediately

Other

� UNION vs. UNION ALL

� Data Types� Do you really need BIGINT?

� Requires storage

� Requires time to read

� Requires memory

BIGINT INT Delta (MB) Delta (%)

Primary Key 1267 977 290 23%

By_PropID_CountryID 813 741 72 9%

By_String 303 230 73 24%

SQL Resource Governor

� Shared DB Server

� Prioritize queries� Reporting vs. Dashboard

� Memory, CPU

� Connections, Parallelism

� RG Configuration� Pools

� Workload Groups

� Custom classification function

� Available in SQL Server Enterprise

Test Under Load

� Life’s complicated

� Concurrent queries

� Multiple user

� Cached Buffers

� Tools

� SQL Server Profiler Replay

� SQL Server 2012 Distributed Replay� Snap In @ http://dreplaygui.codeplex.com/

� SQL Load Generator� http://sqlloadgenerator.codeplex.com/

Summary

� Steps

� Capture/Analysis/Optimization

� Measure the improvement

� Maintain

� Remember

� Reoccurring improvement

� Ideally you should analyze all SQL

� Changes can affect SQL performance� in data size

� result set size

DEMO 5

� Trace Replay

� SQL Load Generator

� Resource Governor

Other Approaches

� Caching� Application (Static)

� Session

� Context

� WCF Session

� Entity Framework Caching

� Paging

� Lazy loading

� Parallel binding

� Historical / Live data tables

SQL Saturday #152 Sponsors