SQL Server 2005 Performance Enhancements for Large Queries Joe Chang [email protected] .

SQL Server 2005 Performance Enhancements for Large Queries

Joe [email protected]/joe_chang.asp

When & Why Migrate to 2005

Performance gains in large queriesIn memory

Disk performance

Data Warehouse ApplicationsEasiest to migrate

Usually internal users

Read-only queries

TPC-H benchmark

Decision SupportLarge volumes of data

Complex queries – 22 queries, 2 data refresh

Power & Throughput metricsPower – run 1 queries at a time

Throughput – run multiple concurrent streams

System Details

2003 Oct: SQL Server 2000 EE build 782

16 Itanium 2 1.5GHz/6M, 64GB, 215 disks

2004 Aug: Oracle 10g EE16 Itanium 2 1.5GHz/6M, 64GB, 166 disks

Unisys ES7000 Aries 420

2005 Jun: SQL 2005 HP Integrity rx8620

16 Itanium 2 1.6GHz/9M, 64GB, 342 disks

2005 Jul: Oracle 10g R2 - 236 disks

OS: Windows Server 2003 Datacenter, SP1 for last

TPC-H 1000GB Results

Power Throughput

Composite

SQL 2000 7,330.6 3,687.4 5,199.1

Oracle 10g 13,614.2 7,131.3 9,853.3

SQL 2005* 19,241.0 9,666.3 13,637.8

Oracle 10g R2*

20,385.3 11,140.4 15,069.9

S2K5 SP1* 23,346.3 13,458.6 17,725.9

*1.6GHz/9M processor, others: 1.5GHz/6M

10

100

1,000

10,000

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22 RF1 RF2

se

co

nd

s

SQL 2000 Oracle 10g

SQL 2005 Oracle 10g R2

TPC-H 1000GB - Power

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18Q19 Q20 Q21 Q22

SQL 2000 2530.8 60.2 873.2 807.4 1058.7 70.8 838.2 822.8 5085.8655.4 224.5 666.0 728.7 149.4 110.9 269.1 181.3 3214.4283.2 125.7 3489.6 156.3

Oracle 10g 1068 117.5 79.2 84 334.4 67 420.7 242.3 1268189.6 172.8 182.8 1251 48.5 122.9 227.9 372.7 2146460.1 156.3 1908 189.2

SQL 2005 987.3 26.2 59.2 76.4 235.7 49 236.9 163.9 1084104.7 112.4 292 405.7 74.1 42.1 210.4 90.5 1456193 76.4 1686 182.6

10X

TPC-H 1000GB - Power

10

100

1,000

10,000

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11

se

co

nd

s

SQL 2000 Oracle 10g

SQL 2005 Oracle 10g R2

10

100

1,000

10,000

Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22

se

co

nd

s

SQL 2000 Oracle 10g

SQL 2005 Oracle 10g R2

TPC-H 1000GB - Throughput

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18Q19 Q20 Q21 Q22

SQL 2000 7111 523.7 4892 3816 5009 185.3 5545 20526 243412241 1464 3677 5320 1108 884.6 3057 5198 152997042 4029 15943 643.4

Oracle 10g 4394 1294 501 366.5 1909 252.5 2594 2496 57481137 866.3 1315 3857 262.3 639.5 770.6 1733 215801140 1236 9459 791.5

SQL 2005 5460 197.1 1486 1640 2447 161.4 1861 2311 64641857 1006 1555 2256 580.4 253.6 821 1630 41261037 2376 5320 644.1

100

1,000

10,000

100,000

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22 RF1 RF2

Av

era

ge

se

co

nd

s

SQL 2000 Oracle 10g

SQL 2005 Oracle 10g R2

TPC-H 1000GB - Throughput

100

1,000

10,000

100,000

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11

Av

era

ge

se

co

nd

s

SQL 2000 Oracle 10g

SQL 2005 Oracle 10g R2

100

1,000

10,000

100,000

Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22

Av

era

ge

se

co

nd

s

SQL 2000 Oracle 10g

SQL 2005 Oracle 10g R2

TPC-H Query 1

SQL 2000 to 20052.56X Power1.30X Throughput

SELECT L_RETURNFLAG ,L_LINESTATUS ,SUM(L_QUANTITY) AS SUM_QTY, SUM(L_EXTENDEDPRICE) AS SUM_BASE_PRICE , SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)) AS SUM_DISC_PRICE, SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)*(1+L_TAX)) AS SUM_CHARGE, AVG(L_QUANTITY) AS AVG_QTY ,AVG(L_EXTENDEDPRICE) AS AVG_PRICE, AVG(L_DISCOUNT) AS AVG_DISC,COUNT(*) AS COUNT_ORDERFROM LINEITEM WHERE L_SHIPDATE <= DATEADD(dd, -90, cast('1998/12/01' as smalldatetime))GROUP BY L_RETURNFLAG ,L_LINESTATUS ORDER BY L_RETURNFLAG ,L_LINESTATUS

TPC-H Query 3

SQL 2000 to 200514.7X Power3.3X Throughput

SELECT TOP 10 L_ORDERKEY, SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)) AS REVENUE, O_ORDERDATE ,O_SHIPPRIORITYFROM CUSTOMER INNER JOIN ORDERS ON C_CUSTKEY = O_CUSTKEY INNER JOIN LINEITEM ON L_ORDERKEY = O_ORDERKEYWHERE C_MKTSEGMENT = 'BUILDING' AND O_ORDERDATE < '1995-03-15' AND L_SHIPDATE > '1995-03-15'GROUP BY L_ORDERKEY ,O_ORDERDATE ,O_SHIPPRIORITY ORDER BY REVENUE DESC ,O_ORDERDATE

Query 8

SQL 2000 to 20055.0X Power8.9X Throughput

100

1,000

10,000

100,000

1,000,000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

DOP 1 DOP 2 DOP 4 DOP 8 DOP 16

100

1,000

10,000

100,000

1,000,000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

DOP 1 DOP 2 DOP 4 DOP 8 DOP 16

10GB Line Item in memory test

SQL Server 2000SP4Query time in milli-seconds versus Max Degree of Parallelism

Limited parallelism gains beyond 4-8PSQL Server 2005June CTPContinued parallelism gains beyond 4P

100

1,000

10,000

100,000

1,000,000

1 2 3 4 5 6 7 8 9 10 11

DOP 1 DOP 2 DOP 4 DOP 8 DOP 16

100

1,000

10,000

100,000

1,000,000

1 2 3 4 5 6 7 8 9 10 11

DOP 1 DOP 2 DOP 4 DOP 8 DOP 16

10GB Line Item in memory test 1

SQL Server 2000SP4Query time in milli-seconds versus Max Degree of Parallelism

Limited parallelism gains beyond 4-8PSQL Server 2005June CTPContinued parallelism gains beyond 4P

100

1,000

10,000

100,000

1,000,000

12 13 14 15 16 17 18 19 20 21 22

DOP 1 DOP 2 DOP 4 DOP 8 DOP 16

100

1,000

10,000

100,000

1,000,000

12 13 14 15 16 17 18 19 20 21 22

DOP 1 DOP 2 DOP 4 DOP 8 DOP 16

10GB Line Item in memory test 2

SQL Server 2000SP4Query time in milli-seconds versus Max Degree of Parallelism

Limited parallelism gains beyond 4-8PSQL Server 2005June CTPContinued parallelism gains beyond 4P

10GB in memory test

Max Degree of Parallelism 1

Max DOP 4

Max DOP 16

30% reduction in total time at 1P

50% reduction in total time at 16P

100

1,000

10,000

100,000

1,000,000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

SQL 2000 SQL 2005

100

1,000

10,000

100,000

1,000,000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

SQL 2000 SQL 2005

100

1,000

10,000

100,000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

SQL 2000 SQL 2005

In-Memory Table Scan

Xeon systems have better table scan performance on SQL 2000SQL 2000 default table scan performance depends on build

0

500

1,000

1,500

2,000

2,500

3,000

3,500

4,000

4,500

5,000

Default RowLock PagLock TabLock NoLock

MB

/sec

SQL 2000 Clust. Index Scan SQL 2000 Heap Table Scan

SQL 2005 Clust. Index Scan SQL 2005 Heap Table

Table Scan to Disk

SQL 2005 Table Scan performance probably limited by disk system

0

200

400

600

800

1,000

1,200

1,400

1,600

Default RowLock PagLock TabLock NoLock

MB

/sec

SQL 2000 Clust. Index Scan SQL 2000 Heap Table Scan

SQL 2005 Clust. Index Scan SQL 2005 Heap Table

Loop Join – SQL 2000 & 2005

SQL 2000

SQL 2005

No scaling from 1-2P

Better overall performance, scaling from 1-8P

0

50

100

150

200

250

300

350

400

100 1,000 10,000 100,000000's rows per query

00

0's

ro

ws

/se

c

DOP 1 DOP 2 DOP 4 DOP 8 DOP 16

0

100

200

300

400

500

600

100 1,000 10,000 100,000000's rows per query

00

0's

ro

w/s

ec

DOP 1 DOP 2 DOP 4 DOP 8 DOP 16

Hash Join

SQL 2000

SQL 2005

Unpredictable behavior below 3M rowsLimited scaling

Consistent behavior < 1M rowsNot as much fall off in large joins

0

500

1,000

1,500

2,000

2,500

3,000

100 1,000 10,000 100,000000's rows per query

00

0's

ro

ws

/se

c

DOP 1

DOP 2

DOP 4

DOP 8

DOP 16

0

500

1,000

1,500

2,000

2,500

3,000

100 1,000 10,000 100,000000's rows per query

00

0's

ro

ws

/se

c

DOP 1 DOP 2 DOP 4 DOP 8 DOP 16

Merge Join – SQL 2000

SQL 2005 does not use parallel merge joins? Slight decrease in non-parallel performance.

SQL 2000 parallel merge joins has serious problems, runs slower, consumes more CPU. It should be possible to performance a parallel merge efficiently?

0

500

1,000

1,500

2,000

2,500

100 1,000 10,000 100,000000's rows per query

00

0's

ro

ws

/se

c

DOP 1 DOP 2 DOP 4 DOP 8 DOP 16

LiteSpeed Backup Performance

0

200

400

600

800

1000

1200

2.6:1 5:1 8:1Compression Ratio

MB

/sec

Threads 6 8 10 12

HP rx8620, 16 x 1.5GHz Itanium 2 processors2 EVA 5000 (4 HSV110 controllers, 8 enclosures x14 disks each8 x 2Gbit/sec FC ports

SQL 2005 Performance Summary

Significant improvements in large queries

Data Warehouse Applications

Improved disk performance

More Information

www.sql-server-performance.com/joe_chang.asp

SQL Server Quantitative Performance SQL Server Quantitative Performance AnalysisAnalysis

Server System ArchitectureServer System Architecture

Processor PerformanceProcessor Performance

Direct Connect Gigabit NetworkingDirect Connect Gigabit Networking

Parallel Execution PlansParallel Execution Plans

Large Data OperationsLarge Data Operations

Transferring StatisticsTransferring Statistics

SQL Server Backup Performance with SQL Server Backup Performance with LiteSpeedLiteSpeed

[email protected]

System Configuration

rx8620

16 Itanium 21.5GHz

HSV110 HSV110 HSV110 HSV110

8 2Gb/sFC ports 6 SCSI Disks

rx862016 Itanium 2

1.5GHz

HSV110 HSV110 HSV110 HSV110

8 2Gb/sFC ports 6 SCSI Disks

Bookmark Lookup Performance

0

200,000

400,000

600,000

800,000

1,000,000

1,200,000

1,400,000

1,600,000

1,800,000

2,000,000

1 2 4 8 16Degree of Parallelism

Ro

ws/

sec

2000 Clustered Index

2005 Clustered Index

2000 Heap

2005 Heap

SQL 2005 has better bookmark lookup performance to Clustered Indexes SQL 2000 has better bookmark lookup performance to Heap organized tables