Front cover
DS8000 Performance Monitoring and TuningUnderstand the
performance aspects of the DS8000 architecture Configure the DS8000
to fully exploit its capabilities Use planning and monitoring tools
with the DS8000
Bert Dufrasne Brett Allison John Barnes Jean Iyabi Rajesh
Jeyapaul Peter Kimmel
Chuck Laing Anderson Nobre Rene Oehme Gero Schmidt Paulus
Usong
ibm.com/redbooks
International Technical Support Organization DS8000 Performance
Monitoring and Tuning March 2009
SG24-7146-01
Note: Before using this information and the product it supports,
read the information in Notices on page xiii.
Second Edition (March 2009) This edition applies to the IBM
System Storage DS8000 with Licensed Machine Code 5.4.1.xx.xx (Code
bundles 64.1.x.x).
Copyright International Business Machines Corporation 2009. All
rights reserved. Note to U.S. Government Users Restricted Rights --
Use, duplication or disclosure restricted by GSA ADP Schedule
Contract with IBM Corp.
ContentsNotices . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . xiii Trademarks . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . xiv Preface . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . xvii The team that wrote this IBM Redbooks
publication . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . xvii Special thanks . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . xix Become a published author . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
xx Comments welcome. . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx
Chapter 1. DS8000 characteristics . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . 1.1 The storage
server challenge . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 1.1.1 Performance numbers .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 1.1.2 Recommendations and rules . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.3 Modeling your workload . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.4
Allocating hardware components to workloads. . . . . . . . . . . .
. . . . . . . . . . . . . . . . 1.2 Meeting the challenge: DS8000 .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 1.2.1 DS8000 models and characteristics . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3
DS8000 performance characteristics overview . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 1.3.1 Advanced caching
techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . 1.3.2 IBM System Storage multipath
Subsystem Device Driver (SDD) . . . . . . . . . . . . . . 1.3.3
Performance characteristics for System z. . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 1 2 2 2 3 3 3 4 5 6 6 6
Chapter 2. Hardware configuration . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.1 Storage
system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 Processor
memory and cache . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 12 2.2.1 Cache and I/O
operations . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 12 2.2.2 Determining the right amount
of cache storage . . . . . . . . . . . . . . . . . . . . . . . . .
. 16 2.3 RIO-G interconnect and I/O enclosures . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.3.1 RIO-G
loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 17 2.3.2 I/O
enclosures . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 17 2.4 Disk
subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.4.1 Device
adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 18 2.4.2 Fibre Channel
disk architecture in the DS8000. . . . . . . . . . . . . . . . . .
. . . . . . . . . 18 2.4.3 Disk enclosures. . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 19 2.4.4 Fibre Channel drives compared to FATA and SATA
drives . . . . . . . . . . . . . . . . . 20 2.4.5 Arrays across
loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 20 2.4.6 Order of installation . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . 21 2.4.7 Performance Accelerator feature
(Feature Code 1980) . . . . . . . . . . . . . . . . . . . . 23 2.5
Host adapters . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.5.1
Fibre Channel and FICON host adapters . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 24 2.5.2 ESCON host adapters . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 25 2.5.3 Multiple paths to Open Systems servers. .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.5.4 Multiple paths to System z servers . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 26 2.5.5 Spreading
host attachments . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 26 2.6 Tools to aid in hardware
planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . 27 2.6.1 White papers . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 27 2.6.2 Disk Magic . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 27 2.6.3 Capacity Magic . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
Copyright IBM Corp. 2009. All rights reserved.
iii
Chapter 3. Understanding your workload . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 3.1 General workload
types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 3.1.1 Standard workload . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 3.1.2 Read intensive cache unfriendly workload .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.3
Sequential workload . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 3.1.4 Batch jobs
workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 3.1.5 Sort jobs workload . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 3.2 Database workload . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . 3.2.1 DB2 query workload . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.2 DB2 logging workload . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 DB2
transaction environment workload . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 3.2.4 DB2 utilities workload . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 3.3 Application workload . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 3.3.1 General file serving . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.2 Online transaction processing. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 3.3.3 Data mining .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 3.3.4 Video on demand . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 3.3.5 Data warehousing . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 3.3.6 Engineering and scientific applications . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.7 Digital
video editing . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 3.4 Profiling workloads
in the design phase . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . 3.5 Understanding your workload type . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 3.5.1 Monitoring the DS8000 workload . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Monitoring
the host workload . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . Chapter 4. Logical configuration
concepts and terminology . . . . . . . . . . . . . . . . . . . .
4.1 RAID levels and spares. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.1 RAID
5 overview . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 4.1.2 RAID 6 overview .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 4.1.3 RAID 10 overview . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 4.1.4 Spare creation. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 4.2 The abstraction layers for logical configuration . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Array
sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Arrays . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 4.2.3 Ranks . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 4.2.4 Extent pools . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . 4.2.5 Logical volumes . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . 4.2.6 Space Efficient volumes . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.7
Allocation, deletion, and modification of LUNs and CKD volumes . .
. . . . . . . . . . 4.2.8 Logical subsystems (LSS). . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.9 Address groups . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.10
Volume access . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 4.2.11 Summary of the
logical configuration hierarchy . . . . . . . . . . . . . . . . . .
. . . . . . . 4.3 Understanding the array to LUN relationship . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 How
extents are formed together to make DS8000 LUNs . . . . . . . . . .
. . . . . . . . 4.3.2 Understanding data I/O placement on ranks and
extent pools . . . . . . . . . . . . . . . Chapter 5. Logical
configuration performance considerations . . . . . . . . . . . . .
. . . . . 5.1 Basic configuration principles for optimal
performance. . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1
Workload isolation. . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 5.1.2 Workload
resource-sharing . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 5.1.3 Workload spreading . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 5.1.4 Using workload isolation, resource-sharing,
and spreading . . . . . . . . . . . . . . . . . 5.2 Analyzing
application workload characteristics . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . ivDS8000 Performance Monitoring and
Tuning
29 30 30 30 30 30 30 31 32 32 32 32 32 33 34 34 34 34 35 35 35
38 38 38 41 42 42 43 43 44 45 45 45 46 47 48 49 50 53 53 54 54 55
56 61 63 64 64 65 66 67 68
5.2.1 Determining isolation requirements. . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 68 5.2.2 Reviewing
remaining workloads for feasibility of resource-sharing. . . . . .
. . . . . . 70 5.3 Planning allocation of disk and host connection
capacity . . . . . . . . . . . . . . . . . . . . . . . 70 5.3.1
Planning DS8000 hardware resources for isolated workloads . . . . .
. . . . . . . . . . 70 5.3.2 Planning DS8000 hardware resources for
resource-sharing workloads . . . . . . . . 70 5.4 Planning volume
and host connection spreading . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 71 5.4.1 Spreading volumes for isolated and
resource-sharing workloads. . . . . . . . . . . . . 71 5.4.2
Spreading host connections for isolated and resource-sharing
workloads . . . . . . 72 5.5 Planning array sites. . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 73 5.5.1 DS8000 configuration example 1: Array
site planning considerations . . . . . . . . . 73 5.5.2 DS8000
configuration example 2: Array site planning considerations . . . .
. . . . . 75 5.5.3 DS8000 configuration example 3: Array site
planning considerations . . . . . . . . . 77 5.6 Planning RAID
arrays and ranks . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 81 5.6.1 RAID-level performance
considerations . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . 81 5.6.2 RAID array considerations . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.6.3 Rank considerations . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.7 Planning
extent pools . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 91 5.7.1 Single-rank
and multi-rank extent pools . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . 92 5.7.2 Extent allocation methods for
multi-rank extent pools. . . . . . . . . . . . . . . . . . . . . .
96 5.7.3 Balancing workload across available resources . . . . . .
. . . . . . . . . . . . . . . . . . . 103 5.7.4 Assigning workloads
to extent pools . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 104 5.7.5 Planning for multi-rank extent pools . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
5.7.6 Planning for single-rank extent pools . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 111 5.8 Plan address
groups, LSSs, volume IDs, and CKD PAVs . . . . . . . . . . . . . .
. . . . . . . 118 5.8.1 Volume configuration scheme using
application-related LSS/LCU IDs . . . . . . . 120 5.8.2 Volume
configuration scheme using hardware-bound LSS/LCU IDs . . . . . . .
. . 124 5.9 Plan I/O port IDs, host attachments, and volume groups
. . . . . . . . . . . . . . . . . . . . . . 131 5.9.1 DS8000
configuration example 1: I/O port planning considerations . . . . .
. . . . . 133 5.9.2 DS8000 configuration example 2: I/O port
planning considerations . . . . . . . . . . 136 5.9.3 DS8000
configuration example 3: I/O port planning considerations . . . . .
. . . . . 140 5.10 Implement and document DS8000 logical
configuration . . . . . . . . . . . . . . . . . . . . . . 143
Chapter 6. Performance management process . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 6.1 Introduction . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 6.2 Purpose . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 6.3 Operational performance subprocess .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 6.3.1 Inputs . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.3.2 Outputs . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.3
Tasks, actors, and roles . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 6.3.4 Performance
troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 6.4 Tactical performance subprocess . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 6.4.1 Inputs . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4.2 Outputs . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.3
Tasks, actors, and roles . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 6.5 Strategic
performance subprocess . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 6.5.1 Inputs . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 6.5.2 Outputs . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 6.5.3 Tasks, actors, and roles . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 7. Performance planning tools . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 7.1 Disk Magic. . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 7.1.1 The need for performance
planning and modeling tools. . . . . . . . . . . . . . . . . . .
7.1.2 Overview and characteristics . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 147 148 149 151 152
153 153 154 155 156 156 157 158 158 158 159 161 162 162 163
Contents
v
7.1.3 Output information . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.4 Disk
Magic modeling . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 7.2 Disk Magic for System z
(zSeries) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 7.2.1 Process the DMC file . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 7.2.2 zSeries model to merge the two ESS-800s to a DS8300 . . . .
. . . . . . . . . . . . . . 7.2.3 Disk Magic performance projection
for zSeries model . . . . . . . . . . . . . . . . . . . . 7.2.4
Workload growth projection for zSeries model . . . . . . . . . . .
. . . . . . . . . . . . . . . 7.3 Disk Magic for Open Systems . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . 7.3.1 Process the TotalStorage Productivity Center csv
output file . . . . . . . . . . . . . . . 7.3.2 Open Systems model
to merge the two ESS-800s to a DS8300 . . . . . . . . . . . . 7.3.3
Disk Magic performance projection for an Open Systems model . . . .
. . . . . . . . 7.3.4 Workload growth projection for an Open
Systems model . . . . . . . . . . . . . . . . . . 7.4 Workload
growth projection. . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 7.5 Input data needed for
Disk Magic study . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 7.5.1 z/OS environment . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 7.5.2 Open Systems environment . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 7.6 Configuration
guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . Chapter 8. Practical
performance management. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 8.1 Introduction to practical performance management
. . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Performance
management tools . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 8.2.1 TotalStorage Productivity
Center overview . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 8.2.2 TotalStorage Productivity Center data collection . . .
. . . . . . . . . . . . . . . . . . . . . 8.2.3 TotalStorage
Productivity Center measurement of DS8000 components. . . . . .
8.2.4 General TotalStorage Productivity Center measurement
considerations . . . . . . 8.3 TotalStorage Productivity Center
data collection . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 8.3.1 Timestamps . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.2
Duration . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.3 Intervals
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 8.4 Key performance
metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 8.4.1 DS8000 key performance
indicator thresholds . . . . . . . . . . . . . . . . . . . . . . .
. . . 8.5 TotalStorage Productivity Center reporting options . . .
. . . . . . . . . . . . . . . . . . . . . . . 8.5.1 Alerts . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 8.5.2 Predefined
performance reports in TotalStorage Productivity Center. . . . . .
. . . 8.5.3 Ad hoc reports. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.4
Batch reports. . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . 8.5.5 TPCTOOL . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 8.5.6 Volume Planner . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 8.5.7 TPC Reporter for Disk. . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 8.6 Monitoring performance of a SAN switch or director. . . . . .
. . . . . . . . . . . . . . . . . . . . 8.6.1 SAN configuration
examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 8.6.2 TotalStorage Productivity Center for
Fabric alerts . . . . . . . . . . . . . . . . . . . . . . . 8.6.3
TotalStorage Productivity Center for Fabric reporting. . . . . . .
. . . . . . . . . . . . . . 8.6.4 TotalStorage Productivity Center
for Fabric metrics . . . . . . . . . . . . . . . . . . . . . . 8.7
End-to-end analysis of I/O performance problems . . . . . . . . . .
. . . . . . . . . . . . . . . . . 8.7.1 Performance analysis
examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 8.8 TotalStorage Productivity Center for Disk in
mixed environment . . . . . . . . . . . . . . . . Chapter 9. Host
attachment . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 9.1 DS8000 host attachment. . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 9.2 Attaching Open Systems hosts . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 9.2.1 Fibre Channel . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.2 SAN
implementations . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
163 163 165 165 170 177 179 180 181 188 194 195 197 197 198 198
199 203 204 204 205 205 207 212 214 214 216 217 218 221 222 223 228
229 232 236 239 240 242 243 246 247 248 249 257 263 265 266 266 267
267
vi
DS8000 Performance Monitoring and Tuning
9.2.3 Multipathing. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3
Attaching IBM System z and S/390 hosts . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 9.3.1 ESCON . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . 9.3.2 FICON . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 9.3.3 FICON configuration and performance
considerations . . . . . . . . . . . . . . . . . . . . 9.3.4 z/VM,
z/VSE, and Linux on System z attachment. . . . . . . . . . . . . .
. . . . . . . . . . Chapter 10. Performance considerations with
Windows Servers . . . . . . . . . . . . . . . 10.1 General Windows
performance tuning . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 10.2 I/O architecture overview . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 10.3 Windows Server 2008 I/O Manager enhancements . . . . .
. . . . . . . . . . . . . . . . . . . . 10.4 Filesystem . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 10.4.1 Windows filesystem
overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 10.4.2 NTFS guidelines . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 10.5 Volume management . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.1
Microsoft Logical Disk Manager (LDM) . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 10.5.2 Microsoft LDM software RAID.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 10.5.3 Veritas Volume Manager (VxVM) . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 10.5.4 Determining
volume layout . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 10.6 Multipathing and the port layer . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . 10.6.1 SCSIport scalability issues . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.2
Storport scalability features. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 10.6.3 Subsystem Device
Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 10.6.4 Subsystem Device Driver Device
Specific Module . . . . . . . . . . . . . . . . . . . . . . 10.6.5
Veritas Dynamic MultiPathing (DMP) for Windows . . . . . . . . . .
. . . . . . . . . . . . 10.7 Host bus adapter (HBA) settings. . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 10.8 I/O performance measurement . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.1 Key I/O
performance metrics . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . 10.8.2 Windows Performance console
(perfmon) . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.8.3 Performance log configuration and data export . . . . . . .
. . . . . . . . . . . . . . . . . 10.8.4 Collecting configuration
data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 10.8.5 Correlating performance and configuration
data. . . . . . . . . . . . . . . . . . . . . . . . 10.8.6
Analyzing performance data . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 10.8.7 Windows Server
Performance Analyzer . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 10.9 Task Manager . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 10.9.1 Starting Task Manager . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 10.10 I/O load
testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 10.10.1 Types of tests
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 10.10.2 Iometer . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . Chapter 11. Performance considerations with UNIX
servers . . . . . . . . . . . . . . . . . . . 11.1 Planning and
preparing UNIX servers for performance . . . . . . . . . . . . . .
. . . . . . . . 11.1.1 UNIX disk I/O architecture. . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.2 AIX disk I/O components. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.1 AIX
Journaled File System (JFS) and Journaled File System 2 (JFS2) . .
. . . . 11.2.2 Veritas File System (VxFS) for AIX . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . 11.2.3 General
Parallel FileSystem (GPFS) . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . 11.2.4 IBM Logical Volume Manager (LVM) . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.5
Veritas Volume Manager (VxVM) . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 11.2.6 IBM Subsystem Device Driver
(SDD) for AIX . . . . . . . . . . . . . . . . . . . . . . . . . .
11.2.7 MPIO with SDDPCM . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 11.2.8 Veritas
Dynamic MultiPathing (DMP) for AIX . . . . . . . . . . . . . . . .
. . . . . . . . . . 11.2.9 FC adapters . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . .
270 275 275 276 279 279 281 282 283 283 284 284 284 285 285 286
286 287 288 289 289 290 290 291 291 291 292 294 296 296 297 297 300
301 301 304 304 305 307 308 309 311 312 315 315 316 320 321 321 322
322
Contents
vii
11.2.10 Virtual I/O Server (VIOS). . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . 11.3 AIX
performance monitoring tools . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 11.3.1 AIX vmstat. . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . 11.3.2 pstat . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 11.3.3 AIX iostat. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . 11.3.4 lvmstat. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.3.5 topas . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.6
nmon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.7 fcstat .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 11.3.8 filemon . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 11.4 Solaris disk I/O components
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 11.4.1 UFS . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . 11.4.2 Veritas FileSystem (VxFS) for Solaris. . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . 11.4.3 SUN
Solaris ZFS. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 11.4.4 Solaris Volume
Manager (formerly Solstice DiskSuite). . . . . . . . . . . . . . .
. . . . 11.4.5 Veritas Volume Manager (VxVM) for Solaris . . . . .
. . . . . . . . . . . . . . . . . . . . . 11.4.6 IBM Subsystem
Device Driver for Solaris . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 11.4.7 MPxIO . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 11.4.8 Veritas Dynamic MultiPathing (DMP) for Solaris. . . .
. . . . . . . . . . . . . . . . . . . . 11.4.9 Array Support
Library (ASL) . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . 11.4.10 FC adapter . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . 11.5 Solaris performance monitoring tools . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.1
fcachestat and directiostat . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 11.5.2 Solaris vmstat . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 11.5.3 Solaris iostat . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 11.5.4 vxstat. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 11.5.5 dtrace . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.6 HP-UX Disk I/O architecture . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.1 HP-UX
High Performance File System (HFS). . . . . . . . . . . . . . . . .
. . . . . . . . . 11.6.2 HP-UX Journaled File System (JFS). . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.3 HP
Logical Volume Manager (LVM) . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . 11.6.4 Veritas Volume Manager (VxVM)
for HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . .
11.6.5 PV Links . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.6
Native multipathing in HP-UX . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 11.6.7 Subsystem Device
Driver (SDD) for HP-UX . . . . . . . . . . . . . . . . . . . . . .
. . . . . 11.6.8 Veritas Dynamic MultiPathing (DMP) for HP-UX . . .
. . . . . . . . . . . . . . . . . . . . 11.6.9 Array Support
Library (ASL) for HP-UX . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 11.6.10 FC adapter . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 11.7 HP-UX performance monitoring tools . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 11.7.1 HP-UX sar . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 11.7.2 vxstat. . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 11.7.3 GlancePlus and HP
Perfview/Measureware . . . . . . . . . . . . . . . . . . . . . . .
. . . . 11.8 SDD commands for AIX, HP-UX, and Solaris . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 11.8.1 HP-UX SDD
commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 11.8.2 Sun Solaris SDD commands. . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.9 Testing and verifying DS8000 Storage . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 11.9.1 Using the dd
command to test sequential rank reads and writes . . . . . . . . .
. . 11.9.2 Verifying your system . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 12.
Performance considerations with VMware . . . . . . . . . . . . . .
. . . . . . . . . 12.1 Disk I/O architecture overview . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.2 Multipathing considerations. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 12.3
Performance monitoring tools . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
323 325 326 328 329 334 335 336 339 339 343 343 344 345 346 347
348 348 349 349 350 350 350 351 352 353 354 356 356 357 357 362 362
362 363 363 363 363 363 363 366 366 366 371 373 375 376 377 383 384
386 389
viii
DS8000 Performance Monitoring and Tuning
12.3.1 Virtual Center Performance Statistics . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 12.3.2 Performance
monitoring with esxtop. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 12.3.3 Guest-based performance monitoring. . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4
VMware specific tuning for maximum performance . . . . . . . . . .
. . . . . . . . . . . . . . . 12.4.1 Workload spreading . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . 12.4.2 Virtual Machines sharing the same LUN. . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 12.4.3 ESX filesystem
considerations . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 12.4.4 Aligning partitions . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 12.5 Tuning of Virtual Machines . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter
13. Performance considerations with Linux. . . . . . . . . . . . .
. . . . . . . . . . . . . 13.1 Supported platforms and
distributions. . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 13.2 Linux disk I/O architecture. . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 13.2.1 I/O subsystem architecture . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . 13.2.2 Cache and
locality of reference . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 13.2.3 Block layer . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 13.2.4 I/O device driver . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13.3 Specific configuration for storage performance. . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 13.3.1 Host bus adapter
for Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 13.3.2 Multipathing in Linux . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 13.3.3 Software RAID functions . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.4
Logical Volume Manager . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 13.3.5 Tuning the disk I/O
scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 13.3.6 Filesystems . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 13.4 Linux performance monitoring tools . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 13.4.1 Disk I/O
performance indicators . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . 13.4.2 Finding disk bottlenecks . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . Chapter 14. IBM System Storage SAN Volume Controller
attachment . . . . . . . . . . . 14.1 IBM System Storage SAN Volume
Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 14.1.1 SAN Volume Controller concepts . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 14.1.2 SAN Volume
Controller multipathing. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 14.1.3 SVC Advanced Copy Services . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.2
SAN Volume Controller performance considerations . . . . . . . . .
. . . . . . . . . . . . . . . 14.3 DS8000 performance
considerations with SVC . . . . . . . . . . . . . . . . . . . . . .
. . . . . . 14.3.1 DS8000 array . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.3.2 DS8000 rank format . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 14.3.3 DS8000
extent pool implications. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . 14.3.4 DS8000 volume considerations
with SVC. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.3.5 Volume assignment to SAN Volume Controller . . . . . . . . .
. . . . . . . . . . . . . . . 14.3.6 Managed Disk Group for DS8000
Volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.4
Performance monitoring . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 14.4.1 Using
TotalStorage Productivity Center for Disk to monitor the SVC . . .
. . . . . 14.5 Sharing the DS8000 between a server and the SVC . .
. . . . . . . . . . . . . . . . . . . . . . 14.5.1 Sharing the
DS8000 between Open Systems servers and the SVC . . . . . . . .
14.5.2 Sharing the DS8000 between System i server and the SVC . . .
. . . . . . . . . . . 14.5.3 Sharing the DS8000 between System z
server and the SVC . . . . . . . . . . . . . . 14.6 Advanced
functions for the DS8000 . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 14.6.1 Cache-disabled VDisks . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . 14.7 Configuration guidelines for optimizing
performance . . . . . . . . . . . . . . . . . . . . . . . .
389 390 391 392 392 392 393 396 399 401 402 402 403 404 405 405
406 406 407 409 410 412 414 417 417 418 421 422 422 425 426 427 429
429 429 430 434 434 435 436 436 437 437 438 438 438 438 439
Chapter 15. System z servers . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 441 15.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442 15.2
Parallel Access Volumes. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 442Contents
ix
15.2.1 Static PAV, Dynamic PAV, and HyperPAV . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 15.2.2 HyperPAV compared to
dynamic PAV test . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 15.2.3 PAV and large volumes . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 15.3 Multiple
Allegiance . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 15.4 How PAV and
Multiple Allegiance work . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 15.4.1 Concurrent read operation . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 15.4.2 Concurrent write operation . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . 15.5 I/O Priority
Queuing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 15.6 Logical volume sizes .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . 15.6.1 Selecting the volume size . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 15.6.2 Larger volume compared to smaller volume performance
. . . . . . . . . . . . . . . . 15.6.3 Planning the volume sizes of
your configuration. . . . . . . . . . . . . . . . . . . . . . . .
15.7 FICON . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15.7.1 Extended Distance FICON . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 15.7.2 High Performance
FICON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 15.7.3 MIDAW . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 15.8 z/OS planning and configuration guidelines . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . 15.8.1 Channel
configuration . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 15.8.2 Extent pool . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 15.8.3 Considerations for mixed workloads . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.9
DS8000 performance monitoring tools . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 15.10 RMF . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 15.10.1 I/O response time . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 15.10.2 I/O response time components . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.10.3
IOP/SAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 15.10.4 FICON host
channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . 15.10.5 FICON director . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . 15.10.6 Processor complex . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.10.7
Cache and NVS . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 15.10.8 DS8000 FICON/Fibre
port and host adapter. . . . . . . . . . . . . . . . . . . . . . .
. . . 15.10.9 Extent pool and rank . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.11 RMF
Magic for Windows . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 15.11.1 RMF Magic analysis
process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 15.11.2 Data collection step. . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15.11.3 RMF Magic reduce step . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 15.11.4 RMF Magic
analyze step. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 15.11.5 Data presentation and reporting
step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15.11.6 Hints and tips . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 16.
Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 16.1 DB2 in a z/OS
environment. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 16.1.1 Understanding your database
workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 16.1.2 DB2 overview . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . 16.1.3 DB2
storage objects . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 16.1.4 DB2 dataset types. . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 16.2 DS8000 considerations for DB2 . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.3 DB2 with DS8000 performance recommendations . . . . . . . . .
. . . . . . . . . . . . . . . . . 16.3.1 Know where your data
resides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 16.3.2 Balance workload across DS8000 resources . . .
. . . . . . . . . . . . . . . . . . . . . . . 16.3.3 Take advantage
of VSAM data striping . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 16.3.4 Large volumes . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.3.5 Modified Indirect Data Address Words (MIDAWs) . . . . . . .
. . . . . . . . . . . . . . . 16.3.6 Adaptive Multi-stream
Prefetching (AMP) . . . . . . . . . . . . . . . . . . . . . . . . .
. . . .
442 443 445 446 446 447 448 449 449 450 451 453 454 455 455 457
459 459 461 463 464 464 464 466 468 468 469 470 470 472 474 476 477
478 479 479 479 482 485 486 486 487 487 488 489 489 489 490 490 490
491 491
x
DS8000 Performance Monitoring and Tuning
16.3.7 DB2 burst write . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.3.8
Monitoring DS8000 performance . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 16.4 DS8000 DB2 UDB in an Open
Systems environment . . . . . . . . . . . . . . . . . . . . . . .
16.4.1 DB2 UDB storage concepts . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 16.5 DB2 UDB with DS8000
performance recommendations. . . . . . . . . . . . . . . . . . . .
. . 16.5.1 Know where your data resides . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 16.5.2 Balance
workload across DS8000 resources . . . . . . . . . . . . . . . . .
. . . . . . . . . 16.5.3 Use DB2 to stripe across containers. . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.5.4
Selecting DB2 logical sizes . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 16.5.5 Selecting the DS8000
logical disk sizes . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . 16.5.6 Multipathing. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.6 IMS in a z/OS environment . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 16.6.1 IMS
overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 16.6.2 IMS logging . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 16.7 DS8000 considerations for IMS .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . 16.8 IMS with DS8000 performance recommendations . . . .
. . . . . . . . . . . . . . . . . . . . . . 16.8.1 Know where your
data resides . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 16.8.2 Balance workload across DS8000 resources
. . . . . . . . . . . . . . . . . . . . . . . . . . 16.8.3 Large
volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 16.8.4 Monitoring DS8000
performance . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . Chapter 17. Copy Services performance . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . 17.1 Copy
Services introduction . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 17.2 FlashCopy . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 17.2.1 FlashCopy performance
considerations . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 17.2.2 Performance planning for IBM FlashCopy SE . . . . .
. . . . . . . . . . . . . . . . . . . . 17.3 Metro Mirror. . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 17.3.1 Metro Mirror configuration
considerations. . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 17.3.2 Metro Mirror performance considerations . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 17.3.3 Scalability . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 17.4 Global Copy . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . 17.4.1 Global Copy configuration
considerations. . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 17.4.2 Global Copy performance consideration. . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 17.4.3 Scalability . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 17.5 Global Mirror . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . 17.5.1 Global Mirror performance
considerations . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 17.5.2 Global Mirror Session parameters . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 17.5.3 Avoid unbalanced
configurations . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 17.5.4 Growth within Global Mirror configurations
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.6 z/OS
Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 17.6.1 z/OS Global
Mirror control dataset placement . . . . . . . . . . . . . . . . .
. . . . . . . . 17.6.2 z/OS Global Mirror tuning parameters . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . 17.6.3 z/OS
Global Mirror enhanced multiple reader. . . . . . . . . . . . . . .
. . . . . . . . . . . 17.6.4 zGM enhanced multiple reader
performance improvement . . . . . . . . . . . . . . . 17.6.5 XRC
Performance Monitor . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 17.7 Metro/Global Mirror . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 17.7.1 Metro/Global Mirror performance . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17.7.2 z/OS Metro/Global Mirror . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 17.7.3 z/OS
Metro/Global Mirror performance . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . .
491 491 492 492 497 497 498 499 499 500 502 502 502 503 504 504
504 505 505 506 507 508 509 511 516 518 519 524 526 526 527 529 530
530 533 535 538 541 543 545 545 549 550 552 552 553 553 554
Appendix A. Logical configuration examples . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 555 A.1 Considering
hardware resource availability for throughput. . . . . . . . . . .
. . . . . . . . . . 556 A.2 Resource isolation or sharing . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 556
Contents
xi
Scenario 1: Spreading everything with no isolation . . . . . . .
. . . . . . . . . . . . . . . . . . . . Scenario 2: Spreading data
I/O with partial isolation . . . . . . . . . . . . . . . . . . . .
. . . . . . Scenario 3: Grouping unlike RAID types together in the
extent pool . . . . . . . . . . . . . . Scenario 4: Grouping like
RAID types in the extent pool . . . . . . . . . . . . . . . . . . .
. . . . Scenario 5: More isolation of RAID types in the extent pool
. . . . . . . . . . . . . . . . . . . . . Scenario 6: Balancing
mixed RAID type ranks and capacities . . . . . . . . . . . . . . .
. . . . Appendix B. Windows server performance log collection . . .
. . . . . . . . . . . . . . . . . . B.1 Windows Server 2003 log
file configuration . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . Configuring logging of disk metrics Windows Server
2003 . . . . . . . . . . . . . . . . . . . . . . Saving counter log
settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . Importing counter logs properties
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . Analyzing disk performance from collected data. . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . Retrieving data
from a counter log file . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . Exporting logged data on Windows
Server 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. B.2 Windows Server 2008 log file configuration . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . Windows Server 2008
Export . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . Appendix C. UNIX shell scripts . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . C.1 Introduction . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . C.2 vgmap . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. C.3 lvmap . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
C.4 vpath_iostat . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.5
ds_iostat . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.6
test_disk_speeds . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . C.7
lsvscsimap.ksh . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . C.8
mkvscsimap.ksh . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix D.
Post-processing scripts . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . D.1 Introduction . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . D.2 Dependencies. . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . D.2.1 Running the scripts. . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. Appendix E. Benchmarking . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . E.1 Goals of
benchmarking . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . E.2 Requirements for a
benchmark . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . Define the benchmark architecture . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. Define the benchmark workload . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . Monitoring the
performance . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . Define the benchmark time frame .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . E.3 Caution using benchmark results to design
production . . . . . . . . . . . . . . . . . . . . . . . Related
publications . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . IBM Redbooks
publications . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . Other publications . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . Online resources . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . How to get IBM Redbooks publications
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . Help from IBM . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. .
556 561 562 563 565 566 571 572 572 575 576 576 576 578 580 584
587 588 588 589 590 594 597 598 602 607 608 608 609 623 624 624 625
625 626 626 627 629 629 630 630 630 631
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 633
xii
DS8000 Performance Monitoring and Tuning
NoticesThis information was developed for products and services
offered in the U.S.A. IBM may not offer the products, services, or
features discussed in this document in other countries. Consult
your local IBM representative for information on the products and
services currently available in your area. Any reference to an IBM
product, program, or service is not intended to state or imply that
only that IBM product, program, or service may be used. Any
functionally equivalent product, program, or service that does not
infringe any IBM intellectual property right may be used instead.
However, it is the user's responsibility to evaluate and verify the
operation of any non-IBM product, program, or service. IBM may have
patents or pending patent applications covering subject matter
described in this document. The furnishing of this document does
not give you any license to these patents. You can send license
inquiries, in writing, to: IBM Director of Licensing, IBM
Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A. The
following paragraph does not apply to the United Kingdom or any
other country where such provisions are inconsistent with local
law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS
PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR
IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR
PURPOSE. Some states do not allow disclaimer of express or implied
warranties in certain transactions, therefore, this statement may
not apply to you. This information could include technical
inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in
new editions of the publication. IBM may make improvements and/or
changes in the product(s) and/or the program(s) described in this
publication at any time without notice. Any references in this
information to non-IBM Web sites are provided for convenience only
and do not in any manner serve as an endorsement of those Web
sites. The materials at those Web sites are not part of the
materials for this IBM product and use of those Web sites is at
your own risk. IBM may use or distribute any of the information you
supply in any way it believes appropriate without incurring any
obligation to you. Information concerning non-IBM products was
obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not
tested those products and cannot confirm the accuracy of
performance, compatibility or any other claims related to non-IBM
products. Questions on the capabilities of non-IBM products should
be addressed to the suppliers of those products. This information
contains examples of data and reports used in daily business
operations. To illustrate them as completely as possible, the
examples include the names of individuals, companies, brands, and
products. All of these names are fictitious and any similarity to
the names and addresses used by an actual business enterprise is
entirely coincidental. COPYRIGHT LICENSE: This information contains
sample application programs in source language, which illustrate
programming techniques on various operating platforms. You may
copy, modify, and distribute these sample programs in any form
without payment to IBM, for the purposes of developing, using,
marketing or distributing application programs conforming to the
application programming interface for the operating platform for
which the sample programs are written. These examples have not been
thoroughly tested under all conditions. IBM, therefore, cannot
guarantee or imply reliability, serviceability, or function of
these programs.
Copyright IBM Corp. 2009. All rights reserved.
xiii
TrademarksIBM, the IBM logo, and ibm.com are trademarks or
registered trademarks of International Business Machines
Corporation in the United States, other countries, or both. These
and other IBM trademarked terms are marked on their first
occurrence in this information with the appropriate symbol ( or ),
indicating US registered or common law trademarks owned by IBM at
the time this information was published. Such trademarks may also
be registered or common law trademarks in other countries. A
current list of IBM trademarks is available on the Web at
http://www.ibm.com/legal/copytrade.shtml The following terms are
trademarks of the International Business Machines Corporation in
the United States, other countries, or both:AIX 5L AIX alphaWorks
CICS DB2 Universal Database DB2 DS4000 DS6000 DS8000 ECKD
Enterprise Storage Server ESCON eServer FICON FlashCopy GDPS
Geographically Dispersed Parallel Sysplex GPFS HACMP i5/OS IBM
iSeries Iterations OMEGAMON OS/390 Parallel Sysplex POWER5 POWER5+
POWER6 PowerHA PowerPC PowerVM POWER pSeries Rational Redbooks
Redbooks (logo) RS/6000 S/390 Sysplex Timer System i System p5
System p System Storage System x System z10 System z9 System z
Tivoli Enterprise Console Tivoli TotalStorage xSeries
z/Architecture z/OS z/VM z/VSE z9 zSeries
The following terms are trademarks of other companies: Acrobat,
and Portable Document Format (PDF) are either registered trademarks
or trademarks of Adobe Systems Incorporated in the United States,
other countries, or both. Disk Magic, IntelliMagic, and the
IntelliMagic logo are trademarks of IntelliMagic BV in the United
States, other countries, or both. ITIL is a registered trademark,
and a registered community trademark of the Office of Government
Commerce, and is registered in the U.S. Patent and Trademark
Office. Novell, SUSE, the Novell logo, and the N logo are
registered trademarks of Novell, Inc. in the United States and
other countries. Oracle, JD Edwards, PeopleSoft, Siebel, and
TopLink are registered trademarks of Oracle Corporation and/or its
affiliates. QLogic, and the QLogic logo are registered trademarks
of QLogic Corporation. SANblade is a registered trademark in the
United States. SAP R/3, SAP, and SAP logos are trademarks or
registered trademarks of SAP AG in Germany and in several other
countries. VMotion, VMware, the VMware "boxes" logo and design are
registered trademarks or trademarks of VMware, Inc. in the United
States and/or other jurisdictions. J2EE, Java, JNI, S24, Solaris,
Solstice, Sun, ZFS, and all Java-based trademarks are trademarks of
Sun Microsystems, Inc. in the United States, other countries, or
both. Excel, Internet Explorer, Microsoft, MS-DOS, MS, PowerPoint,
SQL Server, Visual Basic, Windows NT, xivDS8000 Performance
Monitoring and Tuning
Windows Server, Windows, and the Windows logo are trademarks of
Microsoft Corporation in the United States, other countries, or
both. Intel, Intel logo, Intel Inside logo, and Intel Centrino logo
are trademarks or registered trademarks of Intel Corporation or its
subsidiaries in the United States, other countries, or both. UNIX
is a registered trademark of The Open Group in the United States
and other countries. Linux is a trademark of Linus Torvalds in the
United States, other countries, or both. Other company, product, or
service names may be trademarks or service marks of others.
Notices
xv
xvi
DS8000 Performance Monitoring and Tuning
PrefaceThis IBM Redbooks publication provides guidance about how
to configure, monitor, and manage your IBM System Storage DS8000 to
achieve optimum performance. It describes the DS8000 performance
features and characteristics and how they can be exploited with the
various server platforms that attach to the DS8000. Then, in
separate chapters, we detail specific performance recommendations
and discussions that apply for each server environment, as well as
for database and DS8000 Copy Services environments. We also outline
the various tools available for monitoring and measuring I/O
performance for different server environments, as well as describe
how to monitor the performance of the entire DS8000 subsystem.
The team that wrote this IBM Redbooks publicationThis book was
produced by a team of specialists from around the world working
with the International Technical Support Organization, San Jose
Center at the ESCC lab in Mainz, Germany. Bertrand Dufrasne is an
IBM Certified Consulting I/T Specialist and Project Leader for
System Storage disk products at the International Technical Support
Organization, San Jose Center. He has worked at IBM in various I/T
areas. Bertrand has written many IBM Redbooks publications and has
also developed and taught technical workshops. Before joining the
ITSO, he worked for IBM Global Services as an Application Architect
in the retail, banking, telecommunication, and healthcare
industries. He holds a Masters degree in Electrical Engineering
from the Polytechnic Faculty of Mons (Belgium). Brett Allison has
performed distributed systems performance-related work since 1997,
including performance analysis of J2EE applications, UNIX/Windows
NT systems, and SAN Storage technologies. He is currently the
storage performance and capacity technical focal point for IBM
Global Services Technology Delivery. He has designed and developed
tools, processes, and service offerings to support storage
performance and capacity. He has spoken at a number of conferences
and is the author of several White Papers on performance. John
Barnes is a Senior IT Specialist in IBM Global Services in the UK.
John started his IBM career 30 years ago as a Large Systems
hardware CE. After an assignment to the UK Hardware Support Centre,
he moved to a career in Availability Management. John then joined
the UK Storage and SAN Services Team in 2000, specializing in IBM
TotalStorage Enterprise Storage Server (ESS) and SAN
implementations. He now works in the UK STG Storage Services team,
specializing in DS8000 and SAN implementations, including SAN
Volume Controller (SVC) and Copy Services. Jean Iyabi is an active
member of the IBM ESCC (European Storage Competence Center) in
Mainz, Germany since 2001. As a Product Field Engineer, he acted as
last level support for High End storage disk. Jean has extensive
experience in DS8000 support and focuses on Host Attachment (System
z), Extended Copy Services Functions, and Geographically Dispersed
Parallel Sysplex (GDPS). He was assigned for two years as the EMEA
field support interface with the DS8000 development and test teams
in Tucson, AZ. He holds a degree in Electrical Engineering from the
University of Applied Sciences of Wiesbaden (Germany).
Copyright IBM Corp. 2009. All rights reserved.
xvii
Rajesh Jeyapaul is an AIX Development Support Specialist in IBM
India. He has nine years of experience in AIX, specializing in
investigating the performance impact of processes running in AIX.
Currently, he is leading a technical team responsible for providing
Development support to various AIX components. He holds a Masters
Degree in Software Systems from the University of BITS, India, and
an MBA from University of MKU, India. His areas of expertise
include System p, AIX, and High-Availability Cluster
Multi-Processing (HACMP). Peter Kimmel is an IT Specialist and the
ATS team lead of the Enterprise Disk Performance team at the
European Storage Competence Center in Mainz, Germany. He joined IBM
Storage in 1999 and since then worked with SSA, VSS, the various
ESS generations, and DS8000/DS6000. He has been involved in all
Early Shipment Programs (ESPs), early installs for the Copy
Services rollouts, and has co-authored several DS8000 IBM Redbooks
publications so far. Peter holds a Diploma (MSc) degree in Physics
from the University of Kaiserslautern. Chuck Laing is a Senior IT
Architect and Master Certified IT Specialist with The Open Group.
He is also an IBM Certified IT Specialist, specializing in IBM
Enterprise Class and Midrange Disk Storage Systems/Configuration
Management in the Americas ITD. He has co-authored eight previous
IBM Redbooks publications about the IBM TotalStorage Enterprise
Storage Server and the DS8000/6000. He holds a degree in Computer
Science. He has worked at IBM for over ten years. Before joining
IBM, Chuck was a hardware CE on UNIX systems for ten years and
taught Computer Science at Midland College for six and a half years
in Midland, Texas. Anderson Ferreira Nobre is a Certified IT
Specialist and Certified Advanced Technical Expert - IBM System p5
in Strategic Outsourcing in Hortolndia (Brazil). He has 10 years of
experience with UNIX (mainly with AIX). He was assigned to the UNIX
team in 2005 to plan, manage, and support the UNIX, SAN, and
Storage environments for IBM Outsourcing clients. Rene Oehme is an
IBM Certified Specialist for High-End Disk Solutions, working for
the Germany and CEMAAS Hardware Support Center in Mainz, Germany.
Rene has more than six years of experience in IBM hardware support,
including Storage Subsystems, SAN, and Tape Solutions, as well as
System p and System z. Currently, he provides support for clients
and service representatives with High End Disk Subsystems, such as
the DS8000, DS6000, and ESS. His main focus is Open Systems
attachment of High-End Disk Subsystems, including AIX, Windows,
Linux, and VMware. He holds a degree in Information Technology from
the University of Cooperative Education (BA) Stuttgart. Gero
Schmidt is an IT Specialist in the IBM ATS technical sales support
organization in Germany. He joined IBM in 2001 working at the
European Storage Competence Center (ESCC) in Mainz, providing
technical support for a broad range of IBM storage products (SSA,
ESS, DS4000, DS6000, and DS8000) in Open Systems environments with
a primary focus on storage subsystem performance. During his seven
years of experience with IBM storage products, he participated in
various beta test programs for ESS 800 and especially in the
product rollout and beta test program for the DS6000/DS8000 series.
He holds a degree in Physics (Dipl.-Phys.) from the Technical
University of Braunschweig, Germany. Paulus Usong started his IBM
career in Indonesia decades ago. He rejoined IBM at the Santa
Teresa Lab (now known as the Silicon Valley Lab). In 1995, he
joined the Advanced Technical Support group in San Jose. Currently,
he is a Certified Consulting I/T Specialist and his main
responsibilities are handling mainframe DASD performance critical
situations
xviii
DS8000 Performance Monitoring and Tuning
and performing Disk Magic study and remote copy sizing for
clients who want to implement the IBM solution for their disaster
recovery system.
The team: Rene, Bert, John, Brett, Gero, Anderson, Jean, Paulus,
and Peter
Special thanksFor hosting this residency at the ESCC in Mainz,
Germany, we want to thank: Rainer Zielonka - Director ESCC Dr.
Friedrich Gerken - Manager Services and Technical Sales Support
Rainer Erkens - Manager ESCC Service & Support Management Bernd
Mller - Manager Enterprise Disk High-End Solutions Europe, for
dedicating so many resources to this residency Stephan Weyrich -
Opportunity Manager ESCC Workshops We especially want to thank Lee
La Frese (IBM, Tucson) for being our special advisor and
development contact for this book. Many thanks to those people in
IBM in Mainz, Germany, who helped us with access to equipment as
well as technical information and review: Uwe Heinrich Mueller, Uwe
Schweikhard, Guenter Schmitt, Joerg Zahn, Werner Deul, Mike
Schneider, Markus Oscheka, Hartmut Bohnacker, Gerhard Pieper,
Alexander Warmuth, Kai Jehnen, Frank Krueger, and Werner Bauer
Special thanks to: John Bynum DS8000 World Wide Technical Support
Marketing Lead
Preface
xix
Thanks to the following people for their contributions to this
project: Mary Anne Bromley Garry Bennet Jay Kurtz Rosemary
McCutchen Brian J. Smith Sonny Williams IBM US Nick Clayton Patrick
Keyes Andy Wharton Barry Whyte IBM UK Brian Sherman IBM Canada
Become a published authorJoin us for a two-week to six-week
residency program. Help write an IBM Redbooks publication dealing
with specific products or solutions, while getting hands-on
experience with leading edge technologies. You will team with IBM
technical professionals, IBM Business Partners, and clients. Your
efforts will help increase product acceptance and client
satisfaction. As a bonus, you will develop a network of contacts in
IBM development labs, and increase your productivity and
marketability. Find out more about the residency program, browse
the residency index, and apply online at:
ibm.com/redbooks/residencies.html
Comments welcomeYour comments are important to us. We want our
IBM Redbooks publications to be as helpful as possible. Send us
your comments about this or other IBM Redbooks publications in one
of the following ways: Use the online Contact us review IBM
Redbooks publication form found at: ibm.com/redbooks Send your
comments in an e-mail to: [email protected] Mail your comments to:
IBM Corporation, International Technical Support Organization Dept.
HYTD Mail Station P099 2455 South Road Poughkeepsie, NY
12601-5400
xx
DS8000 Performance Monitoring and Tuning
1
Chapter 1.
DS8000 characteristicsThis chapter contains a high level
discussion and introduction to the storage server performance
challenge. Then, we provide an overview of the DS8000 model
characteristics that allow the DS8000 to meet this performance
challenge.
Copyright IBM Corp. 2009. All rights reserved.
1
1.1 The storage server challengeOne of the primary criteria in
judging a storage server is performance: how fast it responds to a
read or write request from an application server. How well a
storage server accomplishes this task is dependent on the design of
its hardware and firmware. Data continually moves from one
component to another component within a storage server. The
objective of server design is to have hardware of sufficient
throughput to keep that data flowing smoothly without having to
wait because a component is busy. When data stops flowing because a
component is busy, a bottleneck has formed. Obviously, it is
desirable to minimize the frequency and severity of bottlenecks.
The ideal storage server is one in which all components are well
utilized and bottlenecks are few. This scenario is the case if: The
machine is designed well, with all hardware components in balance.
To provide this balance over a range of workloads, a storage server
must allow a range of hardware component options. The machine is
sized well for the clients workload. That is, where options exist,
the right quantities of each option were chosen. The machine is set
up well. That is, where options exist in hardware installation and
logical configuration, these options are chosen correctly.
1.1.1 Performance numbersRaw performance numbers provide
evidence that a particular storage server is better than the
previous generation model, or better than the competitions product.
But isolated performance numbers are often out of line with a
production environment. It is important to understand how raw
performance numbers relate to the performance of the storage server
in processing a particular production workload. Throughput numbers
are usually achieved in controlled tests, which have the objective
of pushing as much data as possible through the storage server as a
whole, or perhaps through just a single component. At the point of
maximum throughput, the system is usually so overloaded that
response times are greatly extended. Trying to achieve such
throughput numbers in a normal business environment brings protests
from the users of the system, because response times are extremely
poor. To assure yourself that the DS8000 offers the latest and
fastest technology, take a look at the performance numbers for the
individual disks, adapters, and other components of the DS8000, as
well as for the total device. You will find that the DS8000 uses
the most current technology available. But, use a more rigorous
approach when planning the DS8000 hardware configuration to meet
the requirements of a specific environment.
1.1.2 Recommendations and rulesHardware selections are sometimes
based on general recommendations and rules. A general rule is a
simple guideline for making a selection based on limited
information. The advantage is that it allows you to make a quick
decision, with little effort, that provides a solution that works
acceptably well most of the time. The disadvantage is that it does
not work all the time; sometimes, the solution is not at all what
the client needs. You can increase the chances that the solution
will work by making it more conservative. However, a conservative
solution generally involves more hardware, which means a more
expensive solution. In this chapter, we will provide
recommendations and general rules for different hardware
2
DS8000 Performance Monitoring and Tuning
components. Just remember, only use general rules when there is
no information available to make a more informed decision.
1.1.3 Modeling your workloadA much better way to determine the
hardware requirements for your workload is to run a Disk Magic
model. Disk Magic is a modeling tool, which shows the throughput
and response time of a storage server based on workload
characteristics and the hardware resources of the storage server.
By converting the results of performance runs into mathematical
formulas, Disk Magic allows the results to be applied to a wide
range of workloads. Disk Magic allows many variables of hardware to
be brought together so the effect of each variable is integrated,
producing a result that shows the overall performance of the
storage server. For additional information about this tool, refer
to 7.1, Disk Magic on page 162.
1.1.4 Allocating hardware components to workloadsThere are two
contrasting methods to allocate the use of hardware components to
workloads. The first method is spreading the workloads across
components, which means that you try to share the use of hardware
components across all, or at least many, workloads. The more
hardware components are shared among multiple workloads, the more
effectively the hardware components are utilized, which reduces
total cost of ownership (TCO). For example, to attach multiple
hosts, you can use the same host adapters for all hosts instead of
acquiring a separate set of host adapters for each host. However,
the more that components are shared, the more potential there is
that one workload will dominate use of the component. The second
method is isolating workloads to specific hardware components,
which means that specific hardware components are used for one
workload, and other hardware components are used for different
workloads. The downside of isolating workloads is that certain
components are unused when their workload is not demanding service.
On the upside, it means that when that workload does demand
service, the component is available immediately, and the workload
does not have to contend with other workloads for that resource.
Spreading the workload maximizes the utilization and performance of
the storage server as a whole. Isolating a workload is a way to
maximize that individual workloads performance, making it run as
fast as possible. For a detailed discussion, refer to 5.1, Basic
configuration principles for optimal performance on page 64.
1.2 Meeting the challenge: DS8000The DS8000 is a member of the
DS product family. It offers disk storage servers with a wide range
of hardware component options to fit many workload requirements, in
terms of both type and size. It has the capability to scale very
well to the highest disk storage capacities. The scalability is
supported by design functions that allow installation of additional
components without disruption. The IBM System Storage DS8000 has
the performance to allow multiple workloads to be easily
consolidated into a single storage subsystem.
Chapter 1. DS8000 characteristics
3
1.2.1 DS8000 models and characteristicsThe DS8000 series
currently has three Turbo models available: the DS8100 Turbo Model
931 and the DS8300 Turbo Models 932 and 9B2. The difference in
models is in the processors and in the capability of storage system
logical partitions (LPARs). The predecessors of the DS8000 series
Turbo models were the DS8000 series Models 921, 922, and 9A2. The
base frame houses the processor complexes, including system memory,
up to 16 host adapters, and up to 128 disk modules. The first
expansion frame houses up to 16 additional host adapters (for a
total of 32) and up to 256 additional disk modules (for a total of
384). A second expansion frame houses up to another 256 disk
modules (for a grand total of 640). The third and fourth expansion
frame houses up to 256 and 128 (for a grand total of 1024)
additional disk modules respectively. There are no additional host
adapters installed for the second, third, and fourth expansion
frames. Table 1-1 provides an overview of the DS8000 models,
including processor, memory, host adapter, and disk specifications
for each model. Note that the DS8300 LPAR model is essentially the
same as the non-LPAR model in terms of hardware components.
However, the LPAR model provides a 50/50, a 75/25, or a 25/75 split
of processors and system memory, and up to half the maximum number
of host adapters and disk modules on each system image.Table 1-1
DS8000 processor models overview DS8100 Turbo model 931 Number of
processor complexes Number of processors per complex Number of
Storage Facility Images (SFIs) 2 2 1 DS8300 Turbo model 932 2 4 1
DS8300 Turbo LPAR model 9B2 2 4 2 (Each SFI has half the total
processor resources) 2.2 GHz 32 GB 64 GB 128 GB 256 GB (Each SFI
has half the total memory) 0-4 Model 9AE 4 - 32 (Each SFI can have
2 - 16) 4
Processor speed Processor Memory options (cache)
2.2 GHz 16 GB 32 GB 64 GB 128 GB
2.2 GHz 32 GB 64 GB 128 GB 256 GB
Expansion frames, minimum - maximum Expansion frame model Host
adapters, minimum - maximum
0-1 Model 92E 2 - 16
0-4 Model 92E 2 - 32
Ports per Fibre Channel Protocol (FCP)/ Fibre Channel connection
(FICON) host adapter Ports per Enterprise Systems Connection
(ESCON) host adapter
4
4
2
2
2
4
DS8000 Performance Monitoring and Tuning
DS8100 Turbo model 931 Disk drive modules (DDMs), minimum
maximum 16 - 384
DS8300 Turbo model 932 16 - 1024
DS8300 Turbo LPAR model 9B2 32 - 1024
Next, we provide a short description of the main hardware
components.
POWER5+ processor technologyThe DS8000 series exploits the IBM
POWER5+ technology, which is the foundation of the storage system
LPARs. The DS8100 Model 931 utilizes the 64-bit microprocessors
dual 2-way processor complexes, and the DS8300 Model 932/9B2 uses
the 64-bit dual 4-way processor complexes. Within the POWER5+
servers, the DS8000 series offers up to 256 GB of cache, which is
up to four times as much as the previous ESS models.
Internal fabricThe DS8000 comes with a high bandwidth, fault
tolerant internal interconnection, which is also used in the IBM
System p servers. It is called RIO-2 (Remote I/O) and can operate
at speeds up to 1 GHz and offers a 2 GB/s sustained bandwidth per
link.
Switched Fibre Channel Arbitrated Loop (FC-AL)The disk
interconnection has changed in comparison to the previous ESS.
Instead of the Serial Storage Architecture (SSA) loops, there is
now a switched FC-AL implementation. This implementation offers a
point-to-point connection to each drive and adapter, so that there
are four paths available from the controllers to each disk
drive.
Disk drivesThe DS8000 offers a selection of industry standard
Fibre Channel (FC) disk drives. There are 15k rpm FC drives
available with 146 GB, 300 GB, or 450 GB capacity. The 500 GB Fibre
Channel Advanced Technology Attachment (FATA) drives (7200 rpm)
allow the system to scale up to 512 TB of capacity.
Host adaptersThe DS8000 offers enhanced connectivity with the
availability of four-port Fibre Channel/FICON host adapters. The 4
Gb/s Fibre Channel/FICON host adapters, which are offered in
longwave and shortwave, can also auto-negotiate to 2 Gb/s or 1 Gb/s
link speeds. This flexibility enables immediate exploitation of the
benefits offered by the higher performance, 4 Gb/s storage area
network (SAN)-based solutions, while also maintaining compatibility
with existing 2 Gb/s infrastructures. In addition, the four ports
on the adapter can be configured with an intermix of Fibre Channel
Protocol (FCP) and FICON, which can help protect your investment in
Fibre adapters, and increase your ability to migrate to new
servers. The DS8000 also offers two-port ESCON adapters. A DS8000
can support up to a maximum of 32 host adapters, which provide up
to 128 Fibre Channel/FICON ports.
1.3 DS8000 performance characteristics overviewThe IBM System
Storage DS8000 offers optimally balanced performance, which is over
six times the throughput of the Enterprise Storage Server Model
800. This throughput is possible, because the DS8000 incorporates
many performance enhancements, such as the dual-clustered POWER5+
servers, four-port 4 Gb Fibre Channel/FICON host adapters, new
Fibre Channel disk drives, and the high-bandwidth, fault-tolerant
internal interconnections.Chapter 1. DS8000 characteristics
5
With all these new components, the DS8000 is positioned at the
top of the high performance category. As previously mentioned in
this chapter, the following components contribute to the high
performance of the DS8000: Redundant Array of Independent Disks
(RAID), array across loops (AAL), POWER5+ processors, RIO, and the
FC-AL implementation with a truly switched FC back end. In addition
to these, there are even more contributions to performance as
illustrated in the following sections.
1.3.1 Advanced caching techniquesThe DS8000 benefits from
advanced caching techniques.
Sequential Prefetching in Adaptive Replacemen