Hardware Components and Architecture

IBM System Storage DS800 implementation for open systHardware components and architectu Copyright IBM Corporation 2007Course materials may not be reproduced in whole or in part without the prior written permissioUnit objectives

After completing this unit, you should be able to: After completing this unit, you should be able to: Discuss the hardware and architecture of the DS Discuss the hardware and architecture of the DS Learn virtualization terminology used for configur Learn virtualization terminology used for configu DS8000 subsystemDS8000 subsystem Describe the physical hardware components and Describe the physical hardware components an Describe the models and features provided by e Describe the models and features provided by e Describe the types of disk arrays that can be con Describe the types of disk arrays that can be co DS8000 subsystemDS8000 subsystem Copyright IBM Corporation 2007Agenda

DS8000 highlights DS8000 hardware components DS8000 architecture DS8000 cache management DS8000 RAS features DS8000 layout and cabling rules Copyright IBM Corporation 2007

DS8000 highlights Copyright IBM Corporation 2007Course materials may not be reproduced in whole or in part without the prior written permissioDS8000 highlights

New processor family - POWER5+ RISC (DS800 DS8100 model 931 DS8300 model 932/9B2 Significant extensions to enable scalability 64K logical volumes (CKD, FB, or mixed) Expanded volume sizes, dynamic volume add/delete I/O adapters Fibre Channel/FICON host adapter (4 Ports, 2 or 4Gb/ ESCON host adapter (2 Ports, 18 MB/s) FC-AL device adapter (4 Ports, 2Gb/s) FC-AL disks 146 GB, 300 GB -10K or 73 GB,146 GB and 300 GB 1 FATA disk drives of 500 GB / 7200 rpm Copyright IBM Corporation 2007 DS8000 series models (2107)

Are: High-performance High-capacity series of disk storage Designed to support continuous operations Redundancy Hot replacement / updates Use IBM POWER5 server technology That is integrated with the IBM Virtualization Engine t Consist of: Storage unit One or two (recommended) Management Consoles (M Graphic User Interface (GUI) or Command Line I allows: Performing logical configurations and Copy Services m functions For high availability, hardware components are re Copyright IBM Corporation 2007 DS8000 models

DS8100 (Model 921 or 931) Processor complex (server): Dual Two-way Up to one expansion frame DS8300 (Models 922, 9A2, 932 or 9B2) Processor complex (server): Dual Four-way Up to two expansion frames Models 9A2 and 9B2 support two IBM System Storage System (LPAR) in one storage unit Expansion frame Model 92E attaches to 921,922, 931 & 932 Expansion frame Model 9AE attaches to 9A2 & 9B2 Model conversionsExisting

Optional c

2-way to 4-way (931 to 932)92x/9Ax b

2-way to 4-way LPAR (931 to 9B2) two-step process 4-way to 4-way LPAR (932 to 9B2) 4-way LPAR to 4-way (9B2 to 932) LPAR expansion to expansion frame (9AE to 92E) Expansion frame to LPAR expansion frame (92E to 9AE) Copyright IBM Corporation 2007 DS8000 R2 and R2.4 highlights

R2: Announcing new features for ALL models: IBM POWER5+ processor: new DS8000 Turbo ( Processor memory for POWER 5+ processor 4Gb FCP / FICON adapter (available on all mode and 93x/9Bx) 500GB 7,200 rpm FATA drives (available on all m 92x/9Ax and 93x/9Bx) 3-site Metro / Global Mirror Earthquake resistance kit Ethernet adapter pair (for TPC RM support) Performance Accelerator (Models 932, and 92E R2.4: Announcing new features for ALL models: 300GB 15,000 rpm Fibre Channel drives HyperPAV (System z) Copyright IBM Corporation 2007 DS8000 hardware overview (old models

2-Way (Model 8100 - 2107-921) Two dual processor servers (POWER5) Up to 128 GB cache (16, 32, 64 or 128 GB) 8 to 64 2Gb FC/FICON 4 to 32 ESCON Ports 16 to 384 HDD Intermixable 73/146 GB 15Krpm, 146/300 GB 10Krpm Physical capacity from 1.1TB up to 115TB (384 x 300 GB DDMs) 4-Way (Model 8300 - 2107-922/9A2)) Two four processor servers (POWER5) Up to 256 GB cache (32, 64, 128 or 256 GB) 8 to 128 2Gb FC/FICON 4 to 64 ESCON Ports 16 to 640 HDD Intermixable 73/146 GB 15Krpm, 146/300 GB 10Krpm Physical capacity from 1.1TB up to 192TB

(640 x 300 GB DDMs)

Expansion frame Model 92E at Expansion frame Model 9AE a Copyright IBM Corporation 2007DS8000 Turbo hardware overview

2-Way (Model 8100 - 2107-931) Two dual processor servers (POWER5+) Up to 128 GB cache (16, 32, 64 or 128 GB) 8 to 64 2Gb FC/FICON 4 to 32 ESCON ports 16 to 384 HDD Intermixable 73/146/300 GB 15Krpm, 146/300 GB 10Kr Physical capacity from 1.1TB up to 192TB (384 x 500 GB FATA DDMs) 4-Way (Model 8300 - 2107-932/9B2) Two four-processor servers (POWER5+) Up to 256 GB cache (32, 64, 128 or 256 GB) 8 to 128 2Gb FC/FICON 4 to 64 ESCON ports 16 to 640 HDD Intermixable 73/146/300 GB 15Krpm, 146/300 GB 10Kr Physical capacity from 1.1TB up to 320TB (640 x 500 GB FATA DDMs)

Expansion frame Model 92E Expansion frame Model 9AE Copyright IBM Corporation 2007

Interfaces to DS8000 (1 of 2)

IBM System Storage DS Storage Manager GUI (DS-SM: Web-based GUI) Program interface to perform logical configurations and Copy S management functions Installed via GUI (graphical mode) or unattended (silent mode) Accessed through Web browser Offers: Simulated configuration (offline) Create, modify, save logical configuration when disconnected Apply to a network-attached storage unit Real-time configuration (online) Logical configuration and Copy Services to a network-attached storag Both DS command-line interface (CLI: script-based) OPEN hosts invoke and manage FlashCopy, Metro and Global Batch processes and scripts Check storage unit configuration and perform specific application f For example: Check and verify storage unit configuration Check current copy services configuration used by storage unit Create new logical storage and Copy Services configuration settings Modify or delete logical storage and Copy Services configuration settin Copyright IBM Corporation 2007 Interfaces to DS8000 (2 of 2)

DS Open application programming interface (API Non-proprietary storage management client applicatio Routine LUN management activities (creation, mapping Creation or deletion of RAID5 and RAID10 volume spac Copy Services functions: FlashCopy, PPRC Helps to integrate configuration management support storage resource management (SRM) applications Enables automation of configuration management thro written applications Complements the use of Web-based DS-SM and scrip Implemented through IBM System Storage Common I Model (CIM) agent Middleware application providing CIM-compliant interfac DS Open API uses CIM technology to manage proprie open system devices through storage management ap DS Open API allows these applications to communica storage unit DS API used by TPC for disk Copyright IBM Corporation 2007 IBM system storage management conso

Focal point for Configuration, Copy Services, Maintenance Dedicated workstation installed inside DS8000 Is the eServer Power5 HMC and can be also called S-DS8000 Automatic monitoring the state of system Notify user and IBM when service required (Call Home Can also be connected to network Enabling centralized management through GUI, CLI, or Called SMC on DS6000 External management console (optional) For redundancy with high availability Copyright IBM Corporation 2007DS8000 management console overview

System storage Management Console: MC Other possible names: HMC or S-HMC On DS6000: SMC Storage Management Console is the focal point for conf Copy Services management, and maintenance activities Dedicated workstation physically located (installed) inside your DS8300 and can automatically monitor the state of your system IBM when service is required. The Management Console can also be connected to your netw centralized management of your system using the IBM System Command Line Interface or storage management software that System storage DS Open API. An external Management Console is available as a optional fea used as a redundant management console for environments wi requirements. Internal Management Console feature code: 1100 External Management Console feature code: 1110 Copyright IBM Corporation 2007 DS8000 management console features

DS8000 management console multiple functions: Local service Interface for local service personnel Remote service Call home and call back Storage facility configuration LPAR management (HMC) Supports logical storage configuration via preinstalled s DS Storage Manager in online mode only Network Interface Server for logical configuration and advanced Copy Services functions Service appliance (closed system) Connection to Storage Facility (DS8000) through private Ethernet networks only Copyright IBM Corporation 2007Hardware management console

AIXAIXUnassignedEthernet

Resources

Partition 1Partition 2Status

POWER5 HypervisorCommand/Response

Virtual Consoles

RAMServiceP5 HMC featu

ProcessorsLPARProcessor

-Volatile Logical partition

Mem RegionsAllocationPermTemp

Dynamic logical

I/O SlotsTables

Capacity and re

Non

management

System status

HMC managem

Service function

update, )

Remote HMC in

Copyright IBM Corporation 2007

DS8000 MC and a pair of Ethernet switc

Every DS8000 base frame comes with a pair of Ethernet switch cabled to the processor complex. The MC has Two built-in Ethernet ports The MC private Ethernet ports shown are configured int Ethernet switch to form the private DS8000 networks. One dual-port Ethernet PCI adapter One PCI modem for asynchronous Call home support. The customer Ethernet port indicated is the primary port to be u the customer network. The empty Ethernet port is normally not used. Corresponding private Ethernet ports of the external MC (FC111 plugged into port 2 of the switches as shown in next foil. To interconnect two DS8000 base frames, FC1190 would provi Ethernet cables to connect from port 16 of each switch in the sec into port 15 of the first frame. If the second MC is installed in the second DS8000, it would remain plug Ethernet switches. Copyright IBM Corporation 2007DS8000 MC and Ethernet switches plug

PCIModem Copyright IBM Corporation 2007DS8000 MC network configuration

Storage Management Console network consists of: Redundant private Ethernet networks for connection to the Stor Customer network configured to allow access from the HMC to secure Virtual Private Network (VPN) Call home to IBM Services is possible through Dial-up ( the MC) or Internet connection VPNs Dial-up or Internet connection VPNs is also available for provide Remote Service and Support Recommended configuration is to connect MC to custom network for support Support will use WebSM GUI for all service actions Downloading of problem determination data favors the use of a network Network connectivity and remote support is managed by Copyright IBM Corporation 2007 DS8000 and DS6000 remote access feat

Call Home (Outbound connectivity): Automatic P Reporting IBM DS6000 and DS8000 are designed with a Call H In the event of a failure, the Call Home function generat with the IBM support organization IBM support determines the failing component and disp customer engineer with the replacement part Remote Service and Support (Inbound connectiv With remote support enabled, IBM technical support c management console to troubleshoot a problem, view and traces interactively This can reduce lag time to send such information to I shorten problem determination time In the case of complex problems, IBM technical suppo engage a specialist quickly to resolve the problems as possible Copyright IBM Corporation 2007 DS8000 MC network topology

DS8000 SubsystemsCustomer

NetworkDMZ

VPNRedundantMCOpt. Firewall

Ethernet

Fabricethethprovided by

ethmodemcustomer

InternetInternetintegratedVPN

Firewall

Proxy

MC :DMZ

Hardware Management ConsoleIBM

DMZ:Demilitarized Zone

VPN:Virtual Private NetworkNetwork

eth:Ethernet Port

IBM Remoteinfrastruc Copyright IBM Corporation 2007How virtual private network (VPN) opera

The VPN server is located behind IBM firewall, which is desi secure The VPN client is located behind the customer firewall The customer has control over opening a connection to access the cl Neither IBM technical support nor non-authorized personnel can acc customers permission The VPN Server security complies with IBM corporate secur ITCS104 This is an IBM internal security measure for all IBM secure data. Remote support (Inbound)

Call home (Outbound)


DS8000 MC remote service security

Server authentication via private/public key: Each MC generates a certificate based on the private will use for Secure Sockets Layer (SSL) based encryp decryption. The IBM SSR transmit the certificate for the installed database maintained within the IBM secure network. IBM personnel then will retrieve the MC specific certifi database and use this key (public key) to establish the session with the MC needed service. SSH over VPN for command line access: Secure Shell (SSH) is used for command line access f IBM location (for example: putty ssh session with publ The SSH daemon on the MC accepts client connectio VPN is up, and a Product Engineer is currently logged SSH client authentication is done through private/publ Copyright IBM Corporation 2007 DS8000 Data flow

The normal flow of data for a write is the following: 1. Data is written to cache memory in the owning server. 2. Data is written to NVS memory of the alternate server. 3. The write is reported to the attached host as having been completed. 4. The write is destaged from the cache memory to disk. 5. The write is then discarded from the NVS memory of the alternate server.

Under normal operation, both DS8000 servers are act processing I/O requests.


DS8000 supported operating systems

IBM: System i: OS/400, i5/OS, Linux, and AIX System p: AIX and Linux System z: z/OS, z/VM, and Linux Intel servers: Windows, Linux, VMware, and NetWare Hewlett-Packard: HP-UX AlphaServer: Tru64 UNIX (April 2005) and OpenVMS (April 20 Sun: Solaris Apple Macintosh OSX SGI Origin Servers: IRIX (April 2005)Check the DS8000 series Interoperability

Fujitsu Primepowerand updated Information on this subject.


Host connectivity: IBM SDD and MPIO

SDD provides the following fun Enhanced data availability Automatic path failover Dynamic I/O load-balancing acros Path selection policies for the hos Concurrent download of licensed With DS6000 and DS8000, SD on the following operating syst Windows NetWare AIX HP-UX Sun Solaris Linux Can coexist with RDAC (DS40 driver) on most operating syst manage separate HBAs. Can not be used with most oth drivers (in other words, Veritas Powerpath)MPIO 2.1.0 PCM (Path Control Module) is also supported for AIX 5.2 ML5 (or later) and AIX 5.3 ML 1(or later).Warning: Default MPIO is not supported: sddpcm file sets are required. Copyright IBM Corporation 2007DS8000 enhancements at a glance

Hardware new everything Processors, adapters, internal paths, frames Increased management flexibility via storage system LPARs Enhanced performance Faster or more of almost everything New patent pending cache algorithms Extended logical device addressing Up to 256 logical subsystems (LSS) with virtualized assignment of physical capacity to LSSs Up to 64K logical volumes Extended connectivity Up to 128 host ports (FC or FICON) or 64 ESCON host ports Up to 510 FCP logins per port and 8,192 per Storage LPAR Up to 512 FICON logical paths per logical control unit image and 128,000 per storage facility image Up to 256 FICON logical path groups per control unit image Up to 2,048 FICON logical paths per port

Improved volume man Nondisruptive volume Up to 64K volumes as Logical Subsystems ( contain volumes for m Larger LUNs (over 2 T 65,520 cylinder (55.6 Improved administratio Online and offline con using a Web-based gr (GUI) Ease-of-use improvem ESS Specialist) Command line interfa control of copy service dependencies on GUI Even more attractive T Ownership More flexible feature li Four year standard w Larger capacity volum Increased opportunitie Copyright IBM Corporation 2007DS8000 overview

2-way2-way4-way or LPAR4-way or

(Base Frame(Base Frane+

+

Only)Only)Expan

Expansion Frame

Fra

Processor (DS8000 turbo)2-way2-way4-way4-w

- System p Squadron Power5+2.2GHz2.2GHz2.2GHz2.2G

Cache16 to 128 GB32 to 256 GB32 to 2

Expansion RackYes (1)--Yes (1 or 2)--

Host Adapters2 to 16

- per HA: 4-port FC / FICON (4Gb)(e.g., 8 to 64 FC /2 to 162 to

- per HA: 2-port ESCONFICON ports)

Device Adapters2 to 82 to 82 to 82 to

(e.g., 1-4 FCALs)

Drives - 73 GB (15K rpm)16 to 12816 to 38416 to 12816 to

- 146 GB (10K & 15K rpm)

(Increments of

- 300 GB (10K & 15K rpm)(increments of 16)(Increments of 16)(incremen

16)

- 500 GB FATA (7200 rpm)

Physical Capacity1.1 to 64 TB1.1 to 192 TB1.1 to 64 TB1.1 to 1

PowerThree-PhaseThree-PhaseThree-phaseThree-P

Dimensions76 x 33.25 x 43 in76 x 66.5 x 43 in76 x 33.25 x 43 in76 x 66.5

- Height x Width x Depth

9.93 sq. ft.19.86 sq. ft.9.93 sq. ft.19.86 s

- Footprint


Hardware components Copyright IBM Corporation 2007Course materials may not be reproduced in whole or in part without the prior written permissioDS8000: primary frame topology

RedundantPowerBBU:BatteryBackupUnits

St

Dense HDD Packaging

16 drives per pack

Dual FC-AL Loop Switches

Point to Point Isolation

Two Simultaneous

Operations per loop

Storage Hardware

Maintenance Console

Processor Complex

IBM eServer p5 570

Dual 2-way or Dual 4-way

4 I/O Enclosure Bays

Each bay supports

4 Host Adapters and

2 Device Adapters

Host Adapter

4 FCP/FICON Ports

or 2 ESCON Ports

FrontDevice Adapter

4 FC-AL Ports


DS8000 terminology

Storage complex A group of DS8000s managed by a single Management Console Storage unit A single DS8000 including expansion frames. Processor complex One P5-570 p-Series server Two processor complexes form a redundant pair Divided into one LPAR (models 931 or 932) or two LPARs (model 9B2) Storage server The software that uses an LPAR Has access to a percentage of resources available on the Processor Complex for the LPAR At GA, this percentage is 50% (model 9B2) or 100 % (models 931 or 932) Storage Facility Image (SFI) Union of 2 LPARs, one from each processor complex Each LPAR hosts one storage server. Copyright IBM Corporation 2007

DS8000 hardware components detail

ProceProcessor complex Copyright IBM Corporation 2007DS8000 processor complex

ProceProcessor complex Copyright IBM Corporation 2007DS8000 processor complex: POWER5 s

CEC Enclosures in the Model 921/931 each have one processor CEC Enclosures in the Model 922/932 & 9A2/9B2 each have two cards (4 Way)

CEC: Computer Electronic ComplexCEC Enclosures contain components such as the processor cards, cache CEC hard drives.

Copyright IBM Corporation 20078-way12-way2-way4-wayIBM Eserver p570

Scales from a 1-way to a 16-way SMP using 4U building blocks Dynamic LPAR and Micro Partitioning Simultaneous Multi-threading (SMT) Self Healing features Bit-Steering (bit sparing) Chipkill ECC (8-bit packet correct) ECC on processor cache memories L3 cache line deletes Memory scrubbing Dynamic processor deallocation Other RAS attributes N+1 power and cooling

Hot-plug PCI16-way

In-place service

First fault data capture

Optimized for storage High I/O bandwidth RIO-2Large robust memories

4K memory allocation

FX0

Thread0 activeFX1

LS0

Thread1 activeLS1

No Thread activeFP0

FP1

BRX

CRL

Execution units utilization


DS8000 processor complex PC

IBM eServer System p POWER5 servers (921, 922, and 2-way 1.5 GHz (3X on ESS 800) 4-way 1.9GHz (6X on ESS 800) New DS8000 Turbo (931, 932, and 9B2) are using POW processors 15 % performance improvement. 2.2 GHz for POWER5+ 2- and 4-way The POWER5 processor supports Logical Partitioning The p5 hardware and Hypervisor manage the real to virtual me provide robust isolation between LPARs. IBM has been doing LPARs for 20 years in mainframes and 3 y At GA, LPARs are split 50-50, so: A 4-way has two processors to one LPAR and two processors t Post GA, 25-75 possible. LPARs only possible in the 4-way P5s (RIO-G cannot be share Cache memory from 16 GB - 256 GB Persistent memory 1 GB to 8 GB: dependent on cache s Battery backed for backup to internal disk (4 GB per server) Copyright IBM Corporation 2007 DS8300 model 8A2/9B2 4-way with LPA

Copyright IBM Corporation 2007Server LPAR concept overview

An LPAR: Uses hardware and firmware to logically partition resources Is a subset of logical resources that are capable of supporting a system Consists of CPUs, memory, and I/O slots that are a subset of t resources within a system Very flexible granularity according to AIX level (5.2, 5.3, and so on No need to conform to physical boundaries of building blocks In an LPAR: An operating system instance runs with dedicated (AIX 5.2) or resources: processors, memory, and I/O slots These resources are assigned to the logical partition The total amount of assignable resources is limited by the phys resources in the system LPARs provide: Isolation between LPARs to prevent unauthorized access betw boundaries Fault isolation such that one LPARs operation does not interfer operation of other LPARs Support for multiple independent workloads, different operating operating system levels, applications, and so on Copyright IBM Corporation 2007 LPAR applied to Storage Facility Images

Processor complex 0

LPAR01LPAR02LPARxyx=Processor complex number y=Storage facility number

Processor DS8300

complex 1 Two eSer

Storage(Proce

Each proc

FacilityLPAR11supports

Image 1Currently,

complex d

StorageLPARs

An LPAR in

FacilityLPAR12complex

Image 2Set of res

exec of a

Delivered AS IS, no need using the MC to configure Copyright IBM Corporation 2007DS8000 processor complex

(P Copyright IBM Corporation 2007DS8000 persistent memory

The 2107 does not use NVS cards, NVS batteries, or N chargers Data that would have been stored in the 2105 NVS card 2107 CEC cache memory A part of the system cache is configured to function as NVS sto In case of power failure, if the 2107 has pinned data in c written to an extra set of 4 disk drives located in each of enclosures Six disk drives total in each CEC: 2 for LIC (LVM Mirrored AIX 5.3 + DS8000 code) 4 for pinned data and other CEC functions During the recovery process the pinned data can be rest extra set of CEC disk drives just as it would have been f cards on the ESS 800 Copyright IBM Corporation 2007 DS8000 I/O enclosure

ProceProcessor complex Copyright IBM Corporation 2007RIO-G and I/O enclosures

Also called I/O drawers 6 PCI-X slots: 3.3V, 133 MHz blind swap Hot-plug: 4 port Host Adapter cards with 4 ports each: FCP or FICON adapter ports 2 Device Adapter cards with 4 ports each: 4 FC-AL ports per card 2 FC-AL loops per card Access to cache via RIO-G internal bus Each adapter has its own PowerPC processor Owned by processors in LPAR SPCN: System Power Control Network Used to control and monitor the status of the power and cooling within the I/O enclosure Cabled as a loop between the different I/O enclosures Copyright IBM Corporation 2007

DS8000 I/O enclosures (aka I/O drawers

Copyright IBM Corporation 2007DS8000 RIO-G port: layout example

Up to in sam

Up to to P5-

Max2000 loop

Each RIO-G port can operate at 1 GHz in bidirectional mode and is capable of pa direction on each cycle of the port. Maximum data rate per I/O Enclosure: 4 GB/sIt is designed as a high performance self-healing interconnect.The p5-570 provides two external RIO-G ports, and an adapter card adds two moreTwo ports on each processor complex form a loop.Figure shows an illustration of how the RIO-G cabling is laid out in a DS8000 that h This would only occur if an expansion frame were installed.The DS8000 RIO-G cabling will vary based on the model. Copyright IBM Corporation 2007DS8000 host adapters HA

ProceProcessor complex Copyright IBM Corporation 2007Host adapter with four fibre channel por

Configured as FCP More FICON logical ESS (1024) versu

One FICON channe devicesOne HA card covers devices that a DS80 (64k -256)Up to 16 HA into a into a DS830016 FICON chann single deviceCurrent System z subsystems limite paths per device Front end of 128 ports for DS8 64 ports for DS81 Copyright IBM Corporation 2007DS8000 FCP/FICON host adapters: HA

Four LC 2Gb or 4Gb FC ports (2 Host Adapter m Auto-negotiates to 1Gbps, 2Gbps, or 4 Gbps Each port independently auto-negotiates to either 1/2 on 2 Gb Host Adapter models or 2/4 Gbps link speed Adapter models. Can be independently configured to FCP or FICO The personality of the port is changeable via the DS S Management tools (GUI or CLI). Ports cannot operate as FCP and FICON simulta FCP port can be Long Wave or Short Wave Short wave ports support a distance of 300m (non-rep Long wave ports support a distance of 10Km (non-rep !! For FCP, configure the ports as follows: Switched point-to-point for fabric topology FC-AL for point-to-point topology Copyright IBM Corporation 2007 DS8000 FICON / FCP host adapter

Processor

QDR1 GHz

PPC

750GX

Fibre Channel

Protocol

EngineBuffer

Data Protection

Data Mover

Fibre ChannelASIC

ProtocolFlash

EngineData Mover

Protocol

QDR

Chipset

PCI-X 64 Bit 133 MHz Four 2 or 4Gbps Fibre Cha New High Function/High P Metadata Creation/Checki Configured at port levelSW or LW Copyright IBM Corporation 2007Performance evolution from the model 800 the DS8000 2 GB host adapters

2 Gb Host Adapter

Copyright IBM Corporation 2007DS8000 4 GB host adapter performance

New 4 Gb Host adapters are designed to improve by 50% single port throu

4 Gb / 2 GB HA performa

Copyright IBM Corporation 2007DS8000 device adapter DA

ProceProcessor complex Copyright IBM Corporation 2007Fibre channel device adapters with 2 Gb

DA perform RAI Offload servers o

Each port has up throughput of pre based DA ports DS8000 AAL (ArLoops): RAID-5 or RAID-spread over two Copyright IBM Corporation 2007DS8000 device adapters

Device adapters support RAID-5 or RAID-10 FC-AL switched fabric topology FC-AL dual ported drives are connected to FC switch in enclosure backplane Two FC-AL loops connect disk enclosures to device ada Array across loops is standard configuration option in D Two simultaneous I/O ops per FC-AL connection possible Switched FC-AL or SBOD (switched bunch of disks) used for b Device adapters are attached to a FC switch with the en 4 paths to each drive: 2 FC-AL loops X dual port access (Detailed later with Storage Enclosures cabling) Copyright IBM Corporation 2007 DS8000 RAID device adapter

PPCProcessor

750FX500 MHz

NVRAMBridgeSDRAM

Fibre Channel

Protocol

Engine

RAID -Buffer

Data Protection -

Fibre ChannelData Mover

ASIC

Protocol Engine

Data Mover

Protocol

Chipset

PCI-X 64 Bit 133 MHz Four 2Gbps Fibre C

New High Function/ High Performance AS Metadata checking Copyright IBM Corporation 2007Performance evolution from the model 800 to the DS8000

Device adapter

Copyright IBM Corporation 200716 drive disk enclosure

DS8000: Disk enclosures installed in pairs: one in front an

Copyright IBM Corporation 2007DS8000 storage enclosures

Enclosure hold 16 DDMs Dual ported FC-AL DDMs 73, 146, or 300 GB DDMs 10 or 15K RPM New FATA Disk drives of 500 GB / 7200 rpm are also supported in the same enclosures. Drives can be added in groups of

8 drives by DS8000 storage enclosure Enclosures act as a FC switch connecting drive using point to point connections

The picture ab simultaneous a switched conn each device ad

Copyright IBM Corporation 2007DS8000 / DS6000 switched FC-AL / FC-A

FC-AL

Loop supports only time Arbitration of com Intermittent failure isIncreasing time as n grows Switched FC-ALDrives attached in p connectionFaster arbitration processing

200 MB/sec exter Improved RAS Switch detects in Intermittent / p Copyright IBM Corporation 2007Switched FC-AL advantages

DS6000 and DS8000 use switched FC-AL technology to link the (DA) pairs and the DDMs. Switched FC-AL uses the standard FC-AL protocol, but the physi is different. The key features of switched FC-AL technology are: Standard FC-AL communication protocol from DA to DDMs Direct point to point links are established between DA and DDM : No arbitration and no performance degradation Isolation capabilities in case of DDM failures provide easy problem determi Predictive failure statistics Simplified expansion: no cable rerouting required when adding another disk The DS8000 architecture employs dual redundant switched FC-A of the disk enclosures. The key benefits of doing this are: Two independent switched networks to access the disk enclosures Four access paths to each DDM in DS8000 architecture (dual switches) Each device adapter port operates independently Double the bandwidth over traditional FC-AL loop implementations Each DDM is attached to two separate Fibre Channel switches. This means that with two device adapters, we have four 2Gb/sec effective d When a connection is made between the device adapter and a di is a switched connection, that uses arbitrated loop protocol. This means that a mini-loop is created between the device adapter and the Four simultaneous and independent connections, one from each device ada Copyright IBM Corporation 2007 DS8000: storage enclosure and DA cabl


Architecture Copyright IBM Corporation 2007Course materials may not be reproduced in whole or in part without the prior written permissioDS8000 frames

Base frame: The base frame contains two processor complexes: eServer p5 Each of them contains the processor and memory that drive all fun DS8000. The base frame can contain up to 8 disk enclosures; each can disk drives. In a maximum configuration, the base frame can hold 128 disk driv The base frame contains 4 I/O enclosures. I/O enclosures provide connectivity between the adapters and the The adapters contained in the I/O enclosures can be either device (DAs or HAs) The communication path used for adapter to processor comple the RIO-G loop. Expansion frames: Each expansion frame can hold up to 16 disk enclosures which drives. In a maximum configuration, an expansion frame can hold 256 dis Expansion frames can contain 4 I/O enclosures and adapters if expansion frame that is attached to either a model 932 or a mo Copyright IBM Corporation 2007 IBM System Storage DS8100 (2-way)

Up to 12Power suppliesHMCIBM eSePOWERBatteries I/O draw Copyright IBM Corporation 2007DS8300 (4-way with two expansion fram

Power supplies

HMC

p5 (POWER5) serversBatteriesI/O drawers Copyright IBM Corporation 2007DS8100 (model 921/931) - 2-way

Up to 16 Host Adapters (HA) FCP/FICON HA: 4 independent ports ESCON HA: 2 ports Up to 4 Device Adapter (DA) pairs DA pairs 0 / 1 / 2 / 3 Automatically configured from DDMs Maximum configuration (384 DDMs) DA pair 0 = 128 DDMs DA pair 1 = 64 DDMs DA pair 2 = 128 DDMs DA pair 3 = 64 DDMs Balanced configuration at 256 DDMs: in other words, 64 DDMs per DA pair DA (card) plugging order: 2 / 0 / 3 / 1 Copyright IBM Corporation 2007

2

200C0C10/1 1/02/3 3/2DS8300 (Models 922/932 and 9A2/9B2) -

Up to 32 Host Adapters FCP/FICON HA: 4 independent ports ESCON HA: 2 ports Up to 8 DA pairs DA pairs 0 to 7 Automatically configured from DDMs Maximum configuration (640 DDMs) DA pairs 1, 3-7 = 64 DDMs DA pairs 2, 0 = 128 DDMs Balanced configuration at 512 DDMs: in other words, 64 DDMs per DA pair DA (card) pair plugging order: 2 / 0 / 6 / 4 / 7 / 5 / 3 / 1

26

26

04

04

C07

C17

5

5

0/11/04/55/4

2/33/26/77/6


DS8000 storage enclosure connectivity


DS8000 cache managemenSARC: Simplified adaptive replacement cache Copyright IBM Corporation 2007Course materials may not be reproduced in whole or in part without the prior written permissioSequential prefetching inadaptive replacement cache (SARC)

SARC basically attempts to determine four things When data is copied into the cache Which data is copied into the cache Which data is evicted when the cache becomes full How the algorithm dynamically adapts to different wor SARC uses: Demand paging for all standard disk I/O Sequential pre-fetch for sequential I/O patterns Copyright IBM Corporation 2007DS8000 caching

Best caching algorithms in industryBenefits of adaptiv

Over 20 years experience

Simplified Adaptive Replacement Cache (SARC)

Self-Learning algorithms

Adaptively and dynamically learn what data

should be stored in cache based upon the

recent access and frequency needs of the hosts

Adaptive Replacement Cache

Most advanced and sophisticated algorithms to

determine what data in cache is removed to

accommodate newer data

Pre-fetching

Predictive algorithm to anticipate data prior to a

host request and loads it into cache1

Benefits

Leading performanceRatio0.8

Been proven to improve cache hit by up to0.6

100% over previous IBM caching algorithms and

Hit

improve I/O response time by 25%0.4

Cache

More efficient use of cache

Intelligent caching algorithm profiles host access0.2

patterns to determine what data is stored0

Need less cache than competitors

064128

Cache Siz

Lower cache-to-backstore ratios with outstanding service timNimrod Megiddo and Dharmendra S. Modha, "Outperforming LRU with an Adaptive Replacement Cache Algorithm," 2004. Copyright IBM Corporation 2007

DS8000 RAS features (Reliability, availability, an serviceability) Copyright IBM Corporation 2007Course materials may not be reproduced in whole or in part without the prior written permissioDS8000 hardware components detail

Copyright IBM Corporation 2007Processor complex RAS

Processor complex has the same RAS features o which is an integral part of the DS8000 architectu IBM Server p5 system main RAS features: First Failure Data Capture Boot process and operating system monitoring Environmental monitoring Self-healing Memory reliability, fault tolerance and integrity Error Checking Correction (ECC) Memory scrubbing and thresholding N+1 redundancy Resource deallocation Concurrent maintenance Copyright IBM Corporation 2007Server RAS (1 of 2)

The DS8000 employs similar methodology to the provide data integrity when performing write oper server failover. Metadata check: The metadata is checked by various components to validate the integrity of the data as it m the disk system or sent back to the host. Server failover and failback: LSS and server affinity: LSS with even number have an affinity with server 0 LSS with odd number have an affinity with server 1 When a host operating system issues a write to a logica DS8000 host adapter directs that write to the server tha which that logical volume is a member. Copyright IBM Corporation 2007Server RAS (2 of 2)

Under normal operation, both DS8000 servers are activ I/O requests Each write is placed into the cache memory of the server ownin also into the NVS memory of the alternate server. Failover: In case of one server failure, the remaining ser take over all of its functions RAID arrays which are connected to both servers can be acces device adapters of the remaining server. Since the DS8000 has only one copy of data in cache of remai now take the following mechanism: It de-stages the contents of its NVS to the disk subsystem. The NVS and cache of remaining server are divided in two, half fo half for the even LSSs. Remaining server now begins processing the writes (and reads) fo Failback: When the failed server has been repaired, failb activated It completes in less than 8 seconds and is invisible to the attac Copyright IBM Corporation 2007 Hypervisor storage image independen

Logical

view:Storage Facility image 1Storage Facility

virtual

StorageLICRIO-GLICLICRIO-G

I/OI/OI/O

MemoryMemoryMemory

Facility

ProcessorProcessorProcessor

imagesLPAR Hypervisor

Physicalview: physical storage unit

takes part oftakes part ofRIO-GI/OI/OMemoryMeProcessor P Copyright IBM Corporation 2007

Server failover

Normal flow of data for a write: 1. Data is written to cache memory in the owning server. 2. Data is written to NVS memory of the alternate server. 3. The write is reported to the attached host as having been completed. 4. The write is destaged from the cache memory to disk. 5. The write is then discarded from the NVS memory of the alternate server. After a failover, remaining server is processing all I/Os with cache and NVS divided by two, one for odd LSSs and one for even LSSs. Copyright IBM Corporation 2007

NVS for odd LSSs

Cache memory for even LSSsServer 0

NVS for odd LSSsCache memory for even LSSsServer 0Failover

NVS recovery after complete power loss

DS8000 Battery Backup Units (BBUs) Both power supplies stopped Batteries not used to keeping disks spinning Scenario at power-off All HA I/O blocked Each server copies NVS data to internal disk Two copies per server When copy process complete, each server shuts down AIX When AIX shutdown complete for both servers (or time out expires powered down Scenario at power-on Processor complexes power-on and perform power-on self test Each server boots up During boot-up, server detects NVS data on its disks and destage When battery units reach a certain level of charge, the servers com NVS contents preserved indefinitely Note: the servers will not come online until the batteries charged. Copyright IBM Corporation 2007 Host connection availability

On DS8000 host adapters are shared between th Unlike the DS6000 which uses the concept of preferre It is preferable for hosts to have at least 2 conne separate host adapters in separate I/O enclosure This configuration allows the host to survive a hardwa component on either path. This is also important because during a microcode up enclosure may need to be taken offline. Multi-pathing software: Subsystem Device Driver (SDD) is able to manage bo and preferred path determination. SDD is usable with ESS800, DS6000, DS8000, or SVC. MPIO PCM is also supported with AIX 5.2 ML5 (or late ML1 (or later). Copyright IBM Corporation 2007 Disk subsystem (1 of 2)

RAID5 and RAID10 RAID5 (7+P or 6+P+S) or RAID10 (2x4 or 2x3 + 2S) DS8000 does not support non-RAID configurations Spare disk creation: A minimum of one spare is created for each array site following conditions are met: A minimum of 4 spares per DA pair A minimum of 4 spares of the largest capacity array site A minimum of 2 spares of capacity and RPM greater tha fastest array site of any given capacity on the DA pair Floating spare: The DS8000 microcode may choose to migrate new s more optimal position to better balance the spares acr pairs, the loops, and the enclosures. (Useful after a drive replacement that became a spare d Copyright IBM Corporation 2007 Disk subsystem (2 of 2)

Each DDM attached to two FC switches Each disk has two separate connections on the backp Each DA has a connection to the two switches Hot pluggable DDMs Predictive Failure Analysis (PFA) Failures anticipation Disk scrubbing All disk sectors periodically read and bad bits correcte Copyright IBM Corporation 2007Power and cooling

Completely redundant power and cooling in N+1 Battery Backup Units (BBU) Used for NVS (part of the servers memory) Can be replaced concurrently Rack Power Control cards (RPC) 2 RPC cards for redundancy Each card can control power of an entire DS8000 Power fluctuation protections DS8000s tolerate a momentary power interruption fo 30ms. After that time, servers start copying content of NVS to disks. Copyright IBM Corporation 2007 Microcode update

Concurrent code update (since Bundle level 324 Management console can hold 6 different versions of Each server can hold 3 different versions of code Installation process: Internal Management Console (MC) code update New DS8000 LIC downloaded on the Internal MC LIC uploaded from MC to each DS8000 server interna New firmware can be loaded from MC directly into eac May require server reboot with failover of its Logical Sub other server Update of servers operating system and LIC Each server updated one at a time with failover of its Lo to the other server Host adapters firmware update Each adapter impacted for less than 2.5 s which should connectivity Longer interruption managed by hosts multi-pathing sof Copyright IBM Corporation 2007 Management console

Redundant Ethernet switches Each switch used in a separate Ethernet network with private IP addresses assigned in networks 172.16/16 and 172.17/16 192.168..16.x and 192.168.17 10.0.16.x and 10.0.17.x Redundant Management Console Each DS8000 can be connected via the redundant Eth to both Management Consoles. Copyright IBM Corporation 2007

Backup slides Copyright IBM Corporation 2007Course materials may not be reproduced in whole or in part without the prior written permissio

DS8000 I/O enclosure layou and cabling rules Copyright IBM Corporation 2007Course materials may not be reproduced in whole or in part without the prior written permissioI/O enclosure numbers

Copyright IBM Corporation 2007Model 921/931 two I/O enclosures

I/O enclosure 3Server 0

(EVEN LSS)Loop 0

RIO-G ports

I/O enclosure 2This configuration will not be available at GA.(At GA a model 921 will always ship with four I/O enclosures). Copyright IBM Corporation 2007Model 921/931 four I/O enclosures

I/O enclosure 0I/O enclosure 3

Server 0

(EVEN LSS)Loop 0

RIO-G ports


Four enclosures is the maximum number for a model 921. Copyright IBM Corporation 2007Model 922/932 four I/O enclosures

I/O enclosure 0

Server 0Loop 0

EVEN LSSs

RIO-G ports

I/O enclosure 3

I/O enclosure 1

Loop 1

I/O enclosure 2On Loop 0 on 921

A model 922 has extra hardware to support a second RIO-In this configuration. Copyright IBM Corporation 2007Model 922/932 eight I/O enclosures


Server 0Loop 0

EVEN LSSs

RIO-G ports



Loop 1


Eight enclosures is the maximum number for a model 922. More enclosures need more RIO-G ports.

To get more RIO-G ports we need more processor complexes Copyright IBM Corporation 2007Model 9A2/9B2 four I/O enclosures

I/O enclosure 0

Server 0!!! Opposite / 922

EVEN LSSsLoop 0 belongs to SFI 1Loop 0

(two instances)

RIO-G ports

I/O enclosure 1

I/O enclosure 3

Loop 1 Loop 1 belongs to SFI 2

I/O enclosure 2

The 9A2 is split into two storage facility images (SFIs).Each SFI controls one RIO-G loop and all the enclosures and adapters Resources cannot be shared between SFIs.

Copyright IBM Corporation 2007Model 9A2/9B2 eight I/O enclosures

!!! Opposite / 92


Server 0

EVEN LSSsLoop 0 belongs to SFI 1 Loop 0

(two instances)

RIO-G ports



Loop 1 Loop 1 belongs to SFI 2


The 9A2 is split into two storage facility images(SFIs).

Each SFI controls one RIO-G loop and all the enclosures and adapters Resources cannot be shared between SFIs.

Copyright IBM Corporation 2007Device adapter pair layouts and server front view

DA pairs represent two device adapters.

One DA in each pair is owned by server 0 and the other DA in that pair is owned by server 1.

DAs in even numbered I/O enclosures belong to server 0. DAs in odd numbered I/O enclosures belong to server 1.

The 'outside' slots always get populated firstEven numbered I/O enclosures are always cabled closer to server 0. Odd numbered I/O enclosures are always cabled closer to server 1.I/O Enclosure numbers

DA pair num Fro(the numbers don't cha which they are inBase Frame

Rack 1Front view

Complex 0

Complex 1

10 11 0

32 33 2

The numbers in the I/O enclosures are the DA pairs in those"0 1" means that one card from DA pair 0 and one card from this enclosure.Warning: Because this is a front view, the left-hand card is in enclosure and the right- hand card is in slot 3 of the enclosure

Copyright IBM Corporation 2007DA plug order and affinity model 921/931

Base frameExpansion frame

Rack 1Rack 2

23

2

3

01

01

2

Complex 02

0

Complex 10

2nd 4th 4th 2nd

0110

1st 3rd3rd 1stPlug order

2332

DA pair

The numbers in the storage which storage enclosure pai which DA pairs."2" in the storage enclosure front and rear storage enclos is attached to DA pair 2.The plug order shows the o pairs are added to the DS80"1st 3rd" means that the left-first and the right hand DA isSo using DA pair numbers, t DA pair 2 then 0 then 3 thenThe numbers in the I/O enclo pairs in those enclosures."2 3" means that one card fr card from DA pair 3 are in thiBecause this is a front view, t in slot 6 of the enclosure and is in slot 3 of the enclosure.The DA pairs are added in t

Copyright IBM Corporation 2007DA affinity models 922/932, 9A2 and 9B2 Base FrameExpansion frameExpansion fra

(Rack 1)(Rack 2)(Rack 3)

263

263

041

041

72

Complex 072

50

Complex 150

2nd 8th8th 2ndPlug ord

4th 6th6th 4th

01104554

1st 7th7th 1st3rd 5th5th 3rdDA pair

23326776

Looking at this chart explains why the DA plug order is different machines because the 1st expansion frame cables to the I/O en that frame. This also means a model upgrade requires re-cablinThe DA pairs are added in this order: 2, 0, 6, 4, 7, 5, 3, 1

Copyright IBM Corporation 2007Resource division model 9A2/9B2 frBase FrameExpansion FrameExpansion

26

26

04

04

7

Complex 0SFI1 SFI27

5

Complex 1 SFI1 SFI25

2nd 8th 8th 2ndLoop 04th 6th6th 4thPlug ord

0 11 04 554DA pair

1st 7th7th 1stLoop 13rd 5th5th 3rd

2 33 26 776

Each Storage Facility Image (SFI) is composed of an LPAR on e processor complex and up to four I/O enclosures. DA ownership therefore storage enclosure ownership, is derived from I/O enclosu ownership.

SFI resources are divided by color, blue for SFI1 and purple for SF

Green and yellow are used to distinguish server 0 and server 1 on following pages. Dont confuse servers and SFIs. Copyright IBM Corporation 2007I/O enclosure slot numbering rear view

Host adapterHost adapterDevice adapterRIO-G portsHost adapterHost adapterDevice adapter

Slot1237456

This is a view o the rear of the

The ports on an numbered 0 toThe location cis in the format:

Rack - enclosue.g. the top por 1 of I/O enclosuR1- I1-C1-T0R1 - Rack 1I1 - Enclosure 1C1 - Card 1T0 - Port 0Warning: port R1-I1-C1-T0 is displayed as I0000 under dscli (lsioport comman (Full details in next slides) Copyright IBM Corporation 2007I/O ports numbering DS CLI lsioport display

I0000I0010I0030I0040

I0001I0011I0031I0041

I0002I0012I0032I0042

I0003I0013I0033I0043

Slot 1Slot 2Slot 4Slot 5

Enclosure 1

I0200I0210I0230I0240

I0201I0211I0231I0241

I0202I0212I0232I0242

I0203I0213I0233I0243


Enclosure 3

I0100I0110I0

I0101I0111I0

I0102I0112I0

I0103I0113I0

Slot 1Slot 2Sl

Enclosure

I0300I0310I0

I0301I0311I0

I0302I0312I0

I0303I0313I0

Slot 1Slot 2Sl

Enclosure

Copyright IBM Corporation 2007I/O ports numbering DS CLI lsioport display expansion frame

I0400I0410I0430I0440

I0401I0411I0431I0441

I0402I0412I0432I0442

I0403I0413I0433I0443


Enclosure 5

I0600I0610I0630I0640

I0601I0611I0631I0641

I0602I0612I0632I0642

I0603I0613I0633I0643


Enclosure 7

I0500I0510I0

I0501I0511I0

I0502I0512I0

I0503I0513I0

Slot 1Slot 2Sl

Enclosure

I0700I0710I0

I0701I0711I0

I0702I0712I0

I0703I0713I0

Slot 1Slot 2Sl

Enclosure

Copyright IBM Corporation 2007Example 1 model 921/931 one DA pair - r

DA plug order : 2

0

r

eFibreFibreESCONESCONFibreFibreESCONES

vchannelchannelchannelchannel

rSlot 1Slot 2Slot 3Slot 4Slot 5Slot 6Slot 1Slot 2Slot 3Slot 4Slo

eI/O enclosure 0I/O enclosure 3

SRIO-GRIO-GRIO-G

RIO-G

RIO-GRIO-GI/O enclosure 2RIO-GRIO-GI/O enclosure 1

Owns allSlot 1Slot 2Slot 3Slot 4Slot 5Slot 6Slot 1Slot 2Slot 3Slot 4Slot

even LSS

FibreFibreDAFibreFibre

logical

volumes.channelchannelpair 2ESCONESCONchannelchannelESCONESC

This example depicts a model 921 with one DA pair. Slots 3 and 6 are always reserved for DA pairs.The first DA pair is pair number 2. This configuration supports 64 DDMs.In all these examples each I/O enclosure has 2 x Fibre channel and 2 x ESCON host adapters (it is just an example). Copyright IBM Corporation 2007Example 2 model 921/931 two DA pairs -

DA plug order : 2

0

rReservedReserved

e

FibreDAFibreFibrefor DAESCONESC

FibreESCONESCONfor DA

vchannel channelpair 0pair 1channel channelpair 3

rSlot 1Slot 2Slot 3Slot 4Slot 5Slot 6Slot 1Slot 2Slot 3Slot 4Slo


SRIO-GRIO-GRIO-G

RIO-G


Owns allSlot 1Slot 2Slot 3Slot 4Slot 5Slot 6Slot 1Slot 2Slot 3Slot 4Slot

even LSS

FibreFibreDAReservedFibreFibreReserved

logical

volumes.channelchannelpair 2ESCONESCONfor DAchannelchannelfor DAESCONESC

pair 3pair 1

This example depicts a model 921 with two DA pairs, which means we in each I/O enclosure.This configuration supports 128 DDMs. At this point the base rack is full of DDMs.Additional capacity requires an expansion rack and more DAs. Copyright IBM Corporation 2007Example 3 model 921/931 - four DA pairs -

DA plug order : 2

0

r

eFibreFibreDAESCONESCONDAFibreFibreDAESCONESC

vchannel channelpair 0pair 1channel channelpair 3

rSlot 1Slot 2Slot 3Slot 4Slot 5Slot 6Slot 1Slot 2Slot 3Slot 4ot

Slot


SRIO-GRIO-GRIO-G

RIO-G


Owns allSlot 1Slot 2Slot 3Slot 4Slot 5Slot 6Slot 1Slot 2Slot 3Slot 4Sl

even LSS

FibreFibreDADAFibreFibreDA

logical

volumes.channelchannelpair 2ESCONESCONpair 3channelchannelpair 1ESCONESC

This example depicts a model 921 with four DA pairs. DA pair 3 was added before DA pair 1.The DDMs on DA pairs 3 and 1 go into the expansion rack.When DA pairs 3 and 1 each have 64 DDMs, an additional 64 DDMs ca DA pair 2 and then 64 more DDMs on DA pair 0.At that point the expansion rack is full with 384 DDMs. No more capacity is possible for this model. Copyright IBM Corporation 2007Example 3 model 921/931 four DA pairs:

Server 0Server 1

I/O Enclosure 2

I/O Enclosure 0I/O Enclosure 1

DA Pair100132

DA pair 3

DA pair 1DA pair 0

0000

15151515

FrontRearFrontRear

0000

15151515

FrontRearFrontRear

00

1515

FrontRear

00

1515

FrontRear

00

1515

FrontRear

00

1515

FrontRear

Base Frame Full : 128 DDMsAdd Expansion FrameModel 921 Full : 384 DDMs Copyright IBM Corporation 2007Example 4 model 922/932 two DA pairs

Copyright IBM Corporation 2007Example 5 model 922/932 four DA pairs (1 exp frame)

Copyright IBM Corporation 2007Example 5 model 922/932 six DA pairs (1

Copyright IBM Corporation 2007Example 5 model 922/932 height DA pair (2 exp frame)

Copyright IBM Corporation 2007Example 6 models 922/932 and 9A2/9B2 eight DA pair

Base FrameExpansion frameExpansion frame

(Rack 1)(Rack 2)(Rack 3)

263

263

041

041

72

Complex 072

50

Complex 150

Plug order

2nd 8th8th 2nd4th 6th6th 4th

01104554

1st 7th7th 1st3rd 5th5th 3rdDA pair

2 3326 77 6

This exemple depicts a model 922 or 9A2 with eight DA pairs Each DA pair has 64 DDMs (2 front & 2 rear Storage Enclosure)When DA pairs 3 & 1 each have 64 DDMs, an additional 64 DDMs can b DA pair 2 and then 64 more DDMs on DA pair 0At this point, the base frame and the 2 expansion frames are full with 640 No more capacity is possible on these models. Copyright IBM Corporation 2007LPAR effect

Because RIO-G loops cannot be shared between facility images (SFIs) a model 922/932 is cabled model 9A2/9B2. On a model 922/932 there is only one SFI, which loops. This means we can split the DA pairs across the loops maximum performance. In a model 9A2/9B2, each SFI 'owns' one loop. This means all DA pairs must be on the same loop. That's why some of the examples look very simil small differences are very important. Copyright IBM Corporation 2007 Example 7 model 9A2/9B2 two DA pairs

Copyright IBM Corporation 2007Example 8 model 9A2/9B2 four DA pairs

Copyright IBM Corporation 2007Example 9 model 9A2/9B2 eight DA pairs rear view


DS8000 ordering consideratio Copyright IBM Corporation 2007Course materials may not be reproduced in whole or in part without the prior written permissioInfrastructure overview

Disk Enclosure Pair (#12x0) Holds 32 drives of the same type, 1 front and 1 rear I/O Enclosure Pair (aka: I/O Drawer, #1300) Device Adapters and Host Adapters are installed here Device Adapter Pair (#30x1) Attach storage devices via FC/AL interface RIO-G cables (#131x) Cabling from I/O drawers to processors Disk Drive cables (#121x) Cables from disk drives to Device Adapters Disk drives are shipped in increments of 16 drives of the same type (#2xy6 - #2xy7 **) Capacity and rpm eConfig will generate the appropriate number of these infrastructure features In general, dependent on number and type of disk driv ** for on-demand Copyright IBM Corporation 2007 DS8000 Turbo - feature details

Processor memory features Processor memory features will be released as f POWER5+ Processor Memory for Models 931 and 92 16GB processor memory FC #4011 32GB processor memory FC #4012 64GB processor memory FC #4013 128GB processor memory FC #4014 POWER5+ Processor Memory for models 932, 9B2 9 32GB processor memory FC #4112 64GB processor memory FC #4113 128GB processor memory FC #4114 256GB processor memory FC #4115 For 93x/9Bx models a minimum of one of the proces features is required at order For 92x/9Ax models Minimum of one or a conversio #150x is ordered Chargeable feature, plant and field install, and will car monthly maintenance charge Feature conversions will also be made available. Copyright IBM Corporation 2007 Disk drive / disk enclosure product stru

A disk enclosure holds 16 drives and must be installed in pairs One enclosure is installed / accessed from the front of the machine, while the other enclosure is installed / accessed from the rear An installed disk enclosure pair must be fully populated with disk drives or dummy carriers, or both Disk drives are shipped in increments of 16 drives of the same type (capacity and rpm): drive set A drive set is installed into a single disk enclosure pair and equally spread across the two enclosures comprising the pair 8 drives in the front enclosure and 8 drives in the rear enclosure For initial GA, drive intermix within a disk enclosure pair is not offered Drive intermix (field install only) evaluated as a post-GA offering FrontRearDisk enclosure pair (2 x 16 = 32 drives total)

16 drives same type

16 drives same type as first 16 at GA


Plugging rules

Same capacity / same rpm No gaps allowed Install top-down starting with top disk enclosure pair in Complete partially filled disk enclosure pair before inst next enclosure pair Different capacity or different rpm If a partially filled disk enclosure (in other words, 16 D enclosure fillers installed), then install different capacit next disk enclosure pair Note: These rules apply independently to each S Image. Thus, if non-LPAR then applies to the full but if LPAR then applies independently to the dis for each LPAR. Copyright IBM Corporation 2007Example: 8300 4-ways base frame FICON 128 73GB / 15krpm

ProductDescriptionQty

2107-922 System storage DS83001

0700OEL Indicator1

08055.1 to 10.0 TB capacity1

0900Non-Standby CoD1

1050Battery Assembly3

1091Line Cord (EMEA)1

1100Management Console Internal1

1210Disk Enclosure Pair4

1211Disk Drive Cable Group 11

1300I/O Enclosure Pair2

1313RIO-G Cable Group 31

14219 um Fibre Cable (LC/SC)48

201673 GB 15K Drive Set8

3011Device Adapter Pair2

32112Gb LW FCP/FICON Adapter12

410364 GB Processor Memory1

9091AC Voltage: 380V - 480V1

9100MC Keyboard - US English1

Copyright IBM Corporation 2007Checkpoint

1. T/F. The DS8000 can be equipped with 2-way or 4-way Power and provide redundant control of the storage arrays. 2. T/F. The DS8000 Model 932 provides the capability to divide t logical partitions called LPARs which divides the hardware com separate controller which can operate independently. 3. T/F. The DS8000 System has scalable I/O bays which adds fr adapters and back-end device adapters plus internal bus band within each bay. 4. The DS8000 System supports the following disk devices: a. 73 GB 15 KRPM disk drives b. 146 GB 10 KRPM disk drives c. 300 GB 10 KRPM disk drives d. 500 GB 7.2 KRPM disk drives e. All of the above 5. T/F. The DS8000 System can be configured offline using a Si Storage Manager interface. Copyright IBM Corporation 2007 Checkpoint solutions

1. False, the DS8000 uses Power5 processors. 2. True, the LPAR function is a feature of the Power5 pro the Model 932 can be split into two LPARs. 3. True, the RIO-G bays are a high-bandwidth I/O enclos connect hosts and backend storage the processor co 4. D, all of the above 5. True, the DS8000 Storage Manager supports offline c Copyright IBM Corporation 2007Unit summary

Having completed this unit, you should be able to: Having completed this unit, you should be able to: Discuss the hardware and architecture of the DS Discuss the hardware and architecture of the DS Learn virtualization terminology used for configur Learn virtualization terminology used for configu DS8000 subsystemDS8000 subsystem Describe the physical hardware components and Describe the physical hardware components an Describe the models and features provided by e Describe the models and features provided by e Describe the types of disk arrays that can be con Describe the types of disk arrays that can be co DS8000 subsystemDS8000 subsystem Copyright IBM Corporation 2007

Hardware Components and Architecture

Documents