IBM® Edge2013 - Introduction to IBM XIV Storage System

© 2013 IBM Corporation

Session 1029 Introduction to IBM XIV Storage System and What Makes It Unique

Jack Arnold - IT Management Consultant

15 May 2013


IBM Presentation Template Full Version

2

XIV – IBM System Storage™

� Founders are 5 graduates of Talpiot

� 14th class (XIV=14 in Roman numerals)

� Chief Architect and IBM Fellow - Moshe Yanai –Formerly EMC’s CTO–Led the development of the Symmetrix and DMX systems

� Founded in 2002

� 1st customer system installed in early 2005http://funds.globes.co.il/serveen/globes/docview.asp?did=1000394326&fid=942



3

Do It Yourself Supercomputers

“PlayStations power Air Force supercomputer”http://news.cnet.com/8301-17938_105-20025680-1.html

http://www.af.mil/news/story.asp?id=123233543

(Credit: Air Force Research Laboratory)

“1716 Sony PS3 processors…35th or 36th most powerful supercomputer in the world”



Supercomputers

� Climate change research gets petascale supercomputer

� 1.5-petaflop IBM Yellowstone system runs 72,288 Intel Xeon cores

4

http://www.computerworld.com/s/article/9232382/Climate_change_research_gets_petascale_supercomputer

Image courtesy of



5

Cache

� 24GB of per module x 15 modules

� 360 GB of cache – 45 GB for Linux = 315 GB usable

� More than 1 GB of cache per usable TB!



6

Cache Is Distributed Not Centralized



7

IBM XIV Storage System – 1TB

� 180 disks per rack (12 * 15), each disk is 1TB 7200 RPM

� Single rack provides 79TB usable capacity



8

Full Rack Power Consumption

–7.7 KW, 180 1 TB spindles, 79 TB usable capacity, 97 W / TB–7.3 KW, 180 2 TB spindles, 161 TB usable capacity, 45 W / TB–6.7 KW, 180 2 TB spindles, 161 TB usable capacity, 42 W / TB–7.0 KW, 180 3 TB spindles, 243 TB usable capacity, 29 W / TB



XIV SSD Solution

� High capacity SSDs used as secondary read cache

– 480 GB enterprise-class SSD device added to each module

– 7.2 TB of cache per full rack

– Housed in PCI caddy in rear of module

– Scales with the system – 6 to 15 SSD drives

– Must be added to all modules together

9



10

Why 7200 RPM?



11

Storage Efficiency

0

100

200

300

400

500

100TB 200TB 400TB 800TB

15K 3.5"

7200 3.5"

Tape

RelativePower @ typical R/W operation load

Source: IBM data, 2009



12

IBM XIV Storage System – The Quest

… at 7200 RPM (SATA or NL SAS) cost

Delivering Fibre Channel performance…



13

IBM XIV Storage System – The Promise

Tier 1

- Performance

- Reliability

- Capability

At a Tier 2 price!

Or put another way



14

Industry Standard Components

Processors – Intel Quad-Core (1 or 2)

Connectivity – 1Gb Ethernet or Infiniband

Disks – standard 7200 RPM (SATA or NL SAS)

Rack – was industry standard (APC), now IBM T42



15

Lower TCO

XIV Brand X

(such as E** or H**)

Remote Replication

$$$$ or €€€€

Point in Time Copy

$$$$ or €€€€

Multi-path Drivers

$$$$ or €€€€

Management Software

$$$$ or €€€€

Host Attach $$$$ or €€€€

No additional charge




No additional charge(we use native drivers)



16

LUN Creation

�LUNs are allocated in 17 GB chunks

�Volumes can be dynamically resized (increased only)

–Primary and Secondary in a synchronous mirroring relationship will be resized

�User does not plan the layout of the volumes relative to physical drive resources



17

LUN Creation

Decisions

�What type of disk (SATA, Fibre Channel)

�What size disk (73GB, 146GB, 300GB, 450GB)

�What speed disk (10K RPM, 15K RPM)

�What RAID level?

�Which set of disks (array)?

�Do I need tiers of storage?

XIV – tierless (tear-less!) storage



18

On-going Management Effort: None

All spindles are always utilized

–No management effort or optimization

–No Hot-Spots



19

Global Hot Spare Storage Capacity Concept

� Traditional approach

– Dedicated Hot spare drives

� With the IBM XIV Storage System, each disk is utilizing 95% of its space

– The system provides enough hot spare capacity to tolerate a sequence of three disk failures and one module failure, all while still providing redundancy

� Upon a failure, the system returns to redundancy



20

Reliability



21

XIV Disks Drives 1-12

Drawers 1-15

Break data into1MB chunks



22

How Do You Get From 180TB to 79TB?

180 TB-15 drives for redundant capacity for drive and drawer failures- 7 TB for a year’s worth of performance data, system dumps, metadata=158

158 / 2 (for duplicating data) = 79



23

All Drives Have Same Amount of Activity

Makes predictive analysis very good



24

Aggressive Monitoring of Drives

!



25

Aggressive Monitoring of Drives

X

80% of drives returned have been administratively failed



26

Performance



27

XIV Disks

Drawers 1-15

180 MB LUN(not possible,illustrative purposes only)

Drives 1-12



28

Disk Rebuild Times – 1TB Drive

Hours

XIV Other

30 minutes!(30-35 minutes is maximum on afull system)



29

Rebuilds Take a Long Time

RAID 5: theory & realityhttp://www.zdnet.com/blog/storage/raid-5-theory-reality/983

“A 2 TB drive might take 40 hours or more to restore.”



30

Traditional RAID-5 (IBM, EMC, HDS) Rebuild

Data Data Data Data Data Data Data Spare

Data Data Data Data Data Data Data

X

X

Cache6 drives read… 1 drive writes



31

XIV Disks Participating in “Rebuild” 179!

X

(168 for reads)

Technically, not a rebuild but rather a “re-gaining redundancy”)



32

XIV Disks Drives 1-12

Drawers 1-15

Data is never duplicated inside the same drawer

180-12

168



33

Typical Rebuild



34

Smart “Rebuild” (regaining redundancy)



35

Adding more Storage Capacity

Data is automatically rebalanced



36

Managing StorageStorage Pool

Total Capacity

Consistency group



37

Storage Pools – Regular and Thin Provisioning

256 pools maximum



38

Creating Point in Time Copies (Snapshots)



39

Restore From Point in Time Copies – Brand X



40

Restore From Point in Time Copies – XIV



41

XIV – Space Efficient Snaps



42

IBM XIV Storage System GUI



Value of Redundancy

43



44

IBM XIV Data Migration Feature

Migrates data from brand x boxes to XIV- One outage to connect the box- Then transparent migration of LUNs (can be thin-provisioned)- Tunable: 300GB/hr -> 1TB/hr

EMCHDS



45

IBM XIV Data Migration Feature

One way transfer from brand x boxes to XIVThen two way communication with server- 512 LUNs from 4 different targets

EMCHDS



46

The Problem

Long distance and low bandwidth



47

The Solution - Offline Initialization

Tape Tape



48

App

OS

App

OS

App

OS

App

OS

App

OS

Brand XStorage

Web Hosting on VMware, Brand X



49

Web Hosting on VMware with XIV

App

OS

App

OS

App

OS

App

OS

App

OS


Thank You!


The End

51

Session 1029Introduction to IBM XIV Storage System and What Makes It Unique

IBM® Edge2013 - Introduction to IBM XIV Storage System

Documents