IBM Spectrum Scale Scalable Global Parallel File Systems(GPFS) Spectrum Scale 2019 Josef (Sepp) Weingand Business Development Leader DACH – Data Retention Infrastructure - Tape Storage Infos / Find me on: [email protected], +49 171 5526783 Blog http://sepp4backup.blogspot.de/ Facebook https://www.facebook.com/Sepp4Tape/ http://www.linkedin.com/pub/josef-weingand/2/788/300 http://www.facebook.com/josef.weingand http://de.slideshare.net/JosefWeingand https://www.xing.com/profile/Josef_Weingand https://www.xing.com/net/ibmdataprotection
46
Embed
IBM Spectrum Scale Scalable Global Parallel File Systems(GPFS)konferenz-nz.dlr.de/pages/samfs2019/present/2. Konferenztag/1 - HSM... · Introducing IBM Spectrum Storage for AI with
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
IBM Spectrum Scale
Scalable Global Parallel File Systems(GPFS)
Spectrum Scale 2019
Josef (Sepp) Weingand
Business Development Leader DACH – Data Retention Infrastructure - Tape Storage
Infos / Find me on: [email protected], +49 171 5526783Blog http://sepp4backup.blogspot.de/
• Single name space supporting 250 PB capacity• Total number of files supported is 100B (10 mio files per single directory)• Single Node 16 GB/sec sequential read/write as requested from ORNL• Performs at an aggregate sequential peak read/write bandwidth of 2.5 TB/s• Performs at an aggregate random peak read/write bandwidth of 2.2 TB/s• Provides rich metadata performance - single directory parallel create rate of 50,000/s• Provides rich interactive performance - @32 KiB I/O 2.6 million IOPs
• Storage scale-out from a single 300TB node to 8 Exabytes and a Yottabyte of files
High-Performance to feed the GPUs
• NVMe throughput of 120GB/s in a rack
• Over 40GB/s sustained random read per 2U
Extensible for the AI Data Pipeline
• Support for any tiered storage, including Cloud and Tape
Introducing IBM Spectrum Storage for AI
with NVIDIA DGX
A Scalable, software-defined infrastructure powered by IBM Spectrum Scale and NVIDIA DGX-1 systems. IBM Spectrum Storage for AI with NVIDIA DGX is a powerful engine for your data pipeline.
The workhorse of an AI data infrastructure on which companies can build their shared data service.
• Unique intelligent de-clustered erasure coding (“super RAID”)•High performance, high availability, high reliability storage layer
ESS has its own built-in JBOD storage = enclosures with drives
• But can mix/match using Spectrum Scale support for almost any block storage (disk, flash, etc.)
ESS reduces risk: quick to deploy and grow a Spectrum Scale cluster • Fully validated hardware and software stack• Pre-assembled and pre-configured• Comes with lab services for on-site deployment
Flash, disk, and hybrid ESS models
▪Elastic Storage Server
(ESS) is an Integrated
building block solution
for Spectrum Scale
IBM Systems20
Spectrum Scale Native RAID (Declustered SW Raid)
Spectrum Scale build your own solution
Requires dedicated disk controllers(SAN)
ESS eliminates need for SAN; Implements
de-clustered RAID in Software
▪Integrate de-clustered RAID
into software stack
▪QDR/EDR IB
▪10/40 GigE
IBM Storage & SDI
21
Declustered Raid6 example
IBM SystemsIBM Systems
The data deluge80% of all files created
are inactive
no access in at least 3 months!
=> NAS: Never Access Storage
| 22
Source: D. Anderson, 2013 IEEE Conf. on Massive Data Storage
Pro Jahr verkauften HDDs entsprechen 1,8 Mio Autos an CO2
IBM SystemsIBM Systems
HDD ?!?▪ HDD has reached the limit of (known) materials to produce
larger write fields:
• Areal density/capacity scaling achieved by shrinking the same basic
technology to write smaller and smaller bits on disk
▪ Technologies to go beyond the superparamagnetic limit:
• Two dimensional magnetic recording (TDMR)
• Heat Assisted Magnetic Recording (HAMR)
• Microwave Assisted Magnetic Recording (MAMR)
• Bit Patterned Media (BPM)
▪ Recent Capacity Scaling of HDD: Volumetric Density
• Slow down in areal density scaling partially compensated by adding
more disks: conventional technology has reached space limit (~5
platters)
• Helium filled drive less turbulence thinner disks higher capacity
• WD 6TB (2013) 6 platters
• HGST 10TB Drive (2015) 7 platters - CAGR 29%
• 14 TB 9 platters (2017) – CAGR18%
• Doesn’t scale: No space for more heads and platters!
| 23
Magnetic Media “Trilemma”:
IBM SystemsIBM Systems
Seagate hits density problem with HAMR, WD infects MAMR
▪ Seagate's next-generation HAMR disk drive will be a drop-in replacement while Western Digital's MAMR drive will not
▪ WD's technical product marketing director Eyal Shani told us that MAMR drives would use host-managed shingling, and so would not be drop-in replacements for existing drives.
▪ With shingling, write tracks are partially overlapped, meaning any rewriting of already written data incurs a time penalty as the affected block of write tracks is read, altered with the new data, and then rewritten.
6 TB (L7) 12.0 TBUp to 24 TB Up to 48 TB Up to 96 TB Up to 192 TB
Other Format
Capacities
(Native)
800 GB (L4)(400 GB L3 R/O)
1.5 TB (L5)(800 GB L4 R/O)
2.5 TB (L6)(1.5 TB L5 R/O)
9 TB (M8)
6 TB (L7)
Up to 12 TB (L8)(6 TB L7 R/O)
Up to 24 TB (L9)(12 TB L8 R/O)
Up to 48 TB (L10)(24 TB L8 R/O)
Up to 96 TB (L11)(48 TB L10 R/O)
Native Data
Rate
140 MB/s 160 MB/s 300 MB/s Up to 360
MB/s
Up to 708 MB/s Up to 1100 MB/s
Any statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.
BSI warnt vor gezielten Angriffen auf Unternehmen• „Wir erleben derzeit die massenhafte Verbreitung von raffinierten Angriffsmethoden durch die Organisierte
Kriminalität, die bis vor einigen Monaten nachrichtendienstlichen Akteuren vorbehalten waren….“, so BSI-Präsident Arne
Schönbohm.
• Dabei versuchen die Angreifer etwaige Backups zu manipulieren oder zu löschen und bringen dann selektiv bei
vielversprechenden Zielen koordiniert Ransomware auf den Computersystemen aus. Dabei kommt es teilweise zu
erheblichen Störungen der Betriebsabläufe. Durch dieses aufwändige Vorgehen können Angreifer deutlich höhere
Lösegeldforderungen an die Unternehmen stellen, als es bei bisherigen ungezielten Ransomware-Kampagnen der Fall
war. Neben einzelnen Unternehmen sind zunehmend auch IT-Dienstleister betroffen, über deren Netzwerke sich die
Angreifer dann Zugang zu deren Kunden verschaffen.
• Es droht ein kompletter Datenverlust
Im Gegensatz zu automatisierten und breitangelegten Ransomware-Kampagnen, bedeuten diese manuell ausgeführten
Angriffe einen deutlich höheren Arbeitsaufwand für die Angreifer. Da sie dadurch jedoch gezielt lukrativere Ziele angreifen
und u.U. Backups so manipulieren bzw. löschen, dass diese nicht mehr zur Wiederherstellung der Systeme zur
Verfügung stehen, können die Angreifer wesentlich höhere Lösegeldbeträge fordern. Unternehmen, die über keine
Offline-Backups verfügen, verlieren bei diesem Vorgehen alle Backups, selbst wenn diese auf externen Backup-
Appliances liegen. Dem BSI sind mehrere Fälle bekannt, bei denen die Verschlüsselung aller Systeme sowie der
Backup-Appliances nicht in eine Risikobewertung einbezogen wurde, weshalb die betroffenen Unternehmen alle
▪ No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation.
▪ The performance data contained herein were obtained in a controlled, isolated environment. Results obtained in other operating environments may vary significantly. While IBM has reviewed each item for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. These values do not constitute a guarantee of performance. The use of this information or the implementation of any of the techniques discussed herein is a customer responsibility and depends on the customer's ability to evaluate and integrate them into their operating environment. Customers attempting to adapt these techniques to their own environments do so at their own risk.
▪ Product data has been reviewed for accuracy as of the date of initial publication. Product data is subject to change without notice. This information could include technical inaccuracies or typographical errors. IBM may make improvements and/or changes in the product(s) and/or programs(s) at any time without notice. Any statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only
▪ References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Any reference to an IBM Program Product in this document is not intended to state or imply that only that program product may be used. Any functionally equivalent program, that does not infringe IBM's intellectually property rights, may be used instead. It is the user's responsibility to evaluate and verify the operation of any on-IBM product, program or service.
▪ THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IBM EXPRESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NONINFRINGEMENT.
▪ IBM shall have no responsibility to update this information. IBM products are warranted according to the terms and conditions of the agreements (e.g. IBM Customer Agreement, Statement of Limited Warranty, International Program License Agreement, etc.) under which they are provided. IBM is not responsible for the performance or interoperability of any non-IBM products discussed herein.
▪ Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.
▪ The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents or copyrights. Inquiries regarding patent or copyright licenses should be made, in writing, to:
IBM Director of LicensingIBM CorporationNorth Castle DriveArmonk, NY 10504-1785U.S.A.
IBM SystemsIBM Systems
Trademarks
46
▪ The following terms are trademarks or registered trademarks of the IBM Corporation in either the United States, other countries or both.– IBM, GDPS, Spectrum Storage, Spectrum Archive, Spectrum Scale, System Storage, System z, Virtualization Engine
▪ Linear Tape File System, Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries.
▪ Other company, product or service names may be trademarks or service marks of others