Ibm ltfs fktg 2013

© 2012 IBM Corporation© 2013 IBM Corporation

IT Tape IBM LTFS – Tape File System

IBM Data Protection&Retention

Storedata

Josef Weingand Leading Technical Sales Professional

+49 171 5526783 - [email protected]

Infos / Find me on: http://sepp4backup.blogspot.de/ https://www.xing.com/profile/Josef_Weingandhttp://www.linkedin.com/pub/josef-weingand/2/788/300http://www.facebook.com/josef.weingand http://de.slideshare.net/JosefWeingandhttps://www.xing.com/net/ibmdataprotection

© 2013 IBM Corporation

Source: If applicable, describe source origin

Agenda

Tape Historie und Entwicklung

IBM Tape Entwicklung und IBM Tape Produkte

IBM Tape File System / LTFS


Video Tapes

1956 Quadruplex format (Zweizoll)

1960 Helical Scan


1952: IBM Modell 726 erster Bandspeicher• Lineare Aufzeichnung• 1.44 MB• 7-Spur Technik

720 Meter Bandlänge 100 BPI

Photo 1951 Prototyp Photo 1952 IBM 726 in Betrieb

1952-

201260 Jahre

IBM Bandspeicher


Tape Technologie Entwicklung

Kapazitätssteigerung:–2002: 1 TB Demonstration

• (44h@50Mbps)• Verfügbar 2008 mit TS1130

–2006: 8 TB Demonstration • (355 h@50Mbps)• Verfügbar ~ 2012 – 2015

–2010: 35 TB Demonstration • (1555h@50Mbps)

–Weitere Forschung an 100 TB Tape Medium–Plan: 135 TB Demonstration

Funktionalität–2010: Tape File System (LTFS) für Single Laufwerk–2011: LTFS für Libraries–2012: LTFS Storage Manager


Tape Storage: Demonstrating 29.5 Gb/in2

Record storage density on magnetic tape of 29.5Gbit/in2

– 39x increase over LTO 4 drives– A bit is 49 nm wide and 446 nm high– Made possible thanks to advanced

BaFe media, improved signal processing, and enhanced mechanics

Strawman Operating Point for 100 Gbit/in2

– Linear Density = 800 kbit/in – Track Density = 125 ktrack/in– Bit area = 32 nm × 200 nm– Only 36 magnetic particles per bit!


Storage Bit Cells and Extendability

Scaled Bit Cells

Magnified View of Scaled Bit Cells

TAPE8000 nm x 65 nm1.2 Gbit/in²

HDD74 nm x 13.5 nm635 Gbit/in²

NAND45 nm x 45 nm330Gbit/in²

Patterned Media25 nm x 25 nm1000 Gbit/in²


http://gmailblog.blogspot.com/2011/02/gmail-back-soon-for-everyone.html…. I know what some of you are thinking: how could this happen if we have multiple copies of your data, in multiple data centers? Well, in some rare instances software bugs can affect several copies of the data. That’s what happened here. Some copies of mail were deleted, and we’ve been hard at work over the last 30 hours getting it back for the people affected by this issue.

To protect your information from these unusual bugs, we also back it up to tape. Since the tapes are offline, they’re protected from such software bugs. But restoring data from them also takes longer than transferring your requests to another data center, which is why it’s taken us hours to get the email back instead of milliseconds.

So what caused this problem? We released a storage software update that introduced the unexpected bug, which caused 0.02% of Gmail users to temporarily lose access to their email. When we discovered the problem, we immediately stopped the deployment of the new software and reverted to the old version.

Feb 2011: Google restore gmail from

tape

Hard Lessons Learned

Tape is „last line of defense“!


Tape ist ein integraler Bestandteil einer effizienten Storage Lösung

Tape stellt eine zusätzliche und andere Schicht gegen Datenverlust zur Verfügung – „offline Medium“

– Langzeit, geringe Kosten, geringe TCO, einfache Skalierung

Reduziert die Kosten– Preis pro TB ist ~10% von Tier 2 disk1

– Stromverbrauch ist ~1%, verglichen zu Disk2

Bietet hohe Kapazitäten und schnelle Transferraten

Bietet hohe Datensicherheit– Geringe Bit-Error-Rate– Read-after-Write Verification

Transportierbar– Leicht, kompakt, hohe Kapazität

Bietet Investmentschutz mit skalierbaren und kompatible Systemen

Über 80%

betrachten Tape als integralen Bestandteil des Backup ProcessesSource: Enterprise Strategy Group Research Report, 2010 Data Protection Trends, April 2010

Sources: 1. “Top 10 Strategies for Surviving Unconstrained Data Growth,” Gartner Symposium Presentation, October 2010, slide 212 “In Search of the Long Term Archiving Solution – Tape Delivers Significant TCO Advantage over Disk”, The Clipper Group, Inc., December 2010.


New!

Tape DrivesTape Libraries

Tape Virtualization & D2D

LTO 4• 800 GB• 120 MB/sec

TS1130• 1 000 GB• 160 MB/sec

TS7700• VTS für zOS

TS7650G• DeDup Gateway• Cluster• bis zu 3 200 MB/sec

TS3500• 58 – 20 087 Slots•Shuttle: 15x20 087 = 300 000 Slots

TS3310• 30 - 408 Slots

TS31/3200• 22 / 44 Slots

TS2900• 9 Slots

1 HH

2/4 HH1/2 FH

18 FH

192

Media

LTO 6• 2500 GB• 160 MB/sec

TS7620• SMB Appliance• bis zu 145 MB/s• 6 / 12 / 23 / 35 TB native

B

D2D•DS3500, V3700, V7000, XIV, DS8000

IBM Tivoli Storage Manager IBM FlashCopy Manager TSM for VE IBM FastBack

TS1140• 4 TB• 250 MB/sec

Neu!

IBM ReadVerify Appliance (RVA)

IBM Tape File System/LTFS LTFS Storage Manager

TSLM

Neu!

Neu!

LTO 5• 1500 GB• 140 MB/sec

IBM DP&R Storage / DataProtection Overview


LTO – Über 10 Jahre und mehr…

Oktober

2012

2000 2002 2004 2007 2010 2012


IBM LTO Generation 6

Schneller und größer zum besseren Peis/Leistungsverhältnis

– 2,5 TB native Kapazität -> ca 110 h @ 50Mbps– 6,25 TB mit Kompression (2,5:1)– 160 MB/s native Durchsatz -> ca 25fache Echtzeit Geschwindigkeit

• Bei 2,5:1 Kompression 400 MB/sec – max 745 MB/sec• Mit variabler Geschwindigkeit 40 – 160 MB/sec

- 14 Speed Matching Geschwindigkeiten- Kein Start/Stop Problem, da kleinste Geschwindigkeit wie bei LTO4

und LTO5• 1024 MB Data Buffer

– Bessere Kompression 2,5 : 1

– Dual Ported 8 Gbit FC oder 6 Gbit SAS

– Kompatibilität: LTO5 Lesen und Schreiben, LTO4 lesen

– Geringerer Stromverbrauch

– Tape File System (LTFS)


3592 Generation 4 – TS1140 4 000 GB Kapazität (JC) -> ca 177 h @ 50Mbps

– Media Re-Use mit JB• JB: 700 GB -> 1 TB -> 1,6 TB

250 MB/sec native Datenrate Dual-active 8 Gbit FC

– Bis zu 700 MB/sec Durchsatz Neuer Giant Magnetorestive R/W Head mit Aluminium Überzug

– 32 parallel Tracks (Read/Write)

Verbesserter Durchsatz durch– 1024 MB Pufferspeicher– "Virtuelles Back Hitch„ mit noch besserern Leistung– Speed Matching mit 14 Geschwindigkeiten

Sehr schneller Datenzugriff durch – High Resolution Tape Directory– schnellste Spulgeschwindigkeit mit 12,4 m/Sek.– Read Ahead Feature– Schneller Load-/Unload-Mechanismus– JC Media unwesentlich länger als JB Media

Verbesserte End-to-End Datenintegrität– End-to-End CRC Check und „outboarded“ Audit-Funktionen– Für Langzeitarchive nützlich!

LTFS Support als Single Drive und in der Library– Tape wie USB-Stick– Tape wie Disk als File System sichtbar– Sehr großes (52 PB) File System

Upgrademöglichkeit von TS1130 auf TS1140!

Schnellstes Tapedrive im Markt!

© 2013 IBM Corporation Any statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.

LTO Generations LTO-3 LTO-4 LTO-5 LTO-6* LTO-7 LTO-8

Max Native Capacity

400 GB (L3) 800 GB (L4) 1.5 TB (L5) 2.5 TB (L6) Up to 6.4 TB (L7)

Up to 12.8 TB (L8)

Other Native Capacities

200 GB L2100 GB L1 R/O

400 GB L3200 GB L2 R/O

800 GB L4400 GB L3 R/O

1.5 TB L5800 GB L4 R/O

2.5 TB L61.5 TB L5 R/O

6.4 TB L72.5 TB L6 R/O

Native Data Rate 80 MB/s 120 MB/s 140 MB/s 160 MB/s Up to 315 MB/s

Up to 472 MB/s* Data Compression engine enhancement from 2:1 to 2.5:1

TS1100Generations

Gen-13592 J1A

Gen-2TS1120

Gen-3TS1130

Gen-4TS1140

Gen-5 Gen-6

Max Native Capacity

300 GB (JA) 700 GB (JB) 1.0 TB (JB) 4.0 TB (JC) 8-10 TB (JD)

14-20 TB (JD)

Other Native Capacities with Media Reuse

500 GB JA 640 GB JA 1.6 TB JB640 GB JA R/O

6-8 TB JC 6-8 TB JC

Native Data Rate 40 MB/s 100 MB/s 160 MB/s 250 MB/s Up to 360 MB/s

Up to 540 MB/s

IBM Tape Drive Roadmaps

© 2013 IBM Corporation17

IBM TS3500 (3584) UltraScalable Tape Library

Hochleistungs - Bandarchiv– Schnellstes Robotorsystem weltweit

Modulares skalierbares Design– Von 58 bis 20 087 Kassetten– bis zu 6 260 TB native Kapazität (derzeit)– Unterbrechungsfreie Erweiterungen

Hochverfügbarkeits Features– Dual Accessor – HA (optional)– Dual Gripper (standard)– Redundant Library Kontroller– Redundant Kontrollpfade

Laufwerke – LTO Gen1, Gen2, Gen3, Gen4, Gen5 und Gen6– TS11x0 Gen1, Gen2, Gen3 und Gen4

Geringster Stromverbrauch von nur (durchs.)185 W

Bis zu 1320 Kassetten pro Frame (0,9 m²)

Bis zu 4,4 PB pro m² -> ca 200 000 h @ 50Mbps


Warum IT Tapes für M&E

Formatunabhängig

Größere Kapazität

Kostengünstiger


Tape Entwicklung: Linear Tape File System (LTFS)=> Tape File System

Tape wie „USB-Stick“ – Tape wird als Filesystem dargestellt– Für Single-Drive und Library

• LTO5, LTO6 und TS1140• TS3500, TS31/3200, TS3310 und TS2900

Phase 1: Single Drive Edition– IBM Entwicklung

• Open Source

– LTFS mit einzelnem Laufwerk– Einsatzmöglichkeiten

• Datenaustausch• „Video“ Recorder• Etc

Phase 2: Library Edition– IBM unique– Tape Library stellt sich als File System dar

Phase 3: Software Integration– LTFS Storage Manager


LTFS: Partitionierung für selbstbeschreibende Medien

Partitionierte Medien sind notwendig für LTFS

Partitionierung der Medien ist die logische und physische Teilung des Bandes in Segmente

2 Partitionen– Index Partition (Partition 0): Enthält die Filesystem Info (ca. 37.5 GB)– Data Partition (Partition 1): Enthält die Files (ca.1.5 TB)

Der LTFS Treiber partitioniert das Medium

Verschlüsselung (Encryption) ist unterstützt– Ein Key für die gesamte Cartridge in transparenter Verschlüsselung– Standard Operation für die das Band nutzende Anwendung

BOT

EOT

Index Partition

Data Partition

Guard WrapsLTFS Index XML

File File File

File


Recognition by M&E Industry

Association for Information and Image Management (AIIM) Best Practice Award, for the media asset production and management system developed for Shanghai Media & Entertainment Group.

A member of our industry leadership team was recognized in Consulting Magazine’s “Top 25 Most Influential Consulting Leaders of 2005”

The Advanced Media Workflow Association is working with IBM on future development and direction of the open-standards software connectors based on IBM's ASD for media services.

The IBM Linear Tape File System won Broadcast Engineering’s Pick Hit Award at NAB 2011. LTFS puts a file system on top of tape.

IBM Beta Program Customer FOR-A won Broadcast Engineering’s Pick Hit Award at NAB 2010 for their LTO-5 video deck with IBM Long Term File System: LTR-100HS.

Frost and Sullivan named IBM the Global Digital Media Company of

the Year in 2010

Storage Visions

2012 Visionary Media and Entertainment Company Award

2011 Primetime Emmy Engineering Award

IBC Connected World Award

2011

Hollywood Post Alliance Engineering Excellence

Award


LTFS Single Drive Edition Downloads 2011 - 2012

2011 2012 (thru August)

Total

Linux 953 1,025 1,978

Mac 355 246 601

Windows 1,787 5,571 7,358

Total 3,095 6,842 9,937

LTFS SDE Download Statistics

0

200

400

600

800

1000

1200

1400

1600

1800

Feb Mar Apr May J un J ul Aug Sep Oct Nov Dec J an Feb Mar Apr May J un J ul Aug

2011 2012

WindowsMacLinux

HBO and/or Discovery Channel announce they will accept only LTFS submissionsHBO and/or Discovery Channel announce they will accept only LTFS submissions

“NAB effect”“NAB effect”


LTFS Library Support

LTFS zeigt Filesystem - Directory der kompletten Library– Jede Kassette wird als separates Directory / Ordner angezeigt – Directroy / Filesystem – Informationen werden von LTFS gecached– Das komplette LTFS Filessystem ist sichtbar und nutzbar (Search) ohne laden von Tape

Kassetten– LTFS steuert Tape Library und Tape Drives

• Lädt Kassetten bei Read/Write Anfroderungen

– „True File System“ Support• Jede Kassette und deren Inhalt ist verfügbar für jede Anwendung durch Standard Filesystem

View of Folders of Cartridges


Alle Cartridges werden als Directories mit Barcode-Namen angezeigt

LTO-5 Tape Cartridges in Library Magazine

LTFS Library appears as network drive

Windows 7 Explorer

Phase 2: LTFS-LE (Library Edition)

© 2013 IBM Corporation25

FilesCartridge 1

Library

Cartridge 2


LTFS Vorteile

Tape ist “self-contained”, “self-describing” und portierbar–File Index ist auf Tape

• Hierarchical directory structure• Extended attributes

Tape ermöglicht Cross-Platform Support–Entwickelt für Linux/Mac/Windows

Tape kann nun von jeder Applikation benützt werden –File System (LTFS) wird durch standard OS Interfaces supportet

• Double-click files• Drag-and-Drop• etc.

IBM won

Emmy award

2011!


LTFS Platform Support

Single Drive Edition Library Edition

Supported Tape Systems

TS2250, TS2260, TS2350, TS2360

TS2900, TS3100, TS3200, TS3310, TS3500, TS1140w/TS3500

Supported Operating Systems

RHEL 5.5, 5.6, 5.7, 5.8, 6.1 and 6.2

SLES 11 SP1 and SP2

Windows 7 SP1

Windows Server2008 R2 SP1

Mac OS X 10.6, 10.7

RHEL 5.5, 5.6, 5.7, 5.8, 6.1 and 6.2

SLES 11 SP1 and SP2

Windows Server 2008 R2 SP1

Other components

For details on supported hardware configurations

please refer to the System Storage Interoperation

Center (SSIC)

For details on supported hardware configurations please refer to the System Storage Interoperation

Center (SSIC)


LTFS Ecosystem


IBM Storage Infrastructure

Tape StorageDisk Storage

GPFS / SoFS TSM

CMS MaterialPool

ArchiveCommercials

MAMMaxdome

PlayoutAutomation

FactualArchive

IBM Archive and Essence Manager

IBM Arema & IBM LTFS Storage Manager

LTFS LE


IBM Storage: LTFS Storage Manager V.1.1 Archive Solution für Digital Media

(Vidoe&Audio)– IBM LTFS Storage Manager managed (archiving

/ restore) Files (Media: Video, Audio, etc) auf LTFS Tape Libraries

Reduziert Kosten in traditionellen Digital Media Umgebungen

Verwaltet Index der Daten– Erlaubt Tagging von Files zum besseren Search

des Content

Open Interface für Enterprise Digital Media Applikationen

– Erlaubt leichte Integration für M&E industry Formats

LTFS Storage Manager für alle Kunden die Media Daten (Audio+Video) speichern und auf Tape archivieren wollen

Disk Cache FC

Manual operationMonitoring

LTFS Library Edition

Local File System

LTFS Storage ManagerLTFS Storage

Manager Agents DB2

FTP, Samba,

NFSLTO Library

Pool 1

Pool 2

Drives

Tapes

Easy of use für Digital Media Daten!

© 2013 IBM Corporation 33

Video Acquisition, Distribution, & Archive

FOR-A Model:LTR-100HSLTO-5 Video Archiving Recorder

•The LTR-100HS enables you to build handy video “archive at ingest” solutions•Can be used as material program exchange media server•Supporting any stage of Ingest - Delivery - Archive - Reuse cycle

http://www.for-a.com/products/ltr100hs/ltr100hs.html

http://www.1beyond.com/products/p2wrangler.asp?search=p2workflow

1 Beyond™ Wrangler™2D & 3D Portable Tapeless Workflow - Ingest, Edit, Archive

•Rapidly unload P2, SxS, RED, Phantom, SI-2K memory cards or disks while shooting in-field •Auto protected to LTO-5 LTFS tape•Instantly Review or Add Metadata to video clips •Optional Editing in-field

LTO-5 Tape

LTO-5 Tape


Use case - Video Acquisition, Distribution, & Archive

Direct Broadcast

LTFS LE Archive


Video Archiving in the CloudT3 Media Inc.

Challenges• Needed a low cost delivery platform for enterprise scale Video Supply Chain

as a Service • Information growth of ~100 TB per month• Easy self-serve access required by clients

Solution• IBM Linear Tape File System at several global locations, including some client

facilities • IBM System Storage® TS3200 Tape Library, LTO®-5 tape drives

Benefits• Opened up new business opportunities• Enabled more predictable and transparent pricing for clients• Portable, interoperable, scalable, cost-effective data protection and

long-term storage

‘LTO 5 and LTFS significantly reduce the ancillary costs around storage. This is a real game-changer from IBM’

Mark Lemmons

CTO, T3 Media

TSP03327-USEN-00

T3 with LTFS on Youtube: http://www.youtube.com/watch?v=M7w0jrkQnj4T3 with LTFS on Youtube: http://www.youtube.com/watch?v=M7w0jrkQnj4


LTFS - Infos

IBM Linear Tape File System:

http://www-03.ibm.com/systems/storage/tape/ltfs/index.html

LTFS Info Center:

http://publib.boulder.ibm.com/infocenter/ltfs/cust/index.jsp

http://publib.boulder.ibm.com/infocenter/ltfsle/cust/index.jsp

http://pic.dhe.ibm.com/infocenter/ltfssm/cust/index.jsp

LTFS Format Specification:

http://www.ibm.com/support/docview.wss?rs=577&uid=ssg1S7003166

LTFS, LFV and ITDT Download (Linux, Mac OS X, Windows):

http://www.ibm.com/support/fixcentral/

(Storage Systems / Tape Systems / Tape Dev Driver and Software / Linear Tape File System / )(Storage Systems / Tape Systems / Tape Dev Driver and Software / LTFS Format Verifier / )(Storage Systems / Tape Systems / Tape Dev Driver and Software / Tape Diagnostic Tool / )

Almaden Research Project:

http://www.almaden.ibm.com/storagesystems/projects/ltfs/

Wikipedia:

http://en.wikipedia.org/wiki/LTFS


Infos / Find me on: http://sepp4backup.blogspot.de/https://www.xing.com/profile/Josef_Weingandhttp://www.linkedin.com/pub/josef-weingand/2/788/300http://www.facebook.com/josef.weingandhttps://www.xing.com/net/ibmdataprotection



Disclaimers

The performance data contained herein was obtained in a controlled environment based on the use of specific data. Actual results that may be obtained in other operating environments may vary significantly. These values do not constitute a guarantee of performance.

Product data is accurate as of initial publication and is subject to change without notice.

No part of this presentation may be reproduced or transmitted in any form without written permission from IBM Corporation.

References in this document to IBM products, programs, or services do not imply that IBM intends to make these available in all countries in which IBM operates. Any reference to an IBM program product in this document is not intended to state or imply that only IBM's program product may be used. Any functionally equivalent program may be used instead.

The information provided in this document has not been submitted to any formal IBM test and is distributed "As Is" basis without any warranty either express or implied. The use of this information or the implementation of any of these techniques is a customer responsibility and depends on the customer's ability to evaluate and integrate them into their operating environment. While each item may have been reviewed by IBM for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Customers attempting to adapt these techniques to their own environments do so at their own risk.


Trademarks

The following terms are trademarks or registered trademarks of the IBM Corporation in either the United States, other countries or both.

IBM, S/390, ES/3090, ES/9000, AS/400, RS/6000, MVS/ESA, OS/390, VM/ESA, VSE, TPF, OS/2, OS/400, AIX, DFSMS/MVS, DFSMS/VM, ADSTAR Distributed Storage Manager, DFSMSdfp, DFSMSdss, DFSMShsm, DFSMSrmm, FICON, ESCON, Magstar, Seascape

Other company, product, and service names mentioned may be trademarks or registered trademarks of their respective companies.

Windows NT is a registered trademark of Microsoft Corporation.

Ibm ltfs fktg 2013

Documents

ibm corporationb

ibm modell

ibm corporationhttps

tape storage

ibm lto generation

tb tape mediumplan

nm wide

slots lto