Mainframe Transparent Cloud Tiering (TCT) Overview

© Copyright IBM Corporation 2018.

Mainframe Transparent Cloud Tiering (TCT) Overview

Toy Phouybanhdyt - [email protected] Retention & Tape Client Technical Specialist

mailto:[email protected]

Washington Systems Center - Storage

© Copyright IBM Corporation 2018.© Copyright IBM Corporation 2018.

Acknowledgement

• Eddie Lin• STSM, DS8000 Architecture and Development

• Glenn Wilcock• DFSMS Architect

• Robert Gensler• DFSMSdss Architecture and Development

2



Learning Objectives

• To introduce end user to Mainframe Transparent Cloud Tiering and understand the benefits of each option.

• To introduce end user to what the hardware, the software and the network requirements are to implement DS8000 Transparent Cloud Tiering.

• To preview TS7700 Transparent Cloud Tiering.

3



Agenda

• DS8000 Transparent Cloud Tiering Overview, including use cases

• DS8000 Transparent Cloud Tiering Requirements• Hardware• Software• Network

• TS7700 Transparent Cloud Tiering Preview

4



New Age of Information Life Cycle

Hot

Cool

Cold

5

Data Profile

Flash

SSD

Disk

VTL

Tape

Cloud



Disclaimers

• Roadmaps represent plans and do not constitute a commitment by IBM

• IBM Plans are subject to change at any time without notification

6



DS8000 Transparent Cloud TieringOverview

7



Transparent Cloud Tiering

• What is Transparent Cloud Tiering?• Native cloud storage tier

• Why Transparent Cloud Tiering?• Data in the enterprise is growing at an alarming rate led by growth in unstructured data.

Transparent Cloud Tiering solves this problem by placing inactive data to cloud object storage. This frees up high performance storage capacity that can be used for more active data reducing storage costs.

• Inactive data constitutes a large proportion of data in the enterprise. Migrating cooler/cold data to lower cost cloud storage tier could result in cost savings. Public cloud as well as many of the private cloud storage is charged based on usage, hence (where applicable) this capability allows IT managers to move large portions of data from CAPEX to OPEX (cost reduction).

8



DFSMShsm Still Utilizes a 1970’s Architect

9

16K Blocksize

Continual Recycle

Serial Access to TapeNo Co-location

DFSMS

CPU

Reorgs, Audits,BackupsDSS/HSM Half

Data Movers

OnlineImmediate Access

OfflineDelayed Access

Data movement requires a large amount of CPU resources.



Cloud Storage Options for z/OS Data

10

Transparent Cloud Tiering

Off-premises as a

service

Transparent Cloud Tiering (TCT) improves business efficiency and flexibility while reducing capital and operating expenses with direct data transfer from DS8000 to hybrid cloud environments for simplified data archiving operations on z Systems.

On-premises or Off-

premises as a service

z System IBM DS8000

Migration

On-premises as object

storage target

Transparent

Cloud

Tiering

IBM TS77001

DFSMSDFSMShsm

1 TS7700 as object is in beta program, and the intermix of object and virtual tape library is a future functionality for the TS7700.



Introducing DS8000 Transparent Cloud Tiering

• Transparent Cloud Tiering Off-loads the data movement responsibility to the DS8000 without any impact on performance

• MIPS reduction via direct data transfer from DS8880 to Object Storage• Eliminates Recycle, 16K Block Sizes, Data

Movement thru IBM Z, Serial Access to Tape

• Leverages existing DS8000 data systems avoiding the need for additional hardware infrastructure• Does not require an additional server or gateway• Uses the existing Ethernet ports to access the

cloud resources

11

Focus on client value

Cloud Tiering

DS8K

Policy

DFSMShsm



TS7700 as a target for Transparent Cloud Tiering

• Allows for TS7700 to be an object storage target from DS8000• Embeds GRID technology into DS8000 to move data• Data is stored on TS7700 as objects1, not tape volumes allowing all advantages

of object storage offload.

12

TS77

00

TS7700

GRID Data Server

Spectrum Scale Filesystem

Ethernet

IBM DS8880

•TS7700 Filesystem•Objects Subdir

•Containers•Object (Dataset)•Dataset Metadata

DS8

880

IBM TS7700

GRID Data Movement Engine

DS8000 CEC

DFSMShsmMigration

IBM Z




Cloud Objects vs. Logical Tape Volumes

• TCT stores datasets as individual objects, not as logical tape volumes.

• Allows for parallelism of object retrieval and stores• Eliminates serial access to tape

• DFSMShsm considers objects a separate tier versus ML1, ML2 (Migration Level CLOUD)• VOLSERs are modified to MIGRATC• Direct from L0 to MLC

• Datasets are a collection of objects : metadata and customer data are split• Allows for future work with analytics, data sharing,

and further tiering13

DCSDMC01.GDG.V12S.G0001V00/APPMETA

DCSDMC01.GDG.V12S.G0001V00/DTPDSHDR

DCSDMC01.GDG.V12S.G0001V00/DTPVOLD01/NVSM/META

DCSDMC01.GDG.V12S.G0001V00/DTPVOLD01/NVSM/EXTENTS

VOLSER Z20022

VS.



Metro Mirror / Hyperswap Support

• Metro Mirror Support• Migrate from Metro Mirror Primary• Recall to Metro Mirror Primary

• Data recalled will be replicated to Metro Mirror Secondary

• Hyperswap Support• Migrates in progress during Hyperswap A->B will fail

migration due to change in storage configuration (Hyperswap)• HSM will retry the migrate on the next window

• Recalls in progress during Hyperswap will automatically be redirected to the “new MM Primary”

• GDPS Hyperswap supported• CSM via z/OS Hyperswap supported

• Both DS8880s must be connected to the same Cloud

14

Metro Mirror(Fiber Channel)

Ethernet Ethernet

Hyperswap / DR Supported



FlashCopy Support

• Migrate/Recall to volumes in a Flashcopy Relationship• Migrate data from a Flashcopy Source or Flashcopy Target

• Recall data to a Flashcopy source• Emulates host writes• CS will still do copy source to target as before

• DB2 Image Copy Offload from a Flashcopy Target• MIGRATE STORAGEGROUP (DB2IMGC) CLOUD(NAME)

15

Source

Target

FlashCopy Source

Target

FlashCopy

Data can be recalled to a volume that has PiT copies already

Current data or point in time copy can be migrated to the cloud



Use Case: DFSMShsm MIGRATE

• HSM invokes DSS to migrate data sets to the cloud• HSM inventory manages the Cloud, Container and Object prefix• Transparent to applications and end users• No Recycle• Recall works just as it does today• Audit support• VOLUME and STORAGEGROUP keywords also supported

• VOLSER will be changed to ‘MIGRAT’• ISPF will display ‘MIGRATC’, as opposed to ‘MIGRAT1’ or ‘MIGRAT2’

16

MIGRATE DATASET(dsname) CLOUD(cloud)



Use Case: DFSMShsm RECALL

• DFSMShsm will automatically Recall a dataset to Primary Storage when it is

referenced

• RECALL, HRECALL, and ARCHRCAL support recalling from the cloud. There are no

parameter changes, as all information is stored within the HSM control datasets.

• Common Recall Queue is supported.

• Fast Subsequent Migration

• Remigrated datasets are just reconnected to existing migration objects if the source

dataset was not updated.

• No additional data movement.

17



Use Case: DB2 Image Copy Offload

• Step 1: Create PiT DB2 Images Copies using FlashCopy• Step 2: Wait for background FlashCopy to complete

• Step 3: MIGRATE STORAGEGROUP(DB2IMGC) CLOUD(MYCLOUD)

18

DB2 Source Objects DB2 Image Copies

FlashCopy

Cloud Tier

DFSMS

IBM CloudObject Storage

DB2 Offline PIT Image Copies with the data never going through the host.



Transparent Cloud Tiering – CPU Efficiency Estimator

• IBM has a tool to estimate CPU saving• HSM writes various statistics to SMF record specified by SETSYS SMF(smfid)

• Recommended smfid is 240

• FSR records are written to smfid+1 (241)• FSRCPU records CPU time• Fields include dataset size and amount of data written

• Tool is publically available at: ftp://public.dhe.ibm.com/eserver/zseries/zos/DFSMS/HSM/zTCT

• CPU efficiency estimator analysis• Do it yourself• WSC Storage team, for NA IBMers and BPs, using the tool can be requested by sending an email

request to [email protected]

19

With a few days worth of SMF data, the estimator can determine:1. Size of datasets to target for greatest cost savings2. Estimated amount of CPU cycles saved by using Transparent Cloud Tiering

ftp://public.dhe.ibm.com/eserver/zseries/zos/DFSMS/HSM/zTCT



DS8000 Transparent Cloud TieringRequirements

20



Transparent Cloud Tiering Implementation Requirements

• DS8000 hardware release level• DS8000 code release level• DS8000 Ethernet connections• If planning to use TS7700 as a target for Transparent Cloud Tiering

• TS7700 hardware release level• TS7700 code release level• TS7700 Ethernet connections

• z/OS level• Cloud API support• Authentication Information from Cloud Service Provider or Administrator• TLS/SSL considerations

21



DS8000 Transparent Cloud Tiering Requirements

• DS8880 Model only• R8.2.3 (88.23.19.0) code level, or higher

• R8.2.3 TCT restricted recall of data from cloud object storage to only Simplex volumes

• R8.3 (88.30.137.0) code level, or higher• R8.3 TCT allows for migrate and recall of data to volumes in both Simplex and 2-Site Metro Mirror

relationships and S3 support• R8.3.3 TCT supports 10Gb Ethernet

• Use existing ethernet ports on DS8880 CECs (p8 severs)• 1Gb Ethernet connections• 10Gb Ethernet connections

• No additional hardware

22

DS8000 Hardware Requirements



DS8000 Transparent Cloud Tiering Requirements (continued)

• TS7700 as a target for Transparent Cloud Tiering1

• Only TS7700 models VEB (TS7720) and VEC (TS7760) supported• No FICON attachment• FC0005 DS8000 Offload Process• R4.1 PGA1a (8.41.101.10) code level, + VTD EXEC 285, or higher• Up to 2-way support

• Stand alone and 2-way configurations only• Can mix TS7720 model VEB and TS7760 model VEC• Grid replication not supported

23

TS7700 Hardware Requirements





• DFSMS• V2R1: OA51622, or higher• V2R2: OA50667, or higher• V2R3: Base, or higher

• DFSMShsm• V2R1: OA54020, or higher• V2R2 and V2R3: OA54628, or higher• Auto Migration support

• V2R2 and above: OA52913, or higher

• DS8880 Metro Mirror configuration• V2R1 and V2R2: OA53474, OA52384, or higher• V2R3: OA54035, OA54034, or higher

24

Software Requirements




• DS8880 uses Ethernet to talk to the object storage• Up to 2 network connections per CEC

• 1Gb adapter – included• 10Gb adapter – additional

• Up to 2 network IP addresses per CEC

25

DS8000 Network RequirementsD

S888

0

WAN




• TS7700 as a cloud object storage• 2 or 4 network connections• 2 or 4 network IP addresses

26

TS7700 Network Requirements



Cloud APIs Support

• DS8880 R8.2.3 code support

• Openstack Swift API to connect to object storage systems

• DS8880 is responsible for archiving and retrieving the actual data to the cloud.

• Host communicates with the DS88800 through the FICON - Fibre Channel protocol, and sends the

metadata directly to the cloud through a TCP/IP connection.

27

Openstack Swift API



Cloud APIs Support

• DS8880 R8.3 code support

• S3 and IBM Cloud Object Storage using S3 API

• DS8880 is responsible for archiving and retrieving the actual data to the cloud.

• Host communicates with the DS8880 through the Ethernet TCP/IP link to send the metadata to the

DS8880, which will then store the metadata in the cloud.

28

S3 API



Obtaining information from Cloud Service Provider/Administrator

• Username and password

• z/OS and DS8880 only support single userid and password to access to the cloud.

• Password is suppressed from the logs when configuring DFSMShsm to use the cloud.

• Password is stored in an encrypted form in DFSMShsm MCDS.

• Authentication end point

• The Uniform Resource Identifier (URI) that DS8880 and DFSMS should use for

authentication with the cloud object storage.

• Port

• The port on which DFSMS to communicate with the end point.

• Tenant (if swift)

• The name or project name, usually provide by cloud administrator, that identifies your

object store environment, i.e. production, development, test, etc...

• SSL/TLS version

• The lowest acceptable level of SSL/TLS that should be used when establishing a secure

connection.

• Digital certificates

• External or self-signed certificates

29



Setting up Digital Certificates

• Why do we need to setup SSL/TLS?• DFSMS will send account information (username and password) over the HTTP

connection• Using SSL/TLS will ensure that information is encrypted on the network

• What we need to do• Establish a secure HTTP connection between our z/OS host and our object storage

cloud server

• Two types of authentication• Server authentication – our z/OS host will verify the identity of the object storage cloud

server• Mutual authentication – our z/OS host will verify the identity of the object storage cloud

server and our object storage cloud will verify the identity of our z/OS host

30

Why use SSL/TLS?



TS7700 Transparent Cloud TieringPreview

31



TS7700 Transparent Cloud Tiering

32

TS77

00

Privateor

Public Cloud

IP

Optional Cloud Tier

TS4500

OptionalTape Tier FC

• TS7760 only• Leverage TCT for offload to public or

private cloud• Physical tape and cloud tier are both

policy managed options• One or the other in R4.2

• Once one cluster puts a copy in the cloud, all clusters have access to the object store instance

• Will support IBM COS and AWS S3 in R4.2

• Local and cross regional support AWS S3 support

• TLS 1.2 HTTPS utilized for cloud object store connectivity with optional customer provided certificates• Encryption on the wire



R4.2 TS7760C & TS7760T

• Define partitions• Same partition concepts of traditional tape attach• Content can go to Cloud via TCT just like it can go to tape• Policy managed• Same pre-migration queue size features as used for tape attached

• Cloud attach option• Attach to an Object Store• Cloud locations are shared with peers and accessible from peers if they too have

connectivity to the same object store

33

Cloud Partition 2

Tape Partition 1

Cache ResidentPartition

TS7700 Primary Disk Cache

Policy Managed

or



R4.2 Expanded Cloud Use Case

• Each cluster has optional connectivity to a single cloud• TS7700 Grid remote access can be leveraged to access another peers cloud

content

34

LAN/WAN

zHost(s)

Composite Library

zHost(s)

C) TS7700 Cluster

B) TS7700 Cluster

A) TS7700 Cluster

E) TS7700 Cluster

D) TS7700 Clusters

Cross Region Replication

Cross Region Replication



Reference Materials

• DFSMS Transparent Cloud Tieringhttp://publibz.boulder.ibm.com/zoslib/pdf/OA51622.pdf

• IBM DS8880 and z/os DFSMS: Transparent Cloud Tiering redbookhttp://www.redbooks.ibm.com/abstracts/sg248381.html?Open

• Configuring TS7700 as a target for Transparent Cloud Tieringhttps://www.ibm.com/support/knowledgecenter/ST5GLJ_8.2.3/com.ibm.storage.ssic.help.doc/f2c_trans_cloud_tiering_ts7700.html

35



Summary

• DS8000 Transparent Cloud Tiering feature can be used to direct HSM data movement between DS8800 & Cloud Storage without consuming zOS host MIPs.

• Understand the requirements to implement DS8000 Transparent Cloud Tiering.

• Understand TS7700 Transparent Cloud Tiering offering.

• Understand the user case of each MF Transparent Cloud Tiering.

36



Questions

April 2017Rite Aid/IBM Confidential

37

Mainframe Transparent Cloud Tiering (TCT) Overview

Documents