IBM Global Technology Services June 2008 Implementing Softek zDMF technology. Supporting simple, effective and nondisruptive data migrations
May 20, 2015
IBM Global Technology ServicesJune 2008
Implementing Softek zDMF technology.Supporting simple, effective and nondisruptive data migrations
Introduction
In the mainframe industry, there are many data migration products and many ways to migrate data. Some are controller based or appliance based, while others are host based. This paper focuses on host-based migration products, as they provide the most flexibility and the fewest operational restrictions. In most cases, it is helpful to use combinations of products or tools to accomplish a migration, because no one tool can do everything exactly the way the user may desire.
For example, the user may want to migrate from older to newer technology while, at the same time, moving to larger-capacity devices such as from an 3390-3 device to a 3390-9 system. One option in this scenario would be to have IBM’s System Managed Storage (SMS) do the migration with redirection, but this would not move files that have not been deleted and reallocated in a timely manner.
Another option would be to use Hierarchical Storage Management (HSM) to migrate files after they have been archived and recalled. But again, if the files are in use, HSM would not archive the files. And, in most cases, it would be necessary to take manual action to adjust HSM policies and then reset them after a migration. Other scenarios could include the use of a disk copy utility such as Data Facility Data Set Services (DFDSS) at the disk and/or file level. However, as with the above scenarios, there is the issue of files being in use, not to mention the manual effort of building the control cards and finding an extended window for conducting the migration.
Implementing Softek zDMF technology.Page 2
Contents
2 Introduction
3 zDMF capabilities
5 zDMF architecture
8 The zDMF migration process
19 A typical migration
25 Performance considerations
29 Platform support
29 Restrictions
30 Other Softek migration
solutions
31 Summary
Implementing Softek zDMF technology.Page 3
To address some of these issues, Softek developed z/OS® Dataset Mobility Facility (zDMF) technology, host-based software that can move data at the file/extent level between Direct Access Storage Device (DASD) volumes without interrupting the applications that read/write to those volumes. zDMF software can work with virtually any vendor’s storage that may or may not be under SMS control in a shared or non-shared DASD environment including sysplexes. zDMF software is ideal for the following types of local data migrations:
• Technologyrefresh,especiallyinheterogeneousor
high-availabilityenvironments
• Migrationsfromsmallertolargervolumes
• Implementationoftieredstorage
• Consolidationofstorage
• Dynamicdatamovementtofixperformanceproblems
(e.g.,movingfilesfromstoragehotspots)
The sections that follow provide a general overview of zDMF software’s capabilities, architecture and migration process. They also sketch a sample migration scenario and discuss performance factors.
zDMF capabilities
Simplicity of design
zDMF technology’s distinct value is its simplicity. Built from the ground up as a fully integrated product, it features components that are designed to execute flawlessly together. zDMF software is designed to be simple to install, simple to use and simple to maintain.
Highlights
Softek zDMF technology can help
with several different types of local
data migrations.
Implementing Softek zDMF technology.Page 4
Nondisruptive switch
zDMF software’s key differentiator is its ability to nondisruptively switch the source and target files, which moves the applications dynamically onto new storage. This switchover feature, which occurs under user control, results in redirection of an application’s input/output (I/O) function (e.g., from the origi-nal source to the target). This occurs without any disruption to the application.
Until the redirection, zDMF software continues to synchronously write to the source and target volumes, allowing fallback (the ability to fall back to the original source volume) at any time. Volumes that are no longer in applications’ I/O path can be taken offline without disruption.
For the current release of zDMF technology, a small application bounce may be required after the migration in order to free up the original disk space. The bounce would be needed only for applications that have a persistent file allocation; it can be scheduled to occur when a window is available.
Migration tuning parameters
In order to adjust the performance impact on applications during a migration, zDMF software provides the ability to control migration rate using the follow-ing parameters:
• MAXIO(determinesthemaximumoverallI/Oconcurrency)
• MAX_CHANNEL_IO(determinesthemaximumI/Oconcurrencyper
channelpath)
• MAX_DEVICE_IO(determinesthemaximumI/Oconcurrencyper
individualdevice)
• MAXTRK(specifiesthesizeoftheI/OoperationintracksthatzDMF
softwarewilluse)
Highlights
Softek zDMF software can dynami-
cally move applications onto new
storage by nondisruptively switch-
ing the source and target files.
The software can control the migra-
tion rate using several parameters.
Implementing Softek zDMF technology.Page 5
Migration groups
zDMF software supports the definition and concurrent migration of thousands of files. Because migration projects may involve several hundred files sup-porting a range of application types, zDMF technology provides the ability to logically group files for efficient operational control. Each group’s migration parameters are independently configured and controlled to best suit the business applications supported.
Monitoring features
zDMF provides the ability to monitor the migration process from start to finish. Statistics include details such as elapsed time, copy rate and percent-age complete.
Shared DASD
zDMF works in a shared DASD environment. The DASD can be shared by individual LPARs running on multiple physical CPUs, shared within a sysplex or shared across a multiple sysplex environment.
New Extended Address Volume functionality
zDMF can streamline large-scale migrations via new Extended Address Volume functionality, which exceeds the previous industry-standard storage capacity of 65 cylinders.
zDMF architecture
zDMF technology consists of four major components:
• zDMFTSOMonitor:Thisisauserinterfacetoestablishamigration,issue
commandsandcontrolmigrations.zDMFsoftwarealsooffersacommand
lineinterface(CLI).
• zDMFserver:TheserverisaprimarycomponentofthezDMFproduct
framework.ThezDMFserverisaz/OSStartedTaskthatmustrunoneach
systemthathasaccesstothedatathatwillbemigrated.
Highlights
zDMF software can migrate thou-
sands of files concurrently and
enables monitoring of the process.
Softek zDMF software’s first two
components are the zDMF TSO
Monitor and zDMF server.
Implementing Softek zDMF technology.Page 6
• zDMFI/Omonitor:AsubcomponentofthezDMFserver.ThezDMF
I/OmonitorisresponsibleformonitoringallI/Oactivitytosourcedata
sets(extents).
• zDMFdatabase:Thisfileisusedtostoreandshareinformationabout
zDMFdatamigrationactivity.ThezDMFdatabaseisusedbythezDMF
servertostoreandcommunicatemigrationinformationacrossall
participatingsystems.
ESCON/FICONDirector
zDMFDatabase
zDMF/ServerMVS/A
zDMF/ServerMVS/V
zDMF/ServerMVS/Y
zDMF/ServerMVS/X
zDMF/ServerMVS/Z
zDMF/ServerMVS/W
zDMF
10708881 zDMF ComponentsFigure 3
TSO Monitor
Figure 1 The zDMF components
Highlights
The second two components
are the zDMF I/O monitor and
zDMF database.
Implementing Softek zDMF technology.Page 7
zDMF migration elements
To facilitate data migration at the logical data level, zDMF software must understand the z/OS metadata structures that describe the logical data being migrated. It must also be able to manipulate and update the metadata dynami-cally while applications and the system continue to access and modify the data.
The following items comprise the metadata in a z/OS system.
Volume table of contents (VTOC)
Contains information describing volume contents, including information about specific data sets (number of extents, size of extents, starting locations, etc.).
VTOCIX Used in conjunction with the VTOC for identifying free space on a volume that is SMS managed.
VSAM volume data set (VVDS)
Introduced with ICF catalogs, this contains information about a data set similar to what is contained within the VTOC. Additionally, information such as details on VSAM data sets such as relative block addresses (RBA) also is kept here.
Catalogs There can be, and usually are, multiple catalogs in a z/OS environ-ment. A data set is most typically catalogued, and this tells the system where the data set is located (volume or volumes on which it resides). Catalog information not only is kept on a DASD volume, but also is kept in memory in the catalog address space (CAS).
Coupling facility If enhanced catalog sharing (ECS) is in use, then entries from the VVDS may be cached in the coupling facility.
Highlights
Metadata structures must be
understood by the zDMF software.
Four items make up the metadata
including the volume table of
contents, VTOCIX, VSAM volume
data set, catalogs and the
coupling facility.
Implementing Softek zDMF technology.Page 8
The zDMF migration process
After the installation, an administrator can use the zDMF TSO Monitor to establish a migration that performs all the steps of a migration process. The distinct sequence of steps involved in a migration using zDMF software is as described below.
Migration steps Description
1. Group definition The data set(s) to be migrated are defined in a migration group using the TSO Monitor.
2. Activating a migration group Activating a migration group initiates the data migration process for the defined migration group.
3. Copy phase Data is asynchronously copied from the source data sets to the target data sets that are defined in a migration group.
4. Synchronization phase All final differences between source and target data sets in a migration group are synchronized and the migration group is prepared for mirroring.
5. Mirror phase The migration group is put into a state of synchronous mirroring.
During the mirror phase, updates to source and target data sets in the migration group are applied simultaneously.
6. Diversion phase In this phase, the actual logical relocation of data sets occurs. Source and target data sets, along with the metadata, are modified and all I/O activity is redirected to the new location. At this point, the files have been migrated, and all references and updates to the files will occur at the target location.
7. Completion phase Although the metadata has been modi-fied, applications that were active before diversion will have their I/O redirected to the target location until they de-allocate the data set. For applications that continue to have the original source file allocation, a scheduled bounce will be required in order to free the original space.
8. Post-completion phase Once the migration has completed, migra-tion groups original source data sets and the storage resources they reside upon can be cleaned up.
Highlights
The first four of the eight migration
steps are group definition, activa-
tion of a migration group, copy
phase and synchronization phase.
The last four steps are mirror
phase, diversion phase,
completion phase and post-
completion phase.
Implementing Softek zDMF technology.Page 9
Group definition
Before a data migration can begin, a group definition must first be created. A group definition consists of the following:
• Migrationgroupname
• Migrationoptions
• Sourcedataset(s)
• Excludedataset(s)—optional
• Targetdataset(s)
zDMF software has the ability to create groups for a migration. The grouping is decided upon by the migration administrator. This makes it easier to control the migration with a single command that is applied to the entire group. It is recommended that the grouping follow some convention, such as by application or by array that is being migrated.
To further refine grouping, zDMF software has the ability to group volumes together. The source volume list name references a user-defined population of source volumes, refer to “Define zDMF Group – Panel 2” and “Define zDMF Group – Panel 3a” below.
Define zDMF Group - Panel 1
Define zDMF GroupCommand ===> Scroll ===> CSR
Primary Commands : EXit NExt
Group Name . . . .
Source Options . . . N Replace Existing Data Sets (Y/N) Y Tolerate Allocation Failure (Y/N)
Figure 2 Define zDMF Group – Panel 1
Highlights
First the migration administrator
defines groups and gives them
basic information.
The software can group volumes
together to further refine grouping.
Implementing Softek zDMF technology.Page 10
The zDMF selection criteria for the source data sets can include wild cards. Care should be taken to make sure that the selection closely reflects the actual data sets that the user intends to migrate. Consider using IEHLIST or ISPF 3.4 to validate the selection.
In further refining the selection criteria, it is possible to build a Data Set Exclude list. An example of this may be to exclude zDMFTEST.** data sets from being migrated after selecting all data sets on a particular volume. This would exclude any data set that starts with a high-level qualifier of zDMF TEST from being migrated.
Depending on parameter settings, zDMF software may display one of the fol-lowing panels.
Define zDMF GroupCommand ===> Scroll ===> CSR
Primary Commands : EXit NExt
Group Name . . . . . . . Source Volume List Name . . . . SPMS13Source Volume List Mask SPMS13
Define zDMF Group - Panel 3a
Define zDMF GroupCommand ===> Scroll ===> CSR
Primary Commands : EXit NExt
Group Name . . . . . . . COCOCOSource Data Set Options . . N Trace (Y/N) N AllocSeq (D/S/N) . Y Sphere (Y/N) N Rename UnConditional (Y/N) . N Build Data Set Exclude List (Y/N)Source Data Set Name/MaskSource Volume List Name . . Storage Class . . . . . .Source Volume(s) . .
Define zDMF Group - Panel 2
Figure 3 Define zDMF Group – Panel 2
Figure 4 Define zDMF Group – Panel 3a
Highlights
Wild cards can be selected for
migration as well as exclusion lists.
Panels 2 and 3a from Softek zDMF
are shown here.
Implementing Softek zDMF technology.Page 11
Define zDMF GroupCommand ===> Scroll ===> CSR
Primary Commands : EXit IMport NExt
Group Name . . . . . . . GROUP002Data Set Excludes Mask
Define zDMF Group - Panel 3b
Figure 5 Define zDMF Group – Panel 3b
Figure 6 Define zDMF Group – Panel 3c
Define zDMF GroupCommand ===> Scroll ===> CSR
Primary Commands : EXit NExt
Group Name . . . . . . . . . Group 002Rename Unconditional Prefix (Optional)Rename Unconditional Mask Pairs
Define zDMF Group - Panel 3c
Highlights
Group definition panels 3b and 3c
are shown here.
Implementing Softek zDMF technology.Page 12
After specifying the source data set information, specify the target data set mask, target volume storage class and/or target volume(s) parameters, as described in “Define zDMF Group – Panel 4.”
Once all the relevant group/pair parameters for the new group definition have been entered, type the BUILD command (or BU), and then press Enter. The new group definition’s configuration information is displayed in “Define zDMF Group – Panel 5.”
Define zDMF Group - Panel 5
Define zDMF Group Row 1 to 19 of 21Command ===> Scroll ===> CSR
Primary Commands : EXit MOre EDit SAve VErify PRomote
Group (COCOCO) - Mode (LMIGR()) - TOLORATE_ALLOCATION_FAILURE(YES) MAXRC(8) - REPLACE(NO)SOURCE_VOLUME_LIST SPMS13 (- SPMS13 - ]SET - ALLOCSEQ(NONE) - TRACE(NO) - SPHERE(YES) SOURCE ( - DSN (SXM90.zDMF.TEST.*) - SOURCE_VOLUME_LIST SPMS13 - )TARGET (- DSN (SXM90.zDMF.TEST.*)- VOLUME (- IBM408 -
Define zDMF Group - Panel 4
. . . . . . .
Define zDMF GroupCommand ===> Scroll ===> CSR
Primary Commands : EXit BUild
Group Name . . . . . . . COCOCOTarget Data Set Name/Mask . . . . . Target Volume Storage Class. Target Volumes(s). . . . . .
Figure 7 Define zDMF Group – Panel 4
Figure 8 Define zDMF Group – Panel 5
Highlights
Panels 4 and 5, shown here,
require information regarding
group definition.
Implementing Softek zDMF technology.Page 13
At this point, the primary commands can be used to add more source data sets to the group, or the user can edit, save, verify and promote the newly created group.
Activating a migration group
During the activation phase, zDMF software allocates data sets on the target volumes. If any errors occur during allocation, the migration group will either terminate the entire group or just the data set(s) that are in error, depending on the Tolerate Allocation Failure option selected when defining the group.
Copy phase tasks
During the copy phase, zDMF software initiates and performs the following tasks:
• ThezDMFserverasynchronouslycopiesthesourcedatasetstothetarget
datasets.
• ThezDMFI/Omonitorroutinesmonitorallactivitytoallsourcedatasets
andtrackallmodificationstothesourcedataset.
• Oncetheinitialcopyofsourcedatasetsiscompleted,thezDMFserver
refreshesthechangeddatarepeatedlyuntilitreachesapointwhereitcan
quicklysynchronizethesourceandtargetdatasetswithminimaldisruption
toapplicationsorthesystem.Oncethispointisachieved,zDMFsoftware
automaticallymovestothesynchronizationphaseforthismigrationgroup.
Highlights
During the activation phase, the
software allocates data sets.
The copying phase occurs after a
group is activated.
Implementing Softek zDMF technology.Page 14
In some instances, a migration group may take some time to complete the copy phase across all data sets in the migration group. The copy phase hap-pens after the group is activated.
Managing performance during the copy phase
During the copy phase, depending on the size of a migration group, a good deal of I/O could possibly be driven by zDMF software. To control the pacing of I/O during the zDMF copy phase, the user can modify parameters that the zDMF server uses. The parameters that can be dynamically modified include the following:
• MAX_CHANNEL_IO
• MAX_DEVICE_IO
The optional MAXTRK parameter specifies the size of the I/O operation in tracks that zDMF software should use to transfer data while in copy phase. The MAXTRK value is used to reduce the impact of zDMF software on the application response time immediately following activation. The MAXTRK parameter is defined in the zDMF server startup configuration file for the system that hosts the zDMF owning server.
Highlights
The zDMF software can dynami-
cally modify parameters during
the copy phase.
Implementing Softek zDMF technology.Page 15
Synchronization phase
A migration group enters the synchronization phase when zDMF software determines that it can quickly synchronize all data between source and target data sets in the migration group without causing disruption to the systems and applications using this data.
During the synchronization phase, zDMF software initiates and performs the following tasks:
• ThezDMFI/OmonitorroutinesdynamicallyholdallI/Otothesourcedata
setsofinalsynchronizationcanbeachieved.
• zDMFsoftwarecopiesallremainingdifferencesfromthesourcedatasetsto
thetargetdatasets.Atthispoint,thereisverylittledifferencebetweendata
sets,andthisoperationoccursveryquickly.
• zDMFsoftwareallowsnormalI/Ooperationstocontinueoncesynchroniza-
tionhasbeenachieved.
When the synchronization phase has completed, the migration group indi-cates that it is in mirror state.
Mirror phase
Once the source and target data sets within the migration group have suc-cessfully synchronized, they enter the mirror phase. During the mirror phase, updates to source and target data sets in the migration group are applied simultaneously. If an I/O error occurs at the target volume, the group is sus-pended. Once the mirror phase has been achieved, mirroring continues until one of the following occurs:
Highlights
The synchronization phase occurs
when the software determines that
it can synchronize data without
causing system and application
disruptions.
Once the mirror state has been indi-
cated and the synchronization phase
completed, the mirror phase begins.
Implementing Softek zDMF technology.Page 16
• ADivertcommandisissuedagainstthemigrationgroup.
• ASuspendcommandisissuedagainstthemigrationgroup.
Diversion phase
The diversion phase executes in two subphases:
• Subphase1:processingofthedivertcommand
• Subphase2:diversionofI/Ofromactiveapplicationstothetargetdataset
Subphase 1: processing of the divert command
During this subphase, zDMF software modifies all metadata for the source and target data set pairs within the migration group, effectively swapping the identities of the source and target. To accomplish this, zDMF software:
• Serializesaccesstothemetadataforcollectionsofdatasetscataloguedina
particularsource/targetcatalogpair
• Updatesallmetadatatoaccomplishtheidentityswappingofthesource
andtargetdatasets:
- Modifiesallvolume-basedmetadataintheVTOC,VTOCIXandVVDS
- Modifiesthecatalogentriesforthesourceandtargetdatasetpairs,and
refreshescatalogdatabuffersforthecatalogsinvolvedacrossallz/OS
imagesinasharedstoragecomplex.
Highlights
Updates to data sets in the
migration group are applied
simultaneously.
During subphase 1 of the diversion
phase, zDMF software swaps the
identities of the source and target.
Implementing Softek zDMF technology.Page 17
Subphase 2: diversion of I/O from active applications to target data set
With the source and target data set identities switched, I/O to any of the source data sets previously allocated by ongoing applications is diverted to the target data set instead. It is at this point that the data set(s) has been migrated from the source to the target volume. All references and I/Os to the data set will now be to the target volume.
The original source data set is renamed, is no longer in use and can not be accessed for alteration including being deleted until the migration goes into the completion phase.
Note: Once the divert command processing is completed, any new application allocations will automatically be directed to the target data set.
Post Migration Completion phase
Migration groups will remain in the diversion phase until all applications have relinquished their allocation of migrating data sets and zDMF software no longer has to redirect I/O to the targets. Group completion is achieved by logi-cal partition (LPAR) in the shared storage complex, depending on the data set usage of the applications on each server.
Each server periodically examines allocation information to see if any active address space still has any source data set allocated. Once all such allocations are freed through the normal action or completion of the system or application processing, then it is no longer necessary for zDMF software to divert any I/O. Locally, the zDMF processing is complete for the group. However, the group cannot be marked fully complete until all LPARs reach the same state.
Highlights
During subphase 2, the source
and target data set identities are
switched. All references and I/Os
are set to the target volume.
Group completion occurs when all
LPARs reach the same state.
Implementing Softek zDMF technology.Page 18
Identifying address spaces being diverted
Identifying the z/OS address spaces that are being diverted is a simple process with zDMF software. By using the zDMF TSO Monitor Option 2 – Interact With Promoted Groups, the user can display all address spaces that are cur-rently diverted across all z/OS images in a shared storage complex.
Post-completion phase
Data migrations in many cases are undertaken to free up a storage resource. Post-migration actions with the freed storage resource may include removing it from the system to return to a leaseholder, or reusing the storage for other application data needs. Regardless of the reason, before the resources can be reused, the user must take the following actions.
Initiate post-completion storage resource cleanup
1.Ensurethatthemigrationgrouphascompleted.
2.Deletethemigrationgroupstargetdatasets(renamedoriginalsourcefileon
originalsourcevolume).
3.DeletethemigrationgroupfromthezDMFdatabase.
When a migration group is deleted, zDMF software performs the following tasks:
• ThezDMFserverensuresthatthemigrationgroupisinastatethatwill
allowittobedeleted.
• ThezDMFserverremovesthemigrationgroupfrominternalz/OSstorage
(memory).Thisresultsinvaluablesystemmemorybeingfreed.
• ThezDMFserverdeletesthemigrationgroupfromthezDMFdatabase.
Highlights
zDMF software enables easy
identification of the z/OS address
spaces that are being diverted.
Before freed-up resources
can be used, a user must
initiate post-completion
storage resource cleanup.
Implementing Softek zDMF technology.Page 19
A typical migration
It is very important to be aware that the most critical phase of any migration is establishing a good migration plan. The more time spent in developing a plan, the better the odds are of a successful migration. This section details the steps of a typical storage migration. As with any complex process, the basic questions of why, what, when and how are key to understanding what kind of planning process will be needed for this migration.
• Why—Atechnologyupgradeisbeingperformedbecausethecurrentstorage
subsystemiscomingofflease.
• What—Dataneededtobemigratedfromoldtonewstorage.Whileperform-
ingthisconversion,itwasalsodecidedtoinstallsomenewhigher-capacity
drives.Themigrationparameterswouldbeasfollows:
- 800MOD-3to800MOD-32270GB
- 120MOD-3to40MOD-9340GB
- 200MOD-9to200MOD-91702GB
- 240MOD-9to80MOD-272043GB
- With1360volumesto1120volumes,thetotalis6355GB.
• When—Thenewsubsystemwastoarriveandbeinstalledbythefirstweek
ofMarch.TheoldsubsystemwouldcomeoffleaseonMarch31.Toavoid
costlystorageoverlap,managementwantedthemigrationtobecompleted
bytheendofMarchsotheoldsubsystemcouldbereleased.
• How—Itwasquicklydecidedthatconventionaldisk-to-diskcopywasnot
feasibleduetothelongapplicationoutagewindowthatwouldberequired.
Also,thevendor’shardwaremigrationutilitywasnotusableduetocompat-
ibilityissuesgoingfromanEnterpriseSystemsConnection(ESCON)toFiber
Connectivity(FICON)onthenewsubsystem.Ultimately,acombinationof
host-basedutilitiesandproductswaschosentoperformthemigration.
Highlights
Developing a thorough
migration plan is key to a
successful migration.
Example migration scenario:
A technology upgrade is taking
place because the current storage
system is coming off lease.
The IT organization chose a com-
bination of host-based utilities and
products to perform the migration.
Implementing Softek zDMF technology.Page 20
Because the environment consisted of 80 to 90 percent SMS, the operations staff believed that they could take advantage of SMS redirection, DFDSS to copy unallocated files (as explained in the section “Executing the migration” below, the operations staff eventually decided to use zDMF software in place of DFDSS), Softek Transparent Data Migration Facility (TDMF™) for volume-level migrations, and Softek zDMF for allocated files.
Selection logic
The Softek products were selected by the manager of infrastructure stor-age based on colleagues’ recommendations, as the manager has never used Softek products before. In addition, the fact that Softek products are designed to support vendor-neutral, nondisruptive migration and received a strong endorsement from the storage vendor made for a fairly easy decision.
Executing the migration
It was decided to perform the most complex migration first, i.e., migrating the smaller to larger volumes. There were a number of reasons for this, which are highlighted below.
Step 1: ICKDSF Minimal Init
Rather than initializing the new volumes with VOLID that corresponded to their naming convention and including them in the proper SMS storage pool, operations would simply do an ICKDSF Minimal Init with a VOLID of XXucb#. No SMS updates are done to include these volumes in the SMS pools.
Step 2: Migrating smaller to larger volumes with TDMF
TDMF software was used to migrate the first smaller volume to a larger volume, i.e., one MOD-9 to a MOD-27. This had the advantage of completing one-third of the migration that required resizing of capacity without any disruption to production. It took less than one hour to migrate 40 MOD-3 to MOD-9 vol-umes and about four hours to migrate 80 MOD-9 to MOD-27 volumes.
Highlights
Using Softek software, the team
decided to perform the most com-
plex migration first.
Migrating the smaller volumes to a
larger volume first eliminated pro-
duction disruption.
Implementing Softek zDMF technology.Page 21
As the source VOLID was being copied to the target volume, no SMS changes needed to occur because the volume was already part of their desired SMS rules. Also, the VOLID naming standards followed to the new volume.
Softek TDMF migration was set up to dynamically invoke ICKDSF to reformat and expand a volume’s VTOC. This function is performed when the source VTOC characteristics do not match those of the target device.
TDMF Dynamic ICKDSF EXTVTOC Function
The ICKDSF EXTVTOC option is only invoked if migrating from one size device to a different size device (e.g., smaller to larger) or if a volume had been previously migrated and no REFVTOC had been performed. Only indexed VTOCs are extended. Nonindexed VTOCs, including volumes with damaged indexes, will be “REFVTOCed.”
The user can specify a specific number of tracks for the VTOC, or TDMF software can use its own algorithm as follows:
The minimum size of the new VTOC is the greater of the current VTOC size on the source and target. The VTOC may be extended further depending on the number of data sets on the volume:
• Ifthevolumeislessthanhalffull,theVTOCwillbeextendedto
containthecurrentnumbersofdatasetsmultipliedbytheratioof
targettosourcedevicesize,plus25percent.
• Ifthevolumeismorethanhalffull,theVTOCwillbeexpandedto
handlethesituationwherethetargetvolumeisfullofdatasetswith
thesameaveragesize.
If the index needs to be extended, but the VTOC does not, Softek TDMF software will attempt to extend the VTOC by one track. If the VTOC cannot be extended because there is a data set adjacent to it, REFVTOC will be performed, unless there is insufficient space in the index for the additional VPSMS.
Highlights
The ICKDSF EXTVTOC option
allows the user to specify the
number of tracks for the VTOC.
Implementing Softek zDMF technology.Page 22
Step 3: Migrating using SMS processes
The remaining two-thirds of the source volumes that had been designated as part of the consolidation to larger-capacity volumes were placed in SMS “DISNEW” status. This would automatically migrate files that were deleted and reallocated during daily and weekly production runs to the new higher-capacity devices. Also this would prevent SMS from allocating any new files/extents on the old volumes.
Operations staff could now go on to their other migration tasks and let normal SMS routines manage moving many of the files. At some point, they would need to revisit this task to determine which files SMS did not migrate.
Step 4: Migrating volumes one for one using TDMF
In this step, operations staff used TDMF software to migrate the remaining disks that were one for one. This included 800 MOD-3 to MOD-3 and 200 MOD-9 to MOD-9 DASD. Upon completion of the TDMF migration, operations started the zDMF migration to finalize data migration requirements in step three above that SMS did not migrate.
Note: The TDMF migration in step four took three days to accomplish, working only during regular business hours (8 a.m. to 5 p.m., Monday through Friday). Additional time with no migration activity was allowed so that SMS could reallocate many files to the higher-capacity devices as described in step three.
Highlights
Next, the IT staff began migration
using SMS processes.
The fourth step involved migrating
volumes using Softek TDMF software.
Implementing Softek zDMF technology.Page 23
Step 5: Migrating using zDMF
The only migration that remained to be performed was of the files that had not been reallocated by SMS to higher-capacity disks during steps one through four. This migration would be completed by zDMF software.
The files that remained were files that had not been through a processing cycle, such as weekly job run; files that had been created a long time ago and never got deleted; and files that were constantly in use such as the IBM DB2® and IBM CICS® files.
Consideration was given to using DFDSS to migrate the remaining files not in allocation, but it was determined that by using zDMF software, operations staff could accomplish the same thing and also handle data sets that were in use at the same time. In addition, zDMF software also provided the flexibility of allowing data sets to go through allocation during this process, whereas DFDSS would have had periodic allocation issues as production continued to run.
Step 6: Completion
All files except the ones with persistent allocations were migrated to comple-tion. The files that remained in zDMF diversion mode were identified using the zDMF TSO Monitor; those applications were then scheduled to be bounced over the weekend.
Once zDMF software completed all data set migrations, the storage migration project to the new technology was complete.
Highlights
Next, the teams migrated files that
were not reallocated.
The completion of the migration
involved files that remained in
diversion mode.
Implementing Softek zDMF technology.Page 24
Step 7: Verification
One additional day was spent to verify that all data had migrated as planned and that post-migration tasks had been completed.
Migration timeline
The following was the timeline for the migration described above:
• Premigration:installnewstoragefirstweek
• Migrationstep1:ICKDSFINITnewstorage
• Migrationstep2:TDMFsmallvolumestolargevolumes
• Migrationstep3:SMS“DISNEW”redirection
• Migrationstep4:TDMFequalsizevolumes
• Migrationstep5:zDMFsmalltolargevolume“files”(filesnotmovedby
SMSredirection)
• Migrationstep6:Scheduledapplicationbounce(ifrequired)
• Migrationstep7:Cleanup
• Post-migration:Twoweekstoremoveoldstorage(installation,migration
andde-installationcompleteinonemonth)
Figure 9 Sample migration timeline
Highlights
The staff spent an additional day
verifying that all data was migrated.
This list and diagram summarize
the migration timeline.
Implementing Softek zDMF technology.Page 25
Performance considerations
TDMF performance management options
TDMF software has a number of options to manage performance during a migration. Some specifically set parameters as to how TDMF software will execute the migration, while others are dynamic and self-adjust depending on resource utilization at a given point in time.
zDMF performance management options
zDMF performance can be adjusted during a migration with the following options. In its current release, the dynamic options must be manually altered during the course of a migration because they are not self-adjusting.
Non-dynamic Dynamic
FASTCOPYFULLSPEEDCONCURRENT volume migrationsSYNCgoal
PACING (dynamic or fixed)
Note: Pacing is based on user I/O perfor-mance and CPU memory utilization.
Non-dynamic Dynamic
MAXIOMAXTRK
MAX_CHANNEL_IOMAX_DEVICE_IO
Note: Parameters may be changed during the course of a migration by the use of z/OS console commands.
Highlights
Softek TDMF software manages
performance during a migration.
The zDMF performance can be
adjusted during a migration.
Implementing Softek zDMF technology.Page 26
Performance rule of thumb
All migrations, regardless of method, will vary in performance depending on the particular environment in which they are carried out. Among the factors that can impact performance are:
• Logical-to-physicaldisklayout(a72GBphysicaldiskcanhave
25MOD3logicaldisks)
• Typeandnumberofchannels
• Enduser/applicationI/Oactivity
• Datachangerate
• Amountofcacheandcachehitratios
• Numberofconcurrentdatamigrations.
In addition, host-based migration products can be affected by CPU utilization and by the amount and usage of processor memory. Both TDMF and zDMF software act as typical disk-to-disk utilities with very low CPU utilization, but they are capable of driving the I/O subsystem to its capacity.
The sections below provide estimates for typical TDMF and zDMF throughput during an average migration. (Note that for the purposes of these examples, numbers have been rounded off.) It is also important to notice that the esti-mates for zDMF software are lower than they are for TDMF software (100GB vs. 180GB per hour). Keep in mind that compared to TDMF software, zDMF technology must monitor many more migration entities (thousands of files/extents) and deal with more components (such as adhering to the customer’s SMS policies and modifying metadata).
Highlights
Several factors can affect the migra-
tion performance, including the data
change rate.
The following section gives
estimates for Softek zDMF and
TDMF throughput during an
average migration.
Implementing Softek zDMF technology.Page 27
TDMF throughput estimates
• 180GB(64MOD-3volumes)perhourbasedon12concurrentvolumes
overESCON
Sample TDMF migration:
• 400MOD-3volumes(1.1TB)to400MOD-3volumes—
(64volumes/hr=6hours)
• 400MOD-9volumes(3.4TB)to400MOD-9volumes—
(21volumes/hr=19hours)
• Noapplicationoutage
• CPUlessthan3percentforthemasterandlessthan1percentperagent
(TDMFsoftwareisrequiredtorunoneachLPARthatissharingDASD;one
LPARwillbedesignatedastheTDMFmaster,andaTDMFagentwillrun
oneachoftheotherLPARs.)
zDMF throughput estimates
• 100GBperhour
Sample zDMF migration:
• 800MOD-3volumes(2.3TB)to400MOD-9volumes—(100GB/hr=23hours)
• Note:Minimal(scheduled)applicationoutagemayberequired.Although
thefileshavebeenmigrated,somedatasetswithpersistentallocations
willremainindiversionmodeuntilascheduledapplicationbouncecanbe
scheduled.Thetimeestimateisonlyforfilereplication.
Highlights
Softek TDMF throughput esti-
mates and a sample migration
are given here.
Implementing Softek zDMF technology.Page 28
Tips for conducting large migrations
When a migration includes only like volume-to-volume migrations, consider using a product that migrates at the volume level, such as Softek TDMF soft-ware. When a migration includes different disk size capacities for the source and target volumes, consider using a combination of Softek TDMF and zDMF software. Use TDMF technology to move the first source volume to the target and then use zDMF technology to move the other source volumes to the same target.
In addition to making this migration easier, this method of performing migra-tion has the advantage of not requiring users to introduce a “new” VOLID into the installation that must follow some standard as well as avoiding concern over introducing a new VOLID into the SMS environment.
To use a simple analogy, picture the data to be migrated as a haystack that has to be moved from point A to point B. The farmer could use a forklift (TDMF software) to quickly move the haystack in large chunks, but the farmer would probably wind up having to leave behind a lot of the hay that was too small for the forklift to pick up. Alternatively, the farmer could use a pitchfork (zDMF software) that allows moving small bundles of hay, but it would be harder and take longer to move the haystack. The best way to tackle the job, then, would be to use some combination of both the forklift and the pitchfork, TDMF and zDMF technology, to quickly and completely move the hay.
Highlights
When a migration includes large
volumes of data, businesses
should consider using Softek
TDMF software in conjunction
with zDMF software.
Implementing Softek zDMF technology.Page 29
Platform support
• TDMFsoftware:IBMOS/390®2.10andalllevelsofIBMz/OS
• zDMFsoftware:IBMz/OS1.4andhigher
Restrictions
TDMF software:
• VolumeswithActivePageorSwapdatasets
• VolumecontainingtheactiveTDMFSYSCOMdatasetforthesession
zDMF software:
• Systemtypedatasets(e.g.,datasetsinLinkListorAPFauthorizedto
aspecificvolume)
• VSAMKSDSfileswithIMBED,KEYRANGEandREPLICATEdefined
• Catalogues
• ISAMdatasets
• Individualmemberswithinapartitioneddataset(PDS)
• UNIX®SystemServices(USS)HFSorzFSdatasets
• Pagedatasets
• Undefineddatasets(DSORG=UorDSORG=PSU)
• VSAMVolumedatasets(VVDS)
• VTOCs
• Temporary(&&)datasets
Highlights
Platform support and restrictions
are included here.
Implementing Softek zDMF technology.Page 30
In addition to the above, the following restriction also applies:
• Controlunitsmusthaveequalorhigherfunctionality.Forexample,move-
mentfroma3990controlunittoa2105controlunitisallowed;reversal
ofthismovement,however,isnotallowed.Thereasonforthisrestrictionis
thatsomeapplicationssuchasDB2technologytakeadvantageofexpanded
channelcommandword(CCW)commandsets.Ifsuchamigrationwereto
beallowed,acommandrejectwouldensue,andtheapplicationcouldexpe-
rienceunexpected,andpossiblyproblematic,results.
Other Softek migration solutions
Softek zDMF software is designed and optimized specifically for local data migrations at the file extent level. Softek TDMF for z/OS technology is designed for volume-level migrations involving movement for local or remote distances (also known as “global migrations”). Data can be moved across a TCP/IP net-work (LAN or WAN) or channel extenders.
Softek zDMF and TDMF products can both be used in the same environment to address different migration project requirements. zDMF and TDMF soft-ware is designed to provide a very fast, easy, optimized migration solution for data migrations.
Softek also offers TDMF software for open system platforms.
Highlights
Softek TDMF and zDMF software
can be used in the same envi-
ronment to address different
migration needs.
Implementing Softek zDMF technology.Page 31
Summary
IT organizations looking to take advantage of large-capacity volumes on today’s high-performance storage subsystems are faced with complex and disrup-tive data conversions that can have a negative impact on important business applications. zDMF software can give users the power to consolidate data onto large-capacity volumes without interruption to the 24x7 business environment.
zDMF software can help ensure that applications maintain maximum availabil-ity even as the underlying storage infrastructure is phased out and taken offline. zDMF software was designed specifically to help customers, storage vendors and integrators successfully and quickly move to new storage technology.
For more information
For more information about implementing Softek zDMF technology, visit:
ibm.com/services/datamobility
Highlights
zDMF software is designed to
support the needs of today’s
24x7 business environment.
© Copyright IBM Corporation 2008
IBM Global Services Route 100 Somers, NY 10589 U.S.A.
Produced in the United States of America 06-08 All Rights Reserved
IBM, the IBM logo, CICS, DB2, zDMF, Nonstop Data Mobility, OS/390, Softek, TDMF and z/OS are trade-marks or registered trademarks of International Business Corporation and other companies in the United States, other countries, or both.
Microsoft is a trademark of Microsoft Corporation in the United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Other company, product and service names may be trademarks or service marks of others.
References in this publication to IBM products or services do not imply that IBM intends to make them available in all countries in which IBM operates.
Performance/capacity results or other technical statistics appearing in this document are provided by the author solely for the purposes of illustrat-ing specific technical concepts relating to the Softek products discussed herein. The perfor-mance/capacity results or other technical statistics published herein do not constitute or represent a warranty as to merchantability, operation, or fitness of any Softek product for any particular purpose.
SDW03009-USEN-02