Top Banner
© 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency CICS VSAM Transparency Session Number: C49 Session Number: C49 Satish Tanna Satish Tanna [email protected] [email protected]
64

© 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

Apr 01, 2015

Download

Documents

Forrest Salton
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© 2007 IBM CorporationConference materials may not be reproduced in whole or in part without the prior written permission of IBM.

CICS VSAM TransparencyCICS VSAM Transparency

Session Number: C49 Session Number: C49

Satish TannaSatish Tanna

[email protected][email protected]

Page 2: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

AGENDA

What is CICS VT?

Product highlights

Components

Migration process summary

Other enhancements in CICS VT 1.2 Operational enhancements Performance enhancements

Supported software levels

Page 3: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

WHAT EXACTLY IS CICS VT?

Tool to migrate VSAM files to DB2 without changing application programs

Legacy programs access DB2 data using driver modules CICS VT generates for each migrated data set

Migrated data can be accessed by SQL in new programs

Existing programs can be enhanced using SQL

Lowest risk migration strategy

Page 4: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

WHAT EXACTLY IS CICS VT (continued)?

VSAMFile #2

VSAMFile #1

ApplicationProgram

Before CICS VT

READ

WRITE, REWRITE

Page 5: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

DB2Table #1

CICS

VT

ApplicationProgram

VSAMFile #2

VSAMFile #1

Application programs unchanged

WHAT EXACTLY IS CICS VT (continued)?

After CICS VTINSERT, UPDATE

SELECT

GET, READ

WRITE, REWRITE

Page 6: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

Completely separate from application program

Transparent access to data in DB2

100% static SQL

Migrate on a file by file basis

Single live copy of data Dual mode available for test purposes

Data can be re-engineered

Same data returned to VSAM programs Enhanced data available using SQL

CICS VT HIGHLIGHTS

Page 7: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

Mapping component Establishes relationship between VSAM record layout and

DB2 row

Data migration component Utilities to migrate data to DB2 and re-engineer if required

Testing component Enables application programs to be tested following data

migration

Run time component Intercepts application calls to VSAM data sets migrated to

DB2

CICS VT COMPONENTS

Page 8: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

MAPPING COMPONENT“ The relationship between the record in VSAM and the row in DB2”

One time activity for each data set and alternate index

Manual and automated mapping methods

Automated method should handle at least 70% of all data sets

Run-time driver module generation

Performed by DB2 DBA and application programmer

Includes design of DB2 objects

Page 9: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

MAPPING COMPONENT

VSAM

01 VSAM-file. 02 file-key pic 9999 02 group-field. 03 order-number pic x(3) 03 order-type pic s9(8) comp 02 next-field pic s999 comp-3………………

MAPPING

DATAin DB2

CREATE TABLE VSAM_TAB ( TAB_KEY DEC(5, 0),ORDER_NO CHAR(3),ORDER_TYPE INTEGER,NEXT_FIELD DEC(3,0) PRIMARY KEY(TAB_KEY))

ISPF

Usually one to one, can be one to many

RUN-TIMEDRIVERS

COPYBOOK or DSECT

Page 10: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TYPICAL DB2 TABLE DESIGN - 1

01 ITEM-DET.02 ITEM-NUMBER PIC X(6).02 ITEM-NAME PIC X(12).02 ITEM-DESC. 03 ITEM-COLOUR PIC X(6). 03 ITEM-WEIGHT PIC X(4). 03 ITEM-COST PIC S99999V99 COMP-3.02 ITEM-REORDER-QUANTITY PIC S999.02 ITEM-SUPPLIER-CODE PIC X(3).02 ITEM-DATE-FIRST-SHIP PIC S9(9) COMP-3.02 ITEM-SHELF-LIFE PIC X(2).02 ITEM-DESCRIPTION PIC X(35) .

One VSAM file becomes one DB2 table Each copybook field becomes a DB2 column

Page 11: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TYPICAL DB2 TABLE DESIGN - 2

02 ITEM-NUMBER PIC X(6). 02 ITEM-NAME PIC X(12). 03 ITEM-COLOUR PIC X(6). 03 ITEM-WEIGHT PIC X(4). 03 ITEM-COST PIC S99999V99 COMP-3. 02 ITEM-REORDER-QUANTITY PIC S999. 02 ITEM-SUPPLIER-CODE PIC X(3). 02 ITEM-DATE-FIRST-SHIP PIC S9(9) COMP-3. 02 ITEM-SHELF-LIFE PIC X(2). 02 ITEM-DESCRIPTION PIC X(35) .

Numeric fields re-engineered Date field re-engineered

CREATE TABLE VID_ITEM (ITEM_NUMBER CHAR(6) ,ITEM_NAME CHAR(12) ,ITEM_COLOUR CHAR(6) ,ITEM_WEIGHT CHAR(4) ,ITEM_COST DEC(7,2) ,ITEM_REORDER_NO SMALLINT ,ITEM_SUPP_CODE CHAR(3) ,ITEM_DATE_FSHIP DATE ,ITEM_SHELF_LIFE CHAR(2) ,ITEM_DESCRIPTION CHAR(35)

Page 12: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

MANUAL VERSUS AUTOMATED MAPPING

Manual Mapping

User inserts information for each copybook field User creates DDL Additional mapping for alternate indexes

Automated Mapping

Mapping and DDL generated by CICS VT Capability to edit column names and attributes Automated alternate index mapping

Page 13: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

AUTO-MAPPING REQUIREMENTS

There is a valid copybook or DSECT for a data set Copybook is written in ASM, Cobol or PL/I and will

assemble/compile There are no COPY statements in the copybook The data set will be migrated to a single table There is a maximum of 5 levels of nested redefines The maximum OCCURS clause is 99 MACRO and MEND statements must be removed

from DSECTS User is responsible for copybook/DSECT “quality”

Page 14: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

AUTO-MAPPING PROCESS

Step 1 - Batch utility gathers base cluster information

Step 2 - User specifies copybook or DSECT for file

Step 3 - Optionally edit generated columns

Names and attributes REDEFINES

Step 4 - JCL generated and submitted

Step 5 - driver modules generated

Page 15: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

When data set will be mapped to multiple tables using exits

Multiple record types Repeating groups of may fields To introduce normalisation

DDL created by user

Data set mapped by user

Alternate indexes mapped by user

Batch mapping utility

MANUAL MAPPING

Page 16: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

Three steps in data migration process

1. Unload existing data from VSAM data set

2. Convert data to DB2 format

3. Load DB2 data (using DB2 LOAD utility)

CICS VT utilities provided for 1 and 2

Optionally identifies invalid numeric fields that are mapped to numeric column types (which would cause data conversion errors)

DATA MIGRATION COMPONENT

Page 17: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

STEP 1 - UNLOAD VSAM DATA SET

Supplied batch utility program VIDUNLOD

//VIDUNLOD EXEC PGM=VIDUNLOD //STEPLIB DD DSN=VID.VIDLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //FILEIN DD DSN=vsam.file,DISP=SHR //FILEOUT DD DSN=vsam.unload,UNIT=SYSDA,// DISP=(,CATLG,DELETE),RECFM=FB,LRECL=108,// SPACE=(CYL,(25,5),RLSE)

Creates sequential data set

Must be RECFM=FB LRECL = longest record length plus 8 bytes for KSDS and 12

bytes for RRDS

Page 18: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

STEP 2 - REFORMAT VSAM DATA - 1

Supplied batch utility program VIDLOAD

//VIDLOAD EXEC PGM=VIDLOAD,PARM=dimname //STEPLIB DD DSN=VID.DRIVERS.LOAD,DISP=SHR // DD DSN=VID,LOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //LOADIN DD DSN=vsam.unload,DISP=SHR //LOADOUT DD DSN=vsam.db2load,UNIT=SYSDA,// DISP=(,CATLG,DELETE),RECFM=FB,LRECL=100, // SPACE=(CYL,(25,5),RLSE)

Creates reformatted sequential data set ready for DB2 load

Must be RECFM=FB LRECL=length of longest DB2 record CICS VT invoked to perform data re-engineering

Page 19: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

STEP 3 - LOAD DATA INTO DB2

Uses standard DB2 LOAD utility (the most efficient way to load records into a DB2 table)

LOAD utility control cards generated by DB2 sample program DSNTIAUL

Will require WHEN statements if migrating one file to multiple DB2 tables

Page 20: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

If file is mapped to multiple tables, DD name for output data is different.

//VIDLOAD EXEC PGM=VIDLOAD,PARM=dimname //STEPLIB DD DSN=VID.DRIVERS.LOAD,DISP=SHR // DD DSN=VID,LOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //LOADIN DD DSN=vsam.unload,DISP=SHR //LOADOUTM DD DSN=vsam.db2load,UNIT=SYSDA,// DISP=(,CATLG,DELETE),RECFM=FB,LRECL=100, // SPACE=(CYL,(25,5),RLSE)

Relies on code in IRD exit to identify appropriate record type

REFORMAT VSAM DATA - 2

Page 21: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

Implemented as 4 tables in CICS VT One FBE exit and one IRD exit to manage the data

Table 1 is mapped to CICS VT and contains the key columns Table 2 contains type #1 records and is accessed by the exits Table 3 contains type #2 records and is accessed by the exits Table 4 contains type #3 records and is accessed by the exits

IRD exit must set table indicator in first 2 bytes of output record

REFORMAT VSAM DATA - 3

Example: FILE X has 8-byte key and contains 3 record types. Sub-field of key indicates record type.

Page 22: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

00010000130101000013DATA1000000000000041DATA1000000000000042DATA100000000000

004300020000200202000020DATA20000000011DATA20000000012DATA10000000013DATA2000000

001400020000210202000021DATA20000000021DATA20000000022DATA10000000023DATA2000000

002400020000220202000022DATA20000000031DATA20000000032DATA10000000033DATA2000000

003400030000300303000030DATA30000011DATA30000012DATA30000013DATA30000014DATA300

0001500030000310303000031DATA30000021DATA30000022DATA30000023DATA30000024DATA300

00025

REFORMAT VSAM DATA - 4

Data produced by VIDLOAD with LOADOUTM

LOAD DATA REPLACE INDDN SYSREC00 INTO TABLE TABLE_02 WHEN (1:2) = '02'

KEY

Page 23: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

DATA MIGRATION - THE REALITY

Potentially old legacy files with high probability of inconsistent data

Numeric field values with BLANKS, binary zeroes Inconsistent date values (how about a value of ASAP?)

VIDLOAD can be used to identify invalid numeric values

DB2 LOAD will fail if it encounters invalid column values

Potential to use exits for verification

Page 24: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

Additional DD statement added to VIDLOAD

//VIDLOAD EXEC PGM=VIDLOAD,PARM=dimname //STEPLIB DD DSN=VID.DRIVERS.LOAD,DISP=SHR // DD DSN=VID,LOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //LOADIN DD DSN=vsam.unload,DISP=SHR //LOADOUT DD DUMMY//VIDS0C7 DD SYSOUT=*

IDENTIFY INCONSISTENT DATA

All invalid numeric fields identified

VID DATA EXCEPTION REPORT RECORD 0000000102 OFFSET 0029 DATASET VIDKSDS FIELD NAME VIDF004 LENGTH 0004 VALUE xxxx RECORD 0000000356 OFFSET 0063 DATASET VIDKSDS FIELD NAME VIDF007 LENGTH 0003 VALUE yyy

FBE exit might be required for recurring fields

Page 25: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

INCONSISTENT DATA - THE OPTIONS

Invalid data

Perform data cleansing exercise Map inconsistent fields to CHAR columns, correct

invalid data and remap to numeric columns and re-migrate

Perform data validation/correction using CICS VT exits Correct data during migration

May have to identify and correct the application programs that are creating invalid data

Page 26: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

Dual mode facility (DMF) - major new V1.2 feature

Application program calls to migrated data sets are processed in DB2 and in VSAM

VT automatically compares record areas and call status areas Any difference is highlighted

Will significantly reduce application program test

Two main purposes:

a) Verify initial migrationb) Test user exits

Page 27: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

Using DMF to verify initial data migration The VT utility VIDUNLOD unloads a VSAM file to a sequential

data set The VT utility VIDLOAD converts sequential data set into a

DB2 load file according to the mapping and user exits The DB2 LOAD utility loads the DB2 table(s)

CICS VT V1.1 User runs VIDUNLOD against the migrated data in DB2 Compares unload data sets from VSAM and DB2

CICS VT V1.2: User runs VIDUNLOD enabled for DMF CICS VT automatically compares data

Page 28: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

VIDUNLOD JCL with and without DMF

//VIDUNLOD EXEC PGM=VIDUNLOD //STEPLIB DD DSN=VID.VIDLOAD,DISP=SHR//SYSPRINT DD SYSOUT=*//FILEIN DD SUBSYS=(ssi,db2id,dimname)//FILEOUT DD DSN=vt.unload,UNIT=SYSDA,// DISP=(,CATLG,DELETE),RECFM=FB,LRECL=108,// SPACE=(CYL,(25,5),RLSE)

//VIDUNLOD EXEC PGM=VIDUNLOD//STEPLIB DD DSN=VID.VIDLOAD,DISP=SHR//SYSPRINT DD SYSOUT=*//FILEIN DD SUBSYS=(ssi,db2id,dimname,VSAMDD)//FILEOUT DD DUMMY,LRECL=108,RECFM=FB//VSAMDD DD DSN=original-VSAM-file,DISP=SHR

Any DD name can be specified

Page 29: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

Using DMF for batch application testing

Add 4th parameter to SUBSYS DD statement Add DD statement for original VSAM data set

//RTP050 EXEC PGM=appl-prog//STEPLIB DD DSN=VID.VIDLOAD,DISP=SHR//SYSPRINT DD SYSOUT=*//ORDERFL DD SUBSYS=(ssi,db2id,dimname,ORDERFLV)//ORDERFL@ DD DSN=hlq.ORDERFL.DUMMY,DISP=SHR//ORDERFLV DD DSN=hlq.ORDERFL,DISP=SHR

Any difference detected by DMF causes program abend

U3139 indicates a data difference U3141 indicates a return code or reason code difference

Page 30: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

DMF U3139 abend

VIDSS227 DUALMODE ERROR FOR SUBSYS=VIDZ VSAM DDNAME=ENTITLVRECORD DATA WAS NOT THE SAME AT OFFSET X"0036"

VIDSS219 DUMP OF DB2 RECORD:VIDSS220 +0016 F0F0F0F0 F0F0F0F4 F1E4F4F1 E5F4F100 *000000041U41V41.*VIDSS220 +0026 00004040 40404040 40404040 40404040 *.. *VIDSS220 +0036 40404040 40404040 40404040 40404040 * *VIDSS220 +0046 40404040 40404040 40404040 40404040 * *VIDSS219 DUMP OF VSAM RECORD:VIDSS220 +0016 F0F0F0F0 F0F0F0F4 F1E4F4F1 E5F4F100 *000000041U41V41.*VIDSS220 +0026 00004040 40404040 40404040 40404040 *.. *VIDSS220 +0036 C9C9C9D5 D5D5C6C6 C6404040 40404040 *IIINNNFFF *VIDSS220 +0046 40404040 40404040 40404040 40404040 * *

Hex offset of first byte identified in error message

X’20’ bytes before and after is displayed

Page 31: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

DMF U3141 abend

VIDSS228 DUALMODE ERROR FOR SUBSYS=VIDZ VSAM DDNAME=ENTITLV FEEDBACK DIFFERENT ON PUT DB2 RPLRTNCD=00 RPLERRCD=00 VSAM RPLRTNCD=08 RPLERRCD=08 ACB MACRF=(ADR,SEQ,DIR,IN) RPL OPTCD=(SEQ,KEY) KEYL=0025

Call parameter list is displayed

Non-significant VSAM fields such as component code are ignored by DMF

Most errors will be a result of unsynchronised VSAM and DB2 data

Page 32: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

Using DMF for CICS application testing

New file definition for VSAM data set must be added to CICS File name is derived from original file name Two new DMF parameters in the VT data set table (DST) DMF is activated/deactivated with new VT transaction

VTMA

2 new transaction abend codes for DMF 3139 for data differences 3141 for RESP code differences

Errors written to new DD statement VIDCOMP

Page 33: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

Enabling DMF in CICS

1) Add new CICS file definition for VSAM data set File name is derived from original file name

Add ‘@’ if original file name less than 7 characters Replace 1st character with ‘@’ if file name is 8 characters

2) Two new DMF parameters in the VT data set table (DST) ERRORACT=ABEND|REPORT DUMP=YES|NO (only relevant if

ERRORACT=ABEND)

3) Issue VTMA START

+VIDCINIC - CICS VT DMF successfully initialised.

Page 34: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

Using the VTMA transaction

VTMA STATUS

VTMA START

VTMA STOP

VTMA RESTART

Optionally add VIDCINIC program to PLT to activate DMF at CICS start up if required

Page 35: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

CEMT I FI(*)

STATUS: RESULTS - OVERTYPE TO MODIFY Fil(VIDKSDS ) Vsa Ope Ena Rea Upd Add Bro Del Sha Dsn( VID.VIDKSDS.DUMMY ) Fil(VIDKSDS@) Vsa Ope Ena Rea Upd Add Bro Del Sha Dsn( VID.VIDKSDS ) Fil(ORDERFIL) Vsa Ope Ena Rea Upd Add Bro Del Sha Dsn( VID.ORDERFIL.DUMMY ) Fil(@RDERFIL Vsa Clo Ena Rea Upd Add Bro Del Sha Dsn( VID.ORDERFIL ) Fil(VIDRRDS) Vsa Ope Ena Rea Upd Add Bro Del Sha Dsn( VID.VIDRRDS.DUMMY )

Files VIDKSDS and ORDERFIL will be processed by DMF

Page 36: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

VTMD transaction

03/21/05 ==> CICS VT File Definition List <== 10:32:57

Filename DIM name Org Status Operations CompareDFHCMACD DFHCMACD ENA INI REA BRO DEL REP N

VIDKSDS VIDKSDS KSDS UNE REA UPD ADD BRO DEL VIDKSDS VIDKSDS KSDS UNE REA UPD ADD BRO DEL REP NREP NKSDS01 KSDS01 ENA INI REA UPD ADD BRO DEL REP NRRDS01 RRDS01 ENA INI REA UPD ADD BRO DEL ABEND N

ORDERFIL ORDERFL ENA INI REA UPD ADD BRO DEL ORDERFIL ORDERFL ENA INI REA UPD ADD BRO DEL ABEND YABEND YTVDP01 TVDP01 ENA INI REA UPD ADD BRO DEL ABEND Y

End of File table

Page 37: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

3139 transaction abend – data is different

********************************************************************** CICS VT COMPARE EXCEPTION **********************************************************************TRAN: K1P4 DIM: KSDS01 TERMINAL: 0019 TIME: 15.49.5300 REQ: 0000026

EXEC CICS READNEXT FILE(KSDS01 ) RIDFLD= 000 F0F0F0

RECORD AREA EXCEPTION @ OFFSET 00000017 -....5....0....5...+CICS VT .ORIGINAL.DATA.FOR.K 40D6D9C9C7C9D5C1D340C4C1E3C140C6D6D940D2VSAM .ORIGINAL.BAD.DATA.F 40D6D9C9C7C9D5C1D340C2C1C440C4C1E3C140C6

Offset is hex value – 10 bytes of data either side is dumped

Page 38: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

3141 transaction abend – return codes are different

********************************************************************** CICS VT COMPARE EXCEPTION **********************************************************************TRAN: K1P2 DIM: KSDS01 TERMINAL: 0019 TIME:15.31.3300 REQ:0000009

EXEC CICS READ FILE(KSDS01 ) RIDFLD= YYYYYY E8E8E8E8E8E8 EQUAL UPDATE

RETURN CODE EXCEPTION

RESP RESP2 RCODECICS VT 0000000D 00000050 810000000000VSAM 00000000 00000000 000000000000

Hex return code - NOTFND

Page 39: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

DMF checking varies by call

Data areas, return and reason codes for READ/GET/BROWSE calls

Return and reason codes for WRITE/REWRITE/PUT calls

Return and reason codes for ERASE calls. Count of records deleted is also checked

Data MUST always be synchronised or results are unreliable

Especially important in batch programs – an abend will leave VSAM data partially updated, but updates will be automatically backed out in DB2

Page 40: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

TESTING APPLICATIONS USING DMF

Can DMF be used in production?

If ERRORACT=ABEND then the transaction will fail If ERRORACT=REPORT transaction won’t fail but data may be

out of line, which will cause additional problems What happens if both versions of the file (DB2 and VSAM) are

not available? There is a performance overhead There is no scope to exploit 24x7 or online DB2 utilities You can’t run mixed online and batch workloads

Page 41: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

DATA MIGRATION PROCESS SUMMARY

ANALYSIS

TEST

MIGRATE

MAP

CUTOVER

Data inconsistencies

Data cleansing required Exits for data validation/correction IF SPACES test may dictate

column attributes

Page 42: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

RUN-TIME COMPONENT

Application programs continue to issue calls to VSAM files

CICS VT intercepts call

Using a global user exit (GLUE) in CICS Using an MVS sub-system for batch programs

Translates call to equivalent SQL call and processes in DB2

After call has been processed by CICS VT

Data and RESP codes return to CICS program Data and return and reason codes passed back to batch program

Page 43: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

RUN TIME CALL INTERCEPTION - CICS

CICS

VT

ApplicationProgram

CICS VT

GLUE

CICS VT

TRUE

VSAM

DB2

GLUE performs DST look up

Page 44: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

CICS VT DST - 1

CICS table containing a list of migrated data sets

Macros based on CICS FCT

VIDCDTAB VIDTAB TYPE=INITIAL VIDTAB TYPE=ENTRY, FILE=VIDKSDS, DIM=VIDKSDS, STATUS=ENA, OPEN=S, ADD=YES, BRO=YES, DEL=YES, REA=YES, UPD=YES, RESET=YES, RECORDF=FIX VIDTAB TYPE=FINAL

Page 45: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

CICS VT DST - 2

To add an entry for a migrated data set

Define DIM and DDM drivers, plus user exits to CICS Add data set information to table Assemble new table to DFHRPL library Restart the CICS VT interface

VTMI RESTART

Data set available for processing by CICS VT CICS VT installed and runs in AOR

Remote file requests no longer function-shipped to FOR

Page 46: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

DEFINING DRIVERS TO CICS VT

DIM and DDM drivers must be defined to CICS

Drivers are assembler programs Re-entrant in CICS VT 1.2

Must be defined RELOAD NO Can be defined RESIDENT if required Can be auto-installed (assuming default is RELOAD NO)

DDM driver size reduced in CICS VT 1.2

V1.1 drivers are not compatible with V1.2 and vice versa Must be re-generated

CICS VT user exits RELOAD=NO

Page 47: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

CICS VT - ADDITIONAL TRANSACTIONS

To control the CICS - CICS VT interface

VTMI with either STOP, START (or RESTART)

STOP disables CICS VT GLUE and effectively CICS VT START enables the CICS VT GLUE and activates call interception

Useful control for testing

MUST BE AUTHORISED IN PRODUCTION

Page 48: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

RUN TIME CALL INTERCEPTION - BATCH 1

MVS subsystem used for batch program call interception

SSI defined at CICS VT installation

Dynamic method - no IPL required Permanent method - IPL required

Default CICS VT ssid = VIDS JCL changes required to enable call interception

Page 49: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

RUN TIME CALL INTERCEPTION - BATCH 2

JCL changes

Before:

//ORDERFL DD DSN=ORDER.VSAM.DATASET,DISP=SHR//ORDERFL1 DD DSN=ORDER.VSAM.AIX,DISP=SHR

After:

//ORDERFL DD SUBSYS=(VIDS,ssid,DIM-name)//ORDERFL1 DD SUBSYS=(VIDS,ssid,DIM-name)

VIDS = CICS VT SSI, ssid = DB2 subsystem

DD statements for base cluster and AIX paths

Page 50: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

RUN TIME CALL INTERCEPTION - BATCH 3

Cobol and PL/1 programs use undocumented interface to SHOWCB macro during initialization

CICS VT uses dummy VSAM data set to achieve same

User must create dummy data set with same attributes as migrated data and add DDNAME to batch JCL

Before:

//ORDERFL DD DSN=ORDER.VSAM.DATASET,DISP=SHR

After:

//ORDERFL DD SUBSYS=(VIDS,ssid,DIM-name)//ORDERFL@ DD DSN=ORDER.DUMMY.VSAM.DATASET,DISP=SHR

Page 51: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

PK14457 ENHANCEMENTS - 1

Previously, DB2 plan name had to match DIM name. The same plan/package attributes applied to every application program accessing a migrated file

New DD statement required

Before PK14457:

//ORDERFL DD SUBSYS=(VIDS,ssid,DIM-name) DIM-name = PLAN-name

After PK14457:

//ORDERFL DD SUBSYS=(VIDS,ssid,DIM-name)//VIDPLAN DD *any_plan

Page 52: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

PK14457 ENHANCEMENTS - 2

Benefits for read only programs:

Example 1: Exploit DB2 uncommitted read

Bind DDM package into new collection (VIDUR?) with ISOLATION(UR)

Bind collection into new plan and specify in VIDPLAN DD

Example 2: Exploit DB2 lock avoidance

Bind DDM package into new collection (VIDNOLK?) with CURRENTDATA(NO)

BIND collection into new plan and specify in VIDPLAN DD

Page 53: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

PK14457 ENHANCEMENTS - 3

Example 3: Mixing packages

An application program accesses 3 files as follows:

VSAM1 is updated VSAM2 is read only but cannot access uncommitted updates VSAM3 is read only and can access uncommitted updates

1. Bind package VSAM2 with CURRENTDATA(NO) into collection VIDNOLK

2. Bind package VSAM3 with ISOLATION(UR) into collection VIDUR3. Bind plan job-name PKLIST(VIDCOLL.VSAM1, VIDNOLK.VSAM2,

VIDUR.VSAM3)

4. Specify plan job-name in VIDPLAN DD statement

Page 54: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

BATCH SUBTASKING - 1

Optional facility for existing programs that include SQL calls

Provides multiple DB2 threads from a single batch TCB

UPDATE-KSDS-01. MOVE WS-ITEM-QUANTITY TO ITEM-QUANTITY . MOVE WS-ITEM-NUMBER TO ITEM-NUMBER . REWRITE ITEM-DET. CICS VT migrated data set UPDATE-DB2TAB. EXEC SQL UPDATE ORDER_TAB SET ORDER_DATE = CURRENT DATE COMMIT END-EXEC.

Page 55: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

BATCH SUBTASKING - 2

Without subtasking:

one thread to DB2 updates to natively accessed tables plus CICS VT

accessed tables are COMMITted

With subtasking:

multiple threads to DB2 COMMIT scope is natively access table only Updates to CICS VT DB2 objects determined by outcome of

job

Implications for restart and recovery

Page 56: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

BATCH SUBTASKING - 3

Enabled by adding optional DD statement

Before://ORDERFL DD DSN=ORDER.VSAM.DATASET,DISP=SHR

After://ORDERFL DD SUBSYS=(VIDS,ssid,DIM-name)//ORDERFL@ DD DSN=ORDER.DUMMY.VSAM.DATASET,DISP=SHR//VIDSUBSW DD DUMMY

Mandatory for existing DB2 programs using DSN attach as opposed to CAF

Measurable performance overhead

Page 57: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

MIGRATION PROCESS SUMMARY

ANALYSIS

TEST

MIGRATE

MAP

CUTOVER

Application programmer and DBA for migration and testing

Users for testing CICS support for defining

driver modules Assembler skills needed if

exits are required

Skills required

Page 58: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

Other enhancements in CICS VT 1.2

Page 59: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

OPERABILITY ENHANCEMENTS

Maximum number of DB2 columns supported by CICS VT is increased;

180 was maximum in V1.1 No limitation imposed by VT 1.2 (750 limit in DB2)

Needed VT exits to circumvent in V1.1

More files are now eligible for auto-mapping

Improves support for Cobol OCCURS and PL/I arrays

DB2 overhead increases as number of table columns increases

Page 60: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

OPERABILITY ENHANCEMENTS

New commands available in auto-mapper

V1.1 supported only the MAP command SUSPEND command allows the mapping to be saved for later

resumption RESUME command recovers previously suspended mapping

information SAVE command writes the generated DDL to a user-specified

dataset. Very useful if person doing mapping does not have the necessary DB2 privileges

SUSPEND/RESUME supports a single file

Page 61: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

OPERABILITY ENHANCEMENTS Auto-mapper now supports INSERTing new columns

Assume there are 4 2-byte copybook fields for CC, YY, MM and DD to be migrated to a single DATE column in DB2

V1.1; manually edit copybook to replace 4 fields with a single 8-byte field and map to a DATE column

V1.2; use auto-mapper to delete 4 2-byte fields and insert a single 8-byte field

Need to manipulate copybooks prior to auto-mapping is reduced

RENAME column name function now displays original field name and attributes (V1.1 displayed only column information)

Page 62: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

PERFORMANCE ENHANCEMENTS

Improvements to WRITE and REWRITE calls

V1.1; Each field built separately V1.2; Adjacent fields with same attributes are built in a single

process Same function for GET calls was introduced in V1.1 by a PTF

Page 63: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

SUPPORTED SW LEVELS

VT 1.2 has been tested in the following environments

z/OS 1.3 to 1.6

CICS TS V1.3 to V3.1

DB2 V7.1

DB2 V8.1 (more on next slide)

Page 64: © 2007 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. CICS VSAM Transparency.

© IBM Corporation 2007 IBM European WebSphere Technical Conference

SUPPORTED SW LEVELS

There are 3 modes of DB2 version 8

Compatibility mode – this is the result of migration from V7.1

Enabling new function mode – the process of migrating to new function mode is underway

New function mode – the process of migrating to new function mode has completed (fully unicode enabled)

Extended object names supported in new function mode only

VT1.2 supports new function mode but does not support extended object names