IMS High Availability Large Database (HALDB) (subset of course CMW46) The world depends on it John Harper - IBM IMS Development [email protected]
IMSHigh Availability Large Database
(HALDB)(subset of course CMW46)
The world depends on it
John Harper - IBM IMS Development
Topics
Overview Database StructuresDefinitionPartition SelectionDBRC SupportDatabase CommandsUtilitiesDatabase Creation and MigrationChanging DatabasesSecondary IndexesApplication ConsiderationsMisc. Considerations HALDB Online ReorganizationSummary
HALDB Overview
HALDB (High Availability Large Database)Large Database
Databases are partitionedUp to 1001 partitions per databasePartitions have up to 10 data set groups
High Availability Database
Partition independenceAllocation, authorization, reorganization, and recovery are by partition
Self healing pointersReorganization of partition does not require changes to secondary indexes or logically related databases
HALDB (High Availability Large Database)Benefits
Greater database capacity Without application changes
Increased database availabilityPartitions, not databases, are removed from systemShortened reorganization processBatch window is shortened with concurrent processing
Improved manageabilityData sets may be smaller
Highlights
Hierarchic structure is maintained
A database record resides in one partition
Minimal (or no) application changes required
Cannot initially load logical child segmentsNew status code for load programs
Some secondary index segment formats change May require I/O area changes when secondary index is processed as a database
New database types
PHDAM - partitioned HDAMPHIDAM - partitioned HIDAM
Index is partitionedPSINDEX - partitioned secondary index
Highlights
OSAM and VSAM (ESDS and KSDS) are supported
Partition selection is done by key range or user exit routine
Logical relationships and secondary indexes are supported
Secondary indexes may be partitionedDBRC is required
Databases must be registeredDynamic allocation uses DBRC information
DFSMDA is not used
Definition Process
DBDGEN
Used to define databaseHierarchic structure, data set group boundaries, pointer options, logical relationships, secondary indexes,...
HALDB Partition Definition Utility
ISPF basedUsed to define partitions in database
Partition selection, space characteristics, randomizers,...
Or
DBRC Commands
Indirect Pointers
HALDB uses both direct and indirect pointers for logical relationships and secondary indexes
Direct pointers point to segments
Indirect pointers "point" to Indirect List Entries (ILEs) in Indirect List Data Set (ILDS)
ILEs have token for keyILEs contain direct pointer to segment
ILDS is KSDS associated with a Partition
Extended Pointer Set
Extended Pointer Set (EPS) is used for logical relationships and secondary indexes
EPS is not updated by reorganizations!
EPS contains direct pointer, reorganization number, and ILK
If reorg number is current, direct pointer is usedIf reorg number is not current, ILK is use to find ILE in ILDSILE contains pointer to segment
Direct pointer and reorg number is EPS are updated when ILE is used
Self healing pointers!
HALDB Database Structure
Each partition in a database has a unique partition ID (PID)
A reorganization number is maintained in each partition
Incremented by each reorganization reload
Each segment in PHDAM or PHIDAM database is assigned a unique token when created
Indirect List Entry Key (ILK)8 bytes stored in segment prefix
Self-Healing PointersUsing an Extended Pointer Set (EPS)
Since reorg # in EPS matches reorg # in Partition DBDS, we use EPS RBA pointer
EPSPartition IDReorg # : 1RBAILK...
Segment A
ILEILKSegment CodePartition IDCurrent reorg # : 1Current RBA...
ILDS
ILE
KSDS
Current reorg # :1
Segment B
Partition DBDS
ILK
Indirect List Key (ILK) is stored in segment prefix.
EPS KEY
Self-Healing PointersAfter reorganization of Partition
Reorg moves segment B, updates reorg # in ILE and Partition DBDS, and changes Current RBA in ILE
EPSPartition IDReorg # : 1RBAILK...
Segment A
ILEILKSegment CodePartition IDCurrent reorg # : 2Current RBA...
ILE
KSDS
ILDS
Current reorg # :2
Segment B
Partition DBDS
ILK
"broken" pointer
Indirect List Key (ILK) is stored in segment prefix.
EPS KEY
Self-Healing PointersUsing the EPS after the reorganization
Since reorg # in EPS does not match reorg # in Partition DBDS, we use the ILE RBA pointer
EPSPartition IDReorg # : 1RBAILK...
Segment A
ILEILKSegment CodePartition IDCurrent reorg # : 2Current RBA...
ILE
KSDS
ILDS
Segment B
Partition DBDS
Current reorg # :2
ILK
"broken" pointer
Indirect List Key (ILK) is stored in segment prefix.
EPS KEY
Self-Healing Pointers"Healing" the EPS
When we use the ILE, we update the reorg # and RBA pointer in the EPS
EPSPartition IDReorg # : 2RBAILK...
Segment A
ILEILKSegment CodePartition IDCurrent reorg # : 2Current RBA...
ILE
KSDS
ILDS
Segment B
Partition DBDS
ILK
Current reorg # :2
"healed" pointer
Indirect List Key (ILK) is stored in segment prefix.
EPS KEY
HALDB Database Data Sets
Each HALDB database has up to 1001 partitions
PHIDAM has index, ILDS, and up to 10 data set groups per partition
3 to 12 data sets per partition3 to 12,012 data sets per database
PHDAM has ILDS and up to 10 data set groups per partition
2 to 11 data sets per partition2 to 11,011 data sets per database
PSINDEX has no ILDS or data set groups
1 data set per partition1 to 1001 data sets per secondary index
HALDB Database Data Sets
PHIDAM PHDAM PSINDEX
...
...
...
...
...
...
...
...Index
ILDS
Data ...
1 to 1001 partitions 1 to 1001 partitions
1 to 1001 partitions
1 to 10 data set groups... ... ...
Partition Independence
Commands
Allowed on both databases and partitions
Availability
Partitions are allocated and authorized independently
Scheduling
Based on database availabilityPCBs and INQY calls report database availabilityPartition may be unavailable with available database
Database Utilities
Allowed on individual partitionsConcurrent processing of multiple partitions allowed
HALDB Overview
Database candidates for HALDB
Very large databasesApproaching 4GB limitationTheoretical limit is now over 40 terabytes per database
Medium and large databasesParallel processing to meet time deadlinesLess frequent reorganizations due to more free space
Any databaseMore frequent reorganizationsMaking only parts of the data unavailable for database maintenance
HALDB Database Structures
Database Structures
Logical relationships
Virtual pairing is not allowedLimited to unidirectional or physically paired
Logical child segments cannot be initially loadedMust be added by update
Symbolic pointers are not used
Secondary indexes must have unique keys
/SX or /CK may be used to create uniqueness/SX is increased from 4 to 8 bytes (ILK)
Symbolic pointers are not used
Database Structures
PHIDAM primary indexes are not separately defined
Defined as part of the PHIDAM database
Applies to DBDGEN and system definition
Parent pointers
All segments have physical parent pointers
Database Data SetsPHIDAM PHDAM PSINDEX
The data sets in a partition have generated data set names and DDNAMEs.Letters are used to distinguish them.
X - PHIDAM indexL - ILDSA through J - Data data setsA - PSINDEX
...
...
...
...
...
...
...
...
Index
ILDS
Data ...
1 to 1001 partitions 1 to 1001 partitions
1 to 1001 partitions
1 to 10 data set groups
X
L L
A-J
...
A
A-J
... ...
Database Data Sets
DDNAMEs
Partition name is basis for DDNAMEUp to 7 charactersAssigned in HALDB Partition Definition Utility or DBRC
commandLetter is used as suffix
X for PHIDAM index L for ILDSA for PSINDEXA through J for data
Example:Partition name: FRANCEDDNAME for PHIDAM Index: FRANCEX
Database Data Sets
Data set names
Begin with data set name prefix for the partitionUp to 37 charactersAssigned in HALDB Partition Definition Utility or DBRC
command
Letter and Partition ID are used as suffixX for PHIDAM index L for ILDSA for PSINDEXA through J for data
Example:Partition data set name prefix IMP0.DB.INV23.FRPartition ID: 00004Data set name PHIDAM Index: IMP0.DB.INV23.FR.X00004
Database Data Sets
Data set size limitations
Maximum data set size is 4GBApplies to OSAM and VSAM
OSAM block sizes must be even
Partition DDNAMEs and Data Set Names
Data set DDNAME Data Set Name
Data set group 1 Partition_nameA DSN_prefix.ApartitionID
Data set group 2 Partition_nameB DSN_prefix.BpartitionID
Data set group 3 Partition_nameC DSN_prefix.CpartitionID
... ... ...
Data set group 10 Partition_nameJ DSN_prefix.JpartitionID
ILDS Partition_nameL DSN_prefix.LpartitionID
PHIDAM Index Partition_nameX DSN_prefix.XpartitionID
Secondary Index Partition_nameA DSN_prefix.ApartitionID
Partition_name assigned by user in HALDB Partition Definition Utility or DBRC command
DSN_prefix assigned by user in HALDB Partition Definition Utility or DBRC command
PartitionID assigned by IMS in HALDB Partition Definition Utility or DBRC command
Partition DDNAMEs and Data Set Names
Partition_name of FRANCE
DSN_prefix of IMP0.DB.INV23.FRANCE
PartitionID of 00004
Example: PHIDAM with 10 data set groups, FRANCE partition
Data set DDNAME Data Set Name
Data set group 1 FRANCEA IMP0.DB.INV23.FRANCE.A00004
Data set group 2 FRANCEB IMP0.DB.INV23.FRANCE.B00004
Data set group 3 FRANCEC IMP0.DB.INV23.FRANCE.C00004
... ... ...
Data set group 10 FRANCEJ IMP0.DB.INV23.FRANCE.J00004
ILDS FRANCEL IMP0.DB.INV23.FRANCE.L00004
PHIDAM Index FRANCEX IMP0.DB.INV23.FRANCE.X00004
Partition DDNAMEs and Data Set Names
Partition_name of CANADA
DSN_prefix of IMP0.DB.INV23.CANADA
PartitionID of 00011
Data set DDNAME Data Set Name
Data set group 1 CANADAA IMP0.DB.INV23.CANADA.A00011
Data set group 2 CANADAB IMP0.DB.INV23.CANADA.B00011
Data set group 3 CANADAC IMP0.DB.INV23.CANADA.C00011
... ... ...
Data set group 10 CANADAJ IMP0.DB.INV23.CANADA.J00011
ILDS CANADAL IMP0.DB.INV23.CANADA.L00011
PHIDAM Index CANADAX IMP0.DB.INV23.CANADA.X00011
Example: PHIDAM with 10 data set groups, CANADA partition
Extended Pointer Set
Extended pointer set (EPS)
Used for logical relationships and secondary indexes
Data:
Reorg number of target partitionTarget partition IDPointer to logical parent or sec. index targetPointer to paired logical child for physical pairingDatabase record lock ID of segmentILK
28 bytes
ILDS Data Sets
Indirect List Entries (ILEs)
Created or updated by reorg reload
Reorgs do not update pointers in segments
Not created or updated by non-reload processing
This processing updates pointers in segments
Initial load does not create ILEs
ILDS Data Sets
ILE keys (9 bytes)
ILK (8 bytes)
RBA of segment at its creation time (4 bytes)
Partition id at creation time (2 bytes)
Reorg number at creation time (2 bytes)
Segment code (1 byte)
ILDS Data Sets
ILE data (50 bytes)
Key (ILK and segment code)
Flags
Old and new copies of:Partition IDReorg numberPointer to logical parent or sec. index targetDatabase record lock ID for segmentPointer to paired logical child for physical pairing
HALDB Segment Format
EPS in logical child segments and secondary indexes
ILK in all PHDAM and PHIDAM segments
Target root key in all PSINDEX segments
PHDAM or PHIDAM segment (not a logical child)SC DB Counter and pointers ILK Data
|-----------------------------Prefix-----------------------------------|
PHDAM or PHIDAM logical child segment
|-----------------------------------Prefix-------------------------------------------|
SC DB Pointers EPS ILK LPCK FID
PSINDEX segmentDB EPS Root Key Index Data
|--------------Prefix--------------------|
SC - segment code DB - delete byte LPCK - log. parent concatenated key FID - fixed intersection data
DFSVSMxx and DFSVSAMPand
Dynamic Allocation
DFSVSMxx Member or DFSVSAMP DD
Data sets assigned to subpool with alphabetic characters
Replaces data set numbers
DBD=dbdname(data set identifier,id)
dbdname - partition name or master database name
data set identifier - Letter A-J, L, or XA-J for user data setsA for secondary indexL for Indirect List Data Set (ILDS)X for PHIDAM primary index
id - subpool id
DFSVSMxx member or DFSVSAMP DDPHDAM example
DSG=A
ILDS
PART1 PART2 PART3
DSG=B
LDDN=PART1L
LDDN=PART2L
LDDN=PART3L
ADDN=PART1A
ADDN=PART2A
ADDN=PART3A
B
DDN=PART1B
B
DDN=PART2B
B
DDN=PART3B
SubpoolSP1
SubpoolSP2
SubpoolSP4
SubpoolSP3
DBD=MASTDB(L,SP1)DBD=MASTDB(A,SP2)DBD=PART3(A,SP3)DBD=MASTDB(B,SP4)
DFSVSMxx member:PHDAM databaseDatabase name: MASTDB3 partitions2 data set groups
Dynamic Allocation
Dynamic allocation uses DBRC information
HALDB Partition Definition utility or DBRC commands create this information
DFSMDA is not used with HALDB
Dynamic allocation should always be used
One exception: The Database Recovery utility (DFSURDB0) requires a DD statement for the database data set
DD statements cannot be used to override data set names
HALDB Partition Selection
Order of Segments in a PHDAM DatabasePHDAM with key range partitioning
Keys are sequential between partitions, random within a partition
PHDAM with Partition Selection Exit routineKeys are not sequential between partitions, random within a
partition
874321456
171215321219
241220122567
921090209791
...
201245167718
241278238900
87417123432
654977209791
...
Order of Segments in a PHIDAM DatabasePHIDAM with key range partitioning
Keys are sequential between partitions, sequential within a partition
PHIDAM with Partition Selection Exit routineKeys are not sequential between partitions, sequential
within a partition
...201245167718
241278238900
87417123432
654977209791
121915321712
321456874
201224122567
902092109791
...
Order of Segments in a PSINDEX DatabasePSINDEX with key range partitioning
Keys are sequential between partitions, sequentialwithin a partition
PSINDEX with Partition Selection Exit routineKeys are not sequential between partitions, sequential
within a partition
321456874
121915321712
201224122567
902092109791
...
201245167718
241278238900
87417123432
654977209791
...
Changing HALDB Databases
Database Changes
Changes affecting all partitions of the database
DBD changes
Partition selection exit
Share level
Nonrecoverable attribute
RSR GSG name and/or tracking level
These changes cannot be made if any partition is authorized
Partition Changes
Changes affecting only one partition
DSN prefixRandomizing module nameNumber of root anchor pointsBytes parameterFree spaceBlock size for OSAMCI size for VSAM*
Changes cannot be made if the partition is authorized
VSAM changes by IDCAMS, not PDU
Database Changes
Changing high keys of partitions
Changes not allowed if affected partitions are authorized
Affected partitions must be unloaded before changes
PINIT flag is set for affected partitions
Affected partitions must be reloaded after initialization
"Affected partitions" are those whose key range is changed
Database records may be moved into or out of affected partitions
Example 1: Splitting a Partition
If partition PB with high key 4000000 needs to be split
–Unload partition PB
HD Unload or HP Unload
–Define new partition PE
Sets PINIT flag for PB and PE
–Initialize partitions PB and PE
–Reload partitions PE and PB
HD Reload or HP Load
–Partitions PA, PC, and PD
are not affected
PA2000000
PB4000000
PC6000000
PD8000000
PA2000000
PE3000000
PC6000000
PD8000000
PB4000000
HD Unload
HD Reload
Define new partition
Example 2: Modifying Partition Boundaries
If records need to be moved from partition PB to PC
–Unload partitions PB and PCHD Unload or HP Unload
–Change high key for partition PB from 4000000 to 3000000Sets PINIT flag for PB and PC
–Initialize partitions PB and PC
–Reload partitions PB and PCHD Reload or HP Load
–Partitions PA and PD are not affected
PA2000000
PB4000000
PC6000000
PD8000000
PA2000000
PC6000000
PD8000000
PB3000000
HD Unload
HD Reload
Change high key
See warning in notes for this page
If partitions PB and PC with high keys 4000000 and 6000000 need to be combined–Unload partitions PB and PC
HD Unload or HP Unload–Delete definition of partition PB
Sets PINIT flag for PC–Initialize partition PC–Reload partition PC
HD Reload or HP Load–Partitions PA and PD are not affected
PA2000000
PB4000000
PC6000000
PD8000000
PA2000000
PC6000000
PD8000000
HD Unload
HD Reload
Example 3: Combining Partitions
Delete partition definition
Some databases have dates as the high-order part of the key
To add a partition for a set of dates (higher keys)–Define it and
initialize it
To delete the partition with the lowest dates (keys) and all of its data–Delete the partition
definition–Turn off PINIT flag for PB
Unloads and reloads are not required for these changes
Databases with Dates for Keys
PA2002xx
PB2003xx
PC2004xx
PD2005xx
PA2002xx
PB2003xx
PC2004xx
PD2005xx
PE2006xx
PA2002xx
PB2003xx
PC2004xx
PD2005xx
PE2006xx
Partition Selection Exit Routine Changes
Changing Partition Selection Exit routine
Change not allowed if affected partitions are authorized
User determines which partitions are affected
Must issue CHANGE.DB PINIT command
or
Use specific partition initialization
Affected partitions must be initialized
Partition Selection Exit Changes
Partition Selection Exit
Adding an exit routine
Preserves high keys in definition as key strings
All partitions must be initialized
Deleting an exit routine
Partition strings cleared
High keys must be defined
All partitions must be initialized
Master Database Changes
Changing Partition Selection Exit
Adding a new partition
Partition must be initialized
User must determine other affected partitions
Deleting a partition
User must determine other affected partitions
Changing a partition string or other changes
User must determine affected partitions
Partition Changes
Partition Data DBDS Changes
GENMAXRECOVPDDEFLTJCLICJCLOICJCLRECOVJCLRECVJCL
Partition can be authorized
Does not require partition initialization
Partition Changes
Partition ID numbers are never reusedDeletion and recreation of same partition name results
in new partition IDData set names are not reused because they include partition ID number
1. PARTC exists 2. PARTC deleted 3. PARTC recreated
Partition Name Partition ID
PARTA 00001PARTB 00002PARTC 00003
Partition Name Partition ID
PARTA 00001PARTB 00002PARTC 00004
Partition Name Partition ID
PARTA 00001PARTB 00002
Partition Changes
Disabling and enabling partitions was introduced by APAR PQ73858 for IMS V8
Included in IMS V9
Disabling partitionsDefinitions and information remain in RECONs
Includes partition IDs, DSN prefixes, and recovery information
Partitions are not usedPartitions are ignored by IMS processing
Disabled partitions may be enabledEnabled partitions are made activeEnabled partitions are marked ‘recovery needed’
Partition Changes
Use of disabling and enabling partitionsDisabling is normally done prior to deleting a partition
Keeps recovery information, partition ID, DSN names prefix, etc.If testing is successful, partition is deleted
Deletion removes all informationIf testing is not successful, partition is enabled
Partition is recovered and becomes activeOther partitions may require timestamp recovery
PDU support of disabling and enabling‘Partition status’ field on ‘Change Partition’ panel
DBRC commands for disabling and enabling
Partition Changes
ILDS or PHIDAM Prime Index DBDSs
Changes cannot be made by HALDB Partition Definition Utility
Changes are made only by rebuilding the data sets
Secondary Indexes with HALDB
Secondary Indexes
Partition selection exit may be able to partition a secondary index like its target database
US UK FR
Target Database
US UK FR
Secondary Index
Secondary Indexes
Shared indexes are not supported
Must have unique keys
/SX and /CK may be used
Symbolic pointing is not allowed
Extended Pointer Set (EPS) contains direct pointer
Target root key is stored in index segment
Sparse indexing is supported
Secondary Indexes
Migration of secondary index
Must be migrated with target database
Migration uses HD Unload and ReloadIndexed database is readMIGRATX=YES option is preferred
Migration Aid utility provides limited information about secondary indexes
Creation of secondary index
Created with initial load of target databaseAn alternative will be covered later
Work files not createdPrefix Resolution, HISAM Unload and Reload not used
Secondary Indexes
Addition of secondary indexCannot use standard IMS utilities to add an indexIBM IMS Index Builder will add indexesAlternative for adding secondary index:
Add definitions to target databaseDefine secondary indexApplication program replaces source segments with different source dataApplication program re-replaces source segments with the original data
Database could be initially loaded again with secondary index definitions
Secondary Indexes
Reorganization of target database does not affect secondary index
Secondary index does not have ILDS
Partition Selection Exit may be used
Secondary indexes may need to be reorganized
HALDBMiscellaneous Considerations
Extended Pointer Set (EPS) Adjustments
When out of date pointer is found it is corrected on DASD if:
Access intent is update or exclusive
PROCOPT is update
Locking considerations
Read programs with update PROCOPTs may hold many locks
If block level data sharing is used, block locks are held until sync point
Logging
No logging of "after images" for PHIDAM indexes
Rebuilt with DFSPREC0 utility"Before images" are not archived
Only "after images" are logged for EPS adjustments
Adjustments are not backed out
Database change log records include partition name instead of master database name
No logging for ILDS
Only updated by HD Reload utility or Online Reorganization
OSAM Sequential Buffering
HALDB supports OSAM Sequential Buffering (SB)
Same support as for non-HALDB databases
Recommended for batch sequential processing
SB monitors by data set
Sequential buffering is turned on/off due to access patterns
Sequential buffering may be invoked for some, but not all partitions
Well-organized vs. poorly-organized partitions
Data Set Groups
HALDB supports data set groups (DSGs)
DSGs are not needed to address database size limitations
With non-HALDB databases, DSGs were often used to deal with data set size limitations
DSGs remain useful
May improve performance by moving rarely accessed data to another data set
Reevaluate need for DSGs when migrating databases to HALDB
Database Exit Routines
Database Exit Routines
Routines:Segment Edit/Compression Exit RoutinesData Capture Exit RoutinesData Conversion User Exit RoutineSecondary Index Database Maintenance Exit Routines
Defined for entire database
All partitions in database use the same exit routines
Exit routines are not given partition information
Data Set Allocation
Dynamic allocation uses DBRC information
DFSMDA is not used
Users cannot override data set information with DD statements for database data sets
Data Sharing
Data sharing is supported
SHARELVL specified on master database level
Authorization is by partition
Authorization occurs on first call to partition
HALDB Summary
HALDB Database Candidates
Very large databases
Approaching 4GB (VSAM) or 8GB (OSAM) limitations
To allow for growth
To make databases more manageable
Previously partitioned databases
Using IMS/ESA Partition Support Product (PDB)
User partitioning
HALDB Database Candidates
Medium and large databases
Parallel processing to meet time deadlines
Application programs
Utilities
HALDB Database Candidates
Any size databaseFaster reorganizations
May be done more frequently
Partition independenceMaking only parts of the data unavailable for database maintenance
Online reorganization in IMS Version 9
Plan migrations of large databases to HALDBLogical relationships may add significant timePay attention to PHDAM randomization parameters
HALDB Migration Aid utility can analyze existing HALDB databases
Useful when planning repartitioning
Deleting a partition definition deletes its recovery information
Disabling a partition keeps its recovery information
Secondary indexes may require reorganizationsThey are not rebuilt when the indexed database is reorganized
Some Things to Remember
Secondary index cannot be rebuilt from database with IMS utilities
Don't make them non-recoverable unless you have a tool like the IBM Index Builder
PHIDAM indexes and ILDSs have a different recovery process
They are rebuilt with Index/ILDS Rebuild Utility (DFSPREC0)
Plan your scheme for creating HALDB test databases
DBRC registration is required for all databases
Some Things to Remember
HALDB Summary
Benefits
Greater database capacity
Without application changes
Increased database availability
Partitions, not databases, are removed from system
Shortened reorganization process
Batch window is shortened with concurrent processing
WWW.IBM.COM/IMS (The IMS webpage!)
WWW.IBM.COM/SUPPORT/TECHDOCS
Search on "HALDB"
Results:IMS HALDB Database Administration
IMS HALDB Reorganization Number Verification
IMS Version 9 HALDB Online Reorganization
Converting IMS Logically Related Database DBDs to HALDB
Application Design and Programming with HALDB
Migrating to IMS HALDB (High Availability Large Database)
More Information on HALDB