Top Banner
© 2015 IBM Corporation IMS Database Solution Pack Overview with Autonomics Richard Phipps IBM IMS and IMS Tools [email protected] Philadelphia IMS Regional User Group Meeting 21 May 2015
82

IMS Database Solution Pack: user group Philadelphia 2015

Jul 22, 2015

Download

Technology

IBM IMS
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation

IMS Database Solution Pack

Overview with Autonomics

Richard Phipps IBM IMS and IMS Tools

[email protected]

Philadelphia IMS Regional User Group

Meeting 21 May 2015

Page 2: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 2

IMS Tools – Why ?

The IMS base product includes a number of basic utilities for the

administration and management of IMS and it’s underlying data bases

• Backup, Recovery, Reorganization, Performance reporting, etc.

• Sufficient for some customers with lesser demands from IMS data

For many IBM customers, the ‘business critical’ nature of IMS data

demands more from what IMS utilities provide.

IMS Tools are needed for:

• Better Performance

• Enriched Features and Functions

• Higher Data Availability

• Integrated Capabilities

• Ease of Use

“Doing more with less” … this is the role of IMS Tools

Page 3: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 3

IMS Tools from IBM

Providing Modernization and Autonomics

IMS Fast Path Solution Pack

Autonomics IMS HP Fast Path Utilities IMS DIB Repair Facility IMS HP Image Copy IMS Library Integrity Utilities

IMS Recovery Solution Pack

IMS HP Image Copy IMS Database Recovery Facility IMS HP Change Accumulation IMS Index Builder IMS DRF Extended Functions

IMS Performance Solution Pack IMS Connect Extensions IMS Performance Analyzer IMS Problem Investigator

IMS Database Solution Pack Autonomics IMS DB Reorganization Expert - Unload, Load, Index Build, Prefix Resolution/Update IMS HP Image Copy IMS HP Pointer Checker IMS Library Integrity Utilities IMS HALDB Toolkit IMS Online Reorg Facility

IMS Sequential Randomizer Generator Data Base Administration

IMS Cloning Tool IMS Database Control Suite

Utility Management

IMS HP Image Copy IMS DEDB Fast Recovery IMS Recovery Expert

Backup and Recovery Batch Terminal Simulator

Program Restart Facility Application Management

Performance Management

IBM Transaction Analysis Workbench IMS Buffer Pool Analyzer IMS Network Compression Facility

System / TM Administration

System IMS Command Control Facility IMS ETO Support IMS HP Sysgen Tools IMS Queue Control Facility IMS Workload Router TM IMS Configuration Manager IMS Sysplex Manager

Regulatory Compliance

Guardium ITAP for IMS IBM Infosphere Guardium Data Encryption for DB2 and IMS Databases

IBM Tools Base for z/OS

Page 4: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 4

Today - Typical Concerns and Challenges

Increased use of data assets - Bigger database size - Higher demands for continuous data availability

New business requirements - Enabling host data for access outside of the zSeries (host) platform - New governmental regulations

Cost containment - Reduce CPU, I/O, and elapsed execution times - Budgetary Concerns

Support staff concerns - Head count reductions - Retirements - Experience and knowledge level

The “shrinking skill set” - A top concern for all

Page 5: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 5

The “Shrinking Skill Set” – Some observations

Issue: The “z” support community is getting older which creates an impending skills

problem that IT organizations are already beginning to deal with Facts:

80% of IMS DBAs are over the age of 55 Within the next 10 years, this entire group will be moving on to retirement

Most companies have IMS DBAs with 25+ years of experience

Estimates are it takes 5-7 years for someone to become a truly effective DBA

Industry statistics show that on average, new employees will remain on the same job for between 3-5 years

Response:

IT organizations have some “big shoes to fill”

Provide tools that can capture the wealth of experience

Put new tools into place to accelerate the learning curve for next generation IT support people

Page 6: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 6

IMS Automated Database Solutions

Checks data base status on a regular basis

User specified thresholds for key indicators

Performs reorganization only when necessary

Performs auxiliary functions: IC, PC, IB, etc.

Keeps data bases performing optimally

Saves human resources

Saves computer resources

Cost efficient solution for both predictable and unpredictable data base growth and

activity

Page 7: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 7

“No charge” value-added products to support

extended use of IBM data management tools

** Capabilities designed to address

concerns of a “shrinking skill set” Provides IMS database compression tool to address

needs of growing IMS database sizes

GUI to provide detailed insight into the health and

availability of IMS databases through an easily

accessible Web interface

Autonomics capability to implement the automation of

tools execution using detailed understanding of the

current state of the IMS databases

Report repository of historical execution of IMS

utilities used in managing the IMS databases

IBM Tools Base

IMS Tools provided:

IMS Hardware Database

Compression Extended

Autonomics Director for

IMS & DB2

IMS Tools Knowledge Base

…and More

Page 8: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 8

IMS Hardware Compression Extended

Productivity aid for implementing Hardware Assisted Data Compression

Functions:

Extends IMS basic HDC support with utilities that ease compression

implementation and provide additional flexibility and function

Works on image copies, High Performance Unload files and IMS unload

files

Includes sample Compression Dictionaries

Enables effectiveness of existing dictionaries to be assessed

Allows monitoring of dictionary effectiveness over time

Builds DBD and reload JCL

IBM Tools Base

Page 9: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 9

IMS Tools Knowledge Base

Implement a report repository for IMS Tools

Creates a historical perspective of IMS databases for the DBAs

Reporting stored in the IMS Tools Knowledge Base can be searched, retrieved, and viewed effortlessly • Through ISPF interface or IMS Admin Console

Facilitates communication between IMS Tools products

Tools Registration

Environment Information

Sensor Data

Policies

Autonomics Engine

IBM Tools Base

Page 10: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 10

IMS Tools Vision - Autonomics

Sensors collect resource statistics

Policies evaluate sensor data and identify potential problems

Automation orchestrates the collection and evaluation of sensor data

Modernization presents an interactive modern interface for managing the system

Putting information to work

Sensors Policies

Automation Modernization

Not a vision, but a reality

IBM Tools Base

Page 11: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 11

Sensors: Collecting the Basic Information You Need

Statistical point-in-time sensor data on your FF/FP Databases

Stored in IMS Tools Knowledge Base repository

Historically maintained per user specifications

Over 60 separate data elements related to space usage, optimization, and

fragmentation

• data set extents, DASD volume usage, data set free space, roots distribution, RAP usage, CI/CA

splits, and IMS free space, etc

Two methods of collection:

Standalone database sensor utilities for full-function and Fast Path databases

Integrated with existing IMS Tools

Integrated Tools support

High Performance Image Copy, High Performance Pointer Checker

Fast Path Analyzer, Fast Path Online Pointer Checker

Sensors

IBM Tools Base

Page 12: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 12

List of Full Function Sensor Data collected

Database Record Statistics (per database or HALDB partition) Nbr. of DB records Avg. DB record length

Randomizer Statistics (per HDAM or PHDAM partition) Nbr. of total RAPs Nbr. of unused RAPs % of number of unused RAPs Nbr. of synonyms

% of number of synonyms Nbr. of root not on home block % of root not on home block % of segment data in overflow

Nbr. of roots in overflow % of number of roots in overflow Bytes of segments in RAA

Volume/Extents Statistics (per data set) Allocation type (CYL, TRK, …) Primary allocation amount Secondary allocation amount SMS-managed or not

Max. nbr of extents for the d.s. Max. nbr. of extents for the volume Nbr. of extents allocated Nbr. of volumes used

Nbr. of unused volumes Nbr. of unused assigned volumes Nbr. of unused candidate volumes

Nbr. of available remaining extents determined by the max. nbr. of data set extents and the max. nbr. of extents available on volumes assigned to the data set

Data Set Space Usage Statistics (per data set) Block/CI size Nbr. of blocks/CIs used Max. size of the data set % of data set size against the max.

High-Allocated-RBA High-Used-RBA

IMS Space Utilization Statistics (per data set) Total bytes of segment data Total bytes of free spaces Total bytes of slack bytes % of free spaces

% of segment data % of unused bytes in the data set Total nbr. of segments Total nbr. of VL segments

Total nbr. of VL-split segments % of nbr. of VL-split segments Total nbr. of slack bytes Avg. nbr. of slack bytes per block

Total nbr. of FSEs Avg. nbr. of FSEs per block Nbr. of FSEs valid for shortest segments Nbr. of FSEs valid for longest segments

Avg. nbr. of non-reusable FSEs Total nbr. of pointers Total nbr. of ptrs pointing external block % of nbr. Of ptrs pointing ext. block

HISAM/SHISAM Statistics (for HISAM) Logical record length Total nbr. of CI splits % of nbr. of CI splits Total nbr. of CA splits

% of nbr. of CA splits Total nbr. of HISAM delete bytes % of nbr. of HISAM delete bytes

Sensors

IBM Tools Base

Page 13: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 13

Policies: Using Sensor Data to Make Decisions

Policy definitions are used to evaluate specific database states

Threshold values are compared against sensor data for a given database or group of databases

When thresholds are met or exceeded, exceptions occur

Works “out of the box”

Ships with predefined policies and threshold values

Full ISPF interface provided for policy management

Customizable to fit your shop

You can define your own sets of threshold values

Customize the messages sent when exceptions do occur

Specify who receives which messages and how • WTO, e-mail, or text

Policies

IBM Tools Base

Page 14: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 14

Autonomics Director for IMS & DB2

IBM Tools Autonomics Director (Passive)

Automates collection and analysis of Sensor Data

Recommends when databases should be reorganized

• With email or text notifications

Provides a scheduling feature that allows you to control how frequently sensor data is

collected and how frequently policies are evaluated

Flexible scheduling around pre-defined PEAK times

IBM Tools Autonomic Director (Active)

Actively initiate recommended actions on user-defined database groups • Discovery feature for identifying related database groups

• Ability to manage and coordinate reorganization of multiple IMS database groups as if

reorganizing a single database

• Flexible scheduling only in pre-defined Maintenance windows

Automation

IBM Tools Base

Help prevent problems before they occur – Higher availability and performance through optimally organized data

Consistent and reliable monitoring of database status

– Regular collection/evaluation at predefined intervals

Page 15: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 15

© 2014 IBM 15

Changing the “face” of the IMS platform and

Management of System z

IBM Management Console for IMS & DB2 Databases for z/OS

Modernization Modernization

Page 16: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 16

Provide a single holistic, easy-to-use interface to manage IMS/DB2 systems and

databases

Consolidate information from various tools to paint a more complete picture of IMS/DB2

systems and databases

Leverage the latest web technologies for a richer user experience

Access from anywhere via the Internet using standard web browsers

Prepare the next generation of DBMS DBAs and System Programmers

through an integrated, context sensitive help system

IBM Management Console for IMS & DB2 Databases for z/OS

Page 17: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 17

Object Health

and Autonomics Enterprise-wide

Navigation

Integrated Help

Graphical Visualization

of data not possible in ISPF

Provides a single, holistic easy-to-use interface to manage IMS and DB2

Zero-install web-based interface

Consolidate information from IMS, DB2 and tools from across the entire enterprise

Reduced time for problem identification and resolution through tight integration with IMS and DB2 Autonomics

Dramatically reduced learning curve for new users of IMS and DB2

Now a separate product available no-charge (5655-TAC)

Extensible by growing number of products and solution packs adding additional value

IMS HP

Tools

Autonomics

Director Discovery

Sensors

IBM Management Console for IMS & DB2 Databases for z/OS

Page 18: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 18

Drill down from the IMSPLEX level through:

IMSPLEX members

IMS Connect

IMS Subsystems

Transactions

Programs

Routing Code

Databases

Identify, stop, start IMS resources

Explore resource relationships

OM Command Builder

IBM Management Console for IMS & DB2 Databases for z/OS

Page 19: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 19

Graphical control of IMS

Autonomics Director

Define monitor lists, setup

peak times and

maintenance windows

Web access to IMS reports

from a variety of Tools

Search and filter by type,

date, completion code, etc

ITKB Reports

Autonomics Control

IBM Management Console for IMS & DB2 Databases for z/OS

Page 20: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 20

Extended IMS Autonomics and

Database information

Easily identify and diagnose

symptoms and recommended

actions through IMS Database

Autonomics

Identify trends and make projections

through database sensor charts

Visualize IMS FF/FP Databases

through Library Integrity Utilities

LIU Reports

Extended with the IMS FF/FP Solution Pack

IBM Management Console for IMS & DB2 Databases for z/OS

Page 21: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 21

Progressive drill down through

variety of DB2 object dashboards

Autonomics Director for DB2 for

z/OS (in the no-charge Tools

Base) enables:

Charting of DB2 object statistics

through RTS snapshots with

Autonomics control to define

profiles and maintenance

windows

Integrated support for the DB2

Admin Task Scheduler

DB2 Subsystem

Maintenance Windows

IBM Management Console for IMS & DB2 Databases for z/OS

Page 22: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 22

Identification and Diagnosis of

symptoms and recommended

actions for REORGs, ICs and

Runstats

Reporting on historical utility

execution including timestamp,

elapse time, system output, etc.

Graphical interfaces to define

Automation Tool Object, Utility,

Exception, and Job Profiles

Extended with the DB2 Utility Solution Pack

IBM Management Console for IMS & DB2 Databases for z/OS

Page 23: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 23

Quickly identify and drill down to databases and objects that need your attention from a single starting point

Easily manage by exception and recommendation, taking action before problems occur

Rapidly interpret statistical trends to verify and project

Shorten the learning curve for new administrators

…all from a unified IMS and DB2 interface

IBM Management Console for IMS & DB2 Databases for z/OS

Page 24: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 24

© 2014 IBM 24

Our Vision – Putting information to work for you

“No charge” systems and data

management solutions from IBM

This is critical technology that we

want all our customers to have

and use now and into the future

Sensors Policies

Automation Modernization

© 2015 IBM Corporation

Page 25: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 25

Designed to reduce technical expertise by providing

an integrated, high performance solution

Provides a complete set of high performance utilities to unload,

load, reorganize, verify, and report on Full Function and HALDB

databases

Functions to assist DBAs in identifying and performing the tasks

of maintaining the health, performance, and recoverability of IMS

databases

Features for the 'smart' reorganization of IMS databases that

can help improve system availability

IMS Database Solution Pack for z/OS

IMS Tools provided: • IMS Online Reorg Facility

• IMS DB Reorg Expert

• HP Unload

• HP Load

• IMS Index Builder

• HP Prefix Resolution

• HP Image Copy

• HP Pointer Checker

• IMS Library Integrity Utilities

• HALDB Toolkit

Utilities to validate, compare, map, recover, and report on IMS libraries

Provides tool for fast backup of IMS databases

Simplified use of IMS data management processes through tool integration

Page 26: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 26

IMS Database Reorganization Today – How are you doing it now?

Workload for data management increases as your business grows

Increasing volume of repetitive tasks such as free space defrag

Processing efficiency is becoming important

More frequent needs for database logical/physical structure changes

Deep IMS DBA skills are needed for these complex tasks

Plan/Do Actions Diagnose Databases

Pointer

Checker

Reports % of segment splits

Avg nbr of FSEs

% of max. dataset size

Number of extents

Statistics Values

. . .

Diagnosis Result

Exception 1

Exception 4

. . .

Each may be simple,

but the total work volume

is getting larger

. . .

Processing efficiency

is becoming important

How to do it

this time?

When can

it be done?

Deep IMS DBA

skills are needed

in planning actions

Exception 2

Exception 3

Check Key

Stats Items

IMS DBA

Simple Repetitive Actions

(e.g. free space defrag)

Complex Actions

(e.g. Structure Change)

Page 27: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 27

IMS Database Reorganization Expert

Provides a capability to perform a database reorganization while the database

remains in read only status

Provides automated one step batch job process to:

Reorganize IMS databases to shadow datasets

Rebuild the indexes

Perform image copy after the reorg with hash pointer checking

Rename shadow datasets to original production name

Perform automated /DBR and /STA

Page 28: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 28

IMS Online Reorganization Facility

Provides a capability to perform a database reorganization while the database

remains online and available for reads and updates

Designed to address customer needs for 24x7 continuous data availability

Provides automated one step batch job process to:

• Reorganize IMS databases to shadow datasets

• Rebuild the indexes

• Perform image copy after the reorg

• Capture changes to the original datasets

• Apply captured changes to the shadow datasets

• Rename shadow datasets to original production name

• Perform automated /DBR and /STA

Page 29: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 29

Conditional Reorganization – Only reorg what needs it!

Conditional Reorganization Process Steps

DB Administrator submits conditional reorganization job

Reorganization utility collects database statistics and stores them in repository

• Tools Base (Policy Services) evaluates the statistics by predefined policies

Reorganization utility reorganizes the database only when it is necessary

Reorganization utility generates diagnosis report and stores it in repository

DB Solution Pack

Reorganization Utility

• DB Reorg Expert

• Online Reorg Facility

TSO

Submit REORG job

IBM Tools Base

IMS Tools Knowledge Base

Policy

Repository

Store reports

Statistics

Evaluation

Results Diagnosis Report

Page 30: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 30

Specifying a Conditional Reorg Mode of Operation

The following modes are supported in both DB Reorg Expert and ORF:

Conditional Reorganization mode

• Database is reorganized only when necessary.

• DB Reorg Expert: CONDREORG=YES

• ORF: CONDREORG(Y)

Reorganization and Diagnosis mode

• Database is always reorganized and Diagnosis report is generated.

• DB Reorg Expert: CONDREORG=YES,REORGDIAG

• ORF: CONDREORG(Y,REORGDIAG)

Diagnosis-only mode

• Database is not reorganized. DB statistics are collected and evaluated.

• DB Reorg Expert: CONDREORG=YES,DIAGONLY

• ORF: CONDREORG(Y,DIAGONLY)

Page 31: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 31

Conditional Online Reorganization Work Flow

Benefits of Conditional Online Reorganization

A database can be reorganized only when it is really needed

The Diagnosis Report provides an evidence of the necessity of reorganization and

effectiveness of the reorganization

IMS Online Reorganization Facility

Dia

gn

os

is o

f O

nli

ne

DB

Renamed to

original name Renamed to

original name

Renamed to

original name Renamed to

original name

Log R

ecovery

Pro

cessin

g

Shadow DBD Shadow DBDS

Index Index Shadow Index

IC

IC IC

DB is Offline

NO

TIF

Y.R

EO

RG

DB

DS

Nam

e S

wappin

g

CH

AN

GE

.DB

AU

TH

RE

AD

OF

F

Co

py T

as

k

NO

TIF

Y.I

C/U

IC

Resta

rtin

g D

B

/STO DB

&

/STA DB

/STA DB

Online Online

Original Index Original Index

Original Index

Original DBDS Original DBDS Renamed to

original name

Online Reorganization Workflow Control (used only when reorg needed)

Sta

rt L

og C

aptu

re

Reorganization need is checked

z/OS consoles (and/or TSO users)

Detected or

remaining exceptions

are notified

(based on the policy)

Un

loa

d T

as

k

/STO DB

&

/STA DB

Sw

itch to C

hange C

aptu

re

Unload

Data Set

Relo

ad

Ta

sk

Shadow DBDS Shadow DBDS

Ind

ex

Bu

ild

Ta

sk

Ima

ge

Co

py T

as

k

(can

in

clu

de P

oin

ter

Ch

eck)

/DBR DB

Makin

g D

B O

fflin

e

Ap

ply

Ta

sk

Dia

gn

os

is o

f O

nli

ne

DB

Co

nd

itio

na

l R

eo

rg

Po

st

Pro

ce

ss

ing

Online Online

Page 32: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 32

Conditional Online Reorganization – Sample JCL

Three types of policy selection

By database type

Default

By database name

Used to specify a database

specific policy

By policy name

Used to specify a policy for a

group of databases

//ORFJOB EXEC PGM=HRFREORG,REGION=80M,DYNAMNBR=999 //STEPLIB DD DISP=SHR,DSN=ITB.SFOILOAD // DD DISP=SHR,DSN=ITB.SHKTLOAD // DD DISP=SHR,DSN=DBSP.SHRFLOAD // DD DISP=SHR,DSN=DBSP.SHPSLMD0 // DD DISP=SHR,DSN=IMS.SDFSRESL //IMSDALIB DD DISP=SHR,DSN=IMS.MDALIB //IMS DD DISP=SHR,DSN=IMS.DBDLIB //SYSPRINT DD SYSOUT=* //HRFSYSIN DD * REORG DBD(SAMPLEDB) - LOG1DSN(DATADS.LOG.&DBD.&DATA1.&TIME2) - IC1DSN(DATADS.IC1.&DBD.&DATA1.&TIME2) - CONDREORG(Y) - ITKBSERVER(FPQSVR00) - CR.POLICYBY(DBTYPE) /*

CR.POLICYBY(DBTYPE)

CR.POLICYBY(DBDNAME)

CR.POLICYBY(NAME) CR.POLICYNM(MY.POLICY.GROUP1)

By database type

By database name

By policy name

Policy to be selected

SYS.DBDTYPE.type

SYS.DBDNAME.dbdname

MY.POLICY.GROUP1

(type = HDAM, HIDAM, PHDAM,

PHIDAM, HISAM, or SHISAM)

Specifies Conditional Reorg Mode

Specifies XCF Group Name for

ITKB Server to connect

Specifies how a policy is to be selected

,DIAGONL(Y) -

Specifies Diagnosis Mode

Page 33: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 33

Database Diagnosis Report

Processing result of Smart Reorg job is summarized in the Database

Diagnosis Report

The report provides the following information:

Summary of Policy Evaluation

•Policy that was applied to the database

•Reorganization need

•Result of policy evaluation

•Result of policy evaluation after reorganization (if performed)

•Evaluation summary message (in the previous chart)

Sensor data values before and after reorganization

•Differences between the two

Page 34: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 34

Database Diagnosis Report - Example

Basic attributes of the subject database is printed in the title page.

IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 1 5655-S35 Date: 12/16/2009 Time: 00.05.14 Summary of Database Definition ------------------------------- Database.......................... BKDB Database Type..................... HIDAM Data Set Organization............. VSAM Number of Data Set Groups......... 1

The following sections follow this title page:

Summary of Policy Evaluation

Sensor data values before and after reorganization

1. Database Record Statistics section

2. Randomizing Statistics section (only for HDAM and PHDAM)

3. Volume and Extent Statistics section

4. Data Set Space Usage Statistics section

5. IMS Space Utilization Statistics section

6. HISAM Statistics section (only for HISAM and SHISAM)

※ Sections 3 through 5 are printed for each data set group

Page 35: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 35

Database Diagnosis Report – Summary of Policy Evaluation IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 2 5655-S35 Date: 12/16/2009 Time: 00.05.14 Summary of Policy Evaluation (DBD: BKDB ) ----------------------------------------------------------------- Name of Policy Applied............ SYS.DBDTYPE.HIDAM Policy Locale..................... Global Reorganization Need............... Yes Exceptions before Reorganization -------------------------------------------------------------------------------- The number of available extents for a data set of BKDB is small Class: DATA_SET_EXTENTS_AVAILABILITY Level: CRITICAL Rule: G:IBM.DBDS_EXTENTS.10 Threshold Set: HIGH The size of a data set in BKDB, which still has a certain amount of free space, has increased Class: GROWING_DBDS_WITH_FREE_SPACES Level: CRITICAL -> REORG Rule: G:IBM.DBDS_GROWTH.20 Threshold Set: HIGH The fragmentation of free space in BKDB has increased Class: FRAGMENTED_FREE_SPACES Level: CRITICAL -> REORG Rule: G:IBM.FRAGMENTATION.10 Threshold Set: HIGH A data set of BKDB has many pointers that point to other bocks or CIs Class: EXCESSIVE_SEGMENT_SCATTERING Level: SEVERE Rule: G:IBM.SEGM_SPREAD.10 Threshold Set: MED -------------------------------------------------------------------------------- Exceptions after Reorganization -------------------------------------------------------------------------------- BSN2800I GENERAL STATUS: RESOURCE=BKDB ACTION_NAME=REORG EXECUTION_STATUS=SUCCESSFUL The number of available extents for a data set of BKDB is small Class: DATA_SET_EXTENTS_AVAILABILITY Level: CRITICAL Rule: G:IBM.DBDS_EXTENTS.10 Threshold Set: HIGH -------------------------------------------------------------------------------- Summary of Policy Evaluation and Action: -------------------------------------------------------------------------------- BSN2904I BKDB HAS BEEN REORGANIZED, BUT IT IS STILL IN A CRITICAL STATE --------------------------------------------------------------------------------

Policy name

The locale where the

policy is defined

Result of decision on

reorganization need

Summary message, which shows that a

CRICTICAL exception still remains after

the reorganization.

Exceptions detected before the

reorganization. In this example,

Three CRITICAL exceptions were

detected

Two of them can be removed by

REORG (unload and reload)

One sever exception was detected

Exceptions that still remain after the

reorganization. In this case, one CRITICAL

exception remains.

Result of the reorganization

processing (success/failure)

1

2

3

4

1’

Page 36: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 36

Database Diagnosis Report – Database Record Statistics

IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 3 5655-S35 Date: 12/16/2009 Time: 00.05.14 Database Statistics (DBD: BKDB ) -------------------------------------------------------------------------------- ============================== Database Record Statistics ============================== Note: The mark * in column P means that the data element is used in the policy. Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_NUM_ROOT * 8,560,000 8,560,000 0 DB_AVG_DBREC_LENGTH * 288.06 288.06 0

The number of

database records

Average database

record length

Page 37: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 37

Database Diagnosis Report – Volume/Extent Statistics ... Data Set Statistics (DBD: BKDB , DSG: 01) -------------------------------------------------------------------------------- ============================ Volume/Extent Statistics ============================ Note: The mark * in column P means that the data element is used in the policy. Data set definition and environment information ================================================= Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_FLAG_SPACE_TYPE C C No change DB_NUM_PRI_SPACE 1,000 1,000 0 DB_NUM_SEC_SPACE 100 100 0 DB_FLAG_SMS * Y Y No change DB_MAX_EXT_DS 251 251 0 DB_MAX_EXT_VOL 123 123 0 Data set usage information ============================ Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_NUM_EXT 37 32 -5 DB_NUM_VOL 3 3 0 DB_NUM_UNUSED_VOL 0 0 0 DB_NUM_UNUSED_VOL_SER 0 0 0 DB_NUM_UNUSED_VOL_CAND * 0 0 0 Remaining available data set extent estimation ================================================ Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_AVAIL_EXT_LESS_100 * Y Y No change DB_NUM_AVAIL_EXT * 0 11 +11 DB_AVAIL_EXT_LIMIT VOL_FREE_EXTENTS VOL_FREE_EXTENTS No change Remark: If DB_NUM_UNUSED_VOL_CAND is not zero, more extents than those indicated by DB_NUM_AVAIL_EXT might be available.

Information on extents and used/unused

volumes for the subject data set

Estimation for extent availability

Information on data set allocation

parameters and DFSMS environment

Page 38: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 38

Database Diagnosis Report – Data Set Space Usage Statistics

IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 5 5655-S35 Date: 12/16/2009 Time: 00.05.14 Data Set Statistics (DBD: BKDB , DSG: 01) -------------------------------------------------------------------------------- =================================== Data Set Space Usage Statistics =================================== Note: The mark * in column P means that the data element is used in the policy. Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_BLOCK_SIZE 4,096 4,096 0 DB_NUM_DBDS_BLOCKS 999,179 673,780 -325,399 DB_MAX_DS_SIZE 4G 4G 0G DB_PCT_OF_MAX_DS_SIZE * 96% 65% -31 DB_RBA_HIGH_ALLOC 4,128,768,000 2,801,664,000 -1,327,104,000 DB_RBA_HIGH_USED 4,092,641,280 2,759,806,976 -1,332,834,304

This shows that, within data elements in

this category, only the value for the data

element DB_PCT_OF_MAX_DS_SIZE is

referrred to in one or more rules defined in

the selected policy.

You can see that the size of data set is

reduced by the reorganization.

Page 39: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 39

Database Diagnosis Report – IMS Space Utilization Statistics

IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 6 5655-S35 Date: 12/16/2009 Time: 00.05.14 Data Set Statistics (DBD: BKDB , DSG: 01) -------------------------------------------------------------------------------- ==================================== IMS Space Utilization Statistics ==================================== Note: The mark * in column P means that the data element is used in the policy. Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_BYTES_SEG 2,465,800,000 2,465,800,000 0 DB_BYTES_FREE_SPACE 1,610,723,680 283,136,678 -1,327,587,002 DB_BYTES_UNIDENTIFIED * 110 0 -110 DB_PCT_BYTES_FREE_SPACE * 39% 10% -29 DB_PCT_BYTES_SEG * 60% 89% +29 DB_PCT_UNUSED_BYTES * 1% 1% 0 DB_NUM_SEG 85,620,000 85,620,000 0 DB_NUM_VLSEG 8,560,000 8,560,000 0 DB_NUM_VLSEG_SPLIT * 0 0 0 DB_PCT_NUM_VLSEG_SPLIT * 0% 0% 0 DB_NUM_UNIDENTIFIED * 51 0 -51 DB_AVG_NUM_UNIDENTIFIED * 0 0 0 DB_NUM_FSE * 23,728,638 673,759 -23,054,879 DB_AVG_NUM_FSE * 23.75 1.00 -22.75 DB_NUM_FSE_MIN * 23,558,660 673,759 -22,884,901 DB_NUM_FSE_MAX * 718,662 673,759 -44,903 DB_AVG_NUM_NOREUSE_FSE * .17 0 -.17 DB_NUM_PTR 85,070,408 84,946,241 -124,167 DB_NUM_PTR_DIFF_BLK * 32,424,879 663,666 -31,761,213 DB_PCT_NUM_PTR_DIFF_BLK * 38% 1% -37

Free spaces fragmentation on each

database block has been resolved by

unload and reload of the database.

Segments of a database record fit in a

single database block in almost all

database records.

Allocated space is more efficiently used by

segment data.

Page 40: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 40

IMS High Performance Unload

IMS High Performance Unload (HP Unload) provides high performance database

retrieval service called High Speed Sequential Retrieval (HSSR) Engine

Provides DL/I call analyzer, segment retriever, and buffer handler that are optimized for GN

calls

• Similar to OSAM Sequential Buffering, but HSSR Engine supports VSAM ESDS also

Supports various unload record formats (HD, SH, CS, CP, F1, F2, F3, F6, A1, A2, A3, or A4)

Supports user-defined unload record formats

Provides an Unload API

• DL/I call and EXEC DLI commands: GU, GHU, GN and GHN call

HP Unload provides two unload utilities (FABHURG1 and FABHFSU)

HP Unload provides “Unload API”, which enables user DL/I unload applications to run on HP Unload without modification

IMS DLIBATCH Native DL/I Call Handler

Native Buffer Handler

IMS

HD Unload

utility

User

Unload

Application

User

Unload

Application

IMS HP Unload

Unload

utility

FABHURG1

JCL & output compatibility

HSSR Engine

No change is needed in many cases

Page 41: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 41

HP Unload – Major features

Common to FABHURG1 and FABHFSU utilities

Can unload a database without decompressing compressed segments

─This can decrease the CPU time and elapsed time

Supports multiple pre-defined unload record formats other than the

standard HD format

Can use HSSR Engine’s optional features to be explained later

IMS High Performance Unload

Page 42: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 42

HP Unload – FABHURG1 unload utility

Is a simple alternative for IMS HD Reorganization Unload

utility

Can run with using the same JCL statements

Can run on DLI and DBB regions in addition to ULU region

Supports a system exit for function extensions:

An exit routine name is specified in SYSIN

IBM-provided exit routines:

• FABHEXTR

• Creating a database extract for database tuning experiments or testing

• FABHKEYX

• Separating output data set at migration unload based on a set of high key

values specified

Users can write their own exit routines to add functions

IMS High Performance Unload

Page 43: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 43

HP Unload – FABHFSU unload utility

Is an unload utility for special purposes Uses JCL statements similar to IMS DLIBATCH

Supports not only ULU region, but also DLI region with a PSB having multiple PCBs for multiple databases

• Sensitivity can be controlled by using PCBs

Can produce multiple outputs with different formats for each output by a single run

Additional segment editing can be done by using user exit routines Different user exit routine can be specified for each output

Provides Parallel Scan Facility • For the compatibility with the old product FSU II

IMS High Performance Unload

Page 44: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 44

HP Unload – Unload API

Optimized for GN and

GNP calls with an

unqualified SSA

Enables an IMS DL/I batch

application program, which

reads a database

sequentially, to use HSSR

Engine without being

recompiled or relink-edited

Supports DL/I calls and

EXEC DLI commands

Through this API, the application program can retrieve and process

a large number of segments efficiently

IMS High Performance Unload

Page 45: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 45

HP Unload – HSSR Engine’s optional features

Report Database Tuning Statistics

Provides information about:

• the need for database reorganization

• the quality of HDAM or PHDAM randomizing

Report information on long database records

DB records that have the following characteristics:

• DB records whose length are greater than the specified bytes

• DB records that would require more than the specified number of I/Os for

retrieval

Information written for each DB record:

• The length of the DB record

• The number of database I/Os required to read all database segments of the

DB record

• The key value of the root segment of the DB record

IMS High Performance Unload

Page 46: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 46

HP Unload – HSSR Engine’s other features

Ability to continue processing after sequence errors Optional sequence-key checks for twin chains

Three options on an error: • ABEND

• Returns a GG status code and not unload the segment containing the error, or its dependents

• Returns a GX status code and unloads the segment in error

Reading a corrupted database Two options:

• Bypassing pointer errors

• Force the roots of HIDAM/PHIDAM DB to be accessed from the primary index

Detailed diagnostic information can be provided for sequence errors and pointer errors

IMS High Performance Unload

Page 47: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 47

IMS High Performance Load

Provides a high performance database load service

Processes compressed segment data as input

If unload dataset created by HP Unload utility

Supports various input record formats, including

Formats supported by HP Unload

User-defined unload record formats produced by HP Unload utility

Generates statistics reports to aid in tuning the database

Checks for database updates in between the unload and reload to prevent data

corruption

IMS DLIBATCH Native DL/I Call Handler

Native Buffer Handler

IMS

HD Reload

utility

User

Load

Application

DB

User

Load

Application

IMS HP Load

Reload

utility

No change is needed in many cases

JCL & input compatibility

HP Load service

Page 48: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 48

Can initialize an empty database or partition

HDAM/PHDAM partition: the entire root addressable area is preformatted

HIDAM/PHIDAM partition: a root segment with a key of all X'FF' is inserted

Provides options for whether/how to build ILDSs for HALDB partitions

Provides a user exit facility for additional user processing of each segment

WF1 record processing

WF1 records can be split into two groups:

• Records for logical relationship maintenance

• Records for secondary index maintenance

and be written into separate outputs

WF1 record generation can be suppressed

IMS High Performance Load

Page 49: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 49

Sorting before reload

Segment records in the input data set can be sorted internally before the reload processing starts

With this function, the SORT step before the reload step can be eliminated when the randomizing for HDAM/PHDAM has been changed between unload and reload

The function can be activated automatically if the input data set was created by HP Unload with certain options specified

IMS HP Load provides a Load API

Load API is optimized for ISRT calls

User initial load program can run on HP Load much faster than that on IMS DL/I

• The load program must specify a PCB with PROCOPT=L or LS

• For HDAM, internal physical sequence sort will be made before actual insert

for better performance

Calls that use AIB is supported

EXEC DLI is supported

Load API reduces time and cost for the initial load process

IMS High Performance Load

Page 50: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 50

IMS Index Builder

Build or rebuild index databases

Full-function non-partitioned databases

Secondary indexes: multiple indexes in one job step

Primary index of HIDAM or PHIDAM partitions

HALDB Indirect List Data Sets (ILDS)

Streamlines index creation, reorganization, maintenance, and recovery.

Eliminates the need for the traditional unload and reload processes at the time of reorganization of the primary database

Eliminates the need to image copy indexes

Exploits parallel processing by overlapping and integrating processes

Using subordinate address spaces (SAS) for parallel prime data scans and

index data sorts

Loading records into each index database in parallel

Overlapping DB scan steps and the index record sort steps overlap

Page 51: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 51

Creating secondary indexes by using DFSURWF1/DFSURIDX as input

DFSURWF1 data set that was created by the IMS HD Reorganization Reload utility or HP Load can be used

Data needed for prefix resolution are extracted from the DFSURWF1 file and written to a new data set, improving the performance of prefix resolution

DFSURIDX data set created by IMS Prefix Resolution utility can be used

PSINDEX support

Applies parallel partition scan to HALDB

Rebuilds selected PSINDEX partitions, and offers support for using Partition Selection Exits (PSE) when rebuilding PSINDEXes.

Building primary indexes

Builds a copy of the primary index of HIDAM database using the existing primary index as input

Creates new primary indexes of HIDAM and PHIDAM databases where input is based on a scan of the physical database without using the existing primary index

Index data set allocation and initialization

Supports IDCAMS delete/define functionality for data sets being built based on a user-provided procedure library so that no preliminary IDCAMS job steps are required

Can only initialize IMS ILDS, primary, and secondary index data sets by using an empty or dummy DFSURIDX data set as input

IMS Index Builder

Page 52: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 52

IMS High Performance Prefix Resolution

Enables you to resolve and update prefixes of IMS databases that are involved in

logical relationships as a single job step

Eliminates the WF2 and WF3 data sets

Executes prefix resolution and prefix update functions

Supports IMS Parallel Reorganization, single job step execution of database

reorganization, prefix resolution, and prefix update tasks

Supports the following three modes of operation:

Stand-Alone Prefix Resolution

• Replacement of IMS Database Prefix Resolution utility (DFSURG10)

Stand-Alone Prefix Update

• Replacement of the IMS Database Prefix Update utility (DFSURGP0),

Concurrent Prefix Update

• Running prefix resolution and prefix update in a single job step

• This function is not available in IMS standard utilities

Page 53: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 53

IMS High Performance Image Copy

High performance utility to create database back ups easily and quickly

For both Full Function (including HALDB) and Fast Path DEDB

‒ Batch IC and Concurrent IC

‒ Standard IC and DFSMS COPY IC

Integration with reorganization utilities included in the Database Solution Pack

IMS Reorganization Expert

IMS Online Reorganization Facility

Integration with High Performance Pointer Checker to ensure integrity of the

backup taken

Can be executed as a stand alone utility to create backups necessary to support a

recovery strategy

Provides a quick copy method (FlashCopy) with simple interface

Create stacked tape easily without special JCL control

Autonomics capability: switch copy technique based on database state

Page 54: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 54

HPIC Quick Copy Method for Stacked Image Copies

Requirement:

Reduce DB outage time when

creating stacked image copy

Solution: HPIC

Create shadow DB using FlashCopy

DB is brought back online

automatically

Read DB records from shadow

database

Create stacked output

DBDS1

DBDS2

DBDS3

Stop HDAM1 database

Read record in DBDS1

Put record into ICDS

Read record in DBDS2

Put record into ICDS

Read record in DBDS3

Put record into ICDS STACK

HDAM1

Delete shadow

Create Shadow by FlashCopy

Start HDAM1 database

DBDS1

DBDS2

DBDS3

Shadow

IMS command

IMS command

IMS High Performance Image Copy

Page 55: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 55

Autonomics: Internally switches IC process to CIC

Requirement:

Daily backup needed

Batch Image Copy is ideal but

active database may be busy when

job is submitted

Solution: HPIC

Checks DB authorization status,

and switch Image Copy process

Batch Image Copy (IC) to

Concurrent Image Copy (CIC)

automatically

55

DB1 DB2

Program Program Program

Suspend DB by DB QUIESCE

DB1 Busy (Command timeout )

Stop DB2

Check DB authorization status

RECONs

CIC DB1

Request Clean IC

IC DB2

IMS High Performance Image Copy

Page 56: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 56

Integration with HP Pointer Checker

Pseudo Online Pointer Check

DB suspended by DB QUIESCE

Shadow DB created by FlashCopy

after DB is quiesced

Checking data in parallel

DBDS1

DBDS2

DBDS3

Suspend DB by DB QUIESCE

Read record in DBDS1

Run pointer check

Read record in DBDS2

Run pointer check

Read record in DBDS3

Run pointer check

HDAM1

Delete shadow

Create Shadow by FlashCopy

Stop DB QUIESCE

DBDS1

DBDS2

DBDS3

Shadow Pointer Evaluation

IMS High Performance Image Copy

Page 57: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 57

IMS High Performance Pointer Checker

Ensures IMS Full Function and HALDB databases are operational, tuned,

repaired, and ready for use.

Consists of six utilities:

HD Pointer Checker (HPPC) – Main Component

• Two modes of execution

• Standard (Full) checking

• Hash checking

• Stand alone execution or integration with High Performance Image Copy

Space Monitor

HD Tuning Aid

DB Historical Data Analyzer

DB Segment Restructure

IMS Database Repair Facility

Page 58: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 58

58

HPPC reports condition of your IMS databases

Database validation – reports if databases

are error free or not

Statistics analysis – generates statistical

reports and keeps data for historical reasons

DBA needs to know database status

Damaged

Faulty

operation

Software or hardware error

Disaster

Space Shortage

Growth of data

Fragmentation

Data update

……

Reorganization

Backup

Recovery

Database Statistics

Analysis

© 2015 IBM Corporation

IMS High Performance Pointer Checker

Page 59: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 59

© 2014 IBM 59

HD Pointer Checker

Detecting errors in a database •Finds errors by checking direct pointers (physical pointers, logical pointers, hierarchical pointers,

pointers to free space, and index pointers) in IMS Full-Function and HALDB databases.

Describing the database

Prints a detailed description of the condition of the IMS database. This includes comprehensive

statistical information about pointers, segments, and free space.

Printing HDAM, HIDAM, PHDAM and PHIDAM database blocks (BLOCKMAP)

•Prints hexadecimal and character dumps of database control intervals or blocks. Each dump

includes a map of all the segments and free space elements in the block, as well as the relative

byte address (RBA) for each printed line.

Finding absolute disk addresses

•Converts an RBA into the corresponding absolute disk address. It calculates the cylinder, track,

record, and offset for each input RBA

Finding all pointers to a target segment

•Lists the RBAs of all pointers (in other segments, possibly in other databases) that point to a

specific target segment

Creating history records using HPPC ‘HISTORY dataset’

© 2015 IBM Corporation

IMS High Performance Pointer Checker

Page 60: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 60

60

Space Monitor Utility

It reports space information of database data set and monitors threshold for space information.

Space Monitor can run as a stand-alone program and can run in HDPC.

HDPC analyzes statistics from the aspect of IMS data, and Space Monitor analyzes them from the aspect of physically data set. You can get both kinds of statistics when you use HDPC and SPMN together.

//HDPC JOB

// EXEC

// :

//PROCCTL DD *

OPTION SPMN=YES, THRESHOLDS=…

//SPMNSPDT DD DSN=

HDPC reports

Space Monitor reports

DB HDPC

SPMN

© 2015 IBM Corporation

IMS High Performance Pointer Checker

Page 61: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 61

61

Space Monitor gets data set information from catalog and VTOC, analyzes them, and generates space statistics reports. For example,

Current space information:

• Data set size, Used space, the number of extents, free space, CI/CA Splits, allocated volumes, and etc..

Estimate available space, available extents, available volumes, and etc..

You can set thresholds for the space information by PROCCTL control statement or SPMNIN control card. If both of them are specified, PROCCTL is used.

It is recommended to use PROCCTL. It is easier than SPMNIN.

The benefits of SPMNIN is it can be used for both SPMN stand alone and HDPC.

//PROCCTL DD *

PROC ….

OPTION SPMNIN=YES, THRESHOLDS=(EXTENTS=13,USEDSPC%=95) DATABASE …. /*

//SPMNIN DD *

DBNAMEX DDNAMEX SAMPDSNAME 12020 10

/*

© 2015 IBM Corporation

IMS High Performance Pointer Checker

Space Monitor Utility

Page 62: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 62

HD Tuning Aid

Evaluate HDAM & HIDAM randomizer settings

Assist with tuning analysis:

– Current randomizer performance in (P)HDAM databases

– Current root segment location in (P)HIDAM databases

– Potential randomizer performance in (P)HDAM databases

– Potential randomizer performance if (P)HIDAM converted to HDAM

"KEYSIN" dataset must be created to run HD Tuning Aid

– Created from HDPC scan execution

IMS High Performance Pointer Checker

Analyzes the KEYSIN data set

• KEYSIN is a unique data set of HPPC

• KEYSIN is created by HDPC. It contains root key

HDTA cannot run in HDPC. Use HPPC sample cataloged procedure FABPPTA. It contains HDPC job step and HDTA job step in one job.

DB HDPC

KEYSIN

HDTA

Page 63: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 63

63

Managing Historical Statistics Data

The database statistics information are stored in :

IMS Tools KB output repository Common Repository of IMS Tools

History data set Unique file of HPPC

DB

HDPC

Statistics data

ITKB Server

History data set Output Repository

Statistics are recorded historically

HDPC &SPMN Reports

SPMN

© 2015 IBM Corporation

IMS High Performance Pointer Checker

Page 64: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 64

64

DB Historical Data Analyzer

Data collected and reported by HP Pointer Checker can be used to create

several reports.

History Data by DB-DS Report

History Data set by Key Date Report

HD Pointer Checker Summary Report

HD Analysis Report

Typical Uses:

Summarization of historical Pointer Checker information

To provide a graphical representation in the form of graphs,charts or tables showing

dataset allocation and usage of IMS databases.

Export the database statistics data from the HISTORY dataset to a flat file that can

be processed by user application programs

© 2015 IBM Corporation

IMS High Performance Pointer Checker

Page 65: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 65

65

DB Segment Restructure

Can perform tasks that would otherwise be extremely difficult including:

Initializing fields with specific values.

Moving data to different positions in a segment.

Changing the DB hierarchy by:

• combining segments

• splitting segments

• deleting segments

• moving segments to new hierarchic location

Creating a test database using part of a larger database

Converting database organization type

• HDAM database to HIDAM

• HIDAM and HISAM databases to HDAM

• HSAM database to HISAM

• HDAM, HIDAM, and HISAM databases to DEDB

Typical Uses:

- Restructure DBs (unload followed by load)

- Convert DB to another type

- Create test DB

© 2015 IBM Corporation

IMS High Performance Pointer Checker

Page 66: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 66

66

DB Segment Restructure

Old Hierarchy

New Hierarchy

with History

segment moved

which changes

segment codes for

History, Deliver

and Schedule

© 2015 IBM Corporation

IMS High Performance Pointer Checker

Page 67: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 67

67

• Repairs IMS databases that contain pointer or data errors

• Interactive Mode:

– Verify data interactively in ISPF

– View data

RBA Block Key

– Navigate by following pointers

– Repair (ZAP) pointers or data

– Can undo mistakes, or save changes to a batch job

• Batch mode: – Replaces VSAM Zapper product (but handles OSAM or VSAM) – Can verify data before replacing it but no “UNDO” capability

– Pre-checking of VER and REP

• Audit trail of all database repair activity

Database Repair Facility

© 2015 IBM Corporation

IMS High Performance Pointer Checker

Page 68: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 68

HALDB Toolkit

Provides functions that can be used for:

Simulating conversion to HALDB

Converting to HALDB

Re-partitioning of a HALDB

Maintaining a HALDB

Analyzer function

DBRC handling

Miscellaneous utilities which assist in HALDB handling tasks

• Indirect list key (ILK) rebuild

• This utility enables you to rebuild all ILKs and their references, and allows you to resolve

conflicting ILKs.

Page 69: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 69

CONVERT command

Backup

HALDB Toolkit

CONVERT

DBDLIB/ACBLIB

RECON FF DB

MDALIB

DBDLIB

RECON

IC datasets

HALDB

backup

//IHCSYSIN DD *

CONVERT –

DBD(HDAM1) –

DSNPREF(HALDB.HDAM1) –

PARTNUM(3) -

:

HSSR Engine

HP Load

HPIC+HPPC Hash

Page 70: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 70

Convert / Migrate to HALDB

CONVERT command can be used to:

•convert to HALDB in a single step batch job

•Integrate with ORF to allow ‘online’ conversion

Partitioning can be determined by:

•PARTNUM: specify the number of partitions

•PARTSIZE: specify the size of partitions

•KEYS: specify the high keys

ISPF Interface available to help orderly tracking of large conversion projects

Database Types that can be converted:

•HDAM to PHDAM

•HIDAM to PHIDAM

•HISAM to PHIDAM

•SHISAM to PHIDAM

•PDB/PDF to PHDAM/PHIDAM

Page 71: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 71

Operations of DL/I Batch program for HALDB with HALDB Toolkit

• HALDB Toolkit provides a function to treat a HALDB as if it is not registered.

(DBRC=N) • No SUBSYS record created

• No batch log created

• Normal procedure:

(1) /DBR

(2) Create an image copy#1

(3) Run DL/I Batch program

(4) Create an image copy#2

(5) /STA

DL/I Batch program

(DBRC=Y)

Create IC

IC#1

/DBR DB Create IC

IC#2

/STA DB

• Recovery procedure:

(1) Re-allocate DBDS

(2) Recovery from image copy#1

(3) Re-build ILDS/Primary Index

(4) Re-run DL/I Batch program

from the beginning

RECON info. SUBSYS record

Batch log record IC record IC record

RECON

X

Page 72: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 72

Library Integrity Utilities

A set of utilities designed to validate, compare, map, report, and regenerate a number of

IMS libraries.

IMS libraries supported

• ACB,DBD,PSB, and MFS libraries

• RECON data sets

• IMS Catalog

LIU utilities

• DBD/PSB/ACB Reversal

• DBD/PSB/ACB Mapper

• DBD/PSB/ACB Compare

• Consistency Checker

• Multiple Resource Checker

• Advanced ACBGEN, ACBLIB Analyzer

• MFS Reversal, MFS Compare

• Integrity Checker

• Catalog Manager =

Page 73: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 73

Advanced ACBGEN Utility

Advanced ACBGEN is faster than IMS standard ACBGEN.

Reduces I/O, elapsed time and CPU time significantly.

In our test system, IMS ACBGEN takes 97 seconds, Advanced ACBGEN takes 14

seconds to build 2600 members.

Preview function

Prior to an ACBGEN, a user can know how many members and which members will be

built.

Getting ACBGEN result in more concise format and unique information

A user can know which members are built.

Information of processed members Before and after ACBGEN.

Page 74: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 74

Multiple Resource Checker

DBD Library PSB Library

ACB Library RECON

DBD Library DBD Library

PSB Library PSB Library

ACB Library ACB Library RECON

RECON

Resource Check

Summary Report

Multiple

Resource

Checker

Input: Multiple Resources (DBD, PSB, ACB and RECON)

Up to 10 DBD/PSB/ACB libraries, 10 RECONs

If a user has many resources…..

• Other compare utilities • Process 1-1 for two resources in one job step.

• Need to run many jobs and analyze many reports for many members.

• Multiple Resource Checker compares multiple IMS resource in one job step. • Prints a consolidated summary report.

Page 75: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 75

IMS Catalog – An issue

DBDLIB

PSBLIB

ACBLIB ACBGEN

IMS ACBLIB and IMS Catalog must be synchronized.

However, there is no method to confirm whether ACBLIB, IMS Catalog and

directory are consistent.

Are IMS Catalog and

ACBLIB

synchronized?

? Populate

Are ACBs in the catalog consistent

to ACBLIB and consistent within IMS Catalog?

DBD

PSB

IMS Catalog

Directory

ACB (DMB) ACB (PSB)

Solution: LIU Catalog Manager Utility

Page 76: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 76

Catalog Manager Utility

Ensures DBDs and PSBs in the IMS catalog are maintained correctly.

Time-stamp validation

• Checks time-stamp of each DBD and PSB between ACBLIB and IMS Catalog

• Checks time-stamp of each ACB (DMB and PSB) member in the directory, If IMS

loading ACB from Catalog is activated

Database Version validation

• If database versioning is enabled, the utility also checks whether the database

version numbers are the same for each DBD.

WTO message & IMS Catalog Validation report

Page 77: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 77

Catalog Manager Utility - WTO messages

RC=00 : All of DBDs and PSBs are consistent between ACBLIB and IMS Catalog

RC=04 : Some of DBDs and PSBs are inconsistent between ACBLIB and IMS

Catalog.

If inconsistency is found, run the ACB Generation and Catalog Populate utility

(DFS3UACB) or the IMS Catalog Population utility (DFS3PU00) to correct the

IMS catalog.

Page 78: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 78

IMS Catalog Validation Report

Valid : same time-stamp and DB Version between ACBLIB and IMS Catalog

Invalid : different time stamp or member does not exist in IMS catalog

This report is available

for IMS V12 or V13

Page 79: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 79

Summary

IMS Database Solution Pack is a single, simplified, integrated solution which includes all

the necessary components to effectively and efficiently manage your IMS Full Function

Database Environment

IMS Online Reorganization Facility

IMS High Availability Large Database Toolkit

IMS Database Reorganization Expert

IMS High Performance Unload

IMS High Performance Load

IMS High Performance Prefix Resolution

IMS Index Builder

IMS High Performance Image Copy

IMS High Performance Pointer Checker

IMS Database Repair Facility (delivered as part of HP Pointer Checker)

IMS Library Integrity Utilities

Page 80: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 80

IMS Tools from IBM

Providing Modernization and Autonomics

IMS Fast Path Solution Pack

Autonomics IMS HP Fast Path Utilities IMS DIB Repair Facility IMS HP Image Copy IMS Library Integrity Utilities

IMS Recovery Solution Pack

IMS HP Image Copy IMS Database Recovery Facility IMS HP Change Accumulation IMS Index Builder IMS DRF Extended Functions

IMS Performance Solution Pack IMS Connect Extensions IMS Performance Analyzer IMS Problem Investigator

IMS Database Solution Pack Autonomics IMS DB Reorganization Expert - Unload, Load, Index Build, Prefix Resolution/Update IMS HP Image Copy IMS HP Pointer Checker IMS Library Integrity Utilities IMS HALDB Toolkit IMS Online Reorg Facility

IMS Sequential Randomizer Generator Data Base Administration

IMS Cloning Tool IMS Database Control Suite

Utility Management

IMS HP Image Copy IMS DEDB Fast Recovery IMS Recovery Expert

Backup and Recovery Batch Terminal Simulator

Program Restart Facility Application Management

Performance Management

IBM Transaction Analysis Workbench IMS Buffer Pool Analyzer IMS Network Compression Facility

System / TM Administration

System IMS Command Control Facility IMS ETO Support IMS HP Sysgen Tools IMS Queue Control Facility IMS Workload Router TM IMS Configuration Manager IMS Sysplex Manager

Regulatory Compliance

Guardium ITAP for IMS IBM Infosphere Guardium Data Encryption for DB2 and IMS Databases

IBM Tools Base for z/OS

Page 81: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 81

Thank You

Richard Phipps IBM IMS and IMS Tools

[email protected]

Page 82: IMS Database Solution Pack: user group Philadelphia 2015

© 2015 IBM Corporation 82

Legal Disclaimer

• © IBM Corporation 2014. All Rights Reserved.

• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained

in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are

subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing

contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and

conditions of the applicable license agreement governing the use of IBM software.

• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or

capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to

future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by

you will result in any specific sales, revenue growth or other results.

• If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will

experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage

configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

• If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs

and performance characteristics may vary by customer.

• Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM

Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server).

Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your

presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in

your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International

Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.

• If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other

countries.

• If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

• If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:

Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:

Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States

and other countries.

• If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:

UNIX is a registered trademark of The Open Group in the United States and other countries.

• If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of

others.

• If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta

Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration

purposes only.