DB2DART - DB2Night Show October 2011

DB2DARTDART

The Information Management Specialists

Iqbal Goralwalla

DB2Night Show

October 14, 2011

About Triton

• Formed in 1996

• IBM Premier Business Partner

• Largest independent DB2 Consultancy in the UK


• Largest independent DB2 Consultancy in the UK

• Strong team of highly experienced and certified DB2 specialists

• Work with customers across a range of sectors providing them with DB2 consultancy and support for both DB2 LUW and DB2 z/OS

• Preferred DB2 services supplier to IBM UK and others

About Me

• Head of Midrange (DB2 on LUW) at Triton Consulting

• Principal Consultant on DB2 LUW


• Principal Consultant on DB2 LUW

• Experience of DB2 LUW since DB2 Common Server

• IBM Champion for Data Management

• Tendency to talk too much

DB2DART

• DB2DART – An Overview

• My 3 Key DB2DARTs


• My 3 Key DB2DARTs

• My bonus DB2DART

• Anything on DB2DART and DPF?

• What about INSPECT?

• GOTCHAS

db2pd

db2top

db2exfmt

• One of the less used performance reporting tools

DB2DART – An Overview


db2exfmt

db2support

db2diag

db2trc

db2dart

Why am I here?

tools

• Historically used with IBM support

• DBAs can use the tool –increases productivity


• Database Analysis and Reporting Tool

• “Offline” tool for checking the architectural correctness of a database


database� Checks validity of meta-data structures, page headers, row

headers, etc.

• Critical for investigating problems involving data corruption

• Runs against the data on disk� Not aware of what’s in the buffer pool

� May show false errors if users are connected to the database


• Granularity� Database: db2dart <db> /DB

� Tablespace: db2dart <db> /TS


� Tablespace: db2dart <db> /TS

� Table: db2dart <db> /T /TSI /OI

• Options for inspecting, formatting, and repairing data

• Run db2dart /H to see all of the supported options

DB2DART – Inspecting data


Examples:

db2dart <db> /DB

db2dart <db> /TS /TSI 2

db2dart <db> /T /TSI 2 /OI 6

DB2DART – Formatting data


Example: db2dart <db> /DD /TSI 2 /OI 6

DB2DART – Repairing data


DB2DART – My 3 Key DB2DARTsDARTs

1. Storage Reclamation and High Water Mark

2. Index Corruption


2. Index Corruption

3. Extracting Data

Storage Reclamation – the DBA

dilemma

Deleted thousands of rows

Why is the space not being released?


rows

Dropped a BIG table

Reorged the table

Storage Reclamation – HWM for DMS

tablespaces• High Water Mark (HWM) is the highest numbered page

that is currently allocated in a DMS tablespace

• HWM can be much higher than the number of pages


• HWM can be much higher than the number of pages currently in use due to:� Dropped tables or delete activity

� Offline table reorg using same tablespace

� Index reorg using “allow read or write access”

• Makes it difficult to release unused disk space for reuse

• HWM can be seen using LIST TABLESPACES SHOW DETAIL or GET SNAPSHOT FOR TABLESPACES

Storage Reclamation – HWM for DMS

tablespaces

• LIST TABLESPACES SHOW DETAIL


HWM – Dropped table (1)

DMS TABLESPACE

T1 T1

DMS TABLESPACE


Table Space HWMdoes NOT change

T2

T1

Drop table T2

Free

T1

5 pages

3 pages

2 pages

Used pages = 10 Used pages = 8

HWM = 10 HWM = 10

5 pages

3 pages

2 pages

HWM – Dropped table (2)

DMS TABLESPACE

T1 Free

DMS TABLESPACE


Table Space HWMDOES change

T2

T1

Drop table T1

Free

5 pages

3 pages

2 pages


HWM = 10 HWM = 8

5 pages

3 pages

2 pages

T2

HWM – Offline Table Reorg (1)

DMS TABLESPACE

T1

REORG TABLE T1

Free

DMS TABLESPACE

T1

INTERMEDIATE STATE


Table Space HWMINCREASES

Free

T1


HWM = 5 HWM = 11

5 pages

4 pages

6 pages

7 pages

T1Shadow Copy

HWM – Offline Table Reorg (2)

DMS TABLESPACE

Free

REORG TABLE T1

T1

DMS TABLESPACE

T1

INTERMEDIATE STATE

Shadow


Table Space HWMDECREASES

T1Free

Used pages = 4Used pages = 4

HWM = 11 HWM = 4

7 pages

7 pages4 pages

4 pages

T1

Shadow Copy

HWM headaches

• Space not released even though free extents available

• Redirected RESTORE needs space equal to or greater than the HWM


than the HWM

� All extents in DMS table spaces up to the HWM copied to the backup image.

� Tablespaces containers cannot be reduced below HWM

• ALTER TABLESPACE <drop container/reduce containers size> only affects extents above the HWM

Storage Reclamation – the DBA

dilemma (DB2 9.1/9.5)

Shall I call IBMSupport? DB2DART


Support?

Not when I’m around!

Storage Reclamation – DB2DART 1DART 1

• db2dart /DHWM – display table space and high-water mark information� A map of the extents in the table space, showing


� A map of the extents in the table space, showing objects owning them

� The ID and type of the object holding the high-water mark extent

� Amount of free extents and in-use extents below the high-water mark

� TIP: If online, use db2pdcfg –flushbp first (V9.1FP5, V9.5FP1)

db2dart /DHWM


LIST TABLESPACES SHOW DETAIL

db2dart /DHWM – Drop table ID 5


LIST TABLESPACES SHOW DETAIL

Storage Reclamation – DB2DART

• db2dart /LHWM – Assists in lowering the high-water mark� Table space ID and desired high-water mark are required


� Table space ID and desired high-water mark are required

� May not be possible to actually lower it to the desired level

� Use "0" for the desired high-water mark to go as low as possible

� Result is a list of operations to execute (EXPORT/LOAD, REORG, etc.)

� Estimate of resulting in use and free extents below the high-water mark is displayed for each step

db2dart /LHWM


db2dart /LHWM – after REORG



• db2dart /RHWM – Removes empty SMP extents holding up the high-water mark


� DB2 9.5 – ALTER TABLESPACE <tbsp> REDUCE/DROP

►Drops/reduces containers to the table space’s high-water mark

► First removes empty SMP extents

►No need for db2dart /RHWM


enhancements• V9.1 FP4 – db2dart /DHWM and /LHWM can be

run against active/inconsistent database� IZ04647 http://www-


� IZ04647 http://www-01.ibm.com/support/docview.wss?uid=swg1IZ04647

• V9.1 FP5 – db2dart /DHWM is not correct when an empty SMP extent is holding the HWM� IZ07663 http://www-

01.ibm.com/support/docview.wss?uid=swg1IZ07663

Storage Reclamation – DB2 9.7

• For a DMS or Automatic Storage table space created in DB2 9.7, you can use reclaimable storage to return unused storage to the system for reuse.

• For Automatic Storage table spaces, the REDUCE MAX


• For Automatic Storage table spaces, the REDUCE MAX option of ALTER TABLESPACE can be used to release all unused spaced

• For DMS, two steps are required to release the unused extents� The High Water can be reduced to match the number of

used extents with the LOWER HIGH WATER MARK option� The REDUCE, RESIZE or DROP options then be used

• Reclaiming storage is an online operation

Index Corruption

• Possible causes� Hardware or software failure

� Killing an executing process (e.g., LOAD)


� Killing an executing process (e.g., LOAD)

� After migration from one system to another

• Symptoms� Bad page errors

� SQL0980C during a backup database command

� SQL0902C – database marked bad

Index Corruption – the DBA dilemma

Shall I call IBMSupport?

Restore from backup?

DB2DART



Index Corruption – DB2DART 2DART 2

• Find out which indexes are problematic� db2dart <db> /DD /TSI <pool id> /OI <object id>� db2dart <db> /DI /TSI <pool id> /OI <object id>


� db2dart <db> /DI /TSI <pool id> /OI <object id>

• Mark indexes as invalid� db2dart <db> /MI /TSI <pool id> /OI <object id>

• Indexes recreated based on INDEXREC (database configuration parameter)

► RESTART► ACCCESS► Monitor using db2diag.log

Index Corruption – DB2DART

• Indexes recreated:� First access of invalid index

� RESTART DATABASE


� RESTART DATABASE

� LOAD – Build phase

� Classic offline table reorg

• To prevent access to problematic index� LOCK TABLE IN EXCLUSIVE MODE

Index Corruption – DB2DART –

Partitioned tables/indexes

• db2dart /MI requires table space ID and index object ID.� For partitioned indexes, use INDPARTITIONOBJECTID


� For partitioned indexes, use INDPARTITIONOBJECTID and INDPARTITIONTBSPACEID from SYSCAT.INDEXPARTITIONS.

Extracting Data

• Table space or database is corrupt or inaccessible• Access to “bad/damaged” data causes an

instance crash


instance crash• Need to salvage data

� Cannot use SQL to select data� Cannot use Export� Restore from backup image

►Does one exist?►Circular logging?

Extracting Data – the DBA dilemma

Shall I call IBMSupport? DB2DART


Support?


Extracting Data – DB2 DART 3DART 3

• db2dart <db> /DDEL• Dumped in delimited ASCII format• Prompted for


• Prompted for � table name (or ID), � table space ID, � starting page, � number of pages, and � output file name

• LONG VARCHAR or LOB data will not be dumped

Extracting Data – DB2DART

• May require multiple invocations� Dump up to “bad/damaged” page using /DDEL option

Aborts on reaching corruption


►Aborts on reaching corruption

� Dump “bad/damaged” page using /DD option► Incorrupt data can be viewed

� Dump rest of pages after “bad/damaged” page using /DDEL option and appropriate start page

• Some data loss inevitable

Backup Pending State (aka “Forced

backup”)

• LOAD with COPY NO (default)

• Logging strategy changed (circular to


• Logging strategy changed (circular to archival)

• LOGARCHIVE enabled

• LOGFILSIZ changed while using raw device for logging

Backup Pending State – the DBA

dilemma

Do I really needto do a backup? DB2DART


to do a backup?


Backup Pending State – bonus DB2DARTDART

• db2dart <db> /CHST /WHAT DBBP OFF


DB2DART – Does it work with DPF?

• Yes!

• Run db2dart on desired partition or on all partitions


partitions

• Use db2_all to run db2dart on all partitions in a single invocation

• Default destination of report file is in diagnostic directory

DB2DART – IBM service options

• /ETS – extends the table limit in a 4 KB table space (DMS only), if possible

• /MT – marks table with drop-pending state


• /MT – marks table with drop-pending state� Needs password – supplied by IBM

• /IP – initializes the data page of a table as empty� Needs password – supplied by IBM

What about INSPECT?

• Online “equivalent” of DB2DART

• Integrated into runtime engine resulting in performance gains


performance gains� Makes use of prefetchers and buffer pools

• Runs on all logical nodes in a partitioned system� Can be directed to run on specific nodes

• Can estimate row compression results for table, dictionary based on sample

What about INSPECT?

• Granular: inspection can be on database, tablespace, or a table

• Results (unformatted) are written to the


• Results (unformatted) are written to the specified file (placed in diagnostic directory path). Use db2inspf to format results

DB2DART vs. INSPECT

• INSPECT does not provide any of the formatting or repair options that db2dart provides

• Unlike DB2DART, INSPECT of a tablespace will only


• Unlike DB2DART, INSPECT of a tablespace will only process objects that reside in that tablespace. It will not inspect:� Corresponding index objects residing in a different

tablespace� LOB objects residing in a different tablespace� Note: running DB2DART on an INDEX tablespace would

not seek out the parent data

DB2DART – Gotchas

• Security – user with authority to run the db2dart tool or with access to the db2dart output can gain access to information that they might not be


gain access to information that they might not be authorized for

• LOAD – ALLOW READ ACCESS option not supported if indexes marked invalid

Summary & Conclusions

• DB2DART is a very powerful tool

• Can be used by DBAs to deal with:


• Can be used by DBAs to deal with:

� Storage Reclamation

� Index Corruption

� Extracting Data

� Database backup pending state

• Increases productivity

DB2DART - DB2Night Show October 2011

Documents