IBM Support

Unload in native mode from legacy table spaces can return incorrect data (INCORROUT)

Troubleshooting


Problem

Db2 High Performance Unload for z/OS can return incorrect data when processing, in native mode, the data from a table space that is created with a version of Db2 for z/OS prior to V5.1 and that has never been reorganized or loaded in REPLACE mode. The concerned table spaces are non-LARGE table spaces that have both:
  • Data pages with a page number (PGNUM) with non-zero leftmost byte (PgLogID field)
  • Data pages with a page number with a zero leftmost byte
A non-zero value in the PgLogID field for a non-LARGE page set indicates that the page hasn't been updated, and the page set has not been rebuilt by a Db2 utility, since V3, if at all.

Symptom

Incorrect Data out, data loss

Cause

The data from a table space created with a version of Db2 for z/OS prior to V5.1 and never reorganized or loaded in REPLACE mode.
A non-zero value in the PgLogID field for a non-LARGE page set indicates that the page hasn't been updated, and the page set has not been rebuilt by a Db2 utility, since V3, if at all.

Environment

Db2 11 for z/OS and Db2 12 for z/OS

Diagnosing The Problem

Determining the effected objects

To determine which table spaces are effected you can choose either of the following method:
 a) Set the VUX038/CHECKINT parmlib parameter to LEVEL1 so that Db2 HPU fails with ABENDUU1021 and REASON=0000000C whenever such an object is processed
 b) List the table spaces that:
  1. Don't have RTS  or have RTS but have not been reorganized or loaded in REPLACE mode
  2. Were created after the Db2 subsystem was upgraded to Db2 5 or later
    The list of such objects is given by the following query against the Db2 catalog tables
SELECT A.DBNAME, A.NAME, A.CREATEDTS
FROM SYSIBM.SYSTABLESPACE A
WHERE DBNAME NOT IN('DSNDB01', 'DSNDB06', 'DSNDB04')
AND
(
NOT EXISTS (SELECT * FROM SYSIBM.SYSTABLESPACESTATS B
WHERE A.DBNAME = B.DBNAME AND A.NAME = B.NAME)
OR
EXISTS (SELECT * FROM SYSIBM.SYSTABLESPACESTATS C
WHERE A.DBNAME = C.DBNAME AND A.NAME = C.NAME AND C.REORGLASTTIME IS NULL AND C.LOADRLASTTIME IS NULL)
)
AND A.CREATEDTS < date-of-the-upgrade-to-Db2_V5
Where date-of-the-upgrade-to-Db2_V5 is the date when the Db2 subsystem was upgraded to Db2 version 5 or later.
Be aware that this list can contain table spaces that are not currently effected by the issue (incorrect output) but that might be effected later if the object is updated. Therefore, it is advisable to apply the solution to all the table spaces listed by this query.

Resolving The Problem

Considerations about the solution implementation

Take into account your exact situation regarding the number of tables potentially effected as well as your constraints/requirements and choose one of the following solutions or develop the most suitable solution based on them.

Use Db2 HPU jobs to identify the concerned table spaces

Use the method (a) previously described. Whenever a Db2 HPU step with ABENDUU1021 and REASON=0000000C, determine the table space that was being run when the ABEND occurred and apply the previously described solution.  

The next run of the step should complete successfully unless another table space for which processing was aborted because of the ABENDU1021 is also effected by the issue. In the latter case, apply the solution to the newly effected table space, run the unload step again, and proceed on in the same way until the step completes successfully.
The solution is complete and comprehensive when all the jobs invoking Db2 HPU have been run successfully.

This solution ensures that Db2 HPU cannot produce incorrect data when run against a table space effected by the issue.

The drawbacks of this solution:
▪ Db2 HPU jobs will face at least as many failures as there are jobs that involve effected table spaces
▪ Performance of Db2 HPU jobs run in native mode is decreased because of VUX038/CHECKINT being set to LEVEL1

Fix all concerned table spaces and start using Db2 HPU again

Use the second method (b) and apply the solution to all the table spaces listed by the query.
The solution is complete when all the table spaces from the list have been processed.

This solution avoids any further failures of Db2 HPU jobs.

The drawbacks of this solution:
▪ Your system might freeze while you reorganize the whole set of concerned table spaces
▪ More table spaces than necessary might be processed or some effected table spaces might not be processed if the date when the Db2 subsystem was upgraded to Db2 5 or later cannot be accurately determined

To fix every effected table space before enabling the use of Db2 HPU, but avoid potential problems later, consider building a dedicated job for every table space from the Db2 subsystem with the following steps:
  1. A Db2 HPU step that requires a physical unload into a dummy data set from the table space
  2. A conditional REORG step (that can be replaced by a DSNUTILB UNLOAD step followed by a LOAD REPLACE step) run only if the unload stops with ABENDUU1021 and REASON=0000000C.
The solution is complete when all the dedicated jobs have completed either with a successful Db2 HPU step or a successful REORG step.

Fix the concerned table spaces as quickly as possible without stopping using Db2 HPU

You can use both methods (a and b) as follows:
  1. For safety - First set VUX038/CHECKINT to LEVEL1 (a) to make sure Db2 HPU does not unload incorrect data when processing a table space as described previously (an ABENDUU1021 with REASON=0000000C would occur instead)
  2. To fix as quickly as possible - Run the query from method (b) and apply the solution to the listed table spaces, starting with the oldest ones and as soon as possible without stopping running the Db2 HPU jobs. While this task is performed, apply the solution to any table space that triggers an ABENDUU1021 and REASON=0000000C in a Db2 HPU step
  3. Wait for fix completion - Wait until all the Db2 HPU steps from your jobs have been successfully run one time
  4. Restore initial performance - Set VUX038/CHECKINT back to BASIC so that the Db2 HPU performance is restored (performance is slightly decreased when LEVEL1 is used)

Recommendation for new Db2 HPU jobs

Important: During the fix and after it is complete, use the CHECK_INTEGRITY LEVEL1 option for a first run of a new job and to reorganize the involved table space if an ABENDUU1021 with REASON=0000000C failure occurs. After a run with CHECK_INTEGRITY LEVEL1 has completed successfully, the CHECK_INTEGRITY LEVEL1 can be removed.
 

Document Location

Worldwide

[{"Line of Business":{"code":"LOB10","label":"Data and AI"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSAUUV","label":"IBM Db2 High Performance Unload for z\/OS"},"ARM Category":[{"code":"a8m0z0000001hFbAAI","label":"Db2 High Performance Unload for z\/OS"}],"ARM Case Number":"","Platform":[{"code":"PF035","label":"z\/OS"}],"Version":"5.1.0"}]

Document Information

Modified date:
08 February 2021

UID

ibm16406218