IT20368: SQLB_BADHDR ERROR DURING SQLB_VERIFY_PAGE CAUSING FODC DUMPS ANDDB MARKED BAD.

Db2 Version 11.1 Mod 3 Fix Pack 3 for Linux, UNIX, and Windows
Db2 Version 11.1 Mod3 Fix Pack3 iFix001 for Linux, UNIX, and Windows
Db2 Version 11.1 Mod3 Fix Pack3 iFix002 for Linux, UNIX, and Windows

APAR status

• Closed as program error.

Error description

• Some partial writes/reads may occur during rebalance/reduce max
  or lower high water mark operations causing SQLB_BADHDR errors,
  FODC dumps and bringing down db2.
  
  With this defect fix, a new registry variable is being
  introduced so that when set, it will verify extents in the
  following manner to prevent any future bad pages and/or dump
  detailed diagnostic data into db2diag.log and into a binary dump
  for further root cause analysis:
  
  -New registry variable DB2_EM_PAGE_VERIFICATION (db2set
  DB2_EM_PAGE_VERIFICATION=TRUE)
     -Extent verification during rebalance operations
        -Verifies that one full extent is read successfully, it
  does not verify data consistency of extent's pages
        -If extent read was not successful, retry read 2 more
  times
        -Verbose logging that dumps diagnostic data on failed
  reads, and states if re-reads were successful
        -If read fails both re-reads, release latches and wait for
  1 minute then retry extent move operation (retry forever)
     -Extent verification during reclaim storage (i.e. LOWER HIGH
  WATER MARK, REDUCE MAX)
        -Verifies that each page of extent has correct checksum
  pending the page header is not corrupted
        -If bad page identified, re-read extent up to 2 more times
        -If all re-reads fail, stop extent movement and return
  error
        -Verbose logging that dumps diagnostic data on failed
  reads, and states if re-reads were successful
  
  With this defect fix, there is also an update to an existing DB2
  registry:
  
  -Updated registry variable DB2_BCKP_PAGE_VERIFICATION (db2set
  DB2_BCKP_PAGE_VERIFICATION=TRUE)
     -Extent verification during offline/online backup
        -Same page verification as extent verification during
  reclaim storage
        -Updated retry on failure to re-read extent up to 2 more
  times
  

Local fix

• No Local Fix
  
  The corrupted tables need to be re-created and re-loaded with
  data. Some data loss is expected.
  
  The data from corrupted tables can be extracted via db2dart
  /DDEL command to be imported into newly created tables.
  
  Link to db2dart command for more details:
  
  https://www.ibm.com/support/knowledgecenter/SSEPGG_11.1.0/com.ib
  m.db2.luw.admin.cmd.doc/doc/r0003477.html
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * ALL                                                          *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See Error Description                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Upgrade to Db2 11.1 Mod 3 Fix Pack 3 or higher               *
  ****************************************************************
  

Problem conclusion

• First fixed in Db2 11.1 Mod 3 Fix Pack 3
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

  IT20332

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  DB2 FOR LUW

• Fixed component ID

  DB2FORLUW

Applicable component levels

• RB10 PSN

     UP