IC92576: DELETE VOLUME MAY HANG WITH MESSAGE ANR9999D_1671797418 ON DEDUP STORAGE POOL VOLUME

IBM Tivoli Storage Manager V6.3 Fix Pack 5 (6.3.5.000) Server Downloads

APAR status

• Closed as program error.

Error description

• When multiple DELETE VOLUME processes are running concurrently,
  continuously or closely after each other, the Tivoli Storage
  Manager server may hang due to the concurrent dedup chunks
  processing.
  Multiple threads can process the same chunks or  table records
  in different order, thus multiple dedup delete threads may
  remain in a waiting state, and a DB2 deadlock may occur.
  Once the deadlock is resolved by the timeout mechanism, the
  Tivoli Storage Manager server aborts one of the database
  deadlock transactions, that will be rolled back and the Tivoli
  Storage Manager activity log displays an  ANR9999D_1671797418
  bfUncatalogDedupBitfile message.
  Customer/L2 Diagnostics:
  The Tivoli Storage Manager activity log shows the following
  messages:
  ANR0159E tbrsql.c(1430): Database deadlock detected on 23:5.
  ANR0162W Supplemental database diagnostic information:
  -1:40001:-911 ([IBM][CLI Driver][DB2/SUN64] SQL0911N  The
  current transaction has been rolled back because of a deadlock
  or timeout.  Reason code "2".  SQLSTATE=40001).
  [...]
  ANR9999D_1671797418 bfUncatalogDedupBitfile(bfdedup.c:2063)
  Thread<>: Error 87 checking extents for object 562386510 in
  pool -1~
  ANR9999D Thread<> issued message 9999 from:~
  ANR9999D Thread<>  00000001807E9566 OutDiagToCons()+1a6~
  ANR9999D Thread<>  00000001807E3372 outDiagfExt()+f2~
  ANR9999D Thread<>  00000001801C992C
  bfUncatalogDedupBitfile()+1cc~
  ANR9999D Thread<>  00000001800AF0FF AdmVolDelThread()+41f~
  ANR9999D Thread<>  0000000180006865 startThread()+125~
  ANR9999D Thread<>  00000000754F2FDF endthreadex()+47~
  ANR9999D Thread<>  00000000754F3080 endthreadex()+e8~
  ANR9999D Thread<>  0000000077A8652D BaseThreadInitThunk()+d~
  ANR9999D Thread<>  0000000077BBC521 RtlUserThreadStart()+21~
  Tivoli Storage Manager Versions Affected:
  Tivoli Storage Manager server 6.1 6.2 6.3 on all supported
  platforms
  Initial Impact:
  Low
  Additional Keywords:
  zz61 zz62 zz63 TSM deduplication hang hung stgpool
  

Local fix

• Run single 'delete volume' commands at a time against the
  affected dedup storage pool.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All Tivoli Storage Manager server users of   *
  *                 the DELETE VOLUME command.                   *
  ****************************************************************
  * PROBLEM DESCRIPTION: See ERROR DESCRIPTION.                  *
  ****************************************************************
  * RECOMMENDATION: Apply fixing level when available. This      *
  *                 problem is currently projected to be fixed   *
  *                 in levels 6.3.5. Note that this is subject   *
  *                 to change at the discretion of IBM.          *
  ****************************************************************
  *
  

Problem conclusion

• This problem was fixed.
  Affected platforms:  AIX, HP-UX, Solaris, Linux, and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R63A PSY

     UP

• R63H PSY

     UP

• R63L PSY

     UP

• R63S PSY

     UP

• R63W PSY

     UP