IT37363: ANR0162W, ANR0106E, 100% CPU - DATA IS NOT EXPIRED FROM CONTAINER WHEN ELIGIBLE CHUNK DELETION IN THE BILLIONS

8.1.12.200-IBM-SPSRV-WindowsX64
8.1.12.200-IBM-SPSRV-Linuxx86_64
8.1.12.200-IBM-SPSRV-Linuxs390x
8.1.12.200-IBM-SPSRV-Linuxppc64le
8.1.12.200-IBM-SPSRV-AIX
IBM Spectrum Protect Server V8.1 Fix Pack 13 (V8.1.13) Downloads
IBM Spectrum Protect Server V8.1.12.X interim fix downloads

APAR status

• Closed as program error.

Error description

• If the number of container pool chunks eligible for deletion
  reaches the order of billions, then backup processing to
  directory container pool maybe be slower than otherwise. Server
  usage of 100% CPU may also occur, along with ANR0162W arithmetic
  overflow/exception and ANR0106E error 2342 messages logged in
  the activity log. When this symptom occurs, no data is expired
  from the directory container and query extendupdates command
  hangs and does not respond.
  
  
  Problem affects both directory and cloud container pools
  
  Customer/Support diagnostics:
  
  When IBM Spectrum Protect Server has chunks to delete and
  reaches the order of billions of chunks, thus the backup process
  to be stored in the storage pool container becomes slow, keeping
  it almost full and dsmserv process may use 100% CPU. When there
  is an attempt to decrease chunks, the error below is generated
  stating that there was an arithmetic overflow is not enough to
  hold the total count of chunks to delete them.
  
  At query actlog shows the message:
  
  ANR0162W Supplemental database diagnostic information:
  -1:22003:-802 ([IBM][CLI Driver][DB2/AIX64] SQL0802N Arithmetic
  overflow or other arithmetic exception occurred.
  SQLSTATE=22003).
  ANR0106E sdutil.c(14286): Unexpected error 2343 fetching row in
  table "SD.Chunk.Locations".
  
  Regarding the operations, it covers the chunk deletion engine,
  container defragmentation process and query extentupdates
  command.
  
  To check the number of chunks to delete, run against IBM
  Spectrum Protect server command line the command and check
  'Number of Extents Eligible for Deletion:':
  
  query extentupdates
  
  In this scenario, the query extentupdates command hangs and does
  not respond.
  
  IBM Spectrum Protect versions affected:  V7.1 and V8.1 on all
  supported platforms
  
  Additional Keywords:
  TSM "Spectrum Protect" TS005164517 ANR0106E ANR0162W
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * All IBM Spectrum Protect server users.                       *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See error description.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be fixed in levels 8.1.12.200 and 8.1.13. Note  *
  * that this is subject to change at the discretion of IBM.     *
  ****************************************************************
  

Problem conclusion

• This problem was fixed.
  Affected platforms for reported release:  AIX, Linux, and
  Windows.
  Platforms fixed:  AIX, Linux, and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R81A PSY

     UP