IT41007: VERY SLOW RUNNING REPLICATION STORAGE RULE MIGHT LEAD TO IBM SPECTRUM PROTECT SERVER ACTIVE LOG PIN

8.1.15.100-IBM-SPSRV-WindowsX64
8.1.15.100-IBM-SPSRV-Linuxx86_64
8.1.15.100-IBM-SPSRV-Linuxs390x
8.1.15.100-IBM-SPSRV-Linuxppc64le
8.1.15.100-IBM-SPSRV-AIX
8.1.16.000-IBM-SPCMS-WindowsX64
8.1.16.000-IBM-SPCMS-WindowsI32
8.1.16.000-IBM-SPCMS-Linuxx86_64
8.1.16.000-IBM-SPOC-WindowsX64
8.1.16.000-IBM-SPOC-Linuxx86_64
8.1.16.000-IBM-SPOC-Linuxs390x
8.1.16.000-IBM-SPOC-LinuxPPC64le
8.1.16.000-IBM-SPOC-AIX
8.1.16.000-IBM-SPSRV-WindowsX64
8.1.16.000-IBM-SPSRV-Linuxx86_64
8.1.16.000-IBM-SPSRV-Linuxs390x
8.1.16.000-IBM-SPSRV-Linuxppc64le
8.1.16.000-IBM-SPSRV-AIX
8.1.14.200-IBM-SPSRV-WindowsX64
8.1.14.200-IBM-SPSRV-Linuxx86_64
8.1.14.200-IBM-SPSRV-Linuxs390x
8.1.14.200-IBM-SPSRV-Linuxppc64le
8.1.14.200-IBM-SPSRV-AIX

APAR status

• Closed as program error.

Error description

• A very slow running replication storage rule might lead to an
  active log pin condition. IBM Spectrum Protect Server can crash
  due to sever active log pinned after at least one failed
  replication storage rule and a restart of the IBM Spectrum
  Protect Server has occurred. The pin occurs from slow running
  replication storage rule transactions that become slower and
  slower due to enormous amount of extents that needing updates
  and they need to be updated in database table
  SD_REFCOUNT_UPDATES. This table is used to manage the reference
  counts for the deduplication catalog. The issue is the backlog
  of the SD_REFCOUNT_UPDATES table since upon startup the thread
  that is responsible for managing that table gets into a single
  threaded mode trying to resolve inflight entries from the failed
  replication. The queries into this table, used by the
  replication storage rule process, become slower and slower and
  eventually the IBM Spectrum Protect Server active log fills up
  as there are a lot of entries in SD_REFCOUNT_UPDATES table from
  the failed replication storage rule. This will only occur on the
  target side of IBM Spectrum Protect Server.
  
  This APAR can be identified by running following command on
  target server:
  QUERY EXTENTUPDATES <STORAGE_POOL_NAME>
  
  If the result of the query is Billions of entries under the
  "Number of Extents Pending Update" column than this APAR may
  apply. The other requirement is if replication storage rules
  were, or are, in use.
  
  IBM Spectrum Protect Versions Affected:
  IBM Spectrum Protect Server 8.1.13 and above on all supported
  platforms
  
  Additional Keywords:
  Spectrum Protect; TSM; stgrule; replication; crash; DB2; active
  log, EXTENTUPDATES, Number of Extents Pending Update,
  TS009154410
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * All IBM Spectrum Protect server users.                       *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See error description.                                       *
  *                                                              *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be                                              *
  * fixed in levels 8.1.14.200, 8.1.15.100 and 8.1.16.  Note     *
  * that this is subject to change at the discretion of IBM.     *
  ****************************************************************
  

Problem conclusion

• This problem was fixed.
  Affected platforms for reported release:  AIX, Linux, and
  Windows.
  Platforms fixed:   AIX, Linux, Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels