IT48310: REPLICATION STORAGE RULE HANGS AND LEAVES SESSIONS ON TARGET SERVER

APAR status

• Closed as program error.

Error description

• This APAR applies specifically to environments where the target
  replication server has multiple directory or cloud pools (or a
  combination of both)
  
  Replication  storage rule process will appear to hang on the
  source server. The target server sessions remain active
  requiring a server restart to terminate.
  The source server will indicate a timeout in the dsmffdc.log
  file:
  
  [ FFDC_REPLICATION ]: [2138734](nrmain.c:14173) Waiting phase
  completed condition timeout after 60 minutes, wait start time
  04/16/2025 12:16:43 PM.
  
  The target server sessions remain active when the source server
  sessions are ended. A restart of the target server is required
  to clear these sessions.
  Subsequent replication storage rule processes report ANR1652E
  Unresolved Chunks error.
  
  Replication process on the source server may appear hung when
  the maximum of 100 inflight replication phases is reached. In
  this case, all of the NrReplicateBatch threads will look similar
  to:
  
  Thread 80321, Parent 80119: NrReplicateBatch, Storage 8640,
  AllocCnt 34658 HighWaterAmt 9520448
  tid=35dc1, ptid=215f7, det=0, zomb=0, join=0, result=0, sess=0,
  procToken=14, sessToken=52664
  Stack trace:
  0x090000000013966c __fd_select
  0x0000000100015398 pkDelayThread
  0x0000000100e61ad4 NrReplicateBatch
  0x00000001006bd4e4 PcConsumerThread
  0x0000000100ecf334 NrReplicateBatchStackTrackEntry
  0x0000000100011a70 StartThread
  Thread context:
  COMMAND: START STGRULE
  COMMMETHOD: SSL
  PORT: 10.xx.xx.xx:46918
  SESSION: 52668
  HOSTADDR: IBM
  PROCESS_NUMBER: 14
  PROCESS_DESC: Replication Storage Rule STGR_REPLICATE
  THREAD_TYPE: PROCESS
  JOB_ID: 697
  SESSION_TYPE: ADMIN
  ADMIN_NAME: ADMIN
  HOSTPORT: 46810
  
  The key part is
  
  0x090000000013966c __fd_select
  0x0000000100015398 pkDelayThread
  0x0000000100e61ad4 NrReplicateBatch
  
  The threads will remain in this state until the number of
  inflight dependency phases reduces below 100.
  IBM Storage Protect Versions Affected: All IBM Storage Protect
  Server 8.1.x versions on all supported platforms.
  
  Additional Keywords: TS014217640 replication storage rule
  stgrule hang
  

Local fix

• Restart the target server.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * All IBM Storage Protect server users.                        *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See error description.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  

Problem conclusion

• This problem is currently projected to be fixed in level
  8.1.27.100 and 8.2.0.
  Note that this is subject to change at the discretion of IBM.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels