IT37106: ANR0551E LOCK CONFLICT REPORTED AFTER OFFSITE RECLAMATION LEAVESORPHANED TRANSACTION

8.1.12.200-IBM-SPSRV-WindowsX64
8.1.12.200-IBM-SPSRV-Linuxx86_64
8.1.12.200-IBM-SPSRV-Linuxs390x
8.1.12.200-IBM-SPSRV-Linuxppc64le
8.1.12.200-IBM-SPSRV-AIX
IBM Spectrum Protect Server V8.1 Fix Pack 13 (V8.1.13) Downloads
IBM Spectrum Protect Server V8.1.12.X interim fix downloads

APAR status

• Closed as program error.

Error description

• Offsite reclamation process might result in orphaned
  transaction. An orphan transaction is a one that does not end
  after offsite reclamation completes. This transaction holds
  locks on objects which would then affect other
  processes/operations trying to access those objects.
  
  
  L2/Customer diagnostics:
  
  
  The symptom is seen when a lock conflict is encountered and
  lots of ANR0538I messages get reported:
  
  
  <date> <time>      ANR0538I A resource waiter has been aborted.
  
  <date> <time>      ANR0551E The client operation failed for
  session 13354
                            for node node-name on the (Linux
  x86-64) operating
                            system because of a lock conflict.
  (SESSION: 13354)
  
  
  SHOW TXNT and SHOW THREADS can be used to identify the orphaned
  transaction. It is the one listed by SHOW TXNT where the
  corresponding thread no longer exists.
  
  
  Here is an example:
  
  
  SHOW TXNT
  slot -> 232:
  Tsn=0:4104936, Resurrected=False, InFlight=True,
  Distributed=False, Persistent=True, Addr 0x7f181dd9eb90
  Start ThreadId=22716, Timestamp=04/29/2021 12:28:59 PM,
  Creator=sctierbatch.c(2809)
  Last known in use by ThreadId=22716
  Participants=1, summaryVote=ReadOnly
  EndInFlight False, endThreadId 0, tmidx 0, processBatchCount 0,
  mustAbort False.
    Participant DB: voteReceived=False, ackReceived=False
      DB: Txn 0x7f181c1f6c50, ReadOnly(YES), connP=0x7f181c1f69a0,
  applHandle=36382, openTbls=0:
  Locks held by Tsn=0:4104936 :
    Type=20009(SD Object ID), Number held: 1144
  
  SHOW LOCK
  slot -> 21313:
  LockDesc: Type=20009(SD Object ID), NameSpace=0, SummMode=xLock,
  Key='23394643'
  Holder: (sctierbatch.c:3796 Thread 22716) Tsn=0:4104936,
  Mode=xLock
  Waiter: (sddelete.c:2937 Thread 240) Tsn=0:145401243, Mode=xLock
  
  
  
  If thread 22716 is not in SHOW THREADS, the transaction is
  orphaned.  The Timestamp of that transaction can be used to
  correlate to the activity log to get more context.
  

Local fix

• Increase the RESOURCETIMEOUT server option so no lock conflict
  is detected and recycle the Spectrum Protect server instance to
  free the locked resources.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * All IBM Spectrum Protect server users.                       *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See error description.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be fixed in levels 8.1.12.200, and 8.1.13. Note *
  * that this is subject to change at the discretion of IBM.     *
  ****************************************************************
  

Problem conclusion

• This problem was fixed.
  Affected platforms for reported release:  AIX, Linux, and
  Windows.
  Platforms fixed:  AIX, Linux, and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R81A PSY

     UP