IT31675: ANR0546W RETRIEVE OR RESTORE FAILS FROM PENDING CONTAINER

APAR status

• Closed as program error.

Error description

• The IBM Spectrum Protect Client error log may show a message at
  the time of the restore or retrieve failure like:
  
  ANS1301E (RC1)  This operation cannot continue due to an error
   on the IBM Spectrum Protect server.
  
  The IBM Spectrum Protect Server actlog will show messages at
  the time of the failure like:
  
   ANR9999D_1288456981 SmRetrieveBitfile(smnqr.c:2560)
    Thread<nnn>:
    Unknown result code (4812) from bfRtrv. (SESSION: xyz)
  .........
     Locks held by Tsn=0:nnnnnnnnn :(SESSION: xyz)
       Type=20001(SD Container), NameSpace=nn, SummMode=sLock,
       Mode=sLock, Key='1234567891123456.dcf' File=sdcntr.c
       Line=4668(SESSION: xyz)
       Type=20002(SD Pool), NameSpace=nn, SummMode=ixLock,
       Mode=isLock, Key='' File=sdcntr.c
       Line=4668(SESSION: xyz)
   ANR0546W Retrieve or restore failed for session xyz for node
    NODENAME (PLATFORM) - internal server error detected.
    (SESSION: xyz)
  
  .... where '1234567891123456.dcf' is an exemplary name of a
  storage pool container.
  The QUERY CONTAINER output will show that the involved
  container is in the "Pending" state.
  
  Creating and running a macro with SHOW INVO queries for the
  affected chunks will show that these chunks belong to objects
  for the node running the restore or retrieve operation:
  
  OS prompt>
  db2 -x "select distinct 'SHOW INVO ' || sdro.objid from
  sd_recon_order sdro where exists (select 1 from sd_all_chunks
  sdac where sdro.chunkid=sdac.chunkid and exists (select 1 from
  sd_containers sdcn where sdcn.cntrid=sdac.cntrid and
  sdcn.state=3))" > show_invo.macro
  
  dsmadmc prompt>
  macro show_invo.macro > showinvo.out
  
  =======================
  Affected versions:
  IBM Spectrum Protect Server 8.1
  on all supported Windows, AIX and Linux platform versions.
  
  =======================
  Initial Impact: Medium
  
  =======================
  Additional Keywords:
  TS003181563 tsm stgpool
  

Local fix

• 1. Set defrag triggers to -1 to prevent new pending volumes
  from being created using the following commands run at a
  dsmadmc prompt:
   setopt defragFsTrigger -1
   setopt defragCntrTrigger -1
  
  2. Re-start the Spectrum Protect Server.
  
  3. Save the list of all containers in the "Pending" state. Then
  reset the state of the pending containers as the Server
  instance owner at an AIX/Linux prompt or on Windows in a
  db2cmd:
   db2 connect to tsmdb1
   db2 set schema tsmdb1
   db2 "update sd_containers sdc set state=0 where sdc.state=3
    and exists (select 1 from sd_all_chunks sdac where
    sdac.cntrid=sdc.cntrid)"
  
  4. Run the restore or retrieve operation.
  
  5. Reset defrag triggers using the following commands run
  at a dsmadmc prompt:
   setopt defragFsTrigger 95
   setopt defragCntrTrigger 90
  
  6. Run at a dsmadmc prompt:
   AUDIT CONTAINER [CONTAINER NAME] WAIT=YES
  .... for each of the containers having been in the "Pending"
  state.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * All IBM Spectrum Protect server users.                       *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See error description.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be fixed in level 8.1.8. Note that this is      *
  * subject to change at the discretion of IBM.                  *
  ****************************************************************
  

Problem conclusion

• This problem was fixed.
  Affected platforms for reported release:  AIX, Linux, and
  Windows.
  Platforms fixed:  AIX, Linux, and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels