IT39091: NDMP RESTORE CAN FAIL WITH AN ANR1095E ERROR SAYING THE OPERATION CANNOT BE STARTED, IF A PREVIOUS RESTORE ATTEMPT FAILED

8.1.23.000-IBM-SPSRV-WindowsX64
8.1.23.000-IBM-SPSRV-Linuxx86_64
8.1.23.000-IBM-SPSRV-Linuxs390x
8.1.23.000-IBM-SPSRV-Linuxppc64le
8.1.23.000-IBM-SPSRV-AIX

APAR status

• Closed as returned (APAR cannot be resolved without additional info from IBM or customer.)

Error description

• After a failed server or client initiated NDMP restore of data
  from a container storage pool on IBM Spectrum Protect server,
  subsequent restore attempts for the same node and filespace
  can fail with error:
  
  ANR1095E Data mover operation for node NODE1 and
  
  file space /filespace1 cannot be
  
  started - an operation involving this node and file
  
  space is already in progress.
  
  
  This is occurring as the failed restore process is not being
  cleaned up correctly.
  Only restore from a container storage pool is affected.
  
  Despite 'query process' showing no running process(es),
  "show threads" will show a 'restore node' command still being
  processed:
  
  Thread 403, Parent 373: AfRtrvRemoteThread, Storage 0, AllocCnt
  101 HighWaterAmt 69072
  tid=140728128947968, ptid=140732091188992, det=1, zomb=0,
  join=0, result=0, sess=0, procToken=1, sessToken=18
  lwp=49625
  Stack traces are disabled by option.
   Awaiting cond sessP->sinkThreadIdle (0x0x27f0ab0), using mutex
  sessP->mutex (0x0x7ffe3c0fa3b0), at sdutil.c(2054)
  Thread context:
  
    COMMMETHOD: SSL
    COMMAND: RESTORE NODE
    PORT: localhost.localdomain[127.0.0.1]:41084
    SESSION: 18
    THREAD_TYPE: UNKNOWN
    SESSION_TYPE: ADMIN
    ADMIN_NAME: ADMIN1
  
  This AfRtrvRemoteThread is hanging.
  
  
  IBM Spectrum Protect versions affected:
  IBM Spectrum Protect Server V8.1 and later on all supported
  platforms
  
  
  
  
  Additional Keywords:
  TSM "Spectrum Protect" #TS006572252  restore node nas ndmp
  

Local fix

• Correct the reason for the original NDMP restore failure.
  Restart the Spectrum Protect server process/service, and then
  retry the failing NDMP restore.
  

Problem summary

Problem conclusion

Temporary fix

Comments

• This APAR is being returned for documentation.  If this problem
  is seen, collect the following documentation and contact IBM
  support with servermon as well as SPI SPID BFREMOTE SDRTRV
  BRNODE trace for failed restore.
  Affected platforms:  AIX, Linux, and Windows.
  

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

Applicable component levels