IT33281: A SUBSET OF VMS MAY FAIL TO BACKUP DUE TO STALE FILE HANDLE IN PROXY

APAR status

• Closed as program error.

Error description

• During a large VMware backup job with multiple proxies involved,
  a subset of VMs may fail to backup. The error messages seen in
  the job log can vary. Here are some common errors:
  
  Unprotected vm: <vm-name>. Last error: [Unknown error (evddk-1)]
  
  Unprotected vm: <vm-name>. Last error: [close() failed for one
  or more disks. (ebackup-close-file)]
  
  Unprotected vm: <vm-name>. Last error: [Open of
  (<directory-name>: no such file or directory)
  (ebackup-open-file) (createDiskDescriptorFile() failed)
  (verifyDestDisk() failed)]
  
  Looking further into the VADP proxy logs, the common symptom is
  that open/write/close of a file fails with "stale file handle"
  error.
  
  <date time> [I] <VDDK> VixDiskLib: Detected DiskLib error
  7602185 (Stale file handle).
  
  The issue occurs when one proxy has already mounted the vSnap
  volume using NFS and started writing a VM backup to it. Later in
  the job, another proxy wants to access the same volume to backup
  a different VM in the same volume. SPP calls the vSnap API to
  update the NFS share to allow access for the IP of the second
  proxy. When the vSnap updates the share to add access for the
  second proxy, it can cause a brief disconnection for the first
  proxy which is already accessing the share. This causes the
  first proxy to experience a stale file handle.
  
  Versions affected: 10.1.*
  

Local fix

• N/A
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Spectrum Protect Plus levels 10.1.5 and 10.1.6.          *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See Error Description                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be fixed in IBM Spectrum Protect Plus level     *
  * 10.1.6.ifix2 and 10.1.7. Note that this is subject to change *
  * at the discretion of IBM.                                    *
  ****************************************************************
  

Problem conclusion

• The issue occurs when one proxy has already mounted the vSnap
  volume using NFS and started writing a VM backup to it. Later in
  the job, when another proxy wants to access the same volume to
  backup a different VM in the same volume, the NFS share is
  updated to allow access for the IP of the second proxy. When the
  vSnap updates the share to add access for the second proxy, it
  can cause a brief disconnection for the first proxy which is
  already accessing the share. This causes the first proxy to
  experience a stale file handle.
  
  The problem has been resolved by fixing the share create/update
  process on vSnap to ensure existing clients are not affected
  when adding access for additional clients to an NFS share.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SP PLUS

• Fixed component ID

  5737SPLUS

Applicable component levels