IT27962: VMWARE CLONE RESTORE JOB CREATES THE VIRTUAL MACHINE BUT FAILS TO CLEANUP THE VOLUME ON THE VSNAP

APAR status

• Closed as program error.

Error description

• During an IBM Spectrum Protect Plus clone restore the
  virtual machine is restored, but there is a failure
    reported on the vSnap during the cleanup of the volume.
    Below are the errors that are reported:
  
      Dismount snapshot copies: Failed to delete filesystem
        (SPP_1012_1542034331013_00_spp_1007_2102_166ec8afee2__
        166ec8b3f50_50_DS3500_DISK_POOL_1) cloned from snapshot
        (spp_1007_2102_4_166fbff2e35).
        Error = (StorageVolumeDestroyException).
      Dismount snapshot copies: Failed to delete filesystem
        (SPP_1012_1542034331013_00_spp_1007_2102_166ec8afee2__
        166ec8b3f50_50_DS3500_DISK_POOL_1).
        Exception (...serviceprovider.common.storage.exception.
        StorageVolumeDestroyException) reports: (Failed to
        delete storage Volume message :"Failed to delete
        volume: Command failed: zfs destroy -rf vpool1/fs8"type
        :"VolumeDeleteError")
      No selected item is recoverable.
    This is due the vSnap trying to destroy the volume, but it
    is reported as being busy. The following error can be see
    in the vSnap logs:
  
      ERROR pid-9667 vsnap.linux.system  Ouput: ['umount:
        /vsnap/vpool5/fs23: target is busy.', '(In some cases
        useful info about processes that use', 'the device is
        found by lsof(8) or fuser(1))', "cannot unmount
        '/vsnap/vpool5/fs23': umount failed"]
      ERROR pid-9667 vsnap.api    Traceback (most recent call
  Customer/L2 Diagnostics (If Applicable)
    N/A
  
  Initial Impact:
    Low
  Additional Keywords:
    TS001592224 SPP ISPP Clone Restore timeout ZFS
  

Local fix

•   Find the command that failed, and try and run the command
    from the command line of the vSnap server as root.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Spectrum Protect Plus level 10.1.2                       *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See ERROR DESCRIPTION                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be fixed IBM Spectrum Protect Plus level 10.1.3 *
  * patch 1. and 10.1.4. Note that this is subject to change at  *
  * the discretion of IBM.                                       *
  ****************************************************************
  

Problem conclusion

• A change was made in Spectrum Protect Plus 10.1.3 in which
  cleanup on vSnap happens asynchronously during a background
  process instead of synchronously during job execution. This
  prevents timing issues which can cause cleanup failures on
  vSnap.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SP PLUS

• Fixed component ID

  5737SPLUS

Applicable component levels