IT35357: OFFLOAD STOPS WITH MESSAGE CTGGA0309 ABOUT INSUFFICIENT FREE SPACE IN /OPT/VSNAP-DATA DESPITE CORRECT SIZING

APAR status

• Closed as program error.

Error description

• The IBM Spectrum Protect Plus offload volume cache file size is
  incorrectly set to 6 GB instead of the expected 512 MB.
  Per cloud volume, there will be such a cache file :
  /opt/vsnap-data/offload<CloudVolumeID>/objectcache
  
  It can be
  checked as follows :
  [serveradmin@vsnap ~]$ sudo find
  /opt/vsnap-data -name objectcache -exec du -h {} \;
  6.2G
  /opt/vsnap-data/offload<CloudVolumeID_1>/objectcache
  6.2G
  /opt/vsnap-data/offload<CloudVolumeID_2>/objectcache
  6.2G
  /opt/vsnap-data/offload<CloudVolumeID_3>/objectcache
  ...
  
  This causes an unexpected usage in the offload cache location of
  the
  vSnap host despite the sizing being correctly set according to
  the BluePrints (https://www.ibm.com/support/pages/ibm-spectrum-p
  rotect-plus-blueprints) recommendations.
  
  When the available
  space is exhausted, the following messages will be seen
  in a failed offload job log :
  ERROR,<timestamp>,CTGGA0309, Copy
  failed for snapshot (ID: xxxx) from source [server:
  <SourcevSnapHostIP>, volume: <SourceVolume>, snapshot:
  <SourceSnapshot>] to target [server: <TargetvSnapHostIP>,
  volume: <TargetVolume>]. Error: Exception: Failed to create
  gateway device: Cloud temporary data location /opt/vsnap-data
  has yy GB free space which is less than the required
  25.00G
  
  IBM Spectrum Protect Versions Affected:
  IBM Spectrum
  Protect Plus 10.1.7
  
  | MDVREGR 10.1.6-5737SPLUS |
  
  Initial
  Impact: Medium
  
  Additional Keywords: SPP, SPPLUS, TS003990321,
  objectcache
  

Local fix

• 1. Set the following preference on each vSnap:
  vsnap system
  pref set --name cloudRestoreCacheSize --value 536870912
  2.
  Delete the existing objectcache files while no jobs are
  running.
  The cache will be recreated at the smaller size during
  the next job run:
  sudo find /opt/vsnap-data -name objectcache
  -delete
  3. Once the fixing IBM Spectrum Protect vSnap version
  is applied, remove the option setting :
  vsnap system pref clear
  --name cloudRestoreCacheSize
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Spectrum Protect Plus level 10.1.7.                      *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See Error Description.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply the fixing level when available. This problem is fixed *
  * in IBM Spectrum Protect Plus levels 10.1.7 iFix2 and 10.1.8. *
  * Note that this is subject to change at the discretion of     *
  * IBM.                                                         *
  ****************************************************************
  

Problem conclusion

• An incorrect preference value was used while constructing the
  object cache during copy to cloud, resulting in unnecessarily
  large cache files. The problem has been resolved by using the
  correct preference value. Upon upgrading to the fixed version,
  the existing object cache files will initially remain at the
  same size but they will gradually be deleted and recreated with
  the new smaller size as and when cloud copy jobs run.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SP PLUS

• Fixed component ID

  5737SPLUS

Applicable component levels