IT47725: CTGGA0076 FAILED TO SNAPSHOT STORAGE VOLUME MESSAGE ON VM BACKUP

APAR status

• Closed as program error.

Error description

• Protection for VMs with IBM Storage Protect Plus can stop with
  the following messages seen in the job log :
  
  ... DETAIL,..,CTGGA0576,Initializing backup of vm: <VMName>
                      (host: <ESXiHost>) data center:
                      <DataCenterName>
  DETAIL,..,CTGGA0671,Backing up VM (<VMName>) from remote proxy
                      (IP: <vADPIP>  Host name: <vADPHost>)
    INFO,..,CTGGA2110,Will do incremental backup for VM <VMName>
    INFO,..,CTGGA0496,Finished backup of VM (Incremental)
                      <VMName> with status complete.
  ...  ERROR,..,CTGGA0076,Unprotected VM: <VMName>. Last error:
                      [Failed to snapshot storage volume]
  
  In the corresponing virgo log, the following message is seen :
  
  INFO .. failed to take snapshot reason com.catalogic.ecx.
         serviceprovider.common.storage.exception.
         StorageSnapshotCreateException:
         Failed to create snapshot on the volume <VolumeName>
         Could not extract response: no suitable
         HttpMessageConverter found for response type
         [class com.fasterxml.jackson.databind.JsonNode] and
         content type [text/html]
  
  In the vSnap host /var/log/messages, the uWSGI process is seen
  to crash and is automatically respawned multiple times.
  
  For example :
  ... [<Timestamp>] ...
  .. !!! uWSGI process 3991604 got Segmentation Fault !!!
  .. *** backtrace of 3991604 ***
  .. /usr/sbin/uwsgi(uwsgi_backtrace+0x2e) [0x46a37e]
  .. /usr/sbin/uwsgi(uwsgi_segfault+0x27) [0x46a767]
  .. /lib64/libc.so.6(+0x4eb50) [0x7f77456d0b50]
  .. /opt/vsnap/venv/lib64/python3.6/site-packages/greenlet/
    _greenlet.cpython-36m-x86_64-linux-gnu.so(_ZN24ThreadState_
    DestroyNoGIL19DestroyQueueWithGILEPv+0x230) [0x7f7740a8c8e0]
  .. /usr/lib64/libpython3.6m.so.1.0(+0x6ff97) [0x7f7744b67f97]
  .. /usr/lib64/libpython3.6m.so.1.0(_PyEval_EvalFrameDefault
    +0x6d44) [0x7f7744c9b5f4]
  .. /usr/lib64/libpython3.6m.so.1.0(_PyFunction_FastCallDict
    +0x122) [0x7f7744bf2e82]
  .. /usr/lib64/libpython3.6m.so.1.0(_PyObject_FastCallDict+0x70e)
  
    [0x7f7744bf3c5e]
  .. /usr/lib64/libpython3.6m.so.1.0(_PyObject_CallMethodId+0x16e)
  
    [0x7f7744bf3f8e]
  .. /usr/lib64/libpython3.6m.so.1.0(+0x1f6f60) [0x7f7744ceef60]
  .. /usr/lib64/libpython3.6m.so.1.0(Py_FinalizeEx+0x17)
    [0x7f7744d04627]
  .. /usr/sbin/uwsgi(uwsgi_plugins_atexit+0x71) [0x468531]
  .. /lib64/libc.so.6(+0x5126c) [0x7f77456d326c]
  .. /lib64/libc.so.6(on_exit+0) [0x7f77456d33a0]
  .. /usr/sbin/uwsgi() [0x41c655]
  .. /usr/sbin/uwsgi(end_me+0x2b) [0x46701b]
  .. /usr/sbin/uwsgi(uwsgi_ignition+0x147) [0x46a967]
  .. /usr/sbin/uwsgi(uwsgi_worker_run+0x276) [0x46e106]
  .. /usr/sbin/uwsgi(uwsgi_run+0x44c) [0x46e67c]
  .. /usr/sbin/uwsgi() [0x418f84]
  .. /lib64/libc.so.6(__libc_start_main+0xe5) [0x7f77456bcd85]
  .. /usr/sbin/uwsgi(_start+0x2e) [0x418fbe]
  .. *** end of backtrace ***
  ...
  ... [<Timestamp>] GET ...
  .. worker 2 killed successfully (pid: 3991604)
  .. uWSGI worker 2 cheaped.
  .. Respawned uWSGI worker 2 (new pid: 4157206)
  .. mounting rest:app on /api
  .. /opt/vsnap/venv/lib64/python3.6/site-packages/pymongo/
    pyopenssl_context.py:26: CryptographyDeprecationWarning:
    Python 3.6 is no longer supported by the Python core team.
    Therefore, support for it is deprecated in cryptography.
    The next release of cryptography will remove support for
    Python 3.6. from cryptography.x509 import
    load_der_x509_certificate as _load_der_x509_certificate
  ... announcing my loyalty to the Emperor...
  
  This is due to a dependency/version problem with the
  python-greenlet package.
  
  |MDVREGR  10.1.7.2 5737SPLUS|
  
  IBM Storage Protect Plus Versions Affected:
  IBM Storage Protect Plus 10.1.8 and later
  
  Additional Keywords: SPP, SPPLUS, TS017959853, greenlet,
                      python 3.6
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Storage Protect Plus users 10.1.10 and later             *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See Error Description                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply the fixing level when available. This problem is fixed *
  * in IBM Storage Protect Plus level 10.1.17.1.                 *
  ****************************************************************
  

Problem conclusion

• In IBM Storage Protect Plus 10.1.17.1, code fixes were
  implemented to explicitly specify the python library version of
  affected transitive dependency module, ensuring the vsnap uWSGI
  service will not get segmentation fault as the fixed version of
  python library is fetched during vsnap upgrade.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SP PLUS

• Fixed component ID

  5737SPLUS

Applicable component levels