IT20371: ABSTRACT: IBM SPECTRUM PROTECT CLIENT MAY HANG DURING PROCESS MGMTCLASS

IBM Spectrum Protect Client 8.1.4 Downloads and READMEs

APAR status

• Closed as program error.

Error description

• IBM Spectrum Protect Backup/Archive Client can hang during
  process management class if the received data is corrupted
  during network transfer.
  .
  L2/Customer diagnostics
  .
  When the hang happens:
  -the hang process stack  stuck at ProcessMC:
  
  For example On AIX:
  
  procstack pid_of_hang_dsmc:
  
  
  ---------- tid# 15991521 (pthread ID:      1) ----------
  0x00000001002330a0  ProcessMC(unsigned char,char*,PolicySet*)()
  + 0x100
  0x000000010023166c  psqSendQuery(policyObject_t*,Sess_o*,const
  char*)() + 0x40c
  0x00000001001f11d0  CheckSession(Sess_o*,sessLoadPolicy_t)() +
  0x2f0
  0x0000000100015974  DscIncrBack() + 0xb14
  0x0000000100003364  DoFunc(DsmcGlobalCB_t*)() + 0x2a4
  0x0000000100001b4c  dscLoop(int,char**)() + 0x14c
  0x0000000100000b1c  dscmain(int,char**)() + 0x7c
  0x0000000100000828  main() + 0x28
  0x00000001000002b0  __start() + 0x70
  
  
  
  Note:
  Use pstack for Linux and Windows process monitor for Windows
  platform
  .
  -a client SERVICE shows we stuck at:
  
  03/16/17   06:20:03.236 [22544486] [1] : session.cpp
  (1793): Recv Verb:
  03/16/17   06:20:03.237 [22544486] [1] : session.cpp
  (1794):   Length:  3992 Code: 000000A1 Type:    <-
  PSQryResp
  03/16/17   06:20:03.237 [22544486] [1] : cups.cpp            (
  368): cuGetPSQryResp: Got a PSQryResp of length: 3984
  
  -A SERVICE and COMMFULL will show details of the corrupted
  packet:
  
  An example shows  we received data overlaid with  zeros:
  
  04/26/17   19:02:02.536 [19267962] [515] : commtcp.cpp
  (1985): TcpRead: data:
  ...
  04010014 100e4241 00000000 00000000 00000000 00000000 00000000
  00000000
  00000000 00000000 00000000 00000000 00000000 00000000 00000000
  00000000
  00000000 00000000 00000000 00000000 00000000 00000000 00000000
  00000000
  00000000 00000000 00000000 00000000 00000000 00000000 00000000
  00000000
  00000000 00000000 00000000 00000000 00000000 00000000 00000000
  00000000
  00000000 00000000 00000000 00000000 000000
  ...
  
  -a topas or similar tool shows dsmc process was active consuming
  CPU and memory:
  
  For example:
  
  Name           PID          CPU%         PgSp Owner
  dsmc       23265904      14.8            5.02M root
  oracle     28311642        1.6             17.2M oracle
  ...
  
  
  Note: the APAR is opened to address the client hang, , Users
  experience the issue should engage Network support to resolve
  the packet corruption during network transfer issue.
  In above example, the issue was caused by down level AIX network
  adapter Firmware ( missing mandatory 10240180)  and
  mis-configured etherchannel.
  
  
  Platform /Version Affected:
  IBM Spectrum Protect Backup/Archive Client all supported
  versions on all platform
  
  Additional KeyWords:
  
  ProcessMC PSQryResp cuGetPSQryResp
  

Local fix

• There is no local fix to prevent the hang, when the hang
  happens, force terminate the hanging process can release the
  holding CPU
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * Backup-archive client versions 6.3, 6.4, 7.1 and 8.1 running *
  * on all platforms.                                            *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See ERROR DESCRIPTION                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * This issue is projected to be fixed in the Backup-Archive    *
  * Client version 8.1.4 on all platforms.                       *
  * Note: This is subject to change at the discretion of IBM.    *
  ****************************************************************
  

Problem conclusion

• After the fix b/a client should not hang.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Modules/Macros

• dsmc
  

Fix information

• Fixed component name

  TSM CLIENT

• Fixed component ID

  5698ISMCL

Applicable component levels