IC90451: HIGH CPU LOAD SEEN ON THE TVOLI STORAGE MANAGER SERVER WITH PARALLEL SYSTEM STATE BACKUPS

APAR status

• Closed as program error.

Error description

• The Tivoli Storage Manager server might see high CPU utilization
  when running many parallel client system state backups.
  
  Under the APAR condition, server instrumentation trace will show
  a high percentage of time spent in DB2 Fetch Exec, in the
  following example it is about 50%:
  
   Thread 532938 psSessionThread parent=49
  19:17:05.225-->19:37:52.969
   Operation       Count  Tottime  Avgtime  Mintime  Maxtime
  InstTput  Total KB
   ---------------------------------------------------------------
  -------------
   Network Recv      707  325.931    0.461    0.000   22.756
  1.1       347
   Network Send      446    0.045    0.000    0.000    0.000
  51471.9      2357
   DB2 Fetch Prep      1    0.018    0.018    0.018    0.018
   DB2 MFtch Prep   9234   39.492    0.004    0.000    0.063
   DB2 Updat Prep      1    0.037    0.038    0.037    0.037
   DB2 Fetch Exec  52628  603.845    0.011    0.000    7.276
   DB2 MFtch Exec   9368  140.705    0.015    0.000    8.129
   DB2 Inser Exec  42496   25.233    0.001    0.000    0.190
   DB2 Updat Exec  42497   28.085    0.001    0.000    0.206
   DB2 Fetch       52627    1.258    0.0000    0.000    0.000
   DB2 MFetch      14514    0.272    0.0000    0.000    0.003
   DB2 Commit        167    1.134    0.007    0.000    0.015
   DB2 Reg Exec     4617   71.907    0.016    0.000    0.254
   DB2 Reg Fetch    9230    0.126    0.000    0.000    0.000
   Unknown                  9.647
   ---------------------------------------------------------------
  -------------
   Total                 1247.743
  2.2      2704
  
   To match the APARs condition, in addition looking for
  tbCliSRFetch calls in the stack dumps, these will nearly all be
  under the imGetNextGroupMember() function:
   ..
   0x0000000100130598  tbCliSRFetch(??, ??, ??, ??, ??, ??, ??,
  ??) + 0x118
   0x000000010091b358  imGetNextGroupMember(??) + 0x6d8
   0x0000000100681860  DoBackQryGroups(??, ??, ??) + 0x8a0
   0x000000010066e384  SmNodeSession(??, ??) + 0x2824
   0x0000000100199768  HandleNodeSession(??, ??, ??) + 0x988
   0x000000010018e298  smExecuteSession(??, ??, ??, ??, ??, ??,
  ??, ??) + 0x1318
   0x0000000100058284  psSessionThread(??) + 0x1e4
   0x0000000100020adc  StartThread(??) + 0x9c
   0x0900000000660d90  _pthread_body(??) + 0xf0
  
   The stack dumps can be found in the show.txt files obtained
  using the servermonV6.pl script, which is documented in the
  following technote:
  http://www.ibm.com/support/docview.wss?uid=swg21432937
   The imGetNextGroupMember() is undesirable and leads to the CPU
  load symptom reported.
  
    Tivoli Storage Manager Versions Affected: all V6
    Customer/L2 Diagnostics (If Applicable) collect data as
  requested with swg21432937
    Initial Impact: High
    Additional Keywords: ZZ61 ZZ62 ZZ63 TSM
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED: All Tivoli Storage Manager server users.     *
  ****************************************************************
  * PROBLEM DESCRIPTION: See error description.                  *
  ****************************************************************
  * RECOMMENDATION: Apply fixing level. This problem is fixed    *
  *                 in server level 6.3.3. Also, when running    *
  *                 server level 6.3.3, upgrading to             *
  *                 Tivoli Storage Manager client version 6.4    *
  *                 might give additional benefit.               *
  ****************************************************************
  *
  

Problem conclusion

• This problem was fixed.
  Affected platforms:  AIX, HP-UX, Solaris, Linux and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R63A PSN

     UP

• R63H PSN

     UP

• R63L PSN

     UP

• R63S PSN

     UP

• R63W PSN

     UP