IC53325: TSM SERVER HANG AFTER UPDATE OR DELETE PATH COMMAND IS ISSUED

APAR status

• Closed as program error.

Error description

• Under certain conditions, the TSM Server may get into a hang
  state after a delete or update path command is issued. When this
  problem happens, it is possible to connect to TSM, but any
  subsequent command that requires a library/drive resource, such
  as QUERY MOUNT, QUERY DRIVE or QUERY PATH will hang.
  Depending on how active the TSM Server is at the time of the
  hang, other operations and/or queries may hang as well giving
  the appearance of a complete hang.
  
  .
  Platforms affected: All 5.3 and 5.4 TSM Servers
  .
  Customer/L2 Diagnostics
  Using the SHOW THREADS command, or any other utility that
  shows the callstack of all active TSM Server threads
  (i.e. pstack/procstack), the following thread will be found for
  the hung DELETE PATH command:
  
  .
  Thread 94: SmHttpCommandThread
   tid=24159, ktid=105887, ptid=43, det=0, zomb=0, join=1,
  result=0, sess=0
    Holding mutex NAV->mutex (0x56b645d4), acquired at 101118e0
  (naDeletePath)
    Awaiting cond latchP->sFree (0x56895740), using mutex
  PVRV->mutex (0x56b67bb4), in AcquireLatchSpecific (0x1002cef4)
    Stack trace:
      0xd0064104 _cond_wait_global
      0xd0064cb8 _cond_wait
      0xd0065884 pthread_cond_wait
      0x1000d970 pkWaitCondition
      0x1002cef8 AcquireLatchSpecific
      0x102c7d6c pvrGetLibraryDevClasses
      0x1010f70c DeleteClassDir
      0x10111990 naDeletePath
      0x107b904c AdmDeletePath
      0x1020c508 AdmCommandLocal
      0x1020d674 admCommand
      0x10563098 HtRunCommands
      0x1056391c htPostForm
      0x1056088c SmHttpCommandThread
      0x1000ea00 StartThread
      0xd004c7e8 _pthread_body
  .
  The update path command will differ for all calls below
  DeleteClassDir.
  .
  NOTE: The key functions will be DeleteClassDir and
  pvrGetLibraryDevClasses.
  .
  Initial Impact: Medium
  .
  Additional Keywords: upd del lib share sharing mount
                       AcquireLatchSpecific hung stop wait hang
                       accesslatch naTestPath nalockuniverse NAV
                       mutex
  

Local fix

• 1) Do not issue the update/delete path command when there
  is library activity taking place
  2) Recycle the TSM Server
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All TSM servers levels 5.3 and above.        *
  ****************************************************************
  * PROBLEM DESCRIPTION: See ERROR DESCRIPTION.  This issue can  *
  *                      also apply to DEFINE PATH.              *
  ****************************************************************
  * RECOMMENDATION: Apply fixing level when available. This      *
  *                 problem is currently projected to be fixed   *
  *                 in level 5.3.7 and 5.4.2.  Note that this is *
  *                                                              *
  *                 subject to change at the discretion of IBM.  *
  ****************************************************************
  See PROBLEM DESCRIPTION.
  

Problem conclusion

• The described problem has been resolved.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R53A PSY

     UP

• R53H PSY

     UP

• R53L PSY

     UP

• R53S PSY

     UP

• R53W PSY

     UP

• R54A PSY

     UP

• R54H PSY

     UP

• R54L PSY

     UP

• R54S PSY

     UP

• R54W PSY

     UP