IV10851: HUB MEMORY LEAK DUE TO RTEMS RECONNECTS

IBM Tivoli Monitoring 6.2.2 Fix Pack 9 (6.2.2-TIV-ITM-FP0009)

APAR status

• Closed as program error.

Error description

• Memory usage increases at the HUB monitoring server each time
  a remote monitoring server reconnects to the HUB. Usually the
  memory increase is gradual over time and does not pose
  significant problem. However, if the reconnections occur too
  often then memory can become exhausted at the HUB and a crash
  can occur.
  
  RECREATE INSTRUCTIONS:
  - Start the HUB and many RTEMS
  - Cause the RTEMS to reconnect to the HUB repeatedly
  - Once you get enough reconnects and enough updates being sent
    to each RTEMS the crash may occur.
  
  This problem is difficult to recreate.
  

Local fix

• Restart the HUB monitoring server.
  

Problem summary

• Hub monitoring server may leak memory due to remote monitoring
  server reconnects.
  
  
  Certain reconnects from a remote monitoring server to the hub
  monitoring server can result in an abandoned data queue at the
  hub monitoring server. The suspect reconnects occur without a
  physical network issue. These abandoned queues will continue to
  collect data.  They are periodically cleaned, but a large number
  of these queues holding a large amount of information can result
  in memory deficiencies and a possible crash at the hub
  monitoring server.
  

Problem conclusion

• Code was updated to prevent data from being added to queues that
  had been abandoned.
  
  The fix for this APAR is contained in the following maintenance
  packages:
    | fix pack | 6.2.2-TIV-ITM-FP0009
    | fix pack | 6.2.3-TIV-ITM-FP0002
  

Temporary fix

• Recycling the hub monitoring server will release the abandoned
  queues and the storage owned by them. However, this condition is
  triggered by a remote monitoring server performing multiple
  reconnects to the hub monitoring server. Identifying and
  eliminating the cause of the reconnects by the remote monitoring
  server will prevent this problem.
  

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  OA39813

Fix information

• Fixed component name

  TEMS

• Fixed component ID

  5724C04MS

Applicable component levels

• R622 PSY

     UP