IT40339: INCREASED MEMORY AND DISK USAGE CAUSING BACKUPS TO FAIL BECAUSE OF UNRESPONSIVE RABBITMQ

APAR status

• Closed as program error.

Error description

• IBM Spectrum Protect Plus memory and disk usage can increase,
  causing Jobs to fail or restart frequently.
  
  
  This happens when the RabbitMQ component in IBM Spectrum
  Protect Plus is being overloaded and becomes unresponsive.
  
  
  In this specific case, the issue with RabbitMQ is caused by too
  many log files being generated in the background.
  
  
  Review virgo logs at /opt/virgo/servicability/logs and look for
  repeating entries of the quartz scheduler.
  
  
  [date] INFO  eduler_QuartzSchedulerThread
  com.syncsort.dp.xsb.scheduleprovider.quartz.CronWeeklyTrigger
    getFireTimeAfter res: <next start>
  [date] INFO  eduler_QuartzSchedulerThread
  com.syncsort.dp.xsb.scheduleprovider.quartz.CronWeeklyTrigger
    getNextFireTime from last fire time: <last start>
  [date] INFO  eduler_QuartzSchedulerThread
  com.syncsort.dp.xsb.scheduleprovider.quartz.CronWeeklyTrigger
    getFireTimeAfter: <next start>
  [date] INFO  eduler_QuartzSchedulerThread
  com.syncsort.dp.xsb.scheduleprovider.quartz.CronWeeklyTrigger
    getFireTimeAfter res: <next start>
  [date] INFO  eduler_QuartzSchedulerThread
  com.syncsort.dp.xsb.scheduleprovider.quartz.CronWeeklyTrigger
    getNextFireTime from last fire time: <last start>
  
  
  This defect is related to APAR  IT40184
  
  
  
  
  | MDVPARTL 10.1.8.0-TIV_5737SPLUS | IT40184
  
  
  IBM Spectrum Protect Plus Versions Affected:
  IBM Spectrum Protect Plus 10.1.8 and higher
  
  
  Additional Keywords: SPP, SPPlus, TS008034459
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Spectrum Protect Plus level 10.1.8 iFix 2, 10.1.9,       *
  * 10.1.10, and 10.1.11.                                        *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See ERROR description.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply the fixing level when available. This problem is       *
  * currently projected to be fixed in IBM Spectrum Protect Plus *
  * level 10.1.12. Note that this is subject to change at the    *
  * discretion of IBM.                                           *
  ****************************************************************
  

Problem conclusion

• Repeated log statements have been removed from scheduler code,
  which reduces the volume of log messages printed when a job is
  scheduled. This in turn reduces the volume of logging messages
  sent to both the Virgo log and sent through RabbitMQ during job
  scheduling.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Modules/Macros

• Platform
  

Fix information

• Fixed component name

  SP PLUS

• Fixed component ID

  5737SPLUS

Applicable component levels