ZZ00525: THRASHING CPU ON CENTRAL SERVER 2.

IBM Cloud Orchestrator Fix Pack 4 (2.4.0.4) for 2.4

APAR status

• Closed as program error.

Error description

• On ICO Central Server 2, vmstat reports show 100% CPU
  utilization over time.  There are also CPU starvation related
  messages in BPM system out log:
  [1/10/16 6:05:42:732 UTC] 0000004b CoordinatorCo W   HMGR0152W:
  CPU Starvation detected. Current thread scheduling delay is 20
  seconds.
  TOP results showed approximately 100% system CPU.  As this was
  consistently the case, the problem is related to how Linux OS
  is trying to resolve memory requests in certain circumstances.
  It is trying to reclaim local memory because in old chip
  architecture local memory was faster. Not the case any more and
  this code can get what is effectively an infinite loop.
  Two changes to fix this issue.  One for a running system and
  the other to insure the fix is in place when the system is
  rebooted.
  1) Run the following to change the running system:
  sysctlvm.zone_reclaim_mode=0  (Note: turning zone_reclaim_mode
  off)
  2) Add the following at the end of /etc/sysctl.conf so the
  setting will be in place during the next reboot:
  vm.zone_reclaim_mode = 0  (Note:
  turning zone_reclaim_mode off)
  You may only seen this problem on CS2, but it would make sense
  to make this change on all servers.
  

Local fix

• 1) Run the following to change the running system:
  sysctlvm.zone_reclaim_mode=0  (Note: turning zone_reclaim_mode
  off)
  2) Add the following at the end of /etc/sysctl.conf so the
  setting will be in place during the next reboot:
  vm.zone_reclaim_mode = 0  (Note:
  turning zone_reclaim_mode off)
  

Problem summary

• On ICO Central Server 2, vmstat reports show 100% CPU
  
  utilization over time. There are also CPU starvation related
  
  messages in BPM system out log
  

Problem conclusion

• Based on the platforms / virtualization experts the system
  
  setting recommended for vm.zone_reclaim_mode should be 0
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SMRTCLOUD ORCHS

• Fixed component ID

  5725H2800

Applicable component levels

• R240 PSY

     UP