CPU plugging

Edit online

To vary the amount of active CPUs in a system, the CPUs are enabled or disabled via sysfs from the cpuplugd daemon.

Note: This changes the amount of CPUs within the range of CPUs defined to the guest, either via CPU statements in the user directory or via CP DEFINE CPU command.
  • UPDATE="1"

    The update parameter determines the frequency of the evaluation of the rules in seconds, 1 is the smallest value. We could not identify any overhead related to a 1 second interval. A larger interval would produce a system that reacts more slowly. The recommendation is to use 1 second intervals for a fast system reaction time. If the objective is not to react immediately to each value change from a certain parameter, the evaluation of that parameter might cover values from several intervals.

  • CPU_MIN="1"
CPU_MAX="0"

    These parameters define the range within cpuplugd daemon varies the amount of CPUs. The lower limit is set to 1, the maximum value is set to '0' which means unlimited, so it is possible to use all of the CPUs that the guest is defined with. In case a middleware works better with two than with one CPU, CPU_MIN would be set to '2'.

  • user_0="(cpustat.user[0] - cpustat.user[1])"
nice_0="(cpustat.nice[0] - cpustat.nice[1])"
system_0="(cpustat.system[0] - cpustat.system[1])"
user_2="(cpustat.user[2] - cpustat.user[3])"
nice_2="(cpustat.nice[2] - cpustat.nice[3])"
system_2="(cpustat.system[2] - cpustat.system[3])"

    These rules calculate user, system, and nice CPU values from the last interval and the third previous interval. The cpustat.<parm> values are counting the CPU ticks for a certain type of load accumulated from the system start, that means they are continually increasing. At each update interval the parameters are determined and saved. They are referred to by an index, which starts at 0 for the most current value. The user CPU from the last interval is the difference between the most current user value cpustat.user[0] minus the value before cpustat.user[1].

    Note: These values are accumulated values from all CPUs and counted in the number of CPU ticks spent for that type of CPU usage! 
  • CP_Active0="(user_0 + nice_0 + system_0)/ (cpustat.total_ticks[0] - cpustat.total_ticks[1])"
CP_Active2="(user_2 + nice_2 + system_2)/ (cpustat.total_ticks[2] - cpustat.total_ticks[3])"

    The differences in CPU ticks for a certain type of load must be normalized with the total amount of CPU ticks in which they are gathered, because the length of the intervals always varies slightly.

    Note:
    • Even the UPDATE interval is specified with a fixed value in seconds, depending on the load level the real interval length might differ more or less. Therefore it is highly recommended to use the values from the cpustat.total_ticks array. The index has the same semantics as the values array, [0] is the most current value, [1] the one before, and so on.
    • cpustat.total_ticks values are accumulated CPU ticks from all CPUs since system start! If the system is 100% busy this means that the number of CPU ticks spent for user, system and nice is equal to cpustat.total_ticks.
    The actively used CPU value for this calculation is composed of the user, system, and nice CPU values.

  • CP_ActiveAVG="(CP_Active0+CP_Active2) / 2"

    We use the average of the current and the third previous interval to cover a certain near term interval.

  • idle_0="(cpustat.idle[0] - cpustat.idle[1])"
iowait_0="(cpustat.iowait[0] - cpustat.iowait[1])"
idle_2="(cpustat.idle[2] - cpustat.idle[3])"
iowait_2="(cpustat.iowait[2] - cpustat.iowait[3])"
CP_idle0="(idle_0 + iowait_0)/ (cpustat.total_ticks[0] - cpustat.total_ticks[1])"
CP_idle2="(idle_2 + iowait_2)/ (cpustat.total_ticks[2] - cpustat.total_ticks[3])"
CP_idleAVG="(CP_idle0 + CP_idle2) / 2"

    The considerations are the same as for the elements which contribute to CP_ActiveAVG as described in the previous paragraph. The states idle and iowait contribute to idle.

    Note: We did not include steal time in these formulas. Never count steal time as active CPU, because adding CPUs triggered by steal time will worsen the situation. In case of very high CPU overcommitment rates, it might make sense to include steal to idle and to remove a CPU if steal time becomes too high. This reduces the level of CPU overcommitment and allow for a low prioritized system to relieve the CPU pressure. For productive systems, we recommend that you ignore it, especially if it appears for a limited period only.
  • HOTPLUG="((1 - CP_ActiveAVG) * onumcpus) < 0.08"
HOTUNPLUG="(CP_idleAVG * onumcpus) > 1.15"
    • onumcpus is the current number of CPUs which are online.
    • Table 1 shows the interpretation of the variables CP_Active AVG and CP_idleAVG:
      Table 1. Interpretation of the variables CP_Active AVG and CP_idleAVG
      Variable Range Interpretation Comment
      CP_Active AVG 0-1

      0 means: no CPU is doing work
      1 means: all CPUs are actively used

      		 includes user, system and nice
      CP_idleAVG 0-1

      0 means: no idle time
      1 means: all CPUs are fully idling

      		 includes idle and iowait, steal time is not included

    (1-CP_ActiveAVG) represents the unused capacity of the system as value between 0 and 1. The multiplication with onumcpus creates a value with a unit in multiples of CPUs. Due to that the comparison with 0.08 refers to 8% of a single CPU independent to the size of the system.

    The rules above:
    • Add another CPU when only less than 8% of one (a single) CPU's capacity is available
    • Remove a CPU when more than 1.15 CPUs are in the state idle or iowait.
    This is the recommended CPU plugging setup for a fast reacting system. If a system that acts in a restrictive manner is required, a loadavg base rule as described in DB2 UDB tuning can be used.