Controlling vertical polarization (HiperDispatch)

7.1 LPAR mode

HiperDispatch automatically adjusts capacity values for vertical polarization. You can influence the decision making of the HiperDispatch function by setting sysfs attributes.

About this task

HiperDispatch uses vertical high polarization to concentrate the entitlement of the LPAR to as few CPUs as possible. Workloads that benefit from this approach are especially workloads that run CPU-heavy processes.

HiperDispatch manages vertically polarized CPUs by adjusting their capacities as presented to the Linux kernel. Vertical high and medium CPUs receive higher capacity allocations, while vertical low CPUs are assigned reduced capacities, especially if HiperDispatch detects excessive steal time on the vertical lows. When no steal time is observed, all CPUs are granted full capacity. The sysfs attributes described here allow you to influence HiperDispatch's decision-making process:

hd_steal_threshold. Controls at which threshold of overall average steal time on vertical lows the capacities are adjusted.
hd_delay_factor. Defines how much time must pass between two HiperDispatch decisions in multiples of 250ms.

HiperDispatch's decision is based on whether steal time is encountered on vertical low CPUs. With the two sysfs options it is possible to control at which threshold the capacities are adjusted and how much time must pass between two HiperDispatch decisions.

To help decide whether to change the capacity values, three debug counters are available. Assuming that debugfs is mounted at /sys/kernel/debug, these counters are available at /sys/kernel/debug/s390/hiperdispatch/.

adjustment_count. Number of state transitions since boot. A high value indicates that the system might be going back and forth between the two states. You might consider increasing the steal time threshold or the delay factor.
conservative_time_ms. Milliseconds since the boot where the capacity of the vertical-low CPUs was restricted.
greedy_time_ms. Milliseconds since boot where the capacity of the vertical-low CPUs was not restricted.

HiperDispatch enablement and disablement

HiperDispatch is enabled by default:

As of Red Hat® Enterprise Linux® 10 and SUSE Linux Enterprise Server 16, Linux boots in vertical mode with HiperDispatch active.
Red Hat Enterprise Linux 9.6 and SUSE Linux Enterprise Server 15 SP7 boot horizontally by default, but activate HiperDispatch automatically when you switch to vertical polarization.

You can turn off HiperDispatch, but doing so reintroduces the traditional risks of vertical mode and is discouraged. Use the sysctl s390.hiperdispatch attribute to toggle HiperDispatch on or off.

Older Linux distributions have no HiperDispatch support. Switching to vertical CPU polarization usually results in a system with different types of vertical CPUs. Running a system with different types of vertical CPUs can result in significant performance degradation. If possible, use only one type of vertical CPUs. Set all other CPUs offline and deconfigure them.

Procedure

Set the percentage of steal time.
Go to /sys/devices/system/cpu/hd_steal_threshold.
The threshold value defines the percentage of steal time that the LPAR must experience on its non-idle vertical-low and vertical-medium CPUs before restricting the capacity on vertical-low CPUs. The default value is 30%. A lower value makes the system react faster to steal time occurrences. 0% implies permanent restriction, whereas 100% implies that capacity is never restricted on vertical-low CPUs.
A value of 0 or 100 sets the capacities permanently until you set another value. A value of 100 is the same as deactivating HiperDispatch.
If you see that HiperDispatch enters a state of back-and-forth, increasing the delay factor can slow this behavior down as HiperDispatch then considers a longer time frame. To determine whether the system is in a state of back-and-forth, check whether the adjustment_count debug attribute is growing quickly.
For example, to set the threshold to 40%, issue a command like:
```
# echo 40 > hd_steal_threshold
```
Set the delay factor.
Go to /sys/devices/system/cpu/hd_delay_factor.
The steal time is evaluated every 250 ms. To prevent a fast back-and-forth behavior between the two states of the vertical-low CPUs, a delay is put in place after each state transition. The default delay factor is 4, which translates to a minimal delay of 4*250 ms until the next state transition can take place. Reducing this value can make the system more responsive to changes in the workload. Increasing the value makes the system less susceptible to sudden changes in steal time.
For example, to set the delay factor to 5, issue a command like:
```
# echo 5 > hd_delay_factor
```