Logical partitioning (LPAR) hardware allocations for NUMA-based Power servers

Memory and CPU provisioning for LPAR

If an IBM® Power® server is running with more than one LPAR, then you need to validate the way memory and CPU are provisioned to each LPAR. It is a good idea to ensure that the processes on the operating system are dispatched to a CPU on the same NUMA node to which the memory is assigned. This takes advantage of memory locality, and ensures that workloads are equally distributed across NUMA nodes. The goal is to minimize the number of NUMA nodes that an LPAR is assigned to. Determine the amount of memory available for LPARs in a NUMA node by allocating around 5-10% of each node for the server hypervisor. If the required memory for an LPAR and the amount that is required by the hypervisor is greater than what is available in a single NUMA node, then the LPAR needs to allocate memory from multiple NUMA nodes. Allocating from multiple NUMA nodes, instead of a single NUMA node, allows for the memory to be successfully assigned to the LPAR. However, there can be an associated performance cost because non-local memory accesses will have higher memory latencies. After the LPAR boots up, you can check whether all the memory is assigned from one partition with the numactl -H.

The Dynamic Platform Optimizer (DPO), automatically optimizes processor and memory affinity. It is run from the HMC command line. The dynamic nature of the DPO allows it to be run while the partitions are active. For more information, see Dynamic Platform Optimizer in IBM POWER8® Documentation.

For a new installation, while you are configuring the system to run for the first time, run DPO while the partitions are shut down. This process ensures that allocation can be done quickly without migrating active pointers and data. For more information, see Running Dynamic Platform Optimizer (DPO) to optimize an LPAR.

Sample Scenario for Power server tuning

A sample scenario can be cited where you configure 4 LPARs on an IBM Power server with 512 GB of memory and 20 processors (across two processor modules or NUMA nodes). The partition profiles for each LPAR must be set to Dedicated processor mode, in this case, with five processors for each LPAR. For memory, the partition profiles for each LPAR must be set with 115 GB desired and 120 GB max memory. This memory allocation reserves about 10% of the memory for the server hypervisor use. Configuring the LPARs, by using this process, allows the DPO to assign each LPAR to a dedicated NUMA node, thus providing the best performance for each LPAR. While running the DPO, depending on hardware and hypervisor restrictions, the amount of memory can be refined to reduce the amount of wasted memory.

If adapters must also be virtualized, a VIO server may need to be configured. Two VIO servers help to ensure high availability. In such a scenario, the memory allocated to each LPAR needs to be reduced to allow the VIO servers to run as well.