Setting the resource limits, number of Db2 replicas, and total number of partition for MPP

Before you install a Db2® Warehouse massively parallel processing (MPP) configuration on Red Hat® OpenShift®, you need to decide about your memory allocation and the optimal number of partitions to create.

About this task

Before installing, consider the following numbers:

  • The number of Db2 engine replica pods. This number controls the compute parallelism because each Db2 engine pod runs on a separate worker node.
  • The number of partitions or multiple logical node (MLN)s to allocate per each pod. Best practice is to evenly distribute the total number of MLNs over all the selected Db2 engine pods. When you choose the number of MLNs, you need to consider also the memory-to-vCPU (CPU core count) ratio.

For help with determining your memory allocation and the optimal number of partitions to create, you can use the Db2 Warehouse online configuration tool.

For the procedure, it is assumed that you have access to three worker nodes, each with system RAM of 128Gi and vCPU count of 48:
  • Leaving some headroom for OpenShift/Kubernetes and Linux® OS daemons, and so on, you can give each pod 96Gi memory resource limit.
  • For optimal performance, you need to maintain a memory-to-vCPU ratio of 8:1 to 32:1, ideally 16:1. If you choose a 8:1 ratio, for example, cpu resource limit must to be set to 96 / 8 = 12 vCPUs.
  • Typically, the total number of MLNs to allocate is a function of amount of data (current and future projections) to be processed by each partition. To get a more accurate sizing, you can use the Db2 Warehouse online configuration tool. In general, you should use 64GB per MLN for heavy workloads, whereas 32GB per MLN is used for a low or starter configuration. Considering this fact, you choose 2 MLNs per pod, that is, 96 / 2 = 48GB per MLN. Therefore, the total number of MLNs is 2 (MLNs per pod) x 3 (number of replicas) = 6.

Procedure

  1. Deploy the Db2 MPP configuration on Red Hat OpenShift by using the db2u-install script in advanced mode.
    For the advanced mode, you must supply a helm_options file that is used by the db2u-install script. In addition to the storage-specific options, you must specify the following helm options in the helm_options file: subType,images.db2u.replicas, and mln.total.

    For example, for the assumed setup, specify:

    subType="mpp"
images.db2u.replicas="3"
mln.total="6"
  2. Define the correct resource limits when invoking the db2u-install script.
    For example, for the assumed setup, you can use the following options to set the vCPU and memory resource limits:
    --cpu-size 12
--memory-size 96