Dynamic pagepool limitations

  • The dynamic pagepool cannot be used on nodes with NSD devices. The dynamicPagepoolEnabled parameter must be set to no on nodes that are NSD servers, except on nodes with only LROC devices. For nodes that are currently not NSD servers, but that is configured as servers with mmchnsd or mmcrnsd commands, set dynamicPagepoolEnabled to no and restart the GPFS daemon.
  • Nodes with ECE devices also do not use the dynamic pagepool and revert to a static pagepool.
  • The dynamic pagepool with RDMA uses large RDMA memory registrations (one for each 256 KiB of pagepool memory). Some older versions of RDMA driver stacks showed problems with large RDMA memory registrations. It is recommended to run a current version of the RDMA stack. If a large number of memory registrations causes any problems, contact your RDMA vendor.
  • Omni-path RDMA is not supported if the dynamic pagepool is enabled on a node.
  • It is recommended to set the pagepool parameter to default on the nodes where the dynamic pagepool is enabled. Avoid a cluster-wide setting of pagepool=default because NSD nodes cannot use the dynamic pagepool and still need a properly sized static pagepool. If the pagepool parameter is set, it serves as an extra limit to the maximum size of the dynamic pagepool. The effective maximum-allowed size of the dynamic pagepool is a smaller value of pagepool and the calculated value from pagepoolMaxPhysMemPct.
  • In systems with a heavy file system activity, where applications also consume memory, memory can be insufficient for a node. Occasionally, the increased rate of memory demand might exceed the rate of reduction in the use of pagepool (as the pagepool size is shrunk), and memory in the system might be exhausted. If this happens, reduce the value of the pagepoolMaxPhysMemPct parameter.
  • The dynamic pagepool is not supported on systems with less than 6 GiB of memory. The dynamic pagepool ensures that at least 4 GiB of memory is available outside the pagepool. However, some of this memory is used by other parts of GPFS. When the dynamic pagepool is enabled, the minimum size is adjusted to at least 1 GiB.
  • Start of changeA check of the Linux kernel, which is used in older Linux distributions (RHEL7 and RHEL8), does not allow single memory allocations if they are larger than available memory on a node. Because Linux does not know that GPFS shrinks its pagepool. This limitation is not applicable for Linux distributions that are based on kernel 5.2 or higher.
    Workarounds to solve this issue are as follows:
    • Free swap space on a node is also considered as available memory, so swap space can be a workaround.
    • You can set the vm.overcommit_memory sysctl parameter value to 1 to disable the Linux kernel check.
    End of change

Other considerations

  • Memory that is used in the dynamic pagepool is shown as the used memory. This used memory might be misleading because the dynamic pagepool relinquishes memory whenever it is requested from the Linux kernel.
  • This consideration is for job scheduling software that queries available memory on the cluster nodes and schedule jobs depending on the available memory. If the dynamic pagepool is close to its maximum size, large jobs might not be scheduled on the node.
  • If the job scheduling software allows modifying the memory check, the software can be used to consider only the minimum size of the pagepool for scheduling decisions. Otherwise, it might be necessary to adjust the dynamic pagepool size to meet the scheduling needs.