Working with hugetlbfs huge-page support

Ubuntu 22.04 LTS LPAR mode z/VM guest KVM guest

Typical tasks for working with hugetlbfs huge-page support include reading the current number of huge pages, changing the number of huge pages, and display information about available huge pages.

About this task

The huge-page memory can be used through mmap() or SysV shared memory system calls. More detailed information, including implementation examples, can be found in the hugetlbpage.txt file, available in the linux-doc package.

Your database product might support huge-page memory. See your database documentation to find out if and how it can be configured to use huge-page memory.

Depending on your version of Java™, you might require specific options to make a Java program use the huge-page feature. For IBM® SDK, Java Technology Edition 7 and 8, specify the -Xlp option. If you use the SysV shared memory interface, which includes java -Xlp, you must adjust the shared memory allocation limits to match the workload requirements. Use the following sysctl attributes:
/proc/sys/kernel/shmall
Defines the global maximum amount of shared memory for all processes, specified in number of 4 KB pages.
/proc/sys/kernel/shmmax
Defines the maximum amount of shared memory per process, specified in number of Bytes.
For example, the following commands would set both limits to 20 GB:
# echo 5242880 > /proc/sys/kernel/shmall
# echo 21474836480 > /proc/sys/kernel/shmmax

Procedure

  • Specify the hugepages= kernel parameter with the number of huge pages to be allocated at boot time. To read the current number of huge pages, issue:
    # cat /proc/sys/vm/nr_hugepages
  • To change the number of huge pages dynamically during runtime, write to procfs:
    # echo 12 > /proc/sys/vm/nr_hugepages
    If there is not enough contiguous memory available to fulfill the request, the maximum possible number of huge pages are reserved.
  • To obtain information about the number of huge pages currently available and the huge-page size, issue:
    # cat /proc/meminfo
    
    ...
    HugePages_Total: 20
    HugePages_Free: 14
    HugePages_Rsvd: 0
    HugePages_Surp: 0
    ...
    Hugepagesize: 1024 KB
    ...
  • To adjust characteristics of a huge-page pool, when more than one pool exists, use the sysfs attributes of the pool.
    These can be found under
    /sys/kernel/mm/hugepages/hugepages-<size>/nr_hugepages
    Where <size> is the page size in KB.

Example

To allocate 2 GB huge pages:
  1. Specify 2 GB huge pages and pre-allocate them to the page pool at boot time. Use the following kernel boot parameters:
    default_hugepagesz=2G hugepagesz=2G hugepages=4
  2. After booting, read /proc/meminfo to see information about the amount of huge pages currently available and the huge-page size:
    cat /proc/meminfo
    ...
    HugePages_Total: 4
    HugePages_Free: 4
    HugePages_Rsvd: 0
    HugePages_Surp: 0
    Hugepagesize: 2097152 kB
    ...