IBM Support

Changing the large page pool size

Question & Answer


Question

How to change the large page pool size?

Answer

This document assumes that you have a configured system and application with the large page support and that you need to change the current large page pool size, changing the size here means decreasing or increasing the large page pool size.
 
CHANGE NOTES
Decreasing the large page pool size
  • You need to be sure there are enough free (unused) large pages before shrinking the current configured large page pool.
  • The number of the free (unused) large pages is the maximum number of large pages (the maximum amount of memory) that could be deleted out from the large page pool. 
  • Shrinking the large page pool by removing a higher number of large pages than the free (unused) large pages will fail with an error.
Increasing the large page pool size
  • Make sure your server has enough physical memory when increasing the large page pool size.
  • Confirm the computational memory is within the expected range for normal operation.
  • Subtract the large page pool size (the pinned size ) from the total computational memory to get the amount of memory consumption by the system.
To change the current large page pool size, you need to run the vmo command with a new value for lgpg_regions tunable, this change is dynamic, the amount of physical memory that you specify takes effect immediately and does not require a system reboot.
There are multiple methods to check the current pinned large page pool size, you can use one of the following:
COMMAND NOTES
# vmstat -P 16M Check siz, then multiply by 16MB to get a size in MB
kdb> vmstat|grep 16M Check 16M number of frames, size in GB
# vmstat -l Check flp size, then multiply by 16MB to get a size in MB
 INCREASING SIZE
  • Assuming a server has a large page pool with a size of 1.0 GB.
  • Increasing the current working large page pool size to 2.0 GB. 
  • Change to 2.0 GB by using vmo command:
    # vmo -p -o lgpg_regions=128 -o lgpg_size=16777216
  • Note that the change done on lgpg_regions only.
  • Tunable lgpg_size remains the same as 16777216.
  • Check change using the KDB (vmstat) command.
    # echo "vmstat" | kdb | grep 16M
      16M number of frames: 00000080   2.0GB
  • The pool size updated to 2.0 GB and is ready for use.
MARKING BINARY
  • Using sar command, run ldedit command.
  • Ldedit command will mark the sar header to use large pages.
    # ldedit -blpdata /usr/sbin/sar
      ldedit:  File /usr/sbin/sar updated.
  • Start a test with sar command, let it in the background:
    # sar 1 600 > /tmp/sar.test&
      [1] 7536648
  • Check the large pages used by vmstat -P
    # vmstat -P 16M
      pgsz            memory                           page  
      ----- -------------------------- ------------------------------------             
               siz       avm      fre    re    pi    po    fr     sr    cy    
      16M      128        96       32     0     0     0     0      0     0
  • The output shows a utilization of 96 of large pages.
DECREASING SIZE
  • Now try to decrease the assigned large page pool size, try to decrease to 50.
  • The output shows an error that the system was unable to satisfy the request.
  • This is because we tried to shrink to a size lower than the current used large pages.
    # vmo -p -o lgpg_regions=50 -o lgpg_size=16777216
   
      Setting lgpg_size to 16777216 in nextboot file  
      Setting lgpg_regions to 50 in nextboot file  
   
      Warning: although the change concerns a dynamic tunable, 
      bosboot should be run to optimally restore these settings at reboot
   
      vmo: 1485-124 Error setting lgpg_size lgpg_regions  
      The system was unable to completely satisfy the request
  • Check what the vmo command did in the below snippet.
    # vmstat -P 16M  
      pgsz            memory                           page  
      ----- --------------------------    ---------------------------------
                 siz      avm      fre    re    pi    po    fr     sr    cy    
    16M           96       96        0     0     0     0     0      0     0
  • The vmo command decreased the large page pool size from 128 to 96.
  • The program is still using 96 large pages.
Although decreasing the large page pool size to a lower size than the current large pages usage of 96 failed with an error, the vmo command decreased already the large page pool size to the large pages current utilization, which is in this example 96, check the figure below.
image 7678
NOTES!
  • If the needed change in the large page pool size is an increase, use the vmo command to change to the size needed.
  • Just make sure you have enough total memory assigned to cover this increase. 
  • You will need to change the lgpg_regions tunable value, lgpg_size remains the same as 16777216.
  • No impact in increasing large page pool size with the current working application's large pages.
  • The application will be able to use pages from the whole large page pool when needed, including the new pages.
  • With some applications, you might need to restart the application to pick up the additional large pages.
  • To decrease the large page pool size, make sure there are free (unused) large pages enough needed to shrink the large page pool.
  • The test in this document was just to explain the large page configuration needed from both the application and operating system sides.
  • The large page support is used with production applications that are certified to use large pages.
  • The sar command works better with non-large pages.

Cheers, Mahmoud M. Elshafey

[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG10","label":"AIX"},"ARM Category":[{"code":"a8m0z0000001fMuAAI","label":"AIX General Support"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)","Line of Business":{"code":"LOB08","label":"Cognitive Systems"}}]

Document Information

Modified date:
11 January 2021

UID

ibm16253913

Page Feedback