Many WebSphere Application Server based applications can benefit in terms of performance from using large page support provided by the Java virtual machine and the operating systems. Using large pages for the Java Heap can improve performance - especially when using large heap sizes.
WebSphere Portal Server is no exception here and the tuning guide (https://www-10.lotus.com/ldd/portalwiki.nsf/dx/IBM_WebSphere_Portal_V_8.5_Performance_Tuning_Guide) recommends this as well. In addition the tuning guide provides instructions on how the set-up and enable large page support on various operating systems.
For Linux it is required to configure huge pages on the OS level before these can be used for the Java Heap. When using large page support for the Java Heap on Linux environments you need to make sure that the whole heap fits into the available HugePages. While it is simple to calculate the number of huge pages (Max-Heap / Huge-Page-Size) there is another feature in Linux which needs to be considered here namely the anonymous huge pages (https://access.redhat.com/solutions/46111). If this feature is enabled some processes being started before the WebSphere Portal Server JVM (like for example the nodeagent) might reserve some huge pages before the WebSphere Portal Server JVM is started. Hence the required number of HugePages is not sufficient any more causing the JVM to use "normal" pages for the Heap. This usually causes heavy swapping of the system as the allocated HugePages are not used for the heap and the remaining "normal" pages are not sufficient to fit into real memory.
Therefore you need to allocate some more huge pages (depending on what's running) on the Linux if anonymous huge page support is enabled to ensure that - before you start WebSphere Portal Server JVM - sufficient HugePages are free (grep Huge /proc/meminfo --> HugePages_Free) . The value displayed must be greater than or equal to the number of HugePages needed to keep your configured maximum heap.
The process to use large page support on Linux should be:
- Calculate the number of HugePages needed for the heap based on the huge page size of your Linux installation
- Add 512 HugePages (which is for example 1GB for RedHat on Intel) to the calculated number
- Update vm.nr_hugepages=<nnnn> in /etc/sysctl.conf.
- If you are running WebSphere Portal as non-root update soft memlock and hard memlock values for the primary group of the user running WebSphere Portal Server in /etc/security/limits.conf to match the allocated huge pages (1024*1024*<GB-of-huge-pages>)
- Start-up everything up to before WebSphere Portal
- Verify the number of available huge pages on the system (grep Huge /proc/meminfo --> check HugePages_Free)
- If the value for HugePages_Free is >= the number of huge pages needed for the maximum Java heap proceed to step 9. Otherwise increase the number of allocated huge pages (by minimum HugePages_Rsvd or simply add another 512 if you are generous) to be allocated and go to step 3.
- Make sure that the generic JVM argument "-Xlp" is set for the WebSphere_Portal JVM
- Start WebSphere Portal server
- Inspect the verbose:gc log file and verify the pageSize attribute in the initialized section of the verboseGC output. This line should look like the following when using 2MB huge pages for the heap:
<attribute name="pageSize" value="0x200000" />
- If you verified that large pages are used you are done. Otherwise it's time for problem determination.