Topic
6 replies Latest Post - ‏2012-09-26T20:51:19Z by mauricfo
PowerLinuxTeam
PowerLinuxTeam
8 Posts
ACCEPTED ANSWER

Pinned topic Using IBM Java and Power's 16MB large pages

‏2012-09-25T15:35:55Z |
Looking for the simple steps for enabling 16MB large pages with IBM's Java with Linux.
 
According to this article, Java Performance on POWER7 - this should do it.

1)    echo 100 > /proc/sys/vm/nr_hugepages
2)    Start java using -Xlp
 
Are these two steps enough?      Or do I need more?
Updated on 2012-09-26T20:51:19Z at 2012-09-26T20:51:19Z by mauricfo
  • rfolco
    rfolco
    6 Posts
    ACCEPTED ANSWER

    Re: Using IBM Java and Power's 16MB large pages

    ‏2012-09-26T15:40:24Z  in response to PowerLinuxTeam
     
    Correct. You have to allocate the large pages and tell Java to use large pages with -Xlp option.
     
     1) Large pages allocation in Linux
    echo N > /proc/sys/vm/nr_hugepages
    mkdir -p /libhugetlbfs
    mount -t hugetlbfs hugetlbfs /libhugetlbfs
    cat /proc/meminfo | grep Huge
     
    2) Java -Xlp option
     java -Xlp16m
     
    Updated on 2012-09-26T15:40:24Z at 2012-09-26T15:40:24Z by rfolco
    • ThinkOpenly
      ThinkOpenly
      40 Posts
      ACCEPTED ANSWER

      Re: Using IBM Java and Power's 16MB large pages

      ‏2012-09-26T15:29:51Z  in response to rfolco
      "TCMALLOC"? That would be for use only when actually using "tcmalloc", correct?
      • rfolco
        rfolco
        6 Posts
        ACCEPTED ANSWER

        Re: Using IBM Java and Power's 16MB large pages

        ‏2012-09-26T15:40:40Z  in response to ThinkOpenly
        Yes, please ignore the first line. This is for tcmalloc only.
         
        just removed line:
        TCMALLOC_MEMFS_MALLOC_PATH=/libhugetlbfs/ 
        Updated on 2012-09-26T15:40:40Z at 2012-09-26T15:40:40Z by rfolco
  • mauricfo
    mauricfo
    8 Posts
    ACCEPTED ANSWER

    Re: Using IBM Java and Power's 16MB large pages

    ‏2012-09-25T20:10:57Z  in response to PowerLinuxTeam
     There should be a hugetlbfs mount in place as well.
     
    Check with: 
    # hugeadm --list-all-mounts
    hugeadm:ERROR: No hugetlbfs mount points found 
     
    This should handle it: 
    # hugeadm --create-global-mounts
     
    Checking mounts.. 
    # hugeadm --list-all-mounts
    Mount Point                             Options
    /var/lib/hugetlbfs/global/pagesize-16MB rw,relatime,pagesize=16777216
    /var/lib/hugetlbfs/global/pagesize-16GB rw,relatime,pagesize=17179869184
     
    (pools for 16MB and 16GB hugepages are in place)
     
    Allocating 100 16MB (default size) hugepages, and verifying the hugepage pools..
    # echo 100 > /proc/sys/vm/nr_hugepages
     # hugeadm --pool-list
          Size  Minimum  Current  Maximum  Default
      16777216      100      100      100        *
    17179869184        0        0        0
     
    Ok. 100 16MB hugepages allocated. 
     
     
    If the hugetlbfs mount is properly set up but java -Xlp still doesn't use hugepages, you may want to check shared memory settings.
     
    Specifically, the files shmmax (max shared memory segment size, in bytes) and shmall (number of pages) under /proc/sys/kernel..
    their values must be big enough to hold the the size of all allocated hugepages.
    The defaults are usually fine, but an application may need tuning.
     
    From man proc command:
     
           /proc/sys/kernel/shmall
                  This file contains the system-wide limit on the total number of pages of System V shared memory.

           /proc/sys/kernel/shmmax
                  This file can be used to query and set the run-time limit on the maximum (System V IPC) shared memory  segment  size
                  that can be created.
     
    For 100 16MB (100 * 16 * 1024^2 bytes) hugepages: 
    # echo  $((100*16*1024**2)) > /proc/sys/kernel/shmmax
     
    (or trying w/ the size of a few more hugepages, to fit overhead) 
     
    The default value of  /proc/sys/kernel/shmall shoud handle most cases, however may be worth checking too.
    • pperf
      pperf
      6 Posts
      ACCEPTED ANSWER

      Re: Using IBM Java and Power's 16MB large pages

      ‏2012-09-26T14:32:46Z  in response to mauricfo
       Hi Rafael & Mauricio,
       
      Thanks for the info,  They are very helpful in my recent Java benchmark testing.
      • mauricfo
        mauricfo
        8 Posts
        ACCEPTED ANSWER

        Re: Using IBM Java and Power's 16MB large pages

        ‏2012-09-26T20:51:19Z  in response to pperf
        You're welcome.
        Hope you get good numbers! :-)