Topic
  • 5 replies
  • Latest Post - ‏2015-01-16T18:11:13Z by FKGK_karthikeyan_krishnamoorth
SystemAdmin
SystemAdmin
2402 Posts

Pinned topic MEM tab in NMON reports

‏2008-12-29T19:31:15Z |
It has recently become my task to take NMON Analyzer reports from 6 servers (4 application servers, 2 database) and consolidate CPU, memory, network, disk, etc. metrics into a single report. Thanks to incredible documentation and extensive forum support, I've managed to gain a grasp on what these reports are saying in the week I've been working on this project, but I'm having trouble with the MEM tab. Specifically, Real Free % and Virtual Free %.

From what I understand (complete AIX novice) the real free % is free physical RAM, and the virtual free % is free virtual memory. According to what I've read, AIX tries to grab up as much memory as it can and allocate it to either to the system or the user. The way I understand this is that if there 5% Real Free, and 99% virtual free, there is about 95% physical ram allocated to the system/user, but only 1% of that is being currently utilized. Please correct me if I'm wrong as I'm just trying to clarify this so I can read the data correctly.

The servers I'm looking at typically have 1-8% real free memory, and 85-99% virtual free. Each server (I think there is more than one in a physical machine) has 4 CPUs and around 64g of ram.

Am I on track here, or am I missing the boat?

Thanks!
Updated on 2009-01-05T15:48:45Z at 2009-01-05T15:48:45Z by SystemAdmin
  • nagger
    nagger
    1638 Posts

    AIX Memory

    ‏2009-01-05T11:33:21Z  
    This question is mostly to do with AIX rather than nmon.

    First, the AIX kernel owns the random access memory (memory) - it is not a question of grabbing it.
    This is put on a free list so the kernel knows which parts of memory can be allocated.
    The memory is in 4KB pages so it keeps track of the pages via the free list.
    Memory then gets allocated to various uses like:
    • the kernel itself = system
    • programs that are running i.e. processes = user
    • file system cache of disk blocks

    With all these demands on memory, eventually, it is all used up - this is a good thing as it improves performance.
    Then the kernel has to free some memory up and does this a couple of ways.

    If the memory page has been assigned but it is a copy of what is on disk but has not been used for a long time (like the read-only instruction part of a program or filesystem cache of a disk block) then the page can be put back on the free list to be reused.

    If the memory page is unique and not a copy of something on a disk (like the private temporary memory of a process) then is is copied to the paging space before it can be released and added to the free list.

    This paging space is often referred to as virtual memory. Running programs think they are entirely in memory but in fact while some of the program is in memory, a lot of it is on the paging space disks. The AIX kernel does many tricks to pretend that it is all in and copies it in an out on demand = called paging.

    In this way, the AIX system has Real memory plus Virtual memory in which to run programs.
    Back to the questions
    Yes, "real free" refers to the real physical random access memory (RAM) of the machine.
    "Virtual free" is the paging space.

    Your statement on "95% allocated but only 1% used" in exactly WRONG.

    You have 95% of real physical memory used PLUS 1% of virtual memory i.e. paging space used.
    Think of the virtual memory as the over flow area to be used when all of RAM has been used.

    A simple typical example, might be
    • 64 GB RAM at 99% used
    • 100 GB paging space at 10% used

    Then you are using ~64GB Physical Real memory PLUS 10 GB Virtual memory = 74GB

    Hope this helps, Nigel
  • SystemAdmin
    SystemAdmin
    2402 Posts

    Re: MEM tab in NMON reports

    ‏2009-01-05T15:48:45Z  
    Wow. Nigel, this is exactly the answer I was looking for. Thanks for explaining it so clearly! Embarrassing that I didn't understand such a simple concept, but helpful nonetheless.

    Thanks again!

    James
  • NickCool
    NickCool
    1 Post

    Re: MEM tab in NMON reports

    ‏2013-06-09T08:30:06Z  

    This post is very useful. I still have another question following this one, why does the kernel begin to use paging space(1% used) while there's still free physical real memory in system(5% free) in this case? Is there any linkage between the used virtual memory and the free real memory?

    Updated on 2013-06-09T08:53:34Z at 2013-06-09T08:53:34Z by NickCool
  • Steve_ATS
    Steve_ATS
    40 Posts

    Re: MEM tab in NMON reports

    ‏2013-06-12T19:14:10Z  
    • NickCool
    • ‏2013-06-09T08:30:06Z

    This post is very useful. I still have another question following this one, why does the kernel begin to use paging space(1% used) while there's still free physical real memory in system(5% free) in this case? Is there any linkage between the used virtual memory and the free real memory?

    Usually only special kernel structures, like a paging table map that are backed to paging space, probably for recovery features.

  • FKGK_karthikeyan_krishnamoorth
    1 Post

    Re: AIX Memory

    ‏2015-01-16T18:11:13Z  
    • nagger
    • ‏2009-01-05T11:33:21Z
    This question is mostly to do with AIX rather than nmon.

    First, the AIX kernel owns the random access memory (memory) - it is not a question of grabbing it.
    This is put on a free list so the kernel knows which parts of memory can be allocated.
    The memory is in 4KB pages so it keeps track of the pages via the free list.
    Memory then gets allocated to various uses like:
    • the kernel itself = system
    • programs that are running i.e. processes = user
    • file system cache of disk blocks

    With all these demands on memory, eventually, it is all used up - this is a good thing as it improves performance.
    Then the kernel has to free some memory up and does this a couple of ways.

    If the memory page has been assigned but it is a copy of what is on disk but has not been used for a long time (like the read-only instruction part of a program or filesystem cache of a disk block) then the page can be put back on the free list to be reused.

    If the memory page is unique and not a copy of something on a disk (like the private temporary memory of a process) then is is copied to the paging space before it can be released and added to the free list.

    This paging space is often referred to as virtual memory. Running programs think they are entirely in memory but in fact while some of the program is in memory, a lot of it is on the paging space disks. The AIX kernel does many tricks to pretend that it is all in and copies it in an out on demand = called paging.

    In this way, the AIX system has Real memory plus Virtual memory in which to run programs.
    Back to the questions
    Yes, "real free" refers to the real physical random access memory (RAM) of the machine.
    "Virtual free" is the paging space.

    Your statement on "95% allocated but only 1% used" in exactly WRONG.

    You have 95% of real physical memory used PLUS 1% of virtual memory i.e. paging space used.
    Think of the virtual memory as the over flow area to be used when all of RAM has been used.

    A simple typical example, might be
    • 64 GB RAM at 99% used
    • 100 GB paging space at 10% used

    Then you are using ~64GB Physical Real memory PLUS 10 GB Virtual memory = 74GB

    Hope this helps, Nigel

    Very useful Nigel. Thanks a lot!