Topic
  • 14 replies
  • Latest Post - ‏2013-05-10T15:44:21Z by manwerjalil
k0da
k0da
7 Posts

Pinned topic Power5 and amount of memory assigned

‏2011-06-16T13:23:29Z |
Hi,
I'm running SLES11SP1 and have confused with amount of memory that kernel detects is lower, that partition have assigned.
HMC reports, I have 1GB memory assigned for LPARs
~> lshwres -r mem -m shark --level lpar --filter "lpar_ids=7"
lpar_name=07-sunrise,lpar_id=7,curr_min_mem=768,curr_mem=1024,curr_max_mem=1024,pend_min_mem=768,pend_mem=1024,pend_max_mem=1024,run_min_mem=768,run_mem=1024

But OS shows only 875M
  1. cat /proc/meminfo | grep MemTotal
MemTotal: 896832 kB
  1. free -k
total used free shared buffers cached
Mem: 896832 808384 88448 0 268224 373440
-/+ buffers/cache: 166720 730112
Swap: 1534016 107456 1426560
  1. uname -r
2.6.32.36-0.5-ppc64

Other partitions have 1GB memory as well, but OS there reports different numbers.
Another example:
~> lshwres -r mem -m shark --level lpar --filter "lpar_ids=10"
lpar_name=09-kiwipower,lpar_id=10,curr_min_mem=768,curr_mem=1024,curr_max_mem=1024,pend_min_mem=768,pend_mem=1024,pend_max_mem=1024,run_min_mem=768,run_mem=1024
  1. cat /proc/meminfo | grep MemTotal
MemTotal: 1018496 kB
  1. free -k
total used free shared buffers cached
Mem: 1018496 240000 778496 0 40000 88256
-/+ buffers/cache: 111744 906752
Swap: 7872 0 7872

Is that behavior normal?
Updated on 2011-06-23T09:57:24Z at 2011-06-23T09:57:24Z by k0da
  • jscheel
    jscheel
    31 Posts

    Re: Power5 and amount of memory assigned

    ‏2011-06-17T15:51:30Z  
    Yes, what you are observing is that the Hypervisor is using memory on behalf of the partition. That means what one allocates for memory on the HMC never reflects what's actually in the partition as shown by the OS. If you are concerned about what's technically going on, I suspect we'll find more details in a Redbook. I'm just unsure in which one.

    However, I can with confidence tell you that you are not imagining things, nor have you stumbled on a defect. It's working as designed.

    Hope that helps. Let me know if you want more information and I'll track that down for you.
    -Jeff
  • k0da
    k0da
    7 Posts

    Re: Power5 and amount of memory assigned

    ‏2011-06-17T16:11:12Z  
    • jscheel
    • ‏2011-06-17T15:51:30Z
    Yes, what you are observing is that the Hypervisor is using memory on behalf of the partition. That means what one allocates for memory on the HMC never reflects what's actually in the partition as shown by the OS. If you are concerned about what's technically going on, I suspect we'll find more details in a Redbook. I'm just unsure in which one.

    However, I can with confidence tell you that you are not imagining things, nor have you stumbled on a defect. It's working as designed.

    Hope that helps. Let me know if you want more information and I'll track that down for you.
    -Jeff
    I will appreciate, if you provide me with Redbook name.

    I though, that memory should be assigned as is or at least with no such big drawbacks
  • jscheel
    jscheel
    31 Posts

    Re: Power5 and amount of memory assigned

    ‏2011-06-17T18:15:22Z  
    • k0da
    • ‏2011-06-17T16:11:12Z
    I will appreciate, if you provide me with Redbook name.

    I though, that memory should be assigned as is or at least with no such big drawbacks
    Would you mind doing a "cat /proc/ppc64/lparcfg" and telling me what the DesMem field says? I suspect it matches the value for which you are looking, but please confirm.
  • jscheel
    jscheel
    31 Posts

    Re: Power5 and amount of memory assigned

    ‏2011-06-17T18:17:57Z  
    • jscheel
    • ‏2011-06-17T18:15:22Z
    Would you mind doing a "cat /proc/ppc64/lparcfg" and telling me what the DesMem field says? I suspect it matches the value for which you are looking, but please confirm.
    If you're looking for information on all of the lparcfg fields, it can be found here: http://www.ibm.com/developerworks/wikis/display/LinuxP/lparcfg+documentation
  • k0da
    k0da
    7 Posts

    Re: Power5 and amount of memory assigned

    ‏2011-06-17T18:19:51Z  
    • jscheel
    • ‏2011-06-17T18:15:22Z
    Would you mind doing a "cat /proc/ppc64/lparcfg" and telling me what the DesMem field says? I suspect it matches the value for which you are looking, but please confirm.
    1. cat /proc/ppc64/lparcfg | grep DesMem
    DesMem=1024
  • jscheel
    jscheel
    31 Posts

    Re: Power5 and amount of memory assigned

    ‏2011-06-20T13:59:22Z  
    • k0da
    • ‏2011-06-17T18:19:51Z
    1. cat /proc/ppc64/lparcfg | grep DesMem
    DesMem=1024
    Ok, as expected. This matches your "cur_mem" field from the HMC. So, it appears the partition has seen all the memory but Linux is not totalling it as available.

    We'll dig deeper and see what we can find out. I still suspect this is working as designed but may not be documented. Once we have a definitive, technical answer, we can correct the documentation part. Thanks for your patience.
  • AntonBlanchard
    AntonBlanchard
    4 Posts

    Re: Power5 and amount of memory assigned

    ‏2011-06-21T22:14:49Z  
    Run dmesg and look for the line with physicalMemorySize, eg:

    physicalMemorySize = 0xf30000000

    This should be the amount of memory the hypervisor has given us. If they are the same, then Linux is responsible for the difference.

    One of the big consumers of memory at boot for Linux is kdump. Check /proc/cmdline on both boxes to make sure they have the same kdump settings.
  • k0da
    k0da
    7 Posts

    Re: Power5 and amount of memory assigned

    ‏2011-06-22T13:56:33Z  
    Run dmesg and look for the line with physicalMemorySize, eg:

    physicalMemorySize = 0xf30000000

    This should be the amount of memory the hypervisor has given us. If they are the same, then Linux is responsible for the difference.

    One of the big consumers of memory at boot for Linux is kdump. Check /proc/cmdline on both boxes to make sure they have the same kdump settings.
    I have physicalMemorySize = 0x40000000, so if I had converted correctly it is 1Gb.

    So Kernel bug?
  • k0da
    k0da
    7 Posts

    Re: Power5 and amount of memory assigned

    ‏2011-06-22T13:57:53Z  
    • k0da
    • ‏2011-06-22T13:56:33Z
    I have physicalMemorySize = 0x40000000, so if I had converted correctly it is 1Gb.

    So Kernel bug?
    Ah, forgot to mention. I have no kdump configured