Topic
14 replies Latest Post - ‏2013-05-10T15:44:21Z by manwerjalil
k0da
k0da
7 Posts
ACCEPTED ANSWER

Pinned topic Power5 and amount of memory assigned

‏2011-06-16T13:23:29Z |
Hi,
I'm running SLES11SP1 and have confused with amount of memory that kernel detects is lower, that partition have assigned.
HMC reports, I have 1GB memory assigned for LPARs
~> lshwres -r mem -m shark --level lpar --filter "lpar_ids=7"
lpar_name=07-sunrise,lpar_id=7,curr_min_mem=768,curr_mem=1024,curr_max_mem=1024,pend_min_mem=768,pend_mem=1024,pend_max_mem=1024,run_min_mem=768,run_mem=1024

But OS shows only 875M
  1. cat /proc/meminfo | grep MemTotal
MemTotal: 896832 kB
  1. free -k
total used free shared buffers cached
Mem: 896832 808384 88448 0 268224 373440
-/+ buffers/cache: 166720 730112
Swap: 1534016 107456 1426560
  1. uname -r
2.6.32.36-0.5-ppc64

Other partitions have 1GB memory as well, but OS there reports different numbers.
Another example:
~> lshwres -r mem -m shark --level lpar --filter "lpar_ids=10"
lpar_name=09-kiwipower,lpar_id=10,curr_min_mem=768,curr_mem=1024,curr_max_mem=1024,pend_min_mem=768,pend_mem=1024,pend_max_mem=1024,run_min_mem=768,run_mem=1024
  1. cat /proc/meminfo | grep MemTotal
MemTotal: 1018496 kB
  1. free -k
total used free shared buffers cached
Mem: 1018496 240000 778496 0 40000 88256
-/+ buffers/cache: 111744 906752
Swap: 7872 0 7872

Is that behavior normal?
Updated on 2011-06-23T09:57:24Z at 2011-06-23T09:57:24Z by k0da
  • jscheel
    jscheel
    31 Posts
    ACCEPTED ANSWER

    Re: Power5 and amount of memory assigned

    ‏2011-06-17T15:51:30Z  in response to k0da
    Yes, what you are observing is that the Hypervisor is using memory on behalf of the partition. That means what one allocates for memory on the HMC never reflects what's actually in the partition as shown by the OS. If you are concerned about what's technically going on, I suspect we'll find more details in a Redbook. I'm just unsure in which one.

    However, I can with confidence tell you that you are not imagining things, nor have you stumbled on a defect. It's working as designed.

    Hope that helps. Let me know if you want more information and I'll track that down for you.
    -Jeff
    • k0da
      k0da
      7 Posts
      ACCEPTED ANSWER

      Re: Power5 and amount of memory assigned

      ‏2011-06-17T16:11:12Z  in response to jscheel
      I will appreciate, if you provide me with Redbook name.

      I though, that memory should be assigned as is or at least with no such big drawbacks
      • jscheel
        jscheel
        31 Posts
        ACCEPTED ANSWER

        Re: Power5 and amount of memory assigned

        ‏2011-06-17T18:15:22Z  in response to k0da
        Would you mind doing a "cat /proc/ppc64/lparcfg" and telling me what the DesMem field says? I suspect it matches the value for which you are looking, but please confirm.
        • jscheel
          jscheel
          31 Posts
          ACCEPTED ANSWER

          Re: Power5 and amount of memory assigned

          ‏2011-06-17T18:17:57Z  in response to jscheel
          If you're looking for information on all of the lparcfg fields, it can be found here: http://www.ibm.com/developerworks/wikis/display/LinuxP/lparcfg+documentation
        • k0da
          k0da
          7 Posts
          ACCEPTED ANSWER

          Re: Power5 and amount of memory assigned

          ‏2011-06-17T18:19:51Z  in response to jscheel
          1. cat /proc/ppc64/lparcfg | grep DesMem
          DesMem=1024
          • jscheel
            jscheel
            31 Posts
            ACCEPTED ANSWER

            Re: Power5 and amount of memory assigned

            ‏2011-06-20T13:59:22Z  in response to k0da
            Ok, as expected. This matches your "cur_mem" field from the HMC. So, it appears the partition has seen all the memory but Linux is not totalling it as available.

            We'll dig deeper and see what we can find out. I still suspect this is working as designed but may not be documented. Once we have a definitive, technical answer, we can correct the documentation part. Thanks for your patience.
  • AntonBlanchard
    AntonBlanchard
    4 Posts
    ACCEPTED ANSWER

    Re: Power5 and amount of memory assigned

    ‏2011-06-21T22:14:49Z  in response to k0da
    Run dmesg and look for the line with physicalMemorySize, eg:

    physicalMemorySize = 0xf30000000

    This should be the amount of memory the hypervisor has given us. If they are the same, then Linux is responsible for the difference.

    One of the big consumers of memory at boot for Linux is kdump. Check /proc/cmdline on both boxes to make sure they have the same kdump settings.
    • k0da
      k0da
      7 Posts
      ACCEPTED ANSWER

      Re: Power5 and amount of memory assigned

      ‏2011-06-22T13:56:33Z  in response to AntonBlanchard
      I have physicalMemorySize = 0x40000000, so if I had converted correctly it is 1Gb.

      So Kernel bug?
      • k0da
        k0da
        7 Posts
        ACCEPTED ANSWER

        Re: Power5 and amount of memory assigned

        ‏2011-06-22T13:57:53Z  in response to k0da
        Ah, forgot to mention. I have no kdump configured
  • MikeWolf
    MikeWolf
    2 Posts
    ACCEPTED ANSWER

    Re: Power5 and amount of memory assigned

    ‏2011-06-22T14:44:28Z  in response to k0da
    I have configured a partition with 1 GB of memory and installed SLES11SP1 on it.
    MemTotal: 896576 kB
    DesMem=1024
    root=/dev/sda4 quiet sysrq=1 insmod=sym53c8xx insmod=ipr crashkernel=512M-:128M

    which is nearly identical to what you are reporting. If I take the "crashkernel=" off
    of the cmdline I then see:
    MemTotal: 1016832 kB
    DesMem=1024
    root=/dev/sda4 quiet sysrq=1 insmod=sym53c8xx insmod=ipr

    which is similiar to the other system you compared to. Please post the kernel commandline
    'cat /proc/cmdline' and double check about crashkernel on it. SLES configures that by default
    • k0da
      k0da
      7 Posts
      ACCEPTED ANSWER

      Re: Power5 and amount of memory assigned

      ‏2011-06-22T15:04:24Z  in response to MikeWolf
      Yes, got 994MB by disabling kdump on SLES LPAR.

      Ok, but where 30M goes to?

      I'll appreciate, if somebody can provide RedBook name where I can read about it.
  • MikeWolf
    MikeWolf
    2 Posts
    ACCEPTED ANSWER

    Re: Power5 and amount of memory assigned

    ‏2011-06-22T15:49:38Z  in response to k0da
    Some of it is used by the Run Time Abstraction Service (rtas)
    some is taken up by OF-lite. To see the size rtas is taking
    up.

    od -x /proc/device-tree/rtas/rtas-size
    • k0da
      k0da
      7 Posts
      ACCEPTED ANSWER

      Re: Power5 and amount of memory assigned

      ‏2011-06-23T09:57:24Z  in response to MikeWolf
      Thanks, now it is clear for me.
  • manwerjalil
    manwerjalil
    2 Posts
    ACCEPTED ANSWER

    Re: Power5 and amount of memory assigned

    ‏2013-05-10T15:44:21Z  in response to k0da

    Guys I am in same trouble. I have one IBM p701 with 16GB memory running PPC64 linux. This is physical machine and it shows only 2GB memory out of 16GB. I have reinstalled this server again to resolve this issue but still same problem.

    Please help me.