Pinned topic Power5 and amount of memory assigned

‏2011-06-16T13:23:29Z | Tags:

Answered question This question has been answered.

Unanswered question This question has not been answered yet.

Hi,
I'm running SLES11SP1 and have confused with amount of memory that kernel detects is lower, that partition have assigned.
HMC reports, I have 1GB memory assigned for LPARs
~> lshwres -r mem -m shark --level lpar --filter "lpar_ids=7"
lpar_name=07-sunrise,lpar_id=7,curr_min_mem=768,curr_mem=1024,curr_max_mem=1024,pend_min_mem=768,pend_mem=1024,pend_max_mem=1024,run_min_mem=768,run_mem=1024

But OS shows only 875M

cat /proc/meminfo | grep MemTotal

MemTotal: 896832 kB

free -k

total used free shared buffers cached
Mem: 896832 808384 88448 0 268224 373440
-/+ buffers/cache: 166720 730112
Swap: 1534016 107456 1426560

uname -r

2.6.32.36-0.5-ppc64

Other partitions have 1GB memory as well, but OS there reports different numbers.
Another example:
~> lshwres -r mem -m shark --level lpar --filter "lpar_ids=10"
lpar_name=09-kiwipower,lpar_id=10,curr_min_mem=768,curr_mem=1024,curr_max_mem=1024,pend_min_mem=768,pend_mem=1024,pend_max_mem=1024,run_min_mem=768,run_mem=1024

cat /proc/meminfo | grep MemTotal

MemTotal: 1018496 kB

free -k

total used free shared buffers cached
Mem: 1018496 240000 778496 0 40000 88256
-/+ buffers/cache: 111744 906752
Swap: 7872 0 7872

Is that behavior normal?

Updated on 2011-06-23T09:57:24Z at 2011-06-23T09:57:24Z by k0da

jscheel 0600025BWM

31 Posts
Re: Power5 and amount of memory assigned

‏2011-06-17T15:51:30Z

This is the accepted answer. This is the accepted answer.

Yes, what you are observing is that the Hypervisor is using memory on behalf of the partition. That means what one allocates for memory on the HMC never reflects what's actually in the partition as shown by the OS. If you are concerned about what's technically going on, I suspect we'll find more details in a Redbook. I'm just unsure in which one.

However, I can with confidence tell you that you are not imagining things, nor have you stumbled on a defect. It's working as designed.

Hope that helps. Let me know if you want more information and I'll track that down for you.
-Jeff
- Log in to reply.
k0da 060000J588

7 Posts
Re: Power5 and amount of memory assigned

‏2011-06-17T16:11:12Z

This is the accepted answer. This is the accepted answer.
jscheel 0600025BWM

‏2011-06-17T15:51:30Z

Yes, what you are observing is that the Hypervisor is using memory on behalf of the partition. That means what one allocates for memory on the HMC never reflects what's actually in the partition as shown by the OS. If you are concerned about what's technically going on, I suspect we'll find more details in a Redbook. I'm just unsure in which one.

However, I can with confidence tell you that you are not imagining things, nor have you stumbled on a defect. It's working as designed.

Hope that helps. Let me know if you want more information and I'll track that down for you.
-Jeff

More...
I will appreciate, if you provide me with Redbook name.

I though, that memory should be assigned as is or at least with no such big drawbacks
- Log in to reply.
jscheel 0600025BWM

31 Posts
Re: Power5 and amount of memory assigned

‏2011-06-17T18:15:22Z

This is the accepted answer. This is the accepted answer.
k0da 060000J588

‏2011-06-17T16:11:12Z

I will appreciate, if you provide me with Redbook name.

I though, that memory should be assigned as is or at least with no such big drawbacks

More...
Would you mind doing a "cat /proc/ppc64/lparcfg" and telling me what the DesMem field says? I suspect it matches the value for which you are looking, but please confirm.
- Log in to reply.
jscheel 0600025BWM

31 Posts
Re: Power5 and amount of memory assigned

‏2011-06-17T18:17:57Z

This is the accepted answer. This is the accepted answer.
jscheel 0600025BWM

‏2011-06-17T18:15:22Z

Would you mind doing a "cat /proc/ppc64/lparcfg" and telling me what the DesMem field says? I suspect it matches the value for which you are looking, but please confirm.

More...
If you're looking for information on all of the lparcfg fields, it can be found here: http://www.ibm.com/developerworks/wikis/display/LinuxP/lparcfg+documentation
- Log in to reply.
k0da 060000J588

7 Posts
Re: Power5 and amount of memory assigned

‏2011-06-17T18:19:51Z

This is the accepted answer. This is the accepted answer.
jscheel 0600025BWM

‏2011-06-17T18:15:22Z

Would you mind doing a "cat /proc/ppc64/lparcfg" and telling me what the DesMem field says? I suspect it matches the value for which you are looking, but please confirm.

More...
1. cat /proc/ppc64/lparcfg | grep DesMem
DesMem=1024
- Log in to reply.
jscheel 0600025BWM

31 Posts
Re: Power5 and amount of memory assigned

‏2011-06-20T13:59:22Z

This is the accepted answer. This is the accepted answer.
k0da 060000J588

‏2011-06-17T18:19:51Z

cat /proc/ppc64/lparcfg | grep DesMem

DesMem=1024

More...
Ok, as expected. This matches your "cur_mem" field from the HMC. So, it appears the partition has seen all the memory but Linux is not totalling it as available.

We'll dig deeper and see what we can find out. I still suspect this is working as designed but may not be documented. Once we have a definitive, technical answer, we can correct the documentation part. Thanks for your patience.
- Log in to reply.
AntonBlanchard 060001CVXR

4 Posts
Re: Power5 and amount of memory assigned

‏2011-06-21T22:14:49Z

This is the accepted answer. This is the accepted answer.

Run dmesg and look for the line with physicalMemorySize, eg:

physicalMemorySize = 0xf30000000

This should be the amount of memory the hypervisor has given us. If they are the same, then Linux is responsible for the difference.

One of the big consumers of memory at boot for Linux is kdump. Check /proc/cmdline on both boxes to make sure they have the same kdump settings.
- Log in to reply.
k0da 060000J588

7 Posts
Re: Power5 and amount of memory assigned

‏2011-06-22T13:56:33Z

This is the accepted answer. This is the accepted answer.
AntonBlanchard 060001CVXR

‏2011-06-21T22:14:49Z

Run dmesg and look for the line with physicalMemorySize, eg:

physicalMemorySize = 0xf30000000

This should be the amount of memory the hypervisor has given us. If they are the same, then Linux is responsible for the difference.

One of the big consumers of memory at boot for Linux is kdump. Check /proc/cmdline on both boxes to make sure they have the same kdump settings.

More...
I have physicalMemorySize = 0x40000000, so if I had converted correctly it is 1Gb.

So Kernel bug?
- Log in to reply.
k0da 060000J588

7 Posts
Re: Power5 and amount of memory assigned

‏2011-06-22T13:57:53Z

This is the accepted answer. This is the accepted answer.
k0da 060000J588

‏2011-06-22T13:56:33Z

I have physicalMemorySize = 0x40000000, so if I had converted correctly it is 1Gb.

So Kernel bug?

More...
Ah, forgot to mention. I have no kdump configured
- Log in to reply.

Forums

Linux for Power Architecture

▼Topic Tags ?

Pinned topic Power5 and amount of memory assigned

Re: Power5 and amount of memory assigned

Re: Power5 and amount of memory assigned

Re: Power5 and amount of memory assigned

Re: Power5 and amount of memory assigned

Re: Power5 and amount of memory assigned

Re: Power5 and amount of memory assigned

Re: Power5 and amount of memory assigned

Re: Power5 and amount of memory assigned

Re: Power5 and amount of memory assigned