Local, Nar & Far Memory part 5 - Low Entitlement has a Bad Side Effect
nagger 100000MRSJ Comment (1) Visits (10707)
The title should read "Local, Near & Far ..." - I will not correct it or links might fail.
With a shared processor virtual machine (I am calling this "VM" but was called LPAR!) there are various suggestions of setting Entitlement ("Desired processing units" on the LPAR profile on the HMC, I am calling this "E") and Virtual Processor numbers (I am calling this "VP").
For Capped, the Entitlement is the maximum guaranteed CPU time that you can't go over and you round up the Entitlement to the next whole number (there is no point of having higher VP as AIX will fold them away as high numbers of CPU are inefficient).
For Uncapped I have seen many possibilities like the following options. For illustration purposes let us take a VM which averages in busy periods 16 physical CPUs, peaks regularly for a few minutes to 18 physical CPUs and we have a shared processor pool of say 48 physical CPUs. Four scenarios:
All of these involve some compromise but there are two hidden side effects that you should know about. The first is covered below and the virtual Processor numbers in the next AIXpert blog (part 6).
To Low an Entitlement
If you go for suggestion 1 (above) the VM is gaurenteed the E CPU time but then needs to compete in the pool for further CPU time. If the weight factor is the same for all VM then it will get its fair share. You may have decided that it is production (or more important than others in some way) and mad the wight factor larger - a good move this just gets the shared of CPU time larger but there is a scheduling side effect that can slow your VM down.
Let me go through the below diagrams to show you the effect:
Below we have the 10 millisecond dispatch cycle which the Hypervisor uses to run virtual machines (VM) on the physical processor - we have just one processor here but a number of VM to run on it. The Entitlement is guaranteed to each VM within these 10 millisecond windows.
Below the other VM run and either use up there Entitlement can get forced off the processor or run out of work and voluntarily yield the processor for some other work.
Below - we have 4 milliseconds "spare" to allocate.
Below - we have some virtual machines that don't want any CPU time (they are waiting to an external event = interrupt) but some are still runnable and would like more CPU time. The weight factor is used to allocate more time and we let them run again.
Below - but again some stop and in this example only our VM is still runnable and wants more CPU cycles so it is scheduled again and runs to the end of the 10 millisecond dispatch cycle.
Below - we have used all the CPU cycles available between the VM running and every VM has got its Entitlement and uncapped VM got extra CPU cycles based on the weight factor so every one is happy - now we get to start over in the next dispatch cycle.
Below - You may have notices that our VM was scheduled 3 times in 10 milliseconds.
Below is the bit you may not know. The virtual machines are all running on the same physical CPU which has a single Level 1 and Level 2 cache for the program code and data of the programs running in the VM. They are highly secure so one VM can't access the cache lines of another VM (this is normal virtual memory features) but as one VM runs it brings in its memory and knocks out cache lines for other VM. So when our VM starts it may have to reload cache lines that are now "missing" while it waited for CPU time.
The below is a grossly exaggerated worst case scenario to make the point but don't forget to keep the pictures simple our VM got on the CPU only 3 times - it could be much higher numbers of on and off the CPU.
Below we highlight the cache warming up again periods
Below - is the alternative is we give the virtual machine a high and more appropriate Entitlement - then it does not have to give up the CPU until it has used its entire allocated CPU time
In the next AIXpert Blog, we look at the issues of to Many a Virtual Processors.