Potentially Lower Performance on AIX with shared processors and CPU intensive applications
kgibm 0600027VAP Visits (3237)
Update: I discussed this issue with Nigel Griffiths (the 'n' in nmon) and he has posted information on mpstat -d which is much easier to use and gives basically the same (and in some senses more) information about process and memory affinity: http
When using shared processors (mic
The benefit of Micro-Partitioning is that it allows for increased overall utilization of system resources by applying only the required amount of processor resource needed by each partition. But due to the overhead associated with maintaining online virtual processors, consider the capacity requirements when choosing values for the attributes. For optimal performance, ensure that you create the minimal amount of partitions, which decreases the overhead of scheduling virtual processors. CPU-intensive applications, like high performance computing applications, might not be suitable for a Micro-Partitioning environment. If an application uses most of its entitled processing capacity during execution, you should use a dedicated processor partition to handle the demands of the application.
PerfPMR is the recommended deep-dive script to diagnose performance problems on AIX (not to be confused with aixperf.sh which comes from the WAS Performance, hang, or high CPU MustGather and is usually a precursor to using PerfPMR). PerfPMR is downloaded from ftp:
To check for virtual processor affinity, we can use PerfPMR to gather kernel trace and then use curt:
Run PerfPMR (exa
If you only want to collect processor affinity information, and you don't want everything else in PerfPMR, then you can collect just the kernel trace that's needed (this example is for 10 seconds):
Process the PerfPMR output dire
Run curt on the trace:
Open curt.out. The report is split up into system, per-CPU, and per-thread analysis. For each thread (section starts with "Report for Thread Id"), find the "processor affinity:" line.
The ideal affinity is 1.0 (meaning that the virtual processor is always going back to the same physical processor, thus maximizing cache hits, etc.) and the worst affinity is 0. Affinity may be low if a partition is above its entitlement and the shared processor pool does not have extra capacity or is in flux, because the partition will constantly have to take cycles from other processors.
Perform this before the performance problem occurs (under full load) and during the problem and compare the affinities. If affinity decreased during the problem, then the lack of entitlement may be making things worse. Be careful with cause and effect here: it's unlikely (though possible) that the decreased affinity in and of itself caused the problem, but instead was a secondary symptom that made things worse.
Processor affinity may be worse depending on the "spread" over the physical processors with a large number of configured virtual processors. Recent versions of AIX introduced processor folding which tries to optimize the use of the least number of virtual processors both to increase affinity and to decrease processor management overhead. Nevertheless, it may help to have the number of virtual processors not much higher than the entitled capacity or the effectively used capacity (see the processor folding link on how to calculate virtual processors).