Processor statistics

Edit online

Processor statistics contain the utilization statistics of logical partition (LPAR), which is recorded in the nmon recording file.

The following sections in the nmon recording file are used to identify the processor statistics:

CPU_ALL

Records the overall LPAR processor utilization based on the Processor Utilization Resource register (PURR) entries. This section is recorded by default. This section contains the following fields:

User %

Average percentage of processor utilization as against the total processor utilization of the LPAR, when the LPAR is in user mode.

System %

Average percentage of processor utilization as against the total processor utilization of the LPAR, when the LPAR is in kernel mode. This percentage includes donated PURR and stolen PURR values.

Donated PURR is the number of processor cycles that are donated by the LPAR to any other LPAR.
Stolen PURR is the number of processor cycles that are used by the hypervisor from the LPAR.

Wait %

Average percentage of processor utilization against the overall processor utilization of the LPAR when the LPAR is in I/O wait mode. This recording information contains following scenarios:

In a shared LPAR, if the total dispatch time is less than the entitlement units assigned to the LPAR, the unused PURR value is added to the value of the Wait % field.
In a dedicated LPAR, if the LPAR is capable of giving idle processor cycles to other LPARs, the busy stolen PURR value and the donated PURR values are added to the sys PURR value. Idle, stolen, and the donated PURR values are added to either the wait PURR value or the idle PURR value.

Idle%

Average percentage of the processor utilization as against the overall processor utilization of the LPAR in idle mode. This recording information contains following scenarios:

In a shared LPAR, if the total dispatch time is less than the entitlement units assigned to the LPAR, the unused PURR value is added to the Wait% value.
In a dedicated LPAR, if the LPAR is capable of giving idle processor cycles to other LPARs, then the busy stolen PURR value and the donated PURR values are accounted to sys PURR value. Idle, stolen, and donated PURR are accounted to either the wait PURR value or the idle PURR values.

Busy

Percentage of physical processor that is used for the actual processing.

Physical CPUs

Total number of logical CPUs that are available.

User

Total physical processing units that are used in the user mode.

System

Total physical processing units that are used in the kernel mode.

Wait

Total physical processing units that are used in the wait mode.

Idle

Total physical processing units that are used in the idle mode.

Entitled capacity

Physical processing units that are allocated to the LPAR.

SCPU_ALL

Records the overall CPU utilization of the LPAR in the Scalable Processor Utilization Resource register (SPURR). This recording is performed only for shared partitions. The SCPU_ALL and PCPU_ALL sections are same on LPARs that are running in nominal frequency. This section contains the following fields:

User: Total physical processing units that are used in the user mode based on SPURR metrics.
System: Total physical processing units that are used in the system mode based on SPURR metrics.
Wait: Total physical processing units that are used in the wait mode based on SPURR metrics.
Idle: Total physical processing units that are used in the idle mode based on SPURR metrics.

CPU* metrics

Records the utilization statistics per logical processor level.

Notes

Default logical processor utilization in percentage based on clock ticks.
Logical processor utilization in percentage based on PURR ticks.

This metric does not completely determine the logical processor utilization. This metric provides an overview of ratio of processor utilization in each of the following modes: user mode, system mode, wait mode and idle mode. This section contains the following fields:

User%

Logical processor utilization in the user mode contains the following scenarios:

Average percentage of logical processor utilization in user mode as against the total clock ticks used.
Average percentage of logical processor utilization in user mode as against the total time that is spent by the LPAR in the CPU.

System%

System utilization percentage contains the following scenarios:

Average percentage of logical processor utilization in wait mode as against the total clock ticks used.
Average percentage of logical processor utilization in wait mode as against the total time that is spent by the LPAR in the CPU.

Wait%

Logical processor utilization in wait mode contains the following scenarios:

Average percentage of logical processor utilization in idle mode as against the total clock ticks used.
Average percentage of logical processor utilization in idle mode as against the total time that is spent by the LPAR in the CPU.

Idle%

Logical processor utilization in idle mode contains the following scenarios:

Average percentage of logical processor utilization in idle mode as against the total clock ticks used.
Average percentage of logical processor utilization in idle mode as against the total by the LPAR in the CPU.

PCPU* metric

Records the statistics of utilization in physical processing units per logical CPU.

Notes:

Sum of user, sys, wait, and idle values of the PCPU metric is equal to the PCPU_ALL value.
Sum of User and sys values of each physical CPU is equal to the Physb value of the logical CPU.

This section contains the following fields:

User: Physical processing units that are used in user mode by the logical CPU.
system: Physical processing units that are used in kernel mode by the logical CPU.
wait: Physical processing units that are used in wait mode by the logical CPU.
idle: Physical processing units that are used in idle mode by the logical CPU.

SCPU* metric

Records utilization statistics for each logical CPU based on Scaled Processor Utilization Resource Register (SPURR) value.

Notes:

Sum of user, sys, wait, idle values must match with the SCPU_ALL value of each logical CPU.
Sum of User and sys values of each physical CPU is equal to the Physb value of the logical CPU.
The SCPU values must match with the PCPU values when the logical CPUs run in nominal frequency.

This section contains the following fields:

User: Physical processing units that are used in user mode based on the SPURR value.
Sys: Physical processing units that are used in kernel mode based on the SPURR value.
Wait: Physical processing units that are used in wait mode based on the SPURR vallue.
Idle: Physical processing units that are used in idle mode based on the SPURR value.

BBBL

Records logical partition configuration details. This section is recorded only once in the recording file. This section contains the following fields:

BBBL,01,lparno: Partition ID.
BBBL,02,lparname: Partition name that is assigned in the Hardware Management Console (HMC).
BBBL,03,CPU in sys: Number of active licensed physical CPUs in the system (CEC).
Note: This number is same across all partitions of the same server.
BBBL,04,Virtual CPU: Number of current online virtual CPUs in the LPAR.
BBBL,05,Logical CPU: Number of online logical CPUs in the LPAR.
BBBL,06,smt threads: Number of Symmetric Multi Threading (SMT) threads that are enabled when the recording started.
BBBL,07,capped: Flag to indicate whether a partition is capped or uncapped. A value of 1 indicates that the partition is capped and a value of 0 indicates that the partition is uncapped. Capped partitions are not allowed to exceed the entitlement capacity. Uncapped partitions are allowed to exceed the entitlement capacity.
BBBL,08,min Virtual: Minimum number of virtual CPUs in the LPAR definition.
BBBL,09,max Virtual: Maximum number of virtual CPUs the LPAR can support.
BBBL,10,min Logical: Minimum number of logical CPUs in the LPAR.
Note: This value is based on number of virtual CPUs and each virtual CPU is running minimum one thread.
BBBL,11,max Logical: Maximum number of Logical CPUs the LPAR can support.
BBBL,12,min Capacity: The minimum unit at which you can change the entitled capacity. A value in multiples of a whole number indicates a dedicated LPAR.
BBBL,13,max Capacity: The maximum unit at which you can change the entitled capacity. This value is based on the maximum virtual processors of the LPAR.
BBBL,14,Entitled Capacity: The number of processing units that are entitled to the LPAR.
BBBL,15,min Memory MB: Minimum memory in MB configured in the LPAR definition.
BBBL,16,max Memory MB: Maximum memory in MB supported for the LPAR.
BBBL,17,online Memory: Amount of Memory in MB that is currently online.
BBBL,18,Pool CPU: Number of physical CPUs in the shared processor pool of the LPAR.
BBBL,19,Weight: Variable processor capacity weight.
Note: Variable processor capacity weight is the priority weight assigned to the LPAR using the lparstat command which controls how idle capacity is allocated to it. A weight of -1 indicates a soft cap is in place.
This parameter is used to determine the share of extra capacity that is provided to uncapped partitions. Uncapped partitions can only use the processor cycles based on its variable capacity weight and virtual processors. This value is a number in the range 0 - 255 and represents the share of extra capacity that the LPAR is eligible to receive. A partition's share is computed by dividing its variable capacity weight by the sum of the variable capacity weights of all uncapped partitions. Therefore, a value of 0 might be used to prevent a partition from receiving extra capacity. This method is also called as soft cap.
BBBL,20,pool id: Shared processor pool ID of the LPAR.

LPAR

Records logical partition processor utilization statistics. This statistics is recorded only for the shared partitions. This section contains the following fields:

PhysicalCPU

Amount of CPU used by the LPAR in physical processing units.

virtualCPUs

Number of current online virtual CPUs in the LPAR.

logicalCPUs

Number of online logical CPUs in the LPAR.

poolCPUs

Number of physical CPUs in the shared processor pool of the LPAR.

Entitled

The number of processing units the LPAR is entitled to receive.

Weight

Variable processor capacity units. This parameter is used to determine the share of extra capacity that is provided to uncapped partitions.

PoolIdle

Available physical processor units in the shared processor pool.

usedAllCPU%

Percentage of physical processors that are used as against the physical processors that are available in the server.

usedPoolCPU%

Percentage of physical processors that are used as against the available physical processors in the shared pool. This value is always 0 for dedicated partitions.

SharedCPU

This value is 1 for shared partitions and 2 for dedicated partitions.

Capped

A value of 1 for this field indicates a capped partition and a value of 2 for this partition indicates an uncapped partition.

EC_User%

Physical processor consumption in the user mode contains the following cases:

Physical processor consumption is less than the entitlement percentage of entitlement that is used in user mode.
Physical processor consumption is greater than the entitlement percentage of physical processor that is used in user mode.

Note: Sum of EC_User %, EC_Sys%, EC_Wait% fields, and EC_Idle % fields is equal to following values: In the first case, it is equal to the percentage of entitlement used and in the second case, it is equal to100%.

EC_Sys%

Physical processor consumption in system mode contains the following scenarios:

Physical processor consumption is less than the entitlement percentage of entitlement that is used in system mode.
Physical processor consumption is greater than the entitlement percentage of physical processor that is used in system mode.

Note: Sum of EC_User %, EC_Sys%, EC_Wait%, and EC_Idle % is equal to following values: In the first case, it is equal to the percentage of entitlement used and in the second case, it is equal to 100%.

EC_Wait%

Physical processor consumption in wait mode contains the following cases:

Physical Processor consumption is less than the entitlement percentage of entitlement that is used in the wait mode.
Physical processor consumption is greater than the entitlement percentage of physical processor that is used in the wait mode.

EC_Idle%

Physical processor consumption in idle mode contains the following cases:

Physical Processor consumption is less than the entitlement percentage of entitlement that is used in the idle mode.
Physical processor consumption is greater than the entitlement percentage of physical processor that is used in the idle mode.

VP_User%

Percentage of virtual CPU consumption in user mode.

VP_Sys%

Percentage of virtual CPU consumption in system mode.

VP_Wait%

Percentage of virtual CPU consumption in I/O wait mode.

VP_Idle%

Percentage of virtual CPU consumption in idle mode.

Folded

Number of virtual processors that are folded. Folding virtual processor means that moving idle virtual processors into a hibernation state to conserve physical processor resources.

Pool_id

Shared processor pool ID of the LPAR.

POOLS

Records multiple shared processor pool statistics. This section is recorded for partitions of POWER6 processor-based systems.

shcpus_in_sys: Number of physical processors that are allocated for shared processor use, across all shared processors pools.
max_pool_capacity: Maximum processor capacity of shared processor pool of the partition.
entitled_pool_capacity: Entitled processor capacity of shared processor pool of the partition.
pool_max_time: Sum of the maximum time, in seconds, that is used by the shared processor pool of the partition.
pool_busy_time: Sum of busy time, in seconds, accumulated across all partitions in the shared processor pool of the partition.
shcpu_tot_time: Total time, in seconds, accumulated across all physical processors that are allocated to the shared processor pools.
shcpu_busy_time: Total busy time, in seconds, that is accumulated across all physical processors that are allocated to shared processors pools.
Pool_id: ID of the shared processor pool of the partition.
entitled: The number of processing units the LPAR is entitled to receive.

DONATE

Records Donate statistics of the logical partition from a dedicated partition with capable of donating its resources. This section is recorded only if the partition is dedicated and is capable of donating its resources.

Note: The processing capacity is fixed in the dedicated processing mode and the capacity gets wasted during low workload processing.

For enhanced use of dedicated processing capacity, donating of processor units started with POWER6 technology. Optionally dedicated partitions can be configured as processor donors. In this mode, the unused cycles are donated to the physical shared-processor pool associated with the Micro-Partitioning value. Maximum processor utilization throughout the system is achieved because of the utilization of unused cycles. The power hypervisor ensures that the only spare CPU cycles are donated. This feature increases system utilization, without affecting the critical partitions configured in a dedicated processor mode.

When the CPU utilization of the CPU core goes below a threshold, and all the Symmetric Multi Threading (SMT ) threads of the CPU are idle from a hypervisor perspective, the CPU is donated to the shared processor pool. The donated processor is returned within micro-seconds to the dedicated processor partition when the timer of one of the SMT threads on the donated CPU receives an instruction to run.

pUser: Total Physical processing units that are used in the user mode.
pSys: Total Physical processing units that are used in the system mode.
pWait: Total Physical processing units that are used in the wait I/O mode.
pIdle: Total Physical processing units that are used in the Idle mode.
pDonateIdle: Idle physical processing units that are donated to the shared processor pool.
pDonateBusy: Busy physical processing units that are donated to the shared processor pool.
pStolenIdle: Idle processing units that are stolen by hypervisor from the dedicated partitions. The hypervisor might forcibly steal cycles from the dedicated partition to perform hypervisor activity. Though the hypervisor might generally steal cycles when the processors in the partition are idle, it might also steal cycles when the processors in the partition are waiting for some hypervisor activity to complete.
In cases where stealing idle cycles is not sufficient for hypervisor to complete activity, it can also borrow cycles when processor is in busy state. This stealing of cycles is orthogonal to donation enablement on the processor and any partition settings.
pStolenBusy: Busy physical processing units that are stolen by the hypervisor from the dedicated partitions.

RAWLPAR

Records raw LPAR statistics that is memory dump of the perfstat_partition_total_t structure that is defined in the libperfstat.h header file. You can use the -K flag to enable the recording of these statistics. This statistics provides the flexibility to the you to utilize the statistics as per your requirement.

Name: Name of the LPAR.
Type: Bits representing partition details. This field is of the perfstat_partition_type type.
lpar_id: Partition ID.
cpu_group_id: ID of the LPAR group to which the partition belongs.
cpu_pool_id: Shared processor pool ID of the partition.
online_cpus: Number of current online virtual CPUs in the LPAR.
max_cpus: Maximum number of virtual CPUs the LPAR can support.
min_cpus: Minimum number of Virtual CPUs in the LPAR definition.
online_memory: Amount of memory, in MB, that is online.
max_memory: Maximum memory, in MB, supported by the LPAR.
min_memory: Minimum memory, in MB, configured for the LPAR definition.
entitled_proc_capacity: The number of processing units this LPAR is entitled to receive.
max_proc_capacity: The maximum unit at which you can change the entitled capacity. This value is based on the maximum virtual processors of the LPAR.
min_proc_capacity: The minimum unit at which you can change the entitled capacity. A whole number indicates a dedicated LPAR.
proc_capacity_increment: Unit at which you can change entitled processor capacity.
unalloc_proc_capacity: Deallocated processor units in the shared processor pool of the partition.
var_proc_capacity_weight: Variable processor capacity weight. This parameter is used to determine the share of extra capacity that is provided to uncapped partitions.
unalloc_var_proc_capacity_weight: Number of variable processor capacity weight currently deallocated in the shared processor pool of this partition.
online_phys_cpus_sys: Number of active licensed physical CPUs in the system.
max_phys_cpus_sys: Maximum number of physical CPUs in the system that includes unlicensed and hot-pluggable CPUs.
phys_cpus_pool: Number of physical CPUs in the shared processor pool of the LPAR.
Puser: Number of physical processor tics in the user mode.
Psys: Number of physical processor tics in the system mode.
Pidle: Number of physical processor tics in the idle mode.
Pwait: Number of physical processor tics in the wait I/O mode.
pool_idle_time: Idle time of shared processor pool in nanoseconds.
phantintrs: Number of phantom interrupts that are received by this partition.
Note: Phantom interrupts are interrupts that are targeted for a different partition but are sharing physical processor of the LPAR.
invol_virt_cswitch: Number of involuntary virtual CPU context switches. Involuntary context switches occur when the time slot allocated to the LPAR on the physical processor is exhausted and it must be allocated to the next LPAR.
vol_virt_cswitch: Number of voluntary virtual CPU context switches.
Note: Voluntary virtual CPU context switches occur when virtual processor voluntarily provides the processor cycles to hypervisor after it completes the work of the processor.
timebase_last: Total number of clock ticks.
Note: This clock ticks every time irrespective of whether the LPAR is running on the actual physical processor.

RAWCPUTOTAL

Records the perfstat_cpu_total_t memory dump from the libperfstat.h header file without performing any calculations on the memory dump. You can use the -K flag to enable the recording of this statistics in the nmon recording file. This section contains the following fields:

ncpus

Number of active logical processors.

ncpus_cfg

Number of configured processors.

description

Description of the processor in the format <type_official name>. For POWER7 processor-based systems, the description is POWERPC_POWER7.

processorHZ

Speed of the processor in Hertz.

user

Number of processor clock ticks in the user mode.

sys

Raw number of processor clock ticks in the system mode.

idle

Raw number of processor clock ticks in the idle mode.

wait

Raw number of processor clock ticks in the I/O wait mode.

pswitch

Number of processor context switches that occur when a new process is dispatched.

Note: Context switch preempts the current running thread, saves its machine state in the mstsave structure, restores the machine state of another thread, and resumes the execution of the newly dispatched thread.

syscall

Number of system calls that are run.

sysread

Number of read system calls that are run.

syswrite

Number of write system calls that are run.

sysfork

Number of fork system calls that are run.

sysexec

Number of exec system calls that are run.

readch

Number of characters that are transferred by using the read system call.

writech

Number of characters that are transferred by using the write system call.

devintrs

Number of device interrupts.

softintrs

Number of software interrupts. A software interrupt is similar to a hardware interrupt that saves the current state of the machine and runs another subroutine that has interrupted. Software interrupt instruction calls the system call handler routine.

lbolt

Average number of runnable processes during the last 1 minute.

loadavg5

Average number of runnable processes during the last 5 minutes.

loadavg15

Average number of runnable processes during the last 15 minutes.

runque

Length of the current run queue.

swpque

Number of ready processes that are waiting to be paged-in currently.

bread

Total number of blocks that are read by the LPAR.

bwrite

Total number of blocking write operations by the LPAR.

lread

Number of logical read requests sent to the LPAR. When a logical read or write operation to a block device is performed, a logical transfer size of less than a full block size can be requested. The system accesses the physical device units of complete blocks and saves the accessed blocks in the kernel buffers. These buffers are called the block I/O cache. This cache area is managed by the kernel so that multiple logical read and write operations to the block device can access previously buffered data from the cache. These read and write operations does not involve real I/O operation on the device.

Read and write operation requests to the block devices by the application are reported statistically as logical read and write operations. The block I/O operations that are performed by the kernel to the block device are reported as block read and write operations.

lwrite

Number of logical write requests.

phread

Number of physical read operations. These operations are performed on raw devices.

Note: Requested I/O to raw character devices is not buffered as it is for block devices. The I/O operation is performed by the device directly.

phwrite

Number of physical write operations. These write operations are made to raw devices.

runocc

Counter that is incremented when run queue is occupied.

Note: This counter is used to calculate the average number of ready processes.

swpocc

Counter that is incremented when the swap queue is occupied. This counter is used to calculate the average number of processes that are waiting to be paged-in.

iget

Number of inode lookup calls performed to search a file or directory associated with an inode.

namei

Number of vnode lookup calls performed to search a file or directory associated with a path name.

dirblk

Number of 512-byte blocking read operations performed by the directory search routine to locate an entry for a file.

msg

Number of interprocess communication (IPC) message operations.

Note: Message queue structurally passes small amounts of information between processes. Information that must be communicated is placed in a predefined message structure.

sema

Number of IPC semaphore operations.

Note: Semaphores are specialized data structures that are used to coordinate access to a non-sharable resource.

rcvint

Number of tty interrupts that are received by the process.

xmtint

Number of tty transmission interrupts that are received by the process.

mdmint

Number of tty modem interrupts that are received by the process.

tty_rawinch

Number of tty input queue characters.

tty_caninch

Number of tty canonical input queue characters.

tty_rawoutch

Number of tty output queue characters.

ksched

Number of kernel processes created.

koverf

Number of times kernel processes are not created because of the process threshold limit is crossed. Reasons for reaching threshold limit follow:

The user has run fork system call to a maximum limit.
The configuration limit of processes is reached.

kexit

Number of kernel processes that became zombie processes.

rbread

Number of remote read requests.

rcread

Number of cached remote read operations.

rbwrt

Number of remote write operations.

rcwrt

Number of cached remote write operations.

traps

Number of asynchronous trap notifications.

ncpus_high

Maximum number of active processors that are online during boot time.

puser

Raw number of physical processor tics in the user mode.

psys

Raw number of physical processor tics in system mode.

pidle

Raw number of physical processor tics in the idle mode.

pwait

Raw number of physical processor tics in the I/O mode.

decrintrs

Number of decrementer tic interrupts.

The decrementer is a binary counter that generates a clock interrupt each time the clock moves to zero. The tic is the value of a decrement. On few systems, time is decremented in nanoseconds. Therefore each tick is equal to 1 nanosecond. On other systems, the value of the decrement depends on the system.

mpcrintrs

Numbers of MPC interrupts that are received.

Note: The kernel proactively generates an extra interrupt to ensure rapid response to a cross-CPU preemption request when the preempting thread is considered as real-time thread. Without this extra interrupt (called an MPC), the preempted thread might continue to run without interruption until the next regularly scheduled timer tick, or up to 10 ms. MPC interrupts reduce preemption latency.

mpcsintrs

Number of MPC send interrupts.

The kernel proactively generates an extra interrupt to provide a rapid response to a cross-CPU preemption request by a real-time thread. The preempted thread continues to run until the next regularly scheduled timer tick, or up to 10 milliseconds unless interrupted by an MPC interrupt. MPC interrupts reduce preemption latency.

phantintrs

Number of phantom interrupts that are received by this partition.

Note: Phantom interrupts are interrupts that are targeted for a different partition but are sharing the physical processor of the LPAR.