Useful microprocessor options
There are many useful microprocessor-related options for the sar command.
The most useful options are:
- sar -P
The -P option reports per-processor statistics for the specified processors. By specifying the ALL keyword, statistics for each individual processor and an average for all processors is reported. When -P ALL is used inside a workload partition environment, RSET-wide statistics will be displayed in addition to system wide statistics and the processors belonging to the RSET will be prefixed with an asterisk (*) symbol. RSET-wide statistics will be displayed only if the workload partition environment is associated with an RSET. Of the flags which specify the statistics to be reported, only the -a, -c, -m, -u, and -w flags are meaningful with the -P flag.
The following example shows the per-processor statistic while a microprocessor-bound program was running on processor number 0:# sar -P ALL 2 2 AIX tooltime2 1 6 00CA52594C00 04/02/08 System configuration: lcpu=4 mode=Capped 05:23:08 cpu %usr %sys %wio %idle physc 05:23:11 0 8 92 0 0 1.00 1 0 51 0 49 0.00 2 0 0 0 100 0.51 3 0 0 0 100 0.48 - 4 46 0 50 1.99 05:23:13 0 10 89 0 0 1.00 1 0 7 0 93 0.00 2 0 3 2 95 0.51 3 0 0 0 100 0.49 - 5 45 0 49 2.00 Average 0 9 91 0 0 1.00 1 0 12 0 88 0.00 2 0 2 1 98 0.51 3 0 0 0 100 0.48 - 5 46 0 49 1.99The last line of every stanza, which starts with a dash (-) in the cpu column, is the average for all processors. An average (-) line displays only if the -P ALL option is used. It is removed if processors are specified. The last stanza, labeled with the word Average instead of a time stamp, keeps the averages for the processor-specific rows over all stanzas.
The following example shows the vmstat output during this time:System configuration: lcpu=4 mem=44570MB kthr memory page faults cpu ----- ----------- ------------------------ ------------ ----------- r b avm fr re pi po fr sr cy in sy cs us sy id wa 2 0 860494 6020610 0 0 0 0 0 0 16 14061 409 5 45 49 0 2 0 860564 6020540 0 0 0 0 0 0 4 14125 400 5 45 50 0 1 0 860669 6020435 0 0 0 0 0 0 3 14042 388 5 46 49 0 2 0 860769 6020335 0 0 0 0 0 0 3 13912 398 5 45 50 0The first numbered line is the summary since startup of the system. The second line reflects the start of the sar command, and with the third row, the reports are comparable. The vmstat command can only display the average microprocessor utilization over all processors. This is comparable with the dashed (-) rows from the microprocessor utilization output from the sar command.
When run inside a WPAR environment, the same command produces the following output:
AIX wpar1 1 6 00CBA6FE4C00 04/01/08 wpar1 configuration: lcpu=2 memlim=204MB cpulim=0.06 rset=Regular 05:23:08 cpu %usr %sys %wio %idle physc 05:23:11 *0 8 92 0 0 1.00 *1 0 51 0 49 0.00 2 0 0 0 100 0.51 3 0 0 0 100 0.48 R 4 71 0 24 1.00 - 4 46 0 50 1.99 05:23:13 *0 10 89 0 0 1.00 *1 0 7 0 93 0.00 2 0 3 2 95 0.51 3 0 0 0 100 0.49 R 5 48 0 46 1.00 - 5 45 0 49 2.00 Average *0 9 91 0 0 1.00 *1 0 12 0 88 0.00 2 0 2 1 98 0.51 3 0 0 0 100 0.48 R 4 51 0 44 1.00 - 5 46 0 49 1.99The WPAR has an associated RSET registry. Processors 0 and 1 are attached to the RSET. The R line displays the use by the RSET associated with the WPAR. The processors present in the RSET are prefixed by the asterisk (*) symbol.
- sar –P RST is used to display the use
metrics of the processors present in the RSET. If there is no RSET
associated with the WPAR environment, all of the processor’s metrics
are displayed.
The following example shows sar –P RST run inside a WPAR environment:
AIX wpar1 1 6 00CBA6FE4C00 04/01/08 wpar1 configuration: lcpu=2 memlim=204MB cpulim=0.06 rset=Regular 05:02:57 cpu %usr %sys %wio %idle physc 05:02:59 0 20 80 0 0 1.00 1 10 0 0 90 0.00 R 15 40 0 45 1.00 05:03:01 0 20 80 0 0 1.00 1 8 0 0 92 0.00 R 14 40 0 46 1.00 Average 0 20 80 0 0 1.00 1 9 0 0 91 0.00 R 14 40 0 46 1.00 - sar -u
This displays the microprocessor utilization. It is the default if no other flag is specified. It shows the same information as the microprocessor statistics of the vmstat or iostat commands.
During the following example, a copy command was started:# sar -u 3 3 AIX wpar1 1 6 00CBA6FE4C00 04/01/08 wpar1 configuration: lcpu=2 memlim=204MB cpulim=0.06 rset=Regular 05:02:57 cpu %usr %sys %wio %idle physc 05:02:59 0 20 80 0 0 1.00 1 10 0 0 90 0.00 R 15 40 0 45 1.00 05:03:01 0 20 80 0 0 1.00 1 8 0 0 92 0.00 R 14 40 0 46 1.00 Average 0 20 80 0 0 1.00 1 9 0 0 91 0.00 R 14 40 0 46 1.00When run inside a workload partition, the same command produces the following output:AIX wpar1 1 6 00CBA6FE4C00 04/01/08 wpar1 configuration: lcpu=2 memlim=204MB cpulim=0.06 rset=Regular 05:07:16 %usr %sys %wio %idle physc %resc 05:07:19 17 83 - - 0.11 181.6 05:07:22 19 81 - - 0.08 133.5 05:07:26 16 84 - - 0.10 173.4 Average 17 83 - - 0.10 164.3This displays the %resc information for workload partitions that have processor resource limits enforced. This metric indicates the percentage of processor resource consumed by the workload partition.
- sar -c The -c option shows the system call rate.
# sar -c 1 3 19:28:25 scall/s sread/s swrit/s fork/s exec/s rchar/s wchar/s 19:28:26 134 36 1 0.00 0.00 2691306 1517 19:28:27 46 34 1 0.00 0.00 2716922 1531 19:28:28 46 34 1 0.00 0.00 2716922 1531 Average 75 35 1 0.00 0.00 2708329 1527While the vmstat command shows system call rates as well, the sar command can also show if these system calls are read(), write(), fork(), exec(), and others. Pay particular attention to the fork/s column. If this is high, then further investigation might be needed using the accounting utilities, the trace command, or the tprof command.
- sar -q The -q option shows the run-queue size and the swap-queue size.
# sar -q 5 3 19:31:42 runq-sz %runocc swpq-sz %swpocc 19:31:47 1.0 100 1.0 100 19:31:52 2.0 100 1.0 100 19:31:57 1.0 100 1.0 100 Average 1.3 95 1.0 95- runq-sz
- The average number of threads that are runnable per second and the percentage of time that the run queue was occupied (the % field is subject to error).
- swpq-sz
- The average number of threads in the VMM wait queue and the % of time that the swap queue was occupied. (The % field is subject to error.)
The -q option can indicate whether you have too many jobs running (runq-sz) or have a potential paging bottleneck. In a highly transactional system, for example Enterprise Resource Planning (ERP), the run queue can be in the hundreds, because each transaction uses small amounts of microprocessor time. If paging is the problem, run the vmstat command. High I/O wait indicates that there is significant competing disk activity or excessive paging due to insufficient memory.