vmstat

Use this command to give multiple statistics on the system. The vmstat command reports statistics about kernel threads in the run and wait queue, memory paging, interrupts, system calls, context switches, and CPU activity.

The CPU activity is percentage breakdown of user mode, system mode, idle time, and waits for disk I/O.

The general syntax of this command is:
vmstat  <time_between_samples_in_seconds> <number_of_samples> -t
A typical output looks like this:
kthr     memory             page              faults        cpu        time
----- ----------- ------------------------ ------------ ----------- --------
 r  b   avm   fre  re  pi  po  fr   sr  cy  in   sy  cs us sy id wa hr mi se
 0  0 45483   221   0   0   0   0    1   0 224  326 362 24  7 69  0 15:10:22
 0  0 45483   220   0   0   0   0    0   0 159   83  53  1  1 98  0 15:10:23
 2  0 45483   220   0   0   0   0    0   0 145  115  46  0  9 90  1 15:10:24
In this output, look for:
  • Columns r (run queue) and b (blocked) starting to increase, especially beyond 10. This rise usually indicates that you have too many processes competing for CPU.
  • Values in the pi, po (page in/out) columns at non-zero, possibly indicating that you are paging and need more memory. The stack size might be set too high for some of your JVM instances.
  • cs (contact switches) going very high compared to the number of processes. You might have to tune the system with vmtune.
  • In the cpu section, us (user time) indicating the time being spent in programs. Assuming Java™ is the first in the list in tprof, you need to tune the Java application. In the cpu section, if sys (system time) is higher than expected, and you still have id (idle) time remaining, you might have lock contention. Check the tprof for lock–related calls in the kernel time. You might want to try multiple instances of the JVM.
  • The -t flag, which adds the time for each sample at the end of the line.