Technical Blog Post
Abstract
Java 6.0.1+ Uses Multiple Compiler Threads - Can Lead to High CPU in JIT
Body
When examining the threads that use CPU, you may find that the JIT threads continuously use the majority of CPU:
Example of high CPU in JIT from the sleep.prof output
(collected as part of the "MustGather: Performance, hang, or high CPU issues with WebSphere Application Server on AIX" for WebSphere Application Server- aixperf.sh script in the section "Collecting Data Manually")
Process Freq Total Kernel User Shared Other
========== ====== ====== ====== ====== ====== ======
/usr/java 246 95.65 3.28 0.00 68.33 24.04
...
PID TID CPU% HEX
========= ========== ======= =======
12124366 41287697 7.61 2760011 JIT Compilation Thread-0
12124366 52232239 7.35 31d002f JIT Compilation Thread-1
12124366 26017843 7.30 18d0033 JIT Compilation Thread-2
12124366 46858399 7.21 2cb009f JIT Compilation Thread-3
...
Totals
======
...
Total % For All Processes
(/usr/IBM/WebSphere/AppServer/java/jre/lib/ppc64/compressedrefs/libj9jit26.so) = 41.57
...
Checking the processor core entitlement as noted in the vmstat output, we see that the system has 16 CPUs but the lpar is only entitled to 2:
System configuration: lcpu=16 ... ent=2.00
Why you may see high CPU in JIT after updating to a 6.0.1 JVM or later from a 6.0 JVM or earlier:
In Java 6.0.1 and up, JIT uses multiple compiler threads (up to 4 max), where in Java 6.0.0 and lower, JIT would only use one thread.
Solution:
-XcompilationThreads1
To go back to the Java 6.0.0 single JIT thread use the "-XcompilationThreads1" generic JVM argument which will prevent the JIT from using any more then 1 CPU core.
After Java 6.0.1, by default, JIT will use up to 4 threads depending on the number of CPU cores that are available. JIT having use of only one CPU (-XcompilationThreads1) allows the Java threads to have more use of the overall CPU time, but also reduces the number of methods JIT can compile at a given time. Using only one CPU could lengthen ramp-up time.
If running on limited physical CPUs (i.e. 2 CPUs) while the OS is reporting more than the limited number (i.e. 16 CPUs), then Java will use the OS reported number of CPUs by creating (up to) 4 JIT threads. The additional threads dedicated to JIT compiling causes the overall CPU usage for JIT to increase. When the JIT thread number exceeds the number of dedicated cores, the "-XcompilationThreads1" option is a perfect solution.
See the -XcompilationThreads option in the product documentation.
Related content in the product documentation:
title image (modified) credit: (cc) Some rights reserved by ClkerFreeVectorImages
UID
ibm11081311