Diagnosing problems with class data sharing
If you encounter problems with class data sharing, VM messages are typically generated that point to an underlying cause. In some situations, a cache might fail to initialize correctly. In other situations classes might not be found or stored in the shared classes cache. To provide more information about a problem, you can generate verbose output, use diagnostic cache utilities, or use the Eclipse OpenJ9™ trace facility.
Initialization problems
If you do not specify a directory for the shared classes cache by using the cacheDir
suboption, the cache is created in the javasharedresources
directory in the following default location:
- On Windows® systems, this directory is created in the user's
C:\Users\<username>\AppData\Local\
directory. - On z/OS® systems, this directory is created in the
/tmp
directory. - On other systems, this directory is located in the
.cache
directory in the user's home directory. Do not set the user's home directory on an NFS mount or a shared mount across systems or LPARs.
Initialization problems can occur on systems other than Windows because caches are created with read/write access for the user only and subsequent users do not have permission to write to the home directory. If you specify the -Xshareclasses:groupAccess
suboption, the cache is created in the /tmp
directory instead where all users have permission to write to the cache.
For a persistent cache, initialization problems can also occur if there is insufficient disk space available or if you are attempting to locate the cache on a remote networked file system, which is not supported.
For a non-persistent cache, a shared memory area is required. Check that your system is configured with sufficient resources as described in Setting shared memory values. If initialization problems persist, try deleting the cache by using the -Xshareclasses:destroyAll
cache utility, which removes all known shared memory areas and semaphores for caches in the cacheDir
specified or the default directory. These actions force the VM to re-create the cache.
Problems finding or storing classes
The most common cause for classes not being stored in the cache is due to space constraints. Make sure that you set an appropriate size for the cache, as described in Creating a shared classes cache. Setting a soft maximum size is recommended, because you can adjust the soft maximum size that is set for the cache after it is created. See Setting a soft maximum size.
Storing classes can also be a problem if the cache is opened read-only or if the class does not exist on the file system because it is sourced from a URL location.
If you are attempting to share modified bytecode, you must use a modification context, as described in Sharing modified bytecode. Otherwise, classes are stored in a private area that is not accessible to other VMs.
If you are using a custom class loader, class path entries in the SharedClassURLClasspathHelper
must be confirmed before classes can be found for these entries. More information about confirmed entries is available in the SharedClassURLClasspathHelper
interface in the com.ibm.oti.shared
API documentation.
If you are instrumenting bytecode by using a JVMTI agent or java.lang.instrument
agent, the following rules apply:
- Redefined classes are never stored in the cache.
- Retransformed classes are stored only if you specify the
-Xshareclasses:cacheRetransformed
suboption when you start your application.
If a running application uses its own class loader and you are using a SecurityManager
, you must grant the class loader permission to SharedCachePermission
before they can share classes. For more information, see Support for custom class loaders.
In very rare cases, problems with finding or storing classes might be due to cache corruption. If the VM detects that a cache is corrupt, it attempts to destroy the cache and re-create it. If the VM cannot re-create the cache, it starts only if the -Xshareclasses:nonfatal
suboption is specified on the command line, but without using the shared cache. Try using the -Xshareclasses:destroy
cache utility to remove the specific cache and re-create it. You might need to specify the cacheDir=<directory>
and name=<cache_name>
suboptions if the cache is not using the default settings.
Generating verbose output
A number of -Xshareclasses
suboptions are available for generating verbose output during class data sharing operations, which can help you identify the root cause of a problem.
verbose
The -Xshareclasses:verbose
suboption provides basic output on cache usage. In the following example, a persistent cache is opened and attached to the VM for class sharing. Information is provided about the size of the cache, the unstored bytes due to the setting of a soft maximum size, and maximum AOT and JIT data size.
java -Xshareclasses:name=myCache,verbose HelloWorld
[-Xshareclasses persistent cache enabled]
[-Xshareclasses verbose output enabled]
JVMSHRC237I Opened shared classes persistent cache myCache
JVMSHRC246I Attached shared classes persistent cache myCache
JVMSHRC765I Memory page protection on runtime data, string read-write data and partially filled pages is successfully enabled
Hello World
JVMSHRC168I Total shared class bytes read=2532416. Total bytes stored=268156
JVMSHRC818I Total unstored bytes due to the setting of shared cache soft max is 0. Unstored AOT bytes due to the setting of -Xscmaxaot is 0. Unstored JIT bytes due to the setting of -Xscmaxjitdata is 0.
verboseIO
The -Xshareclasses:verboseIO
suboption provides more detailed information about class sharing operations. In the following example, some classes are found when the cache is accessed. However, class openj9/internal/tools/attach/target/CommonDirectory
is not found and is therefore stored for sharing.
java -Xshareclasses:name=myCache,verboseIO HelloWorld
[-Xshareclasses verbose I/O output enabled]
Found class java/lang/Object in shared cache for class-loader id 0.
Found class java/lang/J9VMInternals in shared cache for class-loader id 0.
Found class com/ibm/oti/vm/VM in shared cache for class-loader id 0.
Found class java/lang/J9VMInternals$ClassInitializationLock in shared cache for class-loader id 0.
...
Failed to find class openj9/internal/tools/attach/target/CommonDirectory in shared cache for class-loader id 0.
Stored class openj9/internal/tools/attach/target/CommonDirectory in shared cache for class-loader id 0 with URL /root/sdk/jre/lib/amd64/compressedrefs/jclSC180/vm.jar (index 0).
...
The bootstrap class loader has an ID of 0
; other class loaders are given a unique ID. Class loaders follow the class loader hierarchy by asking the parent class loader for a class. If a parent fails to find the class in the cache, the child class loader stores the class in the cache.
In some situations, verbose output might not show classes being found. For example, classes are typically not found if the class is stale, as described in Class data sharing operations. Stale classes are redeemed if the same class is subsequently fetched by the class loader from another VM and checked against the stale class in the cache.
verboseAOT
To troubleshoot AOT problems, use the -Xshareclasses:verboseAOT
suboption on the command line, which generates output about AOT code that is found or stored in the cache.
In the following example output, a populated cache is being accessed to look for compiled AOT code. Some AOT code is found, which can be shared, and some AOT code is stored for reuse.
java -Xshareclasses:name=myCache,verboseAOT HelloWorld
[-Xshareclasses AOT verbose output enabled]
Found AOT code for ROMMethod 0x00007F658005C180 in shared cache.
Found AOT code for ROMMethod 0x00007F65800723EC in shared cache.
Found AOT code for ROMMethod 0x00007F6580071D14 in shared cache.
Stored AOT code for ROMMethod 0x00007F65801847B8 in shared cache.
Stored AOT code for ROMMethod 0x00007F65800D38A4 in shared cache.
Stored AOT code for ROMMethod 0x00007F65800723CC in shared cache.
Found AOT code for ROMMethod 0x00007F65800D38A4 in shared cache.
Stored AOT code for ROMMethod 0x00007F65800724C4 in shared cache.
...
verboseHelper
To troubleshoot problems with custom class loaders that use the Java SharedClassHelper
API, specify the -Xshareclasses:verboseHelper
suboption. Information messages and error messages are generated in the output, which can help you diagnose problems with finding or storing classes in the shared cache.
The following example output shows only information messages:
java -Xshareclasses:name=myCache,verboseHelper HelloWorld
[-Xshareclasses Helper API verbose output enabled]
Info for SharedClassURLClasspathHelper id 1: Verbose output enabled for SharedClassURLClasspathHelper id 1
Info for SharedClassURLClasspathHelper id 1: Created SharedClassURLClasspathHelper with id 1
Info for SharedClassURLClasspathHelper id 2: Verbose output enabled for SharedClassURLClasspathHelper id 2
Info for SharedClassURLClasspathHelper id 2: Created SharedClassURLClasspathHelper with id 2
Info for SharedClassURLClasspathHelper id 1: There are no confirmed elements in the classpath. Returning null.
Info for SharedClassURLClasspathHelper id 2: There are no confirmed elements in the classpath. Returning null.
Info for SharedClassURLClasspathHelper id 2: setClasspath() updated classpath. No invalid URLs found
Info for SharedClassURLClasspathHelper id 2: Number of confirmed entries is now 1
Hello World
Diagnostic cache utilities
These utilities display information about the contents of a shared classes cache. Run the utilities by specifying them as suboptions of -Xshareclasses
. The utilities run on the default cache unless you specify a cache by adding the name=<cache_name>
and cacheDir=<directory>
suboptions.
printStats
-Xshareclasses:printStats
-Xshareclasses:printStats,name=<cache_name>
-Xshareclasses:printStats=<data_type1>[+<data_type2>][...],name=<cache_name>
Displays summary information about the cache. For layered caches, -Xshareclasses:printStats
shows some information for the top layer cache, and summary information (bytes and counts only) for all layers combined. To see information for the top layer cache only, use printTopLayerStats
.
You can request more detail about items of a specific data type that are stored in the shared cache by using printStats=<data_type>
. Use the plus symbol (+) to separate the data types. For example, use printStats=romclass+url,name=myCache
to see information about ROMClass
and URL
items in all the layer caches of the cache called Cache1
. The valid data types are as follows (case insensitive):
help
(displays the list of valid data types)all
(equivalent toprintAllStats
)classpath
url
token
romclass
rommethod
aot
jitprofile
jithint
zipcache
stale
startuphint
Example output for a traditional cache (no cache layers: cache layer = 0
), with summary information only:
Current statistics for cache "Cache1":
Cache created with:
-Xnolinenumbers = false
BCI Enabled = true
Restrict Classpaths = false
Feature = cr
Cache contains only classes with line numbers
base address = 0x00007F60B807A000
end address = 0x00007F60B905E000
allocation pointer = 0x00007F60B81BE3A8
cache layer = 0
cache size = 16776608
softmx bytes = 16776608
free bytes = 12740572
Reserved space for AOT bytes = -1
Maximum space for AOT bytes = -1
Reserved space for JIT data bytes = -1
Maximum space for JIT data bytes = -1
Metadata bytes = 30440
Metadata % used = 0%
Class debug area size = 1331200
Class debug area used bytes = 189358
Class debug area % used = 14%
ROMClass bytes = 1328040
AOT bytes = 98404
JIT data bytes = 168
Zip cache bytes = 1133704
Startup hint bytes = 0
Data bytes = 114080
# ROMClasses = 452
# AOT Methods = 2
# Classpaths = 1
# URLs = 0
# Tokens = 0
# Zip caches = 21
# Startup hints = 0
# Stale classes = 0
% Stale classes = 0%
Cache is 24% full
Cache is accessible to current user = true
Example output for a cache with 2 layers (cache layer = 1
), with summary information only:
Current statistics for top layer of cache "Cache1":
Cache created with:
-Xnolinenumbers = false
BCI Enabled = true
Restrict Classpaths = false
Feature = cr
base address = 0x00007FCAB2766000
end address = 0x00007FCAB374A000
allocation pointer = 0x00007FCAB2766000
cache layer = 1
cache size = 16776608
softmx bytes = 16776608
free bytes = 15299372
Reserved space for AOT bytes = -1
Maximum space for AOT bytes = -1
Reserved space for JIT data bytes = -1
Maximum space for JIT data bytes = -1
Class debug area size = 1331200
Class debug area used bytes = 0
Class debug area % used = 0%
Cache is 8% full
Cache is accessible to current user = true
---------------------------------------------------------
Current statistics for all layers of cache "Cache1":
ROMClass bytes = 1328040
AOT bytes = 128924
JIT data bytes = 812
Zip cache bytes = 1133704
Startup hint bytes = 0
Data bytes = 114080
# ROMClasses = 452
# AOT Methods = 20
# Classpaths = 1
# URLs = 0
# Tokens = 0
# Zip caches = 21
# Startup hints = 0
# Stale classes = 0
% Stale classes = 0%
The Cache created with
section indicates the options that were used when the cache was created. BCI Enabled
relates to the -Xshareclasses:enableBCI
option (enabled by default) and Restrict Classpaths
relates to the -Xshareclasses:restrictClasspaths
option. Feature = cr
indicates that the cache is a 64-bit compressed references cache.
Line number information for classes in the cache is then shown with one of the following messages:
Cache contains only classes with line numbers
: VM line number processing was enabled for all the classes that were stored in this shared cache (the-Xlinenumbers
option is enabled by default). All classes in the cache contain line numbers if the original classes contained line number data.Cache contains only classes without line numbers
: The-Xnolinenumbers
option was used to disable VM line number processing for all the classes that were stored in this shared cache, so none of the classes contain line numbers.Cache contains classes with line numbers and classes without line numbers
: VM line number processing was enabled for some classes and disabled for others (the-Xnolinenumbers
option was specified when some of the classes were added to the cache).
The following summary data is displayed:
baseAddress
andendAddress
- The boundary addresses of the shared memory area that contains the classes. These addresses vary from run to run, depending on where the operating system allocates the memory.
allocation pointer
- The address where
ROMClass
data is currently being allocated in the cache.
- The address where
cache layer
- The layer number that the cache stats relate to.
cache size
andfree bytes
cache size
shows the total size of the shared memory area in bytesfree bytes
shows the free bytes that remain. The free space is not necessarily all available for storing new classes. The cache contains separate areas for different data, and can reserve space for AOT and JIT data, as shown by subsequent summary data.
softmx bytes
- The soft maximum size for the cache. For more information, see
-Xscmx
.
- The soft maximum size for the cache. For more information, see
ROMClass bytes
- The number of bytes of class data in the cache, which does not include data that is stored in the class debug area (see separate output for the class debug area).
AOT bytes
- The number of bytes of AOT-compiled code in the cache.
Reserved space for AOT bytes
- The number of bytes reserved for AOT-compiled code in the cache.
Maximum space for AOT bytes
- The maximum number of bytes of AOT-compiled code that can be stored in the cache.
JIT data bytes
- The number of bytes of JIT-related data stored in the cache.
Reserved space for JIT data bytes
- The number of bytes reserved for JIT-related data in the cache.
Maximum space for JIT data bytes
- The maximum number of bytes of JIT-related data that can be stored in the cache.
Zip cache bytes
- The number of zip entry cache bytes stored in the cache. On Java 11 and later, this value is zero unless a
jar
file is added to the boot classpath.
- The number of zip entry cache bytes stored in the cache. On Java 11 and later, this value is zero unless a
Startup hint bytes
- The number of bytes of data stored to describe startup hints.
Data bytes
- The number of bytes of non-class data stored by the VM.
Metadata bytes
- The number of bytes of data stored to describe the cached classes. Note: This field is available only in the top layer cache output or when a cache is not layered.
Metadata % used
- The proportion of metadata bytes to class bytes, which indicates how efficiently cache space is being used. The value shown does consider the
Class debug area size
.
- The proportion of metadata bytes to class bytes, which indicates how efficiently cache space is being used. The value shown does consider the
Class debug area size
- The size in bytes of the class debug area. This area is reserved to store
LineNumberTable
andLocalVariableTable
class attribute information.
- The size in bytes of the class debug area. This area is reserved to store
Class debug area bytes used
- The size in bytes of the Class Debug Area that contains data.
Class debug area % used
- The percentage of the Class Debug Area that contains data.
ROMClasses
- The number of classes in the cache. The cache stores
ROMClasses
(the class data itself, which is read-only) and information about the location from which the classes were loaded. This information is stored in different ways, depending on the JavaSharedClassHelper
API that was used to store the classes. For more information, see Support for custom class loaders.
- The number of classes in the cache. The cache stores
AOT methods
- Optionally,
ROMClass
methods can be compiled and the AOT code stored in the cache. TheAOT methods
information shows the total number of methods in the cache that have AOT code compiled for them. This number includes AOT code for stale classes.
- Optionally,
Classpaths
,URLs
, andTokens
- The number of class paths, URLs, and tokens in the cache. Classes stored from a
SharedClassURLClasspathHelper
are stored with a Classpath. Classes stored using aSharedClassURLHelper
are stored with a URL. Classes stored using aSharedClassTokenHelper
are stored with a Token. Most class loaders, including the bootstrap and application class loaders, use aSharedClassURLClasspathHelper
. The result is that it is most common to see class paths in the cache. - The number of Classpaths, URLs, and Tokens stored is determined by a number of factors. For example, every time an element of a Classpath is updated, such as when a
.jar
file is rebuilt, a new Classpath is added to the cache. Additionally, if partitions or modification contexts are used, they are associated with the Classpath, URL, or Token. A Classpath, URL, or Token is stored for each unique combination of partition and modification context. For more information, see Sharing modified bytecode andSharedClassHelper
cache partitions.
- The number of class paths, URLs, and tokens in the cache. Classes stored from a
Zip caches
- The number of
.zip
files that have entry caches stored in the shared cache. On Java 11 and later, this value is zero unless ajar
file is added to the boot classpath.
- The number of
Startup hints
- The number of startup hints stored in the cache. There can be a startup hint for each unique set of command line options used to start the VM.
Stale classes
- The number of classes that have been marked as "potentially stale" by the cache code, because of a VM or Java application update. See Class data sharing operations.
% Stale classes
- The percentage of classes in the cache that are stale.
Cache is XXX% full
- The percentage of the cache that is currently used. This line is displayed only if the soft maximum size is not set. This value is calculated as follows:
% Full = (('Cache Size' - 'Free Bytes') * 100) / ('Cache Size')
- The percentage of the cache that is currently used. This line is displayed only if the soft maximum size is not set. This value is calculated as follows:
Cache is XXX% soft full
- The percentage of the soft maximum size that is currently used. This line is displayed only if the soft maximum size is set. The
free bytes
in the cache statistics means the free bytes within the soft maximum limit. This value is calculated as follows:% soft Full = (('Soft max bytes' - 'Free Bytes') * 100) / ('Soft max bytes')
- For more information about the soft maximum size, see
-Xscmx
.
- The percentage of the soft maximum size that is currently used. This line is displayed only if the soft maximum size is set. The
Cache is accessible to current user
- Whether the current user can access the cache.
printAllStats
-Xshareclasses:printAllStats
-Xshareclasses:printAllStats,name=<cache_name>
Displays the contents of the cache in chronological order. You can use this output to see the history of updates that were made to the cache. For layered caches, some information is shown for the top layer cache only, and some is shown for all layers combined. To see information for the top layer cache only, use printTopLayerStats=all
.
Each entry in the output starts with a VM ID, so you can see which VM wrote the associated data. Here are example entries for various types of cache data, with explanations:
Class paths
The following example shows one class path with 4 entries:
1: 0x2234FA6C CLASSPATH
/myVM/Apps/application1.jar
/myVM/Apps/application2.jar
/myVM/Apps/application3.jar
/myVM/Apps/application4.jar
1
: the ID of the VM that wrote this data.0x2234FA6C
: the address where this data is stored.CLASSPATH
: the type of data that was written.
ROMClasses
This example shows an entry for a single ROMClass
:
1: 0x2234F7DC ROMCLASS: java/lang/Runnable at 0x213684A8
Index 1 in class path 0x2234FA6C
1
: the ID of the VM that wrote this data.0x2234F7DC
: the address where the metadata about the class is stored.ROMCLASS
: the type of data that was stored.java/lang/Runnable
: the name of the class.0x213684A8
: the address where the class was stored.Index 1
: the index in the class path where the class was loaded from.0x2234FA6C
: the address of the class path against which this class is stored.
Stale classes are marked with !STALE!
. Any partition or modification context that is used when the class is stored is also shown.
AOT methods
This example shows an entry for one AOT-compiled method:
1: 0x00007F841A800554 AOT: hashCode Signature: ()I Address: 0x00007F83F6859280
for ROMClass java/lang/Object at 0x00007F83F6859000.
1
: the ID of the VM that wrote this data.0x00007F841A800554
: the address where the data is stored.AOT
: the type of data that was stored.hashCode
: the method for which AOT-compiled code is stored.()I
: the signature of the ROM method.0x00007F83F6859280
: the ROM method address.java/lang/Object
: the class that contains the method.0x00007F83F6859000
: the address of the class that contains the method.
Stale methods are marked with !STALE!
.
URLs and tokens
The output for these data types has the same format as that for class paths, but with a single entry. A Token
is a string that is passed to the Java SharedClassHelper
API.
Zip entry caches
The following example shows 4 separate entries for zip entry caches:
1: 0x042FE07C ZIPCACHE: luni-kernel.jar_347075_1272300300_1 Address: 0x042FE094 Size: 7898
1: 0x042FA878 ZIPCACHE: luni.jar_598904_1272300546_1 Address: 0x042FA890 Size: 14195
1: 0x042F71F8 ZIPCACHE: nio.jar_405359_1272300546_1 Address: 0x042F7210 Size: 13808
1: 0x042F6D58 ZIPCACHE: annotation.jar_13417_1272300554_1 Address: 0x042F6D70 Size: 1023
1
: the ID of the VM that wrote this data.0x042FE07C
: the address where the metadata for the zip entry cache is stored.ZIPCACHE
: the type of data that was stored.luni-kernel.jar_347075_1272300300_1
: the name of the zip entry cache.0x042FE094
: the address where the data is stored.7898
: the size of the stored data, in bytes.
JIT data
Information about JIT data is shown in JITPROFILE
and JITHINT
entries. For example:
1: 0xD6290368 JITPROFILE: getKeyHash Signature: ()I Address: 0xD55118C0
for ROMClass java/util/Hashtable$Entry at 0xD5511640.
2: 0xD6283848 JITHINT: loadClass Signature: (Ljava/lang/String;)Ljava/lang/Class; Address: 0xD5558F98
for ROMClass com/ibm/oti/vm/BootstrapClassLoader at 0xD5558AE0.
Startup hints
Information about startup hints is shown in STARTUP HINTS KEY
and STARTUP HINTS DETAIL
. For example:
1: 0x000000002237C6E0 STARTUP HINTS KEY: -Xoptionsfile=jre\bin\compressedrefs\options.default -Xlockword:mode=default -Xjcl:jclse29 -Dcom.ibm.oti.vm.bootstrap.library.path=jre\bin\compressedrefs;jre\bin -Djava.home=jre -Djava.ext.dirs=jre\lib\ext -Duser.dir=bin -Djava.class.path=. -Dsun.java.launcher=SUN_STANDARD Address: 0x000000002237C700 Size: 96
STARTUP HINTS DETAIL Flags: 1 DATA1: 1677721 DATA2: 5033165
printTopLayerStats
Use this utility with a layered cache. This utility works in the same way as printStats
. By default, this utility shows information for the top layer cache. To view statistics for a specific layer, use the layer=<number>
option. For example, to show statistics for the second layer in a 2-layer cache, run printTopLayerStats,layer=1
.
Example output:
Current statistics for cache "Cache1":
Cache created with:
-Xnolinenumbers = false
BCI Enabled = true
Restrict Classpaths = false
Feature = cr
base address = 0x00007F234C054000
end address = 0x00007F234D038000
allocation pointer = 0x00007F234C054000
cache layer = 1
cache size = 16776608
softmx bytes = 16776608
free bytes = 15299372
Reserved space for AOT bytes = -1
Maximum space for AOT bytes = -1
Reserved space for JIT data bytes = -1
Maximum space for JIT data bytes = -1
Metadata bytes = 792
Metadata % used = 0%
Class debug area size = 1331200
Class debug area used bytes = 0
Class debug area % used = 0%
ROMClass bytes = 0
AOT bytes = 30520
JIT data bytes = 644
Zip cache bytes = 0
Startup hint bytes = 0
Data bytes = 114080
# ROMClasses = 0
# AOT Methods = 18
# Classpaths = 0
# URLs = 0
# Tokens = 0
# Zip caches = 0
# Startup hints = 0
# Stale classes = 0
% Stale classes = 0%
Cache is 8% full
Cache is accessible to current user = true
Using the trace facility
Trace output contains very detailed information that might be used by a VM developer to diagnose complex problems with class data sharing. Trace is configured by using the -Xtrace
option and the binary output of the trace facility must be processed by the Trace formatter into a human-readable format.
The trace component for class data sharing is j9shr
. Five levels of trace are available, starting from basic initialization and runtime information in level 1 up to the most detailed trace output in level 5.
To trace memory-mapped files, shared memory, and shared semaphores, include the j9prt
trace component.
To trace operations with Java Helper API methods, include the j9jcl
trace component.