Troubleshooting
You might encounter the following issues with Instana monitoring.
- Unsupported JVM 8
- Incompatible agent detected
- Dynamic attachment issues
- Network visibility issues
- Attach tools missing
- Attach directory not found
- Insufficient disk space for storing temporary files
- Agent cannot bind to the default port
- Container attachment failed
- IBM J9 class sharing enabled
- Generic JVM attachment issue
Unsupported JVM 8
Monitoring issue type: java_8_unmonitored_version
This monitoring issue indicates that your Java application is running on an unsupported Java 8 hotspot virtual machine with a version earlier than 1.8.0_40. Instana does not monitor Java applications that are running on such a hotspot virtual machine due to severe bugs in the latter that lead to incompatibilities and crashes.
To resolve the issue, update the JVM to a more recent version, preferably a JDK 8 252 or later, or to another Java long-term support release.
For more information about the JVMs supported by Instana, see the JVM support documentation.
Incompatible agent detected
Monitoring issue type: jvm_incompatible_agent_detected
This monitoring issue indicates that the Instana agent detected an incompatible agent that is attached to the JVM. To prevent issues from occurring with the JVM, the Instana agent does not collect tracing data from the application that is running inside the JVM. This interruption might result in missing data that is related to application perspectives, Services, and Endpoints, and spurious alerts about drop in calls to application perspectives, Services, and Endpoints.
To resolve the issue, remove the incompatible agent from your JVM.
Dynamic attachment issues
Monitoring issue type: jvm_attach_socket
This monitoring issue indicates that the attachment facility of the JVM is stuck, which is known to happen occasionally due to long-standing bugs of the JVM. As a result, the JVM is unable to connect to a communication socket used during dynamic attach.
To fix this issue, restart the JVM.
Network visibility issues
Monitoring issue type: jvm_attach_network
This issue indicates the failure of JVM instrumentation due to network visibility issues from the JVM and the host agent. To prevent this failure, make sure that the host agent is reachable from the JVM as described in the Network Requirements documentation. In containerized environments, check the network policies and configurations to prevent this issue.
Attach tools missing
Monitoring issue type: jvm_attach_tools
This monitoring issue indicates that the JDK that is used during dynamic attach does not include the required attach code. For more information, see the Java 8 or earlier and Java 9 or later sections.
Attach directory not found
Monitoring issue type: jvm_attach_directory
This issue indicates the failure of dynamic attachment because the attachment directory was not found.
To resolve this issue, do one of the following actions:
- Make sure that the JVM process of the host agent and the target JVM process are started with the same settings for environment variables that control the location of temporary directories
- Instruct the target JVM process to place working files for dynamic attachment where the agent expects them by default. If IBM J9 is used on a POSIX-compliant system, include the following command-line argument when the target JVM process is started:
-Dcom.ibm.tools.attach.directory=/tmp/.com_ibm_tools_attach
Insufficient disk space for storing temporary files
Monitoring issue type: insufficient_disk_space_for_storing_temp_files
During runtime, Instana host agent creates temporary JAR files in the $TMP/.instana
directory. These JAR files are required for the JVM monitoring. This issue indicates that the necessary JAR files cannot be loaded on the host system
or inside application containers that the agent attempts to monitor due to the storage capacity limit.
You can mitigate this issue by setting a flag IN_MEMORY_CLASSLOADER=true
as a system property or as an environment variable. After you enable this option, the required storage for JVM monitoring is reduced to 3.5 MB. For more information,
see Environment Variables - Host Agent.
Agent cannot bind to the default port
Monitoring issue type:
This monitoring issue indicates that the agent cannot bind to the default port 42699. As a result, various sensors fail and monitoring in full capacity might not be possible. For more information, see Network requirements.
To fix the issue, ensure that the port 42699 is available, and restart the agent to recover full monitoring.
Container attachment failed
Monitoring issue type: jvm_attach_container_command
This monitoring issue indicates that the command that is used to dynamically attach to a containerized JVM did not succeed.
For more information, see the exit code from the specific event description.
IBM J9 class sharing enabled
Monitoring issue type: ibm_jvm_class_sharing_enabled
This monitoring issue indicates that the class-sharing feature is enabled for this JVM. Therefore, you cannot attach and instrument. For more information about how to resolve this issue, see IBM J9 limitations.
Generic JVM attachment issue
Monitoring issue type: jvm_attach_generic
This monitoring issue indicates that an unknown issue is attached to the JVM. Therefore, this process cannot be traced nor any JVM metrics captured. The Agent logs must provide more details about the reason for the attachment failure. To resolve this issue, file a ticket and include the logs.