Troubleshooting and debugging

Collect cluster information and debugging logs to troubleshoot issues.

Collecting information

Create an archive file with information about your cluster. You can use the information in the file to troubleshoot issues, or share the file with the support team.

The archive file collects the following information:

  • Container logs
  • Resource manifests (in YAML format)
  • stanctl logs
  • System information that includes memory, CPU, and CPU usage
  • Disk mounts and their usage
  • Open files (allocated, free, and maximum)

Use the following command to create the archive file:

stanctl debug

After you run the command, you see the following messages. When you see Done! in the messages, it means that your archive file is ready.

./stanctl debug
⠼ Streaming container logs  [26s] ✓
⠸ Gathering resource manifests  [27s] ✓
⠋ Gathering stanctl config files  [0s] ✓
⠋ Gathering system information  [0s] ✓
⠹ Creating tar file  [0s] ✓

----------------------
Done!
Debug package -> debug_20231027111737
Compressed debug package -> debug_20231027111737.tar.gz
----------------------

Troubleshooting

Resolve these issues.

Instana agent is not displayed in the UI

After you delete the Instana agent that was configured for remote monitoring and install the Instana agent for self monitoring, the agent might not be displayed on the Instana UI.

The agent might be trying to connect to the remote Instana backend instead of the local Instana backend.

To resolve this issue, install the agent and specify the backend endpoint host and an agent key:

stanctl agent apply --agent-cluster-name <cluster-name> --agent-endpoint-host acceptor.instana-core --agent-endpoint-port 8600 --agent-zone-name <zone-name> --agent-key <agent-key-of-local-backend>

Kafka pods show CrashLoopBackOff status

Kafka pods do not restart after a shutdown of the Instana backend host. You might see a CrashLoopBackOff status of the Kafka pods.

To resolve the issue, restart the Instana backend.

  1. Shut down the backend.
    stanctl down
  2. Start the backend.
    stanctl up

After the backend is restarted, check the status of Kafka pods.

kubectl get pods --all-namespaces | grep kafka

The Kafka pod status should show as Running.

Scheduled Synthetic tests are not running after Instana backup and restore

After Instana backend and agent data are restored, the scheduled Synthetic tests are not running.

To resolve this issue, restart the synthetic-pop-controller pod on the cluster where it is installed.

Host agent cannot connect to the Instana backend on SLES hosts

After you install the host agent on the local host on SUSE Linux Enterprise Server (SLES) 15 SP5 hosts for self monitoring, the agent does not automatically connect to the Instana backend.

You must use the agent external URL to connect to the backend as a remote host.

Use the following command:

stanctl agent apply --agent-endpoint-host agent-acceptor.<base_domain> --agent-endpoint-port 8443

Standard Edition installation on RHEL 9.3 fails

Red Hat® Enterprise Linux® 9.3 uses iptables 1.8.8.

If you are installing Standard Edition on RHEL 9.3, the installation might fail due to iptables 1.8.8.

To work around the issue, upgrade your host to RHEl 9.4, which also upgrades the iptables to version 1.8.10.

Contacting support

If you are unable to resolve the issue, contact Instana support. Provide the archive file that you created to the support team.

For more information, see Opening a support case with IBM Support.

  • For product, select Instana Observability.
  • Specify that the issue is related to the Self-Hosted Standard Edition.