Known issues and limitations

Review the known issues and limitations for IBM Instana observability.

Known issues

Incompatibility of CrowdStrike Falcon sidecar with the Instana agent

The Instana agent container is incompatible with the CrowdStrike Falcon sidecar. You can use Falcon only as a DaemonSet.

Memory issue with Instana agent on Windows Server 2016

On Windows Server 2016 hosts, a library used by the Instana agent gradually consumes memory without freeing it correctly, which might cause increased memory usage on the affected Windows systems. To keep memory consumption under control, the Instana agents are restarted automatically. The automatic restart feature is to prevent the memory issue from impacting automatic agent updates. All the Instana agents that are deployed on Windows Server 2016 restart every 7 days between 05:00 and 05:30 AM, by default. This feature is available since Instana agent bundle 1.1.702. The agent bundle version can be checked in the Instana agent startup logs.

To change the default restart behavior, set the following environment variables before you initiate the Instana agent.

Environment Variable Name Default Note
INSTANA_AUTO_RESTART_WINDOWS_SERVER_2016_ENABLED true The flag to enable or disable automatic restart.
INSTANA_AUTO_RESTART_WINDOWS_SERVER_2016_RESTART_TIME 5:15 The time at which the Instana agent restarts.
INSTANA_AUTO_RESTART_WINDOWS_SERVER_2016_JITTER_IN_MIN 30 The jitter in minutes to set the time range in which the agent restarts. The jitter is to prevent the Instana agents from restarting simultaneously. The restart time is randomly selected within the specified range. To modify the randomization of restart time, set a value greater than 1. The time range for the restart time is calculated by using the formula INSTANA_AUTO_RESTART_WINDOWS_SERVER_2016_RESTART_TIME - (INSTANA_AUTO_RESTART_WINDOWS_SERVER_2016_JITTER_IN_MIN/2) and INSTANA_AUTO_RESTART_WINDOWS_SERVER_2016_RESTART_TIME + (INSTANA_AUTO_RESTART_WINDOWS_SERVER_2016_JITTER_IN_MIN/2). For example, if the restart time is set to 5.15 AM with a jitter of 30 minutes, the restart occurs between 05:00 and 05:30 AM.
INSTANA_AUTO_RESTART_WINDOWS_SERVER_2016_RESTART_MIN_INTERVAL_IN_DAYS 7 The interval before the next restart takes place. Set a value greater than zero.

Known limitations

Analyzing traces and calls

When calls are grouped by log.level or log.message, the special tag group Tag not present is not displayed in the same way as other tags.

For more information about this feature, see Analyzing traces and calls.

Filtering with Dynamic Focus

For more information about this feature, see Filtering with Dynamic Focus

Syntax constraints

  • White space is not allowed between the ! and the query to be negated. Use !<query> or NOT <query> instead.
  • When multiple of the same entity.application.name, entity.service.name, entity.endpoint.name Dynamic Focus keywords are used together, they can only be joined by the OR operator. E.g. the queries entity.application.name:foo AND entity.application.name:bar, entity.application.name:foo entity.application.name:bar are not supported. The query entity.application.name:foo OR entity.application.name:bar is supported.
  • When multiple of different entity.application.name, entity.service.name, entity.endpoint.name Dynamic Focus keywords are used together, they can only be joined by the AND operator. E.g. the query entity.application.name:foo OR entity.service.name:bar is not supported, whereas the query entity.application.name:foo AND entity.service.name:bar is supported.
  • Queries containing a negation of application, service or endpoint name are not supported. Consequently, neither NOT entity.application.name:foo, !entity.service.name:bar nor NOT entity.endpoint.name:foobar are valid. However, filtering out events of an application, service or endpoint is still possible by using NOT event.entity.label:foo instead.
  • Dynamic Focus queries entity.application.name:<term>, entity.service.name:<term> and entity.endpoint.name:<term> are evaluated using the string contains logic, e.g. Dynamic Focus Query entity.application.name:shop would also match entity.application.name:shop-frontend.
  • Dynamic Focus query keywords entity.application.name, entity.service.name and entity.endpoint.name are only supported in boolean queries (see boolean logic constraints above), term queries, phrase queries and prefix queries.

Query limitations

  • Dynamic Focus query on event.text containing hyphens might return unexpected results. For e.g. event.text:"foo-bar-random" would also return events containing just foo or bar or random due to tokenization of the Lucene analyzer.
  • Entities as follows a Docker container, such as Host or Availability Zone, cannot be selected when scoping on most Kubernetes keywords. As an example, the query entity.kubernetes.deployment.name:my-K8s-deployment AND entity.selfType:host does not return any host. In contrast, the query entity.kubernetes.deployment.name:my-K8s-deployment AND entity.selfType:docker returns all Docker containers related to the deployment. However, there are a few Kubernetes keywords that can be used to select Host entites, nameley entity.kubernetes.cluster.* and entity.kubernetes.node.*. For instance, the query entity.kubernetes.cluster.label:my-K8s-cluster AND entity.selfType:host returns all related hosts of the cluster.

Infrastructure metrics

  • Entities are limited to show at most 3000 metrics in the Instana UI for performance reasons. Dashboards with more metrics than that limit will have missing metrics.
  • Negative metric values are not displayed.

Monitoring IBM App Connect Enterprise (ACE)

  • If HTTPS is enabled for REST API interface, you need to specify keystore and keystorePassword parameters in <agent_install_dir>/etc/instana/configuration.yaml. For keystore type, only JKS or P12 is supported.

For more information about this feature, see Monitoring IBM App Connect Enterprise (ACE)

Monitoring IBM API Connect application

  • DataPower Tracing supports only the API gateway and doesn't support other services such as the multiprotocol gateway and web service proxy.

For more information about this feature, see Monitoring IBM API Connect application

Jaeger tracing

  • The tracing data collected from Jaeger will not be correlated with the tracing data collected via AutoTrace, resulting in separate traces even if the systems traced by Jaeger and Instana AutoTrace, respectively, were directly interacting with one another.

  • Since the Jaeger tracing data does not convey a notion of which process is sending them to the host agent, Instana correlates those traces to the host underpinning the host agent. This prevents the association with the process, and therefore also container and platform hierarchy (for example, Kubernetes pod, namespace, and cluster).

  • Jaeger has no notion of user monitoring (although this may eventually change with the adoption of W3C TraceContext.md); as such, beacons collected with Instana Website Monitoring will not be correlated with backend traces collected from Jaeger.

  • The host agent supports the collection of Jaeger traces only over HTTP. UDP, which is the protocol used if you configure the JAEGER_AGENT_HOST and JAEGER_AGENT_PORT environment variables, is not supported.

For more information about this feature, see Jaeger

OpenTelemetry tracing

  • OpenTelemetry links are not supported.
  • Mixing Instana default context propagator with context propagators other than W3C Trace Context is not supported. W3C Trace Context is the default context propagator of OpenTelemetry.

For more information about this feature, see OpenTelemetry Tracing

Zipkin tracing

  • The tracing data collected from Zipkin will not be correlated with the tracing data collected via AutoTrace, resulting in separate traces even if the systems traced by Zipkin and Instana AutoTrace, respectively, were directly interacting with one another.

  • Zipkin has no notion of user monitoring (although this may eventually change with the adoption of W3C TraceContext.md); as such, beacons collected with Instana Website Monitoring will not be correlated with backend traces collected from Zipkin.

  • The host agent supports the collection of Zipkin traces only over HTTP, which corresponds to the COLLECTOR_HTTP_ENABLED setting of Zipkin.

For more information about this feature, see Zipkin

Monitoring Containerd

The CPU Total Normalized and Memory Working Set Usage % metrics are available only on systems that are using Control Group v1 (cgroupv1).

Monitoring Amazon MSK

The Amazon MSK sensor supports monitoring only for the provisioned MSK cluster type. The serverless cluster type is not supported.

ALPN support

The Instana agent and backend support the Application-Layer Protocol Negotiation (ALPN) connection when the following requirements are met:

  • Netty 4.1.49.Final or later
  • Java Development Kit (JDK) 1.8.0_251 or later

Addressing high CPU usage

High CPU usage is observed in hotspot-based agents with agent container 1.263.3 and earlier. To resolve this issue, reinstall the agent. The reserved code cache size is now increased to support high CPU usage. For the agents that run on a host environment, remove the agent and then reinstall it.

If you are using Kubernetes (k8s) or OpenShift Container Platform (OCP), complete the following steps:

  1. Uninstall agent:

    helm uninstall instana-agent

  2. Install agent:

    To install the Instana agent, use Helm Chart on Kubernetes or OpenShift Container Platform.

Synthetic test attributes

Jest as a scriptType is not supported for Browser Simple, API Script, and API Simple tests on UI and API. Jest as a scriptType can be used only with Browser Script tests.

Some Synthetic test attributes are not supported in the Instana UI. To set these attributes, use the Open API or the synctl tool. The unsupported attributes in the Instana UI are listed in the following table:

Test attribute Browser Script Browser Simple API Simple API Script
browser ☒ ☒ N/A N/A
recordVideo ☒ ☒ N/A N/A
expectExists N/A N/A ☒ N/A
expectNotEmpty N/A N/A ☒ N/A
markSyntheticCall ☒ ☒ ☒ ☒
retries ☒ ☒ ☒ ☒
retryInterval ☒ ☒ ☒ ☒
timeout ☒ ☒ ☒ ☒
ScriptType ☒ N/A N/A ☒
HEAD (Operation) N/A N/A ☒ N/A
DELETE (Operation) N/A N/A ☒ N/A
OPTIONS (Operation) N/A N/A ☒ N/A
PATCH (Operation) N/A N/A ☒ N/A
POST (Operation) N/A N/A ☒ N/A
PUT (Operation) N/A N/A ☒ N/A

The symbol ☒ indicates that the attribute is not supported in the Instana UI.

N/A indicates that the attribute is not applicable for the test type.