Logging
With logging, Instana collects application and service logs automatically, correlate them with metrics and traces, and enhances traces or calls with extra messages.
- Collect application and service logs automatically. For more information, see Log messages.
- Correlate logs with metrics and traces by using the logs within the context of Kubernetes and containers. For more information, see Analyze Kubernetes logs.
- Enrich traces or calls with extra messages. For more information, see Instana tracing SDK.
- Integrate with a dedicated logging product. For more information, see Logging integrations.
Viewing logs
To view logging information, complete the following steps:
-
From the navigation menu in the Instana UI, click Analytics.
-
From the drop-down list, select Logs to view the Analytics logs dashboard.
Figure 1. Viewing log analytics
Analyzing logs
You can examine logs in detail with Unbounded analytics, where you can see all the log-related information, and slice and dice them to gain valuable insights during troubleshooting.
Filtering and grouping logs
You can filter and group logs by using the following approaches:
- Query builder
- Directly from within a log
- Filter sidebar
You can use each of the approaches individually, but the best results are achieved when you combine these approaches.
Using a query builder
To filter and group logs with the query builder, complete the following steps:
-
In the Filter field on the Analytics logs dashboard, click Add filter.
-
From the drop-down list, select the required filter option. A constructor appears on the filtering area with an equals operator as default.
You can filter by using any attribute of a log, such as level, message, stream, custom tags, snapshot, trace IDs, and exceptions. More filters that are related to technologies that are observed by Instana are also available. A quick search function can help to access your filtering options quickly.
-
Enter a relevant value in the input field of the equals operator. For standard values or related Instana entities, select values from the drop-down list.
Figure 2. Filtering and grouping -
To troubleshoot problems, filter by using the is present operator to make a quick query on exceptions. You can change the operator by clicking it.
Figure 3. Filtering and grouping -
To add more than one filter, an AND boolean operator appears as default.
-
To remove any of the filtering or the operators, click the filter or operator, and then click the x symbol.
Figure 4. Filtering and grouping -
To apply grouping, click Add group, and select one of the tags. A common use case for grouping is to find out which services or hosts are generating more logs, which can help scope down the search.
Figure 5. Filtering and grouping In this example, you can focus on the group with the hostname
worker0
by clicking the Focus in the group icon near the Number of logs data. Then, Instana adds this hostname as a filter and remove the host grouping.
Directly from within a log
When the log message contains custom tags, they are highlighted in gray.
To filter directly from within the log message, complete the following steps:
-
Identify custom tags in the log message. The custom tags are highlighted in gray.
-
To add the custom tags as filter from within the log message, click the highlighted custom tag, and select Add as filter.
In the following example, you can see the remote address
10.255.201.71
and the remote host10.255.201.71
are custom tags.Figure 6. Filtering from log message The following image shows the result of adding the custom tags as filter from within the log message.
Figure 7. Custom tag in query builder -
To view detailed information that is related to the log, expand the log. The log tag table is displayed.
-
Hover over each row to find contextual actions.
-
Use the following contextual actions within a log to execute the actions and display a log list with the source log.
-
Group by Tag icon: Displays a view where all the logs for the specific time frame and filters are grouped by that specific tag and the different values that it takes. This view is convenient to get hints of log volume for a specific tag.
To add a specific tag and value as a filter and enable endless scrolling, click the focus on this group icon. If you expand a group, only an overview is provided and endless scrolling is not available in this step.
Figure 8. Group by tag -
Add tag as filter icon: Adds the tag and its value as a filter. The default is added as
AND
.Figure 9. Add tag as filter In the following example, the stream is added to the former applied filter
WARN
.Figure 10. Result tag added as filter -
Copy to clipboard icon: Copies the tag value to the clipboard. The tag value can be used when you create a troubleshooting ticket for your team.
Figure 11. Copy to clipboard
-
-
To get information on an entity's health, check the health indicator before the entity name as shown in the following image. The color codes relate with the Smart Alerts color-code. Hover over it to see the number of issues for that entity, which gives a hint on the magnitude of a specific problem without changing the context.
Figure 12. Entities health
From side filter bar
The side filter bar provides a flexible way of filtering and grouping in combination with the query builder and directly from within a log.
-
To filter and group with the side filter bar, use the following tags:
- Log levels
- Stream
- Services
-
To view the number of logs for each different value of that tag even before you apply any filters, look at the number that is displayed near the category. In the following example, you can see that 30.3k logs are provided for the error tag.
-
To add a value to the filter, select the checkbox for the value.
-
To remove the value, clear the checkbox for the value.
You can also group by a filter directly from the icon near the main category title.

Sharing information with your team
You can work with logs that are needed to share information with other members of your team. By clicking the link icon on a specific log, you can share a short link with your team. When anyone else uses the link, Instana shows the same screen with the time frame, filters, and source log that are highlighted, open, and centered in the screen, to facilitate collaboration.

Handling multiline log messages and stack traces
Instana supports multiline log messages and stack traces from containers (Docker and containerd). Multiline log messages and stack traces are aggregated together as a single event, which simplifies troubleshooting issues.
Multiline log messages
Multiline log messages usually come in as separate single log messages. Instana identifies individual log messages that are part of a single multiline log message based on the timestamp. Starting with the most recent log message with a timestamp, each subsequent log message without a timestamp is considered to be a part of a multiline log message.
The following examples show multiline log messages:
Example 1
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v3.2.5)
Example 2
2023-10-08T08:49:17.645+00:00 | INFO | nstana-sensor-scheduler-thread-1 | PackageInstaller | com.instana.discovery-python - 1
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 223, in _main
status = self.run(options, args)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 180, in wrapper
return func(self, options, args)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 454, in run
options.target_dir, target_temp_dir, options.upgrade
File "/usr/local/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 461, in _handle_target_dir
ensure_dir(target_dir)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/utils/misc.py", line 117, in ensure_dir
os.makedirs(path)
File "/usr/lib64/python3.6/os.py", line 210, in makedirs
makedirs (head, mode, exist_ok)
File "/usr/lib64/python3.6/os.py", line 210, in makedirs
makedirs (head, mode, exist_ok)
File "/usr/lib64/python3.6/os.py", line 210, in makedirs
makedirs (head, mode, exist_ok)
[Previous line repeated 1 more time]
File "/usr/lib64/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/proc/1710'
2023-10-08T08:49:21.317+00:00 | INFO_ | d8f6c8dac361e3387053486ae4957736 | Docker| com.instana.sensor-docker – 1.1.5
The following image shows a Java™ multi-line log message example in the Instana logs:

Stack traces
Instana supports stack traces from Java™, Python, and Go applications. Instana considers only a subset of log formats that are defined as standard from these languages. Stack traces are identified and grouped by using regular expressions based on standard for each language.
Example for Java™ stack trace
// Example 1:
Exception in thread "main" java.lang.RuntimeException: A test exception
at com.stackify.stacktrace.StackTraceExample.methodB(StackTraceExample.java:13)
at com.stackify.stacktrace.StackTraceExample.methodA(StackTraceExample.java:9)
at com.stackify.stacktrace.StackTraceExample.main(StackTraceExample.java:5)
// Exception name: java.lang.RuntimeException
// Exception message: A test exception
// Stack trace: all lines
// Example 2:
com.instana.backend.common.exception.InstanaException: java.io.InterruptedIOException: Connection has been shut down
at com.instana.backend.common.client.AbstractHttpClient.execute(AbstractHttpClient.java:217)
at com.instana.groundskeeper.client.GroundskeeperClient.getHttpEndpointConfigs(GroundskeeperClient.java:360)
at com.instana.spanprocessing.stream.common.ReloadingConfigCache.loadInternal(ReloadingConfigCache.java:79)
at com.instana.spanprocessing.stream.common.ReloadingConfigCache$1.reload(ReloadingConfigCache.java:68)
at com.instana.spanprocessing.stream.common.ReloadingConfigCache$1.reload(ReloadingConfigCache.java:59)
at com.github.benmanes.caffeine.cache.CacheLoader.lambda$asyncReload$2(CacheLoader.java:190)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.io.InterruptedIOException: Connection has been shut down
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:342)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at com.instana.backend.common.client.AbstractHttpClient.executeInternalRequest(AbstractHttpClient.java:238)
at com.instana.backend.common.client.AbstractHttpClient.execute(AbstractHttpClient.java:212)
... 9 common frames omitted
Caused by: org.apache.http.impl.conn.ConnectionShutdownException: null
at org.apache.http.impl.conn.CPoolProxy.getValidConnection(CPoolProxy.java:77)
at org.apache.http.impl.conn.CPoolProxy.getSSLSession(CPoolProxy.java:137)
at org.apache.http.impl.client.DefaultUserTokenHandler.getUserToken(DefaultUserTokenHandler.java:82)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:326)
... 16 common frames omitted
// Exception name: com.instana.backend.common.exception.InstanaException
// Exception message: java.io.InterruptedIOException: Connection has been shut down
// Stack trace: all lines
// Example 3 (suppressed exceptions):
Exception in thread "main" java.lang.RuntimeException: I wanted to access this resource. Bad luck. Its dirty resource !!!
at DirtyResource.accessResource(DirtyResource.java:9)
at SuppressedExceptionDemoWithTryWithResource.main(SuppressedExceptionDemoWithTryWithResource.java:12)
Suppressed: java.lang.NullPointerException: Remember me. I am your worst nightmare !! I am Null pointer exception !!
at DirtyResource.close(DirtyResource.java:19)
at SuppressedExceptionDemoWithTryWithResource.main(SuppressedExceptionDemoWithTryWithResource.java:13)
Caused by: org.apache.http.impl.conn.ConnectionShutdownException: null
at org.apache.http.impl.conn.CPoolProxy.getValidConnection(CPoolProxy.java:77)
// Exception name: java.lang.RuntimeException
// Exception message: I wanted to access this resource. Bad luck. Its dirty resource !!!
// Stack trace: all lines
The following image shows a Java™ stack trace example from Instana logs:

Example for Python stack trace
# Example 1:
Traceback (most recent call last):
File "example.py", line 5, in <module>
say('Micheal')
File "example.py", line 3, in say
print('Hello, ' + nam)
NameError: name 'nam' is not defined
# Exception type: NameError
# Exception message: name 'nam' is not defined
# Stack trace: all lines
# Example 2 (handling an exception raises another exception, with logger used to log):
ERROR:root:Everything is broken
Traceback (most recent call last):
File "/tmp/spam2.py", line 13, in throws
1 / 0
ZeroDivisionError: division by zero
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/spam2.py", line 18, in <module>
f1()
File "/tmp/spam2.py", line 2, in f1
f2()
File "/tmp/spam2.py", line 5, in f2
f3()
File "/tmp/spam2.py", line 8, in f3
throws()
File "/tmp/spam2.py", line 15, in throws
raise Exception("boom")
Exception: boom
# Exception type: ZeroDivisionError
# Exception message: division by zero
# Stack trace: all lines
# Example 3 (source not available):
Traceback (most recent call last):
File "example.py", line 5, in <module>
File "example.py", line 3, in say
NameError: name 'nam' is not defined
# Exception type: NameError
# Exception message: name 'nam' is not defined
# Stack trace: all lines
Example for Golang stack trace
Panic: Want something
goroutine 1 [running]:
main.Example(0x2080c3f50, 0x2, 0x4, 0x425c0, 0x5, 0xa)
/Users/bill/Spaces/Go/Projects/src/github.com/goinaction/code/temp/main.go:9 +0x64
main.main()
/Users/bill/Spaces/Go/Projects/src/github.com/goinaction/code/temp/main.go:5 +0x85
goroutine 2 [runnable]:
runtime.forcegchelper()
/Users/bill/go/src/runtime/proc.go:90
runtime.goexit()
/Users/bill/go/src/runtime/asm_amd64.s:2232 +0x1
goroutine 3 [runnable]:
runtime.bgsweep()
/Users/bill/go/src/runtime/mgc0.go:82
runtime.goexit()
/Users/bill/go/src/runtime/asm_amd64.s:2232 +0x1
# Exception type: null
# Exception message: Want something
# Stack trace: all lines
Managing logs
You can extend the duration for which logs are retained in Instana by configuring the log retention period settings. Further, you can also manage your log volume reports.
Extended log retention
Extended log retention is an add-on feature for Instana logging. With extended log retention, you can retain logs for 30, 60, and 90 days, compared to the 7 days of default retention time of Instana core logging feature.
For more information about changing the log retention period, see Configuring extended log retention.
Log volume reports
Log volume reports are an add-on feature for Instana logging. With log volume reports, you can see the amount of ingested log message volume that is associated with a specific time frame with monthly granularity.
For more information about viewing log volume reports, see Viewing log volume reports.
Integrating logs
Instana supports integrating logs from the external logging providers to enable a fluid workflow between Instana and these logging providers. For more information, see Integrating logs.
Deleting logs
If Personal Identifiable Information (PII) is revealed in log messages, you can delete all the logs.
To delete all logs, complete the following steps:
-
In the Instana UI, click Settings > Team settings > Log Management > Delete logs.
In this Settings page, you can see a brief explanation of this function, a button to initiate log deletion, a summary table that lists previous log deletions, and the following information:
- The exact date that the deletion was initiated.
- The reason for deletion.
- The number of logs that were deleted.
- The initiator who deleted the logs.
- The status of the deletion.
Figure 17. Delete logs table -
Click Delete logs.
-
Enter a reason for deletion, and type
LOGS
in the designated field to confirm your decision. -
Click Delete logs.
You can close the dialog because it does not interrupt the process. When the deletion is complete, the outcome (success or failure) is displayed in the dialog, and a new entry is made in the summary table.
