OpenTelemetry tracing in Db2 for z/OS
With OpenTelemetry (OTel), you can generate, collect, and export telemetry data to help site reliability engineers (SRE) and application-focused operations team members triage application incidents involving z/OS applications and databases, to quickly engage the correct z/OS product SMEs.
OpenTelemetry (OTel) is an industry-standard observability framework that consists of a collection of APIs, SDKs, and tools that site reliability engineers (SRE) can use OTel to instrument, generate, collect, and export telemetry data to help observe the behavior of their application, across multiple software components.
OTel traces track the flow of an application through a system, enabling you to observe the behavior of applications, across multiple software components. The flow normally incorporates several different application components, potentially running in different environments. Traces can provide insights into the behavior of each individual application and make it easier to understand where an application might be encountering issues, or performance problems.
Db2 for z/OS supports inbound OTel trace context propagation, and the Db2 activity can be reported in a trace span record for each unique Db2 unit of work. OTel trace context propagation in Db2 for z/OS is based on the Trace Context - W3C Recommendation 23 November 2021.
Activating Db2 accounting trace classes 1 and 2 is optional when you enable OTel trace propagation. However, if these trace classes are not active, the OTel trace spans that Db2 generates omit any CPU, zIIP, and elapsed time statistics.
What are OpenTelemetry traces?
An OTel trace represents the end-to-end flow of control of a distributed application through a system, and consists of multiple spans. A span contains detailed information about a specific segment of the flow, including the duration, start, and end times. Each OTel-enabled component that is involved in the processing of a transaction contributes to the trace by emitting a span.
The first span serves as the parent or root span, and more spans be nested under it. The root span captures the overall journey of a request from start to finish. Child spans are generated by using the current active span as the parent, creating relationships between the spans and thereby enabling the overall anatomy of a request to be understood.
OpenTelemetry traces in Db2 for z/OS
Db2 for z/OS supports inbound OTel trace context propagation, with Db2 activity reported by an OTel trace span record for each unique Db2 unit of work. OTel trace context propagation refers to the ability of a trace context to be passed between components of an application. If Db2 receives OTel trace context from an incoming request, it propagates this context through the use of the traceparent to the unit of work that processes the incoming request.
OpenTelemetry trace propagation
- Manual propagation: the application writes code to propagate the context between components by associating it with each request that it makes
- Automatic propagation: the environment automatically propagates the context without the application having to write code, or even be aware that trace context is being propagated
Db2 supports manual or automatic inbound trace context propagation for the following Db2 for z/OS application environments when they are also configured for OTel context propagation.
- Db2 for z/OS native REST services
- JDBC Type-4/Db2 JCC type-4 Client driver
- CICS-Db2 Attachment Facility
- CICS Liberty-JDBC type-2 (RRSAF)
- IMS-Db2 Attachment (ESAF)
- IMS-Db2 Java Adapter-JDBC type-2 (RRSAF)
- z/OS Connect-Db2 Rest service (HTTP header)
Db2 for z/OS currently supports inbound trace context propagation only. That is, Db2 propagates an OTel trace context for a unit of work only when it receives a trace context in an incoming request from an application environment that is also configured for OTel context propagation.
Example of trace context propagation from IBM MQ to CICS to Db2
The following diagram illustrates a context propagation scenario. As context is propagated from IBM® MQ to CICS to Db2, each product updates the parent ID to reflect the new span ID, allowing the trace to be reconstructed. This enables the span IDs to build up the parent-child relationships in a trace.
In the example, when IBM MQ sends a message to CICS, it includes its own span ID as the parent ID, and CICS then uses this parent ID as its own parent ID when sending a request to Db2. This way, the trace can be followed across multiple products and systems, allowing for end-to-end visibility of the distributed transaction. For more information about OTel trace propagation for these products, see OpenTelemetry tracing with IBM MQ on z/OS and CICS and OpenTelemetry.