[MQ 9.4.3 Jun 2025][z/OS]

Span emission

If a message contains trace context then the OTELTRAC attribute on the queue manager, or on the queue, controls whether IBM® MQ emits a span when a message is sent to, or destructively received from, a queue.

Spans can be emitted both when application, and queue manager processes, send messages to or receive messages from a queue. For example, spans can be generated for:
  • CICS®, IMS, RRS, JMS, or batch applications.
  • Applications connected over a server-connection channel.
  • The channel initiator moving messages over a sender-receiver channel, or via a cluster.
  • The intra-group queuing agent.
  • JMS delivery delay.
  • When report messages are generated.

The OTELTRAC attribute

The OTELTRAC attribute can be configured on the queue manager, or on local, model or remote queues. By default all local, model or remote queues are configured with OTELTRAC(QMGR) which means that changing the OTELTRAC on the queue manager can enable, or disable trace for all queues.

OTELTRAC on local, model or remote queues can be set to:
  • QMGR: the default, use the value of OTELTRAC from the queue manager.
  • OFF: do not emit spans, even if messages contain trace context.
  • ON: emit spans if messages contain trace context.
OTELTRAC on the queue manager can be set to:
  • OFF: do not emit spans, even if messages contain trace context.
  • ON: emit spans if messages contain trace context.
  • NONE: the queue manager never emits spans, even if OTELTRAC(ON) is specified on individual queues. Setting OTELTRAC(ON) on a queue results in the fast put to waiting getter optimization being disabled.
  • OTELTRAC(QMGR)

How span emission works - when sending a message

When a message is sent to a queue which has OTELTRAC(ON) configured, then the queue manager checks the message to see if it contains trace context. If it does, and the message was successfully put to the queue then the queue manager generates a producer span containing information about the sending application, and the message that was sent. For a list of attributes that can be in a span, see Span attributes.

Spans for messages sent out of sync point are emitted by the queue manager immediately. Spans for messages that are sent in sync point are emitted when the current unit of work commits, or rolls back.

How span emission works - when receiving a message

A queue manager only emits a span when a message is received, if a span was emitted when the message was sent. A queue manager does not check the value of OTELTRAC on a queue when receiving messages from it. The criteria for emitting a span are as follows:
  1. A span was emitted when the message was sent to a queue.
  2. OTELTRAC(NONE) is not set on the queue manager.

If both these criteria are met, then the queue manager generates a consumer span containing information about the receiving application, and the message that was received. For a list of attributes that can be in a span, see Span attributes.

Spans for messages received out of sync point are emitted by the queue manager immediately. Spans for messages that are received in sync point are emitted when the current unit of work commits, or rolls back.

Distributed queuing and clustering considerations

OTELTRAC can be configured on remote queue definitions, and transmission queues. The queue manager applies queue name resolution rules which affect whether OTELTRAC comes from a remote queue definition or a transmission queue:
  • If a remote queue definition is used for queue name resolution then the value of OTELTRAC on that queue applies, regardless of the value of OTELTRAC on the transmission queue.
  • Otherwise the value of OTELTRAC on the transmission queue applies.

OTELTRAC is not a clustered attribute, to emit spans when messages are sent to a remote clustered queue requires that OTELTRAC(ON) is specified on the relevant cluster transmission queue.

Example configuration one

By default, the queue manager does not emit any spans even if messages contain trace context:

  • The queue manager defaults to OTELTRAC(OFF).
  • Queues default to OTELTRAC(OFF) for certain system queues, or OTELTRAC(QMGR) for user defined queues or certain system queues.

Example configuration two

Altering the queue manager to OTELTRAC(ON) means that any messages sent to the queue manager with trace context result in the queue manager emitting spans.

Example configuration three

If a local queue called APPLICATION.QUEUE.ONE was configured with OTELTRAC(ON), then any message sent to, or received from, that queue with trace context generates a span.

Example configuration four

In Figure 1, APP1 and APP2 are exchanging messages between a pair of z/OS queue managers connected by a sender-receiver channel. Setting OTELTRAC(ON) on both the remote and local definition of APP2.REQUEST means that four spans are emitted if APP1 sends a message with trace context:
  • A producer span is emitted by MQA when APP1 sends the message to APP2.REQUEST.
  • A consumer span is emitted by MQA when the channel initiator takes the message off the transmission queue prior to sending it over the channel.
  • A producer span is emitted by MQB when the channel initiator receives the message and puts it to APP2.REQUEST.
  • A consumer span is emitted by MQB when APP2 receives the message.
Figure 1. Example configuration 4
This figure is described in the surrounding text.

Example configuration five

In Figure 2, APP3 and APP4 are exchanging messages between a distributed queue manager and a z/OS queue manager connected by a sender-receiver channel. The distributed queue manager has the IBM MQ OpenTelemetry tracing service (see OpenTelemetry tracing with IBM MQ on AIX, Linux, and Windows) installed and configured so that spans are emitted when messages are sent to and received from APP4.REQUEST. On the z/OS queue manager, setting OTELTRAC(ON) on APP4.REQUEST means that four spans are emitted when APP3 sends a message. The IBM MQ OpenTelemetry tracing service automatically adds trace context when messages are sent to queues that are enabled for OpenTelemetry trace.
  • A producer span is emitted by MQC when APP3 sends the message to APP4.REQUEST.
  • A consumer span is emitted by MQC when the MCA process takes the message off the transmission queue prior to sending it over the channel.
  • A producer span is emitted by MQD when the channel initiator receives the message and puts it to APP4.REQUEST.
  • A consumer span is emitted by MQD when APP4 receives the message.
Figure 2. Example configuration 5
This figure is described in the surrounding text.

When spans are not emitted

IBM MQ prioritizes the integrity of the queue manager over the emission of span data. If a span cannot be emitted for any reason, the span data is discarded. SMF data, either in the message manager, or per queue is collected to indicate the reasons for discarding span data.

Span data might be discarded for one of the following reasons:

  • z/OS has not been configured to collect span data for IBM MQ. Tracked in SMF via QMSTOSFR and QMSTOTSR.
  • z/OS has been configured to collect span data for IBM MQ, but has not got capacity to collect span data. Tracked in SMF via QMSTOSFC and QMSTOTSC.
  • IBM MQ has reached the limit of how much span data can be stored before it is emitted. Tracked in SMF via QMSTOSFS.
  • A transaction has generated more than 1000 spans. Tracked in SMF via QMSTOSPL, QMSTOSCL, QQSTOSPL and QQSTOSCL.
  • The queue manager is shutting down with MODE FORCE or RESTART, span data that has not been emitted is discarded.
  • The queue manager is shutting down with MODE QUIESCE, after all applications have disconnected the queue manager waits for a maximum of one second for remaining span data to be emitted. Any remaining data is discarded.