Monitoring basics

You can configure message flows to emit events, which can then be read and used by other applications for transaction monitoring and auditing. Monitoring events are also used by the built-in business transaction monitoring capability that is provided by IBM® App Connect Enterprise, and by the record and replay capability.

The following sections introduce the concepts of monitoring:

For more information about business transaction monitoring, see Monitoring business transactions and Business transaction monitoring overview.

Monitoring events

A monitoring event is a message that is published when a message flow that is configured with a monitoring event completes the source action on which the event is configured. Monitoring events can be configured on two source types:
  • Transaction, including Transaction Started, Transaction Completed.
  • Terminal, when a message flow node terminal propagates.
Monitoring event messages that are published IBM MQ or MQTT are in the form of XML or JSON documents that conform to the monitoring event schema. Monitoring event messages that are sent to a Logstash input in an Elasticsearch, Logstash, and Kibana (ELK) stack are emitted in JSON format.
Each event contains the following information:
  • The source of the event.
  • The name of the event.
  • A sequence number and creation time.
  • A correlation ID for events that are emitted by the same transaction or unit of work.
Additionally, a monitoring event can contain the following items:
  • Application data that is extracted from the message.
  • Part or all of the message bit stream.
You can create a message model for the monitoring event by completing the following steps in the IBM App Connect Enterprise Toolkit:
  1. Click File.
  2. Click New.
  3. Select Message Model.
  4. Select IBM supplied.
  5. Click Next.
  6. Select Create an XML schema file by importing an IBM supplied definition.
  7. Click Next.
  8. Select an existing Application or Library, or create a new one.
  9. Select IBM App Connect Enterprise Monitoring Event from the list of IBM supplied messages.
  10. Click Finish.

You can also find the monitoring event schema at: install_location/server/sample/Monitoring/MonitoringEventV2.xsd or install_location/server/sample/Monitoring/MonitoringEventV1.json.

Event sources

A message flow can emit two kinds of event:
Transaction events
Transaction events are emitted from input nodes only.
Terminal events
Terminal events are emitted from any terminal of any node, including input nodes.

For message processing nodes (for example, Computenode, Tracenode, Filternode), IBM App Connect Enterprise generates monitoring events for the output terminal regardless of whether the output terminal is wired to downstream nodes or not.

Output nodes (for example, FileOutput node, MQOutput node, HTTPReply node), terminate the transaction and do not propagate the message further down the output terminal if they are not wired further. If no message is propagated to the output terminal, the event of type 'out' terminal is not emitted by these nodes.

An individual message flow can emit transaction events, terminal events, or both kinds of event. You can configure, enable, and disable both types of event by using the monitoring properties of the message flow, or by using a monitoring profile.

An event source address identifies an event source in a message flow.

Terminal events can be emitted from any node in a message flow. Therefore, these events can be used as an alternative to dedicated event-emitting nodes or subflows, such as the example supplied in SupportPac IA9V.

Event sources emit events only if monitoring is activated for the message flow.

Transaction events

Each input node in a message flow contains three events sources, in addition to terminal events.
Event source Event source address Description
Transaction start Nodename.transaction.Start The event is emitted when the message is read from the transport.
Transaction end Nodename.transaction.End The event is emitted when IBM App Connect Enterprise completes all processing of the message.
Transaction rollback Nodename.transaction.Rollback The event is emitted instead of transaction end if the message flow issues an exception that is not caught and processed in the flow.

Events are emitted subject to the evaluation of the eventFilter expression, as described in Event output options.

If a message flow handles its own exceptions, a transaction end event is issued instead of a transaction rollback event because the flow takes control of the error and terminates normally. In this case, if you need to distinguish errors, you can configure terminal events at appropriate nodes in the flow.

Terminal events

Any terminal in a message flow can be an event source. If the event source is active, it emits an event each time a message passes through the terminal, subject to the evaluation of the eventFilter expression. This expression is described in Event output options.

Event output options

When you configure an event source, you can define a filter to control whether the event is emitted. You can tailor event emission to your business requirements by filtering out events that do not match a set of rules. For example, you might decide to emit events only for transactions over a minimum amount:
$Body/StockTrade/Details/Value > 10000
The use of a filter can reduce the number of events that are emitted, and reduce the workload on your monitoring application.
You might filter out the transaction.Start and transaction.End events that are emitted by the MQInput node for IBM MQ Backout transaction events after a backout threshold is reached. An event monitoring application can then collect appropriate data.
3 >= $Root/MQMD/BackoutCount

Publishing monitoring events

Events are published to a topic, where they can be read by multiple subscribers. They can be published to IBM MQ or MQTT to enable subscribers to receive them. The topic name has the following structure:
  • For events published on the IBM MQ pub/sub broker:
    • XML:
      $SYS/Broker/integrationNodeName/Monitoring/integrationServerName/flow_name
    • JSON:
      $SYS/Broker/integrationNodeName/MonitoringEvents/JSON/integrationServerName/flow_name
  • For events published on the MQTT pub/sub broker:
    • XML: 
      IBM/IntegrationBus/integrationNodeName/Monitoring/integrationServerName/flow_name
    • JSON:
      IBM/IntegrationBus/integrationNodeName/MonitoringEvents/JSON/integrationServerName/flow_name

Events can also be written to the file system in rotatable log files. The default path where the monitoring events are written to is the workDir/Common/MonitoringEvents subdirectory.

Monitoring events can also be published to a Logstash input plug-in in an Elasticsearch, Logstash, and Kibana (ELK) stack, so that you can display the reported information in a Kibana dashboard. For more information, see Reporting logs and monitoring events to a Logstash input in an ELK stack.

For more information about configuring the publication of event messages, see Configuring the publication of event messages.

The hierarchical structure allows subscribers to filter the events that they receive. One subscriber can receive events from all message flows in the integration node, while another receives only the events from a single integration server. For more information about publishing and subscribing to events, see Publish/subscribe overview.

You decide whether events participate in transactions when you configure a monitoring event source:
  • If you want an event to be emitted only if the message flow transaction commits, configure the event source to coordinate the events with the message flow transaction.
  • If you want an event to be emitted regardless of whether the message flow transaction commits or rolls back, configure the event source to emit events out of sync point. Such events are available immediately.
  • A group of events can be emitted together regardless of whether the message flow transaction commits or rolls back. If you want this option, you must configure the event source to emit events in an independent unit of work.
Note:
The monitoring event is not emitted if a parser exception occurs in the following situations:
  • Constructing the monitoring event from the Event Name, Event Filter, or Event Payload.
  • Including the bit stream in the payload.
In these situations, the parser exception is thrown again and the message is rolled back.