Configuring monitoring for data capture

Determine which data you want to record, and configure your message flows to emit this data, by using monitoring events. Learn about the differences between how you configure event sources for data capture and for business-level monitoring.

Before you begin

Create and configure a database, and specify the IBM® Integration Bus runtime properties that are required for recording data. See Recording data.

About this task

Data capture is based on a publish/subscribe model. To specify the data that you want to record to a database, you configure your message flows to emit event messages. These messages contain the data that you want to record. The messages are published to a topic that you specify in a DataCaptureSource configurable service. This configurable service also contains the name of the DataCaptureStore configurable service that represents the database to which the data is to be recorded. The integration node subscribes to the topic and routes the published messages to the database.

The mechanism that is used for configuring message flows to emit event messages for data capture is also used for business-level monitoring. This mechanism is summarized in the concept topic Configuring monitoring for message flows, and is described further on in this topic. Because data capture and business-level monitoring can share event sources and monitoring events, you must ensure that the configuration for one does not inadvertently affect the other. Before you begin to configure monitoring for data capture, you need to be aware of some specific considerations that relate to the use of the monitoring mechanism for capturing data for record and replay.

  • Typically, the type of data that you want to capture for record and replay differs from the type of data that is useful for business monitoring. When you are capturing data for record and replay, you probably want to capture part or all of the message bitstream data. However, if you are monitoring transactions or business processes, you might want to include only particular message fields that represent significant thresholds. See Monitoring scenarios. The requirements for data capture and for business-level monitoring might therefore conflict with one another.
  • When you view your recorded data in the web administration Data Viewer, you might see one or more errors in the Exception column. You can retrieve the exception data for these errors only if you included the exception list ($ExceptionList) when you created your monitoring events. However, if the monitoring events are exported in order to create a monitoring model for IBM Business Monitor, the inclusion of the exception list results in a warning. See Creating a monitor model for IBM Business Monitor V7 or later.
  • You can record messages from many different providers and sources. However, you can replay messages to WebSphere MQ destinations only.

    When you configure a monitoring event, you can choose to include all of the message bitstream data, the headers only, or the message body only. When an integration node replays a message, it needs to include a WebSphere MQ message header. It handles this requirement in one of the following ways:

    • If the captured data includes a WebSphere MQ header, the integration node uses the existing WebSphere MQ message header.
    • If the captured data does not include a WebSphere MQ header, the integration node generates a WebSphere MQ header. All other headers are appended to the message payload.

Procedure

To configure monitoring on a message flow to emit events for capture, use one of the following methods.
  • Configure and enable event sources, and activate monitoring for the message flow, by completing the steps in Configuring monitoring for message flows.

    You do not need to subscribe to the monitoring topic; the integration node manages the subscription to the topic specified in the DataCaptureSource configurable service.

  • Configure monitoring event sources by using the sample monitoring profile that is provided at install_dir/server/sample/RecordReplay/basicMonitoringProfile.xml.

    The sample monitoring profile works for a single input node. You must update the profile to replace NODENAME with the name of your input node. To enable this monitoring profile, run the mqsicreateconfigurableservice, mqsichangeproperties, and mqsichangeflowmonitoring commands against the integration node to which you deployed the flow. For details about running these commands, see Configuring monitoring event sources by using a monitoring profile.

The subscription to the monitoring events is durable, which results in messages being put to the queue specified in the queueName property of the DataCaptureStore configurable service. Messages can be put to this queue even when the integration node is not running. For more information about how you can monitor the queue depth and tune data capture, see Tuning data capture.

What to do next

Review the steps that you completed for recording data; see Recording data. Now you are ready to view the data that you recorded; see Viewing recorded data.