Generating events in WebSphere Message Broker for transaction monitoring and auditing

This article shows you how to configure and generate monitoring events in a WebSphere Message Broker message flow. Monitoring events are very useful built-in features for transaction monitoring and auditing, and this article describes them in detail.

Shenfu (Mike) Fan (sfan@us.ibm.com), Certified IT Architect and IT Specialist, IBM

Shenfu (Mike) Fan is a Senior Managing Consultant with IBM Software Services for WebSphere (ISSW). He is primarily focused on architectural design, development, and implementation of enterprise application integration solutions using WebSphere products that include WebSphere Message Broker, WebSphere MQ, WebSphere Business Events, WebSphere Service Registry and Repository, WebSphere ESB, and others.


developerWorks Contributing author
        level

11 November 2009

Also available in Chinese

Introduction

Monitoring events are a new feature in IBM® WebSphere® Message Broker V6.1.0.3 or later (hereafter called Message Broker). This feature lets you configure a message flow to emit monitoring events without any coding. As the configured events occur in the message flow, XML documents are generated and published as pre-defined topics by the broker on which the message flow is deployed. These events can be read and consumed by applications such as WebSphere Business Monitor to track and report message flow transaction-level performance data, and to support transaction monitoring, transaction auditing, and business process monitoring, as shown in Figure 1:

Figure 1. Conceptual view of monitoring events
Figure 1. Conceptual view of monitoring events

Readers should have some experience with Message Broker and know how to create and deploy message flows.

What is a monitoring event?

A message flow can be configured at node level to emit monitoring events. A monitoring event is represented as a XML document that conforms to the monitoring event schema. Each event always contains the following information:

  • Source of the event (eventSourceAddress)
  • Name of the event (eventName)
  • Creation time (creationTime)
  • Correlation ID for events emitted by the same transaction or unit of work (localTransactionId, parentTransactionId, and globalTransactionId)
  • Details of the message flow (messageFlowData)

Optionally, a monitoring event can also contain the following information:

  • Application data extracted from the message (applicationData)
  • Part or all of the message bit stream (bitstream)

Message Broker event schema

The predefined Message Broker event schema WMBEvent.xsd defines the structure of the monitoring events. To retrieve the WMBEvent.xsd file:

  1. In the Application Development perspective of the Message Broker Toolkit, right-click a message set under a message set project (create a new message set project if one does not exist) and select New => Message Definition File From => IBM Supplied Message.
  2. On the window that opens, scroll down the list of IBM supplied messages and select Message Broker Monitoring Event and then click Finish. You should see the file WMBEvent.mxsd under the package com.ibm.www.xmlns.prod.websphere.messagebroker._610.monitoring.event.
  3. Right-click the message set again and select Generate => XML Schemas.
  4. In Generate XML Schemas window, choose an external directory to which you want to export the schema file.
  5. Click Finish. A zip file containing the file WMBEvent.xsd is created in that directory.

Event types

You can configure a message flow to emit two types of events: transaction events and terminal events. There are three types of transaction events: start, end, and rollback. The transaction events are emitted only from input nodes such as MQInput and HTTPInput. Terminal events are emitted from any terminal of any node.

Event output

The monitoring event feature leverages the publish/subscribe (pub/sub) mechanism of Message Broker. Event messages are published to the specific topics, which can be registered by subscribing applications. The form of the topic name is:

$SYS/Broker/<brokerName>/Monitoring/<executionGroupName>/<flowName>

The topic hierarchical structure enables subscribing applications to filter the events. For example, one application can receive events from all message flows deployed on all execution groups in a broker, while another can receive only the events from all message flows in a single execution group.

Configuring a message flow to generate monitoring events

As mentioned previously, you can configure monitoring events to emit without any coding. This section provides an example to show you how to configure the monitoring events in a message flow and how to consume these events. In order to run this example, you must deploy the message flow Message Broker V6.1.0.3 or later. The broker name BRK1 and the execution group name default are used in all the commands in this article. Here are the high-level steps in this section:

  1. Creating a message flow
  2. Setting up monitoring properties
  3. Deploying the message flow
  4. Enabling the event sources
  5. Activating the events
  6. Subscribing to the events

1. Creating a message flow

A pre-built message flow as shown in Figure 2 below is provided in the Message Broker Project Interchange file MonitoringEvents_PI.zip, which you can download below , along with other artifacts such as message sets, BAR files, monitoring event schemas, and test messages. This simple flow, which does not include an the error-handling routine, is used to demonstrate how to configure and generate monitoring events. It receives a XML message in an input queue to request a service for temperature conversion. This service is provided on the Web: Visual DataFlex Web service for temperature conversion. Based on the content of the input message, a service is called for conversion from Celsius to Fahrenheit using the HTTP Request node HTTP_Request_C2F, or Fahrenheit to Celsius using the node HTTP_Request_F2C. The response from the service request is returned in an output queue. For more information, see Creating a message flow in the Message Broker information center.

Figure 2. Message flow MonitoringEvents_MF
Figure 2. Message flow MonitoringEvents_MF

The node connection is described in Table 1:

Table 1. Details of node connection on the message flow
NodeNameNodeTypeTerminalToNodeNameNodeTypeTerminal
QINMQInputOut=>Build_HTTP_Request_MsgComputeIn
Build_HTTP_Request_MsgComputeOut=>HTTP_Request_C2FHTTPRequestIn
Build_HTTP_Request_MsgComputeOut1=>HTTP_Request_F2CHTTPRequestIn
HTTP_Request_C2FHTTPRequestOut=>Build_MQ_Response_MsgComputeIn
HTTP_Request_F2CHTTPRequestOut=>Build_MQ_Response_MsgComputeIn
Build_MQ_Response_MsgComputeOut=>QOUTMQOutputIn

2. Setting up monitoring events

You can set up the monitoring events using either monitoring properties or a monitoring profile. In this section, you configure the monitoring events of the message flow using monitoring properties, as shown below. Use of the monitoring profile is described in the section Configuring event sources using a monitoring profile. The monitoring events to be configured include all transaction events, Start, End, and Rollback on MQInput node QIN, and terminal events In and Out on each of the HTTP Request nodes.

  1. In the Broker Application Development perspective of the Message Broker Toolkit, open the flow MonitoringEvents_MF in the Message Flow Editor.
  2. On the Monitoring tab on the properties of the MQInput node QIN, click Add to configure the event sources, as shown in Figure 3:
    Figure 3. Monitoring properties of MQInput node QIN
    Figure 3. Monitoring properties of MQInput node QIN
  3. Select Transaction Start from the Event source list. This event is triggered whenever a message passes through the flow's input node. Change the literal event name from the default QIN.TransactionStart to FlowStart, as shown in Figure 4. Of course, a character field in the message tree in the message assembly could be chosen as the event name.
    Figure 4. Add a transaction event on MQInput node QIN
    Figure 4. Add a transaction event on MQInput node QIN
  4. Click Add for the Event Payload to optionally obtain data from a simple or complex field in the input message tree. As shown in Figure 5 below, after you click Edit for data location, The XPath Expression Builder page opens, where you can add data type from the input message. In order to be seen in the Data Type Viewer, the input message TemperatureConversion in this case must be built in a message set (MRM). If it is not, then you need to manually type the value into the XPath Expression.
    Figure 5. Add data types from the input message
    Figure 5. Add data types from the input message
  5. As shown in Figure 6 below, for demonstration purpose, both the complex field TemperatureConversion from the message body, and the simple field MsgID from the MQMD header are selected using an XPath expression. The payload bitstream data is also included in the monitoring event. Keep in mind that adding a large payload and bitstream may reduce performance.
    Figure 6. Event payload data for transaction event FlowStart
    Figure 6. Event payload data for transaction event FlowStart
  6. Select the Correlation tab to define the correlators, which are used by a monitoring application to match multiple events emitted in the flow transaction. A local transaction correlator links the events emitted by the single invocation of a message flow. A parent transaction correlator links the events from a message flow to a parent message flow or to an external application. A global transaction correlator links events from a message flow to one or more related message flows or external applications. Accept the default value Automatic for all of the correlators, which causes Message Broker to generate a unique correlator 36 characters long that is the same for all events occurring in one transaction at each level.
    Figure 7. Event correlation
    Figure 7. Event correlation
  7. Select the Sequence tab. The Event Sequence is used by the monitoring application to reorder event messages, since events are published at an unpredictable order. In Message Broker V6.1.0.3, the event sequence is set using creation time in the format yyyy-mm-ddThh:mm:ss.fff.
    Figure 8. Event sequence
    Figure 8. Event sequence
  8. Repeat Steps 3 through 7 above to add the transaction events Transaction end and Transaction rollback, as well as the terminal events in and out for both the HTTP_Request_C2F and HTTP_Request_F2C nodes. Only MsgID from the MQMD header is added to payload data for these events. The event setting is summarized in the following table:
    NodeNameEventNameEventSourcePayloadBitstreamTransactionCorrelatorSequence
    QINFlowStartTransaction.start$Root/MQMD/MsgId $Body/Temperature ConversionAll(base64Binary)Local - AutomaticParent - AutomaticGlobal - AutomaticCreation time
    QINFlowEndTransaction.end$Root/MQMD/MsgId-Local - AutomaticParent - AutomaticGlobal - AutomaticCreation time
    QINFlowRollbackTransaction.rollback$Root/MQMD/MsgId-Local - AutomaticParent - AutomaticGlobal - AutomaticCreation time
    HTTP_Request_C2FIn_C2FIn.terminal$Root/MQMD/MsgId-Local - AutomaticParent - AutomaticGlobal - AutomaticCreation time
    HTTP_Request_C2FOut_C2FOut.terminal$Root/MQMD/MsgId-Local - AutomaticParent - AutomaticGlobal - AutomaticCreation time
    HTTP_Request_F2CIn_F2CIn.terminal$Root/MQMD/MsgId-Local - AutomaticParent - AutomaticGlobal - AutomaticCreation time
    HTTP_Request_F2COut_F2COut.terminal$Root/MQMD/MsgId-Local - AutomaticParent - AutomaticGlobal - AutomaticCreation time
  9. Click the monitoring tab in the Message Flow properties window in the Message Flow Editor. All configured events are listed as shown in Figure 9 below, by default. these events are all enabled:
    Figure 9. List of configured monitoring events
    Figure 9. List of configured monitoring events

3. Deploying the message flow

Deploy the message flow into the execution group default on broker BRK1 using the broker archive (BAR) file. Run all of the commands described in the following sections on this broker. Changes are required if you download the sample solution below and deploy it into the environment with different names for the execution group and broker. For more information, see Deploying message flows to an execution group on a broker in the Message Broker information center.

4. Enabling the event sources

As described above, by default, all of the configured events in a message flow are enabled. You can enable or disable these events from the Message Flow Editor using the Message Broker Toolkit. If the message flow is already deployed to an execution group, you must redeploy the flow for the monitoring event changes to take effect. You can use the command mqsichangeflowmonitoring to enable or disable the events without having to redeploy the message flow. The changes take effect immediately at runtime. For example,

mqsichangeflowmonitoring BRK1 -e default -f MonitoringEvents_MF -s 
"QIN.transaction.start,QIN.transaction.end,QIN.transaction.rollback" -i enable
mqsichangeflowmonitoring BRK1 -e default -f MonitoringEvents_MF -s 
"HTTP_Request_C2F.terminal.in,HTTP_Request_C2F.terminal.out" -i disable

5. Activating the events

After the event sources are configured, run the following command in order to activate the monitoring engine and emit the event messages:

mqsichangeflowmonitoring BRK1 -e default -f MonitoringEvents_MF -c active

Conversely, to inactivate the message flow monitoring process, run the same command with the parameter -c inactive.

Event activation status does not change after you stop and restart the broker. To check the status of the monitoring, including the node enablement state, use the following command:

mqsireportflowmonitoring BRK1 -e default -f MonitoringEvents_MF -a

6. Subscribing to the events

When the configured monitoring events occur, the event messages are published with the topics in the following format:

$SYS/Broker/<BrokerName>/Monitoring/<ExecutionGroupName>/<MessageFlowName>

In this case:

$SYS/Broker/BRK1/Monitoring/default/MonitoringEvents_MF

The topic is case-sensitive. Before receiving event messages, the topics need to be subscribed to, which you can do using the rfhutil tool and a message flow.

Using the rfhutil tool

You can download the free Message Broker display, test, and performance tool rfhutil from the IBM Message Broker SupportPac site and use it to subscribe to the events. To subscribe to a topic using the tool.

  1. Open the rfhutil tool and click on the Pub/Sub tab.
  2. Select Sub for Request Type
  3. Type $SYS/Broker/BRK1/Monitoring/default/MonitoringEvents_MF in the Topic(s) field
  4. Type WMBQM in the Subscription Queue Manager field
  5. Type QSUB in the Subscription Queue field
  6. Click Write Msg:
    Figure 10. Subscribing the event topic using rfhutil tool
    Figure 10. Subscribing the event topic using rfhutil tool

You can use the rfhutil tool to unsubscribe from a topic using a procedure similar to subscription, except that you select Unsub instead of Sub for the request Type in Step 2 above.

Using a message flow

You can create a simple message flow, as shown in Figure 11 below, to subscribe to or unsubscribe from a topic programmatically:

Figure 11. Message flow for subscribing topics
Figure 11. Message flow for subscribing topics

An input XML message that contains subscribing or unsubscribing data as shown below is sent to the queue QSUB. Use the Compute node to build a pub/sub command under the MQRFH2 header based on the input message. The command is sent to the system queue SYSTEM.BROKER.CONTROL.QUEUE, from which the Message Broker pub/sub engine processes the command messages to generate topic subscriptions.

Listing 1. A subscribing message
<SubMsg>
<Command>RegSub</Command>
<Topic>$SYS/Broker/BRK1/Monitoring/default/MonitoringEvents_MF</Topic>
<QMgrName>WMBQM</QMgrName>
<QName>QEVENT</QName>
</SubMsg>

To unsubscribe from a topic, change the element Command in the above input XML message from RegSub to DeregSub. To check the topic subscription, click Subscriptions on the Domains panel at the left bottom corner of the Broker Administration perspective in the Message Broker Toolkit. Clicking Query to display a list of topics, including the one being subscribed to:

Figure 12. List of topic subscriptions
Figure 12. List of topic subscriptions

Generating monitoring events

The following scenarios are used to emit monitoring events that are configured on the message flow. In order to run tests, make sure that the machine where the Message Broker server is located has an internet connection, and that Visual DataFlex Web service for temperature conversions is available.

  • A successful transaction using a correct input message as shown in Listing 2 or 3 below.
  • An uncompleted transaction with an exception caused by an invalid input message is shown in Listing 4 below.
Listing 2. Test message C2F.xml
<?xml version="1.0" encoding="UTF-8"?>
<TemperatureConversion>
  <Input>
  <Unit>Celsius</Unit>
  <Degree>28.5</Degree>
  </Input>
  <Output>
  <Unit></Unit>
  <Degree></Degree>
  </Output>
</TemperatureConversion>
Listing 3. Test message F2C.xml
<?xml version="1.0" encoding="UTF-8"?>
<TemperatureConversion>
  <Input>
  <Unit>Fahrenheit</Unit>
  <Degree>80</Degree>
  </Input>
  <Output>
  <Unit></Unit>
  <Degree></Degree>
  </Output>
</TemperatureConversion>
Listing 4. Test message Invalid.xml
<?xml version="1.0" encoding="UTF-8"?>
<TemperatureConversion>
  <Input>
  <Unit>Wrong</Unit>
  <Degree>80</Degree>
  </Input>
  <Output>
  <Unit></Unit>
  <Degree></Degree>
  </Output>
</TemperatureConversion>

Use the rfhutil tool to send a message into the input queue QIN.

  • Open rfhutil and select the queue manager, such as WMBQM, and the queue QIN.
  • Click Open File to import a test message file such as C2F.xml. You should see the content of the test message when you select the Data tab.
  • Select the Main tab and click Write Q. The test message is sent to the queue QIN to invoke the message flow.
  • Check the messages in the generated QEVENT for the monitoring events.

Generating monitoring events with a successful transaction

Send the test message for temperature conversion from Celsius to Fahrenheit to the input queue QIN. As expected, two transaction events FlowStart and FlowEnd, and two terminal events In_C2F and Out_C2F, are emitted in the transaction. These event messages are received in the subscribing queue QEVENT. The local transaction ID is the same in each of the messages, and so is the parent and global transaction IDs. The payload data and its bitstream are included in the XML message for the event FlowStart, as shown in Listing 5:

Listing 5. Event message for FlowStart
<wmb:event xmlns:wmb="http://www.ibm.com/xmlns/prod/websphere/
  messagebroker/6.1.0/monitoring/event">
  <wmb:eventPointData>
    <wmb:eventData wmb:productVersion="6103" wmb:eventSchemaVersion="6.1.0.3" 
      wmb:eventSourceAddress="QIN.transaction.Start">
      <wmb:eventIdentity wmb:eventName="FlowStart"/>
      <wmb:eventSequence wmb:creationTime="2009-10-08T02:16:14.500"/>
      <wmb:eventCorrelation wmb:localTransactionId="ae3f4f63-1ad6-45e7-b84d-0fc9f2f35454" 
        wmb:parentTransactionId="6bf33d39-0e92-4a7f-872e-96c01ea6c65d" 
        wmb:globalTransactionId="4fabdaba-0964-4b94-93f2-14cd5f4a62b8"/>
    <wmb:eventData/>
    <wmb:messageFlowData>
      <wmb:broker wmb:name="BRK1" wmb:UUID="2d750f67-2301-0000-0080-96dbba41cd3f"/>
      <wmb:executionGroup wmb:name="default" 
        wmb:UUID="12feee26-2401-0000-0080-9bc38c4d3910"/>
      <wmb:messageFlow wmb:uniqueFlowName="BRK1.default.MonitoringEvents_MF" 
        wmb:name="MonitoringEvents_MF" wmb:UUID="8c284827-2401-0000-0080-c826dabb2ad0" 
        wmb:threadId="9316"/>
      <wmb:node wmb:nodeLabel="QIN" wmb:nodeType="ComIbmMQInputNode" wmb:detail="QIN"/>  
    <wmb:messageFlowData/>   
  </wmb:eventPointData>
  <wmb:applicationData>
    <wmb:simpleContent wmb:name="MsgId" 
      wmb:value="414d5120574d42514d20202020202020be56ca4a202fe112" 
      wmb:dataType="hexBinary"/>
  </wmb:applicationData>
  <wmb:bitstreamData>
    <wmb:bitstream wmb:encoding="base64Binary">TUQgIAIAAAAAAAAACAAAAP////8AAAAAIgIAALUBAAA
      gICAgICAgIAAAAAAAAAAAQU1RIFdNQlFNICAgICAgIL5WykogL+ESAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
      AAAAAACAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFdNQlFNICAgICA
      gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHNmYW4gICAgICAgIBYBBRUAAACEYU++63/
      PV2Bwonj0AQAAAAAAAAAAAAALICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICALAAAAQzpcTXlCaW5
      ccmZodXRpbFxyZmh1dGlsLmV4ZTIwMDkxMDA4MDIxNjE0NTEgICAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
      AAQAAAAAAAAAAAAAA/////zw/eG1sIHZlcnNpb249IjEuMCIgZW5jb2Rpbmc9IlVURi04Ij8+DQo8VGVtcGV
      yYXR1cmVDb252ZXJzaW9uPg0KICA8SW5wdXQ+DQogICAgPFVuaXQ+Q2Vsc2l1czwvVW5pdD4NCiAgICA8RGV
      ncmVlPjI4LjU8L0RlZ3JlZT4NCiAgPC9JbnB1dD4NCiAgPE91dHB1dD4NCiAgICA8VW5pdD48L1VuaXQ+DQo
      gICAgPERlZ3JlZT48L0RlZ3JlZT4NCiAgPC9PdXRwdXQ+DQogIDxFcnJvcj4NCiAgICA8Q29kZT48L0NvZGU
      +DQogICAgPERlc2M+PC9EZXNjPg0KICA8L0Vycm9yPg0KPC9UZW1wZXJhdHVyZUNvbnZlcnNpb24+DQo=/>
  </wmb:bitstreamData>
</wmb:event>
Listing 6. Event message for FlowEnd
<wmb:event xmlns:wmb="http://www.ibm.com/xmlns/prod/websphere/
  messagebroker/6.1.0/monitoring/event">
  <wmb:eventPointData>
    <wmb:eventData wmb:productVersion="6103" wmb:eventSchemaVersion="6.1.0.3" 
      wmb:eventSourceAddress="QIN.transaction.End">
      <wmb:eventIdentity wmb:eventName="FlowEnd"/>
      <wmb:eventSequence wmb:creationTime="2009-10-08T02:16:15.237"/>
      <wmb:eventCorrelation wmb:localTransactionId="ae3f4f63-1ad6-45e7-b84d-0fc9f2f35454" 
        wmb:parentTransactionId="6bf33d39-0e92-4a7f-872e-96c01ea6c65d" 
        wmb:globalTransactionId="4fabdaba-0964-4b94-93f2-14cd5f4a62b8"/>
    <wmb:eventData/>
    <wmb:messageFlowData>
      <wmb:broker wmb:name="BRK1" wmb:UUID="2d750f67-2301-0000-0080-96dbba41cd3f"/>
      <wmb:executionGroup wmb:name="default" 
        wmb:UUID="12feee26-2401-0000-0080-9bc38c4d3910"/>
      <wmb:messageFlow wmb:uniqueFlowName="BRK1.default.MonitoringEvents_MF" 
        wmb:name="MonitoringEvents_MF" wmb:UUID="8c284827-2401-0000-0080-c826dabb2ad0" 
        wmb:threadId="8152"/>
      <wmb:node wmb:nodeLabel="QIN" wmb:nodeType="ComIbmMQInputNode" wmb:detail="QIN"/>  
    <wmb:messageFlowData/>   
  </wmb:eventPointData>
  <wmb:applicationData>
    <wmb:simpleContent wmb:name="MsgId" 
      wmb:value="414d5120574d42514d20202020202020be56ca4a202fe112" 
      wmb:dataType="hexBinary"/>
  </wmb:applicationData>
</wmb:event>
Listing 7. Event message for In_C2F
<wmb:event xmlns:wmb="http://www.ibm.com/xmlns/prod/websphere/
  messagebroker/6.1.0/monitoring/event">
  <wmb:eventPointData>
    <wmb:eventData wmb:productVersion="6103" wmb:eventSchemaVersion="6.1.0.3" 
      wmb:eventSourceAddress="HTTP_Request_C2F.terminal.in">
      <wmb:eventIdentity wmb:eventName="In_C2F"/>
      <wmb:eventSequence wmb:creationTime="2009-10-08T02:16:14.609"/>
      <wmb:eventCorrelation wmb:localTransactionId="ae3f4f63-1ad6-45e7-b84d-0fc9f2f35454" 
        wmb:parentTransactionId="6bf33d39-0e92-4a7f-872e-96c01ea6c65d" 
        wmb:globalTransactionId="4fabdaba-0964-4b94-93f2-14cd5f4a62b8"/>
    <wmb:eventData/>
    <wmb:messageFlowData>
      <wmb:broker wmb:name="BRK1" wmb:UUID="2d750f67-2301-0000-0080-96dbba41cd3f"/>
      <wmb:executionGroup wmb:name="default" 
        wmb:UUID="12feee26-2401-0000-0080-9bc38c4d3910"/>
      <wmb:messageFlow wmb:uniqueFlowName="BRK1.default.MonitoringEvents_MF" 
        wmb:name="MonitoringEvents_MF" wmb:UUID="8c284827-2401-0000-0080-c826dabb2ad0" 
        wmb:threadId="10024"/>
      <wmb:node wmb:nodeLabel="HTTP_Request_C2F" wmb:nodeType="ComIbmWSRequestNode" 
        wmb:terminal="in"/>  
    <wmb:messageFlowData/> 
    <wmb:applicationData>
      <wmb:simpleContent wmb:name="MsgId" 
        wmb:value="414d5120574d42514d20202020202020be56ca4a202fe112" 
        wmb:dataType="hexBinary"/>
    </wmb:applicationData>  
  </wmb:eventPointData>
</wmb:event>
Listing 8. Event message for Out_C2F
<wmb:event xmlns:wmb="http://www.ibm.com/xmlns/prod/websphere/
  messagebroker/6.1.0/monitoring/event">
  <wmb:eventPointData>
    <wmb:eventData wmb:productVersion="6103" wmb:eventSchemaVersion="6.1.0.3" 
      wmb:eventSourceAddress="HTTP_Request_C2F.terminal.out">
      <wmb:eventIdentity wmb:eventName="Out_C2F"/>
      <wmb:eventSequence wmb:creationTime="2009-10-08T02:16:15.235"/>
      <wmb:eventCorrelation wmb:localTransactionId="ae3f4f63-1ad6-45e7-b84d-0fc9f2f35454" 
        wmb:parentTransactionId="6bf33d39-0e92-4a7f-872e-96c01ea6c65d" 
        wmb:globalTransactionId="4fabdaba-0964-4b94-93f2-14cd5f4a62b8"/>
    <wmb:eventData/>
    <wmb:messageFlowData>
      <wmb:broker wmb:name="BRK1" wmb:UUID="2d750f67-2301-0000-0080-96dbba41cd3f"/>
      <wmb:executionGroup wmb:name="default" 
        wmb:UUID="12feee26-2401-0000-0080-9bc38c4d3910"/>
      <wmb:messageFlow wmb:uniqueFlowName="BRK1.default.MonitoringEvents_MF" 
        wmb:name="MonitoringEvents_MF" wmb:UUID="8c284827-2401-0000-0080-c826dabb2ad0" 
        wmb:threadId="5864"/>
      <wmb:node wmb:nodeLabel="HTTP_Request_C2F" wmb:nodeType="ComIbmWSRequestNode" 
        wmb:terminal="out"/>  
    <wmb:messageFlowData/>
    <wmb:applicationData>
      <wmb:simpleContent wmb:name="MsgId" 
        wmb:value="414d5120574d42514d20202020202020be56ca4a202fe112" 
        wmb:dataType="hexBinary"/>
    </wmb:applicationData>    
  </wmb:eventPointData>
</wmb:event>

Send the test message for the Fahrenheit to Celsius temperature conversion to the input queue QIN. Two transaction events FlowStart and FlowEnd and two terminal events In_F2C and Out_F2C are emitted in the transaction, which generates four messages in the subscribing queue QEVENT.

Generate monitoring events with an exception

Send the invalid test message into the input queue QIN of the flow in order to generate an exception and an unsuccessful transaction. Only the transaction events FlowStart, FlowRollback, and FlowEnd are emitted, with three corresponding XML messages received in the QEVENT queue. Since the exception is thrown at the Compute node Build_HTTP_Request_Msg before passing through the HTTPRequest node, the terminal events are not invoked. Events do not participate in transactions -- in other words, if a transaction rollback occurs in a message flow, then the emitted events are not rolled back and remain published. That is why the messages for the transaction events FlowStart and FlowEnd are received in the QEVENT queue. The event message for the FlowRollback is shown in Listing 9:

Listing 9. Event message for FlowRollback
<wmb:event xmlns:wmb="http://www.ibm.com/xmlns/prod/websphere/
  messagebroker/6.1.0/monitoring/event">
  <wmb:eventPointData>
    <wmb:eventData wmb:productVersion="6103" wmb:eventSchemaVersion="6.1.0.3" 
      wmb:eventSourceAddress="QIN.transaction.Rollback">
      <wmb:eventIdentity wmb:eventName="FlowRollback"/>
      <wmb:eventSequence wmb:creationTime="2009-10-08T02:57:16.653"/>
      <wmb:eventCorrelation wmb:localTransactionId="45efc2e9-a166-43d8-8a27-f0c201f442b6" 
        wmb:parentTransactionId="e5095bf9-ae05-4ca2-89b6-547b9bd9d0e8" 
        wmb:globalTransactionId="ab206c81-13a3-499e-9fa3-87ac236c3c96"/>
    </wmb:eventData>
    <wmb:messageFlowData>
      <wmb:broker wmb:name="BRK1" wmb:UUID="2d750f67-2301-0000-0080-96dbba41cd3f"/>
      <wmb:executionGroup wmb:name="default" 
        wmb:UUID="12feee26-2401-0000-0080-9bc38c4d3910"/>
      <wmb:messageFlow wmb:uniqueFlowName="BRK1.default.MonitoringEvents_MF" 
        wmb:name="MonitoringEvents_MF" wmb:UUID="8c284827-2401-0000-0080-c826dabb2ad0" 
        wmb:threadId="24096"/>
      <wmb:node wmb:nodeLabel="QIN" wmb:nodeType="ComIbmMQInputNode" wmb:detail="QIN"/>
    </wmb:messageFlowData>
  </wmb:eventPointData>
  <wmb:applicationData>
    <wmb:simpleContent wmb:name="MsgId" 
      wmb:value="414d5120574d42514d20202020202020be56ca4a2039df0b" 
      wmb:dataType="hexBinary"/>
  </wmb:applicationData>
</wmb:event>

A rollback event can also be generated when the flow makes a Web service call using the input message C2F.xml or F2C.xml and the machine hosting the Message Broker server does not have Internet connectivity.

Disabling events dynamically

Whenever you don't need them, you can dynamically disable the configured events without redeploying the message flow. For example, to disable all terminal events, run the following command:

mqsichangeflowmonitoring BRK1 -e default -f MonitoringEvents_MF -s
"HTTP_Request_C2F.terminal.in,HTTP_Request_C2F.terminal.out,HTTP_Request_F2C.terminal.in,
HTTP_Request_F2C.terminal.out" -i disable

Use the event source address in the command to disable individual events at the node level, assuming that the monitoring is already activated for the message flow. Of course, you can also inactivate monitoring for a message flow, which prevents any events from being emitted within the message flow.

Configuring event sources using a monitoring profile

If you need to change the enablement status or payload data of events in a message flow, you can configure them using the monitoring properties in the Message Flow editor. In this case, you need to redeploy the flow in order to override the setting of the event properties.

You can also use a monitoring profile to configure event sources. The advantage of doing so is that event sources and their associated properties can be configured dynamically at runtime without flow redeployment. A monitoring profile is an XML document that conforms to the XML Schema file MonitoringProfile.xsd. and specifies the event sources and associated properties in a message flow that will emit events. Use the following command to create the monitoring profile XML file for a message flow that has monitoring properties configured as shown below. This monitoring profile can be used as a starting point to create a new monitoring profile. The output file MyEventMonitoringProfile.xml is listed in Listing 8 above.

mqsireportflowmonitoring BRK1 -e default -f MonitoringEvents_MF 
-x -p C:/MyEventMonitoringProfile.xml
Listing 10. Monitoring profile of MyEventMonitoringProfile.xm
<profile:monitoringProfile xmlns:profile="http://www.ibm.com/xmlns/prod/websphere/
  messagebroker/6.1.0.3/monitoring/profile" profile:version="2.0">
  <profile:eventSource profile:eventSourceAddress="QIN.transaction.start"
    profile:enabled="true">
    <profile:eventPointDataQuery>
      <profile:eventIdentity>
        <profile:eventName profile:literal="FlowStart"/>
      </profile:eventIdentity>
      <profile:eventCorrelation>
        <profile:localTransactionId profile:sourceOfId="automatic"/>
        <profile:parentTransactionId profile:sourceOfId="automatic"/>
        <profile:globalTransactionId profile:sourceOfId="automatic"/>
      </profile:eventCorrelation>
      <profile:eventSequence profile:name="creationTime"/>
    </profile:eventPointDataQuery>   
    <profile:applicationDataQuery>
      <profile:complexContent>
        <profile:payloadQuery profile:queryText="$Root/MQMD/MsgId"/>
      </profile:complexContent>
      <profile:complexContent>
        <profile:payloadQuery profile:queryText="$Body/TemperatureConversion"/>
      </profile:complexContent>
    </profile:applicationDataQuery>
    <profile:bitstreamDataQuery profile:bitstreamContent="all" 
      profile:encoding="base64Binary"/>
  <profile:eventSource>
  <profile:eventSource profile:eventSourceAddress="QIN.transaction.end" 
    profile:enabled="true">
    ......
  </profile:eventSource>
  <profile:eventSource profile:eventSourceAddress="QIN.transaction.rollback" 
    profile:enabled="true">
    ......
  </profile:eventSource> 
  <profile:eventSource profile:eventSourceAddress="HTTP_Request_C2F.terminal.in"
    profile:enabled="true">
    <profile:eventPointDataQuery>
      <profile:eventIdentity>
        <profile:eventName profile:literal="In_C2F"/>
      </profile:eventIdentity>
      <profile:eventCorrelation>
        <profile:localTransactionId profile:sourceOfId="automatic"/>
        <profile:parentTransactionId profile:sourceOfId="automatic"/>
        <profile:globalTransactionId profile:sourceOfId="automatic"/>
      </profile:eventCorrelation>
      <profile:eventSequence profile:name="creationTime"/>
    </profile:eventPointDataQuery>
    <profile:applicationDataQuery>
      <profile:complexContent>
        <profile:payloadQuery profile:queryText="$Root/MQMD/MsgId"/>
      </profile:complexContent>
    </profile:applicationDataQuery>
    <profile:bitstreamDataQuery profile:bitstreamContent="none"
      profile:encoding="none"/> 
  </profile:eventSource>
  <profile:eventSource profile:eventSourceAddress="HTTP_Request_C2F.terminal.out" 
    profile:enabled="true">
    ......
  </profile:eventSource>
  <profile:eventSource profile:eventSourceAddress="HTTP_Request_F2C.terminal.in" 
    profile:enabled="true">
    ......
  </profile:eventSource>
  <profile:eventSource profile:eventSourceAddress="HTTP_Request_F2C.terminal.out" 
    profile:enabled="true">
    ......
  </profile:eventSource>  
</profile:monitoringProfile>

It is quite easy to create a new monitoring profile from scratch or by modifying an existing one. The monitoring profile must define the event sources in the message flow that can emit such events and the properties of each event. In other words, a monitoring profile can only be applied to a message flow with nodes and terminals to which corresponding events can be added. For example, for eventSourceAddress QIN.transaction.start, the message flow must have an input node named QIN.

You can apply the monitoring profile using the Broker Archive Editor:

  1. Open the Broker Application Development perspective in the Message Broker Toolkit.
  2. Open a BAR file using the Broker Archive Editor.
  3. Click the Manage tab.
  4. Click the message flow on which you want to set a monitoring profile configurable service.
  5. Enter name of a monitoring profile in the Monitoring Profile Name field:.
    Figure 13. Apply monitoring profile using BAR file
    Figure 13. Apply monitoring profile using BAR file
  6. Save and deploy the BAR file.

You can also apply the monitoring profile using the following commands

  1. Create a configurable service:
    mqsicreateconfigurableservice BRK1 -c MonitoringProfiles -o MyMonitoringEventProfile
  2. Associate your monitoring profile with the configurable service:
    mqsichangeproperties BRK1 -c MonitoringProfiles -o MyMonitoringEventProfile 
    -n profileProperties -p C:/MyEventMonitoringProfile.xml
  3. Apply a monitoring profile configurable service to a message flow. For example, apply to the flow MonitoringEvents_MF on execution group default:
    mqsichangeflowmonitoring BRK1 -e default -f MonitoringEvents_MF 
    -m MyMonitoringEventProfile

Consuming the monitoring events

After being subscribed, the monitoring event messages can be used by other applications to

  • Generate PKI data
  • Provide statistical and accounting reports
  • Catch any rollback transactions
  • Store payload data for audit

For example, the number of transactions that successfully passed through the message flow can be calculated based on the summation of the event FlowEnd generated at a certain interval. The time to take for completion of the temperature conversion Web service request call can be calculated based on the difference in the creationTime of the events In_C2F and Out_C2F. To learn more about analyzing data from the monitoring event messages, see the sample in the Message Broker Samples Gallery that shows how such events can be monitored by WebSphere Business Monitor.

Other related facilities

As mentioned above, you can use the monitoring events to generate useful data on transaction performance and statistics reports. Message Broker provides other facilities, such as statistics and pub/sub reports at the broker level, and statistics and accounting reports at the message flow level, to capture information at runtime for the same purposes.

Broker statistics reports

You can generate broker statistics reports in XML format to provide information about the performance of a broker network and the throughput between the broker and clients connected to the broker. For example, to enable broker statistics reports on the default execution group of broker BRK1 at one minute intervals, run the following command on the Message Broker Command Console. The change takes effect immediately -- you do not need to restart the broker:

mqsichangeproperties BRK1 -e default -o DynamicSubscriptionEngine 
-n statsInterval -v 60000

To disable the reports, use the same command, but set the value of the statsInterval property to 0. After they are enabled, the statistics reports as shown in Listing 11 below are published by the broker every minute. The statistics reports are distributed to all subscribers with the following topic:

$SYS/Broker/<broker name>/ExecutionGroup/<execution group>/Statistics

In this case:

$SYS/Broker/BRK1/ExecutionGroup/default/Statistics
Listing 11. Example of the broker statistics report
<Broker uuid="2d750f67-2301-0000-0080-96dbba41cd3f" label="BRK1" version="1">
  <ExecutionGroup uuid="12feee26-2401-0000-0080-9bc38c4d3910" label="default">
    <DynamicSubscriptionEngine>
      <ReportResponse>
      <ExecutionGroupWideStatistics clientCount="0" neighbourCount="0"
        subscriptionCount="9" timeStamp="2009-10-05 09:18:53"/>
        <ClientStatistics bytesQueued="0" msgsSent="0" bytesSent="0" bytesCutThru="0"
          msgsReceived="0" bytesReceived="0" msgsDropped="0" bytesDropped="0" 
          disconnectMsgsDropped="0" disconnectBytesDropped="0"/>
        <NeighbourStatistics bytesQueued="0" msgsSent="0" bytesSent="0" bytesCutThru="0" 
          msgsReceived="0" bytesReceived="0" msgsDropped="0" bytesDropped="0"
          disconnectMsgsDropped="0" disconnectBytesDropped="0"/>
      </ReportResponse>
    <DynamicSubscriptionEngine>
  </ExecutionGroup>
</Broker>

Message flow accounting and statistics reports

Message flow accounting and statistics reports contain the dynamic information about the runtime behavior of a message flow that can be collected by a broker to record performance and operating details of the flow execution, including number of messages processed, size of messages, processor usage, and elapsed processing times. For more information, see Message flow accounting and statistics data in the Message Broker information center.

The accounting and statistics data (snapshot, archive, or both) can be collected on a message flow or execution group basis. If needed, you can also collect thread and node-level statistics. Issue the following command to generate snapshot data with a target destination of XML format publication messages on the flow MonitoringEvents_MF:

mqsichangeflowstats BRK1 -s -e default -f MonitoringEvents_MF -c active -o xml

The reports are available to be subscribed to the topic:

$SYS/Broker/BRK1/StatisticsAccounting/SnapShot/default/MonitoringEvents_MF
Listing 12. Example of the flow statistics report
<WMQIStatisticsAccounting RecordType="SnapShot" RecordCode="SnapShot">
  <MessageFlow BrokerLabel="BRK1" BrokerUUID="2d750f67-2301-0000-0080-96dbba41cd3f" 
   ExecutionGroupName="default" ExecutionGroupUUID="12feee26-2401-0000-0080-9bc38c4d3910" 
   MessageFlowName="MonitoringEvents_MF" StartDate="2009-10-14" StartTime="23:58:57.288" 
   EndDate="2009-10-14" EndTime="23:59:16.706" TotalElapsedTime="18127000" 
   MaximumElapsedTime="1168000" MinimumElapsedTime="552000" TotalCPUTime="46875" 
   MaximumCPUTime="15625" MinimumCPUTime="15625" CPUTimeWaitingForInputMessage="0" 
   ElapsedTimeWaitingForInputMessage="1288000" TotalInputMessages="28" 
   TotalSizeOfInputMessages="18452" MaximumSizeOfInputMessages="659" 
   MinimumSizeOfInputMessages="659" NumberOfThreadsInPool="1" 
   TimesMaximumNumberOfThreadsReached="28" TotalNumberOfMQErrors="0" 
   TotalNumberOfMessagesWithErrors="0" TotalNumberOfErrorsProcessingMessages="0" 
   TotalNumberOfTimeOutsWaitingForRepliesToAggregateMessages="0" 
   TotalNumberOfCommits="28" TotalNumberOfBackouts="0" AccountingOrigin="Anonymous"/>
  <Threads Number="0"/>
  <Nodes Number="0"/>
</WMQIStatisticsAccounting>

Conclusion

This article showed you how you can easily configure the built-in monitoring events on WebSphere Message Broker V6.1 or later to publish event messages in XML format. The event documents can be subscribed to predefined topics and consumed by other applications, such as WebSphere Business Monitor, to monitor flow transaction data, store audit data, track message flow usage, report metrics, and provide performance KPI. The article also showed you how the monitoring events configuration can be updated dynamically on demand at runtime. Finally, the article described broker statistics, and message flow statistics and accounting reports, which you can use in conjunction with the monitoring events.

Acknowledgements

The author would like to thank Neal Mulvenna from the IBM Software Services for WebSphere (ISSW) team, and Sudhakar Bodapati from the IBM Global Business Services (GBS) team for reviewing and helping to improve this article.


Download

DescriptionNameSize
Project interchange fileMonitoringEvents_PI.zip10 KB

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=446411
ArticleTitle=Generating events in WebSphere Message Broker for transaction monitoring and auditing
publish-date=11112009