Configuring IBM Integration Bus for data enablement

Edit online

For some data to be available in the Cloud APM console, you must configure IBM® Integration Bus to enable the required data collection.

Before you begin

Make sure that the IBM Integration Bus agent is configured.

Remember: Transaction tracking enablement requires you to restart the integration node (broker).

About this task

Archive statistics and resource statistics can be monitored by the IBM Integration Bus agent only after the data collection is enabled for the integration node (broker). Similarly, if you want to see the transaction tracking in the middleware and topology dashboards, you must enable transaction tracking within the integration node (broker) before you enable transaction tracking for the IBM Integration Bus agent.

Decide what type of data that you want to monitor with the IBM Integration Bus agent and complete the following steps according to your needs.

Integration servers that are owned by the integration node have a default server.conf.yaml configuration file for each integration server that is stored in a subdirectory of the integration node directory. Any properties that you set for the integration node in the node.conf.yaml file are inherited by the integration servers that it owns. Nonetheless, you can change an integration server's properties by modifying them in its server.conf.yaml file. (For more information, see Configuring an integration node by modifying the node.conf.yaml file in the IBM App Connect Enterprise documentation.)

Procedure

To enable archive statistics data collection for the integration node (broker), see Enabling archive accounting and statistics data collection.
To enable resource statistics data for an integration node (broker), see Enabling JVM resource statistics.
To enable transaction tracking for message flows within an integration node (broker), see Enabling transaction tracking.
If you no longer want transaction tracking data, remember to disable transaction tracking for the integration node (broker) on which it was enabled. See Disabling transaction tracking.

Enabling archive accounting and statistics data collection

Edit online

About this task

To enable archive accounting and statistics collection for message flows that belong to the integration node (broker), issue the mqsichangeflowstats command from the bin directory of the integration node (broker) installation directory.

Remember: Issue the mqsichangeflowstats command to the integration node (broker) according to your requirements for monitoring data. It is recommended that you enable only the statistics that you require, because there can be a lot of data and processing when you have many message flows. For more detailed information about the mqsichangeflowstats command, refer to IBM Integration Bus documentation.

Important: IBM Cloud Application Performance Management does not support snapshot accounting and statistics data due to the amount of data and processing required for the set 20 second snapshot interval. Archive data provides the same exact attributes as snapshot data, and is more suitable for the regular production monitoring provided by IBM Cloud Application Performance Management. If you have enabled snapshot data collection for the integration node (broker), remember to configure the IBM Integration Bus agent not to store the snapshot data. For instructions, see Disabling snapshot data collection for the agent.

Procedure

To get most data for message flows, issue the following command. This command is recommended because it does not enable the most detailed terminal statistics that provide invocation counts per terminal per node. The terminal level consumes a lot of storage.
```
mqsichangeflowstats BrokerName -a -g -j -c active -t none -n basic -o xml
```

In ACE version 11, to get most data for message flows, modify the node.conf.yaml or server.conf.yaml file as follows. These properties are recommended because they do not enable the most detailed terminal statistics that provide invocation counts per terminal per node. The terminal level consumes a lot of storage.

Statistics:
  # Application message flows will by default inherit Snapshot and Archive values
  # set here
  Snapshot:
    #publicationOn: 'inactive' # choose 1 of : active|inactive, default inactive
                               # Ensure Events.OperationalEvents.MQ|MQTT 
                               # is set for outputFormat json,xml
    #accountingOrigin: 'none'  # choose 1 of : none|basic
    #nodeDataLevel: 'none'     # choose 1 of : none|basic|advanced
    #outputFormat: 'usertrace' # comma separated list of : 
                               #csv,bluemix,json,xml,usertrace
    #threadDataLevel: 'none'   # choose 1 of : none|basic
  Archive:
    archivalOn: 'active'     # choose 1 of : active|inactive,
                             # default inactive
                               # Ensure Events.OperationalEvents.MQ|MQTT
                               # is set for outputFormat xml
    #accountingOrigin: 'none'  # choose 1 of : none|basic
    #majorInterval: 60         # Sets the interval in minutes at which 
                              #archive statistics are published
    nodeDataLevel: 'basic'         # choose 1 of : none|basic|advanced
    outputFormat: 'xml'  # comma separated list of : csv,xml,usertrace
    #threadDataLevel: 'none'   # choose 1 of : none|basic

Note: If you want to disable this setting, comment out the lines of archivalOn: 'active', nodeDataLevel: 'basic', and outputFormat: 'xml'.

To get all the data supported by the IBM Integration Bus agent, issue the following command:
```
mqsichangeflowstats BrokerName -a -g -j -c active -t none -n advanced -o xml
```

In ACE version 11, to get all the data supported by the IBM Integration Bus agent, modify the node.conf.yaml or server.conf.yaml file as follows:

Statistics:
  # Application message flows will by default inherit Snapshot and Archive values
  # set here
  Snapshot:
    #publicationOn: 'inactive' # choose 1 of : active|inactive, default inactive
                               # Ensure Events.OperationalEvents.MQ|MQTT
                               # is set for outputFormat json,xml
    #accountingOrigin: 'none'  # choose 1 of : none|basic
    #nodeDataLevel: 'none'     # choose 1 of : none|basic|advanced
    #outputFormat: 'usertrace' # comma separated list of :
                               # csv,bluemix,json,xml,usertrace
    #threadDataLevel: 'none'   # choose 1 of : none|basic
  Archive:
    archivalOn: 'active'    # choose 1 of : active|inactive, default inactive
                               # Ensure Events.OperationalEvents.MQ|MQTT
                               # is set for outputFormat xml
    #accountingOrigin: 'none'  # choose 1 of : none|basic
    #majorInterval: 60       # Sets the interval in minutes at which
                             # archive statistics are published
    nodeDataLevel: 'advanced'     # choose 1 of : none|basic|advanced
    outputFormat: 'xml' # comma separated list of : csv,xml,usertrace
    #threadDataLevel: 'none' # choose 1 of : none|basic

Note: If you want to disable this setting, comment out the lines of archivalOn: 'active', nodeDataLevel: 'advanced', and outputFormat: 'xml'.

To reduce the amount of data but still reasonably monitor all message flows without further details, issue the following command:
```
mqsichangeflowstats BrokerName -a -g -j -c active -t none -n none -o xml
```

In ACE version 11, to reduce the amount of data but still reasonably monitor all message flows without further details, modify the node.conf.yaml or server.conf.yaml file as follows:

Statistics:
  # Application message flows will by default inherit Snapshot and Archive values
  #set here
  Snapshot:
    #publicationOn: 'inactive' # choose 1 of : active|inactive, default inactive
                               # Ensure Events.OperationalEvents.MQ|MQTT
                               # is set for outputFormat json,xml
    #accountingOrigin: 'none'  # choose 1 of : none|basic
    #nodeDataLevel: 'none'     # choose 1 of : none|basic|advanced
    #outputFormat: 'usertrace' # comma separated list of :
                               # csv,bluemix,json,xml,usertrace
    #threadDataLevel: 'none'   # choose 1 of : none|basic
  Archive:
    archivalOn: 'active'    # choose 1 of : active|inactive, default inactive
                               # Ensure Events.OperationalEvents.MQ|MQTT
                               # is set for outputFormat xml
    #accountingOrigin: 'none'  # choose 1 of : none|basic  
    #majorInterval: 60       # Sets the interval in minutes at which
                             # archive statistics are published
    nodeDataLevel: 'none'         # choose 1 of : none|basic|advanced
    outputFormat: 'xml' # comma separated list of : csv,xml,usertrace
    #threadDataLevel: 'none' # choose 1 of : none|basic

Note: If you want to disable this setting, comment out the lines of archivalOn: 'active', nodeDataLevel: 'none', and outputFormat: 'xml'.

If you have a large number of message flows and want to reduce the amount of data, you can specify which message flows to monitor by replacing the -g or -j option in the previously mentioned commands.
- To specify a particular integration server (execution group) for enablement, replace -g with -e IntegrationServerName .
- To identify a particular message flow for enablement, replace -j with -f MessageFlowName.
- If you have grouped your message flows into applications, to specify a particular application for enablement, add -k ApplicationName to the -j option.
The IBM Integration Bus agent collects archive accounting and statistics data at the interval of 5 minutes. To set the interval at which the integration node (broker) produces the archive accounting and statistics data to the same interval, issue the following command with the integration node (broker) stopped, and then restart the integration node (broker):
```
mqsichangebroker BrokerName -v 5
```

In ACE version 11, the IBM Integration Bus agent collects archive accounting and statistics data at the interval of 5 minutes. To set the interval at which the integration node (broker) produces the archive accounting and statistics data to the same interval, modify the node.conf.yaml or server.conf.yaml file as follows:

Statistics:
  # Application message flows will by default inherit Snapshot and Archive values
  # set here
  Snapshot:
    #publicationOn: 'inactive' # choose 1 of : active|inactive, default inactive
                               # Ensure Events.OperationalEvents.MQ|MQTT
                               # is set for outputFormat json,xml
    #accountingOrigin: 'none'  # choose 1 of : none|basic
    #nodeDataLevel: 'none'     # choose 1 of : none|basic|advanced
    #outputFormat: 'usertrace' # comma separated list of :
                               # csv,bluemix,json,xml,usertrace
    #threadDataLevel: 'none'   # choose 1 of : none|basic
  Archive:
    archivalOn: 'active'     # choose 1 of : active|inactive, default inactive
                               # Ensure Events.OperationalEvents.MQ|MQTT
                               # is set for outputFormat xml
    #accountingOrigin: 'none'  # choose 1 of : none|basic   
    majorInterval: 5         # Sets the interval in minutes at which
                             # archive statistics are published
    nodeDataLevel: 'none'         # choose 1 of : none|basic|advanced
    outputFormat: 'xml' # comma separated list of : csv,xml,usertrace  
    #threadDataLevel: 'none'       # choose 1 of : none|basic

Results

After the IBM Integration Bus agent is configured and started, the message flow accounting and statistics data is displayed in the following group widgets:

Message Flow Dashboard
- Commits & Backouts
- CPU Microseconds
- Elapsed Microseconds
- Input Byte Rate
- Input Message Rate
- Input Message Size
- Input Message Wait CPU Microseconds
- Input Message Wait Elapsed Microseconds
- Message Flow Errors
- Message Processing Node Statistics
Processing Node Dashboard
- CPU Microseconds
- Elapsed Microseconds
- Invocations
- Processing Node Status
- Terminal Statistics

Enabling JVM resource statistics

Edit online

About this task

To enable JVM resource statistics for integration servers that belong to the integration node (broker), issue the mqsichangeresourcestats command from the bin directory of the integration node (broker) installation directory.

Remember: The JVM resource statistics are considered optional because only a few attributes of data are displayed for the high cost of the agent processing this data every 20 seconds. Be sure to consider whether you really need the JVM resource statistics data.

Procedure

To enable the statistics across all integration servers in the integration node (broker), issue the following command:
```
mqsichangeresourcestats BrokerName -c active
```

In ACE version 11, to enable the statistics across all integration servers in the integration node (broker), modify the node.conf.yaml file as follows:

Statistics:
  # Application message flows will by default inherit Snapshot and Archive values
  # set here
  Snapshot:
    #publicationOn: 'inactive' # choose 1 of : active|inactive, default inactive
                               # Ensure Events.OperationalEvents.MQ|MQTT
                               # is set for outputFormat json,xml
    #accountingOrigin: 'none'  # choose 1 of : none|basic
    #nodeDataLevel: 'none'     # choose 1 of : none|basic|advanced
    #outputFormat: 'usertrace' # comma separated list of :
                               # csv,bluemix,json,xml,usertrace
    #threadDataLevel: 'none'   # choose 1 of : none|basic
  Archive:
    archivalOn: 'active'    # choose 1 of : active|inactive, default inactive
                               # Ensure Events.OperationalEvents.MQ|MQTT
                               # is set for outputFormat xml
    #accountingOrigin: 'none'  # choose 1 of : none|basic
    majorInterval: 5         # Sets the interval in minutes at which
                             # archive statistics are published
    nodeDataLevel: 'advanced'     # choose 1 of : none|basic|advanced
    outputFormat: 'xml' # comma separated list of : csv,xml,usertrace
    threadDataLevel: 'basic'      # choose 1 of : none|basic
   Resource:
     reportingOn: true       # choose 1 of : true|false, default false
......

Note: If you want to disable this setting, comment out reportingOn: true.

To enable the statistics for a given integration server in the integration node (broker), issue the following command:
```
mqsichangeresourcestats BrokerName -e IntegrationServerName -c active
```

In ACE version 11, to enable the statistics for a given integration server in the integration node (broker), modify the server.conf.yaml file as follows:

Statistics:
  # Application message flows will by default inherit Snapshot and Archive values
  # set here
  Snapshot:
    #publicationOn: 'inactive' # choose 1 of : active|inactive, default inactive
                               # Ensure Events.OperationalEvents.MQ|MQTT
                               # is set for outputFormat json,xml
    #accountingOrigin: 'none'  # choose 1 of : none|basic
    #nodeDataLevel: 'none'     # choose 1 of : none|basic|advanced
    #outputFormat: 'usertrace' # comma separated list of : 
                               # csv,bluemix,json,xml,usertrace
    #threadDataLevel: 'none'   # choose 1 of : none|basic
  Archive:
    archivalOn: 'active'    # choose 1 of : active|inactive, default inactive
                               # Ensure Events.OperationalEvents.MQ|MQTT
                               # is set for outputFormat xml
    #accountingOrigin: 'none'  # choose 1 of : none|basic
    majorInterval: 5         # Sets the interval in minutes at which
                             # archive statistics are published
    nodeDataLevel: 'advanced'     # choose 1 of : none|basic|advanced
    outputFormat: 'xml' # comma separated list of : csv,xml,usertrace
    threadDataLevel: 'basic'   # choose 1 of : none|basic
  Resource:
    reportingOn: true        # choose 1 of : true|false, default false

Note: If you want to disable this setting, comment out reportingOn: true.

Results

The JVM resource statistics data is displayed in the following group widgets:

Garbage Collection Count
Garbage Collection Duration
JVM Non-Heap Memory
JVM Heap Memory

Enabling transaction tracking

Edit online

Before you begin

Make sure that the IBM Integration Bus agent is installed. A user exit named KQIUserExit is provided to enable IBM Integration Bus for transaction tracking.
Make sure that the user who will start the integration node (broker) has access to the KQI User Exit module directory. That is, ensure that you add the user ID that is used to start the integration node (broker) to the group under which you installed the IBM Integration Bus agent.

About this task

You must deploy the KQIUserExit user exit to the integration node (broker). Otherwise, no data is available in the middleware and topology dashboards even after you have enabled the IBM Integration Bus agent for transaction tracking.

Tip: The following IBM Integration Bus nodes are included in the middleware and topology dashboards by the KQIUserExit user exit as uninstrumented services:

Database and compute nodes where an ODBC data source is specified
TCP/IP nodes
File nodes for remote FTP or FTPS servers
MQ nodes, unless already instrumented

Procedure

To enable transaction tracking for IBM Integration Bus, complete the following steps:

Close any broker shells that have loaded the MQSI environment.
Open an IBM Integration Bus command console with one of the following methods. If you have multiple versions of integration nodes (brokers) installed, ensure that you start the command console for the correct version.
- Click Start > IBM Integration Bus > IBM Integration Console
- From the bin directory of integration node (broker) installation directory, issue the mqsiprofile command.
Stop the integration node (broker) that you want to configure with the mqsistop command.
Enable transaction tracking for the message flow within the integration node (broker) by adding the KQIUserExit user exit with the mqsichangebroker command.
- To enable transaction tracking for all message flows within the integration node (broker), run the following command:
```
mqsichangebroker broker_name -e "KQIUserExit"
```
- To enable transaction tracking for a specific message flow within the integration node (broker), run the following command:
```
mqsichangeflowuserexits broker_name -e execution_group_name -k application_name -f 
message_flow_name -a "KQIUserExit"
```
Alternatively, in Ace version 11, enable transaction tracking for the message flow within the integration node (broker) by adding the KQIUserExit to the node.conf.yaml or the server.conf.yaml file.
```
UserExits:
activeUserExitList: 'KQIUserExit' # Specify the name 
#of an installed user exit to activate.
```
Note: If you want to disable the transaction tracking, comment out activeUserExitList: 'KQIUserExit'.
Restart the integration node (broker) with the mqsistart command.

Disabling transaction tracking

Edit online

Procedure

To disable transaction tracking for IBM Integration Bus, complete the following steps:

Open an IBM Integration Bus command console with one of the following methods. If you have multiple versions of integration nodes (brokers) installed, ensure that you start the command console for the correct version.
- Click Start > IBM Integration Bus > IBM Integration Console
- From the bin directory of integration node (broker) installation directory, issue the mqsiprofile command.
Disable transaction tracking for the message flow within an integration node (broker) with one of the following methods:
- To disable transaction tracking for a specific message flow, use the mqsichangeflowuserexits command:
```
mqsichangeflowuserexits broker_name -e execution_group_name 
-f message_flow_name -a ""
```
- To disable transaction tracking for all message flows within the integration node (broker), first stop the integration node (broker) with the mqsistop command and then issue the mqsichangebroker command:
```
mqsichangebroker broker_name -e ""
```

What to do next

For transaction tracking, after you enable transaction tracking for IBM Integration Bus, you must also enable transaction tracking for the agent. For instructions, see Configuring transaction tracking for the IBM Integration Bus agent.
If you have enabled snapshot data collection for your integration node (broker), configure the IBM Integration Bus agent not to store any snapshot data. Cloud APM does not support snapshot accounting and statistics data due to the amount of data and processing required for the set 20 second snapshot interval. For instructions, see Disabling snapshot data collection for the agent.