IBM webMethods Broker Document Logging

Edit online

IBM webMethods Broker Document Logging

Edit online

You can configure your system to log documents that Broker clients publish to or subscribe to, on Brokers. Integration Server logs those documents to the IS Core Audit Log database component. Configuring your system to log documents that Broker clients publish to or subscribe to, on Brokers, involves these steps:

  • During installation of IBM webMethods Broker, you selected a preconfigured storage session for the Broker Server. Make sure the Broker Server storage files can accommodate the volume of document logging you expect. For instructions on adjusting the size of the storage file, see The Storage Files.
  • Install IBM webMethods Logging Utility on Integration Server. The Logging Utility functions as a logging agent between Integration Server and Brokers. Also, install the IS Core Audit Log database component, if it is not already installed. For instructions, see Installing IBM webMethods Products.
  • Configure Integration Server to connect to a Broker on which Broker clients will publish or subscribe to documents you want to log. For instructions, see IBM webMethods Integration Server Administrator’s Guide.
  • Set up document logging, as follows:
    • Enable document logging on the connected Broker.
    • Identify the documents you want the connected Broker to log, and specify whether to log the documents on publish or on subscriber acknowledgement.
    • Logging Utility (WmLogUtil package) is an Integration Server package. Therefore, it can communicate only with the Broker that is connected to Integration Server. If you want other Brokers in a territory or cluster also to perform document logging then you must configure Integration Server and Logging Utility for that Broker also.
    • Optionally, specify your own document IDs for logged documents. Doing so lets you provide a uniform business context for these documents that will be helpful when you view them later in My webMethods.
    • After document logging is set up, the connected Broker begins creating copies of the documents you identified. These copies are called logged documents, and are stored in Broker's Logging Utility client queue. Identify the number of logged documents Logging Utility must remove from Broker's Logging Utility client queue at one time and forward to Integration Server to store in the IS Core Audit Log database component.

      For instructions, see Setting Up Document Logging.

The following figure illustrates document logging with the Logging Utility.

Logging Utility Client Groups and Clients

Edit online

When the Logging Utility connects to Broker for the first time, Broker creates two client groups and a client in each client group for Logging Utility:

  • LogOnPublish client group. For grouping the Logging Utility client queues that store documents logged during publish to Broker.
  • LogOnAck client group. For grouping the Logging Utility client queues that store documents logged during subscription acknowledgement to Broker.

Both client groups are created with the property "AllCanSubscribe," which means clients belonging to these client groups can subscribe to any document without the need to add that document to the client group's "cansubscribe" list.

Note: When you create a new Broker, the LogOnPublish and LogOnAck client groups are also created simultaneously with the default client groups.

Each client group contains the following clients:

  • <Integration Server client prefix>_LogOnPublish. Broker creates this client in the LogOnPublish client group. The queue of this client stores the documents logged during publish to Broker.

  • <Integration Server client prefix>_LogOnAck. Broker creates this client in the LogOnAck client group. The queue of this client stores the documents logged during subscriber acknowledgement to Broker.

    Note: Logging Utility clients on Broker and Broker clients are different from one another. Do not consider them to be functionally similar.

Client ID

Edit online

When you are connecting multiple Integration Servers to Broker, the Logging Utility on the Integration Server that first connects to the Broker creates the LogOnPublish and LogOnAck client groups. When the remaining Integration Servers connect to the Broker, only clients are created in the respective client groups.

Each Logging Utility client has an associated Client ID. The first part of the Client ID identifies the Integration Server with which the client is associated, and the second part identifies the associated Logging Utility client group.

Integration Server Client prefix is a string that identifies Integration Server to Broker. By default, Integration Server uses its license key for the prefix. For ease of use, you can replace it with a name of your choice.

Client ID has the following formats:

  • <Integration Server client prefix>_LogOnPublish

    For example, in the client ID, Wjb5tM+bn8pBOTAWIc4XWbfdh5w_LogOnPublish, Wjb5tM+bn8pBOTAWIc4XWbfdh5w is the name of the Integration Server, and LogOnPublish is the name of the Logging Utility client group.

  • <Integration Server client prefix>_LogOnAck

    For example, in the client ID, Wjb5tM+bn8pBOTAWIc4XWbfdh5w_LogOnAck, Wjb5tM+bn8pBOTAWIc4XWbfdh5w is the name of the Integration Server, and LogOnAck is the name of the Logging Utility client group.

Note: To be able to set up document logging effectively, the client prefixes of all Integration Servers in a cluster must be the same.

Identifying a Logging Utility Client by Client ID

Edit online

About this task

Use the following procedure to identify a logging utility client by client ID.

To identify a Logging Utility client by client ID

Procedure

  1. Set the client prefix from Integration Server Administrator: Settings > Messaging > IBM webMethods  Messaging Settings. For complete instructions, see IBM webMethods Integration Server Administrator’s Guide.
    1. Click the Broker connection alias link.
    2. Click Edit Broker Connection Alias and set the client prefix.
  2. After setting the client prefix, restart Integration Server and log on to IBM My webMethods Server.
  3. In My webMethods: Messaging > Clients.
  4. Specify a filter to select the Broker that is connected to Integration Server, and click Go to display the list of clients.
  5. Search for the client ID with the client prefix you set in the Integration Server.

Setting Up Document Logging

Edit online

You must set up document logging so that the Logging Utility clients in the LogOnPublish client group can log documents when publish activity occurs at Broker, and clients in the LogOnAck client group can log subscriber acknowledgement activity at Broker.

To set up document logging

  1. Enable document logging. For more information, see Enable Document Logging.
  2. Identify the document types you want to log. For more information, see Identify Document Types to Log.
  3. Subscribe to document types from other Brokers. For more information, Subscribe to Document Types from Other Brokers in the Territory or Cluster.
  4. Specify document IDs. For more information, see Specify User-Defined Document IDs.
  5. Specify how many documents to pass to Integration Server. For more information, see Specify Number of Logged Documents to Give to Integration Server at One Time.
  6. Configure any additional settings, as needed. For more information, see Other Configuration Settings.

Enable Document Logging

Edit online

About this task

Use the following procedure to enable document logging on Broker.

To enable document logging on Broker

Procedure

  1. In My webMethods: Administration > Messaging >  Broker Server s > Servers.
  2. Click the name of the Broker Server that hosts the Broker to which Integration Server will connect. On the Broker Server Details page, do the following:
    1. Click the Brokers tab and then click the name of the Broker to which Integration Server will connect.
    2. Click the Configuration tab, select the Document logging check box, and then click Apply.

Identify Document Types to Log

Edit online

About this task

Use the following procedure to identify document types you want the connected Broker to log. Specify whether to log the documents on publish or on subscriber acknowledgement.

To identify document types to log

Procedure

  1. In My webMethods: Administration > Messaging >  Broker Server s > Clients.
  2. In the Client List, click the client ID of the Broker client (<Integration Server client prefix>_LogOnPublish and <Integration Server client prefix>_LogOnAck) for which you want to register document type subscriptions. If the Broker client does not appear in the list, use the Search tab to locate it.
  3. On the Client Details page, click the Subscriptions tab. You will see a list of all the document types to which the Broker client has "cansubscribe" permissions.
  4. Select the check box next to each document type to which you want the Broker client to subscribe.
  5. Click Subscribe. On the Subscription tab, My webMethods displays a "Yes" in the Subscription column for each document type to which the Broker client subscribes.

Subscribe to Document Types from Other Brokers in the Territory or Cluster

Edit online

About this task

Use the following procedure to subscribe to document types from other Brokers.

To subscribe to document types from other Brokers

Procedure

  1. For each of the non-connected Broker in the territory or cluster for which to log documents, do the following:
    1. In My webMethods: Administration > Messaging >  Broker Server s > Servers.
    2. Click the name of the Broker Server that hosts one of the Brokers. On the Broker Server Details page, do the following:
      1. Click the Broker s tab and then click the name of the Broker to which Integration Server will connect.
      2. Click the Configuration tab, clear the Document logging check box, and then click Apply.
    3. Use the procedure in Identify Document Types to Log to identify the document types to log for the Broker.
  2. On the Integration Server that hosts the Logging Utility, create triggers that subscribe to the documents to log. For instructions, see the Publish-Subscribe Developer’s Guide.

Specify User-Defined Document IDs

Edit online

About this task

Use the following procedure to specify user-defined document IDs.

To specify user-defined document IDs

Procedure

  1. In Integration Server Administrator, go to the Settings > Extended page.
  2. Click Edit Extended Settings.
  3. In the Extended Settings box, set the watt.server.auditDocIdField property as follows: 
    watt.server.auditDocIdField=user_defined_field

    user_defined_field must be of type Broker unicode string and can be up to 128 characters long. user_defined_field is case sensitive.

    • For BrokerEvents published using the Java client API, define user_defined_field in one of the following ways:
      • watt.server.auditDocIdField=_env. field_name, where field_name is an envelope field. For example, if uuid is set in the envelope, set watt.server.auditDocIdField as follows:

        watt.server.auditDocIdField=_env.uuid

      • watt.server.auditDocIdField=field_name , where field_name is a body field. For example, if SSN is set in the body, set watt.server.auditDocIdField as follows:

        watt.server.auditDocIdField=SSN

    • For JMS messages published using the JMS API, define user_defined_field in one of the following ways:
      • watt.server.auditDocIdField=_env.field_name , where field_name is an envelope field. For example, if Message.setJMSMessageID(String id) is set in the envelope, set watt.server.auditDocIdField as follows:

        watt.server.auditDocIdField=_env.jms_message_id

      • watt.server.auditDocIdField=Properties. field_name, where field_name is a property of the message. For example, if Message.setStringProperty(customId, XYZ) is set on the message before publishing, set watt.server.auditDocIdField as follows:

        watt.server.auditDocIdField=Properties.customId

  4. Click Save Changes, and then restart Integration Server.
  5. Add user_defined_field to your document types, as described in Identify Document Types to Log.

Specify Number of Logged Documents to Give to Integration Server at One Time

Edit online

About this task

Use the following procedure to specify the number of logged documents to pass to Integration Server at one time.

To specify the number of logged documents to pass to Integration Server at one time

Procedure

  1. In Integration Server Administrator, go to the Packages > Management page.
  2. In the Packages list, click in the row for the WmLogUtil package. Integration Server Administrator displays the Logging Utility home page.
  3. Go to the Settings > Configure page and click Change Settings.
  4. In the Size box, type the number of logged documents for the Logging Utility to remove from the Broker log queue and pass to Integration Server at one time. You can improve performance by retrieving a large number of documents from a Broker log queue at one time. However, too large a batch can degrade performance. The number of documents you retrieve at one time depends on environment-specific factors such as the amount of RAM on the Broker host machine, your database, and the size of the logged documents. To determine the optimum batch size, run an activation and tune the number.
    Note: Although the Logging Utility allows you to specify any value for batch size, Broker can retrieve a maximum of 160 documents.
  5. Click Save Changes.

Other Configuration Settings

Edit online

You can configure the SharedEventOrdering and BrokerTimeout parameters in the WmLogUtil\config\LoggingUtility.cnf file.

Parameter Description
SharedEventOrdering

Specifies how the documents will be arranged. To handle high document volume in a clustered Integration Server environment, set the SharedEventOrdering parameter to none. This will allow multiple Integration Servers to process the logged documents in parallel. Default value is Publisher. Allowed values are Publisher and none.

For example, if you want Integration Servers to process the documents ordered by publishers, set SharedEventOrdering=Publisher.

BrokerTimeout

Specifies how long (in milliseconds) Integration Server will wait before timing out the Broker. The default value is 30 seconds.

For example, to configure the Broker timeout value to one minute, set BrokerTimeout=600000

Logging Utility Management

Edit online

The Logging Utility starts automatically when you start Integration Server. The Logging Utility home page shows the utility's status.

Managing the Logging Utility

Edit online

About this task

Use the following procedure to manage the Logging Utility.

To manage the Logging Utility

Procedure

  1. To open the Logging Utility home page, in Integration Server Administrator, go to the Packages > Management page.
  2. In the Packages list, click in the row for the WmLogUtil package. Integration Server Administrator displays the Logging Utility home page. The Status column shows the utility's current status. You can change that status as follows:
    Task Description
    Start Starts the Logging Utility. The utility begins pulling logged documents from the Broker log queue.
    Stop Stops the Logging Utility after completely processing any logged documents the utility has already pulled from the Broker log queue. The value in the Status column changes to Stop Pending.
    Note: Depending on your system, you might have to click Refresh to see when the status changes to Stop.
    Suspend Pauses the Logging Utility after completely processing any logged documents the utility has already pulled from the Broker log queue. The value in the Status column changes to Suspend Pending.
    Note: Depending on your system, you might have to click Refresh to see when the status changes to Suspended.
    Resume Resumes the suspended Logging Utility.
    Refresh Updates the information in the Status column.
  3. If you want the Logging Utility to write debug entries in the Integration Server server log, click the word Off in the Debug Trace column. The word changes to On.

Built-In Services

Edit online

You can use Logging Utility built-in services to work with the log queue of the Broker that is connected to the Logging Utility-equipped Integration Server. You can use these services to check the number of documents in the queue, view problematic documents in the queue, or delete a problematic document from the queue. The services below are available in the WmLogUtil package:

Service Description
pub.loggingUtility.util:brokerLoggingLength

Retrieves the number of logged documents in the Broker's log queue.

pub.loggingUtility.util:brokerLoggingPeek

Displays logged documents in the Broker's log queue.

pub.loggingUtility.util:brokerLoggingSingleExtract

Removes a logged document from the Broker's log queue.

pub.loggingUtility.util:brokerLoggingLength

Edit online

Retrieves the number of logged documents in the Broker's log queue.

Input Parameters

None.

Output Parameters

LoggingLength String Number of logged documents in the Broker's log queue.

Usage Notes

To use this service, run it directly from IBM webMethods Designer and view the results on Designer's Service Result tab. Do not call the service from Integration Server services at run time.

pub.loggingUtility.util:brokerLoggingPeek

Edit online

Displays logged documents in the Broker's log queue.

Input Parameters

NumberToPeek String Number of logged documents to view, starting with the oldest logged document in the log queue.

Output Parameters

BrokerEventStrings String List Collection of logged documents. Each element in BrokerEventStrings contains an entire logged document represented in String form.

Usage Notes

You might use this service to locate a problematic logged document.

To use this service, run it directly from Designer and view the results on Designer's Service Result tab. Do not call the service from Integration Server services at run time.

pub.loggingUtility.util:brokerLoggingSingleExtract

Edit online

Removes a logged document from the Broker's log queue.

Input Parameters

DeleteEvent String Indicates whether to remove the oldest logged document from the log queue. Values are y, yes, true, or on and are case insensitive.

Output Parameters

BrokerEventString String Removed logged document in string form.

Usage Notes

To use this service, run it directly from Designer and view the results on Designer's Service Result tab. Do not call the service from Integration Server services at run time.