About Cloud Messaging

Messaging is one of the integration patterns that are used by the enterprise applications to solve integration challenges. Messaging systems allow independent applications to work together to accomplish some common tasks by exchanging messages without requiring direct connections. Messaging is also known as message-oriented middleware (MOM).

MOM uses a Messaging Provider to mediate messaging operations. MOM systems are composed of three basic components: clients, messages, and a MOM provider, including an API and administrative tools. The basic process of message exchange is as follows.

  1. A client (publisher) sends a message by using an API call to a destination managed by a Messaging Provider and the client continues to do other tasks while the message is being processed.
  2. The Messaging Provider routes and delivers the message to one or more clients (subscribers).
  3. The client (subscribers) receives the message and acknowledges after processing it successfully.

Using the webMethods Integration messaging solution you can implement the send or /receive (publish or subscribe) or point-to-point messaging styles by bringing the JMS-based messaging capabilities. It provides the infrastructure for implementing message-based solutions that are built on the publish-and-subscribe model.

Note:

  • Messaging is available upon request for a tenant. Contact IBM support.

  • Messaging is available for all projects in a tenant after it is enabled for that tenant.

Terminology

  • Publisher - A flowservice or a workflow that is configured to publish messages to the Messaging Provider. Other names for publisher are Sender or Producer.
  • Subscriber - A flowservice or a workflow that is configured to receive messages from the Messaging Provider. Other names for subscriber are Receiver or Consumer.
  • Destination - A virtual location through which the clients exchange messages. The publisher publishes messages to a destination that the messaging provider manages. The subscriber accesses that destination to consume the messages. The types of destinations are:
    • Queue - A queue is used when a message is specified to be consumed by one subscriber. Messages are saved in the queue and delivered to subscribers. Thus, a queue guarantees that each message is delivered to one subscriber only. This type of messaging model is known as point-to-point messaging.

    • Topic - A topic is used when a message is specified to be consumed by more than one subscriber. When you send a message to a topic, all subscribers that are associated to that topic receive the message. This type of messaging model is known as publisher-subscriber.

  • Message - A message is a data packet that is sent to a destination. A data packet can contain simple text and documents. A message has three parts: Header, Properties, and Body. The header is the part of the message that is used for routing messages through the system. The properties include application-specific properties that are used to facilitate communication between applications. The body of a message is called the payload.
  • Messaging Provider - The messaging provider is a communication platform that receives, stores, and delivers messages between applications. A messaging provider can be either Internal or External.
    • Internal Messaging Provider - This Messaging Provider is maintained by IBM. You can publish and consume messages only from workflows or flowservices in IBM webMethods Integration. The messaging destinations such as queues and topics can be managed from IBM webMethods Integration only. You can publish messages to the Internal Messaging Provider by using the Messaging connector.
    • External Messaging Provider - This Messaging Provider is provided by an external vendor. It is hosted on an external system other than the IBM webMethods Integration for you to publish or consume messages. You can publish messages to an External Messaging Provider by using the JMS (Java Messaging Service) connector.
  • Messaging Provider Account – An account that enables the user to establish a connection to the Messaging Provider. Users can create accounts from the Connectors page. To connect to an Internal Messaging Provider, you must use a pre-configured default account.
  • Messaging Service Trigger- A trigger for starting the workflow when a message arrives (when a subscriber receives a message).
  • Message Routes - A list of routing rules that can be added to a subscriber. An arriving message is evaluated against each rule in the specified order by the subscriber. If the evaluation criterion is met, the invocation that is specified in the routing rule is initiated to do further tasks.
  • Routing Rule - A rule that defines how messages must be handled as they arrive at the destination. It consists of the following components:
    • Filter - The expression that is used for evaluating an incoming message.
    • Invocation - A subscriber-initiated workflow or flow service that performs more tasks.
  • Default Invocation - The default workflow or flow service that is initiated by the subscriber if no rules match.

Messaging Exchange Process in IBM webMethods Integration

  1. The publisher publishes messages that are received to a destination (queue or topic) specified in the publisher's workflow or flow service by using the connectors - Messaging or JMS.
    • The Messaging connector is used to publish messages to the Internal Messaging Provider.
    • The JMS connector is used to publish messages to the External Messaging Provider.
  2. The Messaging Provider (internal or external) sends the messages to the registered subscribers.
  3. The subscriber evaluates the messages in accordance to the routing rules defined. If no rule satisfies the criteria, the default invocation is initiated. If default invocation is not specified, then the subscriber acknowledges the message and no invocation happens.
  4. The initiated workflow or flow service does the subsequent tasks as configured in IBM webMethods Integration.

The Messaging Provider delivers messages in the order that they are received.

Message Restrictions in Cloud Messaging

The following section lists the message limits that are supported in cloud messaging:

  • Maximum Pending Messages

  • Message Size Limits

Maximum Pending Messages

The Messaging Provider can hold up to 10000 messages per queue or topic. All messages arriving at the destinations are dropped if the hold capacity limit is reached until the number of messages waiting to be delivered decreases. The dropped messages cannot be recovered. Delivered messages are no longer available for consumption by any other subscriber. If there are no subscribers, based on the destination type, the message retention is as follows for internal messaging providers:

  • Queue - Messages are retained for 30 days.
  • Topic - Messages are removed immediately. However, if there are registered subscribers that are not in the active state, then the message is retained for 30 days. During this period, if the subscriber returns to the active state, the message is delivered and removed from the topic.
Note: In the case of an external messaging provider, the retention period is based on the configurations set in the external messaging provider.

Message Size Limits

The maximum message size is 50 MB. However, depending on the situation the size might vary as follows.

  • If you are using the body/string and body/object fields, then the size limit is less than or equal to 50 MB.
  • If you are using the body/data fields for complex messages, then the size limit is slightly less than 25 MB. This is because during encoding, metadata is included for each field or value in the body/data and published. Thus, it is essential to pay attention to the cumulative size of all fields and values.
  • If you want to run a workflow on receiving messages, then the maximum message size can be up to 10 MB.

High-level Tasks for Configuring Messaging

Task Description
Enable the messaging functions Contact Global Support to enable the messaging functions for your tenant in IBM webMethods Integration.
Create a destination Set up a topic or queue to receive messages from publishers and store them until a subscriber consumes them.
Create a publisher flow service or Workflow Create a flow service or workflow to publish messages to a queue or topic by using the Messaging Service connector. The input data to the publisher flow service or workflow can be a string or a document.
Create a subscriber Create a subscriber to perform the following tasks:
  • Subscribe (listen) for messages that are received by a queue or topic.
  • Define the default invocation.
  • Define message routes, message acknowledgment, message processing, and manage errors.
Create a consumer flow service or workflow Create a flow service or workflow to consume messages from a queue or topic. The consumer is associated with a subscriber and runs when a queue or topic receives messages.

Managing Messaging Destinations

You can create messaging destinations that can be local to a project or shared across other projects. Also, you can also edit and delete messaging destinations using the Actions menu on the Queues and Topics home page, respectively.

Creating Queues

  1. Go to Events > Messaging
  2. Click Messaging Destinations.

    By default, all the existing Queues created in the user tenant are listed.

  3. Click Add Queue to create a new queue.

    The listing page shows the details under Name and Visibility, for example, whether a queue is Local or Shared and in the Actions columns. You can select a queue from the list to view the queue metrics.

  4. Enter the following details:

    • Name - Provide the name of the queue.
    • Capacity – Set the maximum number of messages the queue can hold while waiting for subscribers to consume them. If the capacity exceeds the maximum allowed limit for a tenant (the default limit is 10,000), an error message appears.
      Note: The maximum capacity limit can be adjusted upon request for a tenant by contacting IBM support.
    • Shared across all projects - Select Yes or No to set the queue as local or shared by using this option. The default setting for the Shared across all projects option is No, which means that the queue is project-specific and not accessible in other projects.

Click Save. The newly created Queues listing page appears.

Editing Queues

  1. Navigate to Queues from the left navigation page to access the Queues listing page, which displays all queues that are created in a project, including both shared and local queues.
  2. Select the Edit option in the Actions column to edit a specific queue.

  3. Confirm the edit when the confirmation message appears.
Note: Editing a queue leads to deletion of the existing queue and creating a new queue, interrupting the subscribers by using the existing queue.

Deleting Queues

  1. Navigate to Queues from the left navigation page to access the Queues listing page, which displays all queues that are created in a project, including both shared and local queues.

  2. Select the Delete option in the Actions column to remove a specific queue.

  3. Confirm the deletion when the confirmation message appears.

Note: When a queue is deleted, all the messages in the queue are deleted, and the workflows or flow services become incapable of publishing messages to the queue, potentially disrupting the existing execution.

Creating Topics

  1. Go to Events > Messaging.
  2. Click Messaging Destinations. By default, the Queues listing page appears. Select Topics from the left navigation page to view the topics listing page. All the existing topics that are created in the user tenant are listed.

  3. Click Add Topic to create a new topic:

    The listing page shows the details under Name and Visibility, for example, whether a topic is Local or Shared and in the Actions columns. You can select a topic from the list to view its metrics.

  4. Enter the following details.
    • Name - Provide the name of the topic.
    • Capacity – Set the maximum number of messages the topic can hold while waiting for subscribers to consume them. If the capacity exceeds the maximum allowed limit for a tenant (default limit is 10,000), an error message appears. The maximum capacity limit can be adjusted upon request for a tenant by contacting IBM support.
    • Shared across all projects - Select Yes or No to set the topic as local or shared by using this option. The default setting for the Shared across all projects option is No, which means that the queue is project-specific and not accessible in other projects.

    Click Save. The newly created is listed in the Topic details page appears.

Editing Topics

  1. Navigate to Topics from the left navigation page to access the Topics listing page, which displays all topics that are created in a project, including both shared and local topics.
  2. Select the Edit option in the Actions column to edit a specific topic.
  3. Confirm the edit when the confirmation message appears. Editing a topic leads to the deletion of the existing topic and the creation of a new one, which may interrupt subscribers by using the existing topic.

Deleting Topics

  1. Navigate to Topics from the left navigation page to access the Topics listing page, which displays all topics that are created in a project, including both shared and local queues.
  2. Select the Delete option in the Actions column to remove a specific topic.

  3. Confirm the deletion when the confirmation message appears.

Note: If a topic is deleted, all the messages in the topic are deleted, and the workflows or flow services become incapable of publishing messages to the topic, potentially disrupting the existing execution.

Metrics Associated with Queues and Topics

After you create a queue or a topic, the metrics available for a queue or a topic appear when you click the queue or the topic link to view the details.

  • Name - Name of the queue or the topic.
  • Capacity -The maximum number of messages the queue or topic can hold while waiting for subscribers to consume the pending messages.
  • Rates
    • Current Publish Rate - Rate in seconds at which messages are being published to the queue or to the topic.
    • Current Consumption Rate - Rate in seconds at which messages are currently being consumed from the queue or from the topic.
  • Last Activity
    • Last Published on - The date and time a message was last written to the queue or to the topic.
    • Last Consumed on - The date and time a message was last read from the queue or from the topic.

  • Messages:

    • Published - Number of messages that are published to the queue or topic.
    • Consumed - Number of messages that are consumed from the queue or topic.
    • Pending - Number of messages pending in the queue or topic for consumption.
    • Last Message ID - Message ID of the last consumed message.