Performance and scalability of MQ Telemetry

Capacity Planning

For information about performance reports for MQ Telemetry, see MQ Performance documents.

Connections

If you are sending messages more than one per minute, keep your connection open to avoid the cost of initiating a new connection. If you are sending messages less than one every 10 - 15 minutes, consider dropping your connection to avoid the cost of keeping it open. You might want to keep a TLS connection open, but idle, for longer periods because it is more expensive to set up.

Additionally, consider the capability of the client. If there is a store and forward facility on the client then you might batch up messages and drop the connection between sending the batches. However, if the client is disconnected, then it is not possible for the client to receive a message from the server. Therefore the purpose of your application has a bearing on the decision.

If your system has one client sending many messages, for example file transfers, do not wait for a server response per message. Instead, send all messages and check at the end that they have all been received. Alternatively, use Quality of Service (QoS).

You can vary the QoS by message, delivering unimportant messages using QoS 0 and important messages using a QoS of 2. The message throughput can be around twice as high with a QoS of 0 than with a QoS of 2.

Naming conventions

If you are designing your application for many clients, implement an effective naming convention. In order to map each client to the correct ClientIdentifier, make the ClientIdentifier meaningful. A good naming convention makes it easier for the Administrator to work out which clients are running. A naming convention helps the administrator filter a long list of clients in IBM® MQ Explorer, and helps with problem determination; see Client identifier.

Throughput

The length of topic names affects the number of bytes that flow across the network. When publishing or subscribing, the number of bytes in a message might be important. Therefore limit the number of characters in a topic name. When an MQTT client subscribes for a topic IBM MQ gives it a name of the form:

 ClientIdentifier: TopicName

To view all of the subscriptions for an MQTT client, you can use the IBM MQ MQSC DISPLAY command:

 DISPLAY SUB(' ClientID1:*')

Defining resources in IBM MQ for use by MQTT clients

An MQTT client connects to an IBM MQ remote queue manager. There are two basic methods for an IBM MQ application to send messages to an MQTT client: set the default transmission queue to SYSTEM.MQTT.TRANSMIT.QUEUE or use queue manager aliases. Define the default transmission queue of a queue manager, if there are large numbers of MQTT clients. Using the default transmission queue setting simplifies the administration effort; see Configure distributed queuing to send messages to MQTT clients.

Improving scalability by avoiding subscriptions.

When an MQTT V3 client subscribes to a topic, a subscription is created by the telemetry (MQXR) service in IBM MQ. The subscription routes publications for the client onto SYSTEM.MQTT.TRANSMIT.QUEUE. The remote queue manager name in the transmission header of each publication is set to the ClientIdentifier of the MQTT client that made the subscription. If there are many clients, each making their own subscriptions, this results in many proxy subscriptions being maintained throughout the IBM MQ publish/subscribe cluster or hierarchy. For information about not using publish/subscribe, but using a point to point based solution instead, see Sending a message to a client directly.

Managing large numbers of clients

To support many concurrently connected clients, increase the memory available for the telemetry (MQXR) service by setting the JVM parameters -Xms and -Xmx. Follow these steps:

1. Find the java.properties file in the telemetry service configuration directory; see Telemetry (MQXR) service configuration directory on Windows or Telemetry service configuration directory on Linux® .
2. Follow the directions in the file; a heap of 1 GB is sufficient for 50,000 concurrently connected clients.

   
   # Heap sizing options - uncomment the following lines to set the heap to 1G
   #-Xmx1024m
   #-Xms1024m
   

3. Add other command-line arguments to pass to the JVM running the telemetry (MQXR) service in the java.properties file; see Passing JVM parameters to the telemetry (MQXR) service.

To increase the number of open file descriptors on Linux, add the following lines to /etc/security/limits.conf/, and log in again.

@mqm soft nofile 65000
@mqm hard nofile 65000

Each socket requires one file descriptor. The telemetry service require some additional file descriptors, so this number must be larger than the number of open sockets required.

The queue manager uses an object handle for each nondurable subscription. To support many active, nondurable subscriptions increase the maximum number of active handles in the queue manager; for example:

echo ALTER QMGR MAXHANDS(999999999) | runmqsc qMgrName

echo "ALTER QMGR MAXHANDS(999999999)" | runmqsc qMgrName

Other considerations

When planning your system requirements, consider the length of time taken to restart the system. The planned downtime might have implications for the number of messages that queue up, waiting to be processed. Configure the system so that the messages can be successfully processed in an acceptable time. Review disk storage, memory, and processing power. With some client applications, it might be possible to discard messages when the client reconnects. To discard messages, set CleanSession in the client connection parameters; see Clean sessions. Alternatively, publish and subscribe using the best effort Quality of Service, 0, in an MQTT client; see Quality of service. Use non-persistent messages when sending messages from IBM MQ. Messages with these qualities of service are not recovered when the system or connection restarts.