AES-GCM cipher restriction

A guide to the restrictions that are imposed on AES-GCM ciphers when used for TLS Cryptography. These restrictions are imposed by the IETF and NIST organizations and require that the same session key must not be used to securely transfer more than 2^24.5TLS records when using AES-GCM ciphers.

For more information about these restrictions, see RFC 9325 Section 4.4 Limits on Key Usage and RFC 8446 section 5.5.

IBM® MQ does not implement cryptographic functionality directly. Instead, several different cryptographic libraries are used to provide TLS and Advanced Message Security functionality. On Windows, Linux®, and AIX® operating systems, the cryptographic library that IBM MQ uses is GSKit. For applications, the C and unmanaged .NET libraries use GSKit for cryptographic functionality. The implementation of the AES-GCM encryption algorithms by GSKit includes the restrictions that are specified by the standards group. Also, these restrictions are enabled by default. As such, IBM MQ TLS communication, when using AES-GCM ciphers, terminates if more than 2^24.5TLS records are transmitted using the same session key.

Note: This restriction is not present on IBM i, IBM Z or IBM MQ for HPE NonStop platforms or Java/JMS, managed .NET applications because different cryptographic libraries are used, and these libraries have not implemented the same restriction.

If an IBM MQ channel remains running for long enough that more than 2^24.5TLS records are transmitted using the same session key, the underlying cryptographic library terminates the connection. This causes the channel to terminate and an AMQ9288E error message is generated. Applications that have their communication terminated in this way receive an MQRC_CONNECTION_BROKEN return code from whichever IBM MQ operation was being performed.

The termination of the connection can be performed at either end of the communication, but only on ends that are using GSKit for cryptographic functionality.

Advice for mitigating the restriction

Some options for how to prevent or handle communications that are terminated due to this restriction are as follows:

Use reconnectable clients

Applications can be configured to automatically attempt a reconnection, should a connection fail. This includes connections that are terminated due to the GCM restriction. When configured for reconnection, the client application is restored automatically at any point of failure and any handles to open objects are restored. This is done without returning to the application code.

For more information, see Automatic client reconnection.

Set a secret key reset value

IBM MQ can be configured to request a session key reset after a configurable number of bytes has been transferred over a channel. Upon reaching this limit, IBM MQ requests that the cryptographic layer performs a session key reset, resulting in a new session key.

It is important to note that the value specified is the number of bytes transferred, which relates to the size of the messages being sent by IBM MQ. The restriction is on the number of TLS records that are sent. There is not a direct mapping between message bytes and TLS records as a TLS record can send a maximum number of bytes dependent on the Maximum Transmission Unit (MTU) of the network. Any messages that are sent that are larger than this value are transmitted as multiple TLS records. The MTU value varies between networks. Also, there are other reasons why a TLS record might need to be sent outside of transmitting IBM MQ message data, for example IBM MQ Heartbeat checks, TLS alerts, other IBM MQ protocol messages. These additional TLS records count toward the maximum number of TLS records but are not counted in the IBM MQ secret key reset value.

Regularly resetting a session key using secret key reset can prevent the channel from being terminated due to the AES-GCM restriction.

For more information, see Resetting SSL and TLS secret keys.

Use TLS 1.3 cipherspecs

While the AES-GCM restriction is still present when using the TLS 1.3 protocol, the TLS 1.3 protocol supports automatically performing a session key reset without the need to interrupt TLS communications. This allows GSKit to manage resetting the session key when it is necessary without IBM MQ needing to request a secret key reset.

For more information, see Using TLS 1.3 in IBM MQ in Enabling CipherSpecs.

Disable the AES-GCM restriction

If needed, the restriction can be disabled by setting the environment variable GSK_ENFORCE_GCM_RESTRICTION=GSK_FALSE to disable the AES-GCM restriction. Doing so allows any number of TLS records to be sent using the same session key. If choosing this mitigation, the environment variable must be set on each end of communication that uses GSKit for secure communications.

Warning: This option is not recommended as, after more than 2^24.5 TLS records have been sent, it is possible for attackers to perform analysis on the sent records to determine the session key in use. Once the session key has been determined, all existing and future communication using that session key is compromised.