How MFT agents use Java heap and native heap memory

An IBM® MQ Managed File Transfer agent runs as a Java process. As such, the agent runs in the virtualized environment of the Java Virtual Machine (JVM).

The JVM itself is a native process, is bounded by the hardware and operating system. The JVM maintains two memory areas:
  • The Java heap

    This contains the instances of Java objects and is managed by garbage collection processing. The maximum size of the Java heap is allocated during JVM startup using the -Xmx JVM option.

  • The native heap

    The native heap contains resources for the JVM itself; for example, the Just-In-Time Compiler, Classes, and ClassLoaders.

An agent primarily uses the Java heap. When performing managed transfers, the agent uses the Java heap to create Java objects that are required for the transfer. Any file data that is read into buffers by the agent is also stored in Java heap memory.

An agent does not itself contain any code that uses the native heap. However, there is native code in the Java message queuing interface (JMQI) that the agent uses to communicate with its agent queue manager.

This native code is used when an agent connects to its agent queue manager using the BINDINGS transport. This is a local shared memory connection (sometimes referred to as interprocess communication, or IPC), rather than a TCP/IP connection which is used if an agent connects using the CLIENT transport. When an agent is configured to use the BINDINGS transport, the native heap is used to pass messages and commands between the agent and the agent queue manager.

This means that a heavily loaded agent that is connected to its agent queue manager using the BINDINGS transport makes more extensive use of the native heap, when compared to an equivalent agent which is connected using the CLIENT transport.

One common misconception is that the Java heap for an agent must be equal to (or greater than) the size of the largest file that is to be transferred. This is not correct as file data is read into memory in stages.

As a guide, the maximum amount of Java heap that is used to store file data for each transfer can be roughly calculated as follows:
Memory allocated for a transfer = agentCheckpointInterval * 
agentFrameSize * agentWindowSize * agentChunkSize

How Java heap and native heap usage affects agents

When a java.lang.OutOfMemoryError occurs, you might think it reasonable to increase the amount of Java heap available to the application, using the -Xmx Java System Property. For example, the following property setting attempts to allocate a maximum Java heap size of 2GB:
-Xmx2048M

However, allocating too much Java heap for an application can cause a java.lang.OutOfMemoryError to occur, due to native heap exhaustion. This is because, as the Java heap space grows, the native heap must shrink to accommodate it.

For information about how to prevent java.lang.OutOfMemoryErrors that are caused by native heap exhaustion, see What to do if your MFT agent ABENDS with a java.lang.OutOfMemoryError due to native memory exhaustion.