Getting more memory in AIX for your Java applications
This article explains how 32-bit Java allocates its heap in AIX, and pertains to all the latest versions of 32-bit IBM Java available for AIX versions 4.3.3, 5.1, and 5.2.
You will learn how to distinguish between various memory areas of a Java process under AIX, and how to tweak their sizes to suit your needs. The information in this article will allow your Java-based applications to break the barrier of 1 GB Java heap with versions of Java prior to 1.4; I'll also describe how Java 1.4 does it automatically for you. For applications that use databases and middleware, this article provides the basis for advanced topics, such as segment collision avoidance.
This article has been updated to include information about Java 1.4, and also to give examples of commands you can use to increase your understanding of the topic. The fence analogy, and much of the enhanced description, were provided by Mark Bluemel. Along with Mark, Ashok Ambati and Greg Hodgins reviewed the article to make it much more readable. The update itself was triggered by feedback from readers like you; thanks to everyone who sent their comments.
Native versus Java heaps
A heap is a storage management structure for tracking and allocating memory. When we talk about Java processes, there are two different types of memory areas, and hence two types of heaps, that must be distinguished.
The Java Virtual Machine (JVM) has an address space, just like any process, from which it can allocate chunks of memory for various purposes. Probably the most important (and definitely the most familiar to Java programmers) of these chunks is the Java heap. The Java heap is used for allocating the Java objects, and you can monitor its usage through verbosegc traces. The size of a Java heap can be explicitly controlled using -Xms and -Xmx command line arguments. This is true for all platforms where Java is supported, hence the name Java heap.
The rest of the chunks are allocated by the JVM on an as-needed basis, and are usually platform dependent. We refer to them collectively as data. From a Java point of view, a more appropriate term for these chunks is native heap, due to its platform-specific nature. For example, on AIX the native heap contains several critical components of a JVM process:
- Thread stacks for every thread except primordial
- Buffers, lookup tables, and so on for ZIP-related operations (such as
- Buffers and structures related to native GUIs underlying Swing/AWT (for example, Motif)
- Data used by JNI code, such as native database drivers, MQ-Series, and so on
- Just-in-time (JIT) compiler and Mixed-Mode-Interpreter (MMI) support routines
- Executable code for JIT-compiled methods
and more. The native heap size cannot be controlled through a command line argument.
On AIX, depending on the Java version in use and the Java heap size selected, the Java heap may or may not itself be allocated from the native heap.
In this article, the term heap, if not further qualified, refers to the Java heap, not the process (native) heap. Also, the term data is used interchangeably with native heap.
The o_maxdata on AIX
Before we can discuss how to allocate larger heaps using Java, we have to understand the process address space in AIX. The information in this section applies to all processes under AIX, including the JVM. See Related topics for links to more information on this topic.
On AIX, the 32-bit address space (= 232 or 4 GB) is divided into 16 segments, each 256 MB (256 MB*16 = 4 GB). When a 32-bit process is loaded in memory, its address space looks Figure 1.
Figure 1.Segments in 32-bit AIX model
In the default address-space model, segment 2 contains the user stack,
kernel stack and u-block, as well as data (area used by
malloc()). This translates to a limit of 256 MB for data,
stacks, and u-block for the whole process. Segments 3 through C are free
to be used as shared memory, using
If a process needs to allocate more data, it can switch to the large space
model, where the native heap starts from segment 3 onwards, and grows to
an offset specified by the
o_maxdata value in the XCOFF
header to a non-zero value. It is essential to understand the significance
o_maxdata to learn how to allocate larger Java heaps.
You can think of
o_maxdata to be setting a "fence" in segments
3 through A. Figure 2 shows the fence in action
(segments 0, 1 and D-F omitted for clarity).
Figure 2. The fence
If the fence moves "up" (that is, the Fence line moves towards Segment 2),
we get more space for shared memory and, correspondingly, less space for
the native heap. Moving the fence down allows the native heap to grow,
while reducing shared memory. For a setting of
o_maxdata = N,
the fence is placed at 0x30000000+N. For several good reasons, it is
recommended to set
o_maxdata to a value that is the start of
a particular segment, such as 0xn0000000. In this case, the fence sits
between segments 2+n and 3+n, which translates to n segments for the
native heap, and 10-n segments for shared memory.
o_maxdata is zero, the entire native heap is allocated from
segment 2. This is the default space model; here, all segments from 3
through C (10 segments = 2.56 GB) can be used for shared memory. The
native heap size is constrained by the available memory in segment 2,
(less than 256 MB). When
o_maxdata is set to a non-zero
value, we switch to the large space model, which implies the native heap
is allocated from segment 3 onwards. The top-most position of the fence in
Figure 2 shows the scenario where
o_maxdata is greater than zero. Other structures, such as
stacks and u-blocks, still get allocated from segment 2.
Segments B and C are used only for shared memory, so
can move the fence all the way down to segment A, depicted as the
bottom-most position of the fence, leaving segments 3 through A for native
heap (8 segments = 2 GB), and B & C (2 segments =512 MB) for shared
As an example, setting n=2 (
o_maxdata = 0x20000000) would mean
that the fence sits at segment 5, leaving 2 segments (=512 MB) for native
heap, and 8 segments (=2 GB) for shared memory.
Java heap allocation values
The preceding discussion might make you think that if you want a large Java
heap, you must patch the Java binary to have
o_maxdata set to
0x80000000. Unfortunately, things are not so simple.
If you look at the 32-bit Java binary, using the
o_maxdata is already set to 0x80000000. The example listing
below shows the output for Java 1.3.1.
$ dump -ov /usr/java131/jre/bin/java /usr/java131/jre/bin/java: ***Object Module Header*** # Sections Symbol Ptr # Symbols Opt Hdr Len Flags 4 0x000079bc 627 72 0x1002 Flags=( EXEC DYNLOAD ) Timestamp = "Aug 07 03:17:57 2003" Magic = 0x1df (32-bit XCOFF) ***Optional Header*** Tsize Dsize Bsize Tstart Dstart 0x000056df 0x000002f1 0x00000818 0x10000128 0x20000807 SNloader SNentry SNtext SNtoc SNdata 0x0004 0x0002 0x0001 0x0002 0x0002 TXTalign DATAalign TOC vstamp entry 0x0005 0x0003 0x200009e0 0x0001 0x20000984 maxSTACK maxDATA SNbss magic modtype 0x00000000 0x80000000 0x0003 0x010b UR
o_maxdata shows up in the last line of the output, and is
labeled maxDATA. Yet, an attempt to allocate a heap larger than 1 GB will
fail. You can test this very quickly by using the command
java -mx1200m -version. This command will fail with the
default JVM settings. Note that older versions of Java 1.1.8 and 1.2.2
differed in this behavior. If you are not using the latest service
refreshes of your JVM version, you might observe a different behavior.
The variability is because of the dual-personality heap allocation method
used by Java. For up to 1 GB of heap, Java uses
allocate the Java heap. The Java heap is allocated from the native heap.
But if you go above 1 GB, or if the environment
IBM_JAVA_MMAP_JAVA_HEAP is defined, Java reverts to
mmap() for the Java heap, which means it uses shared memory.
This allows Java to allocate a heap as large as the maximum size of shared
memory, or 2.56 GB.
o_maxdata is set to 0x80000000, leaving 2 GB for
native heap and 512 MB for shared memory. If you attempt to allocate a
Java heap larger than 1 GB, it fails because Java tries to use shared
memory for heap, and there is only 512 MB of shared memory available. If
IBM_JAVA_MMAP_JAVA_HEAP in the environment and try to
allocate a heap larger than 512 MB, JVM will be unable to allocate the
heap. The solution is to adjust
o_maxdata in such a way that
the size of shared memory grows large enough to accommodate the Java heap.
The next section shows you how to do this.
Stretching the limits
So how do you go to a larger Java heap? You need to change
o_maxdata to increase the amount of shared memory address
space. You can use the following calculations to come up with the
appropriate value for
o_maxdata. Supposing you need a maximum
heap size of J bytes, you would invoke Java as
java -mxJ <other arguments>
If J is less than 1 GB, and IBM_JAVA_MMAP_JAVA_HEAP is not set, the default setup will suffice. If J is > 1 GB, or if IBM_JAVA_MMAP_JAVA_HEAP is set, use
o_maxdata = 0xn0000000
n = (10 - ceil(J/256M)) or 8
, whichever is smaller. The function
ceil rounds up the
argument to the next integer.
For example, if you need to allocate 1500 MB of heap, we have
n = (10 - ceil(1500M/256M)) = (10 - 6) = 4. If you set
o_maxdata = 0x40000000, you will be able to allocate the
needed size of heap. To change
o_maxdata, set the following
LDR_CNTRL=MAXDATA=<new o_maxdata value>
The above example would set the following environment variable:
To verify that your calculation is accurate, you can try the following commands:
$ export LDR_CNTRL=MAXDATA=0x40000000 $ java -mx1500m -version
The command works, indicating that your calculations were accurate. Change 4 to 3 in the above example, and you will see that Java fails to load.
If you need a heap larger than 2.25 GB, some older releases of AIX had problems interpreting 0x00000000. You can use LDR_CNTRL=MAXDATA=0 instead.
So what would be the maximum heap size we would be able to allocate with
Java on AIX? Setting
o_maxdata to zero will allow all 10
segments to be used for shared memory, letting us allocate up to 2.56 GB
of heap. (And even more for Java 1.4!) But, see Balancing memory for some caveats associated with very large
Reverse-engineering a running process
If you use svmon on a running JVM, you can get a picture that is quite a parallel of Figure 2. For example, given a JVM process with process ID 389354, the following is the output of svmon -P 389354 -m :
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd LPage 389354 java 14837 3155 951 14566 N Y N Vsid Esid Type Description LPage Inuse Pin Pgsp Virtual 289aa d work loader segment - 6899 0 0 6899 0 0 work kernel segment - 6375 3144 951 6375 1c324 3 work working storage - 993 0 0 993 4d8b3 - pers large file - 236 0 - - /dev/build:1737514 4b772 - work - 135 0 0 135 7c2dc f work shared library data - 103 0 0 103 102e7 2 work process private - 38 4 0 38 13504 - work - 23 7 0 23 46471 - pers large file - 13 0 - - /dev/build:1737510 7637d - pers large file - 10 0 - - /dev/build:2163665 4322 1 pers code,large file - 9 0 - - /dev/build:1720874 55f95 - pers large file - 2 0 - - /dev/build:2163666 3956e - clnt - 1 0 - - 23608 7 mmap mapped to sid 4b772 - 0 0 - - 5b536 - work - 0 0 0 0 1b666 a mmap mapped to sid 82e1 - 0 0 - - 5f557 8 mmap mapped to sid 702df - 0 0 - - 2b76a - work - 0 0 0 0 82e1 - work - 0 0 0 0 702df - work - 0 0 0 0 521d6 6 work working storage - 0 0 0 0 7701 5 work working storage - 0 0 0 0 777bd b mmap mapped to sid 5b536 - 0 0 - - 737bc 9 mmap mapped to sid 2b76a - 0 0 - - 242ca 4 work working storage - 0 0 0 0
Can you tell what the value of
o_maxdata is, by looking at
the above output? The segment numbers show up under Esid; the
above shows segments 3, 4, 5, and 6 being used for "work," and segments 7,
8, 9, a and b are using mmap. So, svmon seems to indicate that this
process was using
o_maxdata set to 0x40000000, and was using
a heap larger than 1 GB but smaller than 1.25 GB (since only 7 through B
are used, hence 5 segments). This is exactly right; the process was
o_maxdata set to 0x40000000, and with a maximum
heap size of 1100m. See Related topics for
more examples with svmon.
If you are new to AIX platform, and the first version of Java you have used on AIX is Java 1.4, you may be wondering what I am talking about. Java 1.4 is more intelligent than its predecessors; it no longer requires you to do calculations and set environment variables. Based on the requested heap size, Java 1.4 launcher sets the appropriate o_maxdata value automatically. The exact behavior of Java 1.4 is a bit different from the one I explained above, and it may change in future releases. You can use svmon to find out for yourself how Java 1.4 and above manipulate o_maxdata. Note, though, that you can still use the techniques you learnt in this article with Java 1.4. If any LDR_CNTRL=MAXDATA values are set in the environment, Java 1.4 will not override it, even if it results in an error during startup. Conversely, if you do want Java 1.4 to manage the heap sizing for you, make sure you remove any LDR_CNTRL=MAXDATA entries from the environment settings.
If you use Java 1.4 on AIX 5.2, another new feature becomes available. Let's see if you can locate what the size of heap is, from the svmon output below (I have removed all lines with Esid marked "-" for clarity):
------------------------------------------------------------------------------- Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd LPage 507904 java 58146 3155 951 52487 N Y N Vsid Esid Type Description LPage Inuse Pin Pgsp Virtual 2f549 2 work process private - 45801 4 0 45801 0 0 work kernel segment - 6383 3144 951 6383 3966e 1 clnt code - 18 0 - - 5bfb4 c work overflow heap - 17 0 0 17 2ffa9 c mmap mapped to sid 23faa - 0 0 - - bda0 e mmap mapped to sid 1bfa4 - 0 0 - - 27fab b mmap mapped to sid 3bfac - 0 0 - - 2f409 5 mmap mapped to sid 1b544 - 0 0 - - 47fb3 7 mmap mapped to sid 7ffbd - 0 0 - - 4bfb0 8 mmap mapped to sid 43fb2 - 0 0 - - 3ffad 9 mmap mapped to sid 37faf - 0 0 - - 57fb7 4 mmap mapped to sid 53fb6 - 0 0 - - 3f42d 3 mmap mapped to sid 5ffb5 - 0 0 - - 2bfa8 a mmap mapped to sid 33fae - 0 0 - - 63fba 6 mmap mapped to sid 6bfb8 - 0 0 - - 23f8a f mmap mapped to sid 13f86 - 0 0 - - 13fa6 d mmap mapped to sid 1ffa5 - 0 0 - -
Except for 0, 1 and 2, all the other segments are being shown as "mmap". This is the Very Large Memory Model (VLMM). By setting LDR_CNTRL=MAXDATA to a "DSA" value, the segments D, E and F can also be used for mmap. Though VLMM is available in AIX 5.1 as well, Java uses it only in AIX 5.2 onwards. This results in a total of up to 3.25 GB available for Java heap. To see this in action, try the following command:
java -mx3328m -version
Congratulations, you just broke the 3 GB barrier! You need to be on at least AIX 5.2, at least Java 1.4, and LDR_CNTRL=MAXDATA must not be set.
If the above svmon output is compared to Figure 1, you will note that the shared library code and data are no longer in segments D and F respectively. So if you are trying to use a tool like tprof to profile the application, you will no longer be able to get library- and subroutine- specific information. Plus, all the caveats associated with large heaps, covered by the next section, are applicable here as well.
See the SDK Guide accompanying Java 1.4 for more details.
There are at least three factors you need to consider before tweaking the heap size to maximum. In no particular order, they are:
- Garbage collection (GC) performance
- The GC needs to scan the whole heap, so the time taken to scan the heap grows as the heap gets larger. See Related topics for information on GC tuning, including tuning the heap size as one of the critical parts.
- Segment collisions
- If an in-process component of the JVM attempts to use shared memory, and the Java heap is using all the available shared memory, you can run into a situation where JVM or the other component overwrite each others' structures. This can lead to misleading error messages and random crashes (or worse). Segment collision is a major topic by itself, and is beyond the scope of this article. Stay tuned for an article on segment collisions, soon to be published here at eServer Developer Domain.
- Running out of native heap
- Occasionally, you may run out of native heap. If you set
o_maxdatato 0, only a portion of segment 2 is available, which may not be enough for the stuff Java needs to allocate. If you are using JNI code that consumes a lot of memory, even larger values of
o_maxdatamay not prove sufficient.
The symptoms of running out of native heap vary. You might not be able to start threads beyond a certain limit, or you may get error messages indicating allocation failures in native code. The verbosegc output usually shows Java heap to still be available, but OutOfMemory exceptions are still thrown.
To find out if you are running low on native heap, observe the "InUse" column of svmon output. This column lists the pages, each of size 4 KB, being used. Observing the value for each segment belonging to native heap gives you a good idea of native heap usage. The maximum value possible for InUse for a segment is 256MB/4KB = 65536. A good debugging technique for detecting memory leaks in native code is to use
svmon -P <pid> -m -i <interval>, which takes a snapshot at a specified interval.
In some cases, you can tune the configuration to avoid running out of native heap:
- Try reducing the stack size for threads (the -Xss parameter). However, deeply nested methods may force a thread stack overflow in case of insufficient stack size.
- For middleware products, if you are using an in-process version of the driver, it is usually possible to find an out-of-process driver that can have a significant effect on the native memory requirements. For example, you can use Type 4 JDBC drivers (DB2's "Net" drivers, Oracle's "Thin" drivers), MQ-Series can be switched from Bindings mode to Client mode, and so on. Refer to the documentation for the products in question for more details. Also see "Segment collisions" above.
Striking the right balance between Java and native heap becomes unavoidable as you go to larger heaps. Java 1.4 goes a long way towards making the configuration work easier for you. But if multiple components are involved, you might still have to take the address space management into your own hands. I hope this article has provided you with enough information to get started.
- The IBM Diagnostics Guides provide exhaustive details about Java debugging on all supported platforms, straight from the IBM Java team.
- You can find the developer kits, tutorials, whitepapers and more at the developerWorks Java Technolozy zone. The Java 1.4 SDK Guide (or README in earlier versions) accompanies the Java Developer Kits and provides up-to-date information about each Java release.
- AIX Documentation deals with the memory models and various APIs mentioned in this article.
- Looking for help? Visit IBM eServer technical support for a comprehensive set of resources, all focused on helping you learn about, choose, implement and use the right IBM e-server solution for your IT infrastructure needs.