OutOfMemoryErrors with Large Objects Held by Threads
kgibm 0600027VAP Visits (1605)
One common type of OutOfMemoryError (OOM) occurs when an application thread accumulates too much memory. When looking at a PHD, however, you may only see a large object (and potentially uninteresting child objects) as GC roots, so it's hard to figure out the root cause. Even if there are clear package names in the large object, that still may be insufficient information for the developers to figure out the problem. For example, here is an example PHD heapdump as seen in the Memory Analyzer Tool and HeapAnalyzer:
In both tools, all we can see is that there are some large GC roots In this case of HeapAnalyzer, the reference path snakes its way into the connection pool infrastructure, so MAT's analysis is much clearer. At first, it may seem that the JDBC ResultSet may not be a GC root; however, if we run "Path to GC Roots," we end up at another uninformative GC root:
So we have a few JDBC objects, an ArrayList, and some primitives inside these two which are causing the OOM, but there is not enough information to point the finger at anyone. This is where system dumps come in. Since I was running WAS >= 18.104.22.168, I got a system dump in addition to the PHD heapdump. Loading the system dump into MAT (jextract is not required for recent Java versions), we go to the thread stacks view (just click Finish on the dialog that pops up):
Next, we expand the threads that are consuming memory and we expand the stack frames until we find the offending code. We can even go further and look at all the Strings and primitives in the objects, but usually the stack frame is enough. In this case, you can even see the SQL query that caused too much data to be queried from the database.
Connecting objects to stack frames is one of the most powerful features of system dumps and the Memory Analyzer Tool.