IBM Support

Resolving an OutOfMemoryError when running a large query in Hive

Troubleshooting


Problem

You are running a large query in Hive and receive an OutOfMemoryError.

Symptom

You ran a large query in Hive shell and received the following error:


JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2013/06/14 09:29:55 - please wait.
JVMDUMP032I JVM requested Heap dump using '/home/xxxx/heapdump.20130614.092955.30523.0001.phd' in response to an event
JVMDUMP010I Heap dump written to /home/xxxx/heapdump.20130614.092955.30523.0001.phd
JVMDUMP032I JVM requested Java dump using '/home/xxxx/javacore.20130614.092955.30523.0002.txt' in response to an event
JVMDUMP010I Java dump written to /home/xxxx/javacore.20130614.092955.30523.0002.txt
JVMDUMP032I JVM requested Snap dump using '/home/xxxx/Snap.20130614.092955.30523.0003.trc' in response to an event
JVMDUMP010I Snap dump written to /home/xxxx/Snap.20130614.092955.30523.0003.trc
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2013/06/14 09:29:59 - please wait.
JVMDUMP032I JVM requested Heap dump using '/home/xxxx/heapdump.20130614.092959.30523.0004.phd' in response to an event
JVMDUMP010I Heap dump written to /home/xxxx/heapdump.20130614.092959.30523.0004.phd
JVMDUMP032I JVM requested Java dump using '/home/xxxx/javacore.20130614.092959.30523.0005.txt' in response to an event
JVMDUMP010I Java dump written to /home/xxxx/javacore.20130614.092959.30523.0005.txt
JVMDUMP032I JVM requested Snap dump using '/home/xxxx/Snap.20130614.092959.30523.0006.trc' in response to an event
JVMDUMP010I Snap dump written to /home/xxxx/Snap.20130614.092959.30523.0006.trc
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at org.apache.derby.client.am.Cursor.allocateCharBuffer(Unknown Source)
at org.apache.derby.client.net.NetStatementReply.parseSQLDTARDarray(Unknown Source)
at org.apache.derby.client.net.NetStatementReply.parseQRYDSC(Unknown Source)
at org.apache.derby.client.net.NetStatementReply.parseOpenQuery(Unknown Source)
at org.apache.derby.client.net.NetStatementReply.parseOPNQRYreply(Unknown Source)
at org.apache.derby.client.net.NetStatementReply.readOpenQuery(Unknown Source)
at org.apache.derby.client.net.StatementReply.readOpenQuery(Unknown Source)
at org.apache.derby.client.net.NetStatement.readOpenQuery_(Unknown Source)

[{"Product":{"code":"SSCRJT","label":"IBM Db2 Big SQL"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Component":"Open Source Tools","Platform":[{"code":"PF016","label":"Linux"}],"Version":"1.4.0;2.0.0;2.1.0","Edition":"","Line of Business":{"code":"LOB76","label":"Data Platform"}}]

Log InLog in to view more of this document

This document has the abstract of a technical article that is available to authorized users once you have logged on. Please use Log in button above to access the full document. After log in, if you do not have the right authorization for this document, there will be instructions on what to do next.

Document Information

Modified date:
18 July 2020

UID

swg21641548

Page Feedback