IBM Support

Executing a code with in R connecting to Impala via JDBC ( RJDBC) results in: Error in .jcall(rp, "I", "fetch", stride) : java.lang.OutOfMemoryError: Java heap space RJDBC(dbGetQuery) - GC overhead limit exceeded

Troubleshooting


Problem

Executing a code with in R connecting to Impala via JDBC ( RJDBC) results in: Error in .jcall(rp, "I", "fetch", stride) : java.lang.OutOfMemoryError: Java heap space
RJDBC(dbGetQuery) - GC overhead limit exceeded

Symptom

Executing a code with in R connecting to Impala via JDBC ( RJDBC) results in: Error in .jcall(rp, "I", "fetch", stride) : java.lang.OutOfMemoryError: Java heap space
RJDBC(dbGetQuery) - GC overhead limit exceeded

Cause

java.lang.OutOfMemoryError: Java heap space

Environment

Linux

DSXL 1.2.0.3

Resolving The Problem

The java.lang.OutOfMemoryError on fetch means that the result set is bigger than the default heap size for rJava: 512MB.
To increase the java heap size use the following in your code:
options(java.parameters = "-Xmx4g") where 4g is the size you want to set.   It might be different depending on specific needs and availability.
Note:  that this step must be performed prior to loading any packages and it will be the first line in the code snippet.

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSKKD2","label":"IBM Data Science Experience Local"},"Component":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"1.2.0.3","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
24 October 2018

UID

ibm10737231

Page Feedback