IBM Support

Container killed by YARN for exceeding memory limits.

Troubleshooting


Problem

You are running an IBM SPSS Modeler Stream in IBM SPSS Analytic Server.  You are watching the Execution Feedback log and see the stream terminate with an error like "Stream fails with ASL execution error.". 

Symptom

You review the 'as_trace.log' file and see an error like this:

2018-09-15 13:41:46,676 | 10.85.128.45:9080 | ibm | admin |  | 165de0893a5-eb27836496a4554f | ERROR | spark.SparkJobRunner | OO-E(10)-P-4-T-6 => Fail to run spark job org.apache.spark.SparkException: Job aborted due to stage failure: Task 6 in stage 1000.0 failed 4 times, most recent failure: Lost task 6.3 in stage 1000.0 (TID 102082, spssas123.mydomain): ExecutorLostFailure (executor 491 exited caused by one of the running tasks) Reason: Container killed by YARN for exceeding memory limits. 27.5 GB of 27.5 GB physical memory used. Consider boosting spark.yarn.executor.memoryOverhead.

[{"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSWLVY","label":"IBM SPSS Analytic Server"},"Component":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB76","label":"Data Platform"}}]

Log InLog in to view more of this document

This document has the abstract of a technical article that is available to authorized users once you have logged on. Please use Log in button above to access the full document. After log in, if you do not have the right authorization for this document, there will be instructions on what to do next.

Document Information

Modified date:
20 September 2018

UID

ibm10732345

Page Feedback