Log files for errors related to YARN

Examine the log files when you receive an error that is related to YARN.

Information about YARN operations is stored in three types of log files:
  • YARN client logs
  • Application Master logs
  • Resource Manager and Node Manager logs.

YARN client logs

The YARN client starts Application Masters that run the jobs on your Hadoop cluster. Errors that occur when you are starting a YARN client are logged in /tmp/yarn_client.out. Errors that occur after the YARN client is started are logged in $APT_ORCHHOME/logs/yarn_logs/yarn_client.out.

Application Master logs

The Application Master is what requests resources from YARN and runs job processes on the allocated resources. Two kinds of log files are generated:

Common log file
A single Application Manager can run multiple jobs. The common log file contains all messages that are not specific to a specific job. The name of the file starts application_ and includes a string of numbers. The file extension is .stdout. For example, the file might be application_1437631989065_0009.stdout.
Individual job log
An Application Master runs multiple jobs, which use Orchestrate Shell (OSH) commands for parallel processing. This log file is specific to a single job. The file name begins with oshjob., followed by the ID of the Application Master in the cluster, followed by an underscore (_) and the number of the job in the order that the Application Manager runs the jobs. For example, the file name oshjob.0009_0 indicates that the Application Master ID is 0009 and the job is the first in the series.
Application Master logs are stored on the node where the jog runs. Because jobs might run on any node in the cluster, open the job log in the InfoSphere® DataStage® and QualityStage® Designer client and look for messages similar to these messages:
  • Connecting to YARN Application Master at node_name:port_number
  • Application Master log location is path
. The first message provides the name of the node (computer), where the log is. The second message provides the path to both the individual and common log files on that node.

If you enable log aggregation by setting the configuration parameter yarn.log-aggregation-enable to true, the log files are moved to HDFS after the Application Master completes. To find the log files on HDFS, retrieve the job-tracking URL from the job log in the Designer client.

Resource Manager and Node Manager log files

If an error occurs at the YARN level, you might have to examine the log files for the Resource Manager and node managers. These files are on the computers that host the Resource Manager and each Node Manager. You would usually consult the Resource Manager log first. From that log, you can determine which Node Manager logs to check, if necessary.

The log files have a default location that differs based on the distribution you use. Unless you know that the default location is not used, look in the default location first. The following table shows the default location for log files for each distribution and the location where the default location can be overridden.

Distribution Default location for Resource Manager and Node Manager log files Where the default location can be overridden
Hortonworks /var/log/hadoop-yarn/yarn In the Ambari user interface, find the YARN Log Dir Prefix property in the YARN configuration file.
Cloudera /var/log/hadoop-yarn In the Cloudera Manager, find the hadoop.log.dir parameter in the Yarn configuration file.
BigInsights /var/log/hadoop-yarn In the Ambari user interface, find the YARN Log Dir Prefix property in the YARN configuration file.