MapReduce logs
Log files are an essential troubleshooting tool during testing and production and contain important runtime information about the general health of workload daemons and system services. IBM® Spectrum Symphony uses the log4j logging framework for MapReduce logging.
You can configure the system to be able to retrieve application logs that are written on all compute hosts on the grid from one central location, through the cluster management console.
Log files and location
The MapReduce framework in IBM Spectrum Symphony provides the following log files, which are located under the $PMR_HOME/logs/ directory. The following table describes the log files and lists their location.
| Component | Log file description | Log file location |
|---|---|---|
| MapReduce service | Service instance log that contains details related to the MapReduce framework and startup on the service side. Each service instance has a separate log, in addition to separate logs for each task attempt. | $PMR_HOME/logs/user.pmr.service.application.index.log (for example, $PMR_HOME/logs/root.pmr.service.MapReduce7.3..1.log). |
| MapReduce task | Task log that contains details relating to user-defined MapReduce code on the service side. Each task records its task log messages (syslog), standard output (stdout) and errors (stderr) to separate files in this directory. Knowing the job and task IDs can be very useful when debugging MapReduce jobs. |
|
| Shuffle daemon | MRSS service log that captures messages, events, and errors related to the MapReduce host. | $PMR_HOME/logs/mrss.hostname.log |
| MapReduce API | API log that captures details relating to API calls from EGO. | api.hostname.log in the directory where the job was submitted |
| MapReduce client | MapReduce client messages are not recorded in a log file. You can only view them on the client console. | None |