IBM Support

Mapreduce program failing while executing hive queries

Troubleshooting


Problem

Running a select count query in hive client generates an error Error during job, obtaining debugging information... Unable to retrieve URL for Hadoop Task logs. Unable to find job tracker info port. Exception in thread "Thread-79" java.lang.RuntimeException: Error while reading from task log url at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getStackTraces(TaskLogProcessor.java:255)

Symptom

Hive CLI:
----------
hive> select count(*) from <table>;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201610100613_3213, Tracking URL = httx://localhost:50030/jobdetails.jsp?jobid=job_201610100613_3213
Kill Command = /opt/ibm/biginsights/IHC/bin/hadoop job -kill job_201610100613_3213
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2016-10-17 06:17:09,095 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201610100613_3213 with errors
Error during job, obtaining debugging information...
Job Tracking URL: httx://localhost:50030/jobdetails.jsp?
jobid=job_201610100613_3213
Examining task ID: task_201610100613_3213_m_000002 (and more) from job job_201610100613_3213
Examining task ID: task_201610100613_3213_m_000001 (and more) from job job_201610100613_3213
Unable to retrieve URL for Hadoop Task logs. Unable to find job tracker info port.
Exception in thread "Thread-79" java.lang.RuntimeException: Error while reading from task log url
at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.
getStackTraces(TaskLogProcessor.java:255)
at org.apache.hadoop.hive.ql.exec.mr.JobDebugger.
showJobFailDebugInfo(JobDebugger.java:285)
at org.apache.hadoop.hive.ql.exec.mr.JobDebugger.run
(JobDebugger.java:118)
at java.lang.Thread.run(Thread.java:853)
Caused by: java.io.FileNotFoundException: httx://localhost:50060/tasklog?attemptid=attempt_201610100613_3213_m_000002_3
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.ja…
at java.net.URL.openStream(URL.java:1049)
at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getStackTraces(TaskLogProcessor.java:196)
... 3 more

[{"Product":{"code":"SSCRJT","label":"IBM Db2 Big SQL"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Component":"Open Source Tools","Platform":[{"code":"PF016","label":"Linux"}],"Version":"3.0.0.2","Edition":"","Line of Business":{"code":"LOB76","label":"Data Platform"}}]

Log InLog in to view more of this document

This document has the abstract of a technical article that is available to authorized users once you have logged on. Please use Log in button above to access the full document. After log in, if you do not have the right authorization for this document, there will be instructions on what to do next.

Document Information

More support for:
IBM Db2 Big SQL

Software version:
3.0.0.2

Operating system(s):
Linux

Document number:
555195

Modified date:
18 July 2020

UID

swg21992821

Manage My Notification Subscriptions