Troubleshooting
Problem
IBM BigInsights: Hive join queries aborts with error code state=08S01,code=1
Symptom
Queries without joins run fine, however join queries abort with following error messages
ERROR : /var/log/hive/hiveserver2.log
Error: Error while processing statement: FAILED: Execution Error,
return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
(state=08S01,code=1)
Cause
Column stats missing for the table.
This can be confirmed by disabling join optimization in hive by running following command.
SET hive.auto.convert.join=false
After disabling the optimization queries will run.
Environment
Linux
Diagnosing The Problem
Generate explain plan by running EXPLAIN EXTENDED <QUERY>
The explain plan will show what kind of join operation is being performed and if the column stats are found
Following is an example output from explain:
Statistics: Num rows: 138781 Data size: 27894993 Basic stats: COMPLETE Column stats: NONE |
| Map Join Operator |
| condition map: |
| Inner Join 0 to 1 |
Column Stats: NONE indicates no stats were found for the join column.
Resolving The Problem
Run Hive Analyze for the table columns
More details about hive analyze command and its syntax is documented in following apache hive link
https://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ExistingTables
Was this topic helpful?
Document Information
Modified date:
18 July 2020
UID
swg22001681