Topic
  • 3 replies
  • Latest Post - ‏2012-04-19T23:46:46Z by Morgan Sousa
ttadeo
ttadeo
2 Posts

Pinned topic hadoop-0.20.2-examples.jar Job Fails

‏2012-03-19T21:00:07Z |
Hi All,

Using BI v1.3 Enterprise. Tried to execute the "examples.jar" job and it fails with the following errors. Looking through the output below I found where it can not find files/directories in the /input directory. However, they are there:

biadmin@bi01 ~$ hadoop fs -ls /input

Found 2 items

drwxr-xr-x - biadmin supergroup 0 2012-03-15 13:45 /input/etcdrwxr-xr-x - biadmin supergroup 0 2012-03-15 13:48 /input/statsFed

Any help would be appreciated. Thanks.

biadmin@bi01 ~$ hadoop jar /opt/ibm/biginsights/IHC/hadoop-0.20.2-examples.jar wordcount /input output12/03/15 13:52:23 INFO input.FileInputFormat: Total input paths to process : 212/03/15 13:52:24 INFO mapred.JobClient: Running job: job_201203110932_001412/03/15 13:52:25 INFO mapred.JobClient: map 0% reduce 0%12/03/15 13:52:40 INFO mapred.JobClient: Task Id : attempt_201203110932_0014_m_000000_0, Status : FAILEDjava.io.IOException: Cannot open filename /input/etc at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1497)at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1488)at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:376)at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:178)at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:356)at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:67)at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:418)at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:620)at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)at org.apache.hadoop.mapred.Child.main(Child.java:170)
12/03/15 13:52:40 INFO mapred.JobClient: Task Id : attempt_201203110932_0014_m_000001_0, Status : FAILEDjava.io.IOException: Cannot open filename /input/statsFed at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1497)at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1488)at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:376)at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:178)at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:356)at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:67)at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:418)at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:620)at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)at org.apache.hadoop.mapred.Child.main(Child.java:170)
Updated on 2012-04-19T23:46:46Z at 2012-04-19T23:46:46Z by Morgan Sousa
  • SystemAdmin
    SystemAdmin
    603 Posts

    Re: hadoop-0.20.2-examples.jar Job Fails

    ‏2012-03-19T22:30:11Z  
    Hi,

    In order for the worcount example to work, your input directory should contain files only. You get this error because it finds sub-directies (i.e. etc) and tries to open them as files.

    Hope this helps.

    Thanks,

    Zach
  • ttadeo
    ttadeo
    2 Posts

    Re: hadoop-0.20.2-examples.jar Job Fails

    ‏2012-03-21T14:10:20Z  
    Thanks for the help Zach. Your solution was correct. Worth noting there was already a directory that was named "input" which had directories and files present previously. This was something that the install did. So I just used another directory i created with the name inpu_lab, copied my input files and now the execution was successful.

    Thanks again !
  • Morgan Sousa
    Morgan Sousa
    7 Posts

    Re: hadoop-0.20.2-examples.jar Job Fails

    ‏2012-04-19T23:46:46Z  
    I updated the V1.4 Problems and Workarounds topic with this issue.