Topic
  • 1 reply
  • Latest Post - ‏2012-12-08T14:18:41Z by SystemAdmin
SystemAdmin
SystemAdmin
603 Posts

Pinned topic Help getting hadoop WordCount example to compile

‏2012-12-07T00:36:02Z |
I tore down all my VM's and built 10 new ones with the new version of big insights, a master and 9 data nodes using the new image (hadoop 1.0.0). After finally resolving where the new examples are and the proper classpath JAR file, I still get this error message:
idcuser@vhost0526 examples$ cd /mnt/biginsights/opt/ibm/biginsights/hdm/IHC/src/examples/org/apache/hadoop/examples
idcuser@vhost0526 examples$ javac -classpath ${HADOOP_HOME}/hadoop-core-1.0.0.jar -d wordcount_classes WordCount.java
WordCount.java:53: cannot access org.apache.commons.cli.Options
class file for org.apache.commons.cli.Options not found
String[] otherArgs = new GenericOptionsParser(conf, args).getRemainingArgs();
^
1 error

I can not find any other file it include in the classpath to make this work. Does anyone have an idea? I need to get this working so I can do modifications for my work.

Thanks
Bob
Updated on 2012-12-08T14:18:41Z at 2012-12-08T14:18:41Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    603 Posts

    Re: Help getting hadoop WordCount example to compile

    ‏2012-12-08T14:18:41Z  
    Hi Bob,

    The "class file for org.apache.commons.cli.Options not found" told us the commons-cli jar is missing, which you could find under $HADOOP_HOME/lib. Actually, you are not only missing this single jar, I would suggest you add all jars under $HADOOP_HOME/lib into java classpath, please try:

    javac -cp /opt/ibm/biginsights/IHC/hadoop-core-1.0.0.jar:$HADOOP_HOME/lib/* -d wordcount_classes WordCount.java

    Actually, if you simply want to try word count, you could use the hadoop-example-*.jar instead, like:
    hadoop jar $HADOOP_HOME/hadoop-examples-*.jar wordcount <input_path> <output_path>

    Hope this helps.