Topic
  • 3 replies
  • Latest Post - ‏2012-05-07T14:48:47Z by Brenagwynn
Brenagwynn
Brenagwynn
3 Posts

Pinned topic Problems with external access to HDFS

‏2012-04-27T10:05:02Z |
Hi all!
I'm trying to use BI as an HDFS file storage. While I'm able to access all my files using the web console and JAQL inside the box, the same does not apply to the Java application I've created which runs externally, on another machine, not in BI. I get the well-known

INFO: Could not obtain block blk_.................._.... from any node:  java.io.IOException: No live nodes contain current block
I've raised dfs.datanode.max.xcievers to 4096 (and even to 8192) and max open file limits to 32K, to no avail. Kinda frustrated already, and google doesn't seem to help anymore. There are no exceptions logged in both namenode and datanode logs. Hadoop fsck says that the filesystem is clean.
I'm running BI 1.3, with 1 datanode running on the same machine as namenode.

Does anyone know what to do with this?
Thanks.
Updated on 2012-05-07T14:48:47Z at 2012-05-07T14:48:47Z by Brenagwynn
  • Brenagwynn
    Brenagwynn
    3 Posts

    Re: Problems with external access to HDFS

    ‏2012-04-27T10:48:38Z  
    More on this. I'm seeing that my application actually tries to access the file in the namenode log:
    
    2012-04-27 05:45:48,839 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=.......  ip=/.....  cmd=open  src=/test/src_email.txt dst=
    
    null perm=
    
    null
    


    But nothing appears in datanode log. Strange...
  • LukeHadoop
    LukeHadoop
    5 Posts

    Re: Problems with external access to HDFS

    ‏2012-04-30T21:26:32Z  
    Sounds like a firewall issue, as you're able to use jaql inside the box. You can confirm that by running your java application on the same box.
  • Brenagwynn
    Brenagwynn
    3 Posts

    Re: Problems with external access to HDFS

    ‏2012-05-07T14:48:47Z  
    Sounds like a firewall issue, as you're able to use jaql inside the box. You can confirm that by running your java application on the same box.
    Thanks, it was really a firewall problem - ports 50010, 50075, 50020, 50070, 50090 were closed for external access. After opening them everything works fine.