java 访问hdfs,我如何通过java的hdfs协议访问hadoop?

I found a way to connect to hadoop via hftp, and it works fine, (read only) :

uri = "hftp://172.16.xxx.xxx:50070/";

System.out.println( "uri: " + uri );

Configuration conf = new Configuration();

FileSystem fs = FileSystem.get( URI.create( uri ), conf );

fs.printStatistics();

However, I want to read/write as well as copy files, that is, I want to connect over hdfs . How can I enable hdfs connections so that i can edit the actual , remote filesystem ?

I tried to change the protocol above from "hftp" -> "hdfs", but I got the following exception ...

(forgive my poor knowledge of url protocols and hadoop , I assume this is a somewhat strange question im asking, but any help would really be appreciated !)

Exception in thread "main" java.io.IOException: Call to

/172.16.112.131:50070 failed on local exception: java.io.EOFException

at org.apache.hadoop.ipc.Client.wrapException(Client.java:1139) at

org.apache.hadoop.ipc.Client.call(Client.java:1107) at

org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226) at

$Proxy0.getProtocolVersion(Unknown Source) at

org.apache.hadoop.ipc.RPC.getProxy(RPC.java:398) at

org.apache.hadoop.ipc.RPC.getProxy(RPC.java:384) at

org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:111)

at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:213) at

org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:180) at

org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)

at

org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1514)

at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:67) at

org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1548)

at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1530)

at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:228) at

sb.HadoopRemote.main(HadoopRemote.java:24)

解决方案

Regarding hadoop : You need to make sure the core-site.xml namenode entry is serving to 0.0.0.0 instead of 127.0.0.1(localhost) in your hadoop configuration. Importantly, clouderas vm distro defaults to localhost for some reason.

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值