在使用hadoop上传文件时候失败
在master中直接报WARN hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/root/input/configuration.xsl could only be replicated to 0 nodes, instead of 1
用hadoop dfsadmin -report 查看文件系统使用情况
网上一查,都是什么防火墙,目录等等错误
实际还是要以datanode日志为主
日志中错误
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
at org.apache.hadoop.ipc.Client
Connection.setupConnection(Client.java:682)atorg.apache.hadoop.ipc.Client
Connection.setupIOstreams(Client.java:778)
at org.apache.hadoop.ipc.Client
Connection.access
3500(Client.java:410)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1544)
at org.apache.hadoop.ipc.Client.call(Client.java:1375)
at org.apache.hadoop.ipc.Client.call(Client.java:1339)
at org.apache.hadoop.ipc.ProtobufRpcEngine
Invoker.invoke(ProtobufRpcEngine.java:227)atorg.apache.hadoop.ipc.ProtobufRpcEngine
Invoker.invoke(ProtobufRpcEngine.java:116)
at com.sun.proxy.$Proxy14.versionRequest(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB.versionRequest(DatanodeProtocolClientSideTranslatorPB.java:274)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.retrieveNamespaceInfo(BPServiceActor.java:215)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:261)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:750)
at java.lang.Thread.run(Thread.java:745)
2017-11-28 19:51:25,051 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: s0/192.168.133.200:8020
就是datanode 不能通过8020端口连接namenode
原因是我在namenode中core-site.xml
fs.defaultFS
hdfs://s0:9000
用了9000端口,但是在datanode中没有写9000端口,所以默认是8020端口,当然连接不上
把namenode配置改成
fs.defaultFS
hdfs://s0
hadoop上传文件失败 hadoop fs -put 1901 /userdata
最新推荐文章于 2024-06-24 18:59:20 发布