hadoop fs -put optexerciseds.txt ds.txt 执行简单的上传文件命令遇到错误1： could only be replicated to 0 nodes inst-CSDN博客

[root@master hadoop-2.6.5]# hadoop fs -put optexerciseds.txt ds.txt

  hadoop fs -put /opt/execerise_data/Hadoop/chapter2/ds.txt ds.txt 

  管用的： 

 
 http://blog.csdn.net/yangkai_hudong/article/details/18792267 

 
 17/08/01 19:45:45 WARN hdfs.DFSClient: DataStreamer Exception 

 
 org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/root/ds.txt._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1). 

 
 org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/root/ds.txt._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1). There are 0 datanode(s) running and no node(s) are excluded in this operation. 

 
 at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1559) 

 
 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3245) 

 
 at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:663) 

 
 at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:482) 

 
 at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) 

 
 at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) 

 
 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:975) 

 
 at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040) 

 
 at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2036) 

 
 at java.security.AccessController.doPrivileged(Native Method) 

 
 at javax.security.auth.Subject.doAs(Subject.java:422) 

 
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692) 

 
 at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2034) 

 
 at org.apache.hadoop.ipc.Client.call(Client.java:1470) 

 
 at org.apache.hadoop.ipc.Client.call(Client.java:1401) 

 
 at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) 

 
 at com.sun.proxy.$Proxy9.addBlock(Unknown Source) 

 
 at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:399) 

 
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 

 
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 

 
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 

 
 at java.lang.reflect.Method.invoke(Method.java:498) 

 
 at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) 

 
 at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) 

 
 at com.sun.proxy.$Proxy10.addBlock(Unknown Source) 

 
 at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1528) 

 
 at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1345) 

 
 at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:587) 

 
 put: File /user/root/ds.txt._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1). There are 0 datanode(s) running and no node(s) are excluded in this operation. 

 
 可能原因 ： 

 
 1. 防火墙状态 

 
 service iptables status可以查看到iptables服务的当前状态。 

 
 但是即使服务运行了,防火墙也不一定起作用,你还得看防火墙规则的设置 iptables -L 

 
 在此说一下关于启动和关闭防火墙的命令: 

 
 1) 重启后生效 

 
 开启： chkconfig iptables on 

 
 关闭： chkconfig iptables off 

 
 2) 即时生效，重启后失效 

 
 开启： service iptables start 

 
 关闭： service iptables stop 

 
 关闭虚拟机防火墙： 

 
 关闭命令：  service iptables stop 

 
 永久关闭防火墙：chkconfig iptables off 

 
 两个命令同时运行，运行完成后查看防火墙关闭状态 

 
 service iptables status 

 
 1 关闭防火墙-----service iptables stop  

 
 2 启动防火墙-----service iptables start  

 
 3 重启防火墙-----service iptables restart  

 
 4 查看防火墙状态--service iptables status  

 
 5 永久关闭防火墙--chkconfig iptables off  

 
 6 永久关闭后启用--chkconfig iptables on 

 
 2.节点启动原因： 

 
 2.1 hadoop dfsadmin -report看集群的状态，估计两个节点都dead了 

 
 在你的datanode上jps一下。看有没有datanode 

 
 启动后执行jps，查看进程 

 
 20131 SecondaryNameNode 

 
 20449 NodeManager --failed 

 
 19776 NameNode 

 
 21123 Jps 

 
 19918 DataNode --failed 

 
 20305 ResourceManager -failed 

  DEPRECATED: Use of this script to execute hdfs command is deprecated. 

  Instead use the hdfs command for it. 

  Configured Capacity: 0 (0 B) 

  Present Capacity: 0 (0 B) 

  DFS Remaining: 0 (0 B) 

  DFS Used: 0 (0 B) 

  DFS Used%: NaN% 

  Under replicated blocks: 0 

  Blocks with corrupt replicas: 0 

  Missing blocks: 0 

 
 是内存不够，修改HDFS文件系统的目录就可以了！ 

 
 这个问题是由于没有添加节点的原因， 
 也就是说需要先启动namenode，再启动datanode，然后启动jobtracker和tasktracker。这样 
 就不会存在这个问题了。 目前解决办法是分别启动节点# 
 hadoop 
 -daemon.sh start namenode #$hadoop-daemon.sh start datanode 

 
 1.   重新启动namenode 

 
 # hadoop-daemon.sh start namenode 

 
 starting namenode, logging to /usr/hadoop-0.21.0/bin/../logs/hadoop-root-namenode-www.keli.com.out 

 
 hadoop-daemon.sh start  
 tasktracker 

 
 2.   重新启动datanode 

 
 # hadoop-daemon.sh start datanode 

 
 starting datanode, logging to /usr/hadoop-0.21.0/bin/../logs/hadoop-root-datanode-www.keli.co 