dfsput操作报“There are 0 datanode(s) running and no node(s) are excluded in this operation”

.  There are 0 datanode(s) running and no node(s) are excluded in this operation.
  $ bin/hdfs dfs -mkdir /user
  $ bin/hdfs dfs -mkdir /user/<username>
  $bin/hdfs dfs -put etc/hadoop input  报如下错:
  17/06/28 00:00:36 WARN hdfs.DataStreamer: DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/root/input/hadoop/capacity-scheduler.xml._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1).  There are 0 datanode(s) running and no node(s) are excluded in this operation.
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1733)
        at org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.chooseTargetForNewBlock(FSDirWriteFileOp.java:265)


vi /usr/src/hadoop/logs/hadoop-root-datanode-centos128.log  查看日志,发现clusterIDs不兼容。想到重建的原因。故想删除历史文件
2017-06-28 00:48:21,085 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /tmp/hadoop-root/dfs/data/in_use.lock acquired by nodename 2533@centos128
2017-06-28 00:48:21,089 WARN org.apache.hadoop.hdfs.server.common.Storage: Failed to add storage directory [DISK]file:/tmp/hadoop-root/dfs/data/
java.io.IOException: Incompatible clusterIDs in /tmp/hadoop-root/dfs/data: namenode clusterID = CID-00699a28-9ec0-4301-ae92-5a04cf302565; datanode clusterID = CID-3c15bdd3-4886-41ab-a3a8-d899d0b607b9


[root@centos128 hadoop]# sbin/stop-dfs.sh     清空/tmp下相关的文件
Stopping namenodes on [localhost]
localhost: stopping namenode
localhost: no datanode to stop
Stopping secondary namenodes [0.0.0.0]
0.0.0.0: stopping secondarynamenode



[root@centos128 hadoop]# ll /tmp
total 0
drwxr-xr-x. 3 root root 20 Jun 27 23:46 hadoop
drwxr-xr-x. 4 root root 31 Jun 27 23:46 hadoop-root
drwxr-xr-x. 2 root root  6 Jun 28 00:57 hsperfdata_root
drwxr-xr-x. 3 root root 21 Jun 28 00:48 Jetty_0_0_0_0_50070_hdfs____w2cu08
drwxr-xr-x. 3 root root 21 Jun 28 00:48 Jetty_0_0_0_0_50090_secondary____y6aanv
drwxr-xr-x. 3 root root 21 Jun 27 23:01 Jetty_localhost_33451_datanode____ihzion
drwxr-xr-x. 3 root root 21 Jun 28 00:19 Jetty_localhost_34384_datanode____gijfbp
drwxr-xr-x. 3 root root 21 Jun 28 00:48 Jetty_localhost_38324_datanode____wwolqv
drwxr-xr-x. 3 root root 21 Jun 27 23:49 Jetty_localhost_40770_datanode____.cx9tc5
drwxr-xr-x. 3 root root 21 Jun 27 23:59 Jetty_localhost_43407_datanode____.hgiwfx
drwxr-xr-x. 3 root root 16 Jun  9 06:34 pip-build-hv2v27iz
drwx------. 3 root root 17 Jun 28 00:27 systemd-private-fc429f3080ff45c493f8d0afc4a92699-vmtoolsd.service-j5KhNK
[root@centos128 hadoop]# cd /tmp
[root@centos128 tmp]# rm -rf hsp* Jett* systemd* hadopp*
[root@centos128 tmp]# ll
total 0
drwxr-xr-x. 3 root root 20 Jun 27 23:46 hadoop
drwxr-xr-x. 4 root root 31 Jun 27 23:46 hadoop-root
drwxr-xr-x. 3 root root 16 Jun  9 06:34 pip-build-hv2v27iz
[root@centos128 tmp]# rm -rf hadoop*
[root@centos128 tmp]# ll
total 0
drwxr-xr-x. 3 root root 16 Jun  9 06:34 pip-build-hv2v27iz


重新format namenode
bin/hdfs namenode -format
sbin/start-dfs.sh


Browse the web interface for the NameNode; by default it is available at:
    NameNode - http://192.168.44.128:50070/
发现有一个live nodes ,并且Datanode Information 有信息

[root@centos128 hadoop]# bin/hdfs dfs -mkdir /user
[root@centos128 hadoop]# bin/hdfs dfs -mkdir /user/root
[root@centos128 hadoop]# bin/hdfs dfs -put etc/hadoop input
没有报错,正常。下面继续操作

[root@centos128 hadoop]# bin/hdfs dfs -ls
Found 2 items
drwxr-xr-x   - root supergroup          0 2017-06-28 01:05 input
drwxr-xr-x   - root supergroup          0 2017-06-28 01:06 output
[root@centos128 hadoop]# bin/hdfs dfs -ls output
Found 2 items
-rw-r--r--   1 root supergroup          0 2017-06-28 01:06 output/_SUCCESS
-rw-r--r--   1 root supergroup        220 2017-06-28 01:06 output/part-r-00000
[root@centos128 hadoop]# bin/hdfs dfs -ls input/hadoop
ls: `input/hadoop': No such file or directory
[root@centos128 hadoop]# bin/hdfs dfs -ls input
Found 29 items
-rw-r--r--   1 root supergroup       4942 2017-06-28 01:05 input/capacity-scheduler.xml
-rw-r--r--   1 root supergroup       1335 2017-06-28 01:05 input/configuration.xsl
-rw-r--r--   1 root supergroup        318 2017-06-28 01:05 input/container-executor.cfg
-rw-r--r--   1 root supergroup        884 2017-06-28 01:05 input/core-site.xml
-rw-r--r--   1 root supergroup       3804 2017-06-28 01:05 input/hadoop-env.cmd
-rw-r--r--   1 root supergroup       4696 2017-06-28 01:05 input/hadoop-env.sh
-rw-r--r--   1 root supergroup       2490 2017-06-28 01:05 input/hadoop-metrics.properties
-rw-r--r--   1 root supergroup       2598 2017-06-28 01:05 input/hadoop-metrics2.properties
-rw-r--r--   1 root supergroup       9683 2017-06-28 01:05 input/hadoop-policy.xml
-rw-r--r--   1 root supergroup        867 2017-06-28 01:05 input/hdfs-site.xml
-rw-r--r--   1 root supergroup       1449 2017-06-28 01:05 input/httpfs-env.sh
-rw-r--r--   1 root supergroup       1657 2017-06-28 01:05 input/httpfs-log4j.properties
-rw-r--r--   1 root supergroup         21 2017-06-28 01:05 input/httpfs-signature.secret
-rw-r--r--   1 root supergroup        620 2017-06-28 01:05 input/httpfs-site.xml
-rw-r--r--   1 root supergroup       3518 2017-06-28 01:05 input/kms-acls.xml
-rw-r--r--   1 root supergroup       1611 2017-06-28 01:05 input/kms-env.sh
-rw-r--r--   1 root supergroup       1631 2017-06-28 01:05 input/kms-log4j.properties
-rw-r--r--   1 root supergroup       5546 2017-06-28 01:05 input/kms-site.xml
-rw-r--r--   1 root supergroup      13661 2017-06-28 01:05 input/log4j.properties
-rw-r--r--   1 root supergroup        951 2017-06-28 01:05 input/mapred-env.cmd
-rw-r--r--   1 root supergroup       1383 2017-06-28 01:05 input/mapred-env.sh
-rw-r--r--   1 root supergroup       4113 2017-06-28 01:05 input/mapred-queues.xml.template
-rw-r--r--   1 root supergroup        758 2017-06-28 01:05 input/mapred-site.xml.template
-rw-r--r--   1 root supergroup         10 2017-06-28 01:05 input/slaves
-rw-r--r--   1 root supergroup       2316 2017-06-28 01:05 input/ssl-client.xml.example
-rw-r--r--   1 root supergroup       2697 2017-06-28 01:05 input/ssl-server.xml.example
-rw-r--r--   1 root supergroup       2250 2017-06-28 01:05 input/yarn-env.cmd
-rw-r--r--   1 root supergroup       4567 2017-06-28 01:05 input/yarn-env.sh
-rw-r--r--   1 root supergroup        690 2017-06-28 01:05 input/yarn-site.xml
[root@centos128 hadoop]# bin/hdfs dfs -get output output
[root@centos128 hadoop]# cat output/*
cat: output/output: Is a directory
1    dfsadmin













  • 0
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值