环境
Centos7 - 1台虚拟机
hadoop-3.2.1
OpenJDK1.8.0_181
root用户 - 伪分布式
执行bin/hdfs dfs -put etc/hadoop/*.xml input失败
[root@ip10 hadoop-3.2.1]# bin/hdfs dfs -put etc/hadoop/*.xml input
2020-05-17 21:28:33,784 WARN hdfs.DataStreamer: DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/root/input/capacity-scheduler.xml._COPYING_ could only be written to 0 of the 1 minReplication nodes. There are 0 datanode(s) running and 0 node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:2219)
at
..........
经多方考证以及测试发现,理论上应该是多次格式化导致的原因,我是因为开始使用2.10.0版本,后来换的hadoop-3.2.1
解决方案:删除hdfs默认配置的格式化的目录,重新格式化。根据官网步骤,未配置路径,默认在/tmp下
#我是新的虚拟机,所以直接干掉了/tmp下所有文件,
#如果有顾虑,可以仅删除与hadoop相关目录,具体目录,等我看到了再回来补
[root@ip10 /]# sbin/stop-dfs.sh #关闭服务
[root@ip10 /]# rm -rf /tmp/*
[root@ip10 /]# bin/hdfs namenode -format
嗯。果然解决了如上问题,又有了如下问题
问题2:Name node is in safe mode.
[root@ip10 hadoop-3.2.1]# bin/hdfs dfs -put etc/hadoop/*.xml input
put: Cannot create file/user/root/input/capacity-scheduler.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/core-site.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/hadoop-policy.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/hdfs-site.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/httpfs-site.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/kms-acls.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/kms-site.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/mapred-site.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/yarn-site.xml._COPYING_. Name node is in safe mode.
看描述是name节点默认是以safe模式启动的,关闭safe模式就好了。
解决方案:
[root@ip10 hadoop-3.2.1]# bin/hadoop dfsadmin -safemode leave
WARNING: Use of this script to execute dfsadmin is deprecated.
WARNING: Attempting to execute replacement "hdfs dfsadmin" instead.
Safe mode is OFF
至此,问题解决,执行bin/hdfs dfs -put etc/hadoop/*.xml input
成功复制到了hdfs上。
至于为什么safe模式复制不了,等我学到了再回来补。
问题家家有,这家特别多