从本地往hadoop上复制文件失败

环境

Centos7 - 1台虚拟机
hadoop-3.2.1
OpenJDK1.8.0_181
root用户 - 伪分布式

执行bin/hdfs dfs -put etc/hadoop/*.xml input失败

[root@ip10 hadoop-3.2.1]# bin/hdfs dfs -put etc/hadoop/*.xml input
2020-05-17 21:28:33,784 WARN hdfs.DataStreamer: DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/root/input/capacity-scheduler.xml._COPYING_ could only be written to 0 of the 1 minReplication nodes. There are 0 datanode(s) running and 0 node(s) are excluded in this operation.
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:2219)
	at 
	..........

经多方考证以及测试发现,理论上应该是多次格式化导致的原因,我是因为开始使用2.10.0版本,后来换的hadoop-3.2.1
解决方案:删除hdfs默认配置的格式化的目录,重新格式化。根据官网步骤,未配置路径,默认在/tmp下

#我是新的虚拟机,所以直接干掉了/tmp下所有文件,
#如果有顾虑,可以仅删除与hadoop相关目录,具体目录,等我看到了再回来补
[root@ip10 /]# sbin/stop-dfs.sh #关闭服务
[root@ip10 /]# rm -rf /tmp/*
[root@ip10 /]# bin/hdfs namenode -format

嗯。果然解决了如上问题,又有了如下问题

问题2:Name node is in safe mode.

[root@ip10 hadoop-3.2.1]# bin/hdfs dfs -put etc/hadoop/*.xml input
put: Cannot create file/user/root/input/capacity-scheduler.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/core-site.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/hadoop-policy.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/hdfs-site.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/httpfs-site.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/kms-acls.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/kms-site.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/mapred-site.xml._COPYING_. Name node is in safe mode.
put: Cannot create file/user/root/input/yarn-site.xml._COPYING_. Name node is in safe mode.

看描述是name节点默认是以safe模式启动的,关闭safe模式就好了。
解决方案:

[root@ip10 hadoop-3.2.1]# bin/hadoop dfsadmin -safemode leave
WARNING: Use of this script to execute dfsadmin is deprecated.
WARNING: Attempting to execute replacement "hdfs dfsadmin" instead.

Safe mode is OFF

至此,问题解决,执行bin/hdfs dfs -put etc/hadoop/*.xml input成功复制到了hdfs上。
至于为什么safe模式复制不了,等我学到了再回来补。
在这里插入图片描述

问题家家有,这家特别多

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值