阿里云搭建伪分布式遇到这个问题,正常办法解决不了了的时候,可以一试,步骤如下:
1、/etc/hosts 文件的多余主机名全部注释掉 ,新增加一个你的局域网IP 和 hadoop的用户名;
注意:这里是内网地址,而不是阿里云的外网地址,先 ifconfig 查看本机内网ip
[root@cx hadoop]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.19.44.21 netmask 255.255.240.0 broadcast 172.19.47.255
ether 00:16:3e:1c:28:6e txqueuelen 1000 (Ethernet)
RX packets 675510 bytes 256484775 (244.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 463354 bytes 323423726 (308.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1 (Local Loopback)
RX packets 16893986 bytes 1619851840 (1.5 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 16893986 bytes 1619851840 (1.5 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
2、更改 /etc/hosts 文件,设置成局域网网IP 主机名,多余的注释或者删掉
[root@cx ~]# cat /etc/hosts
#0.0.0.0 cx
172.19.44.21 cx
#127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
#127.0.1.1 localhost
#:::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
3、客户端添加映射,我这里用的是windos远程连接的hdfs ,编辑 C:\Windows\System32\drivers\etc\hosts文件,加入云服务器的公网IP和主机名映射。
# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
# 102.54.94.97 rhino.acme.com # source server
# 38.25.63.10 x.acme.com # x client host
# localhost name resolution is handled within DNS itself.
# 127.0.0.1 localhost
# ::1 localhost
#203.208.46.146 dl.google.com
#203.208.46.146 dl-ssl.google.com
#0.0.0.0 account.jetbrains.com
172.128.120.66 windows10.microdone.cn
106.打码.打码.打码 cx
4、连接hdfs时,设置下配置方式,conf.set(“dfs.client.use.datanode.hostname”, “true”); 设置客户端访问方式使用主机名
public class HdfsTest {
Configuration conf = null;
FileSystem fs = null;
@org.junit.Before
public void conn () throws Exception{
conf =new Configuration(false);
conf.set("fs.defaultFS", "hdfs://cx:8020");
conf.set("dfs.replication", "1");
conf.set("dfs.block.size", "3145728");
conf.set("dfs.client.use.datanode.hostname", "true");
fs = FileSystem.get(conf);
}
}
至此,就可以正确上传文件到hdfs了。