HDFS/Hadoop启动后不能正常连接访问服务端口

HDFS/Hadoop配置中的网络问题

服务正常启用，但是Telnet无法正常访问服务端口

问题配置

hadoop/etc/hadoop/core-site.xml
服务端口通常配置为localhost(127.0.0.1)

<!-- 指定监听ip与运行端口 -->
<property>
    <name>fs.defaultFS</name>
    <value>hdfs://localhost:9000</value>
</property>

这会导致在一些情况下socket端口无法监听网络，下面演示异常的9000端口与正常的50079端口

netstat -tnlp |grep -E ':9000|:50070'
tcp  0  0  127.0.0.1:9000  0.0.0.0:*  LISTEN  1326/java
tcp  0  0  0.0.0.0:50070   0.0.0.0:*  LISTEN  1326/java

是否有看出什么问题呢？

问题原因

对，监听对象仅限于127.0.0.1，也就是lo网络回环口，经过网卡的网络流量是无法流经lo口访问127.0.0.1这个回环口ip的，它只能访问网卡上 的网络ip，而这个ip是没有处于监听状态的。

问题修正

问题原因知道了，解决就比较简单了，需要让监听ip置于在真实的网络访问流量之下就可以了，直接配置本机ip，或者变更/etc/hosts中的主机名。

<!-- 指定监听ip与运行端口 不能使用localhost(127.0.0.1) -->
<property>
    <name>fs.defaultFS</name>
    <value>hdfs://172.16.43.127:9000</value>
</property>

重启服务重新观察服务监听状态

netstat -tnlp |grep -E ':9000|:50070'
tcp  0  0  172.16.43.127:9000  0.0.0.0:*  LISTEN  1326/java
tcp  0  0  0.0.0.0:50070       0.0.0.0:*  LISTEN  1326/java

连接中的其它报错

错误1：

20/12/29 13:49:00 WARN BlockReaderFactory: I/O error constructing remote block reader.
java.net.ConnectException: Connection timed out: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)

错误2：

20/12/29 13:50:21 ERROR TaskSetManager: Task 1 in stage 0.0 failed 1 times; aborting job
Exception in thread “main” org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 0.0 failed 1 times, most recent failure: Lost task 1.0 in stage 0.0 (TID 1, localhost, executor driver): org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-415748031-172.16.43.127-1609137885414:blk_1073741825_1001 file=/test.txt
at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:930)
at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:609)
at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:841)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:889)

错误3：

Caused by: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-415748031-172.16.43.127-1609137885414:blk_1073741825_1001 file=/test.txt
at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:930)
at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:609)
at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:841)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:889)
at java.io.DataInputStream.read(DataInputStream.java:149)