大数据学习_问题解决_设定hdfs短路读取后datanode不启动问题

出现问题：

在进行impala配置时，由于设置了短路读取，导致启动dfs时无法启动datanode，集群也一直处于安全模式中。

# 2.修改hdfs-site.xml
<!--添加如下内容 -->
	<!--打开短路读取开关 -->
	<!-- 打开短路读取配置-->
	<property>
		<name>dfs.client.read.shortcircuit</name>
		<value>true</value>
	</property>
<!--这是⼀个UNIX域套接字的路径，将⽤于DataNode和本地HDFS客户机之间的通信 -->
	<property>
		<name>dfs.domain.socket.path</name>
		<value>/var/lib/hadoop-hdfs/dn_socket</value>
	</property>

原因：

查看日志，由于hadoop-hdfs该文件夹的拥有者并不是root导致问题的发生。

java.io.IOException: The path component: '/var/lib/hadoop-hdfs' in '/var/lib/hadoop-hdfs/dn_socket' has permissions 0755 uid 993 and gid 991. It is not protected because it is owned by a user who is not root and not the effective user: '0'. This might help: 'chown root /var/lib/hadoop-hdfs' or 'chown 0 /var/lib/hadoop-hdfs'. For more information: https://wiki.apache.org/hadoop/SocketPathSecurity
	at org.apache.hadoop.net.unix.DomainSocket.validateSocketPathSecurity0(Native Method)
	at org.apache.hadoop.net.unix.DomainSocket.bindAndListen(DomainSocket.java:193)
	at org.apache.hadoop.hdfs.net.DomainPeerServer.<init>(DomainPeerServer.java:40)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.getDomainPeerServer(DataNode.java:1171)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.initDataXceiver(DataNode.java:1137)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1369)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:495)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2695)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2598)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2645)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2789)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2813)

同时，dn_socket这个文件夹会由hdfs自行创建，自己不可以提前创建，否则会报错：

java.net.BindException: bind(2) error: Address already in use when trying to bind to '/var/lib-hdfs/dn_socket'
	at org.apache.hadoop.net.unix.DomainSocket.bind0(Native Method)
	at org.apache.hadoop.net.unix.DomainSocket.bindAndListen(DomainSocket.java:195)
	at org.apache.hadoop.hdfs.net.DomainPeerServer.<init>(DomainPeerServer.java:40)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.getDomainPeerServer(DataNode.java:1
	at org.apache.hadoop.hdfs.server.datanode.DataNode.initDataXceiver(DataNode.java:1137)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1369)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:495)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2695)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2
	at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2645)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2789)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2813)
2021-02-25 00:08:04,125 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1: java.net.eption: bind(2) error: Address already in use when trying to bind to '/var/lib/hadoop-hdfs/dn_
2021-02-25 00:08:04,340 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at centos7-1/192.168.80.100
************************************************************/

解决办法：

在所有集群上设置，然后重启，即可完美解决。

chown root /var/lib/hadoop-hdfs