hadoop 不使用hostname，使用ip遇到的问题（Datanode denied communication with namenode because hostname cann）及解决方案

“Hadoop provides a mechanism to prevent unauthorized access. User can add hosts.deny and hosts.allow files to limit the access to a hadoop cluster. When a data node joins the name node, the name node will check the DNS name of the data node, if the DNS name cannot be resolved or the host is not allowed based on hosts.deny and hosts.allow files, the NN will reject the data node join.

org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException): Datanode denied communication with namenode because hostname cannot be resolved (ip=10.5.x.x, hostname=10.5.x.x): DatanodeRegistration(0.0.0.0, datanodeUuie12e2d92-e215-4af9-a09b-d22ef4e3cc5ainfoPort=9102, ipcPort=9201, storageInfo=lv=-55;cid=CID-f50d3808-5d7f-4520-b696-0234f07f89f7;nsid=1758720717;c=0)

If set dfs.namenode.datanode.registration.ip-hostname-check=false, the name node will not do the check, which could be useful if your cluster is inside AWS VPC and you do not have proper reverse DNS.

Another usage of this feature is to decommission a data node by listing it in the hosts.deny file.”

也就是说在namenode的hdfs-site.xml 里面添加

<property>
                      <name>dfs.namenode.datanode.registration.ip-hostname-check</name>
                      <value>false</value>
                  </property>

即可