云服务器上搭建hadoop外网web页面9870无法访问及解决

最新推荐文章于 2024-07-31 23:40:10 发布

Eternalizes

最新推荐文章于 2024-07-31 23:40:10 发布

阅读量1.2k

点赞数 8

文章标签： hadoop hdfs

本文链接：https://blog.csdn.net/qq_42926439/article/details/136086760

版权

申请了一个华为云服务器学习hadoop搭建环境时遇到如下问题今天分享出来大家一起避坑

1.配置好环境启动发现只有DataNode

[root@hecs-96341 conf.d]# start-dfs.sh 
Starting namenodes on [node1]
Last login: Fri Feb  9 15:20:16 CST 2024 on pts/0
node1: ERROR: Cannot set priority of namenode process 6960
Starting datanodes
Last login: Fri Feb  9 15:20:30 CST 2024 on pts/0
Starting secondary namenodes [node1]
Last login: Fri Feb  9 15:20:33 CST 2024 on pts/0
[root@hecs-96341 conf.d]# jps
7091 DataNode
7403 Jps

2.检查nameNode日志提示端口被占用（这里是个坑 kill没有用）

tail -n 50 hadoop-root-namenode-hecs-96341.log

2024-02-09 15:20:32,102 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NameNode metrics system...
2024-02-09 15:20:32,102 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system stopped.
2024-02-09 15:20:32,102 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system shutdown complete.
2024-02-09 15:20:32,102 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.
java.net.BindException: Port in use: node1:9870
        at org.apache.hadoop.http.HttpServer2.constructBindException(HttpServer2.java:1344)
        at

3.问题出在hosts文件，主节点不能配置外网搭建集群的同学们可以试试把其他节点暴露在外网上

/etc/hosts

::1     localhost       localhost.localdomain   localhost6      localhost6.localdomain6
127.0.0.1       localhost       localhost.localdomain   localhost4      localhost4.localdomain4
127.0.0.1       hecs-96341      hecs-96341

#我写的是ipadd里的
192.168.0.208 node1

4.修改完ip成功启动没有映射外网端口外网还是无法访问

[root@hecs-96341 logs]# start-dfs.sh 
Starting namenodes on [node1]
Last login: Fri Feb  9 15:34:31 CST 2024 on pts/0
Starting datanodes
Last login: Fri Feb  9 15:34:46 CST 2024 on pts/0
Starting secondary namenodes [node1]
Last login: Fri Feb  9 15:34:48 CST 2024 on pts/0
[root@hecs-96341 logs]# jps
9111 DataNode
8936 NameNode
9513 Jps
9322 SecondaryNameNode

5.使用ngxin做代理实现外网访问

server{
       listen 80;
       charset utf-8;
       server_name 127.0.0.1;

       location / {
          proxy_pass http://192.168.0.208:9870;
          proxy_redirect default;
       }
    }

外网成功访问美滋滋