Hadoop cannot find namenode pid file when shutdown

最新推荐文章于 2023-04-24 23:33:27 发布

软猫克鲁

最新推荐文章于 2023-04-24 23:33:27 发布

阅读量1.5k

点赞数

分类专栏： Java Hadoop 云计算文章标签： hadoop namenode kill 关闭

本文链接：https://blog.csdn.net/lj0425/article/details/17920111

版权

Java 同时被 3 个专栏收录

4 篇文章 0 订阅

订阅专栏

Hadoop

3 篇文章 0 订阅

订阅专栏

云计算

2 篇文章 0 订阅

订阅专栏

By default settings Hadoop use /tmp/ folder to track NameNode and JobTracker process id alive. But /tmp folder is cleanup once a week. So after some time, you will get error when you try to shutdown Hadoop cluster. You have to shutdown hadoop cluster manually. And don't use default setting to track pid files.

1. Shutdown hadoop cluster manually.

jps | grep 'TaskTracker\|DataNode' | sed 's/ .*//g' | sed 's/^/kill -9 /g' | sh

This code is ugly but works. You can try some graceful way to do this, like :

jps | grep 'TaskTracker\|DataNode' | awk 'echo $1' | xargs kill -9 # I've not try this.

2. Change Hadoop default setting.

Revise hadoop-env.sh file in ${HADOOP_HOME}/conf, add somethings like below.

export HADOOP_PID_DIR=${HADOOP_HOME}/pid

3. Restart your cluster, you should find some pid files in that folder.

ls -l
-rw-r--r-- 1 ana grid 6 Nov 13 01:54 hadoop-ana-jobtracker.pid
-rw-r--r-- 1 ana grid 6 Nov 13 01:54 hadoop-ana-namenode.pid
-rw-r--r-- 1 ana grid 6 Nov 13 01:54 hadoop-ana-secondarynamenode.pid