Removing and adding DataNodes in cluster by hadoop

最新推荐文章于 2022-10-06 13:34:21 发布

huanggang028

最新推荐文章于 2022-10-06 13:34:21 发布

阅读量1.8k

点赞数

本文链接：https://blog.csdn.net/huanggang028/article/details/9787217

版权

本文介绍了如何通过配置Hadoop集群的dfs.hosts和dfs.hosts.exclude属性来添加和移除DataNodes。同时，也提到了mapred.hosts和mapred.hosts.exclude用于管理TaskTracker。此外， dfs.balance.bandwidthPerSec参数用于设置每个DataNode进行数据平衡的最大带宽。

摘要由CSDN通过智能技术生成

You may want to remove or add some DataNodes from your HDFS cluster at some point. In fact ,Removing or adding nodes in Hadoop can be straightforward.Like this, we only do some simply operations, in which we will not affect ongoing other jobs.But in order to removing or adding more safe and efficient ,we must note replication of blocks and other points.

I think you should known that:
dfs.xxx ==> datanode ==> hdfs-site.xml
mapred.xxx ==> tasktracker ==> mapred-site.xml
It's means:you can do some operations on datanode and tasktracker respectively. Because they are hardly in the same of operations.

1.Removing DataNodes
Modify your cluster file of hdfs-site.xml on namenode:add some property like this