Hadoop集群配置文件及修改镜像源

最新推荐文章于 2023-01-09 20:03:48 发布

旅行者10086

最新推荐文章于 2023-01-09 20:03:48 发布

阅读量1.4k

点赞数 1

文章标签： hadoop mapreduce

本文链接：https://blog.csdn.net/avulunter/article/details/121429141

版权

本文详细介绍了Hadoop集群的核心配置文件设置，包括core-site.xml、yarn-site.xml、hdfs-site.xml及mapred-site.xml等关键文件的具体参数，如HDFS的地址、资源管理器的主机名、内存分配等。

摘要由CSDN通过智能技术生成

core-site.xml

<property>
<name>fs.defaultFS</name>
<value>hdfs://hadoop01:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/export/servers/hadoop-2.7.4/tmp</value>
</property>

yarn-site.xml

<property>
<name>yarn.resourcemanager.hostname</name>
<value>hadoop01</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>2048</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>2048</value>
</property>
<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>1</value>
</property>

hdfs-site.xml

<property>
<name>dfs.replication</name>
<value>3</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>hadoop02:50090</value>
</property>

mapred-site.xml

<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>

slaves

hadoop01
hadoop02
hadoop03

修改镜像源

cd /etc/yum.repos.d/

ls

mv CentOS-Base.repo CentOS-Base.repo.backup

vi CentOS-Base.repo

修改内容为

[centos-office]
name=centos-office
failovermethod=priority
baseurl=https://vault.centos.org/6.10/os/x86_64/
gpgcheck=1
gpgkey=https://vault.centos.org/6.10/os/x86_64/RPM-GPG-KEY-CentOS-6