集群hadoop的配置文件（细）

配置文件

一、常用配置文件的文件

core-site.xml
hdfs-site.xml	hdfs-env.sh
mapred-site.xml	mapred-env.sh
yarn-site.xml	yarn-env.sh
slaves

二、配置文件中的内容

①核心配置文件：core-site.xml

[root@HadoopMaster module]# `cd /opt/module/hadoop-2.7.2/etc/hadoop/`
[root@HadoopMaster hadoop]# `vi core-site.xml`

<configuration>
	<!-- 指定HDFS中NameNode的地址 -->
	<property>
		<name>fs.defaultFS</name>
		<value>hdfs://HadoopMaster:9000</value>
	</property>
	<!-- 指定Hadoop运行时产生文件的存储目录 -->
	<property>
		<name>hadoop.tmp.dir</name>
		<value>/opt/module/hadoop-2.7.2/data/tmp/</value>
	</property>
</configuration>

②HDFS配置文件：hdfs-env.sh和hdfs-site.xml

[root@HadoopMaster module]# `cd /opt/module/hadoop-2.7.2/etc/hadoop/`
[root@HadoopMaster hadoop]# vi hadoop-env.sh 

export JAVA_HOME=/opt/module/jdk1.8.0

[root@HadoopMaster module]# cd /opt/module/hadoop-2.7.2/etc/hadoop/
[root@HadoopMaster hadoop]# vi hdfs-site.xml

<configuration>
	<!-- 指定文件备份数量 -->
	<property>
		<name>dfs.replication</name>
		<value>3</value>
	</property>
	<!-- 指定Hadoop辅助名称节点地址 -->
	<property>
		<name>dfs.namenode.secondary.http-address</name>
		<value>HadoopSlaver02:50090</value>
	</property>
</configuration>

③YARN配置文件：yarn-env.sh和yarn-site.xml

[root@HadoopMaster module]# cd /opt/module/hadoop-2.7.2/etc/hadoop/
[root@HadoopMaster hadoop]# vi yarn-env.sh

export JAVA_HOME=/opt/module/jdk1.8.0

[root@HadoopMaster module]# cd /opt/module/hadoop-2.7.2/etc/hadoop/
[root@HadoopMaster hadoop]# vi yarn-site.xml

<configuration>
	<!-- 指定Reducer获取数据的方式 -->
	<property>
		<name>yarn.nodemanager.aux-services</name>
		<value>mapreduce_shuffle</value>
	</property>
	<!-- 指定YARN的ResourceManager的地址 -->
	<property>
		<name>yarn.resourcemanager.hostname</name>
		<value>HadoopSlaver01</value>
	</property>
</configuration>

④MapReduce配置文件：mapred-env.sh和mapred-site.xml

[root@HadoopMaster module]# cd /opt/module/hadoop-2.7.2/etc/hadoop/
[root@HadoopMaster hadoop]# vi mapred-env.sh

export JAVA_HOME=/opt/module/jdk1.8.0

[root@HadoopMaster module]# cd /opt/module/hadoop-2.7.2/etc/hadoop/
[root@HadoopMaster hadoop]# mv mapred-site.xml.template mapred-site.xml
[root@HadoopMaster hadoop]# vi mapred-site.xml

<configuration>
	<!-- 指定MR运行在YARN上 -->
	<property>
		<name>mapreduce.framework.name</name>
		<value>yarn</value>
	</property>
</configuration>

⑤配置slaves

[root@HadoopMaster ~]# cd /opt/module/hadoop-2.7.2/etc/hadoop/
[root@HadoopMaster hadoop]# vi slaves 

HadoopMaster
HadoopSlaver01
HadoopSlaver02

本次文献根据钟老师的笔记所进行修改