配置文件
一、常用配置文件的文件
| |
---|
core-site.xml | |
hdfs-site.xml | hdfs-env.sh |
mapred-site.xml | mapred-env.sh |
yarn-site.xml | yarn-env.sh |
slaves | |
二、配置文件中的内容
①核心配置文件:core-site.xml
[root@HadoopMaster module]# `cd /opt/module/hadoop-2.7.2/etc/hadoop/`
[root@HadoopMaster hadoop]# `vi core-site.xml`
<configuration>
<!-- 指定HDFS中NameNode的地址 -->
<property>
<name>fs.defaultFS</name>
<value>hdfs://HadoopMaster:9000</value>
</property>
<!-- 指定Hadoop运行时产生文件的存储目录 -->
<property>
<name>hadoop.tmp.dir</name>
<value>/opt/module/hadoop-2.7.2/data/tmp/</value>
</property>
</configuration>
②HDFS配置文件:hdfs-env.sh和hdfs-site.xml
[root@HadoopMaster module]# `cd /opt/module/hadoop-2.7.2/etc/hadoop/`
[root@HadoopMaster hadoop]# vi hadoop-env.sh
export JAVA_HOME=/opt/module/jdk1.8.0
[root@HadoopMaster module]# cd /opt/module/hadoop-2.7.2/etc/hadoop/
[root@HadoopMaster hadoop]# vi hdfs-site.xml
<configuration>
<!-- 指定文件备份数量 -->
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
<!-- 指定Hadoop辅助名称节点地址 -->
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>HadoopSlaver02:50090</value>
</property>
</configuration>
③YARN配置文件:yarn-env.sh和yarn-site.xml
[root@HadoopMaster module]# cd /opt/module/hadoop-2.7.2/etc/hadoop/
[root@HadoopMaster hadoop]# vi yarn-env.sh
export JAVA_HOME=/opt/module/jdk1.8.0
[root@HadoopMaster module]# cd /opt/module/hadoop-2.7.2/etc/hadoop/
[root@HadoopMaster hadoop]# vi yarn-site.xml
<configuration>
<!-- 指定Reducer获取数据的方式 -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<!-- 指定YARN的ResourceManager的地址 -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>HadoopSlaver01</value>
</property>
</configuration>
④MapReduce配置文件:mapred-env.sh和mapred-site.xml
[root@HadoopMaster module]# cd /opt/module/hadoop-2.7.2/etc/hadoop/
[root@HadoopMaster hadoop]# vi mapred-env.sh
export JAVA_HOME=/opt/module/jdk1.8.0
[root@HadoopMaster module]# cd /opt/module/hadoop-2.7.2/etc/hadoop/
[root@HadoopMaster hadoop]# mv mapred-site.xml.template mapred-site.xml
[root@HadoopMaster hadoop]# vi mapred-site.xml
<configuration>
<!-- 指定MR运行在YARN上 -->
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
⑤配置slaves
[root@HadoopMaster ~]# cd /opt/module/hadoop-2.7.2/etc/hadoop/
[root@HadoopMaster hadoop]# vi slaves
HadoopMaster
HadoopSlaver01
HadoopSlaver02
本次文献根据钟老师的笔记所进行修改