配置历史服务器
历史服务器配置
停止集群
[dev1@hadoop103 bin]$ stop-yarn.sh
[dev1@hadoop102 hadoop-3.1.3]$ stop-dfs.sh
群发该文件
[dev1@hadoop102 hadoop-3.1.3]$ xsync ./etc/hadoop/mapred-site.xml
内容如下
<!-- 历史服务器端地址 -->
<property>
<name>mapreduce.jobhistory.address</name>
<value>hadoop102:10020</value>
</property>
<!-- 历史服务器web端地址 -->
<property> <name>mapreduce.jobhistory.webapp.address</name>
<name>mapreduce.jobhistory.webapp.address</name>
<value>hadoop102:19888</value>
</property>
群发mapred-site.xml
[dev1@hadoop102 hadoop-3.1.3]$ xsync ./etc/hadoop/mapred-site.xml
启动历史服务
[dev1@hadoop102 hadoop-3.1.3]$ mapred --daemon start historyserver
[dev1@hadoop102 hadoop-3.1.3]$ jps
91373 JobHistoryServer
91534 Jps
客户端查看界面
http://hadoop102:19888/
配置日志的聚集
[dev1@hadoop102 hadoop-3.1.3]$ vim ./etc/hadoop/yarn-site.xml
内容如下:
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<name>yarn.log.server.url</name>
<value>http://hadoop102:19888/jobhistory/logs</value>
</property>
<property>
<name>yarn.log-aggregation.retain-seconds</name>
<value>604800</value>
</property>
群发
[dev1@hadoop102 hadoop-3.1.3]$ xsync ./etc/hadoop/yarn-site.xml
执行wordcount程序
[dev1@hadoop102 hadoop-3.1.3]$ hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.3.jar wordcount /user/dev1/input/word.txt /user/dev1/output3
点击history详情界面的log查看详细日志
配置日志的聚集后运行的程序有详细展示
配置之前的程序没有log详细展示