1.Log Aggregation 日志聚集,应用运行完成以后,将日志信息上传到HDDFS系统上。
配置文件yarn-site.xml
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value> #开启状态
</property>
<property>
<name>yarn.log-aggregation.retain-seconds</name>
<value>640800</value> #日志聚集保存时间(秒)
</property>
2.配置文件
(1)默认配置文件,
core-default.xml hdfs-default.xml
yarn-default.xml mapred-default.xml
(2)自定义配置文件(自定义配置完成,覆盖默认配置)
core-site.xml hdfs-site.xml
yarn-site.xml mapred-site.xml
在core-site.xml可以配置垃圾回收机制:fs.trash.interval
<property>
<name>fs.trash.interval</name>
<value>0</value>
<description>Number of minutes after which the checkpoint
gets deleted. If zero, the trash feature is disabled.
This option may be configured both on the server and the
client. If trash is disabled server side then the client
side configuration is checked. If trash is enabled on the
server side then the value configured on the server is
used and the client configuration value is ignored.
</description>
</property>
3.编译源码
building.txt 有编译说明
4.hdfs shell:打出hdfs dfs 回车,可以看到各命令的说明。
详情可查看官方文档http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html