安装hadoop集群
1.core.site.xml
fs.default.name hdfs://hadoop102:9000 hadoop.tmp.dir /opt/module/hadoop/data/tmp 2.hdfs.site.xml dfs.namenode.secondary.http-address hadoop104:50090 3.mapred-site.xml mapreduce.framework.name yarn mapreduce.jobhistory.address hadoop101:10020 mapreduce.jobhistory.webapp.address hadoop101:19888 4.yarn.site.xml yarn.resourcemanager.hostname hadoop103 yarn.nodemanager.aux-services mapreduce_shuffle yarn.log-aggregation-enable true yarn.log-aggregation.retain-seconds 604800 5.格式化namenode(Hadoop102) hadoop namenode -formate 6.配置slaves群起集群 hadoop102 hadoop103 hadoop104 7.添加lzo压缩方式 hadoop-lzo-0.4.20.jar 放入hadoop/share/hadoop/common/ 配置 core-site.xml io.compression.codecs org.apache.hadoop.io.compress.GzipCodec, org.apache.hadoop.io.compress.DefaultCodec, org.apache.hadoop.io.compress.BZip2Codec, org.apache.hadoop.io.compress.SnappyCodec, com.hadoop.compression.lzo.LzoCodec, com.hadoop.compression.lzo.LzopCodec io.compression.codec.lzo.class com.hadoop.compression.lzo.LzoCodec