hadoop---LZO压缩配置

下载

https://github.com/twitter/hadoop-lzo/archive/master.zip

编译

下载后的文件名是hadoop-lzo-master，它是一个zip格式的压缩包，先进行解压，然后用maven编译。生成hadoop-lzo-0.4.20.jar

上传

HAOOP_HOME/share/hadoop/common/

同步

jar包同步到每一个节点

core-site.xml增加配置支持LZO压缩

<configuration>
	<property>
		<name>io.compression.codecs</name>
		<value>
			org.apache.hadoop.io.compress.GzipCodec,
			org.apache.hadoop.io.compress.DefaultCodec,
			org.apache.hadoop.io.compress.BZip2Codec,
			org.apache.hadoop.io.compress.SnappyCodec,
			com.hadoop.compression.lzo.LzoCodec,
			com.hadoop.compression.lzo.LzopCodec
		</value>
	</property>
	<property>
	    <name>io.compression.codec.lzo.class</name>
	    <value>com.hadoop.compression.lzo.LzoCodec</value>
	</property>
</configuration>

重启

需要重启生效

CDH安装LZO

参考:https://blog.csdn.net/mlljava1111/article/details/70875534
官网:https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_ig_install_gpl_extras.html#xd_583c10bfdbd326ba-3ca24a24-13d80143249–7ec6