Hadoop LZO 安装教程

最新推荐文章于 2022-08-08 10:33:12 发布

babydavic

最新推荐文章于 2022-08-08 10:33:12 发布

阅读量1.9k

点赞数

分类专栏： hadoop

本文链接：https://blog.csdn.net/babydavic/article/details/8530687

版权

hadoop 专栏收录该内容

15 篇文章 0 订阅

订阅专栏

1.安装 hadoop-gpl-compression
1.1 wget http://hadoop-gpl-compression.apache-extras.org.codespot.com/files/hadoop-gpl-compression-0.1.0-rc0.tar.gz
1.2 mv hadoop-gpl-compression-0.1.0/lib/native/Linux-amd64-64/* $HADOOP_HOME/lib/native/Linux-amd64-64/

1.3 cp hadoop-gpl-compression-0.1.0/hadoop-gpl-compression-0.1.0.jar /usr/local/hadoop-1.0.2/lib/

2.安装 lzo
apt-get install gcc
apt-get install lzop

3.在本地测试可以执行压缩及解压缩命令
下载 https://github.com/kevinweil/hadoop-lzo/zipball/master
解压包
export CFLAGS=-m64
export CXXFLAGS=-m64
//用于 hadoop0.20 ，所以1.0未必适用
进行build
ant compile-native tar
将生成的 build/hadoop-lzo-0.4.15.jar cp 到 /usr/local/hadoop-1.0.2/lib

测试解压程序
bin/hadoop jar /usr/local/hadoop-1.0.2/lib/hadoop-lzo-0.4.15.jar com.hadoop.compression.lzo.LzoIndexer /home/hadoop/project_hadoop/aa.html.lzo