HDFS开LZO on redhat/el4/en/x86_64/

最新推荐文章于 2024-09-21 10:38:43 发布

置顶牛_plus_plus

最新推荐文章于 2024-09-21 10:38:43 发布

阅读量4.1k

点赞数

分类专栏： HDFS 文章标签： path ant java

本文链接：https://blog.csdn.net/kalaamong/article/details/7275185

版权

HDFS 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

su root
wget ftp://ftp.univie.ac.at/systems/linux/dag/redhat/el4/en/x86_64/dag/RPMS/lzo-2.04-1.el4.rf.x86_64.rpm
wget ftp://ftp.univie.ac.at/systems/linux/dag/redhat/el4/en/x86_64/dag/RPMS/lzo-devel-2.04-1.el4.rf.x86_64.rpm
rpm -ivh lzo-2.04-1.el4.rf.x86_64.rpm
rpm -ivh lzo-devel-2.04-1.el4.rf.x86_64.rpm
exit

cd ~
wget http://labs.renren.com/apache-mirror//ant/binaries/apache-ant-1.8.2-bin.tar.gz
tar zxvf apache-ant-1.8.2-bin.tar.gz
export JAVA_HOME=~/jdk1.6.0_24/
export PATH=$JAVA_HOME/bin:$PATH
export ANT_HOME=~/apache-ant-1.8.2/
export PATH=$ANT_HOME/bin:$PATH

wget http://hadoop-gpl-compression.apache-extras.org.codespot.com/files/hadoop-gpl-compression-0.1.0-rc0.tar.gz
tar -zxvf hadoop-gpl-compression-0.1.0-rc0.tar.gz
cd hadoop-gpl-compression-0.1.0
cp ~/hadoop-0.20.2-cdh3u0/hadoop-core-0.20.2-cdh3u0.jar lib/
vi src/java/com/hadoop/compression/lzo/LzoCompressor.java

此处添加：
public void reinit(org.apache.hadoop.conf.Configuration conf) {
     // do nothing
    }   

su root
ant compile-native tar

chmod -R 777 *
cp build/hadoop-gpl-compression-0.1.0-dev.jar ~/hadoop-0.20.2-cdh3u0/lib/
cp build/hadoop-gpl-compression-0.1.0-dev.jar ~/hbase-0.90.4/lib/
rsync -auPv lib/native ~/hadoop-0.20.2-cdh3u0/lib/
rsync -auPv lib/native ~/hbase-0.90.4/lib/
vi ~/hadoop-0.20.2-cdh3u0/conf/core-site.xml

此处添加：
<property>
    <name>io.compression.codecs</name>
    <value>com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec</value>
</property>
<property>
    <name>io.compression.codec.lzo.class</name>
    <value>com.hadoop.compression.lzo.LzoCodec</value>
</property>