hadoopsnappy解压_Hadoop的Snappy安装配置

1.snappy编译安装

下载snappy,编译。

前置条件：gcc c++, autoconf, automake, libtool, Java6, JAVA_HOME set, Maven 3

1)运行：autogen.sh

2)运行：

./configure

make

make check

make install

3)验证：查看/usr/local/lib下面是否有libsnappy.*

4)拷贝到其他集群：scp -r ./google-snappy-2d99bd1/ bigdata@xxx:/ssd/bigdata/install/

2. Hadoop Snappy 编译安装

1、下载

2、The built tarball is at target/hadoop-snappy-0.0.1-SNAPSHOT.tar.gz. The tarball includes snappy native library。

解压拷贝：

cp -r * $HADOOP_HOME/lib

scp -r * bigdata@xxx:/ssd/bigdata/software/hadoop/lib

3、配置环境变量：

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native/Linux-amd64-64/:/usr/local/lib/

export

HBASE_LIBRARY_PATH=$HBASE_LIBRARY_PATH:$HBASE_HOME/lib/native/Linux-amd64-64/:/usr/local/lib/

2°、修改$HADOOP_HOME/etc/hadoop/core-site.xml：

io.compression.codecs

org.apache.hadoop.io.compress.GzipCodec,

org.apache.hadoop.io.compress.DefaultCodec,

org.apache.hadoop.io.compress.BZip2Codec,

org.apache.hadoop.io.compress.SnappyCodec

3°、修改$HADOOP_HOME/etc/hadoop/mapred-site.xml中有关压缩属性，测试snappy：

mapreduce.map.output.compress

true

mapreduce.map.output.compress.codec

org.apache.hadoop.io.compress.SnappyCodec

4°、验证

hadoop checknative -a