在hadoop2.2.0集群上安装了一下snappy,安装过程很简单,可以看一下这个博客:http://shitouer.cn/2013/01/hadoop-hbase-snappy-setup-final-tutorial/
我在这里记录一下遇到的问题,
当我用:sudo hbase org.apache.hadoop.hbase.util.CompressionTest hdfs://192.168.24.72:8020/test/output92/part-00000 snappy
测试安装是否成功的时候,出现下面的问题:
刚开始怀疑是找不到本地安装的snappy的lib,把所有相关的lib都拷贝到$HADOOP_HOME/lib 目录下。
再次测试,仍然不好使,仔细看报的异常和警告:
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/q/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
2013-11-12 14:04:28,664 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
怀疑是hadoop的本地lib没有加载成功,但是,apache版本的hadoop2.2.0的lib目录下有native目录,里面也有软连接文件,到网上搜了一下,发现是apache版本的native都是32位的,而生产机都是64位的。需要自己重写编译一下,具体细节见博客:
http://blog.csdn.net/lalaguozhe/article/details/10580727
重新配置,测试:
在hbase中建表,以snappy压缩方式: