hadoopsnappy解压_修改Hadoop集群具有Snappy压缩方式

1．查看hadoop checknative命令使用[atguigu@hadoop104 hadoop-2.7.2]$ hadoop

checknative [-a|-h]  check native hadoop and compression libraries availability

2．查看hadoop支持的压缩方式[atguigu@hadoop104 hadoop-2.7.2]$ hadoop checknative

17/12/24 20:32:52 WARN bzip2.Bzip2Factory: Failed to load/initialize native-bzip2 library system-native, will use pure-Java version

17/12/24 20:32:52 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library

Native library checking:

hadoop:  true /opt/module/hadoop-2.7.2/lib/native/libhadoop.so

zlib:    true /lib64/libz.so.1

snappy:  false

lz4:     true revision:99

bzip2:   false

3．将编译好的支持Snappy压缩的hadoop-2.7.2.tar.gz包导入到hadoop102的/opt/software中

4．解压hadoop-2.7.2.tar.gz到当前路径[atguigu@hadoop102 software]$ tar -zxvf hadoop-2.7.2.tar.gz

5．进入到/opt/software/hadoop-2.7.2/lib/native路径可以看到支持Snappy压缩的动态链接库[atguigu@hadoop102 native]$ pwd

/opt/software/hadoop-2.7.2/lib/native

[atguigu@hadoop102 native]$ ll

-rw-r--r--. 1 atguigu atguigu  472950 9月 1 10:19 libsnappy.a

-rwxr-xr-x. 1 atguigu atguigu     955 9月 1 10:19 libsnappy.la

lrwxrwxrwx. 1 atguigu atguigu      18 12月 24 20:39 libsnappy.so -> libsnappy.so.1.3.0

lrwxrwxrwx. 1 atguigu atguigu      18 12月 24 20:39 libsnappy.so.1 -> libsnappy.so.1.3.0

-rwxr-xr-x. 1 atguigu atguigu  228177 9月 1 10:19 libsnappy.so.1.3.0

6．拷贝/opt/software/hadoop-2.7.2/lib/native里面的所有内容到开发集群的/opt/module/hadoop-2.7.2/lib/native路径上

[atguigu@hadoop102 native]$ cp ../native/* /opt/module/hadoop-2.7.2/lib/native/

7．分发集群[atguigu@hadoop102 lib]$ xsync native/

8．再次查看hadoop支持的压缩类型[atguigu@hadoop102 hadoop-2.7.2]$ hadoop checknative

17/12/24 20:45:02 WARN bzip2.Bzip2Factory: Failed to load/initialize native-bzip2 library system-native, will use pure-Java version

17/12/24 20:45:02 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library

Native library checking:

hadoop:  true /opt/module/hadoop-2.7.2/lib/native/libhadoop.so

zlib:    true /lib64/libz.so.1

snappy:  true /opt/module/hadoop-2.7.2/lib/native/libsnappy.so.1

lz4:     true revision:99

bzip2:   false

9．重新启动hadoop集群和hive