问题提出:
在运行kylin sample时出现以下错误:
org.apache.hadoop.hive.ql.metadata.HiveException: native snappy library not available: this version of libhadoop was built without snappy support.
造成以上错误的原因是Hadoop的二进制安装包中没有snappy支持,需要手工重新编译。
操作过程:
1. 下载所需要的源码包
snappy-1.1.1.tar.gz
protobuf-2.5.0.tar.gz
hadoop-2.7.2-src.tar.gz
2. 准备编译环境
yum install svn
yum install autoconf automake libtool cmake
yum install ncurses-devel
yum install openssl-devel
yum install gcc*
3. 编译安装snappy
# 用root用户执行以下命令
tar -zxvf snappy-1.1.1.tar.gz
cd snappy-1.1.1/
./configure
make
make install
# 查看snappy库文件
ls -lh /usr/local/lib |grep snappy
4. 编译安装protobuf
# 用root用户执行以下命令
tar -zxvf protobuf-2.5.0.tar.gz
cd protobuf-2.5.0/
./configure
make
make install
# 查看protobuf版本以测试是否安装成功
protoc --version
5. 编译hadoop native
tar -zxvf hadoop-2.7.2-src.tar.gz
cd hadoop-2.7.2-src/
mvn clean package -DskipTests -Pdist,native -Dtar -Dsnappy.lib=/usr/local/lib -Dbundle.snappy
http://www.micmiu.com/bigdata/hadoop/hadoop-snappy-install-config/
http://www.linuxidc.com/Linux/2014-12/110857.htm
在运行kylin sample时出现以下错误:
org.apache.hadoop.hive.ql.metadata.HiveException: native snappy library not available: this version of libhadoop was built without snappy support.
造成以上错误的原因是Hadoop的二进制安装包中没有snappy支持,需要手工重新编译。
操作过程:
1. 下载所需要的源码包
snappy-1.1.1.tar.gz
protobuf-2.5.0.tar.gz
hadoop-2.7.2-src.tar.gz
2. 准备编译环境
yum install svn
yum install autoconf automake libtool cmake
yum install ncurses-devel
yum install openssl-devel
yum install gcc*
3. 编译安装snappy
# 用root用户执行以下命令
tar -zxvf snappy-1.1.1.tar.gz
cd snappy-1.1.1/
./configure
make
make install
# 查看snappy库文件
ls -lh /usr/local/lib |grep snappy
4. 编译安装protobuf
# 用root用户执行以下命令
tar -zxvf protobuf-2.5.0.tar.gz
cd protobuf-2.5.0/
./configure
make
make install
# 查看protobuf版本以测试是否安装成功
protoc --version
5. 编译hadoop native
tar -zxvf hadoop-2.7.2-src.tar.gz
cd hadoop-2.7.2-src/
mvn clean package -DskipTests -Pdist,native -Dtar -Dsnappy.lib=/usr/local/lib -Dbundle.snappy
执行成功后,hadoop-dist/target/hadoop-2.7.2.tar.gz即为新生成的二进制安装包。
参考:http://www.micmiu.com/bigdata/hadoop/hadoop-snappy-install-config/
http://www.linuxidc.com/Linux/2014-12/110857.htm