CDH hadoop 编译

最好是 163.yum  epel.repo 的yum 资源 

准备工作

yum install svn autoconf automake libtool cmake ncurses-devel openssl-devel gcc*
wget http://ftp.cuhk.edu.hk/pub/packages/apache.org/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz -O apache-maven-3.3.9-bin.tar.gz 
wget ftp://ftp.netbsd.org/pub/pkgsrc/distfiles/protobuf-2.5.0.tar.gz -O protobuf-2.5.0.tar.gz
 wget http://pkgs.fedoraproject.org/repo/pkgs/snappy/snappy-1.1.1.tar.gz/8887e3b7253b22a31f5486bca3cbc1c2/snappy-1.1.1.tar.gz
tar xf apache-maven-3.3.9-bin.tar.gz -C /home/hadoop/app/



cat /home/hadoop/.bash_profile | grep MAVEN
export MAVEN_HOME=/home/hadoop/app/apache-maven-3.3.9
export PATH=$PATH:$MAVEN_HOME/bin



tar xf protobuf-2.5.0.tar.gz
cd protobuf-2.5.0
./configure 
make
make install
sudo make install
sudo ldconfig
protoc --version








tar xf snappy-1.1.1.tar.gz
cd ../snappy-1.1.1
./configure
make
make install
sudo make install
ls -lh /usr/local/lib |grep snappy




tar xf hadoop-2.6.0-cdh5.7.0-src.tar.gz
cd hadoop-2.6.0-cdh5.7.0


mvn clean package -Pdist,native -DskipTests -Dtar    如果报错的 就用下面的尝试重新编译 ，且下面的编译效率更高些 


mvn clean package -Pdist -Dtar -Dmaven.javadoc.skip=true -DskipTests -fail-at-end -Pnative  





[hadoop@hadoop001 hadoop-2.6.0-cdh5.7.0]$ bin/hadoop checknative
17/08/03 23:00:36 INFO bzip2.Bzip2Factory: Successfully loaded & initialized native-bzip2 library system-native
17/08/03 23:00:36 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:
hadoop:  true /home/hadoop/source/hadoop-2.6.0-cdh5.7.0/hadoop-dist/target/hadoop-2.6.0-cdh5.7.0/lib/native/libhadoop.so.1.0.0
zlib:    true /lib64/libz.so.1
snappy:  true /usr/lib64/libsnappy.so.1
lz4:     true revision:99
bzip2:   true /lib64/libbz2.so.1
openssl: true /usr/lib64/libcrypto.so