hadoop 2.x 编译

Hadoop 2.x 编译

编译前准备 （hadoop_src/BUILDING.txt）

Requirements:

* Unix System

* JDK 1.6+

* Maven 3.0 or later   (建议maven 3.0.5)

* Findbugs 1.3.9 (if running findbugs)

* ProtocolBuffer 2.5.0

* CMake 2.6 or newer (if compiling native code), must be 3.0 or newer on Mac

* Zlib devel (if compiling native code)

* openssl devel ( if compiling native hadoop-pipes )

* Internet connection for first build (to fetch all Maven and Hadoop dependencies)

yum install autoconf automake libtool cmake

yum install ncurses-devel

yum install openssl-devel

yum install lzo-devel zlib-devel gcc gcc-c++

安装protobuf

$ wget https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz

$ tar zxf protobuf-2.5.0.tar.gz

$ cd protobuf-2.5.0

$ ./configure

$ make

$ make install

编译前执行，否则会报内存溢出的错误

export MAVEN_OPTS="-Xms256m -Xmx512m”

开始编译hadoop

cd hadoop-src/

mvn package -Pdist,native,docs,src -DskipTests -Dtar

编译好后的hadoop在/opt/hadoop-2.6.0-src/hadoop-dist/target中

查看hadoop编译的native位数

$ file $HADOOP_PREFIX/lib/native/libhadoop.so.1.0.0