编译hadoop2.7.3源码

hadoop2.7.3 重新编译 64 位本地库

一、环境准备：

虚拟机 VM，操作系统 64 位 CentOS 6.6

下载重新编译需要的软件包：

apache-ant-1.9.4-bin.tar.gz、findbugs-3.0.0.tar.gz、protobuf-2.5.0.tar.gz、apache-maven-3.3.9-bin.tar.gz

下载 hadoop2.7.3 的源码包：

hadoop-2.7.3-src.tar.gz

解压解源码包

$>tar -zxvf hadoop-2.7.3-src.tar.gz

二、安装编译所需软件包：
1）安装 MAVEN
$>tar -zxvf apache-maven-3.3.9-bin.tar.gz -C /opt/
修改/etc/profile 配置，增加 MAVEN 环境配置，保存后使用 source /etc/profile 使修改配置即时生效
$>source /etc/profile
$>mvn -v // 使用 mvn -v 命令进行验证，如图所示表示安装配置成功

2)安装 ANT
$>tar -zxvf apache-ant-1.9.4-bin.tar.gz -C /opt/
修改/etc/profile 配置，增加 ANT 环境配置，保存后使用 source /etc/profile 使修改配置即时生效
$>source /etc/profile
使用 ant -version 命令进行验证，如图所示表示安装配置成功

3）安装 FINDBUGS
$>tar -zxvf findbugs-3.0.0.tar.gz -C /opt/
修改/etc/profile 配置，增加 FINDBUGS 环境配置，保存后使用 source /etc/profile 使修改配置即时生效（同上）
使用 findbugs -version 命令进行验证，如图所示表示安装配置成功

4）安装 PROTOBUF

注意：编译 Hadoop 2.4.0，需要 protobuf 的编译器protoc，一定要是 protobuf 2.5.0 以上
$>tar -zxvf protobuf-2.5.0.tar.gz
安装 protobuf，依次执行如下命令
$> cd protobuf-2.5.0

5）执行上步操作需要安装好C++编译软件，安装如下：（若系统已经安装不用在重复安装）
# yum install gcc-c++
# ./configure
# make
# make check
# make install
6）使用 protoc --version 命令进行验证，如图所示表示安装配置成功

7） 安装依赖包
安装 cmake,openssl-devel,ncurses-devel  依赖包（注意：需要root用户且能够连上互联网）
# yum install cmake
# yum install openssl-devel
# yum install ncurses-devel

三、编译 64  位本地库
进入已压解的 hadoop 源码目录
$>cd hadoop-2.7.3-src
执行 mvn clean install -Dskip Tests 命令，等待完成(会自动联网下载很多东西)
$>mvn clean install -Dskip Tests
执行 mvn package -Pdist,native -DskipTests -Dtar 命令，开始编译，等待完成
$ mvn package -Pdist,native -DskipTests -Dtar
出现如下信息
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS[6.304s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [26.555s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS[2.757s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.216s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [19.592s]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [2.715s]
[INFO] Apache Hadoop MiniKDC ............................. SUCCESS [2.360s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [2.950s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS[2.119s]
[INFO] Apache Hadoop Common .............................. SUCCESS [1:22.302s]
[INFO] Apache Hadoop NFS ................................. SUCCESS [5.095s]
[INFO] Apache Hadoop Common Project...................... SUCCESS [0.026s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [2:06.178s]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [1:09.142s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [14.457s]
[INFO] Apache Hadoop HDFS-NFS ............................SUCCESS [2.859s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.030s]
[INFO] hadoop-yarn ....................................... SUCCESS [0.029s]
[INFO] hadoop-yarn-api ................................... SUCCESS [59.010s]
[INFO] hadoop-yarn-common ................................ SUCCESS [20.743s]
[INFO] hadoop-yarn-server ................................ SUCCESS[0.026s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [7.344s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [11.726s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [2.508s]
[INFO] hadoop-yarn-server-applicationhistoryservice ...... SUCCESS [4.041s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [10.370s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [0.374s]
[INFO] hadoop-yarn-client ................................ SUCCESS [4.791s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [0.025s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [2.242s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [1.553s]
[INFO] hadoop-yarn-site .................................. SUCCESS [0.024s]
[INFO] hadoop-yarn-project ............................... SUCCESS [3.261s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.082s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [18.549s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [13.772s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [2.441s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [6.866s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [6.280s]
[INFO] hadoop-mapreduce-client-jobclient .................SUCCESS [3.510s]
[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [1.725s]
[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [4.641s]
[INFO] hadoop-mapreduce .................................. SUCCESS[3.002s]
[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [3.497s]
[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [5.847s]
[INFO] Apache Hadoop Archives ............................ SUCCESS [1.791s]
[INFO] Apache Hadoop Rumen ............................... SUCCESS [4.693s]
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [3.235s]
[INFO] Apache Hadoop Data Join........................... SUCCESS [2.349s]
[INFO] Apache Hadoop Extras .............................. SUCCESS [2.488s]
[INFO] Apache Hadoop Pipes ............................... SUCCESS [5.863s]
[INFO] Apache Hadoop OpenStack support ................... SUCCESS [3.776s]
[INFO] Apache Hadoop Client .............................. SUCCESS [5.235s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [0.070s]
[INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [3.935s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [4.392s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [0.022s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [21.274s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10:25.147s
[INFO] Finished at: Mon Jul 28 16:09:56 CST 2014
[INFO] Final Memory: 75M/241M
[INFO] ------------------------------------------------------------------------
表示编译成功

进入hadoop-2.7.3-src/hadoop-dist/target/hadoop-2.7.3/lib/native 检查，使用 file * 命 令
注意事项：
1.不要敲错命令，负责会出现失败。
2.需要保持网络畅通，如果出现缺少某个文件，则要先清理maven(使用命令 mvn clean) 再重新编译。
3.如果总是出现同一个错误导致编译失败，则可能是缺少某个库或者工具，检查上述工具是否都安装成功，并且版本正确。