hadoop 2.x linux平台源码编译

下载环境需要的软件
 apache-ant-1.9.4-bin.tar.gz
 findbugs-3.0.0.tar.gz
 protobuf-2.5.0.tar.gz(不能低于此版本)
 jdk-7u45-linux-x64.tar.gz(建议使用jdk1.7)
 apache-maven-3.0.5-bin.tar.gz
下载 hadoop-2.6.4 的源码包
hadoop-2.6.4-src.tar.gz(建议不要使用最新版本,因为很多不兼容)

使用hadoop用户安装
安装在/home/hadoop/apps 下面
cd /home/hadoop
mkdir apps

将所有的软件和源码上传到hadoop下面
解压
tar –zxvf jdk-7u45-linux-x64.tar.gz -C ./apps
tar –zxvf protobuf-2.5.0.tar.gz -C ./apps
tar –zxvf findbugs-3.0.1.tar.gz -C ./apps
tar –zxvf apache-maven-3.0.5-bin.tar.gz -C ./apps
tar –zxvf apache-ant-1.9.7-bin.tar.gz -C ./apps
tar –zxvf hadoop-2.6.4-src.tar.gz -C ./apps

如图所示

配置环境变量
使用root 配置环境变量
su root
vi /etc/profile
export JAVA_HOME=/home/hadoop/apps/jdk1.7.0_45
export M2_HOME=/home/hadoop/apps/maven
export ANT_HOME=/home/hadoop/apps/ant
export FINDBUGS_HOME=/home/hadoop/apps/findbugs

export PATH=$JAVA_HOME/bin:$M2_HOME/bin:$ANT_HOME/bin:$FINDBUGS_HOME/bin:$PATH

然后让配置文件生效
source /etc/profile
验证
java -version
出现如下信息说明安装成功

mvn –v
出现如下信息说明安装成功

ant –version
出现如下信息说明安装成功

findbugs –version
出现如下信息说明安装成功

安装protobuf
1. 首先安装gcc yum –y install gcc
2. 再安装gcc-c++ yum –y install gcc-c++
3. ./configure 出现如下信息说明没有错误 还可查看protobuf下有没有Makefile文件

4. make出现如下信息说明没有错误

5. make check出现如下信息说明没有错误

6. make install 出现如下信息说明没有错误

7. 出现如下信息说明protobuf 安装成功

安装依赖包
 yum –y install cmake
 yum –y install openssl-devel
 yum install ncurses-devel

编译源码
 进入hadoop源码目录 cd hadoop-2.6.4-src
 执行 mvn clean install -DskipTests 命令，等待完成(会自动联网下载很多东西)
出现如下信息表示执行成功

 执行 mvn package -Pdist,native -DskipTests -Dtar 命令，开始编译，等待完成
如果出现如下错误需要手动下载apache-tomcat-6.0.41.tar.gz到
/home/hadoop/apps/hadoop-2.6.4-src/hadoop-common-project/hadoop-kms/downloads/ 目录和
/home/hadoop/apps/hadoop-2.6.4-src/hadoop-hdfs-project/hadoop-hdfs-httpfs/downloads/ 目录下

使用maven编译可以配置成自己的中央仓库加快编译

如果出现如下信息说明编译成功。
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ………………………….. SUCCESS [13.935s]
[INFO] Apache Hadoop Project POM ……………………. SUCCESS [14.310s]
[INFO] Apache Hadoop Annotations ……………………. SUCCESS [29.127s]
[INFO] Apache Hadoop Assemblies …………………….. SUCCESS [2.032s]
[INFO] Apache Hadoop Project Dist POM ……………….. SUCCESS [9.899s]
[INFO] Apache Hadoop Maven Plugins ………………….. SUCCESS [30.329s]
[INFO] Apache Hadoop MiniKDC ……………………….. SUCCESS [22.650s]
[INFO] Apache Hadoop Auth ………………………….. SUCCESS [28.397s]
[INFO] Apache Hadoop Auth Examples ………………….. SUCCESS [21.498s]
[INFO] Apache Hadoop Common ………………………… SUCCESS [8:47.281s]
[INFO] Apache Hadoop NFS …………………………… SUCCESS [37.891s]
[INFO] Apache Hadoop KMS …………………………… SUCCESS [55.542s]
[INFO] Apache Hadoop Common Project …………………. SUCCESS [0.223s]
[INFO] Apache Hadoop HDFS ………………………….. SUCCESS [12:06.030s]
[INFO] Apache Hadoop HttpFS ………………………… SUCCESS [1:14.458s]
[INFO] Apache Hadoop HDFS BookKeeper Journal …………. SUCCESS [1:03.599s]
[INFO] Apache Hadoop HDFS-NFS ………………………. SUCCESS [35.209s]
[INFO] Apache Hadoop HDFS Project …………………… SUCCESS [0.281s]
[INFO] hadoop-yarn ………………………………… SUCCESS [0.274s]
[INFO] hadoop-yarn-api …………………………….. SUCCESS [5:47.763s]
[INFO] hadoop-yarn-common ………………………….. SUCCESS [2:20.337s]
[INFO] hadoop-yarn-server ………………………….. SUCCESS [0.456s]
[INFO] hadoop-yarn-server-common ……………………. SUCCESS [1:03.513s]
[INFO] hadoop-yarn-server-nodemanager ……………….. SUCCESS [1:22.911s]
[INFO] hadoop-yarn-server-web-proxy …………………. SUCCESS [21.555s]
[INFO] hadoop-yarn-server-applicationhistoryservice …… SUCCESS [43.859s]
[INFO] hadoop-yarn-server-resourcemanager ……………. SUCCESS [1:32.575s]
[INFO] hadoop-yarn-server-tests …………………….. SUCCESS [37.084s]
[INFO] hadoop-yarn-client ………………………….. SUCCESS [48.894s]
[INFO] hadoop-yarn-applications …………………….. SUCCESS [0.282s]
[INFO] hadoop-yarn-applications-distributedshell ……… SUCCESS [18.888s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher …. SUCCESS [15.902s]
[INFO] hadoop-yarn-site ……………………………. SUCCESS [0.258s]
[INFO] hadoop-yarn-registry ………………………… SUCCESS [36.312s]
[INFO] hadoop-yarn-project …………………………. SUCCESS [21.744s]
[INFO] hadoop-mapreduce-client ……………………… SUCCESS [0.624s]
[INFO] hadoop-mapreduce-client-core …………………. SUCCESS [1:45.227s]
[INFO] hadoop-mapreduce-client-common ……………….. SUCCESS [1:25.513s]
[INFO] hadoop-mapreduce-client-shuffle ………………. SUCCESS [30.676s]
[INFO] hadoop-mapreduce-client-app ………………….. SUCCESS [1:00.069s]
[INFO] hadoop-mapreduce-client-hs …………………… SUCCESS [51.428s]
[INFO] hadoop-mapreduce-client-jobclient …………….. SUCCESS [28.007s]
[INFO] hadoop-mapreduce-client-hs-plugins ……………. SUCCESS [16.043s]
[INFO] Apache Hadoop MapReduce Examples ……………… SUCCESS [34.093s]
[INFO] hadoop-mapreduce ……………………………. SUCCESS [15.412s]
[INFO] Apache Hadoop MapReduce Streaming …………….. SUCCESS [28.932s]
[INFO] Apache Hadoop Distributed Copy ……………….. SUCCESS [43.127s]
[INFO] Apache Hadoop Archives ………………………. SUCCESS [19.688s]
[INFO] Apache Hadoop Rumen …………………………. SUCCESS [42.575s]
[INFO] Apache Hadoop Gridmix ……………………….. SUCCESS [28.627s]
[INFO] Apache Hadoop Data Join ……………………… SUCCESS [19.835s]
[INFO] Apache Hadoop Ant Tasks ……………………… SUCCESS [21.745s]
[INFO] Apache Hadoop Extras ………………………… SUCCESS [22.839s]
[INFO] Apache Hadoop Pipes …………………………. SUCCESS [19.852s]
[INFO] Apache Hadoop OpenStack support ………………. SUCCESS [38.304s]
[INFO] Apache Hadoop Amazon Web Services support ……… SUCCESS [31.721s]
[INFO] Apache Hadoop Client ………………………… SUCCESS [42.984s]
[INFO] Apache Hadoop Mini-Cluster …………………… SUCCESS [1.176s]
[INFO] Apache Hadoop Scheduler Load Simulator ………… SUCCESS [32.627s]
[INFO] Apache Hadoop Tools Dist …………………….. SUCCESS [30.490s]
[INFO] Apache Hadoop Tools …………………………. SUCCESS [0.207s]
[INFO] Apache Hadoop Distribution …………………… SUCCESS [1:49.560s]
[INFO] ————————————————————————
[INFO] BUILD SUCCESS
[INFO] ————————————————————————
[INFO] Total time: 1:00:47.533s
[INFO] Finished at: Fri Dec 09 07:03:01 CST 2016
[INFO] Final Memory: 81M/237M
[INFO] ————————————————————————

然后进入
/home/hadoop/apps/hadoop-2.6.4-src/hadoop-dist/target/hadoop-2.6.4/lib/native
使用file * 检查 如果出现如下信息表示编译成功

---