hadoop-2.8.2编译64位教程 CentOS7 jdk1.8

一.编译前的准备

1.CentOS7系统的机器

2.jdk1.8 

安装命令：rpm -ivh jdk-8u151-linux-x64.rpm

验证命令：java -version

3.源码包hadoop-2.8.2-src.tar.gz

解压命令：tar -zxvf hadoop-2.8.2-src.tar.gz

4.安装Apache-Maven

解压命令：tar -zxvf apache-maven-3.5.2-bin.tar.gz

重命名命令：mv apache-maven-3.5.2 maven

验证命令：mvn -version

5.安装Apache-ant

解压命令：tar -zxvf apache-ant-1.10.1-bin.tar.gz

重命名命令：mv apache-ant-1.10.1 ant

验证命令：ant -version

6.安装glibc-headers、g++、make和cmake等编译需要的工具

yum install glibc-headers

yum install gcc

yum install gcc-c++

yum install make

yum install cmake

yum install openssl-devel

yum install ncurses-devel

7.安装protobuf(goole序列化工具)

解压命令：tar -zxvf protobuf-2.5.0.tar.gz

解压完成后进入protobuf的主目录

执行以下命令：

./autogen.sh  命令执行完后会生成configure文件

./configure 
make && make install 
ldconfig 

验证命令：protoc --version

以上每次安装完都需要配置环境变量并使用/etc/profile命令刷新配置文件后，才可以验证

配置完后的配置文件：

JAVA_HOME=/usr/java/jdk1.8.0_151
JRE_HOME=/usr/java/jdk1.8.0_151/jre
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
export JAVA_HOME JRE_HOME PATH CLASSPATH
export MAVEN_HOME=/usr/local/maven
export PATH=$PATH:$MAVEN_HOME/bin
export ANT_HOME=/usr/local/ant
export PATH=$PATH:$ANT_HOME/bin
export LD_LIBRARY_PATH=/root/protobuf/protobuf-2.5.0

二.编译阶段

1.cd到hadoop-2.8.2-src目录下

2.执行命令：mvn package -Pdist,native -DskipTests -Dtar 需要等待一段时间

最终会出现

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop Main ................................. SUCCESS [  1.757 s]
[INFO] Apache Hadoop Build Tools .......................... SUCCESS [  1.438 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [  1.393 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [  2.716 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.257 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  1.993 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  4.192 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [  5.291 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [  6.596 s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [  4.516 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [02:59 min]
[INFO] Apache Hadoop NFS .................................. SUCCESS [ 13.017 s]
[INFO] Apache Hadoop KMS .................................. SUCCESS [ 38.222 s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [  0.071 s]
[INFO] Apache Hadoop HDFS Client .......................... SUCCESS [03:36 min]
[INFO] Apache Hadoop HDFS ................................. SUCCESS [02:38 min]
[INFO] Apache Hadoop HDFS Native Client ................... SUCCESS [ 15.576 s]
[INFO] Apache Hadoop HttpFS ............................... SUCCESS [ 42.793 s]
[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [02:04 min]
[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [  5.893 s]
[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [  0.060 s]
[INFO] Apache Hadoop YARN ................................. SUCCESS [  0.085 s]
[INFO] Apache Hadoop YARN API ............................. SUCCESS [ 20.962 s]
[INFO] Apache Hadoop YARN Common .......................... SUCCESS [03:02 min]
[INFO] Apache Hadoop YARN Server .......................... SUCCESS [  0.091 s]
[INFO] Apache Hadoop YARN Server Common ................... SUCCESS [ 10.500 s]
[INFO] Apache Hadoop YARN NodeManager ..................... SUCCESS [ 23.289 s]
[INFO] Apache Hadoop YARN Web Proxy ....................... SUCCESS [  3.668 s]
[INFO] Apache Hadoop YARN ApplicationHistoryService ....... SUCCESS [ 38.188 s]
[INFO] Apache Hadoop YARN ResourceManager ................. SUCCESS [ 36.054 s]
[INFO] Apache Hadoop YARN Server Tests .................... SUCCESS [  3.700 s]
[INFO] Apache Hadoop YARN Client .......................... SUCCESS [  7.952 s]
[INFO] Apache Hadoop YARN SharedCacheManager .............. SUCCESS [  4.912 s]
[INFO] Apache Hadoop YARN Timeline Plugin Storage ......... SUCCESS [  3.788 s]
[INFO] Apache Hadoop YARN Applications .................... SUCCESS [  0.043 s]
[INFO] Apache Hadoop YARN DistributedShell ................ SUCCESS [  2.988 s]
[INFO] Apache Hadoop YARN Unmanaged Am Launcher ........... SUCCESS [  2.406 s]
[INFO] Apache Hadoop YARN Site ............................ SUCCESS [  0.051 s]
[INFO] Apache Hadoop YARN Registry ........................ SUCCESS [  5.572 s]
[INFO] Apache Hadoop YARN Project ......................... SUCCESS [  6.461 s]
[INFO] Apache Hadoop MapReduce Client ..................... SUCCESS [  0.267 s]
[INFO] Apache Hadoop MapReduce Core ....................... SUCCESS [ 35.973 s]
[INFO] Apache Hadoop MapReduce Common ..................... SUCCESS [ 26.100 s]
[INFO] Apache Hadoop MapReduce Shuffle .................... SUCCESS [  6.938 s]
[INFO] Apache Hadoop MapReduce App ........................ SUCCESS [ 11.804 s]
[INFO] Apache Hadoop MapReduce HistoryServer .............. SUCCESS [  6.960 s]
[INFO] Apache Hadoop MapReduce JobClient .................. SUCCESS [ 25.182 s]
[INFO] Apache Hadoop MapReduce HistoryServer Plugins ...... SUCCESS [  2.292 s]
[INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [  6.715 s]
[INFO] Apache Hadoop MapReduce ............................ SUCCESS [  3.803 s]
[INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [ 13.631 s]
[INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [ 26.277 s]
[INFO] Apache Hadoop Archives ............................. SUCCESS [  2.932 s]
[INFO] Apache Hadoop Archive Logs ......................... SUCCESS [  2.705 s]
[INFO] Apache Hadoop Rumen ................................ SUCCESS [  5.781 s]
[INFO] Apache Hadoop Gridmix .............................. SUCCESS [  4.443 s]
[INFO] Apache Hadoop Data Join ............................ SUCCESS [  2.363 s]
[INFO] Apache Hadoop Ant Tasks ............................ SUCCESS [  1.989 s]
[INFO] Apache Hadoop Extras ............................... SUCCESS [  3.201 s]
[INFO] Apache Hadoop Pipes ................................ SUCCESS [ 11.117 s]
[INFO] Apache Hadoop OpenStack support .................... SUCCESS [  6.124 s]
[INFO] Apache Hadoop Amazon Web Services support .......... SUCCESS [ 46.188 s]
[INFO] Apache Hadoop Azure support ........................ SUCCESS [ 24.456 s]
[INFO] Apache Hadoop Client ............................... SUCCESS [  8.949 s]
[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [  1.340 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [  9.491 s]
[INFO] Apache Hadoop Azure Data Lake support .............. SUCCESS [01:14 min]
[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [  8.251 s]
[INFO] Apache Hadoop Tools ................................ SUCCESS [  0.032 s]
[INFO] Apache Hadoop Distribution ......................... SUCCESS [01:02 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27:03 min
[INFO] Finished at: 2017-11-06T22:10:35+08:00
[INFO] Final Memory: 105M/237M
[INFO] ------------------------------------------------------------------------

出现以上的信息表明编译成功！

三.注：

本人在编译过程中遇到的唯一的问题就是protobuf的版本问题，本人用的版本过高，用的是3.4的，随后换成2.5的版本后，一次成功。