Linux环境编译Hadoop源码包

hadoop2.7.4

centos6.5

1.准备的资料

源码根目录下有个BUILDINT.txt，打开即可看见里面关于编译hadoop的一些环境要求

/opt/software/ 存放软件安装包

/opt/module/  存放解压包

2.安装JDK(1.8)

[root@master-node software]# tar -zxvf jdk-8u211-linux-x64.tar.gz  -C /opt/module/
[root@master-node software]# vim /etc/profile
#Java配置
export JAVA_HOME=/opt/module/jdk1.8.0_211
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$JAVA_HOME/bin:$PATH
[root@master-node software]# source /etc/profile
[root@master-node software]# java -version

3.安装maven(3.6.1)

[root@master-node software]# tar -zxvf apache-maven-3.6.1-bin.tar.gz -C /opt/module/
[root@master-node apache-maven-3.6.1]# vim conf/settings.xml
 <mirrors>  
    <!--添加阿里镜像-->
    <mirror>
      <id>alimaven</id>
      <mirrorOf>central</mirrorOf>
      <name>aliyun maven</name>
      <url>https://maven.aliyun.com/repository/central</url>
    </mirror>
    <mirror> 
      <id>aliyun-maven</id> 
      <mirrorOf>*</mirrorOf> 
      <name>aliyun maven</name> 
      <url>http://maven.aliyun.com/nexus/content/groups/public</url> 
    </mirror>
  </mirrors>
[root@master-node software]# vim /etc/profile
#Maven配置
export MAVEN_HOME=/opt/module/apache-maven-3.6.1
export PATH=$PATH:$MAVEN_HOME/bin
[root@master-node software]# source /etc/profile
[root@master-node software]# mvn -version

4.安装cmake,zlib-devel,openssl-devel 依赖库

[root@master-node software]# yum -y install cmake zlib-devel openssl-devel

5.安装ProtocolBuffer 2.5.0 

[root@slave-node1 software]# tar -zxvf protobuf-2.5.0.tar.gz -C /opt/module/
[root@slave-node1 software]# cd /opt/module/protobuf-2.5.0/
[root@master-node protobuf-2.5.0]# ./configure --prefix=/opt/module/protobuf
[root@master-node protobuf-2.5.0]# make
[root@master-node protobuf-2.5.0]# make check
[root@master-node protobuf-2.5.0]# make install
[root@master-node protobuf-2.5.0]# vim /etc/profile
#Protobuf配置
export PROTOC_HOME=/opt/module/protobuf
export PATH=$PATH:$PROTOC_HOME/bin
[root@master-node protobuf-2.5.0]# source /etc/profile
[root@master-node protobuf-2.5.0]# protoc --version

到此，编译工具安装基本完成

6.编译Hadoop

1）解压源码到/opt/module/目录

[root@master-node software]# tar -zxvf hadoop-2.7.4-src.tar.gz -C /opt/module/

2）进入到Hadoop源码主目录，通过maven命令执行编译

[root@master-node software]# cd /opt/module/hadoop-2.7.4-src/
[root@master-node hadoop-2.7.4-src]# mvn package -Pdist,native -DskipTests -Dtar

编译成功

3）编译成功的64位hadoop包在/opt/module/hadoop-2.7.4-src/hadoop-dist/target目录下

[root@master-node target]# cd /opt/module/hadoop-2.7.4-src/hadoop-dist/target