CentOS7 Hadoop 3.1.0 编译安装

 

1.配置环境变量

JAVA_HOME=/jdk1.8.0_131
ANT_HOME=/apache-ant-1.10.5
MAVEN_HOME=/apache-maven-3.5.4
FINDBUGS_HOME=/findbugs-3.0.1
PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$FINDBUGS_HOME/bin:$MAVEN_HOME/bin:$PATH
CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar

export JAVA_HOME ANT_HOME MAVEN_HOME FINDBUGS_HOME PATH CLASSPATH

2.安装必须依赖

yum -y install lzo-devel zlib-devel autoconf automake libtool cmake openssl-devel patch gcc gcc-c++
yum -y install protobuf.x86_64 protobuf-devel.x86_64

3.对于2步骤安装的cmake可能版本过低，如果编译Hadoop源码时，出现cmake错误，可以手动编译安装cmake

yum remove cmake
tar -zxvf cmake-3.12.0.tar.gz 
cd cmake-3.12.0
 ./configure && make && make install

4.解压hadoop源码，增加dynamodb-local-oregon repository

vi hadoop-3.1.0-src/pom.xml

<repositories>

    <repository>

        <id>dynamodb-local-oregon</id>

        <name>DynamoDB Local Release Repository</name>

        <url>https://s3-us-west-2.amazonaws.com/dynamodb-local/release</url>

    </repository>

    ...

  </repositories>

参照amazon网站数据：
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html#DynamoDBLocal.DownloadingAndRunning

5.运行maven命令

pwd

hadoop-3.1.0-src/

mvn package -Pdist,native,docs -DskipTests -Dtar