mac系统编译3.2.1版本hadoop

缘由

编译hadoop主要是为了解决安装hadoop缺少lib/native，发生的“WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform using builtin-java classes where applicable”警告，这个警告虽然不影响hadoop的正常运行
中间想过放弃编译直接下载安装包，用其中的lib/native，发现mac下不能用.so的包。

中间走的弯路，没有效果，是进到hadoop-hdfs-native-client中的src目录下运行的
cmake -DJVM_ARCH_DATA_MODEL=64 -DOPENSSL_ROOT_DIR=/usr/local/Cellar/openssl@1.1/1.1.1g  -DOPENSSL_LIBRARIES=/usr/local/Cellar/openssl@1.1/1.1.1g/lib -DOPENSSL_INCLUDE_DIR=/usr/local/Cellar/openssl@1.1/1.1.1g/include -DPROTOBUF_INCLUDE_DIR=/usr/local/Cellar/protobuf/include -DPROTOBUF_LIBRARY=/usr/local/Cellar/protobuf/lib/libprotoc.dylib

步骤

下载hadoop源码

从官网上下载：下载地址
解压后注意阅读BUILDING.txt文件
Requirements:

• Unix System
• JDK 1.8
• Maven 3.3 or later
• ProtocolBuffer 2.5.0
• CMake 3.1 or newer (if compiling native code)
• Zlib devel (if compiling native code)
• Cyrus SASL devel (if compiling native code)
• One of the compilers that support thread_local storage: GCC 4.8.1 or later, Visual Studio,
  Clang (community version), Clang (version for iOS 9 and later) (if compiling native code)
• openssl devel (if compiling native hadoop-pipes and to get the best HDFS encryption performance)
• Linux FUSE (Filesystem in Userspace) version 2.6 or above (if compiling fuse_dfs)
• Doxygen ( if compiling libhdfspp and generating the documents )
• Internet connection for first build (to fetch all Maven and Hadoop dependencies)
• python (for releasedocs)
• bats (for shell code testing)
• Node.js / bower / Ember-cli (for YARN UI v2 building)

安装maven

brew install maven

#或者下载maven的包解压到/usr/local/maven，在bash_profile中配置

export MAVEN_HOME=/usr/local/maven
export PATH=$PATH:$MAVEN_HOME/bin


#配置settings
<mirrors>
      <mirror>
        <id>alimaven</id>
        <name>aliyun maven</name>
        <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
        <mirrorOf>central</mirrorOf>        
      </mirror>
      <mirror>
        <id>repo1</id>
        <mirrorOf>central</mirrorOf>
        <name>repo1 maven</name>
        <url>http://repo1.maven.org/maven2/</url>
      </mirror>
     <mirror>
         <id>repo2</id>
         <mirrorOf>central</mirrorOf>
         <name>repo2 maven</name>
         <url>http://repo2.maven.org/maven2/</url>
    </mirror>
</mirrors>

安装openssl