hadoop 编译 版本(hadoop-2.6.0-cdh5.15.1)

最新推荐文章于 2021-07-29 11:19:21 发布
最新推荐文章于 2021-07-29 11:19:21 发布
阅读量3.7k 收藏 4
点赞数 1
分类专栏： 大数据相关博文 文章标签： hadoop编译 hadoop cdh5.15.1编译 com.amazonaws:DynamoDBLocal:jar maven仓库地址
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/yz972641975/article/details/98405720
版权

编译前准备工作

系统环境配置

操作系统版本：CentOS Linux release 7.5.1804

需要提前支持的类库：

# 安装 gcc、gcc-c++
yum install gcc
yum install glibc-headers
yum install gcc-c++ 

# 安装 cmake
yum install cmake

# 安装 openssl-devel
yum install openssl-devel

# 安装 zlib-devel、ncurses-devel
yum -y install zlib-devel 
yum -y install ncurses-devel

# 卸载 google protobuf
rm `which protoc`
需要的包
  • 1.hadoop-2.6.0-cdh5.15.1源码包，下载地址http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.15.1-src.tar.gz
  • 2.jdk环境包。需要jdk1.7.X版本，下载地址https://download.oracle.com/otn/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz
  • 3.maven环境包。需要maven3.0.X版本，下载地址https://archive.apache.org/dist/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz
  • 4.protobuf包。需要protobuf-2.5.0版本，从这个地址https://github.com/protocolbuffers/protobuf/releases中寻找所需版本
配置环境变量。

  • 1.jdk，maven环境变量配置不在赘述

  • 2.protobuf 配置
    protobuf下载的是源码，所有需要经过linux源码编译的三个步骤，configure,make ,make install

  • 编译protobuf源码
#配置编译信息  --prefix 是编译好的文件存放位置
./configure --prefix=/opt/module/protobuf-2.5.0
#编译
make
#安装
make install
  • 编译中出现的报错
checking dependency style of g++... none
checking how to run the C++ preprocessor... /lib/cpp
configure: error: in `/opt/module/protobuf-2.5.0':
configure: error: C++ preprocessor "/lib/cpp" fails sanity check
See `config.log' for more details

问题的根源是缺少必要的C++库。如果是CentOS系统，运行，如下命令解决：

yum install -y glibc-headers
yum install -y gcc-c++
  • 配置环境变量
PROTOBUF=/opt/module/protobuf-2.5.0
export PATH=$PROTOBUF/bin:$PATH
  • 验证是否安装成功
$ protoc --version
libprotoc 2.5.0

protobuf安装配置完成

配置maven信息

打开maven下面的conf目录下的settings文件。增加仓库地址:可从中选择需要的地址。把四个都加上也没什么问题，需要配置一个mirrorOf为central的地址。


	<mirror>
		<id>central</id>
		<name>central</name>
		<url>http://central.maven.org/maven2/</url>
		<mirrorOf>*</mirrorOf>
	</mirror>
	
	<mirror>
		<id>cdh</id>
		<mirrorOf>*</mirrorOf>
		<name>cdh</name>
		<url>http://repository.cloudera.com/artifactory/cloudera-repos/</url>
	</mirror>
	
	<mirror>
		<id>dynamodb-local-oregon</id>
		<name>DynamoDB Local Release Repository</name>
		<url>https://s3-ap-southeast-1.amazonaws.com/dynamodb-local-singapore/release</url>
		<mirrorOf>*</mirrorOf>
	</mirror>
	<mirror>
        <id>aliyun</id>
        <mirrorOf>central</mirrorOf>
        <name>aliyun</name> 
        <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
    </mirror>

解压hadoop源码文件

tar -zxvf hadoop-2.6.0-cdh5.15.1-src.tar.gz,可查看解压出来后的目录下的BUILDING.txt帮助文档。

这里编译只依赖ProtocolBuffer 2.5.0

进入到解压目录。开始执行编译mvn clean package -Pdist,native -DskipTests -Dtar
这会进入一个漫长的等待过程。有时候还是遇到编译报错的问题，不过这些一般都是jar找不到。可以使用更换镜像源的方式进行重试。

编译成功

[INFO] Executed tasks
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ hadoop-dist ---
[INFO] Building jar: /opt/soft/hadoop-2.6.0-cdh5.15.1/hadoop-dist/target/hadoop-dist-2.6.0-cdh5.15.1-javadoc.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop Main ................................ SUCCESS [2.983s]
[INFO] Apache Hadoop Build Tools ......................... SUCCESS [1.059s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [0.959s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [3.006s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.787s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [1.746s]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [3.699s]
[INFO] Apache Hadoop MiniKDC ............................. SUCCESS [4.050s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [5.775s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [2.983s]
[INFO] Apache Hadoop Common .............................. SUCCESS [1:47.114s]
[INFO] Apache Hadoop NFS ................................. SUCCESS [6.032s]
[INFO] Apache Hadoop KMS ................................. SUCCESS [13.665s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.060s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [2:49.461s]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [18.896s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [4.894s]
[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [4.443s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.051s]
[INFO] hadoop-yarn ....................................... SUCCESS [0.051s]
[INFO] hadoop-yarn-api ................................... SUCCESS [1:27.662s]
[INFO] hadoop-yarn-common ................................ SUCCESS [26.418s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.057s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [9.688s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [16.387s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [3.505s]
[INFO] hadoop-yarn-server-applicationhistoryservice ...... SUCCESS [6.166s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [18.295s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [0.819s]
[INFO] hadoop-yarn-client ................................ SUCCESS [5.365s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [0.047s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [2.641s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [1.794s]
[INFO] hadoop-yarn-site .................................. SUCCESS [0.048s]
[INFO] hadoop-yarn-registry .............................. SUCCESS [4.993s]
[INFO] hadoop-yarn-project ............................... SUCCESS [5.834s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.152s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [18.929s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [16.882s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [4.145s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [9.546s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [5.960s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [5.755s]
[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [1.877s]
[INFO] hadoop-mapreduce-client-nativetask ................ SUCCESS [1:31.489s]
[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [5.611s]
[INFO] hadoop-mapreduce .................................. SUCCESS [5.630s]
[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [4.339s]
[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [9.237s]
[INFO] Apache Hadoop Archives ............................ SUCCESS [2.097s]
[INFO] Apache Hadoop Archive Logs ........................ SUCCESS [2.211s]
[INFO] Apache Hadoop Rumen ............................... SUCCESS [5.501s]
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [4.200s]
[INFO] Apache Hadoop Data Join ........................... SUCCESS [2.701s]
[INFO] Apache Hadoop Ant Tasks ........................... SUCCESS [2.444s]
[INFO] Apache Hadoop Extras .............................. SUCCESS [3.119s]
[INFO] Apache Hadoop Pipes ............................... SUCCESS [7.162s]
[INFO] Apache Hadoop OpenStack support ................... SUCCESS [5.200s]
[INFO] Apache Hadoop Amazon Web Services support ......... SUCCESS [9.485s]
[INFO] Apache Hadoop Azure support ....................... SUCCESS [4.569s]
[INFO] Apache Hadoop Client .............................. SUCCESS [3.313s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [0.671s]
[INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [5.148s]
[INFO] Apache Hadoop Azure Data Lake support ............. SUCCESS [3.159s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [16.046s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [0.042s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [52.728s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14:13.400s
[INFO] Finished at: Sat Aug 03 00:12:42 EDT 2019
[INFO] Final Memory: 257M/858M
[INFO] ------------------------------------------------------------------------

看到这个日志，是不是很欣慰，终于出来了。(表示本人弄了2天才完成！！！笑哭)

查看编译完成后的目录，及测试

  • 编译好的目录会在解压目录下的hadoop-dist/target文件找到， 这里就是编译输出的目录。
  • 解压出hadoop-2.6.0-cdh5.15.1.tar.gz到指定位置。开始测试。
  • 需要给解压的目录配置一个环境变量（方便测试）
HADOOP_HOME=/opt/module/hadoop-2.6.0-cdh5.15.1
export PATH=$HADOOP_HOME/bin:$PATH
  • 在当前目录下配置一个建立一个文件夹并放入文件(mkdir -p ./test/input/),文件随意
  • 执行hadoop自带的测试程序hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0-cdh5.15.1.jar wordcount ./test/input/ ./test/output/。如果没有报错，并且在/test/output/目录下有输出结果，恭喜，编译没有问题！！

编译的时候遇到的问题

  • NO.1
[heasy@demo hadoop-2.6.0-cdh5.15.1]$ mvn clean package -Pdist,native -DskipTests -Dtar
[INFO] Scanning for projects...
Downloading from cdh.repo: https://repository.cloudera.com/artifactory/cloudera-repos/com/cloudera/cdh/cdh-root/5.15.1/cdh-root-5.15.1.pom
Downloading from nexus-cdh: https://repository.cloudera.com/artifactory/cloudera-repos/com/cloudera/cdh/cdh-root/5.15.1/cdh-root-5.15.1.pom
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for org.apache.hadoop:hadoop-main:2.6.0-cdh5.15.1: Could not transfer artifact com.cloudera.cdh:cdh-root:pom:5.15.1 from/to cdh.repo (https://repository.cloudera.com/artifactory/cloudera-repos): Remote host closed connection during handshake and 'parent.relativePath' points at wrong local POM @ line 19, column 11
 @ 
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.apache.hadoop:hadoop-main:2.6.0-cdh5.15.1 (/opt/module/hadoop-2.6.0-cdh5.15.1/pom.xml) has 1 error
[ERROR]     Non-resolvable parent POM for org.apache.hadoop:hadoop-main:2.6.0-cdh5.15.1: Could not transfer artifact com.cloudera.cdh:cdh-root:pom:5.15.1 from/to cdh.repo (https://repository.cloudera.com/artifactory/cloudera-repos): Remote host closed connection during handshake and 'parent.relativePath' points at wrong local POM @ line 19, column 11: SSL peer shut down incorrectly -> [Help 2]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException

解决方案：前往本地仓库.m2/repository/com/cloudera/cdh/cdh-root/5.15.1到目标文件目录，然后 通过wget文件，来成功获取该文件，重新执行编译命令，将需要的jar直接放到本地仓库

  • NO.2
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project hadoop-auth: Could not resolve dependencies for project org.apache.hadoop:hadoop-auth:jar:2.6.0-cdh5.15.1: Failed to collect dependencies at org.mortbay.jetty:jetty-util:jar:6.1.26.cloudera.4: Failed to read artifact descriptor for org.mortbay.jetty:jetty-util:jar:6.1.26.cloudera.4: Could not transfer artifact org.mortbay.jetty:jetty-util:pom:6.1.26.cloudera.4 from/to nexus-cdh (https://repository.cloudera.com/artifactory/cloudera-repos/): Remote host closed connection during handshake: SSL peer shut down incorrectly -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hadoop-auth

多重试几次，或者换下中央仓库地址在试试

  • NO.3
Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/1.5.8/plexus-utils-1.5.8.jar (0 B at 0 B/s)
[WARNING] Rule 1: org.apache.maven.plugins.enforcer.RequireJavaVersion failed with message:
Detected JDK Version: 1.8.0-151 is not in the allowed range [1.7.0,1.7.1000}].

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce (clean) on project hadoop-main: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
  • NO.4
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.0:process (default) on project hadoop-main: Execution default of goal org.apache.maven.plugins:maven-remote-resources-plugin:1.0:process failed: Plugin org.apache.maven.plugins:maven-remote-resources-plugin:1.0 or one of its dependencies could not be resolved: Failed to collect dependencies at org.apache.maven.plugins:maven-remote-resources-plugin:jar:1.0 -> org.apache.maven:maven-core:jar:2.0.5: Failed to read artifact descriptor for org.apache.maven:maven-core:jar:2.0.5: Could not transfer artifact org.apache.maven:maven-core:pom:2.0.5 from/to cdh.releases.repo (https://repository.cloudera.com/content/repositories/cdh-releases-rcs): Connection reset -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException

下载这两个jar

        <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-remote-resources-plugin -->
        <dependency>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-remote-resources-plugin</artifactId>
            <version>1.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.maven/maven-core -->
        <dependency>
            <groupId>org.apache.maven</groupId>
            <artifactId>maven-core</artifactId>
            <version>2.0.5</version>
        </dependency>
  • NO.5
[ERROR] Failed to execute goal on project hadoop-aws: Could not resolve dependencies for project org.apache.hadoop:hadoop-aws:jar:2.6.0-cdh5.15.1: Failed to collect dependencies at com.amazonaws:DynamoDBLocal:jar:[1.11.86,2.0): No versions available for com.amazonaws:DynamoDBLocal:jar:[1.11.86,2.0) within specified range -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hadoop-aws

解决：
参考：https://www.e-learn.cn/content/qita/769641和https://docs.aws.amazon.com/zh_cn/amazondynamodb/latest/developerguide/DynamoDBLocal.html
个人解决方案（如有更好方式，欢迎探讨）：在idea编辑器中建立一个先空的maven项目。然后在pom.xml中增加依赖

<!--Dependency:-->
<dependencies>
    <dependency>
       <groupId>com.amazonaws</groupId>
       <artifactId>DynamoDBLocal</artifactId>
       <version>[1.11,2.0)</version>
    </dependency>
</dependencies>
<!--Custom repository:-->
<repositories>
    <repository>
       <id>dynamodb-local-oregon</id>
       <name>DynamoDB Local Release Repository</name>
       <url>https://s3-us-west-2.amazonaws.com/dynamodb-local/release</url>
    </repository>
</repositories>

。然后在把这个下载好的文件拷贝到服务器上，因为这个下载的是一个很广泛的版本，在下载好后通过查看maven版本，得到一个可用的版本。这里的可用版本是1.11.477,修改hadoop-2.6.0-cdh5.15.1中的hadoop-project项目中的pom.xml文件中对应DynamoDBLocal的版本为1.11.477，重新编译即可。

这里有个问题,现在这个https://s3-us-west-2.amazonaws.com/dynamodb-local/release的文件需要翻墙。。

附：仓库文件和软件

链接：https://pan.baidu.com/s/13-w-A6-sMCUo0pXY4S0RRg
提取码：cdia

在这里插入图片描述

楚码小生
关注
专栏目录
hadoop-2.6.0-cdh5.15.1.zip
07-27
Hadoop的CDH版本是什么？ 基于Apache协议，100%开源。 版本管理清晰 提供了部署、安装、配置工具，大大提高了集群部署的效率 运维简单。提供了管理、监控、诊断、配置修改的工具，管理配置方便，定位问题快速、准确，使运维工作简单，有效。
hadoop-2.6.0-cdh5.15.1.tar.gz
04-24
大数据/Linux安装包-hadoop-2.6.0-cdh5.15.1.tar.gz 大数据/Linux安装包-hadoop-2.6.0-cdh5.15.1.tar.gz 大数据/Linux安装包-hadoop-2.6.0-cdh5.15.1.tar.gz
使用阿里云主机 编译 hadoop-2.6.0-cdh5.15.1
xinji
08-09 334
使用阿里云主机 编译 hadoop-2.6.0-cdh5.15.1 1.准备 1.系统环境配置 操作系统版本：CentOS Linux release 7.5.1804 依赖库的安装 [root@hadoop001 ~]# yum install -y svn ncurses-devel [root@hadoop001 ~]# yum install -y gcc gcc-c++ make cma...
CDH最新版本5.15.1下载
11-19
CDH最新版本的安装包下载，CDH5.15.1下载，直接下载安装
hadoop-2.6.0-cdh5.15.2编译版本(CentOS 7.3)
04-27
带编译所需的maven库，hadoop-2.6.0-cdh5.15.2在CentOS Linux release 7.3.1611重新编译的版本
阿里云主机hadoop2.6.0-cdh5.15.1-HA环境搭建
whiteblacksheep的博客
08-23 641
1.版本 组件 版本 备注 centos centos7.2 JRE jdk-8u45-linux-x64.gz - hadoop hadoop-2.6.0-cdh5.15.1.tar.gz 自己通过源码编译的，具有压缩工具 zookeeper zookeeper-3.4.6.tar.gz 热切，yarn存储数据使用的协调服务 2.阿里云环境搭建 具体步骤在这不详...
spark2.4.2源码编译集成hadoop-2.6.0-cdh5.15.1
cllblogs
09-06 585
1.准备工作 1.1 组件及源码下载 组件名称 组件版本 centos centos6.4 jdk jdk-8u80-linux-x64.tar.gz maven apache-maven-3.6.1-bin.tar.gz scala scala-2.11.6.tgz hadoop Hadoop-2.6.0-cdh5.15.1-src.tar.gz ⚠️ 版本要...
hadoop-2.6.0-cdh5.15.1源码编译安装遇到的问题
whiteblacksheep的博客
08-01 1351
编译安装步骤还是安装之前的hadoop-2.6.0-cdh5.7.0,这里就不说了 遇到的问题： [FATAL] Non-resolvable parent POM for org.apache.hadoop:hadoop-main:2.6.0-cdh5.15.1: Could not transfer artifact com.cloudera.cdh:cdh-root:pom:5.15....
hadoop-2.6.0-cdh5.15.1源码编译添加压缩支持
SUDDEV的博客
08-05 1320
文章目录1.前言2.环境准备 直接从 Cloudera官网下载的hadoop-2.6.0-cdh5.15.1是不支持压缩的，而生产上往往需要这方面的支持，所以需要自行下载源码并编译，这里记录一下自己编译的过程。 1.前言 首先我们使用命令hadoop checknative查看官方下载的hadoop-2.6.0-cdh5.15.1对压缩支持的情况 [root@suddev bin]# hado...
hadoop 2.6 编译
hadoop_在路上
02-06 462
配置依赖包     1.jdk1.7安装 环境变量配置（不会的自行百度）     2.maven安装       wget http://apache.fayea.com/maven/maven-3/3.2.5/binaries/apache-maven-3.2.5-bin.tar.gz        修改 setting.xml        镜像地址配置为开源中国的
hadoop-2.6.0.tar.gz
04-24
hadoop-2.6.0.tar.gz
hadoop-2.6.0-cdh5.14.2.tar.gz
09-12
hadoop安装包，需要的自行下载
hadoop-2.6.0-cdh5.10.0.tar.gz
03-15
Apache Hadoop 2.6.0-cdh5.10.1是2.x.y发行版中的次要发行版，它建立在先前的稳定版本2.4.1的基础上。
hadoop2.6.0版本-hadoop-2.6.0.tar.gz
10-28
用于在ubuntu上搭建hadoop集群的工具，使用wget下载太慢，放到csdn上来下载，速度贼快
hadoop-2.6.0源码编译
大讲台
09-19 652
运行hadoop环境时，常常会出现这种提示 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable  意思是无法加载本地native库。 这里就需要我们自己本地进行编译hadoop源码，用生成
编译hadoop java_hadoop 2.6.0源码编译
weixin_29574585的博客
02-15 167
这里面选择的HADOOP的版本为2.6.编译前需要准备的如下工具：HADOOP: hadoop-2.6.0-src.tar.gzJDK: jdk-7u71-linux-x64.tar.gzMAVEN: apache-maven-3.0.5-bin.tar.gzPROTOBUF:protobuf-2.5.0.tar.gzFINDBUGS: findbugs-3.0.0.tar.gzA...
hadoop-2.6.0.tar.gz下载
热门推荐
Jock的博客
02-22 1万+
本来就是开源的，结果csdn上下载还得要积分，这里给出官网链接，造福社会https://archive.apache.org/dist/hadoop/common/hadoop-2.6.0/打开可以找到hadoop-2.6.0.tar.gz，看清楚了，别下错了，复制链接用迅雷下载速度挺快的。...
Hadoop【环境搭建 04】【hadoop-2.6.0-cdh5.15.2.tar.gz 基于ZooKeeper搭建高可用集群】（部分图片来源于网络）
シ風
07-29 397
1.高可用简介 Hadoop 高可用 (High Availability) 分为 HDFS 高可用和 YARN 高可用，两者的实现基本类似，但HDFS NameNode 对数据存储及其一致性的要求比 YARN ResourceManger 高得多，所以它的实现也更加复杂。 HDFS的高可用实质是主备共享存储系统，不管是zookeeper还是QJM都是这个原理，一下是基于zookeeper的高可用集群搭建。 1.1 高可用整体架构 HDFS 高可用架构如下： HDFS 高可用架构主要由以下组件所构成： A
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值