java远程提交yarn到集群_Eclipse远程提交hadoop集群任务

文章概览：

1、前言

2、Eclipse查看远程hadoop集群文件

3、Eclipse提交远程hadoop集群任务

4、小结

1 前言

Hadoop高可用品台搭建完备后，参见《Hadoop高可用平台搭建》，下一步是在集群上跑任务，本文主要讲述Eclipse远程提交hadoop集群任务。

2 Eclipse查看远程hadoop集群文件

2.1 编译hadoop eclipse 插件

Hadoop集群文件查看可以通过webUI或hadoop Cmd，为了在Eclipse上方便增删改查集群文件，我们需要编译hadoop eclipse 插件，步骤如下：

① 环境准备

JDK环境配置　　配置JAVA_HOME，并将bin目录配置到path

ANT环境配置　　配置ANT_HOME，并将bin目录配置到path

在cmd查看：

② 软件准备

hadoop-2.6.0

eclipse-jee-luna-SR2-win32-x86_64

③ 编译

注：软件位置为自己机器上位置，请勿照搬。

E:\>cd E:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin

E:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin>ant jar -Dve

rsion=2.6.0 -Declipse.home=E:\eclipse -Dhadoop.home=E:\hadoop\hadoop-2.6.0Buildfile: E:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\b

uild.xml

check-contrib:

init:

[echo] contrib: eclipse-plugin

init-contrib:

ivy-probe-antlib:

ivy-init-antlib:

ivy-init:

[ivy:configure] :: Ivy2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::

[ivy:configure] :: loading settings :: file = E:\hadoop\hadoop2x-eclipse-plugin-master\ivy\ivysettings.xml

ivy-resolve-common:

ivy-retrieve-common:

[ivy:cachepath] DEPRECATED:'ivy.conf.file' is deprecated, use 'ivy.settings.fil

e'instead

[ivy:cachepath] :: loading settings :: file = E:\hadoop\hadoop2x-eclipse-plugin-master\ivy\ivysettings.xml

compile:

[echo] contrib: eclipse-plugin

[javac] E:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\

build.xml:76: warning: 'includeantruntime' was not set, defaulting to build.sysc

lasspath=last; set to false forrepeatable builds

jar:

BUILD SUCCESSFUL

Total time:10 seconds

成功编译，生成如下图：

④ 将改文件拷贝到Eclipse中plugins目录下，重启Eclipse会出现：

2.2 配置hadoop选项

打开Map/Reduce Locations

编辑Map/Reduce配置项：

根据上一篇，我们配置用户hadoop&#