在cygwin上用ant编译hadoop的eclipse插件

最新推荐文章于 2021-06-20 17:28:48 发布

wenlong237

最新推荐文章于 2021-06-20 17:28:48 发布

阅读量1.1k

点赞数

分类专栏： hadoop

本文链接：https://blog.csdn.net/wenlong237/article/details/17531041

版权

hadoop 专栏收录该内容

5 篇文章 0 订阅

订阅专栏

在Cygwin中尝试使用ant编译Hadoop的Eclipse插件时遇到了一系列错误，包括找不到软件包org.apache.hadoop.fs、类org.eclipse.jdt.debug.ui.launchConfigurations.JavaApplicationLaunchShortcut等。通过更新Eclipse的插件解决部分问题，但最终还是选择下载已编译的hadoop-eclipse-plugin-1.0.4.jar放入Eclipse的plugins目录以使插件正常工作，实现了Map/Reduce Locations的配置。

摘要由CSDN通过智能技术生成

1.参考了http://bigboy.iteye.com/blog/1766382，和其不同的地方为：

<!--
         <copyfile="${hadoop.root}/build/hadoop-core-${version}.jar"tofile="${build.dir}/lib/hadoop-core.jar"verbose="true"/>
         <copyfile="${hadoop.root}/build/ivy/lib/Hadoop/common/commons-cli-${commons-cli.version}.jar"  todir="${build.dir}/lib"verbose="true"/>
   -->
         <copyfile="${hadoop.root}/build/hadoop-core-${version}.jar"tofile="${build.dir}/lib/hadoop-core.jar"verbose="true"/>
         <copyfile="${hadoop.root}/build/ivy/lib/Hadoop/common/commons-cli-1.2.jar"  todir="${build.dir}/lib"verbose="true"/>
         <copyfile="${hadoop.root}/lib/commons-configuration-1.6.jar"todir="${build.dir}/lib" verbose="true"/>
         <copyfile="${hadoop.root}/lib/commons-httpclient-3.0.1.jar"todir="${build.dir}/lib" verbose="true"/>
         <copyfile="${hadoop.root}/lib/commons-lang-2.4.jar"todir="${build.dir}/lib" verbose="true"/>
         <copyfile="${hadoop.root}/lib/jackson-core-asl-1.0.1.jar"todir="${build.dir}/lib" verbose="true"/>
         <copyfile="${hadoop.root}/lib/jackson-mapper-asl-1.0.1.jar"todir="${build.dir}/lib" verbose="true"/>

2.最后在%hadoop_root%src\contrib\eclipse-plugin下ant jar报错：软件包org.apache.hadoop.fs不存在

参考http://blog.csdn.net/yundixiaoduo/article/details/7451753

3.ant -v jar提示找不到类org.eclipse.jdt.debug.ui.launchConfigurations.JavaApplicationLaunchShortcut

发现是因为自己用的eclipse太老了，在plugins文件夹中找到的jar包为org.eclipse.jdt.debug.ui_3.2.100.v20070531-1800.jar 2007年的jar包中没有这个类，遂随便下了org.eclipse.jdt.debug.ui_3.6.0.v20110512.jar，包中有此类，放在ant/lib下不再提示找不到此类

4.ant -v jar报错：

[javac] C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\src\java\org\apache\hadoop\eclipse\launch\HadoopApplicationLaunchShortcut.java:48:错误: 无法访问ILaunchShortcut2
   [javac] public class HadoopApplicationLaunchShortcut extends
   [javac]        ^
   [javac]   找不到org.eclipse.debug.ui.ILaunchShortcut2的类文件
   [javac]C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\src\java\org\apache\hadoop\eclipse\launch\HadoopApplicationLaunchShortcut.java:66:错误: 找不到符号
   [javac]        super.findLaunchConfiguration(type, configType);
   [javac]         ^
   [javac]   符号:   变量 super
   [javac]   位置: 类HadoopApplicationLaunchShortcut
   [javac]C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\src\java\org\apache\hadoop\eclipse\launch\HadoopApplicationLaunchShortcut.java:67:错误: 找不到符号
   [javac]     if (iConf == null)iConf = super.createConfiguration(type);
   [javac]                               ^
   [javac]   符号:   变量 super
   [javac]   位置: 类 HadoopApplicationLaunchShortcut
   [javac]C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\src\java\org\apache\hadoop\eclipse\launch\HadoopApplicationLaunchShortcut.java:60:错误: 方法不会覆盖或实现超类型的方法
   [javac]   @Override
   [javac]   ^
   [javac] 注:C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\src\java\org\apache\hadoop\eclipse\server\JarModule.java使用或覆盖了已过时的API。
   [javac] 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
   [javac] 注: 某些输入文件使用了未经检查或不安全的操作。
   [javac] 注: 有关详细信息, 请使用 -Xlint:unchecked 重新编译。
   [javac] 4 个错误
 
BUILD FAILED

服了，彻底投降！！

5.在网上随便找了hadoop-eclipse-plugin-1.0.4.jar放在eclipse/plugins/下重启eclipse，在showview>other中找到Map/Reduce Locations 配置host：127.0.0.1 Port：9000 9001

最终，eclipse中显示了hdfs的文件列表

6.总结：

1）第一次使用ant对其编译原理和结构不熟，一头雾水的瞎搜，瞎搞。。。

2）对xml认识不足，只能照着别人的瞎改。。。

build.xml的细致分析参见：http://blog.sina.com.cn/s/blog_4ed8b87701011c8h.html

7.最终原因

最终原因为装的eclipse版本太低（3.3）07年发布的版本，在网上查到说要3.3以上的版本才能顺利的编译hadoop的eclipse插件，否则会出现各种找不到jar包，就是上面我遇到的各种问题，最终下了最新Kepler Service Release 1，64位的，后eclipse无法打开：

Failed to load the JNI shared library

原因是原来的eclipse是32位的，jdk也是32位的，现在是64位的eclipse 32位的jdk，不匹配。卸载32位的jdk，下载64位的jdk，重新配置环境变量，问题解决。

在cygwin中 cd /hadoophome/src/contrib/eclipse_plugins ant jar 一切顺利，在hadoop-1.0.0\build\contrib\eclipse-plugin\下生成了hadoop-eclipse-plugin-1.0.0.jar

E5430 ~/hadoop-1.0.0/src/contrib/eclipse-plugin
$ ant jar
Buildfile: C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\build.xml

check-contrib:

init:
     [echo] contrib: eclipse-plugin

init-contrib:

ivy-download:
      [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
      [get] To: C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\ivy\ivy-2.1.0.jar
      [get] Not modified - so not downloaded

ivy-probe-antlib:

ivy-init-antlib:

ivy-init:
[ivy:configure] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\ivy\ivysettings.xml

ivy-resolve-common:
[ivy:resolve] :: resolving dependencies :: org.apache.hadoop#eclipse-plugin;working@ZhangWenL-E5430
[ivy:resolve]   confs: [common]
[ivy:resolve]   found commons-logging#commons-logging;1.0.4 in maven2
[ivy:resolve]   found log4j#log4j;1.2.15 in maven2
[ivy:resolve] :: resolution report :: resolve 209ms :: artifacts dl 5ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      common      |   2   |   0   |   0   |   0   ||   2   |   0   |
        ---------------------------------------------------------------------

ivy-retrieve-common:
[ivy:retrieve] :: retrieving :: org.apache.hadoop#eclipse-plugin [sync]
[ivy:retrieve]  confs: [common]
[ivy:retrieve]  0 artifacts copied, 2 already retrieved (0kB/8ms)
[ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead
[ivy:cachepath] :: loading settings :: file = C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\ivy\ivysettings.xml

compile:
     [echo] contrib: eclipse-plugin
    [javac] C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\build.xml:76: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
    [javac] Compiling 45 source files to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\classes
    [javac] 注: 某些输入文件使用或覆盖了已过时的 API。
    [javac] 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
    [javac] 注: 某些输入文件使用了未经检查或不安全的操作。
    [javac] 注: 有关详细信息, 请使用 -Xlint:unchecked 重新编译。

jar:
     [copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
     [copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\hadoop-core-1.0.0.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\hadoop-core.jar
     [copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
     [copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\ivy\lib\Hadoop\common\commons-cli-1.2.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\commons-cli-1.2.jar
     [copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
     [copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\lib\commons-configuration-1.6.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\commons-configuration-1.6.jar
     [copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
     [copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\lib\commons-httpclient-3.0.1.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\commons-httpclient-3.0.1.jar
     [copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
     [copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\lib\commons-lang-2.4.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\commons-lang-2.4.jar
     [copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
     [copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\lib\jackson-core-asl-1.0.1.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\jackson-core-asl-1.0.1.jar
     [copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
     [copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\lib\jackson-mapper-asl-1.0.1.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\jackson-mapper-asl-1.0.1.jar
      [jar] Building jar: C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\hadoop-eclipse-plugin-1.0.0.jar

BUILD SUCCESSFUL
Total time: 7 seconds