Linux下成功编译hadoop-eclipse-plugin-${version}.jar

最新推荐文章于 2020-11-08 13:53:24 发布

Flood_Dragon

最新推荐文章于 2020-11-08 13:53:24 发布

阅读量1.3k

点赞数

分类专栏： hadoop

hadoop 专栏收录该内容

76 篇文章 2 订阅

订阅专栏

最近在学习hadoop，首先第一步学着编译了下hadoop-eclipse插件，中间遇到了不少问题，但最终还是解决了。现将编译过程记录下来，便于后面使用。
      这里是基于hadoop-1.0.4和eclipse Indigo版本。
      这里以HADOOP_HOME来表示您电脑上hadoop的安装目录。

1、编辑${HADOOP_HOME}/src/contrib/下的build-contrib.xml文件
添加{version}和{eclipse.home}属性：

<project name="hadoopbuildcontrib" xmlns:ivy="antlib:org.apache.ivy.ant">     
  <!-- 这里定义了 version & eclipse.home -->  
   <property name="version" value="1.0.4"/>  
   <property name="eclipse.home" value="/home/chqz/systool/eclipse/eclipse"/>
  
   <property name="name" value="${ant.project.name}"/>  
   <property name="root" value="${basedir}"/>   
   <property name="hadoop.root" location="${root}/../../../"/>

2、编辑${HADOOP_HOME}/src/contrib/ eclipse-plugin/下的build.xml文件
(1) 添加hadoop-jars path，并同时加入到classpath中：

  <!-- 这里添加了 hadoop-jars -->
  <path id="hadoop-jars">
    <fileset dir="${hadoop.root}/">
       <include name="hadoop-*.jar"/>
    </fileset>
  </path>

  <!-- Override classpath to include Eclipse SDK jars -->
  <path id="classpath">
    <pathelement location="${build.classes}"/>
    <pathelement location="${hadoop.root}/build/classes"/>
    <path refid="eclipse-sdk-jars"/>
    <!-- 将 hadoop-jars 添加到这里 -->
    <path refid="hadoop-jars"/>
  </path>

(2) 设置includeantruntime=on，防止compile时报warning：

 <target name="compile" depends="init, ivy-retrieve-common" unless="skip.contrib">
    <echo message="contrib: ${name}"/>
    <javac
     encoding="${build.encoding}"
     srcdir="${src.dir}"
     includes="**/*.java"
     destdir="${build.classes}"
     debug="${javac.debug}"
     deprecation="${javac.deprecation}"
     <!-- 设置includeantruntime=on,防止compile报warning -->
     includeantruntime="on">
     <classpath refid="classpath"/>
    </javac>
  </target>

(3) 添加将要打包到plugin中的第三方包列表：

 <!-- Override jar target to specify manifest -->
  <target name="jar" depends="compile" unless="skip.contrib">
    <mkdir dir="${build.dir}/lib"/>
   
    <!-- 这里主要修改的是file中的值，注意路径一定要正确 -->
    <copy file="${hadoop.root}/hadoop-core-${version}.jar" 
          tofile="${build.dir}/lib/hadoop-core.jar" verbose="true"/>
    <copy file="${hadoop.root}/lib/commons-cli-1.2.jar" 
          todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.root}/lib/commons-lang-2.4.jar" 
          todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.root}/lib/commons-configuration-1.6.jar" 
          todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.root}/lib/jackson-mapper-asl-1.8.8.jar" 
          todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.root}/lib/jackson-core-asl-1.8.8.jar" 
          todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.root}/lib/commons-httpclient-3.0.1.jar" 
          todir="${build.dir}/lib" verbose="true"/>
    
   <jar
      jarfile="${build.dir}/hadoop-${name}-${version}.jar"
      manifest="${root}/META-INF/MANIFEST.MF">
      <fileset dir="${build.dir}" includes="classes/ lib/"/>
      <fileset dir="${root}" includes="resources/ plugin.xml"/>
    </jar>
  </target>

3、执行ant命令以生成hadoop-eclipse-plugin-${version}.jar包：
进入到 ${HADOOP_HOME}/src/contrib/ eclipse-plugin/目录下，
然后执行ant命令，输出结果如下：

[chqz@chqz eclipse-plugin]$ ant
Buildfile: /home/chqz/plugin/hadoop/hadoop-1.0.4/src/contrib/eclipse-plugin/build.xml

check-contrib:

init:
     [echo] contrib: eclipse-plugin

init-contrib:

ivy-download:
      [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
      [get] To: /home/chqz/plugin/hadoop/hadoop-1.0.4/ivy/ivy-2.1.0.jar
      [get] Not modified - so not downloaded

ivy-probe-antlib:

ivy-init-antlib:

ivy-init:
[ivy:configure] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /home/chqz/plugin/hadoop/hadoop-1.0.4/ivy/ivysettings.xml

ivy-resolve-common:
[ivy:resolve] :: resolving dependencies :: org.apache.hadoop#eclipse-plugin;working@chqz
[ivy:resolve]  confs: [common]
[ivy:resolve]  found commons-logging#commons-logging;1.0.4 in maven2
[ivy:resolve]  found log4j#log4j;1.2.15 in maven2
[ivy:resolve] :: resolution report :: resolve 152ms :: artifacts dl 4ms
 ---------------------------------------------------------------------
 |                  |            modules            ||   artifacts   |
 |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
 ---------------------------------------------------------------------
 |      common      |   2   |   0   |   0   |   0   ||   2   |   0   |
 ---------------------------------------------------------------------

ivy-retrieve-common:
[ivy:retrieve] :: retrieving :: org.apache.hadoop#eclipse-plugin [sync]
[ivy:retrieve]  confs: [common]
[ivy:retrieve]  0 artifacts copied, 2 already retrieved (0kB/8ms)
[ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead
[ivy:cachepath] :: loading settings :: file = /home/chqz/plugin/hadoop/hadoop-1.0.4/ivy/ivysettings.xml

compile:
     [echo] contrib: eclipse-plugin

jar:

BUILD SUCCESSFUL
Total time: 2 seconds

最后成功生成的 hadoop-eclipse-plugin-${version}.jar在${HADOOP_HOME}/build/contrib/eclipse-plugin下。

drwxrwxr-x. 3 chqz chqz 4096 1月 15 22:50 classes drwxrwxr-x. 2 chqz chqz 4096 1月 15 21:49 examples -rw-rw-r--. 1 chqz chqz 5625159 1月 15 23:03 hadoop-eclipse-plugin-1.0.4.jar drwxrwxr-x. 2 chqz chqz 4096 1月 15 23:03 lib drwxrwxr-x. 3 chqz chqz 4096 1月 15 21:49 system drwxrwxr-x. 3 chqz chqz 4096 1月 15 21:49 test