四步来完成编译eclipse插件的过程

虽然使用Java编写MapReduce作业有点繁琐，但是对于刚入门hadoop的新手来说，有利于理解hadoop的一些底层实现，这无疑也是一种巨大的收获。目前大部分的MapReduce作业都是使用Hive，Pig完成的，当然也有用其他语言实现的，通过使用HadoopStreaming的方式，用原生Java写MR作业的只有很少一部分。在Hadoop2.x中，已经不支持eclipse的插件了，不过可以在配置的XML文件里，加入远程监控的端口，来监控hadoop集群。


由于1.0之后hadoop已经不自带eclipse的插件包了，所以得需要我们自己编译源码生成插件包，建议在Linux下编译，Windows下可能有时候会报一些莫名其妙的错误，先介绍下散仙的环境，Centos6.4的版本，hadoop1.2.0的版本，
hadoop的目录在/root/hadoop-1.2.0下面
eclipse的目录在/root/eclipse下面
总结一下如下的四步来完成编译eclipse插件的过程

 

 

 

步骤

操作

1

进入/root/hadoop-1.2.0/src/contrib/eclipse-plugin下面，修改build.xml

2

进入/root/hadoop-1.2.0/src/contrib下面，修改build-contrib.xml

3

/root/hadoop-1.2.0/src/contrib/eclipse-plugin/META-INF下面，修改MANIFEST.MF文件

4

使用shell命令进入/root/hadoop-1.2.0/src/contrib/eclipse-plugin下面，执行ant命令进行构建

下面散仙就来详细介绍下，每一步具体怎么操作，在第一步里我们需要向build.xml里面，加入的内容有
1,eclipse的根目录.
2.hadoop的版本号.
3.hadoop的一些引用包.
4，在javac里加入  includeantruntime="on".
内容如下所示：

Xml代码  

1.   <!-- 1,2步的配置  -->  
2.   <property name="eclipse.home" location="/root/eclipse"/>  
3.   <property name="version" value="1.2.0"/>  
4.   <!--第3步的配置  -->  
5.   <copy file="${hadoop.root}/hadoop-core-${version}.jar" tofile="${build.dir}/lib/hadoop-core.jar" verbose="true"/>  
6. <copy file="${hadoop.root}/lib/commons-cli-${commons-cli.version}.jar"  tofile="${build.dir}/lib/commons-cli.jar" verbose="true"/>  
7. <copy file="${hadoop.root}/lib/commons-configuration-1.6.jar"  tofile="${build.dir}/lib/commons-configuration.jar" verbose="true"/>  
8. <copy file="${hadoop.root}/lib/commons-httpclient-3.0.1.jar"  tofile="${build.dir}/lib/commons-httpclient.jar" verbose="true"/>  
9. <copy file="${hadoop.root}/lib/commons-lang-2.4.jar"  tofile="${build.dir}/lib/commons-lang.jar" verbose="true"/>  
10. <copy file="${hadoop.root}/lib/jackson-core-asl-1.8.8.jar"  tofile="${build.dir}/lib/jackson-core-asl.jar" verbose="true"/>  
11. <copy file="${hadoop.root}/lib/jackson-mapper-asl-1.8.8.jar"  tofile="${build.dir}/lib/jackson-mapper-asl.jar" verbose="true"/>  
12.  <!--第4步的配置  -->  
13.  includeantruntime="on">  

  <!-- 1,2步的配置  -->
  <property name="eclipse.home" location="/root/eclipse"/>
  <property name="version" value="1.2.0"/>
  <!--第3步的配置  -->
  <copy file="${hadoop.root}/hadoop-core-${version}.jar" tofile="${build.dir}/lib/hadoop-core.jar" verbose="true"/>
<copy file="${hadoop.root}/lib/commons-cli-${commons-cli.version}.jar"  tofile="${build.dir}/lib/commons-cli.jar" verbose="true"/>
<copy file="${hadoop.root}/lib/commons-configuration-1.6.jar"  tofile="${build.dir}/lib/commons-configuration.jar" verbose="true"/>
<copy file="${hadoop.root}/lib/commons-httpclient-3.0.1.jar"  tofile="${build.dir}/lib/commons-httpclient.jar" verbose="true"/>
<copy file="${hadoop.root}/lib/commons-lang-2.4.jar"  tofile="${build.dir}/lib/commons-lang.jar" verbose="true"/>
<copy file="${hadoop.root}/lib/jackson-core-asl-1.8.8.jar"  tofile="${build.dir}/lib/jackson-core-asl.jar" verbose="true"/>
<copy file="${hadoop.root}/lib/jackson-mapper-asl-1.8.8.jar"  tofile="${build.dir}/lib/jackson-mapper-asl.jar" verbose="true"/>
 <!--第4步的配置  -->
 includeantruntime="on">

散仙会在下文贴出整个XML，现在我们看第二大步的配置，修改build-contrib.xml，内容如下：

Xml代码  

1. <!--  原始的为off -->  
2. lt;property name="javac.deprecation" value="on"/>  

 <!--  原始的为off -->
<property name="javac.deprecation" value="on"/>

第三步的修改MANIFEST.MF内容如下：

Xml代码  

1. Bundle-ClassPath: classes/,lib/commons-cli.jar,lib/commons-httpclient.jar,lib/hadoop-core.jar,lib/jackson-mapper-asl.jar,lib/commons-configuration.jar,lib/commons-lang.jar,lib/jackson-core-asl.jar  

Bundle-ClassPath: classes/,lib/commons-cli.jar,lib/commons-httpclient.jar,lib/hadoop-core.jar,lib/jackson-mapper-asl.jar,lib/commons-configuration.jar,lib/commons-lang.jar,lib/jackson-core-asl.jar

第四步执行命令：截图如下：


最后，插件类生成的目录，截图如下：




下面散仙给出所有配置的全部信息，以供参考，下图的是build.xml里面的内容

Xml代码  

1. <?xml version="1.0" encoding="UTF-8" standalone="no"?>  
2.   
3. <!--  
4.    Licensed to the Apache Software Foundation (ASF) under one or more  
5.    contributor license agreements.  See the NOTICE file distributed with  
6.    this work for additional information regarding copyright ownership.  
7.    The ASF licenses this file to You under the Apache License, Version 2.0  
8.    (the "License"); you may not use this file except in compliance with  
9.    the License.  You may obtain a copy of the License at  
10.   
11.        http://www.apache.org/licenses/LICENSE-2.0  
12.   
13.    Unless required by applicable law or agreed to in writing, software  
14.    distributed under the License is distributed on an "AS IS" BASIS,  
15.    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  
16.    See the License for the specific language governing permissions and  
17.    limitations under the License.  
18. -->  
19.   
20.              <!-- build.xml -->  
21. <project default="jar" name="eclipse-plugin">  
22.   
23.   <import file="../build-contrib.xml"/>  
24.   
25.      
26.   <property name="eclipse.home" location="/root/eclipse"/>  
27.   <property name="version" value="1.2.0"/>  
28.     
29.   <path id="eclipse-sdk-jars">  
30.     <fileset dir="${eclipse.home}/plugins/">  
31.       <include name="org.eclipse.ui*.jar"/>  
32.       <include name="org.eclipse.jdt*.jar"/>  
33.       <include name="org.eclipse.core*.jar"/>  
34.       <include name="org.eclipse.equinox*.jar"/>  
35.       <include name="org.eclipse.debug*.jar"/>  
36.       <include name="org.eclipse.osgi*.jar"/>  
37.       <include name="org.eclipse.swt*.jar"/>  
38.       <include name="org.eclipse.jface*.jar"/>  
39.   
40.       <include name="org.eclipse.team.cvs.ssh2*.jar"/>  
41.       <include name="com.jcraft.jsch*.jar"/>  
42.     </fileset>   
43.   </path>  
44.   
45.   <!-- Override classpath to include Eclipse SDK jars -->  
46.   <path id="classpath">  
47.     <pathelement location="${build.classes}"/>  
48.     <pathelement location="${hadoop.root}/build/classes"/>  
49.      <fileset dir="${hadoop.root}">  
50.         <include name="**/*.jar" />  
51.     </fileset>  
52.     <path refid="eclipse-sdk-jars"/>  
53.   </path>  
54.   
55.   <!-- Skip building if eclipse.home is unset. -->  
56.   <target name="check-contrib" unless="eclipse.home">  
57.     <property name="skip.contrib" value="yes"/>  
58.     <echo message="eclipse.home unset: skipping eclipse plugin"/>  
59.   </target>  
60.   
61.  <target name="compile" depends="init, ivy-retrieve-common" unless="skip.contrib">  
62.     <echo message="contrib: ${name}"/>  
63.     <javac  
64.      encoding="${build.encoding}"  
65.      srcdir="${src.dir}"  
66.      includes="**/*.java"  
67.      destdir="${build.classes}"  
68.      debug="${javac.debug}"  
69.      deprecation="${javac.deprecation}"  
70.      includeantruntime="on">  
71.      <classpath refid="classpath"/>  
72.     </javac>  
73.   </target>  
74.   
75.   <!-- Override jar target to specify manifest -->  
76.   <target name="jar" depends="compile" unless="skip.contrib">  
77.     <mkdir dir="${build.dir}/lib"/>  
78.   <!--  <copy file="${hadoop.root}/build/hadoop-core-${version}.jar" tofile="${build.dir}/lib/hadoop-core.jar" verbose="true"/> -->  
79.    <!--  <copy file="${hadoop.root}/build/ivy/lib/Hadoop/common/commons-cli-${commons-cli.version}.jar"  todir="${build.dir}/lib" verbose="true"/>-->  
80.   <copy file="${hadoop.root}/hadoop-core-${version}.jar" tofile="${build.dir}/lib/hadoop-core.jar" verbose="true"/>  
81. <copy file="${hadoop.root}/lib/commons-cli-${commons-cli.version}.jar"  tofile="${build.dir}/lib/commons-cli.jar" verbose="true"/>  
82. <copy file="${hadoop.root}/lib/commons-configuration-1.6.jar"  tofile="${build.dir}/lib/commons-configuration.jar" verbose="true"/>  
83. <copy file="${hadoop.root}/lib/commons-httpclient-3.0.1.jar"  tofile="${build.dir}/lib/commons-httpclient.jar" verbose="true"/>  
84. <copy file="${hadoop.root}/lib/commons-lang-2.4.jar"  tofile="${build.dir}/lib/commons-lang.jar" verbose="true"/>  
85. <copy file="${hadoop.root}/lib/jackson-core-asl-1.8.8.jar"  tofile="${build.dir}/lib/jackson-core-asl.jar" verbose="true"/>  
86. <copy file="${hadoop.root}/lib/jackson-mapper-asl-1.8.8.jar"  tofile="${build.dir}/lib/jackson-mapper-asl.jar" verbose="true"/>  
87.   
88.   
89.   <jar  
90.       jarfile="${build.dir}/hadoop-${name}-${version}.jar"  
91.       manifest="${root}/META-INF/MANIFEST.MF">  
92.       <fileset dir="${build.dir}" includes="classes/ lib/"/>  
93.       <fileset dir="${root}" includes="resources/ plugin.xml"/>  
94.     </jar>  
95.   </target>  
96.   
97. </project>  

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->

             <!-- build.xml -->
<project default="jar" name="eclipse-plugin">

  <import file="../build-contrib.xml"/>

   
  <property name="eclipse.home" location="/root/eclipse"/>
  <property name="version" value="1.2.0"/>
  
  <path id="eclipse-sdk-jars">
    <fileset dir="${eclipse.home}/plugins/">
      <include name="org.eclipse.ui*.jar"/>
      <include name="org.eclipse.jdt*.jar"/>
      <include name="org.eclipse.core*.jar"/>
      <include name="org.eclipse.equinox*.jar"/>
      <include name="org.eclipse.debug*.jar"/>
      <include name="org.eclipse.osgi*.jar"/>
      <include name="org.eclipse.swt*.jar"/>
      <include name="org.eclipse.jface*.jar"/>

      <include name="org.eclipse.team.cvs.ssh2*.jar"/>
      <include name="com.jcraft.jsch*.jar"/>
    </fileset> 
  </path>

  <!-- Override classpath to include Eclipse SDK jars -->
  <path id="classpath">
    <pathelement location="${build.classes}"/>
    <pathelement location="${hadoop.root}/build/classes"/>
	 <fileset dir="${hadoop.root}">
        <include name="**/*.jar" />
    </fileset>
    <path refid="eclipse-sdk-jars"/>
  </path>

  <!-- Skip building if eclipse.home is unset. -->
  <target name="check-contrib" unless="eclipse.home">
    <property name="skip.contrib" value="yes"/>
    <echo message="eclipse.home unset: skipping eclipse plugin"/>
  </target>

 <target name="compile" depends="init, ivy-retrieve-common" unless="skip.contrib">
    <echo message="contrib: ${name}"/>
    <javac
     encoding="${build.encoding}"
     srcdir="${src.dir}"
     includes="**/*.java"
     destdir="${build.classes}"
     debug="${javac.debug}"
     deprecation="${javac.deprecation}"
	 includeantruntime="on">
     <classpath refid="classpath"/>
    </javac>
  </target>

  <!-- Override jar target to specify manifest -->
  <target name="jar" depends="compile" unless="skip.contrib">
    <mkdir dir="${build.dir}/lib"/>
  <!--  <copy file="${hadoop.root}/build/hadoop-core-${version}.jar" tofile="${build.dir}/lib/hadoop-core.jar" verbose="true"/> -->
   <!--  <copy file="${hadoop.root}/build/ivy/lib/Hadoop/common/commons-cli-${commons-cli.version}.jar"  todir="${build.dir}/lib" verbose="true"/>-->
  <copy file="${hadoop.root}/hadoop-core-${version}.jar" tofile="${build.dir}/lib/hadoop-core.jar" verbose="true"/>
<copy file="${hadoop.root}/lib/commons-cli-${commons-cli.version}.jar"  tofile="${build.dir}/lib/commons-cli.jar" verbose="true"/>
<copy file="${hadoop.root}/lib/commons-configuration-1.6.jar"  tofile="${build.dir}/lib/commons-configuration.jar" verbose="true"/>
<copy file="${hadoop.root}/lib/commons-httpclient-3.0.1.jar"  tofile="${build.dir}/lib/commons-httpclient.jar" verbose="true"/>
<copy file="${hadoop.root}/lib/commons-lang-2.4.jar"  tofile="${build.dir}/lib/commons-lang.jar" verbose="true"/>
<copy file="${hadoop.root}/lib/jackson-core-asl-1.8.8.jar"  tofile="${build.dir}/lib/jackson-core-asl.jar" verbose="true"/>
<copy file="${hadoop.root}/lib/jackson-mapper-asl-1.8.8.jar"  tofile="${build.dir}/lib/jackson-mapper-asl.jar" verbose="true"/>


  <jar
      jarfile="${build.dir}/hadoop-${name}-${version}.jar"
      manifest="${root}/META-INF/MANIFEST.MF">
      <fileset dir="${build.dir}" includes="classes/ lib/"/>
      <fileset dir="${root}" includes="resources/ plugin.xml"/>
    </jar>
  </target>

</project>

build-contrib.xml的核心配置如下：只有一个地方，notepad++，定位截图如下：

Xml代码  

1. <property name="javac.deprecation" value="on"/>  

  <property name="javac.deprecation" value="on"/>

下面是MANIFEST.MF里面的内容

Java代码  

1. Manifest-Version: 1.0  
2. Bundle-ManifestVersion: 2  
3. Bundle-Name: MapReduce Tools for Eclipse  
4. Bundle-SymbolicName: org.apache.hadoop.eclipse;singleton:=true  
5. Bundle-Version: 0.18  
6. Bundle-Activator: org.apache.hadoop.eclipse.Activator  
7. Bundle-Localization: plugin  
8. Require-Bundle: org.eclipse.ui,  
9.  org.eclipse.core.runtime,  
10.  org.eclipse.jdt.launching,  
11.  org.eclipse.debug.core,  
12.  org.eclipse.jdt,  
13.  org.eclipse.jdt.core,  
14.  org.eclipse.core.resources,  
15.  org.eclipse.ui.ide,  
16.  org.eclipse.jdt.ui,  
17.  org.eclipse.debug.ui,  
18.  org.eclipse.jdt.debug.ui,  
19.  org.eclipse.core.expressions,  
20.  org.eclipse.ui.cheatsheets,  
21.  org.eclipse.ui.console,  
22.  org.eclipse.ui.navigator,  
23.  org.eclipse.core.filesystem,  
24.  org.apache.commons.logging  
25. Eclipse-LazyStart: true  
26. Bundle-ClassPath: classes/,lib/commons-cli.jar,lib/commons-httpclient.jar,lib/hadoop-core.jar,lib/jackson-mapper-asl.jar,lib/commons-configuration.jar,lib/commons-lang.jar,lib/jackson-core-asl.jar  
27. Bundle-Vendor: Apache Hadoop  

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: MapReduce Tools for Eclipse
Bundle-SymbolicName: org.apache.hadoop.eclipse;singleton:=true
Bundle-Version: 0.18
Bundle-Activator: org.apache.hadoop.eclipse.Activator
Bundle-Localization: plugin
Require-Bundle: org.eclipse.ui,
 org.eclipse.core.runtime,
 org.eclipse.jdt.launching,
 org.eclipse.debug.core,
 org.eclipse.jdt,
 org.eclipse.jdt.core,
 org.eclipse.core.resources,
 org.eclipse.ui.ide,
 org.eclipse.jdt.ui,
 org.eclipse.debug.ui,
 org.eclipse.jdt.debug.ui,
 org.eclipse.core.expressions,
 org.eclipse.ui.cheatsheets,
 org.eclipse.ui.console,
 org.eclipse.ui.navigator,
 org.eclipse.core.filesystem,
 org.apache.commons.logging
Eclipse-LazyStart: true
Bundle-ClassPath: classes/,lib/commons-cli.jar,lib/commons-httpclient.jar,lib/hadoop-core.jar,lib/jackson-mapper-asl.jar,lib/commons-configuration.jar,lib/commons-lang.jar,lib/jackson-core-asl.jar
Bundle-Vendor: Apache Hadoop

至此，已经ant编译完毕，截图如下



启动eclipse后，截图如下：





至此，插件已经完美生成，并运行，因为受上传限制的影响，所以散仙在这里上传不了这个jar包，