linux基于源码安装eclipse,linux下基于eclipse的hadoop开发

太水了，要不是不会java和eclipse也不会浪费我这么多时间了，做个笔记。

1.安装eclipse。yum install eclipse,版本3.3.2。

2.下载MapReduce Tools for Eclipse插件。http://www.alphaworks.ibm.com/tech/mapreducetools。

3.查找eclipse安装路径，看插件放在哪里。打开eclipse,Help->About Fedora Eclipse->Configuration Details,，，安装路径：/usr/share/eclipse，下面有个plugins，插件放这里。

4.重启eclipse。

5.设置hadoop路径。Windows->Preferences->Hadoop Home Directory,填写hadoop路径。

6.建MapReduce项目。File->New->Project,选择MapReduce Project,,,,填写项目名，如wordcount

7.写代码。可以new file，也可以右键点击wordcount->import->general->FileSystem,,,导入文件到workspace/wordcount/src，，，主要是写Map，Reduce类，main函数。

WordCount：http://hadoop.apache.org/common/docs/r0.20.1/cn/mapred_tutorial，(WordCount v1.0)。

8.设置运行时参数。Run->Open Run Dialog,选中要运行的项目，->Arguments。

这里的路径是相对于workspace的，不是相对于hadoop安装路径的。shit。run。。。当然hadoop要start。

WordCount：

1. package org.myorg;

2.

3. import java.io.IOException;

4. import java.util.*;

5.

6. import org.apache.hadoop.fs.Path;

7. import org.apache.hadoop.conf.*;

8. import org.apache.hadoop.io.*;

9. import org.apache.hadoop.mapred.*;

10. import org.apache.hadoop.util.*;

11.

12. public class WordCount {

13.

14. public static class Map extends MapReduceBase implements Mapper {

15. private final static IntWritable one = new IntWritable(1);

16. private Text word = new Text();

17.

18. public void map(LongWritable key, Text value, OutputCollector output, Reporter reporter) throws IOException {

19. String line = value.toString();

20. StringTokenizer tokenizer = new StringTokenizer(line);

21. while (tokenizer.hasMoreTokens()) {

22. word.set(tokenizer.nextToken());

23. output.collect(word, one);

24. }

25. }

26. }

27.

28. public static class Reduce extends MapReduceBase implements Reducer {

29. public void reduce(Text key, Iterator values, OutputCollector output, Reporter reporter) throws IOException {

30. int sum = 0;

31. while (values.hasNext()) {

32. sum += values.next().get();

33. }

34. output.collect(key, new IntWritable(sum));

35. }

36. }

37.

38. public static void main(String[] args) throws Exception {

39. JobConf conf = new JobConf(WordCount.class);

40. conf.setJobName("wordcount");

41.

42. conf.setOutputKeyClass(Text.class);

43. conf.setOutputValueClass(IntWritable.class);

44.

45. conf.setMapperClass(Map.class);

46. conf.setCombinerClass(Reduce.class);

47. conf.setReducerClass(Reduce.class);

48.

49. conf.setInputFormat(TextInputFormat.class);

50. conf.setOutputFormat(TextOutputFormat.class);

51.

52. FileInputFormat.setInputPaths(conf, new Path(args[0]));

53. FileOutputFormat.setOutputPath(conf, new Path(args[1]));

54.

55. JobClient.runJob(conf);

56. }

57. }

参考：

http://www.ibm.com/developerworks/cn/opensource/os-cn-hadoop2/

...

public class WordCount extends Configured implements Tool {

代码清单1

代码清单2

代码清单3

}