Hadoop-2.9.2运行WordCount.jar实现单词计数功能

最新推荐文章于 2022-10-10 09:27:40 发布

原创

最新推荐文章于 2022-10-10 09:27:40 发布 · 663 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#Hadoop

本文档详细介绍了如何在Hadoop-2.9.2环境下，通过编写并运行WordCount.java程序来实现单词计数功能。首先，需要编写WordCount.java源代码，接着进行编译。然后，配置好环境变量，进行编译和打包生成WordCount.jar。最后，通过指定jar包路径和HDFS的输入、输出路径，运行WordCount.jar，将在/output/part-r-00000文件中得到统计结果。

编写WordCount.java文件

//Java Code
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

import java.io.IOException;
import java.util.StringTokenizer;

public class WordCount {
   
   
    public static class TokenizerMapper extends Mapper<Object, Text, Text, IntWritable> {
   
   
        private final static IntWritable one = new IntWritable(1);
        private Text word =<