JAVA程序实现mapreduce的wordcount

最新推荐文章于 2024-10-14 22:15:56 发布

_ 吱吱唔

最新推荐文章于 2024-10-14 22:15:56 发布

阅读量448

点赞数

本文链接：https://blog.csdn.net/weixin_42495364/article/details/90551695

版权

本文介绍如何用JAVA编写MapReduce程序来实现经典的WordCount例子，通过分割文本，统计单词出现次数，展示MapReduce处理大数据的基本流程。

摘要由CSDN通过智能技术生成

JAVA程序实现mapreduce的wordcount

程序代码：

package hdfs.files;

import java.io.IOException;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;

public class WordCountDriver {
	
	public static class WordCountMapper extends Mapper<LongWritable, Text, Text, IntWritable>{
		   
		 protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {

		   String line = value.toString();
		 
		   String[] words=line.split(" ");
		  
		   for(String w:words) {
		  
		   context.write(new Text(w), ne