WordCount
道法—自然
不积跬步,无以至千里;不积小流,无以成江海。——荀子
展开
-
用Python统计单词的个数写wordcount
''' Created on 2018年9月22日 @author: Administrator ''' from pyspark.conf import SparkConf from pyspark.context import SparkContext from pyspark.streaming.tests import result from test.test_importlib.n...原创 2018-09-23 09:58:31 · 3060 阅读 · 0 评论 -
Spark写WordCount
package com.bjsxt.cn; import java.util.Arrays; import org.apache.spark.SparkConf; import org.apache.spark.api.java.JavaPairRDD; import org.apache.spark.api.java.JavaRDD; import org.apache.spark.api...原创 2018-09-23 09:53:58 · 318 阅读 · 0 评论 -
Spark部分:Java版Wordcount(包括flatmap切割,maptopair转换,reducebykey排序,foreach遍历输出)【Java版纯代码】
package com.bjsxt.scala; import org.apache.spark.SparkConf; import org.apache.spark.api.java.JavaPairRDD; import org.apache.spark.api.java.JavaRDD; import org.apache.spark.api.java.JavaSparkContext;...原创 2018-07-21 00:01:10 · 1226 阅读 · 0 评论 -
MapReduce版Wordcount的书写
主方法: package com.bjsxt.sn; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.mapreduce.Job; import org...原创 2018-09-07 08:38:00 · 134 阅读 · 0 评论