WordCount
道法—自然
不积跬步,无以至千里;不积小流,无以成江海。——荀子
展开
-
用Python统计单词的个数写wordcount
'''Created on 2018年9月22日@author: Administrator'''from pyspark.conf import SparkConffrom pyspark.context import SparkContextfrom pyspark.streaming.tests import resultfrom test.test_importlib.n...原创 2018-09-23 09:58:31 · 3089 阅读 · 0 评论 -
Spark写WordCount
package com.bjsxt.cn;import java.util.Arrays;import org.apache.spark.SparkConf;import org.apache.spark.api.java.JavaPairRDD;import org.apache.spark.api.java.JavaRDD;import org.apache.spark.api...原创 2018-09-23 09:53:58 · 324 阅读 · 0 评论 -
Spark部分:Java版Wordcount(包括flatmap切割,maptopair转换,reducebykey排序,foreach遍历输出)【Java版纯代码】
package com.bjsxt.scala;import org.apache.spark.SparkConf;import org.apache.spark.api.java.JavaPairRDD;import org.apache.spark.api.java.JavaRDD;import org.apache.spark.api.java.JavaSparkContext;...原创 2018-07-21 00:01:10 · 1248 阅读 · 0 评论 -
MapReduce版Wordcount的书写
主方法:package com.bjsxt.sn;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.Path;import org.apache.hadoop.io.IntWritable;import org.apache.hadoop.mapreduce.Job;import org...原创 2018-09-07 08:38:00 · 147 阅读 · 0 评论