- 博客(10)
- 资源 (1)
- 收藏
- 关注
原创 spark自定义Accumulator高级应用(JAVA)
public class SessionAggrStatAccumulator implements AccumulatorParam { private static final long serialVersionUID = 6311074555136039130L; /** * zero方法,其实主要用于数据的初始化 * 那么,我们这里,就返回一个值,就是
2017-03-24 16:45:43 2707 3
原创 Spark编写UDAF自定义函数(JAVA)
maven: org.apache.spark spark-core_2.10 1.6.0 org.apache.spark spark-sql_2.10 1.6.0 org.apache.spark spark-hive_2.10 1.6.0工具类 --> com.google.guava
2017-03-21 14:51:02 3325
原创 Spark编写UDF自定义函数(JAVA)
maven: org.apache.spark spark-core_2.10 1.6.0 org.apache.spark spark-sql_2.10 1.6.0 org.apache.spark spark-hive_2.10 1.6.0工具类 --> com.google.guava
2017-03-21 13:25:47 8680
原创 sparksql分组后topN(JAVA)
org.apache.spark spark-core_2.10 1.6.0 org.apache.spark spark-sql_2.10 1.6.0 org.apache.spark spark-hive_2.10 1.6.0public class RowNumberTopN { public static
2017-03-21 10:56:17 1497
原创 sparkSQL用jdbc访问mysql
maven: org.apache.spark spark-core_2.10 1.6.0 org.apache.spark spark-sql_2.10 1.6.0工具类 --> com.google.guava guava 18.0public class JDBCDataSource {
2017-03-17 17:51:18 2986
原创 spark对分组后value值进行排序(JAVA)
maven: org.apache.spark spark-core_2.10 1.6.0groupsort.txt:spark 100storm 90kafka 75hadoop 60zookeeper 100impala 80hbase 65hive 90flume 95elasticsearch 100sp
2017-03-03 13:24:58 1778
原创 spark二次排序简单例子(JAVA)
maven依赖: org.apache.spark spark-core_2.10 1.6.0public class SecondarySortBean implements Ordered,Serializable { private int first; private int second; public SecondarySo
2017-03-02 16:55:07 464
原创 spark对wordcount结果集排序(JAVA)
maven依赖: org.apache.spark spark-core_2.10 1.6.0public class WordCountSort { public static void main(String[] args) { /** * 创建spark配置对象SparkConf,设置spark运行时配置信息,
2017-03-02 15:44:42 1387
原创 spark入门cogroup简单例子(JAVA)
maven依赖: org.apache.spark spark-core_2.10 1.6.0public class CoGroup { public static void main(String[] args) { /** * 创建spark配置对象SparkConf,设置spark运行时配置信息,
2017-03-02 13:45:49 721
原创 spark入门wordcount详解(JAVA)
maven依赖: org.apache.spark spark-core_2.10 1.6.0public class WordCount { public static void main(String[] args) { /** * 创建spark配置对象SparkConf,设置spark运行时配置信息,
2017-03-02 09:59:10 855
空空如也
TA创建的收藏夹 TA关注的收藏夹
TA关注的人