Spark Streaming
leedsjung
这个作者很懒,什么都没留下…
展开
-
SparkStreaming 核心概念与编程
核心概念:1)StreamingContext2)DStream(batched RDDs)3)Input DStream 、Receiver4)Transformation、Output Operation(RDD的转化、行动操作)StreamingContext常用构造方法:def this(sparkContext: SparkContext, batchDuration: Duration...原创 2018-03-19 13:44:28 · 228 阅读 · 0 评论 -
SparkStreaming 进阶与案例实战
updateStateByKey算子需求:统计到目前为止累积出现的单词的个数(需要保持住以前的状态)UpdateStateByKey OperationThe updateStateByKey operation allows you to maintain arbitrary state while continuously updating it with new information. T...原创 2018-03-19 13:44:52 · 260 阅读 · 0 评论