错误1:
ERROR StreamingContext: Error starting the context, marking it as stopped requirement failed: No output operations registered, so nothing to execute
意思是:StreamingContext错误:启动上下文时出错,将其标记为已停止的需求失败:没有注册输出操作,因此没有要执行的内容
报错原因是:检查点的时间至少是10s以上,一般设置时间是batch的5-10倍。一个batch设置5s,检查点应该设置它的5-10倍,25s或者50s.
package day09
import org.apache.log4j.{Level, Logger}
import org.apache.spark.{SparkConf, SparkContext}
import org.apache.spark.streaming.{Seconds, StreamingContext}
object UpdataState01 {
def main(args: Array[String]): Unit = {
//控制日志输出
Logger.getLogger("org").setLevel(Level.ERROR)
val conf: SparkConf = new SparkConf().setAppName(this.getClass.getSimpleName).setMaster("local[2]")
val sc = new SparkContext(conf)
val ssc = new StreamingContext(sc, Seconds(5))
ssc.checkpoint