第13课:Spark Streaming源码解读之Driver容错安全性
/* 王家林老师授课http://weibo.com/ilovepains 每天晚上20:00YY频道现场授课频道68917580*/
1、数据层面:ReceivedBlockTracker 是专门负责管理Spark Streaming运行的元数据。跟踪数据需维持状态。
2、调度层面:DStream和JobGenerator 框架调度的层面,从业务角度考虑的。
JobGenerator(作业生成,进展到什么地步了)。
ReceivedBlockTracker 注释写的好精彩!藏龙卧虎之地
/**
* Class that keep track of all the received blocks, and allocate them to batches
* when required. All actions taken by this class can be saved to a write ahead log
* (if a checkpoint directory has been provided), so that the state of the tracker
* (received blocks