spark小笔记

spark running mode
local mode: one independent process with multiple threads running;
standalone（cluster)：each exists in independent process form and form cluster env;
hadoop yarn mode:running in yarn container;
kubernetes mode: running in kubernetes containner;
cloud mode: cloud platform.
yarn features
resource manage aspect
resource manager(the whole cluster) ;
node manager(single server)
task calculation aspect
applicationmaster#当前计算任务管家#
task#干活的工人
spark features
master :cluster manager
worker(driver#responsible for one signel task+executor)