文章目录
spark作业核心概念:
Application:
1.a driver program =》 driver
2.executors on the cluster =》 executors
spark作业:
1.sparkcontext
2. web ui
Application jar:
spark作业生成的jar包 =》 spark作业 main方法 =》 部署服务器上
Driver program:
1.main 方法
2.创建sparkcontext
Cluster manager:
An external service for acquiring resources
Deploy mode:
spark jar 提交服务器去运行 =》 yarn
cluster:
driver inside of the cluster driver跑在yarn所在的机器里面
client
driver 运行在cluster之外
Worker node:
can run application code
补充:
spark作业 yarn
nodemanager node
Executor:
containe