Spark-Core启动脚本流程|Master和Worker得启动脚本过程|找到Master和Worker的程序执行入口原因

进入启动命令

查看start-all.sh脚本信息

查看spark-config.sh加载信息

查看start-master.sh脚本信息

spark-daemon.sh

查看bin/spark-class

添加一行打印，查看信息

echo "${CMD[@]}" >> ./test.log

启动bin/master-all.sh 查看test.log

/opt/module/jdk1.8.0_144/bin/java -cp /opt/module/spark-standalone/conf/:/opt/module/spark-standalone/jars/* -Dspark.deploy.recoveryMode=ZOOKEEPER -Dspark.deploy.zookeeper.url=hadoop102,hadoop103,hadoop104 -Dspark.deploy.zookeeper.dir=/spark -Xmx1g org.apache.spark.deploy.master.Master --host hadoop102 --port 7077 --webui-port 8080
/opt/module/jdk1.8.0_144/bin/java -cp /opt/module/spark-standalone/conf/:/opt/module/spark-standalone/jars/* -Dspark.deploy.recoveryMode=ZOOKEEPER -Dspark.deploy.zookeeper.url=hadoop102,hadoop103,hadoop104 -Dspark.deploy.zookeeper.dir=/spark -Xmx1g org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://hadoop102:7077

查看进程

总结

1. 通过Java命令启动的进程有main方法
2. 那么我们在Scala中找到Master和Worker源码就可以根据main方法(程序的入口)进行源码跟踪查看执行流程

master

worker