spark spark_deploy_mode配置详解

spark的配置里有个：spark_deploy_mode，可以是client 或cluster

–deploy-mode: Whether to deploy your driver on the worker nodes
(cluster) or locally as an external client (client) (default: client)

注意这里的client和cluster模式的区别在于driver的位置

A common deployment strategy is to submit your application from a
gateway machine that is physically co-located with your worker
machines (e.g. Master node in a standalone EC2 cluster). In this
setup, client mode is appropriate. In client mode, the driver is
launched directly within the spark-submit process which acts as a
client to the cluster. The input and output of the application is
attached to the console. Thus, this mode is especially suitable for
applications that involve the REPL (e.g. Spark shell).

In client mode, the driver is launched in the same process as the
client that submits the ap