Spark On K8S

部署流程：

1. 下载spark环境包https://www.apache.org/dyn/closer.lua/spark/spark-2.4.4/spark-2.4.4-bin-hadoop2.7.tgz
2. 切换目录至根目录：cd spark-2.4.4-bin-hadoop2.7
3. builddocker应用程序镜像及push至repo：docker build -t xxx/spark:2.4.4 -f kubernetes/dockerfiles/spark/Dockerfile . && docker push xxx/spark:2.4.4
4. 创建aks serviceaccount: kubectl create serviceaccount spark
5. 创建clusterrolebinding:kubectl create clusterrolebinding spark-role --clusterrole=edit --serviceaccount=default:spark --namespace=default
6. 本地提交应用程序：

bin/spark-submit \
    --master k8s://172.22.3.107:443 \
    --deploy-mode cluster \
    --conf spark.kubernetes.namespace=default \ 
    --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark    
    --name spark-pi \
    --class org.apache.spark.examples.SparkPi \
    --conf spark.executor.instances=5 \
    --conf spark.kubernetes.container.image=xxx/spark:2.4