"Exception: Python in worker has different version xx than that in driver xxx

即问题原因是pyspark的python环境与driver也就是主节点的python环境版本不一致

---

spark-submit提交程序，如若还是这个问题

在python代码中添加

import os
os.environ['PYSPARK_PYTHON']='/usr/local/bin/pythonXXX' 

即可运行

---

也可以在/etc/spark/conf/spark-env.sh中添加如下代码，就无需在python代码中添加环境变量了

export PYSPARK_PYTHON=/usr/bin/python
export PYSPARK_DRIVER_PYTHON=/usr/bin/python