mac pyspark运行报错

最新推荐文章于 2024-02-20 16:18:36 发布

l_ibrary

最新推荐文章于 2024-02-20 16:18:36 发布

阅读量430

点赞数

本文链接：https://blog.csdn.net/l_ibrary/article/details/106625636

版权

1.20/06/08 17:58:27 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0) org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "/usr/local/spark/python/lib/pyspark.zip/pyspark/worker.py", line 469, in main ("%d.%d" % sys.version_info[:2], version)) Exception: Python in worker has different version 2.7 than that in driver 3.7, PySpark cannot run with different minor versions.Please check environment variables PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON are correctly set.
原因 :运行的是python3 **.py文件
改为: python **.py文件就可以了

问题:
每次打开命令行都需要重新导入? 关于pyspark和py4j导入模块报错.参加如下链接
https://blog.csdn.net/l_ibrary/article/details/106616054

关于pyspark为何链接到了python3:
https://blog.csdn.net/l_ibrary/article/details/106614373
是因为之前设置了吗?