pycharm 运行pyspark报错：py4j.protocol.Py4JJavaError

在pycharm 上运行spark程序报错如下：

File "pyex.py", line 5, in <module>
    sc = SparkContext()   File "/usr/local/lib/python3.4/site-packages/pyspark/context.py", line 118, in __init__
    conf, jsc, profiler_cls)   File "/usr/local/lib/python3.4/site-packages/pyspark/context.py", line 195, in _do_init
    self._encryption_enabled = self._jvm.PythonUtils.getEncryptionEnabled(self._jsc)   File "/usr/local/lib/python3.4/site-packages/py4j/java_gateway.py", line 1487, in __getattr__
    "{0}.{1} does not exist in the JVM".format(self._fqn, name)) py4j.protocol.Py4JError: org.apache.spark.api.python.PythonUtils.getEncryptionEnabled does not exist in the JVM

 

解决方法：配置spark环境变量 并添加 pyspark和py4j

1.配置spark环境

PYTHONUNBUFFERED=1;

SPARK_HOME=/home/spark-2.4.6;

SPARK_PYTHON=/home/spark-2.4.6/pyhton

 

2.添加 pyspark和py4j

重新运行spark 程序，执行成功，这回可以在pycharm上进行开发啦 。