pyspark环境配置报错解决
异常描述
环境:win10, spark3.1.2版本,hadoop3.3.1,java1.8
在pycharm或直接在pyspark shell环境中执行如下测试代码报错:
pyspark3.1: Python worker failed to connect back
from pyspark.sql import SparkSession
from pyspark.sql.types import StructType, StructField, LongType, StringType, IntegerType
if __name__ == "__main__":
spark = SparkSession.builder.master('local[1]').getOrCreate()
spark_rdd = spark.sparkContext.parallelize([