问题描述:
spark 运行遇到如下问题
AttributeError: 'PipelinedRDD' object has no attribute 'toDF'
解决方案:
参考了如下
https://stackoverflow.com/questions/32788387/pipelinedrdd-object-has-no-attribute-todf-in-pyspark
加入如下代码
from pyspark.sql import SparkSession
spark = SparkSession(sc)