解决pySpark导入模型报错：AttributeError: Can‘t get attribute ‘XXX‘ on ＜module ‘pyspark.worker‘ from...＞

hhhhh_s

已于 2023-12-08 09:28:26 修改

阅读量899

点赞数 9

文章标签： spark python

于 2023-12-08 08:44:37 首次发布

本文链接：https://blog.csdn.net/niong/article/details/134868519

版权

需求

自定义一个类

读取一个自定义类的对象的集合，并转换为DataFrame进行操作

错误

from SparkUtils import SparkUtils

class MemberOrderInfo:
	__slots__ = ["area", "memberType", "product", "price"]

	def __init__(self, area, memberType, product, price):
		self.area = area
		self.memberType = memberType
		self.product = product
		self.price = price

if __name__ == '__main__':
	spark = SparkUtils.getSparkSession("测试多维分析", "local")

	sc = spark.sparkContext
	data = [MemberOrderInfo("深圳", "钻石会员", "钻石会员1个月", 25),
			MemberOrderInfo("深圳", "钻石会员", "钻石会员1个月", 25),
			MemberOrderInfo("深圳", "钻石会员", "钻石会员3个月", 70),
			MemberOrderInfo("深圳", "钻石会员", "钻石会员12个月", 300),
			MemberOrderInfo("深圳", "铂金会员", "铂金会员3个月", 60),