python计算结果传给spark_将数组传递给Python Spark Lit函数

数组内置函数中的for循环

您可以使用array内置函数作为a = [1,2,3,4,5,6,7,8,9,10]

df = spark.createDataFrame([['a b c d e f g h i j '],], ['col1'])

df = df.withColumn("NewColumn", F.array([F.lit(x) for x in a]))

df.show(truncate=False)

你应该得到+--------------------+-------------------------------+

|col1 |NewColumn |

+--------------------+-------------------------------+

|a b c d e f g h i j |[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]|

+--------------------+-------------------------------+

root

|-- col1: string (nullable = true)

|-- NewColumn: array (nullable = false)

| |-- element: integer (containsNull = false)

使用udf函数#udf function

def arrayUdf():

return a

callArrayUdf = F.udf(arrayUdf, T.ArrayType(T.IntegerType()))

#calling udf function

df = df.withColumn("NewColumn", callArrayUdf())

输出与for循环方式相同

已更新

我在粘贴下面@pault的评论You can hide the loop using map: df.withColumn("NewColumn", F.array(map(F.lit, a)))