Spark sql实现两列结合形成一个Json结构
原始数据
+----+-------+-----+
| id | name | age |
+----+-------+-----+
| 1 | John | 25 |
| 2 | Alice | 30 |
+----+-------+-----+
结果数据
+----+-------------------------+
| id | json_str |
+----+-------------------------+
| 1 | {"John":"25"} |
| 2 | {"Alice":"30"} |
+----+-------------------------+
SQL代码演示
SELECT
id,
to_json(map_from_entries(transform(collect_list(named_struct('name', name, 'age', age)), x -> (x['name'], x['age'])))) AS json_str
FROM
tb_name
GROUP BY
id
DataFrame版本代码
df.groupby("id") \
.agg(map_from_entries(F.collect_list(struct(F.col("name"),F.col("age"))))