Spark sql实现两列结合形成一个Json结构

Spark sql实现两列结合形成一个Json结构

原始数据

+----+-------+-----+
| id |  name | age |
+----+-------+-----+
|  1 |  John |  25 |
|  2 | Alice |  30 |
+----+-------+-----+

结果数据

+----+-------------------------+
| id |         json_str         |
+----+-------------------------+
|  1 | {"John":"25"}           |
|  2 | {"Alice":"30"}          |
+----+-------------------------+

SQL代码演示

SELECT
  id,
  to_json(map_from_entries(transform(collect_list(named_struct('name', name, 'age', age)), x -> (x['name'], x['age'])))) AS json_str
FROM
  tb_name
GROUP BY
  id

DataFrame版本代码

df.groupby("id") \
    .agg(map_from_entries(F.collect_list(struct(F.col("name"),F.col("age"))))