场景:项目重构时由于数据表结构的更换 需要把原始数据抽取到新的表中 ,使用sparksql 抽取 mysql 中的数据
遇到的问题:
org.apache.spark.sql.AnalysisException: expression 'score' is neither present in the group by, nor is it an aggregate function. Add to group by or wrap in first() (or first_value) if you don't care which value you get.;
报错代码:
更改后的代码:
问题的原因:
In Spark SQL, We have to wrap it in a function either first(column_name) or last(column_name) or any aggregate function when column_name is not present in group by clause. It will take either first or last value from the fetched rows respectively