![](https://img-blog.csdnimg.cn/20201014180756918.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
spark
rookie_bigdata
菜鸟一枚
展开
-
spark sql 查询表时报错:。。。 not a file
现象:spark sql 查询表时报错:。。。 not a file解决:如果是在终端启动 spark shell,那么就执行 spark-sql --conf spark.hadoop.mapreduce.input.fileinputformat.input.dir.recursive=true --conf spark.hive.mapred.supports.subdirectories=true 如果是代码,那么就设置下边两个参数 .set("spark.hadoop.m.原创 2020-07-02 09:55:45 · 1189 阅读 · 1 评论 -
spark sql dataframe 写 hdfs 文本 报错:text datasource support only a single colum, and you have 2 columns
现象:spark sql 程序中resultFrame.coalesce(1).write.text(outputPath) 报错text datasource support only a single colum, and you have 2 columns解决:将结果连接为一列,使用concat_ws如select concat_ws('_', name, age) from students;...原创 2020-07-02 09:50:15 · 848 阅读 · 0 评论 -
spark sql 报错:attribute name “count(1)“ contains invalid charater(s) among
现象:spark sql 运行时报错attribute name "count(1)" contains invalid charater(s) among ,please use alias to rename it解决:将 count(1) 起个别名,如 select a , count(1) num from ...原创 2020-07-02 09:44:11 · 1401 阅读 · 0 评论