Spark读取es数据的array类型

最新推荐文章于 2024-05-16 22:05:01 发布

西瓜投手二号

最新推荐文章于 2024-05-16 22:05:01 发布

阅读量2k

点赞数 2

文章标签： elasticsearch spark

本文链接：https://blog.csdn.net/weixin_59111546/article/details/121763693

版权

最近西瓜用Spark读取ES上的数据时，发现报了一个错误：

21/12/07 10:44:35 WARN ScalaRowValueReader: Field 'vod_cat_tags' is backed by an array but the associated Spark Schema does not reflect this;
              (use es.read.field.as.array.include/exclude) 
21/12/07 10:44:35 WARN ScalaRowValueReader: Field 'vod_cat_tags' is backed by an array but the associated Spark Schema does not reflect this;
              (use es.read.field.as.array.include/exclude) 
21/12/07 10:44:35 WARN ScalaRowValueReader: Field 'vod_cat_tags' is backed by an array but the associated Spark Schema does not reflect this;
              (use es.read.field.as.array.include/exclude) 
21/12/07 10:44:35 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
org.elasticsearch.hadoop.rest.EsHadoopParsingException: org.elasticsearch.hadoop.EsHadoopIllegalStateException: Field 'vod_cat_tags.level5_name' not found; typically this occurs with arrays which are not mapped as single v