spark读取elasticsearch中数组类型的字段

 

之前做的一个项目需要用sparksql读取elasticsearch的数据，当读取的类型中包含数组时报错.

读取方式大概是

val options = Map("pushdown" -> "true",
  "strict" -> "false",
  "es.nodes" -> "127.0.0.1",
  "es.port" -> "9200")
val df = spark.read.format("es").options(options).load("spark/scorearray")

报错信息如下：

WARN ScalaRowValueReader: Field 'array' is backed by an array but the associated Spark Schema does not reflect this;
              (use es.read.field.as.array.include/exclude) 
ERROR Executor: Exception in task 2.0 in stage 1.0 (TID 3)
java.lang.ClassCastException: scala.collection.convert.Wrappers$JListWrapper cannot be cast to java.lang.Long
    at scala.runtime.BoxesRunTime.unboxToLong(BoxesRunTime.java:105)
    at org.apache.spark.sql.catalyst.expressions.BaseGener