JavaEsSpark.esJsonRDD函数读取ES数据报错

错误信息如下
Exception in thread "main" org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'  
    at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:196)  
    at org.elasticsearch.hadoop.rest.RestService.findPartitions(RestService.java:231)  
    at org.elasticsearch.spark.rdd.AbstractEsRDD.esPartitions$lzycompute(AbstractEsRDD.scala:61)  
    at org.elasticsearch.spark.rdd.AbstractEsRDD.esPartitions(AbstractEsRDD.scala:60)  
    at org.elasticsearch.spark.rdd.AbstractEsRDD.getPartitions(AbstractEsRDD.scala:27)  
    at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)  
    at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)  
    at scala.Option.getOrElse(Option.scala:120)  
    at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)  
    at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)  
    at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)  
    at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)  
    at scala.Option.getOrElse(Option.scala:120)  
    at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)  
    at org.apache.spark.rdd.RDD$$anonfun$take$1.apply(RDD.scala:1307)  
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)  
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)  
    at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)  
    at org.apache.spark.rdd.RDD.take(RDD.scala:1302)  
    at org.apache.spark.rdd.RDD$$anonfun$first$1.apply(RDD.scala:1342)  
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)  
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)  
    at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)  
    at org.apache.spark.rdd.RDD.first(RDD.scala:1341)  
    at com.anchiva.test.TestEs$.main(TestEs.scala:17)  
    at com.anchiva.test.TestEs.main(TestEs.scala)  
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)  
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
    at java.lang.reflect.Method.invoke(Method.java:601)  
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)  
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Unsupported/Unknown Elasticsearch version 5.0.0  
    at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:191)  
    ... 30 more  


查了一圈儿资料,原因是我使用的jar包版本有误,原来用的elasticsearch-spark_2.11,pom引用如下

<!-- https://mvnrepository.com/artifact/org.elasticsearch/elasticsearch-spark -->
<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch-spark_2.11</artifactId>
    <version>2.4.5</version>
</dependency>

改成elasticsearch-spark-20_2.11就可以了,pom引用如下

<!-- https://mvnrepository.com/artifact/org.elasticsearch/elasticsearch-spark-20 -->
<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch-spark-20_2.11</artifactId>
    <version>5.6.4</version>
</dependency>



  • 1
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 2
    评论
评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值