org.elasticsearch.common.io.stream.NotSerializableExceptionWrapper: too_many_clauses: maxClauseCount

异常:

Caused by: org.elasticsearch.common.io.stream.NotSerializableExceptionWrapper: too_many_clauses: maxClauseCount is set to 1024


用了es的in查询,in中id大于1024个,导致es报错,es默认支持元素数量为1024个。

解决办法:

编辑elasticsearch.yml,添加如下配置:

index.query.bool.max_clause_count: 10240

注意:必须在最前面添加一个空格,即和其他配置首字母对齐,不然es启动报错。

已标记关键词 清除标记
哪位知道这个怎么回事啊? 使用flume-ng 使用elasticsearch 作为sink的时候报错。 20 三月 2014 22:28:09,417 INFO [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.loadChannels:150) - Creating channels 20 三月 2014 22:28:09,438 INFO [conf-file-poller-0] (org.apache.flume.channel.DefaultChannelFactory.create:40) - Creating instance of channel c1 type memory 20 三月 2014 22:28:09,451 INFO [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.loadChannels:205) - Created channel c1 20 三月 2014 22:28:09,453 INFO [conf-file-poller-0] (org.apache.flume.source.DefaultSourceFactory.create:39) - Creating instance of source r1, type spooldir 20 三月 2014 22:28:09,478 INFO [conf-file-poller-0] (org.apache.flume.sink.DefaultSinkFactory.create:40) - Creating instance of sink: k1, type: elasticsearch 20 三月 2014 22:28:09,486 ERROR [conf-file-poller-0] (org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run:145) - Failed to start agent because dependencies were not found in classpath. Error follows. java.lang.NoClassDefFoundError: org/elasticsearch/common/transport/TransportAddress at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:190) at org.apache.flume.sink.DefaultSinkFactory.getClass(DefaultSinkFactory.java:67) at org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:41) at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:415) at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:103) at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Caused by: java.lang.ClassNotFoundException: org.elasticsearch.common.transport.TransportAddress at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
做报表统计的时候,需要调用一个第三方接口根据IP或许该IP的实际地址。 我先写了一个接口,从数据库中取出每条的ip 再调用第三方接口进行查询,把查到的地址信息存入数据库。 测试的时候出现429 too many requests问题,于是按照同事的建议做了些优化,并添加了线程池,最后测试的时候,没有出现因为 429 too many requests 问题而直接报500的错,但是 更新数据库中的数据1500条数据,一共花了一个小时,控制台打印显示大概是,每更新二三十条数据(调用二三十次第三方接口)它就会因为 频繁请求等问题而停顿,尝试再次发起请求。 <br/><br/> 我主要想问的就是: <br/><br/> **如何利用线程池解决调用第三接口出现 429 too many requests的问题** <br/><br/> 或者 <br/><br/> **不使用线程池的话有没有其他方法可以解决这个问题** <br/><br/> 第三方接口: http://ip-api.com/json/ip地址?lang=zh-CN <br/><br/> 下面是线程池相关部分的代码 ``` RejectedExecutionHandler handler = new RejectedExecutionHandler() { @Override public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) { if(!executor.isShutdown()){ try{ executor.getQueue().put(r); }catch (InterruptedException e){ e.printStackTrace(); } } } }; //线程池 ThreadPoolExecutor pool = new ThreadPoolExecutor(10, 50, 30*1000,TimeUnit.MILLISECONDS, new LinkedBlockingDeque<Runnable>(1000),handler); for (ModuleQueryLogEntity moduleQueryLogEntity : moduleQueryLogDtos){ pool.execute(() -> { try { ModuleQueryUrlResponse response = new ModuleQueryUrlResponse(); //传入ip地址调用第三方接口查询 response = getUrlResponse(moduleQueryLogEntity.getIp(), ModuleQueryUrlResponse.class); //如果查询成功,把查到的信息存入数据库 if(response.getStatus().equals(SUCCESS)){ moduleQueryLogEntity.setCity(response.getCity()); moduleQueryLogEntity.setCountry(response.getCountry()); moduleQueryLogRepository.save(moduleQueryLogEntity); } //沉睡 Thread.sleep(100); } catch (InterruptedException e) { e.printStackTrace(); } }); } //下面是调用第三方接口的代码 public <T> T getUrlResponse(String ip,Class<T> responseType){ HttpHeaders headers = new HttpHeaders(); headers.add("Retry-After","100"); StringBuilder url = new StringBuilder(FINE_IP); url.append(ip); url.append("?lang=zh-CN"); log.info(url.toString()); ResponseEntity<String> response = restTemplate.exchange( url.toString(), HttpMethod.GET, new HttpEntity<String>(headers), String.class); return JSON.parseObject(response.getBody(), responseType); } ```
©️2020 CSDN 皮肤主题: 大白 设计师:CSDN官方博客 返回首页