由dubbo引起的 java OutOfMemoryError unable to create new native thread

先说明结果:
由于linux默认限制1024个线程,运行7个tomcat,一个大概150个线程左右,外加其中有两个dubbo占用了200*2个线程,超过1024限制时会出此错。
解决方法:
1. 修改dubbo配置,降低线程数,50-100个足够。
2. 修改linux配置 /etc/security/limits.conf 把 soft nproc 1024 的1024修改大。

分析工具:jstack,dubbo源码
分析,dubbo为什么要占用这么多线程:
1. dubbo官方文档标称,默认使用fixedThreadPool 并且是 100 个。然而我却发现它的代码里面写死的是 200 。
2. 其次dubbo使用的是 netty,nio啊,为什么要用这么多线程。其实netty 只是用来通信的,nio 工作线程数是cpu线程数+1个,而上面的 200 是属于业务线程。
3. 下面我会用一段不明朗的语言描述这个过程,之后会有一个简单的处理过程图来描述。

语言:
原理是,NettyServer 类创建,创建了cpu线程数+1个工作线程,即new NioServerSocketChannelFactory()的参数workerCount。这是属于 netty 的 nio 处理范围。netty获取到请求之后,需要处理请求。dubbo 默认情况下使用 dubbo 协议,使用 AllChannelHandler 来处理所有请求,而个类的处理 ChannelEventRunnable extend Runnable 处理,且有域 executor,即通过线程池处理,而此线程池是使用的 ThreadPool 类扩展点,它的注解 @SPI(“fixed”) 决定了它是使用的 FixedThreadPool 类型的线程池,这个类里面的实现是直接使用的 Executors.newFixedThreadPool() 的实现。所以会有200个线程。
图:
dubbo线程模型简单代码说明

综上所述,dubbo 会占用较多的线程,原因是什么呢?
netty只用来接收请求,处理很快。后方需要处理业务,可能业务处理会比较慢,所以需要更多的线程来完成业务处理。

所以,根据自己的业务量来调整 dubbo 线程数量,或者修改linux配置,增加线程数。

发布了91 篇原创文章 · 获赞 72 · 访问量 68万+
展开阅读全文

调用dubbo服务,服务端zk内存溢出 求助

12-02

遇到了这样一个问题,定时任务调用dubbo做操作,调用完成后更新定时任务状态。 很突然的出现了一个问题,定时任务调用dubbo服务后就不向下执行了,但是查看服务端记录,调用的方法是正常执行完的。而且定时任务和服务端都没有报错…………求助,可能是什么问题? 分析日志后发现服务端 会报一个错误 14:54:09,510 [ERROR] [org.I0Itec.zkclient.ZkEventThread] :77 - Error handling event ZkEvent[New session event sent to com.alibaba.dubbo.remoting.zookeeper.zkclient.ZkclientZookeeperClient$1@7ac2e933] java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOfRange(Arrays.java:3664) at java.lang.String.<init>(String.java:207) at java.lang.StringBuilder.toString(StringBuilder.java:407) at java.util.AbstractCollection.toString(AbstractCollection.java:464) at java.lang.String.valueOf(String.java:2994) at java.lang.StringBuilder.append(StringBuilder.java:131) at com.alibaba.dubbo.registry.support.FailbackRegistry.recover(FailbackRegistry.java:282) at com.alibaba.dubbo.registry.zookeeper.ZookeeperRegistry.access$000(ZookeeperRegistry.java:45) at com.alibaba.dubbo.registry.zookeeper.ZookeeperRegistry$1.stateChanged(ZookeeperRegistry.java:76) at com.alibaba.dubbo.remoting.zookeeper.support.AbstractZookeeperClient.stateChanged(AbstractZookeeperClient.java:86) at com.alibaba.dubbo.remoting.zookeeper.zkclient.ZkclientZookeeperClient.access$300(ZkclientZookeeperClient.java:17) at com.alibaba.dubbo.remoting.zookeeper.zkclient.ZkclientZookeeperClient$1.handleNewSession(ZkclientZookeeperClient.java:36) at org.I0Itec.zkclient.ZkClient$4.run(ZkClient.java:472) at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71) 求助是什么原因 问答

dubbo 导致tomcat内存溢出

11-30

SEVERE: The web application [/superapi] appears to have started a thread named [DubboRegistryFailedRetryTimer-thread-1] but has failed to stop it. This is very likely to create a memory leak. Nov 29, 2017 7:10:48 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads SEVERE: The web application [/superapi] appears to have started a thread named [localhost-startStop-1-SendThread(hfftautzk02:2181)] but has failed to stop it. This is very likely to create a memory leak. Nov 29, 2017 7:10:48 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads SEVERE: The web application [/superapi] appears to have started a thread named [DubboSaveRegistryCache-thread-1] but has failed to stop it. This is very likely to create a memory leak. Nov 29, 2017 7:10:48 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads SEVERE: The web application [/superapi] appears to have started a thread named [DubboClientReconnectTimer-thread-1] but has failed to stop it. This is very likely to create a memory leak. Nov 29, 2017 7:10:48 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads SEVERE: The web application [/superapi] appears to have started a thread named [NettyClientBoss-thread-1] but has failed to stop it. This is very likely to create a memory leak. Nov 29, 2017 7:10:48 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads SEVERE: The web application [/superapi] appears to have started a thread named [New I/O client worker #1-1] but has failed to stop it. This is very likely to create a memory leak. Nov 29, 2017 7:10:48 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads SEVERE: The web application [/superapi] appears to have started a thread named [dubbo-remoting-client-heartbeat-thread-1] but has failed to stop it. This is very likely to create a memory leak. Nov 29, 2017 7:10:48 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads SEVERE: The web application [/superapi] appears to have started a thread named [DubboClientReconnectTimer-thread-2] but has failed to stop it. This is very likely to create a memory leak. Nov 29, 2017 7:10:48 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads SEVERE: The web application [/superapi] appears to have started a thread named [New I/O client worker #1-2] but has failed to stop it. This is very likely to create a memory leak. Nov 29, 2017 7:10:48 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads SEVERE: The web application [/superapi] appears to have started a thread named [dubbo-remoting-client-heartbeat-thread-2] but has failed to stop it. This is very likely to create a memory leak. Nov 29, 2017 7:10:48 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads SEVERE: The web application [/superapi] appears to have started a thread named [DubboResponseTimeoutScanTimer] but has failed to stop it. This is very likely to create a memory leak. Nov 29, 2017 7:10:48 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads SEVERE: The web application [/superapi] appears to have started a thread named [Thread-4] but has failed to stop it. This is very likely to create a memory leak. Nov 29, 2017 7:10:48 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads SEVERE: The web application [/superapi] appears to have started a thread named [Thread-5] but has failed to stop it. This is very likely to create a memory leak. Nov 29, 2017 7:10:48 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads 问答

没有更多推荐了,返回首页

©️2019 CSDN 皮肤主题: 技术黑板 设计师: CSDN官方博客

分享到微信朋友圈

×

扫一扫,手机浏览