现象:
发起的请求一直报超时,服务器却未见任何日志打印。
排查:
发起多次请求,服务器导出java堆栈信息
jstack -l #{pid} >jstack.txt
查看线程状态:
"http-nio-127.0.0.1-8093-exec-36" #102 daemon prio=5 os_prio=0 tid=0x00007f3a84823000 nid=0x5f5b waiting for monitor entry [0x00007f3a59788000]
java.lang.Thread.State: BLOCKED (on object monitor)
at sun.security.provider.NativePRNG$RandomIO.implNextBytes(NativePRNG.java:545)
- waiting to lock <0x00000000c0b2c488> (a java.lang.Object)
at sun.security.provider.NativePRNG$RandomIO.access$400(NativePRNG.java:331)
at sun.security.provider.NativePRNG$Blocking.engineNextBytes(NativePRNG.java:268)
at java.security.SecureRandom.nextBytes(SecureRandom.java:468)
.....................
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.logging.log4j.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:71)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
- locked <0x00000000f54207a0> (a org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-8093-exec-35" #101 daemon prio=5 os_prio=0 tid=0x00007f3a84821800 nid=0x5f54 waiting for monitor entry [0x00007f3a59889000]
....................
发现很多线程处于:java.lang.Thread.State: BLOCKED
线程状态
- NEW,未启动的。不会出现在Dump中。
- RUNNABLE,在虚拟机内执行的。运行中状态,可能里面还能看到locked字样,表明它获得了某把锁。
- BLOCKED,受阻塞并等待监视器锁。被某个锁(synchronizers)給block住了。
- WATING,无限期等待另一个线程执行特定操作。等待某个condition或monitor发生,一般停留在park(), wait(), sleep(),join() 等语句里。
- TIMED_WATING,有时限的等待另一个线程的特定操作。和WAITING的区别是wait() 等语句加上了时间限制 wait(timeout)。
- TERMINATED,已退出的
可从堆栈信息看出请求在执行某个filter时获取随机数,被某个上锁方法一直阻塞了;
此时再用关键词求助于google或度娘 “SecureRandom.nextBytes卡死”
解决方案可参照: