使用jstack打印后,发现
1)写入disruptor线程卡死在
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338)
at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:137)
at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105)
at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:263)
2)其他正常的disruptor消费线程都卡在
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000601b5d890> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:45)
at com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56)
at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
at java.lang.Thread.run(Thread.java:745)
但是有问题的线程没有了。
原因分析:
参考:
http://gitlab.htzq.htsc.com.cn/liangjian/manager/-/merge_requests/726
1)disruptor并不是线程池,而是启动时就分配了线程。disruptor的消费逻辑中抛异常会导致线程。
2)日志打印不是走的log4j,走的是jdk原生的日志框架jul,会打日志到console log里面。本次问题发现是消费端抛了kafka超时异常。
解决方法:
增加异常处理逻辑或者disruptor添加exceptionHandler配置,比如ignore