线上的一个spring boot项目每两个周会出现系统卡死,不能正常提供api服务,重启后恢复。经过查看日志发现大量的“java.lang.OutOfMemoryError: GC overhead limit exceeded”日志。这个异常的官方解释:
Exception in thread thread_name: java.lang.OutOfMemoryError: GC Overhead limit exceeded
Cause: The detail message “GC overhead limit exceeded” indicates that the garbage collector is running all the time and Java program is making very slow progress. After a garbage collection, if the Java process is spending more than approximately 98% of its time doing garbage collection and if it is recovering less than 2% of the heap and has been doing so far the last 5 (compile time constant) consecutive garbage collections, then a java.lang.OutOfMemoryError is thrown. This exception is typically thrown because the amount of live data barely fits into the Java heap having little free space for new allocations.
Action: Increase the heap size. The java.lang.OutOfMemoryError exception for GC Overhead limit exceeded can be turned off with the command line flag -XX:-UseGCOverheadLimit.
JVM用了98%的时间进行垃圾回收,而只得到2%可用的内存,频繁的进行内存回收。
结合现象,可以推测程序中某些实例的数量在缓慢的增长,但是一直不能被回收。虽然异常信息不是常见的“java.lang.OutOfMemoryError: Java heap space”,但是原因却是相同的。
那我就开始查找原因吧!先说一下笔者的思路:
1、从生成获取dump文件
2、使用jvisualvm.exe或Eclipse Memory Anal