A fatal error has been detected by the Java Runtime Environment

现象

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fcf660bf067, pid=795, tid=0x00007fcf09a65700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_92-b14) (build 1.8.0_92-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.92-b14 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# V  [libjvm.so+0x438067]  ciObjectFactory::create_new_metadata(Metadata*)+0x327
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# hs_err_pid795.log
#
# Compiler replay data is saved as:
# replay_pid795.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp

过程

查看日志

hs_err_pid795.log(只截取了重要信息)

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fcf660bf067, pid=795, tid=0x00007fcf09a65700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_92-b14) (build 1.8.0_92-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.92-b14 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# V  [libjvm.so+0x438067]  ciObjectFactory::create_new_metadata(Metadata*)+0x327
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x00007fcf60804800):  JavaThread "C2 CompilerThread2" daemon [_thread_in_vm, id=839, stack(0x00007fcf09965000,0x00007fcf09a66000)]

siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x0000000000000013

Registers:
RAX=0x000000000000000b, RBX=0x00000000000000bb, RCX=0x00000037800182a0, RDX=0x00007fcf09a65a20
RSP=0x00007fcf09a62090, RBP=0x00007fcf09a62100, RSI=0x00007fcf3152bce8, RDI=0x00007fcf3152bce8
R8 =0x0000000000000000, R9 =0x0000000000000000, R10=0x0000000000000000, R11=0x000000377f75a290
R12=0x00007fcf3152bce8, R13=0x00007fcef8e79940, R14=0x00007fcf60804800, R15=0x00007fcef8e79940
RIP=0x00007fcf660bf067, EFLAGS=0x0000000000010202, CSGSFS=0x0000000000000033, ERR=0x0000000000000004
  TRAPNO=0x000000000000000e

Top of Stack: (sp=0x00007fcf09a62090)
0x00007fcf09a62090:   00007fcf09a620d0 00007fcf660cd796
0x00007fcf09a620a0:   00007fcf09a62110 00007fcf09a62190
0x00007fcf09a620b0:   01007fcf09a620d0 00007fcf09a62190
0x00007fcf09a620c0:   00007fceec499b80 000000000000000a
0x00007fcf09a620d0:   00007fcf09a62110 00000000000000bb
0x00007fcf09a620e0:   00007fcf3152bce8 00007fcef8e79940
0x00007fcf09a620f0:   00000000000000c1 00007fcf18017120
0x00007fcf09a62100:   00007fcf09a62140 00007fcf660bf495
0x00007fcf09a62110:   00007fcf09a62210 0000000000000000
0x00007fcf09a62120:   00007fcf3152bce8 0000000000000000
0x00007fcf09a62130:   00007fcf66c05aa0 00007fceec499c20
0x00007fcf09a62140:   00007fcf09a62180 00007fcf660b7167
0x00007fcf09a62150:   0000000000000010 00007fceec499c20
0x00007fcf09a62160:   00007fcf1801a450 00007fcf09f3ffc8
0x00007fcf09a62170:   00007fcf09a62190 00007fcf09f3fe98
0x00007fcf09a62180:   00007fcf09a622c0 00007fcf660b7478
0x00007fcf09a62190:   00007fcf60208030 000000004e62974e
0x00007fcf09a621a0:   00007fcf09a621f0 00007fcf666b5265
0x00007fcf09a621b0:   00007fcf60804800 3c494703ec499b80
0x00007fcf09a621c0:   00007fcf09a62230 00007fcf09a62230
0x00007fcf09a621d0:   00007fcf60804c40 0000000000000000
0x00007fcf09a621e0:   00007fcf60804c40 0000000000000000
0x00007fcf09a621f0:   00007fcf09a62280 00007fcf66531832
0x00007fcf09a62200:   0000000000000000 00007fcf09a622a0
0x00007fcf09a62210:   00007fceec498c10 00007fceec499b80
0x00007fcf09a62220:   00007fceec4a0bf8 0000000000018360
0x00007fcf09a62230:   00007fcf313861c0 00007fcf66560b52
0x00007fcf09a62240:   0000000000000010 00007fcf1801a2b0
0x00007fcf09a62250:   00007fcf09a62270 0000000000000003
0x00007fcf09a62260:   00000001000ef3e0 0000000000000003
0x00007fcf09a62270:   0000000000000003 00007fcef0882620
0x00007fcf09a62280:   00007fcf09a622c0 00007fcf660ba062 

Instructions: (pc=0x00007fcf660bf067)
0x00007fcf660bf047:   83 c6 08 39 c7 7c e2 44 8d 48 ff 45 89 08 e9 7b
0x00007fcf660bf057:   fd ff ff 66 0f 1f 44 00 00 49 8b 04 24 4c 89 e7
0x00007fcf660bf067:   ff 50 08 84 c0 0f 84 05 fd ff ff 41 8b 44 24 08
0x00007fcf660bf077:   85 c0 0f 8e f8 fc ff ff 41 f6 84 24 0e 01 00 00 

Register to memory mapping:

RAX=0x000000000000000b is an unknown value
RBX=0x00000000000000bb is an unknown value
RCX=0x00000037800182a0: <offset 0x2182a0> in /lib64/libpthread.so.0 at 0x000000377fe00000
RDX=0x00007fcf09a65a20 is pointing into the stack for thread: 0x00007fcf60804800
RSP=0x00007fcf09a62090 is pointing into the stack for thread: 0x00007fcf60804800
RBP=0x00007fcf09a62100 is pointing into the stack for thread: 0x00007fcf60804800
RSI=0x00007fcf3152bce8 is pointing into metadata
RDI=0x00007fcf3152bce8 is pointing into metadata
R8 =0x0000000000000000 is an unknown value
R9 =0x0000000000000000 is an unknown value
R10=0x0000000000000000 is an unknown value
R11=0x000000377f75a290: <offset 0x15a290> in /lib64/libc.so.6 at 0x000000377f600000
R12=0x00007fcf3152bce8 is pointing into metadata
R13=0x00007fcef8e79940 is an unknown value
R14=0x00007fcf60804800 is a thread
R15=0x00007fcef8e79940 is an unknown value


Stack: [0x00007fcf09965000,0x00007fcf09a66000],  sp=0x00007fcf09a62090,  free space=1012k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x438067]  ciObjectFactory::create_new_metadata(Metadata*)+0x327
V  [libjvm.so+0x438495]  ciObjectFactory::get_metadata(Metadata*)+0x85
V  [libjvm.so+0x430167]  ciSpeculativeTrapData::translate_from(ProfileData const*)+0x67
V  [libjvm.so+0x430478]  ciMethodData::load_extra_data()+0xa8
V  [libjvm.so+0x4325e8]  ciMethodData::load_data()+0x268
V  [libjvm.so+0x425344]  ciMethod::method_data()+0xb4
V  [libjvm.so+0x94a675]  Parse::Parse(JVMState*, ciMethod*, float)+0x5f5
V  [libjvm.so+0x3f3168]  ParseGenerator::generate(JVMState*)+0x88
V  [libjvm.so+0x56b506]  Parse::do_call()+0x1e6
V  [libjvm.so+0x953e9a]  Parse::do_one_bytecode()+0x330a
V  [libjvm.so+0x944438]  Parse::do_one_block()+0x178
V  [libjvm.so+0x948647]  Parse::do_all_blocks()+0x127
V  [libjvm.so+0x94a879]  Parse::Parse(JVMState*, ciMethod*, float)+0x7f9
V  [libjvm.so+0x3f3168]  ParseGenerator::generate(JVMState*)+0x88
V  [libjvm.so+0x56b506]  Parse::do_call()+0x1e6
V  [libjvm.so+0x953e9a]  Parse::do_one_bytecode()+0x330a
V  [libjvm.so+0x944438]  Parse::do_one_block()+0x178
V  [libjvm.so+0x948647]  Parse::do_all_blocks()+0x127
V  [libjvm.so+0x94a879]  Parse::Parse(JVMState*, ciMethod*, float)+0x7f9
V  [libjvm.so+0x3f3168]  ParseGenerator::generate(JVMState*)+0x88
V  [libjvm.so+0x56b506]  Parse::do_call()+0x1e6
V  [libjvm.so+0x953e9a]  Parse::do_one_bytecode()+0x330a
V  [libjvm.so+0x944438]  Parse::do_one_block()+0x178
V  [libjvm.so+0x948647]  Parse::do_all_blocks()+0x127
V  [libjvm.so+0x94a879]  Parse::Parse(JVMState*, ciMethod*, float)+0x7f9
V  [libjvm.so+0x3f3168]  ParseGenerator::generate(JVMState*)+0x88
V  [libjvm.so+0x4a42aa]  Compile::Compile(ciEnv*, C2Compiler*, ciMethod*, int, bool, bool, bool)+0x127a
V  [libjvm.so+0x3f1b38]  C2Compiler::compile_method(ciEnv*, ciMethod*, int)+0x198
V  [libjvm.so+0x4aea9a]  CompileBroker::invoke_compiler_on_method(CompileTask*)+0xc9a
V  [libjvm.so+0x4afa46]  CompileBroker::compiler_thread_loop()+0x5d6
V  [libjvm.so+0xa6cc5f]  JavaThread::thread_main_inner()+0xdf
V  [libjvm.so+0xa6cd8c]  JavaThread::run()+0x11c
V  [libjvm.so+0x91fad8]  java_start(Thread*)+0x108
C  [libpthread.so.0+0x79d1]


Current CompileTask:
C2: 165880 16901       4       io.dongtai.iast.thirdparty.org.apache.http.client.utils.URLEncodedUtils::urlDecode (183 bytes)

我们可以在其中捕捉到一些重要信息

• 线程：JavaThread “C2 CompilerThread2”
• 崩溃的方法：ciObjectFactory::create_new_metadata(Metadata*)+0x327
• 崩溃的根本原因：io.dongtai.iast.thirdparty.org.apache.http.client.utils.URLEncodedUtils::urlDecode (183 bytes)

原因分析

C2编译器CompilerThread2线程在执行URLEncodedUtils::urlDecode时试图访问未分配给自己的内存, 或试图往没有写权限的内存地址写数据

解决方案

直接找到urlDecode所在的包，然后升级到新版本（我也不知道老版本运行这么久了怎么突然有问题）

• 找到问题所在的包
  
• 发现不是直接引入的，mvn dependency:tree找到依赖树
  
• 屏蔽掉，然后引入新版本

<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.5.13</version>
        </dependency>

参考文章