gdb调试core分析jvm（JNI）奔溃原因

  前几天服务（服务中使用了JNI调用了C++的so库）在并发测试几天后jvm突然奔溃，只在控制台打印出了一句话： terminate called after throwing an instance of '._0'。因为只根据这句话无法确定奔溃原因，于是查看linux系统日志，进入/var/log下，打开message文件，看到下面的话：

Sep 16 13:44:04 localhost abrt[37667]: Saved core dump of pid 4865 (/home/dear/smb/environments/jdk/bin/java) to /var/spool/abrt/ccpp-2017-09-16-13:27:35-4865 (6602772480 bytes)
Sep 16 13:44:04 localhost abrtd: Directory 'ccpp-2017-09-16-13:27:35-4865' creation detected
Sep 16 13:44:07 localhost abrtd: Executable '/home/dear/smb/environments/jdk/bin/java' doesn't belong to any package and ProcessUnpackaged is set to 'no'
Sep 16 13:44:07 localhost abrtd: 'post-create' on '/var/spool/abrt/ccpp-2017-09-16-13:27:35-4865' exited with 1
Sep 16 13:44:07 localhost abrtd: Deleting problem directory '/var/spool/abrt/ccpp-2017-09-16-13:27:35-4865'

是因为系统设置的问题，没有生成core dump文件，执行ulimit -c为0，于是修改系统配置，使其为unlimited，再次启动服务，期待问题浮现。果然在运行了几天后，又出现了 同样的问题，且生成了core dump文件。使用gdb对core文件进行分析，执行命令：gdb /home/dear/smb/environments/jdk/bin/java（java命令路径） core.4865（core dump文件名称）。然后键入bt命令，出现下列信息：

#0  0x0000003fe60323ae in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:67
#1  0x0000003fe60337f5 in abort () at abort.c:93
#2  0x00007f06386efcbd in __gnu_cxx::__verbose_terminate_handler() () at ../../../../gcc-4.9.2/libstdc++-v3/libsupc++/vterminate.cc:95
#3  0x00007f06386b2956 in __cxxabiv1::__terminate(void (*)()) () at ../../../../gcc-4.9.2/libstdc++-v3/libsupc++/eh_terminate.cc:47
#4  0x00007f06386b29a1 in std::terminate() () at ../../../../gcc-4.9.2/libstdc++-v3/libsupc++/eh_terminate.cc:57
#5  0x00007f06386b27c8 in __cxa_throw () at ../../../../gcc-4.9.2/libstdc++-v3/libsupc++/eh_throw.cc:87
#6  0x00007f06386a8f29 in ApLicenser::CutEffInfo (this=Unhandled dwarf expression opcode 0xf3
) at ap_license.cpp:64
#7  0x00007f06386a907e in ApLicenser::<lambda()>::operator()(void) const (__closure=0x7f0630290210) at ap_license.cpp:171
#8  0x00007f06386aa652 in ApLicenser::CatchInfo (this=Unhandled dwarf expression opcode 0xf3
) at ap_license.cpp:182
#9  0x00007f06386ab46e in ApLicenser::CheckLicense (this=0x7f0640a28700, mark=Unhandled dwarf expression opcode 0xf3
) at ap_license.cpp:269
#10 0x00007f06386abe0a in ApLicenser::CopyrightCheck (this=Unhandled dwarf expression opcode 0xf3
) at ap_license.cpp:414
#11 0x00007f06386a07b6 in ApHandler::CopyrightCheck (this=0x7f06407befd0, request_ip=..., user_num=-1, model_num=-1, verify_num=-1) at ../ap/ap_server.cpp:28
#12 0x00007f06386a2bdd in Java_com_dear_smb_bus_jni_NativeJNIInterface_checkLocalIpInfo (env=0x7f062ae279e8, clazz=Unhandled dwarf expression opcode 0xf3
) at vpr_model.cpp:387
#13 0x00007f0649a89d98 in ?? ()
#14 0xffffffffffffffff in ?? ()
#15 0xffffffffffffffff in ?? ()
#16 0x00000000afd36b70 in ?? ()
#17 0x00000000afd36b98 in ?? ()
#18 0x0000000000000000 in ?? ()

上面的信息列出了出错的native堆栈，因为gdb无法分析到java的层面，所以下面会打出??。通过上面的信息，可以看到，出错的C++源码部分，即，in ApLicenser::CutEffInfo (this=Unhandled dwarf expression opcode 0xf3) at ap_license.cpp:64，查看ap_license.cpp的64行，如下

void ApLicenser::CutEffInfo(const std::string str_order, std::string &result) {
    FILE* fp;  
    char   buf[1024];        
    bzero(buf, sizeof(buf)); 
    fp = popen( str_order.c_str(), "r" ); 
    if (fp == NULL) {
        throw AP_UNKNOWN_ERROR;
    }
    fread( buf, sizeof(char), sizeof(buf),  fp);   //64行
    pclose( fp ); 
    result = buf;
}

通过分析发现，此方法被调用时，上层使用catch (int err)进行捕捉，而AP_UNKNOWN_ERROR为枚举类型，改为throw (int)AP_READ_LICENSE_FAILED;（进行强制转换），问题解决。