gdb调试 程序退出没有堆栈信息([Inferior 1 (process 12867) exited with code 0177])

上周有新任务开发，然后周五开发完了，和其他同事联调(不能远程调试)，发现客户端上报给server之后，serever莫名其妙的就挂了，然后被重新启动(重启是自己设置的，只要进程不存在就回去启动程序)，只要上报了新增的功能相关的，server就会挂掉，上报原来有的都是正常的。但是并不是被kill掉了，也不是崩溃了。因为程序里收到SIGSEGV和SIGABRT会打印堆栈信息到一个文件里面。但是没有发现有这个文件生成。

后面给了一个抓堆栈的脚本，然后那边同事抓给我的结果是文件为空。我以为没抓到，然后看了下日志，server重启的时间和文件生成时间是相同的，看来是没有堆栈的。然后看了一下nohup文件

GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Attaching to process 12867
Reading symbols from /opt/leagsoft/LeagView/Bin/UniServer...done.
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libUniServer_oracle.so...done.
Loaded symbols for /lib64/libUniServer_oracle.so
Reading symbols from /usr/lib/oracle/11.2/client64/lib/libocci.so.11.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/oracle/11.2/client64/lib/libocci.so.11.1
Reading symbols from /usr/lib/oracle/11.2/client64/lib/libclntsh.so.11.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/oracle/11.2/client64/lib/libclntsh.so.11.1
Reading symbols from /usr/lib/oracle/11.2/client64/lib/libnnz11.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/oracle/11.2/client64/lib/libnnz11.so
Reading symbols from /lib64/libodbc.so.2...Reading symbols from /lib64/libodbc.so.2...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libodbc.so.2
Reading symbols from /lib64/libstdc++.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libstdc++.so.6
Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libgcc_s.so.1
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
[New LWP 24958]
[New LWP 4110]
[New LWP 14818]
[New LWP 14817]
[New LWP 14816]
[New LWP 14452]
[New LWP 14451]
[New LWP 4675]
[New LWP 30711]
[New LWP 30710]
[New LWP 19032]
[New LWP 29647]
[New LWP 29090]
[New LWP 29089]
[New LWP 13380]
[New LWP 13379]
[New LWP 13099]
[New LWP 13098]
[New LWP 13094]
[New LWP 13093]
[New LWP 13092]
[New LWP 13091]
[New LWP 13090]
[New LWP 13089]
[New LWP 13088]
[New LWP 13084]
[New LWP 13073]
[New LWP 13067]
[New LWP 13066]
[New LWP 13064]
[New LWP 13063]
[New LWP 13062]
[New LWP 13061]
[New LWP 13060]
[New LWP 13059]
[New LWP 13056]
[New LWP 13055]
[New LWP 13054]
[New LWP 13053]
[New LWP 13052]
[New LWP 13051]
[New LWP 12925]
[New LWP 12914]
[New LWP 12913]
[New LWP 12912]
[New LWP 12911]
[New LWP 12908]
[New LWP 12905]
[New LWP 12904]
[New LWP 12903]
[New LWP 12902]
[New LWP 12899]
[New LWP 12898]
[New LWP 12897]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnsl.so.1
Reading symbols from /lib64/libaio.so.1...Reading symbols from /lib64/libaio.so.1...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libaio.so.1
Reading symbols from /lib64/libltdl.so.7...Reading symbols from /lib64/libltdl.so.7...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libltdl.so.7
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /usr/lib64/gconv/ISO8859-1.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/gconv/ISO8859-1.so
Reading symbols from /usr/lib64/psqlodbcw-11.01.0000.so...done.
Loaded symbols for /usr/lib64/psqlodbcw-11.01.0000.so
Reading symbols from /usr/pgsql-12/lib/libpq.so.5...Reading symbols from /usr/pgsql-12/lib/libpq.so.5...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /usr/pgsql-12/lib/libpq.so.5
Reading symbols from /lib64/libodbcinst.so.2...Reading symbols from /lib64/libodbcinst.so.2...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libodbcinst.so.2
Reading symbols from /lib64/libssl.so.10...Reading symbols from /lib64/libssl.so.10...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libssl.so.10
Reading symbols from /lib64/libcrypto.so.10...Reading symbols from /lib64/libcrypto.so.10...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libcrypto.so.10
Reading symbols from /lib64/libgssapi_krb5.so.2...Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libgssapi_krb5.so.2
Reading symbols from /lib64/libldap_r-2.4.so.2...Reading symbols from /lib64/libldap_r-2.4.so.2...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libldap_r-2.4.so.2
Reading symbols from /lib64/libkrb5.so.3...Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /lib64/libk5crypto.so.3...Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libk5crypto.so.3
Reading symbols from /lib64/libz.so.1...Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libz.so.1
Reading symbols from /lib64/libkrb5support.so.0...Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/liblber-2.4.so.2...Reading symbols from /lib64/liblber-2.4.so.2...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/liblber-2.4.so.2
Reading symbols from /lib64/libsasl2.so.3...Reading symbols from /lib64/libsasl2.so.3...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libsasl2.so.3
Reading symbols from /lib64/libssl3.so...Reading symbols from /lib64/libssl3.so...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libssl3.so
Reading symbols from /lib64/libsmime3.so...Reading symbols from /lib64/libsmime3.so...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libsmime3.so
Reading symbols from /lib64/libnss3.so...Reading symbols from /lib64/libnss3.so...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libnss3.so
Reading symbols from /lib64/libnssutil3.so...Reading symbols from /lib64/libnssutil3.so...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libnssutil3.so
Reading symbols from /lib64/libplds4.so...Reading symbols from /lib64/libplds4.so...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libplds4.so
Reading symbols from /lib64/libplc4.so...Reading symbols from /lib64/libplc4.so...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libplc4.so
Reading symbols from /lib64/libnspr4.so...Reading symbols from /lib64/libnspr4.so...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libnspr4.so
Reading symbols from /lib64/libselinux.so.1...Reading symbols from /lib64/libselinux.so.1...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libselinux.so.1
Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /lib64/libpcre.so.1...Reading symbols from /lib64/libpcre.so.1...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libpcre.so.1
Reading symbols from /lib64/libfreebl3.so...Reading symbols from /lib64/libfreebl3.so...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/libfreebl3.so
Reading symbols from /usr/lib/oracle/11.2/client64/lib/libociei.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/oracle/11.2/client64/lib/libociei.so
Reading symbols from /usr/lib64/libnuma.so.1...Reading symbols from /usr/lib64/libnuma.so.1...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libnuma.so.1
Reading symbols from /lib64/libnss_dns.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnss_dns.so.2
0x00007feb65451e2d in nanosleep () from /lib64/libc.so.6
[Thread 0x7feace7fc700 (LWP 24958) exited]
[New Thread 0x7feace7fc700 (LWP 26087)]
[Thread 0x7feb12ffd700 (LWP 4110) exited]
[New Thread 0x7feb12ffd700 (LWP 31063)]
[Thread 0x7feb12ffd700 (LWP 31063) exited]
[Thread 0x7feace7fc700 (LWP 26087) exited]
[Thread 0x7feb5e638700 (LWP 12897) exited]
[Thread 0x7feb5d636700 (LWP 12899) exited]
[Thread 0x7feb5ce35700 (LWP 12902) exited]
[Thread 0x7feb4ffff700 (LWP 12903) exited]
[Thread 0x7feb4f7fe700 (LWP 12904) exited]
[Thread 0x7feb4effd700 (LWP 12905) exited]
[Thread 0x7feb4d7fa700 (LWP 12908) exited]
[Thread 0x7feb31ef7700 (LWP 12911) exited]
[Thread 0x7feb316f6700 (LWP 12912) exited]
[Thread 0x7feb30ef5700 (LWP 12913) exited]
[Thread 0x7feb699fc700 (LWP 12914) exited]
[Thread 0x7feb13fff700 (LWP 12925) exited]
[Thread 0x7feb127fc700 (LWP 13051) exited]
[Thread 0x7feb11ffb700 (LWP 13052) exited]
[Thread 0x7feb117fa700 (LWP 13053) exited]
[Thread 0x7feb10ff9700 (LWP 13054) exited]
[Thread 0x7feaeffff700 (LWP 13055) exited]
[Thread 0x7feaef7fe700 (LWP 13056) exited]
[Thread 0x7feaedffb700 (LWP 13059) exited]
[Thread 0x7feaed7fa700 (LWP 13060) exited]
[Thread 0x7feaecff9700 (LWP 13061) exited]
[Thread 0x7feacffff700 (LWP 13062) exited]
[Thread 0x7feacf7fe700 (LWP 13063) exited]
[Thread 0x7feaceffd700 (LWP 13064) exited]
[Thread 0x7feacdffb700 (LWP 13066) exited]
[Thread 0x7feacd7fa700 (LWP 13067) exited]
[Thread 0x7feabdffb700 (LWP 13073) exited]
[Thread 0x7feab87f0700 (LWP 13084) exited]
[Thread 0x7feb4e7fc700 (LWP 13088) exited]
[Thread 0x7feab67ec700 (LWP 13089) exited]
[Thread 0x7feab5feb700 (LWP 13090) exited]
[Thread 0x7feab57ea700 (LWP 13091) exited]
[Thread 0x7feab4fe9700 (LWP 13092) exited]
[Thread 0x7feab47e8700 (LWP 13093) exited]
[Thread 0x7feab3fe7700 (LWP 13094) exited]
[Thread 0x7feab37e6700 (LWP 13098) exited]
[Thread 0x7feab2fe5700 (LWP 13099) exited]
[Thread 0x7feab97f2700 (LWP 13379) exited]
[Thread 0x7feab9ff3700 (LWP 13380) exited]
[Thread 0x7feab7fef700 (LWP 29089) exited]
[Thread 0x7feaee7fc700 (LWP 29090) exited]
[Thread 0x7feb227fc700 (LWP 29647) exited]
[Thread 0x7feb4dffb700 (LWP 19032) exited]
[Thread 0x7feb21ffb700 (LWP 30710) exited]
[Thread 0x7feb23fff700 (LWP 30711) exited]
[Thread 0x7feaccff9700 (LWP 4675) exited]
[Thread 0x7feb20ff9700 (LWP 14451) exited]
[Thread 0x7feb217fa700 (LWP 14452) exited]
[Thread 0x7feab6fed700 (LWP 14816) exited]
[Thread 0x7feb237fe700 (LWP 14817) exited]
[Thread 0x7feb137fe700 (LWP 14818) exited]
[Thread 0x7feb69a45740 (LWP 12867) exited]
[Inferior 1 (process 12867) exited with code 0177]
//它这里没有报undefined symbol，可能是因为串口日志被禁用了，所以没有报具体的undefined symbol。前面这句话是从网络上看到的(指的是——>串口日志被禁用了，所以没打印)

这里程序退出了有一个code 0177 百度了一下大致都是undefined symbol，我让同事ldd -r xxxx(程序名)，发现我新增的两个函数都是undefined symbol。然后我又让同事执行nm -A xxx.so | grep xxx(函数名) ，看看动态库里面有没有我们我们新增的函数的实现，发现我新增的函数它是有实现的

man nm //可以看到T 代表的是已经实现的函数  U 为未实现的函数
           "T"
           "t" The symbol is in the text (code) section.

           "U" The symbol is undefined.

所以我们提供的新版本的动态库是有新增的函数实现的，但是我们的程序，使用ldd -r查看时却显示未定义的符号，由此我们推断时so文件的位置放错了，因为server的路径我是知道的，后面发现果然是so路径的问题。

查看我们提供的so的符号信息时，是有实现的，但是server依赖的还是旧的so导致了未定义的符号，而我们在代码中调用了这个只有声明没有定义的接口，程序会直接退出，不是崩溃，没有堆栈信息，也不是被kill。