生成独立的gcc调试文件及利用调试文件调试core文件

1、生成独立的gcc调试文件

编译调试信息：

gcc -g -o main main.c

分离调试信息：

objcopy --only-keep-debug main main.debug

或者

cp main main.debug strip --only-keep-debug main.debug

从原始文件中剥离调试信息：

objcopy --strip-debug main

或者

strip --strip-debug --strip-unneeded main

通过debuglink模式进行调试：

objcopy --add-gnu-debuglink main.debug main gdb main

您还可以分别使用exec文件和符号文件：

gdb -s main.debug -e main

或者

gdb (gdb) exec-file main (gdb) symbol-file main.debug

有关详细信息：

(gdb) help exec-file (gdb) help symbol-file

 

 

2、gdb结合调试符号文件和可执行程序、core文件来定位出错位置

如果是符号表与二进程序分离的程序进行所产生的core，可以通过以下方式调试：

gdb -c core.1234 -e bin/local_agent_server -s debug/local_agent_server.debug
其中：
-c 指定core文件
-e 指定binary，用线上的binary即可
-s 指定符号表，也就是我们新生成的符号表

 

详细参见：与符号表分离的程序和动态库，如何用gdb调试

 

3、调试中发现没有加载到某些库时

可能还是出现找不到符号，也就定位不到出错的信息，这时需要在gdb模式下设置库的查找路径，set solib-search-path，如下：

(gdb) set solib-search-path /home/chengkang/attr_output/bvs-algo/attr_so/:/home/chengkang/attr_output/bvs-algo/

 

下面是gdb调试流程举例：

当没有符号信息时是这样：

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f75c5312de6 in ?? ()
[Current thread is 1 (LWP 9972)]
(gdb) bt
#0  0x00007f75c5312de6 in ?? ()
#1  0x00007ffe5a0ca6d0 in ?? ()
#2  0x00007ffe5a0ca260 in ?? ()
#3  0x000000000262b9c0 in ?? ()
#4  0x00000000039cba70 in ?? ()
#5  0x00007ffe5a0ca148 in ?? ()
#6  0x00007f75dffe9890 in ?? ()
#7  0x00007ffe5a0ca970 in ?? ()
#8  0x00007ffe5a0ca3c0 in ?? ()
#9  0x00007ffe5a0ca380 in ?? ()
#10 0x00007ffe5a0cac10 in ?? ()
#11 0x00007ffe5a0ca370 in ?? ()
#12 0x00007f75c4536ed6 in ?? ()
#13 0x00007ffe5a0ca190 in ?? ()
#14 0x00007ffe5a0ca130 in ?? ()
#15 0x00007f7607efad40 in std::(anonymous namespace)::collate_c () from /opt/compiler/gcc-8.2/lib/libstdc++.so.6
#16 0x00007ffe5a0ca8d0 in ?? ()
#17 0x0000000000000000 in ?? ()
(gdb) disassemble
No function contains program counter for selected frame.
(gdb) f 0
#0  0x00007f75c5312de6 in ?? ()

当加载上debug调试文件、设置了库路径后是这样： 

[root@shdx bvs-algo]# gdb -s bin/body_attr.debug -e bin/body_attr -c core9972
GNU gdb (GDB) 8.3.1
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from bin/body_attr.debug...
[New LWP 9972]
[New LWP 9973]
[New LWP 9974]
[New LWP 9975]
[New LWP 9978]
[New LWP 9976]
[New LWP 9977]
[New LWP 9979]

warning: .dynamic section for "/opt/compiler/gcc-8.2/lib/libz.so.1" is not at the expected address (wrong library or version mismatch?)

warning: .dynamic section for "/opt/compiler/gcc-8.2/lib/libstdc++.so.6" is not at the expected address (wrong library or version mismatch?)

warning: .dynamic section for "/opt/compiler/gcc-8.2/lib/libgcc_s.so.1" is not at the expected address (wrong library or version mismatch?)

warning: .dynamic section for "/opt/compiler/gcc-8.2/lib/libuuid.so.1" is not at the expected address (wrong library or version mismatch?)

warning: .dynamic section for "/opt/compiler/gcc-8.2/lib/libreadline.so.6" is not at the expected address (wrong library or version mismatch?)

warning: .dynamic section for "/opt/compiler/gcc-8.2/lib/libncurses.so.6" is not at the expected address (wrong library or version mismatch?)

warning: .dynamic section for "/lib64/libcuda.so.1" is not at the expected address (wrong library or version mismatch?)


warning: Could not load shared library symbols for 5 libraries, e.g. /root/apps/bvs-algo/./attr_so/libpaddle_fluid.so.
Use the "info sharedlibrary" command to see the complete listing.
Do you need "set solib-search-path" or "set sysroot"?

warning: File "/opt/compiler/gcc-8.2/lib/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
To enable execution of this file add
	add-auto-load-safe-path /opt/compiler/gcc-8.2/lib/libthread_db-1.0.so
line to your configuration file "/root/.gdbinit".
To completely disable this security protection add
	set auto-load safe-path /
line to your configuration file "/root/.gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
	info "(gdb)Auto-loading safe path"

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.

warning: File "/opt/compiler/gcc-8.2/lib/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
Core was generated by `./bin/body_attr --systemResource.gpuIndex.use=2 --system.auth_server=66.100.128'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f75c5312de6 in ?? ()
[Current thread is 1 (LWP 9972)]
(gdb) pwd
Working directory /home/chengkang/attr_output/bvs-algo.
(gdb) set solib-search-path /home/chengkang/attr_output/bvs-algo/attr_so/:/home/chengkang/attr_output/bvs-algo/
Reading symbols from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libpaddle_fluid_gpu_cudnn7.so...
Reading symbols from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libiomp5.so...
(No debugging symbols found in /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libiomp5.so)
Reading symbols from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libmklml_intel.so...
Reading symbols from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libdnnl.so.1...
Reading symbols from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libnvinfer.so...
(No debugging symbols found in /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libnvinfer.so)
(gdb) bt
#0  0x00007f75c5312de6 in paddle::inference::tensorrt::OpConverter::ConvertBlockToTRTEngine(paddle::framework::BlockDesc*, paddle::framework::Scope const&, std::vector<std::string, std::allocator<std::string> > const&, std::unordered_set<std::string, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, paddle::inference::tensorrt::TensorRTEngine*) ()
   from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libpaddle_fluid_gpu_cudnn7.so
#1  0x00007f75c662e908 in paddle::inference::analysis::TensorRtSubgraphPass::CreateTensorRTOp(paddle::framework::ir::Node*, paddle::framework::ir::Graph*, std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> >*) const ()
   from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libpaddle_fluid_gpu_cudnn7.so
#2  0x00007f75c662ff52 in paddle::inference::analysis::TensorRtSubgraphPass::ApplyImpl(paddle::framework::ir::Graph*) const ()
   from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libpaddle_fluid_gpu_cudnn7.so
#3  0x00007f75c673e990 in paddle::framework::ir::Pass::Apply(paddle::framework::ir::Graph*) const ()
   from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libpaddle_fluid_gpu_cudnn7.so
#4  0x00007f75c66b8166 in paddle::inference::analysis::IRPassManager::Apply(std::unique_ptr<paddle::framework::ir::Graph, std::default_delete<paddle::framework::ir::Graph> >) ()
   from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libpaddle_fluid_gpu_cudnn7.so
#5  0x00007f75c669fcff in paddle::inference::analysis::IrAnalysisPass::RunImpl(paddle::inference::analysis::Argument*) ()
   from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libpaddle_fluid_gpu_cudnn7.so
#6  0x00007f75c66340d5 in paddle::inference::analysis::Analyzer::RunAnalysis(paddle::inference::analysis::Argument*) ()
   from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libpaddle_fluid_gpu_cudnn7.so
#7  0x00007f75c43beb44 in paddle::AnalysisPredictor::OptimizeInferenceProgram() ()
   from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libpaddle_fluid_gpu_cudnn7.so
#8  0x00007f75c43bf47f in paddle::AnalysisPredictor::PrepareProgram(std::shared_ptr<paddle::framework::ProgramDesc> const&) ()
   from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libpaddle_fluid_gpu_cudnn7.so
#9  0x00007f75c43bf5ff in paddle::AnalysisPredictor::Init(std::shared_ptr<paddle::framework::Scope> const&, std::shared_ptr<paddle::framework::ProgramDesc> const&) ()
   from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libpaddle_fluid_gpu_cudnn7.so
#10 0x00007f75c43bfa09 in std::unique_ptr<paddle::PaddlePredictor, std::default_delete<paddle::PaddlePredictor> > paddle::CreatePaddlePredictor<paddle::AnalysisConfig, (paddle::PaddleEngineKind)2>(paddle::AnalysisConfig const&) () from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libpaddle_fluid_gpu_cudnn7.so
#11 0x00007f75c43c0341 in std::unique_ptr<paddle::PaddlePredictor, std::default_delete<paddle::PaddlePredictor> > paddle::CreatePaddlePredictor<paddle::AnalysisConfig>(paddle::AnalysisConfig const&) () from /home/chengkang/attr_output/bvs-algo/vis/attr/1.6.3.2.7/output/fg-human-attribute/so/libpaddle_fluid_gpu_cudnn7.so
#12 0x00007f7604082415 in vis::Predictor::init (this=0x2638060, conf_file=..., prefix=...) at baidu/vis-open/paddle/src/predictor.cpp:552
#13 0x00007f7604083291 in vis::create_cnnpredict (conf_file=..., prefix=...) at baidu/vis-open/paddle/src/predictor.cpp:591
#14 0x00007f76061751e6 in vis::GeneralClassifyProcessor::init (this=0x21ece50, conf_file_name=..., prefix=...)
    at baidu/idl-finegrain/fg-human-attribute/src/general_classify_processor.cpp:310
#15 0x00007f760617e486 in vis::GeneralClassifyProcessorFactory::init (this=0x21e6670, conf_path=..., conf_file=...)
    at baidu/idl-finegrain/fg-human-attribute/src/general_classify_processor.cpp:1395
#16 0x00007f75fe4be33e in init_body_attr (sopath=<optimized out>, confdir=0x21b4638 "conf", conffile=0x21b4e08 "attr", processor_num=3)
    at /home/opt/compiler/gcc-8.2/include/c++/8.2.0/ext/new_allocator.h:79
#17 0x00007f75fe4c5cf5 in sub_main (argc=<optimized out>, argv=<optimized out>) at /home/opt/compiler/gcc-8.2/include/c++/8.2.0/bits/basic_string.h:5119
#18 0x000000000040b74d in load_and_run_sub_main (argc=7, argv=0x7ffe5a0ce5f8, soname=<optimized out>) at baidu/aicd/bvs-algo/common/include/main_templ.hpp:104
#19 0x00007f7607e34160 in std::system_error::system_error (this=0x0, __ec=..., __what=0x19ad0 <error: Cannot access memory at address 0x19ad0>)
    at /root/yangsong/code/baidu/bcloud/baseenv/devel/toolchain/gcc-8.2.0/build_p2/x86_64-pc-linux-gnu/libstdc++-v3/include/system_error:353
#20 0x00007f7607ef8e20 in std::cerr () from /opt/compiler/gcc-8.2/lib/libstdc++.so.6
#21 0x00007f7607ef8e20 in std::cerr () from /opt/compiler/gcc-8.2/lib/libstdc++.so.6
#22 0x00007f7607efa080 in (anonymous namespace)::collate_c () from /opt/compiler/gcc-8.2/lib/libstdc++.so.6
#23 0x00007ffe5a0ce0a8 in ?? ()
#24 0x000000000040c0df in daemon_run_ex<void (*)(), int (*)(int, char**, char const*), int, char**, char const*> (et=<optimized out>, f=0x7, args#0=32766,
--Type <RET> for more, q to quit, c to continue without paging--