使用c++filt查看trafficserver堆栈调用信息

在实际运维中，我们经常遇到Apache Traffic Server遇到段错误时，在traffic.out中留下的堆栈调用信息，比如下面的

FATAL: ../.././ats-4.2.0/proxy/http/HttpSM.cc:2080: failed assert `!t_state.host_db_info.reverse_dns`
/opt/ats/bin/traffic_server - STACK TRACE:
/opt/ats/lib/libtsutil.so.4(+0x1f468)[0x2b5f6be67468]
/opt/ats/lib/libtsutil.so.4(+0x1da6f)[0x2b5f6be65a6f]
/opt/ats/bin/traffic_server(_ZN6HttpSM19process_hostdb_infoEP10HostDBInfo+0x67b)[0x51bbeb]
/opt/ats/bin/traffic_server(_ZN15HostDBProcessor13getbyname_immEP12ContinuationMS0_FvP10HostDBInfoEPKciRKNS_7OptionsE+0x2c8)[0x5e4078]
/opt/ats/bin/traffic_server(_ZN6HttpSM16do_hostdb_lookupEv+0xed)[0x519d1d]
/opt/ats/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0xbb0)[0x531ee0]
/opt/ats/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x37a)[0x530f8a]
/opt/ats/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x2b0)[0x524d30]
/opt/ats/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x1f2)[0x531522]
/opt/ats/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x37a)[0x530f8a]
/opt/ats/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x2b0)[0x524d30]
/opt/ats/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x1f2)[0x531522]
/opt/ats/bin/traffic_server(_ZN6HttpSM21state_cache_open_readEiPv+0xfe)[0x5258be]
/opt/ats/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0xd8)[0x5283d8]
/opt/ats/bin/traffic_server(_ZN11HttpCacheSM21state_cache_open_readEiPv+0x1b2)[0x50bb12]
/opt/ats/bin/traffic_server(_ZN7CacheVC8callcontEi+0x53)[0x5f1353]
/opt/ats/bin/traffic_server(_ZN7CacheVC17openReadStartHeadEiP5Event+0x8d4)[0x65ba44]
/opt/ats/bin/traffic_server(_ZN7CacheVC14handleReadDoneEiP5Event+0x1ed)[0x63af3d]
/opt/ats/bin/traffic_server(_ZN5Cache9open_readEP12ContinuationP7INK_MD5P7HTTPHdrP21CacheLookupHttpConfig13CacheFragTypePci+0x3f7)[0x65d407]
/opt/ats/bin/traffic_server(_ZN14CacheProcessor9open_readEP12ContinuationP3URLbP7HTTPHdrP21CacheLookupHttpConfigl13CacheFragType+0xab)[0x63a0cb]
/opt/ats/bin/traffic_server(_ZN11HttpCacheSM9open_readEP3URLP7HTTPHdrP21CacheLookupHttpConfigl+0x94)[0x50b564]
/opt/ats/bin/traffic_server(_ZN6HttpSM24do_cache_lookup_and_readEv+0xf3)[0x519323]
/opt/ats/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x722)[0x531a52]
/opt/ats/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x37a)[0x530f8a]
/opt/ats/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x2b0)[0x524d30]
/opt/ats/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x1f2)[0x531522]
/opt/ats/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x1e2)[0x531512]
/opt/ats/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x37a)[0x530f8a]
/opt/ats/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x2b0)[0x524d30]
/opt/ats/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x1f2)[0x531522]
/opt/ats/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x37a)[0x530f8a]
/opt/ats/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x2b0)[0x524d30]
/opt/ats/bin/traffic_server(_ZN6HttpSM18state_api_callbackEiPv+0x8b)[0x5285cb]
/opt/ats/bin/traffic_server(TSHttpTxnReenable+0x354)[0x4baeb4]
/opt/ats/libexec/trafficserver/mframe.so(_Z24ReadRequestHdrHookHandleP10tsapi_cont7TSEventPv+0x57)[0x2b5f775d5a67]
/opt/ats/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x114)[0x524b94]
/opt/ats/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x1f2)[0x531522]
/opt/ats/bin/traffic_server(_ZN6HttpSM32state_read_client_request_headerEiPv+0x226)[0x521fb6]
/opt/ats/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0xd8)[0x5283d8]
/opt/ats/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x2dc)[0x530eec]
/opt/ats/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x2b0)[0x524d30]
/opt/ats/bin/traffic_server(_ZN6HttpSM21attach_client_sessionEP17HttpClientSessionP14IOBufferReader+0x684)[0x52b2f4]
/opt/ats/bin/traffic_server(_ZN17HttpClientSession16state_keep_aliveEiPv+0xa8)[0x50c3c8]
/opt/ats/bin/traffic_server[0x68873b]
/opt/ats/bin/traffic_server[0x68ae54]
/opt/ats/bin/traffic_server(_ZN10NetHandler12mainNetEventEiP5Event+0x1f2)[0x680a92]
/opt/ats/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x6ad32f]
/opt/ats/bin/traffic_server(_ZN7EThread7executeEv+0x44b)[0x6adb6b]
/opt/ats/bin/traffic_server[0x6ac6ba]
/lib64/libpthread.so.0[0x3095a07851]
/lib64/libc.so.6(clone+0x6d)[0x30956e890d]

我们发现出错的地方是

FATAL: ../.././ats-4.2.0/proxy/http/HttpSM.cc:2080: failed assert `!t_state.host_db_info.reverse_dns`

为了更方便地分析函数之间的堆栈调用关系，我们可以使用c++filt来转换一下函数名称(demangle name)，得到更易读的调用关系图，为此我将上面的信息记为stack.txt，使用

cat stack.txt | c++filt

来得到下面的信息

这样的话，调用关系一下子清晰多了，更加便于调错。针对我目前给出的这个错误，我发现这个错误与host.db有关，经过仔细分析，是host.db文件存在权限问题，将其更改为tserver用户和组就可以了。

关于c++filt的参见文献详细解释可以参见

[1].http://www.sourceware.org/binutils/docs/binutils/c_002b_002bfilt.html

[2].http://book.51cto.com/art/201005/197787.htm