On android, when a process crashes in native code, the call stack of the process will be saved to a log file in /data/tombstomes/, and written to logcat as well. The information is helpful for debugging.
Unfortunately, the call stack doesn't show in human readable format, file name, function name. Instead, it's shown as module name (e.g., libc.so) and memory address of the instruction. We can use addr2line to translate the address to corresponding file name and function name if we have the binary of the module that contains symbol information.
To make it easier to use, this function is included in agdb tool (see here for more). We can use "agdb.py -r -e module_name address" to find out the function name of specified address within the module.
When we have a long call stack, instead of running the command above for each line in the call stack manually, we can feed the whole call stack to agdb through pipe and get the full resolved call stack. For example, use "adb logcat | agdb.py -r" command for adb logcat output with below contents:
22 F/ASessionDescription( 33): frameworks/base/media/libstagefright/rtsp/ASessionDescription.cpp:264 CHECK_GT( end,s) failed: vs.
23 I/DEBUG ( 30): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
24 I/DEBUG ( 30): Build fingerprint: 'generic/generic/generic:2.3.1/GINGERBREAD/eng.raymond.20101222.130550:eng/test-keys'
25 I/DEBUG ( 30): pid: 33, tid: 450 >>> /system/bin/mediaserver <<<
26 I/DEBUG ( 30): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr deadbaad
27 I/DEBUG ( 30): r0 deadbaad r1 0000000c r2 00000027 r3 00000000
28 I/DEBUG ( 30): r4 00000080 r5 afd46668 r6 40806c10 r7 00000000
29 I/DEBUG ( 30): r8 8031db1d r9 0000fae0 10 00100000 fp 00000001
30 I/DEBUG ( 30): ip ffffffff sp 40806778 lr afd19375 pc afd15ef0 cpsr 00000030
31 I/DEBUG ( 30): #00 pc 00015ef0 /system/lib/libc.so
32 I/DEBUG ( 30): #01 pc 00001440 /system/lib/liblog.so
33 I/DEBUG ( 30):
34 I/DEBUG ( 30): code around pc:
35 I/DEBUG ( 30): afd15ed0 68241c23 d1fb2c00 68dae027 d0042a00
36 I/DEBUG ( 30): afd15ee0 20014d18 6028447d 48174790 24802227
37 I/DEBUG ( 30): afd15ef0 f7f57002 2106eb56 ec92f7f6 0563aa01
38 I/DEBUG ( 30): afd15f00 60932100 91016051 1c112006 e818f7f6
39 I/DEBUG ( 30): afd15f10 2200a905 f7f62002 f7f5e824 2106eb42
40 I/DEBUG ( 30):
41 I/DEBUG ( 30): code around lr:
42 I/DEBUG ( 30): afd19354 b0834a0d 589c447b 26009001 686768a5
43 I/DEBUG ( 30): afd19364 220ce008 2b005eab 1c28d003 47889901
44 I/DEBUG ( 30): afd19374 35544306 d5f43f01 2c006824 b003d1ee
45 I/DEBUG ( 30): afd19384 bdf01c30 000281a8 ffffff88 1c0fb5f0
46 I/DEBUG ( 30): afd19394 43551c3d a904b087 1c16ac01 604d9004
47 I/DEBUG ( 30):
48 I/DEBUG ( 30): stack:
49 ........................
92 I/DEBUG ( 30): 408067e4 6f697470
93 I/BootReceiver( 75): Copying /data/tombstones/tombstone_09 to DropBox (SYSTEM_TOMBSTONE)
we get:
22 F/ASessionDescription( 33): frameworks/base/media/libstagefright/rtsp/ASessionDescription.cpp:264 CHECK_GT( end,s) failed: vs.
23 I/DEBUG ( 30): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
24 I/DEBUG ( 30): Build fingerprint: 'generic/generic/generic:2.3.1/GINGERBREAD/eng.raymond.20101222.130550:eng/test-keys'
25 I/DEBUG ( 30): pid: 33, tid: 450 >>> /system/bin/mediaserver <<<
26 I/DEBUG ( 30): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr deadbaad
27 I/DEBUG ( 30): r0 deadbaad r1 0000000c r2 00000027 r3 00000000
28 I/DEBUG ( 30): r4 00000080 r5 afd46668 r6 40806c10 r7 00000000
29 I/DEBUG ( 30): r8 8031db1d r9 0000fae0 10 00100000 fp 00000001
30 I/DEBUG ( 30): ip ffffffff sp 40806778 lr afd19375 pc afd15ef0 cpsr 00000030
31 I/DEBUG ( 30): #00 pc 00015ef0 /system/lib/libc.so
32 I/DEBUG ( 30): #00 __libc_android_abort: abort.c:82
33 I/DEBUG ( 30): #01 pc 00001440 /system/lib/liblog.so
34 I/DEBUG ( 30): #01 __android_log_assert: logd_write.c:235
35 I/DEBUG ( 30):
36 I/DEBUG ( 30): code around pc:
37 I/DEBUG ( 30): afd15ed0 68241c23 d1fb2c00 68dae027 d0042a00
38 I/DEBUG ( 30): afd15ee0 20014d18 6028447d 48174790 24802227
39 I/DEBUG ( 30): afd15ef0 f7f57002 2106eb56 ec92f7f6 0563aa01
Similarly, we can use "echo tombstone_01.txt | agdb.py -r" command to resolve call stack addresses in a tombstone log file.
Unfortunately, the call stack doesn't show in human readable format, file name, function name. Instead, it's shown as module name (e.g., libc.so) and memory address of the instruction. We can use addr2line to translate the address to corresponding file name and function name if we have the binary of the module that contains symbol information.
To make it easier to use, this function is included in agdb tool (see here for more). We can use "agdb.py -r -e module_name address" to find out the function name of specified address within the module.
When we have a long call stack, instead of running the command above for each line in the call stack manually, we can feed the whole call stack to agdb through pipe and get the full resolved call stack. For example, use "adb logcat | agdb.py -r" command for adb logcat output with below contents:
22 F/ASessionDescription( 33): frameworks/base/media/libstagefright/rtsp/ASessionDescription.cpp:264 CHECK_GT( end,s) failed: vs.
23 I/DEBUG ( 30): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
24 I/DEBUG ( 30): Build fingerprint: 'generic/generic/generic:2.3.1/GINGERBREAD/eng.raymond.20101222.130550:eng/test-keys'
25 I/DEBUG ( 30): pid: 33, tid: 450 >>> /system/bin/mediaserver <<<
26 I/DEBUG ( 30): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr deadbaad
27 I/DEBUG ( 30): r0 deadbaad r1 0000000c r2 00000027 r3 00000000
28 I/DEBUG ( 30): r4 00000080 r5 afd46668 r6 40806c10 r7 00000000
29 I/DEBUG ( 30): r8 8031db1d r9 0000fae0 10 00100000 fp 00000001
30 I/DEBUG ( 30): ip ffffffff sp 40806778 lr afd19375 pc afd15ef0 cpsr 00000030
31 I/DEBUG ( 30): #00 pc 00015ef0 /system/lib/libc.so
32 I/DEBUG ( 30): #01 pc 00001440 /system/lib/liblog.so
33 I/DEBUG ( 30):
34 I/DEBUG ( 30): code around pc:
35 I/DEBUG ( 30): afd15ed0 68241c23 d1fb2c00 68dae027 d0042a00
36 I/DEBUG ( 30): afd15ee0 20014d18 6028447d 48174790 24802227
37 I/DEBUG ( 30): afd15ef0 f7f57002 2106eb56 ec92f7f6 0563aa01
38 I/DEBUG ( 30): afd15f00 60932100 91016051 1c112006 e818f7f6
39 I/DEBUG ( 30): afd15f10 2200a905 f7f62002 f7f5e824 2106eb42
40 I/DEBUG ( 30):
41 I/DEBUG ( 30): code around lr:
42 I/DEBUG ( 30): afd19354 b0834a0d 589c447b 26009001 686768a5
43 I/DEBUG ( 30): afd19364 220ce008 2b005eab 1c28d003 47889901
44 I/DEBUG ( 30): afd19374 35544306 d5f43f01 2c006824 b003d1ee
45 I/DEBUG ( 30): afd19384 bdf01c30 000281a8 ffffff88 1c0fb5f0
46 I/DEBUG ( 30): afd19394 43551c3d a904b087 1c16ac01 604d9004
47 I/DEBUG ( 30):
48 I/DEBUG ( 30): stack:
49 ........................
92 I/DEBUG ( 30): 408067e4 6f697470
93 I/BootReceiver( 75): Copying /data/tombstones/tombstone_09 to DropBox (SYSTEM_TOMBSTONE)
we get:
22 F/ASessionDescription( 33): frameworks/base/media/libstagefright/rtsp/ASessionDescription.cpp:264 CHECK_GT( end,s) failed: vs.
23 I/DEBUG ( 30): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
24 I/DEBUG ( 30): Build fingerprint: 'generic/generic/generic:2.3.1/GINGERBREAD/eng.raymond.20101222.130550:eng/test-keys'
25 I/DEBUG ( 30): pid: 33, tid: 450 >>> /system/bin/mediaserver <<<
26 I/DEBUG ( 30): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr deadbaad
27 I/DEBUG ( 30): r0 deadbaad r1 0000000c r2 00000027 r3 00000000
28 I/DEBUG ( 30): r4 00000080 r5 afd46668 r6 40806c10 r7 00000000
29 I/DEBUG ( 30): r8 8031db1d r9 0000fae0 10 00100000 fp 00000001
30 I/DEBUG ( 30): ip ffffffff sp 40806778 lr afd19375 pc afd15ef0 cpsr 00000030
31 I/DEBUG ( 30): #00 pc 00015ef0 /system/lib/libc.so
32 I/DEBUG ( 30): #00 __libc_android_abort: abort.c:82
33 I/DEBUG ( 30): #01 pc 00001440 /system/lib/liblog.so
34 I/DEBUG ( 30): #01 __android_log_assert: logd_write.c:235
35 I/DEBUG ( 30):
36 I/DEBUG ( 30): code around pc:
37 I/DEBUG ( 30): afd15ed0 68241c23 d1fb2c00 68dae027 d0042a00
38 I/DEBUG ( 30): afd15ee0 20014d18 6028447d 48174790 24802227
39 I/DEBUG ( 30): afd15ef0 f7f57002 2106eb56 ec92f7f6 0563aa01
Similarly, we can use "echo tombstone_01.txt | agdb.py -r" command to resolve call stack addresses in a tombstone log file.