系统HANG住分析工具及方法

如果系统HANG住了，这个时候做一个SYSTEM STATE DUMP，对于分析HANG的原因十分重要。但是很多情况下，系统HANG住了就无法登录，那么如何进行分析呢？
METALINK DOC :359536.1提供了一些好的办法，在这里和大家共享：
支持的数据库版本：9.2.0.1-10.2
平台（部分是UNIX平台，部分适用于所有平台）

1、10G以前，使用DBX,GDB之类的工具：

找到一个前台进程的PID（ps -ef|grep LOCAL)，然后用gdb $ORACLE_HOME/bin/oracle 进行跟踪
oracle@gurufl02:~> ps -ef|grep LOCAL
oracle   26358 26357  0 09:52 ?        00:00:00 oracleora9 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
oracle   27848 27810  0 10:41 pts/3    00:00:00 grep LOCAL
oracle@gurufl02:~> gdb $ORACLE_HOME/bin/oracle 26358
然后执行：print ksudss(10)
(gdb) print ksudss(10)
[Switching to Thread 4154533552 (LWP 26358)]
warning: Unable to restore previously selected frame.
$1 = 181773020
Current language:  auto; currently asm

这个时候在UDUMP或者BDUMP下会生成一个TRACE文件。里面包含SYSTEM STATE DUMP

例如：
1：
SQL> select spid from v$session s,v$process p where s.paddr=p.addr and sid in(select distinct sid from v$mystat);  


SPID
------------------------
6093


SQL> update scott.emp set comm=1000 where empno=7369;
2：
[root@test ~]# gdb $ORACLE_HOME/bin/oracle 6093
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
/bin/oracle: No such file or directory.
Attaching to process 6093
Reading symbols from /u01/app/product/11.2.0/dbhome_1/bin/oracle...(no debugging symbols found)...done.
Reading symbols from /u01/app/product/11.2.0/dbhome_1/lib/libodm11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/product/11.2.0/dbhome_1/lib/libodm11.so
Reading symbols from /u01/app/product/11.2.0/dbhome_1/lib/libcell11.so...done.
Loaded symbols for /u01/app/product/11.2.0/dbhome_1/lib/libcell11.so
Reading symbols from /u01/app/product/11.2.0/dbhome_1/lib/libskgxp11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/product/11.2.0/dbhome_1/lib/libskgxp11.so
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /u01/app/product/11.2.0/dbhome_1/lib/libnnz11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/product/11.2.0/dbhome_1/lib/libnnz11.so
Reading symbols from /u01/app/product/11.2.0/dbhome_1/lib/libclsra11.so...done.
Loaded symbols for /u01/app/product/11.2.0/dbhome_1/lib/libclsra11.so
Reading symbols from /u01/app/product/11.2.0/dbhome_1/lib/libdbcfg11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/product/11.2.0/dbhome_1/lib/libdbcfg11.so
Reading symbols from /u01/app/product/11.2.0/dbhome_1/lib/libhasgen11.so...done.
Loaded symbols for /u01/app/product/11.2.0/dbhome_1/lib/libhasgen11.so
Reading symbols from /u01/app/product/11.2.0/dbhome_1/lib/libskgxn2.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/product/11.2.0/dbhome_1/lib/libskgxn2.so
Reading symbols from /u01/app/product/11.2.0/dbhome_1/lib/libocr11.so...done.
Loaded symbols for /u01/app/product/11.2.0/dbhome_1/lib/libocr11.so
Reading symbols from /u01/app/product/11.2.0/dbhome_1/lib/libocrb11.so...done.
Loaded symbols for /u01/app/product/11.2.0/dbhome_1/lib/libocrb11.so
Reading symbols from /u01/app/product/11.2.0/dbhome_1/lib/libocrutl11.so...done.
Loaded symbols for /u01/app/product/11.2.0/dbhome_1/lib/libocrutl11.so
Reading symbols from /lib64/libaio.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libaio.so.1
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnsl.so.1
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /usr/lib64/libnuma.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libnuma.so.1
Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /u01/app/product/11.2.0/dbhome_1/lib/libnque11.so...(no debugging symbols found)...done.
Loaded symbols for /u01/app/product/11.2.0/dbhome_1/lib/libnque11.so
0x000000388300e530 in __read_nocancel () from /lib64/libpthread.so.0
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.107.el6.x86_64 libaio-0.3.107-10.el6.x86_64 numactl-2.0.7-6.el6.x86_64
(gdb) 
(gdb) b kslgetl
Breakpoint 1 at 0x93f9818
(gdb) c
Continuing.


Breakpoint 1, 0x00000000093f9818 in kslgetl ()
(gdb) info all-register 
rax            0x0      0
rbx            0x75414010       1967210512
rcx            0xf97    3991
rdx            0x75413fe0       1967210464
rsi            0x1      1
rdi            0x6010f6c8       1611724488
rbp            0x7fff09c28b40   0x7fff09c28b40
rsp            0x7fff09c28b40   0x7fff09c28b40
r8             0xf97    3991
r9             0xc0cc9e0        202164704
r10            0x75413fe0       1967210464
r11            0xf97    3991
r12            0xc0cc9e0        202164704
r13            0x6010f6c8       1611724488
r14            0x1      1
r15            0x1      1
rip            0x93f9818        0x93f9818 <kslgetl+4>
eflags         0x246    [ PF ZF IF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0
st0            0        (raw 0x00000000000000000000)
st1            0        (raw 0x00000000000000000000)
st2            0        (raw 0x00000000000000000000)
st3            0        (raw 0x00000000000000000000)
st4            0        (raw 0x00000000000000000000)
st5            0        (raw 0x00000000000000000000)
st6            105264   (raw 0x400fcd98000000000000)
st7            8156     (raw 0x400bfee0000000000000)
fctrl          0x27f    639
fstat          0x0      0
ftag           0xffff   65535
fiseg          0x0      0
fioff          0x14b99c1        21731777
foseg          0x7fff   32767
fooff          0x9c26f70        163737456
fop            0x55d    1373
xmm0           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm1           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0xa <repeats 16 times>}, v8_int16 = {0xa0a, 0xa0a, 
    0xa0a, 0xa0a, 0xa0a, 0xa0a, 0xa0a, 0xa0a}, v4_int32 = {0xa0a0a0a, 0xa0a0a0a, 0xa0a0a0a, 0xa0a0a0a}, v2_int64 = {0xa0a0a0a0a0a0a0a, 
---Type <return> to continue, or q <return> to quit---
    0xa0a0a0a0a0a0a0a}, uint128 = 0x0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a}
xmm2           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x8000000000000000, 0x0}, v16_int8 = {0x0, 0xff, 0xff, 0xff, 0x0, 0xff, 0xff, 
    0xff, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0xff00, 0xffff, 0xff00, 0xffff, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0xffffff00, 
    0xffffff00, 0x0, 0x0}, v2_int64 = {0xffffff00ffffff00, 0x0}, uint128 = 0x0000000000000000ffffff00ffffff00}
xmm3           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm4           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm5           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm6           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm7           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x20, 0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x0, 0x0, 0x0, 0x20, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x200000, 0x0, 0x0}, v2_int64 = {
    0x20000000000000, 0x0}, uint128 = 0x00000000000000000020000000000000}
xmm8           {v4_float = {0x0, 0x1, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0, 0x38, 0xfa, 0xfe, 0x42, 0x2e, 0xe6, 0x3f, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x3800, 0xfefa, 0x2e42, 0x3fe6, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0xfefa3800, 
    0x3fe62e42, 0x0, 0x0}, v2_int64 = {0x3fe62e42fefa3800, 0x0}, uint128 = 0x00000000000000003fe62e42fefa3800}
xmm9           {v4_float = {0x0, 0x1, 0x0, 0x0}, v2_double = {0x1, 0x0}, v16_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xf0, 0x3f, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x0, 0x0, 0x0, 0x3ff0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x3ff00000, 0x0, 0x0}, 
  v2_int64 = {0x3ff0000000000000, 0x0}, uint128 = 0x00000000000000003ff0000000000000}
xmm10          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm11          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x30, 0x67, 0xc7, 0x93, 0x57, 0xf3, 0x2e, 0x3d, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x6730, 0x93c7, 0xf357, 0x3d2e, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x93c76730, 
    0x3d2ef357, 0x0, 0x0}, v2_int64 = {0x3d2ef35793c76730, 0x0}, uint128 = 0x00000000000000003d2ef35793c76730}
xmm12          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm13          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm14          {v4_float = {0x0, 0x1, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0, 0x38, 0xfa, 0xfe, 0x42, 0x2e, 0xe6, 0x3f, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x3800, 0xfefa, 0x2e42, 0x3fe6, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0xfefa3800, 
    0x3fe62e42, 0x0, 0x0}, v2_int64 = {0x3fe62e42fefa3800, 0x0}, uint128 = 0x00000000000000003fe62e42fefa3800}
xmm15          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x1, 0x0 <repeats 15 times>}, v8_int16 = {0x1, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x1, 0x0, 0x0, 0x0}, v2_int64 = {0x1, 0x0}, uint128 = 0x00000000000000000000000000000001}
mxcsr          0x1fa0   [ PE IM DM ZM OM UM PM ]
(gdb) x/32 0x6010f6c8
0x6010f6c8:     0       0       17012821        336
0x6010f6d8:     7       3991    -1731922304     1
0x6010f6e8:     5094    8656    0       0
0x6010f6f8:     86320   0       96      0
0x6010f708:     0       0       8565    0
0x6010f718:     0       0       1611724576      0
0x6010f728:     1611724576      0       1611724432      0
0x6010f738:     1611724752      0       0       0
(gdb) b kslwtbctx
Breakpoint 2 at 0x93fde48
(gdb) c
Continuing.


Breakpoint 1, 0x00000000093f9818 in kslgetl ()
(gdb) info all-register
rax            0x0      0
rbx            0xc0cc9e0        202164704
rcx            0xfc7    4039
rdx            0x0      0
rsi            0x1      1
rdi            0x6010f628       1611724328
rbp            0x7fff09c26b20   0x7fff09c26b20
rsp            0x7fff09c26b20   0x7fff09c26b20
r8             0xfc7    4039
r9             0xc0cc9e0        202164704
r10            0x0      0
r11            0xfc7    4039
r12            0x600052c0       1610633920
r13            0x6010f628       1611724328
r14            0x1      1
r15            0x1      1
rip            0x93f9818        0x93f9818 <kslgetl+4>
eflags         0x246    [ PF ZF IF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0
st0            0        (raw 0x00000000000000000000)
st1            0        (raw 0x00000000000000000000)
st2            0        (raw 0x00000000000000000000)
st3            0        (raw 0x00000000000000000000)
st4            0        (raw 0x00000000000000000000)
st5            0        (raw 0x00000000000000000000)
st6            105264   (raw 0x400fcd98000000000000)
st7            8156     (raw 0x400bfee0000000000000)
fctrl          0x27f    639
fstat          0x0      0
ftag           0xffff   65535
fiseg          0x0      0
fioff          0x14b99c1        21731777
foseg          0x7fff   32767
fooff          0x9c26f70        163737456
fop            0x55d    1373
xmm0           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm1           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
---Type <return> to continue, or q <return> to quit---
xmm2           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm3           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x8000000000000000, 0x8000000000000000}, v16_int8 = {0xfe, 0xff, 0xff, 0xff, 
    0xff, 0xff, 0xff, 0xff, 0xfe, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}, v8_int16 = {0xfffe, 0xffff, 0xffff, 0xffff, 0xfffe, 0xffff, 
    0xffff, 0xffff}, v4_int32 = {0xfffffffe, 0xffffffff, 0xfffffffe, 0xffffffff}, v2_int64 = {0xfffffffffffffffe, 0xfffffffffffffffe}, 
  uint128 = 0xfffffffffffffffefffffffffffffffe}
xmm4           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm5           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm6           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm7           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x20, 0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x0, 0x0, 0x0, 0x20, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x200000, 0x0, 0x0}, v2_int64 = {
    0x20000000000000, 0x0}, uint128 = 0x00000000000000000020000000000000}
xmm8           {v4_float = {0x0, 0x1, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0, 0x38, 0xfa, 0xfe, 0x42, 0x2e, 0xe6, 0x3f, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x3800, 0xfefa, 0x2e42, 0x3fe6, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0xfefa3800, 
    0x3fe62e42, 0x0, 0x0}, v2_int64 = {0x3fe62e42fefa3800, 0x0}, uint128 = 0x00000000000000003fe62e42fefa3800}
xmm9           {v4_float = {0x0, 0x1, 0x0, 0x0}, v2_double = {0x1, 0x0}, v16_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xf0, 0x3f, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x0, 0x0, 0x0, 0x3ff0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x3ff00000, 0x0, 0x0}, 
  v2_int64 = {0x3ff0000000000000, 0x0}, uint128 = 0x00000000000000003ff0000000000000}
xmm10          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm11          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x30, 0x67, 0xc7, 0x93, 0x57, 0xf3, 0x2e, 0x3d, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x6730, 0x93c7, 0xf357, 0x3d2e, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x93c76730, 
    0x3d2ef357, 0x0, 0x0}, v2_int64 = {0x3d2ef35793c76730, 0x0}, uint128 = 0x00000000000000003d2ef35793c76730}
xmm12          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm13          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm14          {v4_float = {0x0, 0x1, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0, 0x38, 0xfa, 0xfe, 0x42, 0x2e, 0xe6, 0x3f, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x3800, 0xfefa, 0x2e42, 0x3fe6, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0xfefa3800, 
    0x3fe62e42, 0x0, 0x0}, v2_int64 = {0x3fe62e42fefa3800, 0x0}, uint128 = 0x00000000000000003fe62e42fefa3800}
xmm15          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x1, 0x0 <repeats 15 times>}, v8_int16 = {0x1, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x1, 0x0, 0x0, 0x0}, v2_int64 = {0x1, 0x0}, uint128 = 0x00000000000000000000000000000001}
mxcsr          0x1fa0   [ PE IM DM ZM OM UM PM ]
(gdb) x/32 0x6010f628
0x6010f628:     0       0       16702834        336
0x6010f638:     7       3991    -2070483712     1
0x6010f648:     6820    8704    0       0
0x6010f658:     29819   0       124     0
0x6010f668:     0       0       8592    0
0x6010f678:     0       0       1611724416      0
0x6010f688:     1611724416      0       1611724272      0
0x6010f698:     1611724592      0       0       0
(gdb) info breakpoints 
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   0x00000000093f9818 <kslgetl+4>
        breakpoint already hit 2 times
2       breakpoint     keep y   0x00000000093fde48 <kslwtbctx+4>
(gdb) 

rdi/edi是第一个参数， rsi/esi是第二个参数，贴出info all-register命令所显示函数 kslwtbctx() 的两个参数，如下
rsi            0x1      1        ----第二个参数
rdi            0x6010f6c8       1611724488    ----第一个参数

常用的gdb调试命令:

1. file <文件名> :  加载被调试的可执行程序文件.
2. run(简写r也可以): 运行被调试程序,直到遇到断点.
3. c : 继续执行被调试程序,直到下一断点.
4. b : 设置断点.
5. d <编号> : 删除断点.
6. info breakpoints : 显示已设置的断点列表.
7. s : 源码级的单步进入.
8. n : 源码级的单步步过.
9. si/ni : 指令级的单步进入和步过.(需要先运行display /i $pc)
10. info all-register: 显示所有寄存器的值.
11. p /x $eax : 以16进制显示某一特定寄存器的值.
12. q: 退出gdb调试环境.
13. disassemble <0xXXXXXXXX> : 反汇编指定地址处的指令.(加/r选项,显示对应汇编代码的机器码)
14. set disassembly-flavor <intel | att> : 设定汇编显示的格式(Intel格式或者AT&T格式)
15. 修改指定地址的内存数据:set {unsigned char} <0xXXXXXXXX>= <new data>
16. 显示源代码文件: list
17. info stack: 查看栈追踪(栈回溯). 或者使用 backtrace 也可以
18. x /nfu <addr> : 查看指定地址处的内存内容.
     n: 表示个数.   f(format): x,16进制.  d,10进制.   u(每一项的长度): b,单字节. w,4字节. h,双字节.
     例如: x /1xw 0x804a010

2、sqlplus -prelim

如果10G，可以使用sqlplus -prelim选项强制登录
export ORACLE_SID=ora9                              
sqlplus -prelim '/ as sysdba'
oradebug setmypid
oradebug unlimit;
oradebug dump systemstate 10
oradebug tracefile_name


SQL> o radebug setmypid
Statement processed.
SQL> oradebug unlimit
Statement processed.
SQL> oradebug dump systemstate 10
Statement processed.
SQL> oradebug tracefile_name
/u01/app/oracle/admin/emrep/udump/emrep_ora_5112.trc
SQL> oradebug help

3、如果还不行可以使用pstack把堆栈信息取出来，另外也可以使用truss, tusc 之类的工具看看到底在等什么