TNS-12514 & TNS-12505 & service_died（2） - 模拟PMON等待latch free

TNS-12514 & TNS-12505 & service_died
http://space.itpub.net/8242091/viewspace-749738

尝试 SUSPEND PMON，虽然能观察到监听日志中显示service_died，但是监听仍然还注册在上面

那么下一步，打算模拟下PMON卡在某个LATCH上时情况

本打算用gdb跟踪下pmon请求过哪些latch，但是可惜测试环境为64bit的，参数的位置不一样，没测试出来，就只有换一种办法
gdb跟踪进程参考：http://space.itpub.net/8242091/viewspace-676943

在gdb跟踪时，发现pmon会在固定间隔请求一个latch，找到该latch，持有它应该就可以模拟出来

[oracle@zhangqiaoc ~]$ ps -ef|grep pmon
oracle   25040     1  0 19:18 ?        00:00:00 ora_pmon_o9208
oracle   25141 24993  0 19:23 pts/6    00:00:00 grep pmon
[oracle@zhangqiaoc ~]$ gdb -p 25040
GNU gdb Red Hat Linux (6.5-37.el5rh)
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Attaching to process 25040
Reading symbols from /u01/app/oracle/product/9.2/db_1/bin/oracle...(no debugging symbols found)...done.
Using host libthread_db library "/lib64/libthread_db.so.1".
Reading symbols from /u01/app/oracle/product/9.2/db_1/lib/libodm9.so...done.
Loaded symbols for /u01/app/oracle/product/9.2/db_1/lib/libodm9.so
Reading symbols from /u01/app/oracle/product/9.2/db_1/lib/libskgxp9.so...done.
Loaded symbols for /u01/app/oracle/product/9.2/db_1/lib/libskgxp9.so
Reading symbols from /u01/app/oracle/product/9.2/db_1/lib/libskgxn9.so...done.
Loaded symbols for /u01/app/oracle/product/9.2/db_1/lib/libskgxn9.so
Reading symbols from /u01/app/oracle/product/9.2/db_1/lib/libjox9.so...done.
Loaded symbols for /u01/app/oracle/product/9.2/db_1/lib/libjox9.so
Reading symbols from /lib64/libdl.so.2...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libm.so.6...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/libpthread.so.0...done.
[Thread debugging using libthread_db enabled]
[New Thread 47321884053952 (LWP 25040)]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libnsl.so.1...done.
Loaded symbols for /lib64/libnsl.so.1
Reading symbols from /lib64/libc.so.6...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libnss_files.so.2...done.
Loaded symbols for /lib64/libnss_files.so.2
0x0000003cfdcc8fdf in poll () from /lib64/libc.so.6
(gdb) b kslgetl
Breakpoint 1 at 0x6789c0
(gdb) c
Continuing.
[Switching to Thread 47321884053952 (LWP 25040)]

Breakpoint 1, 0x00000000006789c0 in kslgetl ()

******************************************************
SQL> select * from v$latchholder;

no rows selected
******************************************************

(gdb) n
Single stepping until exit from function kslgetl,
which has no line number information.
0x0000000000ca01e1 in kmmlod ()

******************************************************
SQL> /

       PID        SID LADDR
---------- ---------- ----------------
NAME
----------------------------------------------------------------
         2          1 0000000050022528
session timer
******************************************************

(gdb) c
Continuing.

Breakpoint 1, 0x00000000006789c0 in kslgetl ()

******************************************************
SQL> /

no rows selected
******************************************************

(gdb) n
Single stepping until exit from function kslgetl,
which has no line number information.
0x0000000000ca01e1 in kmmlod ()

******************************************************
SQL> /

       PID        SID LADDR
---------- ---------- ----------------
NAME
----------------------------------------------------------------
         2          1 0000000050022528
session timer
******************************************************

下面开始测试：

SQL> oradebug setmypid
Statement processed.
SQL>  oradebug call kslgetl 1342317864 1
Function returned 1


-- 这个时候可以看到，PMON卡在了latch free上
SQL> l
  1* select * from v$session_wait
SQL> /

       SID       SEQ#
---------- ----------
EVENT
----------------------------------------------------------------
P1TEXT                                                                   P1
---------------------------------------------------------------- ----------
P1RAW
----------------
P2TEXT                                                                   P2
---------------------------------------------------------------- ----------
P2RAW
----------------
P3TEXT                                                                   P3
---------------------------------------------------------------- ----------
P3RAW             WAIT_TIME SECONDS_IN_WAIT STATE
---------------- ---------- --------------- -------------------
         1        113
latch free
address                                                          1342317864
0000000050022528
number                                                                  138
000000000000008A
tries                                                                     0
00                       -1              27 WAITED KNOWN TIME


-- 监听日志也有报错
12-DEC-2012 19:29:32 * service_died * o9208 * 12537

-- 再看动态监听已经没有了
[oracle@zhangqiaoc ~]$ lsnrctl status

LSNRCTL for Linux: Version 11.2.0.3.0 - Production on 12-DEC-2012 19:29:56

Copyright (c) 1991, 2011, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=0.0.0.0)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 9.2.0.8.0 - Production
Start Date                12-DEC-2012 15:38:02
Uptime                    0 days 3 hr. 51 min. 53 sec
Trace Level               off
Security                  OFF
SNMP                      OFF
Listener Parameter File   /u01/app/oracle/product/9.2/db_1/network/admin/listener.ora
Listener Log File         /u01/app/oracle/product/9.2/db_1/network/log/listener.log
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=zhangqiaoc)(PORT=1521)))
The listener supports no services
The command completed successfully
[oracle@zhangqiaoc ~]$

-- 当前没没法测试释放后是否能自动注册回去
-- 因为我测试了个PMON经常请求的latch，如session timer，sort extent pool，释放后PMON都会由于报错而宕库
-- 9i PMON和latch的BUG本来就常有发生，9208都遇到过

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/8242091/viewspace-750993/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/8242091/viewspace-750993/