问题描述：
oracle的alert日志信息如下：
Thu Oct 17 10:17:54 2013
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Using LOG_ARCHIVE_DEST_1 parameter default value as USE_DB_RECOVERY_FILE_DEST
Autotune of undo retention is turned on.
IMODE=BR
ILAT =168
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options.
Using parameter settings in server-side spfile D:\APP\ADMINISTRATOR\PRODUCT\11.2.0\DBHOME_1\DATABASE
\SPFILEORCL.ORA
System parameters with non-default values:
processes = 1000
memory_target = 1648M
control_files = "D:\APP\ADMINISTRATOR\ORADATA\ORCL\CONTROL01.CTL"
control_files = "D:\APP\ADMINISTRATOR\FLASH_RECOVERY_AREA\ORCL\CONTROL02.CTL"
db_block_size = 8192
compatible = "11.2.0.0.0"
db_recovery_file_dest = "D:\app\Administrator\flash_recovery_area"
db_recovery_file_dest_size= 3912M
undo_tablespace = "UNDOTBS1"
remote_login_passwordfile= "EXCLUSIVE"
db_domain = ""
dispatchers = "(PROTOCOL=TCP) (SERVICE=orclXDB)"
local_listener = "LISTENER_ORCL"
audit_file_dest = "D:\APP\ADMINISTRATOR\ADMIN\ORCL\ADUMP"
audit_trail = "DB"
db_name = "orcl"
open_cursors = 5000
diagnostic_dest = "D:\APP\ADMINISTRATOR"
Thu Oct 17 10:17:56 2013
PMON started with pid=2, OS id=3028
Thu Oct 17 10:17:56 2013
VKTM started with pid=3, OS id=284 at elevated priority
VKTM running at (10)millisec precision with DBRM quantum (100)ms
Thu Oct 17 10:17:56 2013
GEN0 started with pid=4, OS id=812
Thu Oct 17 10:17:56 2013
DIAG started with pid=5, OS id=2344
Thu Oct 17 10:17:56 2013
DBRM started with pid=6, OS id=3164
Thu Oct 17 10:17:57 2013
PSP0 started with pid=7, OS id=1804
Thu Oct 17 10:17:57 2013
DIA0 started with pid=8, OS id=2600
Thu Oct 17 10:17:57 2013
MMAN started with pid=9, OS id=3232
Thu Oct 17 10:17:57 2013
DBW0 started with pid=10, OS id=3592
Thu Oct 17 10:17:57 2013
LGWR started with pid=11, OS id=2916
Thu Oct 17 10:17:57 2013
CKPT started with pid=12, OS id=3952
Thu Oct 17 10:17:57 2013
SMON started with pid=13, OS id=3132
Thu Oct 17 10:17:57 2013
RECO started with pid=14, OS id=1472
Thu Oct 17 10:17:57 2013
MMON started with pid=15, OS id=1440
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
Thu Oct 17 10:17:57 2013
MMNL started with pid=16, OS id=1464
starting up 1 shared server(s) ...
ORACLE_BASE from environment = D:\app\Administrator
Thu Oct 17 10:17:57 2013
ALTER DATABASE MOUNT
Successful mount of redo thread 1, with mount id 1356851669
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: ALTER DATABASE MOUNT
Thu Oct 17 10:18:02 2013
ALTER DATABASE OPEN
Thread 1 opened at log sequence 111
Current log# 3 seq# 111 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO03.LOG
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
SMON: enabling cache recovery
Successfully onlined Undo Tablespace 2.
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Database Characterset is AL32UTF8
No Resource Manager plan active
Starting background process QMNC
Thu Oct 17 10:18:07 2013
QMNC started with pid=20, OS id=2360
Completed: ALTER DATABASE OPEN
Thu Oct 17 10:18:13 2013
Starting background process CJQ0
Thu Oct 17 10:18:13 2013
CJQ0 started with pid=28, OS id=3836
Thu Oct 17 10:18:14 2013
db_recovery_file_dest_size of 3912 MB is 0.00% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Thu Oct 17 10:18:53 2013
ALTER SYSTEM SET sessions=1000 SCOPE=SPFILE;
Thu Oct 17 10:25:54 2013
Starting background process SMCO
Thu Oct 17 10:25:54 2013
SMCO started with pid=30, OS id=3440
Thu Oct 17 10:39:02 2013
Thread 1 cannot allocate new log, sequence 112
Private strand flush not complete
Current log# 3 seq# 111 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO03.LOG
Thread 1 advanced to log sequence 112 (LGWR switch)
Current log# 1 seq# 112 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO01.LOG
Thu Oct 17 12:31:18 2013
Thread 1 cannot allocate new log, sequence 113
Private strand flush not complete
Current log# 1 seq# 112 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO01.LOG
Thread 1 advanced to log sequence 113 (LGWR switch)
Current log# 2 seq# 113 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO02.LOG
Thu Oct 17 14:26:47 2013
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_2468.trc (incident=23481):
ORA-00600: internal error code, arguments: [kglLockOwnersListDelete], [0x7FF53EB4C50], [0x7FF46671688], [], [],
[], [], [], [], [], [], []
ORA-03135: connection lost contact
Incident details in: d:\app\administrator\diag\rdbms\orcl\orcl\incident\incdir_23481\orcl_ora_2468_i23481.trc
Thu Oct 17 14:34:10 2013
Trace dumping is performing id=[cdmp_20131017143410]
Thu Oct 17 14:34:10 2013
opidcl aborting process unknown ospid (2468) as a result of ORA-600
Thu Oct 17 14:34:11 2013
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_2892.trc (incident=23121):
ORA-00600: internal error code, arguments: [kglLockOwnersListDelete], [0x7FF53EB4C50], [0x7FF4C5EEA20], [], [],
[], [], [], [], [], [], []
ORA-03135: connection lost contact
Incident details in: d:\app\administrator\diag\rdbms\orcl\orcl\incident\incdir_23121\orcl_ora_2892_i23121.trc
Thu Oct 17 14:34:14 2013
Sweep [inc][23481]: completed
Sweep [inc][23121]: completed
Sweep [inc2][23481]: completed
Thu Oct 17 14:34:35 2013
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_pmon_3028.trc (incident=22817):
ORA-00600: internal error code, arguments: [kglLockOwnersListDelete], [0x7FF53EB5D20], [0x7FF46671788], [], [],
[], [], [], [], [], [], []
Incident details in: d:\app\administrator\diag\rdbms\orcl\orcl\incident\incdir_22817\orcl_pmon_3028_i22817.trc
Thu Oct 17 14:34:37 2013
Trace dumping is performing id=[cdmp_20131017143437]
Thu Oct 17 14:35:13 2013
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_pmon_3028.trc:
ORA-00600: internal error code, arguments: [kglLockOwnersListDelete], [0x7FF53EB5D20], [0x7FF46671788], [], [],
[], [], [], [], [], [], []
PMON (ospid: 3028): terminating the instance due to error 472
Thu Oct 17 14:35:14 2013
opiodr aborting process unknown ospid (3288) as a result of ORA-1092
Thu Oct 17 14:35:14 2013
ORA-1092 : opitsk aborting process
Thu Oct 17 14:35:14 2013
opiodr aborting process unknown ospid (3352) as a result of ORA-1092
Thu Oct 17 14:35:15 2013
ORA-1092 : opitsk aborting process
Thu Oct 17 14:36:26 2013
ORA-00600: 内部错误代码, 参数: [kglLockOwnersListDelete], [0x700000893C0B698], [0x7000008AA8231A0], [], [], [],
[], [], [], [], [],
[]
Wed Aug 07 15:41:50 2013
Errors in file /home/oracle/diag/rdbms/ora920/ora920/trace/ora920_ora_53543522.trc (incident=178409):
ORA-00600: 内部错误代码, 参数: [kglLockOwnersListDelete], [0x700000893C0B698], [0x700000888FAEBC8], [], [], [],
[], [], [], [], [],
[]
Errors in file /home/oracle/diag/rdbms/ora920/ora920/trace/ora920_ora_33358836.trc (incident=171978):
ORA-00600: 内部错误代码, 参数: [kglLockOwnersListDelete], [0x700000893C0B698], [0x7000008B3AF0790], [], [], [],
[], [], [], [], [],
[]
Wed Aug 07 15:41:50 2013
Errors in file /home/oracle/diag/rdbms/ora920/ora920/trace/ora920_ora_49742394.trc (incident=178593):
ORA-00600: 内部错误代码, 参数: [kglLockOwnersListDelete], [0x700000893C0B698], [0x70000089D602B80], [], [], [],
[], [], [], [], [],
[]
Wed Aug 07 15:41:50 2013
DDE: Problem Key 'ORA 600 [kglLockOwnersListDelete]' was completely flood controlled (0x6)
Further messages for this problem key will be suppressed for up to 10 minutes
opidcl aborting process unknown ospid (33292816) as a result of ORA-600
Wed Aug 07 15:41:51 2013
opidcl aborting process unknown ospid (44106606) as a result of ORA-600
Wed Aug 07 15:41:52 2013
opidcl aborting process unknown ospid (62718128) as a result of ORA-600
Wed Aug 07 15:41:52 2013
opidcl aborting process unknown ospid (54657144) as a result of ORA-600
Wed Aug 07 15:41:53 2013
opidcl aborting process unknown ospid (60686498) as a result of ORA-600
Wed Aug 07 15:41:53 2013
opidcl aborting process unknown ospid (49611044) as a result of ORA-600
Wed Aug 07 15:41:56 2013
opidcl aborting process unknown ospid (4260306) as a result of ORA-600
Wed Aug 07 15:42:01 2013
opidcl aborting process unknown ospid (20578824) as a result of ORA-600
Wed Aug 07 15:42:01 2013
二、trace文件部分内容如下
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 – 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /home/oracle/product/11.2.0/db_1
System name: AIX
Node name: db
Release: 1
Version: 6
Machine: 00F723C84C00
Instance name: ora920
Redo thread mounted by this instance: 1
Oracle process number: 1475
Unix process pid: 52101706, image: oracle@db
Dump continued from file: /home/oracle/diag/rdbms/ora920/ora920/trace/ora920_ora_52101706.trc
ORA-00600: 内部错误代码, 参数: [kglLockOwnersListDelete], [0x700000893C0B698], [0x7000008A35A98D0], [], [], [],
[], [], [], [], [], []
========= Dump for incident 178153 (ORA 600 [kglLockOwnersListDelete]) ========
—-- Beginning of Customized Incident Dump(s) —--
LibraryHandle: Address=93c0b698 Hash=0 LockMode=N PinMode=0 LoadLockMode=0 Status=VALD
Name: Namespace=SQL AREA(00) Type=CURSOR(00)
Statistics: InvalidationCount=0 ExecutionCount=124675 LoadCount=1 ActiveLocks=0 TotalLockCount=124776
TotalPinCount=249235
Counters: BrokenCount=1 RevocablePointer=1 KeepDependency=0 KeepHandle=0 BucketInUse=0 HandleInUse=0
Concurrency: DependencyMutex=93c0b748(0, 0, 0, 0) Mutex=9378c650(478, 368071, 242463, 6)
Flags=RON/PIN/PN0/EXP/[10012111]
WaitersLists:
Lock=93c0b728[93c0b728,93c0b728]
Pin=93c0b738[93c0b708,93c0b708]
ReferenceList:
Reference: Address=8a34fb8 Handle=9378c528 Flags=CHL[02]
LibraryObject: Address=8a37668 HeapMask=0000-0001-0001 Flags=EXS[0000] Flags2=[0000] PublicFlags=[0000]
Dependencies: count='2' size='16'
Dependency: num='0'
Table=8a38370 Reference=8a37d58 Handle=93c8d650 Position=29 Flags=DEP[0001]
Dependency: num='1'
Table=8a38370 Reference=8a37e00 Handle=b3c017c8 Position=0 Flags=DEP[0001]
Authorizations: count='1' size='16' entryeize='16'
Accesses: count='1' size='16'
Dependency: num='0' Type=0003
DataBlocks:
Block: #='0' name=CCUR^3522a331 pins=0 Change=NONE
Heap=93da7968 Pointer=8a37750 Extent=8a375e8 Flags=I/-/P/A/-/-
FreedLocation=0 Alloc=3.031250 Size=3.937500 LoadTime=43021714352
Block: #='6' name=SQLA^3522a331 pins=0 Change=NONE
Heap=8a34d88 Pointer=f914470 Extent=f913810 Flags=I/-/-/A/-/E
FreedLocation=0 Alloc=9.750000 Size=11.859375 LoadTime=0
NamespaceDump:
Child Cursor: Heap0=700000708a37750 Heap6=70000060f914470 Heap0 Load Time=08-07-2013 10:02:43 Heap6 Load
Time=08-07-2013 10:02:43 —-- End of Customized Incident Dump(s) —--
*** 2013-08-07 15:20:50.701
dbkedDefDump(): Starting incident default dumps (flags=0x2, level=3, mask=0x0)
—-- SQL Statement (None) —--
Current SQL information unavailable – no cursor.
—-- Call Stack Trace —--
calling call entry argument values in hex
location type point (? means dubious value)
——————-- ——-- ——————-- —————————-
skdstdst()+40 bl 107bf3938 FFFFFFFFFFF9740 ? 000002004 ?
000000001 ? 000000003 ?
000000000 ? 000000002 ?
000000001 ? 000000000 ?
ksedst1()+104 call skdstdst() FFFFFFFFFFF8AE0 ? 000002004 ?
11064CE60 ? 10932D70C ?
11064CE60 ? 000000000 ?
FFFFFFFFFFF8C10 ? 700000007 ?
ksedst()+40 call ksedst1() 3030100154764 ? 002050033 ?
10932D700 ? 7000000000247 ?
000000000 ? 000000000 ?
10932CFB0 ? 000000000 ?
dbkedDefDump()+2832 call ksedst() FFFFFFFFFFF8CC0 ?
FFFFFFFFFFF8EF8 ? 110014D48 ?
1108D3938 ? 1108D3930 ?
110014D48 ?
4F52412036303020 ?
300000003 ?
ksedmp()+76 call dbkedDefDump() 31064CE60 ? 110001128 ?
FFFFFFFFFFF92B0 ? 000000020 ?
100167CF8 ? 000000001 ?
000000000 ? 1105DCC18 ?
ksfdmp()+88 call ksedmp() 000000001 ? 7000008518C7DD8 ?
0FFFF9300 ? 109884838 ?
101FDB540 ? 000000000 ?
110813B78 ? 11064CE60 ?
dbgexPhaseII()+1212 call ksfdmp() 70000080102A180 ? 00000028D ?
FFFFFFFFFFF9370 ?
7000000000435E8 ?
FFFFFFFFFFF93F0 ? 000000008 ?
FFFFFFFFFFF9380 ? 108C13A18 ?
dbgexExplicitEndInc call dbgexPhaseII() 11064CE60 ?
()+616 4F52412D30303630 ?
303A20C4DAB2BFB4 ?
EDCEF3B4FAC2EB2C ?
20B2CECAFD3A205B ?
6B676C4C6F636B4F ?
776E6572734C6973 ?
7444656C6574655D ?
dbgeEndDDEInvocatio call dbgexExplicitEndInc 11064CE60 ? 110813B78 ?
nImpl()+644 () 11064CE60 ?
800000000000F032 ?
FFFFFFFFFFFCD40 ? 11064CE60 ?
1001D4BF4 ? 000000016 ?
dbgeEndDDEInvocatio call dbgeEndDDEInvocatio 11064CE60 ? 110813B78 ?
n()+48 nImpl() FFFFFFFFFFFD880 ? 108A96E58 ?
700000000003658 ? 000000080 ?
7000008A35A98D0 ? 11064CE60 ?
kgllkdl()+1572 call dbgeEndDDEInvocatio 000000008 ? 7000008A35A9AD0 ?
n() 100000001 ? 200000002 ?
000000002 ? 700000893C0B698 ?
000000002 ? 7000008A35A98D0 ?
kgllkds()+136 call kgllkdl() FFFFFFFFFFFDE00 ? 000000000 ?
100000001 ? 000000000 ?
FFFFFFFFFFFDE00 ? 000000000 ?
000000000 ? 110960898 ?
kglUnLock()+368 call kgllkds() 000000000 ? 000000000 ?
FFFFFFFFFFFDE80 ? 110960898 ?
FFFFFFFFFFFDE80 ?
9001000A00A59E0 ? 101BDCBD8 ?
11092AEC0 ?
kxsReleaseLookupLoc call kglUnLock() 101BDE1F4 ? 000006840 ?
k()+152 FFFFFFFFFFFFED0 ?
9FFFFFFF000D4D0 ?
BADC0FFEE0DDF00D ?
000000008 ?
BADC0FFEE0DDF00D ?
000000000 ?
kxsUnlock()+36 call kxsReleaseLookupLoc BADC0FFEE0DDF00D ?
k() 110960898 ? 000000000 ?
000000000 ? 000000000 ?
000000000 ? 000000000 ?
000000000 ?
kxsFreeXsc()+232 call kxsUnlock() 11064CE60 ? 9001000A007FF58 ?
1109610A0 ? 000001A40 ?
11090A6B0 ? 0FFFFFFFF ?
1109271A0 ? 000000C20 ?
kgscFreeCachedCurso call kxsFreeXsc() FFFFFFFFFFFE510 ?
r()+696 7000008BD4889C0 ? 108A96E58 ?
700000000003658 ? 000000000 ?
110014D48 ? 7000008518C7DD8 ?
110014D48 ?
kgscCacheResize()+1 call kgscFreeCachedCurso 000000000 ? 000003000 ?
16 r() 00000000B ? 000008000 ?
FFFFFFFFFFFE580 ? 108A77F30 ?
700000860FE74F8 ?
700000000003658 ?
kgscLogOff()+120 call kgscCacheResize() 000000000 ? 1108F8000 ?
FFFFFFFFFFFE640 ? 110908040 ?
101BD3724 ? 000000000 ?
BADC0FFEE0DDF00D ?
BADC0FFEE0DDF00D ?
kkslof()+60 call kgscLogOff() 000000002 ? 11037FF18 ?
FFFFFFFFFFFE6B0 ?
7000008691168B8 ? 110047180 ?
000000000 ? 000000000 ?
7000008691168B8 ?
opifcs()+196 call kkslof() FFFFFFFFFFFFAA8 ?
800200140000000 ?
FFFFFFFFFFFFED0 ?
9FFFFFFF000D4D0 ?
BADC0FFEE0DDF00D ?
BADC0FFEE0DDF00D ?
BADC0FFEE0DDF00D ?
000000000 ?
ksuxds()+1896 call opifcs() 000000000 ? 4300000043 ?
1108DC460 ? 000000000 ?
100000000000001 ? 1022239C8 ?
FFFFFFFFFFFE610 ? 1105DCC18 ?
ksudel()+116 call ksuxds() 7000008691168B8 ? 200000000 ?
BADC0FFEE0DDF00D ?
BADC0FFEE0DDF00D ?
BADC0FFEE0DDF00D ?
BADC0FFEE0DDF00D ?
110001128 ? 000000000 ?
opidcl()+936 call ksudel() 000000020 ? 9001000A04B1980 ?
FFFFFFFFFFFEC50 ?
FFFFFFFFFFFFA90 ?
FFFFFFFFFFFEC50 ?
800200140000000 ?
FFFFFFFFFFFFED0 ?
9FFFFFFF000D4D0 ?
opidrv()+1572 call opidcl() 110050C10 ? 0686F6D65 ?
FFFFFFFFFFFF4E0 ? 01004FCB0 ?
FFFFFFFFFFFF140 ? 1105DCC18 ?
10720B22C ?
72613932305F6F72 ?
sou2o()+192 call opidrv() 3C06F39554 ? 400000000 ?
FFFFFFFFFFFF4E0 ?
C002201B20000 ? 0000F59E8 ?
000000000 ? 9001000A006F870 ?
1105DCC18 ?
opimai_real()+428 call sou2o() FFFFFFFFFFFF550 ?
BADC0FFEE0DDF00D ?
90000000008936C ?
FFFFFFFFFFFF9A8 ?
BADC0FFEE0DDF00D ?
9001000A007FF58 ?
A0000000A000000 ? 109E20450 ?
ssthrdmain()+324 call opimai_real() FFFFFFFFFFFF600 ?
9001000A04B5068 ?
9000000000CA544 ?
9001000A007FF58 ?
900000000177DA4 ?
9001000A007FF58 ?
657465002E667272 ?
9001000A007FF58 ?
main()+216 call ssthrdmain() 2F0003640 ? FFFFFFFFFFFF9A8 ?
FFFFFFFFFFFFA10 ?
9FFFFFFF000D4E8 ?
9FFFFFFF00009A0 ? 000000000 ?
000000000 ? 9FFFFFFF000D4E8 ?
__start()+112 call main() 000000000 ? 000000000 ?
000000000 ? 000000000 ?
000000000 ? 000000000 ?
000000000 ? 000000000 ?
——————— Binary Stack Dump ———————
三、通过对比STACK TRACE，发现与
kgllkdl kgllkds kglUnLock kxsUnlock kxsFreeXsc kgscFreeCachedCursor
kgscCacheResize kgscCacheCursor kksSessionCache kksCloseCursor 有几分类似，而且故障基本上一致，可以初步确定为
Bug 12628845 : TRACKING BUG TO CHANGE OERI KGLLOCKOWNERSLISTDELETE TO INFORMATIVEEXTERNAL ERROR
如果要完全确定该bug，请log sr
CAUSE
The cause of this problem has been identified in Bug:12628845. It is caused by the code handing active lock
counts on the handle associated with a object used by the process (in this case: a cursor) not being able to
handle the huge amount of locks (due to an overflow) causing the assert to be raised.
Also Bug:11797672 indicates that the error can be raised when not properly closing cursors.
SOLUTION
The issue will be fixed in the upcoming Oracle12c release and has been fixed in the 11.2.0.3 patchset.
There is no workaround for this issue other than to check your code to see whether all cursors opened are also
properly closed.
Apply Patch:12628845 if available for your platform.
Note: the fix created will provide a more informative message on the problem occurring instead of raising an
internal error.
对于该问题，我的建议是：如果短期内不能升级到11203，那打上patch 12628845，如果可以升级，尽量安排升级11201本身bug太多
转自：http://www.xifenfei.com/forum/accident/ora-600%E9%94%99%E8%AF%AF%E8%BF%9B%E7%A8%8B%E4%B8%A2%E5%A4%B1
562
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
