ORA-15064: communication failure with ASM instance

最新推荐文章于 2021-08-10 21:47:17 发布

deanza

最新推荐文章于 2021-08-10 21:47:17 发布

阅读量8k

点赞数

分类专栏： Oracle ASM Storage ORA-xxx Oracle 10G RAC Oracle Production DB 文章标签： disk cache path domain file thread

本文链接：https://blog.csdn.net/deanza/article/details/7947798

版权

ORA-xxx 同时被 3 个专栏收录

24 篇文章 1 订阅

订阅专栏

Oracle Production DB

22 篇文章 0 订阅

订阅专栏

Oracle 10G RAC

21 篇文章 0 订阅

订阅专栏

alert_ASM2.log

Wed Sep 05 03:32:56 2012
IPC Send timeout detected.Sender: ospid 13720
Receiver: inst 1 binc 22497 ospid 1431
IPC Send timeout to 1.1 inc 56 for msg type 65518 from opid 10
Wed Sep 05 03:32:58 2012
Communications reconfiguration: instance_number 1
Wed Sep 05 03:32:58 2012
Trace dumping is performing id=[cdmp_20120905033258]
Waiting for clusterware split-brain resolution
Wed Sep 05 03:43:00 2012
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_lmon_13704.trc  (incident=40264):
ORA-29740: evicted by member 1, group incarnation 58
Incident details in: /u01/app/oracle/diag/asm/+asm/+ASM2/incident/incdir_40264/+ASM2_lmon_13704_i40264.trc
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_lmon_13704.trc:
ORA-29740: evicted by member 1, group incarnation 58
LMON (ospid: 13704): terminating the instance due to error 29740
Wed Sep 05 03:43:01 2012
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_lms0_13720.trc  (incident=40280):
ORA-29740: evicted by member , group incarnation
Incident details in: /u01/app/oracle/diag/asm/+asm/+ASM2/incident/incdir_40280/+ASM2_lms0_13720_i40280.trc
Wed Sep 05 03:43:01 2012
System state dump is made for local instance
System State dumped to trace file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_diag_13682.trc
Wed Sep 05 03:43:01 2012
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_mman_13724.trc  (incident=40288):
ORA-29740: evicted by member , group incarnation
Incident details in: /u01/app/oracle/diag/asm/+asm/+ASM2/incident/incdir_40288/+ASM2_mman_13724_i40288.trc
Wed Sep 05 03:43:02 2012
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_smon_13732.trc  (incident=40320):
ORA-29740: evicted by member , group incarnation
Incident details in: /u01/app/oracle/diag/asm/+asm/+ASM2/incident/incdir_40320/+ASM2_smon_13732_i40320.trc
Wed Sep 05 03:43:02 2012
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_gmon_13736.trc  (incident=40336):
ORA-29740: evicted by member , group incarnation
Incident details in: /u01/app/oracle/diag/asm/+asm/+ASM2/incident/incdir_40336/+ASM2_gmon_13736_i40336.trc
Wed Sep 05 03:43:02 2012


***********************************************************************

Fatal NI connect error 12537, connecting to:
 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))

  VERSION INFORMATION:
        TNS for Solaris: Version 11.1.0.7.0 - Production
        Oracle Bequeath NT Protocol Adapter for Solaris: Version 11.1.0.7.0 - Production
  Time: 05-SEP-2012 03:43:02
  Tracing not turned on.
  Tns error struct:
    ns main err code: 12537
    
TNS-12537: TNS:connection closed
    ns secondary err code: 12560
    nt main err code: 0
    nt secondary err code: 0
    nt OS err code: 0
Wed Sep 05 03:43:02 2012
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_rbal_13734.trc  (incident=40328):
ORA-29740: evicted by member , group incarnation
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_rbal_13734.trc  (incident=40329):
ORA-29740: evicted by member , group incarnation
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_rbal_13734.trc  (incident=40330):
ORA-29740: evicted by member , group incarnation
ORA-29740: evicted by member , group incarnation
ERROR: ORA-29740 thrown in RBAL for group number 1
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_rbal_13734.trc:
ORA-29740: evicted by member , group incarnation
ORA-29740: evicted by member , group incarnation
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_rbal_13734.trc:
ORA-29740: evicted by member , group incarnation
ORA-29740: evicted by member , group incarnation
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_rbal_13734.trc  (incident=40331):
ORA-29740: evicted by member , group incarnation
Trace dumping is performing id=[cdmp_20120905034301]
Wed Sep 05 03:43:06 2012
ORA-1092 : opitsk aborting process
Wed Sep 05 03:43:06 2012
License high water mark = 16
Instance terminated by LMON, pid = 13704
USER (ospid: 16961): terminating the instance
Instance terminated by USER, pid = 16961
Wed Sep 05 03:43:12 2012
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Using LOG_ARCHIVE_DEST_1 parameter default value as /u01/app/oracle/product/11.1.0/asm/dbs/arch
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up ORACLE RDBMS Version: 11.1.0.7.0.
Using parameter settings in server-side pfile /u01/app/oracle/product/11.1.0/asm/dbs/init+ASM2.ora
System parameters with non-default values:
  shared_pool_size         = 200M
  large_pool_size          = 12M
  instance_type            = "asm"
  cluster_interconnects    = "192.168.1.4"
  memory_target            = 512M
  memory_max_target        = 1G
  cluster_database         = TRUE
  instance_number          = 2
  asm_diskgroups           = "DATA_DG1"
  asm_diskgroups           = "DATA_DG2"
  asm_diskgroups           = "DATA_DG3"
  asm_diskgroups           = "BACKUP_DG1"
  asm_diskgroups           = "BACKUP_DG2"
  diagnostic_dest          = "/u01/app/oracle"
Cluster communication is configured to use the following interface(s) for this instance
  192.168.1.4
cluster interconnect IPC version:Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
Wed Sep 05 03:43:13 2012
PMON started with pid=2, OS id=17064
Wed Sep 05 03:43:13 2012
VKTM started with pid=3, OS id=17073 at elevated priority
VKTM running at (20)ms precision
Wed Sep 05 03:43:13 2012
DIAG started with pid=4, OS id=17078
Wed Sep 05 03:43:13 2012
PING started with pid=5, OS id=17083
Wed Sep 05 03:43:13 2012
PSP0 started with pid=6, OS id=17085
Wed Sep 05 03:43:13 2012
DIA0 started with pid=7, OS id=17088
Wed Sep 05 03:43:13 2012
LMON started with pid=8, OS id=17091
Wed Sep 05 03:43:13 2012
LMD0 started with pid=9, OS id=17093
Wed Sep 05 03:43:13 2012
LMS0 started with pid=10, OS id=17100 at elevated priority
Wed Sep 05 03:43:13 2012
MMAN started with pid=11, OS id=17104
Wed Sep 05 03:43:13 2012
DBW0 started with pid=12, OS id=17109
Wed Sep 05 03:43:13 2012
LGWR started with pid=13, OS id=17112
Wed Sep 05 03:43:13 2012
CKPT started with pid=14, OS id=17114
Wed Sep 05 03:43:13 2012
SMON started with pid=15, OS id=17116
Wed Sep 05 03:43:13 2012
RBAL started with pid=16, OS id=17119
Wed Sep 05 03:43:13 2012
GMON started with pid=17, OS id=17121
DISM started, OS id=17123
lmon registered with NM - instance id 2 (internal mem no 1)
Reconfiguration started (old inc 0, new inc 62)
ASM instance
List of nodes:
 0 1
 Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
 Communication channels reestablished
 * domain 0 valid = 1 according to instance 0
* allocate domain 1, invalid = TRUE
 * domain 1 valid = 1 according to instance 0
* allocate domain 2, invalid = TRUE
 * domain 2 valid = 1 according to instance 0
* allocate domain 3, invalid = TRUE
 * domain 3 valid = 1 according to instance 0
* allocate domain 4, invalid = TRUE
 * domain 4 valid = 1 according to instance 0
* allocate domain 5, invalid = TRUE
 * domain 5 valid = 1 according to instance 0
 Master broadcasted resource hash value bitmaps
 Non-local Process blocks cleaned out
 LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
 Set master node info
 Submitted all remote-enqueue requests
 Dwn-cvts replayed, VALBLKs dubious
 All grantable enqueues granted
 LMS 0: 0 GCS shadows traversed, 0 replayed
 Submitted all GCS remote-cache requests
 Post SMON to start 1st pass IR
 Fix write in gcs resources
Reconfiguration complete
Wed Sep 05 03:43:14 2012
LCK0 started with pid=18, OS id=17151
ORACLE_BASE from environment = /u01/app/oracle
Wed Sep 05 03:43:15 2012
SQL> ALTER DISKGROUP ALL MOUNT
NOTE: cache registered group BACKUP_DG1 number=1 incarn=0xbcfb9c9b
NOTE: cache began mount (not first) of group BACKUP_DG1 number=1 incarn=0xbcfb9c9b
NOTE: cache registered group BACKUP_DG2 number=2 incarn=0xbd0b9c9c
NOTE: cache began mount (not first) of group BACKUP_DG2 number=2 incarn=0xbd0b9c9c
NOTE: cache registered group DATA_DG1 number=3 incarn=0xbd0b9c9d
NOTE: cache began mount (not first) of group DATA_DG1 number=3 incarn=0xbd0b9c9d
NOTE: cache registered group DATA_DG2 number=4 incarn=0xbd0b9c9e
NOTE: cache began mount (not first) of group DATA_DG2 number=4 incarn=0xbd0b9c9e
NOTE: cache registered group DATA_DG3 number=5 incarn=0xbd0b9c9f
NOTE: cache began mount (not first) of group DATA_DG3 number=5 incarn=0xbd0b9c9f
WARNING::ASMLIB library not found. See trace file for details.
NOTE: Assigning number (1,3) to disk (/dev/rdsk/c3t6006016029012200ACC05F9D295CDE11d0s6)
NOTE: Assigning number (1,2) to disk (/dev/rdsk/c3t6006016029012200ABC05F9D295CDE11d0s6)
NOTE: Assigning number (1,1) to disk (/dev/rdsk/c3t6006016029012200AAC05F9D295CDE11d0s6)
NOTE: Assigning number (1,7) to disk (/dev/rdsk/c3t600601602901220061CD9F90295CDE11d0s6)
NOTE: Assigning number (1,6) to disk (/dev/rdsk/c3t60060160290122005C4A5D75295CDE11d0s6)
NOTE: Assigning number (1,5) to disk (/dev/rdsk/c3t60060160290122005B4A5D75295CDE11d0s6)
NOTE: Assigning number (1,0) to disk (/dev/rdsk/c3t60060160290122005A4A5D75295CDE11d0s6)
NOTE: Assigning number (1,4) to disk (/dev/rdsk/c3t60060160290122004AD4AF6E295CDE11d0s6)
NOTE: Assigning number (2,5) to disk (/dev/rdsk/c3t60060160290122004DFED3A3295CDE11d0s6)
NOTE: Assigning number (2,13) to disk (/dev/rdsk/c3t6006016029012200ED34DD96295CDE11d0s6)
NOTE: Assigning number (2,12) to disk (/dev/rdsk/c3t6006016029012200EC34DD96295CDE11d0s6)
NOTE: Assigning number (2,2) to disk (/dev/rdsk/c3t6006016029012200D2E98A82295CDE11d0s6)
NOTE: Assigning number (2,1) to disk (/dev/rdsk/c3t6006016029012200D1E98A82295CDE11d0s6)
NOTE: Assigning number (2,0) to disk (/dev/rdsk/c3t6006016029012200D0E98A82295CDE11d0s6)
NOTE: Assigning number (2,11) to disk (/dev/rdsk/c3t6006016029012200EAE8F27B295CDE11d0s6)
NOTE: Assigning number (2,7) to disk (/dev/rdsk/c3t6006016029012200E9E8F27B295CDE11d0s6)
NOTE: Assigning number (2,6) to disk (/dev/rdsk/c3t6006016029012200E8E8F27B295CDE11d0s6)
NOTE: Assigning number (2,4) to disk (/dev/rdsk/c3t600601602901220049D4AF6E295CDE11d0s6)
NOTE: Assigning number (2,3) to disk (/dev/rdsk/c3t600601602901220048D4AF6E295CDE11d0s6)
NOTE: Assigning number (2,8) to disk (/dev/rdsk/c3t600601602901220056E36868295CDE11d0s6)
NOTE: Assigning number (2,9) to disk (/dev/rdsk/c3t600601602901220055E36868295CDE11d0s6)
NOTE: Assigning number (2,10) to disk (/dev/rdsk/c3t600601602901220054E36868295CDE11d0s6)
NOTE: Assigning number (3,3) to disk (/dev/rdsk/c3t6006016029012200AC30026A5A5BDE11d0s6)
NOTE: Assigning number (3,2) to disk (/dev/rdsk/c3t6006016029012200AB30026A5A5BDE11d0s6)
NOTE: Assigning number (3,1) to disk (/dev/rdsk/c3t6006016029012200AA30026A5A5BDE11d0s6)
NOTE: Assigning number (3,11) to disk (/dev/rdsk/c3t6006016029012200EE774E625A5BDE11d0s6)
NOTE: Assigning number (3,12) to disk (/dev/rdsk/c3t6006016029012200ED774E625A5BDE11d0s6)
NOTE: Assigning number (3,15) to disk (/dev/rdsk/c3t6006016029012200EC774E625A5BDE11d0s6)
NOTE: Assigning number (3,8) to disk (/dev/rdsk/c3t60060160290122008B73502D5A5BDE11d0s6)
NOTE: Assigning number (3,0) to disk (/dev/rdsk/c3t60060160290122008A73502D5A5BDE11d0s6)
NOTE: Assigning number (3,14) to disk (/dev/rdsk/c3t60060160290122002D9576255A5BDE11d0s6)
NOTE: Assigning number (3,9) to disk (/dev/rdsk/c3t60060160290122002B9576255A5BDE11d0s6)
NOTE: Assigning number (3,5) to disk (/dev/rdsk/c3t600601602901220038040AF9595BDE11d0s6)
NOTE: Assigning number (3,7) to disk (/dev/rdsk/c3t600601602901220037040AF9595BDE11d0s6)
NOTE: Assigning number (3,6) to disk (/dev/rdsk/c3t600601602901220036040AF9595BDE11d0s6)
NOTE: Assigning number (3,13) to disk (/dev/rdsk/c3t60060160290122002DD902F1595BDE11d0s6)
NOTE: Assigning number (3,10) to disk (/dev/rdsk/c3t60060160290122002BD902F1595BDE11d0s6)
NOTE: Assigning number (3,4) to disk (/dev/rdsk/c3t60060160290122002AD902F1595BDE11d0s6)
NOTE: Assigning number (4,2) to disk (/dev/rdsk/c3t60060160290122008973502D5A5BDE11d0s6)
NOTE: Assigning number (4,3) to disk (/dev/rdsk/c3t60060160290122008873502D5A5BDE11d0s6)
NOTE: Assigning number (4,0) to disk (/dev/rdsk/c3t60060160290122002C9576255A5BDE11d0s6)
NOTE: Assigning number (4,1) to disk (/dev/rdsk/c3t60060160290122002CD902F1595BDE11d0s6)
NOTE: Assigning number (5,6) to disk (/dev/rdsk/c3t60060160290122004EFED3A3295CDE11d0s6)
NOTE: Assigning number (5,0) to disk (/dev/rdsk/c3t60060160290122004CFED3A3295CDE11d0s6)
NOTE: Assigning number (5,7) to disk (/dev/rdsk/c3t6006016029012200EE34DD96295CDE11d0s6)
NOTE: Assigning number (5,2) to disk (/dev/rdsk/c3t600601602901220062CD9F90295CDE11d0s6)
NOTE: Assigning number (5,1) to disk (/dev/rdsk/c3t600601602901220060CD9F90295CDE11d0s6)
NOTE: Assigning number (5,3) to disk (/dev/rdsk/c3t6006016029012200F6F4EF89295CDE11d0s6)
NOTE: Assigning number (5,4) to disk (/dev/rdsk/c3t6006016029012200F5F4EF89295CDE11d0s6)
NOTE: Assigning number (5,5) to disk (/dev/rdsk/c3t6006016029012200F4F4EF89295CDE11d0s6)
kfdp_query(): 6
kfdp_queryBg(): 6
NOTE: cache opening disk 0 of grp 1: BACKUP_DG1_0000 path:/dev/rdsk/c3t60060160290122005A4A5D75295CDE11d0s6
NOTE: F1X0 found on disk 0 fcn 0.258
NOTE: cache opening disk 1 of grp 1: BACKUP_DG1_0001 path:/dev/rdsk/c3t6006016029012200AAC05F9D295CDE11d0s6
NOTE: cache opening disk 2 of grp 1: BACKUP_DG1_0002 path:/dev/rdsk/c3t6006016029012200ABC05F9D295CDE11d0s6
NOTE: cache opening disk 3 of grp 1: BACKUP_DG1_0003 path:/dev/rdsk/c3t6006016029012200ACC05F9D295CDE11d0s6
NOTE: cache opening disk 4 of grp 1: BACKUP_DG1_0004 path:/dev/rdsk/c3t60060160290122004AD4AF6E295CDE11d0s6
NOTE: cache opening disk 5 of grp 1: BACKUP_DG1_0005 path:/dev/rdsk/c3t60060160290122005B4A5D75295CDE11d0s6
NOTE: cache opening disk 6 of grp 1: BACKUP_DG1_0006 path:/dev/rdsk/c3t60060160290122005C4A5D75295CDE11d0s6
NOTE: cache opening disk 7 of grp 1: BACKUP_DG1_0007 path:/dev/rdsk/c3t600601602901220061CD9F90295CDE11d0s6
NOTE: cache mounting (not first) group 1/0xBCFB9C9B (BACKUP_DG1)
kjbdomatt send to node 0
NOTE: attached to recovery domain 1
NOTE: LGWR attempting to mount thread 1 for diskgroup 1
NOTE: LGWR mounted thread 1 for disk group 1
NOTE: opening chunk 1 at fcn 0.703535333 ABA
NOTE: seq=534 blk=2990
NOTE: cache mounting group 1/0xBCFB9C9B (BACKUP_DG1) succeeded
NOTE: cache ending mount (success) of group BACKUP_DG1 number=1 incarn=0xbcfb9c9b
kfdp_query(): 7
kfdp_queryBg(): 7
NOTE: cache opening disk 0 of grp 2: BACKUP_DG2_0000 path:/dev/rdsk/c3t6006016029012200D0E98A82295CDE11d0s6
NOTE: F1X0 found on disk 0 fcn 0.169
NOTE: cache opening disk 1 of grp 2: BACKUP_DG2_0001 path:/dev/rdsk/c3t6006016029012200D1E98A82295CDE11d0s6
NOTE: cache opening disk 2 of grp 2: BACKUP_DG2_0002 path:/dev/rdsk/c3t6006016029012200D2E98A82295CDE11d0s6
NOTE: cache opening disk 3 of grp 2: BACKUP_DG2_0003 path:/dev/rdsk/c3t600601602901220048D4AF6E295CDE11d0s6
NOTE: cache opening disk 4 of grp 2: BACKUP_DG2_0004 path:/dev/rdsk/c3t600601602901220049D4AF6E295CDE11d0s6
NOTE: cache opening disk 5 of grp 2: BACKUP_DG2_0005 path:/dev/rdsk/c3t60060160290122004DFED3A3295CDE11d0s6
NOTE: cache opening disk 6 of grp 2: BACKUP_DG2_0006 path:/dev/rdsk/c3t6006016029012200E8E8F27B295CDE11d0s6
NOTE: cache opening disk 7 of grp 2: BACKUP_DG2_0007 path:/dev/rdsk/c3t6006016029012200E9E8F27B295CDE11d0s6
NOTE: cache opening disk 8 of grp 2: BACKUP_DG2_0008 path:/dev/rdsk/c3t600601602901220056E36868295CDE11d0s6
NOTE: cache opening disk 9 of grp 2: BACKUP_DG2_0009 path:/dev/rdsk/c3t600601602901220055E36868295CDE11d0s6
NOTE: cache opening disk 10 of grp 2: BACKUP_DG2_0010 path:/dev/rdsk/c3t600601602901220054E36868295CDE11d0s6
NOTE: cache opening disk 11 of grp 2: BACKUP_DG2_0011 path:/dev/rdsk/c3t6006016029012200EAE8F27B295CDE11d0s6
NOTE: cache opening disk 12 of grp 2: BACKUP_DG2_0012 path:/dev/rdsk/c3t6006016029012200EC34DD96295CDE11d0s6
NOTE: cache opening disk 13 of grp 2: BACKUP_DG2_0013 path:/dev/rdsk/c3t6006016029012200ED34DD96295CDE11d0s6
NOTE: cache mounting (not first) group 2/0xBD0B9C9C (BACKUP_DG2)
kjbdomatt send to node 0
NOTE: attached to recovery domain 2
NOTE: LGWR attempting to mount thread 1 for diskgroup 2
NOTE: LGWR mounted thread 1 for disk group 2
NOTE: opening chunk 1 at fcn 0.544141956 ABA
NOTE: seq=685 blk=8752
NOTE: cache mounting group 2/0xBD0B9C9C (BACKUP_DG2) succeeded
NOTE: cache ending mount (success) of group BACKUP_DG2 number=2 incarn=0xbd0b9c9c
kfdp_query(): 8
kfdp_queryBg(): 8
NOTE: cache opening disk 0 of grp 3: DATA_DG1_0000 path:/dev/rdsk/c3t60060160290122008A73502D5A5BDE11d0s6
NOTE: F1X0 found on disk 0 fcn 0.152
NOTE: cache opening disk 1 of grp 3: DATA_DG1_0001 path:/dev/rdsk/c3t6006016029012200AA30026A5A5BDE11d0s6
NOTE: cache opening disk 2 of grp 3: DATA_DG1_0002 path:/dev/rdsk/c3t6006016029012200AB30026A5A5BDE11d0s6
NOTE: cache opening disk 3 of grp 3: DATA_DG1_0003 path:/dev/rdsk/c3t6006016029012200AC30026A5A5BDE11d0s6
NOTE: cache opening disk 4 of grp 3: DATA_DG1_0004 path:/dev/rdsk/c3t60060160290122002AD902F1595BDE11d0s6
NOTE: cache opening disk 5 of grp 3: DATA_DG1_0005 path:/dev/rdsk/c3t600601602901220038040AF9595BDE11d0s6
NOTE: cache opening disk 6 of grp 3: DATA_DG1_0006 path:/dev/rdsk/c3t600601602901220036040AF9595BDE11d0s6
NOTE: cache opening disk 7 of grp 3: DATA_DG1_0007 path:/dev/rdsk/c3t600601602901220037040AF9595BDE11d0s6
NOTE: cache opening disk 8 of grp 3: DATA_DG1_0008 path:/dev/rdsk/c3t60060160290122008B73502D5A5BDE11d0s6
NOTE: cache opening disk 9 of grp 3: DATA_DG1_0009 path:/dev/rdsk/c3t60060160290122002B9576255A5BDE11d0s6
NOTE: cache opening disk 10 of grp 3: DATA_DG1_0010 path:/dev/rdsk/c3t60060160290122002BD902F1595BDE11d0s6
NOTE: cache opening disk 11 of grp 3: DATA_DG1_0011 path:/dev/rdsk/c3t6006016029012200EE774E625A5BDE11d0s6
NOTE: cache opening disk 12 of grp 3: DATA_DG1_0012 path:/dev/rdsk/c3t6006016029012200ED774E625A5BDE11d0s6
NOTE: cache opening disk 13 of grp 3: DATA_DG1_0013 path:/dev/rdsk/c3t60060160290122002DD902F1595BDE11d0s6
NOTE: cache opening disk 14 of grp 3: DATA_DG1_0014 path:/dev/rdsk/c3t60060160290122002D9576255A5BDE11d0s6
NOTE: cache opening disk 15 of grp 3: DATA_DG1_0015 path:/dev/rdsk/c3t6006016029012200EC774E625A5BDE11d0s6
NOTE: cache mounting (not first) group 3/0xBD0B9C9D (DATA_DG1)
kjbdomatt send to node 0
NOTE: attached to recovery domain 3
NOTE: LGWR attempting to mount thread 1 for diskgroup 3
NOTE: LGWR mounted thread 1 for disk group 3
NOTE: opening chunk 1 at fcn 0.2874008 ABA
NOTE: seq=43 blk=5509
NOTE: cache mounting group 3/0xBD0B9C9D (DATA_DG1) succeeded
NOTE: cache ending mount (success) of group DATA_DG1 number=3 incarn=0xbd0b9c9d
kfdp_query(): 9
kfdp_queryBg(): 9
NOTE: cache opening disk 0 of grp 4: DATA_DG2_0000 path:/dev/rdsk/c3t60060160290122002C9576255A5BDE11d0s6
NOTE: F1X0 found on disk 0 fcn 0.746372
NOTE: cache opening disk 1 of grp 4: DATA_DG2_0001 path:/dev/rdsk/c3t60060160290122002CD902F1595BDE11d0s6
NOTE: cache opening disk 2 of grp 4: DATA_DG2_0002 path:/dev/rdsk/c3t60060160290122008973502D5A5BDE11d0s6
NOTE: cache opening disk 3 of grp 4: DATA_DG2_0003 path:/dev/rdsk/c3t60060160290122008873502D5A5BDE11d0s6
NOTE: cache mounting (not first) group 4/0xBD0B9C9E (DATA_DG2)
kjbdomatt send to node 0
NOTE: attached to recovery domain 4
NOTE: LGWR attempting to mount thread 1 for diskgroup 4
NOTE: LGWR mounted thread 1 for disk group 4
NOTE: opening chunk 1 at fcn 0.915102 ABA
NOTE: seq=31 blk=7869
NOTE: cache mounting group 4/0xBD0B9C9E (DATA_DG2) succeeded
NOTE: cache ending mount (success) of group DATA_DG2 number=4 incarn=0xbd0b9c9e
kfdp_query(): 10
kfdp_queryBg(): 10
NOTE: cache opening disk 0 of grp 5: DATA_DG3_0000 path:/dev/rdsk/c3t60060160290122004CFED3A3295CDE11d0s6
NOTE: F1X0 found on disk 0 fcn 0.0
NOTE: cache opening disk 1 of grp 5: DATA_DG3_0001 path:/dev/rdsk/c3t600601602901220060CD9F90295CDE11d0s6
NOTE: cache opening disk 2 of grp 5: DATA_DG3_0002 path:/dev/rdsk/c3t600601602901220062CD9F90295CDE11d0s6
NOTE: cache opening disk 3 of grp 5: DATA_DG3_0003 path:/dev/rdsk/c3t6006016029012200F6F4EF89295CDE11d0s6
NOTE: cache opening disk 4 of grp 5: DATA_DG3_0004 path:/dev/rdsk/c3t6006016029012200F5F4EF89295CDE11d0s6
NOTE: cache opening disk 5 of grp 5: DATA_DG3_0005 path:/dev/rdsk/c3t6006016029012200F4F4EF89295CDE11d0s6
NOTE: cache opening disk 6 of grp 5: DATA_DG3_0006 path:/dev/rdsk/c3t60060160290122004EFED3A3295CDE11d0s6
NOTE: cache opening disk 7 of grp 5: DATA_DG3_0007 path:/dev/rdsk/c3t6006016029012200EE34DD96295CDE11d0s6
NOTE: cache mounting (not first) group 5/0xBD0B9C9F (DATA_DG3)
kjbdomatt send to node 0
NOTE: attached to recovery domain 5
NOTE: LGWR attempting to mount thread 1 for diskgroup 5
NOTE: LGWR mounted thread 1 for disk group 5
NOTE: opening chunk 1 at fcn 0.1061356 ABA
NOTE: seq=37 blk=10556
NOTE: cache mounting group 5/0xBD0B9C9F (DATA_DG3) succeeded
NOTE: cache ending mount (success) of group DATA_DG3 number=5 incarn=0xbd0b9c9f
kfdp_query(): 11
kfdp_queryBg(): 11
NOTE: Instance updated compatible.asm to 10.1.0.0.0 for grp 1
SUCCESS: diskgroup BACKUP_DG1 was mounted
kfdp_query(): 12
kfdp_queryBg(): 12
NOTE: Instance updated compatible.asm to 10.1.0.0.0 for grp 2
SUCCESS: diskgroup BACKUP_DG2 was mounted
kfdp_query(): 13
kfdp_queryBg(): 13
NOTE: Instance updated compatible.asm to 10.1.0.0.0 for grp 3
SUCCESS: diskgroup DATA_DG1 was mounted
kfdp_query(): 14
kfdp_queryBg(): 14
NOTE: Instance updated compatible.asm to 10.1.0.0.0 for grp 4
SUCCESS: diskgroup DATA_DG2 was mounted
kfdp_query(): 15
kfdp_queryBg(): 15
NOTE: Instance updated compatible.asm to 10.1.0.0.0 for grp 5
SUCCESS: diskgroup DATA_DG3 was mounted
SUCCESS: ALTER DISKGROUP ALL MOUNT
Wed Sep 05 03:43:31 2012
Starting background process ASMB
Wed Sep 05 03:43:31 2012
ASMB started with pid=21, OS id=17551
NOTE: ASMB process exiting due to lack of ASM file activity for 5 seconds

alert_PFRM_A2.log

Wed Sep  5 01:06:39 2012
ALTER SYSTEM ARCHIVE LOG
Wed Sep  5 01:06:39 2012
Thread 2 advanced to log sequence 9994 (LGWR switch)
  Current log# 4 seq# 9994 mem# 0: +BACKUP_DG2/pfrm_a/onlinelog/group_4.266.692725317
  Current log# 4 seq# 9994 mem# 1: +DATA_DG3/pfrm_a/onlinelog/group_4.371.716207107
Wed Sep  5 01:06:43 2012
ALTER SYSTEM ARCHIVE LOG
Wed Sep  5 01:06:46 2012
Thread 2 advanced to log sequence 9995 (LGWR switch)
  Current log# 3 seq# 9995 mem# 0: +BACKUP_DG2/pfrm_a/onlinelog/group_3.265.692725317
  Current log# 3 seq# 9995 mem# 1: +DATA_DG3/pfrm_a/onlinelog/group_3.370.716207101
Wed Sep  5 01:07:00 2012
Starting control autobackup
Control autobackup written to DISK device
        handle '+BACKUP_DG2/pfrm_a/autobackup/2012_09_05/s_793156020.4229.793156023'
Wed Sep  5 03:43:02 2012
Errors in file /u01/app/oracle/product/10.2.0/db/admin/PFRM_A/bdump/pfrm_a2_asmb_14034.trc:
ORA-15064: communication failure with ASM instance
ORA-03113: end-of-file on communication channel
Wed Sep  5 03:43:02 2012
ASMB: terminating instance due to error 15064
Wed Sep  5 03:43:02 2012
System state dump is made for local instance
System State dumped to trace file /u01/app/oracle/product/10.2.0/db/admin/PFRM_A/bdump/pfrm_a2_diag_13894.trc
Wed Sep  5 03:43:05 2012
Shutting down instance (abort)
License high water mark = 22
Wed Sep  5 03:43:06 2012
Trace dumping is performing id=[cdmp_20120905034302]
Wed Sep  5 03:43:07 2012
Instance terminated by ASMB, pid = 14034
Wed Sep  5 03:43:10 2012
Instance terminated by USER, pid = 16926

alert_PFRM_C2.log

Wed Sep  5 03:43:02 2012
Errors in file /u01/app/oracle/product/10.2.0/db/admin/PFRM_C/bdump/pfrm_c2_asmb_14061.trc:
ORA-15064: communication failure with ASM instance
ORA-03113: end-of-file on communication channel
Wed Sep  5 03:43:02 2012
ASMB: terminating instance due to error 15064
Wed Sep  5 03:43:02 2012
Errors in file /u01/app/oracle/product/10.2.0/db/admin/PFRM_C/bdump/pfrm_c2_lmon_13912.trc:
ORA-15064: communication failure with ASM instance
Wed Sep  5 03:43:02 2012
Errors in file /u01/app/oracle/product/10.2.0/db/admin/PFRM_C/bdump/pfrm_c2_lms0_13918.trc:
ORA-15064: communication failure with ASM instance
Wed Sep  5 03:43:02 2012
Errors in file /u01/app/oracle/product/10.2.0/db/admin/PFRM_C/bdump/pfrm_c2_pmon_13902.trc:
ORA-15064: communication failure with ASM instance
Wed Sep  5 03:43:02 2012
System state dump is made for local instance
System State dumped to trace file /u01/app/oracle/product/10.2.0/db/admin/PFRM_C/bdump/pfrm_c2_diag_13906.trc
Wed Sep  5 03:43:06 2012
Shutting down instance (abort)
License high water mark = 37
Wed Sep  5 03:43:07 2012
Trace dumping is performing id=[cdmp_20120905034302]
Wed Sep  5 03:43:07 2012
Instance terminated by ASMB, pid = 14061
Wed Sep  5 03:43:11 2012
Instance terminated by USER, pid = 16959

alert_COXCALL2.log

Wed Sep  5 03:43:01 2012
Errors in file /u01/app/oracle/product/10.2.0/db/admin/COXCALL/bdump/coxcall2_asmb_14369.trc:
ORA-15064: communication failure with ASM instance
ORA-03113: end-of-file on communication channel
Wed Sep  5 03:43:01 2012
ASMB: terminating instance due to error 15064
Wed Sep  5 03:43:01 2012
Errors in file /u01/app/oracle/product/10.2.0/db/admin/COXCALL/bdump/coxcall2_lms1_14277.trc:
ORA-15064: communication failure with ASM instance
Wed Sep  5 03:43:01 2012
Errors in file /u01/app/oracle/product/10.2.0/db/admin/COXCALL/bdump/coxcall2_lms0_14271.trc:
ORA-15064: communication failure with ASM instance
Wed Sep  5 03:43:01 2012
System state dump is made for local instance
System State dumped to trace file /u01/app/oracle/product/10.2.0/db/admin/COXCALL/bdump/coxcall2_diag_14255.trc
Wed Sep  5 03:43:05 2012
Shutting down instance (abort)
License high water mark = 53
Wed Sep  5 03:43:07 2012
Trace dumping is performing id=[cdmp_20120905034301]
Wed Sep  5 03:43:07 2012
Instance terminated by ASMB, pid = 14369
Wed Sep  5 03:43:10 2012
Instance terminated by USER, pid = 16931
Wed Sep  5 03:43:31 2012
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.4.0.
System parameters with non-default values:
  processes                = 1500
  sessions                 = 1800
  __shared_pool_size       = 469762048
  __large_pool_size        = 16777216
  __java_pool_size         = 16777216
  __streams_pool_size      = 16777216
  spfile                   = +BACKUP_DG1/coxcalls/spfilecoxcall.ora
  cluster_interconnects    = 192.168.1.4
  sga_target               = 3221225472
  control_files            = +DATA_DG1/coxcalls/controlfile/current.305.697217041, +BACKUP_DG1/coxcalls/controlfile/current.268.6972
17041
  db_file_name_convert     = +COXCALL_DATA/COXCALL, +DATA_DG1/COXCALLS, +COXCALL_BACKUP/COXCALL, +BACKUP_DG1/COXCALLS
  log_file_name_convert    = +COXCALL_DATA/COXCALL, +DATA_DG1/COXCALLS, +COXCALL_BACKUP/COXCALL, +BACKUP_DG1/COXCALLS
  db_block_size            = 8192
  __db_cache_size          = 2684354560
  compatible               = 10.2.0.2.0
  log_archive_config       = dg_config=(COXCALL,COXCALLS)
  log_archive_dest_1       = LOCATION=USE_DB_RECOVERY_FILE_DEST
  log_archive_dest_2       =
  log_archive_max_processes= 5
  log_archive_format       = %t_%s_%r.arc
  fal_client               = COXCALLS
  fal_server               = COXCALL
  db_file_multiblock_read_count= 16
  cluster_database         = TRUE
  cluster_database_instances= 2
  db_create_file_dest      = +DATA_DG1
  db_recovery_file_dest    = +BACKUP_DG1
  db_recovery_file_dest_size= 1073741824000
  standby_file_management  = auto
  thread                   = 2
  instance_number          = 2
  undo_management          = AUTO
  undo_tablespace          = UNDOTBS2
  undo_retention           = 20000
  remote_login_passwordfile= EXCLUSIVE
  db_domain                =
  service_names            = RKSCALL,PHONETOOLS,BTSUSER,ITV_SVC,ACCALL,COXCALL,COXCALLS,BTA_SVC
  dispatchers              = (PROTOCOL=TCP) (SERVICE=COXCALLXDB)
  local_listener           =
  remote_listener          =
  job_queue_processes      = 10
  background_dump_dest     = /u01/app/oracle/product/10.2.0/db/admin/COXCALL/bdump
  user_dump_dest           = /u01/app/oracle/product/10.2.0/db/admin/COXCALL/udump
  core_dump_dest           = /u01/app/oracle/product/10.2.0/db/admin/COXCALL/cdump
  audit_file_dest          = /u01/app/oracle/product/10.2.0/db/admin/COXCALL/adump
  db_name                  = COXCALL
  db_unique_name           = COXCALLS
  open_cursors             = 500
  pga_aggregate_target     = 1706033152
Cluster communication is configured to use the following interface(s) for this instance
  192.168.1.4
Wed Sep  5 03:43:34 2012
cluster interconnect IPC version:Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
PMON started with pid=2, OS id=17625
DIAG started with pid=3, OS id=17629
PSP0 started with pid=4, OS id=17633
LMON started with pid=5, OS id=17637
LMD0 started with pid=6, OS id=17639
LMS0 started with pid=7, OS id=17643
LMS1 started with pid=8, OS id=17649
MMAN started with pid=9, OS id=17657
DBW0 started with pid=10, OS id=17661
LGWR started with pid=11, OS id=17665
CKPT started with pid=12, OS id=17669
SMON started with pid=13, OS id=17673
RECO started with pid=14, OS id=17677
CJQ0 started with pid=15, OS id=17679
MMON started with pid=16, OS id=17681
Wed Sep  5 03:43:35 2012
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
MMNL started with pid=17, OS id=17683
Wed Sep  5 03:43:35 2012
starting up 1 shared server(s) ...
Wed Sep  5 03:43:36 2012
lmon registered with NM - instance id 2 (internal mem no 1)
Wed Sep  5 03:43:37 2012
Reconfiguration started (old inc 0, new inc 48)
List of nodes:
 0 1
 Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
 Communication channels reestablished
 * domain 0 valid = 1 according to instance 0
Wed Sep  5 03:43:37 2012
 Master broadcasted resource hash value bitmaps
 Non-local Process blocks cleaned out
Wed Sep  5 03:43:37 2012
 LMS 1: 0 GCS shadows cancelled, 0 closed
Wed Sep  5 03:43:37 2012
 LMS 0: 0 GCS shadows cancelled, 0 closed
 Set master node info
 Submitted all remote-enqueue requests
 Dwn-cvts replayed, VALBLKs dubious
 All grantable enqueues granted
Wed Sep  5 03:43:37 2012
 LMS 1: 0 GCS shadows traversed, 0 replayed
Wed Sep  5 03:43:37 2012
 LMS 0: 0 GCS shadows traversed, 0 replayed
Wed Sep  5 03:43:37 2012
 Submitted all GCS remote-cache requests
 Fix write in gcs resources
Reconfiguration complete
LCK0 started with pid=20, OS id=17727
Wed Sep  5 03:43:38 2012
ALTER DATABASE   MOUNT
Wed Sep  5 03:43:39 2012
Starting background process ASMB
ASMB started with pid=22, OS id=17738
Starting background process RBAL
RBAL started with pid=23, OS id=17742
Wed Sep  5 03:43:42 2012
SUCCESS: diskgroup DATA_DG1 was mounted
SUCCESS: diskgroup BACKUP_DG1 was mounted
Wed Sep  5 03:43:46 2012
Setting recovery target incarnation to 2
Wed Sep  5 03:43:46 2012
Successful mount of redo thread 2, with mount id 819078902
Wed Sep  5 03:43:46 2012
Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
Completed: ALTER DATABASE   MOUNT
Wed Sep  5 03:43:47 2012
ALTER DATABASE OPEN
Picked broadcast on commit scheme to generate SCNs
Wed Sep  5 03:43:52 2012
LGWR: STARTING ARCH PROCESSES
ARC0 started with pid=27, OS id=17912
ARC1 started with pid=28, OS id=17914
ARC2 started with pid=29, OS id=17916
ARC3 started with pid=30, OS id=17918
Wed Sep  5 03:43:52 2012
ARC0: Archival started
ARC1: Archival started
ARC2: Archival started
ARC3: Archival started
ARC4: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
ARC4 started with pid=31, OS id=17920
Wed Sep  5 03:43:52 2012
Thread 2 opened at log sequence 100362
  Current log# 10 seq# 100362 mem# 0: +DATA_DG1/coxcalls/onlinelog/group_10.353.697219913
  Current log# 10 seq# 100362 mem# 1: +BACKUP_DG1/coxcalls/onlinelog/group_10.278.697219913
Successful open of redo thread 2
Wed Sep  5 03:43:52 2012
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Sep  5 03:43:52 2012
SMON: enabling cache recovery
Wed Sep  5 03:43:52 2012
ARC0: Becoming the 'no FAL' ARCH
ARC0: Becoming the 'no SRL' ARCH
Wed Sep  5 03:43:52 2012
ARC3: Becoming the heartbeat ARCH
Wed Sep  5 03:43:53 2012
Successfully onlined Undo Tablespace 5
Wed Sep  5 03:43:53 2012
SMON: enabling tx recovery
Wed Sep  5 03:43:53 2012
Database Characterset is WE8ISO8859P1
Opening with internal Resource Manager plan
where NUMA PG = 1, CPUs = 8
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
QMNC started with pid=35, OS id=17983
Wed Sep  5 03:43:57 2012
Completed: ALTER DATABASE OPEN

deanza

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
1
评论
ORA-15064: communication failure with ASM instance

alert_ASM2.logWed Sep 05 03:32:56 2012IPC Send timeout detected.Sender: ospid 13720Receiver: inst 1 binc 22497 ospid 1431IPC Send timeout to 1.1 inc 56 for msg type 65518 from opid 10Wed Sep 05
复制链接

扫一扫

专栏目录