今天启动RAC做实验,发现RAC实例怎么启动不了。
[oracle@rac2 bin]$ crs_stat -t
NameTypeTargetStateHost
------------------------------------------------------------
ora.orcl.dbapplicationONLINEOFFLINE
ora....oltp.cs applicationONLINEOFFLINE
ora....cl1.srv applicationONLINEOFFLINE
ora....cl2.srv applicationONLINEOFFLINE
ora....l1.inst applicationONLINEOFFLINE
ora....l2.inst applicationONLINEOFFLINE
ora....SM1.asm applicationONLINEONLINErac1
ora....C1.lsnr applicationONLINEONLINErac1
ora.rac1.gsdapplicationONLINEONLINErac1
ora.rac1.onsapplicationONLINEONLINErac1
ora.rac1.vipapplicationONLINEONLINErac1
ora....SM2.asm applicationONLINEONLINErac2
ora....C2.lsnr applicationONLINEONLINErac2
ora.rac2.gsdapplicationONLINEONLINErac2
ora.rac2.onsapplicationONLINEONLINErac2
ora.rac2.vipapplicationONLINEONLINErac2
看了一下ASM的磁盘组,都挂载成功。
[oracle@rac2 bin]$export ORACLE_SID=+ASM2
[oracle@rac2 bin]$sqlplus / as sysdba;
SQL*Plus: Release 10.2.0.1.0 - Production on Tue Sep 28 04:33:24 2010
Copyright (c) 1982, 2005, Oracle.All rights reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options
SQL>select name,state from v$asm_diskgroup;
NAMESTATE
------------------------------ -----------
DATAMOUNTED
FLASH_RECOVERY_AREAMOUNTED
SQL>
之前遇到过这种情况情况,所以第一时间就查看了这个ASM磁盘组的信息。
ASM没有问题,只能查看数据库的alert log了。
[oracle@rac1 bdump]$ pwd
/u01/app/oracle/admin/orcl/bdump
[oracle@rac1 bdump]$ ls
alert_orcl1.logorcl1_diag_22153.trcorcl1_lms0_22169.trc
cdmp_20100917173112orcl1_lgwr_22186.trcorcl1_smon_22190.trc
orcl1_arc0_24180.trcorcl1_lmd0_22167.trc
orcl1_arc1_24182.trcorcl1_lmon_22165.trc
[oracle@rac1 bdump]$tail -100 alert_orcl1.log
Tue Sep 28 04:05:20 2010
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
MMON started with pid=15, OS id=5650
Tue Sep 28 04:05:20 2010
starting up 1 shared server(s) ...
Tue Sep 28 04:05:20 2010
lmon registered with NM - instance id 1 (internal mem no 0)
Tue Sep 28 04:05:21 2010
Reconfiguration started (old inc 0, new inc 2)
List of nodes:
0
Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Tue Sep 28 04:05:21 2010
LMS 0: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Post SMON to start 1st pass IR
Tue Sep 28 04:05:21 2010
LMS 0: 0 GCS shadows traversed, 0 replayed
Tue Sep 28 04:05:21 2010
Submitted all GCS remote-cache requests
Post SMON to start 1st pass IR
Fix write in gcs resources
Reconfiguration complete
LCK0 started with pid=19, OS id=5683
Tue Sep 28 04:05:22 2010
ALTER DATABASEMOUNT
Tue Sep 28 04:05:22 2010
This instance was first to mount
Tue Sep 28 04:05:22 2010
Starting background process ASMB
ASMB started with pid=21, OS id=5706
Starting background process RBAL
RBAL started with pid=22, OS id=5710
Loaded ASM Library - Generic Linux, version 2.0.4 (KABI_V2) library for asmlib interface
Tue Sep 28 04:05:26 2010
SUCCESS: diskgroup DATA was mounted
SUCCESS: diskgroup FLASH_RECOVERY_AREA was mounted
Tue Sep 28 04:05:34 2010
Setting recovery target incarnation to 3
Tue Sep 28 04:05:35 2010
Successful mount of redo thread 1, with mount id 1258660482
Tue Sep 28 04:05:35 2010
Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
Completed: ALTER DATABASEMOUNT
Tue Sep 28 04:05:35 2010
ALTER DATABASE OPEN
This instance was first to open
ORA-1589 signalled during: ALTER DATABASE OPEN...
Tue Sep 28 04:05:39 2010
Shutting down instance (abort)
License high water mark = 1
Instance terminated by USER, pid = 6080
从日志里我们发现了一个很重要的信息:
ALTER DATABASE OPEN
This instance was first to open
ORA-1589 signalled during: ALTER DATABASE OPEN...
Tue Sep 28 04:05:39 2010
Shutting down instance (abort)
查看了一下错误代码:
ORA-01589: | must use RESETLOGS or NORESETLOGS option for database open |
Cause: | Either incomplete or backup control file recovery has been performed. After these types of recovery you must specify either the RESETLOGS option or the NORESETLOGS option to open your database. |
Action: | Specify the appropriate option. |
提示我们必须使用resetlogs模式打开数据。开到这才明白,昨天做RMAN恢复后直接把数据库关了,没有用resetlogs打开。杯具中..
手工启动数据库至mount状态,在用resetlogs打开:
[oracle@rac1 bdump]$export ORACLE_SID=orcl1
[oracle@rac1 bdump]$sqlplus /nolog
SQL*Plus: Release 10.2.0.1.0 - Production on Tue Sep 28 04:17:47 2010
Copyright (c) 1982, 2005, Oracle.All rights reserved.
SQL>conn / as sysdba;
Connected to an idle instance.
SQL>startup mount;
ORACLE instance started.
Total System Global Area180355072 bytes
Fixed Size1218388 bytes
Variable Size109054124 bytes
Database Buffers67108864 bytes
Redo Buffers2973696 bytes
Database mounted.
SQL>alter database open resetlogs;
Database altered.
SQL>
在另一个节点,直接正常启动就可以了。
[oracle@rac2 bin]$export ORACLE_SID=orcl2
[oracle@rac2 bin]$sqlplus /nolog
SQL*Plus: Release 10.2.0.1.0 - Production on Tue Sep 28 04:22:50 2010
Copyright (c) 1982, 2005, Oracle.All rights reserved.
SQL>conn / as sysdba
Connected to an idle instance.
SQL>startup mount;
ORACLE instance started.
Total System Global Area180355072 bytes
Fixed Size1218388 bytes
Variable Size88082604 bytes
Database Buffers88080384 bytes
Redo Buffers2973696 bytes
Database mounted.
SQL>alter database open;
Database altered.
SQL>
查看一下RAC进程的状态:
[oracle@rac2 bin]$ crs_stat -t
NameTypeTargetStateHost
------------------------------------------------------------
ora.orcl.dbapplicationONLINEONLINErac2
ora....oltp.cs applicationONLINEONLINErac2
ora....cl1.srv applicationONLINEONLINErac1
ora....cl2.srv applicationONLINEONLINErac2
ora....l1.inst applicationONLINEONLINErac1
ora....l2.inst applicationONLINEONLINErac2
ora....SM1.asm applicationONLINEONLINErac1
ora....C1.lsnr applicationONLINEONLINErac1
ora.rac1.gsdapplicationONLINEONLINErac1
ora.rac1.onsapplicationONLINEONLINErac1
ora.rac1.vipapplicationONLINEONLINErac1
ora....SM2.asm applicationONLINEONLINErac2
ora....C2.lsnr applicationONLINEONLINErac2
ora.rac2.gsdapplicationONLINEONLINErac2
ora.rac2.onsapplicationONLINEONLINErac2
ora.rac2.vipapplicationONLINEONLINErac2
一切正常,从这个小错误中明白,遇到错误,要养成看alert log的习惯。不管是单实例还是RAC实例,看log永远是第一条。还有就是要心细,昨天偷了懒,为了补这个懒,花的时间就比昨天多了很多。杯具啊.
------------------------------------------------------------------------------