ORACLE 11.2.0.3
REDHAT 5.8
突然节点2数据库出故障,直接使用SQLPLUS 登录进行数据库重新启动,
两个节点中的ORACLE数据库已使用单实例方式 启动了。但在RAC状态看到,数据库OFFLINE,并且OEM也不正常。
但再看CRS_STAT -T 状态如下:
[grid@rac1 bin]$ ./crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.CRS.dg ora....up.type ONLINE ONLINE rac1
ora.DATA.dg ora....up.type ONLINE ONLINE rac1
ora....ER.lsnr ora....er.type ONLINE ONLINE rac1
ora....N1.lsnr ora....er.type ONLINE ONLINE rac1
ora....VERY.dg ora....up.type ONLINE ONLINE rac1
ora.asm ora.asm.type ONLINE ONLINE rac1
ora.cvu ora.cvu.type ONLINE ONLINE rac1
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE rac1
ora.oc4j ora.oc4j.type ONLINE ONLINE rac1
ora.ons ora.ons.type ONLINE ONLINE rac1
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application OFFLINE OFFLINE
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip ora....t1.type ONLINE ONLINE rac1
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application OFFLINE OFFLINE
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip ora....t1.type ONLINE ONLINE rac2
ora.racdb.db ora....se.type ONLINE OFFLINE
ora....ry.acfs ora....fs.type ONLINE ONLINE rac1
ora.scan1.vip ora....ip.type ONLINE ONLINE rac1
[grid@rac1 bin]$ ./srvctl stop database -d racdb
PRCC-1016 : racdb 已停止
[grid@rac1 bin]$ ./srvctl start database -d racdb
PRCR-1079 : 无法启动资源 ora.racdb.db
CRS-2800: Cannot start resource 'ora.asm' as it is already in the INTERMEDIATE state on server 'rac1'
CRS-2632: There are no more servers to try to place resource 'ora.racdb.db' on that would satisfy its placement policy
CRS-2800: Cannot start resource 'ora.asm' as it is already in the INTERMEDIATE state on server 'rac2'
[grid@rac1 bin]$
查看 RACDB的资源如下:
[grid@rac1 bin]$ ./crsctl status resource ora.racdb.db -f
NAME=ora.racdb.db
TYPE=ora.database.type
STATE=OFFLINE
TARGET=ONLINE
ACL=owner racle:rwx,pgrp install:rwx,other::r--
ACTION_FAILURE_TEMPLATE=
ACTION_SCRIPT=
ACTIVE_PLACEMENT=1
AGENT_FILENAME=%CRS_HOME%/bin/oraagent%CRS_EXE_SUFFIX%
AUTO_START=restore
CARDINALITY=2
CARDINALITY_ID=0
CHECK_INTERVAL=1
CHECK_TIMEOUT=30
CLUSTER_DATABASE=true
CREATION_SEED=83
DATABASE_TYPE=RAC
DB_UNIQUE_NAME=racdb
DEFAULT_TEMPLATE=PROPERTY(RESOURCE_CLASS=database) PROPERTY(DB_UNIQUE_NAME= CONCAT(PARSE(%NAME%, ., 2), %USR_ORA_DOMAIN%, .)) ELEMENT(INSTANCE_NAME= %GEN_USR_ORA_INST_NAME%) ELEMENT(DATABASE_TYPE= %DATABASE_TYPE%)
DEGREE=1
DESCRIPTION=Oracle Database resource
ENABLED=1
FAILOVER_DELAY=0
FAILURE_INTERVAL=60
FAILURE_THRESHOLD=1
GEN_AUDIT_FILE_DEST=/opt/app/oracle/admin/racdb/adump
GEN_START_OPTIONS=
GEN_START_OPTIONS@SERVERNAME(rac1)=nomount
GEN_START_OPTIONS@SERVERNAME(rac2)=open
GEN_USR_ORA_INST_NAME=
GEN_USR_ORA_INST_NAME@SERVERNAME(rac1)=racdb1
GEN_USR_ORA_INST_NAME@SERVERNAME(rac2)=racdb2
HOSTING_MEMBERS=
ID=ora.racdb.db
INSTANCE_FAILOVER=0
LOAD=1
LOGGING_LEVEL=1
MANAGEMENT_POLICY=AUTOMATIC
NLS_LANG=
NOT_RESTARTING_TEMPLATE=
OFFLINE_CHECK_INTERVAL=0
ONLINE_RELOCATION_TIMEOUT=0
ORACLE_HOME=/opt/app/oracle/product/11.2.0
ORACLE_HOME_OLD=
PLACEMENT=restricted
PROFILE_CHANGE_TEMPLATE=
RESTART_ATTEMPTS=2
ROLE=PRIMARY
SCRIPT_TIMEOUT=60
SERVER_POOLS=ora.racdb
SPFILE=+DATA/racdb/spfileracdb.ora
START_DEPENDENCIES=hard(ora.DATA.dg,ora.RECOVERY.dg) weak(type ra.listener.type,global:type ra.scan_listener.type,uniform ra.ons,global ra.gns) pullup(ora.DATA.dg,ora.RECOVERY.dg)
START_TIMEOUT=600
STATE_CHANGE_TEMPLATE=
STOP_DEPENDENCIES=hard(intermediate ra.asm,shutdown ra.DATA.dg,shutdown ra.RECOVERY.dg)
STOP_TIMEOUT=600
TYPE_VERSION=3.2
UPTIME_THRESHOLD=1h
USR_ORA_DB_NAME=racdb
USR_ORA_DOMAIN=
USR_ORA_ENV=
USR_ORA_FLAGS=
USR_ORA_INST_NAME=
USR_ORA_INST_NAME@SERVERNAME(rac1)=racdb1
USR_ORA_INST_NAME@SERVERNAME(rac2)=racdb2
USR_ORA_OPEN_MODE=open
USR_ORA_OPI=false
USR_ORA_STOP_MODE=immediate
VERSION=11.2.0.3.0
后经高人指点,操作如下:
操作顺序:
1.先关掉CRS,再重启,(./crsctl stop crs)
2.查看数据状态,
3.查看RAC中的数据库各实例状态,(./srvctl status database -d racdb)
4.使用RAC命令启动实例 (./srvctl start database -d racdb)
在第四步重启集群中的数据库实例时,因为各实例已启动。
[root@rac1 bin]# ./crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2790: 正在启动关闭 'rac1' 上集群就绪服务管理的资源的操作
CRS-2673: 尝试停止 'ora.LISTENER.lsnr' (在 'rac1' 上)
CRS-2673: 尝试停止 'ora.cvu' (在 'rac1' 上)
CRS-2673: 尝试停止 'ora.oc4j' (在 'rac1' 上)
CRS-2673: 尝试停止 'ora.CRS.dg' (在 'rac1' 上)
CRS-2673: 尝试停止 'ora.registry.acfs' (在 'rac1' 上)
CRS-2673: 尝试停止 'ora.DATA.dg' (在 'rac1' 上)
CRS-2673: 尝试停止 'ora.RECOVERY.dg' (在 'rac1' 上)
CRS-2673: 尝试停止 'ora.LISTENER_SCAN1.lsnr' (在 'rac1' 上)
CRS-2677: 成功停止 'ora.cvu' (在 'rac1' 上)
CRS-2672: 尝试启动 'ora.cvu' (在 'rac2' 上)
CRS-2677: 成功停止 'ora.LISTENER.lsnr' (在 'rac1' 上)
CRS-2673: 尝试停止 'ora.rac1.vip' (在 'rac1' 上)
CRS-2677: 成功停止 'ora.rac1.vip' (在 'rac1' 上)
CRS-2672: 尝试启动 'ora.rac1.vip' (在 'rac2' 上)
CRS-2676: 成功启动 'ora.cvu' (在 'rac2' 上)
CRS-2677: 成功停止 'ora.LISTENER_SCAN1.lsnr' (在 'rac1' 上)
CRS-2673: 尝试停止 'ora.scan1.vip' (在 'rac1' 上)
CRS-2677: 成功停止 'ora.scan1.vip' (在 'rac1' 上)
CRS-2672: 尝试启动 'ora.scan1.vip' (在 'rac2' 上)
CRS-2677: 成功停止 'ora.registry.acfs' (在 'rac1' 上)
CRS-2676: 成功启动 'ora.rac1.vip' (在 'rac2' 上)
CRS-2676: 成功启动 'ora.scan1.vip' (在 'rac2' 上)
CRS-2672: 尝试启动 'ora.LISTENER_SCAN1.lsnr' (在 'rac2' 上)
CRS-2676: 成功启动 'ora.LISTENER_SCAN1.lsnr' (在 'rac2' 上)
CRS-2677: 成功停止 'ora.oc4j' (在 'rac1' 上)
CRS-2672: 尝试启动 'ora.oc4j' (在 'rac2' 上)
CRS-2676: 成功启动 'ora.oc4j' (在 'rac2' 上)
CRS-2675: 未能停止 'ora.RECOVERY.dg' (在 'rac1' 上)
CRS-2675: 未能停止 'ora.DATA.dg' (在 'rac1' 上)
CRS-2677: 成功停止 'ora.CRS.dg' (在 'rac1' 上)
CRS-2799: 无法关闭资源 'ora.DATA.dg' (在 'rac1' 上)
CRS-2799: 无法关闭资源 'ora.RECOVERY.dg' (在 'rac1' 上)
CRS-2799: 无法关闭资源 'ora.asm' (在 'rac1' 上)
CRS-2799: 无法关闭资源 'ora.rac1.ASM1.asm' (在 'rac1' 上)
CRS-2794: 关闭 'rac1' 上集群就绪服务管理的资源的操作已失败
CRS-2675: Stop of 'ora.crsd' on 'rac1' failed
CRS-2799: Failed to shut down resource 'ora.crsd' on 'rac1'
CRS-2795: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has failed
CRS-4687: Shutdown command has completed with errors.
CRS-4000: Command Stop failed, or completed with errors.
[root@rac1 bin]# ./crsctl start crs
CRS-4640: Oracle High Availability Services is already active
CRS-4000: Command Start failed, or completed with errors.
[root@rac1 bin]# ./crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.CRS.dg ora....up.type ONLINE ONLINE rac2
ora.DATA.dg ora....up.type ONLINE ONLINE rac1
ora....ER.lsnr ora....er.type ONLINE ONLINE rac2
ora....N1.lsnr ora....er.type ONLINE ONLINE rac2
ora....VERY.dg ora....up.type ONLINE ONLINE rac1
ora.asm ora.asm.type ONLINE ONLINE rac1
ora.cvu ora.cvu.type ONLINE ONLINE rac2
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE rac1
ora.oc4j ora.oc4j.type ONLINE ONLINE rac2
ora.ons ora.ons.type ONLINE ONLINE rac1
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE OFFLINE
ora.rac1.gsd application OFFLINE OFFLINE
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip ora....t1.type ONLINE ONLINE rac2
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application OFFLINE OFFLINE
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip ora....t1.type ONLINE ONLINE rac2
ora.racdb.db ora....se.type ONLINE ONLINE rac2
ora....ry.acfs ora....fs.type ONLINE ONLINE rac2
ora.scan1.vip ora....ip.type ONLINE ONLINE rac2
[root@rac1 bin]#
[root@rac1 bin]# ./srvctl status database
PRKO-2082 : 缺少必需的选项 -d
[root@rac1 bin]# ./srvctl status database -d
PRKO-2388 : 命令行选项 -d 后应跟随值。
[root@rac1 bin]# ./srvctl status database -d racdb
实例 racdb1 没有在 rac1 节点上运行
实例 racdb2 正在节点 rac2 上运行
[root@rac1 bin]# ./srvctl start database -d racdb
[root@rac1 bin]# ./srvctl status database -d racdb
实例 racdb1 正在节点 rac1 上运行
实例 racdb2 正在节点 rac2 上运行
URL:
http://www.linuxidc.com/Linux/2012-01/52058.htm