messages报logger: Waiting minutes for filesystem containing crsctl

原创 2013年12月10日 20:10:04
messages报logger: Waiting  minutes for filesystem containing crsctl
现象:
发现每次REBOOT主机之后CRS启动都非常慢,发现messages有如下信息:
Dec 5 18:20:40 saprac4 logger: Oracle Cluster Ready Services starting up automatically.
Dec 5 18:20:40 saprac4 logger: Waiting 10 minutes for filesystem containing /apps/oracle/11gR2/crs/bin/crsctl.
Dec 5 18:20:54 saprac4 ntpd[4060]: synchronized to LOCAL(0), stratum 10
Dec 5 18:21:40 saprac4 logger: Waiting 9 minutes for filesystem containing /apps/oracle/11gR2/crs/bin/crsctl.
Dec 5 18:22:40 saprac4 logger: Waiting 8 minutes for filesystem containing /apps/oracle/11gR2/crs/bin/crsctl.
Dec 5 18:23:40 saprac4 logger: Waiting 7 minutes for filesystem containing /apps/oracle/11gR2/crs/bin/crsctl.
Dec 5 18:24:40 saprac4 logger: Waiting 6 minutes for filesystem containing /apps/oracle/11gR2/crs/bin/crsctl.
Dec 5 18:25:40 saprac4 logger: Waiting 5 minutes for filesystem containing /apps/oracle/11gR2/crs/bin/crsctl.
Dec 5 18:26:40 saprac4 logger: Waiting 4 minutes for filesystem containing /apps/oracle/11gR2/crs/bin/crsctl.
Dec 5 18:27:40 saprac4 logger: Waiting 3 minutes for filesystem containing /apps/oracle/11gR2/crs/bin/crsctl.
Dec 5 18:28:40 saprac4 logger: Waiting 2 minutes for filesystem containing /apps/oracle/11gR2/crs/bin/crsctl.
Dec 5 18:29:40 saprac4 logger: Waiting 1 minutes for filesystem containing /apps/oracle/11gR2/crs/bin/crsctl.

分析:
该主机之前安装过11GR2(当前11GR1),但安装完成之后未正确卸载,导致有残留的文件,导致会先启动之前的CRS(11GR2),
但/apps文件系统已经被删除,所以messages会一致wait该文件系统的挂载,一直等待时间达到9min之后才启动后安装的CRS(11GR1)。
解决方法:
清楚11GR2残留CRS文件:
#cd /etc/init.d
#mkdir bak
#mv init.ohasd bak
#mv ohasd bak


附:
How to Deconfigure/Reconfigure(Rebuild OCR) or Deinstall Grid Infrastructure

A. Grid Infrastructure Cluster - Entire Cluster
 
Deconfigure and reconfigure entire cluster will rebuild OCR and Voting Disk, user resources (database, instance, service, listener etc) will need to be added back to the cluster manually after reconfigure finishes.
 
Why is deconfigure needed?
 
Deconfigure is needed when:
 ?OCR is corrupted without any good backup
 ?Or GI stack will not come up on any nodes due to missing Oracle Clusterware related files in /etc or /var/opt/oracle, i.e. init.ohasd missing etc. If GI is able to come up on at least one node, refer to next Section "B. Grid Infrastructure Cluster - One or Partial Nodes".
 ?$GRID_HOME should be intact as deconfigure will NOT fix $GRID_HOME corruption
 
Steps to deconfigure
 
Before deconfiguring, collect the following as grid user if possible to generate a list of user resources to be added back to the cluster after reconfigure finishes:
 
$GRID_HOME/bin/crsctl stat res -t
 $GRID_HOME/bin/crsctl stat res -p
 $GRID_HOME/bin/crsctl query css votedisk
 $GRID_HOME/bin/ocrcheck
 $GRID_HOME/bin/oifcfg getif
 $GRID_HOME/bin/srvctl config nodeapps -a
 $GRID_HOME/bin/srvctl config scan
 $GRID_HOME/bin/srvctl config asm -a
 $GRID_HOME/bin/srvctl config listener -l <listener-name> -a
 $DB_HOME/bin/srvctl config database -d <dbname> -a
 $DB_HOME/bin/srvctl config service -d <dbname> -s <service-name> -v
 


To deconfigure:
 ?If OCR and Voting Disks are NOT on ASM, or If OCR and Voting Disks are on ASM but there's NO user data in OCR/Voting Disk ASM diskgroup:
 
On all remote nodes, as root execute:
 
# <$GRID_HOME>/crs/install/rootcrs.pl -deconfig -force -verbose
 
Once the above command finishes on all remote nodes, on local node, as root execute:
 
# <$GRID_HOME>/crs/install/rootcrs.pl -deconfig -force -verbose -lastnode
 
To reconfigure, run $GRID_HOME/crs/config/config.sh, refer to note 1354258.1 for details ?If OCR or Voting Disks are on ASM and there is user data in OCR/Voting Disk ASM diskgroup:
 
?If GI version is 11.2.0.3 AND fix for bug 13058611 and bug 13001955 has been applied, or GI version is 11.2.0.3.2 GI PSU (includes both fixes) or higher:
 
On all remote nodes, as root execute:
 
# <$GRID_HOME>/crs/install/rootcrs.pl -deconfig -force -verbose
 
Once the above command finishes on all remote nodes, on local node, as root execute:

# <$GRID_HOME>/crs/install/rootcrs.pl -deconfig -force -verbose -keepdg -lastnode
 
To reconfigure, run $GRID_HOME/crs/config/config.sh, refer to note 1354258.1 for details
?If fix for bug 13058611 and bug 13001955 has NOT been applied:
 
On all nodes, as root execute:
 
# <$GRID_HOME>/crs/install/rootcrs.pl -deconfig -force -verbose
 

To reconfigure:
 

For 11.2.0.1 - deinstall and reinstall with OCR/Voting Disk on a new ASM diskgroup or supported cluster/network filesystem
 
For 11.2.0.2 and onward - run $GRID_HOME/crs/config/config.sh and place OCR/Voting Disk on a new ASM diskgroup or support cluster/network filesystem. Refer to note 1354258.1 for more details of config.sh/config.bat
 
 
 
B. Grid Infrastructure Cluster - One or Partial Nodes
 
This procedure applies only when all the followings are true:
 ?One or partial nodes are having problem, but one or other nodes are running fine - so there's no need to deconfigure the entire cluster
 ?And GI is a fresh installation (NOT upgrad) without any patch set (interim patch or patch set update(PSU) is fine). A direct patch set installation is considered as a fresh installation regardless how long it has been running, as long as there was no Oracle Clusterware running when it is first installed.
 ?And cluster parameters have not been changed since original configuration, eg: OCR/VD on same location, network configuration has not been changed etc
 ?And $GRID_HOME is intact as deconfigure will NOT fix $GRID_HOME corruption
 ?If any of the above is NOT true, node removal/addition procedure should be used
 
Steps to deconfigure and reconfigure
 
As root, on each problematic node, execute:
 

# <$GRID_HOME>/crs/install/rootcrs.pl -deconfig -force
 # <$GRID_HOME>/root.sh
 
C. Grid Infrastructure Standalone (Oracle Restart)
 
Why is deconfigure needed?
 
Deconfigure is needed when:
 ?OLR is corrupted without any good backup
 ?GI stack will not come up due to missing Oracle Clusterware related files in /etc or /var/opt/oracle, i.e. init.ohasd is missing etc
 ?Nodename needs to be changed
 
Steps to deconfigure
 
Before deconfiguring, collect the following if possible:
 
$GRID_HOME/bin/crsctl stat res -t
 $GRID_HOME/bin/crsctl stat res -p
 $GRID_HOME/bin/srvctl config asm -a
 $GRID_HOME/bin/srvctl config listener -l <listener-name> -a
 $DB_HOME/bin/srvctl config database -d <dbname> -a
 $DB_HOME/bin/srvctl config service -d <dbname> -s <service-name> -v
 


To deconfigure:
 
As root execute:
 

# <$GRID_HOME>/crs/install/roothas.pl -deconfig -force -verbose
 
To reconfigure, refer to note 1354258.1
 


D. Grid Infrastructure Deinstall
 
As grid user, execute:
 
$ <$GRID_HOME>/deinstall/deinstall
 

 For details, refer to the following documentation for your platform:
 
Oracle Grid Infrastructure
 Installation Guide
 
How to Modify or Deinstall Oracle Grid Infrastructure
If there's any error, deconfigure the failed GI with steps in Section A - C, and deinstall manually with note 1364419.1
For searchability: recreate OCR, recreate Voting Disk, rebuild OCR

What can I learn right now in just 10 minutes that could be useful for the rest of my life?

What can I learn right now in just 10 minutes that could be useful for the rest of my life?

Linux内核挂载根文件系统出现INIT:Id "S" respawning too fast: disabled for 5 minutes的解决方法

原创作品,转载时请务必以超链接形式标明文章原始出处:http://blog.csdn.net/gqb666/article/details/8260018 一、   问题描述: 使用l...
  • gqb666
  • gqb666
  • 2012年12月05日 14:58
  • 4005

How-to suppress error popup for inline messages __jspx页面弹出消息提示框

Error messages in ADF Faces are displayed in a DHTML popup dialog.  To change this behavior so messa...

10 RAC CRS 2节点执行root.sh报错Waiting for the Oracle CRSD and EVMD 处理方法

1、--在安装10G clusterware 在二节点执行root.sh报错Waiting for the Oracle CRSD and EVMD to start处理方法 [root@nydb2 ...

Hive 元数据“waiting for table metadata lock”

最近hive元数据统计时,老出现“waiting for metadata lock”,造成hive查询、统计的sql执行失败。现象: 33692473 hiveadmin 10.5.18.22...

一次关于Waiting for table metadata lock的处理

一个简单的DDL操作(表数据很少,访问高)执行很久,导致Waiting for table metadata lock。 版本:mysql5.5.17 查看:Innodb_buffer_pool_...

tomcat8在myeclipse中启动不了 出现严重: Error waiting for multi-thread deployment of directories to completehos

问题解决一: 在eclipse中,window-->preferences-->Java-->Installed JREs   选中JRE 点击Edit 写入这句话: -Xms...

HDU2653 Waiting ten thousand years for Love (三维广搜+优先队列)

Waiting ten thousand years for Love Time Limit:2000MS     Memory Limit:32768KB     64bit IO Format:...

MessagingTimeout: Timed out waiting for a reply to message ID

l3中出现大量消息超时错误,对网络的操作各种异常。 报错如下: 2016-02-25 05:54:59.886 15110 ERROR neutron.agent.l3.agent [req-db...
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:messages报logger: Waiting minutes for filesystem containing crsctl
举报原因:
原因补充:

(最多只允许输入30个字)