11.2.0.3 rac io异常 Bug# 13867435 ，11.2.0.3打补丁11.2.0.3.8记录

auto方式，数据库和crs正在运行
stop the EM agent processes running from the database home, 每个节点都执行 
$ /bin/emctl stop dbconsole 

1.像以往一样，先升级OPatch，注意每个节点的grid和db home都要做 
$ unzip -d  
$ /OPatch/opatch version 

2.运行脚本产生OCM (Oracle Configuration Manager) response file，只需grid用户在每个节点做 
As the Grid home owner execute: 
$ /OPatch/ocm/bin/emocmrsp #其实是grid_home 

su - grid 
/u01/app/11.2.0/grid/OPatch/ocm/bin/emocmrsp 

（节点2同样） 

3.分别在grid home和db home检查oracle环境是否满足patch 
$ /OPatch/opatch lsinventory -detail -oh  

/u01/app/11.2.0/grid/OPatch/opatch lsinventory -detail -oh /u01/app/11.2.0/grid 
/u01/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch lsinventory -detail -oh /u01/app/oracle/product/11.2.0/dbhome_1 

（节点2同样） 

4.解压补丁到相应位置，并赋予owner grid及777权限，注意，手动pach时需要Required amount of space(6754.328MB) 
auto方式打补丁，最好两节点的补丁位置一致。 

Note:Do not unzip the patch in the top level /tmp directory. 
cd /tmp/patch 
$ unzip p13867435_112038_Linux-x86-64.zip 

chown -R grid:oinstall /tmp/patch 
chmod -R 777 /tmp/patch 

（节点2同样） 

5.auto方式打补丁。OPatch Automation for GI 
设置opatch命令路径 
$ export PATH=$PATH:/OPatch 
export PATH=$PATH:/u01/app/11.2.0/grid/OPatch 

用root同时打补丁到gi home和oracle rac db home。另有只打gi补丁或db home补丁，详见readme 
# opatch auto -ocmrf  
opatch auto /tmp/patch -ocmrf /home/grid/ocm.rsp 

（节点2同样） 

5-4.To roll back the patch from the GI home and each Oracle RAC database home: 

# opatch auto -rollback -ocmrf  

6.更新DB补丁记录（sysdba用户登录实例）、编辑无效对象。 
readme中没这一步，觉得auto方式应该不需要 
SQL> @?/rdbms/admin/catbundle.sql psu apply 
SQL> @?/rdbms/admin/utlrp.sql 


7.确认补丁状态 
查询DB补丁记录 
set linesize 300 
col action_time for a30 
col version for a15 
col id for 99 
col action for a10 
col namespace for a10 
col COMMENTS for a20 
col BUNDLE_SERIES for a15 
select * from registry$history; 
查询补丁状态grid、oracle皆可 
$ $ORACLE_HOME/OPatch/opatch lspatches 
$ $ORACLE_HOME/OPatch/opatch lsinventory 

（节点2同样） 



因为11.2.0.3 rac的bug 13867435 造成io异常，虚拟机卡死，auto方式升级会hong住，故使用manual方式升级。 

##stop the EM agent processes running from the database home, 每个节点都执行 
$ /bin/emctl stop dbconsole 

（节点2同样） 

##Execute the following on each node of the cluster in non-shared CRS and DB home environment to apply the patch.（每个节点的crs和db home都要apply the patch） 

1.Stop the CRS managed resources running from DB homes. 

If this is a GI Home environment, as the database home owner execute:（oracle用户执行） 
su - oracle 
$ /bin/srvctl stop home -o -s -n  

[ oracle@racnode2 ~]$ /u01/app/oracle/product/11.2.0/dbhome_1/bin/srvctl stop home -o /u01/app/oracle/product/11.2.0/dbhome_1 -s /home/oracle/state_file.txt -n racnode1 
[ oracle@racnode2 ~]$ ls 
oradiag_oracle state_file.txt 
[ oracle@racnode2 ~]$ cat state_file.txt 
db-racdb 

我应该没用11g的 oracle restart home特性，忽略了下面 
If this is an Oracle Restart Home environment, as the database home owner execute: 
$ /bin/srvctl stop home -o -s  

Note: 
You need to make sure that the Oracle ACFS file systems are unmounted (see Section 2.8) and all other Oracle processes are shutdown before you proceed. 

（节点2同样） 

2.Run the pre root script.（root用户停止grid） 

If this is a GI Home, as the root user execute: 
# /crs/install/rootcrs.pl -unlock 

[root@racnode1 ~]# /u01/app/11.2.0/grid/crs/install/rootcrs.pl -unlock 
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params 
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'racnode1' 
CRS-2673: Attempting to stop 'ora.crsd' on 'racnode1' 
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'racnode1' 
CRS-2673: Attempting to stop 'ora.CRS.dg' on 'racnode1' 
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'racnode1' 
CRS-2673: Attempting to stop 'ora.RECOVERY.dg' on 'racnode1' 
CRS-2677: Stop of 'ora.DATA.dg' on 'racnode1' succeeded 
CRS-2677: Stop of 'ora.RECOVERY.dg' on 'racnode1' succeeded 
CRS-2677: Stop of 'ora.CRS.dg' on 'racnode1' succeeded 
CRS-2673: Attempting to stop 'ora.asm' on 'racnode1' 
CRS-2677: Stop of 'ora.asm' on 'racnode1' succeeded 
CRS-2673: Attempting to stop 'ora.ons' on 'racnode1' 
CRS-2677: Stop of 'ora.ons' on 'racnode1' succeeded 
CRS-2673: Attempting to stop 'ora.net1.network' on 'racnode1' 
CRS-2677: Stop of 'ora.net1.network' on 'racnode1' succeeded 
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'racnode1' has completed 
CRS-2677: Stop of 'ora.crsd' on 'racnode1' succeeded 
CRS-2673: Attempting to stop 'ora.ctssd' on 'racnode1' 
CRS-2673: Attempting to stop 'ora.evmd' on 'racnode1' 
CRS-2673: Attempting to stop 'ora.asm' on 'racnode1' 
CRS-2673: Attempting to stop 'ora.mdnsd' on 'racnode1' 
CRS-2677: Stop of 'ora.evmd' on 'racnode1' succeeded 
CRS-2677: Stop of 'ora.ctssd' on 'racnode1' succeeded 
CRS-2677: Stop of 'ora.mdnsd' on 'racnode1' succeeded 
CRS-2677: Stop of 'ora.asm' on 'racnode1' succeeded 
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'racnode1' 
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'racnode1' succeeded 
CRS-2673: Attempting to stop 'ora.cssd' on 'racnode1' 
CRS-2677: Stop of 'ora.cssd' on 'racnode1' succeeded 
CRS-2673: Attempting to stop 'ora.gipcd' on 'racnode1' 
CRS-2677: Stop of 'ora.gipcd' on 'racnode1' succeeded 
CRS-2673: Attempting to stop 'ora.gpnpd' on 'racnode1' 
CRS-2677: Stop of 'ora.gpnpd' on 'racnode1' succeeded 
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'racnode1' has completed 
CRS-4133: Oracle High Availability Services has been stopped. 
Successfully unlock /u01/app/11.2.0/grid 

（节点2同样） 

If this is an Oracle Restart Home, as the root user execute: 
# /crs/install/roothas.pl -unlock 

3.Apply the CRS patch using.（升级crs，grid用户执行） 
As the GI home owner execute: 

$ /OPatch/opatch napply -oh -local /13867435 
$ /u01/app/11.2.0/grid/OPatch/opatch napply -oh /u01/app/11.2.0/grid -local /data/p13867435_112038_Linux-x86-64/13867435 

（节点2同样） 


4.Run the pre script for DB component of the patch.（升级db component组件，oracle执行） 
As the database home owner execute: 

$ /13867435/custom/server/13867435/custom/scripts/prepatch.sh -dbhome  
$ /data/p13867435_112038_Linux-x86-64/13867435/custom/server/13867435/custom/scripts/prepatch.sh -dbhome /u01/app/oracle/product/11.2.0/dbhome_1 

（节点2同样） 

5.Apply the DB patch.（升级db patch，oracle执行） 
As the database home owner execute: 

$ /OPatch/opatch napply -oh -local /13867435/custom/server/13867435 
$ /u01/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply -oh /u01/app/oracle/product/11.2.0/dbhome_1 -local /data/p13867435_112038_Linux-x86-64/13867435/custom/server/13867435 

（节点2同样） 

6.Run the post script for DB component of the patch.（这个是不是更新注册升级信息？） 
As the database home owner execute: 

$ /13867435/custom/server/13867435/custom/scripts/postpatch.sh -dbhome  
$ /data/p13867435_112038_Linux-x86-64/13867435/custom/server/13867435/custom/scripts/postpatch.sh -dbhome /u01/app/oracle/product/11.2.0/dbhome_1 

（节点2同样） 

7.Run the post script. 
As the root user execute: 

# /rdbms/install/rootadd_rdbms.sh 
# /u01/app/11.2.0/grid/rdbms/install/rootadd_rdbms.sh 

If this is a GI Home, as the root user execute:（if？ 这个要执行，下面那个不执行） 
# /crs/install/rootcrs.pl -patch 
# /u01/app/11.2.0/grid/crs/install/rootcrs.pl -patch 

（节点2同样） 

If this is an Oracle Restart Home, as the root user execute: 
# /crs/install/roothas.pl -patch 

8.If the message, "A system reboot is recommended before using ACFS” is shown, then a reboot must be issued before continuing. Failure to do so will result in running with an unpatched ACFS\ADVM\OKS driver. 

9.Start the CRS managed resources that were earlier running from DB homes. 

If this is a GI Home environment, as the database home owner execute:（oracle执行） 
$ /bin/srvctl start home -o -s -n  
[ oracle@racnode2 ~]$ /u01/app/oracle/product/11.2.0/dbhome_1/bin/srvctl start home -o /u01/app/oracle/product/11.2.0/dbhome_1 -s /home/oracle/state_file.txt -n racnode1 
[ oracle@racnode2 ~]$ ls 
oradiag_oracle state_file.txt 
[ oracle@racnode2 ~]$ cat state_file.txt 
db-racdb 

（节点2同样） 

If this is an Oracle Restart Home environment, as the database home owner execute: 
$ /bin/srvctl start home -o -s  


10.更新DB补丁记录（sysdba用户登录实例）、编辑无效对象。 
readme中没这一步，觉得不是必须 
SQL> @?/rdbms/admin/catbundle.sql psu apply 
SQL> @?/rdbms/admin/utlrp.sql 


7.确认补丁状态 
查询DB补丁记录 
set linesize 300 
col action_time for a30 
col version for a15 
col id for 99 
col action for a10 
col namespace for a10 
col COMMENTS for a20 
col BUNDLE_SERIES for a15 
select * from registry$history; 
查询补丁状态grid、oracle皆可 
$ $ORACLE_HOME/OPatch/opatch lspatches 
$ $ORACLE_HOME/OPatch/opatch lsinventory 

（节点2同样） 




++++++++++++++++++++++++++++++++++ 
升级过程中遇到的问题 
1.空间不足 
[ grid@racnode1 ~]$ /u01/app/11.2.0/grid/OPatch/opatch napply -oh /u01/app/11.2.0/grid -local /tmp/p13867435_112038_Linux-x86-64/13867435 
Oracle Interim Patch Installer version 11.2.0.3.6 
Copyright (c) 2013, Oracle Corporation. All rights reserved. 


Oracle Home : /u01/app/11.2.0/grid 
Central Inventory : /u01/app/oraInventory 
from : /u01/app/11.2.0/grid/oraInst.loc 
OPatch version : 11.2.0.3.6 
OUI version : 11.2.0.3.0 
Log file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2014-10-27_12-11-47PM_1.log 

Verifying environment and performing prerequisite checks... 
Prerequisite check "CheckSystemSpace" failed. 
The details are: 
Required amount of space(6754.328MB) is not available. 
UtilSession failed: 
Prerequisite check "CheckSystemSpace" failed. 
Log file location: /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2014-10-27_12-11-47PM_1.log 

OPatch failed with error code 73 

关闭虚拟机，resize sda大小（node2一样） 
D:\ProgramData\VirtualBox>vboxmanage modifyhd "D:\Vm\vxbox\racnode2\linux5 11g rac2.vdi" --resize 40000 

添加新分区 
fdisk -l 
fdisk /dev/sda 

mkfs -t ext3 /dev/sda3 
mkdir -p /data 
mount /dev/sda3 /data 
vi /etc/fstab 
/dev/sda3 /data ext3 defaults 1 2 



--------------------------- 
2.升级造成的权限问题 
搜索词： ORA 01078  ORA 01565  ORA 17503  ORA 27123 
SQL> startup 
ORA-01078: failure in processing system parameters 
ORA-01565: error in identifying file '+DATA/racdb/spfileracdb.ora' 
ORA-17503: ksfdopn:2 Failed to open file +DATA/racdb/spfileracdb.ora 
ORA-01034: ORACLE not available 
ORA-27123: unable to attach to shared memory segment 
Linux-x86_64 Error: 13: Permission denied 
SQL> 
-------- 
[ grid@racnode2 ~]$ ls -l /u01/app/11.2.0/grid/bin/oracle 
-rwxr-x--x 1 grid oinstall 203974257 Oct 17 09:01 /u01/app/11.2.0/grid/bin/oracle 
[ grid@racnode2 ~]$ 
[ grid@racnode2 ~]$ 
[ grid@racnode2 ~]$ chmod 6751 /u01/app/11.2.0/grid/bin/oracle 
[ grid@racnode2 ~]$ 
[ grid@racnode2 ~]$ 
[ grid@racnode2 ~]$ ls -l /u01/app/11.2.0/grid/bin/oracle 
-rwsr-s--x 1 grid oinstall 203974257 Oct 17 09:01 /u01/app/11.2.0/grid/bin/oracle

startup 成功

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/21699095/viewspace-1312027/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/21699095/viewspace-1312027/