环境说明
OS version:Redhat 7.6
db version:19.3.0.0
PSU version:19.9.0.0.201020
patch number:Patch 31720396
安装前以grid用户解压补丁包
chmod -R 777 31750108/
一定要给777的权限,一定要给777的权限,一定要给777的权限,重要的事说三遍。
具体原因没有深入研究,没有777的权限的话首次安装grid PSU会失败;
安装补丁失败,提示文件权限异常,并且2个节点补丁号不一致;
[grid@nmgldb1 ~]$ opatch lspatches
[grid@nmgldb1 ~]$ kfod op=patches
[grid@nmgldb1 ~]$ kfod op=patchlvl
执行结果与2节点完全不一致,回退补丁命令无法执行;
出现补丁不一致的情况可以加-norolling参数回退单节点补丁
# /OPatch/opatchauto rollback /31750108 -oh -norolling
1.1安装grid PSU(以root用户执行,执行前引入grid环境变量)1节点打PSU
[root@nmgldb1 PSU]# export ORACLE_BASE=/oracle/app/grid[root@nmgldb1 PSU]# export ORACLE_HOME=/oracle/app/19.0.0/grid[root@nmgldb1 PSU]# export PATH=${ORACLE_HOME}/bin:${ORACLE_HOME}/OPatch:${PATH}[root@nmgldb1 PSU]# time opatchauto apply /oracle/soft/PSU/31750108 -oh /oracle/app/19.0.0/gridOPatchauto session is initiated at Sat Oct 31 22:34:16 2020System initialization log file is /oracle/app/19.0.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-10-31_10-34-20PM.log.Session log file is /oracle/app/19.0.0/grid/cfgtoollogs/opatchauto/opatchauto2020-10-31_10-34-52PM.logThe id for this session is EC88Executing OPatch prereq operations to verify patch applicability on home /oracle/app/19.0.0/gridPatch applicability verified successfully on home /oracle/app/19.0.0/gridBringing down CRS service on home /oracle/app/19.0.0/gridCRS service brought down successfully on home /oracle/app/19.0.0/gridStart applying binary patch on home /oracle/app/19.0.0/gridBinary patch applied successfully on home /oracle/app/19.0.0/gridStarting CRS service on home /oracle/app/19.0.0/gridCRS service started successfully on home /oracle/app/19.0.0/gridOPatchAuto successful.--------------------------------Summary--------------------------------Patching is completed successfully. Please find the summary as follows:Host:nmgldb1CRS Home:/oracle/app/19.0.0/gridVersion:19.0.0.0.0Summary:==Following patches were SUCCESSFULLY applied:Patch: /oracle/soft/PSU/31750108/31771877Log: /oracle/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-31_22-38-28PM_1.logPatch: /oracle/soft/PSU/31750108/31772784Log: /oracle/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-31_22-38-28PM_1.logPatch: /oracle/soft/PSU/31750108/31773437Log: /oracle/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-31_22-38-28PM_1.logPatch: /oracle/soft/PSU/31750108/31780966Log: /oracle/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-31_22-38-28PM_1.logOPatchauto session completed at Sat Oct 31 22:51:46 2020Time taken to complete the session 17 minutes, 31 secondsreal 17m30.990suser 11m36.815ssys 3m20.087s
检查1节点grid补丁
[grid@nmgldb1 ~]$ opatch lspatches31780966;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31780966)31773437;ACFS RELEASE UPDATE 19.9.0.0.0 (31773437)31772784;OCW RELEASE UPDATE 19.9.0.0.0 (31772784)31771877;Database Release Update : 19.9.0.0.201020 (31771877)OPatch succeeded.[grid@nmgldb1 ~]$ kfod op=patches---------------List of Patches===============31771877317727843177343731780966[grid@nmgldb1 ~]$ kfod op=patchlvl-------------------Current Patch level===================376483838
1.2安装db PSU(以root用户执行,执行前引入db环境变量)
[root@nmgldb1 PSU]# export ORACLE_BASE=/oracle/app/oracle[root@nmgldb1 PSU]# export ORACLE_HOME=/oracle/app/oracle/product/19.0.0/db_1[root@nmgldb1 PSU]# export PATH=${ORACLE_HOME}/bin:${ORACLE_HOME}/OPatch:${PATH}[root@nmgldb1 PSU]# time opatchauto apply /oracle/soft/PSU/31750108 -oh /oracle/app/oracle/product/19.0.0/db_1OPatchauto session is initiated at Sat Oct 31 22:53:47 2020System initialization log file is /oracle/app/oracle/product/19.0.0/db_1/cfgtoollogs/opatchautodb/systemconfig2020-10-31_10-53-51PM.log.Session log file is /oracle/app/oracle/product/19.0.0/db_1/cfgtoollogs/opatchauto/opatchauto2020-10-31_10-54-44PM.logThe id for this session is IVQAExecuting OPatch prereq operations to verify patch applicability on home /oracle/app/oracle/product/19.0.0/db_1Patch applicability verified successfully on home /oracle/app/oracle/product/19.0.0/db_1Verifying SQL patch applicability on home /oracle/app/oracle/product/19.0.0/db_1No step execution required.........Preparing to bring down database service on home /oracle/app/oracle/product/19.0.0/db_1No step execution required.........Performing prepatch operation on home /oracle/app/oracle/product/19.0.0/db_1Perpatch operation completed successfully on home /oracle/app/oracle/product/19.0.0/db_1Start applying binary patch on home /oracle/app/oracle/product/19.0.0/db_1Binary patch applied successfully on home /oracle/app/oracle/product/19.0.0/db_1Performing postpatch operation on home /oracle/app/oracle/product/19.0.0/db_1Postpatch operation completed successfully on home /oracle/app/oracle/product/19.0.0/db_1Preparing home /oracle/app/oracle/product/19.0.0/db_1 after database service restartedNo step execution required.........Trying to apply SQL patch on home /oracle/app/oracle/product/19.0.0/db_1No step execution required.........OPatchAuto successful.--------------------------------Summary--------------------------------Patching is completed successfully. Please find the summary as follows:Host:nmgldb1RAC Home:/oracle/app/oracle/product/19.0.0/db_1Version:19.0.0.0.0Summary:==Following patches were SKIPPED:Patch: /oracle/soft/PSU/31750108/31773437Reason: This patch is not applicable to this specified target type - "rac_database"Patch: /oracle/soft/PSU/31750108/31780966Reason: This patch is not applicable to this specified target type - "rac_database"==Following patches were SUCCESSFULLY applied:Patch: /oracle/soft/PSU/31750108/31771877Log: /oracle/app/oracle/product/19.0.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-31_22-55-24PM_1.logPatch: /oracle/soft/PSU/31750108/31772784Log: /oracle/app/oracle/product/19.0.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-31_22-55-24PM_1.logOPatchauto session completed at Sat Oct 31 22:59:39 2020Time taken to complete the session 5 minutes, 53 secondsreal 5m52.902suser 5m54.019ssys 1m44.563s
2节点打PSU
2.1安装grid PSU(过程跟1节点一样)
[root@nmgldb2 ~]# export ORACLE_BASE=/oracle/app/grid[root@nmgldb2 ~]# export ORACLE_HOME=/oracle/app/19.0.0/grid[root@nmgldb2 ~]# export PATH=${ORACLE_HOME}/bin:${ORACLE_HOME}/OPatch:${PATH}[root@nmgldb2 ~]# time opatchauto apply /oracle/soft/PSU/31750108 -oh /oracle/app/19.0.0/gridOPatchauto session is initiated at Sat Oct 31 23:01:39 2020System initialization log file is /oracle/app/19.0.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-10-31_11-01-44PM.log.Session log file is /oracle/app/19.0.0/grid/cfgtoollogs/opatchauto/opatchauto2020-10-31_11-02-17PM.logThe id for this session is V4G5Executing OPatch prereq operations to verify patch applicability on home /oracle/app/19.0.0/gridPatch applicability verified successfully on home /oracle/app/19.0.0/gridBringing down CRS service on home /oracle/app/19.0.0/gridCRS service brought down successfully on home /oracle/app/19.0.0/gridStart applying binary patch on home /oracle/app/19.0.0/gridBinary patch applied successfully on home /oracle/app/19.0.0/gridStarting CRS service on home /oracle/app/19.0.0/gridCRS service started successfully on home /oracle/app/19.0.0/gridOPatchAuto successful.--------------------------------Summary--------------------------------Patching is completed successfully. Please find the summary as follows:Host:nmgldb2CRS Home:/oracle/app/19.0.0/gridVersion:19.0.0.0.0Summary:==Following patches were SUCCESSFULLY applied:Patch: /oracle/soft/PSU/31750108/31771877Log: /oracle/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-31_23-05-53PM_1.logPatch: /oracle/soft/PSU/31750108/31772784Log: /oracle/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-31_23-05-53PM_1.logPatch: /oracle/soft/PSU/31750108/31773437Log: /oracle/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-31_23-05-53PM_1.logPatch: /oracle/soft/PSU/31750108/31780966Log: /oracle/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-31_23-05-53PM_1.logOPatchauto session completed at Sat Oct 31 23:19:20 2020Time taken to complete the session 17 minutes, 41 secondsreal 17m41.415suser 11m46.140ssys 3m6.374s
检查2节点grid补丁
[grid@nmgldb2 ~]$ opatch lspatches31780966;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31780966)31773437;ACFS RELEASE UPDATE 19.9.0.0.0 (31773437)31772784;OCW RELEASE UPDATE 19.9.0.0.0 (31772784)31771877;Database Release Update : 19.9.0.0.201020 (31771877)OPatch succeeded.[grid@nmgldb2 ~]$ kfod op=patches---------------List of Patches===============31771877317727843177343731780966[grid@nmgldb2 ~]$ kfod op=patchlvl-------------------Current Patch level===================376483838
2.2安装db PSU(首次安装会报错,安装补丁前检查一下/oracle/app/oraInventory/ContentsXML/路径下文件权限是否与1节点一致,不一致的话改成与一节点一致,可能就不会出错了)
[root@nmgldb2 ~]# export ORACLE_BASE=/oracle/app/oracle[root@nmgldb2 ~]# export ORACLE_HOME=/oracle/app/oracle/product/19.0.0/db_1[root@nmgldb2 ~]# export PATH=${ORACLE_HOME}/bin:${ORACLE_HOME}/OPatch:${PATH}[root@nmgldb2 ~]# time opatchauto apply /oracle/soft/PSU/31750108 -oh /oracle/app/oracle/product/19.0.0/db_1OPatchauto session is initiated at Sat Oct 31 23:21:24 2020System initialization log file is /oracle/app/oracle/product/19.0.0/db_1/cfgtoollogs/opatchautodb/systemconfig2020-10-31_11-21-29PM.log.Session log file is /oracle/app/oracle/product/19.0.0/db_1/cfgtoollogs/opatchauto/opatchauto2020-10-31_11-21-49PM.logThe id for this session is WE7LExecuting OPatch prereq operations to verify patch applicability on home /oracle/app/oracle/product/19.0.0/db_1Patch applicability verified successfully on home /oracle/app/oracle/product/19.0.0/db_1Verifying SQL patch applicability on home /oracle/app/oracle/product/19.0.0/db_1No step execution required.........Preparing to bring down database service on home /oracle/app/oracle/product/19.0.0/db_1No step execution required.........Performing prepatch operation on home /oracle/app/oracle/product/19.0.0/db_1Perpatch operation completed successfully on home /oracle/app/oracle/product/19.0.0/db_1Start applying binary patch on home /oracle/app/oracle/product/19.0.0/db_1Failed while applying binary patches on home /oracle/app/oracle/product/19.0.0/db_1Execution of [OPatchAutoBinaryAction] patch action failed, check log for more details. Failures:Patch Target : nmgldb2->/oracle/app/oracle/product/19.0.0/db_1 Type[rac]Details: [---------------------------Patching Failed---------------------------------Command execution failed during patching in home: /oracle/app/oracle/product/19.0.0/db_1, host: nmgldb2.Command failed: /oracle/app/oracle/product/19.0.0/db_1/OPatch/opatchauto apply /oracle/soft/PSU/31750108 -oh /oracle/app/oracle/product/19.0.0/db_1 -target_type rac_database -binary -invPtrLoc /oracle/app/oracle/product/19.0.0/db_1/oraInst.loc -jre /oracle/app/oracle/product/19.0.0/db_1/OPatch/jre -persistresult /oracle/app/oracle/product/19.0.0/db_1/opatchautocfg/db/sessioninfo/sessionresult_nmgldb2_rac_2.ser -analyzedresult /oracle/app/oracle/product/19.0.0/db_1/opatchautocfg/db/sessioninfo/sessionresult_analyze_nmgldb2_rac_2.serCommand failure output:==Following patches FAILED in apply:Patch: /oracle/soft/PSU/31750108/31771877Log: /oracle/app/oracle/product/19.0.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-31_23-22-33PM_1.logReason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException: ApplySession failed in system modification phase... 'ApplySession::apply failed: java.io.IOException: oracle.sysman.oui.patch.PatchException: java.io.FileNotFoundException: /oracle/app/oraInventory/ContentsXML/oui-patch.xml (Permission denied)'After fixing the cause of failure Run opatchauto resume]OPATCHAUTO-68061: The orchestration engine failed.OPATCHAUTO-68061: The orchestration engine failed with return code 1OPATCHAUTO-68061: Check the log for more details.OPatchAuto failed.OPatchauto session completed at Sat Oct 31 23:25:56 2020Time taken to complete the session 4 minutes, 32 secondsopatchauto failed with error code 42real 4m32.534suser 4m51.153ssys 1m34.999s
根据提示报错检查
/oracle/app/oraInventory/ContentsXML/oui-patch.xml文件,与1节点对比,发现权限确实不对,然后按照1节点修改/oracle/app/oraInventory/ContentsXML/路径下所有文件权限;
2.3修改/oracle/app/oraInventory/ContentsXML/路径下文件权限后,第二次安装db PSU
[root@nmgldb2 ~]# time opatchauto apply /oracle/soft/PSU/31750108 -oh /oracle/app/oracle/product/19.0.0/db_1/OPatchauto session is initiated at Sun Nov 1 00:34:40 2020System initialization log file is /oracle/app/oracle/product/19.0.0/db_1/cfgtoollogs/opatchautodb/systemconfig2020-11-01_12-34-45AM.log.Session log file is /oracle/app/oracle/product/19.0.0/db_1/cfgtoollogs/opatchauto/opatchauto2020-11-01_12-35-06AM.logThe id for this session is VQ1KExecuting OPatch prereq operations to verify patch applicability on home /oracle/app/oracle/product/19.0.0/db_1Patch applicability verification failed on home /oracle/app/oracle/product/19.0.0/db_1Execution of [OPatchAutoBinaryAction] patch action failed, check log for more details. Failures:Patch Target : nmgldb2->/oracle/app/oracle/product/19.0.0/db_1 Type[rac]Details: [---------------------------Patching Failed---------------------------------Command execution failed during patching in home: /oracle/app/oracle/product/19.0.0/db_1, host: nmgldb2.Command failed: /oracle/app/oracle/product/19.0.0/db_1/OPatch/opatchauto apply /oracle/soft/PSU/31750108 -oh /oracle/app/oracle/product/19.0.0/db_1 -target_type rac_database -binary -invPtrLoc /oracle/app/oracle/product/19.0.0/db_1//oraInst.loc -jre /oracle/app/oracle/product/19.0.0/db_1/OPatch/jre -persistresult /oracle/app/oracle/product/19.0.0/db_1/opatchautocfg/db/sessioninfo/sessionresult_analyze_nmgldb2_rac_2.ser -analyze -online -prepare_homeCommand failure output:==Following patches FAILED in analysis for apply:Patch: /oracle/soft/PSU/31750108/31772784Log:Reason: Failed during listing in Analysis: java.lang.Exception: oracle.opatch.opatchsdk.OPatchException: Unable to create patchObjectPossible causes are: ORACLE_HOME/inventory/oneoffs/31771877 is corrupted. PatchObject constructor: Input file "/oracle/app/oracle/product/19.0.0/db_1/inventory/oneoffs/31771877/etc/config/actions" or "/oracle/app/oracle/product/19.0.0/db_1/inventory/oneoffs/31771877/etc/config/inventory" does not exist.Patch: /oracle/soft/PSU/31750108/31771877Log:Reason: Failed during listing in Analysis: java.lang.Exception: oracle.opatch.opatchsdk.OPatchException: Unable to create patchObjectPossible causes are: ORACLE_HOME/inventory/oneoffs/31771877 is corrupted. PatchObject constructor: Input file "/oracle/app/oracle/product/19.0.0/db_1/inventory/oneoffs/31771877/etc/config/actions" or "/oracle/app/oracle/product/19.0.0/db_1/inventory/oneoffs/31771877/etc/config/inventory" does not exist.After fixing the cause of failure Run opatchauto resume]OPATCHAUTO-68061: The orchestration engine failed.OPATCHAUTO-68061: The orchestration engine failed with return code 1OPATCHAUTO-68061: Check the log for more details.OPatchAuto failed.OPatchauto session completed at Sun Nov 1 00:35:32 2020Time taken to complete the session 0 minute, 52 secondsopatchauto failed with error code 42real 0m51.975suser 1m24.131ssys 0m18.654s
悲催了,这次直接提示 ORACLE_HOME/inventory/oneoffs/31771877 is corrupted,检查这个路径果然缺文件;
尝试把1节点的文件传过来:
[oracle@nmgldb2 ~]$ scp -r ORACLE_HOME/inventory/oneoffs/31771877 nmgldb2:/oracle/app/oracle/product/19.0.0/db_1/inventory/oneoffs/
2.4把1节点文件同步到2节点后,第三次安装db PSU
[root@nmgldb2 ~]# time opatchauto apply /oracle/soft/PSU/31750108 -oh /oracle/app/oracle/product/19.0.0/db_1/OPatchauto session is initiated at Sun Nov 1 01:13:52 2020System initialization log file is /oracle/app/oracle/product/19.0.0/db_1/cfgtoollogs/opatchautodb/systemconfig2020-11-01_01-13-57AM.log.Session log file is /oracle/app/oracle/product/19.0.0/db_1/cfgtoollogs/opatchauto/opatchauto2020-11-01_01-14-18AM.logThe id for this session is BTXIExecuting OPatch prereq operations to verify patch applicability on home /oracle/app/oracle/product/19.0.0/db_1Patch applicability verified successfully on home /oracle/app/oracle/product/19.0.0/db_1Verifying SQL patch applicability on home /oracle/app/oracle/product/19.0.0/db_1No step execution required.........Preparing to bring down database service on home /oracle/app/oracle/product/19.0.0/db_1No step execution required.........Performing prepatch operation on home /oracle/app/oracle/product/19.0.0/db_1Perpatch operation completed successfully on home /oracle/app/oracle/product/19.0.0/db_1Start applying binary patch on home /oracle/app/oracle/product/19.0.0/db_1Binary patch applied successfully on home /oracle/app/oracle/product/19.0.0/db_1Performing postpatch operation on home /oracle/app/oracle/product/19.0.0/db_1Postpatch operation completed successfully on home /oracle/app/oracle/product/19.0.0/db_1Preparing home /oracle/app/oracle/product/19.0.0/db_1 after database service restartedNo step execution required.........Trying to apply SQL patch on home /oracle/app/oracle/product/19.0.0/db_1No step execution required.........OPatchAuto successful.--------------------------------Summary--------------------------------Patching is completed successfully. Please find the summary as follows:Host:nmgldb2RAC Home:/oracle/app/oracle/product/19.0.0/db_1Version:19.0.0.0.0Summary:==Following patches were SKIPPED:Patch: /oracle/soft/PSU/31750108/31773437Reason: This patch is not applicable to this specified target type - "rac_database"Patch: /oracle/soft/PSU/31750108/31780966Reason: This patch is not applicable to this specified target type - "rac_database"Patch: /oracle/soft/PSU/31750108/31771877Reason: This patch is already been applied, so not going to apply again.==Following patches were SUCCESSFULLY applied:Patch: /oracle/soft/PSU/31750108/31772784Log: /oracle/app/oracle/product/19.0.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-11-01_01-14-59AM_1.logOPatchauto session completed at Sun Nov 1 01:15:58 2020Time taken to complete the session 2 minutes, 6 secondsreal 2m6.060suser 3m13.387ssys 0m30.439s