oracle 19c打补丁遭遇OPATCHAUTO-72043&OPATCHAUTO-68061

    最近，在AIX上的新装oracle 19C数据库基础版本，使用opatchauto打PSU补丁集35037840时遇到了OPATCHAUTO-72043报错，无法正常应用GI补丁。

一、环境描述

操作系统：AIX

数据库版本：oracle rac 19.3.0新装基础版

应用PSU补丁号：35037840

二、问题现象

    opatchauto给GI打补丁时，报错退出，部分核心报错内容如下：

2023-04-23 15:38:33,411 WARNING [1] com.oracle.glcm.patch.auto.credential.CredentialManager - Unable to locate credential for host orcldb01
2023-04-23 15:38:33,475 INFO  [1] com.oracle.glcm.patch.auto.db.framework.core.patch.impl.PatchPackageFactoryImpl - Entering getPatchPackageFromDir, getting patch object for the given patch location /u01/psu/35037840
2023-04-23 15:38:34,394 INFO  [1] com.oracle.glcm.patch.auto.db.framework.core.patch.OPatchBundlePatchValidatorAndGenerator - Bundle.xml does not exist
2023-04-23 15:38:34,394 INFO  [1] com.oracle.glcm.patch.auto.db.framework.core.patch.OPatchSingletonPatchValidatorAndGenerator - inventory.xml found at /u01/psu/35037840/35050331/etc/config/inventory.xml which implies it is an OPatch Singleton patch
2023-04-23 15:38:34,397 INFO  [1] com.oracle.glcm.patch.auto.db.framework.core.patch.PatchMetadaReader - Command for patch metadata::/u01/app/19.0.0/grid/perl/bin/perl  /u01/app/19.0.0/grid/OPatch/auto/database/bin/OPatchAutoBinary.pl query patchinfo -patch_location /u01/psu/35037840/35050331 -result /u01/app/19.0.0/grid/OPatch/auto/dbtmp/result.ser
2023-04-23 15:38:35,850 INFO  [1] com.oracle.glcm.patch.auto.db.product.executor.GISystemCall - Is retry required=false
2023-04-23 15:38:35,851 WARNING [1] com.oracle.glcm.patch.auto.db.integration.model.plan.PatchingProcessInitializer - Patch Collection failed: 
2023-04-23 15:38:36,094 INFO  [1] com.oracle.glcm.patch.auto.db.integration.model.productsupport.DBBaseProductSupport - Space available after session: 150978 MB
2023-04-23 15:38:36,130 SEVERE [1] com.oracle.glcm.patch.auto.OPatchAuto - OPatchAuto failed.
com.oracle.glcm.patch.auto.OPatchAutoException: OPATCHAUTO-72043: Patch collection failed.
OPATCHAUTO-72043: Failed to create bundle patch object.
OPATCHAUTO-72043: Please verify the patch supplied.
	at com.oracle.glcm.patch.auto.db.integration.model.plan.PatchingProcessInitializer.initializePatchPackageBag(PatchingProcessInitializer.java:97)
	at com.oracle.glcm.patch.auto.db.integration.model.plan.PatchingProcessInitializer.processInit(PatchingProcessInitializer.java:66)
	at com.oracle.glcm.patch.auto.db.integration.model.productsupport.DBCommonSupport.initializePatchData(DBCommonSupport.java:126)
	at com.oracle.glcm.patch.auto.db.integration.model.productsupport.DBBaseProductSupport.initializePatchData(DBBaseProductSupport.java:425)
	at com.oracle.glcm.patch.auto.db.integration.model.productsupport.DBCommonSupport.loadTopology(DBCommonSupport.java:163)
	at com.oracle.glcm.patch.auto.db.integration.model.productsupport.DBBaseProductSupport.loadTopology(DBBaseProductSupport.java:201)
	at com.oracle.glcm.patch.auto.db.integration.model.productsupport.DBProductSupport.loadTopology(DBProductSupport.java:69)
	at com.oracle.glcm.patch.auto.OPatchAuto.loadTopology(OPatchAuto.java:1732)
	at com.oracle.glcm.patch.auto.OPatchAuto.prepareOrchestration(OPatchAuto.java:730)
	at com.oracle.glcm.patch.auto.OPatchAuto.orchestrate(OPatchAuto.java:397)
	at com.oracle.glcm.patch.auto.OPatchAuto.orchestrate(OPatchAuto.java:344)
	at com.oracle.glcm.patch.auto.OPatchAuto.main(OPatchAuto.java:212)
Caused by: com.oracle.glcm.patch.auto.db.integration.model.productsupport.patch.PatchCollectionException: 
	at com.oracle.glcm.patch.auto.db.framework.core.patch.OPatchSingletonPatchValidatorAndGenerator.generate(OPatchSingletonPatchValidatorAndGenerator.java:151)
	at oracle.dbsysmodel.patchsdk.PatchFactory.getInstance(PatchFactory.java:246)
	at com.oracle.glcm.patch.auto.db.framework.core.patch.impl.PatchPackageFactoryImpl.getPatchPackageFromDir(PatchPackageFactoryImpl.java:75)
	at com.oracle.glcm.patch.auto.db.product.patch.BundlePatchObject.prcoessPatch(BundlePatchObject.java:131)
	at com.oracle.glcm.patch.auto.db.product.patch.BundlePatchObject.createPatchBagForPatchLocation(BundlePatchObject.java:124)
	at com.oracle.glcm.patch.auto.db.product.patch.BundlePatchObject.<init>(BundlePatchObject.java:77)
	at com.oracle.glcm.patch.auto.db.integration.model.productsupport.patch.PatchInformationInitializer.createPatch(PatchInformationInitializer.java:43)
	at com.oracle.glcm.patch.auto.db.integration.model.plan.PatchingProcessInitializer.initializePatchPackageBag(PatchingProcessInitializer.java:80)
	... 11 more
2023-04-23 15:38:36,131 INFO  [1] com.oracle.cie.common.util.reporting.CommonReporter - Reporting console output : Message{id='null', message='OPATCHAUTO-72043: Patch collection failed.
OPATCHAUTO-72043: Failed to create bundle patch object.
OPATCHAUTO-72043: Please verify the patch supplied.'}
2023-04-23 15:38:36,131 INFO  [1] com.oracle.cie.common.util.reporting.CommonReporter - Reporting console output : Message{id='null', message='OPatchAuto failed.'}

Patch: /u01/psu/35037840/35107512
Log: /u01/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2023-04-23 10-15-26AM 1.log
Reason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException:
Prerequisite check "checkApplicable"failed
After fixing the cause of failure Run opatchauto resume
OPATCHAUTO-68061:The orchestrationengine failed
OPATCHAUTO-68061:The orchestrationengine failed with return code 1
OPATCHAUTO-68061:Check the log for more details .OPatchAuto failed
OPatchauto session completed at Thu APR 23 10:16;40 2023
Time taken to complete the session 7 minutes, 34 second:
opatchauto failed with error code 42

三、问题分析

    关于autopatch GI打补丁报错OPATCHAUTO-72043，查询oracle官网MOS，oracle官网建议将对应数据库版本最新版opatch工具直接解压到oracle和grid的ORACLE_HOME/OPatch下的隐藏文件宿主；经沟通，负责打PSU的同事，在解压opatch之前没有删除原始的OPatch目录而是直接解压。

 

    关于autopatch打补丁过程中提示OPATCHAUTO-68061，经查看日志/u01/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2023-04-23 10-15-26AM 1.log，发现是打补丁过程中程序删除部分文件权限不够：

[Apr 23, 2023 10:16:16 AM] [INFO]   Prereq checkPatchApplicableOnCurrentPlatform Passed on patch :35107512
[Apr 23, 2023 10:16:18 AM] [INFO]   Patch 35042068:
                                    DeleteAction : Destination File "/u01/app/19.0.0/grid/perl/bin/perl" is not writeable.
                                    'oracle.perlint, 5.28.1.0.0': Cannot delete file '/u01/app/19.0.0/grid/perl/bin/perl'
                                    Copy Action: Destination File "/u01/app/19.0.0/grid/perl/bin/perl" is not writeable.
                                    'oracle.perlint, 5.28.1.0.0': Cannot copy file from 'perl' to '/u01/app/19.0.0/grid/perl/bin/perl'
[Apr 23, 2023 10:16:19 AM] [INFO]   Prerequisite check "CheckApplicable" failed.
                                    The details are:
                                    
                                    Patch 35042068:
                                    DeleteAction : Destination File "/u01/app/19.0.0/grid/perl/bin/perl" is not writeable.
                                    'oracle.perlint, 5.28.1.0.0': Cannot delete file '/u01/app/19.0.0/grid/perl/bin/perl'
                                    Copy Action: Destination File "/u01/app/19.0.0/grid/perl/bin/perl" is not writeable.
                                    'oracle.perlint, 5.28.1.0.0': Cannot copy file from 'perl' to '/u01/app/19.0.0/grid/perl/bin/perl'
[Apr 23, 2023 10:16:19 AM] [INFO]   UtilSession failed: 
                                    Prerequisite check "CheckApplicable" failed.
[Apr 23, 2023 10:16:19 AM] [SEVERE] OUI-67073:UtilSession failed: 
                                    Prerequisite check "CheckApplicable" failed.
[Apr 23, 2023 10:16:19 AM] [INFO]   Finishing UtilSession at Thu Apr 23 10:16:19 GMT+08:00 2023
[Apr 23, 2023 10:16:19 AM] [INFO]   Log file location: /u01/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2023-07-20_10-15-26AM_1.log

四、问题处理

    关于autopatch GI打补丁报错OPATCHAUTO-72043，清除掉OPatch目录，重新解压最新的opatch工具并赋权后，重新使用opatchauto打补丁能够执行。

    关于autopatch GI打补丁报错OPATCHAUTO-68061，oracle 官网文档12202307.9中提到可能是已知问题Bug 34998885.

 经排查确实有未退出的ASMCMD客户端，退出后再次重新应用补丁，顺利完成。