oracle not in一个变量,一个全局变量引发的血案 | 信春哥，系统稳，闭眼上线不回滚！...

今天在测试10.2.0.4.4版本的RAC升级11.2.0.3.0版本的RAC时，跑rootupgrade脚本的时候，报关键补丁没有打的错误，开始一直以为是10.2.0.4.4的补丁打的有问题，害得我反复打了好几次补丁，还是不行。

[root@maa1 ~]# /u01/app/grid/product/11.2.0/grid/rootupgrade.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= oracle

ORACLE_HOME= /u01/app/grid/product/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /u01/app/grid/product/11.2.0/grid/crs/install/crsconfig_params

Creating trace directory

User ignored Prerequisites during installation

cluvfy patch check failed, output is

Performing pre-checks for cluster services setup

The cluvfy tool found some mandatory patches are not installed.

These patches need to be installed before the upgrade can proceed.

The pre-upgrade checks failed, aborting the upgrade

/u01/app/grid/product/11.2.0/grid/perl/bin/perl -I/u01/app/grid/product/11.2.0/grid/perl/lib -I/u01/app/grid/product/11.2.0/grid/crs/install /u01/app/grid/product/11.2.0/grid/crs/install/rootcrs.pl execution failed

查看MOS，有两篇文章和这个案例很像，一篇说是CVU的BUG导致，需要更新下CVU，还有一篇文章说是不能设置ORA_CRS_HOME环境变量，否则就可能一刀这个问题。在更新CVU后，问题依旧，那就只能对ORA_CRS_HOME变量下手了，取消这个变量后，问题还没得到解决。

通过debug CVU信息，追踪到CVU的日志信息。

[14571@maa1] [main] [ 2014-03-21 07:36:50.712 CST ] [SRVCTLUtil.getVersion:173] Command output for SRVCTL -V issrvctl version: 11.2.0.3.0

[14571@maa1] [main] [ 2014-03-21 07:36:50.712 CST ] [SRVCTLUtil.getVersionFromResult:208] Parsing Output line:srvctl version: 11.2.0.3.0

[14571@maa1] [main] [ 2014-03-21 07:36:50.712 CST ] [SRVCTLUtil.getVersionFromResult:226] First Token is srvctl version, checking if it contains srvctl before the colon

[14571@maa1] [main] [ 2014-03-21 07:36:50.713 CST ] [Version$VersionEnum.getEnumMember:196] Version Match Successful: returning version object 11.2.0.3

[14571@maa1] [main] [ 2014-03-21 07:36:50.713 CST ] [VerificationUtil.getCRSHome:3053] getCRSHome(): ERROR instantiating the OUIData object

[14571@maa1] [main] [ 2014-03-21 07:36:50.713 CST ] [VerificationUtil.getCRSHome:3054]

PRVF-5311 : File "/etc/oracle/olr.loc" either does not exist or is not accessible on node "localnode".

[14571@maa1] [main] [ 2014-03-21 07:36:50.713 CST ] [VerificationUtil.getCRSActiveVersion:6117] ERROR: CRS home undefined.

Exception in thread "main" java.lang.NullPointerException

at oracle.ops.mgmt.cluster.Version.isPre(Version.java:528)

at oracle.ops.mgmt.cluster.Version.isPre10i(Version.java:388)

at oracle.ops.mgmt.nativesystem.UnixSystem.getConfigLocation(UnixSystem.java:2325)

at oracle.ops.verification.framework.util.VerificationUtil.getOCRLocationsUnix(VerificationUtil.java:9652)

at oracle.ops.verification.framework.util.VerificationUtil.getOCRLocations(VerificationUtil.java:9633)

at oracle.ops.verification.framework.engine.task.TaskFactory.addOCRIntegrityChecks(TaskFactory.java:1371)

at oracle.ops.verification.framework.engine.task.TaskFactory.getTaskListPreCRSInst(TaskFactory.java:855)

at oracle.ops.verification.framework.engine.task.TaskFactory.getTaskList(TaskFactory.java:395)

at oracle.ops.verification.framework.engine.task.TaskFactory.getTaskList(TaskFactory.java:621)

at oracle.ops.verification.framework.engine.stage.ClusterSvcSetupStage.setTaskList(ClusterSvcSetupStage.java:377)

at oracle.ops.verification.framework.engine.stage.Stage.verify(Stage.java:470)

at oracle.ops.verification.framework.engine.ClusterVerifier.verifyStage(ClusterVerifier.java:147)

at oracle.ops.verification.client.CluvfyDriver.main(CluvfyDriver.java:311)

这些错误信息在MOS是上可以找到几乎一模一样的案例，很明确是设置了ORA_CRS_HOME变量导致的，那么为什么取消这个变量还不行呢？

[root@maa1 ~]# env | grep ORA_CRS_HOME

ORA_CRS_HOME=/u01/app/oracle/product/10.2.0/crs_1

[oracle@maa1 ~]$ env | grep ORA_CRS_HOME

ORA_CRS_HOME=/u01/app/oracle/product/10.2.0/crs_1

在取消ORA_CRS_HOME变量后，root和oracle用户都有ORA_CRS_HOME变量，怎么回事？难道是设置了全局变量？

[root@maa1 ~]# cat /etc/profile

alias vi='vim'

alias rm='rm -i'

alias cp='cp -i'

alias mv='mv -i'

LC_MESSAGES=en_US.UTF-8

export LC_MESSAGES

PATH=$PATH:$HOME/bin export PATH

ORACLE_BASE=/u01/app/oracle;export ORACLE_BASE

ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1;export ORACLE_HOME

ORA_CRS_HOME=$ORACLE_BASE/product/10.2.0/crs_1;export ORA_CRS_HOME

ORACLE_TERM=xterm;export ORACLE_TERM

PATH=$ORACLE_HOME/bin:$ORA_CRS_HOME/bin:$PATH;export PATH

ORACLE_OWNER=oracle;export ORACLE_OWNER

ORACLE_SID=orcl;export ORACLE_SID

LD_LIBRARY_PATH=$ORACLE_HOME/lib;export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE/lib:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

CLASSPATH=$CLASSPATH:$ORACLE_HOME/network/jlib;export CLASSPATH

TMP=/tmp;export TMP

TMPDIR=$TMP;export TMPDIR

EDITOR=vim;export EDITOR

太坑人了，原来SA在安装环境的时候，设置了全局变量，所以虽然修改oracle用户的.bash_profile文件注释掉了ORA_CRS_HOME变量，当新会话连接到服务器，会先检查本地的.bash_profile文件，然后会在检查全局的/etc/profile文件，如果用户本地有变量，以本地变量为准，如果本地没有设置变量而全局设置了，就使用全局的变量。所以，虽然注释掉了oracle用户的ORA_CRS_HOME变量，还是使用了全局设置的变量，等于没有取消掉。再取消全局的变量后，再跑rootupgrade脚本就过了。

———————————————————-end——————————————-