转自ORACLE Blogs:ADG switchover之后，数据库版本为何从11.2.0.4变成了11.2.0.2？

最新推荐文章于 2022-07-08 14:42:46 发布

与尔同销万古愁

最新推荐文章于 2022-07-08 14:42:46 发布

阅读量682

点赞数

分类专栏： Oracle 文章标签： oracle

Oracle 专栏收录该内容

51 篇文章 0 订阅

订阅专栏

By: Simon Fu

文章来源：

ADG switchover之后，数据库版本为何从11.2.0.4变成了11.2.0.2？

最近接到了个比较奇怪的问题，switchover之后，新主库居然提示要以upgrade模式打开：

alter database open

ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00704: bootstrap process failure
ORA-39700: database must be opened with UPGRADE option

我的第一反应是，客户的主库备库在switchover之前就是跨版本的。于是问客户要了下两边的alert log，发现确实是跨版本，新主库是11.2.0.2，而新备库是11.2.0.4。

于是建议客户：

sqlplus / as sysdba
startup upgrade
@?/rdbms/admin/catupgrd.sql
@?/rdbms/admin/utlrp.sql
shutdown immediate
startup

然而客户说：我们升级是很久以前的事啦，早就都是11.2.0.4了，为什么switchover之后却变成11.2.0.2呢？

检查alert log历史启动记录，发现客户所言没错，的确是switchover时版本号变了，且新主库上有两个ORACLE HOME：

/oracle/app/oracle/product/11.2.0.4/dbhome_1

/oracle/app/oracle/product/11.2.0/dbhome_1 《----11.2.0.2

<<<新主库alert log：

Starting up:

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production《====switchover之前还是在11.2.0.4 home下运行的

With the Partitioning, Real Application Clusters, OLAP, Data Mining

and Real Application Testing options.

ORACLE_HOME = /oracle/app/oracle/product/11.2.0.4/dbhome_1

System name: AIX

Node name: oradb1

Release: 1

Version: 7

Machine: 00CBC6D54C00

Using parameter settings in server-side pfile /oracle/app/oracle/product/11.2.0.4/dbhome_1/dbs/initora1.ora

...

Tue Aug 08 14:39:32 2017

ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH;

Tue Aug 08 14:40:24 2017

ALTER DATABASE COMMIT TO SWITCHOVER TO PHYSICAL STANDBY WITH SESSION SHUTDOWN

ALTER DATABASE COMMIT TO SWITCHOVER TO PHYSICAL STANDBY [Process Id: 29098052] (ora1)

...

Completed: ALTER DATABASE COMMIT TO SWITCHOVER TO PHYSICAL STANDBY WITH SESSION SHUTDOWN

Reconfiguration complete

Tue Aug 08 14:40:33 2017

ARC2: Becoming the active heartbeat ARCH

Tue Aug 08 14:40:49 2017

Shutting down instance after CTL_SWITCH

DMON (ospid: 23265382): terminating the instance

Instance terminated by DMON, pid = 23265382

Tue Aug 08 14:40:52 2017

Starting ORACLE instance (normal)

sskgpgetexecname failed to get name

LICENSE_MAX_SESSION = 0

LICENSE_SESSIONS_WARNING = 0

2017-08-08 14:40:52.738

[USER(30343272)]CRS-2317:Fatal error: cannot get local GPnP security keys (wallet).

2017-08-08 14:40:52.739

[USER(30343272)]CRS-2316:Fatal error: cannot initialize GPnP, CLSGPNP_ERR (Generic GPnP error).

kggpnpInit: failed to init gpnp

WARNING: No cluster interconnect has been specified. Depending on

the communication driver configured Oracle cluster traffic

may be directed to the public interface of this machine.

Oracle recommends that RAC clustered databases be configured

with a private interconnect for enhanced security and

performance.

Picked latch-free SCN scheme 3

WARNING: db_recovery_file_dest is same as db_create_file_dest

Autotune of undo retention is turned on.

LICENSE_MAX_USERS = 0

SYS auditing is disabled

Starting up:

Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production

With the Partitioning, Real Application Clusters, OLAP, Data Mining

and Real Application Testing options.

Using parameter settings in server-side pfile /oracle/app/oracle/product/11.2.0/dbhome_1/dbs/initora1.ora<===switchover之后，变成了11.2.0.2 home下打开

System parameters with non-default values:

...

Tue Aug 08 14:41:04 2017
ARC2 started with pid=38, OS id=21364806
ARC1: Archival started
Tue Aug 08 14:41:04 2017
ARC3 started with pid=39, OS id=22282270
ARC2: Archival started
ARC1: Becoming the 'no FAL' ARCH
ARC2: Becoming the heartbeat ARCH
ARC2: Becoming the active heartbeat ARCH
ARC2: Becoming the active heartbeat ARCH
Completed: alter database mount
alter database open
Data Guard Broker initializing...
Data Guard Broker initialization complete
This instance was first to open
Beginning standby crash recovery.

Tue Aug 08 14:41:05 2017

SMON: enabling cache recovery

Errors in file /oracle/app/oracle/diag/rdbms/oradg/ora1/trace/ora1_ora_22544618.trc:

ORA-00704: ????????

ORA-39700: ??? UPGRADE ???????

Errors in file /oracle/app/oracle/diag/rdbms/oradg/ora1/trace/ora1_ora_22544618.trc:

ORA-00704: ????????

ORA-39700: ??? UPGRADE ???????

分析过程：

由于这是RAC to RAC的data guard环境，首先怀疑是OCR里的ORACLE_HOME配错了，但检查了下确实是11.2.0.4没错。而环境变量里的ORACLE_HOME也是11.2.0.4的。/etc/oratab下也只写了11.2.0.4的ORACLE_HOME。

由于alert log中显示是用broker切换的，于是怀疑是broker配置错了，但检查了两个home下的启动的初始化参数，都是同样的configuration file。

于是跟客户要了下broker log看，发现broker的确是在switchover后选择了以11.2.0.2的ORACLE_HOME去启动实例，但看不出来原因。

在让客户用ps aeuxwww检查了进程环境变量ORACLE_HOME确认没问题之后，决定让客户对broker做debug（dgmgrl -debug）看下。

主库操作：（s切w）
DGMGRL> switchover to oradg
立即执行切换, 请稍候...
操作要求连接实例 "ora1" (在数据库 "oradg" 上)
正在连接实例 "ora1"...
[W000 08/09 17:51:21.18] Connecting to database using (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=xxxx)(PORT=2931))(ADDRESS=(PROTOCOL=TCP)(HOST=51.131.1.14)(PORT=2931))(ADDRESS=(PROTOCOL=TCP)(HOST=xxxx)(PORT=2932)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=xxxx)(INSTANCE_NAME=ora1))).
[W000 08/09 17:51:21.31] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].
[W000 08/09 17:51:21.31] Broker version is '11.2.0.4.0'
已连接。
新的主数据库 "oradg" 正在打开...《=====
操作要求启动实例 "ora1" (在数据库 "ora" 上)
正在启动实例 "ora1"...
[W000 08/09 17:51:51.23] Connecting to database using ora.
[W000 08/09 17:51:51.31] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].
ORA-01034: ORACLE not available
进程 ID: 0
会话 ID: 0 序列号: 0

ORACLE 例程已经启动。
[W000 08/09 17:51:56.84] Connecting to database using ora.
[W000 08/09 17:51:56.93] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].
[W000 08/09 17:51:56.94] Broker version is '11.2.0.4.0'
alter database mount

数据库装载完毕。
alter database open

数据库已经打开。<==============这儿已经用11.2.0.4 open成功
[W000 08/09 17:52:09.52] Connecting to database using ora.
[W000 08/09 17:52:09.65] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].
[W000 08/09 17:52:09.71] Broker version is '11.2.0.4.0'
切换成功, 新的主数据库为 "oradg"

备库操作：（w切s）
DGMGRL> switchover to ora
立即执行切换, 请稍候...
操作要求连接实例 "ora1" (在数据库 "ora" 上)
正在连接实例 "ora1"...
[W000 08/09 17:55:28.24] Connecting to database using (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=51.131.1.11)(PORT=2931))(ADDRESS=(PROTOCOL=TCP)(HOST=51.131.1.12)(PORT=2931))(ADDRESS=(PROTOCOL=TCP)(HOST=51.131.1.15)(PORT=2932)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=xxx)(INSTANCE_NAME=ora1))).
[W000 08/09 17:55:28.33] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].
[W000 08/09 17:55:28.33] Broker version is '11.2.0.4.0'《==========这儿还是11.2.0.4
已连接。
新的主数据库 "ora" 正在打开...
操作要求启动实例 "ora1" (在数据库 "oradg" 上)
正在启动实例 "ora1"...
[W000 08/09 17:55:55.32] Connecting to database using oradg.
[W000 08/09 17:55:56.20] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].
ORA-01034: ORACLE not available
进程 ID: 0
会话 ID: 0 序列号: 0

ORACLE 例程已经启动。
[W000 08/09 17:56:02.65] Connecting to database using oradg.《====
[W000 08/09 17:56:02.80] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].
[W000 08/09 17:56:02.96] Broker version is '11.2.0.2.0'<=========到这儿却变成了broker version 11.2.0.2
alter database mount

数据库装载完毕。
alter database open

ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00704: bootstrap process failure
ORA-39700: database must be opened with UPGRADE option
进程 ID: 21168174
会话 ID: 1145 序列号: 1

请执行以下步骤以完成切换:
启动实例 "ora1" (属于数据库 "oradg")

从broker的debug输出，我发现了一个细节：切换前broker是使用连接串连到原备库的：

Connecting to database using (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=xxxx)(PORT=2931))(ADDRESS=(PROTOCOL=TCP)(HOST=51.131.1.14)(PORT=2931))(ADDRESS=(PROTOCOL=TCP)(HOST=xxxx)(PORT=2932)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=xxxx)(INSTANCE_NAME=ora1)))

而切换后，则是使用tns的alias连接到这个新主库的：

数据库装载完毕。
alter database open

于是我想到了下面这个可能性：

由于11.2版本RAC开始，监听配置在grid用户下，而在data guard创建时为了duplicate又必须配置静态监听，只有静态监听才会把ORACLE_HOME绑定在tns name alias上。估计是客户在升级后忘了改静态监听或者删掉它！

在跟客户要来了监听配置后发现果然如此：

SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(ORACLE_HOME = /oracle/app/12.2.0/grid)
(SID_NAME = +ASM1)
)
(SID_DESC =
(GLOBAL_DBNAME=ORA)
(ORACLE_HOME = /oracle/app/oracle/product/11.2.0/dbhome_1)《====这里写的是11.2.0.2的ORACLE_HOME
(SID_NAME = ORA1)
)
)