Oracle 10g RAC ASM存储迁移

使用VBOX模拟存储迁移的环境，OS：RHEL5.9，DB：Oracle 10.2.0.4 RAC ASM 2Nodes。

ASM使用Extern Redundancy，不使用ASM的rebalance功能，ocr和votedisk存放在raw里。

迁移包括controlfile，datafile，tempfile，spfile，ocr，votedisk。

迁移前路径

ocr               /dev/sdf1  /dev/raw/raw1,    /dev/sdf2  /dev/raw/raw2

votedisk          /dev/sdg1  /dev/raw/raw3,    /dev/sdg2  /dev/raw/raw4,    /dev/sdg3  /dev/raw/raw5

ASM    +DATA      /dev/sdb

ASM    +FRA       /dev/sdc

迁移后路径

ocr               /dev/sdh1  /dev/raw/raw6,    /dev/sdh2  /dev/raw/raw7

votedisk          /dev/sdi1  /dev/raw/raw8,    /dev/sdi2  /dev/raw/raw9,    /dev/sdi3  /dev/raw/raw10

ASM    +DATAFG    /dev/sdd

ASM    +FRADG     /dev/sde

迁移votedisk和ocr

[oracle@racdb1 ~]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          2
         Total space (kbytes)     :     497744
         Used space (kbytes)      :       4360
         Available space (kbytes) :     493384
         ID                       :  803449477
         Device/File Name         : /dev/raw/raw1
                                    Device/File integrity check succeeded
         Device/File Name         : /dev/raw/raw2
                                    Device/File integrity check succeeded

         Cluster registry integrity check succeeded

[oracle@racdb1 ~]$ 
[oracle@racdb1 ~]$ crsctl query css votedisk
 0.     0    /dev/raw/raw3
 1.     0    /dev/raw/raw4
 2.     0    /dev/raw/raw5

located 3 votedisk(s).

关闭RAC环境的crs

[root@racdb1 ~]# /etc/init.d/init.crs stop
Shutting down Oracle Cluster Ready Services (CRS):
Nov 24 16:06:49.700 | INF | daemon shutting down
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
Shutdown has begun. The daemons should exit soon.

[root@racdb2 ~]# /etc/init.d/init.crs stop
Shutting down Oracle Cluster Ready Services (CRS):
Nov 24 16:07:11.360 | INF | daemon shutting down
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
Shutdown has begun. The daemons should exit soon.

备份votedisk

[root@racdb1 ~]# dd if=/dev/raw/raw3 of=/u01/votedisk.bak

添加votedisk

[root@racdb1 ~]# /u01/app/oracle/product/10.2.0/db_1/bin/crsctl add css votedisk /dev/raw/raw8 -force
Now formatting voting disk: /dev/raw/raw8
successful addition of votedisk /dev/raw/raw8.
[root@racdb1 ~]# /u01/app/oracle/product/10.2.0/db_1/bin/crsctl add css votedisk /dev/raw/raw9 -force
Now formatting voting disk: /dev/raw/raw9
successful addition of votedisk /dev/raw/raw9.
[root@racdb1 ~]# /u01/app/oracle/product/10.2.0/db_1/bin/crsctl add css votedisk /dev/raw/raw10 -force
Now formatting voting disk: /dev/raw/raw10
successful addition of votedisk /dev/raw/raw10.

删除旧的votedisk

[root@racdb1 ~]# /u01/app/oracle/product/10.2.0/db_1/bin/crsctl delete css votedisk /dev/raw/raw3 -force
successful deletion of votedisk /dev/raw/raw3.
[root@racdb1 ~]# /u01/app/oracle/product/10.2.0/db_1/bin/crsctl delete css votedisk /dev/raw/raw4 -force
successful deletion of votedisk /dev/raw/raw4.
[root@racdb1 ~]# /u01/app/oracle/product/10.2.0/db_1/bin/crsctl delete css votedisk /dev/raw/raw5 -force
successful deletion of votedisk /dev/raw/raw5.

启动crs

[root@racdb1 ~]# /u01/app/oracle/product/10.2.0/db_1/bin/crsctl start crs
Attempting to start CRS stack 
The CRS stack will be started shortly

[root@racdb2 ~]# /u01/app/oracle/product/10.2.0/db_1/bin/crsctl start crs
Attempting to start CRS stack 
The CRS stack will be started shortly

检查crs状态，votedisk路径

[oracle@racdb1 ~]$ crsctl check crs
CSS appears healthy
CRS appears healthy
EVM appears healthy
[oracle@racdb1 ~]$ crs_stat -t
Name           Type           Target    State     Host        
------------------------------------------------------------
ora....SM1.asm application    ONLINE    ONLINE    racdb1      
ora....B1.lsnr application    ONLINE    ONLINE    racdb1      
ora.racdb1.gsd application    ONLINE    ONLINE    racdb1      
ora.racdb1.ons application    ONLINE    ONLINE    racdb1      
ora.racdb1.vip application    ONLINE    ONLINE    racdb1      
ora....SM2.asm application    ONLINE    ONLINE    racdb2      
ora....B2.lsnr application    ONLINE    ONLINE    racdb2      
ora.racdb2.gsd application    ONLINE    ONLINE    racdb2      
ora.racdb2.ons application    ONLINE    ONLINE    racdb2      
ora.racdb2.vip application    ONLINE    ONLINE    racdb2      
ora.zhongwc.db application    ONLINE    ONLINE    racdb2      
ora....c1.inst application    ONLINE    ONLINE    racdb1      
ora....c2.inst application    ONLINE    ONLINE    racdb2      
ora.....zwc.cs application    ONLINE    ONLINE    racdb1      
ora....wc1.srv application    ONLINE    ONLINE    racdb1      
[oracle@racdb1 ~]$ 
[oracle@racdb1 ~]$ crsctl query css votedisk
 0.     0    /dev/raw/raw8
 1.     0    /dev/raw/raw9
 2.     0    /dev/raw/raw10

located 3 votedisk(s).

备份ocr

[root@racdb1 bin]# /u01/app/oracle/product/10.2.0/crs/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          2
         Total space (kbytes)     :     497744
         Used space (kbytes)      :       4360
         Available space (kbytes) :     493384
         ID                       :  803449477
         Device/File Name         : /dev/raw/raw1
                                    Device/File integrity check succeeded
         Device/File Name         : /dev/raw/raw2
                                    Device/File integrity check succeeded

         Cluster registry integrity check succeeded

[root@racdb1 bin]# 
[root@racdb1 bin]# /u01/app/oracle/product/10.2.0/crs/bin/ocrconfig -export /u01/ocr_bak -s online

迁移ocr

[root@racdb1 bin]# /u01/app/oracle/product/10.2.0/crs/bin/ocrconfig -replace ocr /dev/raw/raw6
[root@racdb1 bin]# /u01/app/oracle/product/10.2.0/crs/bin/ocrconfig -replace ocrmirror /dev/raw/raw7

check ocr

[oracle@racdb2 ~]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          2
         Total space (kbytes)     :     497744
         Used space (kbytes)      :       4360
         Available space (kbytes) :     493384
         ID                       :  803449477
         Device/File Name         : /dev/raw/raw6
                                    Device/File integrity check succeeded
         Device/File Name         : /dev/raw/raw7
                                    Device/File integrity check succeeded

         Cluster registry integrity check succeeded

迁移datafile，controlfile，logfile，spfile

创建ASM DISKGROUP

asm_diskgroups='DATA','FRA','DATADG','FRADG'

[oracle@racdb1 ~]$ export ORACLE_SID=+ASM1
[oracle@racdb1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.4.0 - Production on Sun Nov 24 16:40:05 2013

Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.


Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options

SQL> col name for a8
SQL> col path for a15
SQL> select a.path,b.name,b.state from v$asm_disk a,v$asm_diskgroup b where a.group_number=b.group_number(+) order by 1;

PATH            NAME     STATE
--------------- -------- ----------------------
/dev/asm-diskb  DATA     MOUNTED
/dev/asm-diskc  FRA      MOUNTED
/dev/asm-diskd
/dev/asm-diske

SQL> create diskgroup datadg external redundancy disk '/dev/asm-diskd';

Diskgroup created.

SQL> create diskgroup fradg external redundancy disk '/dev/asm-diske';

Diskgroup created.

SQL> select a.path,b.name,b.state from v$asm_disk a,v$asm_diskgroup b where a.group_number=b.group_number(+) order by 1;

PATH            NAME     STATE
--------------- -------- ----------------------
/dev/asm-diskb  DATA     MOUNTED
/dev/asm-diskc  FRA      MOUNTED
/dev/asm-diskd  DATADG   MOUNTED
/dev/asm-diske  FRADG    MOUNTED

rac2

SQL> alter diskgroup datadg mount;

Diskgroup altered.

SQL> alter diskgroup fradg mount;

Diskgroup altered.

RAC1，RAC2 rdbms instance 启动 到mount

[oracle@racdb1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.4.0 - Production on Sun Nov 24 16:47:06 2013

Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.


Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options

SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup mount
ORACLE instance started.

Total System Global Area  599785472 bytes
Fixed Size                  2085776 bytes
Variable Size             167775344 bytes
Database Buffers          423624704 bytes
Redo Buffers                6299648 bytes
Database mounted.

[oracle@racdb2 ~]$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.4.0 - Production on Sun Nov 24 16:47:18 2013

Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.


Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options

SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup mount
ORACLE instance started.

Total System Global Area  599785472 bytes
Fixed Size                  2085776 bytes
Variable Size             163581040 bytes
Database Buffers          427819008 bytes
Redo Buffers                6299648 bytes
Database mounted.

使用RMAN迁移datafile

[oracle@racdb1 ~]$ rman target /

Recovery Manager: Release 10.2.0.4.0 - Production on Sun Nov 24 16:55:26 2013

Copyright (c) 1982, 2007, Oracle.  All rights reserved.

connected to target database: ZHONGWC (DBID=207169202, not open)

RMAN> run{
2> copy datafile 1 to '+DATADG/zhongwc/datafile/system.dbf';
3> copy datafile 2 to '+DATADG/zhongwc/datafile/undotbs1.dbf';
4> copy datafile 3 to '+DATADG/zhongwc/datafile/sysaux.dbf';
5> copy datafile 4 to '+DATADG/zhongwc/datafile/users.dbf';
6> copy datafile 5 to '+DATADG/zhongwc/datafile/example.dbf';
7> copy datafile 6 to '+DATADG/zhongwc/datafile/undotbs2.dbf';
8> }

Starting backup at 24-NOV-13
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=859 instance=zhongwc1 devtype=DISK
channel ORA_DISK_1: starting datafile copy
input datafile fno=00001 name=+DATA/zhongwc/datafile/system.256.832285129
output filename=+DATADG/zhongwc/datafile/system.dbf tag=TAG20131124T165656 recid=2 stamp=832352228
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:15
Finished backup at 24-NOV-13

Starting backup at 24-NOV-13
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
input datafile fno=00002 name=+DATA/zhongwc/datafile/undotbs1.258.832285129
output filename=+DATADG/zhongwc/datafile/undotbs1.dbf tag=TAG20131124T165712 recid=3 stamp=832352233
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
Finished backup at 24-NOV-13

Starting backup at 24-NOV-13
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
input datafile fno=00003 name=+DATA/zhongwc/datafile/sysaux.257.832285129
output filename=+DATADG/zhongwc/datafile/sysaux.dbf tag=TAG20131124T165714 recid=4 stamp=832352241
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:15
Finished backup at 24-NOV-13

Starting backup at 24-NOV-13
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
input datafile fno=00004 name=+DATA/zhongwc/datafile/users.259.832285129
output filename=+DATADG/zhongwc/datafile/users.dbf tag=TAG20131124T165729 recid=5 stamp=832352249
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
Finished backup at 24-NOV-13

Starting backup at 24-NOV-13
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
input datafile fno=00005 name=+DATA/zhongwc/datafile/example.264.832285185
output filename=+DATADG/zhongwc/datafile/example.dbf tag=TAG20131124T165730 recid=6 stamp=832352253
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:07
Finished backup at 24-NOV-13

Starting backup at 24-NOV-13
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
input datafile fno=00006 name=+DATA/zhongwc/datafile/undotbs2.265.832285281
output filename=+DATADG/zhongwc/datafile/undotbs2.dbf tag=TAG20131124T165737 recid=7 stamp=832352258
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
Finished backup at 24-NOV-13

RMAN> 

SQL> alter database rename file '+DATA/zhongwc/datafile/system.256.832285129' to '+DATADG/zhongwc/datafile/system.dbf';

Database altered.

SQL> alter database rename file '+DATA/zhongwc/datafile/undotbs1.258.832285129' to '+DATADG/zhongwc/datafile/undotbs1.dbf';

Database altered.

SQL> alter database rename file '+DATA/zhongwc/datafile/sysaux.257.832285129' to '+DATADG/zhongwc/datafile/sysaux.dbf';

Database altered.

SQL> alter database rename file '+DATA/zhongwc/datafile/users.259.832285129' to '+DATADG/zhongwc/datafile/users.dbf';

Database altered.

SQL> alter database rename file '+DATA/zhongwc/datafile/example.264.832285185' to '+DATADG/zhongwc/datafile/example.dbf';

Database altered.

SQL> alter database rename file '+DATA/zhongwc/datafile/undotbs2.265.832285281' to '+DATADG/zhongwc/datafile/undotbs2.dbf';

Database altered.

SQL> select file#,name from v$datafile;

     FILE# NAME
---------- --------------------------------------------------
         1 +DATADG/zhongwc/datafile/system.dbf
         2 +DATADG/zhongwc/datafile/undotbs1.dbf
         3 +DATADG/zhongwc/datafile/sysaux.dbf
         4 +DATADG/zhongwc/datafile/users.dbf
         5 +DATADG/zhongwc/datafile/example.dbf
         6 +DATADG/zhongwc/datafile/undotbs2.dbf

6 rows selected.

迁移tempfile

SQL> select file#,name from v$tempfile;

     FILE# NAME
---------- ---------------------------------------------
         1 +DATA/zhongwc/tempfile/temp.263.832285181

[oracle@racdb1 ~]$ rman target /


Recovery Manager: Release 10.2.0.4.0 - Production on Sun Nov 24 17:15:38 2013


Copyright (c) 1982, 2007, Oracle.  All rights reserved.


connected to target database: ZHONGWC (DBID=207169202, not open)


RMAN> run{
2> set newname for tempfile 1 to '+DATA/zhongwc/tempfile/temp.dbf';
3> switch tempfile all;
4> }


executing command: SET NEWNAME
using target database control file instead of recovery catalog


renamed temporary file 1 to +DATADG/zhongwc/tempfile/temp.dbf in control file

SQL> select file#,name from v$tempfile;

     FILE# NAME
---------- ---------------------------------------------
         1 +DATADG/zhongwc/tempfile/temp.dbf

迁移logfile

SQL> select a.member,a.group#,b.status,b.thread# from v$logfile a,v$log b where a.group#=b.group# order by 2;


MEMBER                                                 GROUP# STATUS              THREAD#
-------------------------------------------------- ---------- ---------------- ----------
+FRA/zhongwc/onlinelog/group_1.257.832285173                1 CURRENT                   1
+DATA/zhongwc/onlinelog/group_1.261.832285173               1 CURRENT                   1
+FRA/zhongwc/onlinelog/group_2.258.832285175                2 INACTIVE                  1
+DATA/zhongwc/onlinelog/group_2.262.832285175               2 INACTIVE                  1
+DATA/zhongwc/onlinelog/group_3.266.832285329               3 CURRENT                   2
+FRA/zhongwc/onlinelog/group_3.259.832285329                3 CURRENT                   2
+DATA/zhongwc/onlinelog/group_4.267.832285331               4 INACTIVE                  2
+FRA/zhongwc/onlinelog/group_4.260.832285331                4 INACTIVE                  2


8 rows selected.

SQL> alter database add logfile thread 1 group 5 '+DATADG' size 500M;

Database altered.

SQL> alter database add logfile thread 1 group 6 '+DATADG' size 500M;

Database altered.

SQL> alter database add logfile thread 1 group 7 '+DATADG' size 500M;

Database altered.

SQL> alter database add logfile thread 2 group 8 '+DATADG' size 500M;

Database altered.

SQL> alter database add logfile thread 2 group 9 '+DATADG' size 500M;

Database altered.

SQL> alter database add logfile thread 2 group 10 '+DATADG' size 500M;

Database altered.

SQL> alter database open;

Database altered.

SQL> !srvctl start service -d zhongwc -s zwc -i zhongwc1

SQL> alter database drop logfile group 1;

Database altered.

SQL> alter database drop logfile group 2;

Database altered.

SQL> alter database drop logfile group 3;

Database altered.

SQL> alter database drop logfile group 4;

Database altered.

SQL> select a.member,a.group#,b.status,b.thread# from v$logfile a,v$log b where a.group#=b.group# order by 2;

MEMBER                                                 GROUP# STATUS              THREAD#
-------------------------------------------------- ---------- ---------------- ----------
+DATADG/zhongwc/onlinelog/group_5.263.832353821             5 INACTIVE                  1
+DATADG/zhongwc/onlinelog/group_6.262.832353833             6 CURRENT                   1
+DATADG/zhongwc/onlinelog/group_7.264.832353849             7 INACTIVE                  1
+DATADG/zhongwc/onlinelog/group_8.265.832353867             8 INACTIVE                  2
+DATADG/zhongwc/onlinelog/group_9.266.832353879             9 INACTIVE                  2
+DATADG/zhongwc/onlinelog/group_10.267.832353891           10 CURRENT                   2

6 rows selected.

迁移controlfile

SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup nomount
ORACLE instance started.

Total System Global Area  599785472 bytes
Fixed Size                  2085776 bytes
Variable Size             163581040 bytes
Database Buffers          427819008 bytes
Redo Buffers                6299648 bytes

[oracle@racdb1 oracle]$ rman target /

Recovery Manager: Release 10.2.0.4.0 - Production on Sun Nov 24 17:51:08 2013

Copyright (c) 1982, 2007, Oracle.  All rights reserved.

connected to target database: zhongwc (not mounted)

RMAN> run{
2> restore controlfile to '+DATADG/zhongwc/controlfile/control01.ctl' from '+DATA/zhongwc/controlfile/current.260.832285171';
3> restore controlfile to '+FRADG/zhongwc/controlfile/control02.ctl' from '+FRA/zhongwc/controlfile/current.256.832285171';
4> }

Starting restore at 24-NOV-13
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=863 instance=zhongwc1 devtype=DISK

channel ORA_DISK_1: copied control file copy
Finished restore at 24-NOV-13

Starting restore at 24-NOV-13
using channel ORA_DISK_1

channel ORA_DISK_1: copied control file copy
Finished restore at 24-NOV-13

SQL> alter system set control_files='+DATADG/zhongwc/controlfile/control01.ctl','+FRADG/zhongwc/controlfile/control02.ctl' scope=spfile sid='*';

System altered.

SQL> shutdown immediate
ORA-01507: database not mounted


ORACLE instance shut down.
SQL> startup
ORACLE instance started.

Total System Global Area  599785472 bytes
Fixed Size                  2085776 bytes
Variable Size             167775344 bytes
Database Buffers          423624704 bytes
Redo Buffers                6299648 bytes
Database mounted.
Database opened.
SQL> show parameter control_files

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
control_files                        string      +DATADG/zhongwc/controlfile/co
                                                 ntrol01.ctl, +FRADG/zhongwc/co
                                                 ntrolfile/control02.ctl
SQL> !srvctl start service -d zhongwc -s zwc -i zhongwc1

修改db_create_file_dest，db_recovery_file_dest，log_archive_dest_1参数

SQL> alter system set db_create_file_dest='+DATADG';

System altered.

SQL> alter system set db_recovery_file_dest='+FRADG';

System altered.

SQL> alter system set log_archive_dest_1='location=+FRADG';

System altered.

迁移spfile，修改ocr

[oracle@racdb1 ~]$ srvctl config database -d zhongwc -a
racdb1 zhongwc1 /u01/app/oracle/product/10.2.0/db_1
racdb2 zhongwc2 /u01/app/oracle/product/10.2.0/db_1
DB_NAME: zhongwc
ORACLE_HOME: /u01/app/oracle/product/10.2.0/db_1
SPFILE: +DATA/zhongwc/spfilezhongwc.ora
DOMAIN: null
DB_ROLE: null
START_OPTIONS: null
POLICY:  AUTOMATIC
ENABLE FLAG: DB ENABLED

[oracle@racdb1 ~]$ srvctl modify database -d zhongwc -p '+DATADG/zhongwc/spfilezhongwc.ora'
[oracle@racdb1 ~]$ 
[oracle@racdb1 ~]$ srvctl config database -d zhongwc -a
racdb1 zhongwc1 /u01/app/oracle/product/10.2.0/db_1
racdb2 zhongwc2 /u01/app/oracle/product/10.2.0/db_1
DB_NAME: zhongwc
ORACLE_HOME: /u01/app/oracle/product/10.2.0/db_1
SPFILE: +DATADG/zhongwc/spfilezhongwc.ora
DOMAIN: null
DB_ROLE: null
START_OPTIONS: null
POLICY:  AUTOMATIC
ENABLE FLAG: DB ENABLED

最后删除旧的DISKGROUP

on rac2

SQL> alter diskgroup data dismount;

Diskgroup altered.

SQL> alter diskgroup fra dismount;

Diskgroup altered.

on rac1

SQL> drop diskgroup fra;

Diskgroup dropped.

SQL> drop diskgroup data;

Diskgroup dropped.

Note:

Linux下添加删除设备可能导致设备名称更改。