本文介绍oracle rac onenode在线共享存储迁移,同时也是ASM共享磁盘扩容的方法。
1. 在线扫描磁盘
[root@rac-test01-db scsi_host]# cd /sys/class/scsi_host
[root@rac-test01-db scsi_host]# ls
host0 host1 host10 host2 host3 host4 host5 host6 host7 host8 host9
#有几个host文件执行几次
echo "- - -" > /sys/class/scsi_host/host0/scan
echo "- - -" > /sys/class/scsi_host/host1/scan
...
echo "- - -" > /sys/class/scsi_host/host10/scan
2. 检查待添加磁盘
[root@rac-test01-db ~]# fdisk -l
...
Disk /dev/sdh: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sdi: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
3. 依次格式化共享磁盘
fdisk /dev/sdh
#输出
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x649675ed.
Command (m for help): n <---------
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): 【回车】<---------
Using default response p
Partition number (1-4, default 1): 【回车】<---------
First sector (2048-10485759, default 2048): 【回车】<---------
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-10485759, default 10485759): +4G <---------
Partition 1 of type Linux and of size 4 GiB is set
Command (m for help): w <---------
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
4. 创建ASM磁盘
[root@rac-test01-db ~]# oracleasm createdisk TEST01 /dev/sdh1
Writing disk header: done
Instantiating disk: done
[root@rac-test01-db ~]# oracleasm createdisk TEST02 /dev/sdi1
Writing disk header: done
Instantiating disk: done
5. 扫描并列出ASM磁盘
[root@rac-test01-db ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
[root@rac-test01-db ~]# oracleasm listdisks
DATA01
FRA01
FRA02
OCRVD01
OCRVD02
OCRVD03
TEST01
TEST02
6. 创建TEST磁盘组,验证新建共享存储可用
这步是官方文档的最佳实践,为的是避免新加入的磁盘有问题,直接加入现有磁盘组导致拉挂整个磁盘组。
先用新加磁盘创建测试磁盘组,确保磁盘组能MOUNT,再删除测试磁盘组,将新加磁盘加入现有磁盘组。
节点1
[root@rac-test01-db ~]# su - grid
Last login: Fri Dec 29 09:58:03 CST 2017 on pts/1
[grid@rac-test01-db ~]$ sqlplus / as sysasm
SQL*Plus: Release 12.1.0.2.0 Production on Wed Jan 3 16:27:54 2018
Copyright (c) 1982, 2014, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
SQL> create diskgroup TEST external redundancy
disk 'ORCL:TEST01','ORCL:TEST02';
Diskgroup created.
SQL> select NAME,TOTAL_MB,FREE_MB,STATE from v$asm_diskgroup;
NAME TOTAL_MB FREE_MB STATE
------------------------------ ---------- ---------- -----------
OCRVD 10752 6279 MOUNTED
FRA 36864 36530 MOUNTED
DATA 46080 43962 MOUNTED
TEST 8192 8140 MOUNTED
节点2
[root@rac-test02-db ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "TEST01"
Instantiating disk "TEST02"
[root@rac-test02-db ~]# oracleasm listdisks
DATA01
FRA01
FRA02
OCRVD01
OCRVD02
OCRVD03
TEST01
TEST02
su - grid
sqlplus / as sysasm
SQL> select NAME,TOTAL_MB,FREE_MB,STATE from v$asm_diskgroup;
NAME TOTAL_MB FREE_MB STATE
------------------------------ ---------- ---------- -----------
OCRVD 10752 6279 MOUNTED
FRA 36864 36530 MOUNTED
DATA 46080 43962 MOUNTED
TEST 0 0 DISMOUNTED
SQL> alter diskgroup TEST MOUNT;
Diskgroup altered.
SQL> select NAME,TOTAL_MB,FREE_MB,STATE from v$asm_diskgroup;
NAME TOTAL_MB FREE_MB STATE
------------------------------ ---------- ---------- -----------
OCRVD 10752 6279 MOUNTED
FRA 36864 36530 MOUNTED
DATA 46080 43962 MOUNTED
TEST 8192 8097 MOUNTED
验证完成后节点2 dismount TEST组
SQL> alter diskgroup TEST DISMOUNT;
Diskgroup altered.
SQL> select NAME,TOTAL_MB,FREE_MB,STATE from v$asm_diskgroup;
NAME TOTAL_MB FREE_MB STATE
------------------------------ ---------- ---------- -----------
OCRVD 10752 6269 MOUNTED
FRA 36864 35892 MOUNTED
DATA 46080 43656 MOUNTED
TEST 0 0 DISMOUNTED
节点1 drop TEST磁盘组
SQL> select NAME,TOTAL_MB,FREE_MB,STATE from v$asm_diskgroup;
NAME TOTAL_MB FREE_MB STATE
------------------------------ ---------- ---------- -----------
OCRVD 10752 6269 MOUNTED
FRA 36864 35892 MOUNTED
DATA 46080 43656 MOUNTED
TEST 8192 8097 MOUNTED
drop diskgroup TEST;
SQL> select NAME,TOTAL_MB,FREE_MB,STATE from v$asm_diskgroup;
NAME TOTAL_MB FREE_MB STATE
------------------------------ ---------- ---------- -----------
OCRVD 10752 6269 MOUNTED
FRA 36864 35892 MOUNTED
DATA 46080 43656 MOUNTED
查看磁盘组与磁盘对应关系
mount_state为CLOSED,header_status为FORMER,表明该磁盘当前已未使用
SQL> select GROUP_NUMBER,path,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE from v$asm_disk;
GROUP_NUMBER PATH MOUNT_STATUS HEADER_STATUS MODE_STATUS STATE
----------------------------------------------------------------------------------------
0 ORCL:TEST02 CLOSED FORMER ONLINE NORMAL
0 ORCL:TEST01 CLOSED FORMER ONLINE NORMAL
1 ORCL:DATA01 CACHED MEMBER ONLINE NORMAL
2 ORCL:FRA01 CACHED MEMBER ONLINE NORMAL
2 ORCL:FRA02 CACHED MEMBER ONLINE NORMAL
3 ORCL:OCRVD01 CACHED MEMBER ONLINE NORMAL
3 ORCL:OCRVD02 CACHED MEMBER ONLINE NORMAL
3 ORCL:OCRVD03 CACHED MEMBER ONLINE NORMAL
8 rows selected.
7. 将新建磁盘加入原有磁盘组
根据rebalance的进度,白天最高power设置为3, 晚上10点之后,可以power为8,早上8点前需要恢复成power为3
alter diskgroup FRA add disk 'ORCL:TEST01' rebalance power 3;
alter diskgroup FRA add disk 'ORCL:TEST02' rebalance power 3;
可以一次添加多块磁盘
alter diskgroup FRA add disk 'ORCL:TEST01','ORCL:TEST02' rebalance power 3;
检查rebalance操作进度(查询结果为空时再继续增/删磁盘操作)
select * from gv$asm_operation;
INST_ID GROUP_NUMBER OPERA PASS STAT POWER ACTUAL SOFAR
---------- ------------ ----- --------- ---- ---------- ---------- ----------
EST_WORK EST_RATE EST_MINUTES ERROR_CODE
---------- ---------- ----------- --------------------------------------------
CON_ID
----------
1 2 REBAL REBALANCE DONE 3 3 179
179 0 0
0
1 2 REBAL COMPACT RUN 3 3 24
0 0 0
0
INST_ID GROUP_NUMBER OPERA PASS STAT POWER ACTUAL SOFAR
---------- ------------ ----- --------- ---- ---------- ---------- ----------
EST_WORK EST_RATE EST_MINUTES ERROR_CODE
---------- ---------- ----------- --------------------------------------------
CON_ID
----------
2 2 REBAL REBALANCE DONE 3
0
2 2 REBAL COMPACT WAIT 3
INST_ID GROUP_NUMBER OPERA PASS STAT POWER ACTUAL SOFAR
---------- ------------ ----- --------- ---- ---------- ---------- ----------
EST_WORK EST_RATE EST_MINUTES ERROR_CODE
---------- ---------- ----------- --------------------------------------------
CON_ID
----------
0
SQL> select * from gv$asm_operation;
no rows selected
SQL> select GROUP_NUMBER,path,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE from v$asm_disk;
GROUP_NUMBER PATH MOUNT_STATUS HEADER_STATUS MODE_STATUS STATE
----------------------------------------------------------------------------------------
0 ORCL:TEST02 CACHED MEMBER ONLINE NORMAL
0 ORCL:TEST01 CACHED MEMBER ONLINE NORMAL
1 ORCL:DATA01 CACHED MEMBER ONLINE NORMAL
2 ORCL:FRA01 CACHED MEMBER ONLINE NORMAL
2 ORCL:FRA02 CACHED MEMBER ONLINE NORMAL
3 ORCL:OCRVD01 CACHED MEMBER ONLINE NORMAL
3 ORCL:OCRVD02 CACHED MEMBER ONLINE NORMAL
3 ORCL:OCRVD03 CACHED MEMBER ONLINE NORMAL
8 rows selected.
上面的步骤已经完成了扩容操作,下面是磁盘替换操作,如果不需要就可以停了。
8. 从原有磁盘组中删除旧磁盘
alter diskgroup FRA drop disk FRA01 rebalance power 3;
alter diskgroup FRA drop disk FRA02 rebalance power 3;
可以一次删除多块磁盘
alter diskgroup FRA drop disk FRA01,FRA02 rebalance power 3;
检查rebalance操作进度(查询结果为空时再继续增/删磁盘操作)
select * from gv$asm_operation;
SQL> select * from gv$asm_operation;
INST_ID GROUP_NUMBER OPERA PASS STAT POWER ACTUAL SOFAR
---------- ------------ ----- --------- ---- ---------- ---------- ----------
EST_WORK EST_RATE EST_MINUTES ERROR_CODE
---------- ---------- ----------- --------------------------------------------
CON_ID
----------
1 2 REBAL REBALANCE RUN 3 3 3
1318 682 1
0
1 2 REBAL COMPACT WAIT 3 3 0
0 0 0
0
INST_ID GROUP_NUMBER OPERA PASS STAT POWER ACTUAL SOFAR
---------- ------------ ----- --------- ---- ---------- ---------- ----------
EST_WORK EST_RATE EST_MINUTES ERROR_CODE
---------- ---------- ----------- --------------------------------------------
CON_ID
----------
2 2 REBAL REBALANCE WAIT 3
0
2 2 REBAL COMPACT WAIT 3
INST_ID GROUP_NUMBER OPERA PASS STAT POWER ACTUAL SOFAR
---------- ------------ ----- --------- ---- ---------- ---------- ----------
EST_WORK EST_RATE EST_MINUTES ERROR_CODE
---------- ---------- ----------- --------------------------------------------
CON_ID
----------
0
SQL> select * from gv$asm_operation;
no rows selected
SQL> select GROUP_NUMBER,path,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE from v$asm_disk;
GROUP_NUMBER PATH MOUNT_STATUS HEADER_STATUS MODE_STATUS STATE
----------------------------------------------------------------------------------------
0 ORCL:FRA01 CLOSED FORMER ONLINE NORMAL
0 ORCL:FRA02 CLOSED FORMER ONLINE NORMAL
1 ORCL:DATA01 CACHED MEMBER ONLINE NORMAL
2 ORCL:TEST01 CACHED MEMBER ONLINE NORMAL
2 ORCL:TEST02 CACHED MEMBER ONLINE NORMAL
3 ORCL:OCRVD01 CACHED MEMBER ONLINE NORMAL
3 ORCL:OCRVD02 CACHED MEMBER ONLINE NORMAL
3 ORCL:OCRVD03 CACHED MEMBER ONLINE NORMAL
8 rows selected.
磁盘替换成功~ 确定替换下的磁盘对应/dev/盘符就可以在虚拟机层面回收啦