ASM磁盘resize操作案例分享

0.起因

群里小伙伴求助,原计划给ASM扩容,结果有人直接把一个LUN在存储端给扩容了。

image.png

大佬给出解决方案

image.png

我也没干过这种场景,让我来动手模拟下实验试试。

1.准备环境

我这的基础实验环境

首先说明一下,这里DATA磁盘组是外部模式

Oracle linux 6.9 2台 安装Oracle 11.2.0.4 RAC

Oracle linux 6.9 1台安装tgt模拟ISCSI共享存储

现有VOTE磁盘组3块3G[sdc-e]、DATA磁盘组1块12G[sdb]

为了模拟实际效果,我再往DATA磁盘组里加两块盘sdf、sdg大小都12G,使磁盘组变成36G。

扩容完DATA,我再把中间的磁盘SDF改成15G,再尝试扩容ASM磁盘组。

如无问题,扩容完DATA磁盘组变成39G。

创建新存储卷

在TGT存储服务器新建2个卷

[root@tgt-server ~]# lvcreate -L 12g -n data02 oravg
  Logical volume "data02" created.
[root@tgt-server ~]# lvcreate -L 12g -n data03 oravg
  Logical volume "data03" created.

显示LV

[root@tgt-server ~]# lvs
  LV      VG       Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  lv_root VolGroup -wi-ao---- 111.70g
  lv_swap VolGroup -wi-ao----   7.81g
  data    oravg    -wi-ao----  12.00g
  data02  oravg    -wi-a-----  12.00g
  data03  oravg    -wi-a-----  12.00g
  vote01  oravg    -wi-ao----   3.00g
  vote02  oravg    -wi-ao----   3.00g
  vote03  oravg    -wi-ao----   3.00g

添加映射给RAC节点

[root@tgt-server oravg]# tgtadm --lld iscsi --mode logicalunit --op new --tid 1 --lun 5 --backing-store /dev/oravg/data02
[root@tgt-server oravg]# tgtadm --lld iscsi --mode logicalunit --op new --tid 1 --lun 6 --backing-store /dev/oravg/data03

RAC两节点扫描新存储

RAC1节点

[root@rac1 ~]# iscsiadm -m session --rescan
Rescanning session [sid: 1, target: iqn.2021.tgt-server.control, portal: 192.168.58.100,3260]
[root@rac1 /]# lsblk
NAME                        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sr0                          11:0    1  1024M  0 rom
sda                           8:0    0   120G  0 disk
├─sda1                        8:1    0   500M  0 part /boot
└─sda2                        8:2    0 119.5G  0 part
  ├─VolGroup-lv_root (dm-0) 253:0    0 103.5G  0 lvm  /
  └─VolGroup-lv_swap (dm-1) 253:1    0    16G  0 lvm  [SWAP]
sdb                           8:16   0    12G  0 disk
sdc                           8:32   0     3G  0 disk
sdd                           8:48   0     3G  0 disk
sde                           8:64   0     3G  0 disk
sdf                           8:80   0    12G  0 disk
sdg                           8:96   0    12G  0 disk

RAC2节点

[root@rac2 /]# iscsiadm -m session --rescan
Rescanning session [sid: 1, target: iqn.2021.tgt-server.control, portal: 192.168.57.100,3260]
[root@rac2 /]# lsblk
NAME                        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sr0                          11:0    1  1024M  0 rom
sda                           8:0    0   120G  0 disk
├─sda1                        8:1    0   500M  0 part /boot
└─sda2                        8:2    0 119.5G  0 part
  ├─VolGroup-lv_root (dm-0) 253:0    0 103.5G  0 lvm  /
  └─VolGroup-lv_swap (dm-1) 253:1    0    16G  0 lvm  [SWAP]
sdb                           8:16   0    12G  0 disk
sdc                           8:32   0     3G  0 disk
sdd                           8:48   0     3G  0 disk
sde                           8:64   0     3G  0 disk
sdf                           8:80   0    12G  0 disk
sdg                           8:96   0    12G  0 disk

生成udev

[root@rac1 ~]# for i in f g ; do echo "KERNEL==\"sd*\", BUS==\"scsi\", PROGRAM==\"/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/\$name\", RESULT==\"`/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$i`\", NAME=\"asm-disk$i\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\""; done
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="1IET_00010005", NAME="asm-diskf", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="1IET_00010006", NAME="asm-diskg", OWNER="grid", GROUP="asmadmin", MODE="0660"

两个节点的配置文件都更新

[root@rac1 ~]# cat /etc/udev/rules.d/80-asm.rules
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="1IET_00010001", NAME="asm-diskb", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="1IET_00010002", NAME="asm-diskc", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="1IET_00010003", NAME="asm-diskd", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="1IET_00010004", NAME="asm-diske", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="1IET_00010005", NAME="asm-diskf", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="1IET_00010006", NAME="asm-diskg", OWNER="grid", GROUP="asmadmin", MODE="0660"

两个节点都执行

[root@rac1 rules.d]# start_udev
Starting udev:                                             [  OK  ]
[root@rac2 rules.d]# start_udev
Starting udev:                                             [  OK  ]

扩容ASM

登录ASM实例

[grid@rac2 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.4.0 Production on Mon Apr 1 17:25:00 2024

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> col name for a40;
set line 200;
col path for a30;
select name,path,state,HEADER_STATUS from v$asm_disk;
SQL>

NAME                                     PATH                           STATE    HEADER_STATU
---------------------------------------- ------------------------------ -------- ------------
                                         /dev/asm-diskg                 NORMAL   CANDIDATE
                                         /dev/asm-diskf                 NORMAL   CANDIDATE
VOTE_0001                                /dev/asm-diskd                 NORMAL   MEMBER
VOTE_0000                                /dev/asm-diskc                 NORMAL   MEMBER
DATA_0000                                /dev/asm-diskb                 NORMAL   MEMBER
VOTE_0002                                /dev/asm-diske                 NORMAL   MEMBER
SQL> alter diskgroup data add disk '/dev/asm-diskf','/dev/asm-diskg' rebalance power 11;

Diskgroup altered.

SQL> col name for a40;
set line 200;
col path for a30;
select name,path,state,HEADER_STATUS from v$asm_disk;
SQL>

NAME                                     PATH                           STATE    HEADER_STATU
---------------------------------------- ------------------------------ -------- ------------
VOTE_0001                                /dev/asm-diskd                 NORMAL   MEMBER
VOTE_0000                                /dev/asm-diskc                 NORMAL   MEMBER
DATA_0000                                /dev/asm-diskb                 NORMAL   MEMBER
VOTE_0002                                /dev/asm-diske                 NORMAL   MEMBER
DATA_0002                                /dev/asm-diskg                 NORMAL   MEMBER
DATA_0001                                /dev/asm-diskf                 NORMAL   MEMBER

6 rows selected.

SQL> select * from v$asm_operation;

GROUP_NUMBER OPERA STAT      POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE EST_MINUTES ERROR_CODE
------------ ----- ---- ---------- ---------- ---------- ---------- ---------- ----------- --------------------------------------------
           1 REBAL RUN          11         11        343        343          0           0

等待reblance完成

2.准备增加存储

扩容asm-diskf磁盘大小

扩容LUN前检查下磁盘使用情况

SQL> SET LINESIZE 200
SET PAGESIZE 100
col groupname format a25 heading 'Disk Group'
col path format a35 heading 'LUN Path'
col diskname format a20 heading 'Disk Name'
col sector_size format 9,999 heading 'Sector |Size'
col block_size format 99,999 heading 'Block |Size'
col state format a10 heading 'State'
col au format 9,999,999 heading 'AU Size'
col total_gig format 999,999 heading 'Group |Total |GB'
col dtotal_gig format 999,999 heading 'Disk |Total |GB'
col free_gig format 999,999 heading 'Group |Free |GB'
col dfree_gig format 999,999 heading 'Disk |Free |GB'
select
  g.name groupname,
  d.path,
  d.name diskname,
  d.total_mb/1024 dtotal_gig,
  d.free_mb/1024 dfree_gig
from
  v$asm_diskgroup g, v$asm_disk d
where
  d.group_number = g.group_number
order by
  g.name, d.disk_number;SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL>   2    3    4    5    6    7    8    9   10   11   12

                                                                                      Disk     Disk
                                                                                     Total     Free
Disk Group                LUN Path                            Disk Name                  GB       GB
------------------------- ----------------------------------- -------------------- -------- --------
DATA                      /dev/asm-diskb                      DATA_0000                  12       11
DATA                      /dev/asm-diskf                      DATA_0001                  12       11
DATA                      /dev/asm-diskg                      DATA_0002                  12       11
VOTE                      /dev/asm-diskc                      VOTE_0000                   3        3
VOTE                      /dev/asm-diskd                      VOTE_0001                   3        3
VOTE                      /dev/asm-diske                      VOTE_0002                   3        3

6 rows selected.

SQL>

开始扩展中间的LUN卷大小asm-sdf,由原来的12G扩到15G

[root@tgt-server oravg]# lvresize -L 15G /dev/mapper/oravg-data02
  Size of logical volume oravg/data02 changed from 12.00 GiB (3072 extents) to 15.00 GiB (3840 extents).
  Logical volume data02 successfully resized.
[root@tgt-server oravg]# lvs
  LV      VG       Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  lv_root VolGroup -wi-ao---- 111.70g
  lv_swap VolGroup -wi-ao----   7.81g
  data    oravg    -wi-ao----  12.00g
  data02  oravg    -wi-ao----  15.00g
  data03  oravg    -wi-ao----  12.00g
  vote01  oravg    -wi-ao----   3.00g
  vote02  oravg    -wi-ao----   3.00g
  vote03  oravg    -wi-ao----   3.00g

tgt模拟这里直接修改磁盘大小后还要重新映射才能生效,我这里模拟就先正常关集群,再重新映射lv,生产的话,在主机侧重新扫描一下存储就OK,可以看“起因”章节里的截图。

再测试,检查磁盘大小与实际大小

ASM里显示的还是12G,lsblk显示实际盘15G大小。

                                                                                     Total     Free
Disk Group                LUN Path                            Disk Name                  GB       GB
------------------------- ----------------------------------- -------------------- -------- --------
DATA                      /dev/asm-diskb                      DATA_0000                  12       11
DATA                      /dev/asm-diskf                      DATA_0001                  12       11
DATA                      /dev/asm-diskg                      DATA_0002                  12       11
VOTE                      /dev/asm-diskc                      VOTE_0000                   3        3
VOTE                      /dev/asm-diskd                      VOTE_0001                   3        3
VOTE                      /dev/asm-diske                      VOTE_0002                   3        3

6 rows selected.

SQL> host lsblk
NAME                        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sr0                          11:0    1  1024M  0 rom
sda                           8:0    0   120G  0 disk
├─sda1                        8:1    0   500M  0 part /boot
└─sda2                        8:2    0 119.5G  0 part
  ├─VolGroup-lv_root (dm-0) 253:0    0 103.5G  0 lvm  /
  └─VolGroup-lv_swap (dm-1) 253:1    0    16G  0 lvm  [SWAP]
sdb                           8:16   0    12G  0 disk
sdc                           8:32   0     3G  0 disk
sdd                           8:48   0     3G  0 disk
sde                           8:64   0     3G  0 disk
sdf                           8:80   0    15G  0 disk
sdg                           8:96   0    12G  0 disk

扩容ASM

扩容ASM磁盘组

[grid@rac2 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.4.0 Production on Mon Apr 1 18:24:30 2024

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> select name, total_mb/(1024) "Total GiB" from v$asm_diskgroup;

NAME                            Total GiB
------------------------------ ----------
DATA                                   36
VOTE                                    9

SQL> alter diskgroup data resize all;

Diskgroup altered.

SQL> select name, total_mb/(1024) "Total GiB" from v$asm_diskgroup;

NAME                            Total GiB
------------------------------ ----------
DATA                                   39
VOTE                                    9

查看磁盘使用情况

                                                                                      Disk     Disk
                                                                                     Total     Free
Disk Group                LUN Path                            Disk Name                  GB       GB
------------------------- ----------------------------------- -------------------- -------- --------
DATA                      /dev/asm-diskb                      DATA_0000                  12       11
DATA                      /dev/asm-diskf                      DATA_0001                  15       14
DATA                      /dev/asm-diskg                      DATA_0002                  12       11
VOTE                      /dev/asm-diskc                      VOTE_0000                   3        3
VOTE                      /dev/asm-diskd                      VOTE_0001                   3        3
VOTE                      /dev/asm-diske                      VOTE_0002                   3        3

6 rows selected.

[grid@rac2 ~]$ asmcmd lsdg
State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N         512   4096  4194304     39936    37844                0           37844              0             N  DATA/
MOUNTED  NORMAL  N         512   4096  1048576      9216     8290             3072            2609              0             Y  VOTE/

调整期间ASM输出日志,期间会自动扩容DATA_0001磁盘,并执行reblance。

NOTE: client orcl2:orcl registered, osid 7534, mbr 0x1
Mon Apr 01 18:24:55 2024
SQL> alter diskgroup data resize all
NOTE: GroupBlock outside rolling migration privileged region
NOTE: requesting all-instance membership refresh for group=1
NOTE: requesting all-instance disk validation for group=1
Mon Apr 01 18:24:57 2024
NOTE: disk validation pending for group 1/0xedb8fd25 (DATA)
SUCCESS: validated disks for 1/0xedb8fd25 (DATA)
NOTE: increased size in header on grp 1 disk DATA_0001
NOTE: membership refresh pending for group 1/0xedb8fd25 (DATA)
Mon Apr 01 18:25:08 2024
GMON querying group 1 at 9 for pid 18, osid 6281
Mon Apr 01 18:25:08 2024
SUCCESS: refreshed membership for 1/0xedb8fd25 (DATA)
Mon Apr 01 18:25:08 2024
SUCCESS: alter diskgroup data resize all
NOTE: starting rebalance of group 1/0xedb8fd25 (DATA) at power 1
Starting background process ARB0
Mon Apr 01 18:25:08 2024
ARB0 started with pid=28, OS id=8764
NOTE: assigning ARB0 to group 1/0xedb8fd25 (DATA) with 1 parallel I/O
cellip.ora not found.
NOTE: Attempting voting file refresh on diskgroup DATA
NOTE: Refresh completed on diskgroup DATA. No voting file found.
Mon Apr 01 18:25:17 2024
NOTE: GroupBlock outside rolling migration privileged region
NOTE: requesting all-instance membership refresh for group=1
Mon Apr 01 18:25:20 2024
NOTE: membership refresh pending for group 1/0xedb8fd25 (DATA)
NOTE: stopping process ARB0
SUCCESS: rebalance completed for group 1/0xedb8fd25 (DATA)
Mon Apr 01 18:25:23 2024
GMON querying group 1 at 10 for pid 18, osid 6281
SUCCESS: refreshed membership for 1/0xedb8fd25 (DATA)
NOTE: Attempting voting file refresh on diskgroup DATA
NOTE: Refresh completed on diskgroup DATA. No voting file found.

总结

虽然这样可行,不过还是建议磁盘大小都一样,新增磁盘来扩容,这样规范些。
这么扩了之后,会不会留下啥坑不太确定,不知道以后别的盘满了,这里怎么重分布???

后记20240403

后续测试了一下,我这12+15+12的磁盘组共39G可用,我尝试使用了38G的表空间,仍然没问题。
但是有个说法,就是以后扩容的时候要求磁盘里要有空间,如果磁盘100%都满了,貌似是无法扩容,这个回头我再测试一下。

参考ACE大佬实验

Resizing Oracle ASM disks - Ron Ekins’ - Oracle Technology, DevOps and Kubernetes Blog

  • 4
    点赞
  • 9
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

徐sir(徐慧阳)

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值