Oracle ASM multipath LUN存储聚合配置（RHEL7）

最新推荐文章于 2021-05-29 20:43:31 发布
最新推荐文章于 2021-05-29 20:43:31 发布
阅读量665 收藏 3
点赞数
分类专栏： 数据库相关 文章标签： 数据库
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_37644222/article/details/117382131
版权

环境：
openfiler两块网卡 eth0:172.16.0.55 eth1:172.16.0.65
一块200G磁盘全部分配到一个Volume Group
然后分了7个LV
3个3G做OCR
3个8G做DATA
1个8G做FRA
openfiler具体配置方法参考结尾引用文章的链接

挂载RHEL安装介质
[root@rac02 ~]# mount /dev/sr0 /mnt
mount: /dev/sr0 is write-protected, mounting read-only
安装iSCSI-initiator
[root@rac02 ~]# yum install iscsi-initiator-utils*
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is not registered with an entitlement server. You can use subscription-manager to register.
Package iscsi-initiator-utils-6.2.0.874-17.el7.x86_64 already installed and latest version
Package iscsi-initiator-utils-iscsiuio-6.2.0.874-17.el7.x86_64 already installed and latest version
Nothing to do
搜索iscsi target 路径1
[root@rac02 ~]# iscsiadm --mode discovery --type sendtargets --portal 172.16.0.55
172.16.0.55:3260,1 iqn.2006-01.com.openfiler:tsn.3c0d16cca859
搜索iscsi target 路径2
[root@rac02 ~]# iscsiadm --mode discovery --type sendtargets --portal 172.16.0.65
172.16.0.65:3260,1 iqn.2006-01.com.openfiler:tsn.3c0d16cca859
挂载iscsi target下面的LUN
[root@rac02 ~]# iscsiadm --mode node --targetname iqn.2006-01.com.openfiler:tsn.3c0d16cca859  –portal 172.16.0.55:3260 --login
Logging in to [iface: default, target: iqn.2006-01.com.openfiler:tsn.3c0d16cca859, portal: 172.16.0.55,3260] (multiple)
Logging in to [iface: default, target: iqn.2006-01.com.openfiler:tsn.3c0d16cca859, portal: 172.16.0.65,3260] (multiple)
Login to [iface: default, target: iqn.2006-01.com.openfiler:tsn.3c0d16cca859, portal: 172.16.0.55,3260] successful.
Login to [iface: default, target: iqn.2006-01.com.openfiler:tsn.3c0d16cca859, portal: 172.16.0.65,3260] successful.
查看挂载的LUN，这里实际上是7块盘，出现14个LUN,因为有两个路径（2块网卡登录两次iscsi target）
[root@rac02 ~]# fdisk -l | grep dev
Disk /dev/sda: 214.7 GB, 214748364800 bytes, 419430400 sectors
/dev/sda1   *        2048      309247      153600   83  Linux
/dev/sda2          309248   348452863   174071808   8e  Linux LVM
Disk /dev/mapper/rhel-root: 107.4 GB, 107374182400 bytes, 209715200 sectors
Disk /dev/mapper/rhel-swap: 17.2 GB, 17179869184 bytes, 33554432 sectors
Disk /dev/mapper/rhel-home: 53.7 GB, 53687091200 bytes, 104857600 sectors
Disk /dev/sdb: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdc: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdd: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sde: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdf: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdg: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdh: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdi: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdj: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdl: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdn: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdk: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdm: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdo: 8589 MB, 8589934592 bytes, 16777216 sectors

查看iscsi target详细信息
[root@rac02 ~]# iscsiadm -m session -P 3
iSCSI Transport Class version 2.0-870
version 6.2.0.874-17
Target: iqn.2006-01.com.openfiler:tsn.3c0d16cca859 (non-flash)
        Current Portal: 172.16.0.65:3260,1
        Persistent Portal: 172.16.0.65:3260,1
                **********
                Interface:
                **********
                Iface Name: default
                Iface Transport: tcp
                Iface Initiatorname: iqn.1994-05.com.redhat:5e1fc327b079
                Iface IPaddress: 172.16.0.102
                Iface HWaddress: <empty>
                Iface Netdev: <empty>
                SID: 1
                iSCSI Connection State: LOGGED IN
                iSCSI Session State: LOGGED_IN
                Internal iscsid Session State: NO CHANGE
                *********
                Timeouts:
                *********
                Recovery Timeout: 120
                Target Reset Timeout: 30
                LUN Reset Timeout: 30
                Abort Timeout: 15
                *****
                CHAP:
                *****
                username: <empty>
                password: ********
                username_in: <empty>
                password_in: ********
                ************************
                Negotiated iSCSI params:
                ************************
                HeaderDigest: None
                DataDigest: None
                MaxRecvDataSegmentLength: 262144
                MaxXmitDataSegmentLength: 131072
                FirstBurstLength: 262144
                MaxBurstLength: 262144
                ImmediateData: No
                InitialR2T: Yes
                MaxOutstandingR2T: 1
                ************************
                Attached SCSI devices:
                ************************
                Host Number: 33 State: running
                scsi33 Channel 00 Id 0 Lun: 0
                        Attached scsi disk sdb          State: running
                scsi33 Channel 00 Id 0 Lun: 1
                        Attached scsi disk sdc          State: running
                scsi33 Channel 00 Id 0 Lun: 2
                        Attached scsi disk sdd          State: running
                scsi33 Channel 00 Id 0 Lun: 3
                        Attached scsi disk sde          State: running
                scsi33 Channel 00 Id 0 Lun: 4
                        Attached scsi disk sdf          State: running
                scsi33 Channel 00 Id 0 Lun: 5
                        Attached scsi disk sdg          State: running
                scsi33 Channel 00 Id 0 Lun: 6
                        Attached scsi disk sdh          State: running
        Current Portal: 172.16.0.55:3260,1
        Persistent Portal: 172.16.0.55:3260,1
                **********
                Interface:
                **********
                Iface Name: default
                Iface Transport: tcp
                Iface Initiatorname: iqn.1994-05.com.redhat:5e1fc327b079
                Iface IPaddress: 172.16.0.102
                Iface HWaddress: <empty>
                Iface Netdev: <empty>
                SID: 2
                iSCSI Connection State: LOGGED IN
                iSCSI Session State: LOGGED_IN
                Internal iscsid Session State: NO CHANGE
                *********
                Timeouts:
                *********
                Recovery Timeout: 120
                Target Reset Timeout: 30
                LUN Reset Timeout: 30
                Abort Timeout: 15
                *****
                CHAP:
                *****
                username: <empty>
                password: ********
                username_in: <empty>
                password_in: ********
                ************************
                Negotiated iSCSI params:
                ************************
                HeaderDigest: None
                DataDigest: None
                MaxRecvDataSegmentLength: 262144
                MaxXmitDataSegmentLength: 131072
                FirstBurstLength: 262144
                MaxBurstLength: 262144
                ImmediateData: No
                InitialR2T: Yes
                MaxOutstandingR2T: 1
                ************************
                Attached SCSI devices:
                ************************
                Host Number: 34 State: running
                scsi34 Channel 00 Id 0 Lun: 0
                        Attached scsi disk sdi          State: running
                scsi34 Channel 00 Id 0 Lun: 1
                        Attached scsi disk sdj          State: running
                scsi34 Channel 00 Id 0 Lun: 2
                        Attached scsi disk sdk          State: running
                scsi34 Channel 00 Id 0 Lun: 3
                        Attached scsi disk sdl          State: running
                scsi34 Channel 00 Id 0 Lun: 4
                        Attached scsi disk sdm          State: running
                scsi34 Channel 00 Id 0 Lun: 5
                        Attached scsi disk sdn          State: running
                scsi34 Channel 00 Id 0 Lun: 6
                        Attached scsi disk sdo          State: running
将iscsid守护进程设置成开机启动
[root@rac02 ~]# systemctl enable iscsid
Created symlink from /etc/systemd/system/multi-user.target.wants/iscsid.service to /usr/lib/systemd/system/iscsid.service.


RAC 两个节点安装multipath（RHEL7默认已经安装）
[root@rac02 ~]# rpm -qa|grep device-mapper
device-mapper-multipath-0.4.9-131.el7.x86_64
device-mapper-1.02.164-7.el7.x86_64
device-mapper-event-libs-1.02.164-7.el7.x86_64
device-mapper-persistent-data-0.8.5-2.el7.x86_64
device-mapper-libs-1.02.164-7.el7.x86_64
device-mapper-event-1.02.164-7.el7.x86_64
device-mapper-multipath-libs-0.4.9-131.el7.x86_64

多路径软件添加到内核
[root@rac01 /mnt/Packages]# modprobe dm-multipath
[root@rac01 /mnt/Packages]# modprobe dm-round-robin
检查内核添加情况
[root@rac01 /mnt/Packages]# lsmod |grep multipath
dm_multipath           27792  1 dm_round_robin
dm_mod                124501  13 dm_multipath,dm_log,dm_mirror
生成multipath的配置文件
[root@rac01 /mnt/Packages]# mpathconf --enable

[root@rac01 /mnt/Packages]# systemctl start multipathd
查看进程状态
[root@rac01 /mnt/Packages]# systemctl status multipathd.service
● multipathd.service - Device-Mapper Multipath Device Controller
   Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2021-05-27 14:50:41 CST; 6s ago
  Process: 14090 ExecStart=/sbin/multipathd (code=exited, status=0/SUCCESS)
  Process: 14088 ExecStartPre=/sbin/multipath -A (code=exited, status=0/SUCCESS)
  Process: 14086 ExecStartPre=/sbin/modprobe dm-multipath (code=exited, status=0/SUCCESS)
Main PID: 14093 (multipathd)
    Tasks: 13
   CGroup: /system.slice/multipathd.service
           └─14093 /sbin/multipathd

May 27 14:50:41 rac01 multipathd[14093]: mpathf: load table [0 16777216 multipath 0 0 2 1 service-time 0 1 1 8:208 1 service-time 0 1 1 8:160 1]
May 27 14:50:42 rac01 multipathd[14093]: mpathg: load table [0 16777216 multipath 0 0 2 1 service-time 0 1 1 8:224 1 service-time 0 1 1 8:192 1]
May 27 14:50:42 rac01 multipathd[14093]: mpatha: event checker started
May 27 14:50:42 rac01 multipathd[14093]: mpathb: event checker started
May 27 14:50:42 rac01 multipathd[14093]: mpathc: event checker started
May 27 14:50:42 rac01 multipathd[14093]: mpathd: event checker started
May 27 14:50:42 rac01 multipathd[14093]: mpathe: event checker started
May 27 14:50:42 rac01 multipathd[14093]: mpathf: event checker started
May 27 14:50:42 rac01 multipathd[14093]: mpathg: event checker started
May 27 14:50:42 rac01 multipathd[14093]: path checkers start up

设置为自动启动
[root@rac01 /mnt/Packages]# systemctl enable multipathd
配置多路径软件/etc/multipath.conf
在RAC节点1查看并获取存储分配给服务器的逻辑盘lun的wwid信息
获取wwid指令
[root@rac01 ~]# multipath -v0
查看wwid信息
[root@rac01 ~]# more /etc/multipath/wwids
# Multipath wwids, Version : 1.0
# NOTE: This file is automatically maintained by multipath and multipathd.
# You should not need to edit this file in normal circumstances.
#
# Valid WWIDs:
/14f504e46494c45007831743376342d42645a362d38646763/
/14f504e46494c45004f6647784c6b2d634348482d4d537443/
/14f504e46494c45004d37476d64492d516f326d2d55747862/
/14f504e46494c4500707a4b7430482d633044672d6158706b/
/14f504e46494c4500763534334d612d363538792d32626e6e/
/14f504e46494c45004a7a416363632d523475692d746f3068/
/14f504e46494c4500423356646a332d4565674d2d74515555/

wwid绑定的别名
[root@rac01 ~]# more /etc/multipath/bindings
# Multipath bindings, Version : 1.0
# NOTE: this file is automatically maintained by the multipath program.
# You should not need to edit this file in normal circumstances.
#
# Format:
# alias wwid
#
mpatha 14f504e46494c45007831743376342d42645a362d38646763
mpathb 14f504e46494c45004f6647784c6b2d634348482d4d537443
mpathc 14f504e46494c45004d37476d64492d516f326d2d55747862
mpathd 14f504e46494c4500707a4b7430482d633044672d6158706b
mpathe 14f504e46494c4500763534334d612d363538792d32626e6e
mpathf 14f504e46494c45004a7a416363632d523475692d746f3068
mpathg 14f504e46494c4500423356646a332d4565674d2d74515555

将这两个文件scp到RAC 节点2
[root@rac01 ~]# cd /etc/multipath/
[root@rac01 /etc/multipath]# scp -rp bindings wwids root@172.16.0.102:${PWD}
root@172.16.0.102's password:
bindings                                                                                                                         100%  599     1.0MB/s   00:00
wwids                                                                                                                            100%  554     1.2MB/s   00:00

在节点2执行确认动作
[root@rac02 ~]# multipath -v0
[root@rac02 ~]# more /etc/multipath/wwids
# Multipath wwids, Version : 1.0
# NOTE: This file is automatically maintained by multipath and multipathd.
# You should not need to edit this file in normal circumstances.
#
# Valid WWIDs:
/14f504e46494c45007831743376342d42645a362d38646763/
/14f504e46494c45004f6647784c6b2d634348482d4d537443/
/14f504e46494c45004d37476d64492d516f326d2d55747862/
/14f504e46494c4500707a4b7430482d633044672d6158706b/
/14f504e46494c4500763534334d612d363538792d32626e6e/
/14f504e46494c45004a7a416363632d523475692d746f3068/
/14f504e46494c4500423356646a332d4565674d2d74515555/

[root@rac02 ~]# more /etc/multipath/bindings
# Multipath bindings, Version : 1.0
# NOTE: this file is automatically maintained by the multipath program.
# You should not need to edit this file in normal circumstances.
#
# Format:
# alias wwid
#
mpatha 14f504e46494c45007831743376342d42645a362d38646763
mpathb 14f504e46494c45004f6647784c6b2d634348482d4d537443
mpathc 14f504e46494c45004d37476d64492d516f326d2d55747862
mpathd 14f504e46494c4500707a4b7430482d633044672d6158706b
mpathe 14f504e46494c4500763534334d612d363538792d32626e6e
mpathf 14f504e46494c45004a7a416363632d523475692d746f3068
mpathg 14f504e46494c4500423356646a332d4565674d2d74515555

[root@rac02 ~]# multipath -ll
mpathe (14f504e46494c4500763534334d612d363538792d32626e6e) dm-7 OPNFILER,VIRTUAL-DISK
size=8.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:4 sdf 8:80  active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:4 sdm 8:192 active ready running
mpathd (14f504e46494c4500707a4b7430482d633044672d6158706b) dm-6 OPNFILER,VIRTUAL-DISK
size=8.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:3 sde 8:64  active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:3 sdl 8:176 active ready running
mpathc (14f504e46494c45004d37476d64492d516f326d2d55747862) dm-5 OPNFILER,VIRTUAL-DISK
size=3.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:2 sdd 8:48  active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:2 sdk 8:160 active ready running
mpathb (14f504e46494c45004f6647784c6b2d634348482d4d537443) dm-4 OPNFILER,VIRTUAL-DISK
size=3.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:1 sdc 8:32  active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:1 sdj 8:144 active ready running
mpatha (14f504e46494c45007831743376342d42645a362d38646763) dm-3 OPNFILER,VIRTUAL-DISK
size=3.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:0 sdb 8:16  active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:0 sdi 8:128 active ready running
mpathg (14f504e46494c4500423356646a332d4565674d2d74515555) dm-9 OPNFILER,VIRTUAL-DISK
size=8.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:6 sdh 8:112 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:6 sdo 8:224 active ready running
mpathf (14f504e46494c45004a7a416363632d523475692d746f3068) dm-8 OPNFILER,VIRTUAL-DISK
size=8.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:5 sdg 8:96  active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:5 sdn 8:208 active ready running


节点1 查看聚合效果
[root@rac01 /etc/multipath]# fdisk -l | grep dev
Disk /dev/sda: 214.7 GB, 214748364800 bytes, 419430400 sectors
/dev/sda1   *        2048      309247      153600   83  Linux
/dev/sda2          309248   348452863   174071808   8e  Linux LVM
Disk /dev/mapper/rhel-root: 107.4 GB, 107374182400 bytes, 209715200 sectors
Disk /dev/mapper/rhel-swap: 17.2 GB, 17179869184 bytes, 33554432 sectors
Disk /dev/mapper/rhel-home: 53.7 GB, 53687091200 bytes, 104857600 sectors
------------------------下面是未聚合的LUN----------------------------------
Disk /dev/sdb: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdc: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sde: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdf: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdd: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdg: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdh: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdi: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/sdj: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdl: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdk: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdm: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdn: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/sdo: 8589 MB, 8589934592 bytes, 16777216 sectors
--------------------------下面是聚合的LUN----------------------------------
Disk /dev/mapper/mpathc: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/mapper/mpathd: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/mapper/mpathb: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/mapper/mpatha: 3221 MB, 3221225472 bytes, 6291456 sectors
Disk /dev/mapper/mpathe: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/mapper/mpathf: 8589 MB, 8589934592 bytes, 16777216 sectors
Disk /dev/mapper/mpathg: 8589 MB, 8589934592 bytes, 16777216 sectors

编辑/etc/multipath.conf
#blacklist代表黑名单，某些设备不做多路径的写进里面，比如本地磁盘
blacklist {
        devnode "^sda"
}

multipaths {
        multipath {
                wwid 14f504e46494c45007831743376342d42645a362d38646763
                alias asm-ocr1
        }
        multipath {
                wwid 14f504e46494c45004f6647784c6b2d634348482d4d537443
                alias asm-ocr2
        }
        multipath {
                wwid 14f504e46494c45004d37476d64492d516f326d2d55747862
                alias asm-ocr3
        }
        multipath {
                wwid 14f504e46494c4500707a4b7430482d633044672d6158706b
                alias asm-data01
        }
        multipath {
                wwid 14f504e46494c4500763534334d612d363538792d32626e6e
                alias asm-data02
        }
        multipath {
                wwid 14f504e46494c45004a7a416363632d523475692d746f3068
                alias asm-data03
        }
        multipath {
                wwid 14f504e46494c4500423356646a332d4565674d2d74515555
                alias asm-fra01
        }

}

重启multipathd守护程序
[root@rac01 /u01/app]# systemctl restart multipathd
查看定义的别名是否生效
[root@rac01 /u01/app]# multipath -ll
asm-fra01 (14f504e46494c4500423356646a332d4565674d2d74515555) dm-9 OPNFILER,VIRTUAL-DISK
size=8.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:6 sdo 8:224 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:6 sdm 8:192 active ready running
asm-data03 (14f504e46494c45004a7a416363632d523475692d746f3068) dm-8 OPNFILER,VIRTUAL-DISK
size=8.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:5 sdn 8:208 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:5 sdk 8:160 active ready running
asm-ocr3 (14f504e46494c45004d37476d64492d516f326d2d55747862) dm-3 OPNFILER,VIRTUAL-DISK
size=3.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:2 sdh 8:112 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:2 sde 8:64  active ready running
asm-data02 (14f504e46494c4500763534334d612d363538792d32626e6e) dm-6 OPNFILER,VIRTUAL-DISK
size=8.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:4 sdl 8:176 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:4 sdi 8:128 active ready running
asm-ocr2 (14f504e46494c45004f6647784c6b2d634348482d4d537443) dm-5 OPNFILER,VIRTUAL-DISK
size=3.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:1 sdg 8:96  active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:1 sdc 8:32  active ready running
asm-data01 (14f504e46494c4500707a4b7430482d633044672d6158706b) dm-7 OPNFILER,VIRTUAL-DISK
size=8.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:3 sdj 8:144 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:3 sdf 8:80  active ready running
asm-ocr1 (14f504e46494c45007831743376342d42645a362d38646763) dm-4 OPNFILER,VIRTUAL-DISK
size=3.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 33:0:0:0 sdd 8:48  active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 34:0:0:0 sdb 8:16  active ready running


检查/dev/mapper
[root@rac01 /u01/app]#  ls -l /dev/mapper/ | grep dm-*
lrwxrwxrwx 1 root root       7 May 28 14:44 asm-data01 -> ../dm-7
lrwxrwxrwx 1 root root       7 May 28 14:44 asm-data02 -> ../dm-6
lrwxrwxrwx 1 root root       7 May 28 14:44 asm-data03 -> ../dm-8
lrwxrwxrwx 1 root root       7 May 28 14:44 asm-fra01 -> ../dm-9
lrwxrwxrwx 1 root root       7 May 28 14:44 asm-ocr1 -> ../dm-4
lrwxrwxrwx 1 root root       7 May 28 14:44 asm-ocr2 -> ../dm-5
lrwxrwxrwx 1 root root       7 May 28 14:44 asm-ocr3 -> ../dm-3
lrwxrwxrwx 1 root root       7 May 28 14:09 rhel-home -> ../dm-2
lrwxrwxrwx 1 root root       7 May 28 14:09 rhel-root -> ../dm-0
lrwxrwxrwx 1 root root       7 May 28 14:09 rhel-swap -> ../dm-1


RAC节点2重复上面的动作

设置权限和属主的参考示例文件：/usr/share/doc/device-mapper-1.02.164/12-dm-permissions.rules

配置权限和UDEV规则，要注意DM_UUID的格式，必须加mpath-的前缀，否则无效
[root@rac01 ~]# vim /etc/udev/rules.d/99-oracle-asmdevice.rules
[root@rac01 ~]# more /etc/udev/rules.d/99-oracle-asmdevice.rules
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c45007831743376342d42645a362d38646763", ACTION=="add|change", OWNER:="grid", GROUP:="asmadmin", MODE:="660", SYMLINK+="asm-ocr1"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c45004f6647784c6b2d634348482d4d537443", ACTION=="add|change", OWNER:="grid", GROUP:="asmadmin", MODE:="660", SYMLINK+="asm-ocr2"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c45004d37476d64492d516f326d2d55747862", ACTION=="add|change", OWNER:="grid", GROUP:="asmadmin", MODE:="660", SYMLINK+="asm-ocr3"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c4500707a4b7430482d633044672d6158706b", ACTION=="add|change", OWNER:="grid", GROUP:="asmadmin", MODE:="660", SYMLINK+="asm-data01"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c4500763534334d612d363538792d32626e6e", ACTION=="add|change", OWNER:="grid", GROUP:="asmadmin", MODE:="660", SYMLINK+="asm-data02"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c45004a7a416363632d523475692d746f3068", ACTION=="add|change", OWNER:="grid", GROUP:="asmadmin", MODE:="660", SYMLINK+="asm-data03"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c4500423356646a332d4565674d2d74515555", ACTION=="add|change", OWNER:="grid", GROUP:="asmadmin", MODE:="660", SYMLINK+="asm-fra01"


udev生效指令
/sbin/udevadm control --reload-rules
/sbin/udevadm trigger --type=devices --action=change

[root@rac01 ~]# ls -l /dev/dm*
brw-rw---- 1 root disk     253, 0 May 28 14:09 /dev/dm-0
brw-rw---- 1 root disk     253, 1 May 28 14:09 /dev/dm-1
brw-rw---- 1 root disk     253, 2 May 28 14:09 /dev/dm-2
brw-rw---- 1 grid asmadmin 253, 3 May 28 14:09 /dev/dm-3
brw-rw---- 1 grid asmadmin 253, 4 May 28 14:09 /dev/dm-4
brw-rw---- 1 grid asmadmin 253, 5 May 28 14:09 /dev/dm-5
brw-rw---- 1 grid asmadmin 253, 6 May 28 14:09 /dev/dm-6
brw-rw---- 1 grid asmadmin 253, 7 May 28 14:09 /dev/dm-7
brw-rw---- 1 grid asmadmin 253, 8 May 28 14:09 /dev/dm-8
brw-rw---- 1 grid asmadmin 253, 9 May 28 14:09 /dev/dm-9
[root@rac01 ~]# ls -l /dev/asm*
lrwxrwxrwx 1 root root 4 May 28 14:09 /dev/asm-data01 -> dm-7
lrwxrwxrwx 1 root root 4 May 28 14:09 /dev/asm-data02 -> dm-6
lrwxrwxrwx 1 root root 4 May 28 14:09 /dev/asm-data03 -> dm-8
lrwxrwxrwx 1 root root 4 May 28 14:09 /dev/asm-fra01 -> dm-9
lrwxrwxrwx 1 root root 4 May 28 14:09 /dev/asm-ocr1 -> dm-4
lrwxrwxrwx 1 root root 4 May 28 14:09 /dev/asm-ocr2 -> dm-5
lrwxrwxrwx 1 root root 4 May 28 14:09 /dev/asm-ocr3 -> dm-3

将99-oracle-asmdevice.rules传输给RAC节点2
[root@rac01 /etc/udev/rules.d]# scp -p 99-oracle-asmdevice.rules root@172.16.0.102:${PWD}
99-oracle-asmdevice.rules                                                                                                            100% 1246     2.7MB/s   00:00
生效udev配置
[root@rac02 ~]# /sbin/udevadm control --reload-rules
[root@rac02 ~]# /sbin/udevadm trigger --type=devices --action=change
检查节点2
[root@rac02 ~]# ls -l /dev/dm*
brw-rw---- 1 root disk     253, 0 May 28 14:49 /dev/dm-0
brw-rw---- 1 root disk     253, 1 May 28 14:49 /dev/dm-1
brw-rw---- 1 root disk     253, 2 May 28 14:49 /dev/dm-2
brw-rw---- 1 grid asmadmin 253, 3 May 28 14:49 /dev/dm-3
brw-rw---- 1 grid asmadmin 253, 4 May 28 14:49 /dev/dm-4
brw-rw---- 1 grid asmadmin 253, 5 May 28 14:49 /dev/dm-5
brw-rw---- 1 grid asmadmin 253, 6 May 28 14:49 /dev/dm-6
brw-rw---- 1 grid asmadmin 253, 7 May 28 14:49 /dev/dm-7
brw-rw---- 1 grid asmadmin 253, 8 May 28 14:49 /dev/dm-8
brw-rw---- 1 grid asmadmin 253, 9 May 28 14:49 /dev/dm-9
[root@rac02 ~]# ls -l /dev/asm*
lrwxrwxrwx 1 root root 4 May 28 14:49 /dev/asm-data01 -> dm-6
lrwxrwxrwx 1 root root 4 May 28 14:49 /dev/asm-data02 -> dm-7
lrwxrwxrwx 1 root root 4 May 28 14:49 /dev/asm-data03 -> dm-8
lrwxrwxrwx 1 root root 4 May 28 14:49 /dev/asm-fra01 -> dm-9
lrwxrwxrwx 1 root root 4 May 28 14:49 /dev/asm-ocr1 -> dm-3
lrwxrwxrwx 1 root root 4 May 28 14:49 /dev/asm-ocr2 -> dm-4
lrwxrwxrwx 1 root root 4 May 28 14:49 /dev/asm-ocr3 -> dm-5
[root@rac02 ~]# ls -l /dev/mapper/
total 0
lrwxrwxrwx 1 root root       7 May 28 14:49 asm-data01 -> ../dm-6
lrwxrwxrwx 1 root root       7 May 28 14:49 asm-data02 -> ../dm-7
lrwxrwxrwx 1 root root       7 May 28 14:49 asm-data03 -> ../dm-8
lrwxrwxrwx 1 root root       7 May 28 14:49 asm-fra01 -> ../dm-9
lrwxrwxrwx 1 root root       7 May 28 14:49 asm-ocr1 -> ../dm-3
lrwxrwxrwx 1 root root       7 May 28 14:49 asm-ocr2 -> ../dm-4
lrwxrwxrwx 1 root root       7 May 28 14:49 asm-ocr3 -> ../dm-5
crw------- 1 root root 10, 236 May 28 14:49 control
lrwxrwxrwx 1 root root       7 May 28 14:49 rhel-home -> ../dm-2
lrwxrwxrwx 1 root root       7 May 28 14:49 rhel-root -> ../dm-0
lrwxrwxrwx 1 root root       7 May 28 14:49 rhel-swap -> ../dm-1

重启操作系统

参考文档：
《使用OpenFiler来模拟存储配置RAC中ASM共享盘及多路径（multipath）的测试》
https://www.cnblogs.com/lhrbest/p/6345157.html
《存储基本概念（lun,volume,HBA,DAS,NAS,SAN,iSCSI,IPSAN)》
https://blog.csdn.net/liukuan73/article/details/45506441

勤学苦练羊
关注
  • 0
    点赞
  • 踩
  • 3
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 1
    评论
专栏目录
oracle asm块大小,如何调整ASM磁盘大小
weixin_42342032的博客
04-03 710
直接修改lun的大小，证明可行！以下是我在11G RAC系统做的一个在存储中修改lun的大小来提升asm磁盘组的容量.1，停止所有rac系统相关节点后，在存储里修改lun大小。2，重启各节点或者安装sg3_utils扫描也可以重新识别扩容的磁盘；[root@oadb02 ~]# fdisk-l磁盘 /dev/sda：599.9 GB, 599932844032 字节，1171743836 个扇...
linux 存储映射lun 给_在Linux中针对物理磁盘和LUN映射Oracle ASM磁盘
weixin_39532019的博客
12-30 422
本文可帮助你查找哪个ASM磁盘映射到哪个Linux分区或DM设备。前言如果你作为Linux管理员长时间退出工作，你可能已经了解了ASM，Device Mapper Multipathing(DM-Multipathing)。有多种方法可以检查这些信息，但是，使用默认命令时，你将获得部分信息。它不会在单个输出中一起显示所有内容。如果要在单个输出中一起检查所有内容，那么我们需要编写一个小的shell脚...
v7000 lun 扩展 oracle asm,【ASM】ASMLIB 系列-- oracleasm
weixin_35926485的博客
04-13 649
【ASM】ASMLIB 系列说明：RHEL6、RHEL7、CentOS6、CentOS7 都可以使用asmlib，只不过需要安装kmod-oracleasm包。RHEL5不需要安装kmod-oracleasm包。下载地址可直接在官网搜索：www.oracle.com----------------------------oracleasm常用命令asmliboracleasm日志：　ta...
Oracle ASM 命令行方式添加聚合磁盘
weixin_37644222的博客
05-29 546
首先在存储上划分LUN，本文以openfiler进行举例 划分两个8G的LUN 一个给DATA磁盘组 一个给FRA 磁盘组 map LUN 在RAC两个节点，分别扫描LUN设备 扫盘脚本如下，root用户执行 [root@rac01 ~]# cat scan-disk.sh #!/bin/bash #scan disk for i in $(seq 0 100) do echo '- - -' > /sys/class/scsi_host/host${i}/scan done &&gt
Oracle ASM 概念、配置及常用管理命令详解
还不算晕的专栏
09-11 3060
大部分转自DAVE，个人整理及增加了使用UDEV绑定ASM磁盘部分。 ASM:Automatic Storage Management 是Oracle 主推的一种面向Oracle的存储解决方案， ASM 和 RDBMS 非常相似，ASM 也是由实例和文件组成， 也可以通过sqlplus 工具来维护。   ASM 实例的创建和删除也可以用DBCA 这个命令来操作。在dbca 的
Oracle12c 实战ASM(12.1) 安装配置
最新发布
03-05
Oracle 12c ASM（12.1）安装配置 Oracle 12c ASM（12.1）是 Oracle 公司推出的自动存储管理（Automatic Storage...Oracle 12c ASM（12.1）安装配置完成后，可以使用 ASM 管理数据库存储资源，提高数据库性能和可靠性。
kmod-oracleasm-2.0.8-17.rhel7.3.x86_64.rpm
10-15
RHEL7.3或者CentOS7.3安装oracle 安装RAC使用ASM磁盘必备依赖包，亲测可用 kmod-oracleasm-2.0.8-17.el7.3.x86_64
（rhel7） oracleasm安装rpm
04-17
（rhel7） oracleasm rpm安装包：kmod-oracleasm-2.0.8-19.el7.x86_64.rpm oracleasmlib-2.0.12-1.el7.x86_64.rpm oracleasm-support-2.1.8-3.el7.x86_64.rpm
RHEL 7环境下19c RAC安装之udev规则配置ASM共享存储.docx
09-23
在RHEL 7环境下搭建Oracle 19c的Real Application Clusters (RAC)时，配置udev规则以确保ASM(Automatic Storage Management)共享存储的稳定性和可预测性至关重要。udev是Linux系统中的设备管理器，它负责管理和命名...
oracle-support oracleasmlib kmod-oracleasm包
12-31
Oracle Support for Oracle ASM (Automatic Storage Management) 和 Kmod-OracleASM 包是Oracle数据库系统在Linux环境下进行存储管理的关键组件。这些包确保了在Red Hat Enterprise Linux (RHEL)、CentOS以及Oracle ...
RHEL 7配置Multipath多路径构建iSCSI模拟环境.docx
05-17
RHEL 7配置Multipath多路径构建iSCSI模拟环境.docx
oracle ASM在线扩容-加盘
anshen_oracle的博客
03-19 664
oracle ASM磁盘组在线加盘 很多时候，我们DBA都会遇到ASM磁盘空间问题需要加盘，这个时候我们DBA一般都会申请停机，而很多时候，一些核心业务是不能停机的，但这个时候往往最尴尬。 接下来我们以AIX 小机和Linux 平台为例讲解ASM在线加盘。 AIX小鸡加盘姿势一： 存储工程师通知到我们存储已将lun 划好映射给主机。（注意：这个时候千万不要忘记了，lun有ldev号，一定跟存储工程师确认好，不然出事就是事故） 1.扫盘操作： cfgmgr 2.确定盘已经...
oracle rebalance参数,【案例】Oracle ASM扩展新LAN加入asm diskgroup asm rebalance 原理
weixin_29587979的博客
04-06 568
天萃荷净Oracle研究中心案例分析：运维DBA反映Oracle数据库的ASM空间不足，需要扩展。通过划新的LAN加入asm diskgroup并分析asm rebalance 原理。本站文章除注明转载外，均为本站原创： 转载自love wife & love life —Roger 的Oracle技术博客本文链接地址: asm rebalance 原理上周6,某客户进行存储扩容(11gR...
hpux oracle rac添加、删除asm磁盘
old boy的博客
07-28 2002
oracle rac数据库的磁盘组空间不足，需要从存储上划分磁盘进行磁盘组的扩容，主要分为以下几步。 环境： 内容 版本 数据库 oracle 11.2.0.4 rac 操作系统 hpux 11.31 一、存储划分磁盘，并同时映射到rac的两个节点 ...
oracle asmlib 7,【ASM】ASMLIB 系列-- oracleasm
weixin_34129222的博客
04-04 871
【ASM】ASMLIB 系列说明：RHEL6、RHEL7、CentOS6、CentOS7 都可以使用asmlib，只不过需要安装kmod-oracleasm包。RHEL5不需要安装kmod-oracleasm包。下载地址可直接在官网搜索：----------------------------oracleasm常用命令asmliboracleasm日志：　tail-f/var/log...
oracle新增磁盘,oracle asm添加磁盘扩容
weixin_34797871的博客
04-09 441
1.存储上面挂载lun2.操作系统上面执行如下操作(不需重启操作系统)：echo "- - -" > /sys/class/scsi_host/host0/scanecho "- - - > /sys/class/scsi_host/host1/scan..........一步操作完成：echo "- - - > /sys/class/scsi_host/host**/sca...
centos 7.6 multipath 之二 multipath 设备名称在各服务器上保持一致
一名数据库爱好者的专栏
08-07 3385
os: centos 7.6 如果需要确保不同服务器上的 multipath 名称一致,就需要使用如下任一种方法. bindings 文件 首先在一台服务器上建立所有 multipath 设备,然后把 bindings 文件拷贝到其余的机器上 # cat /etc/multipath/wwids 3600508b4000156d700012000000b0000 3600508b4000156d7...
使用Device Mapper Multipath
沈启龙的专栏
08-16 8568
Device Mapper Multipath（DM-Multipath）可以将服务器节点和存储阵列之间的多条I/O链路配置为一个单独的设备。这些I/O链路是由不同的线缆、交换机、控制器组成的SAN物理链路。Multipath将这些链路聚合在一起，生成一个单独的新的设备。
multipath问题解决方案
db_alleria的专栏
11-20 6373
Multipath是多路径聚合软件，用于解决SAN环境下，多条I/O路径的流量分配、路径管理。对于OS而言，每条路径都是一块物理盘，在没有多路径聚合软件的情况下，会出现实际上单一盘由于多链路导致系统认到多块盘的情况。多路径软件常用于故障切换和链路冗余和I\O流量的负载均衡 Multipath的应用操作：1、查看服务进程multipathd。 [root@linuxso.com ~
Oracle ASM安装与配置教程
这通常包括创建ASM磁盘组、格式化磁盘以便ASM使用，以及配置Oracle实例来识别和使用ASM存储。以下是基本步骤： 1. 创建ASM磁盘：使用`oradisk`工具发现并添加新添加的硬盘到ASM，如`oradisk -n DISKGROUP_NAME /dev...
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

勤学苦练羊

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值