oracle asmlib 7,从Oracle ASMLIB迁移到ASM Filter Driver

ASM Filter Driver是一个可以取代 ASMLIB 和 udev 设置的新功能，它可以一举取代 ASMLIB 和手动设置 udev rules 文件的繁琐，并且最重要的是 I/O Filter 功能。所谓的I/O Filter功能可以拒绝所有无效的 I/O 请求，最主要的作用是防止意外覆写 ASM 磁盘的底层盘，在后面的测试中可以看到对于 root 用户的 dd 全盘清零这样的变态操作也都是可以过滤的。

1、查询集群状态

[root@odb02 ~]# crsctl check cluster -all

**************************************************************

odb02:

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

**************************************************************

odb03:

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

**************************************************************

2、查询集群模式

[root@odb02 ~]# crsctl get cluster mode status

Cluster is running in "flex" mode

从12c R2以后，默认的集群模式为flex。

3、查询当前的ASM磁盘以及发现路径

[root@odb02 ~]# Oracleasm listdisks | xargs oracleasm querydisk -p

Disk "DATA01" is a valid ASM disk

/dev/sdb1: LABEL="DATA01" TYPE="oracleasm"

Disk "FRA001" is a valid ASM disk

/dev/sdc1: LABEL="FRA001" TYPE="oracleasm"

[root@odb02 ~]# asmcmd dsget

parameter:/dev/oracleasm/disks/*

profile:/dev/oracleasm/disks/*

4、设置新的ASM磁盘发现路径

[grid@odb02 ~]$ asmcmd dsset '/dev/oracleasm/disks/*','AFD:*'

[grid@odb02 ~]$ asmcmd dsget

parameter:/dev/oracleasm/disks/*, AFD:*

profile:/dev/oracleasm/disks/*,AFD:*

5、检查 GI 环境中的节点

[grid@odb02 ~]$ olsnodes

odb02

odb03

6、RAC所有节点停止集群服务

[root@odb02 ~]# crsctl stop crs

[root@odb03 ~]# crsctl stop crs

7、RAC所有节点卸载ASMLIB软件

odb02

[root@odb02 ~]# systemctl disable oracleasm

Removed symlink /etc/systemd/system/multi-user.target.wants/oracleasm.service.

[root@odb02 ~]# oracleasm status

Checking if ASM is loaded: yes

Checking if /dev/oracleasm is mounted: yes

[root@odb02 ~]# oracleasm exit

Unmounting ASMlib driver filesystem: /dev/oracleasm

Unloading module "oracleasm": oracleasm

[root@odb02 ~]# ls -ltr /dev/oracleasm/

total 0

[root@odb02 ~]# yum -y remove oracleasm-support

odb03

[root@odb03 ~]# systemctl disable oracleasm

Removed symlink /etc/systemd/system/multi-user.target.wants/oracleasm.service.

[root@odb03 ~]# oracleasm status

Checking if ASM is loaded: yes

Checking if /dev/oracleasm is mounted: yes

[root@odb03 ~]# oracleasm exit

Unmounting ASMlib driver filesystem: /dev/oracleasm

Unloading module "oracleasm": oracleasm

[root@odb03 ~]# ls -ltr /dev/oracleasm/

total 0

[root@odb03 ~]# yum -y remove oracleasm-support

8、RAC所有节点停止acfsload服务

[root@odb02 ~]# lsmod|grep acfs

[root@odb02 ~]# acfsload stop

[root@odb02 ~]# lsmod|grep acfs

9、RAC所有节点配置AFD

AFD Configure，实际上这是一个解压程序包，安装，并加载 Driver 的过程，需要消耗一些时间。

[root@odb02 ~]# asmcmd afd_configure

ASMCMD-9523: command cannot be used when Oracle Clusterware stack is up

--如果遇到以上报错，先停止has，再执行一次。

[root@odb02 ~]# crsctl stop has

[root@odb02 ~]# asmcmd afd_configure

AFD-627: AFD distribution files found.

AFD-634: Removing previous AFD installation.

AFD-635: Previous AFD components successfully removed.

AFD-636: Installing requested AFD software.

AFD-637: Loading installed AFD drivers.

AFD-9321: Creating udev for AFD.

AFD-9323: Creating module dependencies - this may take some time.

AFD-9154: Loading 'oracleafd.ko' driver.

AFD-649: Verifying AFD devices.

AFD-9156: Detecting control device '/dev/oracleafd/admin'.

AFD-638: AFD installation correctness verified.

Modifying resource dependencies - this may take some time.

配置完成后，验证下状态：

[root@odb02 ~]# asmcmd afd_state

ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'odb02'

[root@odb03 ~]# asmcmd afd_state

ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'odb03'

--如果上述的状态为disabled，则使用下面的命令启用

[root@odb02 ~]# asmcmd afd_filter -e

另外，会生成一个/etc/oracleafd.conf文件，内容如下：

[root@odb02 ~]# cat /etc/oracleafd.conf

afd_diskstring='/dev/oracleasm/disks/*'

如果其他节点没有，就把这个文件复制一份过去，避免启动crs的时候出错。

10、迁移所有ASM磁盘

这个在一个节点操作，在其他的节点中，不再需要作label，而是直接scan即可，这跟使用 ASMLIB的操作非常相似。另外命令后面必须加migrate参数，才可以成功。

[root@odb02 ~]# asmcmd afd_label data01 /dev/sdb1 --migrate

[root@odb02 ~]# asmcmd afd_label fra001 /dev/sdc1 --migrate

[root@odb02 ~]# asmcmd afd_lsdsk

--------------------------------------------------------------------------------

Label Filtering Path

================================================================================

DATA01 ENABLED /dev/sdb1

FRA001 ENABLED /dev/sdc1

其他节点执行扫描操作：

[grid@odb03 ~]$ asmcmd afd_scan

[grid@odb03 ~]$ asmcmd afd_lsdsk

--------------------------------------------------------------------------------

Label Filtering Path

================================================================================

DATA01 ENABLED /dev/sdb1

FRA001 ENABLED /dev/sdc1

最后，更新/etc/oracleafd.conf的内容如下：

[root@odb02 ~]# cat /etc/oracleafd.conf

afd_diskstring='/dev/sd*

11、RAC所有节点启动acfsload

[root@odb02 ~]# acfsload start

ACFS-9391: Checking for existing ADVM/ACFS installation.

ACFS-9392: Validating ADVM/ACFS installation files for operating system.

ACFS-9393: Verifying ASM Administrator setup.

ACFS-9308: Loading installed ADVM/ACFS drivers.

ACFS-9327: Verifying ADVM/ACFS devices.

ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.

ACFS-9156: Detecting control device '/dev/ofsctl'.

ACFS-9322: completed

12、RAC所有节点启动CRS

[root@odb02 ~]# crsctl start crs

[root@odb03 ~]# crsctl start crs

待所有服务启动后，检查下diskstring参数设置：

[grid@odb02 ~]$ asmcmd dsget

parameter:/dev/oracleasm/disks/*, AFD:*

profile:/dev/oracleasm/disks/*,AFD:*

[grid@odb02 ~]$ asmcmd dsset 'AFD:*'

[grid@odb02 ~]$ asmcmd dsget

parameter:AFD:*

profile:AFD:*

[grid@odb02 ~]$ asmcmd lsdsk

Path

AFD:DATA01

AFD:FRA001

最后通过SQLplus执行下面的SQL进行验证：

set linesize 300

col PATH for a20

set pages 20

col LIBRARY for a45

col NAME for a15

select inst_id,group_number grp_num,name,state,header_status header,mount_status mount,path, library

from gv$asm_disk order by inst_id,group_number,name;