配置Centos上使用ACFS集群文件系统

在centos6.8 64位系统上安装完一套oracle 11.2.0.4 rac测试环境后，发现acfs和advm无法使用，ASMCA图形界面的acfs和volume选项卡为灰色！这是由于acfs不支持centos系统导致的，下面就介绍下如何在配置centos上使用acfs集群文件系统！

测试环境：

OS:Centos 6.8

Oracle:11.2.0.4 rac1 rac2 rac3

1.问题描述：

[root@rac1 lib]# su - grid
+ASM1:/home/grid@rac1>export DISPLAY=192.168.180.186:0.0
+ASM1:/home/grid@rac1>asmca

2：修改osds_acfslib.pm脚本代码,所有节点都要执行

[root@rac1 lib]# cd /u01/app/11.2.0/grid_1/lib/

[root@rac1 lib]# cp -p osds_acfslib.pm osds_acfslib.pm.bak

[root@rac1 lib]# vi osds_acfslib.pm

修改以下内容

if ((defined($release)) &&                     # Redhat or OEL if defined
      (($release =~ /^redhat-release/) ||        # straight RH
       ($release =~ /^enterprise-release/) ||    # Oracle Enterprise Linux
       ($release =~ /^oraclelinux-release/)))    # Oracle Linux
  

将上面的代码片段修改如下

 if ((defined($release)) &&                     # Redhat or OEL if defined
      (($release =~ /^redhat-release/) ||        # straight RH
       ($release =~ /^enterprise-release/) ||    # Oracle Enterprise Linux
       ($release =~ /^centos-release/) ||         #CentOS hack
       ($release =~ /^oraclelinux-release/)))    # Oracle Linux

3：安装acfs，配置acfs和advm模块启动自动加载，其他节点做同样的配置

[root@rac1 lib]# /u01/app/11.2.0/grid_1/bin/acfsload start -s

ACFS-9129: ADVM/ACFS not installed 

[root@rac1 lib]# /u01/app/11.2.0/grid_1/bin/acfsroot install
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies - this may take some time.
ACFS-9154: Loading 'oracleoks.ko' driver.
ACFS-9154: Loading 'oracleadvm.ko' driver.
ACFS-9154: Loading 'oracleacfs.ko' driver.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
ACFS-9156: Detecting control device '/dev/ofsctl'.
ACFS-9309: ADVM/ACFS installation correctness verified.
[root@rac1 lib]# /u01/app/11.2.0/grid_1/bin/acfsload start -s
[root@rac1 lib]# lsmod |grep oracle
oracleacfs           1990406  0 
oracleadvm            250040  0 
oracleoks             427672  2 oracleacfs,oracleadvm

[root@rac1 lib]# vi /etc/init.d/acfsload

#!/bin/sh
# chkconfig: 2345 30 21
# description: Load Oracle ASM volume driver on system startup
ORACLE_HOME=/u01/app/11.2.0/grid_1
export ORACLE_HOME
$ORACLE_HOME/bin/acfsload start -s

[root@rac1 lib]# chmod 755 /etc/init.d/acfsload 
[root@rac1 lib]# chkconfig --add acfsload
[root@rac1 lib]# chkconfig --list |grep acfsload
acfsload       0:off 1:off 2:on 3:on 4:on 5:on 6:off

4.添加ora.registry.acfs到ocr配置中（在节点1上执行就可以了）

[root@rac1 template]# /u01/app/11.2.0/grid_1/bin/crsctl add type ora.registry.acfs.type -basetype ora.local_resource.type -file /u01/app/11.2.0/grid_1/crs/template/registry.acfs.type

[root@rac1 ~]# /u01/app/11.2.0/grid_1/bin/crsctl add resource ora.registry.acfs -attr ACL=\'owner:root:rwx,pgrp:oinstall:r-x,other::r--\'  -type ora.registry.acfs.type -f 

查看节是否添加成功
[root@rac1 ~]# su - grid -c crs_stat | grep acfs 
NAME=ora.registry.acfs
TYPE=ora.registry.acfs.type

[root@rac2 ~]# su - grid -c crs_stat | grep acfs 
NAME=ora.registry.acfs
TYPE=ora.registry.acfs.type

[root@rac3 ~]# su - grid -c crs_stat | grep acfs 
NAME=ora.registry.acfs
TYPE=ora.registry.acfs.type

[root@rac1 ~]# /sbin/acfsutil registry

5.创建卷并格式化成acfs集群文件系统并挂载

SQL> conn /as sysasm
Connected.
SQL> alter diskgroup ACFS add volume acfsvol size 1G;


Diskgroup altered.


SQL> select group_number,name,state,type from v$asm_diskgroup;


GROUP_NUMBER NAME    STATE TYPE
------------ ------------------------------ ----------- ------
  1 DATA    MOUNTED EXTERN
  2 FRA_ARC    MOUNTED EXTERN
  3 OCR_VOTING_NEW    MOUNTED NORMAL
  4 ACFS    MOUNTED EXTERN

这时候查看asmca 可以看到：

创建共享磁盘：

root@rac1 ~]# mkdir /shareddisk

root@rac2 ~]# mkdir /shareddisk

root@rac3 ~]# mkdir /shareddisk

[root@rac1 ~]# /sbin/mkfs -t acfs -n acfsvol1  /dev/asm/acfsvol-440 
mkfs.acfs: version                   = 11.2.0.4.0
mkfs.acfs: on-disk version           = 39.0
mkfs.acfs: volume                    = /dev/asm/acfsvol-440
mkfs.acfs: volume size               = 1073741824
mkfs.acfs: Format complete.

[root@rac1 ~]# /sbin/acfsutil registry -a -f /dev/asm/acfsvol-440 /shareddisk/
acfsutil registry: mount point /shareddisk successfully added to Oracle Registry

[root@rac1 ~]# mount.acfs -o all
[root@rac1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_rac1-lv_root
                       50G   19G   29G  40% /
tmpfs                 939M  540M  399M  58% /dev/shm
/dev/sda1             477M   41M  411M  10% /boot
/dev/mapper/vg_rac1-lv_home
                       26G   45M   24G   1% /home
/dev/sdb1              40G  9.4G   28G  25% /u01
/dev/asm/acfsvol-440  1.0G  117M  908M  12% /shareddisk

[root@rac1 ~]# /sbin/acfsutil info fs
/shareddisk
    ACFS Version: 11.2.0.4.0
    flags:        MountPoint,Available
    mount time:   Sat Mar 11 07:45:26 2017
    volumes:      1
    total size:   1073741824
    total free:   951771136
    primary volume: /dev/asm/acfsvol-440
        label:                 acfsvol1
        flags:                 Primary,Available,ADVM
        on-disk version:       39.0
        allocation unit:       4096
        major, minor:          252, 225281
        size:                  1073741824
        free:                  951771136
        ADVM diskgroup         ACFS
        ADVM resize increment: 33554432
        ADVM redundancy:       unprotected
        ADVM stripe columns:   4
        ADVM stripe width:     131072
    number of snapshots:  0
    snapshot space usage: 0
    replication status: DISABLED

至此Centos上使用ACFS集群文件系统配置完成