selinux 引起的 Instantiating disk: failed

asm 初始化创建ASM磁盘组 失败

[root@dga01 ~]# oracleasm createdisk OCR /dev/sde1
Writing disk header: done
Instantiating disk: failed
Clearing disk header: done


开始以为是asm配置错误，于是重新初始化ASM


[root@dga01 ~]# /etc/init.d/oracleasm   configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface [grid]: grid
Default group to own the driver interface [asmadmin]: asmadmin
Start Oracle ASM library driver on boot (y/n) [y]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: [  OK  ]
Scanning the system for Oracle ASMLib disks: [  OK  ]


[root@dga01 ~]# /usr/sbin/oracleasm  init

但是 重新初始化后创建ASM磁盘组 依然失败

[root@dga01 ~]# oracleasm createdisk OCR /dev/sde1
Writing disk header: done
Instantiating disk: failed
Clearing disk header: done
[root@dga01 ~]# /usr/sbin/oracleasm createdisk OCR /dev/sde1
Writing disk header: done
Instantiating disk: failed
Clearing disk header: done



通过linux提供的 dmesg 和 strace 来定位问题

[root@dga01 ~]# dmesg 

sd 2:0:1:0: [sdb] Cache data unavailable
sd 2:0:1:0: [sdb] Assuming drive cache: write through
sd 2:0:1:0: [sdb] Attached SCSI disk
sd 3:0:0:0: [sde] Cache data unavailable
sd 3:0:0:0: [sde] Assuming drive cache: write through
sd 3:0:0:0: [sde] Attached SCSI disk
sd 2:0:3:0: [sdd] Cache data unavailable
sd 2:0:3:0: [sdd] Assuming drive cache: write through
sd 2:0:3:0: [sdd] Attached SCSI disk
EXT4-fs (sda3): mounted filesystem with ordered data mode. Opts: (null)
dracut: Mounted root filesystem /dev/sda3
dracut: Loading SELinux policy
type=1404 audit(1363446394.257:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295
SELinux: 2048 avtab hash slots, 250819 rules.
SELinux: 2048 avtab hash slots, 250819 rules.
SELinux:  9 users, 12 roles, 3762 types, 187 bools, 1 sens, 1024 cats
SELinux:  81 classes, 250819 rules
SELinux:  Permission audit_access in class file not defined in policy.
SELinux:  Permission audit_access in class dir not defined in policy.
SELinux:  Permission execmod in class dir not defined in policy.
SELinux:  Permission audit_access in class lnk_file not defined in policy.
SELinux:  Permission open in class lnk_file not defined in policy.
SELinux:  Permission execmod in class lnk_file not defined in policy.
SELinux:  Permission audit_access in class chr_file not defined in policy.
SELinux:  Permission audit_access in class blk_file not defined in policy.
SELinux:  Permission execmod in class blk_file not defined in policy.
SELinux:  Permission audit_access in class sock_file not defined in policy.
SELinux:  Permission execmod in class sock_file not defined in policy.
SELinux:  Permission audit_access in class fifo_file not defined in policy.
SELinux:  Permission execmod in class fifo_file not defined in policy.
SELinux:  Permission syslog in class capability2 not defined in policy.
SELinux: the above unknown classes and permissions will be allowed


[root@dga01 ~]# strace  -f -o asm.out /usr/sbin/oracleasm createdisk OCR /dev/sde1

3714  brk(0)                            = 0x1677000
3714  brk(0x1698000)                    = 0x1698000
3714  stat("/dev/sde1", {st_mode=S_IFBLK|0660, st_rdev=makedev(8, 65), ...}) = 0
3714  open("/dev/sde1", O_RDWR)         = 4
3714  fstat(4, {st_mode=S_IFBLK|0660, st_rdev=makedev(8, 65), ...}) = 0
3714  fstat(4, {st_mode=S_IFBLK|0660, st_rdev=makedev(8, 65), ...}) = 0
3714  mknod("/dev/oracleasm/disks/OCR", S_IFBLK|0600, makedev(8, 65)) = -1 EACCES (Permission denied)
3714  write(2, "oracleasm-instantiate-disk: ", 28) = 28
3714  write(2, "Unable to create ASM disk \"OCR\":"..., 51) = 51
3714  close(4)  


日志中多次提到selinux 和 3714  mknod("/dev/oracleasm/disks/OCR", S_IFBLK|0600, makedev(8, 65)) = -1 EACCES (Permission denied)

问题可能出在selinux或者防火墙上，查看selinux和防火墙状态


[root@dga01 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED 
ACCEPT     icmp --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp dpt:ssh 
REJECT     all  --  anywhere             anywhere            reject-with icmp-host-prohibited 

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
REJECT     all  --  anywhere             anywhere            reject-with icmp-host-prohibited 

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

  
[root@dga01 ~]# getenforce 
Enforcing

iptables 与selinux均为开启 ，尝试关闭着两个服务

关闭linux 防火墙

[root@dga01 ~]# iptables -F
[root@dga01 ~]# service iptables stop
iptables: Flushing firewall rules: [  OK  ]
iptables: Setting chains to policy ACCEPT: filter [  OK  ]
iptables: Unloading modules: [  OK  ]
[root@dga01 log]# chkconfig  iptables off

关闭selinux 服务
[root@dga01 ~]# setenforce 0

编辑selinux配置文件修改 为SELINUX=disabled
[root@dga01 ~]# vim /etc/selinux/config 

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

再次查看linux防火墙与selinux服务

[root@dga01 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination 


[root@dga01 log]# getenforce 
Permissive


重新创建ASM磁盘组，顺利完成

[root@dga01 ~]# oracleasm createdisk OCR /dev/sde1
Writing disk header: done
Instantiating disk: done