linux下使用裸设备存储选项安装oracle

1.5.1.3 Raw Devices

Raw devices are disk partitions or logical volumes that have not been formatted with a file system. When you use raw devices for database file storage, Oracle writes data directly to the partition or volume, bypassing the operating system file system layer. For this reason, you can sometimes achieve performance gains by using raw devices. However, because raw devices can be difficult to create and administer, and because the performance gains over modern file systems are minimal, Oracle recommends that you choose Automatic Storage Management or file system storage in preference to raw devices.

linux下使用裸设备存储选项安装oracle时可以参照以下方式进行配置

此例中linux下有/dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf
/dev/sda 用于操作系统软件安装 
/dev/sdb 用于做ocr disk
/dev/sdc 用于做votingdisk
/dev/sdd /dev/sde /dev/sdf 用于做共享磁盘存储，每块磁盘按照指定大小（500M）划分分区
磁盘分区情况如下
[root@node1 ~]# fdisk -l

Disk /dev/sda: 16.1 GB, 16106127360 bytes
255 heads, 63 sectors/track, 1958 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes


   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          64      514048+  83  Linux
/dev/sda2             833        1958     9044595   83  Linux
/dev/sda3              65         701     5116702+  83  Linux
/dev/sda4             702         832     1052257+   5  Extended
/dev/sda5             702         832     1052226   82  Linux swap / Solaris


Partition table entries are not in disk order


Disk /dev/sdb: 536 MB, 536870912 bytes
64 heads, 32 sectors/track, 512 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes


   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         512      524272   83  Linux


Disk /dev/sdc: 536 MB, 536870912 bytes
64 heads, 32 sectors/track, 512 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes


   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1         512      524272   83  Linux


Disk /dev/sdd: 3221 MB, 3221225472 bytes
255 heads, 63 sectors/track, 391 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes


   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1         391     3140676    5  Extended
/dev/sdd5               1          62      497952   83  Linux
/dev/sdd6              63         124      497983+  83  Linux
/dev/sdd7             125         186      497983+  83  Linux
/dev/sdd8             187         248      497983+  83  Linux
/dev/sdd9             249         310      497983+  83  Linux
/dev/sdd10            311         391      650601   83  Linux


Disk /dev/sde: 3221 MB, 3221225472 bytes
255 heads, 63 sectors/track, 391 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes


   Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1         391     3140676    5  Extended
/dev/sde5               1          74      594342   83  Linux
/dev/sde6              75         148      594373+  83  Linux
/dev/sde7             149         222      594373+  83  Linux
/dev/sde8             223         296      594373+  83  Linux
/dev/sde9             297         391      763056   83  Linux


Disk /dev/sdf: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes


   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1        1044     8385898+   5  Extended
/dev/sdf5               1          62      497952   83  Linux
/dev/sdf6              63         124      497983+  83  Linux
/dev/sdf7             125         186      497983+  83  Linux
/dev/sdf8             187         248      497983+  83  Linux
/dev/sdf9             249         310      497983+  83  Linux
/dev/sdf10            311         372      497983+  83  Linux
/dev/sdf11            373         434      497983+  83  Linux
/dev/sdf12            435         496      497983+  83  Linux
/dev/sdf13            497         558      497983+  83  Linux
/dev/sdf14            559         620      497983+  83  Linux
/dev/sdf15            621         682      497983+  83  Linux


Disk /dev/sdg: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes


将每个磁盘分区绑定成裸设备
每一条记录的ENV{MAJOR}=="8",ENV{MINOR}=="17" 可以从（ll /dev/sdf*） 设备属性的第5，6列得到。
编辑文件/etc/udev/rules.d/60-raw.rules 添加裸设备绑定
ACTION=="add", KERNEL=="/dev/sdb1",RUN+="/bin/raw /dev/raw/raw1 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="17",RUN+="/bin/raw /dev/raw/raw1 %M %m"
ACTION=="add", KERNEL=="/dev/sdc1",RUN+="/bin/raw /dev/raw/raw2 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="33",RUN+="/bin/raw /dev/raw/raw2 %M %m"


ACTION=="add", KERNEL=="/dev/sdd5",RUN+="/bin/raw /dev/raw/raw3 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="53",RUN+="/bin/raw /dev/raw/raw3 %M %m"
ACTION=="add", KERNEL=="/dev/sdd6",RUN+="/bin/raw /dev/raw/raw4 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="54",RUN+="/bin/raw /dev/raw/raw4 %M %m"
ACTION=="add", KERNEL=="/dev/sdd7",RUN+="/bin/raw /dev/raw/raw5 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="55",RUN+="/bin/raw /dev/raw/raw5 %M %m"
ACTION=="add", KERNEL=="/dev/sdd8",RUN+="/bin/raw /dev/raw/raw6 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="56",RUN+="/bin/raw /dev/raw/raw6 %M %m"
ACTION=="add", KERNEL=="/dev/sdd9",RUN+="/bin/raw /dev/raw/raw7 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="57",RUN+="/bin/raw /dev/raw/raw7 %M %m"
ACTION=="add", KERNEL=="/dev/sdd10",RUN+="/bin/raw /dev/raw/raw8 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="58",RUN+="/bin/raw /dev/raw/raw8 %M %m"




ACTION=="add", KERNEL=="/dev/sde5",RUN+="/bin/raw /dev/raw/raw9 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="69",RUN+="/bin/raw /dev/raw/raw9 %M %m"
ACTION=="add", KERNEL=="/dev/sde6",RUN+="/bin/raw /dev/raw/raw10 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="70",RUN+="/bin/raw /dev/raw/raw10 %M %m"
ACTION=="add", KERNEL=="/dev/sde7",RUN+="/bin/raw /dev/raw/raw11 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="71",RUN+="/bin/raw /dev/raw/raw11 %M %m"
ACTION=="add", KERNEL=="/dev/sde8",RUN+="/bin/raw /dev/raw/raw12 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="72",RUN+="/bin/raw /dev/raw/raw12 %M %m"
ACTION=="add", KERNEL=="/dev/sde9",RUN+="/bin/raw /dev/raw/raw13 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="73",RUN+="/bin/raw /dev/raw/raw13 %M %m"


ACTION=="add", KERNEL=="/dev/sdf5",RUN+="/bin/raw /dev/raw/raw14 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="85",RUN+="/bin/raw /dev/raw/raw14 %M %m"
ACTION=="add", KERNEL=="/dev/sdf6",RUN+="/bin/raw /dev/raw/raw15 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="86",RUN+="/bin/raw /dev/raw/raw15 %M %m"
ACTION=="add", KERNEL=="/dev/sdf7",RUN+="/bin/raw /dev/raw/raw16 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="87",RUN+="/bin/raw /dev/raw/raw16 %M %m"
ACTION=="add", KERNEL=="/dev/sdf8",RUN+="/bin/raw /dev/raw/raw17 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="88",RUN+="/bin/raw /dev/raw/raw17 %M %m"
ACTION=="add", KERNEL=="/dev/sdf9",RUN+="/bin/raw /dev/raw/raw18 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="89",RUN+="/bin/raw /dev/raw/raw18 %M %m"
ACTION=="add", KERNEL=="/dev/sdf10",RUN+="/bin/raw /dev/raw/raw18 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="90",RUN+="/bin/raw /dev/raw/raw19 %M %m"
ACTION=="add", KERNEL=="/dev/sdf11",RUN+="/bin/raw /dev/raw/raw18 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="91",RUN+="/bin/raw /dev/raw/raw20 %M %m"
ACTION=="add", KERNEL=="/dev/sdf12",RUN+="/bin/raw /dev/raw/raw18 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="92",RUN+="/bin/raw /dev/raw/raw21 %M %m"
ACTION=="add", KERNEL=="/dev/sdf13",RUN+="/bin/raw /dev/raw/raw18 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="93",RUN+="/bin/raw /dev/raw/raw22 %M %m"
ACTION=="add", KERNEL=="/dev/sdf14",RUN+="/bin/raw /dev/raw/raw18 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="94",RUN+="/bin/raw /dev/raw/raw23 %M %m"
ACTION=="add", KERNEL=="/dev/sdf15",RUN+="/bin/raw /dev/raw/raw18 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="95",RUN+="/bin/raw /dev/raw/raw24 %M %m"


##修改裸设备的属主和权限
KERNEL=="raw[1-9]", OWNER="oracle", GROUP="oinstall", MODE="640"
KERNEL=="raw1[0-9]", OWNER="oracle", GROUP="oinstall", MODE="640"
KERNEL=="raw2[0-9]", OWNER="oracle", GROUP="oinstall", MODE="640"


重启服务  
start_udev
--手动绑定
raw /dev/raw/raw3 /dev/sdd5
raw /dev/raw/raw4 /dev/sdd6
raw /dev/raw/raw5 /dev/sdd7
raw /dev/raw/raw6 /dev/sdd8
raw /dev/raw/raw7 /dev/sdd9
raw /dev/raw/raw8 /dev/sdd10


raw /dev/raw/raw9 /dev/sde5
raw /dev/raw/raw10 /dev/sde6
raw /dev/raw/raw11 /dev/sde7
raw /dev/raw/raw12 /dev/sde8
raw /dev/raw/raw13 /dev/sde9


注意  
在node1执行绑定后 raw -qa 可以查看到裸设备绑定信息
node2 需要更新使用命令partprobe /dev/sdd 来激活磁盘信息 才能看到绑定信息
创建裸设备连接
mkdir -p /opt/oracle/oradata/
ln -s /dev/raw/raw3 /opt/oracle/oradata/control01
ln -s /dev/raw/raw4 /opt/oracle/oradata/control02
ln -s /dev/raw/raw5 /opt/oracle/oradata/control03
ln -s /dev/raw/raw6 /opt/oracle/oradata/redo11
ln -s /dev/raw/raw7 /opt/oracle/oradata/redo12
ln -s /dev/raw/raw8 /opt/oracle/oradata/redo21
ln -s /dev/raw/raw9 /opt/oracle/oradata/redo22
ln -s /dev/raw/raw10 /opt/oracle/oradata/spfile
ln -s /dev/raw/raw11 /opt/oracle/oradata/system
ln -s /dev/raw/raw12 /opt/oracle/oradata/sysaux
ln -s /dev/raw/raw13 /opt/oracle/oradata/undotbs1
ln -s /dev/raw/raw14 /opt/oracle/oradata/undotbs2
ln -s /dev/raw/raw15 /opt/oracle/oradata/temp
ln -s /dev/raw/raw16 /opt/oracle/oradata/users




编辑安装时的配置文件
# vi /home/oracle/dbca.conf
spfile = /opt/oracle/oradata/spfile
system = /opt/oracle/oradata/system
sysaux = /opt/oracle/oradata/sysaux
undotbs1 = /opt/oracle/oradata/undotbs1
undotbs2 = /opt/oracle/oradata/undotbs2
temp = /opt/oracle/oradata/temp
users = /opt/oracle/oradata/users
redo1_1 = /opt/oracle/oradata/redo11
redo1_2 = /opt/oracle/oradata/redo12
redo2_1 = /opt/oracle/oradata/redo21
redo2_2 = /opt/oracle/oradata/redo22
control1 = /opt/oracle/oradata/control01
control2 = /opt/oracle/oradata/control02
control3 = /opt/oracle/oradata/control03


安装clusterware前检查


[root@node1 cluvfy]# ./runcluvfy.sh stage -pre crsinst -n node1,node2 -verbose


Performing pre-checks for cluster services setup


Checking node reachability...


Check: Node reachability from node "node1"
  Destination Node                      Reachable?
  ------------------------------------  ------------------------
  node1                                 yes
  node2                                 yes
Result: Node reachability check passed from node "node1".




Checking user equivalence...


Check: User equivalence for user "root"
  Node Name                             Comment
  ------------------------------------  ------------------------
  node2                                 passed
  node1                                 passed
Result: User equivalence check passed for user "root".


Checking administrative privileges...


Check: Existence of user "root"
  Node Name     User Exists               Comment
  ------------  ------------------------  ------------------------
  node2         yes                       passed
  node1         yes                       passed
Result: User existence check passed for "root".


Check: Existence of group "oinstall"
  Node Name     Status                    Group ID
  ------------  ------------------------  ------------------------
  node2         exists                    500
  node1         exists                    500
Result: Group existence check passed for "oinstall".


Check: Membership of user "root" in group "oinstall" [as Primary]
  Node Name         User Exists   Group Exists  User in Group  Primary       Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  node2             yes           yes           no            N/A           failed
  node1             yes           yes           no            N/A           failed
Result: Membership check for user "root" in group "oinstall" [as Primary] failed.


Administrative privileges check failed.


Checking node connectivity...




Interface information for node "node2"
  Interface Name                  IP Address                      Subnet
  ------------------------------  ------------------------------  ----------------
  eth0                            132.193.49.14                   132.193.49.0
  eth1                            10.10.10.12                     10.10.10.0




Interface information for node "node1"
  Interface Name                  IP Address                      Subnet
  ------------------------------  ------------------------------  ----------------
  eth0                            132.193.49.13                   132.193.49.0
  eth1                            10.10.10.11                     10.10.10.0




Check: Node connectivity of subnet "132.193.49.0"
  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  node2:eth0                      node1:eth0                      yes
Result: Node connectivity check passed for subnet "132.193.49.0" with node(s) node2,node1.


Check: Node connectivity of subnet "10.10.10.0"
  Source                          Destination                     Connected?
  ------------------------------  ------------------------------  ----------------
  node2:eth1                      node1:eth1                      yes
Result: Node connectivity check passed for subnet "10.10.10.0" with node(s) node2,node1.


Suitable interfaces for VIP on subnet "132.193.49.0":
node2 eth0:132.193.49.14
node1 eth0:132.193.49.13


Suitable interfaces for the private interconnect on subnet "10.10.10.0":
node2 eth1:10.10.10.12
node1 eth1:10.10.10.11


Result: Node connectivity check passed.




Checking system requirements for 'crs'...


Check: Total memory
  Node Name     Available                 Required                  Comment
  ------------  ------------------------  ------------------------  ----------
  node2         462.75MB (473860KB)       512MB (524288KB)          failed
  node1         462.75MB (473860KB)       512MB (524288KB)          failed
Result: Total memory check failed.


Check: Free disk space in "/tmp" dir
  Node Name     Available                 Required                  Comment
  ------------  ------------------------  ------------------------  ----------
  node2         1.46GB (1526816KB)        400MB (409600KB)          passed
  node1         1.37GB (1437380KB)        400MB (409600KB)          passed
Result: Free disk space check passed.


Check: Swap space
  Node Name     Available                 Required                  Comment
  ------------  ------------------------  ------------------------  ----------
  node2         1GB (1052216KB)           1GB (1048576KB)           passed
  node1         1GB (1052216KB)           1GB (1048576KB)           passed
Result: Swap space check passed.


Check: System architecture
  Node Name     Available                 Required                  Comment
  ------------  ------------------------  ------------------------  ----------
  node2         i686                      i686                      passed
  node1         i686                      i686                      passed
Result: System architecture check passed.


Check: Kernel version
  Node Name     Available                 Required                  Comment
  ------------  ------------------------  ------------------------  ----------
  node2         2.6.18-8.el5xen           2.4.21-15EL               passed
  node1         2.6.18-8.el5xen           2.4.21-15EL               passed
Result: Kernel version check passed.


Check: Package existence for "make-3.79"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  node2                           make-3.81-1.1                   passed
  node1                           make-3.81-1.1                   passed
Result: Package existence check passed for "make-3.79".


Check: Package existence for "binutils-2.14"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  node2                           binutils-2.17.50.0.6-2.el5      passed
  node1                           binutils-2.17.50.0.6-2.el5      passed
Result: Package existence check passed for "binutils-2.14".


Check: Package existence for "gcc-3.2"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  node2                           gcc-4.1.1-52.el5                passed
  node1                           gcc-4.1.1-52.el5                passed
Result: Package existence check passed for "gcc-3.2".


Check: Package existence for "glibc-2.3.2-95.27"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  node2                           glibc-2.5-12                    passed
  node1                           glibc-2.5-12                    passed
Result: Package existence check passed for "glibc-2.3.2-95.27".


Check: Package existence for "compat-db-4.0.14-5"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  node2                           missing                         failed
  node1                           missing                         failed
Result: Package existence check failed for "compat-db-4.0.14-5".


Check: Package existence for "compat-gcc-7.3-2.96.128"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  node2                           missing                         failed
  node1                           missing                         failed
Result: Package existence check failed for "compat-gcc-7.3-2.96.128".


Check: Package existence for "compat-gcc-c++-7.3-2.96.128"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  node2                           missing                         failed
  node1                           missing                         failed
Result: Package existence check failed for "compat-gcc-c++-7.3-2.96.128".


Check: Package existence for "compat-libstdc++-7.3-2.96.128"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  node2                           missing                         failed
  node1                           missing                         failed
Result: Package existence check failed for "compat-libstdc++-7.3-2.96.128".


Check: Package existence for "compat-libstdc++-devel-7.3-2.96.128"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  node2                           missing                         failed
  node1                           missing                         failed
Result: Package existence check failed for "compat-libstdc++-devel-7.3-2.96.128".


Check: Package existence for "openmotif-2.2.3"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  node2                           missing                         failed
  node1                           missing                         failed
Result: Package existence check failed for "openmotif-2.2.3".


Check: Package existence for "setarch-1.3-1"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  node2                           setarch-2.0-1.1                 passed
  node1                           setarch-2.0-1.1                 passed
Result: Package existence check passed for "setarch-1.3-1".


Check: Group existence for "dba"
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  node2         exists                    passed
  node1         exists                    passed
Result: Group existence check passed for "dba".


Check: Group existence for "oinstall"
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  node2         exists                    passed
  node1         exists                    passed
Result: Group existence check passed for "oinstall".


Check: User existence for "nobody"
  Node Name     Status                    Comment
  ------------  ------------------------  ------------------------
  node2         exists                    passed
  node1         exists                    passed
Result: User existence check passed for "nobody".


System requirement failed for 'crs'


Pre-check for cluster services setup was unsuccessful on all the nodes.
安装compat-libstdc++-7.3-2.96.128.i386.rpm时提示与其他包冲突，使用--force强制安装
1、安装compat-libstdc++-7.3-2.96.128.i386.rpm
rpm -ivh compat-libstdc++-7.3-2.96.128.i386.rpm --force
2、安装compat-libstdc++-devel-7.3-2.96.128.i386.rpm
3、安装compat-gcc-c++-7.3-2.96.128.i386.rpm
oracle用户安装clusterware
./runInstaller -ignoreSysPreReqs


重新安装crs之前要彻底删掉之前安装的crs
分别在两个节点上执行
$ORA_CRS_HOME/bin/rootdelete.sh
$ORA_CRS_HOME/bin/rootdeinstall.sh
并清空以下两个目录
$ORA_CRS_HOME
/opt/oracle/oraInventory


安装数据库软件
./runInstaller -ignoreSysPrereqs


安装数据库


安装后crs在node2上的inst无法启动
CRS-0215: Could not start resource 'ora.lndw.lndw2.inst'.
#vi  racgvip  找到 FAIL_WHEN_DEFAULTGW_NOT_FOUND = 1  ，把1改为0