（连载）一个关于ORACLE数据库的虚拟工程（五）--配置一台oracle rac的主机

最新推荐文章于 2023-04-12 22:24:12 发布

蔡菜子

最新推荐文章于 2023-04-12 22:24:12 发布

阅读量730

点赞数

分类专栏： oracle linux

本文链接：https://blog.csdn.net/cd4836/article/details/97788185

版权

oracle 同时被 2 个专栏收录

21 篇文章 0 订阅

订阅专栏

linux

11 篇文章 0 订阅

订阅专栏

先克隆一台centos主机

你还记得我们第一章安装了的那台虚拟化centos主机吗？这一章讲详细介绍如何配置成可以安装oracle rac的主机。

选择虚拟机-管理-克隆；

创建链接克隆，这样占用的空间少，速度也非常快，你只要保证你的主机不开，不坏就行了；

你可以命名为rac1，也可以其他，但是位置必须与openfiler的虚拟机同一个目录，完成后就可以开启rac1了；

开机器后，你会看到原来的网卡eth0,eth1,eth2,eth3变了，这是因为Linux 使用udev动态管理设备文件，并根据设备的信息对其进行持久化命名。udev会在系统引导的过程中识别网卡，将mac地址和网卡名称对应起来记录在udev的规则脚本中。而对于新的虚拟机，VMware会自动为虚拟机的网卡生成MAC地址，当你克隆或者重装虚拟机软件时，由于你使用的是以前系统虚拟硬盘的信息；

[root@localhost ~]# ifconfig
eth4      Link encap:Ethernet  HWaddr 00:0C:29:7C:71:88  
          inet addr:192.168.2.131  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe7c:7188/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:13 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3044 (2.9 KiB)  TX bytes:2308 (2.2 KiB)

eth5      Link encap:Ethernet  HWaddr 00:0C:29:7C:71:92  
          inet addr:192.168.2.128  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe7c:7192/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:14 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3386 (3.3 KiB)  TX bytes:1152 (1.1 KiB)

eth6      Link encap:Ethernet  HWaddr 00:0C:29:7C:71:9C  
          inet addr:192.168.2.129  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe7c:719c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:14 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3386 (3.3 KiB)  TX bytes:1152 (1.1 KiB)

eth7      Link encap:Ethernet  HWaddr 00:0C:29:7C:71:7E  
          inet addr:192.168.5.128  Bcast:192.168.5.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe7c:717e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:19 errors:0 dropped:0 overruns:0 frame:0
          TX packets:24 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2384 (2.3 KiB)  TX bytes:2358 (2.3 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:16 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:960 (960.0 b)  TX bytes:960 (960.0 b)

重启配置网卡

你首先要把/etc/udev/rules.d/70-persistent-net.rules这个文件删除，再把/etc/sysconfig/network-scripts目录下的ifctg-eth0、ifctg-eth1、ifctg-eth2、ifctg-eth3文件中的HWADDR行注释掉；

[root@localhost ~]# cd /etc/udev/rules.d/
[root@localhost rules.d]# rm -rf 70-persistent-net.rules
[root@localhost rules.d]# cd /etc/sysconfig/network-scripts
[root@localhost network-scripts]# ls
ifcfg-eth0  ifcfg-lo     ifdown-ippp  ifdown-ppp     ifup          ifup-ippp  ifup-plusb   ifup-sit          net.hotplug
ifcfg-eth1  ifdown       ifdown-ipv6  ifdown-routes  ifup-aliases  ifup-ipv6  ifup-post    ifup-tunnel       network-functions
ifcfg-eth2  ifdown-bnep  ifdown-isdn  ifdown-sit     ifup-bnep     ifup-isdn  ifup-ppp     ifup-wireless     network-functions-ipv6
ifcfg-eth3  ifdown-eth   ifdown-post  ifdown-tunnel  ifup-eth      ifup-plip  ifup-routes  init.ipv6-global
[root@localhost network-scripts]# vi ifcfg-eth0
DEVICE=eth0
#HWADDR=00:0C:29:E0:4E:5E   //注释掉
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.5.111
NETMASK=255.255.255.0
GATEWAY=192.168.5.2
DNS1=114.114.114.114
DNS2=8.8.8.8
TYPE=Ethernet
UUID=75cdaea2-ac8b-465b-b1fe-7280e02b2ab3
NM_CONTROLLED=yes

重启系统之后，我们再把ifctg-eth1、ifctg-eth2、ifctg-eth3文件按照第二章的规划配置IP地址，但是首先，你得先查看你现在每块网卡的MAC地址，比如我这里的网络适配器2（选择高级），MAC地址为00:0C:29:7C:71:88，是能够与ifconfig出来的地址对应得上；

[root@localhost network-scripts]# cat ifcfg-eth0
DEVICE=eth0
#HWADDR=00:0C:29:E0:4E:5E
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.5.111
NETMASK=255.255.255.0
GATEWAY=192.168.5.2
DNS1=114.114.114.114
DNS2=8.8.8.8
TYPE=Ethernet
UUID=75cdaea2-ac8b-465b-b1fe-7280e02b2ab3
NM_CONTROLLED=yes

[root@localhost network-scripts]# vi ifcfg-eth1
DEVICE=eth1
#HWADDR=00:0C:29:E0:4E:68
TYPE=Ethernet
UUID=0cb9bea8-80e4-4905-84fa-674d479fe22b
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.2.1
NETMASK=255.255.255.0

[root@localhost network-scripts]# vi ifcfg-eth2
DEVICE=eth2
#HWADDR=00:0C:29:E0:4E:72
TYPE=Ethernet
UUID=aa14940c-3cc9-4e50-ac14-989b9269eea6
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.3.11
NETMASK=255.255.255.0

[root@localhost network-scripts]# vi ifcfg-eth3
DEVICE=eth3
#HWADDR=00:0C:29:E0:4E:7C
TYPE=Ethernet
UUID=9e0dda42-5b98-4641-af31-c1a7ffe354a8
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.3.33
NETMASK=255.255.255.0

[root@localhost network-scripts]#chkconfig NetworkManager off //NetworkManager管理工具永久关闭服务，需要重启
[root@localhost network-scripts]#service network restart //才能使配置的IP生效
[root@localhost network-scripts]#vi /etc/sysconfig/network   //顺便将hostname改为:rac1
[root@localhost network-scripts]#reboot //如果网络不生效，就重启

测试地址连通性

到了这一步，你需要测试验证一下这台rac1与存储openfiler的网络通不通了；

[root@rac1 ~]# ping 192.168.5.115  //存储地址1
PING 192.168.5.115 (192.168.5.115) 56(84) bytes of data.
64 bytes from 192.168.5.115: icmp_seq=1 ttl=64 time=0.170 ms
64 bytes from 192.168.5.115: icmp_seq=2 ttl=64 time=0.201 ms
^C
--- 192.168.5.115 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1472ms
rtt min/avg/max/mdev = 0.170/0.185/0.201/0.020 ms
[root@rac1 ~]# ping 192.168.3.55  //存储地址2
PING 192.168.3.55 (192.168.3.55) 56(84) bytes of data.
64 bytes from 192.168.3.55: icmp_seq=1 ttl=64 time=1.26 ms
64 bytes from 192.168.3.55: icmp_seq=2 ttl=64 time=0.276 ms
^C
--- 192.168.3.55 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1295ms
rtt min/avg/max/mdev = 0.276/0.770/1.265/0.495 ms
[root@rac1 ~]# ping 192.168.3.66  //存储地址3
PING 192.168.3.66 (192.168.3.66) 56(84) bytes of data.
64 bytes from 192.168.3.66: icmp_seq=1 ttl=64 time=1.77 ms
64 bytes from 192.168.3.66: icmp_seq=2 ttl=64 time=0.361 ms
^C
--- 192.168.3.66 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1480ms
rtt min/avg/max/mdev = 0.361/1.066/1.771/0.705 ms
[root@rac1 ~]# ping www.qq.com  //互联网
PING public.sparta.mig.tencent-cloud.net (14.18.175.154) 56(84) bytes of data.
64 bytes from 14.18.175.154: icmp_seq=1 ttl=128 time=7.11 ms
64 bytes from 14.18.175.154: icmp_seq=2 ttl=128 time=7.15 ms
^C
--- public.sparta.mig.tencent-cloud.net ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1186ms
rtt min/avg/max/mdev = 7.113/7.133/7.154/0.086 ms

rac1安装oracle rac需要的设置

我们这里先配置一台rac1主机，后面再克隆出rac2主机。

配置本地yum

把安装光盘放入光驱，把默认yum源备份；

#mkdir /opt/centos-yum.bak
#mv /etc/yum.repos.d/* /opt/centos-yum.bak/
#mkdir /media/centos
#mount /dev/cdrom /media/centos
mount: block device /dev/sr0 is write-protected, mounting read-only //说明成功了
 
#vi /etc/yum.repos.d/local.repo   //修改本地源
[local]
name=local
baseurl=file:///media/centos
enabled=1
gpgcheck=0
 
#yum clean all
Loaded plugins: fastestmirror
Cleaning repos: local
Cleaning up everything
Cleaning up list of fastest mirrors
#yum makecache        //把yum源缓存到本地，加快软件的搜索好安装速度（可选）
[root@kangvcar ~]#yum list        //列出了本地源的包

关闭防火墙

#setenforce 0  //临时关闭，不用重启机器
setenforce: SELinux is disabled
#vi /etc/sysconfig/selinux //修改下面的语句
SELINUX=disabled
#getenforce //查看命令
Disabled
#service iptables stop
#chkconfig iptables off

修改hosts文件

#vi /etc/hosts  //host文件进行如下配置
 
192.168.5.111 rac1 
192.168.5.113 rac1-vip 
192.168.2.1 rac1-priv
192.168.3.11 rac1-disk1
192.168.3.33 rac1-disk2
 
192.168.5.112 rac2 
192.168.5.114 rac2-vip 
192.168.2.2 rac2-priv
192.168.3.22 rac2-disk1
192.168.3.44 rac2-disk2
 
192.168.3.55 share1
192.168.3.66 share2

192.168.5.222 scan-ip

安装包检查

根据Oracle软件安装需要的软件包（官方文档）以下摘取一部分，可以看到Oracle Linux 6 和 Red Hat Enterprise Linux 6它们对于安装Oracle rac 所需要的安装包是一样的，我们使用centos和redhat是使用同样的内核，所以理论上也是一样的；

Oracle Linux 6 and Red Hat Enterprise Linux 6

The following packages (or later versions) must be installed:

binutils-2.20.51.0.2-5.11.el6 (x86_64)

compat-libcap1-1.10-1 (x86_64)

compat-libstdc++-33-3.2.3-69.el6 (x86_64)

compat-libstdc++-33-3.2.3-69.el6.i686

gcc-4.4.4-13.el6 (x86_64)

gcc-c++-4.4.4-13.el6 (x86_64)

glibc-2.12-1.7.el6 (i686)

glibc-2.12-1.7.el6 (x86_64)

glibc-devel-2.12-1.7.el6 (x86_64)

glibc-devel-2.12-1.7.el6.i686

ksh

libgcc-4.4.4-13.el6 (i686)

libgcc-4.4.4-13.el6 (x86_64)

libstdc++-4.4.4-13.el6 (x86_64)

libstdc++-4.4.4-13.el6.i686

libstdc++-devel-4.4.4-13.el6 (x86_64)

libstdc++-devel-4.4.4-13.el6.i686

libaio-0.3.107-10.el6 (x86_64)

libaio-0.3.107-10.el6.i686

libaio-devel-0.3.107-10.el6 (x86_64)

libaio-devel-0.3.107-10.el6.i686

make-3.81-19.el6

sysstat-9.0.4-11.el6 (x86_64)

运行以下命令安装；

#yum clean all 
#yum makecache 
#yum -y install binutils-* compat-lib* gcc* glibc-2* glibc-devel* ksh libgcc-* libstdc* libaio-* make-* sysstat-*
//可能需要重新mount光盘

Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
Setting up Install Process
Package binutils-devel-2.20.51.0.2-5.36.el6.x86_64 already installed and latest version
Package binutils-2.20.51.0.2-5.36.el6.x86_64 already installed and latest version
Package compat-libtermcap-2.0.8-49.el6.x86_64 already installed and latest version
Package compat-libgfortran-41-4.1.2-39.el6.x86_64 already installed and latest version
Package compat-libf2c-34-3.4.6-19.el6.x86_64 already installed and latest version
Package compat-libstdc++-296-2.96-144.el6.i686 already installed and latest version
Package compat-libcap1-1.10-1.x86_64 already installed and latest version
Package compat-libstdc++-33-3.2.3-69.el6.x86_64 already installed and latest version
Package gcc-4.4.7-4.el6.x86_64 already installed and latest version
Package gcc-gfortran-4.4.7-4.el6.x86_64 already installed and latest version
Package gcc-c++-4.4.7-4.el6.x86_64 already installed and latest version
Package glibc-2.12-1.132.el6.x86_64 already installed and latest version
Package glibc-devel-2.12-1.132.el6.x86_64 already installed and latest version
Package libgcc-4.4.7-4.el6.x86_64 already installed and latest version
Package libstdc++-4.4.7-4.el6.x86_64 already installed and latest version
Package libstdc++-devel-4.4.7-4.el6.x86_64 already installed and latest version
Package libaio-0.3.107-10.el6.x86_64 already installed and latest version
Package 1:make-3.81-20.el6.x86_64 already installed and latest version
Package sysstat-9.0.4-22.el6.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package gcc-gnat.x86_64 0:4.4.7-4.el6 will be installed
--> Processing Dependency: libgnat-devel = 4.4.7-4.el6 for package: gcc-gnat-4.4.7-4.el6.x86_64
--> Processing Dependency: libgnat = 4.4.7-4.el6 for package: gcc-gnat-4.4.7-4.el6.x86_64
---> Package gcc-java.x86_64 0:4.4.7-4.el6 will be installed
--> Processing Dependency: libgcj-devel = 4.4.7-4.el6 for package: gcc-java-4.4.7-4.el6.x86_64
---> Package gcc-objc.x86_64 0:4.4.7-4.el6 will be installed
--> Processing Dependency: libobjc = 4.4.7-4.el6 for package: gcc-objc-4.4.7-4.el6.x86_64
--> Processing Dependency: libobjc.so.2()(64bit) for package: gcc-objc-4.4.7-4.el6.x86_64
---> Package gcc-objc++.x86_64 0:4.4.7-4.el6 will be installed
---> Package ksh.x86_64 0:20120801-10.el6 will be installed
---> Package libaio-devel.x86_64 0:0.3.107-10.el6 will be installed
---> Package libstdc++-docs.x86_64 0:4.4.7-4.el6 will be installed
--> Running transaction check
---> Package libgcj-devel.x86_64 0:4.4.7-4.el6 will be installed
---> Package libgnat.x86_64 0:4.4.7-4.el6 will be installed
---> Package libgnat-devel.x86_64 0:4.4.7-4.el6 will be installed
---> Package libobjc.x86_64 0:4.4.7-4.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================================================================
 Package                                Arch                           Version                                  Repository                     Size
====================================================================================================================================================
Installing:
 gcc-gnat                               x86_64                         4.4.7-4.el6                              local                         8.7 M
 gcc-java                               x86_64                         4.4.7-4.el6                              local                         3.7 M
 gcc-objc                               x86_64                         4.4.7-4.el6                              local                         3.6 M
 gcc-objc++                             x86_64                         4.4.7-4.el6                              local                         4.0 M
 ksh                                    x86_64                         20120801-10.el6                          local                         756 k
 libaio-devel                           x86_64                         0.3.107-10.el6                           local                          13 k
 libstdc++-docs                         x86_64                         4.4.7-4.el6                              local                          14 M
Installing for dependencies:
 libgcj-devel                           x86_64                         4.4.7-4.el6                              local                         1.6 M
 libgnat                                x86_64                         4.4.7-4.el6                              local                         908 k
 libgnat-devel                          x86_64                         4.4.7-4.el6                              local                         3.3 M
 libobjc                                x86_64                         4.4.7-4.el6                              local                          90 k

Transaction Summary
====================================================================================================================================================
Install      11 Package(s)

Total download size: 40 M
Installed size: 137 M
Downloading Packages:
----------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                88 MB/s |  40 MB     00:00     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : libgnat-4.4.7-4.el6.x86_64                                                                                                      1/11 
  Installing : libgnat-devel-4.4.7-4.el6.x86_64                                                                                                2/11 
  Installing : libobjc-4.4.7-4.el6.x86_64                                                                                                      3/11 
  Installing : gcc-objc-4.4.7-4.el6.x86_64                                                                                                     4/11 
  Installing : libgcj-devel-4.4.7-4.el6.x86_64                                                                                                 5/11 
  Installing : gcc-java-4.4.7-4.el6.x86_64                                                                                                     6/11 
  Installing : gcc-objc++-4.4.7-4.el6.x86_64                                                                                                   7/11 
  Installing : gcc-gnat-4.4.7-4.el6.x86_64                                                                                                     8/11 
  Installing : ksh-20120801-10.el6.x86_64                                                                                                      9/11 
  Installing : libaio-devel-0.3.107-10.el6.x86_64                                                                                             10/11 
  Installing : libstdc++-docs-4.4.7-4.el6.x86_64                                                                                              11/11 
  Verifying  : libgnat-devel-4.4.7-4.el6.x86_64                                                                                                1/11 
  Verifying  : libstdc++-docs-4.4.7-4.el6.x86_64                                                                                               2/11 
  Verifying  : libgcj-devel-4.4.7-4.el6.x86_64                                                                                                 3/11 
  Verifying  : gcc-gnat-4.4.7-4.el6.x86_64                                                                                                     4/11 
  Verifying  : gcc-objc-4.4.7-4.el6.x86_64                                                                                                     5/11 
  Verifying  : libaio-devel-0.3.107-10.el6.x86_64                                                                                              6/11 
  Verifying  : gcc-java-4.4.7-4.el6.x86_64                                                                                                     7/11 
  Verifying  : libobjc-4.4.7-4.el6.x86_64                                                                                                      8/11 
  Verifying  : libgnat-4.4.7-4.el6.x86_64                                                                                                      9/11 
  Verifying  : gcc-objc++-4.4.7-4.el6.x86_64                                                                                                  10/11 
  Verifying  : ksh-20120801-10.el6.x86_64                                                                                                     11/11 

Installed:
  gcc-gnat.x86_64 0:4.4.7-4.el6    gcc-java.x86_64 0:4.4.7-4.el6           gcc-objc.x86_64 0:4.4.7-4.el6          gcc-objc++.x86_64 0:4.4.7-4.el6   
  ksh.x86_64 0:20120801-10.el6     libaio-devel.x86_64 0:0.3.107-10.el6    libstdc++-docs.x86_64 0:4.4.7-4.el6   

Dependency Installed:
  libgcj-devel.x86_64 0:4.4.7-4.el6      libgnat.x86_64 0:4.4.7-4.el6      libgnat-devel.x86_64 0:4.4.7-4.el6      libobjc.x86_64 0:4.4.7-4.el6     

Complete!

rlwrap包安装

在Linux下使用SQL*PLUS无法像bush一样上下翻页，也不能退格，只要安装rlwrap即可实现这些功能。

我们把下载的rlwrap-0.41.tar.gz安装包拉入rac1的tmp目录下；

//安装前准备,使用yum安装，下面两个包有些已经安装好了，可以不用理会。
#yum install readline*
#yum install readline-devel*
//下载安装包后安装，安装包需另外从网上下载
#cd /tmp
#tar -zxvf rlwrap-0.41.tar.gz
#cd rlwrap-0.41
#./configure
#make
#make install

安装iSCSI initiator

# rpm -qa|grep iscsi
# cd /media/centos/Packages/ //进入光盘安装包的目录
# ls | grep iscsi //查找关于iscsi的包，有了就安装它
iscsi-initiator-utils-6.2.0.873-10.el6.x86_64.rpm
# rpm -ivh  iscsi-initiator-utils-6.2.0.873-10.el6.x86_64.rpm
# rpm -qa | grep iscsi //查看是否已经安装
iscsi-initiator-utils-6.2.0.873-10.el6.x86_64

iscsi initiator主要通过iscsiadm命令管理，我们先查看提供服务的iscsi target机器上有哪些target，可以看到存储的iSCSI target的编号和名称，由于在虚拟机上我们用于管理，192.168.5.0这个网段是可以连通的，所以3个IP地址都可以看到；

//配置iSCSI（启动器）服务
# service iscsid start
# chkconfig iscsid on
# chkconfig iscsi on
//通过下面的命令，可以发现从openfiler的3个网卡
#iscsiadm --mode discovery --type sendtargets --portal 192.168.3.55
[  OK  ] iscsid: [  OK  ]
192.168.3.55:3260,1 iqn.2006-01.com.openfiler:racdb.fra
192.168.5.115:3260,1 iqn.2006-01.com.openfiler:racdb.fra
192.168.3.66:3260,1 iqn.2006-01.com.openfiler:racdb.fra
192.168.3.55:3260,1 iqn.2006-01.com.openfiler:racdb.data
192.168.5.115:3260,1 iqn.2006-01.com.openfiler:racdb.data
192.168.3.66:3260,1 iqn.2006-01.com.openfiler:racdb.data
192.168.3.55:3260,1 iqn.2006-01.com.openfiler:racdb.orc03
192.168.5.115:3260,1 iqn.2006-01.com.openfiler:racdb.orc03
192.168.3.66:3260,1 iqn.2006-01.com.openfiler:racdb.orc03
192.168.3.55:3260,1 iqn.2006-01.com.openfiler:racdb.orc02
192.168.5.115:3260,1 iqn.2006-01.com.openfiler:racdb.orc02
192.168.3.66:3260,1 iqn.2006-01.com.openfiler:racdb.orc02
192.168.3.55:3260,1 iqn.2006-01.com.openfiler:racdb.ocr01
192.168.5.115:3260,1 iqn.2006-01.com.openfiler:racdb.ocr01
192.168.3.66:3260,1 iqn.2006-01.com.openfiler:racdb.ocr01
//下面用192.168.3.55这个路径登录，可以看到我们是3路径，现实中我们会屏蔽192.168.5.115这个外网
#iscsiadm --mode node --targetname iqn.2006-01.com.openfiler:racdb.ocr01 –portal 192.168.3.55:3260 --login
Logging in to [iface: default, target: iqn.2006-01.com.openfiler:racdb.ocr01, portal: 192.168.3.66,3260] (multiple)
Logging in to [iface: default, target: iqn.2006-01.com.openfiler:racdb.ocr01, portal: 192.168.3.55,3260] (multiple)
Logging in to [iface: default, target: iqn.2006-01.com.openfiler:racdb.ocr01, portal: 192.168.5.115,3260] (multiple)
Login to [iface: default, target: iqn.2006-01.com.openfiler:racdb.ocr01, portal: 192.168.3.66,3260] successful.
Login to [iface: default, target: iqn.2006-01.com.openfiler:racdb.ocr01, portal: 192.168.3.55,3260] successful.
Login to [iface: default, target: iqn.2006-01.com.openfiler:racdb.ocr01, portal: 192.168.5.115,3260] successful.
//我们把另外4个再登录一遍，此处把本来应该是ocr02写成了orc02，但不影响
#iscsiadm --mode node --targetname iqn.2006-01.com.openfiler:racdb.orc02 –portal 192.168.3.55:3260 --login
#iscsiadm --mode node --targetname iqn.2006-01.com.openfiler:racdb.orc03 –portal 192.168.3.55:3260 --login
#iscsiadm --mode node --targetname iqn.2006-01.com.openfiler:racdb.data –portal 192.168.3.55:3260 --login
#iscsiadm --mode node --targetname iqn.2006-01.com.openfiler:racdb.fra –portal 192.168.3.55:3260 --login
//可以看到，每块盘都由3条路径发布出来，本机认到的就是3块盘

//下面命令可以对应关系
[root@rac1 nodes]# (cd /dev/disk/by-path; ls -l *openfiler* | awk '{FS=" "; 
print $9 " " $10 " " $11}')
ip-192.168.3.55:3260-iscsi-iqn.2006-01.com.openfiler:racdb.data-lun-0 -> ../../sdl
ip-192.168.3.55:3260-iscsi-iqn.2006-01.com.openfiler:racdb.fra-lun-0 -> ../../sdo
ip-192.168.3.55:3260-iscsi-iqn.2006-01.com.openfiler:racdb.ocr01-lun-0 -> ../../sdc
ip-192.168.3.55:3260-iscsi-iqn.2006-01.com.openfiler:racdb.orc02-lun-0 -> ../../sdf
ip-192.168.3.55:3260-iscsi-iqn.2006-01.com.openfiler:racdb.orc03-lun-0 -> ../../sdi
ip-192.168.3.66:3260-iscsi-iqn.2006-01.com.openfiler:racdb.data-lun-0 -> ../../sdm
ip-192.168.3.66:3260-iscsi-iqn.2006-01.com.openfiler:racdb.fra-lun-0 -> ../../sdp
ip-192.168.3.66:3260-iscsi-iqn.2006-01.com.openfiler:racdb.ocr01-lun-0 -> ../../sdd
ip-192.168.3.66:3260-iscsi-iqn.2006-01.com.openfiler:racdb.orc02-lun-0 -> ../../sdg
ip-192.168.3.66:3260-iscsi-iqn.2006-01.com.openfiler:racdb.orc03-lun-0 -> ../../sdj
ip-192.168.5.115:3260-iscsi-iqn.2006-01.com.openfiler:racdb.data-lun-0 -> ../../sdk
ip-192.168.5.115:3260-iscsi-iqn.2006-01.com.openfiler:racdb.fra-lun-0 -> ../../sdn
ip-192.168.5.115:3260-iscsi-iqn.2006-01.com.openfiler:racdb.ocr01-lun-0 -> ../../sdb
ip-192.168.5.115:3260-iscsi-iqn.2006-01.com.openfiler:racdb.orc02-lun-0 -> ../../sde
ip-192.168.5.115:3260-iscsi-iqn.2006-01.com.openfiler:racdb.orc03-lun-0 -> ../../sdh

小知识点-删除iscsi链接信息，可用于重新连接。

//首先要删除原有的配置信息
[root@rac1 ~]# cd /var/lib/iscsi/send_targets/   //进入目录
[root@rac1 ~]# ls
192.168.3.55,3260  192.168.3.66,3260
[root@rac1 ~]#rm -rf 192.168*   //删除各个target的情况
[root@rac1 send_targets]# cd /var/lib/iscsi/nodes/ 
[root@rac1 nodes]# ls
iqn.2006-01.com.openfiler:racdb.data  iqn.2006-01.com.openfiler:racdb.ocr01  iqn.2006-01.com.openfiler:racdb.orc03
iqn.2006-01.com.openfiler:racdb.fra   iqn.2006-01.com.openfiler:racdb.orc02
[root@rac1 ~]#rm -rf iqn*

配置多路径multipath

检查centos自带的多路径rpm包是否都已安装；

[root@rac1 ~]# rpm -qa | grep device-mapper
device-mapper-1.02.79-8.el6.x86_64
device-mapper-event-libs-1.02.79-8.el6.x86_64
device-mapper-persistent-data-0.2.8-2.el6.x86_64
device-mapper-libs-1.02.79-8.el6.x86_64
device-mapper-event-1.02.79-8.el6.x86_64

使用yum安装，启动multipath；

[root@rac1 ~]# mount /dev/cdrom /media/centos
mount: block device /dev/sr0 is write-protected, mounting read-only
[root@rac1 ~]# yum install -y device-mapper-multipath
[root@rac1 ~]# rpm -qa | grep device-mapper
device-mapper-multipath-0.4.9-72.el6.x86_64
device-mapper-1.02.79-8.el6.x86_64
device-mapper-event-libs-1.02.79-8.el6.x86_64
device-mapper-multipath-libs-0.4.9-72.el6.x86_64
device-mapper-persistent-data-0.2.8-2.el6.x86_64
device-mapper-libs-1.02.79-8.el6.x86_64
device-mapper-event-1.02.79-8.el6.x86_64
//将多路径软件添加至内核中
[root@rac1 /]# modprobe dm-multipath  
[root@rac1 /]# modprobe dm-round-robin
//检查内核添加情况
[root@rac1 /]# lsmod |grep multipath
dm_multipath           17724  1 dm_round_robin
dm_mod                 84209  9 dm_multipath,dm_mirror,dm_log
//设置为开机自启动
[root@rac1 /]# chkconfig  --level 2345 multipathd on
[root@rac1 /]# chkconfig  --list|grep multipathd
multipathd      0:off   1:off   2:on    3:on    4:on    5:on    6:off
//启动multipath服务
[root@rac1 /]# service multipathd restart
ux_socket_connect: No such file or directory
Stopping multipathd daemon: [  OK  ]
Starting multipathd daemon: [  OK  ]

[root@rac1 /]# multipath -ll
Aug 01 12:32:08 | /etc/multipath.conf does not exist, blacklisting all devices.
Aug 01 12:32:08 | A sample multipath.conf file is located at
Aug 01 12:32:08 | /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf
Aug 01 12:32:08 | You can run /sbin/mpathconf to create or modify /etc/multipath.conf
//默认情况下，/etc/multipath.conf是不存在的，需要用如下命令生成
[root@rac1 /]#  /sbin/mpathconf --enable --find_multipaths y --with_module y --with_chkconfig y
[root@rac1 /]# ll /etc/multipath.conf
-rw------- 1 root root 2775 Aug  1 12:32 /etc/multipath.conf
//查看并获取存储分配的逻辑盘lun的wwid信息
[root@rac1 /]# multipath -v0
[root@rac1 /]# more /etc/multipath/wwids
# Multipath wwids, Version : 1.0
# NOTE: This file is automatically maintained by multipath and multipathd.
# You should not need to edit this file in normal circumstances.
#
# Valid WWIDs:
/14f504e46494c45527134303534722d4e7163472d55355946/
/14f504e46494c45527965523457652d7965304c2d726c7847/
/14f504e46494c4552314d44544b702d4d5a62742d6e724547/
/14f504e46494c45526858596968622d596f4b792d63636464/
/14f504e46494c455250426c5953782d6f586f742d534d514f/
[root@rac1 /]# multipath -ll
mpathe (14f504e46494c455250426c5953782d6f586f742d534d514f) dm-6 OPNFILER,VIRTUAL-DISK
size=20G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 4:0:0:4 sdj 8:144 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 3:0:0:4 sdk 8:160 active ready running
mpathd (14f504e46494c45526858596968622d596f4b792d63636464) dm-5 OPNFILER,VIRTUAL-DISK
size=20G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 4:0:0:3 sdh 8:112 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 3:0:0:3 sdi 8:128 active ready running
mpathc (14f504e46494c4552314d44544b702d4d5a62742d6e724547) dm-4 OPNFILER,VIRTUAL-DISK
size=2.0G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 4:0:0:2 sdf 8:80  active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 3:0:0:2 sdg 8:96  active ready running
mpathb (14f504e46494c45527965523457652d7965304c2d726c7847) dm-3 OPNFILER,VIRTUAL-DISK
size=2.0G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 4:0:0:1 sdd 8:48  active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 3:0:0:1 sde 8:64  active ready running
mpatha (14f504e46494c45527134303534722d4e7163472d55355946) dm-2 OPNFILER,VIRTUAL-DISK
size=2.0G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 4:0:0:0 sdb 8:16  active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 3:0:0:0 sdc 8:32  active ready running
[root@rac1 ~]# fdisk -l | grep dev
Disk /dev/sda: 53.7 GB, 53687091200 bytes
/dev/sda1   *           1          64      512000   83  Linux
/dev/sda2              64        6528    51915776   8e  Linux LVM
Disk /dev/mapper/VolGroup-lv_root: 49.0 GB, 48964304896 bytes
Disk /dev/mapper/VolGroup-lv_swap: 4194 MB, 4194304000 bytes
Disk /dev/sdb: 2147 MB, 2147483648 bytes
Disk /dev/sdc: 2147 MB, 2147483648 bytes
Disk /dev/sdd: 2147 MB, 2147483648 bytes
Disk /dev/mapper/mpatha: 2147 MB, 2147483648 bytes
Disk /dev/sde: 2147 MB, 2147483648 bytes
Disk /dev/sdf: 2147 MB, 2147483648 bytes
Disk /dev/sdg: 2147 MB, 2147483648 bytes
Disk /dev/mapper/mpathb: 2147 MB, 2147483648 bytes
Disk /dev/sdh: 2147 MB, 2147483648 bytes
Disk /dev/sdi: 2147 MB, 2147483648 bytes
Disk /dev/sdj: 2147 MB, 2147483648 bytes
Disk /dev/mapper/mpathc: 2147 MB, 2147483648 bytes
Disk /dev/sdk: 21.5 GB, 21474836480 bytes
Disk /dev/sdl: 21.5 GB, 21474836480 bytes
Disk /dev/sdm: 21.5 GB, 21474836480 bytes
Disk /dev/mapper/mpathd: 21.5 GB, 21474836480 bytes
Disk /dev/sdn: 21.5 GB, 21474836480 bytes
Disk /dev/sdo: 21.5 GB, 21474836480 bytes
Disk /dev/sdp: 21.5 GB, 21474836480 bytes
Disk /dev/mapper/mpathe: 21.5 GB, 21474836480 bytes

到了这里，我建议你按这一章节的先克隆一台centos主机和重启配置网卡的步骤，克隆一台rac2主机，然后修改hostname和IP地址，注意将rac2的文件夹保存在与rac1文件夹相同的位置。