oracle 11gr2 RAC 2nodes on RHEL5.4 32bit 个人实验

最新推荐文章于 2022-04-07 17:39:03 发布

cnk45946

最新推荐文章于 2022-04-07 17:39:03 发布

阅读量230

点赞数

一环境

宿主OS:

# uname -a
Linux SL410 2.6.37-gentoo-r2 #2 SMP PREEMPT Tue Mar 22 09:27:37 CST 2011 i686 Intel(R) Core(TM)2 Duo CPU T5870 @ 2.00GHz GenuineIntel GNU/Linux

vmware版本:

# vmware -v
VMware Workstation 7.1.3 build-324285

二结构

整体结构:

使用vmware 7.1.3 workstation 虚拟出两台装有RHEL5.4 32bit操作系统的主机rac1,rac2做RAC的两个结点.两结点间的共享存储由iscsi解决(宿主机dd出4个5G的文件,然后由tgtd服务发布出去)

宿主机目录结构:

类型	目录
虚拟OS1	/media/124e0965-0e90-4c07-ada8-ec41ed595f55/vmware/RHEL5.4_11gr2_RAC1/
虚拟OS2	/mnt/sdb1/vmware/RHEL5.4_11gr2_RAC2/
共享存储	/mnt/sdb1/vmware/share_disk

虚拟OS中oracle软件目录结构:

/u01/app/11.2.0/grid
/u01/app/oracle

虚拟OS网卡结构(SCAN):

Identity	Home Node	Host Node	Given Name	Type	Address	Address Assigned By	Resolved By
rac1 Public	rac1	rac1	rac1	Public	192.168.2.11	Fixed	DNS
rac1 VIP	rac1	Selected by Oracle Clusterware	rac1-vip	Virtual	192.168.2.21	Fixed	DNS and hosts file
rac1 Private 1	rac1	rac1	rac1-priv1	Private	10.0.0.11	Fixed	DNS and hosts file, or none
rac1 Private 2	rac1	rac1	rac1-priv2	Private	10.0.0.21	Fixed	DNS and hosts file, or none
rac2 Public	rac2	rac2	rac2	Public	192.168.2.12	Fixed	DNS
rac2 VIP	rac2	Selected by Oracle Clusterware	rac2-vip	Virtual	192.168.2.22	Fixed	DNS and hosts file
rac2 Private 1	rac2	rac2	rac2-priv1	Private	10.0.0.12	Fixed	DNS and hosts file, or none
rac2 Private 2	rac2	rac2	rac2-priv2	Private	10.0.0.22	Fixed	DNS and hosts file, or none
SCAN VIP 1	none	Selected by Oracle Clusterware	rac-scan	Virtual	192.168.2.201	Fixed	DNS
SCAN VIP 2	none	Selected by Oracle Clusterware	rac-scan	Virtual	192.168.2.202	Fixed	DNS
SCAN VIP 3	none	Selected by Oracle Clusterware	rac-scan	Virtual	192.168.2.203	Fixed	DNS

表格参考: http://download.oracle.com/docs/cd/E11882_01/install.112/e17214/chklist.htm#BHAGAAGH

虚拟OS存储结构:

设备名	映射关系	ASM磁盘组分配
/dev/sdb1	/dev/raw/raw1	DATA
/dev/sdc1	/dev/raw/raw2	DATA
/dev/sdd1	/dev/raw/raw3	DATA2
/dev/sde1	/dev/raw/raw4	DATA2

虚拟OS用户结构:

用户组结构为个人自行发挥.官方文档的部分还没细看.这里做参考.

用户	组
grid	asmadmin,OSDBA,OSOPER,OSASM
oracle	dba,oinstall,OSDBA,OSOPER,OSASM

三注意事项

1. node 系统的 swap 一定要大于500M,否则在装cluster需要运行root.sh脚本时会报swap空间不足的错误.
2.共享磁盘最好分的大一些.(此例中共分了5GB*4)
3.ASM第一个磁盘组名最好不动,不知道什么原因我曾经改为cluster结果后期root.sh时报找不到cluster磁盘组.
4.虚拟OS物理内存至少1572864.0 KB
5.用户名密码注意:

The Oracle Database Vault Owner user name can be a minimum of 2, and a maximum of 30 characters long. The account password can be a minimum of 8, and a maximum of 30 characters.

The password that you choose for the Oracle Database Vault Owner account must be secure. The following password restrictions are enforced:

The password must include at least one alphabet, one digit, and one nonalphanumeric character (symbol).
The password cannot be the same as the account name.
The password cannot contain any consecutive repeating characters.

引自:http://download.oracle.com/docs/cd/E11882_01/install.112/e17214/racinstl.htm#CIHHEHBF

四步骤

1.概览

Installing Oracle RAC consists of the following steps:
    1.1 Prepare servers (system, storage, and network administration):
        Install the operating system and install the operating system packages and patches to the required version.
        Create the required groups, users, and software homes.
        Set up the domain name forwarding for Grid Naming Service (GNS) if you plan to deploy GNS, and set up the network addresses in the DNS and on the server as needed.
        Set up the required storage.
        (optional) Stage all of the software on one node for installation (the "local node").
    1.2 Install Oracle Grid Infrastructure for a cluster, which includes Oracle Clusterware and Oracle Automatic Storage Management (system and storage administration):
        Install Oracle Grid Infrastructure for a cluster. During installation, Fixup scripts perform. additional configuration of operating system parameters, secure shell (SSH) for installation and user environment variables.
        Patch Oracle Clusterware and Oracle Automatic Storage Management to the latest patchset.
    1.3 Install Oracle RAC (database administration):
        Install Oracle Real Application Clusters.
        Patch Oracle RAC to the latest patchset.
        Complete the postinstallation configuration of the Oracle RAC database.
引自:http://download.oracle.com/docs/cd/E11882_01/install.112/e17214/chklist.htm#BHAEFBJB

2.详细

2.1 虚拟OS安装 (略) 结点rac1存放在" /media/124e0965-0e90-4c07-ada8-ec41ed595f55/vmware/RHEL5.4_11gr2_RAC1/ "目录下,结点rac2存放在" /mnt/sdb1/vmware/RHEL5.4_11gr2_RAC2/" 目录下.每个虚拟系统分配3个网卡. 注意swap分配要超过500MB.

2.2 共享磁盘准备(宿主机)
宿主机
$dd if=/dev/zero f=/mnt/sdb1/vmware/share_disk/sharedisk1 bs=1M count=5000
$dd if=/dev/zero f=/mnt/sdb1/vmware/share_disk/sharedisk2 bs=1M count=5000
$dd if=/dev/zero f=/mnt/sdb1/vmware/share_disk/sharedisk3 bs=1M count=5000
$dd if=/dev/zero f=/mnt/sdb1/vmware/share_disk/sharedisk4 bs=1M count=5000
# /etc/init.d/tgtd start
#tgtadm --lld iscsi --op new --mode target --tid 1 --targetname storage
#tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 --backing-store /mnt/sdb1/vmware/share_disk/sharedisk1
#tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 2 --backing-store /mnt/sdb1/vmware/share_disk/sharedisk2
#tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 3 --backing-store /mnt/sdb1/vmware/share_disk/sharedisk3
#tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 4 --backing-store /mnt/sdb1/vmware/share_disk/sharedisk4
#tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL

虚拟OS机 (每个结点OS机都执行)
#/etc/init.d/iscsi start
#/etc/init.d/iscsid start
#iscsiadm -m discovery -t sendtargets -p 192.168.2.1 -l
#fdisk -l
成功的话会查到有新磁盘加入

参考资料:( 时间太久,忘记了,google linux iscsi )
2.3 网卡配置

Home Node	Given Name	Device Name	Type	Address
rac1	rac1	eth0	Public	192.168.2.11
rac1	rac1-vip		Virtual	192.168.2.21
rac1	rac1-priv1	eth1	Private	10.0.0.11
rac1	rac1-priv2	eth2	Private	10.0.0.21
rac2	rac2	eth0	Public	192.168.2.12
rac2	rac2-vip		Virtual	192.168.2.22
rac2	rac2-priv1	eth1	Private	10.0.0.12
rac2	rac2-priv2	eth2	Private	10.0.0.22

在主机hosts文件添加映射
# cat /etc/hosts

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1        localhost.localdomain localhost
::1        localhost6.localdomain6 localhost6

192.168.2.11    rac1
192.168.2.12    rac2

10.0.0.11    rac1-priv1
10.0.0.12    rac2-priv1
10.0.0.21    rac1-priv2
10.0.0.22    rac2-priv2

192.168.2.21    rac1-vip
192.168.2.22    rac2-vip
2.4 必要的软件包,kernel参数等配置

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586

Note: The minimum value required for shmmax is 0.5 GB. However, Oracle recommends that you set the value of shmmax to 2.0 GB for optimum performance of the system.

参考资料: http://download.oracle.com/docs/cd/E11882_01/install.112/e16766/toc.htm#i1011296
2.5 建立组,用户及安装目录# groupadd OSDBA
# groupadd OSOPER
# groupadd OSASM

#mkdir -p /u01/app/11.2.0/grid
#mkdir -p /u01/app/oracle
#chown -R oracle:oinstall /u01
2.6 配置DNS

NameServer	域名	IP地址
192.168.2.11	rac-scan.11gr2RAC.com	192.168.2.201
192.168.2.11	rac-scan.11gr2RAC.com	192.168.2.202
192.168.2.11	rac-scan.11gr2RAC.com	192.168.2.203

注:由于是自己实验,所以域名服务器选择了rac2主机.在rac2主机上配置DNS服务.

创建named.conf文件
# cp -p /var/named/chroot/etc/named.caching-nameserver.conf /var/named/chroot/etc/named.conf

编辑named.conf

编辑named.conf文件，将

listen-on port 53 { 127.0.0.1; };

allow-query { localhost; };

match-clients { localhost; };

match-destinations { localhost; };

allow-query-cache { localhost; };

中的内容修改成{ any; } 注意两边空格,然后保存退出

注:红字行与红字行之间为原文贴过来的.

dns配置文件
# cat /var/named/chroot/etc/named.rfc1912.zones
// named.rfc1912.zones:
//
// Provided by Red Hat caching-nameserver package
//
// ISC BIND named zone configuration for zones recommended by
// RFC 1912 section 4.1 : localhost TLDs and address zones
//
// See /usr/share/doc/bind*/sample/ for example named configuration files.
//
zone "." IN {
    type hint;
    file "named.ca";
};

zone "localdomain" IN {
    type master;
    file "localdomain.zone";
    allow-update { none; };
};

zone "localhost" IN {
    type master;
    file "localhost.zone";
    allow-update { none; };
};

zone "0.0.127.in-addr.arpa" IN {
    type master;
    file "named.local";
    allow-update { none; };
};

zone "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa" IN {
        type master;
    file "named.ip6.local";
    allow-update { none; };
};

zone "255.in-addr.arpa" IN {
    type master;
    file "named.broadcast";
    allow-update { none; };
};

zone "0.in-addr.arpa" IN {
    type master;
    file "named.zero";
    allow-update { none; };
};

zone "2.168.192.in-addr.arpa" IN {
    type master;
    file "11gr2RAC.com.local";
    allow-update { none; };
};

zone "11gr2RAC.com" IN {
    type master;
    file "11gr2RAC.com.zone";
    allow-update { none; };
};

注:此步红色为自己添加

创建DNS解析文件
# cat /var/named/chroot/var/named/11gr2RAC.com.local
$TTL    86400
@       IN      SOA     11gr2RAC.com. root.11gr2RAC.com. (
                                      1997022700 ; Serial
                                      28800      ; Refresh
                                      14400      ; Retry
                                      3600000    ; Expire
                                      86400 )    ; Minimum
        IN      NS      localhost.
1       IN      PTR     localhost.
201       IN      PTR     rac-scan.11gr2RAC.com.
202       IN      PTR     rac-scan.11gr2RAC.com.
203       IN      PTR     rac-scan.11gr2RAC.com.

# cat /var/named/chroot/var/named/11gr2RAC.com.zone
$TTL    86400
@        IN SOA    11gr2RAC.com       root.11gr2RAC.com (
                    42        ; serial (d. adams)
                    3H        ; refresh
                    15M        ; retry
                    1W        ; expiry
                    1D )        ; minimum

         IN A        127.0.0.1
        IN AAAA        ::1
            IN NS        11gr2RAC.com
rac-scan    IN A        192.168.2.201
rac-scan    IN A        192.168.2.202
rac-scan    IN A        192.168.2.203

参考资料: http://blog.csdn.net/tianlesoftware/article/details/6006247
2.7 配置存储步骤2.2已经将共享存储完成,系统中会发现 sdb,sdc,sdd,sde 4块硬盘.将此4块盘fdisk分好区.映射成raw设备,并将raw设备授予oracle用户组.在两rac结点系统rc.local中添加如下内容.
# cat /etc/rc.local   (显示已经添加好的rc.local文件内容)

#!/bin/sh
#
# This script. will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style. init stuff.

touch /var/lock/subsys/local

raw /dev/raw/raw1 /dev/sdb1
raw /dev/raw/raw2 /dev/sdc1
raw /dev/raw/raw3 /dev/sdd1
raw /dev/raw/raw4 /dev/sde1
sleep 2
chown -R oracle:dba /dev/raw

注:红色字为添加的内容2.8 配置ntp# cat /etc/ntp.conf
# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery

# Permit all access over the loopback interface. This could
# be tightened as well, but to do so would effect some of
# the administrative functions.
restrict 127.0.0.1
restrict -6 ::1

# Hosts on local network are less restricted.
#restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap

# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server 0.rhel.pool.ntp.org
server 1.rhel.pool.ntp.org
server 2.rhel.pool.ntp.org

#broadcast 192.168.1.255 key 42        # broadcast server
#broadcastclient            # broadcast client
#broadcast 224.0.1.1 key 42        # multicast server
#multicastclient 224.0.1.1        # multicast client
#manycastserver 239.255.254.254        # manycast server
#manycastclient 239.255.254.254 key 42    # manycast client

# Undisciplined Local Clock. This is a fake driver intended for backup
# and when no outside source of synchronized time is available.
server    127.127.1.0    # local clock
fudge    127.127.1.0 stratum 10

server 210,72.145.44
fudge 201.72.145.44 stratum 0

# Drift file. Put this in a directory which the daemon can write to.
# No symbolic links allowed, either, since the daemon updates the file
# by creating a temporary in the same directory and then rename()'ing
# it to the file.
driftfile /var/lib/ntp/drift

# Key file containing the keys and key identifiers used when operating
# with symmetric key cryptography.
keys /etc/ntp/keys

# Specify the key identifiers which are trusted.
#trustedkey 4 8 42

# Specify the key identifier to use with the ntpdc utility.
#requestkey 8

# Specify the key identifier to use with the ntpq utility.
#controlkey 8

注:红色为添加部分

修改/etc/sysconfig/ntpd 文件. 将OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid" 改为 PTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid" . 如下
# cat /etc/sysconfig/ntpd

# Drop root to id 'ntp:ntp' by default.
#OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid"
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"

# Set to 'yes' to sync hw clock after successful ntpdate
SYNC_HWCLOCK=no

# Additional options for ntpdate
NTPDATE_OPTIONS=""

2.9 安装grid关于ntp的检查失败的报错信息.

Network Time Protocol (NTP) - This task verifies cluster time synchronization on clusters that use Network Time Protocol (NTP).
Check Failed on Nodes: [rac2, rac1]
Verification result of failed node: rac2
Expected Value
: n/a
Actual Value
: n/a
List of errors:
-
PRVF-5439 : NTP daemon does not have slewing option "-x" set on node "rac2" - Cause: NTP daemon on the specified node does not have the slewing option set. - Action: Shutdown and restart the NTP daemon with the slewing option set. For more information on NTP daemon slewing option refer to NTP daemon's man pages.
Back to Top
Verification result of failed node: rac1
Expected Value
: n/a
Actual Value
: n/a
List of errors:
-
PRVF-5439 : NTP daemon does not have slewing option "-x" set on node "rac1" - Cause: NTP daemon on the specified node does not have the slewing option set. - Action: Shutdown and restart the NTP daemon with the slewing option set. For more information on NTP daemon slewing option refer to NTP daemon's man pages.

解决方法参考:

How to Configure NTP or Windows Time to Resolve CLUVFY Error PRVF-5436 PRVF-9652 [ID 1056693.1]
https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&doctype=PROBLEM&id=1056693.1

http://3qs.com/2011-06/07-prvf-5436-the-ntp-daemon-running-on-one-or-more-nodes-lacks-the-slewing-option-x-648.html

注:前步ntp配置部分已将此问题解决.
2.9 安装db software2.10 asmca2.11 dbca

终注:最终完成由于vmware workstation的共享存储十分不稳定.致使在RAC两结点间数据有时同步有时不同步.最后共享存储方式使用iscsi方式得以顺利完成.并稳定实现.
下面是vmware workstation 共享存储的设置,两结点中按文中方法确实能同时看到.
vmware workstation 创建共享磁盘组 http://candon123.blog.51cto.com/704299/197219

参考文档 RACGuides_Rac11gR2OnLinux 介个自己找地方下载吧,我的是朋友给的.