环境规划:
1.硬件:
1.1节点主机:ESX4.1创建两台同样配置主机;每台计算机两个独立网卡,每个网卡的地址如上表所示。
1.2存储设备:
以root登录ESC4.1主机,使用命令方式创建共享磁盘(node1为第一节点虚拟化文件存储目录)
vmkfstools -c 11G -d eagerzeroedthick -a lsilogic /vmfs/volumes/node1/RACsharedisk.vmdk
1.3 两台网络交换机:
(a)一台用于两台主机Private NIC 连接,上面只连接从两个Private NIC 过来的网线(这里我们在连接ESX4.1服务器的vcenter 上创建一台虚拟交换机),具体创建方法如下:
(b)一台用于两个主机的Public NIC 和其他客户及的互连。
2.软件
实施阶段:
整个过程分为几个模块:主机配置、安装Clusterware、安装oracle database、配置listener、创建ASM、创建database.
1 主机配置
(先配置一台主机,配置完成后通过vmware 的克隆技术克隆出另一台主机)
1.1网络配置
vim/etc/hosts/,添加如下内容:
192.168.32.87b87
192.168.32.88b88
10.0.0.154 b87-priv
10.0.0.155 b88-priv
192.168.32.89b87-vip
192.168.32.90b88-vip
1.2 必须的RPM包:
使用如下命令检测RPM包
rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' \
binutils \
compat-db \
control-center \
gcc \
gcc-c++ \
glibc \
glibc-common \
glibc-devel \
libstdc++ \
libstdc++-devel \
make \
sysstat \
libaio \
compat-libstdc++-33 \
glibc-headers \
kernel-headers \
libXp \
openmotif22 \
compat-libf2c \
compat-libgcc \
libgomp \
libXmu \
elfutils-libelf \
elfutils-libelf-devel \
elfutils-libelf-devel-static \
libaio-devel \
unixODBC \
unixODBC-devel \
libgcc
1.3安装ASMLib 以及补丁
查看OS Kernel 版本,选择的包和补丁要与内核匹配,详细的包如下:
compat-binutils215-2.15.92.0.2-24.i386.rpm
compat-libcwait-2.1-1.i386.rpm
compat-libstdc++-egcs-1.1.2-1.i386.rpm
compat-oracle-el5-1.0-5.i386.rpm
openmotif21-2.1.30-11.EL5.i386.rpm
openmotif21-debuginfo-2.1.30-11.EL5.i386.rpm
oracleasm-2.6.18-194.el5-2.0.5-1.el5.i686.rpm
oracleasm-2.6.18-194.el5debug-2.0.5-1.el5.i686.rpm
oracleasm-2.6.18-194.el5-debuginfo-2.0.5-1.el5.i686.rpm
oracleasm-2.6.18-194.el5PAE-2.0.5-1.el5.i686.rpm
oracleasm-2.6.18-194.el5xen-2.0.5-1.el5.i686.rpm
oracleasmlib-2.0.4-1.el5.i386.rpm
oracleasm-support-2.1.7-1.el5.i386.rpm
xorg-x11-libs-compat-6.8.2-1.EL.33.0.1.i386.rpm
以上补丁可以在以下链接下载:
http://www.oracle.com/technology/software/tech/linux/asmlib/rhel5.html
http://oss.oracle.com/projects/compat-oracle/files/Enterprise_Linux/
1.4创建oracle用户、组
groupadd -g 1000 oinstall
groupadd -g 1001 dba
useradd -u 1000 -g oinstall -G dba oracle
passwd oracle
1.5创建相关目录
mkdir -p /opt/u01/oracle
chown -R oracle:oinstall /opt/u01/
1.6设置oracle环境变量
vim /home/oracle/.bash_profile
export ORACLE_BASE=/opt/u01/oracle
export ORACLE_TERM=xterm
export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
export CRS_HOME=/opt/u01/oracle/product/crs
export ORACLE_HOME=/opt/u01/oracle/product/10.2.0.1
export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data
export PATH=$CRS_HOME/bin:$ORACLE_HOME/bin:$PATH
export ORACLE_SID=RACDB1
加载配置
oracle用户注销重新登录 或 source .bash_profile
1.7配置内核参数
vim /etc/sysctl.conf
kernel.shmall = 2097152
kernel.shmmax = 2147483648
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 1048576
net.core.rmem_max = 1048576
net.core.wmem_default = 262144
net.core.wmem_max = 262144
使配置生效
sysctl -p
1.8 shell限制
vim /etc/security/limits.conf
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 65536
oracle hard nofile 65536
vim /etc/pam.d/system-auth
session required pam_limits.so
1.9磁盘分区
Fdisk /dev/sdb
依次按如下命令:
(n>p>1>+200M,n>p>2>+200M, n>p>3>+5000m, n>p>4>+5000m, w 保存退出)
分区完成后,使用partprobe /dev/sdb 更新分区
1.10配置裸设备
vim/etc/syscofig/rawdevices
/dev/raw/raw1/dev/sdb1
/dev/raw/raw2/dev/sdb2
设置oracle用户对裸设备的权限,redhat5U5需要如下修改
修改/etc/udev/rules.d/50-udev.rules文件,在KERNEL=="raw[0-9]*",NAME="raw/%k" 这行后面加上用户和权限信息
cat /etc/udev/rules.d/50-udev.rules | grep username
KERNEL=="raw[0-9]*", NAME="raw/%k"OWNER="username" GROUP="username", MODE="0660"
确认裸设备服务自动启动
[root@b87~]# chkconfig --list rawdevices;
rawdevices 0:off 1:off 2:off 3:on 4:on 5:on 6:off
启动rawdevices,确认裸设备创建成功,权限正确:
[root@b87~]# service rawdevices status
/dev/raw/raw1: bound to major 8, minor 17
/dev/raw/raw2: bound to major 8, minor 18
[root@b87~]# ll /dev/raw
total 0
crw-rw---- 1oracle oinstall 162, 1 Dec 19 22:19 raw1
crw-rw---- 1oracle oinstall 162, 2 Dec 21 09:39 raw2
1.11:修改/etc/redhat-release把5.5修改成4.5
Red Hat Enterprise Linux Server release 4.5(Tikanga)
在此,使用VMware克隆技术,克隆出另节点主机b88,IP按照文档开头规划表上进行配置.
以下操作默认在b87上操作,如果需要两个节点都执行,会特别注明。
1.11 创建ASM磁盘
运行/etc/init.d/oracleasm configure
[root@b87rules.d]# /etc/init.d/oracleasm configure
Configuringthe Oracle ASM library driver.
This willconfigure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded onboot and what permissions it will have. The current values
will beshown in brackets ('[]'). Hitting <ENTER> without typing an
answer willkeep that current value. Ctrl-C will abort.
Default userto own the driver interface []: oracle
Defaultgroup to own the driver interface []: dba
Start OracleASM library driver on boot (y/n) [n]: y
Scan forOracle ASM disks on boot (y/n) [y]: y
WritingOracle ASM library driver configuration: done
Initializingthe Oracle ASMLib driver: [ OK ]
Scanning thesystem for Oracle ASMLib disks: [ OK ]
再创建磁盘就成功了
[root@b87rules.d]# /etc/init.d/oracleasm createdisk VOL1 /dev/sdb3
Marking disk"VOL1" as an ASM disk: [ OK ]
[root@b87rules.d]# /etc/init.d/oracleasm createdisk VOL2 /dev/sdb4
Marking disk"VOL2" as an ASM disk: [ OK ]
[root@b87rules.d]# /etc/init.d/oracleasm listdisks
VOL1
VOL2
在b88 上执行
[root@b88~]# /etc/init.d/oracleasm scandisks
Scanning thesystem for Oracle ASMLib disks: [ OK ]
[root@b88~]# /etc/init.d/oracleasm listdisks
VOL1
VOL2
[root@b88~]#
1.12 配置节点之间等价性
以oracle用户在两个节点上都要运行:
cd /home/oracle
mkdir .ssh
chmod 700.ssh
cd .ssh
ssh-keygen –trsa
ssh-keygen –tdsa
cat *.pub> authorized_keys
在节点上执行
b87: scpauthoried_keys b88:/home/oracle/.ssh/keys_b87
b88: catkey_b88 >> authorized_keys
b88: scpauthorized_keys b87:/home/oracle/.ssh/
在两节点上执行确认等价性:
ssh b87,sshb87-priv,ssh b88,ssh b88-priv
1.12配置时间同步
如果没有外部时间服务器,使用节点b87作为时间服务器,配置如下:
编辑b87 ntp配置文件
vim/etc/ntp.conf
server127.127.1.0
fudge 127.127.1.0 stratum 11
driftfile/var/lib/ntp/drift
broadcastdelay0.008
编辑b88 ntp 配置文件
vim/etc/ntp.conf
server192.168.32.87 prefer
driftfile/var/lib/ntp/drift
broadcastdelay0.008
1.13 配置hangcheck-timer模块
[root@b88~]# cat /etc/rc.d/rc.local
#!/bin/sh
#
# Thisscript will be executed *after* all the other init scripts.
# You canput your own initialization stuff in here if you don't
# want to dothe full Sys V style init stuff.
touch/var/lock/subsys/local
modprobehangcheck-timer
[root@b88~]# cat /etc/modprobe.conf
alias eth0pcnet32
alias eth1pcnet32
alias scsi_hostadaptermptbase
aliasscsi_hostadapter1 mptspi
aliasscsi_hostadapter2 ata_piix
optionshangcheck-timer hangcheck_tick=10 hangcheck_margin=30
确认加载成功:
grephangcheck /var/log/messages|tail -2
Dec 1914:04:06 b88 kernel: Hangcheck: starting hangcheck timer 0.9.0 (tick is 10seconds, margin is 30 seconds).
Dec 1921:40:35 b88 kernel: Hangcheck: starting hangcheck timer 0.9.0 (tick is 10seconds, margin is 30 seconds).
现在,所有安装前的准备工作已经完成,下面开始安装部分。
2.安装Oracle Clusterware
下载安装包10201_clusterware_linux32.zip,并上传到b87上,使用unzip解压,生成clusterware目录,然后执行安装:
./clusterware/runinstaller,一路next.
这里不在详细阐述和截图,到这里前期的执行都是没有问题的。从节点2开始执行root.sh这一刻起,问题来了,下面就着问题给出正确的解决方案:
解决节点2运行root.sh脚本,需要手动在节点2上执行VIPCA配置,而运行VIPCA又会失败。
解决VIPCA 启动失败问题解决方法如下:
RAC2节点上执行root.sh之前需要编辑两个文件,这两个文件都位于/opt/ora10g/product/10.2.0/crs_1/bin下,文件名分别为vipca和srvctl。首先编辑vipca文件,找到如下内容:
Remove this workaround when the bug 3937317is fixed
arch=`uname -m`
if [ "$arch" = "i686" -o"$arch" = "ia64" ]
then
LD_ASSUME_KERNEL=2.4.19
export LD_ASSUME_KERNEL
fi
#End workaround
在fi 后新添加一行:
unset LD_ASSUME_KERNEL
然后编辑srvctl文件,找到如下内容:
LD_ASSUME_KERNEL=2.4.19
export LD_ASSUME_KERNEL
同样在其后新增加一行:
unset LD_ASSUME_KERNEL
对应的报错信息是:
/opt/ora10g/product/10.2.0/crs_1/jdk/jre//bin/java:error while loading shared libraries: libpthread.so.0:
cannot open shared object file: No such file or directory
修改完成后可以不需要再执行脚本,手动执行VIPCA配置即可。
安装完成后,使用./olsnodes
./crsctl checkcrs
./crs_stat –t 确认安装有效性。
3.安装Oracle database
详细过程省略
安装database--à配置监听
监听配置如下:
SID_LIST_LISTENER_B87=
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtProc)
(ORACLE_HOME =/opt/u01/oracle/product/10.2.0.1)
(PROGRAM = extproc)
)
)
LISTENER_B87=
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST =b87-vip)(PORT = 1521)(IP = FIRST))
(ADDRESS = (PROTOCOL = TCP)(HOST =192.168.32.87)(PORT = 1521)(IP = FIRST))
)
)
4.创建ASM
是用DBCA一路NEXT,前面都很顺利,到挂载磁盘组的时候节点2挂载不上,需要重启节点2上所有服务,如下。
crs_stop –c b88
Crs_start -all
再回到DBCA图形上点击mount all,挂载磁盘组,随后两个节点都能挂载。
5.创建数据库实例
使用DBCA一路NEXT,前面都很顺利,到输入ASM管理员密码时报错,
DBCA提示
Errorwhen connecting to ASM instance on local node:io exception :The network adaptercould not establish the connection.
解决方法如下:
1、使用srvctl停止listener
/opt/ora10g/product/10.2.0/crs_1/bin/srvctl stop listener -n rac1
/opt/ora10g/product/10.2.0/crs_1/bin/srvctl stop listener -n rac2
2、分别在两个节点启动listener
lsnrctl start
3、查看listener状态()
lsnrctlstatus
4、使用srvctl启动listener
/opt/ora10g/product/10.2.0/crs_1/bin/srvctl startlistener -n rac1
/opt/ora10g/product/10.2.0/crs_1/bin/srvctl start listener -n rac2
数据库安装成功后,使用crs_stat –t–v 看到如下界面
到此,安装结束。
6.客户端测试
客户端tnsnames.ora添加以下内容:
rac =
(DESCRIPTION =
(ADDRESS = (PROTOCOL =TCP)(HOST = b87-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL =TCP)(HOST = b88-vip)(PORT = 1521))
(LOAD_BALANCE = yes)
(CONNECT_DATA=
(SERVER = DEDICATED)
(SERVICE_NAME = racdb)
(FAILOVER_MODE =
(TYPE = session)
(METHOD = basic)
(RETRIES=180 )
(DELAY =5)
)
)
)
轮流关闭各节点,使用crs_stat –t –v,测试VIP能否自动漂移.