一、安装环境简介:
1、安装vbox最新版本,可在oracle官网上下载http://www.oracle.com/technetwork/server-storage/virtualbox/downloads/index.html,vbox位于win系统的安装,只要next就ok了,这里不多说了。
2、操作系统:redhat 5.5企业版
3、相关工具:Xmanager Enterprise 4、SecureCRT Version 6.5.0
4、存储:利用udev来做绑定raw
5、rac节点网络配置
节点名 网卡 ip地址 对应名称
trsen01 eth0 192.168.8.145 trsen01.oracle.com
eht1 192.168.8.147 trsen01-vip
10.1.1.8 trsen01-priv
trsen02 eth0 192.168.8.146 trsen02.oracle.com
eth1 192.168.8.148 trsen02-vip
10.1.1.9 trsen02-priv
6、rac存储配置
软件类型 文件类型 本地文件系统 raw asm
cluster CLUSTE SOFTWARE $CRS_HOME / /
OCR / /dev/raw/raw1(root/oinstall) /
VOTEDISK / /dev/raw/raw2(oracle/oinstall) /
database database software $ORACLE_HOME / +trsen_dg
7、硬件要求
至少1GB的物理内存,至少1GB的虚拟内存;
两台机器上需要两张网卡分别作公网和内网用;
二、在vbox上安装rhel 5.5操作系统,这里不废话,找AS去处理这个问题
三、检查两个节点的oracle所需的安装包并安装rpm包
1、检查rpm包
[root@trsen01 ~]# rpm -q binutils compat-db compat-libstdc++-296 control-center gcc gcc-c++ glibc glibc-common libstdc++ libstdc++-devel make sysstat setarch glibc-devel libaio ksh glibc-headers libgnome libgcc libgnomeui libgomp openmotif libXp --queryformat "%{N}-%{VERSION}.%{RELEASE} (%{ARCH})\n"
binutils-2.17.50.0.6.14.el5 (i386)
package compat-db is not installed
compat-libstdc++-296-2.96.138 (i386)
control-center-2.16.0.16.el5 (i386)
package gcc is not installed
package gcc-c++ is not installed
glibc-2.5.49 (i686)
glibc-common-2.5.49 (i386)
libstdc++-4.1.2.48.el5 (i386)
package libstdc++-devel is not installed
make-3.81.3.el5 (i386)
package sysstat is not installed
setarch-2.0.1.1 (i386)
package glibc-devel is not installed
libaio-0.3.106.5 (i386)
ksh-20100202.1.el5 (i386)
package glibc-headers is not installed
libgnome-2.16.0.6.el5 (i386)
libgcc-4.1.2.48.el5 (i386)
libgnomeui-2.16.0.5.el5 (i386)
package libgomp is not installed
package openmotif is not installed
package libXp is not installed
2、配置yum来安装所需的rpm包,安装完包过后,要mout的光盘给umount了。
[root@vrh3 ~]# mount /dev/cdrom /media/disk
mount: block device /dev/cdrom is write-protected, mounting read-only
touch /etc/yum.repos.d/public-yum-el5.repo,并加入以下内容
[rhel5]
name = Enterprise Linux 5.5 DVD
baseurl=file:///media/disk/Server/
gpgcheck=0
enabled=1
[root@trsen01 ~]# yum -y install compat-db gcc gcc-c++ libstdc++-devel sysstat glibc-devel glibc-headers libgomp openmotif libXp
Loaded plugins: rhnplugin, security
This system is not registered with RHN.
RHN support will be disabled.
rhel5 | 1.3 kB 00:00
rhel5/primary | 753 kB 00:00
rhel5 2348/2348
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package compat-db.i386 0:4.2.52-5.1 set to be updated
---> Package gcc.i386 0:4.1.2-48.el5 set to be updated
---> Package gcc-c++.i386 0:4.1.2-48.el5 set to be updated
---> Package glibc-devel.i386 0:2.5-49 set to be updated
---> Package glibc-headers.i386 0:2.5-49 set to be updated
--> Processing Dependency: kernel-headers >= 2.2.1 for package: glibc-headers
--> Processing Dependency: kernel-headers for package: glibc-headers
---> Package libXp.i386 0:1.0.0-8.1.el5 set to be updated
---> Package libgomp.i386 0:4.4.0-6.el5 set to be updated
---> Package libstdc++-devel.i386 0:4.1.2-48.el5 set to be updated
rhel5/filelists | 2.4 MB 00:00
---> Package openmotif.i386 0:2.3.1-2.el5_4.1 set to be updated
---> Package sysstat.i386 0:7.0.2-3.el5 set to be updated
--> Running transaction check
---> Package kernel-headers.i386 0:2.6.18-194.el5 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
============================================================================================================================
Package Arch Version Repository Size
============================================================================================================================
Installing:
compat-db i386 4.2.52-5.1 rhel5 1.7 M
gcc i386 4.1.2-48.el5 rhel5 5.2 M
gcc-c++ i386 4.1.2-48.el5 rhel5 3.4 M
glibc-devel i386 2.5-49 rhel5 2.0 M
glibc-headers i386 2.5-49 rhel5 601 k
libXp i386 1.0.0-8.1.el5 rhel5 22 k
libgomp i386 4.4.0-6.el5 rhel5 70 k
libstdc++-devel i386 4.1.2-48.el5 rhel5 2.8 M
openmotif i386 2.3.1-2.el5_4.1 rhel5 1.5 M
sysstat i386 7.0.2-3.el5 rhel5 170 k
Installing for dependencies:
kernel-headers i386 2.6.18-194.el5 rhel5 1.0 M
Transaction Summary
============================================================================================================================
Install 11 Package(s)
Upgrade 0 Package(s)
Total download size: 18 M
Downloading Packages:
----------------------------------------------------------------------------------------------------------------------------
Total 3.1 GB/s | 18 MB 00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : libXp 1/11
Installing : libgomp 2/11
Installing : openmotif 3/11
Installing : compat-db 4/11
Installing : sysstat 5/11
Installing : libstdc++-devel 6/11
Installing : kernel-headers 7/11
Installing : glibc-headers 8/11
Installing : glibc-devel 9/11
Installing : gcc 10/11
Installing : gcc-c++ 11/11
Installed:
compat-db.i386 0:4.2.52-5.1 gcc.i386 0:4.1.2-48.el5 gcc-c++.i386 0:4.1.2-48.el5 glibc-devel.i386 0:2.5-49
glibc-headers.i386 0:2.5-49 libXp.i386 0:1.0.0-8.1.el5 libgomp.i386 0:4.4.0-6.el5 libstdc++-devel.i386 0:4.1.2-48.el5
openmotif.i386 0:2.3.1-2.el5_4.1 sysstat.i386 0:7.0.2-3.el5
Dependency Installed:
kernel-headers.i386 0:2.6.18-194.el5
Complete!
四、两个节点配置oracle用户,网络及参数文件
A、添加用户组及用户并给与密码
[root@trsen01 ~]# groupadd -g 501 oinstall
[root@trsen01 ~]# groupadd -g 502 dba
[root@trsen01 ~]# groupadd -g 503 oper
[root@trsen01 ~]# useradd -u 501 -g oinstall -G dba,oper oracle
[root@trsen01 ~]# passwd oracle
Changing password for user oracle.
New UNIX password:
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
B、vi /etc/hosts,添加如下信息:
192.168.8.145 trsen01 trsen01.oracle.com
192.168.8.146 trsen02 trsen02.oracle.com
192.168.8.147 trsen01-vip
192.168.8.148 trsen02-vip
10.1.1.8 trsen01-priv
10.1.1.9 trsen02-priv
C、vi /etc/sysctl.conf的参数,添加如下信息
kernel.shmmax = 48719476736
kernel.shmall = 2294967296
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_default = 262144
net.core.wmem_max = 262144
D、vi /etc/security/limits.conf的参数,添加如下信息
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
vi/etc/pam.d/login的参数,添加如下信息
session required pam_limits.so
E、vi /etc/profile,添加如下信息
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
五、利用vbox配置共享存储
1、选中安装节点,“右击”-->"设置"-->"存储"-->"添加虚拟硬盘"-->”创建新虚拟盘“注意在创建的过程,选择"固定大小",这里我创建3个盘来做共享盘,这三个盘的大小一下。
2、选择“管理”-->"虚拟介质管理"-->"虚拟硬盘"-->选中药共享的盘,右击-->"修改"-->选择“可共享".
3、在另个节点上处理时,,“右击”-->"设置"-->"存储"-->"添加虚拟硬盘"-->"选择已有虚拟盘",选择在另个节点创建已共享的虚拟盘。
4、利用fdisk && udev规则做raw和asm盘的绑定
A、查看一下创建的三个盘分别是/dev/sdb /dev/sdc /dev/sdd
[root@trsen01 ~]# fdisk -l
Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 274 2096482+ 82 Linux swap / Solaris
/dev/sda3 275 1305 8281507+ 83 Linux
Disk /dev/sdb: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdc doesn't contain a valid partition table
Disk /dev/sdd: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdd doesn't contain a valid partition table
B、对sdb盘进行分区,做成放置ocr和voting disk的raw设备
[root@trsen01 ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): p
Disk /dev/sdb: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-783, default 1): 1
Last cylinder or +size or +sizeM or +sizeK (1-783, default 783): +3G
Command (m for help): p
Disk /dev/sdb: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 366 2939863+ 83 Linux
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (367-783, default 367):
Using default value 367
Last cylinder or +size or +sizeM or +sizeK (367-783, default 783): +3G
Command (m for help): p
Disk /dev/sdb: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 366 2939863+ 83 Linux
/dev/sdb2 367 732 2939895 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
C、查看分区后的/dev/sdb有两个分区
[root@trsen01 ~]# ll /dev/sd*
brw-r----- 1 root disk 8, 0 Feb 12 16:29 /dev/sda
brw-r----- 1 root disk 8, 1 Feb 12 16:30 /dev/sda1
brw-r----- 1 root disk 8, 2 Feb 12 16:29 /dev/sda2
brw-r----- 1 root disk 8, 3 Feb 12 16:30 /dev/sda3
brw-r----- 1 root disk 8, 16 Feb 12 16:39 /dev/sdb
brw-r----- 1 root disk 8, 17 Feb 12 16:39 /dev/sdb1
brw-r----- 1 root disk 8, 18 Feb 12 16:39 /dev/sdb2
brw-r----- 1 root disk 8, 32 Feb 12 16:29 /dev/sdc
brw-r----- 1 root disk 8, 48 Feb 12 16:29 /dev/sdd
D、利用udev规则将 /dev/sdb1 && /dev/sdb2绑定成/dev/raw/raw1 && /dev/raw/raw2
[root@trsen01 ~]# more /etc/udev/rules.d/60-raw.rules
# Enter raw device bindings here.
#
# An example would be:
# ACTION=="add", KERNEL=="sda", RUN+="/bin/raw /dev/raw/raw1 %N"
# to bind /dev/raw/raw1 to /dev/sda, or
# ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="1", RUN+="/bin/raw /dev/raw/raw2 %M %m"
# to bind /dev/raw/raw2 to the device with major 8, minor 1.
ACTION=="add", KERNEL=="sdb1", RUN+="/bin/raw /dev/raw/raw1 %N"
ACTION=="add", KERNEL=="sdb2", RUN+="/bin/raw /dev/raw/raw2 %N"
ACTION=="add", KERNEL=="raw*", OWNER=="oracle", GROUP=="oinstall", MODE=="0660"
E、利用下面脚本生成需要的 udev规则语句
[root@trsen01 ~]#for i in c d;
do
echo "KERNEL==\"sd*\", BUS==\"scsi\", PROGRAM==\"/sbin/scsi_id -g -u -s %p\", RESULT==\"`scsi_id -g -u -s /block/sd$i`\", NAME=\"asm-disk$i\", OWNER=\"oracle\", GROUP=\"oinstall\", MODE=\"0660\""
done
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VBce25369a-9fc8aff2_", NAME="asm-diskc", OWNER="oracle", GROUP="oinstall", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB9f38ff67-2fdb530d_", NAME="asm-diskd", OWNER="oracle", GROUP="oinstall", MODE="0660"
F、利用udev规则将/dev/sdc && /dev/sdd 做成asm磁盘
[root@trsen01 ~]# more /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VBce25369a-9fc8aff2_", NAME="asm-diskc",
OWNER="oracle", GROUP="oinstall", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB9f38ff67-2fdb530d_", NAME="asm-diskd
", OWNER="oracle", GROUP="oinstall", MODE="0660"
G、启动udev设备
[root@trsen01 ~]# /sbin/start_udev
Starting udev: [ OK ]
F、截至此,共享的raw设备及asm盘全部做好,查看
[root@trsen01 ~]# ll /dev/raw/*
crw-rw---- 1 oracle oinstall 162, 1 Feb 12 16:54 /dev/raw/raw1
crw-rw---- 1 oracle oinstall 162, 2 Feb 12 16:54 /dev/raw/raw2
[root@trsen01 ~]# ll /dev/asm*
brw-rw---- 1 oracle oinstall 8, 32 Feb 12 16:54 /dev/asm-diskc
brw-rw---- 1 oracle oinstall 8, 48 Feb 12 16:54 /dev/asm-diskd
注意:以上的D~F步骤,在所有节点上都要做。
六、配置oracle用户环境变量、创建相关目录、配置hangcheck-timer.ko 模块及ssh通过密钥验证、检查防火墙,selinux
1、配置oracle用户环境变量
[oracle@trsen02 ~]$ more ~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin
export ORACLE_SID=trsendb1
export ORACLE_BASE=/u01/app/oracle
export ORA_CRS_HOME=$ORACLE_BASE/product/10.2.0/crs_1
export ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1
export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
export PATH=$ORACLE_HOME/bin:$PATH
export TNS_ADMIN=$ORACLE_HOME/network/admin
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib:$ORACLE_HOME/oracm/lib:$ORACLE_HOME/lib
export LIBPATH=$LIBPATH:$ORACLE_HOME/lib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/jlib:$ORACLE_HOME/network/jlib:$ORACLE_HOME/jre/
export TEMP=/tmp
export PATH
2、创建文件夹
mkdir /u01/app -p
chown oracle:oinstall /u01 -R
3、配置hangcheck-timer.ko 模块
[root@trsen01 u01]# /sbin/insmod /lib/modules/2.6.18-194.el5/kernel/drivers/char/hangcheck-timer.ko hangcheck_tick=1 hangcheck_margin=10
[root@trsen01 u01]# lsmod | grep hang
hangcheck_timer 8025 0
4、配置ssh密钥验证通信
A、生成密钥
[oracle@trsen01 ~]$ cd .ssh
[oracle@trsen01 .ssh]$ ls
known_hosts
[oracle@trsen01 .ssh]$ ll -al
total 12
drwx------ 2 oracle oinstall 4096 Feb 13 10:54 .
drwx------ 4 oracle oinstall 4096 Feb 13 10:54 ..
-rw-r--r-- 1 oracle oinstall 395 Feb 13 10:54 known_hosts
[oracle@trsen01 .ssh]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
28:1c:ce:41:95:32:0c:68:44:bc:68:e5:f1:7b:d4:bf oracle@trsen01
[oracle@trsen01 .ssh]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
3e:ee:05:e2:b3:dc:11:62:fe:bc:89:d6:bf:39:6e:10 oracle@trsen01
[oracle@trsen01 .ssh]$ ll -al
total 28
drwx------ 2 oracle oinstall 4096 Feb 13 10:56 .
drwx------ 4 oracle oinstall 4096 Feb 13 10:54 ..
-rw------- 1 oracle oinstall 668 Feb 13 10:56 id_dsa
-rw-r--r-- 1 oracle oinstall 604 Feb 13 10:56 id_dsa.pub
-rw------- 1 oracle oinstall 1675 Feb 13 10:55 id_rsa
-rw-r--r-- 1 oracle oinstall 396 Feb 13 10:55 id_rsa.pub
-rw-r--r-- 1 oracle oinstall 395 Feb 13 10:54 known_hosts
B、把所有的节点的.pub内容追加到 一个文件里,并放在各个节点的.ssh目录下
[oracle@trsen01 .ssh]$ cat *.pub > authorized_keys
[oracle@trsen01 .ssh]$ ll -la
total 32
drwx------ 2 oracle oinstall 4096 Feb 13 10:58 .
drwx------ 4 oracle oinstall 4096 Feb 13 10:54 ..
-rw-r--r-- 1 oracle oinstall 1000 Feb 13 10:58 authorized_keys
-rw------- 1 oracle oinstall 668 Feb 13 10:55 id_dsa
-rw-r--r-- 1 oracle oinstall 604 Feb 13 10:55 id_dsa.pub
-rw------- 1 oracle oinstall 1671 Feb 13 10:55 id_rsa
-rw-r--r-- 1 oracle oinstall 396 Feb 13 10:55 id_rsa.pub
-rw-r--r-- 1 oracle oinstall 395 Feb 13 10:54 known_hosts
5、检查防火墙及selinux
[root@trsen01 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
[root@trsen01 ~]# getenforce
Disabled
七、安装cluster软件,并将cluster 从10.2.0.1升级到10.2.0.5即打UPS,利用xmanager工具进入节点1.
[root@trsen01 ~]# /u01/app/oracle/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oracle/oraInventory to 770.
Changing groupname of /u01/app/oracle/oraInventory to oinstall.
The execution of the script is complete
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/root.sh
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
assigning default hostname trsen01 for node 1.
assigning default hostname trsen02 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: trsen01 trsen01-priv trsen01
node 2: trsen02 trsen02-priv trsen02
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting device: /dev/raw/raw2
Format of 1 voting devices complete.
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
trsen01
CSS is inactive on these nodes.
trsen02
Local node checking complete.
Run root.sh on remaining nodes to start CRS daemons.
root@trsen01 ~]# tail -n 50 /u01/app/oracle/oraInventory/logs/installActions2014-02-13_12-35-26PM.log
Checking daemon liveness...
Liveness check passed for "CSS daemon".
Checking daemon liveness...
Liveness check passed for "EVM daemon".
Checking CRS health...
CRS health check passed.
CRS integrity check passed.
Checking node application existence...
Checking existence of VIP node application (required)
Check failed.
Check failed on nodes:
trsen01,trsen02
Checking existence of ONS node application (optional)
Check ignored.
Checking existence of GSD node application (optional)
Check ignored.
Post-check for cluster services setup was unsuccessful on all the nodes.
Command = /u01/app/oracle/product/10.2.0/crs_1/bin/cluvfy has failed
INFO: Configuration assistant "Oracle Cluster Verification Utility" failed
-----------------------------------------------------------------------------
*** Starting OUICA ***
Oracle Home set to /u01/app/oracle/product/10.2.0/crs_1
Configuration directory is set to /u01/app/oracle/product/10.2.0/crs_1/cfgtoollogs. All xml files under the directory will be processed
INFO: The "/u01/app/oracle/product/10.2.0/crs_1/cfgtoollogs/configToolFailedCommands" script contains all commands that failed, were skipped or were cancelled. This file may be used to run these configuration assistants outside of OUI. Note that you may have to update this script with passwords (if any) before executing the same.
-----------------------------------------------------------------------------
SEVERE: OUI-25031:Some of the configuration assistants failed. It is strongly recommended that you retry the configuration assistants at this time. Not successfully running any "Recommended" assistants means your system will not be correctly configured.
1. Check the Details panel on the Configuration Assistant Screen to see the errors resulting in the failures.
2. Fix the errors causing these failures.
3. Select the failed assistants and click the 'Retry' button to retry them.
INFO: User Selected: Yes/OK
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl check crs
CSS appears healthy
CRS appears healthy
EVM appears healthy
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl query crs softwareversion;
CRS software version on node [trsen01] is [10.2.0.1.0]
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl query crs activeversion;
CRS active version on the cluster is [10.2.0.1.0]
升级cluster软件至10.2.0.5来解决 VIP 失败的问题
[root@trsen01 ~]# ps -ef | grep d.b
root 5678 1 0 12:46 ? 00:00:01 /u01/app/oracle/product/10.2.0/crs_1/bin/crsd.bin reboot
oracle 5916 5671 0 12:47 ? 00:00:01 /u01/app/oracle/product/10.2.0/crs_1/bin/evmd.bin
oracle 6229 6207 0 12:47 ? 00:00:03 /u01/app/oracle/product/10.2.0/crs_1/bin/ocssd.bin
root 20020 2539 0 12:58 pts/1 00:00:00 grep d.b
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl stop crs
Stopping resources.
Successfully stopped CRS resources
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
[root@trsen01 ~]# ps -ef | grep d.b
root 20435 2539 0 12:59 pts/1 00:00:00 grep d.b
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/install/root102.sh
Creating pre-patch directory for saving pre-patch clusterware files
Completed patching clusterware files to /u01/app/oracle/product/10.2.0/crs_1
Relinking some shared libraries.
Relinking of patched files is complete.
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
Preparing to recopy patched init and RC scripts.
Recopying init and RC scripts.
Startup will be queued to init within 30 seconds.
Starting up the CRS daemons.
Waiting for the patched CRS daemons to start.
This may take a while on some systems.
.
.
.
.
.
.
.
10205 patch successfully applied.
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully deleted 1 values from OCR.
Successfully deleted 1 keys from OCR.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: trsen01 trsen01-priv trsen01
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
clscfg -upgrade completed successfully
Creating '/u01/app/oracle/product/10.2.0/crs_1/install/paramfile.crs' with data used for CRS configuration
Setting CRS configuration values in /u01/app/oracle/product/10.2.0/crs_1/install/paramfile.crs
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl query crs activeversion;
CRS active version on the cluster is [10.2.0.5.0]
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl query crs softwareversion;
CRS software version on node [trsen01] is [10.2.0.5.0]
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl check crs
CSS appears healthy
CRS appears healthy
EVM appears healthy
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/vipca
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....n01.gsd application ONLINE ONLINE trsen01
ora....n01.ons application ONLINE ONLINE trsen01
ora....n01.vip application ONLINE ONLINE trsen01
ora....n02.gsd application ONLINE ONLINE trsen02
ora....n02.ons application ONLINE ONLINE trsen02
ora....n02.vip application ONLINE ONLINE trsen02
八、装数据库软件
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/db_1/root.sh
Running Oracle10 root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/10.2.0/db_1
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
九、升级数据库软件
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl stop crs
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
[root@trsen01 tmp]# /u01/app/oracle/product/10.2.0/db_1/root.sh
Running Oracle 10g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/10.2.0/db_1
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
十、利用dbca创建数据库
[root@trsen01 tmp]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl start crs
Attempting to start CRS stack
The CRS stack will be started shortly
[root@trsen01 tmp]# /u01/app/oracle/product/10.2.0/crs_1/bin/crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE trsen01
ora....01.lsnr application ONLINE ONLINE trsen01
ora....n01.gsd application ONLINE ONLINE trsen01
ora....n01.ons application ONLINE ONLINE trsen01
ora....n01.vip application ONLINE ONLINE trsen01
ora....SM2.asm application ONLINE ONLINE trsen02
ora....02.lsnr application ONLINE ONLINE trsen02
ora....n02.gsd application ONLINE ONLINE trsen02
1、安装vbox最新版本,可在oracle官网上下载http://www.oracle.com/technetwork/server-storage/virtualbox/downloads/index.html,vbox位于win系统的安装,只要next就ok了,这里不多说了。
2、操作系统:redhat 5.5企业版
3、相关工具:Xmanager Enterprise 4、SecureCRT Version 6.5.0
4、存储:利用udev来做绑定raw
5、rac节点网络配置
节点名 网卡 ip地址 对应名称
trsen01 eth0 192.168.8.145 trsen01.oracle.com
eht1 192.168.8.147 trsen01-vip
10.1.1.8 trsen01-priv
trsen02 eth0 192.168.8.146 trsen02.oracle.com
eth1 192.168.8.148 trsen02-vip
10.1.1.9 trsen02-priv
6、rac存储配置
软件类型 文件类型 本地文件系统 raw asm
cluster CLUSTE SOFTWARE $CRS_HOME / /
OCR / /dev/raw/raw1(root/oinstall) /
VOTEDISK / /dev/raw/raw2(oracle/oinstall) /
database database software $ORACLE_HOME / +trsen_dg
7、硬件要求
至少1GB的物理内存,至少1GB的虚拟内存;
两台机器上需要两张网卡分别作公网和内网用;
二、在vbox上安装rhel 5.5操作系统,这里不废话,找AS去处理这个问题
三、检查两个节点的oracle所需的安装包并安装rpm包
1、检查rpm包
[root@trsen01 ~]# rpm -q binutils compat-db compat-libstdc++-296 control-center gcc gcc-c++ glibc glibc-common libstdc++ libstdc++-devel make sysstat setarch glibc-devel libaio ksh glibc-headers libgnome libgcc libgnomeui libgomp openmotif libXp --queryformat "%{N}-%{VERSION}.%{RELEASE} (%{ARCH})\n"
binutils-2.17.50.0.6.14.el5 (i386)
package compat-db is not installed
compat-libstdc++-296-2.96.138 (i386)
control-center-2.16.0.16.el5 (i386)
package gcc is not installed
package gcc-c++ is not installed
glibc-2.5.49 (i686)
glibc-common-2.5.49 (i386)
libstdc++-4.1.2.48.el5 (i386)
package libstdc++-devel is not installed
make-3.81.3.el5 (i386)
package sysstat is not installed
setarch-2.0.1.1 (i386)
package glibc-devel is not installed
libaio-0.3.106.5 (i386)
ksh-20100202.1.el5 (i386)
package glibc-headers is not installed
libgnome-2.16.0.6.el5 (i386)
libgcc-4.1.2.48.el5 (i386)
libgnomeui-2.16.0.5.el5 (i386)
package libgomp is not installed
package openmotif is not installed
package libXp is not installed
2、配置yum来安装所需的rpm包,安装完包过后,要mout的光盘给umount了。
[root@vrh3 ~]# mount /dev/cdrom /media/disk
mount: block device /dev/cdrom is write-protected, mounting read-only
touch /etc/yum.repos.d/public-yum-el5.repo,并加入以下内容
[rhel5]
name = Enterprise Linux 5.5 DVD
baseurl=file:///media/disk/Server/
gpgcheck=0
enabled=1
[root@trsen01 ~]# yum -y install compat-db gcc gcc-c++ libstdc++-devel sysstat glibc-devel glibc-headers libgomp openmotif libXp
Loaded plugins: rhnplugin, security
This system is not registered with RHN.
RHN support will be disabled.
rhel5 | 1.3 kB 00:00
rhel5/primary | 753 kB 00:00
rhel5 2348/2348
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package compat-db.i386 0:4.2.52-5.1 set to be updated
---> Package gcc.i386 0:4.1.2-48.el5 set to be updated
---> Package gcc-c++.i386 0:4.1.2-48.el5 set to be updated
---> Package glibc-devel.i386 0:2.5-49 set to be updated
---> Package glibc-headers.i386 0:2.5-49 set to be updated
--> Processing Dependency: kernel-headers >= 2.2.1 for package: glibc-headers
--> Processing Dependency: kernel-headers for package: glibc-headers
---> Package libXp.i386 0:1.0.0-8.1.el5 set to be updated
---> Package libgomp.i386 0:4.4.0-6.el5 set to be updated
---> Package libstdc++-devel.i386 0:4.1.2-48.el5 set to be updated
rhel5/filelists | 2.4 MB 00:00
---> Package openmotif.i386 0:2.3.1-2.el5_4.1 set to be updated
---> Package sysstat.i386 0:7.0.2-3.el5 set to be updated
--> Running transaction check
---> Package kernel-headers.i386 0:2.6.18-194.el5 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
============================================================================================================================
Package Arch Version Repository Size
============================================================================================================================
Installing:
compat-db i386 4.2.52-5.1 rhel5 1.7 M
gcc i386 4.1.2-48.el5 rhel5 5.2 M
gcc-c++ i386 4.1.2-48.el5 rhel5 3.4 M
glibc-devel i386 2.5-49 rhel5 2.0 M
glibc-headers i386 2.5-49 rhel5 601 k
libXp i386 1.0.0-8.1.el5 rhel5 22 k
libgomp i386 4.4.0-6.el5 rhel5 70 k
libstdc++-devel i386 4.1.2-48.el5 rhel5 2.8 M
openmotif i386 2.3.1-2.el5_4.1 rhel5 1.5 M
sysstat i386 7.0.2-3.el5 rhel5 170 k
Installing for dependencies:
kernel-headers i386 2.6.18-194.el5 rhel5 1.0 M
Transaction Summary
============================================================================================================================
Install 11 Package(s)
Upgrade 0 Package(s)
Total download size: 18 M
Downloading Packages:
----------------------------------------------------------------------------------------------------------------------------
Total 3.1 GB/s | 18 MB 00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : libXp 1/11
Installing : libgomp 2/11
Installing : openmotif 3/11
Installing : compat-db 4/11
Installing : sysstat 5/11
Installing : libstdc++-devel 6/11
Installing : kernel-headers 7/11
Installing : glibc-headers 8/11
Installing : glibc-devel 9/11
Installing : gcc 10/11
Installing : gcc-c++ 11/11
Installed:
compat-db.i386 0:4.2.52-5.1 gcc.i386 0:4.1.2-48.el5 gcc-c++.i386 0:4.1.2-48.el5 glibc-devel.i386 0:2.5-49
glibc-headers.i386 0:2.5-49 libXp.i386 0:1.0.0-8.1.el5 libgomp.i386 0:4.4.0-6.el5 libstdc++-devel.i386 0:4.1.2-48.el5
openmotif.i386 0:2.3.1-2.el5_4.1 sysstat.i386 0:7.0.2-3.el5
Dependency Installed:
kernel-headers.i386 0:2.6.18-194.el5
Complete!
四、两个节点配置oracle用户,网络及参数文件
A、添加用户组及用户并给与密码
[root@trsen01 ~]# groupadd -g 501 oinstall
[root@trsen01 ~]# groupadd -g 502 dba
[root@trsen01 ~]# groupadd -g 503 oper
[root@trsen01 ~]# useradd -u 501 -g oinstall -G dba,oper oracle
[root@trsen01 ~]# passwd oracle
Changing password for user oracle.
New UNIX password:
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
B、vi /etc/hosts,添加如下信息:
192.168.8.145 trsen01 trsen01.oracle.com
192.168.8.146 trsen02 trsen02.oracle.com
192.168.8.147 trsen01-vip
192.168.8.148 trsen02-vip
10.1.1.8 trsen01-priv
10.1.1.9 trsen02-priv
C、vi /etc/sysctl.conf的参数,添加如下信息
kernel.shmmax = 48719476736
kernel.shmall = 2294967296
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_default = 262144
net.core.wmem_max = 262144
D、vi /etc/security/limits.conf的参数,添加如下信息
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
vi/etc/pam.d/login的参数,添加如下信息
session required pam_limits.so
E、vi /etc/profile,添加如下信息
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
五、利用vbox配置共享存储
1、选中安装节点,“右击”-->"设置"-->"存储"-->"添加虚拟硬盘"-->”创建新虚拟盘“注意在创建的过程,选择"固定大小",这里我创建3个盘来做共享盘,这三个盘的大小一下。
2、选择“管理”-->"虚拟介质管理"-->"虚拟硬盘"-->选中药共享的盘,右击-->"修改"-->选择“可共享".
3、在另个节点上处理时,,“右击”-->"设置"-->"存储"-->"添加虚拟硬盘"-->"选择已有虚拟盘",选择在另个节点创建已共享的虚拟盘。
4、利用fdisk && udev规则做raw和asm盘的绑定
A、查看一下创建的三个盘分别是/dev/sdb /dev/sdc /dev/sdd
[root@trsen01 ~]# fdisk -l
Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 274 2096482+ 82 Linux swap / Solaris
/dev/sda3 275 1305 8281507+ 83 Linux
Disk /dev/sdb: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdc doesn't contain a valid partition table
Disk /dev/sdd: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdd doesn't contain a valid partition table
B、对sdb盘进行分区,做成放置ocr和voting disk的raw设备
[root@trsen01 ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): p
Disk /dev/sdb: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-783, default 1): 1
Last cylinder or +size or +sizeM or +sizeK (1-783, default 783): +3G
Command (m for help): p
Disk /dev/sdb: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 366 2939863+ 83 Linux
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (367-783, default 367):
Using default value 367
Last cylinder or +size or +sizeM or +sizeK (367-783, default 783): +3G
Command (m for help): p
Disk /dev/sdb: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 366 2939863+ 83 Linux
/dev/sdb2 367 732 2939895 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
C、查看分区后的/dev/sdb有两个分区
[root@trsen01 ~]# ll /dev/sd*
brw-r----- 1 root disk 8, 0 Feb 12 16:29 /dev/sda
brw-r----- 1 root disk 8, 1 Feb 12 16:30 /dev/sda1
brw-r----- 1 root disk 8, 2 Feb 12 16:29 /dev/sda2
brw-r----- 1 root disk 8, 3 Feb 12 16:30 /dev/sda3
brw-r----- 1 root disk 8, 16 Feb 12 16:39 /dev/sdb
brw-r----- 1 root disk 8, 17 Feb 12 16:39 /dev/sdb1
brw-r----- 1 root disk 8, 18 Feb 12 16:39 /dev/sdb2
brw-r----- 1 root disk 8, 32 Feb 12 16:29 /dev/sdc
brw-r----- 1 root disk 8, 48 Feb 12 16:29 /dev/sdd
D、利用udev规则将 /dev/sdb1 && /dev/sdb2绑定成/dev/raw/raw1 && /dev/raw/raw2
[root@trsen01 ~]# more /etc/udev/rules.d/60-raw.rules
# Enter raw device bindings here.
#
# An example would be:
# ACTION=="add", KERNEL=="sda", RUN+="/bin/raw /dev/raw/raw1 %N"
# to bind /dev/raw/raw1 to /dev/sda, or
# ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="1", RUN+="/bin/raw /dev/raw/raw2 %M %m"
# to bind /dev/raw/raw2 to the device with major 8, minor 1.
ACTION=="add", KERNEL=="sdb1", RUN+="/bin/raw /dev/raw/raw1 %N"
ACTION=="add", KERNEL=="sdb2", RUN+="/bin/raw /dev/raw/raw2 %N"
ACTION=="add", KERNEL=="raw*", OWNER=="oracle", GROUP=="oinstall", MODE=="0660"
E、利用下面脚本生成需要的 udev规则语句
[root@trsen01 ~]#for i in c d;
do
echo "KERNEL==\"sd*\", BUS==\"scsi\", PROGRAM==\"/sbin/scsi_id -g -u -s %p\", RESULT==\"`scsi_id -g -u -s /block/sd$i`\", NAME=\"asm-disk$i\", OWNER=\"oracle\", GROUP=\"oinstall\", MODE=\"0660\""
done
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VBce25369a-9fc8aff2_", NAME="asm-diskc", OWNER="oracle", GROUP="oinstall", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB9f38ff67-2fdb530d_", NAME="asm-diskd", OWNER="oracle", GROUP="oinstall", MODE="0660"
F、利用udev规则将/dev/sdc && /dev/sdd 做成asm磁盘
[root@trsen01 ~]# more /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VBce25369a-9fc8aff2_", NAME="asm-diskc",
OWNER="oracle", GROUP="oinstall", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB9f38ff67-2fdb530d_", NAME="asm-diskd
", OWNER="oracle", GROUP="oinstall", MODE="0660"
G、启动udev设备
[root@trsen01 ~]# /sbin/start_udev
Starting udev: [ OK ]
F、截至此,共享的raw设备及asm盘全部做好,查看
[root@trsen01 ~]# ll /dev/raw/*
crw-rw---- 1 oracle oinstall 162, 1 Feb 12 16:54 /dev/raw/raw1
crw-rw---- 1 oracle oinstall 162, 2 Feb 12 16:54 /dev/raw/raw2
[root@trsen01 ~]# ll /dev/asm*
brw-rw---- 1 oracle oinstall 8, 32 Feb 12 16:54 /dev/asm-diskc
brw-rw---- 1 oracle oinstall 8, 48 Feb 12 16:54 /dev/asm-diskd
注意:以上的D~F步骤,在所有节点上都要做。
六、配置oracle用户环境变量、创建相关目录、配置hangcheck-timer.ko 模块及ssh通过密钥验证、检查防火墙,selinux
1、配置oracle用户环境变量
[oracle@trsen02 ~]$ more ~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin
export ORACLE_SID=trsendb1
export ORACLE_BASE=/u01/app/oracle
export ORA_CRS_HOME=$ORACLE_BASE/product/10.2.0/crs_1
export ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1
export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
export PATH=$ORACLE_HOME/bin:$PATH
export TNS_ADMIN=$ORACLE_HOME/network/admin
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib:$ORACLE_HOME/oracm/lib:$ORACLE_HOME/lib
export LIBPATH=$LIBPATH:$ORACLE_HOME/lib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/jlib:$ORACLE_HOME/network/jlib:$ORACLE_HOME/jre/
export TEMP=/tmp
export PATH
2、创建文件夹
mkdir /u01/app -p
chown oracle:oinstall /u01 -R
3、配置hangcheck-timer.ko 模块
[root@trsen01 u01]# /sbin/insmod /lib/modules/2.6.18-194.el5/kernel/drivers/char/hangcheck-timer.ko hangcheck_tick=1 hangcheck_margin=10
[root@trsen01 u01]# lsmod | grep hang
hangcheck_timer 8025 0
4、配置ssh密钥验证通信
A、生成密钥
[oracle@trsen01 ~]$ cd .ssh
[oracle@trsen01 .ssh]$ ls
known_hosts
[oracle@trsen01 .ssh]$ ll -al
total 12
drwx------ 2 oracle oinstall 4096 Feb 13 10:54 .
drwx------ 4 oracle oinstall 4096 Feb 13 10:54 ..
-rw-r--r-- 1 oracle oinstall 395 Feb 13 10:54 known_hosts
[oracle@trsen01 .ssh]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
28:1c:ce:41:95:32:0c:68:44:bc:68:e5:f1:7b:d4:bf oracle@trsen01
[oracle@trsen01 .ssh]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
3e:ee:05:e2:b3:dc:11:62:fe:bc:89:d6:bf:39:6e:10 oracle@trsen01
[oracle@trsen01 .ssh]$ ll -al
total 28
drwx------ 2 oracle oinstall 4096 Feb 13 10:56 .
drwx------ 4 oracle oinstall 4096 Feb 13 10:54 ..
-rw------- 1 oracle oinstall 668 Feb 13 10:56 id_dsa
-rw-r--r-- 1 oracle oinstall 604 Feb 13 10:56 id_dsa.pub
-rw------- 1 oracle oinstall 1675 Feb 13 10:55 id_rsa
-rw-r--r-- 1 oracle oinstall 396 Feb 13 10:55 id_rsa.pub
-rw-r--r-- 1 oracle oinstall 395 Feb 13 10:54 known_hosts
B、把所有的节点的.pub内容追加到 一个文件里,并放在各个节点的.ssh目录下
[oracle@trsen01 .ssh]$ cat *.pub > authorized_keys
[oracle@trsen01 .ssh]$ ll -la
total 32
drwx------ 2 oracle oinstall 4096 Feb 13 10:58 .
drwx------ 4 oracle oinstall 4096 Feb 13 10:54 ..
-rw-r--r-- 1 oracle oinstall 1000 Feb 13 10:58 authorized_keys
-rw------- 1 oracle oinstall 668 Feb 13 10:55 id_dsa
-rw-r--r-- 1 oracle oinstall 604 Feb 13 10:55 id_dsa.pub
-rw------- 1 oracle oinstall 1671 Feb 13 10:55 id_rsa
-rw-r--r-- 1 oracle oinstall 396 Feb 13 10:55 id_rsa.pub
-rw-r--r-- 1 oracle oinstall 395 Feb 13 10:54 known_hosts
5、检查防火墙及selinux
[root@trsen01 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
[root@trsen01 ~]# getenforce
Disabled
七、安装cluster软件,并将cluster 从10.2.0.1升级到10.2.0.5即打UPS,利用xmanager工具进入节点1.
[root@trsen01 ~]# /u01/app/oracle/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oracle/oraInventory to 770.
Changing groupname of /u01/app/oracle/oraInventory to oinstall.
The execution of the script is complete
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/root.sh
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
assigning default hostname trsen01 for node 1.
assigning default hostname trsen02 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: trsen01 trsen01-priv trsen01
node 2: trsen02 trsen02-priv trsen02
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting device: /dev/raw/raw2
Format of 1 voting devices complete.
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
trsen01
CSS is inactive on these nodes.
trsen02
Local node checking complete.
Run root.sh on remaining nodes to start CRS daemons.
root@trsen01 ~]# tail -n 50 /u01/app/oracle/oraInventory/logs/installActions2014-02-13_12-35-26PM.log
Checking daemon liveness...
Liveness check passed for "CSS daemon".
Checking daemon liveness...
Liveness check passed for "EVM daemon".
Checking CRS health...
CRS health check passed.
CRS integrity check passed.
Checking node application existence...
Checking existence of VIP node application (required)
Check failed.
Check failed on nodes:
trsen01,trsen02
Checking existence of ONS node application (optional)
Check ignored.
Checking existence of GSD node application (optional)
Check ignored.
Post-check for cluster services setup was unsuccessful on all the nodes.
Command = /u01/app/oracle/product/10.2.0/crs_1/bin/cluvfy has failed
INFO: Configuration assistant "Oracle Cluster Verification Utility" failed
-----------------------------------------------------------------------------
*** Starting OUICA ***
Oracle Home set to /u01/app/oracle/product/10.2.0/crs_1
Configuration directory is set to /u01/app/oracle/product/10.2.0/crs_1/cfgtoollogs. All xml files under the directory will be processed
INFO: The "/u01/app/oracle/product/10.2.0/crs_1/cfgtoollogs/configToolFailedCommands" script contains all commands that failed, were skipped or were cancelled. This file may be used to run these configuration assistants outside of OUI. Note that you may have to update this script with passwords (if any) before executing the same.
-----------------------------------------------------------------------------
SEVERE: OUI-25031:Some of the configuration assistants failed. It is strongly recommended that you retry the configuration assistants at this time. Not successfully running any "Recommended" assistants means your system will not be correctly configured.
1. Check the Details panel on the Configuration Assistant Screen to see the errors resulting in the failures.
2. Fix the errors causing these failures.
3. Select the failed assistants and click the 'Retry' button to retry them.
INFO: User Selected: Yes/OK
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl check crs
CSS appears healthy
CRS appears healthy
EVM appears healthy
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl query crs softwareversion;
CRS software version on node [trsen01] is [10.2.0.1.0]
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl query crs activeversion;
CRS active version on the cluster is [10.2.0.1.0]
升级cluster软件至10.2.0.5来解决 VIP 失败的问题
[root@trsen01 ~]# ps -ef | grep d.b
root 5678 1 0 12:46 ? 00:00:01 /u01/app/oracle/product/10.2.0/crs_1/bin/crsd.bin reboot
oracle 5916 5671 0 12:47 ? 00:00:01 /u01/app/oracle/product/10.2.0/crs_1/bin/evmd.bin
oracle 6229 6207 0 12:47 ? 00:00:03 /u01/app/oracle/product/10.2.0/crs_1/bin/ocssd.bin
root 20020 2539 0 12:58 pts/1 00:00:00 grep d.b
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl stop crs
Stopping resources.
Successfully stopped CRS resources
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
[root@trsen01 ~]# ps -ef | grep d.b
root 20435 2539 0 12:59 pts/1 00:00:00 grep d.b
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/install/root102.sh
Creating pre-patch directory for saving pre-patch clusterware files
Completed patching clusterware files to /u01/app/oracle/product/10.2.0/crs_1
Relinking some shared libraries.
Relinking of patched files is complete.
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
Preparing to recopy patched init and RC scripts.
Recopying init and RC scripts.
Startup will be queued to init within 30 seconds.
Starting up the CRS daemons.
Waiting for the patched CRS daemons to start.
This may take a while on some systems.
.
.
.
.
.
.
.
10205 patch successfully applied.
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully deleted 1 values from OCR.
Successfully deleted 1 keys from OCR.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: trsen01 trsen01-priv trsen01
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
clscfg -upgrade completed successfully
Creating '/u01/app/oracle/product/10.2.0/crs_1/install/paramfile.crs' with data used for CRS configuration
Setting CRS configuration values in /u01/app/oracle/product/10.2.0/crs_1/install/paramfile.crs
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl query crs activeversion;
CRS active version on the cluster is [10.2.0.5.0]
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl query crs softwareversion;
CRS software version on node [trsen01] is [10.2.0.5.0]
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl check crs
CSS appears healthy
CRS appears healthy
EVM appears healthy
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/vipca
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....n01.gsd application ONLINE ONLINE trsen01
ora....n01.ons application ONLINE ONLINE trsen01
ora....n01.vip application ONLINE ONLINE trsen01
ora....n02.gsd application ONLINE ONLINE trsen02
ora....n02.ons application ONLINE ONLINE trsen02
ora....n02.vip application ONLINE ONLINE trsen02
八、装数据库软件
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/db_1/root.sh
Running Oracle10 root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/10.2.0/db_1
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
九、升级数据库软件
[root@trsen01 ~]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl stop crs
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
[root@trsen01 tmp]# /u01/app/oracle/product/10.2.0/db_1/root.sh
Running Oracle 10g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/10.2.0/db_1
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
十、利用dbca创建数据库
[root@trsen01 tmp]# /u01/app/oracle/product/10.2.0/crs_1/bin/crsctl start crs
Attempting to start CRS stack
The CRS stack will be started shortly
[root@trsen01 tmp]# /u01/app/oracle/product/10.2.0/crs_1/bin/crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE trsen01
ora....01.lsnr application ONLINE ONLINE trsen01
ora....n01.gsd application ONLINE ONLINE trsen01
ora....n01.ons application ONLINE ONLINE trsen01
ora....n01.vip application ONLINE ONLINE trsen01
ora....SM2.asm application ONLINE ONLINE trsen02
ora....02.lsnr application ONLINE ONLINE trsen02
ora....n02.gsd application ONLINE ONLINE trsen02
ora....n02.ons application ONLINE ONLINE trsen02
ora....n02.vip application ONLINE ONLINE trsen02
===========================
相关交流信息
QQ群: 330218614
Email: 623009431@qq.com
Blog: http://blog.csdn.net/trsenzhang
============================