测试环境VMware server 1.0, 建立2个虚拟机, 类型为Redhat Enterprise Linux 4, 每个虚拟机器分配700m内存, 一块12G硬盘, 2块网卡. 虚拟机操作系统安装完成后增加2块4G硬盘作为共享存储设备.
在本文中, 如未说明, 该操作必须在2台机器上都完成.
0 操作系统安装
Oracle9i需要禁止使用SELinux和防火墙, 这些设置可以在操作系统安装过程中完成.
安装操作系统时选择安装Legacy Network Server(并从detail中选择rsh-server), Development Tool, X Software Development, Gnome Software Development, Legancy Software Development可以安装Oracle rac需要的主要软件包.
安装libaio-devel-0.3.103-3.i386.rpm(在第三张光盘):
[root@rhel1 ~]# mount /media/cdrom/ mount: block device /dev/hdc is write-protected, mounting read-only [root@rhel1 ~]# cd /media/cdrom/ [root@rhel1 cdrom]# cd RedHat/RPMS/ [root@rhel1 RPMS]# rpm -ivh libaio-devel-0.3.103-3.i386.rpm warning: libaio-devel-0.3.103-3.i386.rpm: V3 DSA signature: NOKEY, key ID db42a60e Preparing... ########################################### [100%] 1:libaio-devel ########################################### [100%] |
安装gnome-libs-devel-1.4.1.2.90-44.1.i386.rpm(在第四张光盘)
[root@rhel1 RPMS]# rpm -ivh gnome-libs-devel-1.4.1.2.90-44.1.i386.rpm ORBit-devel-0.5.17-14.i386.rpm gtk+-devel-1.2.10-33.i386.rpm imlib-devel-1.9.13-23.i386.rpm glib-devel-1.2.10-15.i386.rpm warning: gnome-libs-devel-1.4.1.2.90-44.1.i386.rpm: V3 DSA signature: NOKEY, key ID db42a60e Preparing... ########################################### [100%] 1:glib-devel ########################################### [ 20%] 2:gtk+-devel ########################################### [ 40%] 3:imlib-devel ########################################### [ 60%] 4:ORBit-devel ########################################### [ 80%] 5:gnome-libs-devel ########################################### [100%] |
安装compat-oracle-rhel4-1.0-5.i386.rpm和compat-libcwait-2.0-2.i386.rpm(在oracle补丁4198954中)
[root@rhel1 ~]# unzip p4198954_21_LINUX.zip Archive: p4198954_21_LINUX.zip creating: 4198954/ inflating: 4198954/compat-oracle-rhel4-1.0-5.i386.rpm inflating: 4198954/compat-libcwait-2.0-2.i386.rpm inflating: 4198954/README.txt [root@rhel1 ~]# cd 4198954/ [root@rhel1 4198954]# rpm -ivh compat-libcwait-2.0-2.i386.rpm Preparing... ########################################### [100%] cat: /etc/ld.so.preload: No such file or directory 1:compat-libcwait ########################################### [100%] [root@rhel1 4198954]# cat /etc/ld.so.preload /usr/lib/libcwait.so [root@rhel1 4198954]# rpm -ivh compat-oracle-rhel4-1.0-5.i386.rpm Preparing... ########################################### [100%] 1:compat-oracle-rhel4 ########################################### [100%] |
1 建立oracle用户及需要的用户组
[root@rhel1 ~]# groupadd oinstall [root@rhel1 ~]# groupadd dba [root@rhel1 ~]# useradd oracle -g oinstall -G dba,oinstall [root@rhel1 ~]# passwd oracle Changing password for user oracle. New UNIX password: BAD PASSWORD: it is based on a dictionary word Retype new UNIX password: passwd: all authentication tokens updated successfully. [root@rhel1 ~]# id oracle uid=500(oracle) gid=501(oinstall) groups=501(oinstall),502(dba) [root@rhel1 ~]# id nobody uid=99(nobody) gid=99(nobody) groups=99(nobody) |
在2台机器上运行id oracle的结果必须相同, 否则需要用usermod或者直接修改/etc/passwd和/etc/group的方式使两之相同.
nobody是oracle apache服务器运行时使用的帐号, 必须在服务器上存在.
2 设置共享存储.
在第一台服务器上对共享磁盘进行分区. 分区后, fdisk -l结果如下:
[root@rhel1 ~]# fdisk -l Disk /dev/sda: 12.8 GB, 12884901888 bytes 255 heads, 63 sectors/track, 1566 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 13 104391 83 Linux /dev/sda2 14 1566 12474472+ 8e Linux LVM Disk /dev/sdb: 4294 MB, 4294967296 bytes 255 heads, 63 sectors/track, 522 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdb1 1 62 497983+ 83 Linux /dev/sdb2 63 124 498015 83 Linux /dev/sdb3 125 186 498015 83 Linux /dev/sdb4 187 522 2698920 5 Extended /dev/sdb5 187 205 152586 83 Linux /dev/sdb6 206 224 152586 83 Linux /dev/sdb7 225 243 152586 83 Linux /dev/sdb8 244 268 200781 83 Linux /dev/sdb9 269 281 104391 83 Linux /dev/sdb10 282 294 104391 83 Linux /dev/sdb11 295 307 104391 83 Linux /dev/sdb12 308 320 104391 83 Linux /dev/sdb13 321 333 104391 83 Linux /dev/sdb14 334 346 104391 83 Linux /dev/sdb15 347 359 104391 83 Linux Disk /dev/sdc: 4294 MB, 4294967296 bytes 255 heads, 63 sectors/track, 522 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdc1 1 19 152586 83 Linux /dev/sdc2 20 38 152617+ 83 Linux /dev/sdc3 39 57 152617+ 83 Linux /dev/sdc4 58 522 3735112+ 5 Extended /dev/sdc5 58 76 152586 83 Linux /dev/sdc6 77 95 152586 83 Linux /dev/sdc7 96 114 152586 83 Linux /dev/sdc8 115 176 497983+ 83 Linux /dev/sdc9 177 238 497983+ 83 Linux /dev/sdc10 239 251 104391 83 Linux /dev/sdc11 252 264 104391 83 Linux |
这些分区并不都是必须的, 在Metalink Notes:246205.1给出的必须建立的裸设备及其大小如下:
Raw Volume
|
File Size
|
Sample File Name
|
SYSTEM tablespace |
400 Mb
| db_name_raw_system_400m |
USERS tablespace |
120 Mb
| db_name_raw_users_120m |
TEMP tablespace |
100 Mb
| db_name_raw_temp_100m |
UNDOTBS tablespace per instance |
312 Mb
| db_name_raw_undotbsx_312m |
CWMLITE tablespace |
100 Mb
| db_name_raw_cwmlite_100m |
EXAMPLE |
160 Mb
| db_name_raw_example_160m |
OEMREPO |
20 Mb
| db_name_raw_oemrepo_20m |
INDX tablespace |
70 Mb
| db_name_raw_indx_70m |
TOOLS tablespace |
12 Mb
| db_name_raw_tools_12m |
DRYSYS tablespace |
90 Mb
| db_name_raw_drsys_90m |
First control file |
110 Mb
| db_name_raw_controlfile1_110m |
Second control file |
110 Mb
| db_name_raw_controlfile2_110m |
Two ONLINE redo log files per instance |
120 Mb x 2
| db_name_thread_lognumber_120m |
spfile.ora |
5 Mb
| db_name_raw_spfile_5m |
srvmconfig |
100 Mb
| db_name_raw_srvmconf_100m |
quorum | 20 Mb | quorum_raw_20m |
建立磁盘分区和裸设备的对映列表, 编辑文件/etc/sysconfig/rawdevices, 增加如下内容:
/dev/raw/raw11 /dev/sdb1 /dev/raw/raw12 /dev/sdb2 /dev/raw/raw13 /dev/sdb3 /dev/raw/raw15 /dev/sdb5 /dev/raw/raw16 /dev/sdb6 /dev/raw/raw17 /dev/sdb7 /dev/raw/raw18 /dev/sdb8 /dev/raw/raw19 /dev/sdb9 /dev/raw/raw20 /dev/sdb10 /dev/raw/raw21 /dev/sdb11 /dev/raw/raw22 /dev/sdb12 /dev/raw/raw23 /dev/sdb13 /dev/raw/raw24 /dev/sdb14 /dev/raw/raw25 /dev/sdb15 /dev/raw/raw31 /dev/sdc1 /dev/raw/raw32 /dev/sdc2 /dev/raw/raw33 /dev/sdc3 /dev/raw/raw35 /dev/sdc5 /dev/raw/raw36 /dev/sdc6 /dev/raw/raw37 /dev/sdc7 /dev/raw/raw38 /dev/sdc8 /dev/raw/raw39 /dev/sdc9 /dev/raw/raw40 /dev/sdc10 /dev/raw/raw41 /dev/sdc11 |
linux kernel 2.6 最多支持4096个裸设备. 所以我们可以选择其中的部分裸设备来使用, 上面的对映可以从裸设备的名称方便的知道它对映的磁盘分区. 此外, Extended partition(/dev/sdb4, /dev/sdc4) 不能被oracle使用, 也就不应该出现在爱/etc/sysconfig/rawdevices文件中.
设置裸设备的权限, rhel4使用udev来管理设备, 手动修改/dev/raw/raw??并不能持久有效. 要想使权限修改持久有效, 需要修改文件/etc/udev/permissions.d/50-udev.permissions, 编辑113行
raw/*:root:disk:0660 |
为
raw/*:oracle:oinstall:0660 |
重新启动系统, 可以使用ls -al /dev/raw/*来确认修改.
设置裸设备和数据文件的对映关系, 以oracle用户登录, 建立文件/home/oracle/raw_config, 内容如下
system=/dev/raw/raw11 spfile=/dev/raw/raw20 users=/dev/raw/raw17 temp=/dev/raw/raw21 undotbs1=/dev/raw/raw12 undotbs2=/dev/raw/raw13 example=/dev/raw/raw18 cwmlite=/dev/raw/raw41 xdb=/dev/raw/raw22 odm=/dev/raw/raw23 indx=/dev/raw/raw24 tools=/dev/raw/raw25 drsys=/dev/raw/raw40 control1=/dev/raw/raw15 control2=/dev/raw/raw16 redo1_1=/dev/raw/raw31 redo1_2=/dev/raw/raw32 redo2_1=/dev/raw/raw33 redo2_2=/dev/raw/raw35 srvmconfig=/dev/raw/raw36 quorum=/dev/raw/raw19 |
3 设置网络, 修改/etc/hosts文件, 修改后hosts文件内容如下:
# Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 localhost.localdomain localhost 192.168.1.141 rhel1 192.168.1.142 rhel2 10.10.10.141 rhel1-priv 10.10.10.142 rhel2-priv |
注意127.0.0.1后不应该有本机的主机名称. 在任一主机上ping两台主机的4个主机名都应该可以ping通, 如果无法ping通, 请检测网络设置.
启动rsh和rlogin服务
[root@rhel2 ~]# chkconfig rsh on [root@rhel2 ~]# chkconfig rlogin on [root@rhel2 ~]# service xinetd restart Stopping xinetd: [ OK ] Starting xinetd: [ OK ] |
建立文件/etc/hosts.equiv, 内容如下:
rhel1 rhel2 rhel1-priv rhel2-priv |
设置/etc/hostss.equiv属主为root, 模式为600, 即只允许root用户读写该文件.
切换到oracle用户, 在每台服务器上分别测试rcp可以从2台主机的4个主机名复制文件.
4 检查oracle硬件需求.
Oracle需要的最小硬件需求为
内存: 512M
swap分区, 1G和内存大小x2中的大者
/tmp所在分区, 300M
oracle安装目录所在分区3.5G
检查过程如下
[root@rhel2 ~]# cat /proc/meminfo |grep "Total" MemTotal: 706628 kB HighTotal: 0 kB LowTotal: 706628 kB SwapTotal: 1441784 kB VmallocTotal: 303096 kB HugePages_Total: 0 [root@rhel2 ~]# df -m Filesystem 1M-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 10552 2446 7571 25% / /dev/sda1 99 9 85 10% /boot none 346 0 346 0% /dev/shm |
可查得内存大小为700M, swap大小为1.4g, root分区剩余空间为7.5g, /tmp和oracle安装分区都放在root分区上.
5 检查oracle软件需求.
安装oracle前需要安装如下软件包
compat-db-4.1.25-9 compat-gcc-32-3.2.3-47.3 compat-gcc-32-c++-3.2.3-47.3 compat-oracle-rhel4-1.0-3 compat-libcwait-2.0-1 compat-libgcc-296-2.96-132.7.2 compat-libstdc++-296-2.96-132.7.2 compat-libstdc++-33-3.2.3-47.3 gcc-3.4.3-9.EL4 gcc-c++-3.4.3-9.EL4 gnome-libs-1.4.1.2.90-44 gnome-libs-devel-1.4.1.2.90-44 libaio-devel-0.3.102-1 libaio-0.3.102-1 make-3.80-5 openmotif21-2.1.30-11 xorg-x11-deprecated-libs-devel-6.8.1-23.EL xorg-x11-deprecated-libs-6.8.1-23.EL rsh-server-0.17-5 |
6 设置内核变量, 将如下内容添加到/etc/sysctl.conf
kernel.shmall = 2097152 kernel.shmmax = 2147483648 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 fs.file-max = 65536 net.ipv4.ip_local_port_range = 1024 65000 net.core.rmem_default = 262144 net.core.rmem_max = 262144 net.core.wmem_default = 262144 net.core.wmem_max = 262144 |
然后运行sysctl -p或重新启动机器使设置生效.
7 配置hangcheck-timer, 将下面命令添加到/etc/rc.d/rc.local, 并运行这一命令活着重启机器, 加载hangcheck-timer模块
modprobe hangcheck-timer hangcheck_tick=30 hangcheck_margin=180 |
8 建立oracle安装目录
[root@rhel1 ~]# mkdir -p /u01/app/oracle [root@rhel1 ~]# chown -R oracle:oinstall /u01/app/oracle [root@rhel1 ~]# chmod 755 -R /u01/app/oracle |
9 设置oracle用户环境变量, 以oracle用户登录, 编辑.bash_profile文件, 增加如下内容
export ORACLE_BASE=/u01/app/oracle export ORACLE_HOME=$ORACLE_BASE/product/9.2.0 export PATH=$ORACLE_HOME/bin:$PATH export DBCA_RAW_CONFIG=~/raw_config export LD_ASSUME_KERNEL=2.4.19 export ORACLE_SID=orcl1 |
在rhel2上最后一行为
export ORACLE_SID=orcl2 |
10 安装oracm
在rhel1上以oracle身份登录, 运行Disk1下的runInstaller开始安装oracle
安装时选择Oracle Cluster Manager 9.2.0.4,同时可在Product Language中增加Simplified chinese
输入public node information时在public node1和public node2 中分别输入rhel1和rhel2
输入private node information时在private node1和private node2中分别输入rhel1-priv和rhel2-priv
quorum disk information输入/dev/raw/raw19
安装完成后退出
11 配置运行oracm
修改文件/u01/app/oracle/product/9.2.0/oracm/admin/cmcfg.ora, 添加
KernelModuleName=hangcheck-timer |
修改MissCount=20为MissCount=210
以root身份登录, 运行/u01/app/oracle/product/9.2.0/oracm/bin/ocmstart.sh
[root@rhel1 bin]# export ORACLE_HOME=/u01/app/oracle/product/9.2.0 [root@rhel1 bin]# ./ocmstart.sh oracm </dev/null 2>&1 >/u01/app/oracle/product/9.2.0/oracm/log/cm.out & [root@rhel1 bin]# ps -ef |grep cm root 7201 1 0 23:09 pts/1 00:00:00 oracm root 7211 3358 0 23:09 pts/1 00:00:00 grep cm [root@rhel1 bin]# |
12 安装oracle rac
在rhel1上以oracle身份登录, 运行Disk1下的runInstaller开始安装oracle
选择节点时使用ctrl键选择rhel1和rhel2, 如果没有出现节点选择的画面, 请检查oracm
是否启动.
安装时选择Oracle Database 9.2.0.4.0, 同时可在Product Language中增加Simplified chinese.
安装类型选择Enterprise Edition
选择不创建数据库.
shared configuration field中输入/dev/raw/raw36
安装过程中需要以root身份运行/u01/app/oracle/product/9.2.0/root.sh, 运行这个脚本前, 请建立目录/var/opt/oracle
13 启动gsd
以oracle身份运行gsdctl start启动gsd
[root@rhel2 log]# su - oracle [oracle@rhel2 ~]$ gsdctl start Successfully started GSD on local node [oracle@rhel2 ~]$ gsdctl stat GSD is running on the local node [oracle@rhel2 ~]$ |
14 建立监听器配置文件及tnsnames.ora文件
在rhel1上运行netca使用默认值建立监听器
建立文件/u01/app/oracle/product/9.2.0/network/admin/tnsnames.ora, 内容如下:
listener_orcl1= (DESCRIPTION= (ADDRESS=(PROTOCOL=tcp)(HOST=rhel1)(PORT=1521))) listener_orcl2= (DESCRIPTION= (ADDRESS=(PROTOCOL=tcp)(HOST=rhel2)(PORT=1521))) listeners_orcl= (DESCRIPTION= (ADDRESS=(PROTOCOL=tcp)(HOST=rhel1)(PORT=1521)) (ADDRESS=(PROTOCOL=tcp)(HOST=rhel2)(PORT=1521))) orcl= (description= (ADDRESS=(PROTOCOL=tcp)(HOST=rhel1)(PORT=1521)) (ADDRESS=(PROTOCOL=tcp)(HOST=rhel2)(PORT=1521)) (connect_data= (service_name=orcl) (failover_mode= (type=select)(method=basic)(retries=20)(delay=15)))) orcl1= (description= (ADDRESS=(PROTOCOL=tcp)(HOST=rhel1)(PORT=1521)) (connect_data= (sid=orcl1))) orcl2= (description= (ADDRESS=(PROTOCOL=tcp)(HOST=rhel2)(PORT=1521)) (connect_data= (sid=orcl2))) |
15 建立数据库
在rhel1上运行dbca建立数据库
节点选择时使用Select All选择全部节点
数据库类型选择new database
全局名称输入orcl
初试化参数中可根据需要选择不同的字符集.
create option中选择建立数据库脚本, 取消选择创建数据库. 如果直接使用dbca建立数据库会提示ocr错误, 可能是oracle 的bug, 没有做进一步分析.
在rhel1上以oracle身份运行/u01/app/oracle/admin/orcl/scripts/orcl1.sh创建数据库, 这个过程需要比较长的时间, 在我的机器上运行了将近2个小时. 在rhel上以oracle身份在/u01/app/oracle/admin/orcl/下建立目录bdump, cdump, udump, 使用
/u01/app/oracle/product/9.2.0/bin/orapwd file=/u01/app/oracle/product/9.2.0/dbs/orapworcl2 password=change_on_install |
生成密码文件后就可以启动数据库.
参考资料:
Metalink Notes:246205.1 Configuring Raw Devices for Real Application Clusters on Linux
Metalink Notes:184821.1 Step-By-Step Installation of 9.2.0.5 RAC on Linux
Metalink Notes:189256.1 Script to Verify Installation Requirements for Oracle 8.0.5 to 9.2 versions of RDBMS
Oracle9i Release Notes Release 2 (9.2.0.4.0) for Linux x86
Oracle9i Real Application Clusters Setup and Configuration