11g Rac 配置 Linux

http://blog.csdn.net/seulk_q/article/details/42612393 

环境如下：

Linux操作系统：Centos 6.5 64bit （这个版本的redhat 6内核等OS在安装grid最后执行root.sh时会出现crs-4124，是oracle11.2.0.1的bug）

VMware version：Workstation 8.0.3 build-703057

Oracle版本：11G R2 grid/oracle 64位 11.2.0.1.0 for linux

步骤规划：

1，配置虚拟机、安装Linux操作系统

2，配置OS

3，配置ASM

4，安装Oracle grid

5，安装Oracle database

6，创建ASM磁盘组

7，建立数据库实例

1，配置虚拟机、安装Linux操作系统

1.1  新建虚拟机：RAC1和RAC2，并安装OS

下载Centos的iso后新建虚拟机后安装即可，这里略。注意内存分配，每个虚拟机至少1.5G内存比较好，我的机器8G内存，给每个虚拟机分了2G，给每个虚拟机分配50GB硬盘容量，配置2个桥接网卡。

1.2   配置Share Disks

新建共享硬盘，在DOS VM安装目录下执行：

cd \D F:/Software/vm8/

vmware-vdiskmanager.exe -c -s 1000Mb -a lsilogic -t 2 K:\VMDisk\Sharedisk\ocr.vmdk
vmware-vdiskmanager.exe -c -s 1000Mb -a lsilogic -t 2 K:\VMDisk\Sharedisk\ocr2.vmdk
vmware-vdiskmanager.exe -c -s 1000Mb -a lsilogic -t 2 K:\VMDisk\Sharedisk\votingdisk.vmdk
vmware-vdiskmanager.exe -c -s 20000Mb -a lsilogic -t 2 K:\VMDisk\Sharedisk\data.vmdk
vmware-vdiskmanager.exe -c -s 10000Mb -a lsilogic -t 2 K:\VMDisk\Sharedisk\backup.vmdk

配置虚拟机RAC1 RAC2目录下的vmx虚拟机配置文件，在行最后添加如下：

scsi1.present = "TRUE"
scsi1.virtualDev = "lsilogic"
scsi1.sharedBus = "virtual"

scsi1:1.present = "TRUE"
scsi1:1.mode = "independent-persistent"
scsi1:1.filename = "K:\VMDisk\Sharedisk\ocr.vmdk"
scsi1:1.deviceType = "plainDisk"

scsi1:2.present = "TRUE"
scsi1:2.mode = "independent-persistent"
scsi1:2.filename = "K:\VMDisk\Sharedisk\votingdisk.vmdk"
scsi1:2.deviceType = "plainDisk" 

scsi1:3.present = "TRUE"
scsi1:3.mode = "independent-persistent"
scsi1:3.filename = "K:\VMDisk\Sharedisk\data.vmdk"
scsi1:3.deviceType = "plainDisk"

scsi1:4.present = "TRUE"
scsi1:4.mode = "independent-persistent"
scsi1:4.filename = "K:\VMDisk\Sharedisk\backup.vmdk"
scsi1:4.deviceType = "plainDisk"

scsi1:5.present = "TRUE"
scsi1:5.mode = "independent-persistent"
scsi1:5.filename = "K:\VMDisk\Sharedisk\ocr2.vmdk"
scsi1:5.deviceType = "plainDisk"

disk.locking = "false"
diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.DataCacheMinReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0"

保存退出。

最后虚拟机图大致如下：

再次提醒注意内存、硬盘和网卡配置。

配置虚拟机、安装Linux操作系统完成。

2，配置OS

以下没有特别说明都需要同时在两个节点执行。

2.1 关闭防火墙、selinux(考虑到后面安装时会碰到一些情况，所以需要执行这步骤）

关闭selinux
setenforce 0   
[root@rac1 ~]# vim /etc/selinux/config
SELINUX=disabled   需要重启机器后才会生效！！！！

关闭防火墙
service iptables stop

chkconfig iptables off

2.2 配置网络

修改节点主机名为rac1，rac2

hostname rac1 

hostname rac2

或修改/etc/sysconfig/network

RAC1

eth0: 192.168.1.130

eth1: 10.10.17.221

RAC2

eth0: 192.168.1.140

eth1: 10.10.17.222

host文件，两个节点都配置

 Vim /etc/hosts

192.168.1.130 rac1
192.168.1.140 rac2

192.168.1.131 rac1-vip
192.168.1.141 rac2-vip

10.10.17.221 rac1-priv
10.10.17.222 rac2-priv

192.168.1.200 scan-ip

保存退出。

配置完之后互相ping看看是否连通。

2.3 配置CTSS服务、节点SSH等效

CTSS

   需要把ntp服务停止和删除配置文件及pid文件

   service ntpd stop

   chkconfig ntpd off

   rm -rf /etc/ntpd.conf 

   rm -rf /var/run/ntpd.pid

  查看ctss服务资源

   crsctl check ctss

   

SSH

在grid安装时会提供配置步骤，所以不需要手工配置。

ssh具体配置

rac1 rac2 和oracle/grid用户都需要配置SSH
mkdir ~/.ssh
chmod 700 ~/.ssh
ssh-keygen -t rsa
ssh-keygen -t dsa

rac1 rac2
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

测试SSH

ssh rac2

ssh rac1

配置完成后要用以下名命令首次运行配置生效：

1. ssh rac1 date
2. ssh rac2 date
3. ssh rac1-priv date
4. ssh rac2-priv date

补充以下是我见过最全的SSH配置方法！！！！！！

1. 配置过程如下：
2. 各节点生成Keys：
3. [root@rac1 ~]# su - oracle
4. [oracle@rac1 ~]$ mkdir ~/.ssh
5. [oracle@rac1 ~]$ chmod 700 ~/.ssh
6. [oracle@rac1 ~]$ ssh-keygen -t rsa
7. [oracle@rac1 ~]$ ssh-keygen -t dsa
8. [root@rac2 ~]# su - oracle
9. [oracle@rac2 ~]$ mkdir ~/.ssh
10. [oracle@rac2 ~]$ chmod 700 ~/.ssh
11. [oracle@rac2 ~]$ ssh-keygen -t rsa
12. [oracle@rac2 ~]$ ssh-keygen -t dsa
13. 
    
14. 在节点1上进行互信配置：
15. [oracle@rac1 ~]$ touch ~/.ssh/authorized_keys
16. [oracle@rac1 ~]$ cd ~/.ssh
17. [oracle@rac1 .ssh]$ ssh rac1 cat ~/.ssh/id_rsa.pub >> authorized_keys
18. [oracle@rac1 .ssh]$ ssh rac2 cat ~/.ssh/id_rsa.pub >> authorized_keys
19. [oracle@rac1 .ssh]$ ssh rac1 cat ~/.ssh/id_dsa.pub >> authorized_keys
20. [oracle@rac1 .ssh]$ ssh rac2 cat ~/.ssh/id_dsa.pub >> authorized_keys
21. 
    
22. 在rac1把存储公钥信息的验证文件传送到rac2上
23. [oracle@rac1 .ssh]$ pwd
24. /home/oracle/.ssh
25. [oracle@rac1 .ssh]$ scp authorized_keys rac2:'pwd'
26. oracle@rac2's password:
27. authorized_keys 100% 1644 1.6KB/s 00:00
28. 
    
29. 设置验证文件的权限
30. 在每一个节点执行：
31. $ chmod 600 ~/.ssh/authorized_keys
32. 
    
33. 启用用户一致性
34. 在你要运行OUI的节点以oracle用户运行(这里选择rac1)：
35. [oracle@rac1 .ssh]$ exec /usr/bin/ssh-agent $SHELL
36. [oracle@rac1 .ssh]$ ssh-add
37. Identity added: /home/oracle/.ssh/id_rsa (/home/oracle/.ssh/id_rsa)
38. Identity added: /home/oracle/.ssh/id_dsa (/home/oracle/.ssh/id_dsa)
39. 
    
40. 验证ssh配置是否正确
41. 以oracle用户在所有节点分别执行：
42. ssh rac1 date
43. ssh rac2 date
44. ssh rac1-priv date
45. ssh rac2-priv date
46. 
    
47. 如果不需要输入密码就可以输出时间，说明ssh验证配置成功。必须把以上命令在两个节点都运行，每一个命令在第一次执行的时候需要输入yes。
48. 如果不运行这些命令，即使ssh验证已经配好，安装clusterware的时候也会出现错误：
49. The specified nodes are not clusterable
50. 因为，配好ssh后，还需要在第一次访问时输入yes，才算是真正的无障碍访问其他服务器。

2.4 配置Oracle软件安装用户、组、环境变量、安装目录

#组
 groupadd -g 1300 dba
 groupadd -g 1301 oper
 groupadd -g 1000 oinstall
 groupadd -g 1200 asmadmin
 groupadd -g 1201 asmdba
 groupadd -g 1202 asmoper

#用户
useradd -m -u 1100 -g oinstall -G asmadmin,asmdba,asmoper -s /bin/bash grid  

useradd -m -u 1101 -g oinstall -G dba,oper,asmdba -s /bin/bash oracle
#用户环境变量修改home下的 .bash_profile

ORACLE

 export TMP=/tmp 
 export TMPDIR=$TMP 
 export ORACLE_SID=RAC1  # RAC1
 export ORACLE_SID=RAC2  # RAC2
 export ORACLE_BASE=/u01/app/oracle 
 export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
 export TNS_ADMIN=$ORACLE_HOME/network/admin 
 export PATH=/usr/sbin:$PATH 
 export PATH=$ORACLE_HOME/bin:$PATH 
 export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib 
 export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib 
 umask 022 

 GRID
 export TMP=/tmp 
 export TMPDIR=$TMP 
 export ORACLE_SID=+ASM1  # RAC1

 export ORACLE_SID=+ASM2  # RAC2

 export ORACLE_BASE=/u01/app/grid
 export ORACLE_HOME=/u01/app/11.2.0/grid
 export PATH=/usr/sbin:$PATH 
 export PATH=$ORACLE_HOME/bin:$PATH 
 export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib 
 export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib 
 umask 022 

配置目录
 mkdir -p  /u01/app/11.2.0/grid
 mkdir -p /u01/app/grid
 mkdir -p /u01/app/oracle
 chown grid:oinstall /u01/app/11.2.0/grid
 chown grid:oinstall /u01/app/grid
 chown oracle:oinstall /u01/app/oracle
 chmod -R 775 /u01/
 chown -R grid:oinstall /u01

2.5 配置系统文件

配置oracle、grid用户的shell限制

vim /etc/security/limits.conf

添加

grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536

修改/etc/pam.d/login配置文件

Vim  /etc/pam.d/login 
添加 session    required     pam_limits.so

修改内核配置文件/etc/sysctl.conf

 fs.aio-max-nr = 1048576 
 fs.file-max = 6815744 
 kernel.shmall = 2097152 
 kernel.shmmax = 1054472192 
 kernel.shmmni = 4096 
 kernel.sem = 250 32000 100 128 
 net.ipv4.ip_local_port_range = 9000 65500 
 net.core.rmem_default = 262144 
 net.core.rmem_max = 4194304 
 net.core.wmem_default = 262144 
 net.core.wmem_max = 1048586 
 net.ipv4.tcp_wmem = 262144 262144 262144 
 net.ipv4.tcp_rmem = 4194304 4194304 4194304 
保存退出后执行 sysctl -p 立即生效。

2.6 配置共享磁盘 （在一个节点执行即可）

查看分区

ls /dev/sd*

使用fdisk进行分区，其他区按照自己的规划划分就可以了，例：

fdisk /dev/sdb

[root@rac1 ~]# fdisk /dev/sdb

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1000, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1000, default 1000): +500M

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (502-1000, default 502):
Using default value 502
Last cylinder, +cylinders or +size{K,M,G} (502-1000, default 1000):
Using default value 1000

Command (m for help): w

The partition table has been altered!

最后配置完后查看分区

fidks -l

[root@rac1 ~]# ls /dev/sd*
/dev/sda  /dev/sda1  /dev/sda2  /dev/sdb  /dev/sdb1  /dev/sdc  /dev/sdc1  /dev/sdd  /dev/sdd1  /dev/sdd2  /dev/sde  /dev/sde1  /dev/sde2  /dev/sdf  /dev/sdf1

2.7 安装Oracle grid等需要的包

binutils-2.20.51.0.2-5.11.el6 (x86_64)
compat-libcap1-1.10-1 (x86_64)
compat-libstdc++-33-3.2.3-69.el6 (x86_64)
compat-libstdc++-33-3.2.3-69.el6.i686
gcc-4.4.4-13.el6 (x86_64)
gcc-c++-4.4.4-13.el6 (x86_64)
glibc-2.12-1.7.el6 (i686)
glibc-2.12-1.7.el6 (x86_64)
glibc-devel-2.12-1.7.el6 (x86_64)
glibc-devel-2.12-1.7.el6.i686
ksh
libgcc-4.4.4-13.el6 (i686)
libgcc-4.4.4-13.el6 (x86_64)
libstdc++-4.4.4-13.el6 (x86_64)
libstdc++-4.4.4-13.el6.i686
libstdc++-devel-4.4.4-13.el6 (x86_64)
libstdc++-devel-4.4.4-13.el6.i686
libaio-0.3.107-10.el6 (x86_64)
libaio-0.3.107-10.el6.i686
libaio-devel-0.3.107-10.el6 (x86_64)
libaio-devel-0.3.107-10.el6.i686
make-3.81-19.el6
sysstat-9.0.4-11.el6 (x86_64)

3，配置ASM

3.1 查看操作系统内核版本对应下载

uname -a

下载oracleasm http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html

安装

yum -y install kmod-oracleasm.x86_64

rpm -ivh oracleasmlib-2.0.4-1.el6.x86_64.rpm 
rpm -ivh oracleasm-support-2.1.8-1.el6.x86_64.rpm

3.2 配置ASM磁盘

在两个节点上执行

[root@rac1 oracle]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: grid
Default group to own the driver interface []: asmdba
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]:
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver:                     [  OK  ]
Scanning the system for Oracle ASMLib disks:               [  OK  ]
[root@rac1 oracle]# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks:               [  OK  ]

[root@rac1 ~]# /etc/init.d/oracleasm listdisks
VOL1
VOL2
VOL3
VOL4
VOL5
VOL6
VOL7

注：如果没有查看到asm磁盘组需要查看之前的selinux是否有禁用。

3.2 oracleasm createdisk

3.3 ssh 配置grid oracle

4，安装Oracle grid

安装前可以执行Oracle提供的检查环境工

[grid@node1 grid_sw]$ ./runcluvfy.sh stage -pre crsinst -n node1,node2 -fixup -verbose

我见过最有用的安装GI之前检查修复工具

./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -fixup -verbose 

4.1 在安装目录下执行./runinstaller，如果出现安装grid提示must be configured to display at least 256 colors Failed，就执行如下
 root 下先执行#xhost +
 su  -  grid
 export DISPLAY=:0.0

注意在此建立SSH等效。执行setup后点test测试。如果不成功就手工配置SSH再进行测试。

这个地方忘记截图了，就是类似这个图。定义磁盘组名字，这里我选择normal冗余，需要选择3个VOL。

这里是把OCR和votingdisk安装在所选磁盘。后面还需要新建asm磁盘组来安装其他数据文件。

 这里的software location应该是 /u01/app/11.2.0/grid

通过Oracle的系统环境检查，安装缺失的包和修改系统环境参数等。

安装65%的时候可能会出现卡顿的情况，这个时候需要检查防火墙和selinux是否关闭，详细情况可以点击detail查看安装进度，如出现拷贝文件到RAC2节点的信息则正常。

根据提示执行orainstRoot.sh和root.sh脚本，先执行rac1后再执行rac2。

在执行root .sh时可能在的时候会提示出现crs-4124，这个是oracle的bug，当在redhat 6等OS下安装11.2.0.1版本Oracle的时候会出现此错误。

解决如下：

在执行root.sh时，在出现Adding daemon to inittab时，需要不停地执行命令：

/bin/dd if=/var/tmp/.oracle/npohasd of=/dev/null bs=1024 count=1

出错后，可以回退已安装的root.sh脚本，方法为：

CRS-4124: Oracle High Availability Services startup failed.
CRS-4000: Command Start failed, or completed with errors.
这是11.2.0.1的BUG
原因：/var/tmp/.oracle/npohasd管道文件的权限不正确。
解决办法：
一
删除配置：/u01/app/11.2.0/grid/crs/install/roothas.pl -deconfig -force-verbose
二

/bin/dd if=/var/tmp/.oracle/npohasd of=/dev/null bs=1024 count=1
如果出现
/bin/dd: opening`/var/tmp/.oracle/npohasd': No such file or directory

的时候文件还没生成就继续执行，直到能执行为止，一般出现Adding daemon to inittab这条信息的时候执行dd命令。
三
执行root.sh

[root@rac1 grid]# ./root.sh
Running Oracle 11g root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2015-01-06 23:46:35: Parsing the host name
2015-01-06 23:46:35: Checking for super user privileges
2015-01-06 23:46:35: User has super user privileges
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
Adding daemon to inittab

CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
ADVM/ACFS is not supported on centos-release-6-6.el6.centos.12.2.x86_64



CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded

ASM created and started successfully.

DiskGroup DATA created successfully.

clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk 60b5c5168a574f3fbfabf7d93a17c18f.
Successful addition of voting disk a59fab87e8634f00bf5b44656d6caaf1.
Successful addition of voting disk 228f0986e7c84f20bf66ed6905b69e43.
Successfully replaced voting disk group with +DATA.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   60b5c5168a574f3fbfabf7d93a17c18f (ORCL:VOL1) [DATA]
 2. ONLINE   a59fab87e8634f00bf5b44656d6caaf1 (ORCL:VOL2) [DATA]
 3. ONLINE   228f0986e7c84f20bf66ed6905b69e43 (ORCL:VOL4) [DATA]
Located 3 voting disk(s).
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'rac1'
CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.DATA.dg' on 'rac1'
CRS-2676: Start of 'ora.DATA.dg' on 'rac1' succeeded

rac1     2015/01/06 23:52:15     /u01/app/11.2.0/grid/cdata/rac1/backup_20150106_235215.olr
Preparing packages for installation...
cvuqdisk-1.0.7-1
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Updating inventory properties for clusterware
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 3906 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/oraInventory
'UpdateNodeList' was successful.

root.sh执行完毕。在rac2节点执行也需要此步骤。

检查grid组件启动情况
[grid@rac1 ~]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[grid@rac1 ~]$ crs_stat -v -t
Name           Type           R/RA   F/FT   Target    State     Host
----------------------------------------------------------------------
ora.DATA.dg    ora....up.type 0/5    0/     ONLINE    ONLINE    rac1
ora....ER.lsnr ora....er.type 0/5    0/     ONLINE    ONLINE    rac1
ora....N1.lsnr ora....er.type 0/5    0/0    ONLINE    ONLINE    rac1
ora.asm        ora.asm.type   0/5    0/     ONLINE    ONLINE    rac1
ora.eons       ora.eons.type  0/3    0/     ONLINE    ONLINE    rac1
ora.gsd        ora.gsd.type   0/5    0/     OFFLINE   OFFLINE
ora....network ora....rk.type 0/5    0/     ONLINE    ONLINE    rac1
ora.oc4j       ora.oc4j.type  0/5    0/0    OFFLINE   OFFLINE
ora.ons        ora.ons.type   0/3    0/     ONLINE    ONLINE    rac1
ora....SM1.asm application    0/5    0/0    ONLINE    ONLINE    rac1
ora....C1.lsnr application    0/5    0/0    ONLINE    ONLINE    rac1
ora.rac1.gsd   application    0/5    0/0    OFFLINE   OFFLINE
ora.rac1.ons   application    0/3    0/0    ONLINE    ONLINE    rac1
ora.rac1.vip   ora....t1.type 0/0    0/0    ONLINE    ONLINE    rac1
ora....SM2.asm application    0/5    0/0    ONLINE    ONLINE    rac2
ora....C2.lsnr application    0/5    0/0    ONLINE    ONLINE    rac2
ora.rac2.gsd   application    0/5    0/0    OFFLINE   OFFLINE
ora.rac2.ons   application    0/3    0/0    ONLINE    ONLINE    rac2
ora.rac2.vip   ora....t1.type 0/0    0/0    ONLINE    ONLINE    rac2
ora.scan1.vip  ora....ip.type 0/0    0/0    ONLINE    ONLINE    rac1

备注：

  在重启节点后出现Oracle has不能通信的情况

1，[root@rac1 ~]# crsctl check crs
CRS-4639: Could not contact Oracle High Availability Services

需要在启动时执行 /bin/dd if=/var/tmp/.oracle/npohasd of=/dev/null bs=1024 count=1。

  然后再执行 crsctl check crs查看服务启动情况。

2，如果出现以下情况，需要查看日志提示，有一次是因为内核参数配置值的问题导致了这个错误。

[root@rac1 bin]# ./crsctl check crs
 CRS-4638: Oracle High Availability Services is online
 CRS-4535: Cannot communicate with Cluster Ready Services 
 CRS-4529: Cluster Synchronization Services is online
 CRS-4533: Event Manager is online

5，安装Oracle database软件

需要用oracle用户安装

su - oracle

./runInstaller

仅安装数据库软件即可，这里就根据OUI的提示安装就可以了，最后按照提示执行相关脚本。

找时间补上截图

6，创建ASM磁盘组

需要新建其他磁盘组来安装数据文件等

su - grid

asmca

根据界面提示创建磁盘组。

找时间补上截图

7，建立数据库实例

su - oracle

./runInstaller

根据OUI步骤提示选择rac选项新建数据库实例即可。

完成。

小结：通过实验了解RAC环境的搭建和工作条件，但是这里面还涉及到许多集群的理论知识，需要不断地区学习和巩固，然后手动去实践。