OS:
Redhat linux AS 4U4(2.6.9-42.ELsmp)
Oracle 10G database 10GR2
Oracle Clusterware
Patch for RAC and DB
VMWare 1.0.6
1.Install OS
A:(HDD>=8G,Memory 500M),Include the following:/boot(100M),/(2G)+insfile(2G),swap(1G),/home/compard/crs(<1G),/home/compard/10.2.0(<1.5G)
B.add 4 hdd(2*100M(orc.vmdk,vote.vmdk),2*2GB(asm1.vmdk,asm2.vmdk)) and 1 ethernet card:
Location of new hdd file must in another folder to share with other node.
If you use the partition that used by root,you can execute "partprobe" to take the affect to kernel
C.Add the following to vmware viture machine file(.vmx)
disk.locking = "false"
diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.DataCacheMinReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0"
scsi1:1.deviceType = "disk"
scsi1:2.deviceType = "disk"
scsi1:3.deviceType = "disk"
scsi1:4.deviceType = "disk"
2.Install Tools and PKG:
vnc,telnet,gsftp,vmware-tools
After install vmware-tools, you must use it to sync the time between the two machine.
Need PKG in oracle official document(RHEL4):
make-3.80-5
gcc-3.4.3-9.EL4
compat-db-4.1.25-9
installing order:
glibc-kernheaders-2.4-9.1.98.EL.i386.rpm
glibc-headers-2.3.4-2.25.i386.rpm
glibc-devel-2.3.4-2.25.i386.rpm
gcc-3.4.6-3.i386.rpm
oracleasm-support-2.1.3-1.el4.i386.rpm
oracleasm-2.6.9-42.ELsmp-2.0.3-1.i686.rpm
oracleasmlib-2.0.4-1.el4.i386.rpm
libstdc++-devel-3.4.6-3.i386.rpm(Need by 10GR2)
gcc-c++-3.4.6-3.i386.rpm(Need by 10GR2)
sysstat-5.0.5-11.rhel4.i386.rpm
Check:
rpm -q binutils compat-db compat-libstdc++-296 control-center gcc gcc-c++ glibc glibc-common gnome-libs libstdc++ sysstat libstdc++-devel make pdksh xscreensaver setarch
3.system Setting
A.add the following to the end of the file "/etc/sysctl.conf",use command "sysctl -p" to apply the affect after doing this.
kernel.shmall = 2097152
kernel.shmmax = 2147483648
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default=262144
net.core.wmem_default=262144
net.core.rmem_max=262144
net.core.wmem_max=262144
B./etc/security/limits.conf
compard soft nproc 2047
compard hard nproc 16384
compard soft nofile 1024
compard hard nofile 65536
C.Hangcheck Timer:/etc/rc.d/rc.local,add to the end and execute
modprobe hangcheck-timer hangcheck_tick=30 hangcheck_margin=180
D./etc/hosts,priv IP and host IP are assign to eth1 and eth0
127.0.0.1 localhost.compard.com localhost
192.168.1.106 myrac1.compard.com myrac1
1.1.1.10 myrac1priv.compard.com myrac1priv
192.168.1.116 myrac2.compard.com myrac2
1.1.1.11 myrac2priv.compard.com myrac2priv
10.10.10.10 myrac1vip.compard.com myrac1vip
10.10.10.11 myrac2vip.compard.com myrac2vip
E:create user group dba and compard belong to it.
groupadd dba
useradd -g dba compard
F:/home/compard/.bash_profile
export ORACLE_BASE=/home/compard
export ORACLE_HOME=$ORACLE_BASE/10.2.0
export CRS_HOME=$ORACLE_BASE/crs
export ORA_CRS_HOME=$ORACLE_BASE/crs
export ORACLE_SID=RAC1
export NLS_LANG=AMERICAN_AMERICA.AL32UTF8
export PATH=.:$PATH:$ORACLE_HOME/bin:$ORA_CRS_HOME/bin:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH:/lib:/usr/lib:/usr/local/lib
G:/etc/pam.d/login
session required /lib/security/pam_limits.so
H:stop no use serice
chkconfig cups off
chkconfig sendmail off
chkconfig isdn off
chkconfig smartd off
chkconfig iptables off
chkconfig pcmcia off
4.RAW Setting:
fdisk /dev/sd[b,c,d,e] n --- p --- 1 -- Enter --Enter -- w
A:/etc/sysconfig/rawdevices
/dev/raw/raw1 /dev/sdb1
/dev/raw/raw2 /dev/sdc1
/dev/raw/raw3 /dev/sdd1
/dev/raw/raw4 /dev/sde1
B.execute the following command(root):
/sbin/service rawdevices restart
chown compard:dba /dev/raw/raw[1-4]
chmod 660 /dev/raw/raw[1-4]
ls -lat /dev/raw/raw*
C./etc/udev/permissions.d/50-udev.permissions
# raw devices
ram*:root:disk:0660
#raw/*:root:disk:0660 #mark this
raw/*:compard:dba:0660 #add this
D.check if the user compard have the privilege to write on the raw device:
dd if=/dev/zero of=/dev/raw/raw1 bs=1024 count=1024
5.Clone to a new machine named myrac2
A.copy
B.update the displayName ="<New name>" in file .vmx
C.Start OS,delete all ethernet card,then re-add it
C.Update hostname/IP
D.update ORACLE_SID
6.RAC Setting:
A.SSH(ALL)
su - compard
mkdir ~/.ssh
chmod 755 ~/.ssh
ssh-keygen -t rsa
ssh-keygen -t dsa
(RAC1)
cat ~/.ssh/id_rsa.pub >> ./.ssh/authorized_keys
cat ~/.ssh/id_dsa.pub >> ./.ssh/authorized_keys
ssh myrac2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac2.compard.com cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac2.compard.com cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac2priv cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac2priv cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac2priv.compard.com cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac2priv.compard.com cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac1 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac1 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac1.compard.com cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac1.compard.com cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac1priv cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac1priv cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac1priv.compard.com cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac1priv.compard.com cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys myrac2:~/.ssh/authorized_keys
exec /usr/bin/ssh-agent $SHELL
/usr/bin/ssh-add
(RAC2)
ssh myrac2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac2.compard.com cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac2.compard.com cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac2priv cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac2priv cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac2priv.compard.com cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac2priv.compard.com cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac1 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac1 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac1.compard.com cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac1.compard.com cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac1priv cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac1priv cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh myrac1priv.compard.com cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh myrac1priv.compard.com cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
Check
ssh myrac1 date
ssh myrac1.compard.com date
ssh myrac1priv date
ssh myrac1priv.compard.com date
ssh myrac2 date
ssh myrac2.compard.com date
ssh myrac2priv date
ssh myrac2priv.compard.com date
B.ASM
(ALL)
/etc/init.d/oracleasm configure(compard,dba,y,y)
(RAC1)
/etc/init.d/oracleasm createdisk VOL1 /dev/sdd1
/etc/init.d/oracleasm createdisk VOL2 /dev/sde1
(RAC2)
/etc/init.d/oracleasm scandisks
(Check)
/etc/init.d/oracleasm listdisks
7.Clusterware Install
A.ftp and unzip ,then chmod 0777
you can check before installation,but the error can be ignored
su - compard
cd /usr/insfile/clusterware/cluvfy
./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verbose
B.Verify the time between RAC1 and RAC2.
C.Install ...
Error during install:
The given interface(s), "eth0" is not public. Public interfaces should be used to configure virtual IPs
Solution:
(RAC2),X manager root Login
/home/compard/crs/bin/vipca
After install,use command "olsnodes" to check the result(run in compard).
8.DB software only
9.execute the netca to setup the listener.
name myrac
RAC1:name will auto-change to myrac_myrac1
RAC2:name will auto-change to myrac_myrac2
10.create db instance
After install, use the following SQL to login DB:
Server:
sqlplus system/<PWD>@rac1
sqlplus system/<PWD>@rac2
Client:
tnsnames.ora
RAC_DB =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = myrac1.compard.com)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = myrac2.compard.com)(PORT = 1521))
(LOAD_BALANCE = yes)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = RAC.compard.com)
)
)
C:/WINDOWS/system32/drivers/etc/hosts:
192.168.1.106 myrac1.compard.com
192.168.1.116 myrac2.compard.com
Toad,PL/SQL,SQLProgress.exe to Login:
SQLProgress:
@echo off
set ORACLE_HOME=C:/oracle/product/10.2.0/client_1
set PATH=%ORACLE_HOME%/bin;%PATH%
"D:/Program Files/Compard/sqlprogress.exe"
screen-snapshot when login successfully:
11.additional information when finished:
A.df -k
[compard@myrac2 admin]$ df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda2 3526204 2820012 527068 85% /
/dev/sda1 101086 11422 84445 12% /boot
none 259252 0 259252 0% /dev/shm
/dev/sda5 2482524 2144040 212376 91% /home
/dev/hdc 2191906 2191906 0 100% /media/cdrom
[compard@myrac1 admin]$ df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda2 3526204 2929712 417368 88% /
/dev/sda1 101086 11422 84445 12% /boot
none 259252 0 259252 0% /dev/shm
/dev/sda5 2482524 1971304 385112 84% /home
/dev/hdc 2191906 2191906 0 100% /media/cdrom
B.Tablespace Used:
TABLESPACE_NAME TOTAL(M) USED(M) USED(%) FREE(M) MAX_BLOCK
SYSAUX 250 244.25 97.7 5.75 5.75
SYSTEM 480 470.75 98.073 9.25 8.938
TEMP 20 19 95 1 20
UNDOTBS1 35 33.375 95.357 1.625 0.938
UNDOTBS2 25 4.125 16.5 20.875 20.875
USERS 5 0.437 8.74 4.563 4.563
C.select count(1) from v$session
35
11:Stop/Start RAC
stop
A.listener
B:DB
C:RAC Service
srvctl stop nodeapps -n myrac2(compard)
srvctl stop nodeapps -n myrac1(compard)
/etc/init.d/init.crs stop(root)
start:
srvctl start nodeapps -n myrac1
srvctl start nodeapps -n myrac2
B:DB
C.listener
All Service in RAC(crs_stat -t)
ora....C1.inst application OFFLINE OFFLINE
ora....C2.inst application OFFLINE OFFLINE
ora.RAC.db application OFFLINE OFFLINE
ora....SM1.asm application OFFLINE OFFLINE
ora....C1.lsnr application OFFLINE OFFLINE
ora.myrac1.gsd application OFFLINE OFFLINE
ora.myrac1.ons application OFFLINE OFFLINE
ora.myrac1.vip application OFFLINE OFFLINE
ora....SM2.asm application OFFLINE OFFLINE
ora....C2.lsnr application OFFLINE OFFLINE
ora.myrac2.gsd application OFFLINE OFFLINE
ora.myrac2.ons application OFFLINE OFFLINE
ora.myrac2.vip application OFFLINE OFFLINE
Q&A:
1.某次开启环境后发现有个节点开机后几分钟自动重启,原因不明,最后换成了RHES4U8后正常。也许是低版本中的BUG导致。
2.VMWare1.0.6安装后机器重启慢了一半,最后换成了Vmware workstation版本,目前运行正常。