介绍RAC环境的安装的文章很多,但是介绍RAC环境卸载的并不多见,如果RAC环境卸载的不彻底,很可能会影响RAC环境的再次搭建。
这一篇介绍CLUSTER软件的卸载。
上一篇中卸载了数据库,并删除了数据库软件。
在文章的开头已经提过,RAC环境的卸载不当会导致问题。正常情况下,应该首先卸载CLUSTER所做的修改,然后再卸载CLUSTERWARE。
正常情况下应该在ORA_CLS_HOME目录下执行:/install/rootdelete.sh remote nosharedhome –force,在RAC的每个节点上都应该执行这个操作,然后在其中一个节点上执行/install/rootdeinstall.sh。删除ASM实例对裸设备头的修改。
但是如果卸载的时候没有注意,很容易没有进行CLUSTER的清理就直接卸载了CLUSTER环境。
比如下面不进行清理而直接卸载CLUSTERWARE:
$ cd $ORACLE_HOME/../crs/oui/bin
$ ./runInstaller
Starting Oracle Universal Installer...
Checking installer requirements...
Checking operating system version: must be 5.8, 5.9 or 5.10. Actual 5.8
Passed
Checking Temp space: must be greater than 150 MB. Actual 11267 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
All installer requirements met.
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2010-04-22_05-45-29PM. Please wait ...$ Oracle Universal Installer, Version10.2.0.1.0 Production
Copyright (C) 1999, 2005, Oracle. All rights reserved.
仍然点击Deinstall Products。选择CLUSTERWARE的安装目录,这里是OraCrs10g_home1,然后点击Remove进行删除:
Oracle会弹出卸载的组件详细明细,确认后,开始卸载。
Oracle会弹出一个提示,说明ClusterWare的安装目录/data/oracle/product/10.2/crs将会在卸载后被删除,如果确认删除按yes,不删除这个目录按no,cancel则取消操作。这里选择yes继续。
在删除CLUSTER的过程中也出现了告警,说明在racnode2和racnode3节点上,有些目录没有权限删除,这种警告是正常的,可以忽略。
卸载结束后,检查日志文件,文件的位置仍然是在ORACLE_BASE/oraInventory/log目录下:
INFO: Saving serialized component list for /data/oracle/product/10.2/crs
INFO: Updating XML inventory.
INFO: Saving comps.xml for /data/oracle/product/10.2/crs
INFO: Current Inventory:
Oracle Home: OraCrs10g_home1
INFO: Updating files in Oracle home '/data/oracle/product/10.2/crs' to remote nodes 'racnode2,racnode3'.
INFO: List of files to be excluded from:null
WARNING: Could not rename file '/tmp/OraInstall2010-04-22_06-34-16PM/installCopyFile.lst' to '/tmp/OraInstall2010-04-22_06-34-16PM/i
nstallCopyFile.lst.bak'.
WARNING: Could not rename file '/tmp/OraInstall2010-04-22_06-34-16PM/installRemoveFile.lst' to '/tmp/OraInstall2010-04-22_06-34-16PM
/installRemoveFile.lst.bak'.
SEVERE: java.io.FileNotFoundException: /tmp/OraInstall2010-04-22_06-34-16PM/installRemoveFile.lst (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:106)
at java.io.FileInputStream.<init>(FileInputStream.java:66)
at java.io.FileReader.<init>(FileReader.java:41)
at oracle.sysman.oii.oiix.OiixFileOps.appendFilePw(OiixFileOps.java:2349)
at oracle.sysman.oii.oiix.OiixFileOps.appendFile(OiixFileOps.java:2373)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterLogger.appendFilesToOH(OiipgClusterLogger.java:673)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterLogger.appendFilesToOH(OiipgClusterLogger.java:626)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterLogger.updateListFiles(OiipgClusterLogger.java:319)
at oracle.sysman.oii.oiif.oiifw.OiifwClusterCopyWCCE.doOperation(OiifwClusterCopyWCCE.java:435)
at oracle.sysman.oii.oiif.oiifb.OiifbCondIterator.iterate(OiifbCondIterator.java:171)
at oracle.sysman.oii.oiif.oiifw.OiifwActionsPhaseWCDE.doOperation(OiifwActionsPhaseWCDE.java:606)
at oracle.sysman.oii.oiic.OiicDeinstallSession$OiicActionsThread.run(OiicDeinstallSession.java:2431)
SEVERE: java.io.FileNotFoundException: /tmp/OraInstall2010-04-22_06-34-16PM/installDirFile.lst (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:106)
at java.io.FileInputStream.<init>(FileInputStream.java:66)
at java.io.FileReader.<init>(FileReader.java:41)
at oracle.sysman.oii.oiix.OiixFileOps.appendFilePw(OiixFileOps.java:2349)
at oracle.sysman.oii.oiix.OiixFileOps.appendFile(OiixFileOps.java:2373)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterLogger.appendFilesToOH(OiipgClusterLogger.java:673)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterLogger.appendFilesToOH(OiipgClusterLogger.java:628)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterLogger.updateListFiles(OiipgClusterLogger.java:319)
at oracle.sysman.oii.oiif.oiifw.OiifwClusterCopyWCCE.doOperation(OiifwClusterCopyWCCE.java:435)
at oracle.sysman.oii.oiif.oiifb.OiifbCondIterator.iterate(OiifbCondIterator.java:171)
at oracle.sysman.oii.oiif.oiifw.OiifwActionsPhaseWCDE.doOperation(OiifwActionsPhaseWCDE.java:606)
at oracle.sysman.oii.oiic.OiicDeinstallSession$OiicActionsThread.run(OiicDeinstallSession.java:2431)
SEVERE: java.io.FileNotFoundException: /tmp/OraInstall2010-04-22_06-34-16PM/installCopyFile.lst (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:106)
at java.io.FileInputStream.<init>(FileInputStream.java:66)
at java.io.FileReader.<init>(FileReader.java:41)
at oracle.sysman.oii.oiix.OiixFileOps.appendFilePw(OiixFileOps.java:2349)
at oracle.sysman.oii.oiix.OiixFileOps.appendFile(OiixFileOps.java:2373)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterLogger.appendFilesToOH(OiipgClusterLogger.java:673)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterLogger.appendFilesToOH(OiipgClusterLogger.java:629)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterLogger.updateListFiles(OiipgClusterLogger.java:319)
at oracle.sysman.oii.oiif.oiifw.OiifwClusterCopyWCCE.doOperation(OiifwClusterCopyWCCE.java:435)
at oracle.sysman.oii.oiif.oiifb.OiifbCondIterator.iterate(OiifbCondIterator.java:171)
at oracle.sysman.oii.oiif.oiifw.OiifwActionsPhaseWCDE.doOperation(OiifwActionsPhaseWCDE.java:606)
at oracle.sysman.oii.oiic.OiicDeinstallSession$OiicActionsThread.run(OiicDeinstallSession.java:2431)
INFO: Error in copying cluster list file to central inventory; OUI-10185:Unable to copy '/tmp/OraInstall2010-04-22_06-34-16PM/instal
lCopyFile.lst' to '/data/oracle/oraInventory/logs/installCopyFile.lst.2010-04-22_06-34-16PM'. OUI-10186:The source file '/tmp/OraIns
tall2010-04-22_06-34-16PM/installCopyFile.lst' does not exist.
SEVERE: oracle.sysman.oii.oiip.oiipg.OiipgRemoteFileOperationException: Could not remove listed directories based on '/tmp/OraInstal
l2010-04-22_06-34-16PM/installRemoveDirFile.lst' from nodes 'racnode2,racnode3'. [PRKC-1002 : All the submitted commands did not exe
cute successfully]
----------------------------------------------------------------------------------
racnode2:
rm: Unable to remove directory /data/oracle/product/10.2/crs/OPatch: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/bin: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/cdata: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/cfgtoollogs/vipca: File exists
rm: Unable to remove directory /data/oracle/product/10.2/crs/cfgtoollogs: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/clone: Permission denied
.
.
.
rm: Unable to remove directory /data/oracle/product/10.2/crs/tg4sybs: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/tg4tera: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs: File exists
----------------------------------------------------------------------------------
racnode3:
rm: Unable to remove directory /data/oracle/product/10.2/crs/OPatch: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/bin: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/cfgtoollogs: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/clone: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/crs/admin: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/crs/demo: Permission denied
.
.
.
rm: Unable to remove directory /data/oracle/product/10.2/crs/cdata/crs: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/cdata: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs: File exists
----------------------------------------------------------------------------------
at oracle.sysman.oii.oiip.oiipg.OiipgClusterOps.throwOiipgRemoteFileOperationException(OiipgClusterOps.java:2699)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterOps.removeListedDirsFromNodes(OiipgClusterOps.java:2120)
at oracle.sysman.oii.oiif.oiifw.OiifwClusterCopyWCCE.doOperation(OiifwClusterCopyWCCE.java:700)
at oracle.sysman.oii.oiif.oiifb.OiifbCondIterator.iterate(OiifbCondIterator.java:171)
at oracle.sysman.oii.oiif.oiifw.OiifwActionsPhaseWCDE.doOperation(OiifwActionsPhaseWCDE.java:606)
at oracle.sysman.oii.oiic.OiicDeinstallSession$OiicActionsThread.run(OiicDeinstallSession.java:2431)
Caused by: oracle.ops.mgmt.cluster.RemoteFileOperationException: PRKC-1002 : All the submitted commands did not execute successfully
at oracle.ops.mgmt.cluster.ClusterCmd.removeListedDirsFromNodes(ClusterCmd.java:4274)
at oracle.ops.mgmt.cluster.ClusterCmd.removeListedDirsFromNodes(ClusterCmd.java:4083)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterOps.removeListedDirsFromNodes(OiipgClusterOps.java:2104)
... 4 more
WARNING: Could not remove listed directories based on '/tmp/OraInstall2010-04-22_06-34-16PM/installRemoveDirFile.lst' from nodes 'ra
cnode2,racnode3'. [PRKC-1002 : All the submitted commands did not execute successfully]
----------------------------------------------------------------------------------
racnode2:
rm: Unable to remove directory /data/oracle/product/10.2/crs/OPatch: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/bin: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/cdata: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/cfgtoollogs/vipca: File exists
rm: Unable to remove directory /data/oracle/product/10.2/crs/cfgtoollogs: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/clone: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/crs/admin: Permission denied
.
.
.
rm: Unable to remove directory /data/oracle/product/10.2/crs/tg4tera: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs: File exists
----------------------------------------------------------------------------------
racnode3:
rm: Unable to remove directory /data/oracle/product/10.2/crs/OPatch: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/bin: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/cfgtoollogs: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/clone: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs/crs/admin: Permission denied
.
.
.
rm: Unable to remove directory /data/oracle/product/10.2/crs/cdata: Permission denied
rm: Unable to remove directory /data/oracle/product/10.2/crs: File exists
----------------------------------------------------------------------------------
Refer to '/data/oracle/oraInventory/logs/installActions2010-04-22_06-34-16PM.log' for details. You may fix the errors on the require
d remote nodes. Refer to the install guide for error recovery. Click 'Yes' if you want to proceed. Click 'No' to exit the install. D
o you want to continue?
INFO: User Selected: Yes/OK
INFO: Setting variable 'CLUSTER_NODES' to 'racnode1,racnode2,racnode3,'. Received the value from a code block.
INFO: Local node 'racnode1' is specified by the user.
INFO: Setting variable 'REMOTE_NODES' to 'racnode2,racnode3,'. Received the value from a code block.
INFO: Removing Cluster Oracle homes
INFO: Running command '/tmp/OraInstall2010-04-22_06-34-16PM/oui/bin/runInstaller -jreLoc ../../jre -paramFile /tmp/OraInstall2010-04
-22_06-34-16PM/oui/clusterparam.ini -silent -ignoreSysPrereqs -removeHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/c
rs LOCAL_NODE=racnode2 -remoteInvocation -invokingNodeName racnode1 -logFilePath "/data/oracle/oraInventory/logs" -timestamp 2010-04
-22_06-34-16PM' on the nodes 'racnode2'.
INFO: Invoking OUI on cluster nodes racnode2
INFO: /tmp/OraInstall2010-04-22_06-34-16PM/oui/bin/runInstaller -jreLoc ../../jre -paramFile /tmp/OraInstall2010-04-22_06-34-16PM/ou
i/clusterparam.ini -silent -ignoreSysPrereqs -removeHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs LOCAL_NODE=rac
node2 -remoteInvocation -invokingNodeName racnode1 -logFilePath "/data/oracle/oraInventory/logs" -timestamp2010-04-22_06-34-16PM
INFO: Running command '/tmp/OraInstall2010-04-22_06-34-16PM/oui/bin/runInstaller -jreLoc ../../jre -paramFile /tmp/OraInstall2010-04
-22_06-34-16PM/oui/clusterparam.ini -silent -ignoreSysPrereqs -removeHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/c
rs LOCAL_NODE=racnode3 -remoteInvocation -invokingNodeName racnode1 -logFilePath "/data/oracle/oraInventory/logs" -timestamp 2010-04
-22_06-34-16PM' on the nodes 'racnode3'.
INFO: Invoking OUI on cluster nodes racnode3
INFO: /tmp/OraInstall2010-04-22_06-34-16PM/oui/bin/runInstaller -jreLoc ../../jre -paramFile /tmp/OraInstall2010-04-22_06-34-16PM/ou
i/clusterparam.ini -silent -ignoreSysPrereqs -removeHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs LOCAL_NODE=rac
node3 -remoteInvocation -invokingNodeName racnode1 -logFilePath "/data/oracle/oraInventory/logs" -timestamp2010-04-22_06-34-16PM
INFO: The flags set for this home (/data/oracle/product/10.2/database) are:clean
INFO: Home:/data/oracle/product/10.2/database oraclehomeproperties.xml is not saved because the flags indicate no changes to this fi
le.
INFO: The flags set for this home (/data/oracle/product/10.2/crs) are:localnode|nodelist|crs|removed
INFO: Home:/data/oracle/product/10.2/crs oraclehomeproperties.xml is not saved because the flags indicate no changes to this file.
INFO: Saving serialized component list for /data/oracle/product/10.2/crs
INFO: Updating XML inventory.
INFO: Current Inventory:
INFO: Deleting the directory: /data/oracle/product/10.2/crs
INFO: This deinstallation was successful
在上面的日志文件中可以看到错误主要是由于没有权限删除root用户建立的目录,可以利用root来删除这个目录,CLUSTERWARE的卸载完成。
但是由于没有进行清理工作,因此节点上还有一些进程没有被停止:
bash-2.03$ ps -ef|grep ora
oracle 2534 1 0 Feb 21 ? 0:00 /data/oracle/product/10.2/crs/opmn/bin/ons -d
oracle 15523 15403 0 09:40:28 pts/1 0:00 bash
oracle 1309 1308 0 Feb 21 ? 0:00 /bin/sh -c ulimit -c unlimited; cd /data/oracle/product/10.2/crs/log/racnode1/c
oracle 1932 1 0 Feb 21 ? 1:02 /data/oracle/product/10.2/database/bin/tnslsnr LISTENER1_RACNODE1 -inherit
oracle 1308 1150 0 Feb 21 ? 0:00 sh -c /bin/sh -c 'ulimit -c unlimited; cd /data/oracle/product/10.2/crs/log/rac
root 377 1 0 Feb 21 ? 152:16 /data/oracle/product/10.2/crs/bin/crsd.bin reboot
oracle 375 1 0 Feb 21 ? 0:00 sh -c sh -c 'ulimit -c unlimited; cd /data/oracle/product/10.2/crs/log/racnode1
root 1216 1147 0 Feb 21 ? 0:00 /data/oracle/product/10.2/crs/bin/oprocd run -t 1000 -m 500 -f
oracle 1562 1151 0 Feb 21 ? 1:01 /data/oracle/product/10.2/crs/bin/evmlogger.bin -o /data/oracle/product/10.2/cr
oracle 1151 375 0 Feb 21 ? 16:10 /data/oracle/product/10.2/crs/bin/evmd.bin
oracle 1310 1309 0 Feb 21 ? 85:41 /data/oracle/product/10.2/crs/bin/ocssd.bin
oracle 2536 2534 0 Feb 21 ? 0:03 /data/oracle/product/10.2/crs/opmn/bin/ons -d
oracle 17996 15523 0 09:42:04 pts/1 0:00 grep ora
oracle 15403 15401 0 09:40:23 pts/1 0:00 -sh
bash-2.03$ su -
Password:
Sun Microsystems Inc. SunOS 5.8 Generic Patch October 2001
You have new mail.
# /etc/init.d/init.crs stop
Shutting down Oracle Cluster Ready Services (CRS):
Shutdown has begun. The daemons should exit soon.
# ps -ef|grep ora
oracle 2534 1 0 Feb 21 ? 0:00 /data/oracle/product/10.2/crs/opmn/bin/ons -d
oracle 15523 15403 0 09:40:28 pts/1 0:00 bash
oracle 1309 1308 0 Feb 21 ? 0:00 /bin/sh -c ulimit -c unlimited; cd /data/oracle/product/10.2/crs/log/racnode1/c
oracle 1932 1 0 Feb 21 ? 1:02 /data/oracle/product/10.2/database/bin/tnslsnr LISTENER1_RACNODE1 -inherit
oracle 1308 1150 0 Feb 21 ? 0:00 sh -c /bin/sh -c 'ulimit -c unlimited; cd /data/oracle/product/10.2/crs/log/rac
root 377 1 0 Feb 21 ? 152:16 /data/oracle/product/10.2/crs/bin/crsd.bin reboot
oracle 375 1 0 Feb 21 ? 0:00 sh -c sh -c 'ulimit -c unlimited; cd /data/oracle/product/10.2/crs/log/racnode1
root 1216 1147 0 Feb 21 ? 0:00 /data/oracle/product/10.2/crs/bin/oprocd run -t 1000 -m 500 -f
oracle 1562 1151 0 Feb 21 ? 1:01 /data/oracle/product/10.2/crs/bin/evmlogger.bin -o /data/oracle/product/10.2/cr
oracle 1151 375 0 Feb 21 ? 16:10 /data/oracle/product/10.2/crs/bin/evmd.bin
oracle 1310 1309 0 Feb 21 ? 85:41 /data/oracle/product/10.2/crs/bin/ocssd.bin
root 18632 18287 0 09:42:34 pts/1 0:00 grep ora
oracle 2536 2534 0 Feb 21 ? 0:03 /data/oracle/product/10.2/crs/opmn/bin/ons -d
oracle 15403 15401 0 09:40:23 pts/1 0:00 -sh
可以看到,通过/etc/init.d/init.crs已经无法关闭已启动的cluster进程了。其他两个节点的情况类似。
如果说这些进程重启后就会消失,那么ASM实例对裸设备头的修改没有清除,这样以后重建RAC的时候会发现以前的磁盘组文件仍然存在,且没有办法重新使用。
首先找到ocr对应的裸设备:
bash-2.03$ more /var/opt/oracle/ocr.loc
ocrconfig_loc=/dev/rac/ocr
local_only=FALSE
bash-2.03$ ls -l /dev/rac/ocr
lrwxrwxrwx 1 root other 18 Nov 26 16:52 /dev/rac/ocr -> /dev/rdsk/c2t1d2s1
然后利用dd手工清理ocr设备:
# dd if=/dev/zero f=/dev/rdsk/c2t1d2s1 skip=25 bs=4k count=2560
2560+0 records in
2560+0 records out
清除/var/opt/oracle目录下所有的文件,否则会导致以后安装CLUSTERWARE的时候报错,Oracle会认为系统中已经配置过CLUSTERWARE。
至于留下的cluster相关的进程,无论是KILL的方式,还是REBOOT的方式,总之重启系统是不可避免的了。
oracle视频教程请关注:http://u.youku.com/user_video/id_UMzAzMjkxMjE2.html
转载于:https://blog.51cto.com/19880614/1221583